Home-Network/home-network-k3s-impl
2
0 Fork 0
Code Issues 2 Pull Requests 6 Actions Packages Projects Releases Wiki Activity
Files
d8b25a67feeea46f3e1311d4e8fb15aedaaad8e8
home-network-k3s-impl/decisions/2026-11-08-wg-client-mtu-docs.md
T
jakman85andClaude Opus 5.5 8f60e6103a
gate / gate (pull_request) Successful in 57s
Details
gate / conformance (pull_request) Skipped
Details
fix(vpn): default client profiles to MTU 1280; correct the VPN runbook 
new-peer.sh generated MTU 1420, which stalled traffic on mobile data
despite a healthy handshake (one phone, one carrier, 2026-11-09). Default
to 1280, overridable through WG_MTU.

Correct stale claims in the runbook and the DaemonSet header against
live BPF conntrack: LoadBalancer VIPs see 10.90.0.x as world, not a
SNATed host identity. Add a troubleshooting section.

The DaemonSet render is unchanged, so WireGuard pods do not roll.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-08 21:15:20 -06:00

4.8 KiB
Raw Blame History

date, branch, title
date branch title
2026-11-08 fix/wg-client-mtu-docs WireGuard client profiles default to MTU 1280; VPN runbook corrected and given a troubleshooting section

What changed

  • infra/vpn-ingress/new-peer.sh: WG_MTU default 1420 -> 1280, still overridable, reason in a comment. Only new profiles change; the phone's existing profiles are not touched.
  • runbooks/vpn-remote-access.md: the verification table no longer says the Gateway sees a host-identity source; the limitations no longer say everything arrives SNATed or that IPv6 leaks on Android; the "if nothing handshakes" advice no longer claims wg show separates a bad forward from a bad key. New "Troubleshooting" section covering the four cases in the brief.
  • platform/vpn-ingress/daemonset.yaml: header comment only, SNAT claim corrected.
  • platform/vpn-ingress/README.md: one sentence, so MTU = 1420 reads as the server's value.

Rollout on merge: the WireGuard DaemonSet does not roll. The runbooks ConfigMap is content hashed and vpn-remote-access.md is in it, so the runbooks site rolls.

Left unchanged, for the operator:

  • platform/vpn-ingress/peers.conf still says "No peers enrolled yet". A comment-only edit on a throwaway copy moved the ConfigMap from wg-peers-29mf6k867t to wg-peers-gggkk5d4tm and changed the DaemonSet, so correcting it restarts every wg0. Fix it alongside the next peer change.
  • clusters/base/cluster-vars.yaml coverage comment still omits WG_CLIENT_CIDR (sensitive path).
  • Server wg0 MTU stays 1420, because changing it restarts all WireGuard pods. For TCP a 1280 client should be enough on its own: it advertises an MSS of 1240, which caps what the far end sends. That protection does not reach UDP. A server-to-phone datagram with an inner size of 1281-1420 bytes is still sent as up to 1480 outer bytes and could be lost on the same path; QUIC should recover by probing, large DNS answers are the likeliest casualty. If that shows up, the fix is a lower server MTU, as a deliberate rollout.
  • Other stale text, outside this brief: platform/vpn-ingress/README.md still says no peer has ever completed a handshake, and the runbook's status line still calls the VPN inert.

What I verified

  • Upstream wireguard-android GoBackend.java, master (last commit to that file ecc3c0a, 2026-03-15): builder.setMtu(config.getInterface().getMtu().orElse(1280)), and allowFamily() is skipped when sawDefaultRoute && config.getPeers().size() == 1.
  • Live, read-only, 2026-11-09 UTC, on the k3s-prod-1 pod and agent:
    • wg0 MTU is 1420 on all three pods. Both peers have handshakes and transfer on prod-1 only; the other two pods read 0.
    • BPF conntrack holds UDP SVC 10.90.0.x -> 192.168.2.23:53 and flows to 192.168.2.24:32400. Their backend entries carry SourceSecurityID=2 (world). So the source stays 10.90.0.x through the load balancer on those two VIPs.
    • /proc/net/nf_conntrack is readable from the pod, which has no conntrack binary.
  • Offline, pinned toolbox: kubectl kustomize platform/vpn-ingress gives identical ConfigMap and DaemonSet hashes on origin/main and on this branch.
  • new-peer.sh writes MTU = 1280 by default and MTU = 1420 with WG_MTU=1420. Run with throwaway keys and stdout discarded; profiles shredded; nothing enrolled.
  • No test or conformance check pins the client MTU (grep for WG_MTU and 1420).
  • Gate (pinned toolbox, from this worktree, on the committed tree): rc 3, UNDETERMINED, not green. gitea-settings is the only stage counted as not run (no token locally). conformance (L2/L3) also skipped as the declared no-cluster gap, so R-ENV-01 and R-OBS-04 are not certified. spec-immutability, approval-policy, decisions-log, lint, l0, render-golden and l1 passed. No sensitive path is touched.

What I did NOT verify

  • That MTU was the cause. It is inferred from one before-and-after, on one phone and one carrier, reported by the operator. The path MTU was never measured.
  • That 1280 is the best value. It is the value that worked, and the IPv6 minimum. A higher one may work too.
  • That the phone's app is on GoBackend rather than the kernel backend, so that a blank field meant 1280 on this device. Only the upstream default was read.
  • That TCP MSS behaves as described, or that large UDP replies are actually lost. No packet capture was taken.
  • That the Gateway VIP .20 sees 10.90.0.x / world. Conntrack held no tunnel flows to it when checked.
  • Android's IPv6 blocking on a real device; read from source only.
  • wg-quick IPv6 behaviour on Linux, and any non-Android client at 1280.
  • /proc/net/nf_conntrack output during a key mismatch. The [UNREPLIED] reading comes from the earlier measurement in the brief; I saw only healthy entries.
Reference in new issue View Git Blame Copy Permalink