jakman85andClaude Opus 5.5
8f60e6103a
new-peer.sh generated MTU 1420, which stalled traffic on mobile data despite a healthy handshake (one phone, one carrier, 2026-11-09). Default to 1280, overridable through WG_MTU. Correct stale claims in the runbook and the DaemonSet header against live BPF conntrack: LoadBalancer VIPs see 10.90.0.x as world, not a SNATed host identity. Add a troubleshooting section. The DaemonSet render is unchanged, so WireGuard pods do not roll. Co-Authored-By: Claude Opus 5.5 <[email protected]>
75 lines
4.8 KiB
Markdown
75 lines
4.8 KiB
Markdown
---
|
|
date: 2026-11-08
|
|
branch: fix/wg-client-mtu-docs
|
|
title: WireGuard client profiles default to MTU 1280; VPN runbook corrected and given a troubleshooting section
|
|
---
|
|
|
|
## What changed
|
|
|
|
- **`infra/vpn-ingress/new-peer.sh`**: `WG_MTU` default 1420 -> 1280, still overridable, reason in a
|
|
comment. Only new profiles change; the phone's existing profiles are not touched.
|
|
- **`runbooks/vpn-remote-access.md`**: the verification table no longer says the Gateway sees a
|
|
`host`-identity source; the limitations no longer say everything arrives SNATed or that IPv6 leaks
|
|
on Android; the "if nothing handshakes" advice no longer claims `wg show` separates a bad forward
|
|
from a bad key. New "Troubleshooting" section covering the four cases in the brief.
|
|
- **`platform/vpn-ingress/daemonset.yaml`**: header comment only, SNAT claim corrected.
|
|
- **`platform/vpn-ingress/README.md`**: one sentence, so `MTU = 1420` reads as the server's value.
|
|
|
|
Rollout on merge: the WireGuard DaemonSet does **not** roll. The `runbooks` ConfigMap is content
|
|
hashed and `vpn-remote-access.md` is in it, so the runbooks site rolls.
|
|
|
|
Left unchanged, for the operator:
|
|
|
|
- **`platform/vpn-ingress/peers.conf`** still says "No peers enrolled yet". A comment-only edit on a
|
|
throwaway copy moved the ConfigMap from `wg-peers-29mf6k867t` to `wg-peers-gggkk5d4tm` and changed
|
|
the DaemonSet, so correcting it restarts every `wg0`. Fix it alongside the next peer change.
|
|
- **`clusters/base/cluster-vars.yaml`** coverage comment still omits `WG_CLIENT_CIDR` (sensitive path).
|
|
- **Server `wg0` MTU stays 1420**, because changing it restarts all WireGuard pods. For TCP a 1280
|
|
client should be enough on its own: it advertises an MSS of 1240, which caps what the far end
|
|
sends. That protection does not reach UDP. A server-to-phone datagram with an inner size of
|
|
1281-1420 bytes is still sent as up to 1480 outer bytes and could be lost on the same path; QUIC
|
|
should recover by probing, large DNS answers are the likeliest casualty. If that shows up, the
|
|
fix is a lower server MTU, as a deliberate rollout.
|
|
- **Other stale text, outside this brief:** `platform/vpn-ingress/README.md` still says no peer has
|
|
ever completed a handshake, and the runbook's status line still calls the VPN inert.
|
|
|
|
## What I verified
|
|
|
|
- Upstream `wireguard-android` `GoBackend.java`, master (last commit to that file `ecc3c0a`,
|
|
2026-03-15): `builder.setMtu(config.getInterface().getMtu().orElse(1280))`, and `allowFamily()` is
|
|
skipped when `sawDefaultRoute && config.getPeers().size() == 1`.
|
|
- Live, read-only, 2026-11-09 UTC, on the `k3s-prod-1` pod and agent:
|
|
- `wg0` MTU is 1420 on all three pods. Both peers have handshakes and transfer on prod-1 only; the
|
|
other two pods read `0`.
|
|
- BPF conntrack holds `UDP SVC 10.90.0.x -> 192.168.2.23:53` and flows to `192.168.2.24:32400`.
|
|
Their backend entries carry `SourceSecurityID=2` (`world`). So the source stays `10.90.0.x`
|
|
through the load balancer on those two VIPs.
|
|
- `/proc/net/nf_conntrack` is readable from the pod, which has no `conntrack` binary.
|
|
- Offline, pinned toolbox: `kubectl kustomize platform/vpn-ingress` gives identical ConfigMap and
|
|
DaemonSet hashes on `origin/main` and on this branch.
|
|
- `new-peer.sh` writes `MTU = 1280` by default and `MTU = 1420` with `WG_MTU=1420`. Run with
|
|
throwaway keys and stdout discarded; profiles shredded; nothing enrolled.
|
|
- No test or conformance check pins the client MTU (`grep` for `WG_MTU` and `1420`).
|
|
- Gate (pinned toolbox, from this worktree, on the committed tree): **rc 3, UNDETERMINED, not green.**
|
|
`gitea-settings` is the only stage counted as not run (no token locally). `conformance` (L2/L3)
|
|
also skipped as the declared no-cluster gap, so R-ENV-01 and R-OBS-04 are not certified.
|
|
`spec-immutability`, `approval-policy`, `decisions-log`, `lint`, `l0`, `render-golden` and `l1`
|
|
passed. No sensitive path is touched.
|
|
|
|
## What I did NOT verify
|
|
|
|
- **That MTU was the cause.** It is inferred from one before-and-after, on one phone and one carrier,
|
|
reported by the operator. The path MTU was never measured.
|
|
- **That 1280 is the best value.** It is the value that worked, and the IPv6 minimum. A higher one
|
|
may work too.
|
|
- That the phone's app is on `GoBackend` rather than the kernel backend, so that a blank field meant
|
|
1280 on this device. Only the upstream default was read.
|
|
- That TCP MSS behaves as described, or that large UDP replies are actually lost. No packet capture
|
|
was taken.
|
|
- That the Gateway VIP `.20` sees `10.90.0.x` / `world`. Conntrack held no tunnel flows to it when
|
|
checked.
|
|
- Android's IPv6 blocking on a real device; read from source only.
|
|
- `wg-quick` IPv6 behaviour on Linux, and any non-Android client at 1280.
|
|
- `/proc/net/nf_conntrack` output during a key mismatch. The `[UNREPLIED]` reading comes from the
|
|
earlier measurement in the brief; I saw only healthy entries.
|