fix(vpn): default client profiles to MTU 1280; correct the VPN runbook

new-peer.sh generated MTU 1420, which stalled traffic on mobile data
despite a healthy handshake (one phone, one carrier, 2026-11-09). Default
to 1280, overridable through WG_MTU.

Correct stale claims in the runbook and the DaemonSet header against
live BPF conntrack: LoadBalancer VIPs see 10.90.0.x as world, not a
SNATed host identity. Add a troubleshooting section.

The DaemonSet render is unchanged, so WireGuard pods do not roll.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

apps/wave-0/pihole-dns.yaml

@@ -83,15 +83,25 @@
 # and would also fix the same latent defect in unifi and crafty; it belongs in its own change.
 #
 # -------------------------------------------------------------------------------------------------
-# loadBalancerSourceRanges — THE LAN RESTRICTION, ENFORCED IN THE DATAPATH.
+# loadBalancerSourceRanges — THE LAN + VPN RESTRICTION, ENFORCED IN THE DATAPATH.
 # -------------------------------------------------------------------------------------------------
 # Cilium enforces this list in eBPF on the ingress node, on the VIP, BEFORE load-balancing and
-# before SNAT. A packet to ${PIHOLE_DNS_VIP}:53 from outside ${LAN_CIDR} is dropped by the
-# datapath — it never reaches a backend and never reaches a network policy. This is what makes
+# before SNAT. A packet to ${PIHOLE_DNS_VIP}:53 from outside ${LAN_CIDR} and ${WG_CLIENT_CIDR} is
+# dropped by the datapath — it never reaches a backend and never reaches a network policy. This is what makes
 # "LAN clients only" a real, tested restriction rather than a comment: VERIFIED by narrowing the
 # range to a /24 that excludes the test client, after which the same query that had been answering
 # stopped being answered at all, and restoring it, after which it answered again.
 #
+# ${WG_CLIENT_CIDR} IS LISTED BECAUSE VPN CLIENTS REACH THIS CHECK UNTRANSLATED. wg0 is one of
+# Cilium's kube-proxy-replacement devices (measured 2026-11-08: `cilium-dbg status` lists eno1, wg0,
+# tun0), so a tunnel client's query is load-balanced as it enters from wg0, ahead of the iptables
+# MASQUERADE in platform/vpn-ingress/daemonset.yaml, and the source tested here is still 10.90.0.x
+# rather than the node's address. Without this entry a full-tunnel client's DNS is predicted to die
+# with DROP_NOT_IN_SRC_RANGE (derived from Cilium's source; NOT observed live, because no peer has
+# ever completed a handshake). The range is private and reachable only by a key in peers.conf, so
+# PiHole stays off the internet; conformance/tests/l0-static/pihole-dns-source-ranges fails the gate
+# if either entry goes missing or any range here stops being RFC1918.
+#
 # It is also the layer that survives the SNAT above. A CiliumNetworkPolicy cannot express this
 # restriction once the source has been rewritten: measured on this cluster, the forwarded query
 # arrives from the ingress node's internal router address and Cilium resolves that source to the
@@ -113,6 +123,7 @@ spec:
   externalTrafficPolicy: Cluster
   loadBalancerSourceRanges:
     - ${LAN_CIDR}
+    - ${WG_CLIENT_CIDR}
   selector:
     app.kubernetes.io/name: pihole
   ports:
@@ -133,6 +144,7 @@ spec:
   externalTrafficPolicy: Cluster
   loadBalancerSourceRanges:
     - ${LAN_CIDR}
+    - ${WG_CLIENT_CIDR}
   selector:
     app.kubernetes.io/name: pihole
   ports:
@@ -158,7 +170,9 @@ spec:
 # "Policy denied ... identity world" drops in `cilium-dbg monitor`). So `world` is the only peer
 # selector that can describe LAN traffic that has been through the LoadBalancer, and the set of
 # addresses that can actually BE that traffic is fixed by loadBalancerSourceRanges before this
-# policy is consulted. Together: ${LAN_CIDR} in the datapath, udp/53 + tcp/53 here.
+# policy is consulted. Together: ${LAN_CIDR} + ${WG_CLIENT_CIDR} in the datapath, udp/53 + tcp/53
+# here. A VPN client delivered to a backend on its own ingress node keeps its 10.90.0.x source,
+# which Cilium also classifies as `world`, so this rule needs no new peer for it.
 #
 # This is NOT wider than the precedent it follows. unifi-allow-l4-ingress and crafty-allow-ingress
 # admit `fromEntities: [world, cluster, host]` on their L4 ports with NO source-range filter at all;
@@ -187,7 +201,7 @@ spec:
       app.kubernetes.io/name: pihole
   ingress:
     - fromEntities:
-        - world         # LAN clients, post-LoadBalancer: see the block above for why not fromCIDR
+        - world         # LAN + VPN clients, post-LoadBalancer: see the block above for why not fromCIDR
         - host          # this node's kubelet probes
         - remote-node   # the node-to-node hop that forwards a query to wherever the pod runs
       toPorts:

catalog/pihole.yaml

@@ -85,7 +85,7 @@ env:
   # Default is LOCAL, which answers only clients on the same subnet as the listening interface. The
   # pod's interface is on the CLUSTER pod CIDR, so every LAN client would be refused. ALL is
   # required for a LoadBalancer-fronted resolver; what keeps PiHole from being an OPEN resolver is
-  # the exposure, not this setting — `loadBalancerSourceRanges: [${LAN_CIDR}]` on the DNS Services,
+  # the exposure, not this setting — `loadBalancerSourceRanges: [${LAN_CIDR}, ${WG_CLIENT_CIDR}]` on the DNS Services,
   # enforced by Cilium in the datapath before load-balancing, plus a CiliumNetworkPolicy that admits
   # nothing but udp/53 and tcp/53. See apps/wave-0/pihole-dns.yaml.
   - name: FTLCONF_dns_listeningMode

conformance/tests/l0-static/pihole-dns-source-ranges/chainsaw-test.yaml

@@ -0,0 +1,39 @@
+# =============================================================================
+# L0 STATIC — PiHole's DNS frontends admit the LAN and the VPN, and nothing public
+# Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster.
+#
+# loadBalancerSourceRanges is the only thing keeping PiHole from being an open resolver, and it
+# must also carry the WireGuard client subnet, because Cilium load-balances tunnel traffic on wg0
+# before the iptables MASQUERADE (so the source it tests is still 10.90.0.x). check.sh's header
+# states the assertions and what they do not prove.
+#
+# STATUS: EXECUTABLE. `check.sh` is a pure offline assertion script (standard library + PyYAML);
+# conformance/run.sh --layer l0 runs it directly and this Chainsaw Test wraps the SAME script.
+# =============================================================================
+apiVersion: chainsaw.kyverno.io/v1alpha1
+kind: Test
+metadata:
+  name: pihole-dns-source-ranges
+  labels:
+    conformance.home.k3s/R-DNS-03: "true"
+  annotations:
+    conformance.home.k3s/requirements: "R-DNS-03"
+    conformance.home.k3s/layer: "L0"
+    # WHOLE-REPO SCOPE: ci/touched-requirements.sh maps apps/* to no R-DNS requirement, so a change
+    # to these Services would otherwise select nothing.
+    conformance.home.k3s/scope: "whole-repo"
+    conformance.home.k3s/safety-critical: "false"
+    conformance.home.k3s/executable: "true"
+    conformance.home.k3s/intent: >-
+      Every PiHole LoadBalancer Service lists both the LAN and the WireGuard client subnet in
+      loadBalancerSourceRanges, and every range resolves to RFC1918, so VPN clients can resolve
+      and PiHole stays unreachable from the internet.
+spec:
+  steps:
+    - name: assert-pihole-source-ranges
+      description: >-
+        Collect the PiHole LoadBalancer Services, resolve each source range through cluster-vars,
+        and assert both required ranges are present and every range is private.
+      try:
+        - script:
+            content: bash ./check.sh

conformance/tests/l0-static/pihole-dns-source-ranges/check.sh

@@ -0,0 +1,146 @@
+#!/usr/bin/env bash
+# =============================================================================
+# L0 static check (offline, no cluster) — pihole-dns-source-ranges
+# Serves: R-DNS-03 (LAN split-horizon DNS — spec docs/09).
+#
+# THE BUG CLASS THIS CLOSES. PiHole's LoadBalancer frontends are the only thing stopping it being
+# an open resolver, and their `loadBalancerSourceRanges` is pulled in two opposite directions:
+#   * Too WIDE, and the resolver is open: a public range, or 0.0.0.0/0, in this list is the whole
+#     restriction gone, with nothing else in the path to catch it (the CiliumNetworkPolicy admits
+#     `world` on :53, because that is the only peer that can describe post-LoadBalancer traffic).
+#   * Too NARROW, and VPN DNS dies: wg0 is a Cilium kube-proxy-replacement device, so a tunnel
+#     client's query is load-balanced with its 10.90.0.x source intact, ahead of the iptables
+#     MASQUERADE. A LAN-only list is predicted to drop it as DROP_NOT_IN_SRC_RANGE, and the
+#     tunnel would still report a healthy handshake.
+#
+# GIVEN the repository as git tracks it,
+# WHEN  every type=LoadBalancer Service in namespace `pihole` is collected and each source range
+#       is resolved through cluster-vars (base value AND the prod overlay's replacement),
+# THEN  (1) the frontends exist and cover both udp/53 and tcp/53;
+#       (2) every frontend lists ${LAN_CIDR} and ${WG_CLIENT_CIDR};
+#       (3) every range resolves to a valid IPv4 CIDR inside RFC1918 — so no 0.0.0.0/0, no public
+#           range, and no unresolvable token that could hide one;
+#   AND (4) the check is NON-VACUOUS: a real checkout and a floor on the assertion count.
+#
+# WHAT THIS DOES NOT PROVE: that a query over the tunnel is actually answered. That needs a
+# handshake from a real client; see platform/vpn-ingress/README.md.
+# =============================================================================
+set -euo pipefail
+DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+ROOT="$(cd "${DIR}/../../../.." && pwd)"        # repo root
+PYTHON="${PYTHON:-python3}"
+
+cd "${ROOT}"
+"${PYTHON}" - "${ROOT}" <<'PY'
+import ipaddress, re, subprocess, sys
+from pathlib import Path
+
+try:
+    import yaml
+except ImportError:                                  # pragma: no cover
+    print("check FAIL: PyYAML is required (same dependency as catalog/render.py)",
+          file=sys.stderr)
+    raise SystemExit(1)
+
+ROOT = Path(sys.argv[1])
+FAILURES: list[str] = []
+CHECKS = 0
+REQUIRED_TOKENS = ("LAN_CIDR", "WG_CLIENT_CIDR")
+RFC1918 = [ipaddress.ip_network(n) for n in ("10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16")]
+TOKEN = re.compile(r"^\$\{([A-Z0-9_]+)\}$")
+
+
+def check(cond, msg):
+    global CHECKS
+    CHECKS += 1
+    if not cond:
+        FAILURES.append(msg)
+
+
+def docs_of(rel: str):
+    try:
+        return [d for d in yaml.safe_load_all((ROOT / rel).read_text(errors="replace"))
+                if isinstance(d, dict)]
+    except Exception:
+        return []
+
+
+FILES = [p for p in subprocess.run(["git", "-C", str(ROOT), "ls-files", "-z"],
+                                   capture_output=True, text=True, check=True).stdout.split("\0")
+         if p]
+YAML_FILES = [f for f in FILES if re.search(r"\.ya?ml$", f)]
+check(len(YAML_FILES) > 50, f"only {len(YAML_FILES)} YAML files seen — not a real checkout")
+
+# ------------------------------------------------- every value a ${TOKEN} can take on this tree
+values: dict[str, set[str]] = {}
+for d in docs_of("clusters/base/cluster-vars.yaml"):
+    if d.get("kind") == "ConfigMap":
+        for k, v in (d.get("data") or {}).items():
+            values.setdefault(k, set()).add(str(v))
+for d in docs_of("clusters/prod/kustomization.yaml"):
+    for p in d.get("patches") or []:
+        ops = p.get("patch") if isinstance(p, dict) else None
+        ops = yaml.safe_load(ops) if isinstance(ops, str) else ops
+        for op in ops if isinstance(ops, list) else []:
+            m = re.fullmatch(r"/data/([A-Z0-9_]+)", str(op.get("path", "")))
+            if m and op.get("op") in ("add", "replace"):
+                values.setdefault(m.group(1), set()).add(str(op.get("value")))
+for tok in REQUIRED_TOKENS:
+    check(values.get(tok), f"clusters/base/cluster-vars.yaml defines no {tok} (R-DNS-03)")
+
+
+def assert_private(where: str, cidr: str):
+    try:
+        net = ipaddress.ip_network(cidr, strict=False)
+    except ValueError:
+        check(False, f"{where}: {cidr!r} is not a valid CIDR (R-DNS-03)")
+        return
+    check(net.version == 4 and any(net.subnet_of(r) for r in RFC1918),
+          f"{where}: {cidr} is not inside RFC1918, so it opens PiHole beyond the LAN and the VPN "
+          "(R-DNS-03)")
+
+
+# ---------------------------------------------------------------------------- the frontends
+frontends = []
+for rel in YAML_FILES:
+    for d in docs_of(rel):
+        md, spec = d.get("metadata") or {}, d.get("spec") or {}
+        if d.get("kind") == "Service" and md.get("namespace") == "pihole" \
+                and spec.get("type") == "LoadBalancer":
+            frontends.append((rel, md.get("name"), spec))
+
+protocols = {(p.get("protocol", "TCP"), p.get("port"))
+             for _, _, spec in frontends for p in spec.get("ports") or []}
+for want in (("UDP", 53), ("TCP", 53)):
+    check(want in protocols, f"no PiHole LoadBalancer Service serves {want[0]}/{want[1]} — this "
+                             "check's corpus has moved and must be re-derived (R-DNS-03)")
+
+for rel, name, spec in frontends:
+    where = f"{rel}: Service/{name}"
+    ranges = spec.get("loadBalancerSourceRanges") or []
+    check(ranges, f"{where} has no loadBalancerSourceRanges, so it answers anyone who can reach "
+                  "the VIP (R-DNS-03)")
+    tokens = {m.group(1) for m in (TOKEN.match(str(r)) for r in ranges) if m}
+    for tok in REQUIRED_TOKENS:
+        check(tok in tokens, f"{where} does not list ${{{tok}}} in loadBalancerSourceRanges "
+                             "(R-DNS-03)")
+    for r in ranges:
+        m = TOKEN.match(str(r))
+        if not m:
+            assert_private(where, str(r))
+            continue
+        resolved = values.get(m.group(1))
+        check(resolved, f"{where}: {r} is not defined in cluster-vars, so its value cannot be "
+                        "shown to be private (R-DNS-03)")
+        for v in sorted(resolved or []):
+            assert_private(f"{where} ({r})", v)
+
+check(CHECKS >= 15, f"only {CHECKS} assertions ran — this check has gone vacuous (R-DNS-03)")
+
+if FAILURES:
+    for f in FAILURES:
+        print(f"check FAIL: {f}", file=sys.stderr)
+    raise SystemExit(1)
+print(f"check OK: pihole-dns-source-ranges — {CHECKS} assertions over "
+      f"{len(frontends)} PiHole LoadBalancer Service(s)")
+PY

decisions/2026-11-08-wg-client-mtu-docs.md

@@ -0,0 +1,74 @@
+---
+date: 2026-11-08
+branch: fix/wg-client-mtu-docs
+title: WireGuard client profiles default to MTU 1280; VPN runbook corrected and given a troubleshooting section
+---
+
+## What changed
+
+- **`infra/vpn-ingress/new-peer.sh`**: `WG_MTU` default 1420 -> 1280, still overridable, reason in a
+  comment. Only new profiles change; the phone's existing profiles are not touched.
+- **`runbooks/vpn-remote-access.md`**: the verification table no longer says the Gateway sees a
+  `host`-identity source; the limitations no longer say everything arrives SNATed or that IPv6 leaks
+  on Android; the "if nothing handshakes" advice no longer claims `wg show` separates a bad forward
+  from a bad key. New "Troubleshooting" section covering the four cases in the brief.
+- **`platform/vpn-ingress/daemonset.yaml`**: header comment only, SNAT claim corrected.
+- **`platform/vpn-ingress/README.md`**: one sentence, so `MTU = 1420` reads as the server's value.
+
+Rollout on merge: the WireGuard DaemonSet does **not** roll. The `runbooks` ConfigMap is content
+hashed and `vpn-remote-access.md` is in it, so the runbooks site rolls.
+
+Left unchanged, for the operator:
+
+- **`platform/vpn-ingress/peers.conf`** still says "No peers enrolled yet". A comment-only edit on a
+  throwaway copy moved the ConfigMap from `wg-peers-29mf6k867t` to `wg-peers-gggkk5d4tm` and changed
+  the DaemonSet, so correcting it restarts every `wg0`. Fix it alongside the next peer change.
+- **`clusters/base/cluster-vars.yaml`** coverage comment still omits `WG_CLIENT_CIDR` (sensitive path).
+- **Server `wg0` MTU stays 1420**, because changing it restarts all WireGuard pods. For TCP a 1280
+  client should be enough on its own: it advertises an MSS of 1240, which caps what the far end
+  sends. That protection does not reach UDP. A server-to-phone datagram with an inner size of
+  1281-1420 bytes is still sent as up to 1480 outer bytes and could be lost on the same path; QUIC
+  should recover by probing, large DNS answers are the likeliest casualty. If that shows up, the
+  fix is a lower server MTU, as a deliberate rollout.
+- **Other stale text, outside this brief:** `platform/vpn-ingress/README.md` still says no peer has
+  ever completed a handshake, and the runbook's status line still calls the VPN inert.
+
+## What I verified
+
+- Upstream `wireguard-android` `GoBackend.java`, master (last commit to that file `ecc3c0a`,
+  2026-03-15): `builder.setMtu(config.getInterface().getMtu().orElse(1280))`, and `allowFamily()` is
+  skipped when `sawDefaultRoute && config.getPeers().size() == 1`.
+- Live, read-only, 2026-11-09 UTC, on the `k3s-prod-1` pod and agent:
+  - `wg0` MTU is 1420 on all three pods. Both peers have handshakes and transfer on prod-1 only; the
+    other two pods read `0`.
+  - BPF conntrack holds `UDP SVC 10.90.0.x -> 192.168.2.23:53` and flows to `192.168.2.24:32400`.
+    Their backend entries carry `SourceSecurityID=2` (`world`). So the source stays `10.90.0.x`
+    through the load balancer on those two VIPs.
+  - `/proc/net/nf_conntrack` is readable from the pod, which has no `conntrack` binary.
+- Offline, pinned toolbox: `kubectl kustomize platform/vpn-ingress` gives identical ConfigMap and
+  DaemonSet hashes on `origin/main` and on this branch.
+- `new-peer.sh` writes `MTU = 1280` by default and `MTU = 1420` with `WG_MTU=1420`. Run with
+  throwaway keys and stdout discarded; profiles shredded; nothing enrolled.
+- No test or conformance check pins the client MTU (`grep` for `WG_MTU` and `1420`).
+- Gate (pinned toolbox, from this worktree, on the committed tree): **rc 3, UNDETERMINED, not green.**
+  `gitea-settings` is the only stage counted as not run (no token locally). `conformance` (L2/L3)
+  also skipped as the declared no-cluster gap, so R-ENV-01 and R-OBS-04 are not certified.
+  `spec-immutability`, `approval-policy`, `decisions-log`, `lint`, `l0`, `render-golden` and `l1`
+  passed. No sensitive path is touched.
+
+## What I did NOT verify
+
+- **That MTU was the cause.** It is inferred from one before-and-after, on one phone and one carrier,
+  reported by the operator. The path MTU was never measured.
+- **That 1280 is the best value.** It is the value that worked, and the IPv6 minimum. A higher one
+  may work too.
+- That the phone's app is on `GoBackend` rather than the kernel backend, so that a blank field meant
+  1280 on this device. Only the upstream default was read.
+- That TCP MSS behaves as described, or that large UDP replies are actually lost. No packet capture
+  was taken.
+- That the Gateway VIP `.20` sees `10.90.0.x` / `world`. Conntrack held no tunnel flows to it when
+  checked.
+- Android's IPv6 blocking on a real device; read from source only.
+- `wg-quick` IPv6 behaviour on Linux, and any non-Android client at 1280.
+- `/proc/net/nf_conntrack` output during a key mismatch. The `[UNREPLIED]` reading comes from the
+  earlier measurement in the brief; I saw only healthy entries.

decisions/2026-11-08-wg-home-peer-pihole-sources.md

@@ -0,0 +1,103 @@
+---
+date: 2026-11-08
+branch: fix/wg-home-peer-pihole-sources
+title: Enrol keith-phone-home and admit the WireGuard subnet to PiHole's DNS frontends
+---
+
+## What changed
+
+- **`platform/vpn-ingress/peers.conf`**: new peer `keith-phone-home`, `10.90.0.3/32`, key generated
+  on the phone. It is the `home` half of a two-tunnel profile (Endpoint `192.168.2.10:51820`),
+  because the router has no hairpin NAT and `wan.` is unusable from inside the house.
+- **`apps/wave-0/pihole-dns.yaml`**: `${WG_CLIENT_CIDR}` added after `${LAN_CIDR}` on both
+  `pihole-dns-udp` and `pihole-dns-tcp`. This file is hand-written, not rendered (`catalog/render.py`
+  only lists it in `EXTRA_UNITS`), so it is the source of truth and `--emit-waves` was not run.
+  `catalog/pihole.yaml` changes in one comment only, so nothing renders differently.
+- **New L0 guard `conformance/tests/l0-static/pihole-dns-source-ranges`** (whole-repo scope,
+  R-DNS-03): every PiHole LoadBalancer Service lists both tokens, and every range resolves through
+  cluster-vars (base and prod overlay) to RFC1918.
+- **`platform/vpn-ingress/README.md`**: corrected "clients arrive SNATed" (false for LoadBalancer
+  VIPs) and the IPv6 claim (Android's `GoBackend` blocks IPv6 for a single-peer default route);
+  documented the home/away pattern, why `wan.` fails from inside, and the prod-1 dependency.
+
+Rollout on merge: the `wg-peers` ConfigMap hash changes (`m9kcckcb5t` -> `29mf6k867t`), so the
+DaemonSet rolls, `maxUnavailable: 1`, `maxSurge: 0`, one node at a time in an order the controller
+chooses. Each roll deletes and recreates that node's `wg0`, so an established session on that node
+drops and must re-handshake. `keith-phone` has no session today (see below), so nothing is
+interrupted in practice.
+
+**The one risk worth watching:** `wg0` is in Cilium's auto-detected device list and `devices` is
+still unpinned. On 2026-10-06 a crash-looping `wg0` churned that list and all three nodes lost their
+main-table default route. A rollout is one delete/create per node, not a loop, so it should not
+repeat that, but after merge check `ip route get 1.1.1.1` on each node.
+
+The Service change is an in-place update: same VIP, same Service objects, no pod restart.
+
+Checked on the rest of the path and left unchanged:
+
+- Gateway VIP `.20` (`gateway/cilium-gateway-shared`): no source ranges, live. Nothing needed.
+- `pihole-allow-lan-dns-ingress` admits `world` on :53, which covers an untranslated 10.90.0.x
+  source. `platform/vpn-ingress/networkpolicy.yaml` does not apply to a hostNetwork pod.
+- No L0 check pinned these ranges to the LAN; none needed updating.
+
+Flagged, not changed (out of scope):
+
+- `runbooks/vpn-remote-access.md` "Known limitations" repeats both corrected claims, and step 3 of
+  its verification table says the Gateway sees a `host`-identity source.
+- `platform/vpn-ingress/daemonset.yaml`'s header repeats the SNAT claim.
+- `clusters/base/cluster-vars.yaml`'s substitution-coverage comment does not list
+  `WG_CLIENT_CIDR` under apps-wave-0. The variable is defined and substituted there regardless.
+- `peers.conf` still says "No peers enrolled yet" above the enrolled peers.
+- PiHole's NodePorts are not covered by source ranges (already acknowledged in `pihole-dns.yaml`).
+  Tunnel clients can reach node addresses, so they can reach those too; they are enrolled devices.
+- Pinning Cilium's `devices` remains the open hardening for the churn risk above.
+
+## What I verified
+
+Live, read-only, 2026-11-08:
+
+- `cilium-dbg status` on one agent: `KubeProxyReplacement: True [eno1 ..., wg0 10.90.0.1, tun0 ...]`
+  and `Masquerading: BPF [eno1, wg0, tun0]`. So `wg0` really is a load-balancing device.
+- `pihole-dns-tcp` and `-udp` carry `ranges=[192.168.2.0/24]`; no other LoadBalancer Service has
+  source ranges. Live `cluster-vars`: `WG_CLIENT_CIDR=10.90.0.0/24`, `LAN_CIDR=192.168.2.0/24`.
+- `wg show wg0 latest-handshakes` on all three pods: `keith-phone` = `0`.
+- `wan.home.keithjackson.dev` resolves publicly to a single A record (via 1.1.1.1).
+
+Offline, in the pinned toolbox:
+
+- `flux envsubst --strict` renders both Services to `[192.168.2.0/24, 10.90.0.0/24]`.
+- `kubectl kustomize platform/vpn-ingress`: `wg-peers-m9kcckcb5t` on the old `peers.conf`,
+  `wg-peers-29mf6k867t` on the new one.
+- Guard mutation test, each restored afterwards. Baseline passes (22 assertions, 2 Services). All
+  seven fail with rc 1 and a named message: WG range dropped from one Service; `0.0.0.0/0` added;
+  `8.8.8.0/24` added; `WG_CLIENT_CIDR` made public in cluster-vars; prod overlay sets `LAN_CIDR` to
+  `0.0.0.0/0`; ranges block removed; undefined token added.
+- The Android claim was read from upstream `GoBackend.java` (`wireguard-android`, master): the
+  `allowFamily` calls are skipped when `sawDefaultRoute && config.getPeers().size() == 1`.
+- Gate (pinned toolbox, from this worktree, on the committed tree): **rc 3, UNDETERMINED, not
+  green.** `gitea-settings` is the only stage counted as not run (no token locally). `conformance`
+  (L2/L3) also skipped as the declared no-cluster gap. `spec-immutability`, `decisions-log`, `lint`,
+  `l0` (including the new guard), `render-golden` and `l1` passed. `approval-policy` reported N/A
+  offline; it will be red in CI until the operator overrides, because `conformance/**` is sensitive.
+- `git merge-tree` against `feat/pihole-explicit-lan-names` (#157): no conflicts.
+
+## What I did NOT verify
+
+- **`DROP_NOT_IN_SRC_RANGE` was NOT observed live.** It is a prediction from Cilium's source. Only
+  the `wg0` device membership was measured; the drop itself has never been seen.
+- **Full-tunnel behaviour was NOT observed live**, on either profile. **No handshake has ever been
+  recorded for any peer**: `keith-phone` reads `0` on all three pods (a counter that resets when a
+  pod restarts, last 22 days ago, so it covers only the time since; earlier entries here also
+  record none).
+- That tunnel DNS works after this change, including the reply path from a backend on another node
+  and from one on the ingress node. That needs the phone: `dig` over the tunnel, while watching
+  `cilium-dbg monitor --type drop`.
+- That the Gateway serves tunnel clients. Its source is predicted to be `world`, as for a LAN
+  browser, which is what the apps admit; not tested.
+- That the new peer's key is the one on the phone. I copied the public key from the brief.
+- That the rollout leaves node default routes intact. Reasoned from the 2026-10-06 incident being
+  a loop, not measured.
+- That a LAN client is unaffected while the Service's source ranges are updated. Expected, not
+  measured.
+- IPv6 behaviour on any non-Android client.
+- That the WAN forward targets prod-1: it is off-repo.

infra/vpn-ingress/new-peer.sh

@@ -53,6 +53,7 @@
 #   WG_ENDPOINT_HOST    default wan.home.keithjackson.dev
 #   WG_CLIENT_CIDR      default 10.90.0.0/24   (must match clusters/base/cluster-vars.yaml)
 #   WG_LAN_CIDR         default 192.168.2.0/24 (prod LAN_CIDR)
+#   WG_MTU              default 1280 (client tunnel MTU; see the comment on the default below)
 #   WG_DNS              default 192.168.2.23   (prod PIHOLE_DNS_VIP)
 #
 # REQUIRES: `wg`, `qrencode`, `sops` — all three in the dev container.
@@ -80,7 +81,14 @@ PEERS_FILE="${REPO_ROOT}/platform/vpn-ingress/peers.conf"
 # cannot resolve until it is already connected.
 : "${WG_ENDPOINT_HOST:=wan.home.keithjackson.dev}"
 : "${WG_LISTEN_PORT:=51820}"
-: "${WG_MTU:=1420}"
+# The CLIENT's tunnel MTU. Deliberately lower than the server's 1420 (daemonset.yaml), which only
+# describes the 1500-byte path at the node's end. A phone's path is the carrier's, often behind
+# carrier NAT, and is not known to carry 1500. Measured 2026-11-09, one Android phone on one
+# carrier: at 1420 the handshake was healthy but names and pages failed; clearing the app's MTU
+# field (its userspace backend then uses 1280) fixed it. 1280 is also the IPv6 minimum link MTU, so
+# it stays valid if the tunnel ever gains IPv6. For a device only ever on a wired or known-1500
+# path, WG_MTU=1420 recovers about 10% of per-packet payload.
+: "${WG_MTU:=1280}"
 
 SPLIT=0
 OUT_DIR=""

platform/vpn-ingress/README.md

@@ -42,8 +42,9 @@ no version string to match against a CVE, and no amplification to abuse.
 
 That is the whole security argument, and it is why the exposed port carries no IP allowlist (which
 would also have made roaming clients unusable — a phone's address changes with every cell tower).
-Compare `apps/wave-0/pihole-dns.yaml`, which *does* pin `loadBalancerSourceRanges` to the LAN: a
-resolver answers anybody, so it must be scoped by address. WireGuard answers nobody.
+Compare `apps/wave-0/pihole-dns.yaml`, which *does* pin `loadBalancerSourceRanges` to the LAN and
+this tunnel's client subnet: a resolver answers anybody, so it must be scoped by address. WireGuard
+answers nobody.
 
 What it does **not** protect against is a stolen device — anyone holding a device's private key
 *is* that device. That is what revocation is for: delete the stanza, merge, done (~1 min).
@@ -55,9 +56,9 @@ What it does **not** protect against is a stolen device — anyone holding a dev
  ┌──────────────┐  udp/51820   ┌───────────────┐  forward   ┌────────────────────────────┐
  │ WireGuard app│─────────────▶│ WAN 51820 ──▶ │───────────▶│ wg0 10.90.0.1/24           │
  │ 10.90.0.2/32 │   over the   │ node IP:51820 │            │   decrypt, then FORWARD    │
- └──────────────┘   internet   └───────────────┘            │   SNAT to the node's addr  │
+ └──────────────┘   internet   └───────────────┘            │  SNAT (LAN), eBPF LB (VIPs)│
                                                             └─────────────┬──────────────┘
-                                                                          │ as `host`/`remote-node`
+                                                                          │ translated per destination (below)
                                         ┌─────────────────────────────────┼──────────────────────┐
                                         ▼                                 ▼                      ▼
                                LAN 192.168.2.0/24            Gateway VIP .20 (16 web UIs)   WAN / internet
@@ -86,9 +87,21 @@ resolved, because `apps/wave-0/pihole-dns.yaml`'s ingress policy deliberately om
 entity. A VPN that looks up and is useless is the worst available outcome, and nothing in the
 manifests would have hinted at it.
 
-In the **host** netns there is no pod identity to carry. Forwarded client traffic is SNATed to the
-node's own address, so it reaches the Gateway, PiHole and every LAN host as `host`/`remote-node` —
-byte-for-byte what a LAN client looks like. Three further problems vanish with it:
+In the **host** netns there is no pod identity to carry, which is what avoids the 403s. Clients do
+**not**, however, all arrive SNATed to the node. Measured 2026-11-08: Cilium counts `wg0` among its
+kube-proxy-replacement devices (`cilium-dbg status`: `eno1`, `wg0`, `tun0`), so what a packet looks
+like depends on where it is going:
+
+| destination | handled by | source the destination sees |
+|---|---|---|
+| LAN hosts, the internet | kernel FORWARD, then the iptables MASQUERADE in [`daemonset.yaml`](./daemonset.yaml) | the node's address |
+| LoadBalancer VIPs (Gateway `.20`, PiHole `.23`, ...) | Cilium's eBPF load balancer, as the packet enters from `wg0`, **before** MASQUERADE | `10.90.0.x` at the VIP's source-range check, identity `world` |
+
+The second row is why PiHole's `loadBalancerSourceRanges` has to list `${WG_CLIENT_CIDR}` as well as
+the LAN: left LAN-only, tunnel DNS is predicted to die with `DROP_NOT_IN_SRC_RANGE`. That prediction
+and the row's source column come from reading Cilium's datapath source; **no peer has ever completed
+a handshake, so neither has been observed live.** The Gateway VIP carries no source ranges and needs
+no change. Three further problems vanish with the host netns:
 
 * `net.ipv4.ip_forward` is **already 1** on these nodes (`node_baseline` writes
   `/etc/sysctl.d/99-k3s-cilium.conf`). It is *not* in the kubelet's safe-sysctl allowlist, so a
@@ -99,7 +112,9 @@ byte-for-byte what a LAN client looks like. Three further problems vanish with i
   `conformance/tests/l0-static/l2-announced-vips-serve-from-any-node`) and no
   `externalTrafficPolicy` trade-off to make at all.
 * **No double encapsulation.** Client UDP lands on the node's own NIC, so the underlay is a flat
-  1500-byte path, not Cilium's 1450-byte VXLAN. That is what makes `MTU = 1420` simply correct.
+  1500-byte path, not Cilium's 1450-byte VXLAN. That is what makes `MTU = 1420` correct on the
+  server's `wg0`. It says nothing about a client's own path: `new-peer.sh` defaults client profiles
+  to `1280` (see "Troubleshooting" in `runbooks/vpn-remote-access.md`).
 
 `hostNetwork` is *permitted* here, and not by oversight:
 `platform/kyverno/policies/restrict-host-namespaces.yaml` matches `namespaceSelector:
@@ -163,6 +178,30 @@ internet path. To enforce less *server-side*, add filtering to the `PostUp` chai
 before the accept rules) — the FORWARD chain there is the only enforcement point, since no
 NetworkPolicy applies.
 
+## Phone profiles: one tunnel for home, one for away
+
+A phone that needs the LAN-only apps runs full tunnel (`AllowedIPs = 0.0.0.0/0`, `DNS = 192.168.2.23`)
+everywhere, so it needs **two tunnels**, each its own peer with its own key and address:
+
+| tunnel | peer | address | Endpoint | use when |
+|---|---|---|---|---|
+| `away` | `keith-phone` | `10.90.0.2` | `wan.home.keithjackson.dev:51820` | off the home network |
+| `home` | `keith-phone-home` | `10.90.0.3` | `192.168.2.10:51820` | on the home Wi-Fi |
+
+**`wan.` cannot work from inside the house.** It is an unproxied A record holding the house's WAN
+address (`platform/externaldns/README.md`), and the router has no hairpin NAT, so a LAN client that
+sends to it never comes back in through the port-forward. The Cloudflare Tunnel is no way round: it
+carries HTTP(S) to the Gateway and WireGuard is UDP to a node, so `wan.` has to stay DNS-only and
+`away` has to keep using it.
+
+**`home` depends on `k3s-prod-1`.** Its Endpoint is that node's address, with no failover: if
+prod-1 is down, `home` cannot handshake even though the other two listeners are healthy. The fix
+is client-side: edit the tunnel's Endpoint to `192.168.2.11` or `.12`, since every node answers the
+same keys. No server change and no new key is needed. The same dependency applies to `away` if the
+WAN forward points at prod-1.
+
+Android allows one active VPN at a time, so switch tunnels by hand when leaving or arriving.
+
 ## Operating it
 
 * **Enrol a device:** `./infra/vpn-ingress/new-peer.sh <name> <octet>` → paste the printed `[Peer]`
@@ -215,10 +254,13 @@ right way; one was a defect in my own manifest.
   handshake — and would be a real secret, forcing `peers.conf` into SOPS and costing the
   reviewability above. Revisit only if the threat model changes; if it does, the PSK belongs in the
   `wg-server-key` Secret.
-* **No IPv6.** The server offers no IPv6 tunnel address, so a full-tunnel client with native IPv6
-  still reaches IPv6 sites *outside* the tunnel. Functionally harmless, a privacy leak if the point
-  was to hide the client's network. Fix is an `fd..` ULA on `wg0` plus `::/0` in the client's
-  `AllowedIPs`; not done because nothing here needs it.
+* **No IPv6.** The server offers no IPv6 tunnel address and `new-peer.sh` emits only `0.0.0.0/0`.
+  What happens to a client's IPv6 then depends on the client. The **Android app does not leak it**:
+  its `GoBackend` skips `allowFamily()` when the config is one peer with a default route, so Android
+  blocks IPv6 for the VPN's duration and apps fall back to IPv4. Other clients are not covered by
+  that code; expect `wg-quick` on a dual-stack Linux host to leave IPv6 outside the tunnel. Fix is
+  an `fd..` ULA on `wg0` plus `::/0` in the client's `AllowedIPs`; not done because nothing here
+  needs it.
 * **No in-cluster-service reach beyond what the LAN has.** Clients reach cluster apps the way a LAN
   browser does — through the Gateway VIP. Pod IPs and `10.43.0.0/16` service IPs are not routed
   into the tunnel.

platform/vpn-ingress/daemonset.yaml

@@ -30,9 +30,12 @@
 # `cluster` entity, so VPN clients would not have been able to resolve `*.${BASE_DOMAIN}` either.
 # The VPN would have looked up and been useless, which is the worst failure shape available.
 #
-# In the host netns there is no pod identity to carry: forwarded client traffic is SNATed to the
-# NODE's address, so it reaches the Gateway, PiHole and every LAN host as `host`/`remote-node` —
-# exactly what a LAN client looks like. Three further problems disappear with it:
+# In the host netns there is no pod identity to carry, so a client never looks like a pod. What a
+# destination sees depends on how the packet is handled. LAN hosts and the internet: forwarded,
+# then MASQUERADEd to the NODE's address by the PostUp rule below. LoadBalancer VIPs: Cilium's load
+# balancer takes the packet as it enters from wg0, before that rule, so the source is still
+# 10.90.0.x with identity `world`, like a LAN browser (measured 2026-11-09 for PiHole .23 and Plex
+# .24; see README). Three further problems disappear with it:
 #   * net.ipv4.ip_forward is ALREADY 1 on these nodes (node_baseline writes it to
 #     /etc/sysctl.d/99-k3s-cilium.conf), so no pod sysctl is needed — and `net.ipv4.ip_forward`
 #     is NOT in the kubelet's safe-sysctl allowlist, so a pod-netns version of this would have

platform/vpn-ingress/peers.conf

@@ -90,3 +90,7 @@
 [Peer]
 PublicKey = pBRSHrRyG2hnsuZ6FvHfEzF8hOVTF5stHUb/7fW2Km8=
 AllowedIPs = 10.90.0.2/32
+# keith-phone-home — <owner>, enrolled 2026-11-08 (at-home profile, LAN Endpoint; no hairpin NAT)
+[Peer]
+PublicKey = AT8lBJHqrVkv8RL9n0qWDsh5nyl0e3qgrfTAaYKgBz0=
+AllowedIPs = 10.90.0.3/32

runbooks/vpn-remote-access.md

@@ -252,7 +252,7 @@ cellular, with Wi-Fi off, and check all four — each proves a different hop:
 |---|---|---|
 | 1 | tunnel shows a recent handshake in the WireGuard app | the WAN forward + the server key + the peer stanza |
 | 2 | `ping 192.168.2.1` (the router) | decrypt → FORWARD → LAN, and the SNAT return path |
-| 3 | open `https://plex.home.keithjackson.dev` | PiHole DNS over the tunnel **and** the Gateway accepting a `host`-identity source — the specific thing a pod-networked VPN would have 403'd |
+| 3 | open `https://plex.home.keithjackson.dev` | PiHole DNS over the tunnel **and** the Gateway admitting a tunnel client, which a pod-networked VPN would have 403'd. Expected source identity is `world`, like a LAN browser: measured 2026-11-09 on VIPs `.23` and `.24`, not yet observed on the Gateway's `.20` |
 | 4 | visit an IP-echo site; it should show `136.51.115.252` | full-tunnel WAN egress out the house, not the carrier |
 
 Then confirm server-side:
@@ -260,18 +260,97 @@ Then confirm server-side:
 ```bash
 ./infra/devcontainer/dev.sh run bash -c \
   'kubectl --context k3s-prod -n vpn-ingress exec ds/wireguard -- wg show'
-# a `latest handshake` under ~3 min, and non-zero transfer, on the expected peer.
+# a `latest handshake` under ~3 min, and non-zero transfer, on the expected peer. `ds/wireguard`
+# picks ONE pod; if it reads 0, ask the pod on the node holding the forward (see Troubleshooting).
 ```
 
 **If check 3 fails while 2 passes**, the tunnel is fine and the problem is DNS or the Gateway, not
 WireGuard. Check the client's `DNS = 192.168.2.23` and that PiHole is up.
 
-**If 1 and 2 pass but large transfers hang**, it is MTU. The profile ships `MTU = 1420` (1500 − 60
-bytes of WireGuard encapsulation); drop the client to `1280` and re-test.
+**If 1 and 2 pass but pages or large transfers hang**, suspect the client MTU; see
+Troubleshooting.
 
-**If nothing handshakes at all**, you cannot tell a wrong forward from a wrong key from outside —
-by design. Work inwards: `wg show` on the node holding the forward will show *zero* received bytes
-if the router is not delivering the packets, and a `handshake` with no completed session if it is.
+**If nothing handshakes at all**, you cannot tell a wrong forward from a wrong key from the
+device, by design. `wg show` cannot tell them apart either: it reads the same in both cases. The
+node's conntrack table can; see Troubleshooting.
+
+## Troubleshooting
+
+Ask the pod on **the node carrying the traffic**: the node the WAN forward names for `away`,
+`k3s-prod-1` for `home`. Sessions are per node, and the other two pods read `0`.
+
+```bash
+./infra/devcontainer/dev.sh run bash -c \
+  'kubectl --context k3s-prod -n vpn-ingress get pods -o wide'
+
+./infra/devcontainer/dev.sh run bash -c \
+  'kubectl --context k3s-prod -n vpn-ingress exec <pod> -c wireguard -- wg show wg0 latest-handshakes'
+# likewise: `wg show wg0 endpoints`, `wg show wg0 transfer`
+```
+
+**Use only those three subcommands to diagnose.** Never `wg showconf`, `wg show wg0 private-key` or
+`wg show wg0 preshared-keys`: they print key material into your terminal and transcript.
+
+| output | reading |
+|---|---|
+| `latest-handshakes`: `<public key>  0` | no handshake with this peer since the pod started |
+| `endpoints`: `<public key>  (none)` | the server has not accepted a packet from this peer since the pod started |
+| `transfer`: `<public key>  <rx>  <tx>` | bytes, as the server counts them |
+
+### The app shows TX, no RX and no "Latest handshake"
+
+**Symptom.** The app's transmit counter climbs and receive stays at zero. On the server,
+`endpoints` is `(none)` and `latest-handshakes` is `0` for that peer.
+
+**Cause.** A key mismatch: the server's public key in the profile, or the device's public key in
+`peers.conf`. WireGuard drops an initiation it cannot authenticate and sends nothing back, so
+neither end logs an error. Measured 2026-11-09: a mistyped server public key in the phone's profile.
+
+**Check.** First prove the packets arrive. The pod has no `conntrack` binary, but in `hostNetwork`
+it can read the host's table:
+
+```bash
+./infra/devcontainer/dev.sh run bash -c \
+  'kubectl --context k3s-prod -n vpn-ingress exec <pod> -c wireguard -- grep dport=51820 /proc/net/nf_conntrack'
+```
+
+- A line whose first tuple is `src=<the phone's public address> ... dport=51820`, marked
+  `[UNREPLIED]`: the datagrams reach the node and the server stays silent. It is the keys.
+- No such line: the packets are not arriving. Look at the router forward, the DNS name, or the
+  device's network.
+
+Then compare the keys **by copy and paste, never by eye and never retyped**: the server's public
+half (the first command in Step 2's "Record it in your PASSWORD MANAGER") against the profile's
+`PublicKey`, and the device's public key from the app against its stanza in `peers.conf`.
+
+### Works on Wi-Fi, stalls on mobile data, handshake healthy
+
+**Symptom.** `latest-handshakes` is recent and some traffic flows, but names or pages fail on
+mobile data. The same device is fine on Wi-Fi.
+
+**Cause.** Most likely a client MTU too large for the carrier's path: a tunnel MTU of `1420` means
+outer packets of up to 1480 bytes. This is inferred from one before-and-after, on one phone and one
+carrier (2026-11-09): `1420` failed and clearing the field fixed it. The path MTU itself was not
+measured.
+
+**Check.** In the Android WireGuard app, edit the tunnel, clear the `MTU` field or set it to `1280`,
+save, then turn the tunnel off and on. A blank field means `1280` in the app's default userspace
+backend (`GoBackend`: `getMtu().orElse(1280)`). On other clients set `MTU = 1280` in the profile.
+
+`new-peer.sh` now generates `MTU = 1280` unless `WG_MTU` overrides it. A profile already on a device
+does not change, so edit it by hand. If `1280` does not help, it is not the MTU: go back to
+Step 6's check 3.
+
+### The `away` profile cannot connect from home
+
+**Symptom.** On the home network, `away` never handshakes.
+
+**Cause.** Its Endpoint is `wan.home.keithjackson.dev`, the house's WAN address, and the router has
+no hairpin NAT. WireGuard cannot use the Cloudflare Tunnel instead.
+
+**Check.** Turn Wi-Fi off. If `away` handshakes on mobile data, the profile is fine: use the `home`
+tunnel indoors, as described in
+[Phone profiles: one tunnel for home, one for away](../platform/vpn-ingress/README.md#phone-profiles-one-tunnel-for-home-one-for-away).
 
 ## Revoking a device
 
@@ -293,10 +372,17 @@ top follow-up for this component.** Until then:
 
 ## Known limitations
 
-* **No IPv6 in the tunnel.** A full-tunnel client with native IPv6 still reaches IPv6 sites outside
-  the tunnel. Functionally harmless; a privacy leak if hiding the client's network was the point.
-* **Client IPs are not preserved** on the LAN side: everything arrives SNATed as the node's
-  address. A static route for `10.90.0.0/24` → the forwarded node, added on the router, would
-  preserve them — at the cost of another hand-maintained off-repo route.
+* **No IPv6 in the tunnel.** What a client's IPv6 does then depends on the client. In the Android
+  app's default `GoBackend`, a config with one peer and a default route skips `allowFamily()`, so
+  Android should block IPv6 while the tunnel is up and apps fall back to IPv4. That is read from
+  upstream source, not observed on a device. A `--split` profile has no default route, so its IPv6
+  stays outside the tunnel. Expect `wg-quick` on a dual-stack Linux host to leave IPv6 outside too
+  (not tested).
+* **Client IPs are only partly preserved.** LAN hosts and the internet see the node's address
+  (MASQUERADE). A static route for `10.90.0.0/24` → the forwarded node, added on the router, would
+  preserve them there, at the cost of another hand-maintained off-repo route. LoadBalancer VIPs are
+  different: Cilium handles the packet before MASQUERADE, so the source stays `10.90.0.x`, identity
+  `world` (BPF conntrack on prod-1, 2026-11-09, VIPs `.23` and `.24`). Any `loadBalancerSourceRanges`
+  must therefore list the tunnel subnet, as PiHole's do.
 * **Rollouts briefly drop the forwarded node.** `maxUnavailable: 1`, so one node at a time; clients
   re-handshake automatically (retry every 5s, rekey every 2 min).