docs: record Phase 3 egress steering validated end-to-end on dev (both gates pass)

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
jakman85andClaude Opus 4.8 committed 2026-08-24 08:23:57 -05:00
1 parent 8a555999ee
commit e14048e7fe
1 file changed
+25
+25
View File
@@ -663,3 +663,28 @@ one branch picked up the other's file; caught + cleaned during review. Next para
one, liftable from `platform/vpn-egress/pia-custom-ovpn.sops.yaml`). **VALIDATION FLOW** (post-prereq): run the
playbook → label `k3s-dev-3` `egress-gateway=true` → add `egress-steering.yaml` to the kustomization → run
`egress-class` + `vpn-killswitch` conformance → then retire gluetun in a separate change.
### Phase 3 egress steering — VALIDATED END-TO-END on dev (2026-08-24)
Operator provisioned the host vault; PM drove the deployment + validation on `k3s-dev-3`:
- Playbook ran (after one AUTH_FAILED — operator had a wrong PIA password in the vault, then fixed it; my
earlier vault recipe put the password in YAML *double* quotes, a special-char hazard — use single quotes /
programmatic lift next time). Tunnel up: `tun0` 10.72.193.53, VPN egress **158.173.241.36** (PIA Stockholm).
- **Split-tunnel invariants hold:** node default unchanged (`via 10.73.0.1 dev ens3`), node's own egress =
**WAN 136.51.115.252**, node Ready throughout. `ip rule 5000: from <tun0-addr> lookup vpn_egress`, dedicated
table (tun0 default + blackhole), nft set scoped to the tun0 src — all correct.
- **Steering wired via GitOps:** added `egress-steering.yaml` to `platform/vpn-egress/kustomization.yaml`
(fixed a self-inflicted break first — the file had been dropped from main by the earlier gateway-role merge;
restored from 38fcfad). Flux reconciled (`8a55599`); `CiliumEgressGatewayPolicy vpn-egress-steering` live.
- **egress-class gate PASS:** `egress-class=vpn` pod → 158.173.241.36 (VPN), `egress-class=internet` pod →
136.51.115.252 (WAN). HTTP ×5 + HTTPS ×3 all 200 (early timeouts were post-restart warmup).
- **vpn-killswitch gate PASS (R-NET-06 fault-injection):** stopped the tunnel → `tun0` gone, ip rule removed,
nft set flushed, blackhole retained → vpn-class pod **timed out (no WAN leak)**, internet-class unaffected;
restart recovered cleanly.
- **FINDING (robustness, feeds WS-NET):** fail-closed currently rides on the stale SNAT source (10.72.193.53)
being non-routable, because the **down-script flushes the nft set + removes the ip rule** — disarming the
scoped drops exactly when the tunnel is down. The up-script already flush+re-adds the set on connect, so the
down-script's `nft flush set` is redundant AND harmful: keeping the last tun0 src in the set makes the
existing `ip saddr @set oifname != tun0 drop` rule an *explicit, edge-independent* backstop. Fix = drop the
`nft flush set` line from `vpn-egress-down.sh.j2`. Gate passes without it, but this is defense-in-depth.
- **Cross-agent lesson reinforced:** parallel agents must use `isolation: worktree` (an earlier non-isolated
pair caused the dropped-file break above).