diff --git a/DECISIONS-LOG.md b/DECISIONS-LOG.md index 993cc0a..4f75cfc 100644 --- a/DECISIONS-LOG.md +++ b/DECISIONS-LOG.md @@ -663,3 +663,28 @@ one branch picked up the other's file; caught + cleaned during review. Next para one, liftable from `platform/vpn-egress/pia-custom-ovpn.sops.yaml`). **VALIDATION FLOW** (post-prereq): run the playbook → label `k3s-dev-3` `egress-gateway=true` → add `egress-steering.yaml` to the kustomization → run `egress-class` + `vpn-killswitch` conformance → then retire gluetun in a separate change. + +### Phase 3 egress steering — VALIDATED END-TO-END on dev (2026-08-24) +Operator provisioned the host vault; PM drove the deployment + validation on `k3s-dev-3`: +- Playbook ran (after one AUTH_FAILED — operator had a wrong PIA password in the vault, then fixed it; my + earlier vault recipe put the password in YAML *double* quotes, a special-char hazard — use single quotes / + programmatic lift next time). Tunnel up: `tun0` 10.72.193.53, VPN egress **158.173.241.36** (PIA Stockholm). +- **Split-tunnel invariants hold:** node default unchanged (`via 10.73.0.1 dev ens3`), node's own egress = + **WAN 136.51.115.252**, node Ready throughout. `ip rule 5000: from lookup vpn_egress`, dedicated + table (tun0 default + blackhole), nft set scoped to the tun0 src — all correct. +- **Steering wired via GitOps:** added `egress-steering.yaml` to `platform/vpn-egress/kustomization.yaml` + (fixed a self-inflicted break first — the file had been dropped from main by the earlier gateway-role merge; + restored from 38fcfad). Flux reconciled (`8a55599`); `CiliumEgressGatewayPolicy vpn-egress-steering` live. +- **egress-class gate PASS:** `egress-class=vpn` pod → 158.173.241.36 (VPN), `egress-class=internet` pod → + 136.51.115.252 (WAN). HTTP ×5 + HTTPS ×3 all 200 (early timeouts were post-restart warmup). +- **vpn-killswitch gate PASS (R-NET-06 fault-injection):** stopped the tunnel → `tun0` gone, ip rule removed, + nft set flushed, blackhole retained → vpn-class pod **timed out (no WAN leak)**, internet-class unaffected; + restart recovered cleanly. +- **FINDING (robustness, feeds WS-NET):** fail-closed currently rides on the stale SNAT source (10.72.193.53) + being non-routable, because the **down-script flushes the nft set + removes the ip rule** — disarming the + scoped drops exactly when the tunnel is down. The up-script already flush+re-adds the set on connect, so the + down-script's `nft flush set` is redundant AND harmful: keeping the last tun0 src in the set makes the + existing `ip saddr @set oifname != tun0 drop` rule an *explicit, edge-independent* backstop. Fix = drop the + `nft flush set` line from `vpn-egress-down.sh.j2`. Gate passes without it, but this is defense-in-depth. +- **Cross-agent lesson reinforced:** parallel agents must use `isolation: worktree` (an earlier non-isolated + pair caused the dropped-file break above).