docs: record ADR-0015 impl (merged, not deployed) + escalation ruling + operator prereq

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
jakman85andClaude Opus 4.8 committed 2026-08-24 07:32:58 -05:00
1 parent 33d048d379
commit 83ea050e82
1 file changed
+35
+35
View File
@@ -628,3 +628,38 @@ pod→PIA `to-network FORWARDED`, no drops.)
+ `OPENVPN_MSSFIX=1350` for the real Cilium-VXLAN 1450 pod MTU (data channel); PIA default OpenVPN UDP port.
Re-enabled `vpn-egress`. **Lesson:** when gluetun "can't reach" a provider, suspect a stale bundled server
list FIRST — validate against the provider's live server list before blaming the network/MTU/ports.
### Phase 3 egress steering — ADR-0015 implemented (PM-orchestrated), NOT yet deployed (2026-08-24)
Architect amended the spec to **ADR-0015** (node-level host OpenVPN tunnel + Cilium Egress Gateway + host
nftables kill-switch; supersedes the in-cluster-Gluetun steering — a Gluetun pod's `tun0` can't be steered by
Cilium egress gateway, and hostNetwork Gluetun hijacks the node default route). Operator also fixed the
invariants: **default = local/WAN for everything; VPN strictly opt-in; kill-switch scoped to vpn-class only;
opposite fail directions** (internet fails open, vpn fails closed).
PM dispatched two **WS-NET** agents (feature branches, no deploy); PM reviewed vs the invariants + DoD and
merged to `main` (`33d048d`) — **code only, nothing deployed** (policy not in the kustomization, playbook not
run, working gluetun kept as fallback):
- `infra/ansible/roles/vpn_egress_gateway/` + `playbooks/vpn.egress.gateway.yml` — host OpenVPN systemd
(PIA/OpenVPN-UDP, creds from **Ansible vault**), **split-tunnel** (`route-noexec` + dedicated route table
51820 + `ip rule from <tun0-addr>` → node's main-table default untouched), **scoped nftables kill-switch**
(policy-accept so node/internet fail-open; tun0-sourced/marked traffic dropped when not egressing tun0 →
vpn fail-closed). Inventory group `vpn_egress_gateway` = `k3s-dev-3`.
- `platform/vpn-egress/egress-steering.yaml` — `CiliumEgressGatewayPolicy` (only `egress-class=vpn`, RFC1918
excluded, `interface: tun0`, gateway-node label). Held OUT of `kustomization.yaml` until validated.
PM review: verified the up-script, nft rules, and policy honor all four invariants. **Escalation ruled:** the
tunnel-*fully-down* independent backstop the agent proposed via a fwmark on the CiliumEgressGatewayPolicy is
**infeasible** (the policy has no fwmark field). Decision: keep the scoped nft + routing blackhole + inert
fwmark scaffolding, and **gate on the `vpn-killswitch` conformance test** (force `tun0` down → assert no leak,
empirically proving Cilium fails closed). Build a real backstop (host controller syncing vpn-pod IPs into the
nft set) ONLY if that test leaks — not over-engineered now.
**Cross-agent git hazard noted:** the two agents shared one working dir (I didn't use isolated worktrees), so
one branch picked up the other's file; caught + cleaned during review. Next parallel coding fan-out → use
`isolation: worktree`.
**OPERATOR PREREQ (T-3.2p):** provision `infra/ansible/inventory/group_vars/vpn_egress_gateway/vault.yml` with
`vault_vpn_egress_gateway_pia_{username,password,ovpn}` (the `.ovpn` = the proven `sweden-aes-128-cbc-udp-ip`
one, liftable from `platform/vpn-egress/pia-custom-ovpn.sops.yaml`). **VALIDATION FLOW** (post-prereq): run the
playbook → label `k3s-dev-3` `egress-gateway=true` → add `egress-steering.yaml` to the kustomization → run
`egress-class` + `vpn-killswitch` conformance → then retire gluetun in a separate change.