diff --git a/DECISIONS-LOG.md b/DECISIONS-LOG.md index 4c7678a..993cc0a 100644 --- a/DECISIONS-LOG.md +++ b/DECISIONS-LOG.md @@ -628,3 +628,38 @@ pod→PIA `to-network FORWARDED`, no drops.) + `OPENVPN_MSSFIX=1350` for the real Cilium-VXLAN 1450 pod MTU (data channel); PIA default OpenVPN UDP port. Re-enabled `vpn-egress`. **Lesson:** when gluetun "can't reach" a provider, suspect a stale bundled server list FIRST — validate against the provider's live server list before blaming the network/MTU/ports. + +### Phase 3 egress steering — ADR-0015 implemented (PM-orchestrated), NOT yet deployed (2026-08-24) +Architect amended the spec to **ADR-0015** (node-level host OpenVPN tunnel + Cilium Egress Gateway + host +nftables kill-switch; supersedes the in-cluster-Gluetun steering — a Gluetun pod's `tun0` can't be steered by +Cilium egress gateway, and hostNetwork Gluetun hijacks the node default route). Operator also fixed the +invariants: **default = local/WAN for everything; VPN strictly opt-in; kill-switch scoped to vpn-class only; +opposite fail directions** (internet fails open, vpn fails closed). + +PM dispatched two **WS-NET** agents (feature branches, no deploy); PM reviewed vs the invariants + DoD and +merged to `main` (`33d048d`) — **code only, nothing deployed** (policy not in the kustomization, playbook not +run, working gluetun kept as fallback): +- `infra/ansible/roles/vpn_egress_gateway/` + `playbooks/vpn.egress.gateway.yml` — host OpenVPN systemd + (PIA/OpenVPN-UDP, creds from **Ansible vault**), **split-tunnel** (`route-noexec` + dedicated route table + 51820 + `ip rule from ` → node's main-table default untouched), **scoped nftables kill-switch** + (policy-accept so node/internet fail-open; tun0-sourced/marked traffic dropped when not egressing tun0 → + vpn fail-closed). Inventory group `vpn_egress_gateway` = `k3s-dev-3`. +- `platform/vpn-egress/egress-steering.yaml` — `CiliumEgressGatewayPolicy` (only `egress-class=vpn`, RFC1918 + excluded, `interface: tun0`, gateway-node label). Held OUT of `kustomization.yaml` until validated. + +PM review: verified the up-script, nft rules, and policy honor all four invariants. **Escalation ruled:** the +tunnel-*fully-down* independent backstop the agent proposed via a fwmark on the CiliumEgressGatewayPolicy is +**infeasible** (the policy has no fwmark field). Decision: keep the scoped nft + routing blackhole + inert +fwmark scaffolding, and **gate on the `vpn-killswitch` conformance test** (force `tun0` down → assert no leak, +empirically proving Cilium fails closed). Build a real backstop (host controller syncing vpn-pod IPs into the +nft set) ONLY if that test leaks — not over-engineered now. + +**Cross-agent git hazard noted:** the two agents shared one working dir (I didn't use isolated worktrees), so +one branch picked up the other's file; caught + cleaned during review. Next parallel coding fan-out → use +`isolation: worktree`. + +**OPERATOR PREREQ (T-3.2p):** provision `infra/ansible/inventory/group_vars/vpn_egress_gateway/vault.yml` with +`vault_vpn_egress_gateway_pia_{username,password,ovpn}` (the `.ovpn` = the proven `sweden-aes-128-cbc-udp-ip` +one, liftable from `platform/vpn-egress/pia-custom-ovpn.sops.yaml`). **VALIDATION FLOW** (post-prereq): run the +playbook → label `k3s-dev-3` `egress-gateway=true` → add `egress-steering.yaml` to the kustomization → run +`egress-class` + `vpn-killswitch` conformance → then retire gluetun in a separate change.