# platform/vpn-egress — VPN egress steering (Phase 3, ADR-0015) `vpn`-class services (sonarr, radarr, jackett, qbittorrent) egress **only** through a PIA VPN tunnel; `internet`-class services go out the normal WAN. Serves **R-NET-05/06/07**. Egress is realized per **[ADR-0015](../../../home-network-k3s/decisions/adr-0015-vpn-egress-steering-node-tunnel-cilium.md)** (spec `docs/03`): a **node-level host OpenVPN split-tunnel** on a labeled egress-gateway node, steered by a **Cilium Egress Gateway** policy, with a **host nftables kill-switch**. There is **no in-cluster gateway pod** — the tunnel is host infrastructure, not a Flux workload. > **STATUS: live + conformance-green.** The node tunnel was validated end-to-end; the `egress-class` > (R-NET-05/07) and `vpn-killswitch` (R-NET-06) conformance tests pass against the live cluster. ## How it works (ADR-0015) Three layers, two owners: - **Tunnel — host OpenVPN (Ansible).** A labeled egress-gateway node runs a **host OpenVPN systemd service** (PIA over OpenVPN-UDP — provider/protocol still per ADR-0013). It is **split-tunnel** (`route-noexec` + policy routing): it creates `tun0` but never becomes the node default route, so the node's own traffic (control plane, image pulls, other pods) stays on the WAN. Managed by the Ansible role **`infra/ansible/roles/vpn_egress_gateway/`**, like k3s itself — not a cluster workload. - **Steering — Cilium Egress Gateway (GitOps).** `egress-steering.yaml` holds a `CiliumEgressGatewayPolicy` that selects pods labeled `homelab.keithjackson.dev/egress-class=vpn` (Kyverno-enforced) and SNATs them out the gateway node's `tun0`. Native Cilium eBPF — no client-pod changes, no sidecar, no overlay. `excludedCIDRs` keeps RFC1918 / cluster / LAN traffic on the local datapath; only public destinations leave via the tunnel. - **Kill-switch — host nftables (Ansible).** A host nftables rule on the gateway node DROPS the steered `vpn`-class traffic whenever `tun0` is down (fail-closed, R-NET-06), scoped to vpn-class only so `internet`-class and node traffic are unaffected. ## Resilience — there is NO gateway failover (spec correction, escalated) **ADR-0015 §Decision says "Two egress-gateway nodes + Cilium Egress Gateway HA (gateway failover) provide the active/standby the earlier ADRs asked for", and spec `docs/03` repeats it. That feature does not exist in Cilium OSS.** The supported topology is **exactly one gateway node**. Verified against the CRD actually deployed on this cluster (`kubectl get crd ciliumegressgatewaypolicies.cilium.io -o json`, apiVersion `cilium.io/v2`, Cilium **v1.20.1**) — the field's own description: > **`egressGateway`** — "EgressGateway is the gateway node responsible for SNATing traffic. > **In case multiple nodes are a match for the given set of labels, the first node in lexical > ordering based on their name will be selected.**" Consequences, stated plainly: - **One gateway node is the supported topology.** `infra/ansible/inventory/hosts.ini`'s `[vpn_egress_gateway]` group is singleton by design. - **A second labelled node does NOT provide failover.** It is not a standby; it carries no traffic and takes over nothing when the first node dies. Gateway-node loss = `vpn`-class egress down (fail-closed, per R-NET-06 — no leak, but no service). - **Lexical ordering decides who wins if two are ever labelled.** Labelling `k3s-prod-1` while `k3s-prod-3` is the live gateway would **silently** move the role to `k3s-prod-1` — because `1` sorts before `3` — taking the tunnel and the public egress IP with it, with no event and no alert. - **The plural `egressGateways` field (Cilium 1.20) is not a workaround.** It *shards* endpoints across entries by CiliumEndpoint UID rather than running active/standby: losing one entry's node blackholes that entry's share and re-shards the remainder (breaking live connections), and each shard leaves from a *different* public IP. Upstream confirmation: [cilium/cilium#30157](https://github.com/cilium/cilium/issues/30157) ("egress gateway doesn't support failover between nodes") and [cilium/cilium#18230](https://github.com/cilium/cilium/issues/18230) ("egress gateway does not tolerate single node failure"). **Governance:** ADR-0015 and `docs/03` are specifications and are immutable to agents (spec `AGENTS.md` §11, R-TEST-06, enforced by `ci/spec-immutability.sh`) — they are **not** edited here. The correction is raised as an **escalation with a drafted ADR-0016 supersede** in `DECISIONS-LOG.md` (entry of 2026-09-06) for the operator/architect to accept. Until then this file and `egress-steering.yaml` are the accurate statement of what the implementation can actually do. ## Secrets — host-side now PIA credentials + the `.ovpn` live **on the gateway node via Ansible vault** (see spec `docs/07`), **not** a cluster SOPS Secret. SOPS still governs in-cluster secrets; VPN egress simply no longer needs one. The `pia-credentials.sops.yaml` and `pia-custom-ovpn.sops.yaml` secrets in this directory are **now UNUSED** — leftovers from the retired Gluetun pod, **retained pending a separate cleanup change**. ## Pieces in this directory - `namespace.yaml` — the `vpn-egress` namespace (now holds only the Cilium policy + the retained, unused secrets; no privileged gateway pod under ADR-0015). - `egress-steering.yaml` — the `CiliumEgressGatewayPolicy` (the steering datapath). - `pia-credentials.sops.yaml`, `pia-custom-ovpn.sops.yaml` — **unused**, retained pending cleanup. ## History (brief) ADR-0013 originally placed the VPN in a single shared **Gluetun pod** and deferred the steering mechanism. Implementation proved that a Gluetun-in-a-pod tunnel **cannot be a Cilium Egress Gateway target**: the policy SNATs out a *node* interface, but the pod's `tun0` lives in the pod netns, and running Gluetun `hostNetwork` to expose `tun0` on the node hijacks the node's default route (breaking k3s/etcd/kubelet). ADR-0015 resolved this by moving the tunnel to the **node** (host OpenVPN split-tunnel) and steering with Cilium's native egress gateway. See ADR-0015 for the full rationale. ## Port forwarding (qBittorrent) — follow-up PIA assigns a forwarded port that can change on reconnect. Under ADR-0015 this is re-solved **host-side** (a small pf/natpmp helper on the gateway node that publishes the assigned port to the app) — Phase 6, alongside the qBittorrent app.