jakman85andClaude Opus 4.8
d920aad74a
pia-credentials and pia-custom-ovpn were gluetun's PIA creds; gluetun is retired and the node tunnel uses host-side Ansible vault creds (docs/07), so these cluster Secrets are unused. Remove them from the kustomization + delete the encrypted files (recoverable from git history). Flux will prune the Secrets from the vpn-egress namespace. Co-Authored-By: Claude Opus 4.8 <[email protected]>
platform/vpn-egress — VPN egress steering (Phase 3, ADR-0015)
vpn-class services (sonarr, radarr, jackett, qbittorrent) egress only through a PIA VPN
tunnel; internet-class services go out the normal WAN. Serves R-NET-05/06/07.
Egress is realized per ADR-0015
(spec docs/03): a node-level host OpenVPN split-tunnel on a labeled egress-gateway node, steered
by a Cilium Egress Gateway policy, with a host nftables kill-switch. There is no in-cluster
gateway pod — the tunnel is host infrastructure, not a Flux workload.
STATUS: live + conformance-green. The node tunnel was validated end-to-end; the
egress-class(R-NET-05/07) andvpn-killswitch(R-NET-06) conformance tests pass against the live cluster.
How it works (ADR-0015)
Three layers, two owners:
- Tunnel — host OpenVPN (Ansible). A labeled egress-gateway node runs a host OpenVPN systemd
service (PIA over OpenVPN-UDP — provider/protocol still per ADR-0013). It is split-tunnel
(
route-noexec+ policy routing): it createstun0but never becomes the node default route, so the node's own traffic (control plane, image pulls, other pods) stays on the WAN. Managed by the Ansible roleinfra/ansible/roles/vpn_egress_gateway/, like k3s itself — not a cluster workload. - Steering — Cilium Egress Gateway (GitOps).
egress-steering.yamlholds aCiliumEgressGatewayPolicythat selects pods labeledhomelab.keithjackson.dev/egress-class=vpn(Kyverno-enforced) and SNATs them out the gateway node'stun0. Native Cilium eBPF — no client-pod changes, no sidecar, no overlay.excludedCIDRskeeps RFC1918 / cluster / LAN traffic on the local datapath; only public destinations leave via the tunnel. - Kill-switch — host nftables (Ansible). A host nftables rule on the gateway node DROPS the steered
vpn-class traffic whenevertun0is down (fail-closed, R-NET-06), scoped to vpn-class only sointernet-class and node traffic are unaffected.
Secrets — host-side now
PIA credentials + the .ovpn live on the gateway node via Ansible vault (see spec docs/07), not
a cluster SOPS Secret. SOPS still governs in-cluster secrets; VPN egress simply no longer needs one.
The pia-credentials.sops.yaml and pia-custom-ovpn.sops.yaml secrets in this directory are now
UNUSED — leftovers from the retired Gluetun pod, retained pending a separate cleanup change.
Pieces in this directory
namespace.yaml— thevpn-egressnamespace (now holds only the Cilium policy + the retained, unused secrets; no privileged gateway pod under ADR-0015).egress-steering.yaml— theCiliumEgressGatewayPolicy(the steering datapath).pia-credentials.sops.yaml,pia-custom-ovpn.sops.yaml— unused, retained pending cleanup.
History (brief)
ADR-0013 originally placed the VPN in a single shared Gluetun pod and deferred the steering
mechanism. Implementation proved that a Gluetun-in-a-pod tunnel cannot be a Cilium Egress Gateway
target: the policy SNATs out a node interface, but the pod's tun0 lives in the pod netns, and
running Gluetun hostNetwork to expose tun0 on the node hijacks the node's default route (breaking
k3s/etcd/kubelet). ADR-0015 resolved this by moving the tunnel to the node (host OpenVPN
split-tunnel) and steering with Cilium's native egress gateway. See ADR-0015 for the full rationale.
Port forwarding (qBittorrent) — follow-up
PIA assigns a forwarded port that can change on reconnect. Under ADR-0015 this is re-solved host-side (a small pf/natpmp helper on the gateway node that publishes the assigned port to the app) — Phase 6, alongside the qBittorrent app.