0 Fork 0

Files

T

jakman85andClaude Opus 4.8 aafb4ca378 docs(vpn-egress): reflect ADR-0015 node-tunnel egress, retire gluetun narrative

The in-cluster Gluetun gateway is retired; VPN egress is now a node-level host
OpenVPN split-tunnel (Ansible role vpn_egress_gateway) steered by a Cilium
Egress Gateway policy with a host nftables kill-switch (ADR-0015). Credentials
moved host-side to the Ansible vault. Update docs/comments only:

- platform/vpn-egress/README.md: rewrite for ADR-0015 (live + conformance-green);
  drop the resolved "option 1 not viable with Gluetun" finding to a brief history note.
- platform/kustomization.yaml: Phase 3 comment -> node tunnel + Cilium steering.
- platform/kyverno/.../require-egress-class-label.yaml: ADR-0013 -> ADR-0015 steering.
- schemas/servicedefinition.schema.json: egress=vpn description -> node tunnel + Cilium.
- clusters/base/cluster-vars.yaml: reframe PIA_REGION / CLUSTER_POD_CIDR WHY comments
  to ADR-0015 (Cilium excludedCIDRs); both keys retained (still referenced/documented).
- platform/vpn-egress/namespace.yaml: no privileged gateway pod under ADR-0015.
- pia-credentials.sops.yaml / pia-custom-ovpn.sops.yaml: comment headers only note
  they are now UNUSED (creds host-side), retained pending cleanup; payloads untouched.

No runtime manifest behavior, alert exprs, kustomization resources, or SOPS payloads
changed. No spec-repo edits.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

2026-08-24 14:50:08 -05:00

3.7 KiB

Raw Permalink Blame History

platform/vpn-egress — VPN egress steering (Phase 3, ADR-0015)

vpn-class services (sonarr, radarr, jackett, qbittorrent) egress only through a PIA VPN tunnel; internet-class services go out the normal WAN. Serves R-NET-05/06/07.

Egress is realized per ADR-0015 (spec docs/03): a node-level host OpenVPN split-tunnel on a labeled egress-gateway node, steered by a Cilium Egress Gateway policy, with a host nftables kill-switch. There is no in-cluster gateway pod — the tunnel is host infrastructure, not a Flux workload.

STATUS: live + conformance-green. The node tunnel was validated end-to-end; the egress-class (R-NET-05/07) and vpn-killswitch (R-NET-06) conformance tests pass against the live cluster.

How it works (ADR-0015)

Three layers, two owners:

Tunnel — host OpenVPN (Ansible). A labeled egress-gateway node runs a host OpenVPN systemd service (PIA over OpenVPN-UDP — provider/protocol still per ADR-0013). It is split-tunnel (route-noexec + policy routing): it creates tun0 but never becomes the node default route, so the node's own traffic (control plane, image pulls, other pods) stays on the WAN. Managed by the Ansible role infra/ansible/roles/vpn_egress_gateway/, like k3s itself — not a cluster workload.
Steering — Cilium Egress Gateway (GitOps). egress-steering.yaml holds a CiliumEgressGatewayPolicy that selects pods labeled homelab.keithjackson.dev/egress-class=vpn (Kyverno-enforced) and SNATs them out the gateway node's tun0. Native Cilium eBPF — no client-pod changes, no sidecar, no overlay. excludedCIDRs keeps RFC1918 / cluster / LAN traffic on the local datapath; only public destinations leave via the tunnel.
Kill-switch — host nftables (Ansible). A host nftables rule on the gateway node DROPS the steered vpn-class traffic whenever tun0 is down (fail-closed, R-NET-06), scoped to vpn-class only so internet-class and node traffic are unaffected.

Secrets — host-side now

PIA credentials + the .ovpn live on the gateway node via Ansible vault (see spec docs/07), not a cluster SOPS Secret. SOPS still governs in-cluster secrets; VPN egress simply no longer needs one.

The pia-credentials.sops.yaml and pia-custom-ovpn.sops.yaml secrets in this directory are now UNUSED — leftovers from the retired Gluetun pod, retained pending a separate cleanup change.

Pieces in this directory

namespace.yaml — the vpn-egress namespace (now holds only the Cilium policy + the retained, unused secrets; no privileged gateway pod under ADR-0015).
egress-steering.yaml — the CiliumEgressGatewayPolicy (the steering datapath).
pia-credentials.sops.yaml, pia-custom-ovpn.sops.yaml — unused, retained pending cleanup.

History (brief)

ADR-0013 originally placed the VPN in a single shared Gluetun pod and deferred the steering mechanism. Implementation proved that a Gluetun-in-a-pod tunnel cannot be a Cilium Egress Gateway target: the policy SNATs out a node interface, but the pod's tun0 lives in the pod netns, and running Gluetun hostNetwork to expose tun0 on the node hijacks the node's default route (breaking k3s/etcd/kubelet). ADR-0015 resolved this by moving the tunnel to the node (host OpenVPN split-tunnel) and steering with Cilium's native egress gateway. See ADR-0015 for the full rationale.

Port forwarding (qBittorrent) — follow-up

PIA assigns a forwarded port that can change on reconnect. Under ADR-0015 this is re-solved host-side (a small pf/natpmp helper on the gateway node that publishes the assigned port to the app) — Phase 6, alongside the qBittorrent app.

3.7 KiB Raw Permalink Blame History