From 1a98429434420a907ec7b101bb851561c44043ed Mon Sep 17 00:00:00 2001 From: Keith Jackson Date: Fri, 21 Aug 2026 22:19:22 -0500 Subject: [PATCH] =?UTF-8?q?feat(phase3):=20author=20shared=20VPN=20egress?= =?UTF-8?q?=20gateway=20(Gluetun/PIA/WireGuard)=20=E2=80=94=20author-first?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per ADR-0013: platform/vpn-egress/ = a single shared Gluetun gateway (PIA over WireGuard, built-in kill-switch FIREWALL=on, native port forwarding), namespace (non-app, NET_ADMIN + /dev/net/tun), control-API Service, and a pre-wired (commented) pia-credentials SOPS line. Defined the PIA secret location (platform/vpn-egress, keys username/password → OPENVPN_USER/ OPENVPN_PASSWORD) and updated the SOPS template accordingly. NOT wired into platform/kustomization.yaml (Phase 3, author-first) — deploy needs PIA creds + validation of the egress-steering mechanism (Cilium Egress Gateway → the Gluetun node's wg0; documented in README with options + a fail-closed rationale, deliberately NOT committed as an untested CiliumEgressGatewayPolicy). WireGuard chosen (UDP, protocol != TCP); PIA supports it on all regions and the secret stays username/password. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../pia-credentials.yaml.example | 24 ++--- platform/vpn-egress/README.md | 50 ++++++++++ platform/vpn-egress/gluetun-gateway.yaml | 99 +++++++++++++++++++ platform/vpn-egress/kustomization.yaml | 10 ++ platform/vpn-egress/namespace.yaml | 11 +++ 5 files changed, 179 insertions(+), 15 deletions(-) create mode 100644 platform/vpn-egress/README.md create mode 100644 platform/vpn-egress/gluetun-gateway.yaml create mode 100644 platform/vpn-egress/kustomization.yaml create mode 100644 platform/vpn-egress/namespace.yaml diff --git a/bootstrap/sops/secret-templates/pia-credentials.yaml.example b/bootstrap/sops/secret-templates/pia-credentials.yaml.example index bc4a103..df882ce 100644 --- a/bootstrap/sops/secret-templates/pia-credentials.yaml.example +++ b/bootstrap/sops/secret-templates/pia-credentials.yaml.example @@ -2,30 +2,24 @@ # TEMPLATE — NOT APPLIED, NOT ENCRYPTED (.yaml.example is ignored by kustomize # and never auto-encrypted by SOPS). Reference scaffold only. # -# (a) Copy to: (TO BE DECIDED AT PHASE 3 — VPN component not authored yet) -# (b) kustomization: DO NOT WIRE YET. -# (c) Encrypt: sops -e -i .sops.yaml (once the target path exists) -# (d) Consumed by: Phase-3 Gluetun VPN egress gateway. The consuming -# component AND its namespace DO NOT EXIST yet — this template -# is provisioned upfront. [Phase 3] -# -# !! NAMESPACE UNKNOWN — the metadata.namespace below is a placeholder. Set it to -# the VPN gateway's namespace when that component is authored at Phase 3. !! +# (a) Copy to: platform/vpn-egress/pia-credentials.sops.yaml +# (b) kustomization: uncomment `- pia-credentials.sops.yaml` in platform/vpn-egress/kustomization.yaml +# (c) Encrypt: sops -e -i platform/vpn-egress/pia-credentials.sops.yaml +# (d) Consumed by: the Phase-3 Gluetun VPN egress gateway (platform/vpn-egress/gluetun-gateway.yaml), +# which maps username->OPENVPN_USER, password->OPENVPN_PASSWORD. [Phase 3] # # VALUE NOTES: # - username: PIA (Private Internet Access) user, form pNNNNNNN. # - password: PIA password. -# - Gluetun typically reads OPENVPN_USER / OPENVPN_PASSWORD (OpenVPN) or -# WireGuard credentials. Confirm the chosen protocol when wiring — the -# requirement says NOT TCP. The env var names / key names the Gluetun -# deployment expects may differ from username/password below; adjust to match -# the chosen protocol before encrypting. +# - Protocol is WireGuard (VPN_TYPE=wireguard, UDP — satisfies "not TCP"). PIA on WireGuard STILL +# authenticates with the username/password below (Gluetun derives the WireGuard config from PIA's +# API at runtime), so these two keys are correct as-is — no extra WireGuard key material needed. # ============================================================================= apiVersion: v1 kind: Secret metadata: name: pia-credentials - namespace: REPLACE-ME-PHASE-3-VPN-NAMESPACE + namespace: vpn-egress type: Opaque stringData: username: REPLACE-ME diff --git a/platform/vpn-egress/README.md b/platform/vpn-egress/README.md new file mode 100644 index 0000000..bf27fee --- /dev/null +++ b/platform/vpn-egress/README.md @@ -0,0 +1,50 @@ +# platform/vpn-egress — shared VPN egress gateway (Phase 3, ADR-0013) + +Single shared **Gluetun** gateway: **PIA over WireGuard**, built-in kill-switch, native port +forwarding. `vpn`-class services (sonarr, radarr, jackett, qbittorrent) egress **only** through this +tunnel; `internet`-class services go out the normal WAN. Serves **R-NET-05/06/07**. + +> **STATUS: authored, NOT deployed.** Not wired into `platform/kustomization.yaml` yet. Deploy needs +> (a) the `pia-credentials` SOPS secret, and (b) validation of the egress-steering mechanism below, +> which cannot be tested without a live PIA tunnel. + +## Pieces +- `gluetun-gateway.yaml` — the Gluetun Deployment (`replicas: 1`, `Recreate`) + the `gluetun-control` + Service (control API :8000 for forwarded-port lookup + tunnel-state health). Kill-switch = `FIREWALL=on`. +- `pia-credentials` (SOPS) — PIA username/password only; Gluetun derives the WireGuard config at runtime. + +## Operator hand-off (when deploying Phase 3) +1. Provision `pia-credentials` (see `bootstrap/sops/secret-templates/pia-credentials.yaml.example`, + namespace `vpn-egress`, keys `username`/`password`) → `sops -e -i` → uncomment it in `kustomization.yaml`. +2. Set cluster-vars: `PIA_REGION` (a **port-forwarding-capable** PIA region — e.g. `Netherlands`, + `CA Toronto`, `Switzerland`; NOT a US region, which lack PF) and `CLUSTER_POD_CIDR` (k3s default + `10.42.0.0/16`). +3. Enable `- vpn-egress` in `platform/kustomization.yaml` (Phase 3). + +## Egress steering — the DESIGN DECISION (needs live validation) +The hard part: routing `vpn`-class pods through this ONE shared tunnel. Cilium Egress Gateway +(`CiliumEgressGatewayPolicy`) SNATs matched pods out of a **gateway node's interface** — it does not +natively route through a *pod's* tunnel. Options considered: + +1. **Cilium Egress Gateway → the Gluetun node's tunnel interface (recommended).** Pin Gluetun to a + node; a `CiliumEgressGatewayPolicy` selects pods labeled `homelab.keithjackson.dev/egress-class=vpn` + and egresses them via that node's WireGuard interface (`wg0`). If the tunnel drops, `wg0` is gone → + traffic has no egress path → **fail-closed** (the kill-switch behavior R-NET-06 needs). This is the + most Cilium-native fit but requires Gluetun's tunnel interface to be visible to the node datapath + (hostNetwork Gluetun, or a shared netns) — the detail to VALIDATE. +2. **Gateway-pod routing:** client pods default-route to the Gluetun pod IP (init-container + + NET_ADMIN, or a Multus secondary interface); Gluetun forwards+NATs via the tunnel. Works without + Cilium egress gateway but is more moving parts and pod-IP-fragile. +3. **Per-pod Gluetun sidecar** — explicitly REJECTED by ADR-0013 (single shared gateway preferred). + +**Plan:** validate option 1 against a live PIA tunnel (confirm `wg0` on the Gluetun node + a +`CiliumEgressGatewayPolicy` steering `egress-class=vpn` pods, then run the `egress-class` (R-NET-05/07) +and `vpn-killswitch` (R-NET-06) conformance tests). Fall back to option 2 if the interface isn't +addressable. The `CiliumEgressGatewayPolicy` is intentionally NOT committed until validated, to avoid +shipping an untested datapath policy. + +## Port forwarding (qBittorrent) — follow-up +PIA assigns a forwarded port that can change on reconnect. A small **port-sync helper** +(`qbittorrent-natmap`) reads the current port from the `gluetun-control` API (:8000 +`/v1/openvpn/portforwarded`) and writes it into qBittorrent, keeping it hands-off (R-UPD). Authored +alongside the qBittorrent app (Phase 6 apps / or here when qBittorrent lands). diff --git a/platform/vpn-egress/gluetun-gateway.yaml b/platform/vpn-egress/gluetun-gateway.yaml new file mode 100644 index 0000000..9ff0c6c --- /dev/null +++ b/platform/vpn-egress/gluetun-gateway.yaml @@ -0,0 +1,99 @@ +--- +# Gluetun — the single shared in-cluster VPN egress gateway (ADR-0013): PIA over WireGuard, with +# Gluetun's built-in kill-switch/firewall (fail-closed), health checks, auto-reconnect, and native +# port forwarding for qBittorrent. `vpn`-class pods are steered through this gateway (see README.md +# "Egress steering" — the Cilium-native mechanism, pending live validation with PIA creds). +# +# Credentials: the `pia-credentials` SOPS Secret (keys: username, password). Gluetun uses PIA's +# username/password (via OPENVPN_USER/OPENVPN_PASSWORD — legacy env names it uses for PIA on BOTH +# protocols) to authenticate to PIA's API and derive the WireGuard config at runtime — so WireGuard +# needs no extra key material (ADR-0013). +apiVersion: apps/v1 +kind: Deployment +metadata: + name: gluetun + namespace: vpn-egress + labels: + app: gluetun +spec: + replicas: 1 # ONE shared tunnel (ADR-0013). Active/standby is a later enhancement. + strategy: + type: Recreate # a tunnel is a singleton — never run two overlapping during a rollout + selector: + matchLabels: + app: gluetun + template: + metadata: + labels: + app: gluetun + spec: + containers: + - name: gluetun + image: qmcgaw/gluetun:v3.40.0 # pinned; Renovate bumps + securityContext: + capabilities: + add: ["NET_ADMIN"] # required for the tunnel + firewall + env: + - name: VPN_SERVICE_PROVIDER + value: "private internet access" + - name: VPN_TYPE + value: "wireguard" # UDP — satisfies "protocol != TCP" (R-NET); upgrade over OpenVPN + # PIA credentials from the SOPS secret (Gluetun's PIA creds env, used for WireGuard too): + - name: OPENVPN_USER + valueFrom: + secretKeyRef: + name: pia-credentials + key: username + - name: OPENVPN_PASSWORD + valueFrom: + secretKeyRef: + name: pia-credentials + key: password + - name: SERVER_REGIONS + value: "${PIA_REGION}" # cluster-vars: a PF-CAPABLE PIA region (ADR-0013 PF caveat) + - name: VPN_PORT_FORWARDING + value: "on" # PIA native port forwarding for qBittorrent + - name: FIREWALL + value: "on" # KILL-SWITCH: block all egress except the tunnel (fail-closed, R-NET-06) + - name: FIREWALL_OUTBOUND_SUBNETS + value: "${CLUSTER_POD_CIDR}" # allow steered client pods (pod CIDR) to route out via the tunnel + - name: DOT + value: "off" # DNS via cluster CoreDNS, not Gluetun's DoT + ports: + - name: control + containerPort: 8000 # Gluetun control API (health + forwarded-port lookup) + livenessProbe: + httpGet: { path: /v1/openvpn/status, port: 8000 } # Gluetun control API health + initialDelaySeconds: 20 + periodSeconds: 30 + resources: + requests: + cpu: 50m + memory: 64Mi + limits: + memory: 160Mi + volumeMounts: + - name: tun + mountPath: /dev/net/tun + volumes: + - name: tun + hostPath: + path: /dev/net/tun + type: CharDevice +--- +# Control API Service — the port-sync helper (qbittorrent-natmap, README) reads the PIA-assigned +# forwarded port from here; also used for tunnel-state health/alerting (ADR-0013 "alert on tunnel state"). +apiVersion: v1 +kind: Service +metadata: + name: gluetun-control + namespace: vpn-egress + labels: + app: gluetun +spec: + selector: + app: gluetun + ports: + - name: control + port: 8000 + targetPort: 8000 diff --git a/platform/vpn-egress/kustomization.yaml b/platform/vpn-egress/kustomization.yaml new file mode 100644 index 0000000..7f50742 --- /dev/null +++ b/platform/vpn-egress/kustomization.yaml @@ -0,0 +1,10 @@ +--- +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - namespace.yaml + - gluetun-gateway.yaml + # SOPS-encrypted PIA credentials (keys: username, password). Operator flow: + # copy bootstrap/sops/secret-templates/pia-credentials.yaml.example here as pia-credentials.sops.yaml + # (namespace: vpn-egress), fill + `sops -e -i`, then uncomment. See bootstrap/sops/CREDENTIALS.md. + # - pia-credentials.sops.yaml diff --git a/platform/vpn-egress/namespace.yaml b/platform/vpn-egress/namespace.yaml new file mode 100644 index 0000000..3e4ef14 --- /dev/null +++ b/platform/vpn-egress/namespace.yaml @@ -0,0 +1,11 @@ +--- +# vpn-egress — the single shared VPN egress gateway namespace (ADR-0013, docs/03). +# Platform INFRA, not an application namespace: the Gluetun gateway needs NET_ADMIN + /dev/net/tun, +# so this ns is deliberately NOT labeled `app-namespace=true` (Kyverno restrict-host-namespaces / +# require-*-label policies scope to app namespaces only — see platform/kyverno). +apiVersion: v1 +kind: Namespace +metadata: + name: vpn-egress + labels: + homelab.keithjackson.dev/app-namespace: "false"