feat(phase3): author shared VPN egress gateway (Gluetun/PIA/WireGuard) — author-first

Per ADR-0013: platform/vpn-egress/ = a single shared Gluetun gateway (PIA over WireGuard, built-in kill-switch FIREWALL=on, native port forwarding), namespace (non-app, NET_ADMIN + /dev/net/tun), control-API Service, and a pre-wired (commented) pia-credentials SOPS line. Defined the PIA secret location (platform/vpn-egress, keys username/password → OPENVPN_USER/ OPENVPN_PASSWORD) and updated the SOPS template accordingly. NOT wired into platform/kustomization.yaml (Phase 3, author-first) — deploy needs PIA creds + validation of the egress-steering mechanism (Cilium Egress Gateway → the Gluetun node's wg0; documented in README with options + a fail-closed rationale, deliberately NOT committed as an untested CiliumEgressGatewayPolicy). WireGuard chosen (UDP, protocol != TCP); PIA supports it on all regions and the secret stays username/password. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-08-21 22:19:22 -05:00
1 parent e6c61c97b3
commit 1a98429434
@@ -2,30 +2,24 @@
 # TEMPLATE — NOT APPLIED, NOT ENCRYPTED (.yaml.example is ignored by kustomize
 # and never auto-encrypted by SOPS). Reference scaffold only.
 #
-# (a) Copy to:      (TO BE DECIDED AT PHASE 3 — VPN component not authored yet)
-# (b) kustomization: DO NOT WIRE YET.
-# (c) Encrypt:       sops -e -i <target>.sops.yaml   (once the target path exists)
-# (d) Consumed by:   Phase-3 Gluetun VPN egress gateway. The consuming
-#                    component AND its namespace DO NOT EXIST yet — this template
-#                    is provisioned upfront. [Phase 3]
-#
-# !! NAMESPACE UNKNOWN — the metadata.namespace below is a placeholder. Set it to
-#    the VPN gateway's namespace when that component is authored at Phase 3. !!
+# (a) Copy to:      platform/vpn-egress/pia-credentials.sops.yaml
+# (b) kustomization: uncomment `- pia-credentials.sops.yaml` in platform/vpn-egress/kustomization.yaml
+# (c) Encrypt:       sops -e -i platform/vpn-egress/pia-credentials.sops.yaml
+# (d) Consumed by:   the Phase-3 Gluetun VPN egress gateway (platform/vpn-egress/gluetun-gateway.yaml),
+#                    which maps username->OPENVPN_USER, password->OPENVPN_PASSWORD. [Phase 3]
 #
 # VALUE NOTES:
 #  - username: PIA (Private Internet Access) user, form pNNNNNNN.
 #  - password: PIA password.
-#  - Gluetun typically reads OPENVPN_USER / OPENVPN_PASSWORD (OpenVPN) or
-#    WireGuard credentials. Confirm the chosen protocol when wiring — the
-#    requirement says NOT TCP. The env var names / key names the Gluetun
-#    deployment expects may differ from username/password below; adjust to match
-#    the chosen protocol before encrypting.
+#  - Protocol is WireGuard (VPN_TYPE=wireguard, UDP — satisfies "not TCP"). PIA on WireGuard STILL
+#    authenticates with the username/password below (Gluetun derives the WireGuard config from PIA's
+#    API at runtime), so these two keys are correct as-is — no extra WireGuard key material needed.
 # =============================================================================
 apiVersion: v1
 kind: Secret
 metadata:
  name: pia-credentials
-  namespace: REPLACE-ME-PHASE-3-VPN-NAMESPACE
+  namespace: vpn-egress
 type: Opaque
 stringData:
  username: REPLACE-ME
@@ -0,0 +1,50 @@
+# platform/vpn-egress — shared VPN egress gateway (Phase 3, ADR-0013)
+
+Single shared **Gluetun** gateway: **PIA over WireGuard**, built-in kill-switch, native port
+forwarding. `vpn`-class services (sonarr, radarr, jackett, qbittorrent) egress **only** through this
+tunnel; `internet`-class services go out the normal WAN. Serves **R-NET-05/06/07**.
+
+> **STATUS: authored, NOT deployed.** Not wired into `platform/kustomization.yaml` yet. Deploy needs
+> (a) the `pia-credentials` SOPS secret, and (b) validation of the egress-steering mechanism below,
+> which cannot be tested without a live PIA tunnel.
+
+## Pieces
+- `gluetun-gateway.yaml` — the Gluetun Deployment (`replicas: 1`, `Recreate`) + the `gluetun-control`
+  Service (control API :8000 for forwarded-port lookup + tunnel-state health). Kill-switch = `FIREWALL=on`.
+- `pia-credentials` (SOPS) — PIA username/password only; Gluetun derives the WireGuard config at runtime.
+
+## Operator hand-off (when deploying Phase 3)
+1. Provision `pia-credentials` (see `bootstrap/sops/secret-templates/pia-credentials.yaml.example`,
+   namespace `vpn-egress`, keys `username`/`password`) → `sops -e -i` → uncomment it in `kustomization.yaml`.
+2. Set cluster-vars: `PIA_REGION` (a **port-forwarding-capable** PIA region — e.g. `Netherlands`,
+   `CA Toronto`, `Switzerland`; NOT a US region, which lack PF) and `CLUSTER_POD_CIDR` (k3s default
+   `10.42.0.0/16`).
+3. Enable `- vpn-egress` in `platform/kustomization.yaml` (Phase 3).
+
+## Egress steering — the DESIGN DECISION (needs live validation)
+The hard part: routing `vpn`-class pods through this ONE shared tunnel. Cilium Egress Gateway
+(`CiliumEgressGatewayPolicy`) SNATs matched pods out of a **gateway node's interface** — it does not
+natively route through a *pod's* tunnel. Options considered:
+
+1. **Cilium Egress Gateway → the Gluetun node's tunnel interface (recommended).** Pin Gluetun to a
+   node; a `CiliumEgressGatewayPolicy` selects pods labeled `homelab.keithjackson.dev/egress-class=vpn`
+   and egresses them via that node's WireGuard interface (`wg0`). If the tunnel drops, `wg0` is gone →
+   traffic has no egress path → **fail-closed** (the kill-switch behavior R-NET-06 needs). This is the
+   most Cilium-native fit but requires Gluetun's tunnel interface to be visible to the node datapath
+   (hostNetwork Gluetun, or a shared netns) — the detail to VALIDATE.
+2. **Gateway-pod routing:** client pods default-route to the Gluetun pod IP (init-container +
+   NET_ADMIN, or a Multus secondary interface); Gluetun forwards+NATs via the tunnel. Works without
+   Cilium egress gateway but is more moving parts and pod-IP-fragile.
+3. **Per-pod Gluetun sidecar** — explicitly REJECTED by ADR-0013 (single shared gateway preferred).
+
+**Plan:** validate option 1 against a live PIA tunnel (confirm `wg0` on the Gluetun node + a
+`CiliumEgressGatewayPolicy` steering `egress-class=vpn` pods, then run the `egress-class` (R-NET-05/07)
+and `vpn-killswitch` (R-NET-06) conformance tests). Fall back to option 2 if the interface isn't
+addressable. The `CiliumEgressGatewayPolicy` is intentionally NOT committed until validated, to avoid
+shipping an untested datapath policy.
+
+## Port forwarding (qBittorrent) — follow-up
+PIA assigns a forwarded port that can change on reconnect. A small **port-sync helper**
+(`qbittorrent-natmap`) reads the current port from the `gluetun-control` API (:8000
+`/v1/openvpn/portforwarded`) and writes it into qBittorrent, keeping it hands-off (R-UPD). Authored
+alongside the qBittorrent app (Phase 6 apps / or here when qBittorrent lands).
@@ -0,0 +1,99 @@
+---
+# Gluetun — the single shared in-cluster VPN egress gateway (ADR-0013): PIA over WireGuard, with
+# Gluetun's built-in kill-switch/firewall (fail-closed), health checks, auto-reconnect, and native
+# port forwarding for qBittorrent. `vpn`-class pods are steered through this gateway (see README.md
+# "Egress steering" — the Cilium-native mechanism, pending live validation with PIA creds).
+#
+# Credentials: the `pia-credentials` SOPS Secret (keys: username, password). Gluetun uses PIA's
+# username/password (via OPENVPN_USER/OPENVPN_PASSWORD — legacy env names it uses for PIA on BOTH
+# protocols) to authenticate to PIA's API and derive the WireGuard config at runtime — so WireGuard
+# needs no extra key material (ADR-0013).
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: gluetun
+  namespace: vpn-egress
+  labels:
+    app: gluetun
+spec:
+  replicas: 1                 # ONE shared tunnel (ADR-0013). Active/standby is a later enhancement.
+  strategy:
+    type: Recreate           # a tunnel is a singleton — never run two overlapping during a rollout
+  selector:
+    matchLabels:
+      app: gluetun
+  template:
+    metadata:
+      labels:
+        app: gluetun
+    spec:
+      containers:
+        - name: gluetun
+          image: qmcgaw/gluetun:v3.40.0        # pinned; Renovate bumps
+          securityContext:
+            capabilities:
+              add: ["NET_ADMIN"]               # required for the tunnel + firewall
+          env:
+            - name: VPN_SERVICE_PROVIDER
+              value: "private internet access"
+            - name: VPN_TYPE
+              value: "wireguard"               # UDP — satisfies "protocol != TCP" (R-NET); upgrade over OpenVPN
+            # PIA credentials from the SOPS secret (Gluetun's PIA creds env, used for WireGuard too):
+            - name: OPENVPN_USER
+              valueFrom:
+                secretKeyRef:
+                  name: pia-credentials
+                  key: username
+            - name: OPENVPN_PASSWORD
+              valueFrom:
+                secretKeyRef:
+                  name: pia-credentials
+                  key: password
+            - name: SERVER_REGIONS
+              value: "${PIA_REGION}"           # cluster-vars: a PF-CAPABLE PIA region (ADR-0013 PF caveat)
+            - name: VPN_PORT_FORWARDING
+              value: "on"                      # PIA native port forwarding for qBittorrent
+            - name: FIREWALL
+              value: "on"                      # KILL-SWITCH: block all egress except the tunnel (fail-closed, R-NET-06)
+            - name: FIREWALL_OUTBOUND_SUBNETS
+              value: "${CLUSTER_POD_CIDR}"     # allow steered client pods (pod CIDR) to route out via the tunnel
+            - name: DOT
+              value: "off"                     # DNS via cluster CoreDNS, not Gluetun's DoT
+          ports:
+            - name: control
+              containerPort: 8000              # Gluetun control API (health + forwarded-port lookup)
+          livenessProbe:
+            httpGet: { path: /v1/openvpn/status, port: 8000 }   # Gluetun control API health
+            initialDelaySeconds: 20
+            periodSeconds: 30
+          resources:
+            requests:
+              cpu: 50m
+              memory: 64Mi
+            limits:
+              memory: 160Mi
+          volumeMounts:
+            - name: tun
+              mountPath: /dev/net/tun
+      volumes:
+        - name: tun
+          hostPath:
+            path: /dev/net/tun
+            type: CharDevice
+---
+# Control API Service — the port-sync helper (qbittorrent-natmap, README) reads the PIA-assigned
+# forwarded port from here; also used for tunnel-state health/alerting (ADR-0013 "alert on tunnel state").
+apiVersion: v1
+kind: Service
+metadata:
+  name: gluetun-control
+  namespace: vpn-egress
+  labels:
+    app: gluetun
+spec:
+  selector:
+    app: gluetun
+  ports:
+    - name: control
+      port: 8000
+      targetPort: 8000
@@ -0,0 +1,10 @@
+---
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - namespace.yaml
+  - gluetun-gateway.yaml
+  # SOPS-encrypted PIA credentials (keys: username, password). Operator flow:
+  # copy bootstrap/sops/secret-templates/pia-credentials.yaml.example here as pia-credentials.sops.yaml
+  # (namespace: vpn-egress), fill + `sops -e -i`, then uncomment. See bootstrap/sops/CREDENTIALS.md.
+  # - pia-credentials.sops.yaml
@@ -0,0 +1,11 @@
+---
+# vpn-egress — the single shared VPN egress gateway namespace (ADR-0013, docs/03).
+# Platform INFRA, not an application namespace: the Gluetun gateway needs NET_ADMIN + /dev/net/tun,
+# so this ns is deliberately NOT labeled `app-namespace=true` (Kyverno restrict-host-namespaces /
+# require-*-label policies scope to app namespaces only — see platform/kyverno).
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: vpn-egress
+  labels:
+    homelab.keithjackson.dev/app-namespace: "false"