Operator answers folded into the specs: - OQ-3 VPN: ExpressVPN, cluster-controlled SINGLE shared in-cluster tunnel (OpenVPN UDP / non-TCP), rules enforced outside the service. New ADR-0012 supersedes ADR-0008 (Gluetun-sidecar default dropped). Central kill-switch. - OQ-1 WAN/edge: one commercial WAN, no multi-ISP plans; dumb SOHO edge (no VLAN/VPN/services). Multi-WAN out of scope; segmentation + VPN in-cluster; edge = NAT + manual port-forward. Gateway VIP resolved to Cilium L2 announcement. T-3.3 marked won't-do. - OQ-2 storage: bulk media lives outside the cluster (backing TBD, non-blocking) via a 'bulk' external-mount indirection; Plex metadata DB stays on Longhorn. Split out a separate OQ-BKP for the off-cluster DR backup target (Phase 4). - OQ-4 refined: wildcard record covers reachability but NOT ACME DNS-01 (needs TXT-write access) and does not match *.home.* depth. Still OPEN: Cloudflare API token (Option A) vs acme-dns delegation (Option B). T-2.4/T-2.5 blocked. Updated: docs 00,01,03,04,08,09; ADR-0008/0012 + index; task board. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
3.5 KiB
3.5 KiB
ADR-0012 — VPN egress: single shared in-cluster ExpressVPN gateway
- Status: Accepted
- Date: 2026-08-15
- Serves: R-NET-05, R-NET-06, R-NET-07
- Supersedes: ADR-0008
Context
ADR-0008 offered two VPN-egress patterns (Gluetun sidecar per pod as default, central gateway as alternative) and left the provider open (OQ-3). The operator has now specified (OQ-3):
- Provider is ExpressVPN; the cluster must provision and control the connection (nothing on the edge router — the edge is a dumb SOHO device with no VPN, OQ-1).
- Protocol must not be TCP; UDP or better is required, data must be protected.
- A single shared VPN connection is preferred over per-service sidecars, so network rules are enforced outside the service.
This changes the decision, so ADR-0008 is superseded.
Decision
Adopt a single, shared, in-cluster VPN egress gateway as the mandated model (ADR-0008's "Pattern B" becomes the only pattern; the per-service Gluetun sidecar default is dropped):
- One VPN egress gateway (HA-capable Deployment, run active/standby) holds a single ExpressVPN tunnel using a non-TCP protocol: OpenVPN UDP (via ExpressVPN's manual OpenVPN-UDP config), or Lightway-UDP if a containerizable client proves viable. The cluster — not the edge — provisions and controls the tunnel.
vpn-class pods are steered through the gateway (Cilium Egress Gateway / routing) and have no independent egress path. The kill-switch (fail-closed) and egress allow rules are enforced at the gateway + namespace/policy level — outside each service. A service is merely labeledegress: vpn; it cannot leak on its own even if misconfigured.- Credentials (ExpressVPN manual OpenVPN config + manual-config username/password) are stored SOPS-encrypted (ADR-0007).
- The
egress: internet | vpnservice contract is unchanged (R-NET-07).
Alternatives considered
- ADR-0008 Pattern A (Gluetun sidecar per pod) — rejected per OQ-3: puts VPN inside each service (rules enforced in the service), multiplies tunnels, and isn't the operator's preferred single- connection model.
- VPN on the edge router — impossible: the SOHO edge has no VPN capability (OQ-1); the cluster must own the tunnel.
- WireGuard — ExpressVPN does not offer WireGuard; OpenVPN UDP (or Lightway-UDP) is the non-TCP path.
- OpenVPN TCP — rejected: operator requires non-TCP.
Consequences
- Rules enforced outside the service (operator's goal); one tunnel to manage; simpler per-service workloads (no sidecars).
- Single shared egress IP for all
vpnservices and a single failure domain → mitigate with active/standby gateway and tunnel-state alerting (10). - ExpressVPN is not a first-class containerized provider: expect a custom OpenVPN-UDP setup (e.g. gluetun custom-provider mode or a purpose-built OpenVPN client), and the operator must obtain the manual OpenVPN config + manual credentials. Tunnel + kill-switch must be validated in dev/test (fault-injection test, R-TEST-05).
- Exact pod→gateway steering (Cilium Egress Gateway vs routing/SNAT) is a WS-NET implementation detail; the kill-switch acceptance test is the gate (14).
- If ExpressVPN containerization proves impractical, revisit provider choice via a superseding ADR (escalation to the operator — agents may not change this decision themselves).