Switch VPN provider ExpressVPN -> PIA (ADR-0013 supersedes ADR-0012)

Operator turned off ExpressVPN auto-renew and purchased PIA. The single-shared-
gateway model is retained; provider/client/protocol change to PIA via Gluetun
over WireGuard with native port forwarding (qbittorrent-natmap port-sync).
Simpler than ExpressVPN: no custom OpenVPN config, just PIA user/pass
SOPS-encrypted; Gluetun provides the built-in kill-switch.

Updated: ADR-0013 (new) + ADR-0012 superseded + index; docs 00,01,03,07;
task board (T-3.2p/T-3.2, OQ-3).

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
This commit is contained in:
jakman85andClaude Opus 4.8 committed 2026-08-15 23:16:33 -05:00
1 parent fdca4305cf
commit 2cca9beac7
8 files changed
+86 -23

No files matched your search

+5 -4
View File
@@ -82,7 +82,7 @@ mandatory; `SHOULD` is strongly preferred and requires an ADR to deviate.
- **R-NET-05** — A service MUST be able to declare its **egress class**: `internet` (standard local
network/internet) or `vpn` (routed through a custom VPN connection). VPN routing MUST be enforced
**outside the service** (a single shared in-cluster gateway), so a service cannot leak by
misconfiguration (ADR-0012, OQ-3).
misconfiguration (ADR-0013, OQ-3).
- **R-NET-06** — VPN-class egress MUST enforce a **kill-switch**: if the VPN tunnel is down, the
service MUST NOT leak traffic to the standard internet.
- **R-NET-07** — Egress class MUST be a declarative property of the service, changeable via GitOps
@@ -191,9 +191,10 @@ implementation.
cluster**; the specific backing is **TBD later** and does not block the core platform. Plex's
metadata DB still lives on replicated Longhorn. The **off-cluster DR backup target** (for app
config, Nextcloud, Gitea, etc.) remains to be chosen at Phase 4. See [`04`](04-storage-and-state.md).
- **OQ-3 (VPN)** — **RESOLVED**: **ExpressVPN**, cluster-provisioned, **single shared in-cluster
tunnel**, **non-TCP** (OpenVPN UDP / Lightway-UDP), rules enforced outside the service
(ADR-0012). See [`03`](03-networking-isolation-and-egress.md).
- **OQ-3 (VPN)** — **RESOLVED**: **PIA** via Gluetun, cluster-provisioned, **single shared
in-cluster tunnel** over **WireGuard** (non-TCP), native port forwarding, rules enforced outside
the service (ADR-0013, supersedes ADR-0012's ExpressVPN choice). See
[`03`](03-networking-isolation-and-egress.md).
- **OQ-4 (TLS/DNS challenge access)** — **RESOLVED**: a **scoped Cloudflare API token**
(`Zone·DNS·Edit`, `keithjackson.dev`), SOPS-encrypted, drives cert-manager DNS-01 and ExternalDNS.
+1 -1
View File
@@ -15,7 +15,7 @@ choice here without a superseding ADR.**
| VM workloads (IaaS) | **KubeVirt** (+ CDI) | R-PLAT-03 | [ADR-0005](../decisions/adr-0005-vm-workloads-kubevirt.md) | [06](06-vm-workloads-kubevirt.md) |
| TLS | **cert-manager** + Let's Encrypt (DNS-01) | R-TLS-01..03 | [ADR-0006](../decisions/adr-0006-tls-cert-manager-letsencrypt.md) | [08](08-tls-and-certificates.md) |
| Secrets | **SOPS + age** (Flux-native) | R-SEC-01 | [ADR-0007](../decisions/adr-0007-secrets-sops-age.md) | [07](07-security-and-secrets.md) |
| VPN egress | **Single shared in-cluster ExpressVPN gateway** (OpenVPN UDP) + Cilium Egress Gateway | R-NET-05/06/07 | [ADR-0012](../decisions/adr-0012-vpn-egress-single-expressvpn-gateway.md) (supersedes 0008) | [03](03-networking-isolation-and-egress.md) |
| VPN egress | **Single shared in-cluster PIA/Gluetun gateway** (WireGuard + native PF) + Cilium Egress Gateway | R-NET-05/06/07 | [ADR-0013](../decisions/adr-0013-vpn-provider-pia-gluetun-wireguard.md) (supersedes 0012/0008) | [03](03-networking-isolation-and-egress.md) |
| Automated updates | **Flux image-update-automation** (images) + **Renovate** (charts) | R-UPD-01..03 | [ADR-0009](../decisions/adr-0009-automated-image-updates.md) | [11](11-automated-updates.md) |
| DNS records | **ExternalDNS** (public via Cloudflare) + **PiHole** (internal) | R-DNS-01..03 | — | [09](09-dns-and-ingress.md) |
| Ingress (north-south) | **Gateway API** implemented by Cilium | R-DNS-01, R-TLS-01 | [ADR-0002](../decisions/adr-0002-cni-cilium.md) | [09](09-dns-and-ingress.md) |
+15 -12
View File
@@ -69,7 +69,7 @@ as multi-ISP routing but as **two egress paths selected by routing rules**:
| Profile | Meaning | Mechanism |
|---|---|---|
| `internet` | Out the single commercial WAN | Normal pod egress via node → SOHO edge router → internet |
| `vpn` | Routed through the in-cluster VPN tunnel | Steered to the shared VPN egress gateway (ADR-0012) |
| `vpn` | Routed through the in-cluster VPN tunnel | Steered to the shared VPN egress gateway (ADR-0013) |
- **Physical multi-WAN (multi-ISP) is out of scope.** With a single WAN there is nothing to policy-
route across; the `internet` path is simply the default route out.
@@ -79,15 +79,16 @@ as multi-ISP routing but as **two egress paths selected by routing rules**:
ports) to the ingress VIP** for `public` services — a documented manual step ([09](09-dns-and-ingress.md)).
- The **service-facing contract is unchanged**: a service still just declares `egress: internet|vpn`.
## VPN egress model: single shared ExpressVPN gateway (R-NET-05/06 — ADR-0012, supersedes ADR-0008)
## VPN egress model: single shared PIA/Gluetun gateway (R-NET-05/06 — ADR-0013, supersedes ADR-0012/0008)
VPN-class services: **sonarr, radarr, jackett, qbittorrent**. Per OQ-3 the design is a **single,
shared VPN connection** — not a per-service sidecar — so that **network rules are enforced outside
the service**:
- **One in-cluster VPN egress gateway** (an HA-capable Deployment) holds a **single ExpressVPN
tunnel** using a **non-TCP protocol (OpenVPN UDP; Lightway-UDP if a containerizable client is
viable)** — the cluster provisions and controls the tunnel (nothing on the edge router).
- **One in-cluster VPN egress gateway** (an HA-capable Deployment) runs **Gluetun** configured for
**PIA over WireGuard** (UDP — "non-TCP or better"). The cluster provisions and controls the tunnel
(nothing on the edge router). Gluetun provides the built-in **kill-switch/firewall**, health
checks, and auto-reconnect.
- **`vpn`-class pods are steered through the gateway** (Cilium Egress Gateway / routing); they have
**no independent egress path**. The kill-switch and allow rules are enforced at the **gateway +
namespace/policy level**, i.e. *outside* each service — the service is just labeled `egress: vpn`
@@ -95,13 +96,15 @@ the service**:
- **Kill-switch (fail-closed) is mandatory and enforced centrally:** if the tunnel is down, `vpn`
pods have **no route to the plain internet**; only the tunnel path exists. `internet`-class
services are unaffected.
- **Port forwarding for qBittorrent:** Gluetun **native PF** (`VPN_PORT_FORWARDING=on`) on a
**PF-enabled PIA region**; a **port-sync helper** (e.g. `qbittorrent-natmap`) writes the assigned
port into qBittorrent so it tracks reconnects — kept hands-off. Alert if PF acquisition fails.
- **Trade-off (documented):** one shared tunnel = one shared egress IP and a single failure domain
for `vpn` services → run the gateway **active/standby** for resilience; alert on tunnel state
([10](10-observability-and-alerting.md)).
- **Prerequisite:** ExpressVPN is not a first-class containerized provider — the operator must obtain
an **ExpressVPN manual OpenVPN-UDP config + manual credentials**; the gateway consumes them as a
SOPS-encrypted secret ([07](07-security-and-secrets.md)). This must be validated end-to-end
(tunnel + kill-switch) in dev/test.
- **Prerequisite (simple):** just the **PIA username/password**, stored SOPS-encrypted
([07](07-security-and-secrets.md)) — no manual `.ovpn` extraction. Validate end-to-end (tunnel +
kill-switch + PF) in dev/test.
**Kill-switch is an acceptance gate:** with the tunnel forcibly down, every `vpn` service must fail
closed (no packets to the plain internet). Tested, not assumed.
@@ -128,10 +131,10 @@ Exposure is a declared property: `internal` (LAN-only) vs `public` (port-forward
## Resolved
- **OQ-1** — one commercial WAN, no multi-ISP plans; dumb SOHO edge (no VLAN/VPN/services). All
segmentation + VPN in-cluster; edge does NAT + port-forward only. Multi-ISP routing out of scope.
- **OQ-3** — ExpressVPN, single shared in-cluster tunnel (OpenVPN UDP / non-TCP), rules enforced
outside the service (ADR-0012).
- **OQ-3** — **PIA** via Gluetun, single shared in-cluster tunnel over **WireGuard**, native port
forwarding, rules enforced outside the service (ADR-0013, supersedes ADR-0012's ExpressVPN choice).
## Open questions
- Exact steering mechanism for `vpn` pods → gateway (Cilium Egress Gateway vs routing/SNAT) —
WS-NET implementation detail; the kill-switch acceptance test is the gate.
- Whether a containerized **Lightway-UDP** client is viable (else OpenVPN UDP is the mechanism).
- Which **PF-enabled PIA region** to use, and the port-sync helper wiring for qBittorrent.
+1 -1
View File
@@ -36,7 +36,7 @@ Defense in depth with declarative, auditable controls:
[AGENTS.md §5](../AGENTS.md)).
### Secret categories to manage
- VPN credentials (WireGuard keys / OpenVPN creds) for the egress model.
- VPN credentials (PIA username/password; WireGuard) for the egress model (ADR-0013).
- DNS provider API token (Cloudflare) for ACME DNS-01 + ExternalDNS.
- SMTP credentials for Alertmanager email.
- Service credentials (Nextcloud, Gitea admin, UniFi, Semaphore, Kasm, DB passwords).