diff --git a/agents/task-board.md b/agents/task-board.md index 31fd681..74f2bca 100644 --- a/agents/task-board.md +++ b/agents/task-board.md @@ -50,8 +50,8 @@ Seeded from [`../roadmap/phases.md`](../roadmap/phases.md) + [`../roadmap/workst |---|---|---|---|---|---|---| | T-3.0 | L3 fault-injection harness — **in-cluster Chainsaw Job** (CI-triggered, scoped test SA) using `script`/`try`/`finally` (force tunnel down, kill node, inject bad image, trigger restore) | WS-TEST | R-TEST-05, R-SEC-02 | T-0.4 | — | todo | | T-3.1 | Egress classes; Cilium Egress Gateway wiring (steer `vpn` pods → gateway) | WS-NET | R-NET-04/05 | T-2.2 | — | todo | -| T-3.2p | **Prereq:** obtain ExpressVPN manual OpenVPN-UDP config + manual creds; store SOPS-encrypted | WS-SEC | R-NET-05 | — | — | todo | -| T-3.2 | **Single shared in-cluster ExpressVPN gateway** (OpenVPN UDP, active/standby) + centrally-enforced kill-switch (ADR-0012) | WS-NET | R-NET-05/06 | T-3.1, T-3.2p | — | todo | +| T-3.2p | **Prereq:** provide **PIA username/password**; store SOPS-encrypted (PIA purchased) | WS-SEC | R-NET-05 | — | — | todo | +| T-3.2 | **Single shared in-cluster PIA/Gluetun gateway** (WireGuard, active/standby) + built-in kill-switch + native PF (`qbittorrent-natmap` port-sync) (ADR-0013) | WS-NET | R-NET-05/06 | T-3.1, T-3.2p | — | todo | | ~~T-3.3~~ | ~~Edge-router multi-WAN split~~ — **out of scope** (single WAN, dumb SOHO edge; OQ-1) | WS-NET | R-NET-04 | — | — | won't-do | | T-3.4 | Prove: VPN IP egress, **kill-switch fail-closed (central)**, egress-class flip via git | WS-NET | R-NET-05/06/07 | T-3.0, T-3.2 | — | todo | @@ -100,6 +100,6 @@ Seeded from [`../roadmap/phases.md`](../roadmap/phases.md) + [`../roadmap/workst |---|---|---|---| | OQ-1 | WAN count + edge router | — | ✅ resolved: 1 WAN, dumb SOHO edge; multi-WAN out of scope; VPN + segmentation in-cluster | | OQ-2 | Bulk-media home | — | ✅ resolved: media outside cluster, backing TBD later (not blocking) | -| OQ-3 | VPN provider/protocol | — | ✅ resolved: ExpressVPN, single shared in-cluster tunnel, OpenVPN UDP (ADR-0012) | +| OQ-3 | VPN provider/protocol | — | ✅ resolved: **PIA** via Gluetun, single shared in-cluster tunnel, **WireGuard** + native PF (ADR-0013, supersedes 0012) | | OQ-4 | DNS-01 challenge access | — | ✅ resolved: scoped **Cloudflare API token** (Zone·DNS·Edit), SOPS-encrypted; drives cert-manager + ExternalDNS. Operator provides token (T-2.4p) | | **OQ-BKP** | **Off-cluster DR backup target** (S3/NFS endpoint) | T-4.3 | ⛔ **OPEN** — needed by Phase 4, not Phase 2 | diff --git a/decisions/README.md b/decisions/README.md index eaeb02f..21526ed 100644 --- a/decisions/README.md +++ b/decisions/README.md @@ -53,4 +53,5 @@ serves. **Do not edit an accepted decision in place** — supersede it with a ne | [0009](adr-0009-automated-image-updates.md) | Automated updates: Flux image automation + Renovate | Accepted | R-UPD-01..03 | | [0010](adr-0010-conformance-testing.md) | Conformance testing: automated, reproducible, behavior-based | Accepted | R-TEST-01..06 | | [0011](adr-0011-test-tooling-kyverno-chainsaw.md) | Conformance test tooling: Kyverno + Chainsaw | Accepted | R-TEST-01..05 | -| [0012](adr-0012-vpn-egress-single-expressvpn-gateway.md) | VPN egress: single shared in-cluster ExpressVPN gateway | Accepted | R-NET-05/06/07 | +| [0012](adr-0012-vpn-egress-single-expressvpn-gateway.md) | VPN egress: single shared in-cluster ExpressVPN gateway | ~~Superseded by 0013~~ | R-NET-05/06/07 | +| [0013](adr-0013-vpn-provider-pia-gluetun-wireguard.md) | VPN provider: PIA via Gluetun (WireGuard + port forwarding) | Accepted | R-NET-05/06/07 | diff --git a/decisions/adr-0012-vpn-egress-single-expressvpn-gateway.md b/decisions/adr-0012-vpn-egress-single-expressvpn-gateway.md index d00a84d..0515106 100644 --- a/decisions/adr-0012-vpn-egress-single-expressvpn-gateway.md +++ b/decisions/adr-0012-vpn-egress-single-expressvpn-gateway.md @@ -1,9 +1,14 @@ # ADR-0012 — VPN egress: single shared in-cluster ExpressVPN gateway -- Status: Accepted +- Status: **Superseded by [ADR-0013](adr-0013-vpn-provider-pia-gluetun-wireguard.md)** (2026-08-15) - Date: 2026-08-15 - Serves: R-NET-05, R-NET-06, R-NET-07 - Supersedes: [ADR-0008](adr-0008-vpn-egress-model.md) +> **Superseded.** The **single-shared-gateway model here is retained**, but the operator moved off +> ExpressVPN to **PIA** (Gluetun-native, WireGuard, port forwarding). Only the provider/client/ +> protocol changed — see [ADR-0013](adr-0013-vpn-provider-pia-gluetun-wireguard.md). Retained for +> history. + ## Context ADR-0008 offered two VPN-egress patterns (Gluetun sidecar per pod as default, central gateway as alternative) and left the provider open (OQ-3). The operator has now specified (OQ-3): diff --git a/decisions/adr-0013-vpn-provider-pia-gluetun-wireguard.md b/decisions/adr-0013-vpn-provider-pia-gluetun-wireguard.md new file mode 100644 index 0000000..aad1a7f --- /dev/null +++ b/decisions/adr-0013-vpn-provider-pia-gluetun-wireguard.md @@ -0,0 +1,53 @@ +# ADR-0013 — VPN provider: PIA via Gluetun (WireGuard + native port forwarding) +- Status: Accepted +- Date: 2026-08-15 +- Serves: R-NET-05, R-NET-06, R-NET-07 +- Supersedes: [ADR-0012](adr-0012-vpn-egress-single-expressvpn-gateway.md) + +## Context +ADR-0012 kept the operator's required model — a single shared in-cluster VPN gateway with rules +enforced outside the service — but named **ExpressVPN** as the provider, which forced a custom +OpenVPN-UDP setup because ExpressVPN is **not a first-class containerized (Gluetun) provider**. The +operator has since **turned off ExpressVPN auto-renew and purchased Private Internet Access (PIA)**, +which *is* first-class in Gluetun. The egress **model is unchanged**; only the provider, client, and +protocol change — so ADR-0012 is superseded. + +## Decision +The single shared in-cluster VPN gateway runs **Gluetun** configured for **PIA** over **WireGuard**, +with Gluetun's **native automated port forwarding** for qBittorrent: + +- **Provider/client:** PIA via the **Gluetun** container as the gateway VPN client (Gluetun provides + the built-in **kill-switch/firewall**, health checks, and auto-reconnect). +- **Protocol:** **WireGuard** (UDP) — satisfies "non-TCP, UDP or better" and is a strict upgrade over + ExpressVPN's OpenVPN-UDP. +- **Credentials:** just the **PIA username/password**, stored **SOPS-encrypted** ([ADR-0007](adr-0007-secrets-sops-age.md)). + No manual `.ovpn` extraction (the ExpressVPN prerequisite is gone). +- **Port forwarding:** Gluetun native (`VPN_PORT_FORWARDING=on`, provider `private internet + access`) on a **PF-enabled PIA region**; a small **port-sync helper** (e.g. `qbittorrent-natmap`) + writes the assigned port into qBittorrent so it tracks reconnects — keeping it hands-off (R-UPD). +- **Model retained from ADR-0012:** one shared tunnel; `vpn`-class pods steered through the gateway + with **no independent egress path**; **central kill-switch (fail-closed)**; rules enforced outside + the service; gateway run **active/standby**; alert on tunnel state. + +## Alternatives considered +- **ExpressVPN (ADR-0012)** — dropped: not Gluetun-native, needed a custom OpenVPN workaround; and the + operator's subscription is lapsing. +- **ProtonVPN** — equally good fit (Gluetun-native NAT-PMP PF, WireGuard); viable if PIA + disappoints, but the operator has purchased PIA. +- **AirVPN** — WireGuard works in Gluetun but automated PF is not integrated (static port only); + less hands-off. +- **Mullvad** — no port forwarding since 2023 → degraded qBittorrent seeding; rejected for this + torrent-adjacent stack. + +## Consequences +- **Simpler and cleaner than ADR-0012:** the gateway is a stock Gluetun container (env-var config), + not a bespoke OpenVPN build; kill-switch and port forwarding are built in. +- **PF caveat:** PIA port forwarding is available only on **specific PF-enabled server regions** and + the assigned port can change on reconnect → the **port-sync helper** and a PF-capable region are + required; alert if PF acquisition fails. +- Single shared egress IP + single failure domain remain (mitigated by active/standby + alerting). +- Credentials are a single SOPS-encrypted PIA username/password; rotation = re-encrypt + commit. +- Kill-switch + correct-egress-IP + PF remain **fault-injection acceptance tests** in dev/test + (R-TEST-05, [14](../docs/14-testing-and-conformance.md)). +- If PIA proves unsatisfactory, **ProtonVPN** is the pre-vetted fallback (superseding ADR, operator + decision). diff --git a/docs/00-vision-and-scope.md b/docs/00-vision-and-scope.md index ab783c2..a020d3b 100644 --- a/docs/00-vision-and-scope.md +++ b/docs/00-vision-and-scope.md @@ -82,7 +82,7 @@ mandatory; `SHOULD` is strongly preferred and requires an ADR to deviate. - **R-NET-05** — A service MUST be able to declare its **egress class**: `internet` (standard local network/internet) or `vpn` (routed through a custom VPN connection). VPN routing MUST be enforced **outside the service** (a single shared in-cluster gateway), so a service cannot leak by - misconfiguration (ADR-0012, OQ-3). + misconfiguration (ADR-0013, OQ-3). - **R-NET-06** — VPN-class egress MUST enforce a **kill-switch**: if the VPN tunnel is down, the service MUST NOT leak traffic to the standard internet. - **R-NET-07** — Egress class MUST be a declarative property of the service, changeable via GitOps @@ -191,9 +191,10 @@ implementation. cluster**; the specific backing is **TBD later** and does not block the core platform. Plex's metadata DB still lives on replicated Longhorn. The **off-cluster DR backup target** (for app config, Nextcloud, Gitea, etc.) remains to be chosen at Phase 4. See [`04`](04-storage-and-state.md). -- **OQ-3 (VPN)** — **RESOLVED**: **ExpressVPN**, cluster-provisioned, **single shared in-cluster - tunnel**, **non-TCP** (OpenVPN UDP / Lightway-UDP), rules enforced outside the service - (ADR-0012). See [`03`](03-networking-isolation-and-egress.md). +- **OQ-3 (VPN)** — **RESOLVED**: **PIA** via Gluetun, cluster-provisioned, **single shared + in-cluster tunnel** over **WireGuard** (non-TCP), native port forwarding, rules enforced outside + the service (ADR-0013, supersedes ADR-0012's ExpressVPN choice). See + [`03`](03-networking-isolation-and-egress.md). - **OQ-4 (TLS/DNS challenge access)** — **RESOLVED**: a **scoped Cloudflare API token** (`Zone·DNS·Edit`, `keithjackson.dev`), SOPS-encrypted, drives cert-manager DNS-01 and ExternalDNS. diff --git a/docs/01-architecture-overview.md b/docs/01-architecture-overview.md index 9842163..ac8ad8b 100644 --- a/docs/01-architecture-overview.md +++ b/docs/01-architecture-overview.md @@ -15,7 +15,7 @@ choice here without a superseding ADR.** | VM workloads (IaaS) | **KubeVirt** (+ CDI) | R-PLAT-03 | [ADR-0005](../decisions/adr-0005-vm-workloads-kubevirt.md) | [06](06-vm-workloads-kubevirt.md) | | TLS | **cert-manager** + Let's Encrypt (DNS-01) | R-TLS-01..03 | [ADR-0006](../decisions/adr-0006-tls-cert-manager-letsencrypt.md) | [08](08-tls-and-certificates.md) | | Secrets | **SOPS + age** (Flux-native) | R-SEC-01 | [ADR-0007](../decisions/adr-0007-secrets-sops-age.md) | [07](07-security-and-secrets.md) | -| VPN egress | **Single shared in-cluster ExpressVPN gateway** (OpenVPN UDP) + Cilium Egress Gateway | R-NET-05/06/07 | [ADR-0012](../decisions/adr-0012-vpn-egress-single-expressvpn-gateway.md) (supersedes 0008) | [03](03-networking-isolation-and-egress.md) | +| VPN egress | **Single shared in-cluster PIA/Gluetun gateway** (WireGuard + native PF) + Cilium Egress Gateway | R-NET-05/06/07 | [ADR-0013](../decisions/adr-0013-vpn-provider-pia-gluetun-wireguard.md) (supersedes 0012/0008) | [03](03-networking-isolation-and-egress.md) | | Automated updates | **Flux image-update-automation** (images) + **Renovate** (charts) | R-UPD-01..03 | [ADR-0009](../decisions/adr-0009-automated-image-updates.md) | [11](11-automated-updates.md) | | DNS records | **ExternalDNS** (public via Cloudflare) + **PiHole** (internal) | R-DNS-01..03 | — | [09](09-dns-and-ingress.md) | | Ingress (north-south) | **Gateway API** implemented by Cilium | R-DNS-01, R-TLS-01 | [ADR-0002](../decisions/adr-0002-cni-cilium.md) | [09](09-dns-and-ingress.md) | diff --git a/docs/03-networking-isolation-and-egress.md b/docs/03-networking-isolation-and-egress.md index 24f931c..ba6f481 100644 --- a/docs/03-networking-isolation-and-egress.md +++ b/docs/03-networking-isolation-and-egress.md @@ -69,7 +69,7 @@ as multi-ISP routing but as **two egress paths selected by routing rules**: | Profile | Meaning | Mechanism | |---|---|---| | `internet` | Out the single commercial WAN | Normal pod egress via node → SOHO edge router → internet | -| `vpn` | Routed through the in-cluster VPN tunnel | Steered to the shared VPN egress gateway (ADR-0012) | +| `vpn` | Routed through the in-cluster VPN tunnel | Steered to the shared VPN egress gateway (ADR-0013) | - **Physical multi-WAN (multi-ISP) is out of scope.** With a single WAN there is nothing to policy- route across; the `internet` path is simply the default route out. @@ -79,15 +79,16 @@ as multi-ISP routing but as **two egress paths selected by routing rules**: ports) to the ingress VIP** for `public` services — a documented manual step ([09](09-dns-and-ingress.md)). - The **service-facing contract is unchanged**: a service still just declares `egress: internet|vpn`. -## VPN egress model: single shared ExpressVPN gateway (R-NET-05/06 — ADR-0012, supersedes ADR-0008) +## VPN egress model: single shared PIA/Gluetun gateway (R-NET-05/06 — ADR-0013, supersedes ADR-0012/0008) VPN-class services: **sonarr, radarr, jackett, qbittorrent**. Per OQ-3 the design is a **single, shared VPN connection** — not a per-service sidecar — so that **network rules are enforced outside the service**: -- **One in-cluster VPN egress gateway** (an HA-capable Deployment) holds a **single ExpressVPN - tunnel** using a **non-TCP protocol (OpenVPN UDP; Lightway-UDP if a containerizable client is - viable)** — the cluster provisions and controls the tunnel (nothing on the edge router). +- **One in-cluster VPN egress gateway** (an HA-capable Deployment) runs **Gluetun** configured for + **PIA over WireGuard** (UDP — "non-TCP or better"). The cluster provisions and controls the tunnel + (nothing on the edge router). Gluetun provides the built-in **kill-switch/firewall**, health + checks, and auto-reconnect. - **`vpn`-class pods are steered through the gateway** (Cilium Egress Gateway / routing); they have **no independent egress path**. The kill-switch and allow rules are enforced at the **gateway + namespace/policy level**, i.e. *outside* each service — the service is just labeled `egress: vpn` @@ -95,13 +96,15 @@ the service**: - **Kill-switch (fail-closed) is mandatory and enforced centrally:** if the tunnel is down, `vpn` pods have **no route to the plain internet**; only the tunnel path exists. `internet`-class services are unaffected. +- **Port forwarding for qBittorrent:** Gluetun **native PF** (`VPN_PORT_FORWARDING=on`) on a + **PF-enabled PIA region**; a **port-sync helper** (e.g. `qbittorrent-natmap`) writes the assigned + port into qBittorrent so it tracks reconnects — kept hands-off. Alert if PF acquisition fails. - **Trade-off (documented):** one shared tunnel = one shared egress IP and a single failure domain for `vpn` services → run the gateway **active/standby** for resilience; alert on tunnel state ([10](10-observability-and-alerting.md)). -- **Prerequisite:** ExpressVPN is not a first-class containerized provider — the operator must obtain - an **ExpressVPN manual OpenVPN-UDP config + manual credentials**; the gateway consumes them as a - SOPS-encrypted secret ([07](07-security-and-secrets.md)). This must be validated end-to-end - (tunnel + kill-switch) in dev/test. +- **Prerequisite (simple):** just the **PIA username/password**, stored SOPS-encrypted + ([07](07-security-and-secrets.md)) — no manual `.ovpn` extraction. Validate end-to-end (tunnel + + kill-switch + PF) in dev/test. **Kill-switch is an acceptance gate:** with the tunnel forcibly down, every `vpn` service must fail closed (no packets to the plain internet). Tested, not assumed. @@ -128,10 +131,10 @@ Exposure is a declared property: `internal` (LAN-only) vs `public` (port-forward ## Resolved - **OQ-1** — one commercial WAN, no multi-ISP plans; dumb SOHO edge (no VLAN/VPN/services). All segmentation + VPN in-cluster; edge does NAT + port-forward only. Multi-ISP routing out of scope. -- **OQ-3** — ExpressVPN, single shared in-cluster tunnel (OpenVPN UDP / non-TCP), rules enforced - outside the service (ADR-0012). +- **OQ-3** — **PIA** via Gluetun, single shared in-cluster tunnel over **WireGuard**, native port + forwarding, rules enforced outside the service (ADR-0013, supersedes ADR-0012's ExpressVPN choice). ## Open questions - Exact steering mechanism for `vpn` pods → gateway (Cilium Egress Gateway vs routing/SNAT) — WS-NET implementation detail; the kill-switch acceptance test is the gate. -- Whether a containerized **Lightway-UDP** client is viable (else OpenVPN UDP is the mechanism). +- Which **PF-enabled PIA region** to use, and the port-sync helper wiring for qBittorrent. diff --git a/docs/07-security-and-secrets.md b/docs/07-security-and-secrets.md index 22800d4..63dc886 100644 --- a/docs/07-security-and-secrets.md +++ b/docs/07-security-and-secrets.md @@ -36,7 +36,7 @@ Defense in depth with declarative, auditable controls: [AGENTS.md §5](../AGENTS.md)). ### Secret categories to manage -- VPN credentials (WireGuard keys / OpenVPN creds) for the egress model. +- VPN credentials (PIA username/password; WireGuard) for the egress model (ADR-0013). - DNS provider API token (Cloudflare) for ACME DNS-01 + ExternalDNS. - SMTP credentials for Alertmanager email. - Service credentials (Nextcloud, Gitea admin, UniFi, Semaphore, Kasm, DB passwords).