fix(vpn): enable gluetun server-list updater (bundled list was stale) + re-enable

CORRECTION: the connection failures were gluetun dialing RETIRED PIA IPs (46.246.x
= dead → timeouts, which I mis-read as a network block). A fresh PIA .ovpn connects
from the same host, and PIA's current SE Stockholm servers (158.173.x) are reachable.
Cilium was independently cleared (default-allow, host-fw off, Hubble FORWARDED).

Fix: UPDATER_PERIOD=24h so gluetun fetches PIA's current servers; keep
OPENVPN_MTU/MSSFIX=1350 for the real Cilium-VXLAN 1450 pod MTU (data channel);
PIA default UDP port. Re-enable vpn-egress. Full correction in DECISIONS-LOG.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
jakman85andClaude Opus 4.8 committed 2026-08-22 23:08:18 -05:00
1 parent 50a416077c
commit 457edb7a38
4 files changed
+35 -13

No files matched your search

+15
View File
@@ -581,3 +581,18 @@ WireGuard/thrnz would NOT help — same PIA IPs are blocked; if switching provid
one whose ranges this network permits. Reverted the port/MTU red-herring env from gluetun-gateway.yaml
(back to PIA defaults); `vpn-egress` disabled so it doesn't crashloop/alert. **Config is correct and
complete — the only blocker is the network.**
### Phase 3 VPN — CORRECTION: it was gluetun's STALE server list, NOT a network block (2026-08-22)
The "network blackholes PIA" conclusion above was WRONG. The operator installed a fresh PIA `.ovpn` in GNOME
on the same host/network and it connected fine → the network permits PIA. Verified against PIA's live server
list: current SE Stockholm servers are **158.173.241.x** and ARE reachable (tcp/8080 → RST / "connection
refused" = the server answered). The IPs gluetun was dialing (**46.246.x / 212.112.x**) are **retired**
(timeout = dead hosts). So gluetun's **bundled server list was stale**, and my host reachability test used
those dead IPs → false "blackhole." (Cilium was independently ruled out per the operator's request: policy
mode `default` = default-allow, host firewall disabled, no policy selects these pods, and Hubble showed
pod→PIA `to-network FORWARDED`, no drops.)
**Fix:** gluetun `UPDATER_PERIOD=24h` (fetch PIA's current server list at startup); keep `OPENVPN_MTU=1350`
+ `OPENVPN_MSSFIX=1350` for the real Cilium-VXLAN 1450 pod MTU (data channel); PIA default OpenVPN UDP port.
Re-enabled `vpn-egress`. **Lesson:** when gluetun "can't reach" a provider, suspect a stale bundled server
list FIRST — validate against the provider's live server list before blaming the network/MTU/ports.
+2 -2
View File
@@ -43,8 +43,8 @@ data:
# which lacks PF; NOT a US region, which also lack PF)
CLUSTER_POD_CIDR: "10.0.0.0/8" # Cilium cluster-pool pod CIDR (pods get 10.x); Gluetun
# FIREWALL_OUTBOUND_SUBNETS = allow these to egress via the tunnel
# NOTE: Phase-3 VPN (vpn-egress) is DISABLED — the network blackholes traffic to PIA's IP ranges
# (operator fix needed, not a config issue). Uses PIA's default OpenVPN UDP port. See DECISIONS-LOG.
# NOTE: Phase-3 VPN uses PIA's default OpenVPN UDP port + gluetun's server-list UPDATER (the bundled list
# was stale, dialing retired IPs). Port/region are not cluster-vars (see gluetun-gateway.yaml). See DECISIONS-LOG.
# --- identity / auth (ADR-0014) ---
KEYCLOAK_REPLICAS: "1" # dev 1 | prod 2 (HA)
+5 -7
View File
@@ -17,13 +17,11 @@ resources:
- cert-manager # Let's Encrypt DNS-01 wildcard — needs SOPS `cloudflare-api-token`
- externaldns # auto DNS records — needs SOPS `cloudflare-api-token`
- gateway # Gateway API listeners (ingress; replaces Traefik)
# ---- Phase 3 — Egress / VPN — DISABLED: BLOCKED at the network (operator action needed). Definitive
# finding: this 192.168.2.x WiFi segment BLACKHOLES traffic to PIA's IP ranges — TCP 501/443 to PIA
# time out, UDP + ICMP get no reply across many PIA servers, while Google/Cloudflare work on all.
# It also intercepts DNS/:53. Not MTU, not ports, not the cluster — the config is correct. Fix at
# the network (move to the unfiltered 192.168.1.x LAN / wired, or unblock PIA on the router), then
# re-enable. See DECISIONS-LOG "Phase 3 VPN — network blocks PIA". ----
# - vpn-egress # re-enable once the network permits PIA (no code change needed)
# ---- Phase 3 — Egress / VPN (PIA OpenVPN-UDP, ADR-0013 amendment). Real root cause of the earlier
# failures: gluetun's bundled PIA server list was STALE (dialing retired 46.246.x IPs → timeouts);
# fix = UPDATER_PERIOD (fetch current servers) + OPENVPN_MTU/MSSFIX 1350 for the Cilium-VXLAN 1450
# pod MTU. See DECISIONS-LOG "Phase 3 VPN". ----
- vpn-egress # shared Gluetun PIA/OpenVPN-UDP egress gateway + kill-switch (ADR-0013)
# ---- Phase 4 — Observability (creds provisioned via SOPS: grafana-admin, alertmanager-smtp) ----
- monitoring # kube-prometheus-stack + Loki + Alloy + Alertmanager
# ---- Later phases: enabled when their phase opens ----
+13 -4
View File
@@ -40,10 +40,19 @@ spec:
value: "private internet access"
- name: VPN_TYPE
value: "openvpn" # OpenVPN-UDP — satisfies "protocol != TCP" (R-NET); ADR-0013 amended
# NOTE (2026-08-22): PIA's default OpenVPN UDP (1197) is used. Earlier port/MTU overrides were
# removed — they were chasing symptoms. The real blocker is that this network BLACKHOLES traffic
# to PIA's IP ranges (TCP 501/443 time out, UDP/ICMP no reply — while Google/Cloudflare work).
# Fixable only at the network, not here. See DECISIONS-LOG "Phase 3 VPN — network blocks PIA".
# Root cause of the connection failures: gluetun's BUNDLED PIA server list was STALE — it dialed
# retired IPs (46.246.x → timeouts, which I initially mis-read as a network block). PIA's CURRENT
# Sweden servers are 158.173.x and are reachable; a fresh .ovpn connects fine. Enable the updater
# so gluetun fetches PIA's current server list at startup + periodically:
- name: UPDATER_PERIOD
value: "24h"
# Pod MTU is 1450 (Cilium VXLAN: 1500 − 50 encap; node/host/WAN are 1500). Shrink OpenVPN so
# DATA-channel packets fit (the handshake is small so this wasn't the connect blocker, but the
# data channel needs it). Hardcoded strings (int-coercion gotcha). See DECISIONS-LOG.
- name: OPENVPN_MTU
value: "1350"
- name: OPENVPN_MSSFIX
value: "1350"
# PIA credentials from the SOPS secret (Gluetun's PIA OpenVPN creds env):
- name: OPENVPN_USER
valueFrom: