From 53e09a2610bb3cdf94a24738a7e4c49396ab86ab Mon Sep 17 00:00:00 2001 From: Keith Jackson Date: Sat, 12 Sep 2026 18:12:46 -0500 Subject: [PATCH] docs(vpn): dev recovered the minute prod left the shared PIA server MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes the loop on the collision entry with the recovery evidence rather than leaving it as an open question for the operator: dev's last Inactivity timeout is 23:05:06 UTC — the minute prod switched to .135 — and at 23:12:03 that same session was still the live one, seven minutes unbroken against a two-minute cycle, 4/4 ping replies at 146 ms, 0% loss, and vpn-class pod egress back to 158.173.241.36 on three consecutive probes. Co-Authored-By: Claude Opus 5 --- DECISIONS-LOG.md | 9 +++++++-- platform/vpn-egress/README.md | 4 ++++ 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/DECISIONS-LOG.md b/DECISIONS-LOG.md index 8e7fe31..b20b43f 100644 --- a/DECISIONS-LOG.md +++ b/DECISIONS-LOG.md @@ -2282,6 +2282,12 @@ address is not a credential, so it sits in plain group_vars next to the vault, r **Standing rule this establishes: one gateway per PIA server.** A third gateway, or a rebuild that re-pins, must take an unused address from the pool. +**Dev recovered the moment prod left the shared server.** Dev's last `Inactivity timeout` is +23:05:06 UTC — the minute prod switched to `.135`. Re-checked at 23:12:03: the 23:05:07 session is +still the live one, seven minutes unbroken against a previous cycle of two, and a source-bound ping +through the tunnel returns 4/4 at 146 ms with 0% loss. vpn-class pod egress from dev is back to +`158.173.241.36` on three consecutive probes. + **And the switch handed us the best possible proof of the re-steer fix, unplanned.** Moving prod to `.135` gave it a genuinely NEW tunnel address on a real reconnect — `10.72.192.7` (ifindex 128) -> `10.150.192.86` (ifindex 129) — which is exactly the case Cilium does not follow and exactly what @@ -2300,8 +2306,7 @@ vpn-class egress dead until someone noticed. have distinct exit IPs (dev `158.173.241.36`, prod `158.173.241.11`), so the immediate problem is solved — but they still share one account, and the per-environment PIA server is now a hand-maintained value in `group_vars/vpn_egress_gateway_prod/vars.yml` that must not collide with - dev's. A separate prod account removes that coupling permanently. **Verify dev recovered**: dev - was still inside its flap window when this session ended (see the open item below). + dev's. A separate prod account removes that coupling permanently. 3. **Gateway-node loss is still an outage** — Cilium OSS has no egress-gateway failover, and the ADR-0016 supersede drafted on 2026-09-06 is still open. Nothing here changes that; the single-gateway choice is now recorded in the inventory itself, not just in prose. diff --git a/platform/vpn-egress/README.md b/platform/vpn-egress/README.md index b1c9aa1..6883332 100644 --- a/platform/vpn-egress/README.md +++ b/platform/vpn-egress/README.md @@ -68,6 +68,10 @@ Each reconnect steals `10.72.192.7` on that server; the loser receives nothing, off — every ~2 minutes, indefinitely. Neither environment had reliable vpn-class egress; prod's first probes looked green only because they landed inside an up-window. +Dev recovered the minute prod left `.112`: its last `Inactivity timeout` is 23:05:06 UTC, and at +23:12:03 that same session was still up — seven minutes unbroken against a two-minute cycle — with +a source-bound ping returning 4/4 at 146 ms and vpn-class pod egress back to `158.173.241.36`. + **Fix:** `vpn_egress_gateway_pia_remote` in `group_vars/vpn_egress_gateway_prod/vars.yml` rewrites just the `remote` line, pointing prod at `158.173.241.135` — another member of the same PIA Stockholm LB pool (`sweden.privacy.network` resolves to