Session-limit interrupted the WS-TEST agent. egress-class test updated; vpn-killswitch
test + host fault-injection fixture still outstanding. Preserved for review/continuation.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The CiliumEgressGatewayPolicy authored in 38fcfad was inadvertently removed from main when
feature/vpn-egress-gateway-role (which carried a git-rm of the duplicated file) was merged.
The kustomization now references it, so restore the file. Content identical to 38fcfad.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Host OpenVPN split-tunnel on k3s-dev-3 validated 2026-08-24: tun0 up, VPN egress
158.173.241.36, node default unchanged (WAN), scoped nft kill-switch active. Wire the
CiliumEgressGatewayPolicy into the runtime kustomization so egress-class=vpn pods steer
to the node tun0.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Author the vpn_egress_gateway role + playbook that stands up the NODE-level
PIA OpenVPN tunnel on egress-gateway node(s) — the host half of ADR-0015. The
CiliumEgressGatewayPolicy steering vpn-class pods is separate GitOps against the
same contract (group vpn_egress_gateway, node label
homelab.keithjackson.dev/egress-gateway=true, host iface tun0).
Honors all four ADR-0015 invariants:
- Split-tunnel (inv 1&2): OpenVPN route-noexec + pull-filter ignore
redirect-gateway => installs no routes; policy routing in a dedicated table
(51820) with `ip rule from <tun0-addr>` steers ONLY Cilium-SNAT'd vpn-class
traffic. Node default route stays on the WAN, tunnel up or down.
- Scoped nftables kill-switch (inv 3&4): own inet table, policy accept, drops
only tunnel-sourced (@vpn_egress_gateway_src set) packets not leaving tun0 =>
vpn-class fails closed (blackhole), node/internet-class fail open (WAN),
never matched.
PIA creds + the proven sweden-aes-128-cbc-udp-ip .ovpn come from Ansible vault
(host-side per ADR-0015/docs-07); vault keys documented, no secret values
committed. ansible-lint (production) + yamllint clean. Not deployed: green
conformance is gated on the live tunnel (operator host creds) — see role README.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Add platform/vpn-egress/egress-steering.yaml — a CiliumEgressGatewayPolicy
that steers homelab.keithjackson.dev/egress-class=vpn pods out a gateway
node's host tun0 split-tunnel interface, per ADR-0015 (node-level tunnel +
Cilium Egress Gateway). Honors all four ADR-0015 invariants: only vpn-class
pods are selected (default/internet + node stay on the local/WAN path),
RFC1918 + cluster ranges are excludedCIDRs so only public internet is
tunneled and tun0 is never a default route, and the host nftables kill-switch
(Ansible) fails vpn-class CLOSED while internet-class fails OPEN.
Interface contract shared verbatim with the parallel Ansible host-tunnel role:
node label homelab.keithjackson.dev/egress-gateway=true, interface tun0,
client label homelab.keithjackson.dev/egress-class=vpn.
Cilium Egress Gateway prerequisites confirmed enabled (egressGateway.enabled,
bpf.masquerade, kubeProxyReplacement all true, [parity]) — no HelmRelease change.
Intentionally NOT added to platform/vpn-egress/kustomization.yaml: the datapath
stays out of the runtime until the live tunnel exists and the egress-class /
vpn-killswitch conformance tests pass (README rule; DoD). Gluetun is retained
as the validated fallback until then. Serves R-NET-05/06/07.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Author the internet/vpn egress-class fixture pods for the egress-class + vpn-killswitch
conformance tests. Record the datapath finding: ADR-0013's preferred steering (Cilium
egress gateway -> gluetun tun0) is NOT viable because gluetun must run hostNetwork to
put tun0 on the node, and gluetun then redirects the NODE's default route through the
VPN (breaks k3s/etcd). Escalated: remaining viable path is option 2 (gateway-pod
routing). No steering datapath committed until the mechanism is chosen + validated.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
prometheus-...-0 was OOMKilled (exit 137) every ~4min during TSDB compaction/checkpoint
(261 restarts/37h). Raise limit to 2Gi + request to 512Mi; nodes have headroom.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Once the tunnel redirects the pod default route via tun0, the pod's response to the
kubelet's httpGet liveness probe egresses the tunnel and never returns → probe times
out → pod restarts ~every 2min despite the tunnel being up (PIA egress IP confirmed).
Switch to a local wget exec probe (stays on lo).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Operator provides a new .ovpn with an IP remote (gluetun's custom parser rejects
hostnames), so drop the resolve-remote initContainer/VPN_ENDPOINT_IP attempts and
mount the .ovpn directly (key pia.ovpn). Bump image to latest v3.41.3.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
gluetun's custom parser rejects the .ovpn's 'remote sweden.privacy.network' (needs
an IP). sweden.privacy.network LB pool = 158.173.241.{22,34,55,112,239}; pin .22
(verified to handshake) via VPN_ENDPOINT_IP/PORT while the .ovpn supplies cipher/CA.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Operator's kubectl create secret keyed the file by its name
(sweden-aes-128-cbc-udp-dns.ovpn); mount item now references that key -> pia.ovpn.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
PIA-provider server selection is flaky here (random dead IPs); the operator's .ovpn
uses sweden.privacy.network (load-balanced, reliable). Switch to custom mode
(VPN_SERVICE_PROVIDER=custom, OPENVPN_CUSTOM_CONFIG=/config/pia.ovpn), mount the
pia-custom-ovpn Secret, keep creds + kill-switch + tun-mtu 1300. Drops PIA-provider
server list/updater and native PF (Phase-6 concern). Pending the operator-provisioned
pia-custom-ovpn Secret (not merged until it exists).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
gluetun silently ignored PIA_ENCRYPTION_PRESET (renamed) and defaulted to strong
(RSA-4096) the whole time. The correct v3.x var is
PRIVATE_INTERNET_ACCESS_OPENVPN_ENCRYPTION_PRESET=normal → RSA-2048/AES-128-CBC/1198,
the smaller handshake that fits the 1450 pod MTU.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
RSA-4096 strong handshake wouldn't hold over the pod's 1450 MTU (Cilium VXLAN);
MTU tuning is exhausted (no --tls-mtu in OpenVPN 2.6, PIA rejects --fragment).
Operator downloaded the lower-encryption PIA config (port 1198, RSA-2048,
AES-128-CBC, SHA1) = gluetun's 'normal' preset. Smaller handshake fits 1450.
Dropped the strong-handshake band-aid flags; kept OPENVPN_MTU 1300 for the data channel.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
RSA-4096 strong-preset handshake is marginal over the 1450 pod MTU (Cilium VXLAN).
--mtu-disc no clears DF so oversized control packets fragment through instead of
being silently dropped. Last MTU-path lever (OpenVPN 2.6 has no --tls-mtu).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
OpenVPN 2.6.16 rejects --tls-mtu (not a real option) and PIA doesn't support
--fragment, so control-channel packets can't be shrunk. Keep --hand-window 120 to
give the RSA-4096 handshake more time over a lossy/marginal path, plus tun-mtu 1300.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
RSA-4096 strong-preset handshake is marginal over the pod's 1450 MTU (Cilium VXLAN).
Lower tun-mtu to 1300 and add --tls-mtu 1200 (smaller control packets) + --hand-window
120 (more time to complete the handshake over a lossy path). Empirical test of the
MTU-tuning path (operator's choice over Cilium native routing).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
PIA's PF gateway 10.31.128.1 sits inside the /8 cluster-pool, so
FIREWALL_OUTBOUND_SUBNETS=10.0.0.0/8 made gluetun route it around the tunnel and
port forwarding timed out (getSignature to 10.31.128.1:19999). /16 covers actual
pod allocations and excludes PIA's 10.31.x.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Operator's working config is sweden.privacy.network:1197 with RSA-4096, AES-256-CBC,
SHA256 = PIA's 'strong' preset. gluetun defaults PIA to 'normal' (AES-128-GCM), a
cipher mismatch against the 1197 (strong) endpoint, so the handshake never completed
(silent after UDP link). PIA_ENCRYPTION_PRESET=strong matches it and selects 1197.
Reverts the 8080 port red herring.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
UPDATER_PERIOD only refreshes AFTER the tunnel is ready (chicken-and-egg: can't
connect with stale servers), and gluetun's bundled PIA Sweden list is retired
46.246.x (current is 158.173.x). Add an `update-servers` initContainer running
`gluetun update -providers "private internet access"` into a shared /gluetun
emptyDir the main container reads, so it dials current servers. UPDATER_PERIOD
raised to 720h (docs-compliant) for ongoing in-pod refresh.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
CORRECTION: the connection failures were gluetun dialing RETIRED PIA IPs (46.246.x
= dead → timeouts, which I mis-read as a network block). A fresh PIA .ovpn connects
from the same host, and PIA's current SE Stockholm servers (158.173.x) are reachable.
Cilium was independently cleared (default-allow, host-fw off, Hubble FORWARDED).
Fix: UPDATER_PERIOD=24h so gluetun fetches PIA's current servers; keep
OPENVPN_MTU/MSSFIX=1350 for the real Cilium-VXLAN 1450 pod MTU (data channel);
PIA default UDP port. Re-enable vpn-egress. Full correction in DECISIONS-LOG.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
jakman85andClaude Opus 4.8