ADR-0015: add non-negotiable egress invariants (default=WAN, scoped kill-switch)
Make the operator's requirement explicit: default (every pod + the nodes) egresses the local/private WAN, never the VPN; VPN is strictly opt-in via egress-class=vpn; the kill-switch blackholes ONLY vpn-class. The two classes fail in opposite directions — internet fails open (WAN), vpn fails closed (blackhole). Added as ADR-0015 Invariants + a default-WAN principle in docs/03, gated by the egress-class/vpn-killswitch conformance tests. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
1 parent
6770087d68
commit
c68349f4ab
2 files changed
+31
-1
No files matched your search
@@ -49,6 +49,27 @@ The user-facing contract is unchanged: services are labeled `egress-class=vpn` (
|
||||
independent egress path**, and the rules are **enforced outside the service** — now via a Cilium policy + host
|
||||
nft rather than an in-cluster gateway pod.
|
||||
|
||||
### Invariants (NON-NEGOTIABLE — implementation must satisfy all four)
|
||||
The two egress classes have **opposite fail-safe directions**; getting this wrong either leaks VPN traffic or
|
||||
breaks the cluster. These are hard requirements, not preferences:
|
||||
|
||||
1. **Default is the local/private WAN path — for everything.** Every pod (regardless of namespace) AND the
|
||||
nodes themselves egress via the normal private-network/WAN stack by default. Traffic is on the VPN **only**
|
||||
if a service explicitly opts in with `egress-class=vpn`.
|
||||
2. **The VPN tunnel is never a default route.** Not for the node, not for any pod. `tun0` carries **only** the
|
||||
flows a `CiliumEgressGatewayPolicy` explicitly selects (`egress-class=vpn`). The host tunnel is split-tunnel
|
||||
(`route-noexec`/policy routing) precisely to guarantee this.
|
||||
3. **The kill-switch blackhole is scoped to vpn-class only.** When `tun0` is down, the nftables rule drops
|
||||
**only** the `egress-class=vpn` (Cilium-egress-gateway-marked) traffic. It MUST NOT match or drop default/
|
||||
`internet`-class traffic or node traffic. `internet`-class egress is unaffected whether the tunnel is up
|
||||
or down.
|
||||
4. **Opposite fail directions.** Default/`internet` traffic **fails open** (toward the WAN — never blackholed).
|
||||
`egress-class=vpn` traffic **fails closed** (toward blackhole — never leaks to the WAN). A misconfiguration
|
||||
or partial failure must degrade in these directions, never the reverse.
|
||||
|
||||
These are gated by the `egress-class` (default/internet shows WAN IP; vpn shows VPN IP) and `vpn-killswitch`
|
||||
(tunnel down ⇒ vpn-class blackholed, internet-class unaffected) conformance tests.
|
||||
|
||||
## Consequences
|
||||
- Steering is **Cilium-native and declarative** (a `CiliumEgressGatewayPolicy` in Git) and touches no app pods
|
||||
— flipping a service's `egress-class` label re-routes it with no workload rearchitecting.
|
||||
|
||||
@@ -92,7 +92,16 @@ VPN-class services: **sonarr, radarr, jackett, qbittorrent**. Per OQ-3 the desig
|
||||
shared VPN connection** — not a per-service sidecar — so that **network rules are enforced outside
|
||||
the service**. The steering mechanism was settled by **ADR-0015** after the original in-cluster-Gluetun
|
||||
approach proved un-steerable (a Gluetun *pod*'s `tun0` is invisible to the node datapath, and
|
||||
`hostNetwork` Gluetun hijacks the node's own default route). The model:
|
||||
`hostNetwork` Gluetun hijacks the node's own default route).
|
||||
|
||||
**Default is the WAN; the VPN is strictly opt-in.** Every pod — and the nodes themselves — egress the
|
||||
local/private WAN by default and **never** the VPN. Only services labeled `egress-class=vpn` are steered to
|
||||
the tunnel, and the kill-switch blackholes **only** that class. The two classes fail in **opposite**
|
||||
directions: `internet` fails **open** (WAN, never blackholed); `vpn` fails **closed** (blackhole, never leaks
|
||||
to WAN). See the ADR-0015 invariants — implementation must not route the default through the VPN or blackhole
|
||||
the wrong class.
|
||||
|
||||
The model:
|
||||
|
||||
- **Tunnel on the node, not in a pod.** One or two labeled **egress-gateway nodes** run a **host OpenVPN
|
||||
systemd service** — **PIA over OpenVPN-UDP** (provider/protocol per ADR-0013; "non-TCP or better"),
|
||||
|
||||
Reference in new issue
Block a user