phase3(vpn): author Cilium Egress Gateway steering policy (ADR-0015)
Add platform/vpn-egress/egress-steering.yaml — a CiliumEgressGatewayPolicy that steers homelab.keithjackson.dev/egress-class=vpn pods out a gateway node's host tun0 split-tunnel interface, per ADR-0015 (node-level tunnel + Cilium Egress Gateway). Honors all four ADR-0015 invariants: only vpn-class pods are selected (default/internet + node stay on the local/WAN path), RFC1918 + cluster ranges are excludedCIDRs so only public internet is tunneled and tun0 is never a default route, and the host nftables kill-switch (Ansible) fails vpn-class CLOSED while internet-class fails OPEN. Interface contract shared verbatim with the parallel Ansible host-tunnel role: node label homelab.keithjackson.dev/egress-gateway=true, interface tun0, client label homelab.keithjackson.dev/egress-class=vpn. Cilium Egress Gateway prerequisites confirmed enabled (egressGateway.enabled, bpf.masquerade, kubeProxyReplacement all true, [parity]) — no HelmRelease change. Intentionally NOT added to platform/vpn-egress/kustomization.yaml: the datapath stays out of the runtime until the live tunnel exists and the egress-class / vpn-killswitch conformance tests pass (README rule; DoD). Gluetun is retained as the validated fallback until then. Serves R-NET-05/06/07. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
jakman85andClaude Opus 4.8 committed
1 parent
272a7d7d2f
commit
38fcfadbf0
1 file changed
+88
@@ -0,0 +1,88 @@
|
|||||||
|
---
|
||||||
|
# ============================================================================
|
||||||
|
# Cilium Egress Gateway steering for `egress-class=vpn` pods — ADR-0015.
|
||||||
|
# Serves: R-NET-05, R-NET-06, R-NET-07.
|
||||||
|
#
|
||||||
|
# ##########################################################################
|
||||||
|
# ## NOT WIRED INTO THE RUNTIME YET — DO NOT ADD TO kustomization.yaml ##
|
||||||
|
# ## UNTIL THE HOST tun0 TUNNEL IS LIVE AND THIS DATAPATH IS VALIDATED. ##
|
||||||
|
# ##########################################################################
|
||||||
|
#
|
||||||
|
# This policy steers vpn-class pods out a gateway node's host `tun0` interface. It is INERT and
|
||||||
|
# potentially disruptive until the Ansible-managed host OpenVPN split-tunnel service actually exists
|
||||||
|
# on a node labeled `homelab.keithjackson.dev/egress-gateway=true`. If Flux shipped this before the
|
||||||
|
# tunnel is up, vpn-class pods would be SNAT'd to a non-existent interface — an unvalidated datapath.
|
||||||
|
# Per platform/vpn-egress/README.md: "the steering datapath is intentionally NOT committed [to the
|
||||||
|
# runtime kustomization] until the mechanism is chosen + validated." ADR-0015 chose the mechanism
|
||||||
|
# (node tunnel + Cilium Egress Gateway); this file IS that datapath, authored but held out of the
|
||||||
|
# resource list for the PM to enable after a live-tunnel validation (see TODO at the bottom).
|
||||||
|
#
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# PREREQUISITES CONFIRMED (2026-08-24):
|
||||||
|
# * Cilium Egress Gateway is ENABLED per ADR-0002 — platform/cilium/helmrelease.yaml and
|
||||||
|
# platform/cilium/values.yaml both set (all [parity] with the Ansible bootstrap render):
|
||||||
|
# egressGateway.enabled: true
|
||||||
|
# bpf.masquerade: true (Egress Gateway REQUIRES eBPF masquerade)
|
||||||
|
# kubeProxyReplacement: true
|
||||||
|
# No Cilium HelmRelease change is needed for this policy. (If any of the three were false this
|
||||||
|
# would be an ESCALATION, not a silent HelmRelease edit — they are all true, so we proceed.)
|
||||||
|
#
|
||||||
|
# INTERFACE CONTRACT (shared verbatim with the parallel Ansible host-tunnel role — ADR-0015):
|
||||||
|
# * Gateway node label ...... homelab.keithjackson.dev/egress-gateway=true (applied by Ansible)
|
||||||
|
# * Host tunnel interface .... tun0 (split-tunnel OpenVPN)
|
||||||
|
# * Client selection ......... homelab.keithjackson.dev/egress-class=vpn (Kyverno-enforced)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
apiVersion: cilium.io/v2
|
||||||
|
kind: CiliumEgressGatewayPolicy
|
||||||
|
metadata:
|
||||||
|
name: vpn-egress-steering
|
||||||
|
annotations:
|
||||||
|
homelab.keithjackson.dev/requirements: R-NET-05,R-NET-06,R-NET-07
|
||||||
|
homelab.keithjackson.dev/adr: ADR-0015
|
||||||
|
spec:
|
||||||
|
# --- Client selection (INVARIANTS 1 & 2) --------------------------------
|
||||||
|
# ONLY pods explicitly opted into the vpn class are matched. Default/internet-class pods and node
|
||||||
|
# traffic are never selected, so they stay on the local/private-WAN path (invariant 1: default is
|
||||||
|
# the local path). Because ONLY these flows are steered, tun0 never becomes a default route for the
|
||||||
|
# node or for any other pod (invariant 2). The selector is intentionally cluster-wide (no namespace
|
||||||
|
# scope) so the class label alone is the contract — every namespace's vpn-class pods route here.
|
||||||
|
selectors:
|
||||||
|
- podSelector:
|
||||||
|
matchLabels:
|
||||||
|
homelab.keithjackson.dev/egress-class: vpn
|
||||||
|
# --- What gets tunneled (INVARIANT 1: only PUBLIC internet leaves via the VPN) ------------------
|
||||||
|
# destinationCIDRs is the full internet; excludedCIDRs carves out everything that must stay LOCAL:
|
||||||
|
# all RFC1918 private ranges (which also cover the Cilium cluster-pool pod CIDR 10.0.0.0/8, the k3s
|
||||||
|
# service/pod ranges, and the LAN 192.168.x/10.73.x node network). So intra-cluster east-west,
|
||||||
|
# pod->service, and LAN traffic are NOT SNAT'd out tun0 — only genuinely public destinations are.
|
||||||
|
# Excluded flows fall through to Cilium's normal local datapath (invariant 1).
|
||||||
|
destinationCIDRs:
|
||||||
|
- "0.0.0.0/0"
|
||||||
|
excludedCIDRs:
|
||||||
|
- "10.0.0.0/8" # RFC1918 + Cilium cluster-pool pod CIDR (CLUSTER_POD_CIDR) + k3s service net
|
||||||
|
- "172.16.0.0/12" # RFC1918
|
||||||
|
- "192.168.0.0/16" # RFC1918 (home LAN / node network family)
|
||||||
|
# --- Where it egresses --------------------------------------------------
|
||||||
|
# The gateway node (Ansible-labeled) SNATs matched traffic out its host split-tunnel `tun0`. When
|
||||||
|
# tun0 is absent/down, Cilium has no egress interface here; the host nftables kill-switch (Ansible,
|
||||||
|
# ADR-0015) is the authoritative fail-closed backstop that DROPS this vpn-class traffic — scoped to
|
||||||
|
# vpn-class only, so default/internet-class is unaffected (invariants 3 & 4: vpn fails CLOSED,
|
||||||
|
# internet fails OPEN).
|
||||||
|
egressGateway:
|
||||||
|
nodeSelector:
|
||||||
|
matchLabels:
|
||||||
|
homelab.keithjackson.dev/egress-gateway: "true"
|
||||||
|
interface: tun0
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# TODO(PM — enable after live validation, then retire Gluetun):
|
||||||
|
# 1. Confirm the Ansible host-tunnel role has brought `tun0` up on a node labeled
|
||||||
|
# homelab.keithjackson.dev/egress-gateway=true, and the host nftables kill-switch is in place.
|
||||||
|
# 2. Add `- egress-steering.yaml` to platform/vpn-egress/kustomization.yaml so Flux ships it.
|
||||||
|
# 3. Run the `egress-class` conformance test: vpn-class pod shows the PIA IP, internet-class shows
|
||||||
|
# the WAN IP, and a git label flip re-routes (R-NET-05/07).
|
||||||
|
# 4. Run the `vpn-killswitch` conformance test: force tun0 down => vpn-class blackholed (no leak),
|
||||||
|
# internet-class unaffected (R-NET-06).
|
||||||
|
# 5. Only AFTER both pass green: retire the Gluetun Deployment (remove gluetun-gateway.yaml from
|
||||||
|
# kustomization.yaml) per ADR-0015 ("Gluetun is dropped as the gateway"). Do NOT retire it in
|
||||||
|
# the same change that enables this — keep the validated fallback until steering is proven.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
Reference in new issue
Block a user