# ============================================================================= # L0 STATIC — two egress paths, and the rule set that selects between them (R-NET-04) [EXECUTABLE] # Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster. # Spec: docs/00 R-NET-04 ("The platform MUST support **multiple egress paths** selected by # routing rules ... realized as two paths — the local `internet` path and the # in-cluster `vpn` path — not multi-ISP routing"), elaborated by docs/03 §"Egress # routing: two paths, one physical WAN": "the `internet` path is simply the default # route out"; "**Default is the WAN; the VPN is strictly opt-in.** Every pod — and # the nodes themselves — egress the local/private WAN by default and **never** the # VPN. Only services labeled `egress-class=vpn` are steered to the tunnel, and the # kill-switch blackholes **only** that class." # # WHAT THIS ADDS OVER R-NET-05/07 (already covered, and NOT re-tagged here) # ------------------------------------------------------------------------ # R-NET-05/07 are about a SERVICE — it declares a class, enforcement is outside it, the class # flips via GitOps — and are proven by tests/l1-rendering/egress-class-render and # tests/l2-behavioral/egress-class. R-NET-04 is about the PLATFORM's PATH SET and the RULES # that select among them, which nothing else asserts: # * exactly TWO paths, the same closed set in the schema, the admission policy and the # renderer (a third value would be a path nothing routes; "multi-ISP" is out of scope); # * selection is by a ROUTING RULE and the rule set is TOTAL AND DISJOINT — exactly one # Cilium steering rule, selecting only `egress-class=vpn`, and on the gateway node one # `ip rule` scoped to the tunnel address. NOTHING selects the `internet` class, because # absence of a matching rule IS the default path; # * the two paths COEXIST per DESTINATION: private space is excluded from the tunnel, so a # vpn-class pod's cluster/LAN traffic still takes the local path; # * the vpn path never becomes the DEFAULT — its default route lives in a dedicated routing # table, `route-noexec` is set and a pushed `redirect-gateway` is ignored; # * the kill-switch blackholes ONLY that class — the nft chain's policy is accept and every # DROP is scoped to the vpn class's own source set/fwmark, so `internet` fails OPEN. # (R-NET-06's opposite claim, that `vpn` fails CLOSED, is L3's vpn-killswitch.) # # WHY L0: every clause above is a statement about ALL rules — including rules that must be # ABSENT (nothing selects the default path; no unscoped rule; no unqualified drop; no default # route in the main table). A cluster can show which path two pods took; it cannot show that # no third rule exists or that the vocabulary is closed. # # ENVIRONMENT-AGNOSTIC: no address appears. The private ranges the tunnel must exclude are # checked as CONTAINMENT of the addresses each environment's own Flux overlay declares (pod # CIDR, Gateway VIP, LB pool, API host), so a new environment on a different LAN is checked # against its own values. # # GIVEN the repository as git tracks it, # WHEN the egress vocabulary, every CiliumEgressGatewayPolicy, the gateway node's policy # routing / OpenVPN / nftables templates, and every environment's overlay addresses are # parsed and cross-checked, # THEN there are exactly two paths with `internet` as the default; exactly one steering rule # selects exactly the vpn class by label; the gateway is selected by node label with a # named egress interface; all RFC1918 space and every address any environment assigns # itself is excluded from the tunnel; the tunnel's default route and its fail-closed # blackhole live only in a dedicated routing table behind a scoped `ip rule`; OpenVPN # cannot take the main table; and no nftables DROP is unscoped. # # STATUS: EXECUTABLE. `check.sh` is a pure offline assertion script (git tree + PyYAML; no # cluster, no network); conformance/run.sh --layer l0 and ci/l0.sh run it directly, and this # Chainsaw Test wraps the SAME script. # # KNOWN, DELIBERATE GAPS (enumerated in check.sh's header; this test claims none of them): # * BEHAVIOUR. That an unlabelled pod really takes the WAN path is asserted at L2 by # tests/l2-behavioral/egress-default-path (same requirement label); the two labelled # classes are R-NET-05's L2 test; the kill-switch firing is R-NET-06's L3 test. # * The RUNNING gateway node's routing tables / nftables ruleset. This reads the templates # that generate them; a hand-added rule on the box is invisible here. # * PHYSICAL MULTI-WAN's absence is NOT asserted — it is out of scope per OQ-1 and no # honest repository signal distinguishes "one WAN, correctly" from "one WAN, by default". # * IPv6 egress. # ============================================================================= apiVersion: chainsaw.kyverno.io/v1alpha1 kind: Test metadata: name: egress-paths-selected-by-rules labels: conformance.home.k3s/R-NET-04: "true" annotations: conformance.home.k3s/requirements: "R-NET-04" conformance.home.k3s/layer: "L0" # WHOLE-REPO SCOPE. Universally quantified over tracked YAML: every manifest that carries # an egress-class label/annotation must use a value the policy set actually steers. The # file most likely to break it is a NEW workload under apps/ — which attributes no # requirement — not platform/vpn-egress/ (the only path the map keys to R-NET-04). conformance.home.k3s/scope: "whole-repo" conformance.home.k3s/safety-critical: "false" conformance.home.k3s/executable: "true" conformance.home.k3s/intent: >- Given the tracked tree, when the egress vocabulary, every steering rule and the gateway node's policy-routing/OpenVPN/nftables templates are parsed, then the platform offers exactly two egress paths with the WAN as the default, exactly one rule selects exactly the vpn class, local destinations stay on the local path in every environment, the tunnel never becomes the default route, and nothing unscoped can blackhole the default path. spec: steps: - name: then-two-paths-exist-and-the-rules-select-between-them-totally-and-disjointly description: >- Run the offline assertion script over the tracked tree (same script ci/l0.sh and conformance/run.sh --layer l0 execute). try: - script: content: bash ./check.sh