Files
home-network-k3s-impl/conformance/tests/l2-behavioral/default-deny/chainsaw-test.yaml
T
jakman85andClaude Opus 5 cb1916f092 feat(conformance): cover R-PLAT-01, R-NET-04, R-SEC-03 — 88.2% -> 94.1%, and fix an unrecordable-proof defect
Coverage 45/51 -> 48/51, gaps 6 -> 3 (R-C2-01, R-C2-02, R-STATE-04), 0
safety-critical. --strict-coverage exit 0; run.sh --layer l0 11/11 PASS.

R-PLAT-01 — split L0 + L2, because "3 nodes with production-grade HA control
plane" is half authored and half observable.
  l0-static/ha-control-plane-as-code (92 assertions): environments DERIVED from
  the R-ENV-02 seam (never a hardcoded group name); k3s installed as `server` with
  no --datastore-endpoint/--disable-etcd anywhere; exactly ONE --cluster-init
  founder per environment; odd, quorum-capable server count with tolerance >= 1;
  tls-sans carry the VIP and every server IP; kube-vip leader-elected with
  cp_enable/svc_enable parsed from the template's OWN env pairs; each VIP outside
  its environment's LB pool and not a node IP.
  l2-behavioral/ha-control-plane: control-plane AND etcd role on every node,
  every node a REGISTERED etcd member, exactly 3 when the cluster says
  CLUSTER_ENV=prod, and an in-cluster Job curling /readyz through the VIP AND every
  server's own address — which proves the VIP fronts the API and that every server
  runs an apiserver, not one founder plus two joiners.

  An earlier draft probed those endpoints FROM THE RUNNER: that passes from prod's
  LAN and fails against dev (nodes behind the pfSense NAT), making the test a
  property of where it was launched. Hence the in-cluster Job.

R-NET-04 — what it adds over R-NET-05/07 is the PATH SET and the
totality/disjointness of the selecting rules, not another per-service check.
  l0-static/egress-paths-selected-by-rules (68 assertions): exactly two paths, the
  same closed set in schema + Kyverno policy + renderer; EXACTLY ONE
  CiliumEgressGatewayPolicy with a single selector matching only egress-class=vpn;
  nothing selects `internet` (absence of a rule IS the path); RFC1918 containment
  computed with `ipaddress` plus every address each overlay assigns itself; the
  tunnel default route and blackhole confined to table 51820 behind an ip rule that
  must carry a from/fwmark/iif selector; and the nft chain `policy accept` with
  every drop qualified — the internet-fails-OPEN direction that R-NET-06's L3 test
  does not assert.

R-SEC-03 — refuses the tautology "a test exists, therefore it is auditable".
  l0-static/auditability-proofs-producible (78 assertions) drives the real
  machinery: each of docs/07's five proofs must have an EXECUTABLE test at the layer
  the spec's wording demands (fault injection and DR drill => L3), taken from
  status.py's own verdict; run.sh --requirements must resolve and pass the gate's
  --require assertion; each proof declares a sentinel record_run.py can attribute,
  computed with record_run.parse_sentinels itself; and TWO FORGERIES are mounted
  against the real tools — a green JUnit whose log holds only a SKIP token, and a
  ledger row claiming `proven` at the root commit — which must come back not-proven
  and stale-proof. The five proofs are parsed from docs/07 when the spec is mounted,
  not hand-listed.

*** A REAL DEFECT FOUND AND FIXED, not asserted around. *** l2-behavioral's
default-deny, trusted-tls and gateway-ingress-reachable declared NO pass sentinel,
so record_run.py could never attribute a pass to them — the isolation and TLS audit
proofs were STRUCTURALLY UNRECORDABLE (landing as unknown-no-sentinel-declared,
and status.py requires all of a requirement's tests to be proven). Each now ends
with an additive terminal sentinel step reached only when every prior step passed.
Verified: all three carry it.

TWO HOLLOW DRAFT ASSERTIONS CAUGHT BY THE AGENT ITSELF, both the failure mode this
repo keeps hitting: (a) `svc_enable=false` matched a COMMENT in the kube-vip role
defaults, so flipping the template to true still passed — now parses the DaemonSet
template's env pairs; (b) the nft scoping check searched for the VALUE of a Jinja
variable while the template carries its NAME, matching nothing and passing
vacuously — templates are now rendered against role defaults first. 30 perturbations
across the three checks, all caught, plus 16 synthetic L2 perturbations via a
fake-kubectl shim.

R-C2-01/02 judged IMPLEMENTATION gaps, so no test was written: Homepage's
widgets.yaml carries only datetime and search, kubernetes.yaml is empty,
automountServiceAccountToken is false, and its NetworkPolicy allows DNS egress
only — no health source is even reachable. It has already drifted (crafty authors a
route with no tile). A test today would go red on both; it must ship WITH the
config fix.

VERIFIED: full CI gate PASS. Neither cluster mutated.

Refs R-PLAT-01, R-NET-04, R-SEC-03, R-TEST-01..05.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-09-06 19:35:18 -05:00

173 lines
7.7 KiB
YAML

# =============================================================================
# L2 BEHAVIORAL — Default-deny east-west (R-NET-01) [RUNNABLE]
# Catalog: docs/14-testing-and-conformance.md "Default-deny (R-NET-01)"
# Spec: docs/03-networking-isolation-and-egress.md "Default-deny baseline"
#
# EXEMPLAR: this file is the reference for the tagging convention and step
# style. Other scaffolds follow the same shape (see tests/README.md).
#
# GIVEN a fresh workload in a namespace with the default-deny baseline applied,
# WHEN it attempts any in-cluster connection except DNS,
# THEN the connection FAILS (fail-closed); DNS still succeeds.
#
# This asserts observable BEHAVIOR (a probe Job cannot connect) via ephemeral
# probe pods, not the mere existence of a policy object (R-TEST-01).
#
# RUNNABLE ON THE BASE CLUSTER: the platform-wide default-deny baseline (T-2.2)
# is not yet applied cluster-wide, so this test applies an equivalent baseline
# as its OWN hermetic fixture in the per-test namespace. The fixture is a
# standard Kubernetes NetworkPolicy (Cilium enforces it) — portable and
# version-agnostic; the platform's baseline will be the CiliumNetworkPolicy
# equivalent, but the asserted BEHAVIOR (DNS-only egress) is identical, which
# is what R-TEST-01 requires us to prove. Chainsaw auto-creates/cleans the
# namespace (hermetic); the probe Jobs are ephemeral and self-cleaned with it.
# =============================================================================
apiVersion: chainsaw.kyverno.io/v1alpha1
kind: Test
metadata:
name: default-deny
labels:
# --- requirement tags (authoritative; one label per verified R-* ID) ---
conformance.home.k3s/R-NET-01: "true"
annotations:
conformance.home.k3s/requirements: "R-NET-01"
conformance.home.k3s/layer: "L2"
conformance.home.k3s/safety-critical: "false"
# executable => this test runs real behavior against a cluster (not a scaffold).
conformance.home.k3s/executable: "true"
conformance.home.k3s/intent: >-
Given a bare workload in a namespace with the default-deny baseline, only
DNS egress succeeds; every other in-cluster connection is denied.
spec:
steps:
# -- GIVEN: apply the default-deny baseline to the per-test namespace. Deny all
# ingress + egress for every pod; allow ONLY DNS egress to the in-cluster
# resolver. This is the hermetic stand-in for the platform baseline (T-2.2).
- name: given-default-deny-baseline
try:
- apply:
resource:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny-baseline
spec:
podSelector: {}
policyTypes: [Ingress, Egress]
# no ingress rules => deny all ingress
ingress: []
# only DNS egress to kube-dns (CoreDNS) in kube-system
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
k8s-app: kube-dns
ports:
- protocol: UDP
port: 53
- protocol: TCP
port: 53
# -- THEN(DNS allowed): DNS egress MUST succeed even under default-deny. An
# ephemeral Job resolves an in-cluster name; we assert the Job SUCCEEDS.
# (Chainsaw polls the Job status within the assert timeout — bounded, no sleeps.)
- name: then-dns-succeeds
try:
- apply:
resource:
apiVersion: batch/v1
kind: Job
metadata:
name: dns-probe
spec:
backoffLimit: 4
activeDeadlineSeconds: 120
template:
metadata:
labels: { app: conformance-probe }
spec:
restartPolicy: Never
containers:
- name: probe
image: busybox:1.36.1
command:
- sh
- -c
- |
set -eu
# DNS is the one egress the baseline permits.
nslookup kubernetes.default.svc.cluster.local
echo DNS_OK
- assert:
resource:
apiVersion: batch/v1
kind: Job
metadata: { name: dns-probe }
status: { succeeded: 1 }
# -- THEN(everything else denied): a non-DNS in-cluster connection MUST fail
# closed. The Job attempts to reach the kube-apiserver ClusterIP (always
# present) on 443; the probe EXITS 0 only when that connection is BLOCKED,
# so a green Job proves the deny. A leak (connection succeeds) fails the Job.
- name: then-nondns-denied
try:
- apply:
resource:
apiVersion: batch/v1
kind: Job
metadata:
name: block-probe
spec:
backoffLimit: 4
activeDeadlineSeconds: 120
template:
metadata:
labels: { app: conformance-probe }
spec:
restartPolicy: Never
containers:
- name: probe
image: busybox:1.36.1
command:
- sh
- -c
- |
set -u
# DNS resolves (allowed); the TCP connect must be dropped.
if nc -w 4 kubernetes.default.svc.cluster.local 443 </dev/null 2>/dev/null; then
echo "LEAK: reached kube-apiserver under default-deny" 1>&2
exit 1
fi
echo DENIED_OK
- assert:
resource:
apiVersion: batch/v1
kind: Job
metadata: { name: block-probe }
status: { succeeded: 1 }
# -- chainsaw deletes the namespace (policy + probe Jobs) automatically; this
# block is the explicit hook for any out-of-namespace fixtures (none here).
finally:
- script:
content: 'echo "cleanup handled by chainsaw namespace teardown"'
# -----------------------------------------------------------------------------------
# PROOF-OF-PASS SENTINEL (R-SEC-03). Every step above is an assert/Job-status gate, so
# chainsaw already fails the test before reaching this step if any of them fails — but a
# green Chainsaw JUnit carries NO stdout, and the L2/L3 tests in this suite can self-skip
# GREEN, so lib/record_run.py refuses to treat a bare `passed` as evidence and records
# `unknown-no-sentinel-declared` instead. Without a declared token this test could pass
# forever and NEVER become a recorded proof of R-NET-01 — the isolation half of docs/07's audit proof list
# — which is precisely what docs/07 asks for ("These proofs are part of the platform
# acceptance suite, not one-time checks"). This step exists only to emit that token, and it
# is reached only when everything above has passed. Asserted by
# tests/l0-static/auditability-proofs-producible.
- name: then-emit-the-proof-of-pass-sentinel
try:
- script:
content: echo DEFAULT_DENY_PROVEN_OK
check:
(contains($stdout, 'DEFAULT_DENY_PROVEN_OK')): true