Coverage 45/51 -> 48/51, gaps 6 -> 3 (R-C2-01, R-C2-02, R-STATE-04), 0 safety-critical. --strict-coverage exit 0; run.sh --layer l0 11/11 PASS. R-PLAT-01 — split L0 + L2, because "3 nodes with production-grade HA control plane" is half authored and half observable. l0-static/ha-control-plane-as-code (92 assertions): environments DERIVED from the R-ENV-02 seam (never a hardcoded group name); k3s installed as `server` with no --datastore-endpoint/--disable-etcd anywhere; exactly ONE --cluster-init founder per environment; odd, quorum-capable server count with tolerance >= 1; tls-sans carry the VIP and every server IP; kube-vip leader-elected with cp_enable/svc_enable parsed from the template's OWN env pairs; each VIP outside its environment's LB pool and not a node IP. l2-behavioral/ha-control-plane: control-plane AND etcd role on every node, every node a REGISTERED etcd member, exactly 3 when the cluster says CLUSTER_ENV=prod, and an in-cluster Job curling /readyz through the VIP AND every server's own address — which proves the VIP fronts the API and that every server runs an apiserver, not one founder plus two joiners. An earlier draft probed those endpoints FROM THE RUNNER: that passes from prod's LAN and fails against dev (nodes behind the pfSense NAT), making the test a property of where it was launched. Hence the in-cluster Job. R-NET-04 — what it adds over R-NET-05/07 is the PATH SET and the totality/disjointness of the selecting rules, not another per-service check. l0-static/egress-paths-selected-by-rules (68 assertions): exactly two paths, the same closed set in schema + Kyverno policy + renderer; EXACTLY ONE CiliumEgressGatewayPolicy with a single selector matching only egress-class=vpn; nothing selects `internet` (absence of a rule IS the path); RFC1918 containment computed with `ipaddress` plus every address each overlay assigns itself; the tunnel default route and blackhole confined to table 51820 behind an ip rule that must carry a from/fwmark/iif selector; and the nft chain `policy accept` with every drop qualified — the internet-fails-OPEN direction that R-NET-06's L3 test does not assert. R-SEC-03 — refuses the tautology "a test exists, therefore it is auditable". l0-static/auditability-proofs-producible (78 assertions) drives the real machinery: each of docs/07's five proofs must have an EXECUTABLE test at the layer the spec's wording demands (fault injection and DR drill => L3), taken from status.py's own verdict; run.sh --requirements must resolve and pass the gate's --require assertion; each proof declares a sentinel record_run.py can attribute, computed with record_run.parse_sentinels itself; and TWO FORGERIES are mounted against the real tools — a green JUnit whose log holds only a SKIP token, and a ledger row claiming `proven` at the root commit — which must come back not-proven and stale-proof. The five proofs are parsed from docs/07 when the spec is mounted, not hand-listed. *** A REAL DEFECT FOUND AND FIXED, not asserted around. *** l2-behavioral's default-deny, trusted-tls and gateway-ingress-reachable declared NO pass sentinel, so record_run.py could never attribute a pass to them — the isolation and TLS audit proofs were STRUCTURALLY UNRECORDABLE (landing as unknown-no-sentinel-declared, and status.py requires all of a requirement's tests to be proven). Each now ends with an additive terminal sentinel step reached only when every prior step passed. Verified: all three carry it. TWO HOLLOW DRAFT ASSERTIONS CAUGHT BY THE AGENT ITSELF, both the failure mode this repo keeps hitting: (a) `svc_enable=false` matched a COMMENT in the kube-vip role defaults, so flipping the template to true still passed — now parses the DaemonSet template's env pairs; (b) the nft scoping check searched for the VALUE of a Jinja variable while the template carries its NAME, matching nothing and passing vacuously — templates are now rendered against role defaults first. 30 perturbations across the three checks, all caught, plus 16 synthetic L2 perturbations via a fake-kubectl shim. R-C2-01/02 judged IMPLEMENTATION gaps, so no test was written: Homepage's widgets.yaml carries only datetime and search, kubernetes.yaml is empty, automountServiceAccountToken is false, and its NetworkPolicy allows DNS egress only — no health source is even reachable. It has already drifted (crafty authors a route with no tile). A test today would go red on both; it must ship WITH the config fix. VERIFIED: full CI gate PASS. Neither cluster mutated. Refs R-PLAT-01, R-NET-04, R-SEC-03, R-TEST-01..05. Co-Authored-By: Claude Opus 5 <[email protected]>
173 lines
7.7 KiB
YAML
173 lines
7.7 KiB
YAML
# =============================================================================
|
|
# L2 BEHAVIORAL — Default-deny east-west (R-NET-01) [RUNNABLE]
|
|
# Catalog: docs/14-testing-and-conformance.md "Default-deny (R-NET-01)"
|
|
# Spec: docs/03-networking-isolation-and-egress.md "Default-deny baseline"
|
|
#
|
|
# EXEMPLAR: this file is the reference for the tagging convention and step
|
|
# style. Other scaffolds follow the same shape (see tests/README.md).
|
|
#
|
|
# GIVEN a fresh workload in a namespace with the default-deny baseline applied,
|
|
# WHEN it attempts any in-cluster connection except DNS,
|
|
# THEN the connection FAILS (fail-closed); DNS still succeeds.
|
|
#
|
|
# This asserts observable BEHAVIOR (a probe Job cannot connect) via ephemeral
|
|
# probe pods, not the mere existence of a policy object (R-TEST-01).
|
|
#
|
|
# RUNNABLE ON THE BASE CLUSTER: the platform-wide default-deny baseline (T-2.2)
|
|
# is not yet applied cluster-wide, so this test applies an equivalent baseline
|
|
# as its OWN hermetic fixture in the per-test namespace. The fixture is a
|
|
# standard Kubernetes NetworkPolicy (Cilium enforces it) — portable and
|
|
# version-agnostic; the platform's baseline will be the CiliumNetworkPolicy
|
|
# equivalent, but the asserted BEHAVIOR (DNS-only egress) is identical, which
|
|
# is what R-TEST-01 requires us to prove. Chainsaw auto-creates/cleans the
|
|
# namespace (hermetic); the probe Jobs are ephemeral and self-cleaned with it.
|
|
# =============================================================================
|
|
apiVersion: chainsaw.kyverno.io/v1alpha1
|
|
kind: Test
|
|
metadata:
|
|
name: default-deny
|
|
labels:
|
|
# --- requirement tags (authoritative; one label per verified R-* ID) ---
|
|
conformance.home.k3s/R-NET-01: "true"
|
|
annotations:
|
|
conformance.home.k3s/requirements: "R-NET-01"
|
|
conformance.home.k3s/layer: "L2"
|
|
conformance.home.k3s/safety-critical: "false"
|
|
# executable => this test runs real behavior against a cluster (not a scaffold).
|
|
conformance.home.k3s/executable: "true"
|
|
conformance.home.k3s/intent: >-
|
|
Given a bare workload in a namespace with the default-deny baseline, only
|
|
DNS egress succeeds; every other in-cluster connection is denied.
|
|
spec:
|
|
steps:
|
|
# -- GIVEN: apply the default-deny baseline to the per-test namespace. Deny all
|
|
# ingress + egress for every pod; allow ONLY DNS egress to the in-cluster
|
|
# resolver. This is the hermetic stand-in for the platform baseline (T-2.2).
|
|
- name: given-default-deny-baseline
|
|
try:
|
|
- apply:
|
|
resource:
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: NetworkPolicy
|
|
metadata:
|
|
name: default-deny-baseline
|
|
spec:
|
|
podSelector: {}
|
|
policyTypes: [Ingress, Egress]
|
|
# no ingress rules => deny all ingress
|
|
ingress: []
|
|
# only DNS egress to kube-dns (CoreDNS) in kube-system
|
|
egress:
|
|
- to:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: kube-system
|
|
podSelector:
|
|
matchLabels:
|
|
k8s-app: kube-dns
|
|
ports:
|
|
- protocol: UDP
|
|
port: 53
|
|
- protocol: TCP
|
|
port: 53
|
|
|
|
# -- THEN(DNS allowed): DNS egress MUST succeed even under default-deny. An
|
|
# ephemeral Job resolves an in-cluster name; we assert the Job SUCCEEDS.
|
|
# (Chainsaw polls the Job status within the assert timeout — bounded, no sleeps.)
|
|
- name: then-dns-succeeds
|
|
try:
|
|
- apply:
|
|
resource:
|
|
apiVersion: batch/v1
|
|
kind: Job
|
|
metadata:
|
|
name: dns-probe
|
|
spec:
|
|
backoffLimit: 4
|
|
activeDeadlineSeconds: 120
|
|
template:
|
|
metadata:
|
|
labels: { app: conformance-probe }
|
|
spec:
|
|
restartPolicy: Never
|
|
containers:
|
|
- name: probe
|
|
image: busybox:1.36.1
|
|
command:
|
|
- sh
|
|
- -c
|
|
- |
|
|
set -eu
|
|
# DNS is the one egress the baseline permits.
|
|
nslookup kubernetes.default.svc.cluster.local
|
|
echo DNS_OK
|
|
- assert:
|
|
resource:
|
|
apiVersion: batch/v1
|
|
kind: Job
|
|
metadata: { name: dns-probe }
|
|
status: { succeeded: 1 }
|
|
|
|
# -- THEN(everything else denied): a non-DNS in-cluster connection MUST fail
|
|
# closed. The Job attempts to reach the kube-apiserver ClusterIP (always
|
|
# present) on 443; the probe EXITS 0 only when that connection is BLOCKED,
|
|
# so a green Job proves the deny. A leak (connection succeeds) fails the Job.
|
|
- name: then-nondns-denied
|
|
try:
|
|
- apply:
|
|
resource:
|
|
apiVersion: batch/v1
|
|
kind: Job
|
|
metadata:
|
|
name: block-probe
|
|
spec:
|
|
backoffLimit: 4
|
|
activeDeadlineSeconds: 120
|
|
template:
|
|
metadata:
|
|
labels: { app: conformance-probe }
|
|
spec:
|
|
restartPolicy: Never
|
|
containers:
|
|
- name: probe
|
|
image: busybox:1.36.1
|
|
command:
|
|
- sh
|
|
- -c
|
|
- |
|
|
set -u
|
|
# DNS resolves (allowed); the TCP connect must be dropped.
|
|
if nc -w 4 kubernetes.default.svc.cluster.local 443 </dev/null 2>/dev/null; then
|
|
echo "LEAK: reached kube-apiserver under default-deny" 1>&2
|
|
exit 1
|
|
fi
|
|
echo DENIED_OK
|
|
- assert:
|
|
resource:
|
|
apiVersion: batch/v1
|
|
kind: Job
|
|
metadata: { name: block-probe }
|
|
status: { succeeded: 1 }
|
|
# -- chainsaw deletes the namespace (policy + probe Jobs) automatically; this
|
|
# block is the explicit hook for any out-of-namespace fixtures (none here).
|
|
finally:
|
|
- script:
|
|
content: 'echo "cleanup handled by chainsaw namespace teardown"'
|
|
# -----------------------------------------------------------------------------------
|
|
# PROOF-OF-PASS SENTINEL (R-SEC-03). Every step above is an assert/Job-status gate, so
|
|
# chainsaw already fails the test before reaching this step if any of them fails — but a
|
|
# green Chainsaw JUnit carries NO stdout, and the L2/L3 tests in this suite can self-skip
|
|
# GREEN, so lib/record_run.py refuses to treat a bare `passed` as evidence and records
|
|
# `unknown-no-sentinel-declared` instead. Without a declared token this test could pass
|
|
# forever and NEVER become a recorded proof of R-NET-01 — the isolation half of docs/07's audit proof list
|
|
# — which is precisely what docs/07 asks for ("These proofs are part of the platform
|
|
# acceptance suite, not one-time checks"). This step exists only to emit that token, and it
|
|
# is reached only when everything above has passed. Asserted by
|
|
# tests/l0-static/auditability-proofs-producible.
|
|
- name: then-emit-the-proof-of-pass-sentinel
|
|
try:
|
|
- script:
|
|
content: echo DEFAULT_DENY_PROVEN_OK
|
|
check:
|
|
(contains($stdout, 'DEFAULT_DENY_PROVEN_OK')): true
|