# ============================================================================= # L0 STATIC — the five auditability proofs can be produced on demand (R-SEC-03) [EXECUTABLE] # Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster. # Spec: docs/00 R-SEC-03 ("All the above properties MUST be demonstrable/auditable (an # operator can prove isolation, egress path, cert validity, and backup integrity)"), # made concrete by docs/07 §"Auditability requirements (R-SEC-03) — what an operator # must be able to prove": Isolation (denied-probe test), Egress path (egress IP # check), Kill-switch (fault-injection test), TLS (cert status + external check), # Backups (DR drill) — "These proofs are part of the platform acceptance suite, not # one-time checks", with the acceptance gate "The five auditability proofs above can # be produced on demand." # # HOW THIS AVOIDS BEING A TAUTOLOGY (the whole design question for a meta-requirement) # ----------------------------------------------------------------------------------- # "A test exists for R-NET-01, therefore isolation is auditable" would pass in any suite that # compiles and would keep passing while every proof rotted into a scaffold. Four distinct # failure modes are asserted instead, each by DRIVING the real machinery: # (a) each of the five proofs has an EXECUTABLE test — not a placeholder scaffold — at the layer # the spec's own words require (fault injection / a DR drill => L3), taken from # lib/status.py's own verdict so this check cannot disagree with the status report; # (b) the proofs are PRODUCIBLE by the documented command: `run.sh --requirements ` # must resolve to real test dirs AND pass the merge gate's coverage assertion; # (c) each Chainsaw-layer proof declares a PASS SENTINEL that lib/record_run.py can # attribute — a proof that passes but records nothing leaves the operator with a green # terminal and no evidence; # (d) the evidence chain REFUSES FORGERIES: this check mounts two of its own against the # real tools — a green JUnit for a real proof test whose log holds only that test's SKIP # token, and a ledger row claiming `proven` at the repository's ROOT commit — and # requires record_run.py to report not-proven and status.py to report `stale-proof`. It # also drives `record_run.py --selftest` and `status.py --selftest`, which NOTHING in # ci/ or run.sh ran before this test existed. # The five proofs are not hand-listed where the spec can be read: when the read-only spec repo # is mounted, docs/07's bullet labels are parsed and must match this check's mapping exactly, # so a sixth proof added to the spec fails here instead of being silently unverified. # # WHY L0: every clause is a property of the repository and its own tooling, needs no cluster, # and belongs in the merge gate — an audit chain is worth having only if it is intact BEFORE # anyone needs it. # # GIVEN the repository as git tracks it (plus the read-only spec repo when mounted), # WHEN lib/status.py, lib/common.sh's requirement resolver, run.sh's coverage gate, # lib/record_run.py's sentinel parser and both evidence tools' self-tests are DRIVEN, # and two synthetic forgeries are presented to them, # THEN each of docs/07's five proofs maps to an executable test at the required layer, the # documented one-command invocation selects and asserts them, each is capturable as a # ledger row, and neither a self-skip nor a stale row can be recorded as proof. # # STATUS: EXECUTABLE. `check.sh` is a pure offline assertion script (git tree + the harness's # own stdlib tooling; no cluster, no network); conformance/run.sh --layer l0 and ci/l0.sh run # it directly, and this Chainsaw Test wraps the SAME script. # # WHAT THIS TEST FIXED WHILE BEING WRITTEN (recorded, not hidden): three of the tests that ARE # the automated half of docs/07's isolation and TLS proofs — l2-behavioral/default-deny, # l2-behavioral/trusted-tls and l2-behavioral/gateway-ingress-reachable — declared NO pass # sentinel, so lib/record_run.py could never record them as proven and those two audit proofs # were structurally unrecordable. Each now ends with a proof-of-pass sentinel step. The # assertion here was NOT relaxed to accommodate them (R-TEST-06). # # KNOWN, DELIBERATE GAPS (enumerated in check.sh's header; this test claims none of them): # * That the five proofs PASS. This asserts they are present, strong, runnable by one # command, capturable, and that the chain refuses forgeries — not that isolation holds # today (R-NET-01's test) or that the restore works (R-STATE-03's). # * HUBBLE. docs/07 names Hubble flow logs as part of the isolation proof; no test in this # suite queries Hubble. Only the denied-probe half is automated — a real, named gap. # * LEDGER COMPLETENESS OVER TIME. Nothing offline can prove a row was never removed; that # would need signed history. # * R-SEC-03's wording is broader than docs/07's five bullets. Suite-wide coverage of every # requirement is R-TEST-01/04's claim (tests/meta/harness-selfcheck) and is deliberately # not re-asserted here. # ============================================================================= apiVersion: chainsaw.kyverno.io/v1alpha1 kind: Test metadata: name: auditability-proofs-producible labels: conformance.home.k3s/R-SEC-03: "true" annotations: conformance.home.k3s/requirements: "R-SEC-03" conformance.home.k3s/layer: "L0" # WHOLE-REPO SCOPE. This is a META-invariant over the ENTIRE conformance suite: the five # spec-named proofs must each still have an EXECUTABLE test at the right layer, with a # pass sentinel, resolvable by the documented command. Every one of its inputs lives under # conformance/, which ci/touched-requirements.sh keys to R-TEST-01..04 — never to R-SEC-03. # So downgrading conformance/tests/l3-fault-injection/dr-restore to a scaffold attributes # R-STATE-03/R-TEST-*, and requirement-scoped selection drops the one check that notices # the audit evidence just stopped being producible. conformance.home.k3s/scope: "whole-repo" conformance.home.k3s/safety-critical: "false" conformance.home.k3s/executable: "true" conformance.home.k3s/intent: >- Given the tracked tree and the harness's own tooling, when status.py, the requirement resolver, run.sh's coverage gate, record_run.py's sentinel parser and both evidence tools' self-tests are driven and two synthetic forgeries are presented to them, then each of docs/07's five auditability proofs has an executable test at the layer the spec requires, is selectable and asserted by the documented one-command invocation, is capturable as a ledger row, and neither a self-skip nor a stale proof can be recorded as evidence. spec: steps: - name: then-each-of-the-five-proofs-is-producible-and-the-evidence-chain-refuses-forgeries description: >- Run the offline assertion script over the tracked tree (same script ci/l0.sh and conformance/run.sh --layer l0 execute). try: - script: content: bash ./check.sh # --------------------------------------------------------------------------------------------- # NOTE ON WORDING (deliberate, do not "improve"): this file and check.sh say "placeholder # scaffold" rather than the four-letter marker lib/status.py looks for, because that detector is # a bare substring test over the whole file. Writing the marker in PROSE — even to say a proof # must NOT be one — makes status.py classify THIS test as a scaffold, so the R-SEC-03 test would # report itself as unexecutable. Observed and worked around here; the detector's crudeness is # reported to the PM rather than loosened, since loosening a fake-green guard to suit one file is # the wrong trade (R-TEST-06).