# ============================================================================= # L0 STATIC — every WHOLE-REPO invariant is actually wired into the gate (R-TEST-03) [EXECUTABLE] # Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster. # # WHAT THIS REPLACES, AND WHY IT HAS TO EXIST # ------------------------------------------------------------------------------ # R-TEST-03 gates a merge on the conformance subset the change TOUCHES. That selection has one # blind spot, and it is the worst-shaped kind: for an invariant that quantifies over the WHOLE # tracked tree (or over subtrees the coarse path->requirement map keys to NOTHING — apps/, # infra/, most of platform/), the change most likely to break it attributes no requirement at # all, so requirement-scoped selection filters the check out of EXACTLY that change. # # The repo's first answer was a hand-written "UNCONDITIONAL" stage per such check in ci/l0.sh # (3c subst-safety, 3d keycloak-client-secrets, 3e bulk-tier), and each of those three checks # asserted its OWN wiring by grepping ci/l0.sh for its own check.sh path. That pattern did not # scale — and it failed SILENTLY in the worst way, because whoever added the NEXT whole-repo # invariant had to REMEMBER to hand-write a stage, and if they forgot, the check ran only when # some unrelated requirement happened to be touched. # # The stages are gone. A test now DECLARES `conformance.home.k3s/scope: "whole-repo"` and the # runner unions every declaring test into the selection (`run.sh --include-scope whole-repo`). # This test is what keeps that declarative mechanism honest, for ALL declaring tests at once # instead of three of them separately: # # (1) the mechanism EXISTS end to end — lib/common.sh resolves the annotation, run.sh's # --include-scope unions it into select_dirs, and ci/l0.sh passes it; # (2) it WORKS, proven empirically and falsifiably: with a requirement filter that matches # NO test, `run.sh --layer l0 --print-selection --include-scope whole-repo` must still # resolve to every whole-repo-scoped L0 test — and WITHOUT the flag, to none of them; # (3) a SKIP of a scope-declared check is UNWIRED, NOT GREEN — proven by running the offline # runner against synthetic checks and asserting exit 3 for the declared one and exit 0 # for the undeclared one. This is the rc-3 semantics the deleted stages encoded # ("check SKIPPED (clusters/ absent) — not a pass"), which is the half most likely to # be lost in a refactor because nothing else notices when it disappears; # (4) the bespoke pattern has not crept back: a ci/*.sh that hardcodes a test's check.sh path # must at least belong to a test that ALSO declares the scope, so a hand-written stage # can never be the ONLY thing running a whole-repo check; # (5) ci/gate.sh still BLOCKS the merge when the l0 stage fails (the assertion each of the # three deleted stages' checks carried, kept here once); # (6) the declared set is non-empty and every member is executable — a mechanism that # resolves to nothing would pass every other assertion here while checking nothing. # # This test declares whole-repo scope ITSELF, for the obvious reason: an invariant about the # gate's own selection must not be selectable away by the gate's own selection. # ============================================================================= apiVersion: chainsaw.kyverno.io/v1alpha1 kind: Test metadata: name: whole-repo-invariants-wired labels: conformance.home.k3s/R-TEST-03: "true" annotations: conformance.home.k3s/requirements: "R-TEST-03" conformance.home.k3s/layer: "L0" # WHOLE-REPO SCOPE, and necessarily so: this asserts a property OF the merge gate's # requirement-scoped selection. If it could itself be filtered out by that selection, the # first change to break the mechanism would also be the change that stops checking it. conformance.home.k3s/scope: "whole-repo" conformance.home.k3s/safety-critical: "false" conformance.home.k3s/executable: "true" conformance.home.k3s/intent: >- Every conformance test that declares `conformance.home.k3s/scope: whole-repo` is unconditionally selected by the merge gate regardless of which requirements a change touches, and a SKIP of such a test is reported as UNWIRED (exit 3), never as green. spec: steps: - name: assert-whole-repo-scope-mechanism-is-wired-and-falsifiable description: >- Resolve the gate's own L0 selection with a requirement filter that matches no test and assert every whole-repo-scoped test is still selected; then prove a scope-declared SKIP exits 3 while an undeclared SKIP exits 0. try: - script: content: bash ./check.sh