# ============================================================================= # L0 STATIC — no environment overlay ships a placeholder value, and the guard that owns # one refuses it # Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster. # # THE BUG CLASS THIS CLOSES (it has now bitten three times in one file). A guard in # `bootstrap/standup.sh` keys on the SHAPE of a value that lives in # `clusters/base/cluster-vars.yaml`, and nothing tied the two together. On 2026-09-05 the # S3 placeholder changed from `TBD-PROD-S3-BACKUP-ENDPOINT` to # `https://TBD-R2-ACCOUNT-ID.r2.cloudflarestorage.com` (commit 32d6766); the three guards # matched the glob `TBD-*`, a PREFIX match, and `fnmatch('https://TBD-R2-…','TBD-*')` is # False. All three guards silently stopped matching, and `standup.sh resume` would have # resumed `velero` on prod against a nonexistent hostname: installs clean, reports healthy, # NO BACKUPS EXIST. It was found by accident. # # WHY THIS TEST WAS RESTRUCTURED (2026-09-13). It was PERMANENTLY SKIPPED. `check.sh` exited # 3 when `bootstrap/standup.sh` was absent — and that file has NEVER been on `main`: it was # not deleted (no deletion commit exists anywhere), it landed only on unmerged prod-standup # branches. So an incident-derived test reported SKIP on every gate run, and SKIPPED IS NOT # GREEN here. The skip was also hiding a RED: run against the real `standup.sh` from # `work/prod-standup-campaign`, the old check FAILED 2 of 34 assertions on false positives # from treating the empty string as a placeholder for knobs cluster-vars authors empty on # purpose. Both are fixed. The invariant is now split into a half that needs no guard and a # half that activates when one lands, and THIS CHECK NEVER EXITS 3. # # GIVEN the repository as authored, # THEN HALF A — always, needs no guard, input surface `clusters/`: # (A1) no environment overlay's EFFECTIVE cluster-vars value is placeholder-shaped by # an INDEPENDENT detector, for any knob no guard refuses (base is exempt: it is # not a bootstrap target and its loud sentinels exist to be overridden — but a # sentinel an overlay forgets to override IS caught, as that overlay's value); # (A2) an empty effective value is STATED explicitly by the overlay, never inherited # in silence — cluster-vars' own documented convention, and the rule that # separates "authored empty" from "somebody forgot"; # (A3) a LOUD base sentinel is explicitly overridden by every overlay, so the # `TBD-PROD-*` convention cannot be defeated by an omitted patch op; # (A4) the detector itself recognises the exact 2026-09-05 string MID-STRING, accepts # both real endpoints the repo ships, and does not call the empty string loud; # (A5) ci/l0.sh runs the L0 layer and ci/gate.sh blocks the merge when it fails. # HALF B — when a `*_is_placeholder` predicate exists ANYWHERE in the tree (the guard # is DISCOVERED, not hardcoded, so a guard landing at bootstrap/standup.sh or moving to # a sourceable library activates this half with no edit here): # (B1) every authored value of a guarded knob the detector calls placeholder-shaped is # REFUSED by the REAL predicate, extracted from the real file and executed (never # reimplemented — see check.sh "HOW THE REAL GUARD IS EXERCISED"), and every # usable value is ACCEPTED; # (B2) every function that reads a guarded knob applies the guard to it (the "three # guards died at once" regression); # (B3) the guard matches its own vocabulary ANYWHERE in a value, not just as a prefix # — the exact anchoring defect above — and still accepts a real endpoint; # (B4) the extraction is faithful, exactly one file and one definition own the # vocabulary, and the check fails LOUDLY rather than silently testing a copy if # the guard stops being a self-contained predicate. # # STATUS: EXECUTABLE, and it RUNS — never SKIP. `check.sh` is a pure offline assertion # script (standard library + PyYAML + bash); conformance/run.sh --layer l0 runs it directly # and this Chainsaw Test wraps the SAME script. # # NOT proven: while no guard exists on `main`, HALF B IS NOT PROVEN, and the check SAYS SO on # stdout every run rather than swallowing it as a skip — what is proven instead is the # stronger-at-the-source half, that the values a guard would be asked about are not # placeholders at all. Also not proven: that the endpoint is REACHABLE or that backups exist # (runtime facts — `standup.sh s3check` probes the former from a pod, l3-fault-injection/ # dr-restore proves the latter); that guards for knobs OTHER than the placeholder-refusing # ones are correct; and that the guard runs at all in a real standup (the dispatch path is # shell, executed only against a live cluster). A1 reads cluster-vars, not every manifest: a # placeholder hardcoded in a manifest rather than in a knob is out of scope. R-STATE-03 is # tagged for the narrow static slice only: the authored backup target cannot be a placeholder. # ============================================================================= apiVersion: chainsaw.kyverno.io/v1alpha1 kind: Test metadata: name: placeholder-guard-coupling labels: conformance.home.k3s/R-ENV-02: "true" conformance.home.k3s/R-STATE-03: "true" annotations: conformance.home.k3s/requirements: "R-ENV-02, R-STATE-03" conformance.home.k3s/layer: "L0" # NO `scope` ANNOTATION, DELIBERATELY — this invariant IS requirement-scoped, and was # re-audited as such on 2026-09-13 after the restructure changed the input surface. # HALF A reads clusters/base/cluster-vars.yaml plus clusters//kustomization.yaml; # ci/touched-requirements.sh keys clusters/* to R-ENV-02 + R-GITOPS-01, so every path # that can break HALF A attributes a requirement declared above. HALF B reads whichever # file defines the guard; bootstrap/* is keyed to R-STATE-03 + R-GITOPS-01, which covers # the place a standup guard lands. Requirement-scoped selection therefore cannot hide a # breaking change, and declaring whole-repo scope would only add runtime, not coverage. # The one residual: (A5) reads ci/l0.sh and ci/gate.sh, and ci/* attributes neither # R-ENV-02 nor R-STATE-03 — but that wiring is separately and unconditionally asserted by # the whole-repo-scoped tests/l0-static/whole-repo-invariants-wired, so (A5) here is # belt-and-braces rather than the only proof. conformance.home.k3s/safety-critical: "false" conformance.home.k3s/executable: "true" conformance.home.k3s/intent: >- No environment overlay can ship a placeholder-shaped cluster-vars value that nothing refuses, and where a real bootstrap guard owns such a knob it actually recognises every authored value as a placeholder — so a value change in one file cannot silently disarm a guard in another and enable velero against an endpoint that does not exist. spec: steps: - name: refuse-authored-placeholders-and-couple-them-to-any-real-guard description: >- Screen every authored effective cluster-vars value of every environment overlay against an independent placeholder detector, assert empty values are stated rather than inherited, and — when a *_is_placeholder predicate exists anywhere in the tree — extract and execute the REAL guard against those same values. try: - script: content: bash ./check.sh