diff --git a/DECISIONS-LOG.md b/DECISIONS-LOG.md index 76f07ce..a0decbe 100644 --- a/DECISIONS-LOG.md +++ b/DECISIONS-LOG.md @@ -2869,3 +2869,131 @@ PVC and both change the operator's own login story, so **neither was done**. Fla in these pods), and three ~1 GB files named `...S18E05....zipx` / `...S06E01....exe` / `...S29E01....exe`. Media files with `.exe`/`.zipx` extensions are worth an operator's eye; nothing here read or moved them. +## 2026-09-13 — `placeholder-guard-coupling` was SKIPPING FOREVER, and the skip was hiding a RED (branch `fix/placeholder-guard-coupling-unskipped`) + +An L0 audit flagged `conformance/tests/l0-static/placeholder-guard-coupling` (R-ENV-02, R-STATE-03) +as permanently SKIPPED. Confirmed, and the specifics change what the correct fix is, so they were +measured rather than assumed. + +### Did `bootstrap/standup.sh` ever exist? NO — not on `main`, and it was never deleted + +- `git log --all --diff-filter=D -- bootstrap/standup.sh` → **empty**. There is no deletion commit + for it anywhere in the repo. +- `git ls-tree main bootstrap/` → `README.md`, `flux/`, `gitea/`, `sops/`. No `standup.sh`. +- The file is real and substantial (988 lines) but lives only on **unmerged prod-standup branches**: + `work/prod-standup-campaign`, `prod/*`, `feat/prod-hardening`. First added in `4160fee`, last + touched in `8f9b2e4` ("revives a guard I had killed"). +- So the check was authored against a file that **was planned and never landed**, not one that was + removed. `docs/BULK-NFS-STORAGE.md` §7 had already recorded this, including the consequence: + *"nothing in the gate currently refuses a placeholder-shaped overlay value."* +- The 2026-09-05 incident itself (`32d6766`, the MinIO → Cloudflare R2 move) is likewise **not in + this log on `main`** — it is on the same unmerged branches. The only record on `main` was the + check's own header, `ci/touched-requirements.sh`'s comment, and `dr-restore`'s test header. It is + written down here now so it survives whether or not those branches ever land. + +### The skip was hiding a RED, which is worse + +Dropped the real `standup.sh` from `work/prod-standup-campaign` into the tree and ran the OLD check: +**FAIL, 2 of 34 assertions.** Both were false positives from treating the **empty string** as +placeholder-shaped for knobs cluster-vars authors empty *on purpose* — `BACKUP_CREDENTIALS_SUFFIX` +and `VELERO_BACKUP_PREFIX`, both `""` on dev, and both documented in `cluster-vars.yaml` as +load-bearing ("THE BASE DEFAULT IS THE EMPTY STRING, AND THAT IS LOAD-BEARING … an EMPTY VALUE IS +SAFE UNDER StrictPostBuildSubstitutions: strict mode fails on a variable that is not DEFINED, not on +one defined as `""`"). So the check would have **false-blocked the merge that landed its own input**. + +### NOT deleted — the hazard is not gone. Split into a half that needs no guard, and a half that self-activates + +The invariant has two independent halves and only one needs the guard: + +- **HALF A — the authored values.** No environment overlay may ship a placeholder-shaped *effective* + cluster-vars value. This is the 2026-09-05 **outcome** asserted at the source instead of at the + guard: if prod's `S3_BACKUP_ENDPOINT` is never a placeholder, velero cannot be pointed at a + hostname that does not exist, guard or no guard. Input surface is `clusters/`, all on `main`. + Runs unconditionally, 70 authored overlay values screened today. +- **HALF B — the coupling.** The real predicate extracted and executed against those same values, + the consumer trace, and the mid-string anchoring property. Needs a guard. + +**The check never exits 3 again.** And the guard is now **discovered, not hardcoded**: any tracked +`*.sh` defining `_is_placeholder()` is found, so a guard landing at `bootstrap/standup.sh` — +or moved to a sourceable library, which is the remedy the check already recommends — activates HALF B +**with no edit to the test**. Two files defining it is a FAIL (that is the 2026-09-05 divergence with +a second file's name on it). When no guard exists, the gap is printed loudly on stdout every run +instead of being swallowed as a skip. + +**LOUD vs EMPTY** is the correction that makes it honest in both modes. A loud placeholder (`TBD`, +`TODO`, `REPLACE-ME`, ``, …) is always a defect in an overlay. The empty string is +not, so it is held to a *different real rule*: an empty effective value must be **stated explicitly** +by that overlay, never inherited in silence — which is cluster-vars' own documented convention +("Both overlays state this knob explicitly … so neither inherits it silently"). Emptiness still +counts as a placeholder for a knob a guard covers, because a `""` case-arm is the guard author's +declaration that empty is not a usable target. + +### Proof it runs, and proof it is not vacuous + +`check.sh` exits 0 at **84 assertions** on `main` (`(A1)` 70 authored overlay values, `(A2)` 2 empty +values checked for explicit statement, `(A3)` 0 live base sentinels, detector canaries, load-bearing). +Four deliberate breakages, each reverted: + +| break | result | +| --- | --- | +| prod `S3_BACKUP_ENDPOINT` → `https://TBD-R2-ACCOUNT-ID.r2.cloudflarestorage.com` (the literal 2026-09-05 value) | **FAIL** 1/84 — (A1) | +| delete dev's explicit `VELERO_BACKUP_PREFIX` op so it inherits `""` | **FAIL** 1/84 — (A2) | +| base `BULK_NFS_SERVER` → `TBD-PROD-BULK-NFS-SERVER` + delete prod's override | **FAIL** 2/86 — (A1)+(A3) | +| drop in the real `standup.sh`, then revert `s3_is_placeholder` to the `TBD-*` **prefix** glob | **FAIL** 12/110 — (B3) anchoring | + +With the pristine `standup.sh` present, discovery finds it and the check **PASSES at 110 +assertions** — where the old one failed 2 of 34. A second file defining the predicate is detected and +named. Coverage is unchanged: R-ENV-02 and R-STATE-03 both still list this test (49/51, 2 gaps, +identical before and after), because nothing was deleted. + +**L0 layer: 14 checks, 14 PASS, 0 SKIP, 0 FAIL** (was 13 PASS / 1 SKIP). + +### The other L0 skip-capable checks — audited, and none has this defect + +Every L0 check whose `check.sh` can `exit 3` on a missing input path, with the path verified against +the tree: + +| check | skip condition | on `main` | +| --- | --- | --- | +| `placeholder-guard-coupling` | *was* `bootstrap/standup.sh` | **was the only real one — fixed; it now FAILS rather than skips if `clusters/base/cluster-vars.yaml` goes missing, because that means the knobs moved and the check must be re-pointed in the same change** | +| `bulk-tier-distinct-from-replicated` | `catalog/` or `apps/` absent | both present — never fires | +| `flux-postbuild-substitution` | `clusters/` or `clusters/base/cluster-vars.yaml` absent | present — never fires | +| `keycloak-client-secret-reconciler` | any of 4 `platform/keycloak/*.yaml` absent | all 4 present — never fires | +| `spec-immutability-enforced` | the sibling spec repo unavailable | **legitimate and left alone** — an environmental dependency (`../home-network-k3s`, R-TEST-06), not a repo-state defect; present here, so it PASSES | +| `whole-repo-invariants-wired` | — | its `exit 3`s are *synthetic fixtures* it creates to prove the runner treats a skipped whole-repo test as UNWIRED, not its own skip | + +So the disease was confined to one test. The four defensive guards above are dead code in practice +and were left alone rather than expanding scope. + +### `ci/touched-requirements.sh` — the entry STAYS, the comment was wrong + +`bootstrap/*` → `R-STATE-03 R-GITOPS-01` is **kept**. Its comment claimed `bootstrap/standup.sh` +"carries the placeholder guards", describing a file that is not on `main`; corrected to say +R-STATE-03 there is **anticipatory**. Deleting the entry would mean the change that eventually lands +a placeholder guard is selected by nothing — the 2026-09-05 class left to reintroduce itself — and +with content-based guard discovery that entry is exactly what pulls this test into that change's gate +run. `ci/tests/touched-requirements.test.sh` gained a **path-map section** (12 assertions, up from 8) +locking `bootstrap/*` → R-STATE-03 + R-GITOPS-01 and `clusters/*` → R-ENV-02 + R-GITOPS-01, so the +entry's removal is now a test failure rather than a silent loss of selection. Also corrected: the +check's own header claimed "an edit to `bootstrap/*` attributes NO requirement in the path map", +stale since 2026-09-06. + +### Scope annotation: re-audited and still deliberately requirement-scoped, NOT whole-repo + +HALF A reads `clusters/` (→ R-ENV-02); HALF B reads wherever the guard lands, i.e. `bootstrap/` +(→ R-STATE-03). Every path that can break the check attributes a requirement it declares, so +requirement-scoped selection cannot hide a breaking change and `whole-repo` would add runtime, not +coverage. One residual, recorded rather than papered over: `(A5)` reads `ci/l0.sh` and `ci/gate.sh`, +and `ci/*` attributes neither of this test's requirements — but that wiring is separately and +unconditionally asserted by the whole-repo-scoped `whole-repo-invariants-wired`, so `(A5)` here is +belt-and-braces rather than sole proof. + +### Debt / follow-ups + +- **HALF B is genuinely unproven on `main`,** and that is now *reported on stdout every run* rather + than hidden. It cannot be fixed from `main`: it needs `bootstrap/standup.sh` (or its predicate, + extracted into a sourceable library) to land. When it does, no edit to this test is required. +- **`prom-rules` still reports "skipped (missing tool) — not green"** in the gate; pre-existing, + `promtool` is being baked into the toolchain on another branch. +- **A placeholder hardcoded in a manifest** rather than in a cluster-vars knob is out of HALF A's + scope: it reads the ConfigMap and the overlay patches, not every YAML in the tree. diff --git a/ci/tests/touched-requirements.test.sh b/ci/tests/touched-requirements.test.sh index 36812ea..9e906e1 100755 --- a/ci/tests/touched-requirements.test.sh +++ b/ci/tests/touched-requirements.test.sh @@ -93,6 +93,49 @@ printf 'touched set (removal):\n%s\n---\n' "$OUT2" assert_absent "R-DELDOC-01" "$OUT2" "REMOVED whole-line comment R-ID is NOT attributed" assert_present "R-DELREAL-01" "$OUT2" "REMOVED real-line R-ID IS attributed" +# --------------------------------------------------------------------------------------- +# PATH MAP — bootstrap/* (added 2026-09-13) +# +# WHY THIS IS LOCKED DOWN. path_requirements() keys `bootstrap/*` to R-STATE-03 + +# R-GITOPS-01, and until 2026-09-13 the comment justifying it described +# `bootstrap/standup.sh` as a file that carries the placeholder guards. That file has never +# been on `main` (it landed only on unmerged prod-standup branches), so the comment was +# stale and the entry looked like dead weight someone would tidy away. It is not dead: it +# is what SELECTS conformance/tests/l0-static/placeholder-guard-coupling for the change +# that eventually lands a placeholder guard under bootstrap/ — and that test discovers its +# guard by content rather than by path, so it activates on exactly that change. Removing +# the entry would let the change introducing a guard be gated by nothing, which is the +# 2026-09-05 class. These assertions make the entry's deletion a test failure rather than a +# silent loss of selection. +# +# NOTE the R-ID content scan must not be what produces these: `notes/` is outside the path +# map precisely so the other cases isolate the scan, and here the file content cites no +# R-ID at all, so any R-ID in the output came from the PATH. +git -C "$TMP" checkout -q -b feature3 main +mkdir -p "$TMP/bootstrap/flux" +cat > "$TMP/bootstrap/flux/bootstrap.sh" <<'SH' +#!/usr/bin/env bash +# a bootstrap-path change that cites no requirement id of its own +echo "flux bootstrap" +SH +git -C "$TMP" add -A && git -C "$TMP" commit -qm "touch bootstrap/" +OUT3="$(REPO_ROOT="$TMP" BASE_REF=main "$SCRIPT" 2>/dev/null)" +printf 'touched set (bootstrap/ path map):\n%s\n---\n' "$OUT3" +assert_present "R-STATE-03" "$OUT3" "bootstrap/* attributes R-STATE-03 (selects placeholder-guard-coupling)" +assert_present "R-GITOPS-01" "$OUT3" "bootstrap/* attributes R-GITOPS-01 (it IS the GitOps bootstrap path)" + +# And the neighbouring entry the same test leans on for its other half: clusters/* must +# attribute R-ENV-02, which is what selects placeholder-guard-coupling's HALF A (the +# authored-values sweep over clusters/ effective cluster-vars). +git -C "$TMP" checkout -q -b feature4 main +mkdir -p "$TMP/clusters/base" +printf 'kind: ConfigMap\nmetadata:\n name: cluster-vars\n' > "$TMP/clusters/base/cluster-vars.yaml" +git -C "$TMP" add -A && git -C "$TMP" commit -qm "touch clusters/" +OUT4="$(REPO_ROOT="$TMP" BASE_REF=main "$SCRIPT" 2>/dev/null)" +printf 'touched set (clusters/ path map):\n%s\n---\n' "$OUT4" +assert_present "R-ENV-02" "$OUT4" "clusters/* attributes R-ENV-02 (selects placeholder-guard-coupling HALF A)" +assert_present "R-GITOPS-01" "$OUT4" "clusters/* attributes R-GITOPS-01" + echo if [ "$FAIL" -eq 0 ]; then echo "touched-requirements tests: PASS ($PASS assertions)"; exit 0; else echo "touched-requirements tests: FAIL ($FAIL failed, $PASS passed)"; exit 1; fi diff --git a/ci/touched-requirements.sh b/ci/touched-requirements.sh index 4e93d8f..3f07374 100755 --- a/ci/touched-requirements.sh +++ b/ci/touched-requirements.sh @@ -217,14 +217,23 @@ path_requirements() { # chainsaw-test.yaml), and the correct fix for a path-scoped invariant is a path-map entry. infra/*) echo "R-ENV-01" ;; clusters/*) echo "R-GITOPS-01 R-ENV-02" ;; - # bootstrap/ was UNMAPPED until 2026-09-06, and the omission was load-bearing: - # bootstrap/standup.sh carries the placeholder guards that decide whether `velero` may be - # enabled against a real backup endpoint. A change touching standup.sh ALONGSIDE any - # already-mapped path scoped the L0 run to that other path's requirements and silently - # excluded conformance/tests/l0-static/placeholder-guard-coupling — the check that exists - # precisely to catch a guard/value mismatch. The 2026-09-05 guard bug (a prefix glob that - # stopped matching when the value it guards changed, in a different file) is exactly the - # class that omission would hide. + # bootstrap/ was UNMAPPED until 2026-09-06. The entry STAYS, but its original rationale + # was written against a file that is not here, and that is corrected on 2026-09-13: + # it claimed "bootstrap/standup.sh carries the placeholder guards that decide whether + # `velero` may be enabled against a real backup endpoint". bootstrap/standup.sh HAS NEVER + # BEEN ON `main` — it was not deleted (no deletion commit exists anywhere), it landed only + # on unmerged prod-standup branches. `git ls-tree main bootstrap/` lists README.md, flux/, + # gitea/, sops/. So R-STATE-03 here is ANTICIPATORY, not descriptive. + # + # WHY IT IS STILL RIGHT. conformance/tests/l0-static/placeholder-guard-coupling DISCOVERS + # its guard (any tracked *.sh defining `_is_placeholder()`) instead of hardcoding a + # path, so the moment a standup guard lands under bootstrap/ that test's coupling half + # activates — and this entry is what selects the test for the change that lands it. + # Deleting the entry would mean the change introducing a placeholder guard is gated by + # nothing, which is the 2026-09-05 class (a prefix glob that stopped matching when the + # value it guards changed, in a different file) left to reintroduce itself. R-GITOPS-01 is + # descriptive today: bootstrap/flux/, gitea/ and sops/ ARE the GitOps bootstrap path. + # Locked down by ci/tests/touched-requirements.test.sh's path-map section. bootstrap/*) echo "R-STATE-03 R-GITOPS-01" ;; # R-TEST-06 added 2026-09-12. The comment in changed_diff()'s rule 3 above records why it # was ABSENT: R-TEST-06 used to be enforced by the spec-immutability STAGE and carried by no diff --git a/conformance/tests/l0-static/placeholder-guard-coupling/chainsaw-test.yaml b/conformance/tests/l0-static/placeholder-guard-coupling/chainsaw-test.yaml index 2124761..0597210 100644 --- a/conformance/tests/l0-static/placeholder-guard-coupling/chainsaw-test.yaml +++ b/conformance/tests/l0-static/placeholder-guard-coupling/chainsaw-test.yaml @@ -1,43 +1,71 @@ # ============================================================================= -# L0 STATIC — a placeholder value in cluster-vars is REFUSED by the guard that owns it +# L0 STATIC — no environment overlay ships a placeholder value, and the guard that owns +# one refuses it # Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster. # # THE BUG CLASS THIS CLOSES (it has now bitten three times in one file). A guard in # `bootstrap/standup.sh` keys on the SHAPE of a value that lives in # `clusters/base/cluster-vars.yaml`, and nothing tied the two together. On 2026-09-05 the # S3 placeholder changed from `TBD-PROD-S3-BACKUP-ENDPOINT` to -# `https://TBD-R2-ACCOUNT-ID.r2.cloudflarestorage.com`; the three guards matched the glob -# `TBD-*`, a PREFIX match, and `fnmatch('https://TBD-R2-…','TBD-*')` is False. All three -# guards silently stopped matching, and `standup.sh resume` would have resumed `velero` on -# prod against a nonexistent hostname: installs clean, reports healthy, NO BACKUPS EXIST. -# It was found by accident. The guard is fixed (`s3_is_placeholder`, substring); the -# COUPLING was untested, which is what makes the class reintroducible. +# `https://TBD-R2-ACCOUNT-ID.r2.cloudflarestorage.com` (commit 32d6766); the three guards +# matched the glob `TBD-*`, a PREFIX match, and `fnmatch('https://TBD-R2-…','TBD-*')` is +# False. All three guards silently stopped matching, and `standup.sh resume` would have +# resumed `velero` on prod against a nonexistent hostname: installs clean, reports healthy, +# NO BACKUPS EXIST. It was found by accident. +# +# WHY THIS TEST WAS RESTRUCTURED (2026-09-13). It was PERMANENTLY SKIPPED. `check.sh` exited +# 3 when `bootstrap/standup.sh` was absent — and that file has NEVER been on `main`: it was +# not deleted (no deletion commit exists anywhere), it landed only on unmerged prod-standup +# branches. So an incident-derived test reported SKIP on every gate run, and SKIPPED IS NOT +# GREEN here. The skip was also hiding a RED: run against the real `standup.sh` from +# `work/prod-standup-campaign`, the old check FAILED 2 of 34 assertions on false positives +# from treating the empty string as a placeholder for knobs cluster-vars authors empty on +# purpose. Both are fixed. The invariant is now split into a half that needs no guard and a +# half that activates when one lands, and THIS CHECK NEVER EXITS 3. # # GIVEN the repository as authored, -# WHEN the REAL predicate is extracted from bootstrap/standup.sh and executed by bash -# (never reimplemented — see check.sh "HOW THE REAL GUARD IS EXERCISED"), -# THEN (1) every authored value of a guarded knob that is placeholder-shaped by an -# INDEPENDENT detector is actually refused by that guard, in base and in every -# environment overlay; -# (2) no environment overlay carries a placeholder-shaped value for a knob no guard -# refuses; -# (3) every function in standup.sh that reads a guarded knob applies the guard to it -# (the "three guards died at once" regression); -# (4) the guard matches its own vocabulary ANYWHERE in a value, not just as a prefix -# — the exact anchoring defect above — and still accepts a real endpoint; -# AND (5) the extraction is faithful, and the check fails LOUDLY rather than silently -# testing a copy if the guard stops being a self-contained predicate. +# THEN HALF A — always, needs no guard, input surface `clusters/`: +# (A1) no environment overlay's EFFECTIVE cluster-vars value is placeholder-shaped by +# an INDEPENDENT detector, for any knob no guard refuses (base is exempt: it is +# not a bootstrap target and its loud sentinels exist to be overridden — but a +# sentinel an overlay forgets to override IS caught, as that overlay's value); +# (A2) an empty effective value is STATED explicitly by the overlay, never inherited +# in silence — cluster-vars' own documented convention, and the rule that +# separates "authored empty" from "somebody forgot"; +# (A3) a LOUD base sentinel is explicitly overridden by every overlay, so the +# `TBD-PROD-*` convention cannot be defeated by an omitted patch op; +# (A4) the detector itself recognises the exact 2026-09-05 string MID-STRING, accepts +# both real endpoints the repo ships, and does not call the empty string loud; +# (A5) ci/l0.sh runs the L0 layer and ci/gate.sh blocks the merge when it fails. +# HALF B — when a `*_is_placeholder` predicate exists ANYWHERE in the tree (the guard +# is DISCOVERED, not hardcoded, so a guard landing at bootstrap/standup.sh or moving to +# a sourceable library activates this half with no edit here): +# (B1) every authored value of a guarded knob the detector calls placeholder-shaped is +# REFUSED by the REAL predicate, extracted from the real file and executed (never +# reimplemented — see check.sh "HOW THE REAL GUARD IS EXERCISED"), and every +# usable value is ACCEPTED; +# (B2) every function that reads a guarded knob applies the guard to it (the "three +# guards died at once" regression); +# (B3) the guard matches its own vocabulary ANYWHERE in a value, not just as a prefix +# — the exact anchoring defect above — and still accepts a real endpoint; +# (B4) the extraction is faithful, exactly one file and one definition own the +# vocabulary, and the check fails LOUDLY rather than silently testing a copy if +# the guard stops being a self-contained predicate. # -# STATUS: EXECUTABLE. `check.sh` is a pure offline assertion script (standard library + -# PyYAML + bash); conformance/run.sh --layer l0 runs it directly and this Chainsaw Test -# wraps the SAME script. +# STATUS: EXECUTABLE, and it RUNS — never SKIP. `check.sh` is a pure offline assertion +# script (standard library + PyYAML + bash); conformance/run.sh --layer l0 runs it directly +# and this Chainsaw Test wraps the SAME script. # -# NOT proven: that the endpoint is REACHABLE or that backups exist (runtime facts — -# `standup.sh s3check` probes the former from a pod, l3-fault-injection/dr-restore proves -# the latter); that guards for knobs OTHER than the placeholder-refusing ones are correct; -# and that the guard runs at all in a real standup (the dispatch path is shell, executed -# only against a live cluster). R-STATE-03 is tagged for the narrow static slice only: the -# authored backup target cannot be a placeholder that the guard fails to recognise. +# NOT proven: while no guard exists on `main`, HALF B IS NOT PROVEN, and the check SAYS SO on +# stdout every run rather than swallowing it as a skip — what is proven instead is the +# stronger-at-the-source half, that the values a guard would be asked about are not +# placeholders at all. Also not proven: that the endpoint is REACHABLE or that backups exist +# (runtime facts — `standup.sh s3check` probes the former from a pod, l3-fault-injection/ +# dr-restore proves the latter); that guards for knobs OTHER than the placeholder-refusing +# ones are correct; and that the guard runs at all in a real standup (the dispatch path is +# shell, executed only against a live cluster). A1 reads cluster-vars, not every manifest: a +# placeholder hardcoded in a manifest rather than in a knob is out of scope. R-STATE-03 is +# tagged for the narrow static slice only: the authored backup target cannot be a placeholder. # ============================================================================= apiVersion: chainsaw.kyverno.io/v1alpha1 kind: Test @@ -50,27 +78,32 @@ metadata: conformance.home.k3s/requirements: "R-ENV-02, R-STATE-03" conformance.home.k3s/layer: "L0" # NO `scope` ANNOTATION, DELIBERATELY — this invariant IS requirement-scoped, and was - # audited as such (2026-09-12). Its entire input surface is bootstrap/standup.sh (the - # placeholder guards) plus clusters/base/cluster-vars.yaml and clusters// - # (the values they guard). ci/touched-requirements.sh keys bootstrap/* to - # R-STATE-03 + R-GITOPS-01 and clusters/* to R-ENV-02 + R-GITOPS-01 — so EVERY path that - # can break this check already attributes one of the two requirements it declares above. - # Requirement-scoped selection therefore cannot hide a breaking change here, and declaring - # whole-repo scope would only add runtime, not coverage. + # re-audited as such on 2026-09-13 after the restructure changed the input surface. + # HALF A reads clusters/base/cluster-vars.yaml plus clusters//kustomization.yaml; + # ci/touched-requirements.sh keys clusters/* to R-ENV-02 + R-GITOPS-01, so every path + # that can break HALF A attributes a requirement declared above. HALF B reads whichever + # file defines the guard; bootstrap/* is keyed to R-STATE-03 + R-GITOPS-01, which covers + # the place a standup guard lands. Requirement-scoped selection therefore cannot hide a + # breaking change, and declaring whole-repo scope would only add runtime, not coverage. + # The one residual: (A5) reads ci/l0.sh and ci/gate.sh, and ci/* attributes neither + # R-ENV-02 nor R-STATE-03 — but that wiring is separately and unconditionally asserted by + # the whole-repo-scoped tests/l0-static/whole-repo-invariants-wired, so (A5) here is + # belt-and-braces rather than the only proof. conformance.home.k3s/safety-critical: "false" conformance.home.k3s/executable: "true" conformance.home.k3s/intent: >- - Every placeholder-shaped value authored in cluster-vars, for any environment - overlay, is actually recognised as a placeholder by the real bootstrap guard that - is supposed to refuse it — so a value change in one file cannot silently disarm a - guard in another and enable velero against an endpoint that does not exist. + No environment overlay can ship a placeholder-shaped cluster-vars value that nothing + refuses, and where a real bootstrap guard owns such a knob it actually recognises + every authored value as a placeholder — so a value change in one file cannot silently + disarm a guard in another and enable velero against an endpoint that does not exist. spec: steps: - - name: couple-authored-placeholders-to-the-real-guard + - name: refuse-authored-placeholders-and-couple-them-to-any-real-guard description: >- - Extract the real placeholder predicate from bootstrap/standup.sh, run it against - every authored cluster-vars value the repo actually carries, and assert the guard - refuses each one an independent detector calls a placeholder. + Screen every authored effective cluster-vars value of every environment overlay + against an independent placeholder detector, assert empty values are stated rather + than inherited, and — when a *_is_placeholder predicate exists anywhere in the tree + — extract and execute the REAL guard against those same values. try: - script: content: bash ./check.sh diff --git a/conformance/tests/l0-static/placeholder-guard-coupling/check.sh b/conformance/tests/l0-static/placeholder-guard-coupling/check.sh index d3814e6..8189263 100755 --- a/conformance/tests/l0-static/placeholder-guard-coupling/check.sh +++ b/conformance/tests/l0-static/placeholder-guard-coupling/check.sh @@ -12,20 +12,74 @@ # placeholder. Three of those guards keyed on the SHAPE of `S3_BACKUP_ENDPOINT`, which is # authored in `clusters/base/cluster-vars.yaml`. On 2026-09-05 the placeholder changed # from `TBD-PROD-S3-BACKUP-ENDPOINT` to `https://TBD-R2-ACCOUNT-ID.r2.cloudflarestorage.com` -# (the MinIO -> Cloudflare R2 move). The guards matched the glob `TBD-*` — a PREFIX match — -# and `fnmatch('https://TBD-R2-…','TBD-*')` is FALSE. All three guards silently stopped -# matching. `standup.sh resume` would then have resumed `velero` on prod against a hostname -# that does not exist: the HelmRelease installs clean, the Kustomization reports Ready, and -# NO BACKUPS EXIST. The only reason it was not shipped is that someone noticed by accident. +# (the MinIO -> Cloudflare R2 move, commit 32d6766). The guards matched the glob `TBD-*` — +# a PREFIX match — and `fnmatch('https://TBD-R2-…','TBD-*')` is FALSE. All three guards +# silently stopped matching. `standup.sh resume` would then have resumed `velero` on prod +# against a hostname that does not exist: the HelmRelease installs clean, the Kustomization +# reports Ready, and NO BACKUPS EXIST. It was found by accident. # -# The predicate is fixed (`s3_is_placeholder`, substring match). The COUPLING is what was -# never tested: a guard in one file keying on a value in another, with nothing tying them -# together. This check is that tie, and it is derived from BOTH real files — no current -# string is hardcoded here, so it does not rot when the placeholder or the guard changes. +# WHY THIS CHECK WAS REWRITTEN (2026-09-13) — IT WAS SKIPPING FOREVER +# ------------------------------------------------------------------ +# The original version keyed its ENTIRE existence on `bootstrap/standup.sh` and exited 3 +# (SKIP) when that file was absent. Established by measurement, not assumption: +# +# * `bootstrap/standup.sh` HAS NEVER EXISTED ON `main`. It was not deleted — there is no +# deletion commit for it anywhere (`git log --all --diff-filter=D -- bootstrap/standup.sh` +# is empty). It lives on unmerged prod-standup branches (`work/prod-standup-campaign`, +# `prod/*`, `feat/prod-hardening`) and never landed. `git ls-tree main bootstrap/` lists +# only README.md, flux/, gitea/, sops/. docs/BULK-NFS-STORAGE.md §7 already recorded +# this, including the consequence: "nothing in the gate currently refuses a +# placeholder-shaped overlay value". +# * So this check reported SKIP on every gate run since it landed, and SKIPPED IS NOT +# GREEN in this repo. An incident-derived test that can only ever skip is the quiet +# version of the failure it was written to prevent. +# * Worse, the skip was HIDING A RED. Run against the real `standup.sh` from +# `work/prod-standup-campaign`, the original check FAILED 2 of 34 assertions — both +# false positives from treating the EMPTY STRING as placeholder-shaped for knobs whose +# emptiness is deliberately authored (`BACKUP_CREDENTIALS_SUFFIX` and +# `VELERO_BACKUP_PREFIX`, both `""` on dev). cluster-vars documents that emptiness as +# load-bearing ("THE BASE DEFAULT IS THE EMPTY STRING, AND THAT IS LOAD-BEARING … an +# EMPTY VALUE IS SAFE UNDER StrictPostBuildSubstitutions"). So the check would have +# false-blocked the moment its own input landed. That is fixed below. +# +# THE HAZARD IS NOT GONE, so this coverage is NOT obsolete and was NOT deleted. The +# invariant has two independent halves, and only one of them needs `standup.sh`: +# +# HALF A — THE AUTHORED VALUES (provable on `main` today, needs NO guard). +# No environment overlay may ship a placeholder-shaped effective cluster-vars value. +# This is the OUTCOME the 2026-09-05 guard existed to prevent, asserted at the source +# instead of at the guard: if prod's `S3_BACKUP_ENDPOINT` is never a placeholder, velero +# cannot be pointed at a hostname that does not exist, whether or not a guard catches +# it. Its entire input surface is `clusters/`, all of which is on `main`. +# HALF B — THE COUPLING (activates automatically when a guard lands). +# The real predicate, extracted from the real file and executed, agrees with an +# INDEPENDENT detector about every authored value; every consumer of a guarded knob +# applies the guard; the guard matches its vocabulary ANYWHERE, not as a prefix. +# +# THIS CHECK NEVER EXITS 3. Half A always runs, against real files, with an assertion +# floor. Half B runs when a guard exists and is reported as NOT PROVEN when one does not — +# stated loudly in the output rather than swallowed as a skip. +# +# THE GUARD IS DISCOVERED, NOT HARDCODED (this is the "re-point" made automatic) +# ----------------------------------------------------------------------------- +# The original hardcoded `bootstrap/standup.sh`, which is exactly why a file that never +# landed could disable the whole check. Instead, every tracked `*.sh` in the tree is +# scanned for a `_is_placeholder()` definition: +# * exactly one such FILE -> that is the guard; Half B runs against it, wherever it is. +# A guard that lands at `bootstrap/standup.sh`, or moves to `bootstrap/lib/placeholder.sh`, +# is picked up with no edit to this check. +# * none -> Half A only, with the gap named in the output. +# * more than one file -> FAIL. Two files defining the placeholder vocabulary is the +# 2026-09-05 divergence with a second file's name on it; this check cannot say which +# one a standup run would source. +# `PLACEHOLDER_GUARD_FILE` overrides discovery (repo-relative) for negative fixtures. +# NOTE: discovery is shell-only, because the guard family is shell. If the refusal ever +# moves to a non-shell mechanism (a Kyverno policy, a CEL admission rule), discovery finds +# nothing and says so — re-point this check deliberately in that change. # # HOW THE REAL GUARD IS EXERCISED (and why not a copy) # --------------------------------------------------- -# `bootstrap/standup.sh` is NOT sourceable: at top level it parses argv, hard-exits when +# `standup.sh` is NOT sourceable: at top level it parses argv, hard-exits when # --kubeconfig/--expect-server/ are absent, `cd`s to the repo root, exports # KUBECONFIG, installs an EXIT trap and finally dispatches steps against a live cluster. # So this check EXTRACTS the predicate's own source lines from the file and sources ONLY @@ -35,105 +89,119 @@ # * the extracted body must be a SELF-CONTAINED predicate — `case`/`return` and nothing # else, no command substitution, no call to another function. If the guard ever grows # a dependency, extraction stops being faithful and THIS CHECK FAILS LOUDLY, naming the -# remedy (move the predicate into a small sourceable library that standup.sh sources) -# rather than quietly falling back to testing a copy of the logic; +# remedy (move the predicate into a small sourceable library) rather than quietly +# falling back to testing a copy of the logic; # * a CANARY pins the extraction itself: the extracted function must refuse the empty -# string and accept a realistic endpoint. An extraction that grabbed the wrong lines, -# or a predicate degraded to always-true / always-false, fails there — this repo has -# already been bitten by a check that passed when perturbed because it asked a question -# that was always satisfiable. +# string and accept a realistic endpoint. # There is deliberately NO second implementation of the matching logic anywhere in this file. # -# THE INDEPENDENT DETECTOR (the other half of the comparison) -# ---------------------------------------------------------- -# Asking "does the guard agree with the guard?" proves nothing, so the set of -# placeholder-shaped values is decided by a detector written HERE, whose vocabulary is the -# UNION of two independent sources: +# THE INDEPENDENT DETECTOR (and the empty-string distinction the old version got wrong) +# ------------------------------------------------------------------------------------ +# Asking "does the guard agree with the guard?" proves nothing, so placeholder-shaped is +# decided by a detector written HERE, whose vocabulary is the UNION of two independent +# sources: # * SEED — universal loud-placeholder idioms (TBD, TODO, FIXME, PLACEHOLDER, REPLACE-ME, -# CHANGE-ME, FILL-ME, SET-ME, XXXX), plus `` templates and the empty -# string. These are NOT derived from the guard, so DELETING a guard arm cannot make a -# value stop being placeholder-shaped — it makes assertion (1) fail, which is the point. -# A value using an idiom the guard does not know is a finding either way: fix it by -# giving the knob a real value, or by teaching the guard the idiom. -# * the guard's OWN vocabulary, read out of its case-arms with the glob anchors STRIPPED. -# Stripping the anchors is what makes the comparison non-circular in the dimension that -# actually broke: the detector matches a word ANYWHERE, case-insensitively, and the -# guard must agree. A prefix-anchored guard cannot pass. +# CHANGE-ME, FILL-ME, SET-ME, XXXX), plus `` templates. NOT derived from +# any guard, so DELETING a guard arm cannot make a value stop being placeholder-shaped. +# * the guard's OWN vocabulary (when a guard exists), read out of its case-arms with the +# glob anchors STRIPPED. Stripping the anchors is what makes the comparison non-circular +# in the dimension that actually broke: the detector matches a word ANYWHERE, +# case-insensitively, and the guard must agree. A prefix-anchored guard cannot pass. +# +# LOUD vs EMPTY — the correction. A LOUD placeholder is a positive statement that a value +# was never decided, and it is always a defect in an overlay. The EMPTY STRING is not: +# cluster-vars authors `""` deliberately for suffix/prefix knobs, and Flux strict +# substitution fails on an UNDEFINED variable, not on one defined as `""`. So emptiness is +# NOT treated as a loud placeholder. It is held to a different, real rule instead (A2): an +# empty effective value must be STATED EXPLICITLY by that overlay, never inherited in +# silence — which is cluster-vars' own documented convention ("Both overlays state this +# knob explicitly … so neither inherits it silently"). Emptiness still counts as +# placeholder-shaped for a knob a GUARD covers, because a `""` case-arm in the guard is the +# guard author's declaration that empty is not a usable target for that knob. # # GIVEN the repository as authored, -# WHEN the real predicate(s) are extracted from bootstrap/standup.sh and executed, -# THEN (1) VALUES: for every knob a guard is responsible for, every authored value — the -# base default AND each environment overlay's value — that the independent -# detector calls placeholder-shaped is REFUSED by the real guard, and every value -# it calls usable is ACCEPTED (a guard that refuses a real endpoint leaves velero -# suspended forever, which is the same class of silent failure); -# (2) OVERLAYS: no environment overlay's EFFECTIVE cluster-vars carries a -# placeholder-shaped value for a knob that no guard refuses. (Base alone is -# exempt: it is not a bootstrap target — `flux bootstrap --path=clusters/` — -# and its loud sentinels exist precisely to be overridden. A base placeholder an -# overlay forgets to override IS caught, because it becomes that overlay's -# effective value.) -# (3) CONSUMERS: every function in standup.sh that reads a guarded knob (via `cvar`, -# following one hop of variable aliasing) applies a placeholder guard to it. This -# is the "all three guards died at once" regression stated as a property: the -# guard being correct is worthless if a consumer stops calling it. -# (4) ANCHORING: for each word in the guard's own vocabulary, the guard refuses a -# value carrying that word in the MIDDLE (not at the start) — the precise defect -# of 2026-09-05 — while still accepting a realistic endpoint. -# (5) HARNESS: exactly one definition per predicate, the extracted body is verbatim -# from the file and self-contained, the canary holds, no divergent second copy of -# the matching logic exists elsewhere in standup.sh, and a floor on the number of -# assertions run keeps a pass from degenerating into a no-op. -# AND (6) LOAD-BEARING: ci/l0.sh runs the L0 conformance layer on every gate run and -# ci/gate.sh sets the merge block when that stage fails. +# THEN HALF A (always): +# (A1) VALUES: no environment overlay's EFFECTIVE cluster-vars value is LOUD +# placeholder-shaped, for any knob that no guard refuses. (Base alone is exempt: +# it is not a bootstrap target — `flux bootstrap --path=clusters/` — and its +# loud sentinels exist to be overridden. A base placeholder an overlay forgets to +# override IS caught, because it becomes that overlay's effective value.) +# (A2) EXPLICITNESS: an empty effective value is stated explicitly by that overlay. +# (A3) SENTINEL DISCIPLINE: if base carries a LOUD sentinel for a knob, every overlay +# states that knob explicitly — the `TBD-PROD-*` convention only works if no +# overlay can inherit the sentinel by omission. +# (A4) DETECTOR CANARIES: the detector recognises the exact 2026-09-05 string +# MID-STRING, accepts both real endpoints the repo actually ships, and does NOT +# call the empty string loud. +# (A5) LOAD-BEARING: ci/l0.sh runs the L0 conformance layer on every gate run and +# ci/gate.sh sets the merge block when that stage fails. +# HALF B (only when a guard file is discovered): +# (B1) VALUES: for every knob a guard is responsible for, every authored value — the +# base default AND each overlay's value — that the detector calls placeholder- +# shaped is REFUSED by the real guard, and every value it calls usable is +# ACCEPTED (a guard that refuses a real endpoint leaves velero suspended forever, +# the same class of silent failure). +# (B2) CONSUMERS: every function that reads a guarded knob (via `cvar`, following one +# hop of variable aliasing) applies a placeholder guard to it. This is the "all +# three guards died at once" regression stated as a property. +# (B3) ANCHORING: for each word in the guard's own vocabulary, the guard refuses a +# value carrying that word in the MIDDLE (not at the start) — the precise defect +# of 2026-09-05 — while still accepting a realistic endpoint. +# (B4) HARNESS: exactly one definition per predicate, one guard FILE in the tree, the +# extracted body verbatim and self-contained, the canary holds, no divergent +# second copy of the matching logic, and assertion floors keep a pass from +# degenerating into a no-op. # # WHAT THIS DOES *NOT* PROVE (honest gaps) +# - While no guard exists on `main`, HALF B IS NOT PROVEN AT ALL, and the check says so on +# stdout every run. What is proven is the stronger-at-the-source half: the values a +# guard would be asked about are not placeholders in the first place. This is a real +# narrowing of coverage versus the intent, and it is reported, not hidden. # - Nothing about RUNTIME: not that the endpoint resolves, not that credentials work, not # that a backup exists. `standup.sh s3check` probes reachability from a pod (operator- # run, cluster-gated) and conformance/tests/l3-fault-injection/dr-restore proves restore. -# - It does not execute standup.sh's dispatch path, so it cannot prove the guard is -# REACHED in a real run — only that, when reached, it refuses what it must. The -# consumer property (3) is a static approximation of reachability: it is a text-level -# taint trace over one file, not an interpreter, and a guard whose result is computed -# and then ignored would satisfy it. +# - Half B does not execute the dispatch path, so it cannot prove the guard is REACHED in +# a real run — only that, when reached, it refuses what it must. (B2) is a text-level +# taint trace over one file, not an interpreter. # - It says nothing about guards keyed on shapes OTHER than placeholders (e.g. the # EXTERNALDNS_TXT_OWNER_ID == 'default' test), nor about placeholder-shaped values # outside cluster-vars (the Ansible inventory's BLOCKED/NOT-A-DEVICE sentinels are # asserted by the node_baseline role itself, at play time). -# - Stage 3b of ci/l0.sh scopes the conformance run to the TOUCHED requirements. An edit -# to bootstrap/* attributes NO requirement in ci/touched-requirements.sh's path map, so -# the touched set is empty and the FULL L0 layer runs (verified: ci/lib.sh -# load_touched_requirements) — this check runs. An edit to clusters/* maps to -# R-ENV-02, which this check carries, so a cluster-vars value change also selects it. -# A change that touches standup.sh ALONGSIDE some unrelated mapped path (say -# platform/cilium/) would scope the run away from this check; closing that needs a -# `bootstrap/*` entry in the path map, which is not this check's file to edit. +# - A1 reads cluster-vars, not every manifest. A placeholder hardcoded in a manifest +# rather than in a knob is out of scope here. +# +# SELECTION. `ci/touched-requirements.sh` keys `clusters/*` -> R-ENV-02 (Half A's whole +# input surface) and `bootstrap/*` -> R-STATE-03 (where a guard lands), so every path that +# can break this check attributes a requirement it declares. Deliberately requirement-scoped, +# not whole-repo — see chainsaw-test.yaml. # ============================================================================= set -euo pipefail DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" ROOT="$(cd "${DIR}/../../../.." && pwd)" # repo root PYTHON="${PYTHON:-python3}" -STANDUP_REL="${STANDUP_REL:-bootstrap/standup.sh}" +GUARD_OVERRIDE="${PLACEHOLDER_GUARD_FILE:-}" cd "${ROOT}" -if [ ! -f "${ROOT}/${STANDUP_REL}" ]; then - echo "check SKIPPED (placeholder-guard-coupling): ${STANDUP_REL} is absent — nothing to couple." >&2 - exit 3 +# The authored half of the coupling is the one thing this check cannot do without. +if [ ! -f "${ROOT}/clusters/base/cluster-vars.yaml" ]; then + echo "CHECK FAIL (placeholder-guard-coupling): clusters/base/cluster-vars.yaml is absent — the authored values this check audits do not exist. That is not a SKIP: the knobs this invariant is about have been moved or deleted, and this check must be re-pointed in the SAME change." >&2 + exit 1 fi WORK="$(mktemp -d)" trap 'rm -rf "${WORK}"' EXIT # --------------------------------------------------------------------------------------- -# PHASE 1 (python): extract the real predicate(s), derive the corpus from the real files, -# and run every assertion that does not need a shell. Writes: -# ${WORK}/guards.sh the predicate source lines, verbatim, and nothing else -# ${WORK}/corpus.tsv \t\t\t +# PHASE 1 (python): discover the guard, derive the corpus from the real files, and run +# every assertion that does not need a shell. Writes: +# ${WORK}/guards.sh the predicate source lines, verbatim, and nothing else (may be empty) +# ${WORK}/corpus.tsv \t\t\t (may be empty) # ${WORK}/findings.txt one line per failed assertion # ${WORK}/counts.txt +# ${WORK}/mode.txt the discovered guard's repo-relative path, or empty # --------------------------------------------------------------------------------------- -"${PYTHON}" - "${ROOT}" "${WORK}" "${STANDUP_REL}" <<'PY' +"${PYTHON}" - "${ROOT}" "${WORK}" "${GUARD_OVERRIDE}" <<'PY' import re import sys from pathlib import Path @@ -145,7 +213,7 @@ except ImportError: # pragma: no cover file=sys.stderr) raise SystemExit(1) -ROOT, WORK, STANDUP_REL = Path(sys.argv[1]), Path(sys.argv[2]), sys.argv[3] +ROOT, WORK, OVERRIDE = Path(sys.argv[1]), Path(sys.argv[2]), sys.argv[3] FAILURES: list[str] = [] CHECKS = 0 @@ -164,116 +232,159 @@ def bail(msg): (WORK / "counts.txt").write_text("0\n") (WORK / "corpus.tsv").write_text("") (WORK / "guards.sh").write_text("") + (WORK / "mode.txt").write_text("") raise SystemExit(1) -TEXT = (ROOT / STANDUP_REL).read_text() -LINES = TEXT.splitlines() - -# ------------------------------------------------------------------ (5) extract the guard DEF_RE = re.compile(r"^([a-z0-9_]+_is_placeholder)\(\)\s*\{") -defs: dict[str, list[int]] = {} -for i, line in enumerate(LINES): - m = DEF_RE.match(line) - if m: - defs.setdefault(m.group(1), []).append(i) +SELF_DIR = ROOT / "conformance" / "tests" / "l0-static" / "placeholder-guard-coupling" +SKIP_DIRS = {".git", "_artifacts", "node_modules", ".venv"} -if not defs: - bail(f"{STANDUP_REL} defines no *_is_placeholder predicate. Either the guard was renamed " - "(update this check's discovery pattern in the SAME change) or it was DELETED, which " - "is the 2026-09-05 failure with the guard removed instead of disarmed.") +# ------------------------------------------------------------------ discover the guard FILE +# Hardcoding a path is what let a file that never landed disable this whole check. +if OVERRIDE: + cand = [ROOT / OVERRIDE] + if not cand[0].is_file(): + bail(f"PLACEHOLDER_GUARD_FILE={OVERRIDE} does not exist") +else: + cand = [] + for p in sorted(ROOT.rglob("*.sh")): + if SKIP_DIRS & set(p.parts) or SELF_DIR in p.parents: + continue + try: + text = p.read_text() + except (OSError, UnicodeDecodeError): # pragma: no cover + continue + if any(DEF_RE.match(line) for line in text.splitlines()): + cand.append(p) -# An "argument-shaped" call: "" -CALL_RE = re.compile(r"\b([a-z0-9_]+_is_placeholder)\s+\"([^\"]*)\"") +check(len(cand) <= 1, + "more than one file defines a *_is_placeholder predicate (" + + ", ".join(str(p.relative_to(ROOT)) for p in cand) + + ") — two files owning the placeholder vocabulary is the 2026-09-05 divergence with a " + "second file's name on it, and this check cannot say which one a standup run would " + "source. Keep exactly one, and source it from the other.") +GUARD_REL = str(cand[0].relative_to(ROOT)) if len(cand) == 1 else "" +(WORK / "mode.txt").write_text(GUARD_REL + "\n") + +# --------------------------------------------------------- HALF B setup: extract the guard guard_src: list[str] = [] guard_words: dict[str, list[str]] = {} -ARM_RE = re.compile(r"^(?P[^()]*?)\)\s*return\s+[01]\s*;;$") -ALLOWED = ( - re.compile(r"^$"), - re.compile(r"^#"), - re.compile(r"^case\s+\"\$\{?[0-9A-Za-z_]+\}?\"\s+in$"), - re.compile(r"^esac$"), - re.compile(r"^return\s+[01]$"), - ARM_RE, -) +funcs: dict[str, list[str]] = {} +CALL_RE = re.compile(r"\b([a-z0-9_]+_is_placeholder)\s+\"([^\"]*)\"") -for name, at in sorted(defs.items()): - check(len(at) == 1, - f"{STANDUP_REL} defines {name}() {len(at)} times (lines " - f"{', '.join(str(a + 1) for a in at)}) — which one runs depends on file order, and " - "this check can no longer say which one it extracted. Keep exactly one definition.") - if len(at) != 1: - continue - start = at[0] - end = next((j for j in range(start + 1, len(LINES)) if LINES[j] == "}"), None) - if end is None: - bail(f"{name}() at {STANDUP_REL}:{start + 1} has no closing brace at column 0 — the " - "extraction boundary this check relies on is gone. Restore the conventional " - "layout, or move the predicate into a sourceable library and point this check at it.") - body = LINES[start:end + 1] - block = "\n".join(body) - check(block in TEXT, - f"extracted {name}() is not a verbatim substring of {STANDUP_REL} — extraction bug; " - "the harness would be testing something the script does not contain") +if GUARD_REL: + TEXT = (ROOT / GUARD_REL).read_text() + LINES = TEXT.splitlines() - words: list[str] = [] - for raw in body[1:-1]: - s = raw.strip() - if not any(rx.match(s) for rx in ALLOWED) or "$(" in s or "`" in s: - bail(f"{name}() at {STANDUP_REL}:{start + 1} is no longer a SELF-CONTAINED " - f"case/return predicate (offending line: {s!r}). This check exercises the REAL " - "function by sourcing its own lines; a body that calls out to anything else " - "cannot be extracted faithfully, and silently testing a COPY of the logic is " - "exactly the failure mode this check exists to prevent. REMEDY: move the " - "predicate into a tiny library that standup.sh sources (e.g. " - "bootstrap/lib/placeholder.sh) and point STANDUP_REL/this extractor at it — do " - "NOT reimplement the matching here.") - m = ARM_RE.match(s) - if not m: + defs: dict[str, list[int]] = {} + for i, line in enumerate(LINES): + m = DEF_RE.match(line) + if m: + defs.setdefault(m.group(1), []).append(i) + + ARM_RE = re.compile(r"^(?P[^()]*?)\)\s*return\s+[01]\s*;;$") + ALLOWED = ( + re.compile(r"^$"), + re.compile(r"^#"), + re.compile(r"^case\s+\"\$\{?[0-9A-Za-z_]+\}?\"\s+in$"), + re.compile(r"^esac$"), + re.compile(r"^return\s+[01]$"), + ARM_RE, + ) + + for name, at in sorted(defs.items()): + check(len(at) == 1, + f"{GUARD_REL} defines {name}() {len(at)} times (lines " + f"{', '.join(str(a + 1) for a in at)}) — which one runs depends on file order, and " + "this check can no longer say which one it extracted. Keep exactly one definition.") + if len(at) != 1: continue - for pat in m.group("pats").split("|"): - pat = pat.strip() - if pat in ('""', "''", ""): - continue # the empty-value arm: covered by the canary - words.append(pat.strip("'\"").strip("*")) - check(len(words) >= 1, - f"{name}() has no non-empty placeholder pattern left — every arm that recognises a " - "loud placeholder string has been removed, so the guard now only refuses the empty " - "value and any 'TBD'-style endpoint would sail straight through it") - guard_words[name] = words - guard_src.append(block) + start = at[0] + end = next((j for j in range(start + 1, len(LINES)) if LINES[j] == "}"), None) + if end is None: + bail(f"{name}() at {GUARD_REL}:{start + 1} has no closing brace at column 0 — the " + "extraction boundary this check relies on is gone. Restore the conventional " + "layout, or move the predicate into a sourceable library.") + body = LINES[start:end + 1] + block = "\n".join(body) + check(block in TEXT, + f"extracted {name}() is not a verbatim substring of {GUARD_REL} — extraction bug; " + "the harness would be testing something the script does not contain") -(WORK / "guards.sh").write_text("\n".join(guard_src) + "\n") + words: list[str] = [] + for raw in body[1:-1]: + s = raw.strip() + if not any(rx.match(s) for rx in ALLOWED) or "$(" in s or "`" in s: + bail(f"{name}() at {GUARD_REL}:{start + 1} is no longer a SELF-CONTAINED " + f"case/return predicate (offending line: {s!r}). This check exercises the " + "REAL function by sourcing its own lines; a body that calls out to anything " + "else cannot be extracted faithfully, and silently testing a COPY of the " + "logic is exactly the failure mode this check exists to prevent. REMEDY: " + "move the predicate into a tiny library that the script sources (e.g. " + "bootstrap/lib/placeholder.sh) — discovery will find it — do NOT " + "reimplement the matching here.") + m = ARM_RE.match(s) + if not m: + continue + for pat in m.group("pats").split("|"): + pat = pat.strip() + if pat in ('""', "''", ""): + continue # the empty-value arm: covered by the canary + words.append(pat.strip("'\"").strip("*")) + check(len(words) >= 1, + f"{name}() has no non-empty placeholder pattern left — every arm that recognises a " + "loud placeholder string has been removed, so the guard now only refuses the empty " + "value and any 'TBD'-style endpoint would sail straight through it") + guard_words[name] = words + guard_src.append(block) -# ------------------------------------------------- no divergent second copy of the logic -# Any of the guard's own vocabulary appearing in EXECUTABLE text outside the predicate means -# a second, independently-drifting implementation of the same test (which is how three -# guards came to disagree with one value in the first place). -guard_line_nums = {j for name, at in defs.items() if len(at) == 1 - for j in range(at[0], next(k for k in range(at[0] + 1, len(LINES)) - if LINES[k] == "}") + 1)} -all_words = sorted({w.upper() for ws in guard_words.values() for w in ws if w}) -divergent: list[str] = [] -for j, line in enumerate(LINES): - if j in guard_line_nums or line.strip().startswith("#"): - continue - code = line.split("#", 1)[0] - hit = [w for w in all_words if w in code.upper()] - if hit: - divergent.append(f"{STANDUP_REL}:{j + 1} ({', '.join(hit)}): {line.strip()!r}") -check(not divergent, - "the guard's own placeholder vocabulary appears in EXECUTABLE text outside the " - f"predicate — a SECOND implementation of the same test, which is precisely how a value " - f"change disarmed three guards at once. Call the predicate instead. Sites: " - + " | ".join(divergent)) + if not guard_words: + bail(f"{GUARD_REL} was discovered as the guard file but yielded no usable predicate — " + "the definitions are there but none could be extracted. Fix the extraction in the " + "SAME change; do not let this degrade to a pass.") + + # ------------------------------------------- no divergent second copy of the logic + # The guard's own vocabulary in EXECUTABLE text outside the predicate means a second, + # independently-drifting implementation of the same test (which is how three guards came + # to disagree with one value in the first place). + guard_line_nums = {j for name, at in defs.items() if len(at) == 1 + for j in range(at[0], next(k for k in range(at[0] + 1, len(LINES)) + if LINES[k] == "}") + 1)} + divergent: list[str] = [] + _words = sorted({w.upper() for ws in guard_words.values() for w in ws if w}) + for j, line in enumerate(LINES): + if j in guard_line_nums or line.strip().startswith("#"): + continue + code = line.split("#", 1)[0] + hit = [w for w in _words if w in code.upper()] + if hit: + divergent.append(f"{GUARD_REL}:{j + 1} ({', '.join(hit)}): {line.strip()!r}") + check(not divergent, + "the guard's own placeholder vocabulary appears in EXECUTABLE text outside the " + "predicate — a SECOND implementation of the same test, which is precisely how a value " + "change disarmed three guards at once. Call the predicate instead. Sites: " + + " | ".join(divergent)) + + # function bodies, for the consumer trace + FUNC_RE = re.compile(r"^([a-z0-9_]+)\(\)\s*\{") + i = 0 + while i < len(LINES): + m = FUNC_RE.match(LINES[i]) + if m: + end = next((j for j in range(i + 1, len(LINES)) if LINES[j] == "}"), len(LINES) - 1) + funcs[m.group(1)] = LINES[i + 1:end] + i = end + 1 + continue + i += 1 + check(len(funcs) > 5, f"parsed only {len(funcs)} function(s) out of {GUARD_REL} — the " + "consumer assertion (B2) would be vacuous") + +(WORK / "guards.sh").write_text(("\n".join(guard_src) + "\n") if guard_src else "") # ------------------------------------------------------------- cluster-vars, as authored CV = ROOT / "clusters" / "base" / "cluster-vars.yaml" -if not CV.is_file(): - bail("clusters/base/cluster-vars.yaml is absent — the authored half of the coupling is " - "missing and this check cannot mean anything") - base: dict[str, str] = {} for d in yaml.safe_load_all(CV.read_text()): if isinstance(d, dict) and d.get("kind") == "ConfigMap" \ @@ -282,12 +393,16 @@ for d in yaml.safe_load_all(CV.read_text()): check(len(base) > 5, f"clusters/base/cluster-vars.yaml yielded {len(base)} knob(s) — the " "ConfigMap parse failed, so every value assertion below would be vacuous") +# Effective values per overlay, and WHICH knobs that overlay states explicitly. The second +# half is what lets emptiness be judged as authored-or-forgotten instead of guessed. overlays: dict[str, dict[str, str]] = {} +stated: dict[str, set[str]] = {} for overlay in sorted((ROOT / "clusters").glob("*/kustomization.yaml")): env = overlay.parent.name if env == "base": continue eff = dict(base) + explicit: set[str] = set() try: k = yaml.safe_load(overlay.read_text()) or {} except Exception as exc: # noqa: BLE001 @@ -308,27 +423,15 @@ for overlay in sorted((ROOT / "clusters").glob("*/kustomization.yaml")): m = re.match(r"^/data/([A-Za-z0-9_]+)$", str(op.get("path", ""))) if m and str(op.get("op", "")) in ("replace", "add"): eff[m.group(1)] = "" if op.get("value") is None else str(op["value"]) + explicit.add(m.group(1)) overlays[env] = eff + stated[env] = explicit check(len(overlays) >= 1, - "no clusters//kustomization.yaml overlay was parsed — assertion (2) would be vacuous") - -# ------------------------------------------------ which knobs is a guard responsible for? -# Function bodies of standup.sh, then a one-hop taint trace from `x="$(cvar KEY)"` to a -# `*_is_placeholder "${x}"` call. Both halves are read out of the file; nothing is named here. -FUNC_RE = re.compile(r"^([a-z0-9_]+)\(\)\s*\{") -funcs: dict[str, list[str]] = {} -i = 0 -while i < len(LINES): - m = FUNC_RE.match(LINES[i]) - if m: - end = next((j for j in range(i + 1, len(LINES)) if LINES[j] == "}"), len(LINES) - 1) - funcs[m.group(1)] = LINES[i + 1:end] - i = end + 1 - continue - i += 1 -check(len(funcs) > 5, f"parsed only {len(funcs)} function(s) out of {STANDUP_REL} — the " - "consumer assertion (3) would be vacuous") + "no clusters//kustomization.yaml overlay was parsed — assertion (A1) would be vacuous") +# ------------------------------------------ which knobs is a guard responsible for? (HALF B) +# A one-hop taint trace from `x="$(cvar KEY)"` to a `*_is_placeholder "${x}"` call. Both +# halves are read out of the file; nothing is named here. CVAR_RE = re.compile(r"([A-Za-z_][A-Za-z0-9_]*)=\"\$\(\s*cvar\s+([A-Z0-9_]+)\s*\)\"") ALIAS_RE = re.compile(r"([A-Za-z_][A-Za-z0-9_]*)=\"\$\{([A-Za-z_][A-Za-z0-9_]*)[:}]") @@ -347,8 +450,6 @@ for fname, body in funcs.items(): keys_of_var.setdefault(m.group(1), set()).update(src) called_on = {m.group(2) for m in CALL_RE.finditer(text)} called_vars = {re.sub(r"^\$\{?|\}?$", "", a) for a in called_on} - # Aggregate per (function, knob): the knob is guarded HERE if ANY variable carrying its - # value is passed to a predicate — `live="$(cvar K)"; ep="${live}"; guard "${ep}"` counts. guarded_here: dict[str, bool] = {} for var, keys in keys_of_var.items(): for key in keys: @@ -358,43 +459,119 @@ for fname, body in funcs.items(): if g: guarded_keys.add(key) -if not guarded_keys: - bail(f"no cluster-vars knob in {STANDUP_REL} is passed to a *_is_placeholder predicate. " +if GUARD_REL and not guarded_keys: + bail(f"no cluster-vars knob in {GUARD_REL} is passed to a *_is_placeholder predicate. " "Either the guards were removed, or the way the script reads cluster-vars changed and " "this check's derivation must be updated in the SAME change — it must never silently " "verify an empty set.") -# (3) CONSUMERS: every reader of a guarded knob must apply the guard to it. +# (B2) CONSUMERS: every reader of a guarded knob must apply the guard to it. for fname, key, guarded in sorted(consumers): if key not in guarded_keys: continue # nothing claims responsibility for this knob at all check(guarded, - f"{STANDUP_REL}: {fname}() reads cluster-vars knob {key} but never passes it to a " + f"{GUARD_REL}: {fname}() reads cluster-vars knob {key} but never passes it to a " "*_is_placeholder guard, while other functions do. On 2026-09-05 all three consumers " "of S3_BACKUP_ENDPOINT were disarmed at once; a consumer that reads the knob and skips " "the guard is that failure in one function.") -# ---------------------------------------------------------------- independent detector -# Deliberately NOT derived from the guard alone (see the header): SEED is universal -# loud-placeholder vocabulary, and the guard's own words are added so that a guard whose -# ANCHORING is wrong still has its vocabulary tested against real values. +# ------------------------------------------------------------------ independent detector +# Deliberately NOT derived from the guard alone: SEED is universal loud-placeholder +# vocabulary, and the guard's own words are added so that a guard whose ANCHORING is wrong +# still has its vocabulary tested against real values. SEED = ["TBD", "TODO", "FIXME", "PLACEHOLDER", "REPLACE-ME", "REPLACEME", "CHANGE-ME", "CHANGEME", "FILL-ME", "FILLME", "SET-ME", "XXXX"] +all_words = sorted({w.upper() for ws in guard_words.values() for w in ws if w}) VOCAB = sorted({w.upper() for w in SEED} | set(all_words)) ANGLE = re.compile(r"<[^>\s]+>") -def placeholder_shaped(value: str) -> bool: - if value.strip() == "": - return True # an unset endpoint is not a target +def loud(value: str) -> bool: + """A positive statement that the value was never decided. The EMPTY STRING IS NOT LOUD — + see the header: cluster-vars authors "" deliberately, and treating it as a placeholder is + the false positive that made the original check red against its own input.""" up = value.upper() return any(w in up for w in VOCAB) or bool(ANGLE.search(value)) -rows: list[tuple[str, str, str, str]] = [] # guard, expect, origin, value -GUARD = sorted(guard_words)[0] if len(guard_words) == 1 else None +def guard_shaped(value: str) -> bool: + """What a GUARD must refuse: loud, or empty. Empty counts here only because a `""` + case-arm is the guard author's declaration that empty is not a usable target.""" + return value.strip() == "" or loud(value) -# (1) VALUES: base default and every overlay's value, for every guarded knob. + +# ------------------------------------------------------------------- (A4) DETECTOR CANARIES +# The detector is now the thing doing the refusing in HALF A, so it gets the same +# no-op interrogation the guard does. These strings are the real ones from the incident. +check(loud("https://TBD-R2-ACCOUNT-ID.r2.cloudflarestorage.com"), + "the detector does not recognise the exact 2026-09-05 placeholder " + "'https://TBD-R2-ACCOUNT-ID.r2.cloudflarestorage.com' — it is prefix-anchored, which is " + "the defect this check exists to catch, now in the check itself") +check(loud("TBD-PROD-BULK-NFS-SERVER"), + "the detector does not recognise the repo's own TBD-PROD-* sentinel convention " + "(clusters/README.md) — assertion (A1) would pass over the very values it is about") +check(loud(""), + "the detector does not recognise an template as a placeholder") +check(not loud("https://0123456789abcdef0123456789abcd.r2.cloudflarestorage.com"), + "the detector calls a REAL Cloudflare R2 endpoint a placeholder — it would false-block " + "every prod backup target") +check(not loud("http://192.168.1.9:9000"), + "the detector calls a REAL MinIO endpoint a placeholder — it would false-block dev") +check(not loud(""), + "the detector calls the EMPTY STRING loud. cluster-vars authors \"\" deliberately for " + "suffix/prefix knobs and documents it as load-bearing; emptiness is held to the " + "explicitness rule (A2) instead. This exact false positive made the original check FAIL " + "2 of 34 assertions against the real guard.") + +# --------------------------------- (A1) no overlay ships a LOUD value nothing refuses +a1_rows = 0 +for env, eff in sorted(overlays.items()): + for key, value in sorted(eff.items()): + if key in guarded_keys: + continue # a guard owns this knob: HALF B (B1) asserts it refuses it + a1_rows += 1 + check(not loud(value), + f"clusters/{env}: knob {key} resolves to the placeholder-shaped value {value!r}, " + "and nothing refuses it" + + (f" — no guard in {GUARD_REL} covers this knob." if GUARD_REL + else " — and no placeholder guard exists in the tree at all.") + + " Either give it a real value before it reaches an overlay, or guard it the way " + "the S3 endpoint is guarded. An overlay that ships a placeholder no guard " + "recognises is the 2026-09-05 failure with a different knob's name on it: it " + "applies clean, reports Ready, and the thing it configures does not exist.") + +# ------------------------- (A2) an empty effective value must be STATED, never inherited +a2_rows = 0 +for env, eff in sorted(overlays.items()): + for key, value in sorted(eff.items()): + if value.strip() != "": + continue + a2_rows += 1 + check(key in stated[env], + f"clusters/{env}: knob {key} resolves to the EMPTY STRING by INHERITANCE from " + "clusters/base — the overlay never states it. An empty value can be correct (dev's " + "BACKUP_CREDENTIALS_SUFFIX and VELERO_BACKUP_PREFIX are authored empty on purpose), " + "but only when the overlay SAYS SO: cluster-vars' own convention is that both " + "overlays state such a knob explicitly so neither inherits it silently. An empty " + "value nobody stated is indistinguishable from a knob somebody forgot.") + +# ---------------- (A3) a LOUD base sentinel must be overridden EXPLICITLY by every overlay +a3_rows = 0 +for key, value in sorted(base.items()): + if not loud(value): + continue + for env in sorted(overlays): + a3_rows += 1 + check(key in stated[env], + f"clusters/base/cluster-vars.yaml authors the LOUD sentinel {key}={value!r}, and " + f"clusters/{env} does not state that knob — so that overlay INHERITS the sentinel. " + "The TBD-PROD-* convention only works if a sentinel cannot be inherited by " + "omission; an omitted op is how prod silently ships a value nobody decided.") + +# --------------------------------------- HALF B corpus: authored values + anchoring + canaries +rows: list[tuple[str, str, str, str]] = [] # guard, expect, origin, value + +# (B1) VALUES: base default and every overlay's value, for every guarded knob. for key in sorted(guarded_keys): seen: list[tuple[str, str]] = [] if key in base: @@ -404,24 +581,12 @@ for key in sorted(guarded_keys): check(bool(seen), f"guarded knob {key} is not defined in cluster-vars at all — the guard " "protects a knob nothing authors") for origin, value in seen: - expect = "placeholder" if placeholder_shaped(value) else "usable" + expect = "placeholder" if guard_shaped(value) else "usable" for name in sorted(guard_words): rows.append((name, expect, origin, value)) -# (2) OVERLAYS: a placeholder-shaped effective value for an UNGUARDED knob. -for env, eff in sorted(overlays.items()): - for key, value in sorted(eff.items()): - if key in guarded_keys or not placeholder_shaped(value): - continue - check(False, - f"clusters/{env}: knob {key} resolves to the placeholder-shaped value {value!r}, " - f"and NO guard in {STANDUP_REL} refuses it. Either give it a real value before it " - "reaches an overlay, or guard it the way the S3 endpoint is guarded — an overlay " - "that ships a placeholder no guard recognises is the 2026-09-05 failure with a " - "different knob's name on it.") - -# (4) ANCHORING: the guard's own vocabulary must match ANYWHERE, not just as a prefix, -# and a realistic endpoint must still be accepted. +# (B3) ANCHORING: the guard's own vocabulary must match ANYWHERE, not just as a prefix, +# and a realistic endpoint must still be accepted. for name, words in sorted(guard_words.items()): for w in sorted(set(words)): rows.append((name, "placeholder", @@ -436,15 +601,16 @@ for name, words in sorted(guard_words.items()): # Every guarded knob must reach the oracle once per authored source (base + each overlay), # so a knob silently dropping out of the corpus is a failure rather than a quiet pass. -authored = [r for r in rows if not r[2].startswith(("anchoring:", "canary:"))] -expected_rows = len(guarded_keys) * (1 + len(overlays)) * len(guard_words) -check(len(authored) == expected_rows, - f"{len(authored)} authored value(s) reached the guard, expected {expected_rows} " - f"({len(guarded_keys)} guarded knob(s) x base+{len(overlays)} overlay(s) x " - f"{len(guard_words)} predicate(s)) — a guarded knob is missing from cluster-vars or from " - "an overlay's effective values, and the coupling is only partly exercised") +if GUARD_REL: + authored = [r for r in rows if not r[2].startswith(("anchoring:", "canary:"))] + expected_rows = len(guarded_keys) * (1 + len(overlays)) * len(guard_words) + check(len(authored) == expected_rows, + f"{len(authored)} authored value(s) reached the guard, expected {expected_rows} " + f"({len(guarded_keys)} guarded knob(s) x base+{len(overlays)} overlay(s) x " + f"{len(guard_words)} predicate(s)) — a guarded knob is missing from cluster-vars or " + "from an overlay's effective values, and the coupling is only partly exercised") -# ------------------------------------------------------- (6) the check is load-bearing +# --------------------------------------------------- (A5) the check is load-bearing L0 = (ROOT / "ci" / "l0.sh").read_text() if (ROOT / "ci" / "l0.sh").is_file() else "" GATE = (ROOT / "ci" / "gate.sh").read_text() if (ROOT / "ci" / "gate.sh").is_file() else "" check("--layer l0" in L0, @@ -452,6 +618,14 @@ check("--layer l0" in L0, check(bool(re.search(r"^for s in .*\bl0\b.*; do", GATE, re.M)) and bool(re.search(r"fail\).*BLOCK=1", GATE)), "ci/gate.sh does not set the merge block when the l0 stage fails") +# ------------------------------------------------------- non-vacuity floor for HALF A +# HALF A is the half that runs unconditionally, so it carries its own floor: if the +# authored-value sweep shrinks to nothing, a pass here means nothing. +check(a1_rows >= 20, + f"only {a1_rows} authored overlay value(s) were audited by (A1) — the sweep over " + "clusters/ effective cluster-vars has collapsed, and a pass would be meaningless. " + "Expected roughly (knobs x overlays); check the ConfigMap and overlay parse above.") + # --------------------------------------------------------------------------- handoff with (WORK / "corpus.tsv").open("w") as fh: for name, expect, origin, value in rows: @@ -459,17 +633,31 @@ with (WORK / "corpus.tsv").open("w") as fh: (WORK / "findings.txt").write_text("".join(f"{f}\n" for f in FAILURES)) (WORK / "counts.txt").write_text(f"{CHECKS}\n") -print(f"guards: {', '.join(sorted(guard_words))} " - f"vocabulary: {', '.join(VOCAB)}") -print(f"guarded knobs: {', '.join(sorted(guarded_keys))} " - f"consumers traced: {len(consumers)} overlays: {', '.join(sorted(overlays))}") +if GUARD_REL: + print(f"guard file: {GUARD_REL} predicates: {', '.join(sorted(guard_words))}") + print(f"guarded knobs: {', '.join(sorted(guarded_keys))} " + f"consumers traced: {len(consumers)}") +else: + print("guard file: NONE FOUND in the tree — HALF B (guard/value coupling) IS NOT PROVEN " + "ON THIS TREE. No *_is_placeholder predicate exists to extract: bootstrap/standup.sh " + "has never been on main (no deletion commit exists; it lives on unmerged prod-standup " + "branches). HALF A below runs against the real authored values and is what currently " + "prevents the 2026-09-05 outcome. When a guard lands ANYWHERE in the tree, discovery " + "picks it up and HALF B activates with no edit to this check.") +print(f"detector vocabulary: {', '.join(VOCAB)}") +print(f"HALF A: overlays {', '.join(sorted(overlays))}; {len(base)} base knob(s); " + f"(A1) {a1_rows} authored overlay value(s) screened; (A2) {a2_rows} empty value(s) " + f"checked for explicit statement; (A3) {a3_rows} base-sentinel inheritance check(s)") for name, expect, origin, value in rows: if not origin.startswith(("anchoring:", "canary:")): print(f" authored {origin} = {value!r} -> expect {expect}") PY +GUARD_REL="$(tr -d '\n' < "${WORK}/mode.txt")" + # --------------------------------------------------------------------------------------- # PHASE 2 (bash): source the REAL predicate lines and run the corpus through them. +# Empty when no guard exists — HALF A has already run and carries its own floor. # --------------------------------------------------------------------------------------- # shellcheck disable=SC1090,SC1091 source "${WORK}/guards.sh" @@ -483,9 +671,9 @@ while IFS=$'\t' read -r guard expect origin value; do if [ "${verdict}" != "${expect}" ]; then oracle_failures=$((oracle_failures + 1)) if [ "${expect}" = placeholder ]; then - printf '%s\n' "${guard}() in ${STANDUP_REL} ACCEPTS '${value}' (from ${origin}) as a usable value, but it is placeholder-shaped. This is the 2026-09-05 failure exactly: the guard is asked about a value it does not recognise, returns 'fine', and the tier it protects is enabled against something that does not exist. Fix the predicate (match the idiom anywhere, not as a prefix) or stop authoring that idiom." >> "${WORK}/findings.txt" + printf '%s\n' "${guard}() in ${GUARD_REL} ACCEPTS '${value}' (from ${origin}) as a usable value, but it is placeholder-shaped. This is the 2026-09-05 failure exactly: the guard is asked about a value it does not recognise, returns 'fine', and the tier it protects is enabled against something that does not exist. Fix the predicate (match the idiom anywhere, not as a prefix) or stop authoring that idiom." >> "${WORK}/findings.txt" else - printf '%s\n' "${guard}() in ${STANDUP_REL} REFUSES '${value}' (from ${origin}), which is a usable value. An over-matching guard leaves the tier it protects suspended forever and reports it as an operator decision — a silent failure in the other direction." >> "${WORK}/findings.txt" + printf '%s\n' "${guard}() in ${GUARD_REL} REFUSES '${value}' (from ${origin}), which is a usable value. An over-matching guard leaves the tier it protects suspended forever and reports it as an operator decision — a silent failure in the other direction." >> "${WORK}/findings.txt" fi fi done < "${WORK}/corpus.tsv" @@ -495,11 +683,13 @@ py_checks="$(cat "${WORK}/counts.txt")" total=$((py_checks + oracle_rows)) findings="$(cat "${WORK}/findings.txt")" -# NO-OP GUARD: this repo has been bitten by a check that passed when perturbed because it +# NO-OP GUARDS: this repo has been bitten by a check that passed when perturbed because it # asked a question that was always satisfiable. The floors below are the minimum evidence -# for the verdict to mean anything, not a style rule. -if [ "${oracle_rows}" -lt 6 ]; then - findings="${findings}only ${oracle_rows} value(s) were put to the real guard — the coupling is not actually being exercised +# for the verdict to mean anything, not a style rule. The oracle floor applies only when a +# guard exists — when none does, HALF A's own floor (a1_rows >= 20, asserted in phase 1) is +# what stops this from degenerating, and the absence is reported on stdout. +if [ -n "${GUARD_REL}" ] && [ "${oracle_rows}" -lt 6 ]; then + findings="${findings}only ${oracle_rows} value(s) were put to the real guard in ${GUARD_REL} — the coupling is not actually being exercised " fi if [ "${total}" -lt 20 ]; then @@ -513,4 +703,8 @@ if [ -n "${findings}" ]; then exit 1 fi -echo "check OK: placeholder-guard-coupling — ${total} assertions; ${oracle_rows} value(s) run through the REAL extracted predicate(s) from ${STANDUP_REL} (no reimplementation), covering every authored cluster-vars value of every guarded knob in base and each overlay, the guard's own vocabulary matched mid-string, and the empty/real-endpoint canaries. Static slice only: refusal, not reachability — see the header." +if [ -n "${GUARD_REL}" ]; then + echo "check OK: placeholder-guard-coupling — ${total} assertions. HALF A: no environment overlay ships a placeholder-shaped effective cluster-vars value, every empty value is stated explicitly rather than inherited, and the detector canaries hold (including the exact 2026-09-05 string, matched mid-string). HALF B: ${oracle_rows} value(s) run through the REAL extracted predicate(s) from ${GUARD_REL} (no reimplementation), covering every authored value of every guarded knob in base and each overlay, the guard's vocabulary matched mid-string, and the empty/real-endpoint canaries. Static slice only: refusal, not reachability — see the header." +else + echo "check OK: placeholder-guard-coupling — ${total} assertions. HALF A ONLY: no environment overlay ships a placeholder-shaped effective cluster-vars value, every empty value is stated explicitly rather than inherited, no LOUD base sentinel is inherited by omission, and the detector canaries hold (including the exact 2026-09-05 string 'https://TBD-R2-ACCOUNT-ID.r2.cloudflarestorage.com', matched mid-string). HALF B (guard/value coupling) IS NOT PROVEN: no *_is_placeholder predicate exists anywhere in the tree to extract — see the stdout note above and the header. This is a REPORTED gap, not a skip: what a guard would be asked about is proven not to be a placeholder in the first place." +fi diff --git a/docs/BULK-NFS-STORAGE.md b/docs/BULK-NFS-STORAGE.md index b9af86f..3d99879 100644 --- a/docs/BULK-NFS-STORAGE.md +++ b/docs/BULK-NFS-STORAGE.md @@ -252,18 +252,28 @@ so **plex, sonarr, radarr and nextcloud would sit in ContainerCreating on prod.* revision that carries these bulk volumes.** Dev renders a working value and is where these manifests are meant to be proven first. -### The static guard this convention leans on is missing from `main` +### The static guard this convention leans on is missing from `main` — but the gate now refuses the value anyway `bootstrap/standup.sh` — which carries the `*_is_placeholder` guard family — **does not exist on `main`** (verified: `git ls-tree main bootstrap/` lists only `README.md`, `flux/`, `gitea/`, `sops/`; -the file lives on unmerged prod branches). So -`conformance/tests/l0-static/placeholder-guard-coupling/check.sh` exits **3 SKIPPED**, and **nothing -in the gate currently refuses a placeholder-shaped overlay value.** +the file lives on unmerged prod branches, and there is no deletion commit for it anywhere — it never +landed rather than having been removed). -When `standup.sh` lands, that check's assertions (2) and (3) cover this knob automatically — its -independent detector's SEED vocabulary already matches `TBD` — **provided** the guard gains an arm -for `BULK_NFS_SERVER` and a consumer that calls it. Without the arm, assertion (2) is what fails, by -design. +**UPDATED 2026-09-13.** This section used to end "so +`conformance/tests/l0-static/placeholder-guard-coupling/check.sh` exits **3 SKIPPED**, and nothing in +the gate currently refuses a placeholder-shaped overlay value." That is **no longer true.** That check +was restructured (see DECISIONS-LOG, branch `fix/placeholder-guard-coupling-unskipped`): it never +exits 3, and its HALF A screens **every environment overlay's effective cluster-vars value** against +an independent placeholder detector with no guard required. A `BULK_NFS_SERVER` of +`TBD-PROD-BULK-NFS-SERVER` in `clusters/prod/` is now a **merge-gate FAIL** — verified by deliberately +authoring exactly that value and watching the check fail, which is the one hazard on this page that +CRD validation cannot catch. HALF A also refuses a LOUD base sentinel that an overlay inherits by +*omitting* its patch op, which is the other way this knob can silently ship wrong. + +The guard is now **discovered by content**, not by path, so when `standup.sh` (or its predicate in a +sourceable library) lands, the coupling half activates with no edit to the test — and it then covers +this knob **provided** the guard gains an arm for `BULK_NFS_SERVER` and a consumer that calls it. +Without the arm, HALF A is what holds the line, by design. ---