Home-Network/home-network-k3s-impl
2
0 Fork 0
Code Issues 2 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

merge: the placeholder guard test was hiding a RED, not a pass
gate / gate (push) Has been cancelled
Details
gate / conformance (push) Has been cancelled
Details

Browse Source 
It had been permanently SKIPPED because it expected bootstrap/standup.sh,
which never landed on main — planned, never merged, still living only on the
unmerged prod-standup branches along with the 2026-09-05 incident it came
from. Dropping the real script in and running the old check produced FAIL
2/34: it would have false-blocked the very merge that landed its own input.

Now split into the half that needs no guard (no overlay ships a
placeholder-shaped effective cluster-vars value; an empty value must be
STATED by the overlay, not inherited in silence; a loud base sentinel must
be explicitly overridden) and the half that self-activates. The guard is
discovered by CONTENT — any tracked *.sh defining <name>_is_placeholder() —
so landing one anywhere activates the second half with no edit here, and two
definitions is a failure. The check never exits 3 again.

Unscoped L0 goes 13 PASS / 1 SKIP -> 14 PASS / 0 SKIP.
This commit is contained in:
jakman85 committed 2026-09-13 00:14:17 -05:00
2 parents e49b86eb41 87c3d83dff
commit 5fa893095b
6 files changed
+716 -299
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

DECISIONS-LOG.md
+128
View File
@@ -2869,3 +2869,131 @@ PVC and both change the operator's own login story, so **neither was done**. Fla
in these pods), and three ~1 GB files named `...S18E05....zipx` / `...S06E01....exe` /
`...S29E01....exe`. Media files with `.exe`/`.zipx` extensions are worth an operator's eye; nothing
here read or moved them.
## 2026-09-13 — `placeholder-guard-coupling` was SKIPPING FOREVER, and the skip was hiding a RED (branch `fix/placeholder-guard-coupling-unskipped`)
An L0 audit flagged `conformance/tests/l0-static/placeholder-guard-coupling` (R-ENV-02, R-STATE-03)
as permanently SKIPPED. Confirmed, and the specifics change what the correct fix is, so they were
measured rather than assumed.
### Did `bootstrap/standup.sh` ever exist? NO — not on `main`, and it was never deleted
- `git log --all --diff-filter=D -- bootstrap/standup.sh` → **empty**. There is no deletion commit
for it anywhere in the repo.
- `git ls-tree main bootstrap/` → `README.md`, `flux/`, `gitea/`, `sops/`. No `standup.sh`.
- The file is real and substantial (988 lines) but lives only on **unmerged prod-standup branches**:
`work/prod-standup-campaign`, `prod/*`, `feat/prod-hardening`. First added in `4160fee`, last
touched in `8f9b2e4` ("revives a guard I had killed").
- So the check was authored against a file that **was planned and never landed**, not one that was
removed. `docs/BULK-NFS-STORAGE.md` §7 had already recorded this, including the consequence:
*"nothing in the gate currently refuses a placeholder-shaped overlay value."*
- The 2026-09-05 incident itself (`32d6766`, the MinIO → Cloudflare R2 move) is likewise **not in
this log on `main`** — it is on the same unmerged branches. The only record on `main` was the
check's own header, `ci/touched-requirements.sh`'s comment, and `dr-restore`'s test header. It is
written down here now so it survives whether or not those branches ever land.
### The skip was hiding a RED, which is worse
Dropped the real `standup.sh` from `work/prod-standup-campaign` into the tree and ran the OLD check:
**FAIL, 2 of 34 assertions.** Both were false positives from treating the **empty string** as
placeholder-shaped for knobs cluster-vars authors empty *on purpose* — `BACKUP_CREDENTIALS_SUFFIX`
and `VELERO_BACKUP_PREFIX`, both `""` on dev, and both documented in `cluster-vars.yaml` as
load-bearing ("THE BASE DEFAULT IS THE EMPTY STRING, AND THAT IS LOAD-BEARING … an EMPTY VALUE IS
SAFE UNDER StrictPostBuildSubstitutions: strict mode fails on a variable that is not DEFINED, not on
one defined as `""`"). So the check would have **false-blocked the merge that landed its own input**.
### NOT deleted — the hazard is not gone. Split into a half that needs no guard, and a half that self-activates
The invariant has two independent halves and only one needs the guard:
- **HALF A — the authored values.** No environment overlay may ship a placeholder-shaped *effective*
cluster-vars value. This is the 2026-09-05 **outcome** asserted at the source instead of at the
guard: if prod's `S3_BACKUP_ENDPOINT` is never a placeholder, velero cannot be pointed at a
hostname that does not exist, guard or no guard. Input surface is `clusters/`, all on `main`.
Runs unconditionally, 70 authored overlay values screened today.
- **HALF B — the coupling.** The real predicate extracted and executed against those same values,
the consumer trace, and the mid-string anchoring property. Needs a guard.
**The check never exits 3 again.** And the guard is now **discovered, not hardcoded**: any tracked
`*.sh` defining `<name>_is_placeholder()` is found, so a guard landing at `bootstrap/standup.sh` —
or moved to a sourceable library, which is the remedy the check already recommends — activates HALF B
**with no edit to the test**. Two files defining it is a FAIL (that is the 2026-09-05 divergence with
a second file's name on it). When no guard exists, the gap is printed loudly on stdout every run
instead of being swallowed as a skip.
**LOUD vs EMPTY** is the correction that makes it honest in both modes. A loud placeholder (`TBD`,
`TODO`, `REPLACE-ME`, `<angle-brackets>`, …) is always a defect in an overlay. The empty string is
not, so it is held to a *different real rule*: an empty effective value must be **stated explicitly**
by that overlay, never inherited in silence — which is cluster-vars' own documented convention
("Both overlays state this knob explicitly … so neither inherits it silently"). Emptiness still
counts as a placeholder for a knob a guard covers, because a `""` case-arm is the guard author's
declaration that empty is not a usable target.
### Proof it runs, and proof it is not vacuous
`check.sh` exits 0 at **84 assertions** on `main` (`(A1)` 70 authored overlay values, `(A2)` 2 empty
values checked for explicit statement, `(A3)` 0 live base sentinels, detector canaries, load-bearing).
Four deliberate breakages, each reverted:
| break | result |
| --- | --- |
| prod `S3_BACKUP_ENDPOINT` → `https://TBD-R2-ACCOUNT-ID.r2.cloudflarestorage.com` (the literal 2026-09-05 value) | **FAIL** 1/84 — (A1) |
| delete dev's explicit `VELERO_BACKUP_PREFIX` op so it inherits `""` | **FAIL** 1/84 — (A2) |
| base `BULK_NFS_SERVER` → `TBD-PROD-BULK-NFS-SERVER` + delete prod's override | **FAIL** 2/86 — (A1)+(A3) |
| drop in the real `standup.sh`, then revert `s3_is_placeholder` to the `TBD-*` **prefix** glob | **FAIL** 12/110 — (B3) anchoring |
With the pristine `standup.sh` present, discovery finds it and the check **PASSES at 110
assertions** — where the old one failed 2 of 34. A second file defining the predicate is detected and
named. Coverage is unchanged: R-ENV-02 and R-STATE-03 both still list this test (49/51, 2 gaps,
identical before and after), because nothing was deleted.
**L0 layer: 14 checks, 14 PASS, 0 SKIP, 0 FAIL** (was 13 PASS / 1 SKIP).
### The other L0 skip-capable checks — audited, and none has this defect
Every L0 check whose `check.sh` can `exit 3` on a missing input path, with the path verified against
the tree:
| check | skip condition | on `main` |
| --- | --- | --- |
| `placeholder-guard-coupling` | *was* `bootstrap/standup.sh` | **was the only real one — fixed; it now FAILS rather than skips if `clusters/base/cluster-vars.yaml` goes missing, because that means the knobs moved and the check must be re-pointed in the same change** |
| `bulk-tier-distinct-from-replicated` | `catalog/` or `apps/` absent | both present — never fires |
| `flux-postbuild-substitution` | `clusters/` or `clusters/base/cluster-vars.yaml` absent | present — never fires |
| `keycloak-client-secret-reconciler` | any of 4 `platform/keycloak/*.yaml` absent | all 4 present — never fires |
| `spec-immutability-enforced` | the sibling spec repo unavailable | **legitimate and left alone** — an environmental dependency (`../home-network-k3s`, R-TEST-06), not a repo-state defect; present here, so it PASSES |
| `whole-repo-invariants-wired` | — | its `exit 3`s are *synthetic fixtures* it creates to prove the runner treats a skipped whole-repo test as UNWIRED, not its own skip |
So the disease was confined to one test. The four defensive guards above are dead code in practice
and were left alone rather than expanding scope.
### `ci/touched-requirements.sh` — the entry STAYS, the comment was wrong
`bootstrap/*` → `R-STATE-03 R-GITOPS-01` is **kept**. Its comment claimed `bootstrap/standup.sh`
"carries the placeholder guards", describing a file that is not on `main`; corrected to say
R-STATE-03 there is **anticipatory**. Deleting the entry would mean the change that eventually lands
a placeholder guard is selected by nothing — the 2026-09-05 class left to reintroduce itself — and
with content-based guard discovery that entry is exactly what pulls this test into that change's gate
run. `ci/tests/touched-requirements.test.sh` gained a **path-map section** (12 assertions, up from 8)
locking `bootstrap/*` → R-STATE-03 + R-GITOPS-01 and `clusters/*` → R-ENV-02 + R-GITOPS-01, so the
entry's removal is now a test failure rather than a silent loss of selection. Also corrected: the
check's own header claimed "an edit to `bootstrap/*` attributes NO requirement in the path map",
stale since 2026-09-06.
### Scope annotation: re-audited and still deliberately requirement-scoped, NOT whole-repo
HALF A reads `clusters/` (→ R-ENV-02); HALF B reads wherever the guard lands, i.e. `bootstrap/`
(→ R-STATE-03). Every path that can break the check attributes a requirement it declares, so
requirement-scoped selection cannot hide a breaking change and `whole-repo` would add runtime, not
coverage. One residual, recorded rather than papered over: `(A5)` reads `ci/l0.sh` and `ci/gate.sh`,
and `ci/*` attributes neither of this test's requirements — but that wiring is separately and
unconditionally asserted by the whole-repo-scoped `whole-repo-invariants-wired`, so `(A5)` here is
belt-and-braces rather than sole proof.
### Debt / follow-ups
- **HALF B is genuinely unproven on `main`,** and that is now *reported on stdout every run* rather
than hidden. It cannot be fixed from `main`: it needs `bootstrap/standup.sh` (or its predicate,
extracted into a sourceable library) to land. When it does, no edit to this test is required.
- **`prom-rules` still reports "skipped (missing tool) — not green"** in the gate; pre-existing,
`promtool` is being baked into the toolchain on another branch.
- **A placeholder hardcoded in a manifest** rather than in a cluster-vars knob is out of HALF A's
scope: it reads the ConfigMap and the overlay patches, not every YAML in the tree.
ci/tests/touched-requirements.test.sh
+43
View File
@@ -93,6 +93,49 @@ printf 'touched set (removal):\n%s\n---\n' "$OUT2"
assert_absent "R-DELDOC-01" "$OUT2" "REMOVED whole-line comment R-ID is NOT attributed"
assert_present "R-DELREAL-01" "$OUT2" "REMOVED real-line R-ID IS attributed"
# ---------------------------------------------------------------------------------------
# PATH MAP — bootstrap/* (added 2026-09-13)
#
# WHY THIS IS LOCKED DOWN. path_requirements() keys `bootstrap/*` to R-STATE-03 +
# R-GITOPS-01, and until 2026-09-13 the comment justifying it described
# `bootstrap/standup.sh` as a file that carries the placeholder guards. That file has never
# been on `main` (it landed only on unmerged prod-standup branches), so the comment was
# stale and the entry looked like dead weight someone would tidy away. It is not dead: it
# is what SELECTS conformance/tests/l0-static/placeholder-guard-coupling for the change
# that eventually lands a placeholder guard under bootstrap/ — and that test discovers its
# guard by content rather than by path, so it activates on exactly that change. Removing
# the entry would let the change introducing a guard be gated by nothing, which is the
# 2026-09-05 class. These assertions make the entry's deletion a test failure rather than a
# silent loss of selection.
#
# NOTE the R-ID content scan must not be what produces these: `notes/` is outside the path
# map precisely so the other cases isolate the scan, and here the file content cites no
# R-ID at all, so any R-ID in the output came from the PATH.
git -C "$TMP" checkout -q -b feature3 main
mkdir -p "$TMP/bootstrap/flux"
cat > "$TMP/bootstrap/flux/bootstrap.sh" <<'SH'
#!/usr/bin/env bash
# a bootstrap-path change that cites no requirement id of its own
echo "flux bootstrap"
SH
git -C "$TMP" add -A && git -C "$TMP" commit -qm "touch bootstrap/"
OUT3="$(REPO_ROOT="$TMP" BASE_REF=main "$SCRIPT" 2>/dev/null)"
printf 'touched set (bootstrap/ path map):\n%s\n---\n' "$OUT3"
assert_present "R-STATE-03" "$OUT3" "bootstrap/* attributes R-STATE-03 (selects placeholder-guard-coupling)"
assert_present "R-GITOPS-01" "$OUT3" "bootstrap/* attributes R-GITOPS-01 (it IS the GitOps bootstrap path)"
# And the neighbouring entry the same test leans on for its other half: clusters/* must
# attribute R-ENV-02, which is what selects placeholder-guard-coupling's HALF A (the
# authored-values sweep over clusters/<env> effective cluster-vars).
git -C "$TMP" checkout -q -b feature4 main
mkdir -p "$TMP/clusters/base"
printf 'kind: ConfigMap\nmetadata:\n name: cluster-vars\n' > "$TMP/clusters/base/cluster-vars.yaml"
git -C "$TMP" add -A && git -C "$TMP" commit -qm "touch clusters/"
OUT4="$(REPO_ROOT="$TMP" BASE_REF=main "$SCRIPT" 2>/dev/null)"
printf 'touched set (clusters/ path map):\n%s\n---\n' "$OUT4"
assert_present "R-ENV-02" "$OUT4" "clusters/* attributes R-ENV-02 (selects placeholder-guard-coupling HALF A)"
assert_present "R-GITOPS-01" "$OUT4" "clusters/* attributes R-GITOPS-01"
echo
if [ "$FAIL" -eq 0 ]; then echo "touched-requirements tests: PASS ($PASS assertions)"; exit 0;
else echo "touched-requirements tests: FAIL ($FAIL failed, $PASS passed)"; exit 1; fi
ci/touched-requirements.sh
+17 -8
View File
@@ -217,14 +217,23 @@ path_requirements() {
# chainsaw-test.yaml), and the correct fix for a path-scoped invariant is a path-map entry.
infra/*) echo "R-ENV-01" ;;
clusters/*) echo "R-GITOPS-01 R-ENV-02" ;;
# bootstrap/ was UNMAPPED until 2026-09-06, and the omission was load-bearing:
# bootstrap/standup.sh carries the placeholder guards that decide whether `velero` may be
# enabled against a real backup endpoint. A change touching standup.sh ALONGSIDE any
# already-mapped path scoped the L0 run to that other path's requirements and silently
# excluded conformance/tests/l0-static/placeholder-guard-coupling — the check that exists
# precisely to catch a guard/value mismatch. The 2026-09-05 guard bug (a prefix glob that
# stopped matching when the value it guards changed, in a different file) is exactly the
# class that omission would hide.
# bootstrap/ was UNMAPPED until 2026-09-06. The entry STAYS, but its original rationale
# was written against a file that is not here, and that is corrected on 2026-09-13:
# it claimed "bootstrap/standup.sh carries the placeholder guards that decide whether
# `velero` may be enabled against a real backup endpoint". bootstrap/standup.sh HAS NEVER
# BEEN ON `main` — it was not deleted (no deletion commit exists anywhere), it landed only
# on unmerged prod-standup branches. `git ls-tree main bootstrap/` lists README.md, flux/,
# gitea/, sops/. So R-STATE-03 here is ANTICIPATORY, not descriptive.
#
# WHY IT IS STILL RIGHT. conformance/tests/l0-static/placeholder-guard-coupling DISCOVERS
# its guard (any tracked *.sh defining `<name>_is_placeholder()`) instead of hardcoding a
# path, so the moment a standup guard lands under bootstrap/ that test's coupling half
# activates — and this entry is what selects the test for the change that lands it.
# Deleting the entry would mean the change introducing a placeholder guard is gated by
# nothing, which is the 2026-09-05 class (a prefix glob that stopped matching when the
# value it guards changed, in a different file) left to reintroduce itself. R-GITOPS-01 is
# descriptive today: bootstrap/flux/, gitea/ and sops/ ARE the GitOps bootstrap path.
# Locked down by ci/tests/touched-requirements.test.sh's path-map section.
bootstrap/*) echo "R-STATE-03 R-GITOPS-01" ;;
# R-TEST-06 added 2026-09-12. The comment in changed_diff()'s rule 3 above records why it
# was ABSENT: R-TEST-06 used to be enforced by the spec-immutability STAGE and carried by no
conformance/tests/l0-static/placeholder-guard-coupling/chainsaw-test.yaml
+77 -44
View File
@@ -1,43 +1,71 @@
# =============================================================================
# L0 STATIC — a placeholder value in cluster-vars is REFUSED by the guard that owns it
# L0 STATIC — no environment overlay ships a placeholder value, and the guard that owns
# one refuses it
# Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster.
#
# THE BUG CLASS THIS CLOSES (it has now bitten three times in one file). A guard in
# `bootstrap/standup.sh` keys on the SHAPE of a value that lives in
# `clusters/base/cluster-vars.yaml`, and nothing tied the two together. On 2026-09-05 the
# S3 placeholder changed from `TBD-PROD-S3-BACKUP-ENDPOINT` to
# `https://TBD-R2-ACCOUNT-ID.r2.cloudflarestorage.com`; the three guards matched the glob
# `TBD-*`, a PREFIX match, and `fnmatch('https://TBD-R2-…','TBD-*')` is False. All three
# guards silently stopped matching, and `standup.sh resume` would have resumed `velero` on
# prod against a nonexistent hostname: installs clean, reports healthy, NO BACKUPS EXIST.
# It was found by accident. The guard is fixed (`s3_is_placeholder`, substring); the
# COUPLING was untested, which is what makes the class reintroducible.
# `https://TBD-R2-ACCOUNT-ID.r2.cloudflarestorage.com` (commit 32d6766); the three guards
# matched the glob `TBD-*`, a PREFIX match, and `fnmatch('https://TBD-R2-…','TBD-*')` is
# False. All three guards silently stopped matching, and `standup.sh resume` would have
# resumed `velero` on prod against a nonexistent hostname: installs clean, reports healthy,
# NO BACKUPS EXIST. It was found by accident.
#
# WHY THIS TEST WAS RESTRUCTURED (2026-09-13). It was PERMANENTLY SKIPPED. `check.sh` exited
# 3 when `bootstrap/standup.sh` was absent — and that file has NEVER been on `main`: it was
# not deleted (no deletion commit exists anywhere), it landed only on unmerged prod-standup
# branches. So an incident-derived test reported SKIP on every gate run, and SKIPPED IS NOT
# GREEN here. The skip was also hiding a RED: run against the real `standup.sh` from
# `work/prod-standup-campaign`, the old check FAILED 2 of 34 assertions on false positives
# from treating the empty string as a placeholder for knobs cluster-vars authors empty on
# purpose. Both are fixed. The invariant is now split into a half that needs no guard and a
# half that activates when one lands, and THIS CHECK NEVER EXITS 3.
#
# GIVEN the repository as authored,
# WHEN the REAL predicate is extracted from bootstrap/standup.sh and executed by bash
# (never reimplemented — see check.sh "HOW THE REAL GUARD IS EXERCISED"),
# THEN (1) every authored value of a guarded knob that is placeholder-shaped by an
# INDEPENDENT detector is actually refused by that guard, in base and in every
# environment overlay;
# (2) no environment overlay carries a placeholder-shaped value for a knob no guard
# refuses;
# (3) every function in standup.sh that reads a guarded knob applies the guard to it
# (the "three guards died at once" regression);
# (4) the guard matches its own vocabulary ANYWHERE in a value, not just as a prefix
# — the exact anchoring defect above — and still accepts a real endpoint;
# AND (5) the extraction is faithful, and the check fails LOUDLY rather than silently
# testing a copy if the guard stops being a self-contained predicate.
# THEN HALF A — always, needs no guard, input surface `clusters/`:
# (A1) no environment overlay's EFFECTIVE cluster-vars value is placeholder-shaped by
# an INDEPENDENT detector, for any knob no guard refuses (base is exempt: it is
# not a bootstrap target and its loud sentinels exist to be overridden — but a
# sentinel an overlay forgets to override IS caught, as that overlay's value);
# (A2) an empty effective value is STATED explicitly by the overlay, never inherited
# in silence — cluster-vars' own documented convention, and the rule that
# separates "authored empty" from "somebody forgot";
# (A3) a LOUD base sentinel is explicitly overridden by every overlay, so the
# `TBD-PROD-*` convention cannot be defeated by an omitted patch op;
# (A4) the detector itself recognises the exact 2026-09-05 string MID-STRING, accepts
# both real endpoints the repo ships, and does not call the empty string loud;
# (A5) ci/l0.sh runs the L0 layer and ci/gate.sh blocks the merge when it fails.
# HALF B — when a `*_is_placeholder` predicate exists ANYWHERE in the tree (the guard
# is DISCOVERED, not hardcoded, so a guard landing at bootstrap/standup.sh or moving to
# a sourceable library activates this half with no edit here):
# (B1) every authored value of a guarded knob the detector calls placeholder-shaped is
# REFUSED by the REAL predicate, extracted from the real file and executed (never
# reimplemented — see check.sh "HOW THE REAL GUARD IS EXERCISED"), and every
# usable value is ACCEPTED;
# (B2) every function that reads a guarded knob applies the guard to it (the "three
# guards died at once" regression);
# (B3) the guard matches its own vocabulary ANYWHERE in a value, not just as a prefix
# — the exact anchoring defect above — and still accepts a real endpoint;
# (B4) the extraction is faithful, exactly one file and one definition own the
# vocabulary, and the check fails LOUDLY rather than silently testing a copy if
# the guard stops being a self-contained predicate.
#
# STATUS: EXECUTABLE. `check.sh` is a pure offline assertion script (standard library +
# PyYAML + bash); conformance/run.sh --layer l0 runs it directly and this Chainsaw Test
# wraps the SAME script.
# STATUS: EXECUTABLE, and it RUNS — never SKIP. `check.sh` is a pure offline assertion
# script (standard library + PyYAML + bash); conformance/run.sh --layer l0 runs it directly
# and this Chainsaw Test wraps the SAME script.
#
# NOT proven: that the endpoint is REACHABLE or that backups exist (runtime facts —
# `standup.sh s3check` probes the former from a pod, l3-fault-injection/dr-restore proves
# the latter); that guards for knobs OTHER than the placeholder-refusing ones are correct;
# and that the guard runs at all in a real standup (the dispatch path is shell, executed
# only against a live cluster). R-STATE-03 is tagged for the narrow static slice only: the
# authored backup target cannot be a placeholder that the guard fails to recognise.
# NOT proven: while no guard exists on `main`, HALF B IS NOT PROVEN, and the check SAYS SO on
# stdout every run rather than swallowing it as a skip — what is proven instead is the
# stronger-at-the-source half, that the values a guard would be asked about are not
# placeholders at all. Also not proven: that the endpoint is REACHABLE or that backups exist
# (runtime facts — `standup.sh s3check` probes the former from a pod, l3-fault-injection/
# dr-restore proves the latter); that guards for knobs OTHER than the placeholder-refusing
# ones are correct; and that the guard runs at all in a real standup (the dispatch path is
# shell, executed only against a live cluster). A1 reads cluster-vars, not every manifest: a
# placeholder hardcoded in a manifest rather than in a knob is out of scope. R-STATE-03 is
# tagged for the narrow static slice only: the authored backup target cannot be a placeholder.
# =============================================================================
apiVersion: chainsaw.kyverno.io/v1alpha1
kind: Test
@@ -50,27 +78,32 @@ metadata:
conformance.home.k3s/requirements: "R-ENV-02, R-STATE-03"
conformance.home.k3s/layer: "L0"
# NO `scope` ANNOTATION, DELIBERATELY — this invariant IS requirement-scoped, and was
# audited as such (2026-09-12). Its entire input surface is bootstrap/standup.sh (the
# placeholder guards) plus clusters/base/cluster-vars.yaml and clusters/<env>/
# (the values they guard). ci/touched-requirements.sh keys bootstrap/* to
# R-STATE-03 + R-GITOPS-01 and clusters/* to R-ENV-02 + R-GITOPS-01 — so EVERY path that
# can break this check already attributes one of the two requirements it declares above.
# Requirement-scoped selection therefore cannot hide a breaking change here, and declaring
# whole-repo scope would only add runtime, not coverage.
# re-audited as such on 2026-09-13 after the restructure changed the input surface.
# HALF A reads clusters/base/cluster-vars.yaml plus clusters/<env>/kustomization.yaml;
# ci/touched-requirements.sh keys clusters/* to R-ENV-02 + R-GITOPS-01, so every path
# that can break HALF A attributes a requirement declared above. HALF B reads whichever
# file defines the guard; bootstrap/* is keyed to R-STATE-03 + R-GITOPS-01, which covers
# the place a standup guard lands. Requirement-scoped selection therefore cannot hide a
# breaking change, and declaring whole-repo scope would only add runtime, not coverage.
# The one residual: (A5) reads ci/l0.sh and ci/gate.sh, and ci/* attributes neither
# R-ENV-02 nor R-STATE-03 — but that wiring is separately and unconditionally asserted by
# the whole-repo-scoped tests/l0-static/whole-repo-invariants-wired, so (A5) here is
# belt-and-braces rather than the only proof.
conformance.home.k3s/safety-critical: "false"
conformance.home.k3s/executable: "true"
conformance.home.k3s/intent: >-
Every placeholder-shaped value authored in cluster-vars, for any environment
overlay, is actually recognised as a placeholder by the real bootstrap guard that
is supposed to refuse it — so a value change in one file cannot silently disarm a
guard in another and enable velero against an endpoint that does not exist.
No environment overlay can ship a placeholder-shaped cluster-vars value that nothing
refuses, and where a real bootstrap guard owns such a knob it actually recognises
every authored value as a placeholder — so a value change in one file cannot silently
disarm a guard in another and enable velero against an endpoint that does not exist.
spec:
steps:
- name: couple-authored-placeholders-to-the-real-guard
- name: refuse-authored-placeholders-and-couple-them-to-any-real-guard
description: >-
Extract the real placeholder predicate from bootstrap/standup.sh, run it against
every authored cluster-vars value the repo actually carries, and assert the guard
refuses each one an independent detector calls a placeholder.
Screen every authored effective cluster-vars value of every environment overlay
against an independent placeholder detector, assert empty values are stated rather
than inherited, and — when a *_is_placeholder predicate exists anywhere in the tree
— extract and execute the REAL guard against those same values.
try:
- script:
content: bash ./check.sh
conformance/tests/l0-static/placeholder-guard-coupling/check.sh
+433 -239
View File
@@ -12,20 +12,74 @@
# placeholder. Three of those guards keyed on the SHAPE of `S3_BACKUP_ENDPOINT`, which is
# authored in `clusters/base/cluster-vars.yaml`. On 2026-09-05 the placeholder changed
# from `TBD-PROD-S3-BACKUP-ENDPOINT` to `https://TBD-R2-ACCOUNT-ID.r2.cloudflarestorage.com`
# (the MinIO -> Cloudflare R2 move). The guards matched the glob `TBD-*` — a PREFIX match —
# and `fnmatch('https://TBD-R2-…','TBD-*')` is FALSE. All three guards silently stopped
# matching. `standup.sh resume` would then have resumed `velero` on prod against a hostname
# that does not exist: the HelmRelease installs clean, the Kustomization reports Ready, and
# NO BACKUPS EXIST. The only reason it was not shipped is that someone noticed by accident.
# (the MinIO -> Cloudflare R2 move, commit 32d6766). The guards matched the glob `TBD-*` —
# a PREFIX match — and `fnmatch('https://TBD-R2-…','TBD-*')` is FALSE. All three guards
# silently stopped matching. `standup.sh resume` would then have resumed `velero` on prod
# against a hostname that does not exist: the HelmRelease installs clean, the Kustomization
# reports Ready, and NO BACKUPS EXIST. It was found by accident.
#
# The predicate is fixed (`s3_is_placeholder`, substring match). The COUPLING is what was
# never tested: a guard in one file keying on a value in another, with nothing tying them
# together. This check is that tie, and it is derived from BOTH real files — no current
# string is hardcoded here, so it does not rot when the placeholder or the guard changes.
# WHY THIS CHECK WAS REWRITTEN (2026-09-13) — IT WAS SKIPPING FOREVER
# ------------------------------------------------------------------
# The original version keyed its ENTIRE existence on `bootstrap/standup.sh` and exited 3
# (SKIP) when that file was absent. Established by measurement, not assumption:
#
# * `bootstrap/standup.sh` HAS NEVER EXISTED ON `main`. It was not deleted — there is no
# deletion commit for it anywhere (`git log --all --diff-filter=D -- bootstrap/standup.sh`
# is empty). It lives on unmerged prod-standup branches (`work/prod-standup-campaign`,
# `prod/*`, `feat/prod-hardening`) and never landed. `git ls-tree main bootstrap/` lists
# only README.md, flux/, gitea/, sops/. docs/BULK-NFS-STORAGE.md §7 already recorded
# this, including the consequence: "nothing in the gate currently refuses a
# placeholder-shaped overlay value".
# * So this check reported SKIP on every gate run since it landed, and SKIPPED IS NOT
# GREEN in this repo. An incident-derived test that can only ever skip is the quiet
# version of the failure it was written to prevent.
# * Worse, the skip was HIDING A RED. Run against the real `standup.sh` from
# `work/prod-standup-campaign`, the original check FAILED 2 of 34 assertions — both
# false positives from treating the EMPTY STRING as placeholder-shaped for knobs whose
# emptiness is deliberately authored (`BACKUP_CREDENTIALS_SUFFIX` and
# `VELERO_BACKUP_PREFIX`, both `""` on dev). cluster-vars documents that emptiness as
# load-bearing ("THE BASE DEFAULT IS THE EMPTY STRING, AND THAT IS LOAD-BEARING … an
# EMPTY VALUE IS SAFE UNDER StrictPostBuildSubstitutions"). So the check would have
# false-blocked the moment its own input landed. That is fixed below.
#
# THE HAZARD IS NOT GONE, so this coverage is NOT obsolete and was NOT deleted. The
# invariant has two independent halves, and only one of them needs `standup.sh`:
#
# HALF A — THE AUTHORED VALUES (provable on `main` today, needs NO guard).
# No environment overlay may ship a placeholder-shaped effective cluster-vars value.
# This is the OUTCOME the 2026-09-05 guard existed to prevent, asserted at the source
# instead of at the guard: if prod's `S3_BACKUP_ENDPOINT` is never a placeholder, velero
# cannot be pointed at a hostname that does not exist, whether or not a guard catches
# it. Its entire input surface is `clusters/`, all of which is on `main`.
# HALF B — THE COUPLING (activates automatically when a guard lands).
# The real predicate, extracted from the real file and executed, agrees with an
# INDEPENDENT detector about every authored value; every consumer of a guarded knob
# applies the guard; the guard matches its vocabulary ANYWHERE, not as a prefix.
#
# THIS CHECK NEVER EXITS 3. Half A always runs, against real files, with an assertion
# floor. Half B runs when a guard exists and is reported as NOT PROVEN when one does not —
# stated loudly in the output rather than swallowed as a skip.
#
# THE GUARD IS DISCOVERED, NOT HARDCODED (this is the "re-point" made automatic)
# -----------------------------------------------------------------------------
# The original hardcoded `bootstrap/standup.sh`, which is exactly why a file that never
# landed could disable the whole check. Instead, every tracked `*.sh` in the tree is
# scanned for a `<name>_is_placeholder()` definition:
# * exactly one such FILE -> that is the guard; Half B runs against it, wherever it is.
# A guard that lands at `bootstrap/standup.sh`, or moves to `bootstrap/lib/placeholder.sh`,
# is picked up with no edit to this check.
# * none -> Half A only, with the gap named in the output.
# * more than one file -> FAIL. Two files defining the placeholder vocabulary is the
# 2026-09-05 divergence with a second file's name on it; this check cannot say which
# one a standup run would source.
# `PLACEHOLDER_GUARD_FILE` overrides discovery (repo-relative) for negative fixtures.
# NOTE: discovery is shell-only, because the guard family is shell. If the refusal ever
# moves to a non-shell mechanism (a Kyverno policy, a CEL admission rule), discovery finds
# nothing and says so — re-point this check deliberately in that change.
#
# HOW THE REAL GUARD IS EXERCISED (and why not a copy)
# ---------------------------------------------------
# `bootstrap/standup.sh` is NOT sourceable: at top level it parses argv, hard-exits when
# `standup.sh` is NOT sourceable: at top level it parses argv, hard-exits when
# --kubeconfig/--expect-server/<step> are absent, `cd`s to the repo root, exports
# KUBECONFIG, installs an EXIT trap and finally dispatches steps against a live cluster.
# So this check EXTRACTS the predicate's own source lines from the file and sources ONLY
@@ -35,105 +89,119 @@
# * the extracted body must be a SELF-CONTAINED predicate — `case`/`return` and nothing
# else, no command substitution, no call to another function. If the guard ever grows
# a dependency, extraction stops being faithful and THIS CHECK FAILS LOUDLY, naming the
# remedy (move the predicate into a small sourceable library that standup.sh sources)
# rather than quietly falling back to testing a copy of the logic;
# remedy (move the predicate into a small sourceable library) rather than quietly
# falling back to testing a copy of the logic;
# * a CANARY pins the extraction itself: the extracted function must refuse the empty
# string and accept a realistic endpoint. An extraction that grabbed the wrong lines,
# or a predicate degraded to always-true / always-false, fails there — this repo has
# already been bitten by a check that passed when perturbed because it asked a question
# that was always satisfiable.
# string and accept a realistic endpoint.
# There is deliberately NO second implementation of the matching logic anywhere in this file.
#
# THE INDEPENDENT DETECTOR (the other half of the comparison)
# ----------------------------------------------------------
# Asking "does the guard agree with the guard?" proves nothing, so the set of
# placeholder-shaped values is decided by a detector written HERE, whose vocabulary is the
# UNION of two independent sources:
# THE INDEPENDENT DETECTOR (and the empty-string distinction the old version got wrong)
# ------------------------------------------------------------------------------------
# Asking "does the guard agree with the guard?" proves nothing, so placeholder-shaped is
# decided by a detector written HERE, whose vocabulary is the UNION of two independent
# sources:
# * SEED — universal loud-placeholder idioms (TBD, TODO, FIXME, PLACEHOLDER, REPLACE-ME,
# CHANGE-ME, FILL-ME, SET-ME, XXXX), plus `<angle-bracket>` templates and the empty
# string. These are NOT derived from the guard, so DELETING a guard arm cannot make a
# value stop being placeholder-shaped — it makes assertion (1) fail, which is the point.
# A value using an idiom the guard does not know is a finding either way: fix it by
# giving the knob a real value, or by teaching the guard the idiom.
# * the guard's OWN vocabulary, read out of its case-arms with the glob anchors STRIPPED.
# Stripping the anchors is what makes the comparison non-circular in the dimension that
# actually broke: the detector matches a word ANYWHERE, case-insensitively, and the
# guard must agree. A prefix-anchored guard cannot pass.
# CHANGE-ME, FILL-ME, SET-ME, XXXX), plus `<angle-bracket>` templates. NOT derived from
# any guard, so DELETING a guard arm cannot make a value stop being placeholder-shaped.
# * the guard's OWN vocabulary (when a guard exists), read out of its case-arms with the
# glob anchors STRIPPED. Stripping the anchors is what makes the comparison non-circular
# in the dimension that actually broke: the detector matches a word ANYWHERE,
# case-insensitively, and the guard must agree. A prefix-anchored guard cannot pass.
#
# LOUD vs EMPTY — the correction. A LOUD placeholder is a positive statement that a value
# was never decided, and it is always a defect in an overlay. The EMPTY STRING is not:
# cluster-vars authors `""` deliberately for suffix/prefix knobs, and Flux strict
# substitution fails on an UNDEFINED variable, not on one defined as `""`. So emptiness is
# NOT treated as a loud placeholder. It is held to a different, real rule instead (A2): an
# empty effective value must be STATED EXPLICITLY by that overlay, never inherited in
# silence — which is cluster-vars' own documented convention ("Both overlays state this
# knob explicitly … so neither inherits it silently"). Emptiness still counts as
# placeholder-shaped for a knob a GUARD covers, because a `""` case-arm in the guard is the
# guard author's declaration that empty is not a usable target for that knob.
#
# GIVEN the repository as authored,
# WHEN the real predicate(s) are extracted from bootstrap/standup.sh and executed,
# THEN (1) VALUES: for every knob a guard is responsible for, every authored value — the
# base default AND each environment overlay's value — that the independent
# detector calls placeholder-shaped is REFUSED by the real guard, and every value
# it calls usable is ACCEPTED (a guard that refuses a real endpoint leaves velero
# suspended forever, which is the same class of silent failure);
# (2) OVERLAYS: no environment overlay's EFFECTIVE cluster-vars carries a
# placeholder-shaped value for a knob that no guard refuses. (Base alone is
# exempt: it is not a bootstrap target — `flux bootstrap --path=clusters/<env>` —
# and its loud sentinels exist precisely to be overridden. A base placeholder an
# overlay forgets to override IS caught, because it becomes that overlay's
# effective value.)
# (3) CONSUMERS: every function in standup.sh that reads a guarded knob (via `cvar`,
# following one hop of variable aliasing) applies a placeholder guard to it. This
# is the "all three guards died at once" regression stated as a property: the
# guard being correct is worthless if a consumer stops calling it.
# (4) ANCHORING: for each word in the guard's own vocabulary, the guard refuses a
# value carrying that word in the MIDDLE (not at the start) — the precise defect
# of 2026-09-05 — while still accepting a realistic endpoint.
# (5) HARNESS: exactly one definition per predicate, the extracted body is verbatim
# from the file and self-contained, the canary holds, no divergent second copy of
# the matching logic exists elsewhere in standup.sh, and a floor on the number of
# assertions run keeps a pass from degenerating into a no-op.
# AND (6) LOAD-BEARING: ci/l0.sh runs the L0 conformance layer on every gate run and
# ci/gate.sh sets the merge block when that stage fails.
# THEN HALF A (always):
# (A1) VALUES: no environment overlay's EFFECTIVE cluster-vars value is LOUD
# placeholder-shaped, for any knob that no guard refuses. (Base alone is exempt:
# it is not a bootstrap target — `flux bootstrap --path=clusters/<env>` — and its
# loud sentinels exist to be overridden. A base placeholder an overlay forgets to
# override IS caught, because it becomes that overlay's effective value.)
# (A2) EXPLICITNESS: an empty effective value is stated explicitly by that overlay.
# (A3) SENTINEL DISCIPLINE: if base carries a LOUD sentinel for a knob, every overlay
# states that knob explicitly — the `TBD-PROD-*` convention only works if no
# overlay can inherit the sentinel by omission.
# (A4) DETECTOR CANARIES: the detector recognises the exact 2026-09-05 string
# MID-STRING, accepts both real endpoints the repo actually ships, and does NOT
# call the empty string loud.
# (A5) LOAD-BEARING: ci/l0.sh runs the L0 conformance layer on every gate run and
# ci/gate.sh sets the merge block when that stage fails.
# HALF B (only when a guard file is discovered):
# (B1) VALUES: for every knob a guard is responsible for, every authored value — the
# base default AND each overlay's value — that the detector calls placeholder-
# shaped is REFUSED by the real guard, and every value it calls usable is
# ACCEPTED (a guard that refuses a real endpoint leaves velero suspended forever,
# the same class of silent failure).
# (B2) CONSUMERS: every function that reads a guarded knob (via `cvar`, following one
# hop of variable aliasing) applies a placeholder guard to it. This is the "all
# three guards died at once" regression stated as a property.
# (B3) ANCHORING: for each word in the guard's own vocabulary, the guard refuses a
# value carrying that word in the MIDDLE (not at the start) — the precise defect
# of 2026-09-05 — while still accepting a realistic endpoint.
# (B4) HARNESS: exactly one definition per predicate, one guard FILE in the tree, the
# extracted body verbatim and self-contained, the canary holds, no divergent
# second copy of the matching logic, and assertion floors keep a pass from
# degenerating into a no-op.
#
# WHAT THIS DOES *NOT* PROVE (honest gaps)
# - While no guard exists on `main`, HALF B IS NOT PROVEN AT ALL, and the check says so on
# stdout every run. What is proven is the stronger-at-the-source half: the values a
# guard would be asked about are not placeholders in the first place. This is a real
# narrowing of coverage versus the intent, and it is reported, not hidden.
# - Nothing about RUNTIME: not that the endpoint resolves, not that credentials work, not
# that a backup exists. `standup.sh s3check` probes reachability from a pod (operator-
# run, cluster-gated) and conformance/tests/l3-fault-injection/dr-restore proves restore.
# - It does not execute standup.sh's dispatch path, so it cannot prove the guard is
# REACHED in a real run — only that, when reached, it refuses what it must. The
# consumer property (3) is a static approximation of reachability: it is a text-level
# taint trace over one file, not an interpreter, and a guard whose result is computed
# and then ignored would satisfy it.
# - Half B does not execute the dispatch path, so it cannot prove the guard is REACHED in
# a real run — only that, when reached, it refuses what it must. (B2) is a text-level
# taint trace over one file, not an interpreter.
# - It says nothing about guards keyed on shapes OTHER than placeholders (e.g. the
# EXTERNALDNS_TXT_OWNER_ID == 'default' test), nor about placeholder-shaped values
# outside cluster-vars (the Ansible inventory's BLOCKED/NOT-A-DEVICE sentinels are
# asserted by the node_baseline role itself, at play time).
# - Stage 3b of ci/l0.sh scopes the conformance run to the TOUCHED requirements. An edit
# to bootstrap/* attributes NO requirement in ci/touched-requirements.sh's path map, so
# the touched set is empty and the FULL L0 layer runs (verified: ci/lib.sh
# load_touched_requirements) — this check runs. An edit to clusters/* maps to
# R-ENV-02, which this check carries, so a cluster-vars value change also selects it.
# A change that touches standup.sh ALONGSIDE some unrelated mapped path (say
# platform/cilium/) would scope the run away from this check; closing that needs a
# `bootstrap/*` entry in the path map, which is not this check's file to edit.
# - A1 reads cluster-vars, not every manifest. A placeholder hardcoded in a manifest
# rather than in a knob is out of scope here.
#
# SELECTION. `ci/touched-requirements.sh` keys `clusters/*` -> R-ENV-02 (Half A's whole
# input surface) and `bootstrap/*` -> R-STATE-03 (where a guard lands), so every path that
# can break this check attributes a requirement it declares. Deliberately requirement-scoped,
# not whole-repo — see chainsaw-test.yaml.
# =============================================================================
set -euo pipefail
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT="$(cd "${DIR}/../../../.." && pwd)" # repo root
PYTHON="${PYTHON:-python3}"
STANDUP_REL="${STANDUP_REL:-bootstrap/standup.sh}"
GUARD_OVERRIDE="${PLACEHOLDER_GUARD_FILE:-}"
cd "${ROOT}"
if [ ! -f "${ROOT}/${STANDUP_REL}" ]; then
echo "check SKIPPED (placeholder-guard-coupling): ${STANDUP_REL} is absent — nothing to couple." >&2
exit 3
# The authored half of the coupling is the one thing this check cannot do without.
if [ ! -f "${ROOT}/clusters/base/cluster-vars.yaml" ]; then
echo "CHECK FAIL (placeholder-guard-coupling): clusters/base/cluster-vars.yaml is absent — the authored values this check audits do not exist. That is not a SKIP: the knobs this invariant is about have been moved or deleted, and this check must be re-pointed in the SAME change." >&2
exit 1
fi
WORK="$(mktemp -d)"
trap 'rm -rf "${WORK}"' EXIT
# ---------------------------------------------------------------------------------------
# PHASE 1 (python): extract the real predicate(s), derive the corpus from the real files,
# and run every assertion that does not need a shell. Writes:
# ${WORK}/guards.sh the predicate source lines, verbatim, and nothing else
# ${WORK}/corpus.tsv <guard>\t<expect: placeholder|usable>\t<origin>\t<value>
# PHASE 1 (python): discover the guard, derive the corpus from the real files, and run
# every assertion that does not need a shell. Writes:
# ${WORK}/guards.sh the predicate source lines, verbatim, and nothing else (may be empty)
# ${WORK}/corpus.tsv <guard>\t<expect: placeholder|usable>\t<origin>\t<value> (may be empty)
# ${WORK}/findings.txt one line per failed assertion
# ${WORK}/counts.txt <assertions-run>
# ${WORK}/mode.txt the discovered guard's repo-relative path, or empty
# ---------------------------------------------------------------------------------------
"${PYTHON}" - "${ROOT}" "${WORK}" "${STANDUP_REL}" <<'PY'
"${PYTHON}" - "${ROOT}" "${WORK}" "${GUARD_OVERRIDE}" <<'PY'
import re
import sys
from pathlib import Path
@@ -145,7 +213,7 @@ except ImportError: # pragma: no cover
file=sys.stderr)
raise SystemExit(1)
ROOT, WORK, STANDUP_REL = Path(sys.argv[1]), Path(sys.argv[2]), sys.argv[3]
ROOT, WORK, OVERRIDE = Path(sys.argv[1]), Path(sys.argv[2]), sys.argv[3]
FAILURES: list[str] = []
CHECKS = 0
@@ -164,116 +232,159 @@ def bail(msg):
(WORK / "counts.txt").write_text("0\n")
(WORK / "corpus.tsv").write_text("")
(WORK / "guards.sh").write_text("")
(WORK / "mode.txt").write_text("")
raise SystemExit(1)
TEXT = (ROOT / STANDUP_REL).read_text()
LINES = TEXT.splitlines()
# ------------------------------------------------------------------ (5) extract the guard
DEF_RE = re.compile(r"^([a-z0-9_]+_is_placeholder)\(\)\s*\{")
defs: dict[str, list[int]] = {}
for i, line in enumerate(LINES):
m = DEF_RE.match(line)
if m:
defs.setdefault(m.group(1), []).append(i)
SELF_DIR = ROOT / "conformance" / "tests" / "l0-static" / "placeholder-guard-coupling"
SKIP_DIRS = {".git", "_artifacts", "node_modules", ".venv"}
if not defs:
bail(f"{STANDUP_REL} defines no *_is_placeholder predicate. Either the guard was renamed "
"(update this check's discovery pattern in the SAME change) or it was DELETED, which "
"is the 2026-09-05 failure with the guard removed instead of disarmed.")
# ------------------------------------------------------------------ discover the guard FILE
# Hardcoding a path is what let a file that never landed disable this whole check.
if OVERRIDE:
cand = [ROOT / OVERRIDE]
if not cand[0].is_file():
bail(f"PLACEHOLDER_GUARD_FILE={OVERRIDE} does not exist")
else:
cand = []
for p in sorted(ROOT.rglob("*.sh")):
if SKIP_DIRS & set(p.parts) or SELF_DIR in p.parents:
continue
try:
text = p.read_text()
except (OSError, UnicodeDecodeError): # pragma: no cover
continue
if any(DEF_RE.match(line) for line in text.splitlines()):
cand.append(p)
# An "argument-shaped" call: <name> "<something>"
CALL_RE = re.compile(r"\b([a-z0-9_]+_is_placeholder)\s+\"([^\"]*)\"")
check(len(cand) <= 1,
"more than one file defines a *_is_placeholder predicate ("
+ ", ".join(str(p.relative_to(ROOT)) for p in cand)
+ ") — two files owning the placeholder vocabulary is the 2026-09-05 divergence with a "
"second file's name on it, and this check cannot say which one a standup run would "
"source. Keep exactly one, and source it from the other.")
GUARD_REL = str(cand[0].relative_to(ROOT)) if len(cand) == 1 else ""
(WORK / "mode.txt").write_text(GUARD_REL + "\n")
# --------------------------------------------------------- HALF B setup: extract the guard
guard_src: list[str] = []
guard_words: dict[str, list[str]] = {}
ARM_RE = re.compile(r"^(?P<pats>[^()]*?)\)\s*return\s+[01]\s*;;$")
ALLOWED = (
re.compile(r"^$"),
re.compile(r"^#"),
re.compile(r"^case\s+\"\$\{?[0-9A-Za-z_]+\}?\"\s+in$"),
re.compile(r"^esac$"),
re.compile(r"^return\s+[01]$"),
ARM_RE,
)
funcs: dict[str, list[str]] = {}
CALL_RE = re.compile(r"\b([a-z0-9_]+_is_placeholder)\s+\"([^\"]*)\"")
for name, at in sorted(defs.items()):
check(len(at) == 1,
f"{STANDUP_REL} defines {name}() {len(at)} times (lines "
f"{', '.join(str(a + 1) for a in at)}) — which one runs depends on file order, and "
"this check can no longer say which one it extracted. Keep exactly one definition.")
if len(at) != 1:
continue
start = at[0]
end = next((j for j in range(start + 1, len(LINES)) if LINES[j] == "}"), None)
if end is None:
bail(f"{name}() at {STANDUP_REL}:{start + 1} has no closing brace at column 0 — the "
"extraction boundary this check relies on is gone. Restore the conventional "
"layout, or move the predicate into a sourceable library and point this check at it.")
body = LINES[start:end + 1]
block = "\n".join(body)
check(block in TEXT,
f"extracted {name}() is not a verbatim substring of {STANDUP_REL} — extraction bug; "
"the harness would be testing something the script does not contain")
if GUARD_REL:
TEXT = (ROOT / GUARD_REL).read_text()
LINES = TEXT.splitlines()
words: list[str] = []
for raw in body[1:-1]:
s = raw.strip()
if not any(rx.match(s) for rx in ALLOWED) or "$(" in s or "`" in s:
bail(f"{name}() at {STANDUP_REL}:{start + 1} is no longer a SELF-CONTAINED "
f"case/return predicate (offending line: {s!r}). This check exercises the REAL "
"function by sourcing its own lines; a body that calls out to anything else "
"cannot be extracted faithfully, and silently testing a COPY of the logic is "
"exactly the failure mode this check exists to prevent. REMEDY: move the "
"predicate into a tiny library that standup.sh sources (e.g. "
"bootstrap/lib/placeholder.sh) and point STANDUP_REL/this extractor at it — do "
"NOT reimplement the matching here.")
m = ARM_RE.match(s)
if not m:
defs: dict[str, list[int]] = {}
for i, line in enumerate(LINES):
m = DEF_RE.match(line)
if m:
defs.setdefault(m.group(1), []).append(i)
ARM_RE = re.compile(r"^(?P<pats>[^()]*?)\)\s*return\s+[01]\s*;;$")
ALLOWED = (
re.compile(r"^$"),
re.compile(r"^#"),
re.compile(r"^case\s+\"\$\{?[0-9A-Za-z_]+\}?\"\s+in$"),
re.compile(r"^esac$"),
re.compile(r"^return\s+[01]$"),
ARM_RE,
)
for name, at in sorted(defs.items()):
check(len(at) == 1,
f"{GUARD_REL} defines {name}() {len(at)} times (lines "
f"{', '.join(str(a + 1) for a in at)}) — which one runs depends on file order, and "
"this check can no longer say which one it extracted. Keep exactly one definition.")
if len(at) != 1:
continue
for pat in m.group("pats").split("|"):
pat = pat.strip()
if pat in ('""', "''", ""):
continue # the empty-value arm: covered by the canary
words.append(pat.strip("'\"").strip("*"))
check(len(words) >= 1,
f"{name}() has no non-empty placeholder pattern left — every arm that recognises a "
"loud placeholder string has been removed, so the guard now only refuses the empty "
"value and any 'TBD'-style endpoint would sail straight through it")
guard_words[name] = words
guard_src.append(block)
start = at[0]
end = next((j for j in range(start + 1, len(LINES)) if LINES[j] == "}"), None)
if end is None:
bail(f"{name}() at {GUARD_REL}:{start + 1} has no closing brace at column 0 — the "
"extraction boundary this check relies on is gone. Restore the conventional "
"layout, or move the predicate into a sourceable library.")
body = LINES[start:end + 1]
block = "\n".join(body)
check(block in TEXT,
f"extracted {name}() is not a verbatim substring of {GUARD_REL} — extraction bug; "
"the harness would be testing something the script does not contain")
(WORK / "guards.sh").write_text("\n".join(guard_src) + "\n")
words: list[str] = []
for raw in body[1:-1]:
s = raw.strip()
if not any(rx.match(s) for rx in ALLOWED) or "$(" in s or "`" in s:
bail(f"{name}() at {GUARD_REL}:{start + 1} is no longer a SELF-CONTAINED "
f"case/return predicate (offending line: {s!r}). This check exercises the "
"REAL function by sourcing its own lines; a body that calls out to anything "
"else cannot be extracted faithfully, and silently testing a COPY of the "
"logic is exactly the failure mode this check exists to prevent. REMEDY: "
"move the predicate into a tiny library that the script sources (e.g. "
"bootstrap/lib/placeholder.sh) — discovery will find it — do NOT "
"reimplement the matching here.")
m = ARM_RE.match(s)
if not m:
continue
for pat in m.group("pats").split("|"):
pat = pat.strip()
if pat in ('""', "''", ""):
continue # the empty-value arm: covered by the canary
words.append(pat.strip("'\"").strip("*"))
check(len(words) >= 1,
f"{name}() has no non-empty placeholder pattern left — every arm that recognises a "
"loud placeholder string has been removed, so the guard now only refuses the empty "
"value and any 'TBD'-style endpoint would sail straight through it")
guard_words[name] = words
guard_src.append(block)
# ------------------------------------------------- no divergent second copy of the logic
# Any of the guard's own vocabulary appearing in EXECUTABLE text outside the predicate means
# a second, independently-drifting implementation of the same test (which is how three
# guards came to disagree with one value in the first place).
guard_line_nums = {j for name, at in defs.items() if len(at) == 1
for j in range(at[0], next(k for k in range(at[0] + 1, len(LINES))
if LINES[k] == "}") + 1)}
all_words = sorted({w.upper() for ws in guard_words.values() for w in ws if w})
divergent: list[str] = []
for j, line in enumerate(LINES):
if j in guard_line_nums or line.strip().startswith("#"):
continue
code = line.split("#", 1)[0]
hit = [w for w in all_words if w in code.upper()]
if hit:
divergent.append(f"{STANDUP_REL}:{j + 1} ({', '.join(hit)}): {line.strip()!r}")
check(not divergent,
"the guard's own placeholder vocabulary appears in EXECUTABLE text outside the "
f"predicate — a SECOND implementation of the same test, which is precisely how a value "
f"change disarmed three guards at once. Call the predicate instead. Sites: "
+ " | ".join(divergent))
if not guard_words:
bail(f"{GUARD_REL} was discovered as the guard file but yielded no usable predicate — "
"the definitions are there but none could be extracted. Fix the extraction in the "
"SAME change; do not let this degrade to a pass.")
# ------------------------------------------- no divergent second copy of the logic
# The guard's own vocabulary in EXECUTABLE text outside the predicate means a second,
# independently-drifting implementation of the same test (which is how three guards came
# to disagree with one value in the first place).
guard_line_nums = {j for name, at in defs.items() if len(at) == 1
for j in range(at[0], next(k for k in range(at[0] + 1, len(LINES))
if LINES[k] == "}") + 1)}
divergent: list[str] = []
_words = sorted({w.upper() for ws in guard_words.values() for w in ws if w})
for j, line in enumerate(LINES):
if j in guard_line_nums or line.strip().startswith("#"):
continue
code = line.split("#", 1)[0]
hit = [w for w in _words if w in code.upper()]
if hit:
divergent.append(f"{GUARD_REL}:{j + 1} ({', '.join(hit)}): {line.strip()!r}")
check(not divergent,
"the guard's own placeholder vocabulary appears in EXECUTABLE text outside the "
"predicate — a SECOND implementation of the same test, which is precisely how a value "
"change disarmed three guards at once. Call the predicate instead. Sites: "
+ " | ".join(divergent))
# function bodies, for the consumer trace
FUNC_RE = re.compile(r"^([a-z0-9_]+)\(\)\s*\{")
i = 0
while i < len(LINES):
m = FUNC_RE.match(LINES[i])
if m:
end = next((j for j in range(i + 1, len(LINES)) if LINES[j] == "}"), len(LINES) - 1)
funcs[m.group(1)] = LINES[i + 1:end]
i = end + 1
continue
i += 1
check(len(funcs) > 5, f"parsed only {len(funcs)} function(s) out of {GUARD_REL} — the "
"consumer assertion (B2) would be vacuous")
(WORK / "guards.sh").write_text(("\n".join(guard_src) + "\n") if guard_src else "")
# ------------------------------------------------------------- cluster-vars, as authored
CV = ROOT / "clusters" / "base" / "cluster-vars.yaml"
if not CV.is_file():
bail("clusters/base/cluster-vars.yaml is absent — the authored half of the coupling is "
"missing and this check cannot mean anything")
base: dict[str, str] = {}
for d in yaml.safe_load_all(CV.read_text()):
if isinstance(d, dict) and d.get("kind") == "ConfigMap" \
@@ -282,12 +393,16 @@ for d in yaml.safe_load_all(CV.read_text()):
check(len(base) > 5, f"clusters/base/cluster-vars.yaml yielded {len(base)} knob(s) — the "
"ConfigMap parse failed, so every value assertion below would be vacuous")
# Effective values per overlay, and WHICH knobs that overlay states explicitly. The second
# half is what lets emptiness be judged as authored-or-forgotten instead of guessed.
overlays: dict[str, dict[str, str]] = {}
stated: dict[str, set[str]] = {}
for overlay in sorted((ROOT / "clusters").glob("*/kustomization.yaml")):
env = overlay.parent.name
if env == "base":
continue
eff = dict(base)
explicit: set[str] = set()
try:
k = yaml.safe_load(overlay.read_text()) or {}
except Exception as exc: # noqa: BLE001
@@ -308,27 +423,15 @@ for overlay in sorted((ROOT / "clusters").glob("*/kustomization.yaml")):
m = re.match(r"^/data/([A-Za-z0-9_]+)$", str(op.get("path", "")))
if m and str(op.get("op", "")) in ("replace", "add"):
eff[m.group(1)] = "" if op.get("value") is None else str(op["value"])
explicit.add(m.group(1))
overlays[env] = eff
stated[env] = explicit
check(len(overlays) >= 1,
"no clusters/<env>/kustomization.yaml overlay was parsed — assertion (2) would be vacuous")
# ------------------------------------------------ which knobs is a guard responsible for?
# Function bodies of standup.sh, then a one-hop taint trace from `x="$(cvar KEY)"` to a
# `*_is_placeholder "${x}"` call. Both halves are read out of the file; nothing is named here.
FUNC_RE = re.compile(r"^([a-z0-9_]+)\(\)\s*\{")
funcs: dict[str, list[str]] = {}
i = 0
while i < len(LINES):
m = FUNC_RE.match(LINES[i])
if m:
end = next((j for j in range(i + 1, len(LINES)) if LINES[j] == "}"), len(LINES) - 1)
funcs[m.group(1)] = LINES[i + 1:end]
i = end + 1
continue
i += 1
check(len(funcs) > 5, f"parsed only {len(funcs)} function(s) out of {STANDUP_REL} — the "
"consumer assertion (3) would be vacuous")
"no clusters/<env>/kustomization.yaml overlay was parsed — assertion (A1) would be vacuous")
# ------------------------------------------ which knobs is a guard responsible for? (HALF B)
# A one-hop taint trace from `x="$(cvar KEY)"` to a `*_is_placeholder "${x}"` call. Both
# halves are read out of the file; nothing is named here.
CVAR_RE = re.compile(r"([A-Za-z_][A-Za-z0-9_]*)=\"\$\(\s*cvar\s+([A-Z0-9_]+)\s*\)\"")
ALIAS_RE = re.compile(r"([A-Za-z_][A-Za-z0-9_]*)=\"\$\{([A-Za-z_][A-Za-z0-9_]*)[:}]")
@@ -347,8 +450,6 @@ for fname, body in funcs.items():
keys_of_var.setdefault(m.group(1), set()).update(src)
called_on = {m.group(2) for m in CALL_RE.finditer(text)}
called_vars = {re.sub(r"^\$\{?|\}?$", "", a) for a in called_on}
# Aggregate per (function, knob): the knob is guarded HERE if ANY variable carrying its
# value is passed to a predicate — `live="$(cvar K)"; ep="${live}"; guard "${ep}"` counts.
guarded_here: dict[str, bool] = {}
for var, keys in keys_of_var.items():
for key in keys:
@@ -358,43 +459,119 @@ for fname, body in funcs.items():
if g:
guarded_keys.add(key)
if not guarded_keys:
bail(f"no cluster-vars knob in {STANDUP_REL} is passed to a *_is_placeholder predicate. "
if GUARD_REL and not guarded_keys:
bail(f"no cluster-vars knob in {GUARD_REL} is passed to a *_is_placeholder predicate. "
"Either the guards were removed, or the way the script reads cluster-vars changed and "
"this check's derivation must be updated in the SAME change — it must never silently "
"verify an empty set.")
# (3) CONSUMERS: every reader of a guarded knob must apply the guard to it.
# (B2) CONSUMERS: every reader of a guarded knob must apply the guard to it.
for fname, key, guarded in sorted(consumers):
if key not in guarded_keys:
continue # nothing claims responsibility for this knob at all
check(guarded,
f"{STANDUP_REL}: {fname}() reads cluster-vars knob {key} but never passes it to a "
f"{GUARD_REL}: {fname}() reads cluster-vars knob {key} but never passes it to a "
"*_is_placeholder guard, while other functions do. On 2026-09-05 all three consumers "
"of S3_BACKUP_ENDPOINT were disarmed at once; a consumer that reads the knob and skips "
"the guard is that failure in one function.")
# ---------------------------------------------------------------- independent detector
# Deliberately NOT derived from the guard alone (see the header): SEED is universal
# loud-placeholder vocabulary, and the guard's own words are added so that a guard whose
# ANCHORING is wrong still has its vocabulary tested against real values.
# ------------------------------------------------------------------ independent detector
# Deliberately NOT derived from the guard alone: SEED is universal loud-placeholder
# vocabulary, and the guard's own words are added so that a guard whose ANCHORING is wrong
# still has its vocabulary tested against real values.
SEED = ["TBD", "TODO", "FIXME", "PLACEHOLDER", "REPLACE-ME", "REPLACEME",
"CHANGE-ME", "CHANGEME", "FILL-ME", "FILLME", "SET-ME", "XXXX"]
all_words = sorted({w.upper() for ws in guard_words.values() for w in ws if w})
VOCAB = sorted({w.upper() for w in SEED} | set(all_words))
ANGLE = re.compile(r"<[^>\s]+>")
def placeholder_shaped(value: str) -> bool:
if value.strip() == "":
return True # an unset endpoint is not a target
def loud(value: str) -> bool:
"""A positive statement that the value was never decided. The EMPTY STRING IS NOT LOUD —
see the header: cluster-vars authors "" deliberately, and treating it as a placeholder is
the false positive that made the original check red against its own input."""
up = value.upper()
return any(w in up for w in VOCAB) or bool(ANGLE.search(value))
rows: list[tuple[str, str, str, str]] = [] # guard, expect, origin, value
GUARD = sorted(guard_words)[0] if len(guard_words) == 1 else None
def guard_shaped(value: str) -> bool:
"""What a GUARD must refuse: loud, or empty. Empty counts here only because a `""`
case-arm is the guard author's declaration that empty is not a usable target."""
return value.strip() == "" or loud(value)
# (1) VALUES: base default and every overlay's value, for every guarded knob.
# ------------------------------------------------------------------- (A4) DETECTOR CANARIES
# The detector is now the thing doing the refusing in HALF A, so it gets the same
# no-op interrogation the guard does. These strings are the real ones from the incident.
check(loud("https://TBD-R2-ACCOUNT-ID.r2.cloudflarestorage.com"),
"the detector does not recognise the exact 2026-09-05 placeholder "
"'https://TBD-R2-ACCOUNT-ID.r2.cloudflarestorage.com' — it is prefix-anchored, which is "
"the defect this check exists to catch, now in the check itself")
check(loud("TBD-PROD-BULK-NFS-SERVER"),
"the detector does not recognise the repo's own TBD-PROD-* sentinel convention "
"(clusters/README.md) — assertion (A1) would pass over the very values it is about")
check(loud("<your-account-id>"),
"the detector does not recognise an <angle-bracket> template as a placeholder")
check(not loud("https://0123456789abcdef0123456789abcd.r2.cloudflarestorage.com"),
"the detector calls a REAL Cloudflare R2 endpoint a placeholder — it would false-block "
"every prod backup target")
check(not loud("http://192.168.1.9:9000"),
"the detector calls a REAL MinIO endpoint a placeholder — it would false-block dev")
check(not loud(""),
"the detector calls the EMPTY STRING loud. cluster-vars authors \"\" deliberately for "
"suffix/prefix knobs and documents it as load-bearing; emptiness is held to the "
"explicitness rule (A2) instead. This exact false positive made the original check FAIL "
"2 of 34 assertions against the real guard.")
# --------------------------------- (A1) no overlay ships a LOUD value nothing refuses
a1_rows = 0
for env, eff in sorted(overlays.items()):
for key, value in sorted(eff.items()):
if key in guarded_keys:
continue # a guard owns this knob: HALF B (B1) asserts it refuses it
a1_rows += 1
check(not loud(value),
f"clusters/{env}: knob {key} resolves to the placeholder-shaped value {value!r}, "
"and nothing refuses it"
+ (f" — no guard in {GUARD_REL} covers this knob." if GUARD_REL
else " — and no placeholder guard exists in the tree at all.")
+ " Either give it a real value before it reaches an overlay, or guard it the way "
"the S3 endpoint is guarded. An overlay that ships a placeholder no guard "
"recognises is the 2026-09-05 failure with a different knob's name on it: it "
"applies clean, reports Ready, and the thing it configures does not exist.")
# ------------------------- (A2) an empty effective value must be STATED, never inherited
a2_rows = 0
for env, eff in sorted(overlays.items()):
for key, value in sorted(eff.items()):
if value.strip() != "":
continue
a2_rows += 1
check(key in stated[env],
f"clusters/{env}: knob {key} resolves to the EMPTY STRING by INHERITANCE from "
"clusters/base — the overlay never states it. An empty value can be correct (dev's "
"BACKUP_CREDENTIALS_SUFFIX and VELERO_BACKUP_PREFIX are authored empty on purpose), "
"but only when the overlay SAYS SO: cluster-vars' own convention is that both "
"overlays state such a knob explicitly so neither inherits it silently. An empty "
"value nobody stated is indistinguishable from a knob somebody forgot.")
# ---------------- (A3) a LOUD base sentinel must be overridden EXPLICITLY by every overlay
a3_rows = 0
for key, value in sorted(base.items()):
if not loud(value):
continue
for env in sorted(overlays):
a3_rows += 1
check(key in stated[env],
f"clusters/base/cluster-vars.yaml authors the LOUD sentinel {key}={value!r}, and "
f"clusters/{env} does not state that knob — so that overlay INHERITS the sentinel. "
"The TBD-PROD-* convention only works if a sentinel cannot be inherited by "
"omission; an omitted op is how prod silently ships a value nobody decided.")
# --------------------------------------- HALF B corpus: authored values + anchoring + canaries
rows: list[tuple[str, str, str, str]] = [] # guard, expect, origin, value
# (B1) VALUES: base default and every overlay's value, for every guarded knob.
for key in sorted(guarded_keys):
seen: list[tuple[str, str]] = []
if key in base:
@@ -404,24 +581,12 @@ for key in sorted(guarded_keys):
check(bool(seen), f"guarded knob {key} is not defined in cluster-vars at all — the guard "
"protects a knob nothing authors")
for origin, value in seen:
expect = "placeholder" if placeholder_shaped(value) else "usable"
expect = "placeholder" if guard_shaped(value) else "usable"
for name in sorted(guard_words):
rows.append((name, expect, origin, value))
# (2) OVERLAYS: a placeholder-shaped effective value for an UNGUARDED knob.
for env, eff in sorted(overlays.items()):
for key, value in sorted(eff.items()):
if key in guarded_keys or not placeholder_shaped(value):
continue
check(False,
f"clusters/{env}: knob {key} resolves to the placeholder-shaped value {value!r}, "
f"and NO guard in {STANDUP_REL} refuses it. Either give it a real value before it "
"reaches an overlay, or guard it the way the S3 endpoint is guarded — an overlay "
"that ships a placeholder no guard recognises is the 2026-09-05 failure with a "
"different knob's name on it.")
# (4) ANCHORING: the guard's own vocabulary must match ANYWHERE, not just as a prefix,
# and a realistic endpoint must still be accepted.
# (B3) ANCHORING: the guard's own vocabulary must match ANYWHERE, not just as a prefix,
# and a realistic endpoint must still be accepted.
for name, words in sorted(guard_words.items()):
for w in sorted(set(words)):
rows.append((name, "placeholder",
@@ -436,15 +601,16 @@ for name, words in sorted(guard_words.items()):
# Every guarded knob must reach the oracle once per authored source (base + each overlay),
# so a knob silently dropping out of the corpus is a failure rather than a quiet pass.
authored = [r for r in rows if not r[2].startswith(("anchoring:", "canary:"))]
expected_rows = len(guarded_keys) * (1 + len(overlays)) * len(guard_words)
check(len(authored) == expected_rows,
f"{len(authored)} authored value(s) reached the guard, expected {expected_rows} "
f"({len(guarded_keys)} guarded knob(s) x base+{len(overlays)} overlay(s) x "
f"{len(guard_words)} predicate(s)) — a guarded knob is missing from cluster-vars or from "
"an overlay's effective values, and the coupling is only partly exercised")
if GUARD_REL:
authored = [r for r in rows if not r[2].startswith(("anchoring:", "canary:"))]
expected_rows = len(guarded_keys) * (1 + len(overlays)) * len(guard_words)
check(len(authored) == expected_rows,
f"{len(authored)} authored value(s) reached the guard, expected {expected_rows} "
f"({len(guarded_keys)} guarded knob(s) x base+{len(overlays)} overlay(s) x "
f"{len(guard_words)} predicate(s)) — a guarded knob is missing from cluster-vars or "
"from an overlay's effective values, and the coupling is only partly exercised")
# ------------------------------------------------------- (6) the check is load-bearing
# --------------------------------------------------- (A5) the check is load-bearing
L0 = (ROOT / "ci" / "l0.sh").read_text() if (ROOT / "ci" / "l0.sh").is_file() else ""
GATE = (ROOT / "ci" / "gate.sh").read_text() if (ROOT / "ci" / "gate.sh").is_file() else ""
check("--layer l0" in L0,
@@ -452,6 +618,14 @@ check("--layer l0" in L0,
check(bool(re.search(r"^for s in .*\bl0\b.*; do", GATE, re.M)) and bool(re.search(r"fail\).*BLOCK=1", GATE)),
"ci/gate.sh does not set the merge block when the l0 stage fails")
# ------------------------------------------------------- non-vacuity floor for HALF A
# HALF A is the half that runs unconditionally, so it carries its own floor: if the
# authored-value sweep shrinks to nothing, a pass here means nothing.
check(a1_rows >= 20,
f"only {a1_rows} authored overlay value(s) were audited by (A1) — the sweep over "
"clusters/<env> effective cluster-vars has collapsed, and a pass would be meaningless. "
"Expected roughly (knobs x overlays); check the ConfigMap and overlay parse above.")
# --------------------------------------------------------------------------- handoff
with (WORK / "corpus.tsv").open("w") as fh:
for name, expect, origin, value in rows:
@@ -459,17 +633,31 @@ with (WORK / "corpus.tsv").open("w") as fh:
(WORK / "findings.txt").write_text("".join(f"{f}\n" for f in FAILURES))
(WORK / "counts.txt").write_text(f"{CHECKS}\n")
print(f"guards: {', '.join(sorted(guard_words))} "
f"vocabulary: {', '.join(VOCAB)}")
print(f"guarded knobs: {', '.join(sorted(guarded_keys))} "
f"consumers traced: {len(consumers)} overlays: {', '.join(sorted(overlays))}")
if GUARD_REL:
print(f"guard file: {GUARD_REL} predicates: {', '.join(sorted(guard_words))}")
print(f"guarded knobs: {', '.join(sorted(guarded_keys))} "
f"consumers traced: {len(consumers)}")
else:
print("guard file: NONE FOUND in the tree — HALF B (guard/value coupling) IS NOT PROVEN "
"ON THIS TREE. No *_is_placeholder predicate exists to extract: bootstrap/standup.sh "
"has never been on main (no deletion commit exists; it lives on unmerged prod-standup "
"branches). HALF A below runs against the real authored values and is what currently "
"prevents the 2026-09-05 outcome. When a guard lands ANYWHERE in the tree, discovery "
"picks it up and HALF B activates with no edit to this check.")
print(f"detector vocabulary: {', '.join(VOCAB)}")
print(f"HALF A: overlays {', '.join(sorted(overlays))}; {len(base)} base knob(s); "
f"(A1) {a1_rows} authored overlay value(s) screened; (A2) {a2_rows} empty value(s) "
f"checked for explicit statement; (A3) {a3_rows} base-sentinel inheritance check(s)")
for name, expect, origin, value in rows:
if not origin.startswith(("anchoring:", "canary:")):
print(f" authored {origin} = {value!r} -> expect {expect}")
PY
GUARD_REL="$(tr -d '\n' < "${WORK}/mode.txt")"
# ---------------------------------------------------------------------------------------
# PHASE 2 (bash): source the REAL predicate lines and run the corpus through them.
# Empty when no guard exists — HALF A has already run and carries its own floor.
# ---------------------------------------------------------------------------------------
# shellcheck disable=SC1090,SC1091
source "${WORK}/guards.sh"
@@ -483,9 +671,9 @@ while IFS=$'\t' read -r guard expect origin value; do
if [ "${verdict}" != "${expect}" ]; then
oracle_failures=$((oracle_failures + 1))
if [ "${expect}" = placeholder ]; then
printf '%s\n' "${guard}() in ${STANDUP_REL} ACCEPTS '${value}' (from ${origin}) as a usable value, but it is placeholder-shaped. This is the 2026-09-05 failure exactly: the guard is asked about a value it does not recognise, returns 'fine', and the tier it protects is enabled against something that does not exist. Fix the predicate (match the idiom anywhere, not as a prefix) or stop authoring that idiom." >> "${WORK}/findings.txt"
printf '%s\n' "${guard}() in ${GUARD_REL} ACCEPTS '${value}' (from ${origin}) as a usable value, but it is placeholder-shaped. This is the 2026-09-05 failure exactly: the guard is asked about a value it does not recognise, returns 'fine', and the tier it protects is enabled against something that does not exist. Fix the predicate (match the idiom anywhere, not as a prefix) or stop authoring that idiom." >> "${WORK}/findings.txt"
else
printf '%s\n' "${guard}() in ${STANDUP_REL} REFUSES '${value}' (from ${origin}), which is a usable value. An over-matching guard leaves the tier it protects suspended forever and reports it as an operator decision — a silent failure in the other direction." >> "${WORK}/findings.txt"
printf '%s\n' "${guard}() in ${GUARD_REL} REFUSES '${value}' (from ${origin}), which is a usable value. An over-matching guard leaves the tier it protects suspended forever and reports it as an operator decision — a silent failure in the other direction." >> "${WORK}/findings.txt"
fi
fi
done < "${WORK}/corpus.tsv"
@@ -495,11 +683,13 @@ py_checks="$(cat "${WORK}/counts.txt")"
total=$((py_checks + oracle_rows))
findings="$(cat "${WORK}/findings.txt")"
# NO-OP GUARD: this repo has been bitten by a check that passed when perturbed because it
# NO-OP GUARDS: this repo has been bitten by a check that passed when perturbed because it
# asked a question that was always satisfiable. The floors below are the minimum evidence
# for the verdict to mean anything, not a style rule.
if [ "${oracle_rows}" -lt 6 ]; then
findings="${findings}only ${oracle_rows} value(s) were put to the real guard — the coupling is not actually being exercised
# for the verdict to mean anything, not a style rule. The oracle floor applies only when a
# guard exists — when none does, HALF A's own floor (a1_rows >= 20, asserted in phase 1) is
# what stops this from degenerating, and the absence is reported on stdout.
if [ -n "${GUARD_REL}" ] && [ "${oracle_rows}" -lt 6 ]; then
findings="${findings}only ${oracle_rows} value(s) were put to the real guard in ${GUARD_REL} — the coupling is not actually being exercised
"
fi
if [ "${total}" -lt 20 ]; then
@@ -513,4 +703,8 @@ if [ -n "${findings}" ]; then
exit 1
fi
echo "check OK: placeholder-guard-coupling — ${total} assertions; ${oracle_rows} value(s) run through the REAL extracted predicate(s) from ${STANDUP_REL} (no reimplementation), covering every authored cluster-vars value of every guarded knob in base and each overlay, the guard's own vocabulary matched mid-string, and the empty/real-endpoint canaries. Static slice only: refusal, not reachability — see the header."
if [ -n "${GUARD_REL}" ]; then
echo "check OK: placeholder-guard-coupling — ${total} assertions. HALF A: no environment overlay ships a placeholder-shaped effective cluster-vars value, every empty value is stated explicitly rather than inherited, and the detector canaries hold (including the exact 2026-09-05 string, matched mid-string). HALF B: ${oracle_rows} value(s) run through the REAL extracted predicate(s) from ${GUARD_REL} (no reimplementation), covering every authored value of every guarded knob in base and each overlay, the guard's vocabulary matched mid-string, and the empty/real-endpoint canaries. Static slice only: refusal, not reachability — see the header."
else
echo "check OK: placeholder-guard-coupling — ${total} assertions. HALF A ONLY: no environment overlay ships a placeholder-shaped effective cluster-vars value, every empty value is stated explicitly rather than inherited, no LOUD base sentinel is inherited by omission, and the detector canaries hold (including the exact 2026-09-05 string 'https://TBD-R2-ACCOUNT-ID.r2.cloudflarestorage.com', matched mid-string). HALF B (guard/value coupling) IS NOT PROVEN: no *_is_placeholder predicate exists anywhere in the tree to extract — see the stdout note above and the header. This is a REPORTED gap, not a skip: what a guard would be asked about is proven not to be a placeholder in the first place."
fi
docs/BULK-NFS-STORAGE.md
+18 -8
View File
@@ -252,18 +252,28 @@ so **plex, sonarr, radarr and nextcloud would sit in ContainerCreating on prod.*
revision that carries these bulk volumes.** Dev renders a working value and is where these manifests
are meant to be proven first.
### The static guard this convention leans on is missing from `main`
### The static guard this convention leans on is missing from `main` — but the gate now refuses the value anyway
`bootstrap/standup.sh` — which carries the `*_is_placeholder` guard family — **does not exist on
`main`** (verified: `git ls-tree main bootstrap/` lists only `README.md`, `flux/`, `gitea/`, `sops/`;
the file lives on unmerged prod branches). So
`conformance/tests/l0-static/placeholder-guard-coupling/check.sh` exits **3 SKIPPED**, and **nothing
in the gate currently refuses a placeholder-shaped overlay value.**
the file lives on unmerged prod branches, and there is no deletion commit for it anywhere — it never
landed rather than having been removed).
When `standup.sh` lands, that check's assertions (2) and (3) cover this knob automatically — its
independent detector's SEED vocabulary already matches `TBD` — **provided** the guard gains an arm
for `BULK_NFS_SERVER` and a consumer that calls it. Without the arm, assertion (2) is what fails, by
design.
**UPDATED 2026-09-13.** This section used to end "so
`conformance/tests/l0-static/placeholder-guard-coupling/check.sh` exits **3 SKIPPED**, and nothing in
the gate currently refuses a placeholder-shaped overlay value." That is **no longer true.** That check
was restructured (see DECISIONS-LOG, branch `fix/placeholder-guard-coupling-unskipped`): it never
exits 3, and its HALF A screens **every environment overlay's effective cluster-vars value** against
an independent placeholder detector with no guard required. A `BULK_NFS_SERVER` of
`TBD-PROD-BULK-NFS-SERVER` in `clusters/prod/` is now a **merge-gate FAIL** — verified by deliberately
authoring exactly that value and watching the check fail, which is the one hazard on this page that
CRD validation cannot catch. HALF A also refuses a LOUD base sentinel that an overlay inherits by
*omitting* its patch op, which is the other way this knob can silently ship wrong.
The guard is now **discovered by content**, not by path, so when `standup.sh` (or its predicate in a
sourceable library) lands, the coupling half activates with no edit to the test — and it then covers
this knob **provided** the guard gains an arm for `BULK_NFS_SERVER` and a consumer that calls it.
Without the arm, HALF A is what holds the line, by design.
---