fix(conformance): placeholder-guard-coupling actually RUNS — it was skipping forever
`conformance/tests/l0-static/placeholder-guard-coupling` (R-ENV-02, R-STATE-03) exited 3 SKIPPED on every gate run since it landed, because its check keyed its whole existence on `bootstrap/standup.sh`. Measured, not assumed: that file has NEVER been on `main` — there is no deletion commit for it anywhere; it landed only on unmerged prod-standup branches (`work/prod-standup-campaign`, `prod/*`, `feat/prod-hardening`). It was planned and never merged, not removed. SKIPPED is not green, and a permanently-skipped incident-derived test is the quiet version of the failure it was written to prevent. The skip was also hiding a RED. Run against the real `standup.sh` from `work/prod-standup-campaign`, the old check FAILED 2 of 34 assertions — false positives from treating the empty string as placeholder-shaped for knobs cluster-vars authors empty on purpose (`BACKUP_CREDENTIALS_SUFFIX`, `VELERO_BACKUP_PREFIX`, both `""` on dev and both documented there as load-bearing). It would have false-blocked the merge landing its own input. NOT deleted — the hazard is not gone. Split into two halves, one of which needs no guard: HALF A (always runs): no environment overlay ships a placeholder-shaped EFFECTIVE cluster-vars value; an empty effective value is STATED by the overlay rather than inherited in silence; a LOUD base sentinel is explicitly overridden by every overlay. This is the 2026-09-05 outcome asserted at the source instead of at the guard — if prod's S3_BACKUP_ENDPOINT is never a placeholder, velero cannot be pointed at a hostname that does not exist. Input surface is `clusters/`, all of it on `main`. HALF B (self-activating): the real predicate extracted and executed against those same values, the consumer trace, and the mid-string anchoring property. The check NEVER exits 3 again, and the guard is DISCOVERED by content rather than hardcoded: any tracked `*.sh` defining `<name>_is_placeholder()` is found, so a guard landing at `bootstrap/standup.sh` — or moved into a sourceable library, the remedy the check already recommends — activates HALF B with no edit to the test. Two files defining it is a FAIL. When no guard exists the gap is printed loudly every run instead of swallowed as a skip. LOUD vs EMPTY is the correction that makes it honest in both modes: a loud placeholder is always a defect in an overlay, while `""` is held to the explicitness rule instead — and still counts as a placeholder for a knob a guard covers, because a `""` case-arm is the guard author's declaration that empty is not a usable target. Proof, all reverted: 84 assertions and exit 0 on `main`; prod S3_BACKUP_ENDPOINT set to the literal `https://TBD-R2-ACCOUNT-ID.r2.cloudflarestorage.com` FAILS (A1); deleting dev's explicit VELERO_BACKUP_PREFIX op so it inherits `""` FAILS (A2); a LOUD base BULK_NFS_SERVER with prod's override deleted FAILS 2/86 (A1+A3); and with the real `standup.sh` present, reverting `s3_is_placeholder` to the `TBD-*` PREFIX glob FAILS 12/110 (B3 anchoring). With the pristine `standup.sh` the check PASSES at 110 assertions where the old one failed 2 of 34. L0 layer: 14 checks, 14 PASS, 0 SKIP, 0 FAIL (was 13 PASS / 1 SKIP). Coverage unchanged — R-ENV-02 and R-STATE-03 both still list this test, 49/51 with the same 2 gaps. Also: * `ci/touched-requirements.sh` — `bootstrap/*` -> R-STATE-03 R-GITOPS-01 is KEPT, its stale comment corrected. It claimed standup.sh "carries the placeholder guards", describing a file that is not here; R-STATE-03 there is ANTICIPATORY. Deleting the entry would leave the change that lands a placeholder guard selected by nothing. * `ci/tests/touched-requirements.test.sh` — new path-map section (12 assertions, was 8) locking `bootstrap/*` and `clusters/*`, so removing either entry is a test failure rather than a silent loss of selection. All three ci/tests pass (12 + 31 + 15). * `docs/BULK-NFS-STORAGE.md` §7 — its "nothing in the gate currently refuses a placeholder-shaped overlay value" is no longer true; corrected. * Scope re-audited: still deliberately requirement-scoped, not whole-repo. Every path that can break it attributes a requirement it declares. Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
1 parent
55b90849d1
commit
87c3d83dff
6 files changed
+716
-299
No files matched your search
@@ -1,43 +1,71 @@
|
||||
# =============================================================================
|
||||
# L0 STATIC — a placeholder value in cluster-vars is REFUSED by the guard that owns it
|
||||
# L0 STATIC — no environment overlay ships a placeholder value, and the guard that owns
|
||||
# one refuses it
|
||||
# Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster.
|
||||
#
|
||||
# THE BUG CLASS THIS CLOSES (it has now bitten three times in one file). A guard in
|
||||
# `bootstrap/standup.sh` keys on the SHAPE of a value that lives in
|
||||
# `clusters/base/cluster-vars.yaml`, and nothing tied the two together. On 2026-09-05 the
|
||||
# S3 placeholder changed from `TBD-PROD-S3-BACKUP-ENDPOINT` to
|
||||
# `https://TBD-R2-ACCOUNT-ID.r2.cloudflarestorage.com`; the three guards matched the glob
|
||||
# `TBD-*`, a PREFIX match, and `fnmatch('https://TBD-R2-…','TBD-*')` is False. All three
|
||||
# guards silently stopped matching, and `standup.sh resume` would have resumed `velero` on
|
||||
# prod against a nonexistent hostname: installs clean, reports healthy, NO BACKUPS EXIST.
|
||||
# It was found by accident. The guard is fixed (`s3_is_placeholder`, substring); the
|
||||
# COUPLING was untested, which is what makes the class reintroducible.
|
||||
# `https://TBD-R2-ACCOUNT-ID.r2.cloudflarestorage.com` (commit 32d6766); the three guards
|
||||
# matched the glob `TBD-*`, a PREFIX match, and `fnmatch('https://TBD-R2-…','TBD-*')` is
|
||||
# False. All three guards silently stopped matching, and `standup.sh resume` would have
|
||||
# resumed `velero` on prod against a nonexistent hostname: installs clean, reports healthy,
|
||||
# NO BACKUPS EXIST. It was found by accident.
|
||||
#
|
||||
# WHY THIS TEST WAS RESTRUCTURED (2026-09-13). It was PERMANENTLY SKIPPED. `check.sh` exited
|
||||
# 3 when `bootstrap/standup.sh` was absent — and that file has NEVER been on `main`: it was
|
||||
# not deleted (no deletion commit exists anywhere), it landed only on unmerged prod-standup
|
||||
# branches. So an incident-derived test reported SKIP on every gate run, and SKIPPED IS NOT
|
||||
# GREEN here. The skip was also hiding a RED: run against the real `standup.sh` from
|
||||
# `work/prod-standup-campaign`, the old check FAILED 2 of 34 assertions on false positives
|
||||
# from treating the empty string as a placeholder for knobs cluster-vars authors empty on
|
||||
# purpose. Both are fixed. The invariant is now split into a half that needs no guard and a
|
||||
# half that activates when one lands, and THIS CHECK NEVER EXITS 3.
|
||||
#
|
||||
# GIVEN the repository as authored,
|
||||
# WHEN the REAL predicate is extracted from bootstrap/standup.sh and executed by bash
|
||||
# (never reimplemented — see check.sh "HOW THE REAL GUARD IS EXERCISED"),
|
||||
# THEN (1) every authored value of a guarded knob that is placeholder-shaped by an
|
||||
# INDEPENDENT detector is actually refused by that guard, in base and in every
|
||||
# environment overlay;
|
||||
# (2) no environment overlay carries a placeholder-shaped value for a knob no guard
|
||||
# refuses;
|
||||
# (3) every function in standup.sh that reads a guarded knob applies the guard to it
|
||||
# (the "three guards died at once" regression);
|
||||
# (4) the guard matches its own vocabulary ANYWHERE in a value, not just as a prefix
|
||||
# — the exact anchoring defect above — and still accepts a real endpoint;
|
||||
# AND (5) the extraction is faithful, and the check fails LOUDLY rather than silently
|
||||
# testing a copy if the guard stops being a self-contained predicate.
|
||||
# THEN HALF A — always, needs no guard, input surface `clusters/`:
|
||||
# (A1) no environment overlay's EFFECTIVE cluster-vars value is placeholder-shaped by
|
||||
# an INDEPENDENT detector, for any knob no guard refuses (base is exempt: it is
|
||||
# not a bootstrap target and its loud sentinels exist to be overridden — but a
|
||||
# sentinel an overlay forgets to override IS caught, as that overlay's value);
|
||||
# (A2) an empty effective value is STATED explicitly by the overlay, never inherited
|
||||
# in silence — cluster-vars' own documented convention, and the rule that
|
||||
# separates "authored empty" from "somebody forgot";
|
||||
# (A3) a LOUD base sentinel is explicitly overridden by every overlay, so the
|
||||
# `TBD-PROD-*` convention cannot be defeated by an omitted patch op;
|
||||
# (A4) the detector itself recognises the exact 2026-09-05 string MID-STRING, accepts
|
||||
# both real endpoints the repo ships, and does not call the empty string loud;
|
||||
# (A5) ci/l0.sh runs the L0 layer and ci/gate.sh blocks the merge when it fails.
|
||||
# HALF B — when a `*_is_placeholder` predicate exists ANYWHERE in the tree (the guard
|
||||
# is DISCOVERED, not hardcoded, so a guard landing at bootstrap/standup.sh or moving to
|
||||
# a sourceable library activates this half with no edit here):
|
||||
# (B1) every authored value of a guarded knob the detector calls placeholder-shaped is
|
||||
# REFUSED by the REAL predicate, extracted from the real file and executed (never
|
||||
# reimplemented — see check.sh "HOW THE REAL GUARD IS EXERCISED"), and every
|
||||
# usable value is ACCEPTED;
|
||||
# (B2) every function that reads a guarded knob applies the guard to it (the "three
|
||||
# guards died at once" regression);
|
||||
# (B3) the guard matches its own vocabulary ANYWHERE in a value, not just as a prefix
|
||||
# — the exact anchoring defect above — and still accepts a real endpoint;
|
||||
# (B4) the extraction is faithful, exactly one file and one definition own the
|
||||
# vocabulary, and the check fails LOUDLY rather than silently testing a copy if
|
||||
# the guard stops being a self-contained predicate.
|
||||
#
|
||||
# STATUS: EXECUTABLE. `check.sh` is a pure offline assertion script (standard library +
|
||||
# PyYAML + bash); conformance/run.sh --layer l0 runs it directly and this Chainsaw Test
|
||||
# wraps the SAME script.
|
||||
# STATUS: EXECUTABLE, and it RUNS — never SKIP. `check.sh` is a pure offline assertion
|
||||
# script (standard library + PyYAML + bash); conformance/run.sh --layer l0 runs it directly
|
||||
# and this Chainsaw Test wraps the SAME script.
|
||||
#
|
||||
# NOT proven: that the endpoint is REACHABLE or that backups exist (runtime facts —
|
||||
# `standup.sh s3check` probes the former from a pod, l3-fault-injection/dr-restore proves
|
||||
# the latter); that guards for knobs OTHER than the placeholder-refusing ones are correct;
|
||||
# and that the guard runs at all in a real standup (the dispatch path is shell, executed
|
||||
# only against a live cluster). R-STATE-03 is tagged for the narrow static slice only: the
|
||||
# authored backup target cannot be a placeholder that the guard fails to recognise.
|
||||
# NOT proven: while no guard exists on `main`, HALF B IS NOT PROVEN, and the check SAYS SO on
|
||||
# stdout every run rather than swallowing it as a skip — what is proven instead is the
|
||||
# stronger-at-the-source half, that the values a guard would be asked about are not
|
||||
# placeholders at all. Also not proven: that the endpoint is REACHABLE or that backups exist
|
||||
# (runtime facts — `standup.sh s3check` probes the former from a pod, l3-fault-injection/
|
||||
# dr-restore proves the latter); that guards for knobs OTHER than the placeholder-refusing
|
||||
# ones are correct; and that the guard runs at all in a real standup (the dispatch path is
|
||||
# shell, executed only against a live cluster). A1 reads cluster-vars, not every manifest: a
|
||||
# placeholder hardcoded in a manifest rather than in a knob is out of scope. R-STATE-03 is
|
||||
# tagged for the narrow static slice only: the authored backup target cannot be a placeholder.
|
||||
# =============================================================================
|
||||
apiVersion: chainsaw.kyverno.io/v1alpha1
|
||||
kind: Test
|
||||
@@ -50,27 +78,32 @@ metadata:
|
||||
conformance.home.k3s/requirements: "R-ENV-02, R-STATE-03"
|
||||
conformance.home.k3s/layer: "L0"
|
||||
# NO `scope` ANNOTATION, DELIBERATELY — this invariant IS requirement-scoped, and was
|
||||
# audited as such (2026-09-12). Its entire input surface is bootstrap/standup.sh (the
|
||||
# placeholder guards) plus clusters/base/cluster-vars.yaml and clusters/<env>/
|
||||
# (the values they guard). ci/touched-requirements.sh keys bootstrap/* to
|
||||
# R-STATE-03 + R-GITOPS-01 and clusters/* to R-ENV-02 + R-GITOPS-01 — so EVERY path that
|
||||
# can break this check already attributes one of the two requirements it declares above.
|
||||
# Requirement-scoped selection therefore cannot hide a breaking change here, and declaring
|
||||
# whole-repo scope would only add runtime, not coverage.
|
||||
# re-audited as such on 2026-09-13 after the restructure changed the input surface.
|
||||
# HALF A reads clusters/base/cluster-vars.yaml plus clusters/<env>/kustomization.yaml;
|
||||
# ci/touched-requirements.sh keys clusters/* to R-ENV-02 + R-GITOPS-01, so every path
|
||||
# that can break HALF A attributes a requirement declared above. HALF B reads whichever
|
||||
# file defines the guard; bootstrap/* is keyed to R-STATE-03 + R-GITOPS-01, which covers
|
||||
# the place a standup guard lands. Requirement-scoped selection therefore cannot hide a
|
||||
# breaking change, and declaring whole-repo scope would only add runtime, not coverage.
|
||||
# The one residual: (A5) reads ci/l0.sh and ci/gate.sh, and ci/* attributes neither
|
||||
# R-ENV-02 nor R-STATE-03 — but that wiring is separately and unconditionally asserted by
|
||||
# the whole-repo-scoped tests/l0-static/whole-repo-invariants-wired, so (A5) here is
|
||||
# belt-and-braces rather than the only proof.
|
||||
conformance.home.k3s/safety-critical: "false"
|
||||
conformance.home.k3s/executable: "true"
|
||||
conformance.home.k3s/intent: >-
|
||||
Every placeholder-shaped value authored in cluster-vars, for any environment
|
||||
overlay, is actually recognised as a placeholder by the real bootstrap guard that
|
||||
is supposed to refuse it — so a value change in one file cannot silently disarm a
|
||||
guard in another and enable velero against an endpoint that does not exist.
|
||||
No environment overlay can ship a placeholder-shaped cluster-vars value that nothing
|
||||
refuses, and where a real bootstrap guard owns such a knob it actually recognises
|
||||
every authored value as a placeholder — so a value change in one file cannot silently
|
||||
disarm a guard in another and enable velero against an endpoint that does not exist.
|
||||
spec:
|
||||
steps:
|
||||
- name: couple-authored-placeholders-to-the-real-guard
|
||||
- name: refuse-authored-placeholders-and-couple-them-to-any-real-guard
|
||||
description: >-
|
||||
Extract the real placeholder predicate from bootstrap/standup.sh, run it against
|
||||
every authored cluster-vars value the repo actually carries, and assert the guard
|
||||
refuses each one an independent detector calls a placeholder.
|
||||
Screen every authored effective cluster-vars value of every environment overlay
|
||||
against an independent placeholder detector, assert empty values are stated rather
|
||||
than inherited, and — when a *_is_placeholder predicate exists anywhere in the tree
|
||||
— extract and execute the REAL guard against those same values.
|
||||
try:
|
||||
- script:
|
||||
content: bash ./check.sh
|
||||
@@ -12,20 +12,74 @@
|
||||
# placeholder. Three of those guards keyed on the SHAPE of `S3_BACKUP_ENDPOINT`, which is
|
||||
# authored in `clusters/base/cluster-vars.yaml`. On 2026-09-05 the placeholder changed
|
||||
# from `TBD-PROD-S3-BACKUP-ENDPOINT` to `https://TBD-R2-ACCOUNT-ID.r2.cloudflarestorage.com`
|
||||
# (the MinIO -> Cloudflare R2 move). The guards matched the glob `TBD-*` — a PREFIX match —
|
||||
# and `fnmatch('https://TBD-R2-…','TBD-*')` is FALSE. All three guards silently stopped
|
||||
# matching. `standup.sh resume` would then have resumed `velero` on prod against a hostname
|
||||
# that does not exist: the HelmRelease installs clean, the Kustomization reports Ready, and
|
||||
# NO BACKUPS EXIST. The only reason it was not shipped is that someone noticed by accident.
|
||||
# (the MinIO -> Cloudflare R2 move, commit 32d6766). The guards matched the glob `TBD-*` —
|
||||
# a PREFIX match — and `fnmatch('https://TBD-R2-…','TBD-*')` is FALSE. All three guards
|
||||
# silently stopped matching. `standup.sh resume` would then have resumed `velero` on prod
|
||||
# against a hostname that does not exist: the HelmRelease installs clean, the Kustomization
|
||||
# reports Ready, and NO BACKUPS EXIST. It was found by accident.
|
||||
#
|
||||
# The predicate is fixed (`s3_is_placeholder`, substring match). The COUPLING is what was
|
||||
# never tested: a guard in one file keying on a value in another, with nothing tying them
|
||||
# together. This check is that tie, and it is derived from BOTH real files — no current
|
||||
# string is hardcoded here, so it does not rot when the placeholder or the guard changes.
|
||||
# WHY THIS CHECK WAS REWRITTEN (2026-09-13) — IT WAS SKIPPING FOREVER
|
||||
# ------------------------------------------------------------------
|
||||
# The original version keyed its ENTIRE existence on `bootstrap/standup.sh` and exited 3
|
||||
# (SKIP) when that file was absent. Established by measurement, not assumption:
|
||||
#
|
||||
# * `bootstrap/standup.sh` HAS NEVER EXISTED ON `main`. It was not deleted — there is no
|
||||
# deletion commit for it anywhere (`git log --all --diff-filter=D -- bootstrap/standup.sh`
|
||||
# is empty). It lives on unmerged prod-standup branches (`work/prod-standup-campaign`,
|
||||
# `prod/*`, `feat/prod-hardening`) and never landed. `git ls-tree main bootstrap/` lists
|
||||
# only README.md, flux/, gitea/, sops/. docs/BULK-NFS-STORAGE.md §7 already recorded
|
||||
# this, including the consequence: "nothing in the gate currently refuses a
|
||||
# placeholder-shaped overlay value".
|
||||
# * So this check reported SKIP on every gate run since it landed, and SKIPPED IS NOT
|
||||
# GREEN in this repo. An incident-derived test that can only ever skip is the quiet
|
||||
# version of the failure it was written to prevent.
|
||||
# * Worse, the skip was HIDING A RED. Run against the real `standup.sh` from
|
||||
# `work/prod-standup-campaign`, the original check FAILED 2 of 34 assertions — both
|
||||
# false positives from treating the EMPTY STRING as placeholder-shaped for knobs whose
|
||||
# emptiness is deliberately authored (`BACKUP_CREDENTIALS_SUFFIX` and
|
||||
# `VELERO_BACKUP_PREFIX`, both `""` on dev). cluster-vars documents that emptiness as
|
||||
# load-bearing ("THE BASE DEFAULT IS THE EMPTY STRING, AND THAT IS LOAD-BEARING … an
|
||||
# EMPTY VALUE IS SAFE UNDER StrictPostBuildSubstitutions"). So the check would have
|
||||
# false-blocked the moment its own input landed. That is fixed below.
|
||||
#
|
||||
# THE HAZARD IS NOT GONE, so this coverage is NOT obsolete and was NOT deleted. The
|
||||
# invariant has two independent halves, and only one of them needs `standup.sh`:
|
||||
#
|
||||
# HALF A — THE AUTHORED VALUES (provable on `main` today, needs NO guard).
|
||||
# No environment overlay may ship a placeholder-shaped effective cluster-vars value.
|
||||
# This is the OUTCOME the 2026-09-05 guard existed to prevent, asserted at the source
|
||||
# instead of at the guard: if prod's `S3_BACKUP_ENDPOINT` is never a placeholder, velero
|
||||
# cannot be pointed at a hostname that does not exist, whether or not a guard catches
|
||||
# it. Its entire input surface is `clusters/`, all of which is on `main`.
|
||||
# HALF B — THE COUPLING (activates automatically when a guard lands).
|
||||
# The real predicate, extracted from the real file and executed, agrees with an
|
||||
# INDEPENDENT detector about every authored value; every consumer of a guarded knob
|
||||
# applies the guard; the guard matches its vocabulary ANYWHERE, not as a prefix.
|
||||
#
|
||||
# THIS CHECK NEVER EXITS 3. Half A always runs, against real files, with an assertion
|
||||
# floor. Half B runs when a guard exists and is reported as NOT PROVEN when one does not —
|
||||
# stated loudly in the output rather than swallowed as a skip.
|
||||
#
|
||||
# THE GUARD IS DISCOVERED, NOT HARDCODED (this is the "re-point" made automatic)
|
||||
# -----------------------------------------------------------------------------
|
||||
# The original hardcoded `bootstrap/standup.sh`, which is exactly why a file that never
|
||||
# landed could disable the whole check. Instead, every tracked `*.sh` in the tree is
|
||||
# scanned for a `<name>_is_placeholder()` definition:
|
||||
# * exactly one such FILE -> that is the guard; Half B runs against it, wherever it is.
|
||||
# A guard that lands at `bootstrap/standup.sh`, or moves to `bootstrap/lib/placeholder.sh`,
|
||||
# is picked up with no edit to this check.
|
||||
# * none -> Half A only, with the gap named in the output.
|
||||
# * more than one file -> FAIL. Two files defining the placeholder vocabulary is the
|
||||
# 2026-09-05 divergence with a second file's name on it; this check cannot say which
|
||||
# one a standup run would source.
|
||||
# `PLACEHOLDER_GUARD_FILE` overrides discovery (repo-relative) for negative fixtures.
|
||||
# NOTE: discovery is shell-only, because the guard family is shell. If the refusal ever
|
||||
# moves to a non-shell mechanism (a Kyverno policy, a CEL admission rule), discovery finds
|
||||
# nothing and says so — re-point this check deliberately in that change.
|
||||
#
|
||||
# HOW THE REAL GUARD IS EXERCISED (and why not a copy)
|
||||
# ---------------------------------------------------
|
||||
# `bootstrap/standup.sh` is NOT sourceable: at top level it parses argv, hard-exits when
|
||||
# `standup.sh` is NOT sourceable: at top level it parses argv, hard-exits when
|
||||
# --kubeconfig/--expect-server/<step> are absent, `cd`s to the repo root, exports
|
||||
# KUBECONFIG, installs an EXIT trap and finally dispatches steps against a live cluster.
|
||||
# So this check EXTRACTS the predicate's own source lines from the file and sources ONLY
|
||||
@@ -35,105 +89,119 @@
|
||||
# * the extracted body must be a SELF-CONTAINED predicate — `case`/`return` and nothing
|
||||
# else, no command substitution, no call to another function. If the guard ever grows
|
||||
# a dependency, extraction stops being faithful and THIS CHECK FAILS LOUDLY, naming the
|
||||
# remedy (move the predicate into a small sourceable library that standup.sh sources)
|
||||
# rather than quietly falling back to testing a copy of the logic;
|
||||
# remedy (move the predicate into a small sourceable library) rather than quietly
|
||||
# falling back to testing a copy of the logic;
|
||||
# * a CANARY pins the extraction itself: the extracted function must refuse the empty
|
||||
# string and accept a realistic endpoint. An extraction that grabbed the wrong lines,
|
||||
# or a predicate degraded to always-true / always-false, fails there — this repo has
|
||||
# already been bitten by a check that passed when perturbed because it asked a question
|
||||
# that was always satisfiable.
|
||||
# string and accept a realistic endpoint.
|
||||
# There is deliberately NO second implementation of the matching logic anywhere in this file.
|
||||
#
|
||||
# THE INDEPENDENT DETECTOR (the other half of the comparison)
|
||||
# ----------------------------------------------------------
|
||||
# Asking "does the guard agree with the guard?" proves nothing, so the set of
|
||||
# placeholder-shaped values is decided by a detector written HERE, whose vocabulary is the
|
||||
# UNION of two independent sources:
|
||||
# THE INDEPENDENT DETECTOR (and the empty-string distinction the old version got wrong)
|
||||
# ------------------------------------------------------------------------------------
|
||||
# Asking "does the guard agree with the guard?" proves nothing, so placeholder-shaped is
|
||||
# decided by a detector written HERE, whose vocabulary is the UNION of two independent
|
||||
# sources:
|
||||
# * SEED — universal loud-placeholder idioms (TBD, TODO, FIXME, PLACEHOLDER, REPLACE-ME,
|
||||
# CHANGE-ME, FILL-ME, SET-ME, XXXX), plus `<angle-bracket>` templates and the empty
|
||||
# string. These are NOT derived from the guard, so DELETING a guard arm cannot make a
|
||||
# value stop being placeholder-shaped — it makes assertion (1) fail, which is the point.
|
||||
# A value using an idiom the guard does not know is a finding either way: fix it by
|
||||
# giving the knob a real value, or by teaching the guard the idiom.
|
||||
# * the guard's OWN vocabulary, read out of its case-arms with the glob anchors STRIPPED.
|
||||
# Stripping the anchors is what makes the comparison non-circular in the dimension that
|
||||
# actually broke: the detector matches a word ANYWHERE, case-insensitively, and the
|
||||
# guard must agree. A prefix-anchored guard cannot pass.
|
||||
# CHANGE-ME, FILL-ME, SET-ME, XXXX), plus `<angle-bracket>` templates. NOT derived from
|
||||
# any guard, so DELETING a guard arm cannot make a value stop being placeholder-shaped.
|
||||
# * the guard's OWN vocabulary (when a guard exists), read out of its case-arms with the
|
||||
# glob anchors STRIPPED. Stripping the anchors is what makes the comparison non-circular
|
||||
# in the dimension that actually broke: the detector matches a word ANYWHERE,
|
||||
# case-insensitively, and the guard must agree. A prefix-anchored guard cannot pass.
|
||||
#
|
||||
# LOUD vs EMPTY — the correction. A LOUD placeholder is a positive statement that a value
|
||||
# was never decided, and it is always a defect in an overlay. The EMPTY STRING is not:
|
||||
# cluster-vars authors `""` deliberately for suffix/prefix knobs, and Flux strict
|
||||
# substitution fails on an UNDEFINED variable, not on one defined as `""`. So emptiness is
|
||||
# NOT treated as a loud placeholder. It is held to a different, real rule instead (A2): an
|
||||
# empty effective value must be STATED EXPLICITLY by that overlay, never inherited in
|
||||
# silence — which is cluster-vars' own documented convention ("Both overlays state this
|
||||
# knob explicitly … so neither inherits it silently"). Emptiness still counts as
|
||||
# placeholder-shaped for a knob a GUARD covers, because a `""` case-arm in the guard is the
|
||||
# guard author's declaration that empty is not a usable target for that knob.
|
||||
#
|
||||
# GIVEN the repository as authored,
|
||||
# WHEN the real predicate(s) are extracted from bootstrap/standup.sh and executed,
|
||||
# THEN (1) VALUES: for every knob a guard is responsible for, every authored value — the
|
||||
# base default AND each environment overlay's value — that the independent
|
||||
# detector calls placeholder-shaped is REFUSED by the real guard, and every value
|
||||
# it calls usable is ACCEPTED (a guard that refuses a real endpoint leaves velero
|
||||
# suspended forever, which is the same class of silent failure);
|
||||
# (2) OVERLAYS: no environment overlay's EFFECTIVE cluster-vars carries a
|
||||
# placeholder-shaped value for a knob that no guard refuses. (Base alone is
|
||||
# exempt: it is not a bootstrap target — `flux bootstrap --path=clusters/<env>` —
|
||||
# and its loud sentinels exist precisely to be overridden. A base placeholder an
|
||||
# overlay forgets to override IS caught, because it becomes that overlay's
|
||||
# effective value.)
|
||||
# (3) CONSUMERS: every function in standup.sh that reads a guarded knob (via `cvar`,
|
||||
# following one hop of variable aliasing) applies a placeholder guard to it. This
|
||||
# is the "all three guards died at once" regression stated as a property: the
|
||||
# guard being correct is worthless if a consumer stops calling it.
|
||||
# (4) ANCHORING: for each word in the guard's own vocabulary, the guard refuses a
|
||||
# value carrying that word in the MIDDLE (not at the start) — the precise defect
|
||||
# of 2026-09-05 — while still accepting a realistic endpoint.
|
||||
# (5) HARNESS: exactly one definition per predicate, the extracted body is verbatim
|
||||
# from the file and self-contained, the canary holds, no divergent second copy of
|
||||
# the matching logic exists elsewhere in standup.sh, and a floor on the number of
|
||||
# assertions run keeps a pass from degenerating into a no-op.
|
||||
# AND (6) LOAD-BEARING: ci/l0.sh runs the L0 conformance layer on every gate run and
|
||||
# ci/gate.sh sets the merge block when that stage fails.
|
||||
# THEN HALF A (always):
|
||||
# (A1) VALUES: no environment overlay's EFFECTIVE cluster-vars value is LOUD
|
||||
# placeholder-shaped, for any knob that no guard refuses. (Base alone is exempt:
|
||||
# it is not a bootstrap target — `flux bootstrap --path=clusters/<env>` — and its
|
||||
# loud sentinels exist to be overridden. A base placeholder an overlay forgets to
|
||||
# override IS caught, because it becomes that overlay's effective value.)
|
||||
# (A2) EXPLICITNESS: an empty effective value is stated explicitly by that overlay.
|
||||
# (A3) SENTINEL DISCIPLINE: if base carries a LOUD sentinel for a knob, every overlay
|
||||
# states that knob explicitly — the `TBD-PROD-*` convention only works if no
|
||||
# overlay can inherit the sentinel by omission.
|
||||
# (A4) DETECTOR CANARIES: the detector recognises the exact 2026-09-05 string
|
||||
# MID-STRING, accepts both real endpoints the repo actually ships, and does NOT
|
||||
# call the empty string loud.
|
||||
# (A5) LOAD-BEARING: ci/l0.sh runs the L0 conformance layer on every gate run and
|
||||
# ci/gate.sh sets the merge block when that stage fails.
|
||||
# HALF B (only when a guard file is discovered):
|
||||
# (B1) VALUES: for every knob a guard is responsible for, every authored value — the
|
||||
# base default AND each overlay's value — that the detector calls placeholder-
|
||||
# shaped is REFUSED by the real guard, and every value it calls usable is
|
||||
# ACCEPTED (a guard that refuses a real endpoint leaves velero suspended forever,
|
||||
# the same class of silent failure).
|
||||
# (B2) CONSUMERS: every function that reads a guarded knob (via `cvar`, following one
|
||||
# hop of variable aliasing) applies a placeholder guard to it. This is the "all
|
||||
# three guards died at once" regression stated as a property.
|
||||
# (B3) ANCHORING: for each word in the guard's own vocabulary, the guard refuses a
|
||||
# value carrying that word in the MIDDLE (not at the start) — the precise defect
|
||||
# of 2026-09-05 — while still accepting a realistic endpoint.
|
||||
# (B4) HARNESS: exactly one definition per predicate, one guard FILE in the tree, the
|
||||
# extracted body verbatim and self-contained, the canary holds, no divergent
|
||||
# second copy of the matching logic, and assertion floors keep a pass from
|
||||
# degenerating into a no-op.
|
||||
#
|
||||
# WHAT THIS DOES *NOT* PROVE (honest gaps)
|
||||
# - While no guard exists on `main`, HALF B IS NOT PROVEN AT ALL, and the check says so on
|
||||
# stdout every run. What is proven is the stronger-at-the-source half: the values a
|
||||
# guard would be asked about are not placeholders in the first place. This is a real
|
||||
# narrowing of coverage versus the intent, and it is reported, not hidden.
|
||||
# - Nothing about RUNTIME: not that the endpoint resolves, not that credentials work, not
|
||||
# that a backup exists. `standup.sh s3check` probes reachability from a pod (operator-
|
||||
# run, cluster-gated) and conformance/tests/l3-fault-injection/dr-restore proves restore.
|
||||
# - It does not execute standup.sh's dispatch path, so it cannot prove the guard is
|
||||
# REACHED in a real run — only that, when reached, it refuses what it must. The
|
||||
# consumer property (3) is a static approximation of reachability: it is a text-level
|
||||
# taint trace over one file, not an interpreter, and a guard whose result is computed
|
||||
# and then ignored would satisfy it.
|
||||
# - Half B does not execute the dispatch path, so it cannot prove the guard is REACHED in
|
||||
# a real run — only that, when reached, it refuses what it must. (B2) is a text-level
|
||||
# taint trace over one file, not an interpreter.
|
||||
# - It says nothing about guards keyed on shapes OTHER than placeholders (e.g. the
|
||||
# EXTERNALDNS_TXT_OWNER_ID == 'default' test), nor about placeholder-shaped values
|
||||
# outside cluster-vars (the Ansible inventory's BLOCKED/NOT-A-DEVICE sentinels are
|
||||
# asserted by the node_baseline role itself, at play time).
|
||||
# - Stage 3b of ci/l0.sh scopes the conformance run to the TOUCHED requirements. An edit
|
||||
# to bootstrap/* attributes NO requirement in ci/touched-requirements.sh's path map, so
|
||||
# the touched set is empty and the FULL L0 layer runs (verified: ci/lib.sh
|
||||
# load_touched_requirements) — this check runs. An edit to clusters/* maps to
|
||||
# R-ENV-02, which this check carries, so a cluster-vars value change also selects it.
|
||||
# A change that touches standup.sh ALONGSIDE some unrelated mapped path (say
|
||||
# platform/cilium/) would scope the run away from this check; closing that needs a
|
||||
# `bootstrap/*` entry in the path map, which is not this check's file to edit.
|
||||
# - A1 reads cluster-vars, not every manifest. A placeholder hardcoded in a manifest
|
||||
# rather than in a knob is out of scope here.
|
||||
#
|
||||
# SELECTION. `ci/touched-requirements.sh` keys `clusters/*` -> R-ENV-02 (Half A's whole
|
||||
# input surface) and `bootstrap/*` -> R-STATE-03 (where a guard lands), so every path that
|
||||
# can break this check attributes a requirement it declares. Deliberately requirement-scoped,
|
||||
# not whole-repo — see chainsaw-test.yaml.
|
||||
# =============================================================================
|
||||
set -euo pipefail
|
||||
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ROOT="$(cd "${DIR}/../../../.." && pwd)" # repo root
|
||||
PYTHON="${PYTHON:-python3}"
|
||||
STANDUP_REL="${STANDUP_REL:-bootstrap/standup.sh}"
|
||||
GUARD_OVERRIDE="${PLACEHOLDER_GUARD_FILE:-}"
|
||||
|
||||
cd "${ROOT}"
|
||||
|
||||
if [ ! -f "${ROOT}/${STANDUP_REL}" ]; then
|
||||
echo "check SKIPPED (placeholder-guard-coupling): ${STANDUP_REL} is absent — nothing to couple." >&2
|
||||
exit 3
|
||||
# The authored half of the coupling is the one thing this check cannot do without.
|
||||
if [ ! -f "${ROOT}/clusters/base/cluster-vars.yaml" ]; then
|
||||
echo "CHECK FAIL (placeholder-guard-coupling): clusters/base/cluster-vars.yaml is absent — the authored values this check audits do not exist. That is not a SKIP: the knobs this invariant is about have been moved or deleted, and this check must be re-pointed in the SAME change." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
WORK="$(mktemp -d)"
|
||||
trap 'rm -rf "${WORK}"' EXIT
|
||||
|
||||
# ---------------------------------------------------------------------------------------
|
||||
# PHASE 1 (python): extract the real predicate(s), derive the corpus from the real files,
|
||||
# and run every assertion that does not need a shell. Writes:
|
||||
# ${WORK}/guards.sh the predicate source lines, verbatim, and nothing else
|
||||
# ${WORK}/corpus.tsv <guard>\t<expect: placeholder|usable>\t<origin>\t<value>
|
||||
# PHASE 1 (python): discover the guard, derive the corpus from the real files, and run
|
||||
# every assertion that does not need a shell. Writes:
|
||||
# ${WORK}/guards.sh the predicate source lines, verbatim, and nothing else (may be empty)
|
||||
# ${WORK}/corpus.tsv <guard>\t<expect: placeholder|usable>\t<origin>\t<value> (may be empty)
|
||||
# ${WORK}/findings.txt one line per failed assertion
|
||||
# ${WORK}/counts.txt <assertions-run>
|
||||
# ${WORK}/mode.txt the discovered guard's repo-relative path, or empty
|
||||
# ---------------------------------------------------------------------------------------
|
||||
"${PYTHON}" - "${ROOT}" "${WORK}" "${STANDUP_REL}" <<'PY'
|
||||
"${PYTHON}" - "${ROOT}" "${WORK}" "${GUARD_OVERRIDE}" <<'PY'
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
@@ -145,7 +213,7 @@ except ImportError: # pragma: no cover
|
||||
file=sys.stderr)
|
||||
raise SystemExit(1)
|
||||
|
||||
ROOT, WORK, STANDUP_REL = Path(sys.argv[1]), Path(sys.argv[2]), sys.argv[3]
|
||||
ROOT, WORK, OVERRIDE = Path(sys.argv[1]), Path(sys.argv[2]), sys.argv[3]
|
||||
FAILURES: list[str] = []
|
||||
CHECKS = 0
|
||||
|
||||
@@ -164,116 +232,159 @@ def bail(msg):
|
||||
(WORK / "counts.txt").write_text("0\n")
|
||||
(WORK / "corpus.tsv").write_text("")
|
||||
(WORK / "guards.sh").write_text("")
|
||||
(WORK / "mode.txt").write_text("")
|
||||
raise SystemExit(1)
|
||||
|
||||
|
||||
TEXT = (ROOT / STANDUP_REL).read_text()
|
||||
LINES = TEXT.splitlines()
|
||||
|
||||
# ------------------------------------------------------------------ (5) extract the guard
|
||||
DEF_RE = re.compile(r"^([a-z0-9_]+_is_placeholder)\(\)\s*\{")
|
||||
defs: dict[str, list[int]] = {}
|
||||
for i, line in enumerate(LINES):
|
||||
m = DEF_RE.match(line)
|
||||
if m:
|
||||
defs.setdefault(m.group(1), []).append(i)
|
||||
SELF_DIR = ROOT / "conformance" / "tests" / "l0-static" / "placeholder-guard-coupling"
|
||||
SKIP_DIRS = {".git", "_artifacts", "node_modules", ".venv"}
|
||||
|
||||
if not defs:
|
||||
bail(f"{STANDUP_REL} defines no *_is_placeholder predicate. Either the guard was renamed "
|
||||
"(update this check's discovery pattern in the SAME change) or it was DELETED, which "
|
||||
"is the 2026-09-05 failure with the guard removed instead of disarmed.")
|
||||
# ------------------------------------------------------------------ discover the guard FILE
|
||||
# Hardcoding a path is what let a file that never landed disable this whole check.
|
||||
if OVERRIDE:
|
||||
cand = [ROOT / OVERRIDE]
|
||||
if not cand[0].is_file():
|
||||
bail(f"PLACEHOLDER_GUARD_FILE={OVERRIDE} does not exist")
|
||||
else:
|
||||
cand = []
|
||||
for p in sorted(ROOT.rglob("*.sh")):
|
||||
if SKIP_DIRS & set(p.parts) or SELF_DIR in p.parents:
|
||||
continue
|
||||
try:
|
||||
text = p.read_text()
|
||||
except (OSError, UnicodeDecodeError): # pragma: no cover
|
||||
continue
|
||||
if any(DEF_RE.match(line) for line in text.splitlines()):
|
||||
cand.append(p)
|
||||
|
||||
# An "argument-shaped" call: <name> "<something>"
|
||||
CALL_RE = re.compile(r"\b([a-z0-9_]+_is_placeholder)\s+\"([^\"]*)\"")
|
||||
check(len(cand) <= 1,
|
||||
"more than one file defines a *_is_placeholder predicate ("
|
||||
+ ", ".join(str(p.relative_to(ROOT)) for p in cand)
|
||||
+ ") — two files owning the placeholder vocabulary is the 2026-09-05 divergence with a "
|
||||
"second file's name on it, and this check cannot say which one a standup run would "
|
||||
"source. Keep exactly one, and source it from the other.")
|
||||
|
||||
GUARD_REL = str(cand[0].relative_to(ROOT)) if len(cand) == 1 else ""
|
||||
(WORK / "mode.txt").write_text(GUARD_REL + "\n")
|
||||
|
||||
# --------------------------------------------------------- HALF B setup: extract the guard
|
||||
guard_src: list[str] = []
|
||||
guard_words: dict[str, list[str]] = {}
|
||||
ARM_RE = re.compile(r"^(?P<pats>[^()]*?)\)\s*return\s+[01]\s*;;$")
|
||||
ALLOWED = (
|
||||
re.compile(r"^$"),
|
||||
re.compile(r"^#"),
|
||||
re.compile(r"^case\s+\"\$\{?[0-9A-Za-z_]+\}?\"\s+in$"),
|
||||
re.compile(r"^esac$"),
|
||||
re.compile(r"^return\s+[01]$"),
|
||||
ARM_RE,
|
||||
)
|
||||
funcs: dict[str, list[str]] = {}
|
||||
CALL_RE = re.compile(r"\b([a-z0-9_]+_is_placeholder)\s+\"([^\"]*)\"")
|
||||
|
||||
for name, at in sorted(defs.items()):
|
||||
check(len(at) == 1,
|
||||
f"{STANDUP_REL} defines {name}() {len(at)} times (lines "
|
||||
f"{', '.join(str(a + 1) for a in at)}) — which one runs depends on file order, and "
|
||||
"this check can no longer say which one it extracted. Keep exactly one definition.")
|
||||
if len(at) != 1:
|
||||
continue
|
||||
start = at[0]
|
||||
end = next((j for j in range(start + 1, len(LINES)) if LINES[j] == "}"), None)
|
||||
if end is None:
|
||||
bail(f"{name}() at {STANDUP_REL}:{start + 1} has no closing brace at column 0 — the "
|
||||
"extraction boundary this check relies on is gone. Restore the conventional "
|
||||
"layout, or move the predicate into a sourceable library and point this check at it.")
|
||||
body = LINES[start:end + 1]
|
||||
block = "\n".join(body)
|
||||
check(block in TEXT,
|
||||
f"extracted {name}() is not a verbatim substring of {STANDUP_REL} — extraction bug; "
|
||||
"the harness would be testing something the script does not contain")
|
||||
if GUARD_REL:
|
||||
TEXT = (ROOT / GUARD_REL).read_text()
|
||||
LINES = TEXT.splitlines()
|
||||
|
||||
words: list[str] = []
|
||||
for raw in body[1:-1]:
|
||||
s = raw.strip()
|
||||
if not any(rx.match(s) for rx in ALLOWED) or "$(" in s or "`" in s:
|
||||
bail(f"{name}() at {STANDUP_REL}:{start + 1} is no longer a SELF-CONTAINED "
|
||||
f"case/return predicate (offending line: {s!r}). This check exercises the REAL "
|
||||
"function by sourcing its own lines; a body that calls out to anything else "
|
||||
"cannot be extracted faithfully, and silently testing a COPY of the logic is "
|
||||
"exactly the failure mode this check exists to prevent. REMEDY: move the "
|
||||
"predicate into a tiny library that standup.sh sources (e.g. "
|
||||
"bootstrap/lib/placeholder.sh) and point STANDUP_REL/this extractor at it — do "
|
||||
"NOT reimplement the matching here.")
|
||||
m = ARM_RE.match(s)
|
||||
if not m:
|
||||
defs: dict[str, list[int]] = {}
|
||||
for i, line in enumerate(LINES):
|
||||
m = DEF_RE.match(line)
|
||||
if m:
|
||||
defs.setdefault(m.group(1), []).append(i)
|
||||
|
||||
ARM_RE = re.compile(r"^(?P<pats>[^()]*?)\)\s*return\s+[01]\s*;;$")
|
||||
ALLOWED = (
|
||||
re.compile(r"^$"),
|
||||
re.compile(r"^#"),
|
||||
re.compile(r"^case\s+\"\$\{?[0-9A-Za-z_]+\}?\"\s+in$"),
|
||||
re.compile(r"^esac$"),
|
||||
re.compile(r"^return\s+[01]$"),
|
||||
ARM_RE,
|
||||
)
|
||||
|
||||
for name, at in sorted(defs.items()):
|
||||
check(len(at) == 1,
|
||||
f"{GUARD_REL} defines {name}() {len(at)} times (lines "
|
||||
f"{', '.join(str(a + 1) for a in at)}) — which one runs depends on file order, and "
|
||||
"this check can no longer say which one it extracted. Keep exactly one definition.")
|
||||
if len(at) != 1:
|
||||
continue
|
||||
for pat in m.group("pats").split("|"):
|
||||
pat = pat.strip()
|
||||
if pat in ('""', "''", ""):
|
||||
continue # the empty-value arm: covered by the canary
|
||||
words.append(pat.strip("'\"").strip("*"))
|
||||
check(len(words) >= 1,
|
||||
f"{name}() has no non-empty placeholder pattern left — every arm that recognises a "
|
||||
"loud placeholder string has been removed, so the guard now only refuses the empty "
|
||||
"value and any 'TBD'-style endpoint would sail straight through it")
|
||||
guard_words[name] = words
|
||||
guard_src.append(block)
|
||||
start = at[0]
|
||||
end = next((j for j in range(start + 1, len(LINES)) if LINES[j] == "}"), None)
|
||||
if end is None:
|
||||
bail(f"{name}() at {GUARD_REL}:{start + 1} has no closing brace at column 0 — the "
|
||||
"extraction boundary this check relies on is gone. Restore the conventional "
|
||||
"layout, or move the predicate into a sourceable library.")
|
||||
body = LINES[start:end + 1]
|
||||
block = "\n".join(body)
|
||||
check(block in TEXT,
|
||||
f"extracted {name}() is not a verbatim substring of {GUARD_REL} — extraction bug; "
|
||||
"the harness would be testing something the script does not contain")
|
||||
|
||||
(WORK / "guards.sh").write_text("\n".join(guard_src) + "\n")
|
||||
words: list[str] = []
|
||||
for raw in body[1:-1]:
|
||||
s = raw.strip()
|
||||
if not any(rx.match(s) for rx in ALLOWED) or "$(" in s or "`" in s:
|
||||
bail(f"{name}() at {GUARD_REL}:{start + 1} is no longer a SELF-CONTAINED "
|
||||
f"case/return predicate (offending line: {s!r}). This check exercises the "
|
||||
"REAL function by sourcing its own lines; a body that calls out to anything "
|
||||
"else cannot be extracted faithfully, and silently testing a COPY of the "
|
||||
"logic is exactly the failure mode this check exists to prevent. REMEDY: "
|
||||
"move the predicate into a tiny library that the script sources (e.g. "
|
||||
"bootstrap/lib/placeholder.sh) — discovery will find it — do NOT "
|
||||
"reimplement the matching here.")
|
||||
m = ARM_RE.match(s)
|
||||
if not m:
|
||||
continue
|
||||
for pat in m.group("pats").split("|"):
|
||||
pat = pat.strip()
|
||||
if pat in ('""', "''", ""):
|
||||
continue # the empty-value arm: covered by the canary
|
||||
words.append(pat.strip("'\"").strip("*"))
|
||||
check(len(words) >= 1,
|
||||
f"{name}() has no non-empty placeholder pattern left — every arm that recognises a "
|
||||
"loud placeholder string has been removed, so the guard now only refuses the empty "
|
||||
"value and any 'TBD'-style endpoint would sail straight through it")
|
||||
guard_words[name] = words
|
||||
guard_src.append(block)
|
||||
|
||||
# ------------------------------------------------- no divergent second copy of the logic
|
||||
# Any of the guard's own vocabulary appearing in EXECUTABLE text outside the predicate means
|
||||
# a second, independently-drifting implementation of the same test (which is how three
|
||||
# guards came to disagree with one value in the first place).
|
||||
guard_line_nums = {j for name, at in defs.items() if len(at) == 1
|
||||
for j in range(at[0], next(k for k in range(at[0] + 1, len(LINES))
|
||||
if LINES[k] == "}") + 1)}
|
||||
all_words = sorted({w.upper() for ws in guard_words.values() for w in ws if w})
|
||||
divergent: list[str] = []
|
||||
for j, line in enumerate(LINES):
|
||||
if j in guard_line_nums or line.strip().startswith("#"):
|
||||
continue
|
||||
code = line.split("#", 1)[0]
|
||||
hit = [w for w in all_words if w in code.upper()]
|
||||
if hit:
|
||||
divergent.append(f"{STANDUP_REL}:{j + 1} ({', '.join(hit)}): {line.strip()!r}")
|
||||
check(not divergent,
|
||||
"the guard's own placeholder vocabulary appears in EXECUTABLE text outside the "
|
||||
f"predicate — a SECOND implementation of the same test, which is precisely how a value "
|
||||
f"change disarmed three guards at once. Call the predicate instead. Sites: "
|
||||
+ " | ".join(divergent))
|
||||
if not guard_words:
|
||||
bail(f"{GUARD_REL} was discovered as the guard file but yielded no usable predicate — "
|
||||
"the definitions are there but none could be extracted. Fix the extraction in the "
|
||||
"SAME change; do not let this degrade to a pass.")
|
||||
|
||||
# ------------------------------------------- no divergent second copy of the logic
|
||||
# The guard's own vocabulary in EXECUTABLE text outside the predicate means a second,
|
||||
# independently-drifting implementation of the same test (which is how three guards came
|
||||
# to disagree with one value in the first place).
|
||||
guard_line_nums = {j for name, at in defs.items() if len(at) == 1
|
||||
for j in range(at[0], next(k for k in range(at[0] + 1, len(LINES))
|
||||
if LINES[k] == "}") + 1)}
|
||||
divergent: list[str] = []
|
||||
_words = sorted({w.upper() for ws in guard_words.values() for w in ws if w})
|
||||
for j, line in enumerate(LINES):
|
||||
if j in guard_line_nums or line.strip().startswith("#"):
|
||||
continue
|
||||
code = line.split("#", 1)[0]
|
||||
hit = [w for w in _words if w in code.upper()]
|
||||
if hit:
|
||||
divergent.append(f"{GUARD_REL}:{j + 1} ({', '.join(hit)}): {line.strip()!r}")
|
||||
check(not divergent,
|
||||
"the guard's own placeholder vocabulary appears in EXECUTABLE text outside the "
|
||||
"predicate — a SECOND implementation of the same test, which is precisely how a value "
|
||||
"change disarmed three guards at once. Call the predicate instead. Sites: "
|
||||
+ " | ".join(divergent))
|
||||
|
||||
# function bodies, for the consumer trace
|
||||
FUNC_RE = re.compile(r"^([a-z0-9_]+)\(\)\s*\{")
|
||||
i = 0
|
||||
while i < len(LINES):
|
||||
m = FUNC_RE.match(LINES[i])
|
||||
if m:
|
||||
end = next((j for j in range(i + 1, len(LINES)) if LINES[j] == "}"), len(LINES) - 1)
|
||||
funcs[m.group(1)] = LINES[i + 1:end]
|
||||
i = end + 1
|
||||
continue
|
||||
i += 1
|
||||
check(len(funcs) > 5, f"parsed only {len(funcs)} function(s) out of {GUARD_REL} — the "
|
||||
"consumer assertion (B2) would be vacuous")
|
||||
|
||||
(WORK / "guards.sh").write_text(("\n".join(guard_src) + "\n") if guard_src else "")
|
||||
|
||||
# ------------------------------------------------------------- cluster-vars, as authored
|
||||
CV = ROOT / "clusters" / "base" / "cluster-vars.yaml"
|
||||
if not CV.is_file():
|
||||
bail("clusters/base/cluster-vars.yaml is absent — the authored half of the coupling is "
|
||||
"missing and this check cannot mean anything")
|
||||
|
||||
base: dict[str, str] = {}
|
||||
for d in yaml.safe_load_all(CV.read_text()):
|
||||
if isinstance(d, dict) and d.get("kind") == "ConfigMap" \
|
||||
@@ -282,12 +393,16 @@ for d in yaml.safe_load_all(CV.read_text()):
|
||||
check(len(base) > 5, f"clusters/base/cluster-vars.yaml yielded {len(base)} knob(s) — the "
|
||||
"ConfigMap parse failed, so every value assertion below would be vacuous")
|
||||
|
||||
# Effective values per overlay, and WHICH knobs that overlay states explicitly. The second
|
||||
# half is what lets emptiness be judged as authored-or-forgotten instead of guessed.
|
||||
overlays: dict[str, dict[str, str]] = {}
|
||||
stated: dict[str, set[str]] = {}
|
||||
for overlay in sorted((ROOT / "clusters").glob("*/kustomization.yaml")):
|
||||
env = overlay.parent.name
|
||||
if env == "base":
|
||||
continue
|
||||
eff = dict(base)
|
||||
explicit: set[str] = set()
|
||||
try:
|
||||
k = yaml.safe_load(overlay.read_text()) or {}
|
||||
except Exception as exc: # noqa: BLE001
|
||||
@@ -308,27 +423,15 @@ for overlay in sorted((ROOT / "clusters").glob("*/kustomization.yaml")):
|
||||
m = re.match(r"^/data/([A-Za-z0-9_]+)$", str(op.get("path", "")))
|
||||
if m and str(op.get("op", "")) in ("replace", "add"):
|
||||
eff[m.group(1)] = "" if op.get("value") is None else str(op["value"])
|
||||
explicit.add(m.group(1))
|
||||
overlays[env] = eff
|
||||
stated[env] = explicit
|
||||
check(len(overlays) >= 1,
|
||||
"no clusters/<env>/kustomization.yaml overlay was parsed — assertion (2) would be vacuous")
|
||||
|
||||
# ------------------------------------------------ which knobs is a guard responsible for?
|
||||
# Function bodies of standup.sh, then a one-hop taint trace from `x="$(cvar KEY)"` to a
|
||||
# `*_is_placeholder "${x}"` call. Both halves are read out of the file; nothing is named here.
|
||||
FUNC_RE = re.compile(r"^([a-z0-9_]+)\(\)\s*\{")
|
||||
funcs: dict[str, list[str]] = {}
|
||||
i = 0
|
||||
while i < len(LINES):
|
||||
m = FUNC_RE.match(LINES[i])
|
||||
if m:
|
||||
end = next((j for j in range(i + 1, len(LINES)) if LINES[j] == "}"), len(LINES) - 1)
|
||||
funcs[m.group(1)] = LINES[i + 1:end]
|
||||
i = end + 1
|
||||
continue
|
||||
i += 1
|
||||
check(len(funcs) > 5, f"parsed only {len(funcs)} function(s) out of {STANDUP_REL} — the "
|
||||
"consumer assertion (3) would be vacuous")
|
||||
"no clusters/<env>/kustomization.yaml overlay was parsed — assertion (A1) would be vacuous")
|
||||
|
||||
# ------------------------------------------ which knobs is a guard responsible for? (HALF B)
|
||||
# A one-hop taint trace from `x="$(cvar KEY)"` to a `*_is_placeholder "${x}"` call. Both
|
||||
# halves are read out of the file; nothing is named here.
|
||||
CVAR_RE = re.compile(r"([A-Za-z_][A-Za-z0-9_]*)=\"\$\(\s*cvar\s+([A-Z0-9_]+)\s*\)\"")
|
||||
ALIAS_RE = re.compile(r"([A-Za-z_][A-Za-z0-9_]*)=\"\$\{([A-Za-z_][A-Za-z0-9_]*)[:}]")
|
||||
|
||||
@@ -347,8 +450,6 @@ for fname, body in funcs.items():
|
||||
keys_of_var.setdefault(m.group(1), set()).update(src)
|
||||
called_on = {m.group(2) for m in CALL_RE.finditer(text)}
|
||||
called_vars = {re.sub(r"^\$\{?|\}?$", "", a) for a in called_on}
|
||||
# Aggregate per (function, knob): the knob is guarded HERE if ANY variable carrying its
|
||||
# value is passed to a predicate — `live="$(cvar K)"; ep="${live}"; guard "${ep}"` counts.
|
||||
guarded_here: dict[str, bool] = {}
|
||||
for var, keys in keys_of_var.items():
|
||||
for key in keys:
|
||||
@@ -358,43 +459,119 @@ for fname, body in funcs.items():
|
||||
if g:
|
||||
guarded_keys.add(key)
|
||||
|
||||
if not guarded_keys:
|
||||
bail(f"no cluster-vars knob in {STANDUP_REL} is passed to a *_is_placeholder predicate. "
|
||||
if GUARD_REL and not guarded_keys:
|
||||
bail(f"no cluster-vars knob in {GUARD_REL} is passed to a *_is_placeholder predicate. "
|
||||
"Either the guards were removed, or the way the script reads cluster-vars changed and "
|
||||
"this check's derivation must be updated in the SAME change — it must never silently "
|
||||
"verify an empty set.")
|
||||
|
||||
# (3) CONSUMERS: every reader of a guarded knob must apply the guard to it.
|
||||
# (B2) CONSUMERS: every reader of a guarded knob must apply the guard to it.
|
||||
for fname, key, guarded in sorted(consumers):
|
||||
if key not in guarded_keys:
|
||||
continue # nothing claims responsibility for this knob at all
|
||||
check(guarded,
|
||||
f"{STANDUP_REL}: {fname}() reads cluster-vars knob {key} but never passes it to a "
|
||||
f"{GUARD_REL}: {fname}() reads cluster-vars knob {key} but never passes it to a "
|
||||
"*_is_placeholder guard, while other functions do. On 2026-09-05 all three consumers "
|
||||
"of S3_BACKUP_ENDPOINT were disarmed at once; a consumer that reads the knob and skips "
|
||||
"the guard is that failure in one function.")
|
||||
|
||||
# ---------------------------------------------------------------- independent detector
|
||||
# Deliberately NOT derived from the guard alone (see the header): SEED is universal
|
||||
# loud-placeholder vocabulary, and the guard's own words are added so that a guard whose
|
||||
# ANCHORING is wrong still has its vocabulary tested against real values.
|
||||
# ------------------------------------------------------------------ independent detector
|
||||
# Deliberately NOT derived from the guard alone: SEED is universal loud-placeholder
|
||||
# vocabulary, and the guard's own words are added so that a guard whose ANCHORING is wrong
|
||||
# still has its vocabulary tested against real values.
|
||||
SEED = ["TBD", "TODO", "FIXME", "PLACEHOLDER", "REPLACE-ME", "REPLACEME",
|
||||
"CHANGE-ME", "CHANGEME", "FILL-ME", "FILLME", "SET-ME", "XXXX"]
|
||||
all_words = sorted({w.upper() for ws in guard_words.values() for w in ws if w})
|
||||
VOCAB = sorted({w.upper() for w in SEED} | set(all_words))
|
||||
ANGLE = re.compile(r"<[^>\s]+>")
|
||||
|
||||
|
||||
def placeholder_shaped(value: str) -> bool:
|
||||
if value.strip() == "":
|
||||
return True # an unset endpoint is not a target
|
||||
def loud(value: str) -> bool:
|
||||
"""A positive statement that the value was never decided. The EMPTY STRING IS NOT LOUD —
|
||||
see the header: cluster-vars authors "" deliberately, and treating it as a placeholder is
|
||||
the false positive that made the original check red against its own input."""
|
||||
up = value.upper()
|
||||
return any(w in up for w in VOCAB) or bool(ANGLE.search(value))
|
||||
|
||||
|
||||
rows: list[tuple[str, str, str, str]] = [] # guard, expect, origin, value
|
||||
GUARD = sorted(guard_words)[0] if len(guard_words) == 1 else None
|
||||
def guard_shaped(value: str) -> bool:
|
||||
"""What a GUARD must refuse: loud, or empty. Empty counts here only because a `""`
|
||||
case-arm is the guard author's declaration that empty is not a usable target."""
|
||||
return value.strip() == "" or loud(value)
|
||||
|
||||
# (1) VALUES: base default and every overlay's value, for every guarded knob.
|
||||
|
||||
# ------------------------------------------------------------------- (A4) DETECTOR CANARIES
|
||||
# The detector is now the thing doing the refusing in HALF A, so it gets the same
|
||||
# no-op interrogation the guard does. These strings are the real ones from the incident.
|
||||
check(loud("https://TBD-R2-ACCOUNT-ID.r2.cloudflarestorage.com"),
|
||||
"the detector does not recognise the exact 2026-09-05 placeholder "
|
||||
"'https://TBD-R2-ACCOUNT-ID.r2.cloudflarestorage.com' — it is prefix-anchored, which is "
|
||||
"the defect this check exists to catch, now in the check itself")
|
||||
check(loud("TBD-PROD-BULK-NFS-SERVER"),
|
||||
"the detector does not recognise the repo's own TBD-PROD-* sentinel convention "
|
||||
"(clusters/README.md) — assertion (A1) would pass over the very values it is about")
|
||||
check(loud("<your-account-id>"),
|
||||
"the detector does not recognise an <angle-bracket> template as a placeholder")
|
||||
check(not loud("https://0123456789abcdef0123456789abcd.r2.cloudflarestorage.com"),
|
||||
"the detector calls a REAL Cloudflare R2 endpoint a placeholder — it would false-block "
|
||||
"every prod backup target")
|
||||
check(not loud("http://192.168.1.9:9000"),
|
||||
"the detector calls a REAL MinIO endpoint a placeholder — it would false-block dev")
|
||||
check(not loud(""),
|
||||
"the detector calls the EMPTY STRING loud. cluster-vars authors \"\" deliberately for "
|
||||
"suffix/prefix knobs and documents it as load-bearing; emptiness is held to the "
|
||||
"explicitness rule (A2) instead. This exact false positive made the original check FAIL "
|
||||
"2 of 34 assertions against the real guard.")
|
||||
|
||||
# --------------------------------- (A1) no overlay ships a LOUD value nothing refuses
|
||||
a1_rows = 0
|
||||
for env, eff in sorted(overlays.items()):
|
||||
for key, value in sorted(eff.items()):
|
||||
if key in guarded_keys:
|
||||
continue # a guard owns this knob: HALF B (B1) asserts it refuses it
|
||||
a1_rows += 1
|
||||
check(not loud(value),
|
||||
f"clusters/{env}: knob {key} resolves to the placeholder-shaped value {value!r}, "
|
||||
"and nothing refuses it"
|
||||
+ (f" — no guard in {GUARD_REL} covers this knob." if GUARD_REL
|
||||
else " — and no placeholder guard exists in the tree at all.")
|
||||
+ " Either give it a real value before it reaches an overlay, or guard it the way "
|
||||
"the S3 endpoint is guarded. An overlay that ships a placeholder no guard "
|
||||
"recognises is the 2026-09-05 failure with a different knob's name on it: it "
|
||||
"applies clean, reports Ready, and the thing it configures does not exist.")
|
||||
|
||||
# ------------------------- (A2) an empty effective value must be STATED, never inherited
|
||||
a2_rows = 0
|
||||
for env, eff in sorted(overlays.items()):
|
||||
for key, value in sorted(eff.items()):
|
||||
if value.strip() != "":
|
||||
continue
|
||||
a2_rows += 1
|
||||
check(key in stated[env],
|
||||
f"clusters/{env}: knob {key} resolves to the EMPTY STRING by INHERITANCE from "
|
||||
"clusters/base — the overlay never states it. An empty value can be correct (dev's "
|
||||
"BACKUP_CREDENTIALS_SUFFIX and VELERO_BACKUP_PREFIX are authored empty on purpose), "
|
||||
"but only when the overlay SAYS SO: cluster-vars' own convention is that both "
|
||||
"overlays state such a knob explicitly so neither inherits it silently. An empty "
|
||||
"value nobody stated is indistinguishable from a knob somebody forgot.")
|
||||
|
||||
# ---------------- (A3) a LOUD base sentinel must be overridden EXPLICITLY by every overlay
|
||||
a3_rows = 0
|
||||
for key, value in sorted(base.items()):
|
||||
if not loud(value):
|
||||
continue
|
||||
for env in sorted(overlays):
|
||||
a3_rows += 1
|
||||
check(key in stated[env],
|
||||
f"clusters/base/cluster-vars.yaml authors the LOUD sentinel {key}={value!r}, and "
|
||||
f"clusters/{env} does not state that knob — so that overlay INHERITS the sentinel. "
|
||||
"The TBD-PROD-* convention only works if a sentinel cannot be inherited by "
|
||||
"omission; an omitted op is how prod silently ships a value nobody decided.")
|
||||
|
||||
# --------------------------------------- HALF B corpus: authored values + anchoring + canaries
|
||||
rows: list[tuple[str, str, str, str]] = [] # guard, expect, origin, value
|
||||
|
||||
# (B1) VALUES: base default and every overlay's value, for every guarded knob.
|
||||
for key in sorted(guarded_keys):
|
||||
seen: list[tuple[str, str]] = []
|
||||
if key in base:
|
||||
@@ -404,24 +581,12 @@ for key in sorted(guarded_keys):
|
||||
check(bool(seen), f"guarded knob {key} is not defined in cluster-vars at all — the guard "
|
||||
"protects a knob nothing authors")
|
||||
for origin, value in seen:
|
||||
expect = "placeholder" if placeholder_shaped(value) else "usable"
|
||||
expect = "placeholder" if guard_shaped(value) else "usable"
|
||||
for name in sorted(guard_words):
|
||||
rows.append((name, expect, origin, value))
|
||||
|
||||
# (2) OVERLAYS: a placeholder-shaped effective value for an UNGUARDED knob.
|
||||
for env, eff in sorted(overlays.items()):
|
||||
for key, value in sorted(eff.items()):
|
||||
if key in guarded_keys or not placeholder_shaped(value):
|
||||
continue
|
||||
check(False,
|
||||
f"clusters/{env}: knob {key} resolves to the placeholder-shaped value {value!r}, "
|
||||
f"and NO guard in {STANDUP_REL} refuses it. Either give it a real value before it "
|
||||
"reaches an overlay, or guard it the way the S3 endpoint is guarded — an overlay "
|
||||
"that ships a placeholder no guard recognises is the 2026-09-05 failure with a "
|
||||
"different knob's name on it.")
|
||||
|
||||
# (4) ANCHORING: the guard's own vocabulary must match ANYWHERE, not just as a prefix,
|
||||
# and a realistic endpoint must still be accepted.
|
||||
# (B3) ANCHORING: the guard's own vocabulary must match ANYWHERE, not just as a prefix,
|
||||
# and a realistic endpoint must still be accepted.
|
||||
for name, words in sorted(guard_words.items()):
|
||||
for w in sorted(set(words)):
|
||||
rows.append((name, "placeholder",
|
||||
@@ -436,15 +601,16 @@ for name, words in sorted(guard_words.items()):
|
||||
|
||||
# Every guarded knob must reach the oracle once per authored source (base + each overlay),
|
||||
# so a knob silently dropping out of the corpus is a failure rather than a quiet pass.
|
||||
authored = [r for r in rows if not r[2].startswith(("anchoring:", "canary:"))]
|
||||
expected_rows = len(guarded_keys) * (1 + len(overlays)) * len(guard_words)
|
||||
check(len(authored) == expected_rows,
|
||||
f"{len(authored)} authored value(s) reached the guard, expected {expected_rows} "
|
||||
f"({len(guarded_keys)} guarded knob(s) x base+{len(overlays)} overlay(s) x "
|
||||
f"{len(guard_words)} predicate(s)) — a guarded knob is missing from cluster-vars or from "
|
||||
"an overlay's effective values, and the coupling is only partly exercised")
|
||||
if GUARD_REL:
|
||||
authored = [r for r in rows if not r[2].startswith(("anchoring:", "canary:"))]
|
||||
expected_rows = len(guarded_keys) * (1 + len(overlays)) * len(guard_words)
|
||||
check(len(authored) == expected_rows,
|
||||
f"{len(authored)} authored value(s) reached the guard, expected {expected_rows} "
|
||||
f"({len(guarded_keys)} guarded knob(s) x base+{len(overlays)} overlay(s) x "
|
||||
f"{len(guard_words)} predicate(s)) — a guarded knob is missing from cluster-vars or "
|
||||
"from an overlay's effective values, and the coupling is only partly exercised")
|
||||
|
||||
# ------------------------------------------------------- (6) the check is load-bearing
|
||||
# --------------------------------------------------- (A5) the check is load-bearing
|
||||
L0 = (ROOT / "ci" / "l0.sh").read_text() if (ROOT / "ci" / "l0.sh").is_file() else ""
|
||||
GATE = (ROOT / "ci" / "gate.sh").read_text() if (ROOT / "ci" / "gate.sh").is_file() else ""
|
||||
check("--layer l0" in L0,
|
||||
@@ -452,6 +618,14 @@ check("--layer l0" in L0,
|
||||
check(bool(re.search(r"^for s in .*\bl0\b.*; do", GATE, re.M)) and bool(re.search(r"fail\).*BLOCK=1", GATE)),
|
||||
"ci/gate.sh does not set the merge block when the l0 stage fails")
|
||||
|
||||
# ------------------------------------------------------- non-vacuity floor for HALF A
|
||||
# HALF A is the half that runs unconditionally, so it carries its own floor: if the
|
||||
# authored-value sweep shrinks to nothing, a pass here means nothing.
|
||||
check(a1_rows >= 20,
|
||||
f"only {a1_rows} authored overlay value(s) were audited by (A1) — the sweep over "
|
||||
"clusters/<env> effective cluster-vars has collapsed, and a pass would be meaningless. "
|
||||
"Expected roughly (knobs x overlays); check the ConfigMap and overlay parse above.")
|
||||
|
||||
# --------------------------------------------------------------------------- handoff
|
||||
with (WORK / "corpus.tsv").open("w") as fh:
|
||||
for name, expect, origin, value in rows:
|
||||
@@ -459,17 +633,31 @@ with (WORK / "corpus.tsv").open("w") as fh:
|
||||
(WORK / "findings.txt").write_text("".join(f"{f}\n" for f in FAILURES))
|
||||
(WORK / "counts.txt").write_text(f"{CHECKS}\n")
|
||||
|
||||
print(f"guards: {', '.join(sorted(guard_words))} "
|
||||
f"vocabulary: {', '.join(VOCAB)}")
|
||||
print(f"guarded knobs: {', '.join(sorted(guarded_keys))} "
|
||||
f"consumers traced: {len(consumers)} overlays: {', '.join(sorted(overlays))}")
|
||||
if GUARD_REL:
|
||||
print(f"guard file: {GUARD_REL} predicates: {', '.join(sorted(guard_words))}")
|
||||
print(f"guarded knobs: {', '.join(sorted(guarded_keys))} "
|
||||
f"consumers traced: {len(consumers)}")
|
||||
else:
|
||||
print("guard file: NONE FOUND in the tree — HALF B (guard/value coupling) IS NOT PROVEN "
|
||||
"ON THIS TREE. No *_is_placeholder predicate exists to extract: bootstrap/standup.sh "
|
||||
"has never been on main (no deletion commit exists; it lives on unmerged prod-standup "
|
||||
"branches). HALF A below runs against the real authored values and is what currently "
|
||||
"prevents the 2026-09-05 outcome. When a guard lands ANYWHERE in the tree, discovery "
|
||||
"picks it up and HALF B activates with no edit to this check.")
|
||||
print(f"detector vocabulary: {', '.join(VOCAB)}")
|
||||
print(f"HALF A: overlays {', '.join(sorted(overlays))}; {len(base)} base knob(s); "
|
||||
f"(A1) {a1_rows} authored overlay value(s) screened; (A2) {a2_rows} empty value(s) "
|
||||
f"checked for explicit statement; (A3) {a3_rows} base-sentinel inheritance check(s)")
|
||||
for name, expect, origin, value in rows:
|
||||
if not origin.startswith(("anchoring:", "canary:")):
|
||||
print(f" authored {origin} = {value!r} -> expect {expect}")
|
||||
PY
|
||||
|
||||
GUARD_REL="$(tr -d '\n' < "${WORK}/mode.txt")"
|
||||
|
||||
# ---------------------------------------------------------------------------------------
|
||||
# PHASE 2 (bash): source the REAL predicate lines and run the corpus through them.
|
||||
# Empty when no guard exists — HALF A has already run and carries its own floor.
|
||||
# ---------------------------------------------------------------------------------------
|
||||
# shellcheck disable=SC1090,SC1091
|
||||
source "${WORK}/guards.sh"
|
||||
@@ -483,9 +671,9 @@ while IFS=$'\t' read -r guard expect origin value; do
|
||||
if [ "${verdict}" != "${expect}" ]; then
|
||||
oracle_failures=$((oracle_failures + 1))
|
||||
if [ "${expect}" = placeholder ]; then
|
||||
printf '%s\n' "${guard}() in ${STANDUP_REL} ACCEPTS '${value}' (from ${origin}) as a usable value, but it is placeholder-shaped. This is the 2026-09-05 failure exactly: the guard is asked about a value it does not recognise, returns 'fine', and the tier it protects is enabled against something that does not exist. Fix the predicate (match the idiom anywhere, not as a prefix) or stop authoring that idiom." >> "${WORK}/findings.txt"
|
||||
printf '%s\n' "${guard}() in ${GUARD_REL} ACCEPTS '${value}' (from ${origin}) as a usable value, but it is placeholder-shaped. This is the 2026-09-05 failure exactly: the guard is asked about a value it does not recognise, returns 'fine', and the tier it protects is enabled against something that does not exist. Fix the predicate (match the idiom anywhere, not as a prefix) or stop authoring that idiom." >> "${WORK}/findings.txt"
|
||||
else
|
||||
printf '%s\n' "${guard}() in ${STANDUP_REL} REFUSES '${value}' (from ${origin}), which is a usable value. An over-matching guard leaves the tier it protects suspended forever and reports it as an operator decision — a silent failure in the other direction." >> "${WORK}/findings.txt"
|
||||
printf '%s\n' "${guard}() in ${GUARD_REL} REFUSES '${value}' (from ${origin}), which is a usable value. An over-matching guard leaves the tier it protects suspended forever and reports it as an operator decision — a silent failure in the other direction." >> "${WORK}/findings.txt"
|
||||
fi
|
||||
fi
|
||||
done < "${WORK}/corpus.tsv"
|
||||
@@ -495,11 +683,13 @@ py_checks="$(cat "${WORK}/counts.txt")"
|
||||
total=$((py_checks + oracle_rows))
|
||||
findings="$(cat "${WORK}/findings.txt")"
|
||||
|
||||
# NO-OP GUARD: this repo has been bitten by a check that passed when perturbed because it
|
||||
# NO-OP GUARDS: this repo has been bitten by a check that passed when perturbed because it
|
||||
# asked a question that was always satisfiable. The floors below are the minimum evidence
|
||||
# for the verdict to mean anything, not a style rule.
|
||||
if [ "${oracle_rows}" -lt 6 ]; then
|
||||
findings="${findings}only ${oracle_rows} value(s) were put to the real guard — the coupling is not actually being exercised
|
||||
# for the verdict to mean anything, not a style rule. The oracle floor applies only when a
|
||||
# guard exists — when none does, HALF A's own floor (a1_rows >= 20, asserted in phase 1) is
|
||||
# what stops this from degenerating, and the absence is reported on stdout.
|
||||
if [ -n "${GUARD_REL}" ] && [ "${oracle_rows}" -lt 6 ]; then
|
||||
findings="${findings}only ${oracle_rows} value(s) were put to the real guard in ${GUARD_REL} — the coupling is not actually being exercised
|
||||
"
|
||||
fi
|
||||
if [ "${total}" -lt 20 ]; then
|
||||
@@ -513,4 +703,8 @@ if [ -n "${findings}" ]; then
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "check OK: placeholder-guard-coupling — ${total} assertions; ${oracle_rows} value(s) run through the REAL extracted predicate(s) from ${STANDUP_REL} (no reimplementation), covering every authored cluster-vars value of every guarded knob in base and each overlay, the guard's own vocabulary matched mid-string, and the empty/real-endpoint canaries. Static slice only: refusal, not reachability — see the header."
|
||||
if [ -n "${GUARD_REL}" ]; then
|
||||
echo "check OK: placeholder-guard-coupling — ${total} assertions. HALF A: no environment overlay ships a placeholder-shaped effective cluster-vars value, every empty value is stated explicitly rather than inherited, and the detector canaries hold (including the exact 2026-09-05 string, matched mid-string). HALF B: ${oracle_rows} value(s) run through the REAL extracted predicate(s) from ${GUARD_REL} (no reimplementation), covering every authored value of every guarded knob in base and each overlay, the guard's vocabulary matched mid-string, and the empty/real-endpoint canaries. Static slice only: refusal, not reachability — see the header."
|
||||
else
|
||||
echo "check OK: placeholder-guard-coupling — ${total} assertions. HALF A ONLY: no environment overlay ships a placeholder-shaped effective cluster-vars value, every empty value is stated explicitly rather than inherited, no LOUD base sentinel is inherited by omission, and the detector canaries hold (including the exact 2026-09-05 string 'https://TBD-R2-ACCOUNT-ID.r2.cloudflarestorage.com', matched mid-string). HALF B (guard/value coupling) IS NOT PROVEN: no *_is_placeholder predicate exists anywhere in the tree to extract — see the stdout note above and the header. This is a REPORTED gap, not a skip: what a guard would be asked about is proven not to be a placeholder in the first place."
|
||||
fi
|
||||
Reference in new issue
Block a user