Files
home-network-k3s-impl/conformance/tests/l0-static/placeholder-guard-coupling/check.sh
T
jakman85andClaude Opus 5 6a8953e288 feat(prod): observability for the two failure modes that went unseen, plus OIDC self-healing
Minimal prod delivery, authored directly on main (= prod's live revision d1a6628)
rather than by merging work/prod-standup-campaign. That branch carries the DNS apex
inversion, which would have set prod's BASE_DOMAIN to home.keithjackson.dev; prod
runs external-dns with policy: sync and txtOwnerId: prod, so it would have begun
rewriting apex records dev owns while the non-k3s prod still serves 443 there.
Operator sequencing is "dev migrates first". Nothing under clusters/ is touched
here at all, which is what makes this safe to land now.

Also deliberately EXCLUDED, because prod's live cluster-vars has no CNPG_BACKUP_PATH
key and StrictPostBuildSubstitutions would fail the whole tier: the R2 rework of
platform/velero/ and the CNPG destinationPath changes. Those land as
feat/r2-backup-prod together with the endpoint and bucket knobs.

WHAT THIS FIXES

1. Nothing was watching CNPG. All three prod databases have run since bootstrap with
   ContinuousArchiving=False and firstRecoverabilityPoint=null, and no alert existed
   anywhere. Because archive_mode=on and the archive command fails, Postgres refuses
   to recycle WAL: keycloak-db holds every segment ever written (257 MiB on a 5 GiB
   volume, ~6.8 MiB/h) — roughly 29 days to a disk-full PANIC on the cluster's SSO
   database. Found by hand, 38h late. CNPGWALArchiveBacklog would have fired within
   the hour, and evaluates to 3 firing series on prod today.
   Rules key on cnpg_collector_* rather than cnpg_pg_stat_archiver_*, because
   nextcloud-db's exporter cannot log in to its own database (permission denied for
   database "nextcloud", both clusters) so the pg_stat_archiver family does not exist
   for it — the obvious metric would have been blind to one database in three.

2. No PVC outside namespace=monitoring had any fill alert. New rules are partitioned
   by namespace so they cannot double-fire with MonitoringVolumeFillingUp, pinned to
   job="kubelet" (the apiserver copy of the same series reports namespace=default),
   and guarded by inodes > 0 — block-mode PVCs such as unifi-rootdisk report
   available=0 forever and would otherwise have fired critical permanently.

3. VPNTunnelDown could never clear on a cluster with no gateway node; it has been
   firing critical on prod since 2026-09-05. Now guarded so prod self-arms the moment
   a node carries the egress-gateway label, with dev's behaviour unchanged today.

4. The Keycloak realm import fails OPEN: an unresolved placeholder is stored
   LITERALLY and the import still reports success, then sits at Done=True and never
   re-runs. On 2026-09-06 all 12 prod OIDC clients held the literal
   $(env:..._CLIENT_SECRET) string and every SSO login failed, while every signal was
   green. The CronJob is the level-triggered correction; two alerts make its silence
   observable. Repaired live already — this stops it recurring.

Also: the R-NET-04 conformance scan descended into nested agent worktrees under
.claude/, counting 4 egress policies where 1 is authored.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-09-07 00:59:22 -05:00

517 lines
28 KiB
Bash
Executable File

#!/usr/bin/env bash
# =============================================================================
# L0 static check (offline, no cluster) — placeholder-guard-coupling
# Serves: R-ENV-02 ("Same GitOps config promotes dev/test -> prod (authored once)."
# — spec docs/02-environments-and-provisioning.md)
# R-STATE-03 ("Scheduled off-cluster backups + restore (DR)."
# — spec docs/04-storage-and-state.md)
#
# WHY THIS EXISTS — A GUARD KEYED ON A VALUE THAT LIVES IN ANOTHER FILE
# --------------------------------------------------------------------
# `bootstrap/standup.sh` refuses to resume tiers whose configuration is still a
# placeholder. Three of those guards keyed on the SHAPE of `S3_BACKUP_ENDPOINT`, which is
# authored in `clusters/base/cluster-vars.yaml`. On 2026-09-05 the placeholder changed
# from `TBD-PROD-S3-BACKUP-ENDPOINT` to `https://TBD-R2-ACCOUNT-ID.r2.cloudflarestorage.com`
# (the MinIO -> Cloudflare R2 move). The guards matched the glob `TBD-*` — a PREFIX match —
# and `fnmatch('https://TBD-R2-…','TBD-*')` is FALSE. All three guards silently stopped
# matching. `standup.sh resume` would then have resumed `velero` on prod against a hostname
# that does not exist: the HelmRelease installs clean, the Kustomization reports Ready, and
# NO BACKUPS EXIST. The only reason it was not shipped is that someone noticed by accident.
#
# The predicate is fixed (`s3_is_placeholder`, substring match). The COUPLING is what was
# never tested: a guard in one file keying on a value in another, with nothing tying them
# together. This check is that tie, and it is derived from BOTH real files — no current
# string is hardcoded here, so it does not rot when the placeholder or the guard changes.
#
# HOW THE REAL GUARD IS EXERCISED (and why not a copy)
# ---------------------------------------------------
# `bootstrap/standup.sh` is NOT sourceable: at top level it parses argv, hard-exits when
# --kubeconfig/--expect-server/<step> are absent, `cd`s to the repo root, exports
# KUBECONFIG, installs an EXIT trap and finally dispatches steps against a live cluster.
# So this check EXTRACTS the predicate's own source lines from the file and sources ONLY
# those, then calls the real function. That is the real bytes, not a reimplementation:
# * every function whose name ends `_is_placeholder` is discovered, not named here;
# * a duplicate definition of one is a failure (which of the two would run?);
# * the extracted body must be a SELF-CONTAINED predicate — `case`/`return` and nothing
# else, no command substitution, no call to another function. If the guard ever grows
# a dependency, extraction stops being faithful and THIS CHECK FAILS LOUDLY, naming the
# remedy (move the predicate into a small sourceable library that standup.sh sources)
# rather than quietly falling back to testing a copy of the logic;
# * a CANARY pins the extraction itself: the extracted function must refuse the empty
# string and accept a realistic endpoint. An extraction that grabbed the wrong lines,
# or a predicate degraded to always-true / always-false, fails there — this repo has
# already been bitten by a check that passed when perturbed because it asked a question
# that was always satisfiable.
# There is deliberately NO second implementation of the matching logic anywhere in this file.
#
# THE INDEPENDENT DETECTOR (the other half of the comparison)
# ----------------------------------------------------------
# Asking "does the guard agree with the guard?" proves nothing, so the set of
# placeholder-shaped values is decided by a detector written HERE, whose vocabulary is the
# UNION of two independent sources:
# * SEED — universal loud-placeholder idioms (TBD, TODO, FIXME, PLACEHOLDER, REPLACE-ME,
# CHANGE-ME, FILL-ME, SET-ME, XXXX), plus `<angle-bracket>` templates and the empty
# string. These are NOT derived from the guard, so DELETING a guard arm cannot make a
# value stop being placeholder-shaped — it makes assertion (1) fail, which is the point.
# A value using an idiom the guard does not know is a finding either way: fix it by
# giving the knob a real value, or by teaching the guard the idiom.
# * the guard's OWN vocabulary, read out of its case-arms with the glob anchors STRIPPED.
# Stripping the anchors is what makes the comparison non-circular in the dimension that
# actually broke: the detector matches a word ANYWHERE, case-insensitively, and the
# guard must agree. A prefix-anchored guard cannot pass.
#
# GIVEN the repository as authored,
# WHEN the real predicate(s) are extracted from bootstrap/standup.sh and executed,
# THEN (1) VALUES: for every knob a guard is responsible for, every authored value — the
# base default AND each environment overlay's value — that the independent
# detector calls placeholder-shaped is REFUSED by the real guard, and every value
# it calls usable is ACCEPTED (a guard that refuses a real endpoint leaves velero
# suspended forever, which is the same class of silent failure);
# (2) OVERLAYS: no environment overlay's EFFECTIVE cluster-vars carries a
# placeholder-shaped value for a knob that no guard refuses. (Base alone is
# exempt: it is not a bootstrap target — `flux bootstrap --path=clusters/<env>` —
# and its loud sentinels exist precisely to be overridden. A base placeholder an
# overlay forgets to override IS caught, because it becomes that overlay's
# effective value.)
# (3) CONSUMERS: every function in standup.sh that reads a guarded knob (via `cvar`,
# following one hop of variable aliasing) applies a placeholder guard to it. This
# is the "all three guards died at once" regression stated as a property: the
# guard being correct is worthless if a consumer stops calling it.
# (4) ANCHORING: for each word in the guard's own vocabulary, the guard refuses a
# value carrying that word in the MIDDLE (not at the start) — the precise defect
# of 2026-09-05 — while still accepting a realistic endpoint.
# (5) HARNESS: exactly one definition per predicate, the extracted body is verbatim
# from the file and self-contained, the canary holds, no divergent second copy of
# the matching logic exists elsewhere in standup.sh, and a floor on the number of
# assertions run keeps a pass from degenerating into a no-op.
# AND (6) LOAD-BEARING: ci/l0.sh runs the L0 conformance layer on every gate run and
# ci/gate.sh sets the merge block when that stage fails.
#
# WHAT THIS DOES *NOT* PROVE (honest gaps)
# - Nothing about RUNTIME: not that the endpoint resolves, not that credentials work, not
# that a backup exists. `standup.sh s3check` probes reachability from a pod (operator-
# run, cluster-gated) and conformance/tests/l3-fault-injection/dr-restore proves restore.
# - It does not execute standup.sh's dispatch path, so it cannot prove the guard is
# REACHED in a real run — only that, when reached, it refuses what it must. The
# consumer property (3) is a static approximation of reachability: it is a text-level
# taint trace over one file, not an interpreter, and a guard whose result is computed
# and then ignored would satisfy it.
# - It says nothing about guards keyed on shapes OTHER than placeholders (e.g. the
# EXTERNALDNS_TXT_OWNER_ID == 'default' test), nor about placeholder-shaped values
# outside cluster-vars (the Ansible inventory's BLOCKED/NOT-A-DEVICE sentinels are
# asserted by the node_baseline role itself, at play time).
# - Stage 3b of ci/l0.sh scopes the conformance run to the TOUCHED requirements. An edit
# to bootstrap/* attributes NO requirement in ci/touched-requirements.sh's path map, so
# the touched set is empty and the FULL L0 layer runs (verified: ci/lib.sh
# load_touched_requirements) — this check runs. An edit to clusters/* maps to
# R-ENV-02, which this check carries, so a cluster-vars value change also selects it.
# A change that touches standup.sh ALONGSIDE some unrelated mapped path (say
# platform/cilium/) would scope the run away from this check; closing that needs a
# `bootstrap/*` entry in the path map, which is not this check's file to edit.
# =============================================================================
set -euo pipefail
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT="$(cd "${DIR}/../../../.." && pwd)" # repo root
PYTHON="${PYTHON:-python3}"
STANDUP_REL="${STANDUP_REL:-bootstrap/standup.sh}"
cd "${ROOT}"
if [ ! -f "${ROOT}/${STANDUP_REL}" ]; then
echo "check SKIPPED (placeholder-guard-coupling): ${STANDUP_REL} is absent — nothing to couple." >&2
exit 3
fi
WORK="$(mktemp -d)"
trap 'rm -rf "${WORK}"' EXIT
# ---------------------------------------------------------------------------------------
# PHASE 1 (python): extract the real predicate(s), derive the corpus from the real files,
# and run every assertion that does not need a shell. Writes:
# ${WORK}/guards.sh the predicate source lines, verbatim, and nothing else
# ${WORK}/corpus.tsv <guard>\t<expect: placeholder|usable>\t<origin>\t<value>
# ${WORK}/findings.txt one line per failed assertion
# ${WORK}/counts.txt <assertions-run>
# ---------------------------------------------------------------------------------------
"${PYTHON}" - "${ROOT}" "${WORK}" "${STANDUP_REL}" <<'PY'
import re
import sys
from pathlib import Path
try:
import yaml
except ImportError: # pragma: no cover
print("check FAIL: PyYAML is required (same dependency as catalog/render.py)",
file=sys.stderr)
raise SystemExit(1)
ROOT, WORK, STANDUP_REL = Path(sys.argv[1]), Path(sys.argv[2]), sys.argv[3]
FAILURES: list[str] = []
CHECKS = 0
def check(cond, msg):
global CHECKS
CHECKS += 1
if not cond:
FAILURES.append(msg)
def bail(msg):
"""A failure that makes the rest of the check meaningless: report and stop."""
print(f"CHECK FAIL (placeholder-guard-coupling): {msg}", file=sys.stderr)
(WORK / "findings.txt").write_text(msg + "\n")
(WORK / "counts.txt").write_text("0\n")
(WORK / "corpus.tsv").write_text("")
(WORK / "guards.sh").write_text("")
raise SystemExit(1)
TEXT = (ROOT / STANDUP_REL).read_text()
LINES = TEXT.splitlines()
# ------------------------------------------------------------------ (5) extract the guard
DEF_RE = re.compile(r"^([a-z0-9_]+_is_placeholder)\(\)\s*\{")
defs: dict[str, list[int]] = {}
for i, line in enumerate(LINES):
m = DEF_RE.match(line)
if m:
defs.setdefault(m.group(1), []).append(i)
if not defs:
bail(f"{STANDUP_REL} defines no *_is_placeholder predicate. Either the guard was renamed "
"(update this check's discovery pattern in the SAME change) or it was DELETED, which "
"is the 2026-09-05 failure with the guard removed instead of disarmed.")
# An "argument-shaped" call: <name> "<something>"
CALL_RE = re.compile(r"\b([a-z0-9_]+_is_placeholder)\s+\"([^\"]*)\"")
guard_src: list[str] = []
guard_words: dict[str, list[str]] = {}
ARM_RE = re.compile(r"^(?P<pats>[^()]*?)\)\s*return\s+[01]\s*;;$")
ALLOWED = (
re.compile(r"^$"),
re.compile(r"^#"),
re.compile(r"^case\s+\"\$\{?[0-9A-Za-z_]+\}?\"\s+in$"),
re.compile(r"^esac$"),
re.compile(r"^return\s+[01]$"),
ARM_RE,
)
for name, at in sorted(defs.items()):
check(len(at) == 1,
f"{STANDUP_REL} defines {name}() {len(at)} times (lines "
f"{', '.join(str(a + 1) for a in at)}) — which one runs depends on file order, and "
"this check can no longer say which one it extracted. Keep exactly one definition.")
if len(at) != 1:
continue
start = at[0]
end = next((j for j in range(start + 1, len(LINES)) if LINES[j] == "}"), None)
if end is None:
bail(f"{name}() at {STANDUP_REL}:{start + 1} has no closing brace at column 0 — the "
"extraction boundary this check relies on is gone. Restore the conventional "
"layout, or move the predicate into a sourceable library and point this check at it.")
body = LINES[start:end + 1]
block = "\n".join(body)
check(block in TEXT,
f"extracted {name}() is not a verbatim substring of {STANDUP_REL} — extraction bug; "
"the harness would be testing something the script does not contain")
words: list[str] = []
for raw in body[1:-1]:
s = raw.strip()
if not any(rx.match(s) for rx in ALLOWED) or "$(" in s or "`" in s:
bail(f"{name}() at {STANDUP_REL}:{start + 1} is no longer a SELF-CONTAINED "
f"case/return predicate (offending line: {s!r}). This check exercises the REAL "
"function by sourcing its own lines; a body that calls out to anything else "
"cannot be extracted faithfully, and silently testing a COPY of the logic is "
"exactly the failure mode this check exists to prevent. REMEDY: move the "
"predicate into a tiny library that standup.sh sources (e.g. "
"bootstrap/lib/placeholder.sh) and point STANDUP_REL/this extractor at it — do "
"NOT reimplement the matching here.")
m = ARM_RE.match(s)
if not m:
continue
for pat in m.group("pats").split("|"):
pat = pat.strip()
if pat in ('""', "''", ""):
continue # the empty-value arm: covered by the canary
words.append(pat.strip("'\"").strip("*"))
check(len(words) >= 1,
f"{name}() has no non-empty placeholder pattern left — every arm that recognises a "
"loud placeholder string has been removed, so the guard now only refuses the empty "
"value and any 'TBD'-style endpoint would sail straight through it")
guard_words[name] = words
guard_src.append(block)
(WORK / "guards.sh").write_text("\n".join(guard_src) + "\n")
# ------------------------------------------------- no divergent second copy of the logic
# Any of the guard's own vocabulary appearing in EXECUTABLE text outside the predicate means
# a second, independently-drifting implementation of the same test (which is how three
# guards came to disagree with one value in the first place).
guard_line_nums = {j for name, at in defs.items() if len(at) == 1
for j in range(at[0], next(k for k in range(at[0] + 1, len(LINES))
if LINES[k] == "}") + 1)}
all_words = sorted({w.upper() for ws in guard_words.values() for w in ws if w})
divergent: list[str] = []
for j, line in enumerate(LINES):
if j in guard_line_nums or line.strip().startswith("#"):
continue
code = line.split("#", 1)[0]
hit = [w for w in all_words if w in code.upper()]
if hit:
divergent.append(f"{STANDUP_REL}:{j + 1} ({', '.join(hit)}): {line.strip()!r}")
check(not divergent,
"the guard's own placeholder vocabulary appears in EXECUTABLE text outside the "
f"predicate — a SECOND implementation of the same test, which is precisely how a value "
f"change disarmed three guards at once. Call the predicate instead. Sites: "
+ " | ".join(divergent))
# ------------------------------------------------------------- cluster-vars, as authored
CV = ROOT / "clusters" / "base" / "cluster-vars.yaml"
if not CV.is_file():
bail("clusters/base/cluster-vars.yaml is absent — the authored half of the coupling is "
"missing and this check cannot mean anything")
base: dict[str, str] = {}
for d in yaml.safe_load_all(CV.read_text()):
if isinstance(d, dict) and d.get("kind") == "ConfigMap" \
and (d.get("metadata") or {}).get("name") == "cluster-vars":
base = {str(k): "" if v is None else str(v) for k, v in (d.get("data") or {}).items()}
check(len(base) > 5, f"clusters/base/cluster-vars.yaml yielded {len(base)} knob(s) — the "
"ConfigMap parse failed, so every value assertion below would be vacuous")
overlays: dict[str, dict[str, str]] = {}
for overlay in sorted((ROOT / "clusters").glob("*/kustomization.yaml")):
env = overlay.parent.name
if env == "base":
continue
eff = dict(base)
try:
k = yaml.safe_load(overlay.read_text()) or {}
except Exception as exc: # noqa: BLE001
check(False, f"clusters/{env}/kustomization.yaml does not parse as YAML ({exc}) — this "
"check cannot compute that overlay's effective values")
continue
for p in (k.get("patches") or []):
tgt = p.get("target") or {}
if tgt.get("kind") != "ConfigMap" or tgt.get("name") != "cluster-vars":
continue
try:
ops = yaml.safe_load(p.get("patch") or "") or []
except Exception: # noqa: BLE001
continue
for op in ops if isinstance(ops, list) else []:
if not isinstance(op, dict):
continue
m = re.match(r"^/data/([A-Za-z0-9_]+)$", str(op.get("path", "")))
if m and str(op.get("op", "")) in ("replace", "add"):
eff[m.group(1)] = "" if op.get("value") is None else str(op["value"])
overlays[env] = eff
check(len(overlays) >= 1,
"no clusters/<env>/kustomization.yaml overlay was parsed — assertion (2) would be vacuous")
# ------------------------------------------------ which knobs is a guard responsible for?
# Function bodies of standup.sh, then a one-hop taint trace from `x="$(cvar KEY)"` to a
# `*_is_placeholder "${x}"` call. Both halves are read out of the file; nothing is named here.
FUNC_RE = re.compile(r"^([a-z0-9_]+)\(\)\s*\{")
funcs: dict[str, list[str]] = {}
i = 0
while i < len(LINES):
m = FUNC_RE.match(LINES[i])
if m:
end = next((j for j in range(i + 1, len(LINES)) if LINES[j] == "}"), len(LINES) - 1)
funcs[m.group(1)] = LINES[i + 1:end]
i = end + 1
continue
i += 1
check(len(funcs) > 5, f"parsed only {len(funcs)} function(s) out of {STANDUP_REL} — the "
"consumer assertion (3) would be vacuous")
CVAR_RE = re.compile(r"([A-Za-z_][A-Za-z0-9_]*)=\"\$\(\s*cvar\s+([A-Z0-9_]+)\s*\)\"")
ALIAS_RE = re.compile(r"([A-Za-z_][A-Za-z0-9_]*)=\"\$\{([A-Za-z_][A-Za-z0-9_]*)[:}]")
guarded_keys: set[str] = set()
consumers: list[tuple[str, str, bool]] = [] # (func, key, guarded-in-this-func)
for fname, body in funcs.items():
text = "\n".join(body)
keys_of_var: dict[str, set[str]] = {}
for m in CVAR_RE.finditer(text):
keys_of_var.setdefault(m.group(1), set()).add(m.group(2))
# one hop of aliasing: ep="${live}" carries live's taint
for _ in range(3):
for m in ALIAS_RE.finditer(text):
src = keys_of_var.get(m.group(2))
if src:
keys_of_var.setdefault(m.group(1), set()).update(src)
called_on = {m.group(2) for m in CALL_RE.finditer(text)}
called_vars = {re.sub(r"^\$\{?|\}?$", "", a) for a in called_on}
# Aggregate per (function, knob): the knob is guarded HERE if ANY variable carrying its
# value is passed to a predicate — `live="$(cvar K)"; ep="${live}"; guard "${ep}"` counts.
guarded_here: dict[str, bool] = {}
for var, keys in keys_of_var.items():
for key in keys:
guarded_here[key] = guarded_here.get(key, False) or (var in called_vars)
for key, g in guarded_here.items():
consumers.append((fname, key, g))
if g:
guarded_keys.add(key)
if not guarded_keys:
bail(f"no cluster-vars knob in {STANDUP_REL} is passed to a *_is_placeholder predicate. "
"Either the guards were removed, or the way the script reads cluster-vars changed and "
"this check's derivation must be updated in the SAME change — it must never silently "
"verify an empty set.")
# (3) CONSUMERS: every reader of a guarded knob must apply the guard to it.
for fname, key, guarded in sorted(consumers):
if key not in guarded_keys:
continue # nothing claims responsibility for this knob at all
check(guarded,
f"{STANDUP_REL}: {fname}() reads cluster-vars knob {key} but never passes it to a "
"*_is_placeholder guard, while other functions do. On 2026-09-05 all three consumers "
"of S3_BACKUP_ENDPOINT were disarmed at once; a consumer that reads the knob and skips "
"the guard is that failure in one function.")
# ---------------------------------------------------------------- independent detector
# Deliberately NOT derived from the guard alone (see the header): SEED is universal
# loud-placeholder vocabulary, and the guard's own words are added so that a guard whose
# ANCHORING is wrong still has its vocabulary tested against real values.
SEED = ["TBD", "TODO", "FIXME", "PLACEHOLDER", "REPLACE-ME", "REPLACEME",
"CHANGE-ME", "CHANGEME", "FILL-ME", "FILLME", "SET-ME", "XXXX"]
VOCAB = sorted({w.upper() for w in SEED} | set(all_words))
ANGLE = re.compile(r"<[^>\s]+>")
def placeholder_shaped(value: str) -> bool:
if value.strip() == "":
return True # an unset endpoint is not a target
up = value.upper()
return any(w in up for w in VOCAB) or bool(ANGLE.search(value))
rows: list[tuple[str, str, str, str]] = [] # guard, expect, origin, value
GUARD = sorted(guard_words)[0] if len(guard_words) == 1 else None
# (1) VALUES: base default and every overlay's value, for every guarded knob.
for key in sorted(guarded_keys):
seen: list[tuple[str, str]] = []
if key in base:
seen.append((f"clusters/base/cluster-vars.yaml:{key}", base[key]))
for env, eff in sorted(overlays.items()):
seen.append((f"clusters/{env} effective:{key}", eff.get(key, "")))
check(bool(seen), f"guarded knob {key} is not defined in cluster-vars at all — the guard "
"protects a knob nothing authors")
for origin, value in seen:
expect = "placeholder" if placeholder_shaped(value) else "usable"
for name in sorted(guard_words):
rows.append((name, expect, origin, value))
# (2) OVERLAYS: a placeholder-shaped effective value for an UNGUARDED knob.
for env, eff in sorted(overlays.items()):
for key, value in sorted(eff.items()):
if key in guarded_keys or not placeholder_shaped(value):
continue
check(False,
f"clusters/{env}: knob {key} resolves to the placeholder-shaped value {value!r}, "
f"and NO guard in {STANDUP_REL} refuses it. Either give it a real value before it "
"reaches an overlay, or guard it the way the S3 endpoint is guarded — an overlay "
"that ships a placeholder no guard recognises is the 2026-09-05 failure with a "
"different knob's name on it.")
# (4) ANCHORING: the guard's own vocabulary must match ANYWHERE, not just as a prefix,
# and a realistic endpoint must still be accepted.
for name, words in sorted(guard_words.items()):
for w in sorted(set(words)):
rows.append((name, "placeholder",
f"anchoring:{name}:mid-string({w})",
f"https://account-{w}-id.r2.cloudflarestorage.com"))
rows.append((name, "placeholder", f"anchoring:{name}:tail({w})",
f"https://r2-endpoint.example.invalid/{w}"))
rows.append((name, "placeholder", f"canary:{name}:empty", ""))
rows.append((name, "usable", f"canary:{name}:real-r2",
"https://0123456789abcdef0123456789abcd.r2.cloudflarestorage.com"))
rows.append((name, "usable", f"canary:{name}:real-minio", "http://192.168.1.9:9000"))
# Every guarded knob must reach the oracle once per authored source (base + each overlay),
# so a knob silently dropping out of the corpus is a failure rather than a quiet pass.
authored = [r for r in rows if not r[2].startswith(("anchoring:", "canary:"))]
expected_rows = len(guarded_keys) * (1 + len(overlays)) * len(guard_words)
check(len(authored) == expected_rows,
f"{len(authored)} authored value(s) reached the guard, expected {expected_rows} "
f"({len(guarded_keys)} guarded knob(s) x base+{len(overlays)} overlay(s) x "
f"{len(guard_words)} predicate(s)) — a guarded knob is missing from cluster-vars or from "
"an overlay's effective values, and the coupling is only partly exercised")
# ------------------------------------------------------- (6) the check is load-bearing
L0 = (ROOT / "ci" / "l0.sh").read_text() if (ROOT / "ci" / "l0.sh").is_file() else ""
GATE = (ROOT / "ci" / "gate.sh").read_text() if (ROOT / "ci" / "gate.sh").is_file() else ""
check("--layer l0" in L0,
"ci/l0.sh does not run the L0 conformance layer — this check would never execute in the gate")
check(bool(re.search(r"^for s in .*\bl0\b.*; do", GATE, re.M)) and bool(re.search(r"fail\).*BLOCK=1", GATE)),
"ci/gate.sh does not set the merge block when the l0 stage fails")
# --------------------------------------------------------------------------- handoff
with (WORK / "corpus.tsv").open("w") as fh:
for name, expect, origin, value in rows:
fh.write(f"{name}\t{expect}\t{origin}\t{value}\n")
(WORK / "findings.txt").write_text("".join(f"{f}\n" for f in FAILURES))
(WORK / "counts.txt").write_text(f"{CHECKS}\n")
print(f"guards: {', '.join(sorted(guard_words))} "
f"vocabulary: {', '.join(VOCAB)}")
print(f"guarded knobs: {', '.join(sorted(guarded_keys))} "
f"consumers traced: {len(consumers)} overlays: {', '.join(sorted(overlays))}")
for name, expect, origin, value in rows:
if not origin.startswith(("anchoring:", "canary:")):
print(f" authored {origin} = {value!r} -> expect {expect}")
PY
# ---------------------------------------------------------------------------------------
# PHASE 2 (bash): source the REAL predicate lines and run the corpus through them.
# ---------------------------------------------------------------------------------------
# shellcheck disable=SC1090,SC1091
source "${WORK}/guards.sh"
oracle_failures=0
oracle_rows=0
while IFS=$'\t' read -r guard expect origin value; do
[ -n "${guard}" ] || continue
oracle_rows=$((oracle_rows + 1))
if "${guard}" "${value}"; then verdict=placeholder; else verdict=usable; fi
if [ "${verdict}" != "${expect}" ]; then
oracle_failures=$((oracle_failures + 1))
if [ "${expect}" = placeholder ]; then
printf '%s\n' "${guard}() in ${STANDUP_REL} ACCEPTS '${value}' (from ${origin}) as a usable value, but it is placeholder-shaped. This is the 2026-09-05 failure exactly: the guard is asked about a value it does not recognise, returns 'fine', and the tier it protects is enabled against something that does not exist. Fix the predicate (match the idiom anywhere, not as a prefix) or stop authoring that idiom." >> "${WORK}/findings.txt"
else
printf '%s\n' "${guard}() in ${STANDUP_REL} REFUSES '${value}' (from ${origin}), which is a usable value. An over-matching guard leaves the tier it protects suspended forever and reports it as an operator decision — a silent failure in the other direction." >> "${WORK}/findings.txt"
fi
fi
done < "${WORK}/corpus.tsv"
# ------------------------------------------------------------------------------ verdict
py_checks="$(cat "${WORK}/counts.txt")"
total=$((py_checks + oracle_rows))
findings="$(cat "${WORK}/findings.txt")"
# NO-OP GUARD: this repo has been bitten by a check that passed when perturbed because it
# asked a question that was always satisfiable. The floors below are the minimum evidence
# for the verdict to mean anything, not a style rule.
if [ "${oracle_rows}" -lt 6 ]; then
findings="${findings}only ${oracle_rows} value(s) were put to the real guard — the coupling is not actually being exercised
"
fi
if [ "${total}" -lt 20 ]; then
findings="${findings}only ${total} assertions ran — the check degenerated to a near-no-op and its pass would be meaningless
"
fi
if [ -n "${findings}" ]; then
echo "CHECK FAIL (placeholder-guard-coupling): $(printf '%s' "${findings}" | grep -c . ) of ${total} assertions failed" >&2
printf '%s' "${findings}" | sed 's/^/ - /' >&2
exit 1
fi
echo "check OK: placeholder-guard-coupling — ${total} assertions; ${oracle_rows} value(s) run through the REAL extracted predicate(s) from ${STANDUP_REL} (no reimplementation), covering every authored cluster-vars value of every guarded knob in base and each overlay, the guard's own vocabulary matched mid-string, and the empty/real-endpoint canaries. Static slice only: refusal, not reachability — see the header."