feat(ci): substitution-safety gate + escape 6 comment hazards; strict mode is now SAFE
Adds conformance/tests/l0-static/flux-postbuild-substitution/ — an offline, stdlib-only
L0 check tagged R-ENV-02 + R-GITOPS-01 (deliberately NOT R-TEST-01, which would
overclaim). It reuses lib/status.py's `resources:` walk rather than forking it, so "what a
path renders" has exactly one definition. It asserts: every ${...} on a substituted path
is defined-or-escaped (comments included); no default-operator form on an undefined name;
no unparseable expansion; the rendering path actually substitutes cluster-vars; every
overlay-patched /data/KEY has a base default; and that the gate itself is load-bearing.
WIRED AS AN UNCONDITIONAL STAGE (3c) in ci/l0.sh, and that matters: stage 3b scopes to
touched requirements, and ci/touched-requirements.sh maps only clusters/* to R-ENV-02 —
so an edit to a substituted manifest under apps/ or platform/ would have filtered this
check out of exactly the change that introduces the bug. l0 fail -> BLOCK=1 in gate.sh.
Mutation-tested all five assertion classes (undefined token, indexed expansion, :- form,
missing substituteFrom, un-wired gate); each fails as intended, baseline green.
STRICT MODE IS PROVEN SAFE TO ENABLE — measured, not inspected. For all 11 substituted
paths, `kubectl kustomize --load-restrictor LoadRestrictionsNone <path> | flux envsubst
--strict` exits 0 against cluster-vars base defaults. Negative control confirms the
incident: with K3S_API_HOST unset, non-strict renders k8sServiceHost empty while strict
exits 1 with `variable not set (strict mode)`. The feature gate itself is NOT flipped
here — that is a live-cluster change and the operator's call.
Escapes the six comment hazards that stood in the way (apps/vms/unifi x4,
platform/longhorn x2). All were prose ABOUT the escaping convention; since the repo's own
contract is that $${...} emits a literal ${...}, escaping them is both precise and
self-consistent. kustomize strips these particular comments today, but comment survival is
an implementation detail — a body comment carrying $ WAS observed surviving into built
apps/wave-1 — and a survivor would kill the whole Kustomization.
DOCTRINE CORRECTED (measured against flux 2.4.0): clusters/base/apps-vms.yaml claimed an
indexed expansion is fatal "escaped or not". Escaping is TOTAL — $${ARR[0]} emits the
literal text safely; only the UNESCAPED indexed form is fatal. Corrected in place.
Also empties DOCUMENTED_GAPS in conformance/run.sh with rationale: R-ENV-02 and
R-GITOPS-01 now have a real verifying test, which run.sh's own comment named as the
trigger for removing them.
Advisory output (not failures): 6 dead cluster-vars knobs — BASE_DOMAIN, CLUSTER_ENV,
CLUSTER_POD_CIDR, PIA_REGION, PLATFORM_NODE_SELECTOR_{KEY,VALUE}. The last four are
documented reference-only; BASE_DOMAIN and CLUSTER_ENV look like genuine rot (hostnames
are hardcoded elsewhere) and are worth a follow-up.
Gate: MERGE GATE: PASS. 102 assertions, 0 hazards.
Serves: R-ENV-02, R-GITOPS-01.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
1 parent
3484955903
commit
8a2be824f6
8 files changed
+491
-20
No files matched your search
+15
-13
@@ -57,17 +57,15 @@ FORCE_NO_CLUSTER=0
|
||||
# Documented, out-of-scope coverage gaps exempt from the completeness gate: requirements
|
||||
# owned by another workstream that WS-TEST does not yet scaffold. Listed here so the
|
||||
# exemption is explicit and reviewable, not silent.
|
||||
# R-ENV-02 (dev->prod promote) — owned by WS-INFRA/WS-GITOPS; unscaffolded (a GAP in
|
||||
# coverage.md). The coarse path->requirement map keys
|
||||
# clusters/* AND ci/* to it and the R-ENV-02 doc token
|
||||
# appears in platform/kustomization.yaml's header, so ANY
|
||||
# infra/overlay/CI edit (e.g. enabling a component, bumping
|
||||
# SPEC_PIN) would otherwise false-block on it.
|
||||
# R-GITOPS-01 (state in git/Flux) — same: owned by WS-GITOPS, unscaffolded, and the ci/*
|
||||
# path-map keys to it, so editing any ci/ file (SPEC_PIN
|
||||
# adoption included) would false-block.
|
||||
# These are visible GAPs, not hidden — removing them from this list is the right move the
|
||||
# moment WS-TEST adds their verifying tests.
|
||||
#
|
||||
# EMPTY as of 2026-09-01. R-ENV-02 and R-GITOPS-01 were the last two entries — exempted
|
||||
# because they were unscaffolded while the coarse path->requirement map keys clusters/*
|
||||
# and ci/* to them, so any overlay/CI edit would have false-blocked. They now have a real
|
||||
# verifying L0 test (tests/l0-static/flux-postbuild-substitution), which asserts the
|
||||
# substitution mechanism that IS the "authored once, promoted" contract, so they are
|
||||
# covered rather than exempt and the completeness gate can hold them to it. That test
|
||||
# names its own limits in its header: it proves the STATIC property (no manifest renders
|
||||
# an undefined variable), not the live-cluster half of either requirement.
|
||||
#
|
||||
# FOLLOW-UP (WS-TEST) — HALF DONE. The debt was "make the matcher stop keying on doc-header
|
||||
# tokens so real requirements need not be exempted". The CONTENT-SCAN half is now fixed in
|
||||
@@ -83,7 +81,7 @@ FORCE_NO_CLUSTER=0
|
||||
# R-STATE-02 (node-loss) WAS listed here (owned by WS-STORAGE, "delivered with Longhorn/infra")
|
||||
# but now has a real verifying L3 test — tests/l3-fault-injection/node-loss — so it is no
|
||||
# longer a gap and has been removed from this list; the coverage report counts it as covered.
|
||||
DOCUMENTED_GAPS="R-ENV-02 R-GITOPS-01"
|
||||
DOCUMENTED_GAPS=""
|
||||
|
||||
usage() { sed -n '2,40p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'; }
|
||||
|
||||
@@ -108,7 +106,11 @@ log() { printf '\033[1m[run.sh]\033[0m %s\n' "$*"; }
|
||||
|
||||
# --- Coverage report is always produced (cluster-independent, R-TEST-04) --------------
|
||||
gen_coverage() {
|
||||
local flags=(--allow-gap "${DOCUMENTED_GAPS}")
|
||||
# Pass --allow-gap only when there IS an exemption: an empty DOCUMENTED_GAPS means no
|
||||
# requirement is exempt, and handing the flag an empty string would be a silent no-op
|
||||
# that reads like an exemption in the CLI log.
|
||||
local flags=()
|
||||
[ -n "${DOCUMENTED_GAPS}" ] && flags+=(--allow-gap "${DOCUMENTED_GAPS}")
|
||||
[ "${STRICT_COVERAGE}" = 1 ] && flags+=(--strict)
|
||||
# A tagged subset run is also the merge gate: the touched requirements MUST be covered.
|
||||
[ -n "${REQS}" ] && flags+=(--require "${REQS}")
|
||||
|
||||
Reference in new issue
Block a user