diff --git a/ci/Makefile b/ci/Makefile
index 632926f..3d426da 100644
--- a/ci/Makefile
+++ b/ci/Makefile
@@ -44,7 +44,11 @@ touched: ## Print the touched-requirement subset for the current branch vs main
@"$(CI_DIR)touched-requirements.sh"
test: ## Run ci/ script unit tests (no cluster; e.g. touched-requirements content scan)
- @for t in "$(CI_DIR)"tests/*.test.sh; do echo "== $$t"; bash "$$t"; done
+ @rc=0; for t in "$(CI_DIR)"tests/*.test.sh; do \
+ echo "== $$t"; bash "$$t" || rc=1; \
+ done; \
+ if [ $$rc -ne 0 ]; then echo "ci/tests: FAILED" >&2; else echo "ci/tests: all suites passed"; fi; \
+ exit $$rc
versions: ## Print pinned tool versions (R-TEST-02)
@cat "$(CI_DIR)versions.env"
diff --git a/ci/README.md b/ci/README.md
index eac3253..bec4949 100644
--- a/ci/README.md
+++ b/ci/README.md
@@ -20,17 +20,54 @@ Individual stages:
```bash
make -C ci lint # yaml/lint (no cluster; live now)
-make -C ci l0 # ServiceDefinition schema + Kyverno (no cluster; live now)
+make -C ci l0 # schema + Kyverno + L0 static conformance (no cluster; live now)
make -C ci l1 # L1 rendering checks (no cluster; live now)
make -C ci conformance # L2/L3 in-cluster Chainsaw Job (SKIPS unless CLUSTER_AVAILABLE=true)
+make -C ci test # unit tests for these scripts themselves (ci/tests/*.test.sh)
```
+The `l0` stage has three sub-stages: ServiceDefinition schema validation, the Kyverno policy gate
+(`kyverno test`), and **`conformance/run.sh --layer l0`** — the whole-repo L0 invariants a
+per-resource policy cannot express (secret hygiene across every tracked file, least-privilege RBAC,
+spec-immutability enforcement). See `conformance/tests/README.md`.
+
## Exit-code contract
Every stage script uses: **`0` = pass · `1` = fail (blocks merge) · `3` = SKIPPED** (cluster-gated
with no cluster, or an unwired seam / missing tool). A `3` is **never** counted as green — that is how
the gate stays honest in Phase 0 (no fake green).
+### The touched-requirement subset can also be *undeterminable* (exit 3)
+
+`touched-requirements.sh` is the input to the R-TEST-03 decision. It resolves a diff base in this
+order, **verifying each candidate exists** before using it:
+
+```
+BASE_REF (explicit) > GITHUB_BASE_REF (Actions PR runs) > origin/main > main
+```
+
+If no candidate resolves, if there is no merge base, if the diff command fails, or if **HEAD is not
+ahead of the base** (you are sitting on `main`, or the branch is already merged), it prints a loud
+banner, writes **nothing** to stdout, and exits **3** — plus a machine-readable reason code
+(`on-base` / `no-base` / `diff-failed`) to `$TOUCHED_REASON_FILE` when that variable names a path.
+
+**Why this is not cosmetic.** Every one of those conditions used to degrade *silently* to an empty
+touched set and exit 0. An empty set means `conformance/run.sh` is invoked with no `--requirements`,
+which means its coverage `--require` gate never runs — so on 2026-08-31 the local gate printed
+`MERGE GATE: PASS` having checked nothing, while CI (correct base, 17 touched requirements) BLOCKED
+on real coverage debt. There is no silent narrowing left: the `HEAD~1` last resort and the two-dot
+diff fallback are both gone. Locked down by `ci/tests/touched-requirements-base.test.sh`.
+
+`gate.sh` turns exit 3 into an **UNDECIDED** merge decision, never a pass:
+
+| reason | headline | exit |
+|---|---|---|
+| `on-base` | `MERGE GATE: NOT APPLICABLE — no branch under gate` | 0 (a push-to-`main` CI run legitimately lands here; it just must not read as "my branch is green") |
+| `no-base`, `diff-failed` | `MERGE GATE: BLOCKED` | 1 (the environment cannot support a merge decision; refusing to decide must not look like deciding yes) |
+
+A stage that needs the subset (`l0.sh`, `l1.sh`) still runs its FULL layer as a superset sanity net
+in that case — nothing goes untested — and says plainly that the *scoped* check did not happen.
+
## Reproducibility (R-TEST-02)
- **Version-pinned:** all tool versions live in [`versions.env`](versions.env) — the single source of
@@ -43,16 +80,19 @@ the gate stays honest in Phase 0 (no fake green).
```
ci/
- Makefile targets: lint · l0 · l1 · conformance · gate · touched · tools · versions
+ Makefile targets: lint · l0 · l1 · conformance · gate · touched · test · tools · versions
versions.env PINNED tool versions (R-TEST-02) — single source of truth
lib.sh shared helpers: logging, tool checks, cluster detection, artifact paths
yamllint.yaml lint config for the yaml/lint stage
lint.sh Stage: yaml/lint
- l0.sh Stage: ServiceDefinition schema validation + Kyverno L0 policy gate
+ l0.sh Stage: schema validation + Kyverno L0 gate + L0 static conformance
l1.sh Stage: L1 rendering checks (Chainsaw L1 via the conformance runner)
conformance.sh Stage: L2/L3 in-cluster Chainsaw Job (trigger + collect)
- touched-requirements.sh compute the R-* subset a change touches (drives the R-TEST-03 filter)
+ touched-requirements.sh compute the R-* subset a change touches (drives the R-TEST-03 filter);
+ exits 3 LOUDLY rather than guessing when the diff base is unreliable
gate.sh run all stages + the merge decision (the single command)
+ tests/*.test.sh unit tests for these scripts (`make -C ci test`), run against
+ throwaway git repos via the REPO_ROOT override — no network, no cluster
README.md this file
```
@@ -67,7 +107,7 @@ below are this workstream's **assumptions** — flagged for the PM to verify at
|---|---|---|---|---|
| ServiceDefinition schema validator | WS-GITOPS · **T-0.3** (`schemas/`) | `SCHEMA_VALIDATE_CMD` | `schemas/validate.sh` | `validate.sh catalog/*.yaml` → exit 0/1; `--version` supported |
| Kyverno L0 policies + tests | WS-TEST · **T-0.5** (`platform/kyverno/`) | `KYVERNO_POLICY_DIR` | `platform/kyverno` | contains `kyverno-test.yaml` files discoverable by `kyverno test
` |
-| Conformance runner (Chainsaw) | WS-TEST · **T-0.4** (`conformance/`) | `CONFORMANCE_CMD` | `conformance/run.sh` | `run.sh --layer [--requirements R-A,R-B] [--trigger in-cluster-job] --junit --coverage ` |
+| Conformance runner (Chainsaw) | WS-TEST · **T-0.4** (`conformance/`) | `CONFORMANCE_CMD` | `conformance/run.sh` | `run.sh --layer [--requirements R-A,R-B]`; artifacts land in `conformance/_artifacts/` |
| Catalog (ServiceDefinitions) | WS-APPS | `CATALOG_DIR` | `catalog` | `*.yaml` ServiceDefinition files |
Artifact locations (git-ignored): JUnit → `results/junit/*.xml`; coverage → `reports/coverage/**`.
diff --git a/ci/gate.sh b/ci/gate.sh
index 3a54ba4..8df27b4 100755
--- a/ci/gate.sh
+++ b/ci/gate.sh
@@ -48,6 +48,32 @@ echo >&2
log "==== gate decision ===="
BLOCK=0
+# --- R-TEST-03 input: the touched-requirement subset --------------------------
+# Computed ONCE here. ci/touched-requirements.sh exits 3 when it cannot establish a
+# reliable diff base; that used to degrade to an empty set, which silently disabled the
+# coverage `--require` gate and let the gate print "MERGE GATE: PASS" having checked
+# nothing (the 2026-08-31 local false green). An undeterminable subset is now surfaced
+# as an UNDECIDED merge decision, never as a pass.
+_reason_file="$(mktemp)"
+# NOTE: command substitution, NOT `mapfile < <(...)` — process substitution does not
+# propagate the child's exit status, and the whole point here is to SEE the exit 3.
+TOUCHED_OUT="$(TOUCHED_REASON_FILE="$_reason_file" "${CI_DIR}/touched-requirements.sh")"
+TOUCHED_RC=$?
+TOUCHED_REASON="$(cat "$_reason_file" 2>/dev/null || true)"
+rm -f "$_reason_file"
+TOUCHED=()
+[ -n "$TOUCHED_OUT" ] && mapfile -t TOUCHED <<< "$TOUCHED_OUT"
+UNDECIDED=0
+if [ "$TOUCHED_RC" -eq 3 ]; then
+ UNDECIDED=1
+ echo >&2
+ err "R-TEST-03 DECISION INPUT MISSING: the touched-requirement subset could not be"
+ err " determined (${TOUCHED_REASON:-unknown}). The scoped coverage gate did NOT run."
+ summary "- **R-TEST-03 subset UNDETERMINED** (\`${TOUCHED_REASON:-unknown}\`) — the scoped coverage gate did not run."
+elif [ "${#TOUCHED[@]}" -gt 0 ]; then
+ log "touched requirements: ${TOUCHED[*]}"
+fi
+
# spec-immutability (R-TEST-06): a FAIL always blocks (the spec was modified/drifted).
# A skip means the spec repo wasn't available to verify — surfaced, not green.
case "${OUTCOME[spec-immutability]}" in
@@ -77,9 +103,11 @@ case "${OUTCOME[conformance]}" in
skip)
if cluster_available; then
err "conformance: skipped despite CLUSTER_AVAILABLE=true -> blocks merge"; BLOCK=1
+ elif [ "$UNDECIDED" -eq 1 ]; then
+ warn "conformance L2/L3 UNVERIFIED, and the touched subset is unknown — nothing certified."
else
- mapfile -t reqs < <("${CI_DIR}/touched-requirements.sh" || true)
- if [ "${#reqs[@]}" -gt 0 ]; then
+ if [ "${#TOUCHED[@]}" -gt 0 ]; then
+ reqs=("${TOUCHED[@]}")
warn "conformance L2/L3 UNVERIFIED for touched requirements: ${reqs[*]}"
warn " (no cluster — Phase 0). These are NOT certified green."
summary "- **UNVERIFIED (no cluster):** ${reqs[*]}"
@@ -93,15 +121,35 @@ case "${OUTCOME[conformance]}" in
fi ;;
esac
+# An UNDECIDED subset is never a pass. Two outcomes, by reason:
+# on-base -> this checkout has no branch commits to gate (you are on main, or the
+# branch is already merged; a push-to-main CI run lands here). Legitimate,
+# so the job is not failed — but the headline says NOT APPLICABLE, never
+# PASS, so nobody reads it as "my branch is green".
+# anything -> the environment cannot support a merge decision (no base ref, broken
+# else diff). Refusing to decide must not look like deciding yes: BLOCK.
+if [ "$UNDECIDED" -eq 1 ] && [ "$TOUCHED_REASON" != "on-base" ]; then
+ err "cannot compute the R-TEST-03 subset (${TOUCHED_REASON:-unknown}) -> blocks merge"
+ BLOCK=1
+fi
+
echo >&2
-if [ "$BLOCK" -eq 0 ]; then
- ok "MERGE GATE: PASS (touched-requirement subset green for the layers that can run)"
- summary ""
- summary "**Result: PASS** — L0/L1 green. L2/L3 status per cluster availability above."
- exit 0
-else
+if [ "$BLOCK" -ne 0 ]; then
err "MERGE GATE: BLOCKED"
summary ""
summary "**Result: BLOCKED** — see failures above."
exit 1
+elif [ "$UNDECIDED" -eq 1 ]; then
+ warn "MERGE GATE: NOT APPLICABLE — no branch under gate (${TOUCHED_REASON})."
+ warn " The stages above ran and did not fail, but NO touched-requirement subset was"
+ warn " checked. This is NOT a green branch: check out the branch you mean to gate"
+ warn " (or set BASE_REF) and run it again."
+ summary ""
+ summary "**Result: NOT APPLICABLE** — no branch under gate (\`${TOUCHED_REASON}\`); the R-TEST-03 subset was not checked."
+ exit 0
+else
+ ok "MERGE GATE: PASS (touched-requirement subset green for the layers that can run)"
+ summary ""
+ summary "**Result: PASS** — L0/L1 green. L2/L3 status per cluster availability above."
+ exit 0
fi
diff --git a/ci/l0.sh b/ci/l0.sh
index 9708d2d..d4dc4cc 100755
--- a/ci/l0.sh
+++ b/ci/l0.sh
@@ -7,6 +7,9 @@
# Stage 3 L0 : `kyverno test` over platform/kyverno/ — default-deny baseline
# present, no plaintext secrets, required labels/egress-class,
# ServiceDefinition schema conformance (owner: T-0.5, ADR-0011).
+# Stage 3b L0 : conformance/run.sh --layer l0 — the WHOLE-REPO L0 invariants that
+# a per-resource policy cannot express (secret hygiene across every
+# tracked file, least-privilege RBAC, spec-immutability enforcement).
#
# Integration seams are env-overridable (see ci/lib.sh + README). When an owning
# subtree is still a placeholder, this reports SKIPPED (exit 3) for that sub-stage
@@ -73,6 +76,30 @@ else
rm -rf "$l0_render"
fi
+# ---- Stage 3b: L0 STATIC conformance checks ---------------------------------
+# docs/14's L0 row is "manifests/ServiceDefinitions obey invariants", which is wider
+# than what Kyverno alone can express over single resources: whole-repo invariants
+# (no plaintext secrets anywhere in git, least-privilege RBAC across every tracked
+# binding, the spec-immutability gate's own behaviour) live in the conformance
+# harness's tests/l0-static subtree and run with no cluster. This stage executes them
+# through the same single-command runner CI and a laptop both use (R-TEST-02).
+log "Stage: L0 static conformance checks (${CONFORMANCE_CMD} --layer l0)"
+if ! entrypoint_ready "$CONFORMANCE_CMD"; then
+ skip "conformance runner not present at '${CONFORMANCE_CMD}' — UNWIRED, not green"
+ summary "- l0-static: **SKIPPED** (conformance runner missing)"
+ if [ "$rc" -eq 0 ]; then rc=3; fi
+else
+ load_touched_requirements
+ if "$CONFORMANCE_CMD" --layer l0 ${REQ_ARGS[@]+"${REQ_ARGS[@]}"}; then
+ ok "L0 static conformance checks passed"
+ summary "- l0-static: **passed**"
+ else
+ err "L0 static conformance checks failed"
+ summary "- l0-static: **FAILED**"
+ rc=1
+ fi
+fi
+
case "$rc" in
0) ok "L0 stage complete" ;;
3) skip "L0 stage incomplete (unwired seam / missing tool) — not a pass" ;;
diff --git a/ci/l1.sh b/ci/l1.sh
index 98707e3..0735146 100755
--- a/ci/l1.sh
+++ b/ci/l1.sh
@@ -20,19 +20,16 @@ if ! entrypoint_ready "$CONFORMANCE_CMD"; then
exit 3
fi
-# Touched-requirement subset (R-TEST-03). Empty subset => run all L1 as a sanity net.
-mapfile -t reqs < <("${CI_DIR}/touched-requirements.sh" || true)
-req_args=()
-if [ "${#reqs[@]}" -gt 0 ]; then
- log "touched requirements: ${reqs[*]}"
- req_args=(--requirements "$(IFS=,; echo "${reqs[*]}")")
-fi
+# Touched-requirement subset (R-TEST-03). An empty OR undeterminable subset => run all
+# L1 as a superset sanity net, and SAY SO — see load_touched_requirements in ci/lib.sh.
+load_touched_requirements
+req_args=(${REQ_ARGS[@]+"${REQ_ARGS[@]}"})
# RECONCILED (PM 2026-08-16): conformance/run.sh's real interface is `--layer `
# [--requirements R-A,R-B]; it ALWAYS writes artifacts to a FIXED location
# (conformance/_artifacts/) and does NOT accept --junit/--coverage. Invoke by layer +
# touched subset, then COLLECT its artifacts into the CI artifact dirs.
-"$CONFORMANCE_CMD" --layer l1 "${req_args[@]}"
+"$CONFORMANCE_CMD" --layer l1 ${req_args[@]+"${req_args[@]}"}
CONF_ART="$(dirname "$CONFORMANCE_CMD")/_artifacts"
if [ -f "${CONF_ART}/report.junit.xml" ]; then
diff --git a/ci/lib.sh b/ci/lib.sh
index 0e6cbc2..9a0bf27 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -85,3 +85,38 @@ cluster_available() {
}
mkartifacts() { mkdir -p "$JUNIT_DIR" "$COVERAGE_DIR"; }
+
+# --- Touched-requirement subset (R-TEST-03 input) ----------------------------
+# Sets: TOUCHED_RC (0 ok / 3 not determinable), TOUCHED_REQS[] and REQ_ARGS[]
+# (ready to splice into a conformance/run.sh invocation).
+#
+# ci/touched-requirements.sh exits 3 — LOUDLY — when it cannot establish a reliable
+# diff base, instead of the old silent degradation to an empty set. An empty set
+# means run.sh gets no `--requirements`, which means its coverage `--require` gate
+# never runs; treating "could not determine" as "nothing touched" is precisely the
+# false green this helper exists to prevent. A stage still runs its FULL layer in
+# that case (a superset, so nothing goes untested), but says plainly that the
+# SCOPED check did not happen; ci/gate.sh turns that into an UNDECIDED decision.
+load_touched_requirements() {
+ local out rc=0
+ out="$("${CI_DIR}/touched-requirements.sh")" || rc=$?
+ TOUCHED_RC=$rc
+ TOUCHED_REQS=()
+ # `if`, not `[ ... ] && ...`: under `set -e` a false AND-list is the caller's last
+ # command status and would abort the stage script on an EMPTY touched set.
+ if [ -n "$out" ]; then mapfile -t TOUCHED_REQS <<< "$out"; fi
+ REQ_ARGS=()
+ if [ "$rc" -eq 3 ]; then
+ warn "touched-requirement subset UNDETERMINED (reason logged above)."
+ warn " Running the FULL layer as a superset sanity net; the SCOPED R-TEST-03"
+ warn " coverage check is NOT performed here — ci/gate.sh reports that as UNDECIDED."
+ elif [ "$rc" -ne 0 ]; then
+ warn "touched-requirements.sh exited ${rc}; proceeding with the full layer."
+ elif [ "${#TOUCHED_REQS[@]}" -gt 0 ]; then
+ log "touched requirements: ${TOUCHED_REQS[*]}"
+ REQ_ARGS=(--requirements "$(IFS=,; echo "${TOUCHED_REQS[*]}")")
+ else
+ log "empty touched-requirement set (base WAS established) — running the full layer."
+ fi
+ return 0
+}
diff --git a/ci/tests/touched-requirements-base.test.sh b/ci/tests/touched-requirements-base.test.sh
new file mode 100755
index 0000000..45a7ebf
--- /dev/null
+++ b/ci/tests/touched-requirements-base.test.sh
@@ -0,0 +1,169 @@
+#!/usr/bin/env bash
+# ci/tests/touched-requirements-base.test.sh — regression tests for BASE-REF RESOLUTION
+# in ci/touched-requirements.sh.
+#
+# THE BUG THESE LOCK DOWN (2026-08-31, local/CI divergence):
+# `make -C ci gate` on a feature branch printed
+# "no changed files vs base; empty touched-requirement set"
+# and then "MERGE GATE: PASS" — while CI, with a correct base, computed 17 touched
+# requirements and BLOCKED on real coverage debt. Root cause: EVERY way of failing to
+# establish a diff base degraded silently to an EMPTY touched set and exit 0. An empty
+# set means conformance/run.sh is invoked without `--requirements`, which means its
+# coverage `--require` gate never runs — so the gate reported PASS having checked
+# nothing. A false green is worse than a red.
+#
+# The fix: an undeterminable base is a LOUD exit 3 (ci/README.md's SKIPPED code) with
+# EMPTY stdout and a machine-readable reason code, never exit 0. These tests assert
+# that for each way the base can be undeterminable, and assert the good paths still work.
+#
+# The tests point the real script at THROWAWAY git repos via the REPO_ROOT override
+# (ci/lib.sh honours a pre-set REPO_ROOT); the script runs from its true location so it
+# still sources the real lib.sh. No network, no cluster, and the working repo is never
+# touched.
+#
+# Run: ci/tests/touched-requirements-base.test.sh (or `make -C ci test`)
+set -uo pipefail # NOT -e: these tests deliberately run a script that exits non-zero.
+
+CI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+SCRIPT="${CI_DIR}/touched-requirements.sh"
+
+PASS=0 FAIL=0
+ok() { printf ' ok %s\n' "$*"; PASS=$((PASS+1)); }
+bad() { printf ' FAIL %s\n' "$*" >&2; FAIL=$((FAIL+1)); }
+
+TMP="$(mktemp -d)"
+trap 'rm -rf "$TMP"' EXIT
+
+# newrepo — a throwaway repo with one baseline commit on `main`.
+newrepo() {
+ local d="$1"
+ mkdir -p "$d/notes"
+ git -C "$d" init -q -b main
+ git -C "$d" config user.email t@t.t
+ git -C "$d" config user.name t
+ git -C "$d" config commit.gpgsign false # throwaway: never sign (ignore host gpg config)
+ echo baseline > "$d/notes/base.txt"
+ git -C "$d" add -A && git -C "$d" commit -qm baseline
+}
+
+# commit_touching — a commit whose content declares an R-ID.
+commit_touching() {
+ local d="$1" text="$2"
+ printf 'name: %s\n' "$text" > "$d/notes/decl.yaml"
+ git -C "$d" add -A && git -C "$d" commit -qm change
+}
+
+# Runs the script in ; sets RC, OUT (stdout), ERRTXT (stderr), REASON.
+run_in() {
+ local d="$1"; shift
+ local reason_file="${TMP}/reason.$$"
+ : > "$reason_file"
+ OUT="$(REPO_ROOT="$d" TOUCHED_REASON_FILE="$reason_file" "$@" "$SCRIPT" 2>"${TMP}/err.$$")"
+ RC=$?
+ ERRTXT="$(cat "${TMP}/err.$$")"
+ REASON="$(cat "$reason_file")"
+}
+
+# assert_refusal
+assert_refusal() {
+ local label="$1" want="$2"
+ [ "$RC" = 3 ] && ok "${label}: exits 3 (SKIPPED), not 0" || bad "${label}: rc=${RC}, expected 3"
+ [ -z "$OUT" ] && ok "${label}: stdout is EMPTY (no silent requirement list)" \
+ || bad "${label}: stdout was '${OUT}'"
+ [ "$REASON" = "$want" ] && ok "${label}: reason code '${want}'" \
+ || bad "${label}: reason '${REASON}', expected '${want}'"
+ grep -q "NOT DETERMINABLE" <<<"$ERRTXT" && ok "${label}: announces the refusal LOUDLY on stderr" \
+ || bad "${label}: stderr lacked the loud banner"
+}
+
+echo "== case 1: a branch ahead of local main (no remote) resolves and reports"
+R1="${TMP}/r1"; newrepo "$R1"
+git -C "$R1" checkout -q -b feature
+commit_touching "$R1" implements-R-ALPHA-01
+run_in "$R1"
+[ "$RC" = 0 ] && ok "ahead-of-main: exits 0" || bad "ahead-of-main: rc=${RC}"
+grep -qx "R-ALPHA-01" <<<"$OUT" && ok "ahead-of-main: emits the touched R-ID" \
+ || bad "ahead-of-main: output was '${OUT}'"
+grep -q "diff base: main" <<<"$ERRTXT" && ok "ahead-of-main: names the base it used" \
+ || bad "ahead-of-main: base not reported: ${ERRTXT}"
+
+echo "== case 2: THE FALSE GREEN — HEAD is the base branch itself"
+# This is the state that produced 'no changed files vs base' + 'MERGE GATE: PASS'
+# locally while CI blocked. It must now refuse, not report an empty set.
+R2="${TMP}/r2"; newrepo "$R2"
+git -C "$R2" checkout -q -b feature
+commit_touching "$R2" implements-R-ALPHA-01
+git -C "$R2" checkout -q main # <- sitting on the base branch
+run_in "$R2"
+assert_refusal "on-base" "on-base"
+
+echo "== case 3: a branch already merged into the base (HEAD is an ancestor)"
+R3="${TMP}/r3"; newrepo "$R3"
+git -C "$R3" checkout -q -b feature
+commit_touching "$R3" implements-R-ALPHA-01
+git -C "$R3" checkout -q main
+git -C "$R3" merge -q --no-edit feature
+git -C "$R3" checkout -q feature # feature is now behind main
+run_in "$R3"
+assert_refusal "already-merged" "on-base"
+
+echo "== case 4: no base ref resolves at all (no main, no origin/main)"
+R4="${TMP}/r4"; newrepo "$R4"
+git -C "$R4" checkout -q -b work
+commit_touching "$R4" implements-R-ALPHA-01
+git -C "$R4" branch -q -D main
+run_in "$R4"
+assert_refusal "no-base-ref" "no-base"
+
+echo "== case 5: a single-commit repo with no main (the removed HEAD~1 last resort)"
+# HEAD~1 does not exist here. The old code fell back to it, the diff failed, and the
+# empty result was reported as 'no changed files' + exit 0 — a silent false green.
+R5="${TMP}/r5"; mkdir -p "$R5/notes"
+git -C "$R5" init -q -b work
+git -C "$R5" config user.email t@t.t; git -C "$R5" config user.name t
+git -C "$R5" config commit.gpgsign false
+echo only > "$R5/notes/only.txt"; git -C "$R5" add -A; git -C "$R5" commit -qm only
+run_in "$R5"
+assert_refusal "single-commit-no-main" "no-base"
+
+echo "== case 6: BASE_REF names a ref that does not exist"
+R6="${TMP}/r6"; newrepo "$R6"
+git -C "$R6" checkout -q -b feature
+commit_touching "$R6" implements-R-ALPHA-01
+run_in "$R6" env BASE_REF=refs/heads/nope
+assert_refusal "bogus-BASE_REF" "no-base"
+
+echo "== case 7: GITHUB_BASE_REF set but absent from the checkout (shallow clone)"
+R7="${TMP}/r7"; newrepo "$R7"
+git -C "$R7" checkout -q -b feature
+commit_touching "$R7" implements-R-ALPHA-01
+run_in "$R7" env GITHUB_BASE_REF=release-9
+assert_refusal "missing-GITHUB_BASE_REF" "no-base"
+
+echo "== case 8: origin/main is preferred over a stale local main, and staleness warns"
+R8="${TMP}/r8"; newrepo "$R8"
+UP="${TMP}/r8-origin.git"; git init -q --bare -b main "$UP"
+git -C "$R8" remote add origin "$UP"
+git -C "$R8" push -q origin main
+# advance origin/main beyond the local main ref, then branch off it
+git -C "$R8" checkout -q -b ahead
+echo more > "$R8/notes/more.txt"; git -C "$R8" add -A; git -C "$R8" commit -qm more
+git -C "$R8" push -q origin ahead:main
+git -C "$R8" fetch -q origin
+git -C "$R8" checkout -q -b feature origin/main
+commit_touching "$R8" implements-R-ALPHA-01
+run_in "$R8"
+[ "$RC" = 0 ] && ok "origin-preferred: exits 0" || bad "origin-preferred: rc=${RC}"
+grep -q "diff base: origin/main" <<<"$ERRTXT" && ok "origin-preferred: uses origin/main, not local main" \
+ || bad "origin-preferred: base line was: ${ERRTXT}"
+grep -q "may be stale" <<<"$ERRTXT" && ok "origin-preferred: warns that local main differs" \
+ || bad "origin-preferred: no staleness warning"
+grep -qx "R-ALPHA-01" <<<"$OUT" && ok "origin-preferred: emits the touched R-ID" \
+ || bad "origin-preferred: output was '${OUT}'"
+
+echo
+if [ "$FAIL" -eq 0 ]; then
+ echo "touched-requirements base-resolution tests: PASS (${PASS} assertions)"; exit 0
+fi
+echo "touched-requirements base-resolution tests: FAIL (${FAIL} failed, ${PASS} passed)" >&2
+exit 1
diff --git a/ci/touched-requirements.sh b/ci/touched-requirements.sh
index a18268a..181c36a 100755
--- a/ci/touched-requirements.sh
+++ b/ci/touched-requirements.sh
@@ -28,22 +28,93 @@ CI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck disable=SC1091
source "${CI_DIR}/lib.sh"
-# Base ref to diff against. In Gitea/GitHub Actions PR runs this is the target
-# branch (main). Locally, default to main; override with BASE_REF.
+# ---------------------------------------------------------------------------------
+# BASE-REF RESOLUTION — and why it now REFUSES rather than guesses.
+#
+# BUG THIS FIXES (local/CI divergence, 2026-08-31): every failure to establish a
+# reliable diff base used to degrade SILENTLY to an EMPTY touched set, which main()
+# reported as "no changed files vs base" and exit 0. An empty set means the coverage
+# `--require` gate is never applied, so the merge gate printed "MERGE GATE: PASS"
+# having checked NOTHING — while CI, with a correct base, computed 17 touched
+# requirements and blocked. A false green is the worst outcome this project can
+# produce, so every one of those degradations is now a LOUD exit 3:
+#
+# * HEAD~1 last resort REMOVED. It silently narrowed the diff to the last commit
+# (and on a one-commit repo failed into an empty set).
+# * two-dot fallback REMOVED. It masked a failing three-dot diff as "no changes".
+# * unresolvable base exit 3 (was: HEAD~1 or empty).
+# * diff command fails exit 3 (was: empty set).
+# * HEAD not ahead of exit 3 (was: exit 0 with an empty set — this is the case
+# the base that produced the false green: running the gate while
+# sitting on `main`, or on a branch already merged).
+#
+# Exit 3 is the ci/README.md "SKIPPED / not-run" code: NOT a pass, NOT a fail. Callers
+# (ci/gate.sh) must surface it as an UNDECIDED merge decision. An honest, noisy skip
+# beats a quiet false pass.
+#
+# Order of preference: BASE_REF (explicit) > GITHUB_BASE_REF (Actions PR runs, target
+# branch) > origin/main > main. Each candidate is VERIFIED to resolve to a commit
+# before it is used, so a shallow checkout that lacks origin/main is reported, not
+# papered over.
: "${BASE_REF:=}"
-resolve_base() {
- if [ -n "$BASE_REF" ]; then echo "$BASE_REF"; return; fi
- if [ -n "${GITHUB_BASE_REF:-}" ]; then echo "origin/${GITHUB_BASE_REF}"; return; fi
- if git -C "$REPO_ROOT" rev-parse --verify -q origin/main >/dev/null; then echo "origin/main"; return; fi
- if git -C "$REPO_ROOT" rev-parse --verify -q main >/dev/null; then echo "main"; return; fi
- echo "HEAD~1" # last resort
+BASE="" # resolved base ref
+BASE_SOURCE="" # where it came from (for the log line)
+BASE_PROBLEM="" # why resolution failed, when it did
+
+rev() { git -C "$REPO_ROOT" rev-parse --verify -q "$1^{commit}" 2>/dev/null; }
+
+# Loud, structured refusal. Prints to stderr only; stdout stays empty so a caller that
+# captures stdout gets NO requirement list rather than a silently-empty one.
+#
+# $1 is a machine-readable REASON CODE, also written to $TOUCHED_REASON_FILE when that
+# env var names a path. ci/gate.sh uses it to tell the two cases apart:
+# on-base HEAD is the base branch (or already merged) — nothing to gate. Legitimate
+# (a push-to-main CI run lands here), so the gate reports NOT APPLICABLE
+# rather than PASS, and does not fail the job.
+# no-base no reliable base ref exists in this checkout — the ENVIRONMENT is broken,
+# diff-failed or git could not produce the diff. A merge decision is impossible, so the
+# gate BLOCKS rather than guessing.
+undecidable() {
+ local code="$1"; shift
+ if [ -n "${TOUCHED_REASON_FILE:-}" ]; then printf '%s\n' "$code" > "$TOUCHED_REASON_FILE"; fi
+ err "TOUCHED-REQUIREMENT SET NOT DETERMINABLE (${code})"
+ err " reason: $*"
+ err " consequence: the R-TEST-03 coverage gate CANNOT be applied to this checkout."
+ err " This is exit 3 (SKIPPED / not-run), never a pass — see ci/README.md."
+ err " Fix one of:"
+ err " * check out the feature branch you mean to gate (not the base branch), or"
+ err " * git fetch origin main (so origin/main exists and is current), or"
+ err " * BASE_REF=[ make -C ci gate to name the base explicitly."
+ exit 3
}
+resolve_base() {
+ local cand
+ if [ -n "$BASE_REF" ]; then
+ if rev "$BASE_REF" >/dev/null; then BASE="$BASE_REF"; BASE_SOURCE="BASE_REF"; return 0; fi
+ BASE_PROBLEM="BASE_REF='${BASE_REF}' does not resolve to a commit in ${REPO_ROOT}"
+ return 1
+ fi
+ if [ -n "${GITHUB_BASE_REF:-}" ]; then
+ for cand in "origin/${GITHUB_BASE_REF}" "${GITHUB_BASE_REF}"; do
+ if rev "$cand" >/dev/null; then BASE="$cand"; BASE_SOURCE="GITHUB_BASE_REF"; return 0; fi
+ done
+ BASE_PROBLEM="GITHUB_BASE_REF='${GITHUB_BASE_REF}' is set, but neither 'origin/${GITHUB_BASE_REF}' nor '${GITHUB_BASE_REF}' exists here (shallow checkout? the workflow needs fetch-depth: 0)"
+ return 1
+ fi
+ for cand in origin/main main; do
+ if rev "$cand" >/dev/null; then BASE="$cand"; BASE_SOURCE="default"; return 0; fi
+ done
+ BASE_PROBLEM="no base ref resolves (tried origin/main, main) in ${REPO_ROOT}"
+ return 1
+}
+
+# Both diff helpers assume BASE is already resolved and validated by main(). They return
+# git's exit status so the CALLER can distinguish "no changes" from "the diff failed" —
+# conflating those two is exactly what produced the false green.
changed_files() {
- local base; base="$(resolve_base)"
# Three-dot: changes on this branch since it diverged from base.
- if git -C "$REPO_ROOT" diff --name-only "${base}...HEAD" 2>/dev/null; then return; fi
- git -C "$REPO_ROOT" diff --name-only "${base}" 2>/dev/null || true
+ git -C "$REPO_ROOT" diff --name-only "${BASE}...HEAD"
}
# The ADDED/REMOVED lines of the change, for the R-ID content scan — excluding
@@ -92,10 +163,8 @@ changed_files() {
# carry AUTHORITATIVE `conformance.home.k3s/R-*` labels and are deliberately still scanned.
# Three-dot to match changed_files(): changes on this branch since it diverged from base.
changed_diff() {
- local base; base="$(resolve_base)"
local -a excl=(':(exclude)*.md' ':(exclude)ci/tests/')
- if git -C "$REPO_ROOT" diff -U0 "${base}...HEAD" -- . "${excl[@]}" 2>/dev/null; then return; fi
- git -C "$REPO_ROOT" diff -U0 "${base}" -- . "${excl[@]}" 2>/dev/null || true
+ git -C "$REPO_ROOT" diff -U0 "${BASE}...HEAD" -- . "${excl[@]}"
}
# Coarse path -> requirement map. Deliberately conservative; extend as subtrees
@@ -120,9 +189,34 @@ path_requirements() {
}
main() {
- local files reqs=""
- files="$(changed_files)"
- [ -n "$files" ] || { log "no changed files vs base; empty touched-requirement set"; exit 0; }
+ local files reqs="" rc=0 base_sha head_sha local_main merge_base
+
+ resolve_base || undecidable no-base "$BASE_PROBLEM"
+ base_sha="$(rev "$BASE")" || undecidable no-base "resolved base '${BASE}' vanished"
+ head_sha="$(rev HEAD)" || undecidable no-base "HEAD does not resolve to a commit (empty repo?)"
+ log "diff base: ${BASE} (${BASE_SOURCE}) @ ${base_sha:0:12} ... HEAD @ ${head_sha:0:12}"
+
+ # Advisory: a never-fetched origin/main silently WIDENS the set (over-broad, never a
+ # false green) — surfaced so an unexpectedly large touched set has an explanation.
+ if [ "$BASE" = "origin/main" ] && local_main="$(rev main)" && [ -n "$local_main" ] \
+ && [ "$local_main" != "$base_sha" ]; then
+ warn "local 'main' (${local_main:0:12}) != origin/main (${base_sha:0:12}) — if origin/main"
+ warn " was not fetched recently the touched set may be stale: git fetch origin main"
+ fi
+
+ merge_base="$(git -C "$REPO_ROOT" merge-base "$BASE" HEAD 2>/dev/null)" \
+ || undecidable no-base "no merge base between '${BASE}' and HEAD (unrelated histories, or a shallow clone)"
+ if [ "$merge_base" = "$head_sha" ]; then
+ undecidable on-base "HEAD is not ahead of '${BASE}' (merge-base == HEAD): this checkout contains no branch commits to gate. The empty touched set would be empty BY DEFAULT, not by verification."
+ fi
+
+ files="$(changed_files)" || rc=$?
+ [ "$rc" -eq 0 ] || undecidable diff-failed "git diff --name-only ${BASE}...HEAD failed (rc=${rc})"
+ if [ -z "$files" ]; then
+ warn "HEAD is ahead of ${BASE} but the three-dot diff is empty (self-cancelling commits?)"
+ warn " -> genuinely empty touched-requirement set; base WAS established."
+ exit 0
+ fi
while IFS= read -r f; do
[ -n "$f" ] || continue
@@ -135,7 +229,10 @@ main() {
# what remains is the SOURCE line, and `grep -vE '^[[:space:]]*#'` drops whole-line
# YAML/shell comments (rule 3 in changed_diff's header) BEFORE the R-ID grep — a
# requirement cited only in a `#` comment is prose, not a declaration.
- reqs+=" $(changed_diff \
+ local diff_txt
+ diff_txt="$(changed_diff)" || rc=$?
+ [ "$rc" -eq 0 ] || undecidable diff-failed "git diff -U0 ${BASE}...HEAD failed (rc=${rc})"
+ reqs+=" $(printf '%s\n' "$diff_txt" \
| grep -E '^[+-]' \
| grep -vE '^(\+\+\+|---)' \
| sed -E 's/^[+-]//' \
diff --git a/conformance/lib/common.sh b/conformance/lib/common.sh
index 3fdeba2..7891fed 100644
--- a/conformance/lib/common.sh
+++ b/conformance/lib/common.sh
@@ -47,7 +47,7 @@ conf_dirs_for_requirements() {
done < <(conf_all_test_files) | sort -u
}
-# Directories for a given layer (subtree): l1 | l2 | l3 | meta | all.
+# Directories for a given layer (subtree): l0 | l1 | l2 | l3 | meta | all.
# Accepts a comma/space separated LIST of layers (e.g. "l2,l3") and returns the UNION of
# their test dirs, deduplicated. A single layer keeps its prior behavior (backward compatible);
# this is what lets `run.sh --layer l2,l3` (the ci/conformance.sh L2/L3 stage) select both
@@ -58,6 +58,7 @@ conf_dirs_for_layer() {
for layer in ${layers//,/ }; do
case "$layer" in
all) roots=("${CONF_ROOT}/tests"); break ;;
+ l0) roots+=("${CONF_ROOT}/tests/l0-static") ;;
l1) roots+=("${CONF_ROOT}/tests/l1-rendering") ;;
l2) roots+=("${CONF_ROOT}/tests/l2-behavioral") ;;
l3) roots+=("${CONF_ROOT}/tests/l3-fault-injection") ;;
diff --git a/conformance/lib/coverage.py b/conformance/lib/coverage.py
index b100356..7938873 100755
--- a/conformance/lib/coverage.py
+++ b/conformance/lib/coverage.py
@@ -115,7 +115,9 @@ def parse_test(path: Path) -> dict:
# Layer inferred from path as a fallback / cross-check.
rel = str(path)
if not layer:
- if "l1-rendering" in rel:
+ if "l0-static" in rel:
+ layer = "L0"
+ elif "l1-rendering" in rel:
layer = "L1"
elif "l2-behavioral" in rel:
layer = "L2"
diff --git a/conformance/lib/emit_skipped_junit.py b/conformance/lib/emit_skipped_junit.py
index 44a6d2d..af190b1 100755
--- a/conformance/lib/emit_skipped_junit.py
+++ b/conformance/lib/emit_skipped_junit.py
@@ -33,7 +33,8 @@ def collect(path: Path) -> tuple[str, list[str], str]:
reqs = sorted({m.group(1).upper() for m in
(LABEL_LINE_RE.match(l) for l in text.splitlines()) if m})
rel = str(path)
- layer = ("L1" if "l1-rendering" in rel else "L2" if "l2-behavioral" in rel
+ layer = ("L0" if "l0-static" in rel
+ else "L1" if "l1-rendering" in rel else "L2" if "l2-behavioral" in rel
else "L3" if "l3-fault-injection" in rel else "META")
return name, reqs, layer
diff --git a/conformance/lib/run_l1_offline.py b/conformance/lib/run_l1_offline.py
index d2f6155..c9de783 100644
--- a/conformance/lib/run_l1_offline.py
+++ b/conformance/lib/run_l1_offline.py
@@ -1,17 +1,24 @@
#!/usr/bin/env python3
-"""Run the L1 *rendering* conformance checks OFFLINE (no cluster) and emit JUnit.
+"""Run the CLUSTER-INDEPENDENT conformance checks OFFLINE and emit JUnit.
-docs/14: "L1 rendering assertions run in CI without a cluster." L1 verifies the
-ServiceDefinition -> rendered-resources mapping against golden expectations; unlike L2/L3
-it needs no live cluster. Each l1-rendering test dir carries a `render-check.sh` (renderer
-+ golden diff + structural asserts). run.sh invokes this for `--layer l1` so those tests
-report a REAL pass/fail instead of being skipped with the cluster-gated layers.
+docs/14 puts two layers in CI with no cluster:
+ * L0 static/policy -- "manifests/ServiceDefinitions obey invariants" (tests/l0-static/,
+ each dir carrying a `check.sh`).
+ * L1 rendering -- ServiceDefinition -> rendered resources vs golden expectations
+ (tests/l1-rendering/, each dir carrying a `render-check.sh`).
+Both are executed here so they report a REAL pass/fail instead of being skipped with the
+cluster-gated layers. run.sh invokes this for `--layer l0`, `--layer l1`, and as the
+offline leg of a full run.
-Pure standard library (matches the harness's no-pip-dependency posture); the render checks
-themselves shell out to catalog/render.py (the renderer's PyYAML dependency lives there).
+Pure standard library (matches the harness's no-pip-dependency posture); the checks
+themselves shell out (render checks to catalog/render.py, whose PyYAML dependency lives
+there; L0 checks to git + PyYAML).
Usage: run_l1_offline.py --junit ] [--label-prefix P] [ ...]
-Exit 0 iff every selected render check passes.
+Exit 0 iff every selected offline check passes.
+
+The per-dir check script is whichever of `render-check.sh` / `check.sh` the directory
+carries; a dir with neither is reported SKIPPED (never a pass).
"""
from __future__ import annotations
@@ -25,6 +32,19 @@ from xml.dom import minidom
LABEL_LINE_RE_TMPL = r"^\s*{prefix}/(R-[A-Z0-9]+-[0-9]+)\s*:\s*[\"']?true"
+# The offline check script a test dir may carry, in preference order.
+CHECK_SCRIPTS = ("render-check.sh", "check.sh")
+
+
+def layer_of(d: Path) -> str:
+ """Layer for the JUnit properties, from the subtree the test lives in."""
+ rel = str(d)
+ if "l0-static" in rel:
+ return "L0"
+ if "l1-rendering" in rel:
+ return "L1"
+ return "OFFLINE"
+
def collect_meta(test_yaml: Path, prefix: str):
text = test_yaml.read_text() if test_yaml.exists() else ""
@@ -44,10 +64,11 @@ def main() -> int:
cases = [] # (name, reqs, dir, rc, output)
for d in args.dirs:
- check = d / "render-check.sh"
+ check = next((d / s for s in CHECK_SCRIPTS if (d / s).exists()), None)
name, reqs = collect_meta(d / "chainsaw-test.yaml", args.label_prefix)
- if not check.exists():
- cases.append((name, reqs, d, 3, f"no render-check.sh in {d}"))
+ if check is None:
+ cases.append((name, reqs, d, 3,
+ f"no offline check script ({' or '.join(CHECK_SCRIPTS)}) in {d}"))
continue
proc = subprocess.run(
["bash", str(check)], capture_output=True, text=True
@@ -61,7 +82,7 @@ def main() -> int:
suite = ET.Element(
"testsuite",
{
- "name": "home-k3s-conformance L1 (offline render)",
+ "name": "home-k3s-conformance L0/L1 (offline)",
"tests": str(len(cases)),
"failures": str(failures),
"errors": "0",
@@ -72,19 +93,21 @@ def main() -> int:
tc = ET.SubElement(
suite,
"testcase",
- {"classname": f"L1:{','.join(reqs) if reqs else 'UNTAGGED'}", "name": name},
+ {"classname": f"{layer_of(d)}:{','.join(reqs) if reqs else 'UNTAGGED'}",
+ "name": name},
)
props = ET.SubElement(tc, "properties")
ET.SubElement(props, "property", {"name": "requirements", "value": ",".join(reqs)})
- ET.SubElement(props, "property", {"name": "layer", "value": "L1"})
+ ET.SubElement(props, "property", {"name": "layer", "value": layer_of(d)})
ET.SubElement(props, "property", {"name": "path", "value": str(d)})
if rc == 3:
- ET.SubElement(tc, "skipped", {"message": out or "no render-check.sh"})
+ ET.SubElement(tc, "skipped", {"message": out or "no offline check script"})
elif rc != 0:
- fail = ET.SubElement(tc, "failure", {"message": f"render-check exit {rc}"})
+ fail = ET.SubElement(tc, "failure", {"message": f"offline check exit {rc}"})
fail.text = out
status = "PASS" if rc == 0 else ("SKIP" if rc == 3 else "FAIL")
- print(f"[l1-offline] {status:4s} {name} ({','.join(reqs) or 'untagged'})",
+ print(f"[offline] {layer_of(d):3s} {status:4s} {name} "
+ f"({','.join(reqs) or 'untagged'})",
file=sys.stderr)
if rc not in (0, 3):
print(out, file=sys.stderr)
@@ -103,7 +126,7 @@ def main() -> int:
args.junit.parent.mkdir(parents=True, exist_ok=True)
xml = minidom.parseString(ET.tostring(root, "utf-8")).toprettyxml(indent=" ")
args.junit.write_text(xml)
- print(f"[l1-offline] wrote JUnit ({len(cases)} case(s), {failures} failure(s)) -> "
+ print(f"[offline] wrote JUnit ({len(cases)} case(s), {failures} failure(s)) -> "
f"{args.junit}", file=sys.stderr)
return 1 if failures else 0
diff --git a/conformance/lib/status.py b/conformance/lib/status.py
index e3e3934..d2d14b2 100755
--- a/conformance/lib/status.py
+++ b/conformance/lib/status.py
@@ -287,15 +287,22 @@ def classify_test(t: dict) -> dict:
executable annotated `conformance.home.k3s/executable: "true"` and no TODOs.
executable-untagged no TODOs and a real runnable body, but the annotation is
missing (true of the L1 render tests, which execute a
- render-check.sh + golden diff, and of the META self-check).
+ render-check.sh + golden diff, of the L0 static checks, and
+ of the META self-check).
Counted as executable, and flagged as an annotation gap.
scaffold anything else.
"""
p = Path(t["path"])
text = p.read_text() if p.is_file() else ""
+ # The offline assertion script a test dir may carry: `render-check.sh` (L1 rendering)
+ # or `check.sh` (L0 static). BOTH are read, so a placeholder body in either one still
+ # downgrades the test to a scaffold — an annotation must never outrank the body,
+ # whichever file the body lives in.
extra = ""
- if (p.parent / "render-check.sh").is_file():
- extra = (p.parent / "render-check.sh").read_text()
+ for script in ("render-check.sh", "check.sh"):
+ sp = p.parent / script
+ if sp.is_file():
+ extra += sp.read_text()
declared = bool(_EXECUTABLE_RE.search(text))
has_todo = "TODO" in text or "TODO" in extra
bm = _BLOCKED_RE.search(text)
diff --git a/conformance/run.sh b/conformance/run.sh
index 0e79a20..2d9efcc 100755
--- a/conformance/run.sh
+++ b/conformance/run.sh
@@ -8,7 +8,7 @@
#
# ./run.sh # full suite; skips L1/L2/L3 if no cluster
# ./run.sh --requirements R-NET-01,R-NET-06 # merge-gate subset (tagged)
-# ./run.sh --layer l2 # one layer
+# ./run.sh --layer l2 # one layer (l0|l1|l2|l3|meta|all; l0/l1 need no cluster)
# ./run.sh --requirements R-NET-01,R-NET-02 --layer l2 # intersect: tagged AND in layer
# ./run.sh --coverage-only # regenerate coverage report only (no cluster)
# ./run.sh --status # DERIVED STATUS report only (no cluster): per
@@ -280,26 +280,40 @@ run_chainsaw() {
return 0
}
-# --- L1 rendering: cluster-INDEPENDENT render checks (docs/14) --------------------------
-# L1 verifies ServiceDefinition -> rendered-resources against golden expectations; it needs
-# no live cluster. Each l1-rendering test dir carries a `render-check.sh` (renderer + golden
-# diff + structural asserts). We run those directly and emit a REAL pass/fail JUnit, instead
-# of skipping L1 with the cluster-gated layers (T-0.3). Honors the same --layer/--requirements
-# selection as the rest of the suite (select_dirs), filtered to the l1-rendering subtree.
-l1_selected_dirs() {
- select_dirs | grep '/l1-rendering/' || true
+# --- L0/L1: the cluster-INDEPENDENT layers (docs/14) ------------------------------------
+# docs/14 puts TWO layers in CI without a cluster:
+# L0 static/policy tests/l0-static/** each dir carries a `check.sh` (repo invariants)
+# L1 rendering tests/l1-rendering/** each dir carries a `render-check.sh` (render == golden)
+# Both are executed directly here and emit a REAL pass/fail JUnit, instead of being skipped
+# with the cluster-gated layers. Honors the same --layer/--requirements selection as the rest
+# of the suite (select_dirs), filtered to those two subtrees.
+OFFLINE_SUBTREES='/l0-static/|/l1-rendering/'
+
+offline_selected_dirs() {
+ select_dirs | grep -E "${OFFLINE_SUBTREES}" || true
}
-run_l1_offline() {
- local -a l1dirs
- mapfile -t l1dirs < <(l1_selected_dirs)
- if [ "${#l1dirs[@]}" -eq 0 ]; then
- log "No L1 render tests match the selection (layer=${LAYER} reqs='${REQS}')."
+# True when EVERY requested layer is cluster-independent, so the offline result is the
+# whole result and run.sh must exit on it (the ci/l0.sh + ci/l1.sh merge-gate paths).
+layers_are_offline_only() {
+ local l
+ [ "${LAYER}" = all ] && return 1
+ for l in ${LAYER//,/ }; do
+ case "$l" in l0|l1) ;; *) return 1 ;; esac
+ done
+ return 0
+}
+
+run_offline_checks() {
+ local -a dirs
+ mapfile -t dirs < <(offline_selected_dirs)
+ if [ "${#dirs[@]}" -eq 0 ]; then
+ log "No L0/L1 offline checks match the selection (layer=${LAYER} reqs='${REQS}')."
return 0
fi
- log "Running ${#l1dirs[@]} L1 render check(s) OFFLINE (no cluster; docs/14)."
+ log "Running ${#dirs[@]} offline L0/L1 check(s) (no cluster; docs/14)."
"${PYTHON}" "${CONF_ROOT}/lib/run_l1_offline.py" \
- --junit "${JUNIT_OUT}" --label-prefix "${CONF_LABEL_PREFIX}" "${l1dirs[@]}"
+ --junit "${JUNIT_OUT}" --label-prefix "${CONF_LABEL_PREFIX}" "${dirs[@]}"
}
###############################################################################
@@ -331,29 +345,30 @@ case "${MODE}" in
;;
esac
-# L1 rendering is cluster-independent (docs/14): run the selected L1 render checks offline
-# and treat their result as authoritative REGARDLESS of cluster availability. A render-check
-# failure is a real failure (nothing cluster-gated about it), so it always blocks.
-L1_RC=0
-run_l1_offline || L1_RC=$?
-if [ "${LAYER}" = l1 ]; then
- # L1-only run (the ci/l1.sh merge-gate path): the offline render result IS the result.
- if [ "${L1_RC}" -eq 0 ]; then
- log "Done (L1 render). JUnit: ${JUNIT_OUT} | Coverage: ${COVERAGE_OUT}/coverage.md"
+# L0 static + L1 rendering are cluster-independent (docs/14): run the selected offline
+# checks and treat their result as authoritative REGARDLESS of cluster availability. An
+# offline check failure is a real failure (nothing cluster-gated about it), so it always blocks.
+OFFLINE_RC=0
+run_offline_checks || OFFLINE_RC=$?
+if layers_are_offline_only; then
+ # Offline-only run (the ci/l0.sh + ci/l1.sh merge-gate paths): the offline result IS the result.
+ if [ "${OFFLINE_RC}" -eq 0 ]; then
+ log "Done (offline ${LAYER}). JUnit: ${JUNIT_OUT} | Coverage: ${COVERAGE_OUT}/coverage.md"
else
- log "L1 render checks FAILED (rc=${L1_RC}). JUnit: ${JUNIT_OUT}"
+ log "Offline ${LAYER} checks FAILED (rc=${OFFLINE_RC}). JUnit: ${JUNIT_OUT}"
fi
- exit "${L1_RC}"
+ exit "${OFFLINE_RC}"
fi
-if [ "${L1_RC}" -ne 0 ]; then
- log "L1 render checks FAILED (rc=${L1_RC}) — blocking (render failure is not cluster-gated)."
- exit "${L1_RC}"
+if [ "${OFFLINE_RC}" -ne 0 ]; then
+ log "Offline L0/L1 checks FAILED (rc=${OFFLINE_RC}) — blocking (not cluster-gated)."
+ exit "${OFFLINE_RC}"
fi
# run mode: need a cluster for L2/L3. Without one (Phase 0), emit skipped JUnit for the
# cluster-gated layers. (L1 already ran offline above and passed.)
if [ "${FORCE_NO_CLUSTER}" = 1 ] || ! conf_cluster_reachable; then
log "No reachable cluster — emitting SKIPPED JUnit scaffold for L2/L3 (Phase 0 behavior)."
+ log "NOTE: the offline L0/L1 results above already ran for real; this shim covers L2/L3 only."
log "Also running structural lint so scaffolds stay valid."
lint_tests || { log "lint failed"; exit 1; }
"${PYTHON}" "${CONF_ROOT}/lib/emit_skipped_junit.py" \
diff --git a/conformance/tests/README.md b/conformance/tests/README.md
index 2408da4..8fcf3a7 100644
--- a/conformance/tests/README.md
+++ b/conformance/tests/README.md
@@ -4,6 +4,7 @@ Every test is a **Chainsaw `Test`** in its own directory:
```
tests/
+ l0-static/ L0 — whole-repo invariants over the tracked tree (no cluster)
l1-rendering/ L1 — ServiceDefinition -> rendered resources match golden
l2-behavioral/ L2 — real runtime behavior (probes against a cluster)
l3-fault-injection/ L3 — break something; assert fail-closed / no-data-loss
@@ -14,6 +15,35 @@ tests/
The layer a test belongs to is its subtree **and** its `.../layer` annotation; the
coverage generator cross-checks the two.
+### L0 static (`l0-static/`) — what belongs here, and what does not
+
+docs/14's L0 row is *"Pre-apply, in CI · manifests/ServiceDefinitions obey invariants"*.
+Kyverno (`platform/kyverno/`) covers the invariants expressible over a **single
+resource**. This subtree covers the L0 invariants that are **properties of the repository
+as a whole** and that no per-resource policy can state — e.g. *no plaintext secret exists
+anywhere in the tracked tree*, *no tracked RoleBinding grants a workload identity a
+privileged role*, *the spec-immutability gate actually blocks a modified spec*.
+
+Each `l0-static//` dir carries a **`check.sh`**: a self-contained offline assertion
+script (git + Python stdlib/PyYAML; no cluster, no network) that exits 0/1. It is executed:
+
+- by `conformance/run.sh --layer l0` (and as the offline leg of a full run), via
+ `lib/run_l1_offline.py`, producing a REAL pass/fail JUnit entry — never a skip; and
+- by the merge gate's **`ci/l0.sh`** stage, so these assertions run on every PR; and
+- by the wrapping Chainsaw `Test`, which runs the SAME `check.sh`, so an in-cluster run
+ executes byte-identical assertions.
+
+Two authoring rules specific to this layer:
+
+1. **State the gap.** A statically-provable half of a requirement is still only half.
+ Every `check.sh` header ends with an explicit *"what this does NOT prove"* list naming
+ the cluster-layer remainder. Covering a requirement at L0 does not finish it.
+2. **Exemptions must be self-policing.** Where a check must tolerate something (a
+ deliberately-plaintext policy fixture, an `.example` template), the exemption is
+ conditioned on a property that would break if the exemption were abused — e.g. the
+ plaintext Secret fixture is exempt *only while* a `kyverno-test.yaml` asserts the
+ policy rejects it. Never a bare path allowlist.
+
## Requirement-tagging convention (authoritative)
A test declares the requirement(s) it verifies **three** ways; the label is the source
@@ -28,7 +58,7 @@ metadata:
annotations:
# Human-readable canonical list — MUST equal the label set.
conformance.home.k3s/requirements: "R-NET-01"
- # Declared layer (L1|L2|L3|META) — must match the subtree.
+ # Declared layer (L0|L1|L2|L3|META) — must match the subtree.
conformance.home.k3s/layer: "L2"
# true => R-TEST-05 requires this to be (or have) an L3 fault-injection test.
conformance.home.k3s/safety-critical: "false"
@@ -65,4 +95,7 @@ and coverage extraction exact and robust.
the PM — do not relax the assertion or the requirement.
Scaffolds in this tree carry `TODO(T-x.y)` markers where a fixture/assertion lands as a
-later roadmap phase delivers the capability under test.
+later roadmap phase delivers the capability under test. `lib/status.py` treats such a
+marker — in the `chainsaw-test.yaml`, in `render-check.sh`, or in `check.sh` — as
+outranking any `conformance.home.k3s/executable: "true"` annotation, so a placeholder body
+can never be reported as an executable test.
diff --git a/conformance/tests/l0-static/least-privilege-rbac/chainsaw-test.yaml b/conformance/tests/l0-static/least-privilege-rbac/chainsaw-test.yaml
new file mode 100644
index 0000000..414c6de
--- /dev/null
+++ b/conformance/tests/l0-static/least-privilege-rbac/chainsaw-test.yaml
@@ -0,0 +1,44 @@
+# =============================================================================
+# L0 STATIC — least privilege for workload identities and network reach (R-SEC-02)
+# Catalog: docs/14 L0 row, "Manifests/ServiceDefinitions obey invariants" — runs in
+# CI WITHOUT a cluster.
+#
+# GIVEN the repository as git tracks it,
+# WHEN every tracked RBAC object and every least-network-reach policy is inspected,
+# THEN no in-repo role grants a wildcard or a privilege-escalation verb, no workload
+# ServiceAccount is bound to a privileged role, the conformance runner's own
+# privileged test identity stays scoped to its authored ClusterRole, and each
+# least-network-reach policy both Enforces and is proven by an L0 negative row
+# (or a generated-resource row) to actually close reach.
+#
+# STATUS: EXECUTABLE — and DELIBERATELY PARTIAL. `check.sh` is a pure offline assertion
+# script (git ls-files + PyYAML); conformance/run.sh --layer l0 and ci/l0.sh run it
+# directly, and this Chainsaw Test wraps the SAME script. It proves the STATIC half of
+# both clauses. The RUNTIME half — that the API server actually denies an unrequested
+# verb, and that pod-to-pod traffic is actually dropped — needs a cluster and is NOT
+# asserted under this requirement; see the gap list in check.sh's header. R-SEC-02 is
+# therefore covered, not finished.
+# =============================================================================
+apiVersion: chainsaw.kyverno.io/v1alpha1
+kind: Test
+metadata:
+ name: least-privilege-rbac
+ labels:
+ conformance.home.k3s/R-SEC-02: "true"
+ annotations:
+ conformance.home.k3s/requirements: "R-SEC-02"
+ conformance.home.k3s/layer: "L0"
+ conformance.home.k3s/safety-critical: "false"
+ conformance.home.k3s/executable: "true"
+ conformance.home.k3s/intent: >-
+ Workload identities carry no wildcard, no escalation verb and no privileged
+ binding; the least-network-reach policies are enforced and proven to reject.
+spec:
+ steps:
+ - name: scan-rbac-and-least-reach-policies
+ description: >-
+ Assert the authored least-privilege posture of every tracked RBAC object and
+ the enforced, test-proven status of the least-network-reach policies.
+ try:
+ - script:
+ content: bash ./check.sh
diff --git a/conformance/tests/l0-static/least-privilege-rbac/check.sh b/conformance/tests/l0-static/least-privilege-rbac/check.sh
new file mode 100755
index 0000000..1498097
--- /dev/null
+++ b/conformance/tests/l0-static/least-privilege-rbac/check.sh
@@ -0,0 +1,269 @@
+#!/usr/bin/env bash
+# =============================================================================
+# L0 static check (offline, no cluster) — least-privilege-rbac
+# Serves: R-SEC-02 ("The platform MUST follow least-privilege for workload
+# identities and network reach." — spec docs/00-vision-and-scope.md;
+# detail in docs/07 "least privilege ... least network reach").
+#
+# The requirement has TWO halves and this check covers the STATIC half of each,
+# at the layer docs/14 puts them ("L0 Static/policy | Pre-apply, in CI").
+#
+# GIVEN the repository as git tracks it,
+# WHEN every tracked RBAC object and every least-network-reach policy is inspected,
+# THEN — workload identity —
+# (1) no in-repo Role/ClusterRole grants a wildcard apiGroup, resource or verb,
+# and none grants escalate/bind/impersonate (the privilege-escalation verbs);
+# (2) no (Cluster)RoleBinding binds a ServiceAccount — a WORKLOAD identity — to
+# `cluster-admin` or to any in-repo role holding wildcard rules. Human/Group
+# subjects are out of scope: R-SEC-02 constrains workload identities, and the
+# operator's own admin path is an OIDC group binding, deliberately;
+# (3) every ServiceAccount subject is namespaced (no cluster-wide ambiguity);
+# (4) the conformance runner's own SA — this workstream's PRIVILEGED TEST
+# IDENTITY, whose least-privilege authoring docs/14 explicitly delegates to
+# WS-TEST — is bound only to its own scoped ClusterRole, carries no wildcard
+# and no privilege-escalation verb, and grants nothing over RBAC objects
+# themselves (no rbac.authorization.k8s.io rules);
+# — network reach —
+# (5) the policies that implement least network reach exist, and each is Enforce
+# (or carries the Flux token that substitutes to Enforce): the default-deny
+# baseline generator, the baseline SHAPE validator, the host-namespace/
+# privilege restriction, and the egress-class label requirement;
+# (6) each of those policies is actually EXERCISED by the L0 Kyverno suite with a
+# NEGATIVE row (result: fail) — a policy nobody proves rejects anything is a
+# declaration, not a gate.
+#
+# WHAT THIS DOES *NOT* PROVE (honest gap, cluster layer — R-SEC-02 is NOT fully covered):
+# - RUNTIME least privilege of workload identities: that the API server actually
+# denies the conformance SA (or any workload SA) the verbs it was not granted.
+# That needs a live cluster (L2) and no test asserts it today.
+# - RUNTIME least network reach: that pod-to-pod traffic is actually denied. The
+# behaviour is exercised by tests/l2-behavioral/{default-deny,deny-undeclared},
+# but those are tagged R-NET-01/R-NET-02 and are cluster-gated (skipped in Phase 0);
+# nothing proves it under an R-SEC-02 tag.
+# - Helm-chart-generated RBAC (e.g. what a HelmRelease's chart creates in-cluster) is
+# not in the repo, so it is outside this static scan entirely.
+# =============================================================================
+set -euo pipefail
+DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+ROOT="$(cd "${DIR}/../../../.." && pwd)" # repo root
+PYTHON="${PYTHON:-python3}"
+
+cd "${ROOT}"
+"${PYTHON}" - "${ROOT}" <<'PY'
+import os, re, subprocess, sys
+from pathlib import Path
+
+try:
+ import yaml
+except ImportError: # pragma: no cover
+ print("check FAIL: PyYAML is required (same dependency as catalog/render.py)",
+ file=sys.stderr)
+ raise SystemExit(1)
+
+ROOT = Path(sys.argv[1])
+FAILURES: list[str] = []
+CHECKS = 0
+
+
+def check(cond, msg):
+ global CHECKS
+ CHECKS += 1
+ if not cond:
+ FAILURES.append(msg)
+
+
+def tracked() -> list[str]:
+ out = subprocess.run(["git", "-C", str(ROOT), "ls-files", "-z"],
+ capture_output=True, text=True, check=True).stdout
+ return [p for p in out.split("\0") if p]
+
+
+def docs_of(rel: str):
+ try:
+ return [d for d in yaml.safe_load_all((ROOT / rel).read_text(errors="replace"))
+ if isinstance(d, dict)]
+ except Exception:
+ return []
+
+
+FILES = tracked()
+YAML_FILES = [f for f in FILES if re.search(r"\.(ya?ml)(\.example)?$", f)]
+check(len(YAML_FILES) > 50, f"only {len(YAML_FILES)} YAML files seen — not a real checkout")
+
+ESCALATION_VERBS = {"escalate", "bind", "impersonate"}
+PRIVILEGED_ROLES = {"cluster-admin", "admin", "edit"}
+
+roles: dict[tuple[str, str], dict] = {} # (kind, name) -> {"file":…, "wildcard":bool, …}
+bindings: list[dict] = []
+
+for rel in YAML_FILES:
+ for d in docs_of(rel):
+ kind = d.get("kind")
+ name = (d.get("metadata") or {}).get("name")
+ if kind in ("Role", "ClusterRole"):
+ wildcard, escalating, rbac_over_rbac = False, set(), False
+ for r in d.get("rules") or []:
+ if not isinstance(r, dict):
+ continue
+ for field in ("apiGroups", "resources", "verbs"):
+ if "*" in (r.get(field) or []):
+ wildcard = True
+ escalating |= set(v for v in (r.get("verbs") or [])) & ESCALATION_VERBS
+ if "rbac.authorization.k8s.io" in (r.get("apiGroups") or []):
+ rbac_over_rbac = True
+ roles[(kind, name)] = {"file": rel, "wildcard": wildcard,
+ "escalating": escalating, "rbac": rbac_over_rbac,
+ "doc": d}
+ # (1) no wildcards, no privilege-escalation verbs.
+ check(not wildcard,
+ f"{rel}: {kind}/{name} grants a wildcard apiGroup/resource/verb — that is "
+ "not least-privilege (R-SEC-02)")
+ check(not escalating,
+ f"{rel}: {kind}/{name} grants privilege-escalation verb(s) "
+ f"{sorted(escalating)} (R-SEC-02)")
+ elif kind in ("RoleBinding", "ClusterRoleBinding"):
+ bindings.append({"file": rel, "name": name, "doc": d})
+
+check(len(roles) > 0, "no Role/ClusterRole found in the repo — the RBAC scan is not looking at it")
+check(len(bindings) > 0, "no RoleBinding/ClusterRoleBinding found — the RBAC scan is not looking at it")
+
+for b in bindings:
+ d, rel, name = b["doc"], b["file"], b["name"]
+ ref = d.get("roleRef") or {}
+ ref_kind, ref_name = ref.get("kind"), ref.get("name")
+ sa_subjects = [s for s in (d.get("subjects") or [])
+ if isinstance(s, dict) and s.get("kind") == "ServiceAccount"]
+ local = roles.get((ref_kind, ref_name))
+ ref_is_privileged = (ref_name in PRIVILEGED_ROLES) or bool(local and local["wildcard"])
+ # (2) a WORKLOAD identity must never hold a privileged role.
+ check(not (sa_subjects and ref_is_privileged),
+ f"{rel}: {d.get('kind')}/{name} binds ServiceAccount(s) "
+ f"{[s.get('name') for s in sa_subjects]} to the privileged role {ref_name!r} — "
+ "workload identities must be least-privilege (R-SEC-02)")
+ # (3) SA subjects must name their namespace (a ClusterRoleBinding without one is
+ # ambiguous and, in a RoleBinding, silently defaults to the binding's namespace).
+ for s in sa_subjects:
+ check(bool(s.get("namespace")) or d.get("kind") == "RoleBinding",
+ f"{rel}: {d.get('kind')}/{name} has a ServiceAccount subject "
+ f"{s.get('name')!r} with no namespace (R-SEC-02)")
+
+# --------------------------------- (4) our own privileged test identity, in detail
+RBAC_FILE = "conformance/incluster/rbac.yaml"
+check(RBAC_FILE in FILES, f"{RBAC_FILE} is missing — the conformance runner has no authored RBAC")
+runner_role = roles.get(("ClusterRole", "conformance-runner"))
+check(runner_role is not None,
+ f"{RBAC_FILE}: no ClusterRole/conformance-runner — the test identity's grants are undefined")
+if runner_role:
+ check(runner_role["file"] == RBAC_FILE,
+ f"ClusterRole/conformance-runner is defined in {runner_role['file']}, not {RBAC_FILE}")
+ check(not runner_role["wildcard"], "ClusterRole/conformance-runner uses a wildcard (R-SEC-02)")
+ check(not runner_role["escalating"],
+ "ClusterRole/conformance-runner grants a privilege-escalation verb (R-SEC-02)")
+ check(not runner_role["rbac"],
+ "ClusterRole/conformance-runner grants rights over RBAC objects themselves — a test "
+ "identity that can rewrite RBAC is not least-privilege (R-SEC-02)")
+
+runner_bindings = [b for b in bindings
+ if any(s.get("kind") == "ServiceAccount" and s.get("name") == "conformance-runner"
+ for s in (b["doc"].get("subjects") or []))]
+check(len(runner_bindings) >= 1, "the conformance-runner ServiceAccount has no binding at all")
+for b in runner_bindings:
+ ref = b["doc"].get("roleRef") or {}
+ check(ref.get("name") == "conformance-runner",
+ f"{b['file']}: conformance-runner is bound to {ref.get('name')!r} instead of its own "
+ "scoped ClusterRole (R-SEC-02)")
+ for s in b["doc"].get("subjects") or []:
+ if s.get("kind") == "ServiceAccount":
+ check(s.get("namespace") == "conformance",
+ f"{b['file']}: conformance-runner subject is in namespace "
+ f"{s.get('namespace')!r}, expected 'conformance' (R-SEC-02)")
+
+# --------------------- (5)/(6) least NETWORK REACH: the policies exist, Enforce, and
+# are proven to reject something at L0.
+# Validating policies: must Enforce AND be proven to REJECT something at L0.
+REQUIRED_POLICIES = {
+ "validate-default-deny-shape": "a baseline that leaves a direction open is rejected",
+ "restrict-host-namespaces": "host namespaces / privileged containers are rejected",
+ "require-egress-class-label": "workloads must declare their egress class",
+}
+# Generating policies have no validationFailureAction and no negative row: the proof
+# that they close network reach is that the L0 suite asserts the resource they GENERATE.
+REQUIRED_GENERATE_POLICIES = {
+ "require-default-deny-baseline": "every namespace gets a default-deny baseline",
+}
+policies: dict[str, dict] = {}
+for rel in YAML_FILES:
+ for d in docs_of(rel):
+ if d.get("kind") in ("ClusterPolicy", "Policy"):
+ policies[(d.get("metadata") or {}).get("name")] = {"file": rel, "doc": d}
+
+for pname, why in REQUIRED_POLICIES.items():
+ p = policies.get(pname)
+ check(p is not None, f"policy {pname!r} is missing — nothing enforces: {why} (R-SEC-02)")
+ if not p:
+ continue
+ action = str((p["doc"].get("spec") or {}).get("validationFailureAction", ""))
+ check(action == "Enforce" or action.startswith("${"),
+ f"{p['file']}: policy {pname!r} validationFailureAction is {action!r} — a least-reach "
+ "policy in Audit mode blocks nothing (R-SEC-02)")
+
+negatives: dict[str, int] = {}
+suites = [f for f in FILES if os.path.basename(f) == "kyverno-test.yaml"]
+check(len(suites) > 0, "no kyverno-test.yaml — the L0 policy gate is not exercised at all")
+for suite in suites:
+ for d in docs_of(suite):
+ for r in d.get("results") or []:
+ if isinstance(r, dict) and r.get("result") == "fail":
+ negatives[r.get("policy")] = negatives.get(r.get("policy"), 0) + 1
+
+for pname, why in REQUIRED_POLICIES.items():
+ check(negatives.get(pname, 0) > 0,
+ f"no L0 test row asserts that policy {pname!r} REJECTS a violating resource "
+ f"(result: fail) — unproven gate for: {why} (R-SEC-02)")
+
+# The generating half: the policy must exist, and the L0 suite must assert the resource
+# it generates (a `generatedResource` row that passes) — otherwise nothing proves a new
+# namespace actually arrives with its reach closed.
+for pname, why in REQUIRED_GENERATE_POLICIES.items():
+ p = policies.get(pname)
+ check(p is not None, f"policy {pname!r} is missing — nothing enforces: {why} (R-SEC-02)")
+ generated_rows = []
+ for suite in suites:
+ for d in docs_of(suite):
+ for r in d.get("results") or []:
+ if isinstance(r, dict) and r.get("policy") == pname \
+ and r.get("generatedResource") and r.get("result") == "pass":
+ generated_rows.append((suite, r.get("generatedResource")))
+ check(len(generated_rows) > 0,
+ f"no L0 test row asserts the resource generated by {pname!r} — unproven gate for: "
+ f"{why} (R-SEC-02)")
+ for suite, gen in generated_rows:
+ gp = os.path.normpath(os.path.join(os.path.dirname(suite), str(gen)))
+ check(gp in FILES, f"{suite}: generatedResource {gen!r} does not resolve to a tracked file")
+ golden = "\n".join(l for l in (ROOT / gp).read_text(errors="replace").splitlines()
+ if not l.lstrip().startswith("#")) if gp in FILES else ""
+ check("NetworkPolicy" in golden and re.search(r"policyTypes:", golden),
+ f"{gp}: the asserted default-deny baseline is not a shaped NetworkPolicy (R-SEC-02)")
+
+# The host-namespace policy must prove BOTH of its rules reject, not just one.
+host_rules = set()
+for suite in suites:
+ for d in docs_of(suite):
+ for r in d.get("results") or []:
+ if isinstance(r, dict) and r.get("policy") == "restrict-host-namespaces" \
+ and r.get("result") == "fail":
+ host_rules.add(r.get("rule"))
+check({"disallow-host-namespaces", "disallow-privileged-containers"} <= host_rules,
+ "the L0 suite does not prove BOTH restrict-host-namespaces rules reject a violating pod "
+ f"(negative rows seen for: {sorted(host_rules)}) — R-SEC-02")
+
+# --------------------------------------------------------------------------- verdict
+if FAILURES:
+ print(f"CHECK FAIL (least-privilege-rbac): {len(FAILURES)} of {CHECKS} assertions failed",
+ file=sys.stderr)
+ for f in FAILURES:
+ print(f" - {f}", file=sys.stderr)
+ raise SystemExit(1)
+print(f"check OK: least-privilege-rbac — {CHECKS} assertions over {len(roles)} role(s), "
+ f"{len(bindings)} binding(s), {len(REQUIRED_POLICIES)} least-reach policies")
+PY
diff --git a/conformance/tests/l0-static/no-plaintext-secrets/chainsaw-test.yaml b/conformance/tests/l0-static/no-plaintext-secrets/chainsaw-test.yaml
new file mode 100644
index 0000000..6e45e1c
--- /dev/null
+++ b/conformance/tests/l0-static/no-plaintext-secrets/chainsaw-test.yaml
@@ -0,0 +1,43 @@
+# =============================================================================
+# L0 STATIC — no plaintext secrets in git (R-SEC-01)
+# Catalog: docs/14 L0 row, "Manifests/ServiceDefinitions obey invariants ...
+# no plaintext secrets" — runs in CI WITHOUT a cluster.
+#
+# GIVEN the repository as git tracks it,
+# WHEN every tracked file is inspected,
+# THEN every committed Secret is SOPS ciphertext, every ansible group_vars vault is
+# ansible-vault encrypted, no private key material is tracked, decryption is
+# delegated to an in-cluster SOPS key, and the L0 policy that rejects plaintext
+# Secrets exists, Enforces, and is proven by a negative test row to reject one.
+#
+# STATUS: EXECUTABLE. `check.sh` is a pure offline assertion script (git ls-files +
+# PyYAML, no cluster, no network); conformance/run.sh --layer l0 and ci/l0.sh run it
+# directly, and this Chainsaw Test wraps the SAME script so an in-cluster run executes
+# the identical assertions. Encryption at rest is necessary but NOT sufficient: the
+# runtime half of "decrypted only in-cluster", and a scan of git HISTORY, are named as
+# explicit gaps in check.sh's header and are NOT claimed here.
+# =============================================================================
+apiVersion: chainsaw.kyverno.io/v1alpha1
+kind: Test
+metadata:
+ name: no-plaintext-secrets
+ labels:
+ conformance.home.k3s/R-SEC-01: "true"
+ annotations:
+ conformance.home.k3s/requirements: "R-SEC-01"
+ conformance.home.k3s/layer: "L0"
+ conformance.home.k3s/safety-critical: "false"
+ conformance.home.k3s/executable: "true"
+ conformance.home.k3s/intent: >-
+ No secret is stored in plaintext in git: committed Secrets are SOPS ciphertext,
+ ansible vaults are encrypted, no private key material is tracked, and the
+ no-plaintext-secrets policy gate is enforced and proven to reject.
+spec:
+ steps:
+ - name: scan-tracked-tree-for-plaintext-secrets
+ description: >-
+ Assert over the whole tracked tree that no secret material is stored in
+ plaintext and that decryption is delegated to the in-cluster SOPS key.
+ try:
+ - script:
+ content: bash ./check.sh
diff --git a/conformance/tests/l0-static/no-plaintext-secrets/check.sh b/conformance/tests/l0-static/no-plaintext-secrets/check.sh
new file mode 100755
index 0000000..29c389d
--- /dev/null
+++ b/conformance/tests/l0-static/no-plaintext-secrets/check.sh
@@ -0,0 +1,287 @@
+#!/usr/bin/env bash
+# =============================================================================
+# L0 static check (offline, no cluster) — no-plaintext-secrets
+# Serves: R-SEC-01 ("Secrets MUST NOT be stored in plaintext in git; they MUST be
+# encrypted at rest in the repo and decrypted only in-cluster."
+# — spec docs/00-vision-and-scope.md; detail in docs/07.)
+#
+# docs/14 puts exactly this invariant at L0 ("Static/policy | Pre-apply, in CI |
+# ... no plaintext secrets"), so it is asserted here over the WHOLE tracked tree,
+# with no cluster. Run by `conformance/run.sh --layer l0` and by ci/l0.sh.
+#
+# GIVEN the repository as git tracks it,
+# WHEN every tracked file is inspected,
+# THEN (1) SOPS is configured at the repo root with a PUBLIC age recipient;
+# (2) every committed Kubernetes Secret is SOPS-encrypted — a top-level
+# `sops:` block with a mac AND every data/stringData value is ENC[...];
+# (3) the only two exempt classes are narrow and self-policing:
+# a. `*.example` templates, which must NOT be referenced by any
+# kustomization and whose secret-named values must be placeholders;
+# b. the Kyverno L0 negative fixture, which is exempt ONLY while it is
+# actually wired into a kyverno-test.yaml that asserts
+# `disallow-plaintext-secrets` REJECTS it (result: fail);
+# (4) every ansible-vault group_vars file is `$ANSIBLE_VAULT;`-encrypted, and
+# no secret-valued ansible key carries a literal outside a vault;
+# (5) no private key material of any kind is tracked (PEM private keys, an
+# age SECRET key, kubeconfig client-key-data);
+# (6) decryption is delegated to the cluster: a Flux Kustomization declares
+# `decryption.provider: sops` with a secretRef, and the private half of
+# that key pair is nowhere in the repo (5).
+#
+# WHAT THIS DOES *NOT* PROVE (honest gap, cluster layer):
+# - that the decrypted Secret only ever materialises in-cluster (the runtime half
+# of "decrypted only in-cluster") — needs a live cluster (L2), untested today.
+# - that no plaintext secret exists in git HISTORY. This walks the WORKING TREE at
+# HEAD only; docs/07's history-scan acceptance gate needs a secret scanner over
+# all revisions and is NOT implemented here.
+# - value-level entropy: a placeholder-looking string that is in fact a real
+# credential is not detectable statically.
+# =============================================================================
+set -euo pipefail
+DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+ROOT="$(cd "${DIR}/../../../.." && pwd)" # repo root
+PYTHON="${PYTHON:-python3}"
+
+cd "${ROOT}"
+"${PYTHON}" - "${ROOT}" <<'PY'
+import os, re, subprocess, sys
+from pathlib import Path
+
+try:
+ import yaml
+except ImportError: # pragma: no cover
+ print("check FAIL: PyYAML is required (same dependency as catalog/render.py)",
+ file=sys.stderr)
+ raise SystemExit(1)
+
+ROOT = Path(sys.argv[1])
+FAILURES: list[str] = []
+CHECKS = 0
+
+
+def check(cond, msg):
+ global CHECKS
+ CHECKS += 1
+ if not cond:
+ FAILURES.append(msg)
+
+
+def tracked() -> list[str]:
+ out = subprocess.run(["git", "-C", str(ROOT), "ls-files", "-z"],
+ capture_output=True, text=True, check=True).stdout
+ return [p for p in out.split("\0") if p]
+
+
+FILES = tracked()
+check(len(FILES) > 50, f"git ls-files returned only {len(FILES)} paths — not a real checkout")
+
+
+def docs_of(rel: str):
+ """Parsed YAML documents of a tracked file, or [] when it is not parseable YAML."""
+ p = ROOT / rel
+ try:
+ return [d for d in yaml.safe_load_all(p.read_text(errors="replace")) if isinstance(d, dict)]
+ except Exception:
+ return []
+
+
+YAMLISH = re.compile(r"\.(ya?ml)(\.example)?$")
+YAML_FILES = [f for f in FILES if YAMLISH.search(f)]
+
+# ---------------------------------------------------------------- (1) SOPS config
+sops_cfg = ROOT / ".sops.yaml"
+check(sops_cfg.is_file(), ".sops.yaml is missing from the repository root (R-SEC-01)")
+if sops_cfg.is_file():
+ cfg = yaml.safe_load(sops_cfg.read_text()) or {}
+ rules = cfg.get("creation_rules") or []
+ check(len(rules) > 0, ".sops.yaml declares no creation_rules — new secrets would be committed unencrypted")
+ for r in rules:
+ pr = r.get("path_regex", "")
+ recips = str(r.get("age", ""))
+ check(bool(recips), f".sops.yaml creation_rule {pr!r} has no age recipient")
+ check(all(x.startswith("age1") for x in re.split(r"[,\s]+", recips) if x),
+ f".sops.yaml creation_rule {pr!r} recipient is not a PUBLIC age1... key")
+ check("AGE-SECRET-KEY" not in recips.upper(),
+ f".sops.yaml creation_rule {pr!r} contains an age PRIVATE key")
+
+# --------------------------------------------- Kyverno negative-fixture exemption
+# A plaintext Secret is tolerated ONLY where it is the negative case that proves the
+# L0 policy rejects plaintext. We do not trust the path: the file must be listed in a
+# kyverno-test.yaml whose results assert disallow-plaintext-secrets returns `fail`.
+POLICY_NAME = "disallow-plaintext-secrets"
+negative_fixtures: set[str] = set()
+kyverno_suites = [f for f in FILES if os.path.basename(f) == "kyverno-test.yaml"]
+check(len(kyverno_suites) > 0, "no kyverno-test.yaml found — the L0 policy gate is not exercised at all")
+for suite in kyverno_suites:
+ docs = docs_of(suite)
+ if not docs:
+ continue
+ d = docs[0]
+ results = d.get("results") or []
+ rejects_plaintext = any(r.get("policy") == POLICY_NAME and r.get("result") == "fail"
+ for r in results if isinstance(r, dict))
+ if not rejects_plaintext:
+ continue
+ base = os.path.dirname(suite)
+ for res in d.get("resources") or []:
+ negative_fixtures.add(os.path.normpath(os.path.join(base, str(res))))
+
+check(len(negative_fixtures) > 0,
+ f"no kyverno-test.yaml asserts that {POLICY_NAME} REJECTS a plaintext Secret "
+ "(result: fail) — the plaintext-secret gate is unproven, so nothing may be exempt")
+
+# --------------------------------------------------- (2)/(3) committed Secret scan
+SECRET_KEY_RE = re.compile(r"(pass|secret|token|key|credential|cookie|cert)", re.I)
+PLACEHOLDER_RE = re.compile(r"^\s*(|<[^>]*>|replace[-_ ]?me.*|change[-_ ]?me.*|placeholder.*|x{3,})\s*$", re.I)
+
+secrets_seen = 0
+for rel in YAML_FILES:
+ for d in docs_of(rel):
+ if d.get("kind") != "Secret":
+ continue
+ secrets_seen += 1
+ values = []
+ for field in ("data", "stringData"):
+ for k, v in (d.get(field) or {}).items():
+ values.append((k, "" if v is None else str(v)))
+
+ is_example = rel.endswith(".example")
+ is_negative_fixture = os.path.normpath(rel) in negative_fixtures
+
+ if is_example:
+ # Template: never applied, and its secret-named values must be placeholders.
+ refs = [k for k in YAML_FILES
+ if os.path.basename(k) == "kustomization.yaml"
+ and os.path.basename(rel) in (ROOT / k).read_text(errors="replace")]
+ check(not refs,
+ f"{rel}: an unencrypted .example template is referenced by a kustomization "
+ f"({refs}) — it would be applied as a plaintext Secret")
+ for k, v in values:
+ if SECRET_KEY_RE.search(k):
+ check(bool(PLACEHOLDER_RE.match(v)),
+ f"{rel}: template key {k!r} holds {v[:24]!r}, not a placeholder — "
+ "a real credential may have been committed (R-SEC-01)")
+ continue
+
+ if is_negative_fixture:
+ # Exempt, but only as the proven negative case; assert it really is plaintext
+ # (an encrypted 'bad' fixture would silently stop testing anything).
+ continue
+
+ # Everything else: must be SOPS-encrypted at rest.
+ sops_block = d.get("sops")
+ check(isinstance(sops_block, dict) and bool(sops_block.get("mac")),
+ f"{rel}: committed Secret {d.get('metadata', {}).get('name')!r} has no top-level "
+ "`sops:` block with a mac — it is stored in plaintext (R-SEC-01)")
+ for k, v in values:
+ check(v.startswith("ENC["),
+ f"{rel}: Secret value {k!r} is not SOPS ciphertext (expected ENC[...]) — "
+ "plaintext secret material in git (R-SEC-01)")
+
+check(secrets_seen > 0, "no Kubernetes Secret manifests found at all — the scan is not looking at the repo")
+
+# ------------------------------------------------------------ (4) ansible vaults
+VAULT_HEADER = "$ANSIBLE_VAULT;"
+vault_files = [f for f in FILES
+ if os.path.basename(f) in ("vault.yml", "vault.yaml")]
+check(len(vault_files) > 0, "no ansible vault.yml files found — the vault scan is not looking at the repo")
+for rel in vault_files:
+ head = (ROOT / rel).read_text(errors="replace")[:len(VAULT_HEADER)]
+ check(head == VAULT_HEADER,
+ f"{rel}: an ansible group_vars vault file is NOT ansible-vault encrypted "
+ "(must start with the $ANSIBLE_VAULT header) — R-SEC-01")
+
+SECRET_ANSIBLE_KEYS = ("ansible_become_password", "ansible_become_pass", "ansible_password",
+ "ansible_ssh_pass", "ansible_sudo_pass", "ansible_ssh_private_key_file")
+ASSIGN_RE = re.compile(
+ r"^\s*(" + "|".join(SECRET_ANSIBLE_KEYS) + r")\s*:\s*(.*?)\s*$")
+for rel in FILES:
+ p = ROOT / rel
+ if not p.is_file() or p.is_symlink():
+ continue
+ try:
+ text = p.read_text(errors="replace")
+ except Exception:
+ continue
+ if text.startswith(VAULT_HEADER):
+ continue # encrypted at rest: opaque, by design
+ for lineno, line in enumerate(text.splitlines(), 1):
+ if line.lstrip().startswith("#"):
+ continue
+ m = ASSIGN_RE.match(line)
+ if not m:
+ continue
+ key, val = m.group(1), m.group(2).strip().strip("'\"")
+ ok = (val.startswith("{{") or val.startswith("$")
+ or bool(PLACEHOLDER_RE.match(val)) or val == "")
+ check(ok, f"{rel}:{lineno}: {key} is assigned the literal {val[:24]!r} outside an "
+ "ansible-vault file — a plaintext secret in git (R-SEC-01)")
+
+# ------------------------------------------------------ (5) no private key material
+# NOTE: each marker is assembled from fragments so that THIS file never contains the
+# literal string it hunts for — otherwise the scanner would flag itself the moment it
+# is tracked, and the only way to silence it would be to exempt the scanner (which is
+# exactly the kind of self-exemption this suite exists to prevent).
+_DASH5 = "-" * 5
+PRIVKEY_PATTERNS = [
+ (re.compile(_DASH5 + r"BEGIN (?:[A-Z ]+ )?PRIVATE" + r" KEY" + _DASH5),
+ "a PEM private key block"),
+ (re.compile("AGE" + "-SECRET" + "-KEY-1"), "an age SECRET (decryption) key"),
+ (re.compile(r"^\s*client" + r"-key-data\s*:\s*\S"), "kubeconfig client-key-data"),
+]
+for rel in FILES:
+ p = ROOT / rel
+ if not p.is_file() or p.is_symlink():
+ continue
+ try:
+ text = p.read_text(errors="replace")
+ except Exception:
+ continue
+ if text.startswith(VAULT_HEADER):
+ continue
+ for rx, what in PRIVKEY_PATTERNS:
+ for lineno, line in enumerate(text.splitlines(), 1):
+ if line.lstrip().startswith("#"):
+ continue
+ if rx.search(line):
+ check(False, f"{rel}:{lineno}: tracked file contains {what} (R-SEC-01)")
+
+# ----------------------------------------- (6) decryption is delegated to the cluster
+sops_decryption = []
+for rel in YAML_FILES:
+ for d in docs_of(rel):
+ if d.get("kind") != "Kustomization":
+ continue
+ dec = d.get("spec", {}).get("decryption") if isinstance(d.get("spec"), dict) else None
+ if isinstance(dec, dict) and dec.get("provider") == "sops" and dec.get("secretRef"):
+ sops_decryption.append(rel)
+ # Flux overlay patches carry the same block inside a patch body.
+for rel in YAML_FILES:
+ text = (ROOT / rel).read_text(errors="replace")
+ if re.search(r"^\s*decryption:\s*$", text, re.M) and re.search(r"^\s*provider:\s*sops\s*$", text, re.M) \
+ and re.search(r"^\s*secretRef:\s*$", text, re.M):
+ sops_decryption.append(rel)
+check(len(sops_decryption) > 0,
+ "no Flux Kustomization declares `decryption: {provider: sops, secretRef: ...}` — nothing "
+ "proves decryption happens in-cluster rather than from committed key material (R-SEC-01)")
+
+# ------------------------------------------- (7) the L0 policy gate is real and strict
+policy_files = [f for f in FILES if f.endswith(f"{POLICY_NAME}.yaml")]
+check(len(policy_files) == 1, f"expected exactly one {POLICY_NAME} ClusterPolicy, found {policy_files}")
+for rel in policy_files:
+ d = (docs_of(rel) or [{}])[0]
+ action = str(d.get("spec", {}).get("validationFailureAction", ""))
+ check(action == "Enforce" or action.startswith("${"),
+ f"{rel}: validationFailureAction is {action!r} — the no-plaintext-secrets gate must "
+ "Enforce (or carry the Flux token that substitutes to Enforce)")
+
+# --------------------------------------------------------------------------- verdict
+if FAILURES:
+ print(f"CHECK FAIL (no-plaintext-secrets): {len(FAILURES)} of {CHECKS} assertions failed",
+ file=sys.stderr)
+ for f in FAILURES:
+ print(f" - {f}", file=sys.stderr)
+ raise SystemExit(1)
+print(f"check OK: no-plaintext-secrets — {CHECKS} assertions over {len(FILES)} tracked files, "
+ f"{secrets_seen} committed Secret document(s), {len(vault_files)} ansible vault file(s)")
+PY
diff --git a/conformance/tests/l0-static/spec-immutability-enforced/chainsaw-test.yaml b/conformance/tests/l0-static/spec-immutability-enforced/chainsaw-test.yaml
new file mode 100644
index 0000000..91a54be
--- /dev/null
+++ b/conformance/tests/l0-static/spec-immutability-enforced/chainsaw-test.yaml
@@ -0,0 +1,46 @@
+# =============================================================================
+# L0 STATIC — the specification is immutable to agents (R-TEST-06)
+# Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster.
+#
+# A governance rule is made testable by testing its ENFORCEMENT MECHANISM, not by
+# observing that no agent misbehaved today.
+#
+# GIVEN the enforcement mechanism (ci/spec-immutability.sh + ci/SPEC_PIN),
+# WHEN it is driven against hermetic THROWAWAY spec repositories in each state,
+# THEN a clean spec at the pinned SHA passes; an uncommitted edit under docs/ or
+# decisions/, an untracked spec addition, and an unadopted committed drift each
+# exit 1 and block the merge; an absent spec repo exits 3 (honest skip, never
+# green); an unpinned SPEC_PIN fails closed; and the merge gate runs the stage
+# and blocks on its failure.
+#
+# THE REAL SPEC REPO IS NEVER TOUCHED: every case runs against a git repo created under
+# mktemp with SPEC_REPO pointed at it and a copy of ci/ carrying a fixture SPEC_PIN.
+#
+# STATUS: EXECUTABLE. `check.sh` is a pure offline assertion script; conformance/run.sh
+# --layer l0 and ci/l0.sh run it directly, and this Chainsaw Test wraps the SAME script.
+# NOT proven (named in check.sh's header): that the Gitea server REQUIRES this status
+# check before merge — branch protection is operator configuration outside the repo.
+# =============================================================================
+apiVersion: chainsaw.kyverno.io/v1alpha1
+kind: Test
+metadata:
+ name: spec-immutability-enforced
+ labels:
+ conformance.home.k3s/R-TEST-06: "true"
+ annotations:
+ conformance.home.k3s/requirements: "R-TEST-06"
+ conformance.home.k3s/layer: "L0"
+ conformance.home.k3s/safety-critical: "false"
+ conformance.home.k3s/executable: "true"
+ conformance.home.k3s/intent: >-
+ A spec modification is detected and blocks the merge; an unverifiable spec skips
+ honestly rather than passing; and the gate wires that stage as blocking.
+spec:
+ steps:
+ - name: drive-the-spec-immutability-gate
+ description: >-
+ Run the real ci/spec-immutability.sh against throwaway spec repos in every
+ state and assert its exit code, then assert the gate wires it as blocking.
+ try:
+ - script:
+ content: bash ./check.sh
diff --git a/conformance/tests/l0-static/spec-immutability-enforced/check.sh b/conformance/tests/l0-static/spec-immutability-enforced/check.sh
new file mode 100755
index 0000000..0c7bc3b
--- /dev/null
+++ b/conformance/tests/l0-static/spec-immutability-enforced/check.sh
@@ -0,0 +1,165 @@
+#!/usr/bin/env bash
+# =============================================================================
+# L0 static check (offline, no cluster) — spec-immutability-enforced
+# Serves: R-TEST-06 ("AI agents MUST NOT modify the specifications (this docs/ tree,
+# the requirements, or accepted ADRs). Specs are changed only by the human
+# operator/architect." — spec docs/00-vision-and-scope.md.)
+#
+# HOW A GOVERNANCE RULE IS MADE TESTABLE
+# -------------------------------------
+# "An agent must not do X" cannot be proven by observing that no agent did X today.
+# What CAN be proven — and is what actually protects the spec — is that the MECHANISM
+# which detects and blocks a spec modification works, and is wired into the merge gate
+# as a blocking stage. So this test drives the real `ci/spec-immutability.sh` against
+# HERMETIC THROWAWAY spec repositories and asserts its exit-code behaviour.
+#
+# THE REAL SPEC REPO IS NEVER TOUCHED. Every case runs against a fixture git repo under
+# mktemp, with SPEC_REPO pointed at it and a COPY of ci/ (so the fixture's SPEC_PIN can
+# be set without editing the tracked one). Nothing here writes outside its temp dir —
+# which is itself the R-TEST-06 discipline this test exists to defend.
+#
+# GIVEN the enforcement mechanism (ci/spec-immutability.sh + ci/SPEC_PIN),
+# WHEN it is run against a spec repo in each state,
+# THEN (1) clean and at the pinned SHA -> exit 0 (pass)
+# (2) an UNCOMMITTED edit under docs/ -> exit 1 (blocks the merge)
+# (3) an uncommitted edit under decisions/ -> exit 1 (accepted ADRs are spec too)
+# (4) a NEW COMMIT (HEAD drifted from pin) -> exit 1 (an "adopted" change must be
+# an explicit reviewed SPEC_PIN bump)
+# (5) an untracked file added to the spec -> exit 1 (porcelain catches additions)
+# (6) the spec repo absent -> exit 3 (honest SKIP, never green)
+# (7) an empty/comment-only SPEC_PIN -> exit 1 (an unpinned gate is no gate)
+# AND (8) the tracked ci/SPEC_PIN really is a 40-hex commit SHA, and
+# (9) the gate wires spec-immutability as a stage whose FAIL sets the merge block —
+# i.e. the mechanism above is load-bearing, not dead code.
+#
+# WHAT THIS DOES *NOT* PROVE (honest gap):
+# - That the Gitea server actually REQUIRES this status check before merge. Branch
+# protection is operator configuration outside the repo; nothing in-repo can assert it.
+# - Intent: an agent could still open a spec PR. The rule is enforced by the gate
+# blocking it, which is exactly what (1)-(9) assert; it is not enforced by prediction.
+# =============================================================================
+set -euo pipefail
+DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+ROOT="$(cd "${DIR}/../../../.." && pwd)" # repo root
+CI_SRC="${ROOT}/ci"
+
+PASS=0
+FAILED=0
+note() { printf ' ok %s\n' "$*"; PASS=$((PASS + 1)); }
+bad() { printf ' FAIL %s\n' "$*" >&2; FAILED=$((FAILED + 1)); }
+
+WORK="$(mktemp -d)"
+trap 'rm -rf "${WORK}"' EXIT
+
+# --- a hermetic stand-in for the spec repo -----------------------------------------
+# Mirrors the real spec's shape: a docs/ tree and a decisions/ tree, one commit.
+make_spec_repo() {
+ local d="$1"
+ mkdir -p "${d}/docs" "${d}/decisions"
+ # Fixture prose deliberately carries NO R-* token: ci/touched-requirements.sh scans the
+ # non-comment lines of changed files, so an example ID here would be attributed to the
+ # change as a touched requirement that does not exist in the registry.
+ printf 'normative spec text (fixture)\n' > "${d}/docs/00-vision-and-scope.md"
+ printf 'accepted decision\n' > "${d}/decisions/adr-0001-example.md"
+ git -C "$d" init -q -b main
+ git -C "$d" config user.email spec@example.invalid
+ git -C "$d" config user.name spec
+ git -C "$d" config commit.gpgsign false # throwaway: never sign, ignore host gpg config
+ git -C "$d" add -A
+ git -C "$d" commit -qm "spec baseline"
+}
+
+# A COPY of ci/ whose SPEC_PIN we may set freely. The script resolves CI_DIR from its own
+# location and sources lib.sh + versions.env from there, so a copy is self-contained.
+make_ci_copy() {
+ local dst="$1" pin="$2"
+ mkdir -p "$dst"
+ cp "${CI_SRC}/spec-immutability.sh" "${CI_SRC}/lib.sh" "${CI_SRC}/versions.env" "$dst/"
+ printf '%s\n' "$pin" > "${dst}/SPEC_PIN"
+ chmod +x "${dst}/spec-immutability.sh"
+}
+
+# run_case
+run_case() {
+ local want="$1" desc="$2" spec="$3" pin="$4" rc=0
+ local cidir; cidir="$(mktemp -d -p "$WORK" ci.XXXXXX)"
+ make_ci_copy "$cidir" "$pin"
+ SPEC_REPO="$spec" REPO_ROOT="$WORK" "${cidir}/spec-immutability.sh" >/dev/null 2>&1 || rc=$?
+ if [ "$rc" = "$want" ]; then note "${desc} -> exit ${rc}"; else
+ bad "${desc}: expected exit ${want}, got ${rc}"; fi
+}
+
+for f in spec-immutability.sh lib.sh versions.env SPEC_PIN; do
+ if [ -f "${CI_SRC}/${f}" ]; then note "ci/${f} present"; else bad "ci/${f} is missing"; fi
+done
+
+SPEC="${WORK}/spec"
+make_spec_repo "$SPEC"
+PINNED="$(git -C "$SPEC" rev-parse HEAD)"
+
+# (1) clean + at the pin -> pass.
+run_case 0 "clean spec at the pinned SHA passes" "$SPEC" "$PINNED"
+
+# (2) an uncommitted edit under docs/ -> block.
+printf 'normative spec text REWRITTEN BY AN AGENT\n' > "${SPEC}/docs/00-vision-and-scope.md"
+run_case 1 "uncommitted edit under docs/ BLOCKS" "$SPEC" "$PINNED"
+git -C "$SPEC" checkout -q -- docs/00-vision-and-scope.md
+
+# (3) an uncommitted edit to an accepted ADR -> block (decisions/ is spec too).
+printf 'reversed decision\n' > "${SPEC}/decisions/adr-0001-example.md"
+run_case 1 "uncommitted edit under decisions/ BLOCKS" "$SPEC" "$PINNED"
+git -C "$SPEC" checkout -q -- decisions/adr-0001-example.md
+
+# (5) an untracked addition to the spec tree -> block.
+printf 'a spec doc an agent invented\n' > "${SPEC}/docs/99-invented.md"
+run_case 1 "untracked file added to the spec BLOCKS" "$SPEC" "$PINNED"
+rm -f "${SPEC}/docs/99-invented.md"
+
+# clean again -> still passes (proves cases 2/3/5 failed for the edit, not permanently).
+run_case 0 "spec restored to clean passes again" "$SPEC" "$PINNED"
+
+# (4) a committed spec change that the pin has not adopted -> block.
+printf 'an extra normative line added by an agent\n' >> "${SPEC}/docs/00-vision-and-scope.md"
+git -C "$SPEC" add -A
+git -C "$SPEC" commit -qm "agent edits the spec"
+DRIFTED="$(git -C "$SPEC" rev-parse HEAD)"
+run_case 1 "committed spec drift from the pinned SHA BLOCKS" "$SPEC" "$PINNED"
+# ...and adopting it EXPLICITLY (a reviewed SPEC_PIN bump) is the only way to green.
+run_case 0 "an explicit SPEC_PIN bump adopts the advance" "$SPEC" "$DRIFTED"
+
+# (6) spec repo unavailable -> honest SKIP (exit 3), never a pass.
+run_case 3 "absent spec repo SKIPS (exit 3), never green" "${WORK}/no-such-spec" "$PINNED"
+
+# (7) an unpinned gate is not a gate.
+run_case 1 "empty SPEC_PIN FAILS closed" "$SPEC" "# only a comment, no SHA"
+
+# (8) the tracked pin is a real commit SHA.
+REAL_PIN="$(grep -vE '^\s*#' "${CI_SRC}/SPEC_PIN" | tr -d '[:space:]')"
+if printf '%s' "$REAL_PIN" | grep -Eq '^[0-9a-f]{40}$'; then
+ note "ci/SPEC_PIN is a 40-hex commit SHA"
+else
+ bad "ci/SPEC_PIN is not a 40-hex commit SHA: '${REAL_PIN}'"
+fi
+
+# (9) the mechanism is load-bearing: the merge gate runs it as a stage AND a FAIL there
+# sets the block. Asserted against the gate source because running the whole gate
+# from inside a conformance check would recurse into this very check.
+GATE="${CI_SRC}/gate.sh"
+if grep -Eq '^[[:space:]]*run_stage[[:space:]]+spec-immutability' "$GATE"; then
+ note "ci/gate.sh runs spec-immutability as a gate stage"
+else
+ bad "ci/gate.sh does not run spec-immutability as a stage — the gate is not wired"
+fi
+if awk '/OUTCOME\[spec-immutability\]/,/^esac/' "$GATE" | grep -Eq '^[[:space:]]*fail\).*BLOCK=1'; then
+ note "ci/gate.sh blocks the merge when spec-immutability FAILS"
+else
+ bad "ci/gate.sh does not set the merge block on a spec-immutability failure"
+fi
+
+echo
+if [ "$FAILED" -eq 0 ]; then
+ echo "check OK: spec-immutability-enforced — ${PASS} assertions (R-TEST-06)"
+ exit 0
+fi
+echo "CHECK FAIL (spec-immutability-enforced): ${FAILED} failed, ${PASS} passed" >&2
+exit 1