From 6e77150ae1a8a171299886d3eaf35486789b75f1 Mon Sep 17 00:00:00 2001 From: Keith Jackson Date: Mon, 31 Aug 2026 23:26:49 -0500 Subject: [PATCH] fix(ci): kill the false-green base resolution; cover R-SEC-01/02 + R-TEST-06 (L0) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Unblocks the enforced Gitea gate, which correctly BLOCKED this branch on ['R-SEC-01','R-SEC-02','R-TEST-06'] being touched but uncovered. Tests were authored; the coverage rule was NOT narrowed. === THE FALSE GREEN (root cause) === `make -C ci gate` locally reported PASS having checked NOTHING. resolve_base() validated nothing, and EVERY way of failing to find a base degraded silently to an empty touched set + exit 0: an `HEAD~1` last resort, a two-dot diff fallback masking a failing three-dot diff, and no check that HEAD is even AHEAD of the base. Run while sitting on `main` (main == origin/main), `origin/main...HEAD` is empty -> "no changed files vs base" -> the empty set means run.sh is invoked with NO --requirements -> coverage.py's --require gate never runs at all. Green by vacuum. Now: candidates are BASE_REF > GITHUB_BASE_REF > origin/main > main, each verified to resolve. HEAD~1 and the two-dot fallback are gone. Unresolvable base / no merge base / failed diff / HEAD-not-ahead -> loud banner, EMPTY stdout, exit 3, and a machine-readable reason (on-base | no-base | diff-failed). gate.sh turns exit 3 into an UNDECIDED decision: `on-base` -> "NOT APPLICABLE — no branch under gate" (exit 0 so push-to-main runs don't break, but it never reads as PASS); `no-base`/`diff-failed` -> BLOCKED. It also now prints the diff base it used. A stale local main warns (widens, never falsely greens). Also fixed `make -C ci test`, which was SWALLOWING failing suites. Regression: ci/tests/touched-requirements-base.test.sh — 31 assertions over 8 throwaway-repo scenarios including the exact false-green case, an already-merged branch, no base ref, a single-commit repo, bogus BASE_REF, a shallow GITHUB_BASE_REF, and origin/main-preferred-over-stale-main. === NEW L0 STATIC LAYER (conformance/tests/l0-static/) === Where docs/14 actually puts these invariants. Each check.sh really executes offline via `run.sh --layer l0` and is wired into ci/l0.sh, so it runs on every PR. No scaffolds. R-SEC-01 no-plaintext-secrets (165 assertions / 445 tracked files): .sops.yaml holds only PUBLIC age recipients; every committed Secret is SOPS-wrapped with a mac and ENC[...] values; ansible vault.yml files are $ANSIBLE_VAULT; no secret-valued ansible key carries a literal outside a vault; no PEM key / age secret key / kubeconfig client-key-data is tracked; Flux declares sops decryption; the disallow-plaintext-secrets policy Enforces and has a proving `result: fail` row. Exemptions are SELF-POLICING, not path allowlists — a .example must be referenced by no kustomization and hold only placeholders, and the Kyverno fixture is exempt ONLY while a test asserts the policy rejects it. Proven against 7 injected violations, including deleting the negative row and weakening Enforce->Audit. R-SEC-02 least-privilege-rbac (41 assertions): no wildcard apiGroup/resource/verb and no escalate|bind|impersonate in any in-repo Role/ClusterRole; no ServiceAccount bound to cluster-admin or a wildcard role; the conformance runner's own test identity is scoped and holds no rights over RBAC objects; the default-deny/host-namespace/egress-class policies Enforce and each carry a negative row. Proven against 6 injected violations. R-TEST-06 spec-immutability-enforced (16 assertions): drives the REAL ci/spec-immutability.sh against HERMETIC throwaway spec repos and asserts its exit code in every state — clean-at-pin 0, edits under docs/ or decisions/ 1, untracked addition 1, unadopted drift 1, explicit pin bump 0, ABSENT spec repo 3 (honest skip, never green), unpinned SPEC_PIN 1 — plus that gate.sh sets BLOCK=1 on its failure, so the mechanism is load-bearing. The real spec repo is never written to. Each check documents what it does NOT cover (runtime decryption, git history, runtime least-privilege, branch protection) rather than overclaiming. status.py now reads check.sh too, so a placeholder L0 body still outranks an `executable: true` annotation. Coverage 37/51 -> 40/51 (11 gaps); executable tests 32 -> 35; scaffold-only stays 5 — no sixth scaffold was added. Serves: R-SEC-01, R-SEC-02, R-TEST-01..06. Co-Authored-By: Claude Opus 5 (1M context) --- ci/Makefile | 6 +- ci/README.md | 50 ++- ci/gate.sh | 64 +++- ci/l0.sh | 27 ++ ci/l1.sh | 13 +- ci/lib.sh | 35 +++ ci/tests/touched-requirements-base.test.sh | 169 +++++++++++ ci/touched-requirements.sh | 133 ++++++-- conformance/lib/common.sh | 3 +- conformance/lib/coverage.py | 4 +- conformance/lib/emit_skipped_junit.py | 3 +- conformance/lib/run_l1_offline.py | 61 ++-- conformance/lib/status.py | 13 +- conformance/run.sh | 75 +++-- conformance/tests/README.md | 37 ++- .../least-privilege-rbac/chainsaw-test.yaml | 44 +++ .../l0-static/least-privilege-rbac/check.sh | 269 ++++++++++++++++ .../no-plaintext-secrets/chainsaw-test.yaml | 43 +++ .../l0-static/no-plaintext-secrets/check.sh | 287 ++++++++++++++++++ .../chainsaw-test.yaml | 46 +++ .../spec-immutability-enforced/check.sh | 165 ++++++++++ 21 files changed, 1450 insertions(+), 97 deletions(-) create mode 100755 ci/tests/touched-requirements-base.test.sh create mode 100644 conformance/tests/l0-static/least-privilege-rbac/chainsaw-test.yaml create mode 100755 conformance/tests/l0-static/least-privilege-rbac/check.sh create mode 100644 conformance/tests/l0-static/no-plaintext-secrets/chainsaw-test.yaml create mode 100755 conformance/tests/l0-static/no-plaintext-secrets/check.sh create mode 100644 conformance/tests/l0-static/spec-immutability-enforced/chainsaw-test.yaml create mode 100755 conformance/tests/l0-static/spec-immutability-enforced/check.sh diff --git a/ci/Makefile b/ci/Makefile index 632926f..3d426da 100644 --- a/ci/Makefile +++ b/ci/Makefile @@ -44,7 +44,11 @@ touched: ## Print the touched-requirement subset for the current branch vs main @"$(CI_DIR)touched-requirements.sh" test: ## Run ci/ script unit tests (no cluster; e.g. touched-requirements content scan) - @for t in "$(CI_DIR)"tests/*.test.sh; do echo "== $$t"; bash "$$t"; done + @rc=0; for t in "$(CI_DIR)"tests/*.test.sh; do \ + echo "== $$t"; bash "$$t" || rc=1; \ + done; \ + if [ $$rc -ne 0 ]; then echo "ci/tests: FAILED" >&2; else echo "ci/tests: all suites passed"; fi; \ + exit $$rc versions: ## Print pinned tool versions (R-TEST-02) @cat "$(CI_DIR)versions.env" diff --git a/ci/README.md b/ci/README.md index eac3253..bec4949 100644 --- a/ci/README.md +++ b/ci/README.md @@ -20,17 +20,54 @@ Individual stages: ```bash make -C ci lint # yaml/lint (no cluster; live now) -make -C ci l0 # ServiceDefinition schema + Kyverno (no cluster; live now) +make -C ci l0 # schema + Kyverno + L0 static conformance (no cluster; live now) make -C ci l1 # L1 rendering checks (no cluster; live now) make -C ci conformance # L2/L3 in-cluster Chainsaw Job (SKIPS unless CLUSTER_AVAILABLE=true) +make -C ci test # unit tests for these scripts themselves (ci/tests/*.test.sh) ``` +The `l0` stage has three sub-stages: ServiceDefinition schema validation, the Kyverno policy gate +(`kyverno test`), and **`conformance/run.sh --layer l0`** — the whole-repo L0 invariants a +per-resource policy cannot express (secret hygiene across every tracked file, least-privilege RBAC, +spec-immutability enforcement). See `conformance/tests/README.md`. + ## Exit-code contract Every stage script uses: **`0` = pass · `1` = fail (blocks merge) · `3` = SKIPPED** (cluster-gated with no cluster, or an unwired seam / missing tool). A `3` is **never** counted as green — that is how the gate stays honest in Phase 0 (no fake green). +### The touched-requirement subset can also be *undeterminable* (exit 3) + +`touched-requirements.sh` is the input to the R-TEST-03 decision. It resolves a diff base in this +order, **verifying each candidate exists** before using it: + +``` +BASE_REF (explicit) > GITHUB_BASE_REF (Actions PR runs) > origin/main > main +``` + +If no candidate resolves, if there is no merge base, if the diff command fails, or if **HEAD is not +ahead of the base** (you are sitting on `main`, or the branch is already merged), it prints a loud +banner, writes **nothing** to stdout, and exits **3** — plus a machine-readable reason code +(`on-base` / `no-base` / `diff-failed`) to `$TOUCHED_REASON_FILE` when that variable names a path. + +**Why this is not cosmetic.** Every one of those conditions used to degrade *silently* to an empty +touched set and exit 0. An empty set means `conformance/run.sh` is invoked with no `--requirements`, +which means its coverage `--require` gate never runs — so on 2026-08-31 the local gate printed +`MERGE GATE: PASS` having checked nothing, while CI (correct base, 17 touched requirements) BLOCKED +on real coverage debt. There is no silent narrowing left: the `HEAD~1` last resort and the two-dot +diff fallback are both gone. Locked down by `ci/tests/touched-requirements-base.test.sh`. + +`gate.sh` turns exit 3 into an **UNDECIDED** merge decision, never a pass: + +| reason | headline | exit | +|---|---|---| +| `on-base` | `MERGE GATE: NOT APPLICABLE — no branch under gate` | 0 (a push-to-`main` CI run legitimately lands here; it just must not read as "my branch is green") | +| `no-base`, `diff-failed` | `MERGE GATE: BLOCKED` | 1 (the environment cannot support a merge decision; refusing to decide must not look like deciding yes) | + +A stage that needs the subset (`l0.sh`, `l1.sh`) still runs its FULL layer as a superset sanity net +in that case — nothing goes untested — and says plainly that the *scoped* check did not happen. + ## Reproducibility (R-TEST-02) - **Version-pinned:** all tool versions live in [`versions.env`](versions.env) — the single source of @@ -43,16 +80,19 @@ the gate stays honest in Phase 0 (no fake green). ``` ci/ - Makefile targets: lint · l0 · l1 · conformance · gate · touched · tools · versions + Makefile targets: lint · l0 · l1 · conformance · gate · touched · test · tools · versions versions.env PINNED tool versions (R-TEST-02) — single source of truth lib.sh shared helpers: logging, tool checks, cluster detection, artifact paths yamllint.yaml lint config for the yaml/lint stage lint.sh Stage: yaml/lint - l0.sh Stage: ServiceDefinition schema validation + Kyverno L0 policy gate + l0.sh Stage: schema validation + Kyverno L0 gate + L0 static conformance l1.sh Stage: L1 rendering checks (Chainsaw L1 via the conformance runner) conformance.sh Stage: L2/L3 in-cluster Chainsaw Job (trigger + collect) - touched-requirements.sh compute the R-* subset a change touches (drives the R-TEST-03 filter) + touched-requirements.sh compute the R-* subset a change touches (drives the R-TEST-03 filter); + exits 3 LOUDLY rather than guessing when the diff base is unreliable gate.sh run all stages + the merge decision (the single command) + tests/*.test.sh unit tests for these scripts (`make -C ci test`), run against + throwaway git repos via the REPO_ROOT override — no network, no cluster README.md this file ``` @@ -67,7 +107,7 @@ below are this workstream's **assumptions** — flagged for the PM to verify at |---|---|---|---|---| | ServiceDefinition schema validator | WS-GITOPS · **T-0.3** (`schemas/`) | `SCHEMA_VALIDATE_CMD` | `schemas/validate.sh` | `validate.sh catalog/*.yaml` → exit 0/1; `--version` supported | | Kyverno L0 policies + tests | WS-TEST · **T-0.5** (`platform/kyverno/`) | `KYVERNO_POLICY_DIR` | `platform/kyverno` | contains `kyverno-test.yaml` files discoverable by `kyverno test ` | -| Conformance runner (Chainsaw) | WS-TEST · **T-0.4** (`conformance/`) | `CONFORMANCE_CMD` | `conformance/run.sh` | `run.sh --layer [--requirements R-A,R-B] [--trigger in-cluster-job] --junit --coverage ` | +| Conformance runner (Chainsaw) | WS-TEST · **T-0.4** (`conformance/`) | `CONFORMANCE_CMD` | `conformance/run.sh` | `run.sh --layer [--requirements R-A,R-B]`; artifacts land in `conformance/_artifacts/` | | Catalog (ServiceDefinitions) | WS-APPS | `CATALOG_DIR` | `catalog` | `*.yaml` ServiceDefinition files | Artifact locations (git-ignored): JUnit → `results/junit/*.xml`; coverage → `reports/coverage/**`. diff --git a/ci/gate.sh b/ci/gate.sh index 3a54ba4..8df27b4 100755 --- a/ci/gate.sh +++ b/ci/gate.sh @@ -48,6 +48,32 @@ echo >&2 log "==== gate decision ====" BLOCK=0 +# --- R-TEST-03 input: the touched-requirement subset -------------------------- +# Computed ONCE here. ci/touched-requirements.sh exits 3 when it cannot establish a +# reliable diff base; that used to degrade to an empty set, which silently disabled the +# coverage `--require` gate and let the gate print "MERGE GATE: PASS" having checked +# nothing (the 2026-08-31 local false green). An undeterminable subset is now surfaced +# as an UNDECIDED merge decision, never as a pass. +_reason_file="$(mktemp)" +# NOTE: command substitution, NOT `mapfile < <(...)` — process substitution does not +# propagate the child's exit status, and the whole point here is to SEE the exit 3. +TOUCHED_OUT="$(TOUCHED_REASON_FILE="$_reason_file" "${CI_DIR}/touched-requirements.sh")" +TOUCHED_RC=$? +TOUCHED_REASON="$(cat "$_reason_file" 2>/dev/null || true)" +rm -f "$_reason_file" +TOUCHED=() +[ -n "$TOUCHED_OUT" ] && mapfile -t TOUCHED <<< "$TOUCHED_OUT" +UNDECIDED=0 +if [ "$TOUCHED_RC" -eq 3 ]; then + UNDECIDED=1 + echo >&2 + err "R-TEST-03 DECISION INPUT MISSING: the touched-requirement subset could not be" + err " determined (${TOUCHED_REASON:-unknown}). The scoped coverage gate did NOT run." + summary "- **R-TEST-03 subset UNDETERMINED** (\`${TOUCHED_REASON:-unknown}\`) — the scoped coverage gate did not run." +elif [ "${#TOUCHED[@]}" -gt 0 ]; then + log "touched requirements: ${TOUCHED[*]}" +fi + # spec-immutability (R-TEST-06): a FAIL always blocks (the spec was modified/drifted). # A skip means the spec repo wasn't available to verify — surfaced, not green. case "${OUTCOME[spec-immutability]}" in @@ -77,9 +103,11 @@ case "${OUTCOME[conformance]}" in skip) if cluster_available; then err "conformance: skipped despite CLUSTER_AVAILABLE=true -> blocks merge"; BLOCK=1 + elif [ "$UNDECIDED" -eq 1 ]; then + warn "conformance L2/L3 UNVERIFIED, and the touched subset is unknown — nothing certified." else - mapfile -t reqs < <("${CI_DIR}/touched-requirements.sh" || true) - if [ "${#reqs[@]}" -gt 0 ]; then + if [ "${#TOUCHED[@]}" -gt 0 ]; then + reqs=("${TOUCHED[@]}") warn "conformance L2/L3 UNVERIFIED for touched requirements: ${reqs[*]}" warn " (no cluster — Phase 0). These are NOT certified green." summary "- **UNVERIFIED (no cluster):** ${reqs[*]}" @@ -93,15 +121,35 @@ case "${OUTCOME[conformance]}" in fi ;; esac +# An UNDECIDED subset is never a pass. Two outcomes, by reason: +# on-base -> this checkout has no branch commits to gate (you are on main, or the +# branch is already merged; a push-to-main CI run lands here). Legitimate, +# so the job is not failed — but the headline says NOT APPLICABLE, never +# PASS, so nobody reads it as "my branch is green". +# anything -> the environment cannot support a merge decision (no base ref, broken +# else diff). Refusing to decide must not look like deciding yes: BLOCK. +if [ "$UNDECIDED" -eq 1 ] && [ "$TOUCHED_REASON" != "on-base" ]; then + err "cannot compute the R-TEST-03 subset (${TOUCHED_REASON:-unknown}) -> blocks merge" + BLOCK=1 +fi + echo >&2 -if [ "$BLOCK" -eq 0 ]; then - ok "MERGE GATE: PASS (touched-requirement subset green for the layers that can run)" - summary "" - summary "**Result: PASS** — L0/L1 green. L2/L3 status per cluster availability above." - exit 0 -else +if [ "$BLOCK" -ne 0 ]; then err "MERGE GATE: BLOCKED" summary "" summary "**Result: BLOCKED** — see failures above." exit 1 +elif [ "$UNDECIDED" -eq 1 ]; then + warn "MERGE GATE: NOT APPLICABLE — no branch under gate (${TOUCHED_REASON})." + warn " The stages above ran and did not fail, but NO touched-requirement subset was" + warn " checked. This is NOT a green branch: check out the branch you mean to gate" + warn " (or set BASE_REF) and run it again." + summary "" + summary "**Result: NOT APPLICABLE** — no branch under gate (\`${TOUCHED_REASON}\`); the R-TEST-03 subset was not checked." + exit 0 +else + ok "MERGE GATE: PASS (touched-requirement subset green for the layers that can run)" + summary "" + summary "**Result: PASS** — L0/L1 green. L2/L3 status per cluster availability above." + exit 0 fi diff --git a/ci/l0.sh b/ci/l0.sh index 9708d2d..d4dc4cc 100755 --- a/ci/l0.sh +++ b/ci/l0.sh @@ -7,6 +7,9 @@ # Stage 3 L0 : `kyverno test` over platform/kyverno/ — default-deny baseline # present, no plaintext secrets, required labels/egress-class, # ServiceDefinition schema conformance (owner: T-0.5, ADR-0011). +# Stage 3b L0 : conformance/run.sh --layer l0 — the WHOLE-REPO L0 invariants that +# a per-resource policy cannot express (secret hygiene across every +# tracked file, least-privilege RBAC, spec-immutability enforcement). # # Integration seams are env-overridable (see ci/lib.sh + README). When an owning # subtree is still a placeholder, this reports SKIPPED (exit 3) for that sub-stage @@ -73,6 +76,30 @@ else rm -rf "$l0_render" fi +# ---- Stage 3b: L0 STATIC conformance checks --------------------------------- +# docs/14's L0 row is "manifests/ServiceDefinitions obey invariants", which is wider +# than what Kyverno alone can express over single resources: whole-repo invariants +# (no plaintext secrets anywhere in git, least-privilege RBAC across every tracked +# binding, the spec-immutability gate's own behaviour) live in the conformance +# harness's tests/l0-static subtree and run with no cluster. This stage executes them +# through the same single-command runner CI and a laptop both use (R-TEST-02). +log "Stage: L0 static conformance checks (${CONFORMANCE_CMD} --layer l0)" +if ! entrypoint_ready "$CONFORMANCE_CMD"; then + skip "conformance runner not present at '${CONFORMANCE_CMD}' — UNWIRED, not green" + summary "- l0-static: **SKIPPED** (conformance runner missing)" + if [ "$rc" -eq 0 ]; then rc=3; fi +else + load_touched_requirements + if "$CONFORMANCE_CMD" --layer l0 ${REQ_ARGS[@]+"${REQ_ARGS[@]}"}; then + ok "L0 static conformance checks passed" + summary "- l0-static: **passed**" + else + err "L0 static conformance checks failed" + summary "- l0-static: **FAILED**" + rc=1 + fi +fi + case "$rc" in 0) ok "L0 stage complete" ;; 3) skip "L0 stage incomplete (unwired seam / missing tool) — not a pass" ;; diff --git a/ci/l1.sh b/ci/l1.sh index 98707e3..0735146 100755 --- a/ci/l1.sh +++ b/ci/l1.sh @@ -20,19 +20,16 @@ if ! entrypoint_ready "$CONFORMANCE_CMD"; then exit 3 fi -# Touched-requirement subset (R-TEST-03). Empty subset => run all L1 as a sanity net. -mapfile -t reqs < <("${CI_DIR}/touched-requirements.sh" || true) -req_args=() -if [ "${#reqs[@]}" -gt 0 ]; then - log "touched requirements: ${reqs[*]}" - req_args=(--requirements "$(IFS=,; echo "${reqs[*]}")") -fi +# Touched-requirement subset (R-TEST-03). An empty OR undeterminable subset => run all +# L1 as a superset sanity net, and SAY SO — see load_touched_requirements in ci/lib.sh. +load_touched_requirements +req_args=(${REQ_ARGS[@]+"${REQ_ARGS[@]}"}) # RECONCILED (PM 2026-08-16): conformance/run.sh's real interface is `--layer ` # [--requirements R-A,R-B]; it ALWAYS writes artifacts to a FIXED location # (conformance/_artifacts/) and does NOT accept --junit/--coverage. Invoke by layer + # touched subset, then COLLECT its artifacts into the CI artifact dirs. -"$CONFORMANCE_CMD" --layer l1 "${req_args[@]}" +"$CONFORMANCE_CMD" --layer l1 ${req_args[@]+"${req_args[@]}"} CONF_ART="$(dirname "$CONFORMANCE_CMD")/_artifacts" if [ -f "${CONF_ART}/report.junit.xml" ]; then diff --git a/ci/lib.sh b/ci/lib.sh index 0e6cbc2..9a0bf27 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -85,3 +85,38 @@ cluster_available() { } mkartifacts() { mkdir -p "$JUNIT_DIR" "$COVERAGE_DIR"; } + +# --- Touched-requirement subset (R-TEST-03 input) ---------------------------- +# Sets: TOUCHED_RC (0 ok / 3 not determinable), TOUCHED_REQS[] and REQ_ARGS[] +# (ready to splice into a conformance/run.sh invocation). +# +# ci/touched-requirements.sh exits 3 — LOUDLY — when it cannot establish a reliable +# diff base, instead of the old silent degradation to an empty set. An empty set +# means run.sh gets no `--requirements`, which means its coverage `--require` gate +# never runs; treating "could not determine" as "nothing touched" is precisely the +# false green this helper exists to prevent. A stage still runs its FULL layer in +# that case (a superset, so nothing goes untested), but says plainly that the +# SCOPED check did not happen; ci/gate.sh turns that into an UNDECIDED decision. +load_touched_requirements() { + local out rc=0 + out="$("${CI_DIR}/touched-requirements.sh")" || rc=$? + TOUCHED_RC=$rc + TOUCHED_REQS=() + # `if`, not `[ ... ] && ...`: under `set -e` a false AND-list is the caller's last + # command status and would abort the stage script on an EMPTY touched set. + if [ -n "$out" ]; then mapfile -t TOUCHED_REQS <<< "$out"; fi + REQ_ARGS=() + if [ "$rc" -eq 3 ]; then + warn "touched-requirement subset UNDETERMINED (reason logged above)." + warn " Running the FULL layer as a superset sanity net; the SCOPED R-TEST-03" + warn " coverage check is NOT performed here — ci/gate.sh reports that as UNDECIDED." + elif [ "$rc" -ne 0 ]; then + warn "touched-requirements.sh exited ${rc}; proceeding with the full layer." + elif [ "${#TOUCHED_REQS[@]}" -gt 0 ]; then + log "touched requirements: ${TOUCHED_REQS[*]}" + REQ_ARGS=(--requirements "$(IFS=,; echo "${TOUCHED_REQS[*]}")") + else + log "empty touched-requirement set (base WAS established) — running the full layer." + fi + return 0 +} diff --git a/ci/tests/touched-requirements-base.test.sh b/ci/tests/touched-requirements-base.test.sh new file mode 100755 index 0000000..45a7ebf --- /dev/null +++ b/ci/tests/touched-requirements-base.test.sh @@ -0,0 +1,169 @@ +#!/usr/bin/env bash +# ci/tests/touched-requirements-base.test.sh — regression tests for BASE-REF RESOLUTION +# in ci/touched-requirements.sh. +# +# THE BUG THESE LOCK DOWN (2026-08-31, local/CI divergence): +# `make -C ci gate` on a feature branch printed +# "no changed files vs base; empty touched-requirement set" +# and then "MERGE GATE: PASS" — while CI, with a correct base, computed 17 touched +# requirements and BLOCKED on real coverage debt. Root cause: EVERY way of failing to +# establish a diff base degraded silently to an EMPTY touched set and exit 0. An empty +# set means conformance/run.sh is invoked without `--requirements`, which means its +# coverage `--require` gate never runs — so the gate reported PASS having checked +# nothing. A false green is worse than a red. +# +# The fix: an undeterminable base is a LOUD exit 3 (ci/README.md's SKIPPED code) with +# EMPTY stdout and a machine-readable reason code, never exit 0. These tests assert +# that for each way the base can be undeterminable, and assert the good paths still work. +# +# The tests point the real script at THROWAWAY git repos via the REPO_ROOT override +# (ci/lib.sh honours a pre-set REPO_ROOT); the script runs from its true location so it +# still sources the real lib.sh. No network, no cluster, and the working repo is never +# touched. +# +# Run: ci/tests/touched-requirements-base.test.sh (or `make -C ci test`) +set -uo pipefail # NOT -e: these tests deliberately run a script that exits non-zero. + +CI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +SCRIPT="${CI_DIR}/touched-requirements.sh" + +PASS=0 FAIL=0 +ok() { printf ' ok %s\n' "$*"; PASS=$((PASS+1)); } +bad() { printf ' FAIL %s\n' "$*" >&2; FAIL=$((FAIL+1)); } + +TMP="$(mktemp -d)" +trap 'rm -rf "$TMP"' EXIT + +# newrepo — a throwaway repo with one baseline commit on `main`. +newrepo() { + local d="$1" + mkdir -p "$d/notes" + git -C "$d" init -q -b main + git -C "$d" config user.email t@t.t + git -C "$d" config user.name t + git -C "$d" config commit.gpgsign false # throwaway: never sign (ignore host gpg config) + echo baseline > "$d/notes/base.txt" + git -C "$d" add -A && git -C "$d" commit -qm baseline +} + +# commit_touching — a commit whose content declares an R-ID. +commit_touching() { + local d="$1" text="$2" + printf 'name: %s\n' "$text" > "$d/notes/decl.yaml" + git -C "$d" add -A && git -C "$d" commit -qm change +} + +# Runs the script in ; sets RC, OUT (stdout), ERRTXT (stderr), REASON. +run_in() { + local d="$1"; shift + local reason_file="${TMP}/reason.$$" + : > "$reason_file" + OUT="$(REPO_ROOT="$d" TOUCHED_REASON_FILE="$reason_file" "$@" "$SCRIPT" 2>"${TMP}/err.$$")" + RC=$? + ERRTXT="$(cat "${TMP}/err.$$")" + REASON="$(cat "$reason_file")" +} + +# assert_refusal