diff --git a/ci/conformance.sh b/ci/conformance.sh index b2efff0..18f033a 100755 --- a/ci/conformance.sh +++ b/ci/conformance.sh @@ -61,5 +61,45 @@ if [ -d "${CONF_ART}/coverage" ]; then cp -r "${CONF_ART}/coverage/." "${COVERAGE_DIR}/" fi +# --------------------------------------------------------------------------------------- +# DID IT ACTUALLY RUN? -- the guard without which this stage FAKES A GREEN. +# +# run.sh exits 0 in "scaffold mode" when it cannot reach a cluster: it emits a JUnit file +# whose every testcase is and returns success. That is correct FOR run.sh -- an +# honest skip is its documented no-cluster behaviour. It is catastrophic HERE, because +# reaching this point means the environment DECLARED CLUSTER_AVAILABLE=true, and this stage +# would otherwise print "L2/L3 conformance passed" over 54 tests that never executed. +# +# MEASURED 2026-09-29, before this guard existed: +# CLUSTER_AVAILABLE=true ./ci/conformance.sh -> rc=0 +# "[junit] wrote skipped JUnit for 54 scaffolded test(s)" +# "[ ok ] L2/L3 conformance passed for touched subset" +# One Gitea variable away from a fake green on an enforcing merge gate. That is exactly the +# HARD RULE 4 violation this repo's whole gate design exists to prevent, and nothing caught +# it because nobody had ever set the variable. +# +# A skip while a cluster is DECLARED is a contradiction: either the declaration is wrong or +# the cluster is unreachable. The merge-gate decision matrix already says this case is BLOCK +# ("conformance skipped while CLUSTER_AVAILABLE=true -> BLOCK (it should have run)"). This +# makes the stage itself honour that instead of reporting a pass. +JUNIT="${CONF_ART}/report.junit.xml" +if [ -f "$JUNIT" ]; then + total="$(grep -o ' **CORRECTION (2026-09-29): `CLUSTER_AVAILABLE` is NOT "the only remaining gate", and setting it +> today is actively DANGEROUS.** Measured: with `CLUSTER_AVAILABLE=true` and no reachable cluster, +> `conformance/run.sh` exits 0 in scaffold mode having marked all 54 tests ``, and +> `ci/conformance.sh` then printed `[ ok ] L2/L3 conformance passed for touched subset` — **rc 0, a +> fake green**, on a merge gate that is now an enforcing required status check. The CI runner has +> no cluster credential at all (`automountServiceAccountToken: false`, by design), so the "local +> run" this stage attempts can never succeed; it can only skip and, before the guard added on +> 2026-09-29, lie about it. +> +> `ci/conformance.sh` now BLOCKS (exit 1) when a cluster is declared but every test skipped. **Do +> not set `CLUSTER_AVAILABLE=true` until §3's trigger path actually exists** — the variable does +> not enable L2/L3, it only changes which way this stage fails. ---