fix(ci): CLUSTER_AVAILABLE=true would have FAKED A GREEN over 54 skipped tests

Wiring L2/L3 into CI was a tracked next step, described in the plan as needing only
vars.CLUSTER_AVAILABLE. Attempting it surfaced that the variable does not enable L2/L3 at
all -- it only changes which way the stage fails, and before this commit one of those ways
was a false pass.

MEASURED, before the guard:
  CLUSTER_AVAILABLE=true ./ci/conformance.sh   ->  rc=0
  "[junit] wrote skipped JUnit for 54 scaffolded test(s)"
  "[ ok ] L2/L3 conformance passed for touched subset"

run.sh exits 0 in scaffold mode when it cannot reach a cluster, marking every test
<skipped>. That is correct and honest FOR run.sh. It is catastrophic in ci/conformance.sh,
which reaches that code only because the environment DECLARED a cluster available -- so it
reported a pass over 54 tests that never executed. One Gitea variable away from a fake green
on a gate that is now an ENFORCING required status check, and nothing caught it because
nobody had ever set the variable.

The CI runner holds no cluster credential by design (automountServiceAccountToken: false),
so the local run this stage attempts can never succeed. The in-cluster trigger path that
would make L2/L3 actually run is still unimplemented, and is blocked on a privilege-
escalation design decision that is the operator's to make (plan section 4: any principal
that can `create jobs` in ns conformance can run arbitrary code as the privileged
conformance-runner SA, because Kubernetes places no `use` check on serviceAccountName).

The stage now BLOCKS (exit 1) when a cluster is declared but every test skipped, which is
what the merge-gate decision matrix already said should happen ("conformance skipped while
CLUSTER_AVAILABLE=true -> BLOCK"). Verified: rc=1, naming the count and the reason.

The plan document claimed CLUSTER_AVAILABLE was "the only remaining gate". Corrected in
place with the measurement and an explicit do-not-set-this-yet.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

ci/conformance.sh

@@ -61,5 +61,45 @@ if [ -d "${CONF_ART}/coverage" ]; then
   cp -r "${CONF_ART}/coverage/." "${COVERAGE_DIR}/"
 fi
 
+# ---------------------------------------------------------------------------------------
+# DID IT ACTUALLY RUN? -- the guard without which this stage FAKES A GREEN.
+#
+# run.sh exits 0 in "scaffold mode" when it cannot reach a cluster: it emits a JUnit file
+# whose every testcase is <skipped> and returns success. That is correct FOR run.sh -- an
+# honest skip is its documented no-cluster behaviour. It is catastrophic HERE, because
+# reaching this point means the environment DECLARED CLUSTER_AVAILABLE=true, and this stage
+# would otherwise print "L2/L3 conformance passed" over 54 tests that never executed.
+#
+# MEASURED 2026-09-29, before this guard existed:
+#   CLUSTER_AVAILABLE=true ./ci/conformance.sh   ->  rc=0
+#   "[junit] wrote skipped JUnit for 54 scaffolded test(s)"
+#   "[ ok ] L2/L3 conformance passed for touched subset"
+# One Gitea variable away from a fake green on an enforcing merge gate. That is exactly the
+# HARD RULE 4 violation this repo's whole gate design exists to prevent, and nothing caught
+# it because nobody had ever set the variable.
+#
+# A skip while a cluster is DECLARED is a contradiction: either the declaration is wrong or
+# the cluster is unreachable. The merge-gate decision matrix already says this case is BLOCK
+# ("conformance skipped while CLUSTER_AVAILABLE=true -> BLOCK (it should have run)"). This
+# makes the stage itself honour that instead of reporting a pass.
+JUNIT="${CONF_ART}/report.junit.xml"
+if [ -f "$JUNIT" ]; then
+  total="$(grep -o '<testcase' "$JUNIT" | wc -l | tr -d ' ')"
+  skipped="$(grep -o '<skipped' "$JUNIT" | wc -l | tr -d ' ')"
+  if [ "${total:-0}" -gt 0 ] && [ "${skipped:-0}" -ge "${total:-0}" ]; then
+    err "CLUSTER_AVAILABLE=true, but the L2/L3 suite SKIPPED every one of its ${total} tests"
+    err "  (scaffold mode -- run.sh could not reach a cluster and exited 0)."
+    err "  This stage will NOT report that as a pass. Either the cluster is unreachable from"
+    err "  this runner, or CLUSTER_AVAILABLE is set when it should not be."
+    err "  NOTE: the in-cluster trigger path is still UNIMPLEMENTED -- see"
+    err "  docs/plans/l2-l3-conformance-ci-wiring.md. Setting CLUSTER_AVAILABLE=true does not"
+    err "  by itself make L2/L3 run; it only makes this stage attempt a LOCAL run that the CI"
+    err "  runner has no credential for (automountServiceAccountToken: false, by design)."
+    summary "- conformance-l2-l3: **BLOCKED — declared available but every test skipped** (not a pass)"
+    exit 1
+  fi
+  log "L2/L3 executed: ${total} testcase(s), ${skipped} skipped"
+fi
+
 ok "L2/L3 conformance passed for touched subset"
 summary "- conformance-l2-l3: **passed**"

docs/plans/l2-l3-conformance-ci-wiring.md

@@ -38,8 +38,21 @@ agents. So the credential model must be decided by the operator before anything
 | Runner deployment | `apps/wave-1/gitea-actions.yaml` | act_runner + DinD (Docker executor). **`automountServiceAccountToken: false`** — the runner and its job containers hold NO SA token. Broad egress allowed (`- {}`), so it can *reach* the kube-API VIP `192.168.1.49:6443`, but has no credential. |
 | Secrets encryption | `.sops.yaml` (repo root) | age recipient `age175l8m…mfzg`; encrypts `data`/`stringData` of `*.sops.yaml` under `platform|apps|catalog/**` and `secrets/**`. Private key is the in-cluster `sops-age` Secret Flux uses. |
 
-**Cluster reality:** the "Phase 0 / no cluster" language in these files is stale — the dev cluster
-is LIVE (Phase 6). So `CLUSTER_AVAILABLE` is the only remaining gate, exactly as the blueprint says.
+**Cluster reality:** the "Phase 0 / no cluster" language in these files is stale — the cluster is
+LIVE (and it is now PROD; dev was decommissioned 2026-09-12).
+
+> **CORRECTION (2026-09-29): `CLUSTER_AVAILABLE` is NOT "the only remaining gate", and setting it
+> today is actively DANGEROUS.** Measured: with `CLUSTER_AVAILABLE=true` and no reachable cluster,
+> `conformance/run.sh` exits 0 in scaffold mode having marked all 54 tests `<skipped>`, and
+> `ci/conformance.sh` then printed `[ ok ] L2/L3 conformance passed for touched subset` — **rc 0, a
+> fake green**, on a merge gate that is now an enforcing required status check. The CI runner has
+> no cluster credential at all (`automountServiceAccountToken: false`, by design), so the "local
+> run" this stage attempts can never succeed; it can only skip and, before the guard added on
+> 2026-09-29, lie about it.
+>
+> `ci/conformance.sh` now BLOCKS (exit 1) when a cluster is declared but every test skipped. **Do
+> not set `CLUSTER_AVAILABLE=true` until §3's trigger path actually exists** — the variable does
+> not enable L2/L3, it only changes which way this stage fails.
 
 ---