fix(ci): CLUSTER_AVAILABLE=true would have faked a green #12

Merged
jakman85 merged 1 commits from fix/cluster-available-would-fake-a-green into main 2026-09-29 18:14:56 +00:00
Owner

Item 2 was "wire vars.CLUSTER_AVAILABLE so L2/L3 runs in CI." Attempting it found that the
variable does not enable L2/L3 at all — it only changes which way the stage fails, and one
of those ways was a false pass.

Measured, before this guard

CLUSTER_AVAILABLE=true ./ci/conformance.sh   ->  rc=0
  [junit] wrote skipped JUnit for 54 scaffolded test(s)
  [ ok ] L2/L3 conformance passed for touched subset

run.sh exits 0 in scaffold mode when it can't reach a cluster, marking every test <skipped>.
That's correct and honest for run.sh. It's catastrophic in ci/conformance.sh, which only
reaches that code because the environment declared a cluster available — so it reported a pass
over 54 tests that never ran.

One Gitea variable away from a fake green on a gate that is now an enforcing required status
check.
Nothing caught it because nobody had ever set the variable.

Why the local run can never work

The CI runner holds no cluster credential by design (automountServiceAccountToken: false).
The in-cluster trigger path that would make L2/L3 genuinely run is still unimplemented, and is
blocked on a real security decision that is yours — plan §4: any principal that can create jobs
in ns conformance can run arbitrary code as the privileged conformance-runner SA
, because
Kubernetes places no use check on serviceAccountName.

The fix

The stage now blocks (exit 1) when a cluster is declared but every test skipped — which is what
the merge-gate decision matrix already said should happen ("conformance skipped while
CLUSTER_AVAILABLE=true → BLOCK"
). Verified: rc=1, naming the count and the reason.

docs/plans/l2-l3-conformance-ci-wiring.md claimed CLUSTER_AVAILABLE was "the only remaining
gate"
. Corrected in place with the measurement and an explicit do not set this yet.

Gate: PASS, rc 0, 76s.

Co-Authored-By: Claude Opus 5 (1M context) [email protected]

🤖 Generated with Claude Code

Item 2 was "wire `vars.CLUSTER_AVAILABLE` so L2/L3 runs in CI." Attempting it found that the variable **does not enable L2/L3 at all** — it only changes which way the stage fails, and one of those ways was a **false pass**. ### Measured, before this guard ``` CLUSTER_AVAILABLE=true ./ci/conformance.sh -> rc=0 [junit] wrote skipped JUnit for 54 scaffolded test(s) [ ok ] L2/L3 conformance passed for touched subset ``` `run.sh` exits 0 in scaffold mode when it can't reach a cluster, marking every test `<skipped>`. That's correct and honest *for run.sh*. It's catastrophic in `ci/conformance.sh`, which only reaches that code because the environment **declared** a cluster available — so it reported a pass over 54 tests that never ran. **One Gitea variable away from a fake green on a gate that is now an enforcing required status check.** Nothing caught it because nobody had ever set the variable. ### Why the local run can never work The CI runner holds no cluster credential **by design** (`automountServiceAccountToken: false`). The in-cluster trigger path that would make L2/L3 genuinely run is still unimplemented, and is blocked on a real security decision that is yours — plan §4: *any principal that can `create jobs` in ns `conformance` can run arbitrary code as the privileged `conformance-runner` SA*, because Kubernetes places no `use` check on `serviceAccountName`. ### The fix The stage now **blocks (exit 1)** when a cluster is declared but every test skipped — which is what the merge-gate decision matrix already said should happen (*"conformance skipped while CLUSTER_AVAILABLE=true → BLOCK"*). Verified: `rc=1`, naming the count and the reason. `docs/plans/l2-l3-conformance-ci-wiring.md` claimed `CLUSTER_AVAILABLE` was *"the only remaining gate"*. Corrected in place with the measurement and an explicit **do not set this yet**. Gate: **PASS, rc 0, 76s**. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> 🤖 Generated with [Claude Code](https://claude.com/claude-code)
gitea-admin added 1 commit 2026-09-29 07:59:57 +00:00
fix(ci): CLUSTER_AVAILABLE=true would have FAKED A GREEN over 54 skipped tests
gate / gate (pull_request) Successful in 1m36s
gate / conformance (pull_request) Has been skipped
6586919132
Wiring L2/L3 into CI was a tracked next step, described in the plan as needing only
vars.CLUSTER_AVAILABLE. Attempting it surfaced that the variable does not enable L2/L3 at
all -- it only changes which way the stage fails, and before this commit one of those ways
was a false pass.

MEASURED, before the guard:
  CLUSTER_AVAILABLE=true ./ci/conformance.sh   ->  rc=0
  "[junit] wrote skipped JUnit for 54 scaffolded test(s)"
  "[ ok ] L2/L3 conformance passed for touched subset"

run.sh exits 0 in scaffold mode when it cannot reach a cluster, marking every test
<skipped>. That is correct and honest FOR run.sh. It is catastrophic in ci/conformance.sh,
which reaches that code only because the environment DECLARED a cluster available -- so it
reported a pass over 54 tests that never executed. One Gitea variable away from a fake green
on a gate that is now an ENFORCING required status check, and nothing caught it because
nobody had ever set the variable.

The CI runner holds no cluster credential by design (automountServiceAccountToken: false),
so the local run this stage attempts can never succeed. The in-cluster trigger path that
would make L2/L3 actually run is still unimplemented, and is blocked on a privilege-
escalation design decision that is the operator's to make (plan section 4: any principal
that can `create jobs` in ns conformance can run arbitrary code as the privileged
conformance-runner SA, because Kubernetes places no `use` check on serviceAccountName).

The stage now BLOCKS (exit 1) when a cluster is declared but every test skipped, which is
what the merge-gate decision matrix already said should happen ("conformance skipped while
CLUSTER_AVAILABLE=true -> BLOCK"). Verified: rc=1, naming the count and the reason.

The plan document claimed CLUSTER_AVAILABLE was "the only remaining gate". Corrected in
place with the measurement and an explicit do-not-set-this-yet.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
jakman85 approved these changes 2026-09-29 18:14:51 +00:00
jakman85 merged commit dfd608ac32 into main 2026-09-29 18:14:56 +00:00
Sign in to join this conversation.
No Reviewers
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Home-Network/home-network-k3s-impl#12