jakman85andClaude Opus 5
cd64b4b672
Completes the WIP preserved in the previous commit. `MERGE GATE: PASS` + exit 0 now means
what it says: every stage that was SUPPOSED to run ran, and was green.
THE DEFECT. In gate.sh's decision section a FAILED stage set BLOCK=1 but a SKIPPED stage only
warned, so the gate printed `l0: skipped (unwired seam / missing tool) — not green` and then
`MERGE GATE: PASS`, exit 0. Seen twice for real: `prom-rules` skipped because promtool was
absent, and `l0` skipped because the pinned kyverno CLI was. Exit 0 is what CI and every
`||`-chained script read as merge-approved, so the one component whose entire job is to fail
closed was failing OPEN — in a repo whose stated principle is that SKIPPED IS NOT GREEN, and
whose history already contains the 2026-08-31 "local false green" that UNDECIDED was added to
prevent. The offline stages simply never used that vocabulary.
A SKIP IS NOT ONE THING — the per-stage rule, with the justification in the file header:
* lint / l0 / render-golden / l1 — COULD NOT RUN (a pinned tool absent, a seam unwired).
BLOCKED under strict skips (a CI runner has the toolchain installed for it by
install-tools.sh, so a skip there is a defect); UNDETERMINED (exit 3) otherwise — a
workstation missing a tool is a real local condition, still non-zero, still never a pass.
* spec-immutability — the spec repo is not checked out, so R-TEST-06 COULD NOT BE VERIFIED:
UNDETERMINED. Deliberately NOT upgraded to a block by strictness, because the live runner
does not check the spec repo out at all and blocking would leave the only enforced status
check permanently red for a reason no PR can clear. It is the ONE skip a caller may
knowingly tolerate; REQUIRE_SPEC_VERIFICATION=true makes it hard once wired.
* conformance L2/L3 — CLUSTER_AVAILABLE!=true is the environment DECLARING the capability
absent, not a stage failing to run where it could have. A declared-inapplicable stage was
never "supposed to run", so it does not deny a PASS — but it is now NAMED in the PASS
headline and the step summary, so PASS never implies more than was verified. A skip
DESPITE CLUSTER_AVAILABLE=true still blocks.
declared-not-applicable may pass; could-not-run may not.
STRICTNESS IS DECLARED, NOT SNIFFED. .gitea/workflows/gate.yml sets GATE_STRICT_SKIPS=true on
the gate step — the workflow is the thing that knows it is CI — with GITHUB_ACTIONS/CI (set by
the runner itself) only as the `auto` fallback. No hostname/path/tool-availability heuristic.
A wrong answer in either direction cannot fail open: both branches are non-zero and neither is
a PASS. That step now calls ./ci/gate.sh directly and tolerates exit 3 in one visible line,
because under strict skips exit 3 can ONLY mean the unverifiable spec repo — a missing tool is
exit 1 and untolerated. Tolerating a non-answer is a reviewed choice in the CALLER; there is
deliberately no knob inside gate.sh that turns UNDETERMINED into exit 0.
ALSO, from the unlanded fix/touched-requirements-fail-closed (47068ee), whose reasoning is
carried over rather than its tree: gate.sh special-cased only exit 3 from
touched-requirements.sh, so ANY other non-zero exit left the touched set empty and printed PASS
having applied no coverage gate at all. An unusable subset is not an empty subset: it BLOCKS.
Latent today (the script exits only 0 or 3) and wired now so the hole is already closed when a
fail-closed path map makes exit 1 reachable — the measured state of that map is in the PM report.
TESTED — ci/tests/gate-decision.test.sh, 47 assertions over the whole decision table: the real
gate.sh and lib.sh run in a throwaway CI_DIR against STUB stage scripts whose exit codes the
suite dictates, so it asserts how the gate REASONS, with no tools, no cluster, no network. All
stages green -> PASS/0; each stage FAILED -> BLOCKED/1; each stage SKIPPED -> never PASS and
non-zero; strict skips -> BLOCKED/1; the spec exception; the declared-gap PASS; the
undetermined-touched-subset paths unchanged (on-base -> NOT APPLICABLE/0, no-base and
diff-failed -> BLOCKED/1); UNDETERMINED outranking NOT APPLICABLE; an unusable subset ->
BLOCKED/1; truthy()'s misspelling direction. 32 of the 47 FAIL against the pre-change gate,
including "l0 SKIPPED: printed 'MERGE GATE: PASS'" — the suite is falsifiable, not decorative.
lib.sh: truthy() is now the one spelling of "is this switch on" and cluster_available() uses it
(the comment claimed that; now it is true), and load_touched_requirements' rc!=0 branch says
what actually happened instead of "proceeding with the full layer".
Refs R-TEST-02, R-TEST-03, R-TEST-06.
Co-Authored-By: Claude Opus 5 <[email protected]>
CI merge gate — .gitea/workflows/
Task T-0.2 · Requirements R-GITOPS-01, R-TEST-02, R-TEST-03 · Owner WS-GITOPS (CI).
This is the merge gate for the home k3s + Longhorn platform. It enforces the gating rules in the
spec (docs/14-testing-and-conformance.md,
ADR-0010,
ADR-0011) as a
Gitea Actions pipeline: no change lands on main unless the conformance subset for the requirements
it touches is green (R-TEST-03).
Done is a green, reproducible, requirement-tagged test — never "merged," never a demo. This gate is the machine that enforces it.
The pipeline
lint ─▶ l0 (schema + Kyverno) ─▶ render-golden ─▶ l1 (rendering) ─▶ conformance (L2/L3) ─▶ merge-gate decision
└──────────────── run in CI, NO cluster ────────────────┘ └── in-cluster Chainsaw Job ──┘
live now cluster-gated (SKIPS in Phase 0)
| Stage | Job | Layer | Verifies | Cluster? |
|---|---|---|---|---|
| yaml/lint | lint |
— | YAML well-formed (ci/yamllint.yaml) |
no — live now |
| schema | l0 |
L0 | ServiceDefinitions valid against schemas/ |
no — live now |
| Kyverno L0 | l0 |
L0 | kyverno test platform/kyverno/: default-deny baseline present, no plaintext secrets, required labels/egress-class, schema conformance |
no — live now |
| L1 rendering | l1 |
L1 | ServiceDefinition → rendered resources match golden expectations (Chainsaw L1 via conformance/) |
no — live now |
| conformance | conformance |
L2+L3 | Real runtime behavior + fault-injection, as an in-cluster Chainsaw Job CI triggers and awaits | yes — cluster-gated |
| decision | merge-gate |
— | Applies R-TEST-03 over the above; blocks merge on any failure | no |
Each stage is a thin wrapper over a ci/*.sh script, so CI and a laptop run the identical gate
(R-TEST-02). See ../../ci/README.md.
Why L2/L3 is an in-cluster Job (not run by CI directly)
Per docs/14 §"Where each layer runs"
and ADR-0011: CI holds no broad kubeconfig. L2/L3 run as an in-cluster Chainsaw Job whose own
scoped test ServiceAccount holds cluster access (R-SEC-02). CI only triggers the Job and
collects its artifacts, using a narrow trigger credential referenced by secret name only
(secrets.CONFORMANCE_TRIGGER_KUBECONFIG) — never a plaintext secret, never a cluster-admin kubeconfig.
Honest SKIP — never fake green (Phase 0)
No cluster exists in Phase 0. The gate is deliberately built so unavailable checks SKIP loudly, they do not pass:
- The
conformancejob isif: ${{ vars.CLUSTER_AVAILABLE == 'true' }}— with no cluster it is skipped by Gitea (shown as skipped, not green). Themerge-gatedecision then reports any touched requirement that needs L2/L3 as UNVERIFIED, not satisfied. - Stages whose owning subtree is still a placeholder (
schemas/,platform/kyverno/,conformance/) report SKIPPED — unwired, distinct from a pass, until those seams are delivered.
The decision matrix (scripts/decide.sh):
| Situation | Outcome |
|---|---|
| Any of lint/l0/l1 fails | BLOCK |
| conformance fails/cancelled | BLOCK |
conformance skipped while CLUSTER_AVAILABLE=true |
BLOCK (it should have run) |
| conformance skipped, no cluster, touched reqs need L2/L3 | UNVERIFIED — reported, not green; blocks only if REQUIRE_CLUSTER_TESTS=true |
| lint/l0/l1 green, no cluster-needing reqs touched | PASS |
REQUIRE_CLUSTER_TESTS is false in Phase 0 (bootstrap: seams and cluster don't exist yet) and flips
to true from Phase 1 once a dev/test cluster is declared — from then on, any touched requirement that
can't be verified is a hard block.
Artifacts (R-TEST-04 traceability)
Every layer emits machine-readable output collected by the merge-gate:
- JUnit XML →
results/junit/*.xml(uploaded asl1-junit-coverage/conformance-junit-coverage). - Requirement-coverage report →
reports/coverage/**— maps everyR-*to its verifying test(s); uncovered requirements are visible gaps. Produced by the conformance runner (conformance/, WS-TEST).
Both dirs are git-ignored (see .gitignore).
Triggers & branch model (CONTRIBUTING.md)
pull_request→main: PRs are the enforced merge gate intomain.push→feature/**,fix/**,docs/**,chore/**: fast feedback on the branch. No work onmain.
Transition: generic-CI-now → Gitea-Actions-after-Gitea-is-up
Per CONTRIBUTING.md and
docs/05 §"local bootstrap → self-hosted Gitea":
- Now (interim): there is no remote git host on this network. This workflow file is dormant —
but the identical gate runs locally today via
make -C ci gate(R-TEST-02). Development is local feature-branch →main, the gate enforced by convention + the local runner. - Gitea comes online early (one of the first services onboarded) at
gitea.home.keithjackson.dev. - CI moves to Gitea Actions: this workflow activates unchanged. The only operational steps are
enabling a Gitea Actions runner and setting
vars.CLUSTER_AVAILABLE=true+secrets.CONFORMANCE_TRIGGER_KUBECONFIGonce a dev/test cluster exists. PRs then become the platform-enforced merge gate — the same no-work-in-main rule, now enforced by machine.
Configuration (set in Gitea; referenced by name only)
| Kind | Name | Purpose |
|---|---|---|
| variable | CLUSTER_AVAILABLE |
'true' once a dev/test cluster exists → L2/L3 run and are required; else they SKIP |
| variable | REQUIRE_CLUSTER_TESTS |
'true' (Phase 1+) makes unverified cluster-needing touched reqs a hard block |
| secret | CONFORMANCE_TRIGGER_KUBECONFIG |
narrow creds to create/await the in-cluster Chainsaw Job (not cluster-admin) |
Files
.gitea/workflows/
merge-gate.yaml the pipeline (lint → l0 → l1 → conformance → decision)
scripts/
install-tools.sh installs the pinned tools per stage (from ci/versions.env)
decide.sh the R-TEST-03 merge decision over job results
README.md this file
The stage logic itself lives in ../../ci/ so it is reusable locally and in CI.