diff --git a/ci/l1.sh b/ci/l1.sh index 0aa85b0..d19e9f7 100755 --- a/ci/l1.sh +++ b/ci/l1.sh @@ -30,11 +30,13 @@ req_args=(${REQ_ARGS[@]+"${REQ_ARGS[@]}"}) # (conformance/_artifacts/) and does NOT accept --junit/--coverage. Invoke by layer + # touched subset, then COLLECT its artifacts into the CI artifact dirs. # --include-scope: union the touched subset with every L1 test DECLARING -# `conformance.home.k3s/scope: whole-repo`. No L1 test declares it today, so this is a no-op -# now — it is wired HERE, once, so the NEXT whole-repo invariant (at any layer) is picked up by -# declaring the annotation, instead of needing someone to remember to hand-write a stage. That -# "someone must remember" is the failure mode this whole change removes; see -# conformance/lib/common.sh. rc 3 = a scope-declared check SKIPPED = UNWIRED, never a pass. +# `conformance.home.k3s/scope: whole-repo`. As of 2026-09-18 ALL THREE L1 tests declare it, so this +# is load-bearing rather than a no-op: every L1 golden is checked on every gate run, whatever the +# touched paths attribute. It was wired HERE first, ahead of any declaration, so picking up a new +# whole-repo invariant needs only the annotation instead of someone remembering to hand-write a +# stage — and that "someone must remember" is exactly how the 2026-09-04 external-dns merge gated +# GREEN while leaving all three goldens stale (see the rationale in the Tests themselves). +# rc 3 = a scope-declared check SKIPPED = UNWIRED, never a pass. : "${WHOLE_REPO_SCOPE:=whole-repo}" l1_rc=0 "$CONFORMANCE_CMD" --layer l1 --include-scope "$WHOLE_REPO_SCOPE" \ diff --git a/conformance/tests/l1-rendering/container-workload-render/chainsaw-test.yaml b/conformance/tests/l1-rendering/container-workload-render/chainsaw-test.yaml index 05418e5..ffd2276 100644 --- a/conformance/tests/l1-rendering/container-workload-render/chainsaw-test.yaml +++ b/conformance/tests/l1-rendering/container-workload-render/chainsaw-test.yaml @@ -23,6 +23,17 @@ metadata: conformance.home.k3s/requirements: "R-PLAT-02" conformance.home.k3s/layer: "L1" conformance.home.k3s/safety-critical: "false" + # WHY SCOPE, NOT REQUIREMENTS ALONE: every L1 test exercises ONE shared code path — + # catalog/render.py against schemas/servicedefinition.schema.json — so anything that changes + # rendered output breaks these goldens regardless of which requirement the touched path maps to. + # Requirement-scoped selection cannot see that coupling. Concretely (2026-09-04 external-dns + # merge): `platform/externaldns/helmrelease.yaml` attributes R-ENV-02, R-OBS-01/02/04, + # R-STATE-03 and R-TEST-01/02/03/04 via ci/touched-requirements.sh — NOT R-PLAT-02, R-NET-01 or + # R-NET-05/06/07 — yet it added `external-dns.alpha.kubernetes.io/controller: none` to the + # rendered output of ALL THREE L1 goldens. A requirement-scoped gate selected ZERO L1 tests, went + # GREEN, and left every golden stale; the staleness surfaced only on a later full run. + # ci/l1.sh has passed --include-scope since it was written, waiting for this annotation. + conformance.home.k3s/scope: "whole-repo" conformance.home.k3s/intent: >- A kind: container ServiceDefinition renders a Deployment + Service (the PaaS path). spec: diff --git a/conformance/tests/l1-rendering/default-deny-baseline-render/chainsaw-test.yaml b/conformance/tests/l1-rendering/default-deny-baseline-render/chainsaw-test.yaml index ea46220..7ce99cb 100644 --- a/conformance/tests/l1-rendering/default-deny-baseline-render/chainsaw-test.yaml +++ b/conformance/tests/l1-rendering/default-deny-baseline-render/chainsaw-test.yaml @@ -27,6 +27,17 @@ metadata: conformance.home.k3s/requirements: "R-NET-01" conformance.home.k3s/layer: "L1" conformance.home.k3s/safety-critical: "false" + # WHY SCOPE, NOT REQUIREMENTS ALONE: every L1 test exercises ONE shared code path — + # catalog/render.py against schemas/servicedefinition.schema.json — so anything that changes + # rendered output breaks these goldens regardless of which requirement the touched path maps to. + # Requirement-scoped selection cannot see that coupling. Concretely (2026-09-04 external-dns + # merge): `platform/externaldns/helmrelease.yaml` attributes R-ENV-02, R-OBS-01/02/04, + # R-STATE-03 and R-TEST-01/02/03/04 via ci/touched-requirements.sh — NOT R-PLAT-02, R-NET-01 or + # R-NET-05/06/07 — yet it added `external-dns.alpha.kubernetes.io/controller: none` to the + # rendered output of ALL THREE L1 goldens. A requirement-scoped gate selected ZERO L1 tests, went + # GREEN, and left every golden stale; the staleness surfaced only on a later full run. + # ci/l1.sh has passed --include-scope since it was written, waiting for this annotation. + conformance.home.k3s/scope: "whole-repo" conformance.home.k3s/intent: >- Rendering any service's namespace yields a default-deny baseline policy. spec: diff --git a/conformance/tests/l1-rendering/egress-class-render/chainsaw-test.yaml b/conformance/tests/l1-rendering/egress-class-render/chainsaw-test.yaml index 6ad2275..ab9d0a9 100644 --- a/conformance/tests/l1-rendering/egress-class-render/chainsaw-test.yaml +++ b/conformance/tests/l1-rendering/egress-class-render/chainsaw-test.yaml @@ -32,6 +32,17 @@ metadata: conformance.home.k3s/requirements: "R-NET-05, R-NET-06, R-NET-07" conformance.home.k3s/layer: "L1" conformance.home.k3s/safety-critical: "false" + # WHY SCOPE, NOT REQUIREMENTS ALONE: every L1 test exercises ONE shared code path — + # catalog/render.py against schemas/servicedefinition.schema.json — so anything that changes + # rendered output breaks these goldens regardless of which requirement the touched path maps to. + # Requirement-scoped selection cannot see that coupling. Concretely (2026-09-04 external-dns + # merge): `platform/externaldns/helmrelease.yaml` attributes R-ENV-02, R-OBS-01/02/04, + # R-STATE-03 and R-TEST-01/02/03/04 via ci/touched-requirements.sh — NOT R-PLAT-02, R-NET-01 or + # R-NET-05/06/07 — yet it added `external-dns.alpha.kubernetes.io/controller: none` to the + # rendered output of ALL THREE L1 goldens. A requirement-scoped gate selected ZERO L1 tests, went + # GREEN, and left every golden stale; the staleness surfaced only on a later full run. + # ci/l1.sh has passed --include-scope since it was written, waiting for this annotation. + conformance.home.k3s/scope: "whole-repo" conformance.home.k3s/intent: >- egress: vpn renders the egress-gateway + kill-switch path; internet does not. spec: