fix(runbooks): declare whole-repo scope, drop the bespoke l0 stage, +cnpg-primary-spread

Three corrections after rebasing onto main.

1. THE GUARD CAUGHT A REAL ONE ON ITS FIRST CONTACT WITH main. main had gained
   `CNPGPrimariesCoLocated` (platform/cnpg/alerts.yaml) pointing at
   /runbooks/cnpg-primary-spread, with no runbook. The check failed the rebase exactly as
   designed — 46 annotations, 35 names, 34 present — naming both the missing file AND the
   missing configMapGenerator entry. runbooks/cnpg-primary-spread.md now exists: what
   co-located primaries actually cost here (blast radius, not an outage, because
   instances: 3 + required anti-affinity), why the drift is uncorrectable-by-declaration
   (CNPG has no primary-placement field), the readiness gate that keeps the rule quiet
   during a real node outage, and the ~9s-per-database switchover rebalance, one cluster
   at a time. The 2026-09-07 outage in which all three primaries sat on one node is the
   reason the alert exists at all.

2. WHOLE-REPO SCOPE, NOT A HAND-WRITTEN STAGE. main's refactor replaced ci/l0.sh's
   per-check "UNCONDITIONAL" stages with a declaration:
   `conformance.home.k3s/scope: "whole-repo"`, unioned into every gate run by Stage 3b's
   `--include-scope`. My +38-line Stage 3f is removed (ci/l0.sh is now byte-identical to
   main) and the annotation is declared instead. Assertion (7) of the check was rewritten
   to match: it no longer asserts its own path appears in ci/l0.sh — it asserts the test
   DECLARES the scope and that ci/l0.sh still passes --include-scope, since neither half
   alone makes the check load-bearing. That matters here because
   ci/touched-requirements.sh maps platform/monitoring/* to "R-OBS-01 R-OBS-02", not
   R-OBS-04, and maps platform/kubevirt/* and runbooks/* to nothing at all.

3. A FACTUAL CORRECTION IN ALL 35 RUNBOOKS. Every page claimed `dev.sh` defaults its
   current-context to k3s-dev. main's `9b4744d fix(devcontainer): dev.sh defaults to
   prod` made that false, and a runbook that misstates which cluster a command talks to is
   worse than one that says nothing. The pages now say what is durably true: the default
   has already changed once, so naming `--context k3s-prod` is what makes a command mean
   the same thing next month, and the per-invocation banner is there to be read. The
   check's own failure message was corrected the same way.

Re-verified after all three: check green at 374 assertions (46 annotations, 35/35 runbooks
present, 35/35 projected), re-applied to prod, and re-probed in-cluster through the
Gateway — node-down, keycloak-down, keycloak-client-secrets, cnpg-archiving,
longhorn-faulted, vpn-down, node-io and cnpg-primary-spread all HTTP 200 with
text/plain; charset=utf-8 and 8.9-12.1 KB of real body; /runbooks/ index 200; an unknown
name 404s with a pointer rather than a blank page; and homepage still answers the apex `/`,
confirming Gateway API longest-path precedence.

Full merge gate: PASS (spec-immutability, lint, l0, render-golden, l1 all green;
prom-rules SKIPPED for want of promtool in the image; L2/L3 SKIPPED, no cluster).

Co-Authored-By: Claude Opus 5 <[email protected]>

conformance/tests/l0-static/alert-runbooks-resolvable/chainsaw-test.yaml

@@ -35,8 +35,11 @@
 #           shell example in it (it fails open: DECISIONS-LOG 2026-09-01, the emptied Cilium
 #           k8sServiceHost), and the flux-postbuild-substitution check cannot see it because that check
 #           scans files reachable through `resources:`, not configMapGenerator inputs;
-#   AND (7) the check is LOAD-BEARING: ci/l0.sh runs it on every gate run (Stage 3f) and ci/gate.sh sets
-#           the merge block when the l0 stage fails.
+#   AND (7) the check is LOAD-BEARING: this Test DECLARES
+#           `conformance.home.k3s/scope: whole-repo`, so ci/l0.sh Stage 3b's
+#           `--include-scope whole-repo` unions it into EVERY gate invocation rather than only the runs
+#           whose touched requirements happen to include R-OBS-04; and ci/gate.sh sets the merge block
+#           when the l0 stage fails. A SKIP of a scope-declaring check is UNWIRED, never green.
 #
 # STATUS: EXECUTABLE. `check.sh` is a pure offline assertion script (standard library only);
 # conformance/run.sh --layer l0 and ci/l0.sh run it directly, and this Chainsaw Test wraps the SAME
@@ -59,6 +62,16 @@ metadata:
   annotations:
     conformance.home.k3s/requirements: "R-OBS-04"
     conformance.home.k3s/layer: "L0"
+    # WHOLE-REPO SCOPE — this is what makes the check unconditional. Stage 3b of ci/l0.sh runs
+    # `conformance/run.sh --layer l0 --include-scope whole-repo`, which UNIONS every test carrying this
+    # annotation into the selection regardless of which requirements the change touched. That union is
+    # load-bearing here: ci/touched-requirements.sh maps platform/monitoring/* to "R-OBS-01 R-OBS-02" —
+    # NOT R-OBS-04 — and maps platform/kubevirt/* and runbooks/* to nothing at all, so the single most
+    # likely breaking change (adding an alert with a runbook_url, or adding a runbook without its
+    # configMapGenerator entry) attributes a requirement set that would otherwise FILTER THIS CHECK OUT
+    # of exactly the change that introduces the bug. Declaring the scope replaces the hand-written
+    # per-test ci/l0.sh stage that this repo used to need (and that nobody remembered to add).
+    conformance.home.k3s/scope: "whole-repo"
     conformance.home.k3s/safety-critical: "false"
     conformance.home.k3s/executable: "true"
     conformance.home.k3s/intent: >-

conformance/tests/l0-static/alert-runbooks-resolvable/check.sh

@@ -56,7 +56,8 @@
 # maps platform/kubevirt/* and runbooks/* to nothing at all. So the single most likely breaking
 # change (adding an alert with a runbook_url to platform/monitoring/alerts.yaml, or adding a
 # runbook and forgetting the generator entry) attributes a requirement set that would FILTER THIS
-# CHECK OUT. Same reasoning as stages 3c/3d/3e; the invariant is whole-repo, so it runs every time.
+# CHECK OUT. Declaring `conformance.home.k3s/scope: whole-repo` makes ci/l0.sh Stage 3b union it into
+# EVERY gate invocation instead; a SKIP of a scope-declaring check is UNWIRED, never green.
 #
 # WHAT THIS DOES *NOT* PROVE (honest gaps — this is a STATIC check):
 #   - It does not fetch the URL. That it returns HTTP 200 with real content was proven by an
@@ -122,6 +123,7 @@ def read(p: Path) -> str:
 # `kind: PrometheusRule` are scanned, so a runbook_url quoted inside this check or inside prose
 # elsewhere in the repo is not mistaken for an alert annotation. Commented-out lines are skipped:
 # platform/monitoring/alerts.yaml carries a `# runbook_url points at a docs base` header comment.
+SCOPE_DECL_RE = r"^\s*conformance\.home\.k3s/scope:\s*[\"\']?whole-repo"
 RUNBOOK_URL_RE = re.compile(r"""^\s*runbook_url:\s*["']?([^"'\s#]+)["']?\s*$""")
 
 rule_files = sorted(
@@ -217,8 +219,10 @@ for name, sites in sorted(names.items()):
           f"`infra/devcontainer/dev.sh run bash -c '...'` and the runbook must say so")
     check("--context k3s-prod" in body,
           f"{md.relative_to(ROOT)} never passes --context k3s-prod — dev.sh merges both clusters "
-          f"and defaults to k3s-DEV, so a context-less command silently answers about the wrong "
-          f"cluster in the middle of an incident")
+          f"and picks a current-context PER INVOCATION — a default that has already changed once "
+          f"(k3s-dev until dev was decommissioned on 2026-09-13, k3s-prod now). A context-less command "
+          f"silently answers about whichever cluster the default happens to be, in the middle of an "
+          f"incident")
 
 # ADVISORY, not a failure: a runbook nothing links to yet. Writing the runbook BEFORE the alert is
 # the right order, and the repo's precedent for "authored but unreferenced" is advisory (see the
@@ -353,13 +357,29 @@ check(re.search(rf"^\s*-\s*{re.escape(PREFIX)}\.yaml\b", read(BASE_KUST), re.M)
 # ============================================ (7) this check is load-bearing in the merge gate
 L0 = read(ROOT / "ci" / "l0.sh")
 GATE = read(ROOT / "ci" / "gate.sh")
-check("alert-runbooks-resolvable/check.sh" in L0,
-      "ci/l0.sh does not run alert-runbooks-resolvable/check.sh. The requirement-scoped conformance "
-      "selection in Stage 3b would filter this check out of exactly the change that breaks it: "
-      "ci/touched-requirements.sh maps platform/monitoring/* to 'R-OBS-01 R-OBS-02' (not R-OBS-04) "
-      "and maps platform/kubevirt/* and runbooks/* to nothing, so adding an alert with a new "
-      "runbook_url attributes no requirement this test carries. It must be invoked directly and "
-      "unconditionally, like stages 3c/3d/3e")
+# THIS CHECK IS LOAD-BEARING — asserted against the DECLARATIVE mechanism, in two halves.
+# It is not enough that check.sh exists: the merge gate must SELECT it on every run. The repo used to
+# get that with a hand-written "UNCONDITIONAL" stage per whole-repo check in ci/l0.sh, hardcoding the
+# path; those stages are gone, because the next whole-repo invariant only got one if somebody
+# REMEMBERED to write it. The property is now DECLARED by the test and HONOURED by the gate, so both
+# halves are asserted — neither alone is sufficient. The mechanism itself (that the union really
+# selects, and that a scope-declared SKIP exits 3 rather than passing green) is proven end-to-end and
+# falsifiably by conformance/tests/l0-static/whole-repo-invariants-wired.
+SELF_YAML = (ROOT / "conformance" / "tests" / "l0-static" / "alert-runbooks-resolvable"
+             / "chainsaw-test.yaml")
+SELF = read(SELF_YAML)
+check(re.search(SCOPE_DECL_RE, SELF, re.M) is not None,
+      "alert-runbooks-resolvable/chainsaw-test.yaml does not declare "
+      "conformance.home.k3s/scope: whole-repo, so the merge gate selects this check only when a "
+      "change happens to touch a requirement it is labelled with. ci/touched-requirements.sh maps "
+      "platform/monitoring/* to 'R-OBS-01 R-OBS-02' (NOT R-OBS-04) and maps platform/kubevirt/* and "
+      "runbooks/* to nothing at all — so adding an alert with a new runbook_url, or adding a runbook "
+      "without its configMapGenerator entry, would be filtered out of exactly the change that "
+      "introduces the dead link")
+check("--include-scope" in L0,
+      "ci/l0.sh no longer passes --include-scope to the conformance runner, so the L0 stage is a "
+      "purely requirement-scoped run again and the scope annotation above is inert decoration — this "
+      "check would then run only by coincidence")
 check(re.search(r"^for s in .*\bl0\b.*; do", GATE, re.M) and re.search(r"fail\).*BLOCK=1", GATE),
       "ci/gate.sh does not set the merge block when the l0 stage fails")