fix(runbooks): declare whole-repo scope, drop the bespoke l0 stage, +cnpg-primary-spread
Three corrections after rebasing onto main. 1. THE GUARD CAUGHT A REAL ONE ON ITS FIRST CONTACT WITH main. main had gained `CNPGPrimariesCoLocated` (platform/cnpg/alerts.yaml) pointing at /runbooks/cnpg-primary-spread, with no runbook. The check failed the rebase exactly as designed — 46 annotations, 35 names, 34 present — naming both the missing file AND the missing configMapGenerator entry. runbooks/cnpg-primary-spread.md now exists: what co-located primaries actually cost here (blast radius, not an outage, because instances: 3 + required anti-affinity), why the drift is uncorrectable-by-declaration (CNPG has no primary-placement field), the readiness gate that keeps the rule quiet during a real node outage, and the ~9s-per-database switchover rebalance, one cluster at a time. The 2026-09-07 outage in which all three primaries sat on one node is the reason the alert exists at all. 2. WHOLE-REPO SCOPE, NOT A HAND-WRITTEN STAGE. main's refactor replaced ci/l0.sh's per-check "UNCONDITIONAL" stages with a declaration: `conformance.home.k3s/scope: "whole-repo"`, unioned into every gate run by Stage 3b's `--include-scope`. My +38-line Stage 3f is removed (ci/l0.sh is now byte-identical to main) and the annotation is declared instead. Assertion (7) of the check was rewritten to match: it no longer asserts its own path appears in ci/l0.sh — it asserts the test DECLARES the scope and that ci/l0.sh still passes --include-scope, since neither half alone makes the check load-bearing. That matters here because ci/touched-requirements.sh maps platform/monitoring/* to "R-OBS-01 R-OBS-02", not R-OBS-04, and maps platform/kubevirt/* and runbooks/* to nothing at all. 3. A FACTUAL CORRECTION IN ALL 35 RUNBOOKS. Every page claimed `dev.sh` defaults its current-context to k3s-dev. main's `9b4744d fix(devcontainer): dev.sh defaults to prod` made that false, and a runbook that misstates which cluster a command talks to is worse than one that says nothing. The pages now say what is durably true: the default has already changed once, so naming `--context k3s-prod` is what makes a command mean the same thing next month, and the per-invocation banner is there to be read. The check's own failure message was corrected the same way. Re-verified after all three: check green at 374 assertions (46 annotations, 35/35 runbooks present, 35/35 projected), re-applied to prod, and re-probed in-cluster through the Gateway — node-down, keycloak-down, keycloak-client-secrets, cnpg-archiving, longhorn-faulted, vpn-down, node-io and cnpg-primary-spread all HTTP 200 with text/plain; charset=utf-8 and 8.9-12.1 KB of real body; /runbooks/ index 200; an unknown name 404s with a pointer rather than a blank page; and homepage still answers the apex `/`, confirming Gateway API longest-path precedence. Full merge gate: PASS (spec-immutability, lint, l0, render-golden, l1 all green; prom-rules SKIPPED for want of promtool in the image; L2/L3 SKIPPED, no cluster). Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
1 parent
44983095e1
commit
b7341b329b
40 files changed
+514
-191
No files matched your search
@@ -31,10 +31,14 @@ that hang is not the cluster being down. Every command below runs as:
|
||||
|
||||
infra/devcontainer/dev.sh run bash -c 'kubectl --context k3s-prod get nodes'
|
||||
|
||||
Always pass `--context k3s-prod` explicitly. `dev.sh` merges both clusters into one kubeconfig and
|
||||
defaults its current-context to k3s-**dev**, so a command without `--context` silently answers
|
||||
about the wrong cluster. (`CLUSTER=prod infra/devcontainer/dev.sh run kubectl ...` is equivalent.)
|
||||
Do not run two `dev.sh` invocations at once — they share one container and interfere.
|
||||
Always pass `--context k3s-prod` explicitly. `dev.sh` merges BOTH clusters into one kubeconfig and
|
||||
picks a current-context per invocation, and **that default has already changed once** — it was k3s-dev
|
||||
until dev was decommissioned on 2026-09-13, and is k3s-prod now. A command that names its context means
|
||||
the same thing next month as it does today; one that relies on the default does not, and the failure is
|
||||
silent (it answers confidently about the other cluster). `CLUSTER=prod infra/devcontainer/dev.sh run
|
||||
kubectl ...` selects it explicitly, and every invocation prints a banner naming the cluster it is about
|
||||
to talk to — read it. Do not run two `dev.sh` invocations at once; they share one container and
|
||||
interfere.
|
||||
|
||||
USE FULLY-QUALIFIED RESOURCE NAMES, ALWAYS. Three CRDs on this cluster register the kind `Backup`:
|
||||
`longhorn.io/v1beta2`, `postgresql.cnpg.io/v1` and `velero.io/v1`. The bare short name resolves to
|
||||
|
||||
Reference in new issue
Block a user