--- # Runbooks tier — reconciled by its OWN non-gating post-platform Flux Kustomization # (clusters/base/runbooks.yaml, dependsOn platform-config, wait:false). Same placement and rationale as # platform/homepage: a leaf web server must never sit in the gating `platform` tier, where a UI's # unhealth once stalled the whole chain (DECISIONS-LOG: "Headlamp gating stall"). # # WHY THE MANIFESTS LIVE HERE, ALONGSIDE THE DOCS, RATHER THAN IN platform/runbooks/. # The markdown is the source of truth and the task is to serve it verbatim, which means the ConfigMap must # be GENERATED from these files — there must not be a second hand-maintained copy of 34 documents that can # drift from the originals. `configMapGenerator` does that, and its `files:` paths are resolved relative to # THIS directory. A `platform/runbooks/kustomization.yaml` would therefore need `../../runbooks/.md`, # which depends on kustomize load restrictions being relaxed — true for Flux's kustomize-controller, # refused by a plain `kustomize build`, and so a divergence between what the gate can verify and what the # cluster does. Making this directory the Flux path removes the question entirely: no `../`, identical # behaviour locally and in-cluster, one copy of every document. The serving manifests are tucked under # `serving/` so `runbooks/*.md` stays exactly the flat set of pages the URLs name. apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization namespace: runbooks resources: - serving/namespace.yaml - serving/nginx-config.yaml - serving/deployment.yaml - serving/service.yaml # default-deny baseline + Cilium gateway-ingress allow (DNS-only egress from the baseline). - serving/networkpolicy.yaml - serving/httproute.yaml # apex `/runbooks` prefix (binds the Gateway `https-apex` listener) configMapGenerator: # ONE ConfigMap holding every runbook, keyed by the EXACT name the alert annotations use. # # THE KEYS ARE EXTENSIONLESS ON PURPOSE. The annotations are `https://${BASE_DOMAIN}/runbooks/` # with NO `.md`, and this repo must not change them (they are spread across four alerts.yaml files that # other work is editing). `=` maps `node-down.md` on disk to the key `node-down`, the volume # mount turns each key into a file under the nginx root, and `/runbooks/node-down` becomes a plain static # lookup — no rewrite rule, nothing to get subtly wrong at 3am. Drop the `=` and every key would # carry `.md`, so every alert link would 404 while this file still looked correct. # # The generated name carries kustomize's content hash, so editing any runbook rolls the server # automatically and a stale projection is impossible. # # ADDING A RUNBOOK: add the file AND its line here. The L0 check # conformance/tests/l0-static/alert-runbooks-resolvable asserts both halves, because a file in git with # no generator entry is reviewed, merged, apparently done — and still a 404 in the cluster. - name: runbooks-content options: labels: app.kubernetes.io/name: runbooks files: # The index, served at /runbooks/ (nginx `try_files /runbooks/README`). - README=README.md - backup-failure=backup-failure.md - backup-missed=backup-missed.md - backup-partial=backup-partial.md - cdi-import-stuck=cdi-import-stuck.md - cert-expiry=cert-expiry.md - cert-issuance=cert-issuance.md - cnpg-archiving=cnpg-archiving.md - cnpg-backup-missed=cnpg-backup-missed.md - cnpg-exporter=cnpg-exporter.md - cnpg-primary-spread=cnpg-primary-spread.md - cnpg-wal-growth=cnpg-wal-growth.md - denied-flows=denied-flows.md - flux-reconcile=flux-reconcile.md - flux-suspended=flux-suspended.md - image-automation=image-automation.md - image-policy-stalled=image-policy-stalled.md - keycloak-client-secrets=keycloak-client-secrets.md - keycloak-db=keycloak-db.md - keycloak-down=keycloak-down.md - kubevirt-control-plane=kubevirt-control-plane.md - kubevirt-crashloop=kubevirt-crashloop.md - kubevirt-launch-failed=kubevirt-launch-failed.md - longhorn-capacity=longhorn-capacity.md - longhorn-degraded=longhorn-degraded.md - longhorn-faulted=longhorn-faulted.md - monitoring-capacity=monitoring-capacity.md - node-cpu=node-cpu.md - node-disk=node-disk.md - node-down=node-down.md - node-io=node-io.md - node-memory=node-memory.md - node-memory-swap=node-memory-swap.md # Operator procedure, not an alert page (no alert links to it — the # alert-runbooks-resolvable check treats that as advisory). It is served # because the 4-minute plex.tv claim window is not a thing to go looking # for in git: /runbooks/plex-claim has to resolve the moment it is needed. - plex-claim=plex-claim.md - pvc-capacity=pvc-capacity.md - targets-down=targets-down.md - update-rollback=update-rollback.md - vpn-down=vpn-down.md - vpn-egress-stale=vpn-egress-stale.md