--- # The runbook static server. Deliberately the dumbest thing that can work. # # WHY A POD AT ALL, AND NOT A REDIRECT TO GITEA. The obvious cheaper option is an HTTPRoute whose # only job is a RequestRedirect into the in-cluster Gitea web UI, which already renders these files. # It was rejected for three reasons, in order of weight: # 1. DEPENDENCY DIRECTION. A runbook is read when the platform is broken. Gitea is an app-tier # workload on Longhorn; its own DR path and several of these runbooks' subjects sit UNDER it. # `runbooks/longhorn-faulted` must be readable while Longhorn is faulted, and # `runbooks/keycloak-db` while a CNPG database is down. Pointing the operator at Gitea makes # the documentation fail in exactly the conditions that produce the alert. # 2. NO SUFFIX REWRITE EXISTS. Gateway API's RequestRedirect path modifier offers ReplaceFullPath # and ReplacePrefixMatch only. There is no "append .md", so mapping 34 extensionless names onto # 34 `.md` blobs needs 34 hand-maintained ReplaceFullPath rules — MORE surface than this pod, # and a silent 404 for every runbook someone forgets to add a rule for. # 3. A 302 is not a document. The annotation's contract is that the URL returns the runbook; a # redirect chain through a second app's auth and rendering is a worse contract. # This pod reads only from a ConfigMap projected out of git. It depends on the CNI, the kubelet and # the Gateway, and on nothing else in the platform. # # WHY NOT THE SERVICE CATALOG (catalog/*.yaml + render.py). The catalog is the idiomatic onboarding # surface for a SERVICE, and it was the first thing checked — but its schema cannot express this # route. `ingressRoutes[].host` is pattern-bound to `^[a-z0-9-]+\.(\$\{BASE_DOMAIN\}|home\.keithjackson\.dev)$`: # a single DNS label plus the apex, so the BARE apex is unrepresentable, and there is no `path` field # at all, so a path-prefix route cannot be authored. Bending the schema for one docs server would be # the tail wagging the dog. The repo's OTHER established pattern is exactly right instead: a small # hand-authored platform component reconciled by its own non-gating Flux Kustomization — what # platform/homepage does for the apex `/` prefix, which this mirrors file for file. apiVersion: apps/v1 kind: Deployment metadata: name: runbooks namespace: runbooks labels: app.kubernetes.io/name: runbooks spec: # TWO replicas with anti-affinity, unlike homepage's one. This is the only workload in the # cluster whose whole purpose is to be available during an incident, and a single replica is # unavailable for the minutes a node reboot or a drain takes — which is when it is wanted. It # costs 2 x 16Mi. replicas: 2 revisionHistoryLimit: 3 strategy: type: RollingUpdate rollingUpdate: maxUnavailable: 0 maxSurge: 1 selector: matchLabels: app.kubernetes.io/name: runbooks template: metadata: labels: app.kubernetes.io/name: runbooks spec: automountServiceAccountToken: false # Spread across nodes so losing one node never takes the runbooks with it. `preferred`, not # `required`: on a 3-node cluster with one node cordoned for the very incident being debugged, # a hard rule would leave a replica Pending instead of running somewhere. affinity: podAntiAffinity: preferredDuringSchedulingIgnoredDuringExecution: - weight: 100 podAffinityTerm: topologyKey: kubernetes.io/hostname labelSelector: matchLabels: app.kubernetes.io/name: runbooks # Tolerate a node that is under memory/disk pressure or unreachable for a while. An operator # reading `runbooks/node-disk` is by definition dealing with a node under pressure, and the # default 300s unreachable toleration is long enough; these exist so eviction pressure does # not silently remove the last replica while the cluster is sick. tolerations: - key: node.kubernetes.io/unreachable operator: Exists effect: NoExecute tolerationSeconds: 900 securityContext: runAsNonRoot: true runAsUser: 101 # `nginx` in nginxinc/nginx-unprivileged runAsGroup: 101 fsGroup: 101 seccompProfile: type: RuntimeDefault containers: - name: nginx # Renovate maintains this pin. nginx-unprivileged (not nginx) so the container can honour # runAsNonRoot: the stock nginx image's entrypoint needs root to drop privileges and to # bind :80. `stable-alpine` branch, pinned exactly. image: nginxinc/nginx-unprivileged:1.30.4-alpine imagePullPolicy: IfNotPresent ports: - name: http containerPort: 8080 protocol: TCP resources: requests: cpu: 10m memory: 16Mi limits: memory: 64Mi securityContext: allowPrivilegeEscalation: false capabilities: drop: - ALL # Achievable here, unlike homepage/headlamp: nginx only needs /tmp (its pid file, on the # unprivileged image) and /var/cache/nginx writable, and both are emptyDirs below. readOnlyRootFilesystem: true livenessProbe: httpGet: { path: /healthz, port: http } initialDelaySeconds: 3 periodSeconds: 15 readinessProbe: # Probe a REAL runbook, not just /healthz. nginx answers /healthz from its own config # even if the ConfigMap volume failed to project, so /healthz alone would report a pod # that serves 404 for every runbook as Ready. `node-down` is the canonical alert link # (NodeExporterDown, platform/monitoring/alerts.yaml), so if this path 404s the pod is # useless and must not take traffic. httpGet: { path: /runbooks/node-down, port: http } initialDelaySeconds: 3 periodSeconds: 15 volumeMounts: - name: nginx-config mountPath: /etc/nginx/conf.d/default.conf subPath: default.conf readOnly: true # The runbooks themselves, one extensionless file per key. Whole-volume mount (NOT # subPath): subPath mounts do not update when the ConfigMap changes, and here the # ConfigMap name carries kustomize's content hash so a content change rolls the pod # anyway — the whole-volume mount just means a stale projection is impossible. - name: runbooks mountPath: /usr/share/nginx/html/runbooks readOnly: true - name: tmp mountPath: /tmp - name: cache mountPath: /var/cache/nginx volumes: - name: nginx-config configMap: name: runbooks-nginx - name: runbooks configMap: # Generated by ../kustomization.yaml's configMapGenerator from runbooks/*.md. kustomize # rewrites this reference to the hashed name, so the pods roll whenever a runbook changes. name: runbooks-content - name: tmp emptyDir: {} - name: cache emptyDir: {}