jakman85andClaude Opus 5
c5b5b8672a
A service that reads this repository's Gitea issues, decides what to do about each thread as the Program Manager persona, and runs one worker agent per assignment that authors a change, runs the merge gate and opens a pull request. It reacts to issue creation and to comments; the assignment and all correspondence travel through the issue thread and the PR. It authenticates with an Anthropic subscription token. It never pushes to main and never merges. Three choices carry the design, with alternatives recorded in docs/plans/: - it POLLS rather than receiving webhooks (Gitea's webhook SSRF guard, lost deliveries, and no inbound reach needed at all); - it keeps NO state: the trigger is "the newest thing in this thread was not written by me", so the issue thread is the state — no PVC, no backup obligation; - the model holds NO tools: its reply is one JSON object this code validates, so nothing in the path executes text. The boundary is platform/kyverno/admission/pm-agent-job-shape.yaml (Enforce, failurePolicy: Fail), not the RBAC: Kubernetes places no `use` check on serviceAccountName, so "may create Jobs" is "may run arbitrary code as the namespace's most privileged identity". Six rules admit only the committed shape, and the worker identity is bound to nothing with no token mounted, so there is nothing to escalate to. 28/28 offline Kyverno assertions. Nothing has run: the image has never been built or pushed, and both the image digest and the two credentials are deliberate placeholders. See the handoff entry's "What I did NOT verify". Refs CLAUDE.md §1/§3/§6, R-SEC-02, R-GITOPS-02. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
122 lines
7.2 KiB
YAML
122 lines
7.2 KiB
YAML
---
|
|
# Runbooks tier — reconciled by its OWN non-gating post-platform Flux Kustomization
|
|
# (clusters/base/runbooks.yaml, dependsOn platform-config, wait:false). Same placement and rationale as
|
|
# platform/homepage: a leaf web server must never sit in the gating `platform` tier, where a UI's
|
|
# unhealth once stalled the whole chain (DECISIONS-LOG: "Headlamp gating stall").
|
|
#
|
|
# WHY THE MANIFESTS LIVE HERE, ALONGSIDE THE DOCS, RATHER THAN IN platform/runbooks/.
|
|
# The markdown is the source of truth and the task is to serve it verbatim, which means the ConfigMap must
|
|
# be GENERATED from these files — there must not be a second hand-maintained copy of 34 documents that can
|
|
# drift from the originals. `configMapGenerator` does that, and its `files:` paths are resolved relative to
|
|
# THIS directory. A `platform/runbooks/kustomization.yaml` would therefore need `../../runbooks/<name>.md`,
|
|
# which depends on kustomize load restrictions being relaxed — true for Flux's kustomize-controller,
|
|
# refused by a plain `kustomize build`, and so a divergence between what the gate can verify and what the
|
|
# cluster does. Making this directory the Flux path removes the question entirely: no `../`, identical
|
|
# behaviour locally and in-cluster, one copy of every document. The serving manifests are tucked under
|
|
# `serving/` so `runbooks/*.md` stays exactly the flat set of pages the URLs name.
|
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
kind: Kustomization
|
|
namespace: runbooks
|
|
resources:
|
|
- serving/namespace.yaml
|
|
- serving/nginx-config.yaml
|
|
- serving/deployment.yaml
|
|
- serving/service.yaml
|
|
# default-deny baseline + Cilium gateway-ingress allow (DNS-only egress from the baseline).
|
|
- serving/networkpolicy.yaml
|
|
- serving/httproute.yaml # apex `/runbooks` prefix (binds the Gateway `https-apex` listener)
|
|
|
|
configMapGenerator:
|
|
# ONE ConfigMap holding every runbook, keyed by the EXACT name the alert annotations use.
|
|
#
|
|
# THE KEYS ARE EXTENSIONLESS ON PURPOSE. The annotations are `https://${BASE_DOMAIN}/runbooks/<name>`
|
|
# with NO `.md`, and this repo must not change them (they are spread across four alerts.yaml files that
|
|
# other work is editing). `<key>=<file>` maps `node-down.md` on disk to the key `node-down`, the volume
|
|
# mount turns each key into a file under the nginx root, and `/runbooks/node-down` becomes a plain static
|
|
# lookup — no rewrite rule, nothing to get subtly wrong at 3am. Drop the `<key>=` and every key would
|
|
# carry `.md`, so every alert link would 404 while this file still looked correct.
|
|
#
|
|
# The generated name carries kustomize's content hash, so editing any runbook rolls the server
|
|
# automatically and a stale projection is impossible.
|
|
#
|
|
# ADDING A RUNBOOK: add the file AND its line here. The L0 check
|
|
# conformance/tests/l0-static/alert-runbooks-resolvable asserts both halves, because a file in git with
|
|
# no generator entry is reviewed, merged, apparently done — and still a 404 in the cluster.
|
|
- name: runbooks-content
|
|
options:
|
|
labels:
|
|
app.kubernetes.io/name: runbooks
|
|
files:
|
|
# The index, served at /runbooks/ (nginx `try_files /runbooks/README`).
|
|
- README=README.md
|
|
- backup-failure=backup-failure.md
|
|
- backup-missed=backup-missed.md
|
|
- backup-partial=backup-partial.md
|
|
- cdi-import-stuck=cdi-import-stuck.md
|
|
- cert-expiry=cert-expiry.md
|
|
- cert-issuance=cert-issuance.md
|
|
- cnpg-archiving=cnpg-archiving.md
|
|
- cnpg-backup-missed=cnpg-backup-missed.md
|
|
- cnpg-exporter=cnpg-exporter.md
|
|
- cnpg-primary-spread=cnpg-primary-spread.md
|
|
- cnpg-wal-growth=cnpg-wal-growth.md
|
|
# Operator procedure, not alert-linked (Crafty's local login is a second factor behind
|
|
# the Keycloak gate and its bootstrap password lives only on the PVC).
|
|
- crafty-admin-credential=crafty-admin-credential.md
|
|
- denied-flows=denied-flows.md
|
|
- flux-reconcile=flux-reconcile.md
|
|
- flux-rollout=flux-rollout.md
|
|
- flux-suspended=flux-suspended.md
|
|
- image-automation=image-automation.md
|
|
- image-policy-stalled=image-policy-stalled.md
|
|
- keycloak-client-secrets=keycloak-client-secrets.md
|
|
- keycloak-db=keycloak-db.md
|
|
- keycloak-down=keycloak-down.md
|
|
- kubevirt-control-plane=kubevirt-control-plane.md
|
|
- kubevirt-crashloop=kubevirt-crashloop.md
|
|
- kubevirt-launch-failed=kubevirt-launch-failed.md
|
|
- longhorn-capacity=longhorn-capacity.md
|
|
- longhorn-degraded=longhorn-degraded.md
|
|
- longhorn-faulted=longhorn-faulted.md
|
|
- monitoring-capacity=monitoring-capacity.md
|
|
- node-cpu=node-cpu.md
|
|
- node-disk=node-disk.md
|
|
- node-down=node-down.md
|
|
- node-io=node-io.md
|
|
- node-memory=node-memory.md
|
|
- node-memory-swap=node-memory-swap.md
|
|
# Linked from OAuth2ProxyGateUnobserved, and the diagnostic procedure for any gated app
|
|
# that "is up but does not work" — the gate answers an unauthenticated XHR with a 302,
|
|
# not a 401, so the failure arrives as an HTTP 200 carrying the wrong body.
|
|
- oauth2-proxy-auth-failures=oauth2-proxy-auth-failures.md
|
|
# Operator procedure, not alert-linked. Reading the per-app oauth2-proxy sidecar log to tell
|
|
# Keycloak, the sidecar and the app's own login apart. Cookie collision between two sidecars
|
|
# presents as random session loss in ONE app's log and is only legible with both side by side,
|
|
# so the page has to be reachable at 3am without going looking for it in git.
|
|
# SIBLING PAGE to oauth2-proxy-auth-failures above, and they are deliberately both here: this
|
|
# one is the SESSION-LIFECYCLE procedure (why a session died), that one is the DIAGNOSTIC
|
|
# TRIAGE (which of the three layers refused, and how to tell a 302-to-200 from a real success).
|
|
- oauth2-proxy-session=oauth2-proxy-session.md
|
|
# Operator procedure, not an alert page (no alert links to it — the
|
|
# alert-runbooks-resolvable check treats that as advisory). It is served
|
|
# because the plex.tv claim window is short and is not a thing to go looking
|
|
# for in git: /runbooks/plex-claim has to resolve the moment it is needed.
|
|
- plex-claim=plex-claim.md
|
|
# Operator procedure, not an alert page, and NO alert is possible for it: plex renders no
|
|
# metricsEndpoint, so there is no ServiceMonitor, no transcode-mode metric and nothing a rule
|
|
# could evaluate. Served because the feature it completes is INERT until a human ticks a box in
|
|
# the Plex UI — the one step GitOps cannot reach — and a page nobody can find is how a shipped
|
|
# feature stays switched off for months.
|
|
- plex-quicksync=plex-quicksync.md
|
|
# Operator procedure, not alert-linked: how to STOP the PM agent service (three ways,
|
|
# increasing severity), how to read why it is idle, how to tell a worker that was
|
|
# correctly refused by the gate from one that broke, and the one-year expiry of its
|
|
# Anthropic subscription token — which nothing warns about, because Claude Code's expiry
|
|
# notice only fires for an interactive login, never for CLAUDE_CODE_OAUTH_TOKEN.
|
|
- pm-agent=pm-agent.md
|
|
- pvc-capacity=pvc-capacity.md
|
|
- targets-down=targets-down.md
|
|
- update-rollback=update-rollback.md
|
|
- vpn-down=vpn-down.md
|
|
- vpn-egress-stale=vpn-egress-stale.md
|