home-network-k3s-impl/decisions/2026-10-05-pm-agent-issue-to-pr.md

date, branch, title

date	branch	title
2026-10-05	feat/pm-agent-issue-to-pr	An in-cluster AI Program Manager that turns Gitea issues into pull requests — authored, nothing deployed

What changed

A new service, pm-agent, that runs in the cluster, reads the issues of this Gitea repository, decides what to do about each thread as the Program Manager persona this repository already works under, and runs one worker agent per assignment that authors a change, runs the merge gate and opens a pull request. It reacts to issue creation and to comments; the assignment and all correspondence travel through the issue thread and the PR. It authenticates with an Anthropic subscription token. It never pushes to main and never merges.

Design, alternatives and threat model: docs/plans/pm-agent-issue-to-pr.md. Operating surface: platform/pm-agent/README.md. Operator procedures: runbooks/pm-agent.md.

New — the agent itself (infra/agent/): one image (agent-runner) layered on the pinned dev toolbox plus node and the Claude Code CLI, with two fixed entrypoints and no argument surface. pm-dispatch is the PM; pm-worker is one assignment. pm-persona.md is the PM persona, assembled from CLAUDE.md, the frozen log's PM guardrails and the spec repo's agents/roles.md. worker-job-template.yaml is the only Job shape the dispatcher may create, baked into the image rather than mounted, because that template is the shape the admission policy protects. versions.env pins only what the toolbox lacks — deliberately not a second copy of the toolchain pins. build-image.sh builds, verifies in-image as uid 65532 on a read-only rootfs, and never pushes.

New — the deployment (platform/pm-agent/): namespace (no app-namespace label — this is CI plumbing like gitea-actions; PodSecurity restricted is enforced), rbac.yaml (the dispatcher may create/get/list/watch jobs in its own namespace and nothing else; the worker identity is bound to nothing and mounts no token), the dispatcher Deployment (1 replica, Recreate — two dispatchers would double-trigger every thread), configmap.yaml (every knob, with its reasoning), networkpolicy.yaml (default-deny; egress to gitea:3000, kube-apiserver for the dispatcher only, tcp/443; no ingress at all), and pm-agent-credentials.sops.yaml holding placeholders.

New — the boundary (platform/kyverno/admission/pm-agent-job-shape.yaml, Enforce + failurePolicy: Fail): Kubernetes places no use check on serviceAccountName, so "may create Jobs" is "may run arbitrary code as the namespace's most privileged identity". Six rules admit only the committed shape. It is listed in platform-config/kustomization.yaml, which is the only list that actually applies ClusterPolicies — adding it to platform/kyverno/admission/kustomization.yaml alone is a silent no-op, which is exactly how the conformance boundary shipped green and never reached the cluster on 2026-09-29.

Wiring: clusters/base/pm-agent.yaml (dependsOn apps-wave-0 — what it needs is Gitea, not Keycloak or the Gateway; wait: false, because the pod is expected to be unhealthy while its credentials and image digest are placeholders), its line in clusters/base/kustomization.yaml, a SOPS decryption patch in both overlays, and an inClusterClients entry for the pm-agent namespace on catalog/gitea.yaml with apps/wave-0/gitea.yaml re-rendered by the canonical command.

Three design choices worth naming, with the alternatives recorded in the plan: it polls rather than receiving webhooks (Gitea's webhook SSRF guard, lost deliveries, and no need for inbound reach); it keeps no state — the trigger is "the newest thing in this thread was not written by me", so the thread is the state and there is no PVC and no backup obligation; and the model holds no tools — its reply is one JSON object that this code validates, so nothing in the path executes text.

What I verified

All offline, in the pinned toolbox (./infra/devcontainer/dev.sh run), never on the host.

• kyverno test platform/kyverno/tests/pm-agent → 28 passed, 0 failed under the pinned CLI (v1.19.0). The first run was 27 passed, 1 failed and caught a real defect: the volume rule asserted length(emptyDir) != 0, which denies the valid emptyDir: {}. Fixed by asserting the volume's key set instead, which also denies every other source type without enumerating them. The suite includes the two cases a careless policy admits — a second container while containers[0] is compliant, and a volume named workspace whose source is a hostPath.
• ./ci/gate.sh → exit 3, MERGE GATE: UNDETERMINED. Not green, and not reported as green. Per-stage, quoted from the run: spec-immutability pass (the spec repo is a sibling of this checkout and is clean at the pinned SHA), decisions-log pass, lint pass, l0 pass, render-golden pass (10 services reproduce), l1 pass, approval-policy pass (N/A off a pull-request event), conformance declared-skip (no cluster — Phase 0), and gitea-settings SKIPPED: SETTINGS_API_TOKEN is not set on this workstation, so branch protection and runner registration are UNVERIFIED by this run. The gate names that single stage as the reason for rc 3 ("Stages that did not run: gitea-settings"), so nothing in the change is unexamined — but rc 3 is not a pass and this entry does not call it one.
• The PR's OWN CI run, read from its log rather than from the status tick (PR #57, head 8b5e950b, .gitea/workflows/gate.yml with GATE_STRICT_SKIPS=true and REQUIRE_GITEA_SETTINGS=true): decisions-log pass, lint pass, l0 pass, render-golden pass, l1 pass, gitea-settings pass — including block_admin_merge_override = true, which is the one stage my workstation cannot run at all — spec-immutability SKIPPED (the spec repo is absent in CI, as its own step tolerates), conformance SKIPPED, and approval-policy FAIL → MERGE GATE: BLOCKED. The failure is only the sensitive-path rule: "This PR changes the pipeline or the security posture it rests on, and has NO approving review from anyone other than its author ('gitea-admin')", over eleven paths — clusters/{base,dev,prod}/*, platform-config/kustomization.yaml, platform/kyverno/{admission,tests}/pm-agent*, platform/pm-agent/rbac.yaml and platform/pm-agent/pm-agent-credentials.sops.yaml. That is the gate working, not a defect in the change, and it is the one thing an agent must not clear for itself: the operator posts /approve-override <reason> as a PR review, or a non-author reviews it. CI is therefore red and will stay red until a human acts.
• The commit-status API did not reflect any of that. While the run completed and its log carried the verdict above, GET /commits/{sha}/status still reported gate / gate (pull_request): pending — Waiting to run. Worth knowing before anyone builds automation on that endpoint: on this Gitea the status tick lagged the run indefinitely, so the log is the source of truth. A fresh instance of §2b — "a green tick is not proof anything ran"; here the tick was not even wrong, it was absent.
• The gate caught two real defects in this change before I did, which is worth recording because both were in the parts I was most confident about:
  1. flux-postbuild-substitution → 1 UNDEFINED + 1 FATAL-PARSE. My ConfigMap's comment explaining the envsubst hazard contained two dollar-brace tokens, and envsubst runs over comment lines too — so the paragraph warning about the trap was itself the trap. Rewritten without braces.
  2. no-plaintext-secrets → the unencrypted .example template was read as REFERENCED by a kustomization, because the kustomization's comment named it by filename. Correctly refused: the distance between a filename in a comment and a line under resources: is one careless edit. The comment now points at it in prose.
• infra/agent/probe-validation.sh → 17 refusals, 0 failures. A new, committed negative-path probe for pm-worker's input guards — the things that stand between an injected environment variable and a clone from the wrong host or a push to the wrong ref. Each case asserts a non-zero exit AND the specific message, so a guard that stopped firing cannot pass by failing somewhere else. It refuses: an argument on the pinned command; main, a wrong prefix, feat/x;id, ../../etc/passwd, too-short slugs, and a branch equal to the base; a redirected git host, a real-and-reachable wrong one (github.com — the egress policy permits tcp/443, so this one is genuinely reachable), and the right host with a smuggled path; a malformed owner or repository; an unknown mode; a non-numeric and a zero issue number; and each missing credential. build-image.sh also runs it inside the built image, where the interpreter is the one that will run in the cluster.
• Writing that probe corrected an overclaim in the code. The PM_BRANCH != "main" guard is UNREACHABLE while the pattern check above it holds (main has no feat|fix|docs|chore/ prefix), so the comment claiming it as the defence against pushing to main was wrong about which line does the work. It is kept — as an assertion that does not depend on the regex being right — and the comment now says so.
• Two more defects found by self-review of the unattended paths, both fixed: the worker's EXIT trap ended in a failing test, which under set -e can replace the script's real exit code with that failure; and a provenance log line read a file through a redirect that set -o pipefail would have turned into a dead worker if the file were absent.
• A self-review of the trigger found and fixed one real defect before any of this ran: last_activity() compared RFC 3339 timestamp STRINGS to find the newest comment, which is only correct while every offset Gitea returns is identical — and the failure mode is the worst available one, the PM answering its own last comment in a loop. It now takes the last element of the comments list, which is the order the endpoint returns, so there is nothing about timezones to be right about.
• check-jsonschema --schemafile schemas/servicedefinition.schema.json catalog/gitea.yaml → ok.
• The golden reproduces: python3 catalog/render.py --all catalog -o apps --wave-layout --no-image-automation leaves apps/wave-0/gitea.yaml byte-identical to what is committed, and the only diff versus main is the new gitea-allow-pm-agent-ingress CiliumNetworkPolicy.
• Syntax: bash -n infra/agent/pm-worker and ast.parse on pm-dispatch.py both clean. yaml.safe_load_all parses every new manifest.
• The spec repo has no git remote — git remote -v in ../home-network-k3s prints nothing. That is why a worker's gate can never run spec-immutability in-cluster, and it is measured rather than assumed. The worker's acceptance rule names that stage and gitea-settings explicitly and refuses any other skip.
• PM_SELFTEST=1 python3 infra/agent/pm-dispatch.py → exit 0, 26 assertions, all ok (run in the toolbox against the repository's own files, with PM_WORKER_JOB_TEMPLATE and PM_PERSONA_FILE pointed at infra/agent/). It covers the branch vocabulary in both directions (feature/x, main, feat/AB, feat/x;rm -rf / and ../../etc/passwd all refused), the five result-envelope shapes extract_decision has to survive plus three it must reject, the committed Job template against its whole contract (pinned command, no args, the powerless SA, no mounted token, the volume set, the security context, and every env name inside the dispatcher's allowlist), and that the persona still says the four things it must. build-image.sh runs the same selftest inside the image; that invocation is the one that has not happened, because the image has not been built.
• Claude Code facts were read from the current documentation, not from memory: claude setup-token mints a one-year subscription OAuth token consumed via CLAUDE_CODE_OAUTH_TOKEN; ANTHROPIC_API_KEY/ANTHROPIC_AUTH_TOKEN rank above it in credential precedence (hence the unsets and the env-name allowlist); --bare does not read it (hence neither entrypoint uses that flag); --permission-prompts none requires

  = 2.1.259, which is why the image asserts a version floor. The pin 2.1.285 is the npm registry's stable dist-tag, measured on 2026-10-05.

Merged main in, after PR #55 and #56 landed

One conflict, in clusters/base/kustomization.yaml: PR #55 (vpn-ingress) and this branch both appended a resource line to the same tail. Resolved by keeping both, main's first — vpn-ingress is a platform-config-tier leaf and pm-agent dependsOn apps-wave-0, which is later. Nothing else conflicted; runbooks/kustomization.yaml auto-merged with both entries, and the handoff entries are separate files with nothing to merge, which is exactly what decisions/ being a directory buys.

PR #56 added CLAUDE.md §0 — work in your own worktree, never the shared checkout — while this branch was open, and it named this run's uncommitted directories as its measured example. Two consequences were handled rather than noted:

• This merge, and everything after it, was done in a worktree outside the repository root (/var/home/keith/source/wt-pm-agent), per §0.
• §0 would have misled a worker agent, so infra/agent/worker-rules.md now says so explicitly. A worker reads CLAUDE.md first, and §0's instruction is to create a worktree outside the shared checkout. There is no shared checkout in a worker's container — it holds a fresh clone of its own, which is what §0 is actually asking for — and a worker that obeyed §0 literally would git worktree add, edit somewhere the wrapper does not look, and have its work committed by nobody while the issue got a "changed nothing" report. The rules file now tells it §0 is already satisfied and not to make a worktree.

The operator's approving review goes STALE on each new head, and the gate does not care. I first wrote here that it would need re-approving. That was wrong, and the correction is worth keeping because it is a fact about this repository's gate rather than about this change:

• Measured on the live branch protection: dismiss_stale_approvals = true, ignore_stale_approvals = false — so pushing the merge did stale jakman85's review (confirmed: GET /pulls/57/reviews → APPROVED, stale: true).
• But ci/approval-policy.sh selects reviews on .state == "APPROVED" and .user.login != author and reads nothing about staleness (line 147). So a stale approval still satisfies the sensitive-path rule.
• And required_approvals = 0, so Gitea's own merge button never required one either.

Measured, not inferred: CI run #257 on the merged head printed approved by 'jakman85' (author 'gitea-admin') — sensitive change is human-reviewed. → stage approval-policy: pass, with gitea-settings: pass as well. The job's own exit was MERGE GATE: UNDETERMINED (rc 3 — spec-immutability and conformance skipped, as the workflow tolerates with || [ $? -eq 3 ]), so the required check gate / gate (pull_request) is green and PR #57 reports mergeable: true.

Nothing is therefore waiting on the operator except the merge itself, which is theirs by CLAUDE.md §1 and never an agent's. Worth noting for whoever reads this later: the rule "push again and you must override again" applies to the /approve-override directive, which is staleness-checked against the head commit — not to an ordinary approving review, which this gate accepts stale.

What I did NOT verify

Nothing in this change has run. That is the headline, and everything below is a consequence of it.

• The image has never been built. The nodejs module-stream install, the npm i -g @anthropic-ai/[email protected] layer, the version-floor assertion, the COPY ownership assertions and build-image.sh's whole in-image verification (including PM_SELFTEST=1 pm-dispatch) are unexercised. Expect the build to need iteration.
• It has never been pushed, so the committed image digests are the deliberate all-zero placeholder. Nothing has pulled this image.
• No pod has started. The dispatcher has never authenticated to Gitea, never polled, never triaged an issue, never created a Job. No worker has cloned, branched, run a session, run the gate in-cluster, pushed a branch, opened a pull request or commented on an issue. Every claim about runtime behaviour is a design claim.
• The admission policy has not met a live API server. 28 offline assertions prove the policy refuses; nothing proves the API server enforces it here. That is an L2 test and it needs a cluster. Note also that CLAUDE.md §2b applies to me as much as to anyone: I did not query whether Kyverno is healthy, whether pm-agent-job-shape reconciles, or whether platform-config is Ready.
• The network policies have not been exercised. Whether toEntities: kube-apiserver actually permits kubectl from this namespace on k3s, and whether world:443 reaches api.anthropic.com through this cluster's egress, are both unmeasured.
• Neither credential exists. The Gitea bot account has not been created, so its permission has not been measured — and that measurement is the control that keeps the agent from clearing its own sensitive-path review. Do it before the first real run.
• Gitea's webhook ALLOWED_HOST_LIST was not queried. It is cited as a reason webhooks were rejected; it is not a measured fact about this Gitea.
• The write-can-merge gap is real and unclosed. No Gitea token scope separates "push a branch and open a pull request" from "merge it". The service issues no merge call, which is a property of the code, not of the credential.
• The worker's credentials are inside the model session's reach. Same container, same uid; no isolation exists and none is claimed. Bounded by token scope and egress only.
• This branch touches sensitive paths — clusters/**, platform/kyverno/**, platform-config/kustomization.yaml, **/rbac.yaml, **/*.sops.yaml — so it needs an approving review from a non-author, or an administrator's /approve-override posted by the operator. An agent must not clear this one on its own PR; that is precisely the self-approval the stage exists to prevent.
• No observability. No metrics endpoint, no ServiceMonitor, no alert, and the namespace's default-deny would drop a scrape from monitoring if one were added. Follow-ups are listed in the plan, in the order they are worth doing.