home-network-k3s-impl/decisions/2026-10-05-pm-agent-issue-to-pr.md

---
date: 2026-10-05
branch: feat/pm-agent-issue-to-pr
title: An in-cluster AI Program Manager that turns Gitea issues into pull requests — authored, nothing deployed
---

## What changed

A new service, `pm-agent`, that runs in the cluster, reads the issues of this Gitea
repository, decides what to do about each thread **as the Program Manager persona this
repository already works under**, and runs one worker agent per assignment that authors a
change, runs the merge gate and opens a pull request. It reacts to issue creation and to
comments; the assignment and all correspondence travel through the issue thread and the PR.
It authenticates with an Anthropic **subscription** token. It never pushes to `main` and
never merges.

Design, alternatives and threat model: `docs/plans/pm-agent-issue-to-pr.md`. Operating
surface: `platform/pm-agent/README.md`. Operator procedures:
`runbooks/pm-agent.md`.

**New — the agent itself** (`infra/agent/`): one image (`agent-runner`) layered on the pinned
dev toolbox plus node and the Claude Code CLI, with two fixed entrypoints and no argument
surface. `pm-dispatch` is the PM; `pm-worker` is one assignment. `pm-persona.md` is the PM
persona, assembled from `CLAUDE.md`, the frozen log's PM guardrails and the spec repo's
`agents/roles.md`. `worker-job-template.yaml` is the only Job shape the dispatcher may
create, baked into the image rather than mounted, because that template *is* the shape the
admission policy protects. `versions.env` pins only what the toolbox lacks — deliberately not
a second copy of the toolchain pins. `build-image.sh` builds, verifies in-image as uid 65532
on a read-only rootfs, and **never pushes**.

**New — the deployment** (`platform/pm-agent/`): namespace (no `app-namespace` label — this
is CI plumbing like `gitea-actions`; PodSecurity `restricted` is enforced), `rbac.yaml` (the
dispatcher may `create/get/list/watch jobs` in its own namespace and nothing else; the worker
identity is bound to **nothing** and mounts no token), the dispatcher Deployment (1 replica,
`Recreate` — two dispatchers would double-trigger every thread), `configmap.yaml` (every knob,
with its reasoning), `networkpolicy.yaml` (default-deny; egress to gitea:3000,
kube-apiserver for the dispatcher only, tcp/443; **no ingress at all**), and
`pm-agent-credentials.sops.yaml` holding **placeholders**.

**New — the boundary** (`platform/kyverno/admission/pm-agent-job-shape.yaml`, `Enforce` +
`failurePolicy: Fail`): Kubernetes places no `use` check on `serviceAccountName`, so "may
create Jobs" is "may run arbitrary code as the namespace's most privileged identity". Six
rules admit only the committed shape. It is listed in `platform-config/kustomization.yaml`,
which is the only list that actually applies ClusterPolicies — adding it to
`platform/kyverno/admission/kustomization.yaml` alone is a silent no-op, which is exactly how
the conformance boundary shipped green and never reached the cluster on 2026-09-29.

**Wiring:** `clusters/base/pm-agent.yaml` (dependsOn `apps-wave-0` — what it needs is Gitea,
not Keycloak or the Gateway; `wait: false`, because the pod is *expected* to be unhealthy
while its credentials and image digest are placeholders), its line in
`clusters/base/kustomization.yaml`, a SOPS decryption patch in **both** overlays, and an
`inClusterClients` entry for the `pm-agent` namespace on `catalog/gitea.yaml` with
`apps/wave-0/gitea.yaml` re-rendered by the canonical command.

Three design choices worth naming, with the alternatives recorded in the plan: it **polls**
rather than receiving webhooks (Gitea's webhook SSRF guard, lost deliveries, and no need for
inbound reach); it keeps **no state** — the trigger is "the newest thing in this thread was
not written by me", so the thread is the state and there is no PVC and no backup obligation;
and the **model holds no tools** — its reply is one JSON object that this code validates, so
nothing in the path executes text.

## What I verified

All offline, in the pinned toolbox (`./infra/devcontainer/dev.sh run`), never on the host.

- **`kyverno test platform/kyverno/tests/pm-agent` → 28 passed, 0 failed** under the pinned
  CLI (v1.19.0). The first run was **27 passed, 1 failed** and caught a real defect: the
  volume rule asserted `length(emptyDir) != 0`, which denies the valid `emptyDir: {}`. Fixed
  by asserting the volume's **key set** instead, which also denies every other source type
  without enumerating them. The suite includes the two cases a careless policy admits — a
  second container while `containers[0]` is compliant, and a volume *named* `workspace` whose
  source is a `hostPath`.
- **`./ci/gate.sh` → exit 3, MERGE GATE: UNDETERMINED. Not green, and not reported as
  green.** Per-stage, quoted from the run: spec-immutability **pass** (the spec repo is a
  sibling of this checkout and is clean at the pinned SHA), decisions-log **pass**, lint
  **pass**, l0 **pass**, render-golden **pass** (10 services reproduce), l1 **pass**,
  approval-policy **pass** (N/A off a pull-request event), conformance **declared-skip** (no
  cluster — Phase 0), and `gitea-settings` **SKIPPED**: `SETTINGS_API_TOKEN` is not set on
  this workstation, so branch protection and runner registration are UNVERIFIED by this run.
  The gate names that single stage as the reason for rc 3 ("Stages that did not run:
  gitea-settings"), so nothing in the change is unexamined — but rc 3 is not a pass and this
  entry does not call it one.
- **The PR's OWN CI run, read from its log rather than from the status tick** (PR #57, head
  `8b5e950b`, `.gitea/workflows/gate.yml` with `GATE_STRICT_SKIPS=true` and
  `REQUIRE_GITEA_SETTINGS=true`): decisions-log **pass**, lint **pass**, l0 **pass**,
  render-golden **pass**, l1 **pass**, **gitea-settings pass** — including
  `block_admin_merge_override = true`, which is the one stage my workstation cannot run at
  all — spec-immutability **SKIPPED** (the spec repo is absent in CI, as its own step
  tolerates), conformance **SKIPPED**, and **approval-policy FAIL → MERGE GATE: BLOCKED**.
  The failure is *only* the sensitive-path rule: "This PR changes the pipeline or the
  security posture it rests on, and has NO approving review from anyone other than its author
  ('gitea-admin')", over eleven paths — `clusters/{base,dev,prod}/*`,
  `platform-config/kustomization.yaml`, `platform/kyverno/{admission,tests}/pm-agent*`,
  `platform/pm-agent/rbac.yaml` and `platform/pm-agent/pm-agent-credentials.sops.yaml`.
  **That is the gate working, not a defect in the change**, and it is the one thing an agent
  must not clear for itself: the operator posts `/approve-override <reason>` as a PR *review*,
  or a non-author reviews it. CI is therefore red and will stay red until a human acts.
- **The commit-status API did not reflect any of that.** While the run completed and its log
  carried the verdict above, `GET /commits/{sha}/status` still reported
  `gate / gate (pull_request): pending — Waiting to run`. Worth knowing before anyone builds
  automation on that endpoint: on this Gitea the status tick lagged the run indefinitely, so
  the log is the source of truth. A fresh instance of §2b — "a green tick is not proof
  anything ran"; here the tick was not even wrong, it was absent.
- **The gate caught two real defects in this change before I did**, which is worth recording
  because both were in the parts I was most confident about:
  1. `flux-postbuild-substitution` → 1 UNDEFINED + 1 FATAL-PARSE. My ConfigMap's *comment*
     explaining the envsubst hazard contained two dollar-brace tokens, and envsubst runs over
     comment lines too — so the paragraph warning about the trap was itself the trap. Rewritten
     without braces.
  2. `no-plaintext-secrets` → the unencrypted `.example` template was read as REFERENCED by a
     kustomization, because the kustomization's comment named it by filename. Correctly
     refused: the distance between a filename in a comment and a line under `resources:` is
     one careless edit. The comment now points at it in prose.
- **`infra/agent/probe-validation.sh` → 17 refusals, 0 failures.** A new, committed
  negative-path probe for `pm-worker`'s input guards — the things that stand between an
  injected environment variable and a clone from the wrong host or a push to the wrong ref.
  Each case asserts a non-zero exit AND the specific message, so a guard that stopped firing
  cannot pass by failing somewhere else. It refuses: an argument on the pinned command;
  `main`, a wrong prefix, `feat/x;id`, `../../etc/passwd`, too-short slugs, and a branch equal
  to the base; a redirected git host, a real-and-reachable wrong one (`github.com` — the
  egress policy permits tcp/443, so this one is genuinely reachable), and the right host with
  a smuggled path; a malformed owner or repository; an unknown mode; a non-numeric and a zero
  issue number; and each missing credential. `build-image.sh` also runs it inside the built
  image, where the interpreter is the one that will run in the cluster.
- **Writing that probe corrected an overclaim in the code.** The `PM_BRANCH != "main"` guard
  is UNREACHABLE while the pattern check above it holds (`main` has no `feat|fix|docs|chore/`
  prefix), so the comment claiming it as the defence against pushing to main was wrong about
  which line does the work. It is kept — as an assertion that does not depend on the regex
  being right — and the comment now says so.
- **Two more defects found by self-review of the unattended paths, both fixed:** the worker's
  `EXIT` trap ended in a failing test, which under `set -e` can replace the script's real exit
  code with that failure; and a provenance *log line* read a file through a redirect that
  `set -o pipefail` would have turned into a dead worker if the file were absent.
- **A self-review of the trigger found and fixed one real defect** before any of this ran:
  `last_activity()` compared RFC 3339 timestamp STRINGS to find the newest comment, which is
  only correct while every offset Gitea returns is identical — and the failure mode is the
  worst available one, the PM answering its own last comment in a loop. It now takes the last
  element of the comments list, which is the order the endpoint returns, so there is nothing
  about timezones to be right about.
- **`check-jsonschema --schemafile schemas/servicedefinition.schema.json catalog/gitea.yaml`
  → ok.**
- **The golden reproduces:** `python3 catalog/render.py --all catalog -o apps --wave-layout
  --no-image-automation` leaves `apps/wave-0/gitea.yaml` byte-identical to what is committed,
  and the only diff versus `main` is the new `gitea-allow-pm-agent-ingress`
  CiliumNetworkPolicy.
- **Syntax:** `bash -n infra/agent/pm-worker` and `ast.parse` on `pm-dispatch.py` both clean.
  `yaml.safe_load_all` parses every new manifest.
- **The spec repo has no git remote** — `git remote -v` in `../home-network-k3s` prints
  nothing. That is *why* a worker's gate can never run `spec-immutability` in-cluster, and it
  is measured rather than assumed. The worker's acceptance rule names that stage and
  `gitea-settings` explicitly and refuses any other skip.
- **`PM_SELFTEST=1 python3 infra/agent/pm-dispatch.py` → exit 0, 26 assertions, all ok**
  (run in the toolbox against the repository's own files, with `PM_WORKER_JOB_TEMPLATE` and
  `PM_PERSONA_FILE` pointed at `infra/agent/`). It covers the branch vocabulary in both
  directions (`feature/x`, `main`, `feat/AB`, `feat/x;rm -rf /` and `../../etc/passwd` all
  refused), the five result-envelope shapes `extract_decision` has to survive plus three it
  must reject, the committed Job template against its whole contract (pinned command, no
  args, the powerless SA, no mounted token, the volume set, the security context, and every
  env name inside the dispatcher's allowlist), and that the persona still says the four
  things it must. `build-image.sh` runs the same selftest inside the image; that invocation
  is the one that has not happened, because the image has not been built.
- **Claude Code facts were read from the current documentation, not from memory:**
  `claude setup-token` mints a one-year subscription OAuth token consumed via
  `CLAUDE_CODE_OAUTH_TOKEN`; `ANTHROPIC_API_KEY`/`ANTHROPIC_AUTH_TOKEN` rank **above** it in
  credential precedence (hence the unsets and the env-name allowlist); `--bare` does **not**
  read it (hence neither entrypoint uses that flag); `--permission-prompts none` requires
  >= 2.1.259, which is why the image asserts a version floor. The pin `2.1.285` is the npm
  registry's `stable` dist-tag, measured on 2026-10-05.

## Merged `main` in, after PR #55 and #56 landed

One conflict, in `clusters/base/kustomization.yaml`: PR #55 (vpn-ingress) and this branch
both appended a resource line to the same tail. Resolved by keeping both, main's first —
`vpn-ingress` is a platform-config-tier leaf and `pm-agent` dependsOn `apps-wave-0`, which
is later. Nothing else conflicted; `runbooks/kustomization.yaml` auto-merged with both
entries, and the handoff entries are separate files with nothing to merge, which is exactly
what `decisions/` being a directory buys.

**PR #56 added `CLAUDE.md` §0 — work in your own worktree, never the shared checkout —
while this branch was open, and it named this run's uncommitted directories as its
measured example.** Two consequences were handled rather than noted:

- This merge, and everything after it, was done in a worktree outside the repository root
  (`/var/home/keith/source/wt-pm-agent`), per §0.
- **§0 would have misled a worker agent, so `infra/agent/worker-rules.md` now says so
  explicitly.** A worker reads `CLAUDE.md` first, and §0's instruction is to create a
  worktree outside the shared checkout. There is no shared checkout in a worker's container
  — it holds a fresh clone of its own, which is what §0 is actually asking for — and a
  worker that obeyed §0 literally would `git worktree add`, edit somewhere the wrapper does
  not look, and have its work committed by nobody while the issue got a "changed nothing"
  report. The rules file now tells it §0 is already satisfied and not to make a worktree.

**The operator's approving review goes STALE on each new head, and the gate does not care.**
I first wrote here that it would need re-approving. That was wrong, and the correction is
worth keeping because it is a fact about this repository's gate rather than about this
change:

- Measured on the live branch protection: `dismiss_stale_approvals = true`,
  `ignore_stale_approvals = false` — so pushing the merge did stale `jakman85`'s review
  (confirmed: `GET /pulls/57/reviews` → `APPROVED`, `stale: true`).
- But `ci/approval-policy.sh` selects reviews on `.state == "APPROVED"` and
  `.user.login != author` and reads **nothing about staleness** (line 147). So a stale
  approval still satisfies the sensitive-path rule.
- And `required_approvals = 0`, so Gitea's own merge button never required one either.

Measured, not inferred: CI run **#257** on the merged head printed
`approved by 'jakman85' (author 'gitea-admin') — sensitive change is human-reviewed.` →
`stage approval-policy: pass`, with `gitea-settings: pass` as well. The job's own exit was
`MERGE GATE: UNDETERMINED` (rc 3 — `spec-immutability` and `conformance` skipped, as the
workflow tolerates with `|| [ $? -eq 3 ]`), so the required check
`gate / gate (pull_request)` is **green** and PR #57 reports `mergeable: true`.

Nothing is therefore waiting on the operator except **the merge itself**, which is theirs
by CLAUDE.md §1 and never an agent's. Worth noting for whoever reads this later: the rule
"push again and you must override again" applies to the `/approve-override` **directive**,
which is staleness-checked against the head commit — not to an ordinary approving review,
which this gate accepts stale.

## What I did NOT verify

**Nothing in this change has run.** That is the headline, and everything below is a
consequence of it.

- **The image has never been built.** The nodejs module-stream install, the `npm i -g
  @anthropic-ai/claude-code@2.1.285` layer, the version-floor assertion, the `COPY`
  ownership assertions and `build-image.sh`'s whole in-image verification (including
  `PM_SELFTEST=1 pm-dispatch`) are **unexercised**. Expect the build to need iteration.
- **It has never been pushed,** so the committed image digests are the deliberate all-zero
  placeholder. Nothing has pulled this image.
- **No pod has started.** The dispatcher has never authenticated to Gitea, never polled,
  never triaged an issue, never created a Job. No worker has cloned, branched, run a session,
  run the gate in-cluster, pushed a branch, opened a pull request or commented on an issue.
  Every claim about runtime behaviour is a design claim.
- **The admission policy has not met a live API server.** 28 offline assertions prove the
  policy *refuses*; nothing proves the API server *enforces* it here. That is an L2 test and
  it needs a cluster. Note also that `CLAUDE.md` §2b applies to me as much as to anyone: I
  did not query whether Kyverno is healthy, whether `pm-agent-job-shape` reconciles, or
  whether `platform-config` is Ready.
- **The network policies have not been exercised.** Whether `toEntities: kube-apiserver`
  actually permits `kubectl` from this namespace on k3s, and whether `world:443` reaches
  `api.anthropic.com` through this cluster's egress, are both unmeasured.
- **Neither credential exists.** The Gitea bot account has not been created, so its
  permission has not been measured — and *that measurement is the control* that keeps the
  agent from clearing its own sensitive-path review. Do it before the first real run.
- **Gitea's webhook `ALLOWED_HOST_LIST` was not queried.** It is cited as a reason webhooks
  were rejected; it is not a measured fact about this Gitea.
- **The `write`-can-merge gap is real and unclosed.** No Gitea token scope separates "push a
  branch and open a pull request" from "merge it". The service issues no merge call, which is
  a property of the code, not of the credential.
- **The worker's credentials are inside the model session's reach.** Same container, same
  uid; no isolation exists and none is claimed. Bounded by token scope and egress only.
- **This branch touches sensitive paths** — `clusters/**`, `platform/kyverno/**`,
  `platform-config/kustomization.yaml`, `**/rbac.yaml`, `**/*.sops.yaml` — so it needs an
  approving review from a non-author, or an administrator's `/approve-override` **posted by
  the operator**. An agent must not clear this one on its own PR; that is precisely the
  self-approval the stage exists to prevent.
- **No observability.** No metrics endpoint, no ServiceMonitor, no alert, and the namespace's
  default-deny would drop a scrape from `monitoring` if one were added. Follow-ups are listed
  in the plan, in the order they are worth doing.