Files
home-network-k3s-impl/decisions/2026-10-05-pm-agent-issue-to-pr.md
T
jakman85andClaude Opus 5 8b5e950baa
gate / gate (pull_request) Failing after 1m52s
gate / conformance (pull_request) Has been skipped
test(pm-agent): a committed negative-path probe for the worker's input guards
pm-worker's command is pinned by admission with no args, so everything a caller could
influence arrives in the environment — and the guards at the top of that script are what
stand between an injected variable and a clone from the wrong host or a push to the
wrong ref. Nothing exercised them, which makes them a claim rather than a control.

probe-validation.sh asserts 17 refusals, each on BOTH a non-zero exit and the specific
message, so a guard that stopped firing cannot pass by failing somewhere else. It covers
a redirected git host, a real and reachable WRONG one (the egress policy permits tcp/443,
so that case is not hypothetical), the right host with a smuggled path, 'main' and five
other bad branch shapes, a malformed owner/repo, a bad mode, a bad issue number, and each
missing credential. build-image.sh runs it inside the built image too.

Writing it found three things:
 - the PM_BRANCH != main guard is unreachable while the pattern check above it holds, so
   the comment claiming it as the defence was wrong about which line does the work. Kept
   as an assertion independent of the regex; the comment now says so.
 - the EXIT trap ended in a failing test, which under set -e can replace the script's
   real exit code with that failure.
 - a provenance LOG LINE read a file through a redirect that set -o pipefail would have
   turned into a dead worker if the file were absent.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-10-05 22:34:46 -05:00

190 lines
14 KiB
Markdown

---
date: 2026-10-05
branch: feat/pm-agent-issue-to-pr
title: An in-cluster AI Program Manager that turns Gitea issues into pull requests — authored, nothing deployed
---
## What changed
A new service, `pm-agent`, that runs in the cluster, reads the issues of this Gitea
repository, decides what to do about each thread **as the Program Manager persona this
repository already works under**, and runs one worker agent per assignment that authors a
change, runs the merge gate and opens a pull request. It reacts to issue creation and to
comments; the assignment and all correspondence travel through the issue thread and the PR.
It authenticates with an Anthropic **subscription** token. It never pushes to `main` and
never merges.
Design, alternatives and threat model: `docs/plans/pm-agent-issue-to-pr.md`. Operating
surface: `platform/pm-agent/README.md`. Operator procedures:
`runbooks/pm-agent.md`.
**New — the agent itself** (`infra/agent/`): one image (`agent-runner`) layered on the pinned
dev toolbox plus node and the Claude Code CLI, with two fixed entrypoints and no argument
surface. `pm-dispatch` is the PM; `pm-worker` is one assignment. `pm-persona.md` is the PM
persona, assembled from `CLAUDE.md`, the frozen log's PM guardrails and the spec repo's
`agents/roles.md`. `worker-job-template.yaml` is the only Job shape the dispatcher may
create, baked into the image rather than mounted, because that template *is* the shape the
admission policy protects. `versions.env` pins only what the toolbox lacks — deliberately not
a second copy of the toolchain pins. `build-image.sh` builds, verifies in-image as uid 65532
on a read-only rootfs, and **never pushes**.
**New — the deployment** (`platform/pm-agent/`): namespace (no `app-namespace` label — this
is CI plumbing like `gitea-actions`; PodSecurity `restricted` is enforced), `rbac.yaml` (the
dispatcher may `create/get/list/watch jobs` in its own namespace and nothing else; the worker
identity is bound to **nothing** and mounts no token), the dispatcher Deployment (1 replica,
`Recreate` — two dispatchers would double-trigger every thread), `configmap.yaml` (every knob,
with its reasoning), `networkpolicy.yaml` (default-deny; egress to gitea:3000,
kube-apiserver for the dispatcher only, tcp/443; **no ingress at all**), and
`pm-agent-credentials.sops.yaml` holding **placeholders**.
**New — the boundary** (`platform/kyverno/admission/pm-agent-job-shape.yaml`, `Enforce` +
`failurePolicy: Fail`): Kubernetes places no `use` check on `serviceAccountName`, so "may
create Jobs" is "may run arbitrary code as the namespace's most privileged identity". Six
rules admit only the committed shape. It is listed in `platform-config/kustomization.yaml`,
which is the only list that actually applies ClusterPolicies — adding it to
`platform/kyverno/admission/kustomization.yaml` alone is a silent no-op, which is exactly how
the conformance boundary shipped green and never reached the cluster on 2026-09-29.
**Wiring:** `clusters/base/pm-agent.yaml` (dependsOn `apps-wave-0` — what it needs is Gitea,
not Keycloak or the Gateway; `wait: false`, because the pod is *expected* to be unhealthy
while its credentials and image digest are placeholders), its line in
`clusters/base/kustomization.yaml`, a SOPS decryption patch in **both** overlays, and an
`inClusterClients` entry for the `pm-agent` namespace on `catalog/gitea.yaml` with
`apps/wave-0/gitea.yaml` re-rendered by the canonical command.
Three design choices worth naming, with the alternatives recorded in the plan: it **polls**
rather than receiving webhooks (Gitea's webhook SSRF guard, lost deliveries, and no need for
inbound reach); it keeps **no state** — the trigger is "the newest thing in this thread was
not written by me", so the thread is the state and there is no PVC and no backup obligation;
and the **model holds no tools** — its reply is one JSON object that this code validates, so
nothing in the path executes text.
## What I verified
All offline, in the pinned toolbox (`./infra/devcontainer/dev.sh run`), never on the host.
- **`kyverno test platform/kyverno/tests/pm-agent` → 28 passed, 0 failed** under the pinned
CLI (v1.19.0). The first run was **27 passed, 1 failed** and caught a real defect: the
volume rule asserted `length(emptyDir) != 0`, which denies the valid `emptyDir: {}`. Fixed
by asserting the volume's **key set** instead, which also denies every other source type
without enumerating them. The suite includes the two cases a careless policy admits — a
second container while `containers[0]` is compliant, and a volume *named* `workspace` whose
source is a `hostPath`.
- **`./ci/gate.sh` → exit 3, MERGE GATE: UNDETERMINED. Not green, and not reported as
green.** Per-stage, quoted from the run: spec-immutability **pass** (the spec repo is a
sibling of this checkout and is clean at the pinned SHA), decisions-log **pass**, lint
**pass**, l0 **pass**, render-golden **pass** (10 services reproduce), l1 **pass**,
approval-policy **pass** (N/A off a pull-request event), conformance **declared-skip** (no
cluster — Phase 0), and `gitea-settings` **SKIPPED**: `SETTINGS_API_TOKEN` is not set on
this workstation, so branch protection and runner registration are UNVERIFIED by this run.
The gate names that single stage as the reason for rc 3 ("Stages that did not run:
gitea-settings"), so nothing in the change is unexamined — but rc 3 is not a pass and this
entry does not call it one.
- **The gate caught two real defects in this change before I did**, which is worth recording
because both were in the parts I was most confident about:
1. `flux-postbuild-substitution` → 1 UNDEFINED + 1 FATAL-PARSE. My ConfigMap's *comment*
explaining the envsubst hazard contained two dollar-brace tokens, and envsubst runs over
comment lines too — so the paragraph warning about the trap was itself the trap. Rewritten
without braces.
2. `no-plaintext-secrets` → the unencrypted `.example` template was read as REFERENCED by a
kustomization, because the kustomization's comment named it by filename. Correctly
refused: the distance between a filename in a comment and a line under `resources:` is
one careless edit. The comment now points at it in prose.
- **`infra/agent/probe-validation.sh` → 17 refusals, 0 failures.** A new, committed
negative-path probe for `pm-worker`'s input guards — the things that stand between an
injected environment variable and a clone from the wrong host or a push to the wrong ref.
Each case asserts a non-zero exit AND the specific message, so a guard that stopped firing
cannot pass by failing somewhere else. It refuses: an argument on the pinned command;
`main`, a wrong prefix, `feat/x;id`, `../../etc/passwd`, too-short slugs, and a branch equal
to the base; a redirected git host, a real-and-reachable wrong one (`github.com` — the
egress policy permits tcp/443, so this one is genuinely reachable), and the right host with
a smuggled path; a malformed owner or repository; an unknown mode; a non-numeric and a zero
issue number; and each missing credential. `build-image.sh` also runs it inside the built
image, where the interpreter is the one that will run in the cluster.
- **Writing that probe corrected an overclaim in the code.** The `PM_BRANCH != "main"` guard
is UNREACHABLE while the pattern check above it holds (`main` has no `feat|fix|docs|chore/`
prefix), so the comment claiming it as the defence against pushing to main was wrong about
which line does the work. It is kept — as an assertion that does not depend on the regex
being right — and the comment now says so.
- **Two more defects found by self-review of the unattended paths, both fixed:** the worker's
`EXIT` trap ended in a failing test, which under `set -e` can replace the script's real exit
code with that failure; and a provenance *log line* read a file through a redirect that
`set -o pipefail` would have turned into a dead worker if the file were absent.
- **A self-review of the trigger found and fixed one real defect** before any of this ran:
`last_activity()` compared RFC 3339 timestamp STRINGS to find the newest comment, which is
only correct while every offset Gitea returns is identical — and the failure mode is the
worst available one, the PM answering its own last comment in a loop. It now takes the last
element of the comments list, which is the order the endpoint returns, so there is nothing
about timezones to be right about.
- **`check-jsonschema --schemafile schemas/servicedefinition.schema.json catalog/gitea.yaml`
→ ok.**
- **The golden reproduces:** `python3 catalog/render.py --all catalog -o apps --wave-layout
--no-image-automation` leaves `apps/wave-0/gitea.yaml` byte-identical to what is committed,
and the only diff versus `main` is the new `gitea-allow-pm-agent-ingress`
CiliumNetworkPolicy.
- **Syntax:** `bash -n infra/agent/pm-worker` and `ast.parse` on `pm-dispatch.py` both clean.
`yaml.safe_load_all` parses every new manifest.
- **The spec repo has no git remote** — `git remote -v` in `../home-network-k3s` prints
nothing. That is *why* a worker's gate can never run `spec-immutability` in-cluster, and it
is measured rather than assumed. The worker's acceptance rule names that stage and
`gitea-settings` explicitly and refuses any other skip.
- **`PM_SELFTEST=1 python3 infra/agent/pm-dispatch.py` → exit 0, 26 assertions, all ok**
(run in the toolbox against the repository's own files, with `PM_WORKER_JOB_TEMPLATE` and
`PM_PERSONA_FILE` pointed at `infra/agent/`). It covers the branch vocabulary in both
directions (`feature/x`, `main`, `feat/AB`, `feat/x;rm -rf /` and `../../etc/passwd` all
refused), the five result-envelope shapes `extract_decision` has to survive plus three it
must reject, the committed Job template against its whole contract (pinned command, no
args, the powerless SA, no mounted token, the volume set, the security context, and every
env name inside the dispatcher's allowlist), and that the persona still says the four
things it must. `build-image.sh` runs the same selftest inside the image; that invocation
is the one that has not happened, because the image has not been built.
- **Claude Code facts were read from the current documentation, not from memory:**
`claude setup-token` mints a one-year subscription OAuth token consumed via
`CLAUDE_CODE_OAUTH_TOKEN`; `ANTHROPIC_API_KEY`/`ANTHROPIC_AUTH_TOKEN` rank **above** it in
credential precedence (hence the unsets and the env-name allowlist); `--bare` does **not**
read it (hence neither entrypoint uses that flag); `--permission-prompts none` requires
>= 2.1.259, which is why the image asserts a version floor. The pin `2.1.285` is the npm
registry's `stable` dist-tag, measured on 2026-10-05.
## What I did NOT verify
**Nothing in this change has run.** That is the headline, and everything below is a
consequence of it.
- **The image has never been built.** The nodejs module-stream install, the `npm i -g
@anthropic-ai/claude-code@2.1.285` layer, the version-floor assertion, the `COPY`
ownership assertions and `build-image.sh`'s whole in-image verification (including
`PM_SELFTEST=1 pm-dispatch`) are **unexercised**. Expect the build to need iteration.
- **It has never been pushed,** so the committed image digests are the deliberate all-zero
placeholder. Nothing has pulled this image.
- **No pod has started.** The dispatcher has never authenticated to Gitea, never polled,
never triaged an issue, never created a Job. No worker has cloned, branched, run a session,
run the gate in-cluster, pushed a branch, opened a pull request or commented on an issue.
Every claim about runtime behaviour is a design claim.
- **The admission policy has not met a live API server.** 28 offline assertions prove the
policy *refuses*; nothing proves the API server *enforces* it here. That is an L2 test and
it needs a cluster. Note also that `CLAUDE.md` §2b applies to me as much as to anyone: I
did not query whether Kyverno is healthy, whether `pm-agent-job-shape` reconciles, or
whether `platform-config` is Ready.
- **The network policies have not been exercised.** Whether `toEntities: kube-apiserver`
actually permits `kubectl` from this namespace on k3s, and whether `world:443` reaches
`api.anthropic.com` through this cluster's egress, are both unmeasured.
- **Neither credential exists.** The Gitea bot account has not been created, so its
permission has not been measured — and *that measurement is the control* that keeps the
agent from clearing its own sensitive-path review. Do it before the first real run.
- **Gitea's webhook `ALLOWED_HOST_LIST` was not queried.** It is cited as a reason webhooks
were rejected; it is not a measured fact about this Gitea.
- **The `write`-can-merge gap is real and unclosed.** No Gitea token scope separates "push a
branch and open a pull request" from "merge it". The service issues no merge call, which is
a property of the code, not of the credential.
- **The worker's credentials are inside the model session's reach.** Same container, same
uid; no isolation exists and none is claimed. Bounded by token scope and egress only.
- **This branch touches sensitive paths** — `clusters/**`, `platform/kyverno/**`,
`platform-config/kustomization.yaml`, `**/rbac.yaml`, `**/*.sops.yaml` — so it needs an
approving review from a non-author, or an administrator's `/approve-override` **posted by
the operator**. An agent must not clear this one on its own PR; that is precisely the
self-approval the stage exists to prevent.
- **No observability.** No metrics endpoint, no ServiceMonitor, no alert, and the namespace's
default-deny would drop a scrape from `monitoring` if one were added. Follow-ups are listed
in the plan, in the order they are worth doing.