chore(pm-agent): turn PM_DRY_RUN off (operator-approved), bump config-generation to 4

The run-383 worker image (df36d9b, pinned in #96) bakes imagePullSecrets: pm-agent-registry, which admission requires, and the dispatcher's dry-run triage of #97 was read and approved. PM_REQUIRE_LABEL=agent and the worker caps are unchanged. The dry-run status text in the configmap comments, the README and the plan's follow-up 3 is updated to match. Pre-flight (read-only, k3s-prod): pm-agent-job-shape Ready+Enforce; SA/Role/RoleBinding present and can-i create jobs = yes; pm-agent-registry and pm-agent-credentials exist; both egress CNPs Valid. Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-31 11:45:52 -05:00
1 parent 11267409d7
commit 0d3e51acc4
@@ -0,0 +1,55 @@
+---
+date: 2026-10-31
+branch: chore/pm-agent-go-live
+title: Take pm-agent out of dry run (operator-approved); label scope and worker caps unchanged
+---
+
+## What changed
+
+- `platform/pm-agent/configmap.yaml`: `PM_DRY_RUN` `"true"` -> `"false"`. The operator approved
+  this after reading the dry-run triage of issue #97. `PM_REQUIRE_LABEL: agent`,
+  `PM_MAX_ACTIVE_WORKERS: 2`, `PM_MAX_NEW_WORKERS_PER_CYCLE: 1`, the worker image digest and every
+  other knob are unchanged. The two comments that said dry run was the only mode that worked
+  (because the run-340 digest lacked `imagePullSecrets`) now describe the run-383 digest that
+  PR #96 pinned.
+- `platform/pm-agent/deployment.yaml`: `config-generation` `"3"` -> `"4"`. Without the bump the
+  dispatcher keeps its startup config and stays in dry run.
+- `platform/pm-agent/README.md` and `docs/plans/pm-agent-issue-to-pr.md` (follow-up 3): the
+  "currently dry run" status text now says dry run is off. `runbooks/pm-agent.md` is unchanged.
+  Its "stop it: set `PM_DRY_RUN: "true"`" step and its admission-refusal troubleshooting row
+  are still accurate.
+
+## What I verified
+
+Read-only, `kubectl --context k3s-prod` through the toolbox, 2026-10-31 ~11:45 local, with Flux
+`pm-agent` and `platform-config` both at `main@1126740`:
+
+- ClusterPolicy `pm-agent-job-shape`: `Ready=True`, `validationFailureAction=Enforce`,
+  `background=false`. It has seven rules, including `worker-pull-secret-must-be-the-committed-one`.
+- RBAC: ServiceAccounts `pm-agent` and `pm-agent-worker`, Role `pm-agent` and RoleBinding
+  `pm-agent` (roleRef `pm-agent`, subject `ServiceAccount/pm-agent`) all exist. `auth can-i`
+  as `system:serviceaccount:pm-agent:pm-agent` returned `yes` for `create jobs.batch` and `no`
+  for `delete jobs.batch`.
+- Secrets (key names only): `pm-agent-registry` is `kubernetes.io/dockerconfigjson` with key
+  `.dockerconfigjson`. `pm-agent-credentials` is `Opaque` with keys `claude-code-oauth-token`
+  and `gitea-token`.
+- Egress: `default-deny` NetworkPolicy exists. CiliumNetworkPolicies `pm-agent-allow-egress`
+  and `pm-agent-worker-allow-egress` both report `Valid=True`.
+- Deployment `pm-agent` is 1/1 on `agent-runner@sha256:df36d9b7…` with
+  `config-generation: "3"` live. No Jobs exist in the namespace.
+- `platform/pm-agent/{configmap,deployment}.yaml`, README and docs do not match any
+  `sensitive_paths` glob in `infra/gitea/approval-policy.json`, so no review or override applies.
+
+## What I did NOT verify
+
+- **That a real worker Job is admitted and completes.** Nothing can show this until a labelled
+  issue gets a `start` decision after the merge. The L0 Kyverno suite shows the policy accepts
+  the committed template. Whether the API server admits the Job built from the df36d9b image's
+  baked template has not been tested live.
+- That the worker pod can pull its image with `pm-agent-registry`, reach Gitea and the model
+  through its egress policy, and push a branch and open a PR. None of this has run before.
+- The pod list also shows a stale `pm-agent-65db599649-v678g` (25d, `ContainerStatusUnknown`,
+  from an old ReplicaSet). It does not block anything, but I did not investigate it. Clearing
+  it is an operator action.
+- CI: see the PR for the gate result. Locally the best possible result is rc 3, with only
+  `gitea-settings` not run.
@@ -257,7 +257,9 @@ about runtime behaviour in this document is a design claim, not a measurement.
   bot's permission is `write`, not `admin`** — that measurement is the control that keeps
   the agent out of its own approval gate.
 3. **First run with `PM_DRY_RUN=true` and `PM_REQUIRE_LABEL=agent`**, on one issue written
-   for the purpose. Read what the PM says before letting it write anything.
+   for the purpose. Read what the PM says before letting it write anything. *Done:* dry-run
+   triage of issue #97 read; `PM_DRY_RUN=false` on 2026-10-31
+   (`decisions/2026-10-31-pm-agent-go-live.md`), `PM_REQUIRE_LABEL=agent` kept.
 4. **Tighten the admission image pin** from the `@sha256:*` wildcard to the literal digest.
   One line, reviewed.
 5. **Narrow `world:443` to `toFQDNs: api.anthropic.com`** — after verifying Cilium's DNS
@@ -46,11 +46,12 @@ waive the sensitive-path human review on its own pull request.
 Both are deliberate, and both fail **visibly** rather than silently:

 1. **The image is published by CI, not by a person.** `platform/pm-agent/deployment.yaml` and
-   `PM_WORKER_IMAGE` in `configmap.yaml` carry the digest of the **first** publish (images
-   run 340, `sha-523cd9118f95`), pinned once on 2026-10-30 so going live did not wait on
-   Renovate's first run; every later bump is Renovate's. That digest predates the worker
-   template's `imagePullSecrets` entry, so worker Jobs built from it are refused by
-   admission — which is why `PM_DRY_RUN` starts `true` (see `configmap.yaml`). `.gitea/workflows/images.yaml` builds `agent-runner` on rootless BuildKit
+   `PM_WORKER_IMAGE` in `configmap.yaml` first carried the run-340 publish
+   (`sha-523cd9118f95`), whose baked template predated the `imagePullSecrets` entry admission
+   requires, so the service ran with `PM_DRY_RUN=true`. PR #96 pinned the run-383 digest
+   (`df36d9b`, `sha-ddd04d460ef6`), whose template carries the entry, and dry run was turned
+   off on 2026-10-31 (`decisions/2026-10-31-pm-agent-go-live.md`); every later bump is
+   Renovate's. `.gitea/workflows/images.yaml` builds `agent-runner` on rootless BuildKit
   (`platform/buildkit/`) — the same Containerfile, and the same `verify` stage, that
   `infra/agent/build-image.sh` builds locally — and on a push to `main` publishes it to
   `gitea.home.keithjackson.dev/home-network/agent-runner` under `sha-<commit>` and `main`.
@@ -58,11 +58,11 @@ data:
  # request that also moves the Deployment's image (same marker, same branch). Nobody pushes
  # this image from a workstation.
  #
-  # READ BEFORE TURNING PM_DRY_RUN OFF: this digest PREDATES the worker template's
-  # `imagePullSecrets` entry (PR #61). Its baked template has none, and admission now REQUIRES
-  # it — so a worker Job built from this image is DENIED. The merge that adds the entry
-  # changes infra/agent/**, CI re-publishes agent-runner, and Renovate's bump to that digest
-  # is what makes worker Jobs admissible. Until then, dry run is the only mode that works.
+  # The run-340 digest predated the worker template's `imagePullSecrets` entry (PR #61), which
+  # admission REQUIRES, so its worker Jobs were DENIED. The digest below (df36d9b, run 383,
+  # agent-runner:sha-ddd04d460ef6, pinned in PR #96) bakes a template that carries
+  # `imagePullSecrets: pm-agent-registry`, which is what made turning PM_DRY_RUN off possible
+  # (decisions/2026-10-31-pm-agent-runner-digest.md). A digest older than that would be DENIED.
  # renovate-digest: image=gitea.home.keithjackson.dev/home-network/agent-runner tag=main
  PM_WORKER_IMAGE: "gitea.home.keithjackson.dev/home-network/agent-runner@sha256:df36d9b7fef56d141220a387f64fb864cede8e5af30513ebfeb3ab566d3f771d"

@@ -120,10 +120,11 @@ data:
  # no worker Job and so changes no code. The honest way to watch this service for a day
  # before letting it write anything.
  #
-  # "true" FOR THE FIRST RUN, and not only for caution: the pinned digest above cannot yet
-  # create an admissible worker Job (see the PM_WORKER_IMAGE comment). Flip to "false" in the
-  # same pull request as — or after — Renovate's first agent-runner digest bump.
-  PM_DRY_RUN: "true"
+  # "true" for the first run (2026-10-30 .. 2026-10-31); dry-run triage of issue #97 was read
+  # and the operator approved going live. "false" since decisions/2026-10-31-pm-agent-go-live.md.
+  # Setting it back to "true" (plus a config-generation bump) is the gentlest stop — see
+  # runbooks/pm-agent.md.
+  PM_DRY_RUN: "false"

  # If the gate refuses a worker's change, the branch is NOT pushed by default and the issue
  # gets the gate log instead. Set to "true" to push anyway and open a red pull request —
@@ -48,7 +48,7 @@ spec:
        # It is a hand-maintained marker rather than a computed hash because Flux's
        # Kustomization here does not run a configmap generator; bump it in the same commit
        # as a change to configmap.yaml.
-        pm-agent.homelab.keithjackson.dev/config-generation: "3"
+        pm-agent.homelab.keithjackson.dev/config-generation: "4"
    spec:
      serviceAccountName: pm-agent
      # The kubelet's credential for pulling this image. Not the agent's Gitea credential —