Compare commits

...
Author SHA1 Message Date
jakman85andClaude Opus 5.5 19097e4d4f chore(pm-agent): pin agent-runner to the run-394 digest (466f63e, PR #99), bump config-generation
gate / gate (pull_request) Successful in 2m57s
gate / conformance (pull_request) Has been skipped
Dispatcher image and PM_WORKER_IMAGE both move to
sha256:14a42ba8281ecd98005f5d99ecda4fb729104c783b01584097c32e104ced1bb2, the digest the
registry returns for agent-runner:sha-466f63ec3c3d and :main. config-generation 4 -> 5.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-31 12:17:00 -05:00
jakman85 466f63ec3c Merge pull request 'fix(pm-agent): log the CLI's stdout envelope when triage fails; add PM_DECISION_MAX_TURNS' (#99) from fix/pm-agent-triage-exit into main
gate / gate (push) Successful in 2m56s
images / images (push) Successful in 16s
gate / conformance (push) Has been skipped
Reviewed-on: #99
2026-10-31 17:12:40 +00:00
jakman85andClaude Opus 5.5 664ee82b9a fix(pm-agent): log the CLI's stdout envelope when triage fails; add PM_DECISION_MAX_TURNS
gate / gate (pull_request) Successful in 2m56s
images / images (pull_request) Successful in 13s
gate / conformance (pull_request) Has been skipped
`claude --print --output-format json` reports its own failures (API error, usage limit,
error_max_turns) as the result envelope on stdout and leaves stderr empty, so ask_pm()'s
stderr-only message logged "claude exited 1: " with nothing after it, twice on 2026-10-31.
describe_cli_failure() now logs the envelope's diagnostic fields plus stdout/stderr tails,
redacted (sk-ant-, Bearer/token, URL userinfo). PM_DECISION_MAX_TURNS (default 1, unchanged)
makes the max-turns remedy a ConfigMap change if the envelope turns out to say error_max_turns.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-31 12:04:09 -05:00
jakman85 95e08da2f0 Merge pull request 'chore(pm-agent): take the dispatcher out of dry run (operator-approved), config-generation 4' (#98) from chore/pm-agent-go-live into main
gate / gate (push) Successful in 2m58s
gate / conformance (push) Has been skipped
Reviewed-on: #98
Reviewed-by: Keith Jackson <[email protected]>
2026-10-31 16:52:16 +00:00
jakman85andClaude Opus 5.5 0d3e51acc4 chore(pm-agent): turn PM_DRY_RUN off (operator-approved), bump config-generation to 4
gate / gate (pull_request) Successful in 2m58s
gate / conformance (pull_request) Has been skipped
The run-383 worker image (df36d9b, pinned in #96) bakes imagePullSecrets: pm-agent-registry,
which admission requires, and the dispatcher's dry-run triage of #97 was read and approved.
PM_REQUIRE_LABEL=agent and the worker caps are unchanged. The dry-run status text in the
configmap comments, the README and the plan's follow-up 3 is updated to match.

Pre-flight (read-only, k3s-prod): pm-agent-job-shape Ready+Enforce; SA/Role/RoleBinding
present and can-i create jobs = yes; pm-agent-registry and pm-agent-credentials exist; both
egress CNPs Valid.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-31 11:45:52 -05:00
jakman85 11267409d7 Merge pull request 'chore(pm-agent): pin agent-runner to the run-383 digest (ddd04d4), bump config-generation' (#96) from chore/pm-agent-runner-digest into main
gate / gate (push) Successful in 2m58s
gate / conformance (push) Has been skipped
Reviewed-on: #96
Reviewed-by: Keith Jackson <[email protected]>
2026-10-31 16:34:49 +00:00
8 changed files with 299 additions and 22 deletions

No files matched your search

@@ -0,0 +1,41 @@
---
date: 2026-10-31
branch: chore/pm-agent-digest-99
title: Pin pm-agent dispatcher and worker to the agent-runner digest built from PR #99's merge
---
## What changed
- `platform/pm-agent/deployment.yaml`: the dispatcher `image:` moves from
`agent-runner@sha256:df36d9b7…` (images run 383) to
`agent-runner@sha256:14a42ba8281ecd98005f5d99ecda4fb729104c783b01584097c32e104ced1bb2`.
`config-generation` goes from "4" to "5" so the pod rolls.
- `platform/pm-agent/configmap.yaml`: `PM_WORKER_IMAGE` gets the same digest. A comment line
records where it came from. The `# renovate-digest:` marker still sits directly above both
values.
- No other values changed. `PM_DRY_RUN` stays "false". `PM_DECISION_MAX_TURNS` is not set, so
it keeps its in-code default of 1. No `*.sops.yaml` was touched.
## What I verified
- PR #99's merge commit is `466f63ec3c3df968a1dab25a8811dcd41961efde` (origin/main HEAD at
branch time).
- The Gitea Actions tasks API shows the `images` workflow (images.yaml) on that commit as
**run 394** (task 643), with event `push` and status `success`.
- I resolved the digest read-only from the registry by HEADing the manifests with OCI and
Docker Accept headers. Both `agent-runner:sha-466f63ec3c3d` and `agent-runner:main` returned
`200`, an `application/vnd.oci.image.manifest.v1+json` content type, and
`docker-content-digest: sha256:14a42ba8281ecd98005f5d99ecda4fb729104c783b01584097c32e104ced1bb2`.
The two are identical.
- Toolbox gate: the rc and skipped stages are in the PR description.
## What I did NOT verify
- That the dispatcher actually rolls to the new digest after merge, and that Flux applies it.
Flux's `lastAppliedRevision` and the pod's image were not checked. That is post-merge work.
- That the new image surfaces the triage failure's real cause. It will show only on the next
failing triage, in the dispatcher log.
- That admission (rule 7, `imagePullSecrets`) admits worker Jobs from this digest. The
template comes from `infra/agent/` at PR #99's merge, which still carries the
`imagePullSecrets` entry, but no Job was created to prove it.
- I did not run the image's own selftest against this digest.
+55
View File
@@ -0,0 +1,55 @@
---
date: 2026-10-31
branch: chore/pm-agent-go-live
title: Take pm-agent out of dry run (operator-approved); label scope and worker caps unchanged
---
## What changed
- `platform/pm-agent/configmap.yaml`: `PM_DRY_RUN` `"true"` -> `"false"`. The operator approved
this after reading the dry-run triage of issue #97. `PM_REQUIRE_LABEL: agent`,
`PM_MAX_ACTIVE_WORKERS: 2`, `PM_MAX_NEW_WORKERS_PER_CYCLE: 1`, the worker image digest and every
other knob are unchanged. The two comments that said dry run was the only mode that worked
(because the run-340 digest lacked `imagePullSecrets`) now describe the run-383 digest that
PR #96 pinned.
- `platform/pm-agent/deployment.yaml`: `config-generation` `"3"` -> `"4"`. Without the bump the
dispatcher keeps its startup config and stays in dry run.
- `platform/pm-agent/README.md` and `docs/plans/pm-agent-issue-to-pr.md` (follow-up 3): the
"currently dry run" status text now says dry run is off. `runbooks/pm-agent.md` is unchanged.
Its "stop it: set `PM_DRY_RUN: "true"`" step and its admission-refusal troubleshooting row
are still accurate.
## What I verified
Read-only, `kubectl --context k3s-prod` through the toolbox, 2026-10-31 ~11:45 local, with Flux
`pm-agent` and `platform-config` both at `main@1126740`:
- ClusterPolicy `pm-agent-job-shape`: `Ready=True`, `validationFailureAction=Enforce`,
`background=false`. It has seven rules, including `worker-pull-secret-must-be-the-committed-one`.
- RBAC: ServiceAccounts `pm-agent` and `pm-agent-worker`, Role `pm-agent` and RoleBinding
`pm-agent` (roleRef `pm-agent`, subject `ServiceAccount/pm-agent`) all exist. `auth can-i`
as `system:serviceaccount:pm-agent:pm-agent` returned `yes` for `create jobs.batch` and `no`
for `delete jobs.batch`.
- Secrets (key names only): `pm-agent-registry` is `kubernetes.io/dockerconfigjson` with key
`.dockerconfigjson`. `pm-agent-credentials` is `Opaque` with keys `claude-code-oauth-token`
and `gitea-token`.
- Egress: `default-deny` NetworkPolicy exists. CiliumNetworkPolicies `pm-agent-allow-egress`
and `pm-agent-worker-allow-egress` both report `Valid=True`.
- Deployment `pm-agent` is 1/1 on `agent-runner@sha256:df36d9b7…` with
`config-generation: "3"` live. No Jobs exist in the namespace.
- `platform/pm-agent/{configmap,deployment}.yaml`, README and docs do not match any
`sensitive_paths` glob in `infra/gitea/approval-policy.json`, so no review or override applies.
## What I did NOT verify
- **That a real worker Job is admitted and completes.** Nothing can show this until a labelled
issue gets a `start` decision after the merge. The L0 Kyverno suite shows the policy accepts
the committed template. Whether the API server admits the Job built from the df36d9b image's
baked template has not been tested live.
- That the worker pod can pull its image with `pm-agent-registry`, reach Gitea and the model
through its egress policy, and push a branch and open a PR. None of this has run before.
- The pod list also shows a stale `pm-agent-65db599649-v678g` (25d, `ContainerStatusUnknown`,
from an old ReplicaSet). It does not block anything, but I did not investigate it. Clearing
it is an operator action.
- CI: see the PR for the gate result. Locally the best possible result is rc 3, with only
`gitea-settings` not run.
@@ -0,0 +1,77 @@
---
date: 2026-10-31
branch: fix/pm-agent-triage-exit
title: pm-agent triage fails with "claude exited 1:" and no message; log the CLI's stdout envelope so the cause is visible
---
## What changed
- `infra/agent/pm-dispatch.py`, `ask_pm()`: on a non-zero exit the error now comes from a new
`describe_cli_failure()`, not from `proc.stderr` alone. With `--output-format json` the CLI
reports its own failures as the result envelope on **stdout** and writes nothing to stderr,
which is why both failures today logged `claude exited 1: ` with an empty message. The new
line carries the envelope's `subtype`, `is_error`, `api_error_status`, `num_turns`,
`duration_ms`, `total_cost_usd`, `stop_reason`, its `result`/`error` text, and the tails of
stdout and stderr. It names silence explicitly when both streams are empty.
- `redact()`: everything in that message, and in `extract_decision()`'s "no decision object"
error, passes through it. It covers `sk-ant-…`, `Bearer`/`token` values, and `user:pass@` in
both `http://` and `https://` URLs.
- `PM_DECISION_MAX_TURNS`: a new knob. It defaults to 1, today's value, so behaviour is
unchanged, and it is bounded to 1–5. If the newly visible envelope says `error_max_turns`,
the remedy is a ConfigMap change rather than another image build. Documented in
`platform/pm-agent/README.md`.
- Selftest: four new assertions. They cover an empty stderr with the cause on stdout, a
stderr-only failure, total silence, and redaction.
## What I verified
- **Timeline (Loki, `{namespace="pm-agent"}`, and `kubectl logs`):**
- **16:37:38 to 16:38:00**: pod `pm-agent-898fd8f5f-dp5hs`, image **c4eba5b3**,
config-generation 2, dry run. It triaged #97 in **22 s**, logged `PM says comment`, and
posted the 16:38:00Z comment. The thread then held only the issue body.
- **16:43:09 to 16:44:00**: pod `pm-agent-f7c9b7fcf-j8mzn`, image **df36d9b7**,
config-generation 3, **still dry run**. It logged `claude exited 1: ` with an empty
message after **51 s**.
- **16:55:42 to 16:56:36**: pod `pm-agent-7796db8bcb-p5dsm`, df36d9b7, config-generation
4, live. The same empty failure, after **54 s**.
- The only successful triage ever was the 16:38 one. Every earlier `898fd8f5f`/`5db55d558c`
log line is a startup crash (Gitea `GET /user` 401), not a triage.
- **Going live is NOT the cause.** The first failure happened in dry run. `ask_pm()` has no
dry-run branch: the prompt, flags, tools (`--tools ""`) and cwd are identical in both
modes. `PM_DRY_RUN` gates only `create_job()`, which runs after the model has answered.
- **Ruled out, with evidence:**
- **Timeout**: 51 to 54 s against a 900 s limit, and a timeout raises `TimeoutExpired`,
not rc 1.
- **Budget cap**: the PM call passes no `--max-budget-usd`.
- **Prompt size**: the thread is the issue body (293 chars) plus three comments
(1871 + 404 + 77 chars), and the persona is 8 KB.
- **Writable HOME**: HOME=`/workspace` (emptyDir), and `CLAUDE_CONFIG_DIR=/workspace/.claude`
was written at 16:56.
- **Egress**: a TLS 1.3 handshake to `api.anthropic.com:443` from the dispatcher pod
succeeds in 0.06 s.
- **Model name and CLI version**: the same `claude-opus-5` and CLI 2.1.285 succeeded at
16:38. Between c4eba5b and df36d9b, `infra/agent/` differs only by a template selftest
assertion and the toolbox base digest.
- **A hint, not proof:** `/workspace/.claude/.claude.json` in the live pod has
`cachedExtraUsageDisabledReason: 'out_of_credits'`. No token was read or printed.
- Offline selftest: `PM_SELFTEST=1` against the worktree's template and persona prints
`selftest OK`, including the 4 new assertions.
- Toolbox gate: see the PR description for the rc and the stages that were skipped.
## What I did NOT verify
- **The root cause. UNVERIFIED.** The CLI's own message was discarded, `--no-session-persistence`
leaves no transcript, and I did not run `claude` in the pod. The candidates still open:
- **Usage or rate limit on the subscription seat.** The seat is shared with the operator's
interactive use. The `out_of_credits` extra-usage flag fits this. So does a ~50 s run,
which matches the CLI exhausting its 429/529 retries.
- **`error_max_turns`.** With `--max-turns 1` and `--json-schema`, an answer that fails the
schema cannot be retried. The failing thread now invites `implement` (a branch plus a
long brief), where the 16:38 one produced a short `comment`. ~50 s is about two
generations of the 22 s success.
- **The 17:11 retry will NOT reveal the cause.** It runs the old image, which still discards
stdout. The cause becomes visible only after this PR merges, CI re-publishes agent-runner,
and a follow-up PR bumps the dispatcher digest (`deployment.yaml` and the `PM_WORKER_IMAGE`
in `configmap.yaml`, plus a `config-generation` bump).
- That the CLI's error envelope carries exactly these key names. The parser takes whichever
keys are present and falls back to raw tails.
+3 -1
View File
@@ -257,7 +257,9 @@ about runtime behaviour in this document is a design claim, not a measurement.
bot's permission is `write`, not `admin`** — that measurement is the control that keeps
the agent out of its own approval gate.
3. **First run with `PM_DRY_RUN=true` and `PM_REQUIRE_LABEL=agent`**, on one issue written
for the purpose. Read what the PM says before letting it write anything.
for the purpose. Read what the PM says before letting it write anything. *Done:* dry-run
triage of issue #97 read; `PM_DRY_RUN=false` on 2026-10-31
(`decisions/2026-10-31-pm-agent-go-live.md`), `PM_REQUIRE_LABEL=agent` kept.
4. **Tighten the admission image pin** from the `@sha256:*` wildcard to the literal digest.
One line, reviewed.
5. **Narrow `world:443` to `toFQDNs: api.anthropic.com`** — after verifying Cilium's DNS
+95 -4
View File
@@ -214,6 +214,12 @@ class Config:
# ConfigMap without rebuilding the image.
self.push_on_red = env("PM_PUSH_ON_RED_GATE", "false")
self.pm_timeout = env_int("PM_DECISION_TIMEOUT_SECONDS", 900)
# The PM holds no tools, so the only thing a second turn can be spent on is the CLI's
# structured-output step re-asking the model after an answer that failed the schema.
# 1 is the long-standing value; it is a knob so that, if the failure envelope now
# logged by describe_cli_failure() says `error_max_turns`, the remedy is a ConfigMap
# change rather than another image build.
self.pm_max_turns = env_int("PM_DECISION_MAX_TURNS", 1)
# Opt-in gate. Empty (the default) means every open issue is in scope, which is the
# behaviour the service was asked for. Setting it to a label name — `agent` is the
# conventional one — narrows the agent to issues a human has explicitly handed it,
@@ -231,6 +237,8 @@ class Config:
if self.max_active_workers < 1:
die("PM_MAX_ACTIVE_WORKERS must be >= 1")
if not 1 <= self.pm_max_turns <= 5:
die("PM_DECISION_MAX_TURNS must be between 1 and 5")
if "@sha256:" not in self.worker_image:
# The admission policy requires a digest too, so a tag here would produce a Job
# that is created and then DENIED — a failure visible only in the dispatcher's
@@ -437,7 +445,7 @@ def ask_pm(cfg: Config, thread: str) -> dict:
"--system-prompt-file", cfg.persona_file,
"--tools", "",
"--permission-prompts", "none",
"--max-turns", "1",
"--max-turns", str(cfg.pm_max_turns),
"--output-format", "json",
"--json-schema", json.dumps(DECISION_SCHEMA),
"--no-session-persistence",
@@ -445,10 +453,76 @@ def ask_pm(cfg: Config, thread: str) -> dict:
proc = subprocess.run(cmd, input=thread, capture_output=True, text=True,
timeout=cfg.pm_timeout)
if proc.returncode != 0:
raise RuntimeError(f"claude exited {proc.returncode}: {proc.stderr.strip()[:800]}")
raise RuntimeError(describe_cli_failure(proc.returncode, proc.stdout, proc.stderr))
return extract_decision(proc.stdout)
# Anything that could be a credential in the CLI's output. The CLI does not normally echo its
# token, but an API error body or a proxy's HTML page is not something to bet a leak on. Both
# schemes are matched (the http one is the lesson of a scheme-specific redaction that leaked
# ~/.git-credentials), and `token ...` is Gitea's own header scheme.
_SECRET_PATTERNS = [
(re.compile(r"sk-ant-[A-Za-z0-9_\-]+"), "sk-ant-[REDACTED]"),
(re.compile(r"(?i)\b(bearer|token)(\s+|=|:\s*)[A-Za-z0-9._~+/=\-]{8,}"), r"\1\2[REDACTED]"),
(re.compile(r"(?i)(https?://)[^/\s:@]+:[^/\s@]+@"), r"\1[REDACTED]@"),
]
def redact(text: str) -> str:
for pat, repl in _SECRET_PATTERNS:
text = pat.sub(repl, text)
return text
def describe_cli_failure(rc: int, stdout: str, stderr: str, limit: int = 800) -> str:
"""Say WHY `claude --print` failed, in the dispatcher log, redacted.
WHY THIS EXISTS: with `--output-format json` the CLI reports its own failures — an API
error, a usage limit, `error_max_turns`, a schema the model could not satisfy — as the
JSON result envelope on STDOUT (`is_error`, `subtype`, `result`), and writes nothing to
stderr. Logging only stderr therefore produced `claude exited 1: ` with an empty message
on 2026-10-31, twice, with the cause sitting unread in stdout. So: the envelope's
diagnostic fields first, then the raw tails of both streams.
"""
parts = [f"claude exited {rc}"]
out = (stdout or "").strip()
env = None
if out:
try:
env = json.loads(out)
except ValueError:
# A stream of JSON lines, or a log line ahead of the envelope: the last line that
# parses as an object is the result.
for line in reversed(out.splitlines()):
try:
cand = json.loads(line)
except ValueError:
continue
if isinstance(cand, dict):
env = cand
break
if isinstance(env, dict):
fields = []
for key in ("subtype", "is_error", "api_error_status", "num_turns", "duration_ms",
"duration_api_ms", "total_cost_usd", "stop_reason"):
if key in env:
fields.append(f"{key}={env[key]!r}")
if fields:
parts.append("envelope: " + " ".join(fields))
for key in ("result", "error", "errors", "message"):
if env.get(key):
val = env[key] if isinstance(env[key], str) else json.dumps(env[key])
parts.append(f"{key}: {val[:limit]}")
if out and not isinstance(env, dict):
parts.append(f"stdout tail: {out[-limit:]}")
err_text = (stderr or "").strip()
if err_text:
parts.append(f"stderr tail: {err_text[-limit:]}")
if len(parts) == 1:
parts.append("(no output on stdout or stderr)")
return redact(" | ".join(parts))
def extract_decision(stdout: str) -> dict:
"""Pull the decision object out of whatever `claude --print` wrapped it in.
@@ -501,7 +575,7 @@ def extract_decision(stdout: str) -> dict:
obj = as_obj(c)
if obj and isinstance(obj.get("action"), str):
return obj
raise RuntimeError(f"no decision object in claude output: {stdout[:400]!r}")
raise RuntimeError(redact(f"no decision object in claude output: {stdout[-800:]!r}"))
# --------------------------------------------------------------------------- thread
@@ -988,8 +1062,25 @@ def selftest() -> int:
except RuntimeError:
check(True, f"bad output refused: {label}")
# -- a failed CLI run must say why, and must not leak ---------------------
# The 2026-10-31 failure: rc 1, empty stderr, the cause in the stdout envelope.
env_err = json.dumps({"type": "result", "subtype": "error_max_turns", "is_error": True,
"num_turns": 2, "result": "API Error: 429 rate_limit_error"})
msg = describe_cli_failure(1, env_err, "")
check("error_max_turns" in msg and "429" in msg,
"CLI failure: stdout envelope surfaced when stderr is empty")
msg = describe_cli_failure(1, "", "boom on stderr")
check("boom on stderr" in msg, "CLI failure: stderr tail surfaced")
msg = describe_cli_failure(1, "", "")
check("no output" in msg, "CLI failure: silence is named, not an empty message")
msg = describe_cli_failure(1, "plain text sk-ant-oat01-abcdefABCDEF_123 here",
"Authorization: Bearer abcdefghijklmnop "
"http://u:[email protected]/x")
check("sk-ant-oat01" not in msg and "abcdefghijklmnop" not in msg and ":pw@" not in msg,
"CLI failure: token, bearer and URL credential redacted")
# -- the committed Job template -----------------------------------------
tmpl = os.environ.get("PM_WORKER_JOB_TEMPLATE", "/etc/pm-agent/worker-job-template.yaml")
tmpl =os.environ.get("PM_WORKER_JOB_TEMPLATE", "/etc/pm-agent/worker-job-template.yaml")
try:
selftest_template(tmpl)
check(True, f"worker Job template satisfies its contract ({tmpl})")
+12 -5
View File
@@ -46,11 +46,12 @@ waive the sensitive-path human review on its own pull request.
Both are deliberate, and both fail **visibly** rather than silently:
1. **The image is published by CI, not by a person.** `platform/pm-agent/deployment.yaml` and
`PM_WORKER_IMAGE` in `configmap.yaml` carry the digest of the **first** publish (images
run 340, `sha-523cd9118f95`), pinned once on 2026-10-30 so going live did not wait on
Renovate's first run; every later bump is Renovate's. That digest predates the worker
template's `imagePullSecrets` entry, so worker Jobs built from it are refused by
admission — which is why `PM_DRY_RUN` starts `true` (see `configmap.yaml`). `.gitea/workflows/images.yaml` builds `agent-runner` on rootless BuildKit
`PM_WORKER_IMAGE` in `configmap.yaml` first carried the run-340 publish
(`sha-523cd9118f95`), whose baked template predated the `imagePullSecrets` entry admission
requires, so the service ran with `PM_DRY_RUN=true`. PR #96 pinned the run-383 digest
(`df36d9b`, `sha-ddd04d460ef6`), whose template carries the entry, and dry run was turned
off on 2026-10-31 (`decisions/2026-10-31-pm-agent-go-live.md`); every later bump is
Renovate's. `.gitea/workflows/images.yaml` builds `agent-runner` on rootless BuildKit
(`platform/buildkit/`) — the same Containerfile, and the same `verify` stage, that
`infra/agent/build-image.sh` builds locally — and on a push to `main` publishes it to
`gitea.home.keithjackson.dev/home-network/agent-runner` under `sha-<commit>` and `main`.
@@ -249,6 +250,12 @@ five that matter:
| `PM_MAX_ACTIVE_WORKERS` | `2` | concurrent workers. RAM is the binding constraint (2Gi limit each) and the subscription seat's rate limits are shared with every other use of that seat |
| `PM_MAX_NEW_WORKERS_PER_CYCLE` | `1` | drains a backlog one issue at a time instead of in one stampede the first minute the credentials work |
| `PM_PUSH_ON_RED_GATE` | `false` | a change the gate refused is not pushed; the issue gets the gate log instead |
| `PM_DECISION_MAX_TURNS` | `1` (unset) | turns the triage call may take (1–5). The PM holds no tools, so a second turn is only ever the CLI re-asking after an answer that failed the decision schema. Raise it if a triage error logs `subtype='error_max_turns'` |
**When a triage fails**, the `[error] issue #N: claude exited …` line carries the CLI's result
envelope (`subtype`, `is_error`, `api_error_status`, `num_turns`, `result`) and the tails of
stdout and stderr, redacted. Before 2026-10-31 it logged stderr only, which is empty under
`--output-format json` — see `decisions/2026-10-31-pm-agent-triage-exit.md`.
Changing a knob requires bumping the `config-generation` annotation in `deployment.yaml` in
the same commit — the process reads its configuration once, at startup, so without the bump
+14 -10
View File
@@ -58,13 +58,16 @@ data:
# request that also moves the Deployment's image (same marker, same branch). Nobody pushes
# this image from a workstation.
#
# READ BEFORE TURNING PM_DRY_RUN OFF: this digest PREDATES the worker template's
# `imagePullSecrets` entry (PR #61). Its baked template has none, and admission now REQUIRES
# it — so a worker Job built from this image is DENIED. The merge that adds the entry
# changes infra/agent/**, CI re-publishes agent-runner, and Renovate's bump to that digest
# is what makes worker Jobs admissible. Until then, dry run is the only mode that works.
# The run-340 digest predated the worker template's `imagePullSecrets` entry (PR #61), which
# admission REQUIRES, so its worker Jobs were DENIED. The digest below (df36d9b, run 383,
# agent-runner:sha-ddd04d460ef6, pinned in PR #96) bakes a template that carries
# `imagePullSecrets: pm-agent-registry`, which is what made turning PM_DRY_RUN off possible
# (decisions/2026-10-31-pm-agent-runner-digest.md). A digest older than that would be DENIED.
# The digest below (14a42ba, images run 394, agent-runner:sha-466f63ec3c3d = PR #99's merge)
# adds the CLI failure envelope to triage errors and the PM_DECISION_MAX_TURNS knob
# (decisions/2026-10-31-pm-agent-digest-99.md).
# renovate-digest: image=gitea.home.keithjackson.dev/home-network/agent-runner tag=main
PM_WORKER_IMAGE: "gitea.home.keithjackson.dev/home-network/agent-runner@sha256:df36d9b7fef56d141220a387f64fb864cede8e5af30513ebfeb3ab566d3f771d"
PM_WORKER_IMAGE: "gitea.home.keithjackson.dev/home-network/agent-runner@sha256:14a42ba8281ecd98005f5d99ecda4fb729104c783b01584097c32e104ced1bb2"
# --- the model -------------------------------------------------------------
# The PM only reads a thread and answers with one JSON object, so it holds no tools and
@@ -120,10 +123,11 @@ data:
# no worker Job and so changes no code. The honest way to watch this service for a day
# before letting it write anything.
#
# "true" FOR THE FIRST RUN, and not only for caution: the pinned digest above cannot yet
# create an admissible worker Job (see the PM_WORKER_IMAGE comment). Flip to "false" in the
# same pull request as — or after — Renovate's first agent-runner digest bump.
PM_DRY_RUN: "true"
# "true" for the first run (2026-10-30 .. 2026-10-31); dry-run triage of issue #97 was read
# and the operator approved going live. "false" since decisions/2026-10-31-pm-agent-go-live.md.
# Setting it back to "true" (plus a config-generation bump) is the gentlest stop — see
# runbooks/pm-agent.md.
PM_DRY_RUN: "false"
# If the gate refuses a worker's change, the branch is NOT pushed by default and the issue
# gets the gate log instead. Set to "true" to push anyway and open a red pull request —
+2 -2
View File
@@ -48,7 +48,7 @@ spec:
# It is a hand-maintained marker rather than a computed hash because Flux's
# Kustomization here does not run a configmap generator; bump it in the same commit
# as a change to configmap.yaml.
pm-agent.homelab.keithjackson.dev/config-generation: "3"
pm-agent.homelab.keithjackson.dev/config-generation: "5"
spec:
serviceAccountName: pm-agent
# The kubelet's credential for pulling this image. Not the agent's Gitea credential —
@@ -78,7 +78,7 @@ spec:
# the registry's manifest endpoint on 2026-10-30 and pinned by hand once so the
# dispatcher does not wait on Renovate's first run; every later bump is Renovate's.
# renovate-digest: image=gitea.home.keithjackson.dev/home-network/agent-runner tag=main
image: gitea.home.keithjackson.dev/home-network/agent-runner@sha256:df36d9b7fef56d141220a387f64fb864cede8e5af30513ebfeb3ab566d3f771d
image: gitea.home.keithjackson.dev/home-network/agent-runner@sha256:14a42ba8281ecd98005f5d99ecda4fb729104c783b01584097c32e104ced1bb2
imagePullPolicy: IfNotPresent
command:
- /usr/local/bin/pm-dispatch