fix(pm-agent): log the CLI's stdout envelope when triage fails; add PM_DECISION_MAX_TURNS #99

2026-10-31T17:07:39Z

gitea-admin commented

2026-10-31 17:07:39 +00:00

Why

Since PR #98 took pm-agent live, triage of #97 fails with [error] issue #97: claude exited 1: and nothing after the colon. It has happened twice, at 16:44 and 16:56 UTC.

The message is empty because ask_pm() logs only stderr. Under --output-format json, the CLI writes its failure as the result envelope on stdout (is_error, subtype, result) and leaves stderr empty.

What changed

describe_cli_failure(): on a non-zero exit, the error line now carries:
- the envelope's subtype, is_error, api_error_status, num_turns, duration_ms, total_cost_usd, stop_reason;
- the envelope's result or error text;
- the tails of stdout and stderr.
It says so explicitly when both streams are empty.
redact(): applied to that message and to extract_decision()'s error. It covers sk-ant-…, Bearer/token values, and user:pass@ in both http and https URLs.
PM_DECISION_MAX_TURNS: a new knob. It defaults to 1 (no behaviour change) and is bounded to 1–5. If the envelope says error_max_turns, the remedy becomes a ConfigMap change instead of another image build.
Selftest: four new assertions. They cover the cause on stdout with stderr empty, stderr only, total silence, and redaction.
Docs: a README knob row and the handoff file decisions/2026-10-31-pm-agent-triage-exit.md.

Evidence (full detail in the handoff)

Going live is not the cause. The 16:44 failure happened in dry run, on pod f7c9b7fcf, image df36d9b7, config-generation 3. ask_pm() has no dry-run branch.
The one success ever was 16:37:38 → 16:38:00, which took 22 s. It ran on pod 898fd8f5f with image c4eba5b3, against a thread that held only the issue body. Both failures took 51–54 s.
Ruled out:
- timeout (900 s limit, and a timeout would not exit with rc 1);
- budget (the PM call passes no --max-budget-usd);
- prompt size (~2.6 KB of thread);
- an unwritable HOME (the config dir was written at 16:56);
- egress (TLS to api.anthropic.com from the pod succeeds in 0.06 s);
- the model name and CLI version (unchanged since the success).
Root cause: UNVERIFIED. Two candidates remain open:
- a usage or rate limit on the shared subscription seat (the pod's CLI state has cachedExtraUsageDisabledReason: out_of_credits);
- error_max_turns under --max-turns 1 + --json-schema.

Gate

I ran ./ci/gate.sh in the pinned toolbox from the worktree (DEV_KUBECONFIG=/dev/null plus the ENGINE shim with the .git and spec-repo mounts). It returned rc 3, and the only stage that did not run was gitea-settings: it needs SETTINGS_API_TOKEN, which exists only in CI. Kyverno reported 87 passed and 0 failed. The offline PM_SELFTEST=1 printed selftest OK.

Follow-up required

This changes infra/agent/**, so CI re-publishes agent-runner. The dispatcher only picks up the fix after a second PR bumps the digest in platform/pm-agent/deployment.yaml and PM_WORKER_IMAGE in configmap.yaml, and bumps config-generation. Until then, retries run the old image and still log an empty message.

No path in this PR is approval-policy sensitive.

🤖 Generated with Claude Code

## Why Since PR #98 took pm-agent live, triage of #97 fails with `[error] issue #97: claude exited 1: ` and nothing after the colon. It has happened twice, at 16:44 and 16:56 UTC. The message is empty because `ask_pm()` logs only stderr. Under `--output-format json`, the CLI writes its failure as the result envelope on stdout (`is_error`, `subtype`, `result`) and leaves stderr empty. ## What changed - **`describe_cli_failure()`**: on a non-zero exit, the error line now carries: - the envelope's `subtype`, `is_error`, `api_error_status`, `num_turns`, `duration_ms`, `total_cost_usd`, `stop_reason`; - the envelope's `result` or `error` text; - the tails of stdout and stderr. It says so explicitly when both streams are empty. - **`redact()`**: applied to that message and to `extract_decision()`'s error. It covers `sk-ant-…`, `Bearer`/`token` values, and `user:pass@` in both http and https URLs. - **`PM_DECISION_MAX_TURNS`**: a new knob. It defaults to `1` (no behaviour change) and is bounded to 1–5. If the envelope says `error_max_turns`, the remedy becomes a ConfigMap change instead of another image build. - **Selftest**: four new assertions. They cover the cause on stdout with stderr empty, stderr only, total silence, and redaction. - **Docs**: a README knob row and the handoff file `decisions/2026-10-31-pm-agent-triage-exit.md`. ## Evidence (full detail in the handoff) - **Going live is not the cause.** The 16:44 failure happened in **dry run**, on pod `f7c9b7fcf`, image df36d9b7, config-generation 3. `ask_pm()` has no dry-run branch. - **The one success ever** was 16:37:38 → 16:38:00, which took 22 s. It ran on pod `898fd8f5f` with image **c4eba5b3**, against a thread that held only the issue body. Both failures took 51–54 s. - **Ruled out:** - timeout (900 s limit, and a timeout would not exit with rc 1); - budget (the PM call passes no `--max-budget-usd`); - prompt size (~2.6 KB of thread); - an unwritable HOME (the config dir was written at 16:56); - egress (TLS to api.anthropic.com from the pod succeeds in 0.06 s); - the model name and CLI version (unchanged since the success). - **Root cause: UNVERIFIED.** Two candidates remain open: - a usage or rate limit on the shared subscription seat (the pod's CLI state has `cachedExtraUsageDisabledReason: out_of_credits`); - `error_max_turns` under `--max-turns 1` + `--json-schema`. ## Gate I ran `./ci/gate.sh` in the pinned toolbox from the worktree (`DEV_KUBECONFIG=/dev/null` plus the ENGINE shim with the .git and spec-repo mounts). It returned **rc 3**, and the only stage that did not run was **gitea-settings**: it needs `SETTINGS_API_TOKEN`, which exists only in CI. Kyverno reported 87 passed and 0 failed. The offline `PM_SELFTEST=1` printed `selftest OK`. ## Follow-up required This changes `infra/agent/**`, so CI re-publishes agent-runner. The dispatcher only picks up the fix after a **second PR** bumps the digest in `platform/pm-agent/deployment.yaml` and `PM_WORKER_IMAGE` in `configmap.yaml`, and bumps `config-generation`. Until then, retries run the old image and still log an empty message. No path in this PR is approval-policy sensitive. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

gitea-admin added 1 commit 2026-10-31 17:07:39 +00:00

fix(pm-agent): log the CLI's stdout envelope when triage fails; add PM_DECISION_MAX_TURNS

gate / gate (pull_request) Successful in 2m56s

Details

images / images (pull_request) Successful in 13s

Details

gate / conformance (pull_request) Has been skipped

Details

664ee82b9a

`claude --print --output-format json` reports its own failures (API error, usage limit,
error_max_turns) as the result envelope on stdout and leaves stderr empty, so ask_pm()'s
stderr-only message logged "claude exited 1: " with nothing after it, twice on 2026-10-31.
describe_cli_failure() now logs the envelope's diagnostic fields plus stdout/stderr tails,
redacted (sk-ant-, Bearer/token, URL userinfo). PM_DECISION_MAX_TURNS (default 1, unchanged)
makes the max-turns remedy a ConfigMap change if the envelope turns out to say error_max_turns.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

pm-agent referenced this pull request

2026-10-31 17:12:40 +00:00

Massive amounts of historical data #97

jakman85 merged commit 466f63ec3c into main

2026-10-31 17:12:41 +00:00

jakman85 referenced this issue from a commit

2026-10-31 17:12:41 +00:00

Merge pull request 'fix(pm-agent): log the CLI's stdout envelope when triage fails; add PM_DECISION_MAX_TURNS' (#99) from fix/pm-agent-triage-exit into main

gitea-admin referenced this pull request

2026-10-31 17:20:19 +00:00

chore(pm-agent): pin agent-runner to the run-394 digest (466f63e, PR #99), bump config-generation #100

gitea-admin referenced this issue from a commit