Files
home-network-k3s-impl/decisions/2026-10-05-agents-work-in-their-own-workspace.md
T
jakman85andClaude Opus 5 2cb891c05d
gate / gate (pull_request) Successful in 1m48s
gate / conformance (pull_request) Has been skipped
docs(agents): one workspace per agent run — the shared checkout is off-limits
Agents have been working in the operator's checkout and clobbering each other.
Nothing in the tree told them not to: worktrees appeared in CLAUDE.md only as
§3's gate workarounds, and decisions/README.md described worktree isolation as
something agents already do rather than as a rule.

New CLAUDE.md §0 states the rule, with the commands and an explicit do-not list
for the shared checkout. Placed before §1 so every existing §-reference in the
repo keeps its meaning.

Measured in the shared checkout on 2026-10-05, each row a distinct failure mode:
107 `checkout: moving` reflog entries; sitting on feat/pm-agent-issue-to-pr, not
main; two other runs' untracked trees left in it; local main two commits behind
origin/main; 42 registered worktrees of which 31 are on already-merged branches;
9 gate scripts carrying a `.claude/` exclusion because an in-tree worktree made
find-based discovery count the same manifest up to 18 times.

Corrects a claim carried in earlier briefs: `git worktree remove` is NOT blocked
by the agent sandbox. Provoked on a throwaway worktree — rc 0.

§3 reframed as the normal path and grown from two breakages to three:
spec-immutability's skip is avoidable, not merely tolerated, with the spec repo
mounted read-only in the same ENGINE shim plus -e SPEC_REPO=/home-network-k3s.

CLAUDE.md is a sensitive path, so this PR needs a non-author review or an
administrator /approve-override posted as a PR review. Not posted by me: §1
forbids an agent clearing that on its own PR.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-10-05 22:31:40 -05:00

10 KiB
Raw Blame History

date, branch, title
date branch title
2026-10-05 docs/agents-work-in-their-own-workspace One workspace per agent run — the shared checkout is now documented as off-limits

What changed

Agents have been doing their work in the operator's checkout at /var/home/keith/source/home-network-k3s-impl and clobbering each other. Nothing in the tree told them not to: worktrees appeared in CLAUDE.md only as §3's gate workarounds, i.e. as an awkward edge case, and decisions/README.md asserted worktree isolation as a fact about how agents already behave rather than as a rule they must follow. Nowhere was the rule stated.

Documentation only — no code, no manifests, no pipeline logic.

  • CLAUDE.md: new ## 0. Work in your OWN workspace. Never in the shared checkout. Placed before §1 so the section numbers every other file and handoff entry cites (§1, §2b, §3, §7) all keep their meaning. It gives the three commands (git fetch origin main; git worktree add -b <type>/<slug> <path outside the repo> origin/main; cd), an explicit do-not list for the shared checkout (no checkout/switch, no edits, no add/commit/stash/restore, no removing others' worktrees), a measurement table, the "put it outside the repo root" rule, cleanup, what to do if you are already standing in the shared checkout with edits, and what a PM must put in a brief it writes for another agent.
  • CLAUDE.md intro now says "Start with §0", with the reason: it is the rule you can break before you have read anything else.
  • CLAUDE.md §3 retitled "Running the gate from a worktree" and reframed as the normal path rather than an edge case, since §0 now makes every gate run a worktree run. Its list grew from two breakages to three: spec-immutability's skip is avoidable, not merely tolerated, if the ENGINE shim also mounts the sibling spec repo read-only and passes -e SPEC_REPO=/home-network-k3s.
  • CLAUDE.md §4 gained "One branch, one workspace, one PR".
  • README.md: a fourth non-negotiable rule ("No work in a shared checkout"), and "Working on this repo" now opens with how to get out of the checkout before describing the dev container that runs in it.
  • decisions/README.md: the worktree-isolation sentence now cites §0 as the requirement instead of describing it as something agents happen to do.

I did this work the way the new §0 requires — a worktree at <session scratchpad>/wt-workspace-rule, outside the repo root, branched from origin/main, with the shared checkout left on the branch and with the untracked files I found in it.

What I verified

Every number in the new §0 table was measured in the shared checkout on 2026-10-05, not recalled:

Claim Command Value
shared HEAD is yanked between branches git reflog | grep -c 'checkout: moving' 107
it is not sitting on main git worktree list feat/pm-agent-issue-to-pr
other runs' uncommitted work is in it git status --porcelain 2 trees: infra/agent/ (22:04), platform/pm-agent/ (22:09) — stat -c %y
its main is stale git rev-parse main vs origin/main 4386d14 vs 22c94ef, 2 behind
worktrees registered / already merged git worktree list --porcelain + git merge-base --is-ancestor <b> main 42 branches, 31 merged
nested worktrees are a gate hazard grep -rl '\.claude' conformance/tests platform/monitoring/tests 9 scripts carry an explicit exclusion
the 18× double-count read platform/monitoring/tests/run.sh:115-123 in-tree comment, observed 2026-09-12

The failure mode was observed live, mid-session, not just reconstructed. The two untracked trees I measured at 22:04 and 22:09 were committed at 22:20 by another run — 5334abe feat(pm-agent): an in-cluster AI Program Manager..., 1,000+ lines across infra/agent/, clusters/, catalog/, apps/, docs/plans/ and its own decisions/ entry — onto the shared checkout's checked-out branch. Measured after the fact: the shared checkout's HEAD moved 22c94ef → 5334abe and its working tree went clean between my first git status and my last, with git log -1 naming a commit that is not mine. That is §0's "shared HEAD" row happening during the session that wrote §0, and it is now cited in the doc.

A claim of mine was wrong and is corrected in the doc. I had it recorded that git worktree remove is blocked by the agent sandbox. Provoked rather than assumed: created a throwaway detached worktree in the scratchpad and removed it — rc 0, and git worktree list no longer shows it. §0 therefore tells agents to remove their own, and scopes the "blocked" note to what it actually covers: other sessions' .claude/worktrees/agent-*, some of which are additionally locked on git's own terms. The rework/<pr-number> escape hatch is now scoped to someone else's stale worktree holding a branch, which is a real and separate problem (31 of them). Both of my own worktrees in this session were removed cleanly at rc 0 after pushing.

spec-immutability is avoidable-not-tolerated: taken from decisions/2026-10-01-render-selftest-runs-in-gate.md, which records stage spec-immutability: pass under the two-mount shim plus -e SPEC_REPO=/home-network-k3s.

Gate — rc 3, UNDETERMINED, and I am not calling it green. Pinned toolbox, from this worktree, DEV_KUBECONFIG=/dev/null and an ENGINE shim bind-mounting both the main checkout's .git (at its own host path) and the sibling spec repo read-only with -e SPEC_REPO=/home-network-k3s:

ENGINE=<two-mount shim> DEV_KUBECONFIG=/dev/null       ./infra/devcontainer/dev.sh run bash -c './ci/gate.sh'   # rc 3
stage result
spec-immutability pass — it genuinely RAN; the second mount is what makes it run from a worktree
approval-policy pass, but N/A: event='none'. It cannot see a PR locally, so this green says nothing. See below
decisions-log pass — the three required headings and the frozen-archive pin
lint · l0 · render-golden · l1 pass (L1: 3 offline checks, 0 failures, 0 skips, 0 unwired)
gitea-settings SKIPPED — SETTINGS_API_TOKEN unset; it exists only in CI
conformance (L2/L3) SKIPPED — CLUSTER_AVAILABLE!=true; the decision notes no cluster-needing requirement was touched

gitea-settings is the only stage that did not run, and it cannot be made to run from a workstation, so rc 3 is the ceiling for any local run — not a defect in this branch. The first gate attempt returned rc 3 for a different and real reason (HEAD is not ahead of origin/main: I had not committed yet, so the touched-requirement set was empty by default rather than by verification); it was re-run after committing, and the table above is the post-commit run. §3 now records both expected skips, because it previously named only spec-immutability.

What I did NOT verify

  • GATE: UNVERIFIED at the time of writing. The pinned-toolbox result for this branch and its exit code are reported in the PR, not predicted here. A local rc 3 is not green and will not be described as one.
  • CI gate: RED, measured — MERGE GATE: BLOCKED, one failing stage. Run 245 on PR #56, head b988da8b, 1m47s. The only failure is approval-policy: "a sensitive change lacks a non-author approval", sensitive paths touched: CLAUDE.md, author gitea-admin. Every other stage passed — decisions-log, lint, l0, render-golden, l1, and gitea-settings (which can only run in CI, and reported required_approvals = 0, dismiss_stale_approvals = true). spec-immutability SKIPPED in CI (no sibling spec repo in the CI workspace — the inverse of the local run, where the two-mount shim made it pass) and conformance SKIPPED. Note gate / conformance (pull_request) reports status success with description "Has been skipped" — CLAUDE.md §2b's trap, live on this very PR. Nothing in this branch is broken; the red is the structural human gate, below. Measured at b988da8b; this entry was then amended to record it, so the head SHA in the PR is newer than the run quoted here and CI re-ran on the amended commit.
  • This PR is a sensitive-path PR and is CI-RED on approval-policy until a human acts. CLAUDE.md is in infra/gitea/approval-policy.json's sensitive_paths ("the delivery contract agents follow"), so the stage requires an approving review from someone who is not the author, or an /approve-override <reason> posted by an administrator as a PR review of type "Comment", not in the plain comment box. ci/approval-policy.sh run locally reports N/A (it cannot see a PR), so the local gate cannot catch this and a green local run says nothing about it. I have not posted an override and must not — CLAUDE.md §1 forbids an agent clearing this on its own PR even though gitea-admin holds owner and could.
  • Nothing here is enforced. §0 is prose. There is no gate stage that detects work done in the shared checkout, no check that a worktree is outside the repo root, and no check that a branch was cut from origin/main. Every other rule in CLAUDE.md that agents kept breaking (DECISIONS-LOG.md appends, the three handoff headings) ended up as a gate stage, and that is the honest follow-up here. I did not write one: the ask was documentation, and an enforcement stage lands in ci/**, which is itself a sensitive path and deserves its own PR.
  • I did not clean up the 31 stale merged worktrees or the 35 nested under .claude/worktrees/. They are other sessions' workspaces; removing them is an operator action, and §0 explicitly tells agents not to remove worktrees they did not create.
  • I did not touch the untracked infra/agent/ and platform/pm-agent/ trees sitting in the shared checkout. They are another run's in-flight work. Whose, and whether they are wanted, is unknown to me.
  • docs/plans/ssd-etcd-execution-runbook.md:18 still points at a .claude/worktrees/ path inside the repo. Left as-is deliberately: that runbook is marked COMPLETED/historical and describes the now-decommissioned dev cluster, so the path is a record of what was done, not an instruction. It does mean a grep for in-repo worktree paths is not yet clean.
  • No runtime, cluster or Flux behaviour was touched or checked. This branch changes three Markdown files; kubectl was not run.