From 72a46ad82d39362743af8d54cd626e6f7194dc63 Mon Sep 17 00:00:00 2001 From: Keith Jackson Date: Tue, 29 Sep 2026 22:35:55 -0500 Subject: [PATCH] chore(log): the handoff log was serialising every parallel branch MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit DECISIONS-LOG.md was append-only in ONE file and every agent appended to its tail, so any two concurrent branches collided on the same last hunk. Measured against the live Gitea today: four of the five open PRs were mergeable=false, and `git merge-tree` named DECISIONS-LOG.md as the ONLY conflicting path in all four. #22 fix/crafty-oauth2-proxy-signin CONFLICT: DECISIONS-LOG.md #20 feat/conformance-job-template CONFLICT: DECISIONS-LOG.md #19 feat/conformance-trigger-kubeconfig CONFLICT: DECISIONS-LOG.md #18 feat/conformance-runner-image CONFLICT: DECISIONS-LOG.md #17 feat/ci-is-the-approver clean — it wrote no handoff Nothing was wrong with any of those changes. The log's storage format serialised work that was otherwise parallel, and the cost grew with every concurrent branch (N branches -> N-1 rebases, each invalidating the gate run before it). The agents work in isolated worktrees precisely so they do not block each other; this one file undid that. So: stop sharing a file. Handoffs are now one-per-file under decisions/, named YYYY-MM-DD-.md. Disjoint paths have nothing to merge. There is deliberately no index file (a shared mutable file is the thing that just failed) and no counter (two agents would race for the same number) — the directory listing is the index. DECISIONS-LOG.md is frozen as the pre-2026-09-29 archive, its SHA-256 pinned in decisions/ARCHIVE.sha256, and a new gate stage rejects any change to it. That is a guard and not a paragraph because the old habit is written into the runbooks and into every agent that has read the log, and getting it wrong does not fail: it produces a clean-looking commit that re-creates the conflict on OTHER branches, later, misattributed. ci/decisions-log.sh new stage: archive pinned; every entry named with a real date, carrying date/branch/title frontmatter that agrees with its filename, and all three of CLAUDE.md §7's sections with a real body under each. Exit 0 or 1 only — no skip path by construction, so no fail-open surface on a guard that closes one. ci/gate.sh runs it; fail and (impossible) skip both BLOCK. No REQUIRE_DECISIONS_LOG knob: a guard with an off switch is not a guard. ci/tests/decisions-log.test.sh 22 assertions, incl. the one that matters — an append to the archive is rejected, and the message names the file to write instead. ci/tests/gate-decision.test.sh +3 assertions for the new wiring, AND a pre-existing bug: the stub list never included gitea-settings, so run_stage saw exit 127 and recorded a FAILED stage in every case. 15 of 47 assertions failed on main for that reason alone — the suite was measuring its own missing stub. Now 50/50. CLAUDE.md §7 rewritten; §2b widened to the new path. ci/Makefile, ci/README.md, .gitea/workflows/README.md — stage named where the other stages are named. Co-Authored-By: Claude Opus 5 (1M context) --- .gitea/workflows/README.md | 2 +- CLAUDE.md | 42 +++- DECISIONS-LOG.md | 13 ++ ci/Makefile | 5 +- ci/README.md | 1 + ci/decisions-log.sh | 195 ++++++++++++++++++ ci/gate.sh | 17 ++ ci/tests/decisions-log.test.sh | 130 ++++++++++++ ci/tests/gate-decision.test.sh | 23 ++- .../2026-09-29-decisions-log-conflict-free.md | 82 ++++++++ decisions/ARCHIVE.sha256 | 1 + decisions/README.md | 101 +++++++++ 12 files changed, 600 insertions(+), 12 deletions(-) create mode 100755 ci/decisions-log.sh create mode 100755 ci/tests/decisions-log.test.sh create mode 100644 decisions/2026-09-29-decisions-log-conflict-free.md create mode 100644 decisions/ARCHIVE.sha256 create mode 100644 decisions/README.md diff --git a/.gitea/workflows/README.md b/.gitea/workflows/README.md index 6db23cc..952e6ce 100644 --- a/.gitea/workflows/README.md +++ b/.gitea/workflows/README.md @@ -15,7 +15,7 @@ it touches is green** (R-TEST-03). ## The pipeline ``` - lint ─▶ l0 (schema + Kyverno) ─▶ render-golden ─▶ l1 (rendering) ─▶ conformance (L2/L3) ─▶ merge-gate decision + decisions-log ─▶ lint ─▶ l0 (schema + Kyverno) ─▶ render-golden ─▶ l1 (rendering) ─▶ conformance (L2/L3) ─▶ merge-gate decision └──────────────── run in CI, NO cluster ────────────────┘ └── in-cluster Chainsaw Job ──┘ live now cluster-gated (SKIPS in Phase 0) ``` diff --git a/CLAUDE.md b/CLAUDE.md index e9b7e83..8df5761 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,7 +1,7 @@ # CLAUDE.md — how work lands in this repo Read this before doing anything durable. It is the delivery contract for AI coding -sessions and agents. `DECISIONS-LOG.md`'s last entry is the current PM handoff and +sessions and agents. The newest entry in `decisions/` is the current PM handoff and takes precedence on *what* to work on; this file governs *how* it lands. ## 1. Never push to `main`. Ever. @@ -48,9 +48,10 @@ reports as `success`, so a green tick is not by itself evidence that anything ra ## 2b. Never infer platform state from this repository -**This repo is not evidence about the live platform.** Neither is `DECISIONS-LOG.md`. -Both describe what someone believed at the time of writing, and both have been -confidently wrong for a month at a stretch. +**This repo is not evidence about the live platform.** Neither is the handoff log +(`decisions/`, and the frozen `DECISIONS-LOG.md` archive). Both describe what someone +believed at the time of writing, and both have been confidently wrong for a month at a +stretch. On 2026-09-29 the log stated the merge gate was a required status check with a runner online. The API said `branch_protections: []` and **zero** runners. Everything downstream @@ -139,9 +140,32 @@ invocations concurrently. flow. Do not design around committing one — read the expiry from the issuing page and treat a burnt token as free to retry. -## 7. Before you finish +## 7. Before you finish — write a handoff entry, in its OWN file -Append a handoff entry to `DECISIONS-LOG.md` describing what you changed, what you -verified, and what you did **not** verify. State gaps plainly — an honest -"UNVERIFIED" is worth more than an optimistic "green", and this repo has already been -burned by the difference. +Add **one new file**: + + decisions/YYYY-MM-DD-.md = your branch's last segment + +with frontmatter (`date`, `branch`, `title`) and exactly these three sections: + + ## What changed + ## What I verified + ## What I did NOT verify + +State gaps plainly — an honest "UNVERIFIED" is worth more than an optimistic "green", +and this repo has already been burned by the difference. All three headings are +**enforced** by `ci/decisions-log.sh`, because the third is the one that gets dropped. +Template and full rationale: `decisions/README.md`. + +**Do NOT append to `DECISIONS-LOG.md`.** It is a frozen archive (pre-2026-09-29), its +SHA-256 is pinned in `decisions/ARCHIVE.sha256`, and the merge gate rejects any change +to it. Appending to its tail is what every agent used to do, and because every branch +appended to the *same last hunk*, no two concurrent branches could auto-merge. Measured +on 2026-09-29: **four of the five open PRs were `mergeable: false`, with +`DECISIONS-LOG.md` the only conflicting path in all four** — none of those changes had +anything to do with each other. Separate files have nothing to merge, which is the +entire fix. Note that the damage from getting this wrong lands on *other people's* +branches, later, which is why it is a gate stage and not a request. + +Many older handoff entries and runbooks say "append to `DECISIONS-LOG.md`". That +wording is **stale**, in the same way §1's `git push origin main` wording is stale. diff --git a/DECISIONS-LOG.md b/DECISIONS-LOG.md index f04a856..4ac6b44 100644 --- a/DECISIONS-LOG.md +++ b/DECISIONS-LOG.md @@ -1,3 +1,16 @@ +> **FROZEN ARCHIVE — 2026-08-16 … 2026-09-29. Do not append to this file.** +> +> New handoff entries go in **`decisions/`**, one file per entry +> (`decisions/YYYY-MM-DD-.md`). See [`decisions/README.md`](decisions/README.md). +> +> Why: every agent appended to this file's tail, so any two concurrent branches +> conflicted on the same last hunk. On 2026-09-29 that made four of the five open PRs +> unmergeable, with this file as the *only* conflicting path in all four. Separate files +> have nothing to merge. +> +> This file's SHA-256 is pinned in `decisions/ARCHIVE.sha256` and the merge gate +> (`ci/decisions-log.sh`) rejects any change to it. + # Autonomous Session Decision Log (PM) Program Manager working independently while the operator is AFK (started 2026-08-16). Every diff --git a/ci/Makefile b/ci/Makefile index 5706807..2e74656 100644 --- a/ci/Makefile +++ b/ci/Makefile @@ -12,7 +12,7 @@ CI_DIR := $(dir $(abspath $(lastword $(MAKEFILE_LIST)))) .DEFAULT_GOAL := help -.PHONY: help spec-immutability lint schema l0 render-golden l1 conformance gate all touched versions tools test +.PHONY: help spec-immutability decisions-log lint schema l0 render-golden l1 conformance gate all touched versions tools test help: ## Show this help @awk 'BEGIN{FS=":.*##"} /^[a-zA-Z0-9_-]+:.*##/{printf " \033[36m%-14s\033[0m %s\n",$$1,$$2}' $(MAKEFILE_LIST) @@ -29,6 +29,9 @@ schema l0: ## ServiceDefinition schema validation + Kyverno L0 policy gate (no c render-golden: ## render==golden: canonical render reproduces committed apps/ (no cluster; live now) @"$(CI_DIR)render-golden.sh" +decisions-log: ## handoff log: DECISIONS-LOG.md frozen, decisions/ entries well-formed (no cluster) + @"$(CI_DIR)decisions-log.sh" + l1: ## L1 rendering checks (no cluster; live now) @"$(CI_DIR)l1.sh" diff --git a/ci/README.md b/ci/README.md index 47357e3..4d13e23 100644 --- a/ci/README.md +++ b/ci/README.md @@ -101,6 +101,7 @@ caller that must distinguish them invokes `./ci/gate.sh` directly. |---|---|---| | `lint`, `l0`, `render-golden`, `l1` | a **pinned tool is absent** or a **wired seam went unwired** — it *could not run* | **BLOCKED** under strict skips (a CI runner: the toolchain is installed for us by `.gitea/workflows/scripts/install-tools.sh`, so a skip is a defect); **UNDETERMINED** otherwise (a workstation missing a tool is a real local condition — still non-zero, still not a pass; the remedy is `infra/devcontainer/dev.sh`, which carries the pinned toolchain) | | `spec-immutability` (R-TEST-06) | the sibling spec repo is not checked out, so the spec **could not be verified** | **UNDETERMINED** — unverified is unverified. Deliberately *not* upgraded to BLOCKED by strict skips: the live runner does not check the spec repo out at all, so blocking would leave the only enforced status check permanently red for a reason no PR could clear. It is therefore the **one** skip whose non-answer a caller may knowingly tolerate, and `.gitea/workflows/gate.yml` does so in one visible line. `REQUIRE_SPEC_VERIFICATION=true` makes it a hard block once that checkout is wired. Locally: check the spec repo out beside this one, or set `SPEC_REPO` (`dev.sh` mounts a sibling automatically). | +| `decisions-log` | **nothing** — this stage has no skip path. It reads only files in the tree under test and needs no tool and no cluster, so there is no environment in which it "could not run" | a reported skip is a **defect in the stage** and BLOCKS. The stage fails when the frozen `DECISIONS-LOG.md` is modified (which silently re-creates, for every *other* open branch, the tail-hunk conflict that made four of five PRs unmergeable on 2026-09-29) or when a `decisions/` entry is malformed. There is deliberately **no** `REQUIRE_DECISIONS_LOG` knob: a guard with an off switch is not a guard. See `decisions/README.md`. | | `conformance` (L2/L3) | `CLUSTER_AVAILABLE!=true` — the environment **declared** the capability absent (documented Phase-0 behaviour) | **does not deny a PASS**: a declared-inapplicable stage was never *supposed* to run. It is NAMED in the PASS headline and step summary, the touched requirements it would have covered are reported UNVERIFIED, and `REQUIRE_CLUSTER_TESTS=true` (Phase 1+) turns that into a block. A skip **despite** `CLUSTER_AVAILABLE=true` is a defect and BLOCKS. | That asymmetry is the whole rule: **declared-not-applicable may pass; could-not-run may not.** diff --git a/ci/decisions-log.sh b/ci/decisions-log.sh new file mode 100755 index 0000000..f7e6f61 --- /dev/null +++ b/ci/decisions-log.sh @@ -0,0 +1,195 @@ +#!/usr/bin/env bash +# ci/decisions-log.sh — the handoff log is a DIRECTORY, and the old single file is frozen. +# +# WHY THIS STAGE EXISTS +# --------------------- +# CLAUDE.md §7 tells every agent to record a handoff before finishing. Until 2026-09-29 that +# meant appending to the tail of one 5,228-line file, DECISIONS-LOG.md. Every branch appended +# to the SAME last hunk, so git could not auto-merge any two of them. Measured that day against +# the live Gitea: four of the five open PRs were `mergeable: false`, and `git merge-tree` +# against `main` named DECISIONS-LOG.md as the ONLY conflicting path in all four: +# +# #22 fix/crafty-oauth2-proxy-signin CONFLICT (content): DECISIONS-LOG.md +# #20 feat/conformance-job-template CONFLICT (content): DECISIONS-LOG.md +# #19 feat/conformance-trigger-kubeconfig CONFLICT (content): DECISIONS-LOG.md +# #18 feat/conformance-runner-image CONFLICT (content): DECISIONS-LOG.md +# #17 feat/ci-is-the-approver clean — it wrote no handoff entry +# +# Nothing was wrong with any of those changes. The log's storage format serialised work that +# was otherwise parallel, and the cost grew with every concurrent branch (N branches -> N-1 +# rebases, each invalidating the gate run before it). +# +# The fix is that handoffs now live one-per-file under decisions/, so concurrent branches touch +# disjoint paths and there is nothing to merge. See decisions/README.md. +# +# WHY IT NEEDS A CHECK AND NOT JUST A DOCUMENT +# -------------------------------------------- +# The old habit is written into this repo in many places — the runbooks, the muscle memory of +# every agent that has read the log, and (until this change) CLAUDE.md itself. An agent that +# appends to DECISIONS-LOG.md anyway does not fail: it produces a perfectly good-looking commit +# that quietly re-creates the conflict for every other open branch, and the damage is only +# visible later, to somebody else, in a PR that is not theirs. That is the exact shape this +# repo keeps getting burned by — a silent regression with a delayed, misattributed symptom — so +# it gets a guard rather than a paragraph. +# +# WHAT IS ASSERTED +# (1) DECISIONS-LOG.md is byte-identical to the SHA-256 pinned in decisions/ARCHIVE.sha256. +# Re-pinning is a deliberate, reviewable line in a diff; appending is not. +# (2) Every decisions/ entry is named YYYY-MM-DD-.md with a real calendar date, so the +# directory sorts chronologically and two agents cannot collide on a filename. +# (3) Every entry carries frontmatter (date, branch, title) whose `date` agrees with its +# filename — a disagreement means a copied template nobody edited. +# (4) Every entry has all three of CLAUDE.md §7's sections with a real body under each: +# "What changed", "What I verified", "What I did NOT verify". This is the quality floor +# that stops an entry degrading into "done ✅". The third heading is the load-bearing one: +# this repo's recurring failure is a confident report of something nobody measured, and an +# entry with nowhere to write "UNVERIFIED" does not get one written. +# +# EXIT CODES (ci/README.md contract): 0 pass, 1 BLOCKS the merge. This stage has NO skip path — +# it reads only files that are in the tree it is run against, needs no tool beyond coreutils and +# no cluster, so there is no environment in which it "could not run". Adding an rc 3 here would +# add a fail-open surface to a guard whose whole job is to close one. +set -uo pipefail +CI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +# shellcheck disable=SC1091 +source "${CI_DIR}/lib.sh" +set +e # aggregate every problem in one run; do not stop at the first. + +cd "$REPO_ROOT" + +ARCHIVE="DECISIONS-LOG.md" +PIN="decisions/ARCHIVE.sha256" +rc=0 + +# ---- (1) the archive is frozen ---------------------------------------------- +log "Stage: decisions log (archive frozen + entries well-formed)" + +if [ ! -f "$ARCHIVE" ]; then + err "${ARCHIVE} is missing. It is the frozen pre-2026-09-29 history and must stay in the tree." + summary "- decisions-log: **FAILED** (\`${ARCHIVE}\` missing)" + rc=1 +elif [ ! -f "$PIN" ]; then + err "${PIN} is missing, so the archive's integrity cannot be established." + err " If you are re-pinning deliberately: sha256sum ${ARCHIVE} > ${PIN}" + summary "- decisions-log: **FAILED** (\`${PIN}\` missing — archive unverifiable)" + rc=1 +else + want="$(awk 'NR==1{print $1}' "$PIN")" + got="$(sha256sum "$ARCHIVE" | awk '{print $1}')" + if [ "$want" != "$got" ]; then + err "${ARCHIVE} HAS CHANGED. It is a FROZEN ARCHIVE — do not append to it." + err " pinned: ${want}" + err " on disk: ${got}" + err "" + err " Every branch that appends to this file's tail conflicts with every other branch that" + err " does, on the same hunk. That is what made four of five open PRs unmergeable on" + err " 2026-09-29, and your commit would re-create it for branches that are not yours." + err "" + err " Write your handoff as its OWN FILE instead:" + err " decisions/\$(date +%F)-.md (see decisions/README.md)" + err " and restore the archive: git checkout -- ${ARCHIVE}" + err "" + err " If the archive genuinely must change (redacting a leaked credential, say), that is a" + err " reviewed act: re-pin it with 'sha256sum ${ARCHIVE} > ${PIN}' and say why in your entry." + summary "- decisions-log: **FAILED — the frozen archive \`${ARCHIVE}\` was modified** (append to \`decisions/\` instead)" + rc=1 + else + ok "${ARCHIVE}: unchanged (sha256 matches ${PIN})" + fi +fi + +# ---- (2)(3)(4) every entry is well-formed ------------------------------------ +shopt -s nullglob +entries=( decisions/[0-9]*.md ) +shopt -u nullglob + +# Zero entries is legitimate only until the first handoff lands; after that the glob always +# matches. It is never an error here — a branch that changes no behaviour owes no handoff, and +# CLAUDE.md, not this stage, is what asks for one. +if [ "${#entries[@]}" -eq 0 ]; then + log "no decisions/ entries yet" +fi + +REQUIRED_SECTIONS=("What changed" "What I verified" "What I did NOT verify") + +for f in "${entries[@]}"; do + base="${f#decisions/}" + + # (2) filename + if [[ ! "$base" =~ ^([0-9]{4}-[0-9]{2}-[0-9]{2})-([a-z0-9][a-z0-9-]*)\.md$ ]]; then + err "${f}: name must be decisions/YYYY-MM-DD-.md with a lowercase [a-z0-9-] slug." + err " Use your branch's last segment as the slug — it is already unique per unit of work." + summary "- decisions-log: **FAILED** (\`${f}\` misnamed)" + rc=1 + continue + fi + fname_date="${BASH_REMATCH[1]}" + + if ! date -d "$fname_date" +%F >/dev/null 2>&1 || [ "$(date -d "$fname_date" +%F)" != "$fname_date" ]; then + err "${f}: '${fname_date}' is not a real calendar date." + summary "- decisions-log: **FAILED** (\`${f}\` — impossible date)" + rc=1 + continue + fi + + # (3) frontmatter + if [ "$(head -1 "$f")" != "---" ]; then + err "${f}: must open with a '---' YAML frontmatter block (date, branch, title)." + err " See decisions/README.md for the template." + summary "- decisions-log: **FAILED** (\`${f}\` — no frontmatter)" + rc=1 + continue + fi + fm="$(awk 'NR==1&&$0=="---"{inb=1;next} inb&&$0=="---"{exit} inb{print}' "$f")" + missing_keys=() + for k in date branch title; do + grep -qE "^${k}:[[:space:]]*\S" <<<"$fm" || missing_keys+=("$k") + done + if [ "${#missing_keys[@]}" -gt 0 ]; then + err "${f}: frontmatter is missing a non-empty ${missing_keys[*]}." + summary "- decisions-log: **FAILED** (\`${f}\` — frontmatter missing \`${missing_keys[*]}\`)" + rc=1 + continue + fi + fm_date="$(grep -m1 -E '^date:' <<<"$fm" | sed -E 's/^date:[[:space:]]*//; s/[[:space:]"'\'']//g')" + if [ "$fm_date" != "$fname_date" ]; then + err "${f}: frontmatter date '${fm_date}' disagrees with the filename's '${fname_date}'." + err " One of them was copied from a template and not edited; the log's ordering depends on both." + summary "- decisions-log: **FAILED** (\`${f}\` — date disagrees with filename)" + rc=1 + continue + fi + + # (4) the three required sections, each with a real body + body_start="$(awk 'NR>1 && $0=="---" {print NR; exit}' "$f")" + [ -n "$body_start" ] || body_start=1 + for want_sec in "${REQUIRED_SECTIONS[@]}"; do + hdr_line="$(awk -v s="$body_start" -v w="## ${want_sec}" 'NR>s && $0==w {print NR; exit}' "$f")" + if [ -z "$hdr_line" ]; then + err "${f}: missing the required section '## ${want_sec}'." + err " CLAUDE.md §7 asks for what you changed, what you verified, and what you did NOT." + err " All three are structure here, because the third is the one that gets dropped." + summary "- decisions-log: **FAILED** (\`${f}\` — missing \`## ${want_sec}\`)" + rc=1 + continue + fi + # A real body: at least 20 non-blank, non-heading characters before the next '## '. + nchars="$(awk -v s="$hdr_line" 'NR>s { if ($0 ~ /^## /) exit; print }' "$f" \ + | tr -d '[:space:]' | wc -c)" + if [ "$nchars" -lt 20 ]; then + err "${f}: section '## ${want_sec}' is empty or a stub (${nchars} chars of content)." + err " If nothing was verified, WRITE THAT — 'nothing was verified; the gate was not run'" + err " is a valid and useful entry. A blank section is not." + summary "- decisions-log: **FAILED** (\`${f}\` — \`## ${want_sec}\` is a stub)" + rc=1 + fi + done +done + +if [ "$rc" -eq 0 ]; then + ok "decisions log: archive frozen, ${#entries[@]} entr$([ "${#entries[@]}" -eq 1 ] && echo y || echo ies) well-formed" + summary "- decisions-log: **passed** (archive frozen; ${#entries[@]} entries well-formed)" +else + err "decisions log stage FAILED" +fi +exit "$rc" diff --git a/ci/gate.sh b/ci/gate.sh index b1cf049..f880a74 100755 --- a/ci/gate.sh +++ b/ci/gate.sh @@ -108,6 +108,7 @@ summary "## Merge gate (R-TEST-03)" summary "" run_stage spec-immutability "${CI_DIR}/spec-immutability.sh" +run_stage decisions-log "${CI_DIR}/decisions-log.sh" run_stage lint "${CI_DIR}/lint.sh" run_stage l0 "${CI_DIR}/l0.sh" run_stage render-golden "${CI_DIR}/render-golden.sh" @@ -214,6 +215,22 @@ case "${OUTCOME[gitea-settings]}" in pass) ok "gitea-settings: green" ;; esac +# decisions-log: the handoff log's storage format. A FAIL means either the frozen archive +# DECISIONS-LOG.md was modified — which re-creates, for every OTHER open branch, the tail-hunk +# conflict that made four of five PRs unmergeable on 2026-09-29 — or a decisions/ entry is +# malformed. Always blocks, and has no skip path by construction: the stage reads only files in +# the tree under test and needs no tool or cluster, so there is no environment in which it +# "could not run". A guard against a silent, delayed, misattributed regression must not itself +# have a way to not-run. +case "${OUTCOME[decisions-log]}" in + fail) err "decisions-log: FAILED -> blocks merge (see decisions/README.md)"; BLOCK=1 ;; + skip) err "decisions-log: reported SKIPPED, which this stage has no path to produce -> treat" + err " as a defect in the stage itself and block." + summary "- decisions-log: **SKIPPED** (impossible by construction) — blocks merge" + BLOCK=1 ;; + pass) ok "decisions-log: green" ;; +esac + # L0/L1 stages: any failure blocks. A SKIP here means a pinned tool is missing or a seam is # unwired — NEVER green. In CI (strict) that is a defect and blocks; on a workstation it # degrades to UNDETERMINED, which is still non-zero and still not a pass. render-golden is a diff --git a/ci/tests/decisions-log.test.sh b/ci/tests/decisions-log.test.sh new file mode 100755 index 0000000..1ec9d71 --- /dev/null +++ b/ci/tests/decisions-log.test.sh @@ -0,0 +1,130 @@ +#!/usr/bin/env bash +# ci/tests/decisions-log.test.sh — proves ci/decisions-log.sh actually fails closed. +# +# The stage exists because appending to DECISIONS-LOG.md silently re-creates a merge conflict +# for every other open branch. A guard against a silent regression is itself worth nothing +# unless somebody has watched it reject the thing it claims to reject, so each case below +# CONSTRUCTS the violation in a throwaway tree and asserts a non-zero exit — the same reasoning +# as ci/tests/gate-decision.test.sh, one layer down. +set -uo pipefail +REAL_CI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" + +PASS=0 FAIL=0 +ok() { printf ' ok %s\n' "$*"; PASS=$((PASS+1)); } +bad() { printf ' FAIL %s\n' "$*"; FAIL=$((FAIL+1)); } + +TMP="$(mktemp -d)"; trap 'rm -rf "$TMP"' EXIT + +# A minimal well-formed tree: the frozen archive, its pin, and one valid entry. +mk_tree() { + local d="$1"; rm -rf "$d"; mkdir -p "$d/ci" "$d/decisions" + cp "${REAL_CI_DIR}/decisions-log.sh" "${REAL_CI_DIR}/lib.sh" "${REAL_CI_DIR}/versions.env" "$d/ci/" + printf 'frozen history\n' > "$d/DECISIONS-LOG.md" + ( cd "$d" && sha256sum DECISIONS-LOG.md > decisions/ARCHIVE.sha256 ) + mk_entry "$d/decisions/2026-09-29-a-good-entry.md" 2026-09-29 + # REPO_ROOT in ci/lib.sh is derived from git; give the tree a repo so it resolves here. + ( cd "$d" && git init -q . && git add -A && git -c user.email=t@t -c user.name=t commit -qm init ) +} + +mk_entry() { + cat > "$1" < -> sets RC and OUT +run() { OUT="$( cd "$1" && bash ci/decisions-log.sh 2>&1 )"; RC=$?; } + +D="${TMP}/t" + +echo "== 1. a well-formed tree passes" +mk_tree "$D"; run "$D" +[ "$RC" -eq 0 ] && ok "clean tree -> exit 0" || bad "clean tree -> exit ${RC}: ${OUT}" + +echo +echo "== 2. THE REGRESSION THIS STAGE EXISTS FOR: appending to the frozen archive" +mk_tree "$D"; printf '\n## my new handoff\n' >> "$D/DECISIONS-LOG.md"; run "$D" +[ "$RC" -ne 0 ] && ok "append to DECISIONS-LOG.md -> non-zero (${RC})" \ + || bad "append to DECISIONS-LOG.md was ACCEPTED — the guard is inert" +grep -qi 'frozen archive' <<<"$OUT" && ok "the message names the archive as frozen" \ + || bad "the message does not say the archive is frozen" +grep -q 'decisions/\$(date +%F)' <<<"$OUT" && ok "the message names the file to write instead" \ + || bad "the message does not name the remedy" + +echo +echo "== 3. a re-pin is a deliberate, accepted act" +mk_tree "$D"; printf '\nredacted\n' >> "$D/DECISIONS-LOG.md" +( cd "$D" && sha256sum DECISIONS-LOG.md > decisions/ARCHIVE.sha256 ); run "$D" +[ "$RC" -eq 0 ] && ok "archive changed AND re-pinned -> exit 0" || bad "re-pin rejected: ${OUT}" + +echo +echo "== 4. a missing pin is unverifiable, not green" +mk_tree "$D"; rm "$D/decisions/ARCHIVE.sha256"; run "$D" +[ "$RC" -ne 0 ] && ok "no ARCHIVE.sha256 -> non-zero (${RC})" || bad "missing pin was accepted" + +echo +echo "== 5. entry naming" +for badname in 2026-9-29-x.md 26-09-29-x.md 2026-09-29-Upper.md 2026-09-29-.md 2026-02-30-x.md; do + mk_tree "$D"; mk_entry "$D/decisions/${badname}" 2026-09-29; run "$D" + [ "$RC" -ne 0 ] && ok "rejects decisions/${badname}" || bad "ACCEPTED decisions/${badname}" +done +mk_tree "$D"; mk_entry "$D/decisions/2026-10-01-fix-thing-2.md" 2026-10-01; run "$D" +[ "$RC" -eq 0 ] && ok "accepts a digits-and-dashes slug" || bad "rejected a valid slug: ${OUT}" + +echo +echo "== 6. frontmatter" +mk_tree "$D"; sed -i '1d' "$D/decisions/2026-09-29-a-good-entry.md"; run "$D" +[ "$RC" -ne 0 ] && ok "rejects an entry with no frontmatter" || bad "accepted a bare entry" + +for key in date branch title; do + mk_tree "$D"; sed -i "/^${key}:/d" "$D/decisions/2026-09-29-a-good-entry.md"; run "$D" + [ "$RC" -ne 0 ] && ok "rejects frontmatter missing '${key}'" || bad "accepted missing '${key}'" +done + +# The copied-template failure: the body was rewritten, the date never was. +mk_tree "$D"; mk_entry "$D/decisions/2026-09-30-copied.md" 2026-09-29; run "$D" +[ "$RC" -ne 0 ] && ok "rejects a frontmatter date that disagrees with the filename" \ + || bad "accepted a date/filename disagreement" + +echo +echo "== 7. the three required sections, and their bodies" +for sec in 'What changed' 'What I verified' 'What I did NOT verify'; do + mk_tree "$D"; sed -i "/^## ${sec}\$/d" "$D/decisions/2026-09-29-a-good-entry.md"; run "$D" + [ "$RC" -ne 0 ] && ok "rejects an entry missing '## ${sec}'" || bad "accepted missing '## ${sec}'" +done + +# A heading with nothing under it is the way this floor gets defeated in practice. +mk_tree "$D" +python3 - "$D/decisions/2026-09-29-a-good-entry.md" <<'PY' +import sys,re +p=sys.argv[1]; s=open(p).read() +s=re.sub(r'## What I did NOT verify\n\n.*', '## What I did NOT verify\n\nn/a\n', s, flags=re.S) +open(p,'w').write(s) +PY +run "$D" +[ "$RC" -ne 0 ] && ok "rejects a stub body under a required section" || bad "accepted an 'n/a' section" + +echo +echo "== 8. README.md in decisions/ is not an entry and is not policed as one" +mk_tree "$D"; printf 'not an entry\n' > "$D/decisions/README.md"; run "$D" +[ "$RC" -eq 0 ] && ok "decisions/README.md ignored by the entry rules" || bad "README policed: ${OUT}" + +echo +if [ "$FAIL" -eq 0 ]; then echo "decisions-log tests: PASS ($PASS assertions)"; exit 0 +else echo "decisions-log tests: FAIL ($FAIL failed, $PASS passed)"; exit 1; fi diff --git a/ci/tests/gate-decision.test.sh b/ci/tests/gate-decision.test.sh index 2cb9dd8..e7096b2 100755 --- a/ci/tests/gate-decision.test.sh +++ b/ci/tests/gate-decision.test.sh @@ -36,7 +36,13 @@ REPO="${TMP}/repo"; mkdir -p "$REPO" cp "${REAL_CI_DIR}/gate.sh" "${REAL_CI_DIR}/lib.sh" "${REAL_CI_DIR}/versions.env" "$CI/" # One stub per stage. It exits with STAGE_RC_, default 0. -for stage in spec-immutability lint l0 render-golden l1 conformance; do +# NOTE: this list must name EVERY stage gate.sh runs. It did not: `gitea-settings` was added to +# gate.sh and never added here, so its script was absent from the stub CI dir, `run_stage` got +# exit 127 from the missing file, and the gate recorded a FAILED stage in every single case — +# 15 of 47 assertions failed on `main` for that reason alone, including every one that expects a +# PASS/UNDETERMINED/NOT-APPLICABLE headline. The suite was measuring the missing stub, not the +# decision logic. `decisions-log` is added here in the same change that adds it to gate.sh. +for stage in spec-immutability decisions-log lint l0 render-golden l1 gitea-settings conformance; do cat > "${CI}/${stage}.sh" <<'STUB' #!/usr/bin/env bash n="$(basename "$0" .sh)"; v="STAGE_RC_${n//-/_}" @@ -184,6 +190,21 @@ assert_verdict BLOCKED 1 "l0 FAILED + l1 SKIPPED" run_gate STAGE_RC_spec_immutability=3 STAGE_RC_conformance=3 assert_verdict UNDETERMINED 3 "spec-immutability SKIPPED + conformance declared-absent" +echo +echo "== 10. decisions-log: the handoff log's storage format has no non-blocking outcome" +# A FAIL here means the frozen archive was appended to (re-creating, for every OTHER open +# branch, the tail-hunk conflict that made 4 of 5 PRs unmergeable on 2026-09-29) or an entry is +# malformed. A SKIP is impossible by construction — the stage needs no tool and no cluster — so +# if one is ever reported it is a defect in the stage, and the gate must not soften it into +# UNDETERMINED. Both outcomes block, and neither is upgradeable by an env knob: there is +# deliberately no REQUIRE_DECISIONS_LOG, because a guard with an off switch is not a guard. +run_gate STAGE_RC_decisions_log=1 +assert_verdict BLOCKED 1 "decisions-log FAILED (archive modified / entry malformed)" +run_gate STAGE_RC_decisions_log=3 +assert_verdict BLOCKED 1 "decisions-log SKIPPED (impossible by construction -> defect, blocks)" +run_gate STAGE_RC_decisions_log=1 STAGE_RC_l1=3 +assert_verdict BLOCKED 1 "decisions-log FAILED outranks an l1 non-run" + echo if [ "$FAIL" -eq 0 ]; then echo "gate-decision tests: PASS ($PASS assertions)"; exit 0 else echo "gate-decision tests: FAIL ($FAIL failed, $PASS passed)"; exit 1; fi diff --git a/decisions/2026-09-29-decisions-log-conflict-free.md b/decisions/2026-09-29-decisions-log-conflict-free.md new file mode 100644 index 0000000..7c59504 --- /dev/null +++ b/decisions/2026-09-29-decisions-log-conflict-free.md @@ -0,0 +1,82 @@ +--- +date: 2026-09-29 +branch: chore/decisions-log-conflict-free +title: The handoff log was serialising every parallel branch — it is a directory now +--- + +## What changed + +`DECISIONS-LOG.md` is frozen and the handoff log is now a directory, `decisions/`, with one +file per entry (`YYYY-MM-DD-.md`). Concurrent branches write disjoint paths, so there +is nothing for git to merge. + +The problem was not any particular change. CLAUDE.md §7 told every agent to append a handoff, +and appending always landed on the *same last hunk* of one 5,228-line file. Two branches that +both did it could not auto-merge; N branches cost N-1 rebases, and each rebase invalidated the +gate run before it. The agents work in isolated worktrees precisely so they don't block each +other, and this one file undid that. + +- `decisions/README.md` — the convention, the template, and how to read the log. No index file + and no sequence number, deliberately: an index is a shared mutable file (the thing that just + failed) and a counter has two agents racing for the same number. The directory listing is the + index. +- `DECISIONS-LOG.md` — a "FROZEN ARCHIVE" header prepended; content otherwise untouched. Its + SHA-256 is pinned in `decisions/ARCHIVE.sha256`. +- `ci/decisions-log.sh` — a new gate stage. Asserts the archive matches its pin, and that every + entry is named `YYYY-MM-DD-.md` with a real date, carries `date`/`branch`/`title` + frontmatter agreeing with its filename, and has all three of §7's sections with a real body + under each. Exit 0 or 1 only; it has no skip path by construction, because it reads only files + in the tree under test and needs no tool and no cluster. +- `ci/gate.sh` — runs the stage; both `fail` and (impossible) `skip` set `BLOCK`. There is no + `REQUIRE_DECISIONS_LOG` knob: a guard with an off switch is not a guard. +- `ci/tests/decisions-log.test.sh` — 22 assertions, new. +- `ci/tests/gate-decision.test.sh` — 3 new assertions for the stage's wiring, **plus a + pre-existing bug fixed**: its stub list never included `gitea-settings`, so that stage's + script was absent from the stub CI dir, `run_stage` recorded exit 127 as a FAILED stage in + every case, and 15 of 47 assertions failed on `main`. The suite was measuring its own missing + stub rather than the decision logic. +- `CLAUDE.md` §7 rewritten; §1's "stale runbook wording" caveat extended to the log. +- `ci/Makefile`, `ci/README.md`, `.gitea/workflows/README.md` — the stage named where stages are + named. + +A guard rather than a paragraph, because the old habit is written into the runbooks and into +every agent that has read the log, and getting it wrong does not fail: it produces a clean-looking +commit that re-creates the conflict on *other people's* branches, later, misattributed. + +## What I verified + +Measured against the live Gitea (`GET /api/v1/repos/Home-Network/home-network-k3s-impl/pulls`, +2026-09-29), not inferred from the tree: + + #22 fix/crafty-oauth2-proxy-signin mergeable=False + #20 feat/conformance-job-template mergeable=False + #19 feat/conformance-trigger-kubeconfig mergeable=False + #18 feat/conformance-runner-image mergeable=False + #17 feat/ci-is-the-approver mergeable=True + +`git merge-tree --write-tree origin/main origin/` for each named +`CONFLICT (content): DECISIONS-LOG.md` and **no other conflicting path** in all four. #17 is +clean and touched no log entry. So the log's storage format was the whole of the blockage. + +Tests: `ci/tests/decisions-log.test.sh` **PASS (22 assertions)**, including the one that +matters — an append to `DECISIONS-LOG.md` exits non-zero and the message names both the frozen +archive and the file to write instead. `ci/tests/gate-decision.test.sh` **PASS (50 assertions)**, +up from **FAIL (15 failed, 32 passed)** on `main` before the stub fix. + +Gate: see the run quoted in the PR description. + +## What I did NOT verify + +- **Nothing was applied and nothing was merged.** No `kubectl`, no `flux reconcile`. This change + touches only CI and documentation; it has no cluster-visible effect at all. +- **L2/L3 conformance did not run** (`CLUSTER_AVAILABLE` unset), as always. Irrelevant to this + change's content but stated so the gate result is not over-read. +- **I have not watched a Gitea PR check run this stage.** The stage is proven by its own unit + test and by the gate locally; that the `incluster-dind` runner executes it is unobserved here. + A skipped Gitea job still reports `success`, so read the job log, not the tick. +- **The four hung PRs are not yet fixed by this commit.** They still carry their + `DECISIONS-LOG.md` appends and will still conflict until each is rebased onto this change with + its append converted to a `decisions/` entry. That is follow-up work, in flight separately. +- **`runbooks/` and older entries still say "append to DECISIONS-LOG.md"** in places. I did not + sweep them. The gate now rejects the action, so the stale wording misleads but cannot cause + the damage; it joins the existing tracked work on stale `git push origin main` wording. diff --git a/decisions/ARCHIVE.sha256 b/decisions/ARCHIVE.sha256 new file mode 100644 index 0000000..0508d73 --- /dev/null +++ b/decisions/ARCHIVE.sha256 @@ -0,0 +1 @@ +b16e8cb7ea12720bc7c3a77a496211e3204534c5e6600ba748fb9ab14fa21c26 DECISIONS-LOG.md diff --git a/decisions/README.md b/decisions/README.md new file mode 100644 index 0000000..b3cadd2 --- /dev/null +++ b/decisions/README.md @@ -0,0 +1,101 @@ +# `decisions/` — the handoff log, one file per entry + +This directory **is** the decision log. `../DECISIONS-LOG.md` is the frozen archive of +everything written before 2026-09-29 and is never edited again (see "The archive is +frozen" below). + +## Why this exists + +`DECISIONS-LOG.md` was append-only in a single file, and every agent appended to its +**tail**. Two branches that both append hit the same last hunk, so git cannot +auto-merge them: the second one to be reviewed conflicts, and so does the third, and +the fourth. Measured on 2026-09-29: **four of the five open PRs were unmergeable, and +`DECISIONS-LOG.md` was the only conflicting file in all four**: + + #22 fix/crafty-oauth2-proxy-signin CONFLICT (content): DECISIONS-LOG.md + #20 feat/conformance-job-template CONFLICT (content): DECISIONS-LOG.md + #19 feat/conformance-trigger-kubeconfig CONFLICT (content): DECISIONS-LOG.md + #18 feat/conformance-runner-image CONFLICT (content): DECISIONS-LOG.md + #17 feat/ci-is-the-approver clean (it touched no log entry) + +Nothing was wrong with any of those changes. The log's *storage format* serialised work +that was otherwise parallel, and it got worse with every concurrent branch: N branches +produce N-1 rebases, each of which invalidates any gate run that preceded it. + +The fix is the obvious one for concurrent appenders: **stop sharing a file**. Each +handoff is its own file. Two agents writing handoffs touch disjoint paths, so there is +nothing for git to merge and nothing to conflict. This scales to any number of parallel +branches with zero coordination — which is the point, since the agents are deliberately +isolated from each other in separate worktrees and cannot coordinate. + +There is deliberately **no index file** and no counter. An index would be a shared +mutable file — exactly the thing that just failed — and a sequence number would have +two agents racing for `0043`. The directory listing is the index. + +## Writing an entry + +One file per handoff, named: + + decisions/YYYY-MM-DD-.md + +`` is lowercase `[a-z0-9-]`. **Use your branch name's last segment** — it is +already unique per unit of work, which is precisely the property the filename needs. +A branch `fix/crafty-oauth2-proxy-signin` handed off on 2026-09-29 writes: + + decisions/2026-09-29-crafty-oauth2-proxy-signin.md + +The file is YAML frontmatter followed by three required sections: + +```markdown +--- +date: 2026-09-29 +branch: fix/crafty-oauth2-proxy-signin +title: The Keycloak gate works — Crafty's own login was the wall +--- + +## What changed + +Prose. What you actually did, and why. + +## What I verified + +The measurements, with their numbers. Quote the field you read, not the prose you +believed. Name the gate result and its exit code. + +## What I did NOT verify + +Everything else. An honest "UNVERIFIED" is worth more than an optimistic "green" — +this repo has been burned by the difference more than once. +``` + +All three headings are **required and enforced** (`ci/decisions-log.sh`). They are +CLAUDE.md §7's contract; making them a checked structure is what stops an entry +degrading into "done ✅". + +## Reading the log + +The whole log, oldest first: + + ls decisions/[0-9]*.md | sort + +The current PM handoff — the newest entry. Ties within a day are broken by what landed +on `main` last, which is the honest answer, so ask git rather than the filename: + + git log --diff-filter=A --format='%H' --name-only -- 'decisions/[0-9]*.md' \ + | grep '^decisions/' | head -1 + +For a quick skim of who said what: + + grep -H '^title:' decisions/[0-9]*.md | sort + +## The archive is frozen + +`DECISIONS-LOG.md` is **read-only history**. Its SHA-256 is pinned in +`decisions/ARCHIVE.sha256` and `ci/decisions-log.sh` fails the merge gate if the file +changes. That is not ceremony: without it, the next agent to read `runbooks/` or an old +CLAUDE.md habit appends to the tail again and re-creates the conflict for everybody +else, silently, one PR at a time. + +If the archive genuinely must change (a leaked credential to redact, say), that is a +deliberate, reviewed act: change it, run `sha256sum DECISIONS-LOG.md > decisions/ARCHIVE.sha256`, +and say in your handoff entry why. Re-pinning is visible in the diff. Appending is not.