diff --git a/DECISIONS-LOG.md b/DECISIONS-LOG.md index 195c1dd..588a5f3 100644 --- a/DECISIONS-LOG.md +++ b/DECISIONS-LOG.md @@ -4964,3 +4964,148 @@ account rubber-stamping the human. That is ceremony, not review. Options: keep i the ceremony; drop `required_approvals` to 0 and rely on the status check, which is the gate that actually verifies something; or add a second human account. **This is a decision for the operator, not a default to be chosen silently.** + +## 2026-09-29 (end of session) — **PM HANDOFF.** Six PRs open; read this before touching anything. + +The session that began "set a SOPS secret" ended having rebuilt the delivery pipeline. Plex is +claimed and scanning. `main` is genuinely protected and the gate genuinely enforces. What is +open is listed below with the reason each is not merged. + +### MERGE THESE — both gated green + +| PR | What | Risk | +|---|---|---| +| #12 | `CLUSTER_AVAILABLE=true` would have FAKED A GREEN over 54 skipped tests — now blocks | low; makes a wrong state loud | +| #13 | Admission boundary constraining the conformance CI trigger (R-SEC-02) + a fail-open fix in `ci/l0.sh` | medium; touches Kyverno + adds a Flux Kustomization | + +(#11, the Gitea drift detector, was merged during this session — hence the action below.) + +Order does not matter for correctness; if a later one conflicts, rebase and the gate re-runs +(~1m35s in CI, 76s local). **Each was gated in the pinned toolbox, not on a host.** + +### DO THIS FIRST — #11 is merged and its detector is currently INERT + +`ci/gitea-settings.sh` SKIPS until its token exists, and **a skipped detector detects nothing**: + + ./infra/gitea/set-ci-token.sh # mints + stores GITEA_SETTINGS_TOKEN, never prints it + +Then open any PR and confirm the gate prints `gitea-settings: green`. ONLY THEN uncomment +`REQUIRE_GITEA_SETTINGS: "true"` in `.gitea/workflows/gate.yml`, so a future missing token blocks +instead of skipping. Doing that before seeing it green locks the repo. + +Until then every gate run ends `MERGE GATE: UNDETERMINED` (exit 3) naming `gitea-settings` as +unrun. That is BY DESIGN and `gate.yml` tolerates exit 3, so PRs still pass — but it is not a +green run, and it is saying so honestly. The token is what turns it green. + +### A DIFFERENT COMPUTER NEEDS THREE THINGS THAT ARE NOT IN GIT + +This is the part that will silently waste an hour otherwise. None of these are in the repo, by +design, and without them the toolbox and the cluster are unreachable: + +1. **`~/.config/sops/age/keys.txt`** — the age PRIVATE key. Without it NO `*.sops.yaml` can be + read or edited, and `sops` will fail in a way that looks like a broken file. Copy it from this + workstation; the public recipient in `.sops.yaml` is only half the pair. +2. **`infra/ansible/.cache/prod.kubeconfig`** — untracked, so it exists only in the main checkout + on this machine. Without it `dev.sh` refuses to start. In a git WORKTREE it is absent even + here: use `DEV_KUBECONFIG=/dev/null` for anything that needs no cluster (see `CLAUDE.md` §3). +3. **A Gitea credential** for `gitea.home.keithjackson.dev` (this session used the stored + `~/.git-credentials`). `git push` and every `infra/gitea/*.sh` script needs it. + +### STATE, MEASURED — do not re-derive it from this log, re-measure it + +`CLAUDE.md` §2b exists because this log was confidently wrong for a month. Verify before relying: + + protected:true user_can_push:false enable_status_check:true + status_check_contexts:["gate / gate (pull_request)"] required_approvals:1 + block_admin_merge_override:true runner incluster-dind: online gate ~1m35s + +### NEXT WORK, in priority order + +1. **Finish the L2/L3 wiring** now that the credential model is decided and #13 lands the + boundary. Still missing (`docs/plans/l2-l3-conformance-ci-wiring.md` §3): a runner image with + bash/python3/kubectl/git/chainsaw, an init-container cloning the repo at the PR SHA, results + collection via a base64 tar on stdout, and the `ci/conformance.sh` rewrite to drive the Job. + **Tighten the admission policy's image pattern to the literal digest when that image exists.** +2. **Do NOT set `vars.CLUSTER_AVAILABLE=true`** until step 1 is done. It does not enable L2/L3; it + only changes which way the stage fails (see #12). +3. The `gitea-settings` detector covers branch protection + runners. It does NOT yet cover Gitea + Actions **variables and secrets** — the next-nearest invisible surface. +4. `required_approvals: 1` with two accounts means a PR authored by `jakman85` needs `gitea-admin` + to approve. The operator holds ultimate authority and does not require that review; revisit if + it becomes friction. + +### THE LESSON, because it caused every finding here + +Four times this session the repository was treated as evidence about the live platform, and four +times it was wrong: the push that would "be rejected" (it would not have been), the token that +"could not survive the pipeline" (~2m45s, it fits), the required check that would "deadlock" (a +skipped Gitea job reports SUCCESS), and enforcement that was "on" (admins overrode it). Two +fail-opens in the gate itself were found the same way — `kyverno test` certifying policies that do +not load, and `CLUSTER_AVAILABLE` certifying 54 skipped tests as passed. + +**Measure, then assert. `CLAUDE.md` §2b is the rule; this entry is the evidence for it.** + +## 2026-09-29 (addendum, unattended) — the detector is armed; **I CANNOT MERGE, and that is correct** + +Operator ran `set-ci-token.sh`, uncommented `REQUIRE_GITEA_SETTINGS` locally on `main`, and left +instructions to continue unattended and merge on green CI. Two things came out of that. + +### 1. The detector was INERT, and would have stayed inert silently + +`set-ci-token.sh` PUT the secret as `GITEA_SETTINGS_TOKEN`. **Gitea reserves the `GITEA_` +prefix** for secret names (as GitHub reserves `GITHUB_`) and rejected it: + + HTTP 400 {"message":"invalid secret name"} + +The script sent that PUT to `/dev/null` and printed `==> stored.` unconditionally; `curl -sS` +exits 0 on an HTTP error. **A token existed; no secret did.** So `ci/gitea-settings.sh` would +have skipped on every run forever — and enabling `REQUIRE_GITEA_SETTINGS` on top of that would +have BLOCKED EVERY PR, including the one fixing it. That is the exact lock-out this repo has now +nearly walked into three times. + +Fixed in **PR #15**: the secret is `SETTINGS_API_TOKEN`, and `set-ci-token.sh` now checks the HTTP +status AND reads the secret back before claiming success. The missing verification was the real +defect; the illegal name is only what it hid. The secret is stored and verified present, and +**PR #15's CI is GREEN with `REQUIRE_GITEA_SETTINGS=true`** — which is the only proof that +matters, because a skip would have made it red. + +The operator's local edit could not be pushed to `main` (protected). It is carried in PR #15. + +### 2. No agent-authored PR can be merged without the operator — by design + +Merging PR #12 as `gitea-admin` returned: + + HTTP 405 {"message":"Does not have enough approvals"} + +`required_approvals: 1` + `block_admin_merge_override: true` + two accounts, one of which +(`gitea-admin`) authors every agent PR and cannot self-approve. **So "merge on green CI" is not +something an agent can execute here.** That is the control working, not a fault — and it was +deliberately not circumvented. Self-approving would hollow out the boundary that was just built +and verified. + +**This is an operator decision, not a bug to fix:** either accept that every agent PR waits for a +human approval (the honest reading of the control), or add a second human reviewer account, or +lower `required_approvals` to 0 and rely on the status check — which is the gate that actually +verifies something. Whichever, change the DECLARATION in +`infra/gitea/expected-branch-protection.json` in the same commit, or `gitea-settings` will +correctly fail the gate on the drift. + +### Open PRs, all gated green, none mergeable by an agent + +| PR | What | +|---|---| +| #12 | `CLUSTER_AVAILABLE=true` would have faked a green over 54 skipped tests — now blocks | +| #13 | Conformance-trigger admission boundary (R-SEC-02) + `kyverno test` fail-open fix | +| #14 | This handoff | +| #15 | Secret-name fix + `REQUIRE_GITEA_SETTINGS` enabled — **CI green with REQUIRE active** | + +Suggested order: #15 first (it arms the detector and its CI already proved it), then #12, #13, +#14. If a later one conflicts, rebase; the gate re-runs in ~1m36s. + +### Deliberately NOT done while unattended + +No new feature work was started. Five unmerged PRs already stack review burden, and building the +next L2/L3 piece on top of unmerged branches would create a rebase tangle for no gain — the work +cannot land any faster. The next piece is specified and ready to start once these are merged: +`docs/plans/l2-l3-conformance-ci-wiring.md` §3, with §4's credential model now decided and its +boundary landed in #13.