Merge pull request 'feat(ci): run the merge gate inside the CI-built toolbox image, pinned by digest' (#147) from feat/gate-toolbox-container into main

Reviewed-on: #147
Reviewed-by: Keith Jackson <[email protected]>

decisions/2026-11-08-gate-toolbox-container.md

@@ -0,0 +1,104 @@
+---
+date: 2026-11-08
+branch: feat/gate-toolbox-container
+title: The merge gate runs inside the CI-built toolbox image, pinned by digest — no per-run tool install
+---
+
+## What changed
+
+`.gitea/workflows/gate.yml`, job `gate` (job name, the gate step, its env and its
+`|| [ "$?" -eq 3 ]` tolerance are unchanged, so the required context `gate / gate (pull_request)`
+is unchanged):
+
+- **`container:`** `gitea.home.keithjackson.dev/home-network/toolbox@sha256:83a3501e…ef81` — the
+  image `images.yaml` published from `dd94101` (run 789; it is today's `main` and `sha-dd941016478c`
+  tag). The same image `infra/devcontainer/dev.sh` runs, so local and CI gate on the identical
+  toolchain. A `# renovate-digest: image=… tag=main` marker sits on the line above, the repo's
+  existing convention; `infra/gitea/registry-retention.sh` reads the same marker, so this digest
+  is protected from pruning.
+- **`actions/checkout` replaced by a plain-git step.** The toolbox has no Node (measured:
+  `command -v node` → missing), and `actions/checkout` is a Node action. Chosen over adding Node to
+  the toolbox because that would need two sequential PRs (Containerfile, then a publish, then the
+  pin) and would put a JS runtime into the operator toolbox only to run one action. The step
+  reproduces what checkout@v7 `fetch-depth: 0` actually ran here (read from runs 490 and 495):
+  every branch to `refs/remotes/origin/*`, tags, plus the event SHA; detached for a PR,
+  `-B <branch>` for a push; asserts `HEAD == github.sha`. The job token goes to that one fetch via
+  `GIT_CONFIG_*` env and is **not** persisted in `.git/config` (nothing later needs network git).
+  It sets **`umask 0022`** first: the PR's first CI run (run 500) failed L0
+  `prod-baremetal-not-dev-surface` because act runs steps with umask 0000, every checked-out
+  directory was world-writable, and Ansible (present in the toolbox, absent from the catthehacker
+  image) then ignored `infra/ansible/ansible.cfg`. Reproduced locally in the pinned image: clone
+  under umask 0000 → that check fails with the same message; under 0022 → `check OK`. Side
+  effect worth knowing: the Ansible-driven parts of that check now actually execute in CI.
+- **"Install pinned CI toolchain" step removed**, replaced by **"Toolchain is the one
+  ci/versions.env pins"**: every `*_VERSION` in `ci/versions.env` must equal the image's
+  `/etc/dev-toolchain-versions`, else `::error::DRIFT` and the job fails. Without this, dropping
+  the installer would have made `ci/versions.env` an unenforced claim for everything except
+  promtool (which `alert-rules-unit-tested` already asserts).
+- Header comments rewritten for the new job image (pull auth, cold-pull size, no-Node, why
+  `install-tools.sh` stays).
+
+`renovate.json5`: the `renovate-digest` custom manager now also scans `.gitea/workflows/gate.yml`;
+the `github-actions` manager is disabled for this repo's own images (same reason the `kubernetes`
+manager already was: it would see a digest-only image and look up `latest`).
+
+Comment/message wording only: `ci/gate.sh` (header + the strict-skip error text), `ci/lib.sh`,
+`ci/README.md`, `.gitea/workflows/README.md` — they said the CI toolchain comes from
+`install-tools.sh`.
+
+`.gitea/workflows/scripts/install-tools.sh` is **kept**: `merge-gate.yaml` (dormant blueprint)
+calls it, and `conformance/tests/l0-static/alert-rules-unit-tested/check.sh` asserts its promtool
+wiring (not touched — another agent owns `conformance/`).
+
+**Consequence to know:** every toolbox digest bump is now a Renovate PR touching `.gitea/**`,
+i.e. a sensitive path needing an operator `/approve-override`. That is arguably correct (a new
+toolchain changes what green means — and `ci/versions.env` was already sensitive), but it is a
+new recurring operator touch.
+
+## What I verified
+
+- Registry pull auth: the Home-Network org is `public`; anonymous `/v2/token` → manifest `200`,
+  `docker-content-digest: sha256:83a3501e…ef81` for both `main` and `sha-dd941016478c`. So no
+  `container: credentials:` is needed. (The Actions token could not have been used anyway — no
+  package scope, per images.yaml's header.)
+- Image: 466,912,274 bytes compressed across 6 layers (manifest sum), 1,541,728,125 bytes
+  unpacked (`podman image inspect`). Local cold pull 10.4s on the workstation.
+- Image tool versions equal `ci/versions.env` for all 8 pins (kyverno 1.19.0, kyverno-json 0.0.3,
+  chainsaw 0.2.12, yamllint 1.35.1, kubectl 1.36.4, yq 4.44.6, promtool 3.14.0, check-jsonschema
+  0.29.4) — both via `/etc/dev-toolchain-versions` and by running each binary's `--version`.
+  `sleep`, `tail`, `git` 2.56.0, `bash`, `python3`, `jq`, `curl` present (act's entrypoint is
+  `/bin/sleep 10800`).
+- The new checkout script, run in the pinned image against the public repo (auth header disabled
+  for the test only): PR ref `refs/pull/143/head` → detached at `60e3a24`, 1018 commits; push ref
+  `refs/heads/main` → `On branch main` at `e8d2666`; `origin/main` present in both. The
+  toolchain-drift step passes on the real `ci/versions.env` and exits 1 naming both tools when
+  yamllint/promtool pins are altered.
+- Renovate marker regex (copied from renovate.json5) matches gate.yml exactly once, capturing
+  depName/tag/digest.
+- LOCAL GATE, run **in the pinned CI image itself** (`IMAGE=<the digest above>` through
+  `dev.sh`, `DEV_KUBECONFIG=/dev/null`, ENGINE shim adding the main checkout's `.git` read-only),
+  on commit `9537ca2`: **rc 3**; spec-immutability, decisions-log (78 entries), lint, l0,
+  render-golden, l1, approval-policy (N/A locally — not a PR event) all pass; **gitea-settings
+  the only stage not run** (no `SETTINGS_API_TOKEN` locally); L2/L3 conformance skipped as
+  usual. rc 3 is not green.
+
+- **CI, this PR's own runs** (Gitea API `/actions/runs/{id}/jobs` + job logs):
+  - run 500 (`514ed5d`, cold DinD cache: `Image exists? false`, image pulled): job 151s; set-up
+    incl. pull **9s**, checkout <1s, toolchain check 1s (all 8 pins `ok`), gate 140s → BLOCKED by
+    L0 `prod-baremetal-not-dev-surface` (umask, fixed above) + approval-policy.
+  - run 503 (`27d603f`, warm: `Image exists? true`): job **140s**; set-up 1s, checkout 1s,
+    toolchain check 0s, gate 137s. Every stage passes — l0 (incl. prod-baremetal-not-dev-surface),
+    lint, decisions-log, render-golden, l1, **gitea-settings** — except **approval-policy**,
+    the expected sensitive-path block. spec-immutability skipped (expected, no spec repo in CI).
+  - baseline before (runs 494/495, push on main): job 180s/179s; set-up 1–2s, checkout 1s,
+    **install 11s**, gate 164–165s. The install step is gone; the remaining ~25s gate-step delta
+    is NOT claimed as caused by this change (different base commits, one sample each).
+- Re-gated locally on `27d603f` in the pinned image: rc 3, gitea-settings the only not-run stage.
+
+## What I did NOT verify
+
+- Renovate actually opening a digest-bump PR for gate.yml — only the regex was tested, not a
+  Renovate run.
+- Push-to-main behaviour in CI (only the PR event runs before merge).
+- Whether the gate-step speedup (165s → 137s) is real or run-to-run variance.
+- That the cold-pull cost stays ~9s under a busier node or a bigger digest delta; one sample.