feat(images): build, verify and publish the three images in CI; Renovate carries the digests

Operator rule 2026-10-29: image creation, hosting and lifecycle belong to CI/CD, never to a workstation build/push. Nothing had ever been pushed to the Gitea registry. - .gitea/workflows/images.yaml + scripts/images.sh: PRs build+verify (no credential); main builds, verifies and pushes toolbox, agent-runner (FROM toolbox) and conformance-runner as sha-<commit> + main, reading the digest back from the registry. Only changed images build. - Each image's own build-image.sh does the build and in-image verification, now engine-neutral (DinD: no podman-only calls, probe over stdin not a bind mount). New toolbox build-image.sh. - Digest references carry `# renovate-digest:` markers; renovate.json5 gains a custom.regex manager for them. The worker Job template is deliberately untracked (it would loop). - renovate.yaml pins renovate-version 44.167.0: the action's default (39.264.0) rejects the current config's managerFilePatterns, so every run would have died at CONFIG_VALIDATION. - Push credential: REGISTRY_PUSH_TOKEN secret; the Actions token has no package scope in Gitea 1.24.3 (container.go Authenticate). Retention: infra/gitea/registry-retention.sh. Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-29 23:59:55 -05:00
1 parent 27da5555a4
commit e539109830
@@ -115,14 +115,47 @@ CI. That is a real coverage gap, not a phase.
 | variable | `REQUIRE_CLUSTER_TESTS` | `'true'` (Phase 1+) makes unverified cluster-needing touched reqs a hard block |
 | secret | `CONFORMANCE_TRIGGER_KUBECONFIG` | narrow creds to create/await the in-cluster Chainsaw Job (not cluster-admin) |
 ## How images are built
 This repo's three images — `toolbox` (`infra/devcontainer/`), `agent-runner` (`infra/agent/`,
 built `FROM` the toolbox) and `conformance-runner` (`conformance/incluster/`) — are built,
 verified and published by [`images.yaml`](images.yaml), never from a workstation
 (operator rule, 2026-10-29). The logic is in [`scripts/images.sh`](scripts/images.sh); each
 image is built by **its own** `build-image.sh`, the same script a developer runs locally, so the
 in-image verification (non-root uid, read-only rootfs, tool presence, `PM_SELFTEST=1
 pm-dispatch`, the `conformance-clone` refusals) exists once.
 | Trigger | What happens |
 |---|---|
 | pull request touching an image's inputs | build + verify only. Nothing is pushed and **no registry credential is referenced** |
 | push to `main` touching an image's inputs | build + verify + publish to `gitea.home.keithjackson.dev/home-network/<image>` as `sha-<commit>` (immutable) and `main` (moving); the **registry-returned** digest is in the job summary |
 | `workflow_dispatch` | the same as a push when run on `main` (all images by default); verify-only elsewhere |
 Only images whose inputs changed are built. The publish job **never writes to git**: Renovate
 follows each image's `main` tag through a `# renovate-digest:` marker next to every digest
 reference (`platform/pm-agent/`, `infra/agent/versions.env`, `conformance/versions.env`) and
 opens the bump PR, which goes through the gate like anything else. A PR opened with the Actions
 token would not trigger the required gate, which is why it is Renovate's job and not this one's.
 Credential: the repository secret **`REGISTRY_PUSH_TOKEN`** (scope `write:package`, owned by an
 account with package write on the org). The automatic Actions token cannot push in Gitea 1.24.3
 — the reasoning, with source citations, is in the header of `images.yaml`. Retention:
 [`infra/gitea/registry-retention.sh`](../../infra/gitea/registry-retention.sh), an operator
 action, keeps every digest git pins plus the newest tags.
 Cost on this runner: the job shares the single act_runner slot with the gate, and the DinD store
 is a node `emptyDir`; the job prunes it on exit.
 ## Files
 ```
 .gitea/workflows/
  merge-gate.yaml            the pipeline (lint → l0 → l1 → conformance → decision)
  images.yaml                build/verify/publish this repo's container images (see above)
  scripts/
    install-tools.sh         installs the pinned tools per stage (from ci/versions.env)
    decide.sh                the R-TEST-03 merge decision over job results
    images.sh                the image pipeline's driver (what to build, base, push, digest)
  README.md                  this file
 ```
@@ -0,0 +1,174 @@
 # Container images — built, verified and published by CI, never from a workstation.
 #
 # ═══════════════════════════════════════════════════════════════════════════════════════════════
 # WHY THIS FILE EXISTS (operator rule, 2026-10-29)
 # ═══════════════════════════════════════════════════════════════════════════════════════════════
 # "Container image creation, hosting and lifecycle must be part of CI/CD and the codebase, never a
 # one-off build/push from a workstation." Before this workflow, three images existed only as
 # workstation builds and NOTHING had ever been pushed to the Gitea registry: the Home-Network org's
 # package list was empty, platform/pm-agent/ pinned an all-zero digest and sat in ImagePullBackOff,
 # and conformance/versions.env carried an empty digest. Both build scripts said "publishing is an
 # operator action" — a step with no log, no recorded inputs and no verification anyone could read.
 #
 # WHAT IT DOES — the logic is in scripts/images.sh; this file only wires triggers and secrets.
 #   pull_request -> build + VERIFY only. No push, and no registry credential is referenced.
 #   push to main -> build + verify + PUBLISH to gitea.home.keithjackson.dev/home-network/<image>
 #   dispatch     -> same as a push when run on main; verify-only on any other ref.
 # Images: toolbox (infra/devcontainer), agent-runner (infra/agent, FROM toolbox) and
 # conformance-runner (conformance/incluster). Only images whose inputs changed are built.
 #
 # THE DIGEST LIFECYCLE — this workflow NEVER writes to git.
 #   1. publish pushes `sha-<commit>` (immutable) and `main` (moving), and prints the digest the
 #      REGISTRY returns in the job summary.
 #   2. Renovate (renovate.yaml) follows the `main` tag of each image through the
 #      `# renovate-digest:` markers next to every digest reference, and opens the bump PR.
 #   3. That PR goes through the gate like any other change, and Flux deploys it.
 # The bump PR is Renovate's and not this job's because a PR opened with the Actions token does not
 # trigger workflows, so it could never pass the REQUIRED gate check; Renovate's token is a real
 # user's.
 #
 # THE PUSH CREDENTIAL — `REGISTRY_PUSH_TOKEN`, a repository Actions secret, referenced BY NAME.
 # The automatic Actions token CANNOT be used. Gitea 1.24.3 source, verified:
 #   * services/auth/basic.go (Basic.Verify) turns a task token into the pseudo-user
 #     `gitea-actions` (ActionsUserID -2) and sets NO token scope;
 #   * routers/api/packages/container/container.go (Authenticate, the /v2/token exchange every
 #     docker client uses) then requires read:package / write:package / all scope and answers 401;
 #   * services/context/package.go (determineAccessMode) has `// TODO: ActionUser permission check`
 #     and no branch for that user, so even past login it would get at most READ on a public org.
 # So: a personal access token with scope `write:package`, minted by an account that has package
 # WRITE on the Home-Network org (org owner, or a team with the Packages unit at write). A dedicated
 # bot account in a packages-only team is the least-privilege choice. The username sent with it is
 # not checked by Gitea (basic.go resolves the owner from the token), so `vars.REGISTRY_PUSH_USER`
 # is optional and only labels the login.
 #
 # THE RUNNER (apps/wave-1/gitea-actions.yaml): act_runner 0.2.11, one job at a time, Docker
 # executor on a privileged `docker:27-dind` sidecar whose image store is an emptyDir on the node
 # (no sizeLimit) and whose memory limit is 2Gi — shared by the job container AND every build
 # container. The toolbox is ~1.4 GB uncompressed and agent-runner adds ~0.3 GB on top of it, so the
 # cleanup step below is not optional: without it each run would leave ~3 GB on a node that had
 # 7 GB free when this was written, and the next eviction would take the runner with it.
 name: images
 on:
  pull_request:
    branches: [main]
    paths:
      - infra/devcontainer/Containerfile
      - infra/devcontainer/versions.env
      - infra/devcontainer/build-image.sh
      - infra/agent/**
      - conformance/incluster/Containerfile
      - conformance/incluster/conformance-clone
      - conformance/incluster/build-image.sh
      - .gitea/workflows/images.yaml
      - .gitea/workflows/scripts/images.sh
  push:
    branches: [main]
    paths:
      - infra/devcontainer/Containerfile
      - infra/devcontainer/versions.env
      - infra/devcontainer/build-image.sh
      - infra/agent/**
      - conformance/incluster/Containerfile
      - conformance/incluster/conformance-clone
      - conformance/incluster/build-image.sh
      - .gitea/workflows/images.yaml
      - .gitea/workflows/scripts/images.sh
  workflow_dispatch:
    inputs:
      images:
        description: "Which images: 'all', or a space-separated subset of: toolbox agent-runner conformance-runner"
        required: false
        default: "all"
 concurrency:
  # Never cancel a publish half-way: a pushed `sha-` tag with no matching `main` move is
  # recoverable (a rerun skips what is already published) but pointless to provoke.
  group: images-${{ github.ref }}
  cancel-in-progress: false
 jobs:
  images:
    runs-on: ubuntu-latest
    # Measured on the PR that introduced this file — see its decisions entry. Generous because a
    # cold DinD store downloads the Fedora base and every pinned tool on each run.
    timeout-minutes: 75
    steps:
      - name: Checkout
        uses: actions/checkout@v4
        with:
          fetch-depth: 0   # the per-image change detection diffs against the base commit
      # ── pull requests: build + verify, NO credential ───────────────────────────────────────
      - name: Build and verify (pull request — nothing is pushed)
        if: github.event_name == 'pull_request'
        env:
          MODE: verify
          IMAGES: auto
          PR_BASE_REF: ${{ github.base_ref }}
        run: |
          set -euo pipefail
          BASE_SHA="$(git merge-base "origin/${PR_BASE_REF}" HEAD)"
          export BASE_SHA
          ./.gitea/workflows/scripts/images.sh
      # ── non-main dispatch: verify only ──────────────────────────────────────────────────────
      - name: Build and verify (dispatch off main — nothing is pushed)
        if: github.event_name == 'workflow_dispatch' && github.ref != 'refs/heads/main'
        env:
          MODE: verify
          IMAGES: ${{ inputs.images == 'all' && 'toolbox agent-runner conformance-runner' || inputs.images }}
        run: ./.gitea/workflows/scripts/images.sh
      # ── main: publish ───────────────────────────────────────────────────────────────────────
      - name: Preflight - the push token must exist, and its absence must be LOUD
        if: github.event_name != 'pull_request' && github.ref == 'refs/heads/main'
        env:
          REGISTRY_PUSH_TOKEN: ${{ secrets.REGISTRY_PUSH_TOKEN }}
        run: |
          set -euo pipefail
          if [ -z "${REGISTRY_PUSH_TOKEN:-}" ]; then
            echo "::error::REGISTRY_PUSH_TOKEN is not set on this repository." >&2
            cat >&2 <<'MSG'
          Images cannot be published without a registry credential, and this step fails rather
          than skipping ON PURPOSE: a skipped Gitea job reports `success`, so skipping here would
          make "nothing was published" look exactly like "published". That is how this repo went
          a month with images that existed only on one workstation.
          WHY NOT THE BUILT-IN ACTIONS TOKEN: Gitea 1.24.3 gives it no package scope at the
          registry's /v2/token exchange (routers/api/packages/container/container.go,
          Authenticate) — see the header of .gitea/workflows/images.yaml.
          TO FIX: as an account with package WRITE on the Home-Network org (ideally a dedicated
          bot in a packages-only team), create an access token with scope `write:package` and
          nothing else, then add it to this repository as the Actions secret
          REGISTRY_PUSH_TOKEN. Optionally set the Actions variable REGISTRY_PUSH_USER to that
          account's name. Do not commit the token anywhere. Then re-run this workflow
          (workflow_dispatch on main, images: all).
          MSG
            exit 1
          fi
          echo "REGISTRY_PUSH_TOKEN is present (value not printed)."
      - name: Build, verify and publish (main)
        if: github.event_name != 'pull_request' && github.ref == 'refs/heads/main'
        env:
          MODE: publish
          IMAGES: ${{ github.event_name == 'workflow_dispatch' && (inputs.images == 'all' && 'toolbox agent-runner conformance-runner' || inputs.images) || 'auto' }}
          BASE_SHA: ${{ github.event.before }}
          REGISTRY_TOKEN: ${{ secrets.REGISTRY_PUSH_TOKEN }}
          REGISTRY_USER: ${{ vars.REGISTRY_PUSH_USER }}
        run: ./.gitea/workflows/scripts/images.sh
      # ── always: give the DinD store back ────────────────────────────────────────────────────
      # The runner's image store is a node emptyDir with no sizeLimit (see header). Everything this
      # job built or pulled is removed; `prune -a` cannot remove the job's own image, because the
      # job container is still running on it.
      - name: Clean the DinD image store
        if: always()
        run: |
          docker logout gitea.home.keithjackson.dev >/dev/null 2>&1 || true
          docker image prune -af >/dev/null || true
          docker builder prune -af >/dev/null || true
          docker system df || true
@@ -60,8 +60,9 @@
 # entry's "What I did NOT verify" — that gap is stated, not papered over.
 #
 # OPERATOR PREREQUISITE — ONE SECRET, NOT COMMITTED HERE:
-#   `RENOVATE_TOKEN` — a Gitea access token for the bot account, scopes `write:repository` and
+#   `RENOVATE_TOKEN` — a Gitea access token for the bot account, scopes `write:repository`,
-#   `read:user`. Set it as an Actions secret on this repository. Nothing in this repo mints,
+#   `read:user` and (since 2026-10-29) `read:package`. Set it as an Actions secret on this
 #   repository. Nothing in this repo mints,
 #   stores or references its value; it is consumed by name only.
 #   Until it exists, this workflow FAILS LOUDLY on every scheduled run. That is intended: a
 #   visible red is the correct state for "automation configured but not credentialed".
@@ -139,8 +140,9 @@ jobs:
          seven files claimed "Renovate maintains this pin".
          TO FIX: create a Gitea access token for the bot account with scopes
-          `write:repository` and `read:user`, then add it to this repository as the Actions
+          `write:repository`, `read:user` and `read:package` (the last one lets Renovate read
-          secret RENOVATE_TOKEN. Do not commit it anywhere.
+          this repo's own images in the Gitea registry), then add it to this repository as the
          Actions secret RENOVATE_TOKEN. Do not commit it anywhere.
          MSG
            exit 1
          fi
@@ -179,6 +181,14 @@ jobs:
          RENOVATE_REQUIRE_CONFIG: "required"
          RENOVATE_DRY_RUN: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run == 'true' && 'full' || '' }}
          LOG_LEVEL: ${{ github.event_name == 'workflow_dispatch' && inputs.log_level || 'info' }}
          # ADDED 2026-10-29: registry read access for this repo's OWN images (toolbox,
          # agent-runner, conformance-runner — published by .gitea/workflows/images.yaml and
          # followed through `# renovate-digest:` markers). The Gitea registry answers 401 to
          # anonymous reads, and Renovate scopes the platform token to hostType `gitea`, which a
          # docker lookup never matches — so without this rule every digest lookup fails. Same
          # token; it therefore needs scope `read:package` in addition to the ones listed in the
          # preflight above. Gitea ignores the username for token auth (services/auth/basic.go).
          RENOVATE_HOST_RULES: '[{"hostType":"docker","matchHost":"gitea.home.keithjackson.dev","username":"renovate","password":"${{ secrets.RENOVATE_TOKEN }}"}]'
      - name: Summary
        if: always()
@@ -0,0 +1,291 @@
 #!/usr/bin/env bash
 ###############################################################################
 # .gitea/workflows/scripts/images.sh — the image pipeline's driver (images.yaml calls this).
 #
 # Builds, verifies and (on main) PUBLISHES the three images this repo ships:
 #
 #   toolbox             infra/devcontainer/   the pinned dev/gate toolchain
 #   agent-runner        infra/agent/          FROM toolbox; the PM dispatcher + worker
 #   conformance-runner  conformance/incluster/ the in-cluster L2/L3 Chainsaw runner
 #
 # Each image is built by ITS OWN build-image.sh — the same script a developer runs locally —
 # so the in-image verification (non-root uid, read-only rootfs, tool presence, pm-dispatch
 # selftest, conformance-clone refusals, ...) is written once and runs in both places. This
 # file only decides WHAT to build, in which order, FROM which base, and where it goes.
 #
 # MODES (env MODE):
 #   verify   build + verify, never push, holds NO credential. Pull requests.
 #   publish  build + verify + push to ${REGISTRY}/${NAMESPACE}/<image> with tags
 #            `sha-<12 hex>` (immutable: never re-pushed) and `main` (moving; Renovate follows
 #            it). The digest recorded is the one the REGISTRY returns, read back after the push.
 #
 # WHAT GETS BUILT (env IMAGES): `auto` (default) diffs BASE_SHA..HEAD_SHA against each image's
 # inputs; otherwise a space-separated list. A change to the pipeline itself (images.yaml, this
 # file) builds everything in verify mode, and nothing extra in publish mode: re-publishing
 # unchanged inputs would only mint new digests and churn Renovate PRs.
 #
 # THE agent-runner BASE — the one ordering rule, and the reason this is not three jobs:
 #   * toolbox built in this run  -> agent-runner builds FROM it (in publish mode, FROM the
 #     digest the registry just returned for it).
 #   * verify mode, toolbox not changed -> the toolbox is built anyway as a base. A pull request
 #     holds no registry credential, so it cannot pull the published one.
 #   * publish mode, toolbox not changed -> AGENT_BASE_IMAGE from infra/agent/versions.env,
 #     i.e. the published toolbox digest that Renovate keeps current. If that is still the
 #     all-zero placeholder the toolbox has never been published, so it is built and published
 #     first (this is how the very first run bootstraps).
 #
 # LOOP-FREEDOM. Renovate turns each publish into a digest-bump PR. Those bumps land in
 # platform/pm-agent/ (no image input), conformance/versions.env (no image input) and
 # infra/agent/versions.env (agent-runner input -> ONE more agent-runner build, whose own bump
 # lands in platform/pm-agent/ and stops). The worker Job template is deliberately NOT tracked:
 # it is baked INTO agent-runner, so bumping it would rebuild agent-runner, mint a new digest,
 # bump it again, forever. pm-dispatch overwrites that field at runtime anyway.
 ###############################################################################
 set -euo pipefail
 REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)"
 cd "${REPO_ROOT}"
 MODE="${MODE:-verify}"
 IMAGES="${IMAGES:-auto}"
 ENGINE="${ENGINE:-docker}"
 REGISTRY="${REGISTRY:-gitea.home.keithjackson.dev}"
 NAMESPACE="${NAMESPACE:-home-network}"
 HEAD_SHA="${HEAD_SHA:-$(git rev-parse HEAD)}"
 BASE_SHA="${BASE_SHA:-}"
 SUMMARY="${GITHUB_STEP_SUMMARY:-/dev/null}"
 ZERO_DIGEST="sha256:0000000000000000000000000000000000000000000000000000000000000000"
 ALL_IMAGES=(toolbox agent-runner conformance-runner)
 SHA_TAG="sha-${HEAD_SHA:0:12}"
 MOVING_TAG="main"
 log()  { printf '\n\033[1m==> %s\033[0m\n' "$*"; }
 info() { printf '    %s\n' "$*"; }
 die()  { printf '\033[31mERROR: %s\033[0m\n' "$*" >&2; printf -- '- **ERROR**: %s\n' "$*" >>"${SUMMARY}"; exit 1; }
 case "${MODE}" in verify|publish) ;; *) die "MODE must be verify or publish, got '${MODE}'" ;; esac
 # --- inputs per image: a change under any of these rebuilds that image ---------------------
 # conformance-runner takes its chainsaw/kubectl/fedora pins from infra/devcontainer/versions.env.
 # conformance/versions.env is NOT an input (it only RECORDS the published digest), and must not
 # become one: Renovate writes it after every publish, and an input there would loop.
 inputs() {
  case "$1" in
    toolbox)            echo "infra/devcontainer/Containerfile infra/devcontainer/versions.env infra/devcontainer/build-image.sh" ;;
    agent-runner)       echo "infra/agent/" ;;
    conformance-runner) echo "conformance/incluster/Containerfile conformance/incluster/conformance-clone conformance/incluster/build-image.sh infra/devcontainer/versions.env" ;;
  esac
 }
 PIPELINE_FILES=".gitea/workflows/images.yaml .gitea/workflows/scripts/images.sh"
 declare -A WANT=() BUILT=() DIGEST=() SECS=() NOTE=()
 changed_any() { # <paths...> -> 0 if BASE_SHA..HEAD_SHA touches any
  # shellcheck disable=SC2086
  [ -n "$(git diff --name-only "${BASE_SHA}" "${HEAD_SHA}" -- "$@")" ]
 }
 select_images() {
  if [ "${IMAGES}" != "auto" ]; then
    for i in ${IMAGES}; do
      case " ${ALL_IMAGES[*]} " in *" ${i} "*) WANT[$i]=1; NOTE[$i]="requested" ;; *) die "unknown image '${i}'" ;; esac
    done
    return
  fi
  if [ -z "${BASE_SHA}" ] || [[ "${BASE_SHA}" =~ ^0+$ ]] || ! git cat-file -e "${BASE_SHA}^{commit}" 2>/dev/null; then
    info "no usable diff base ('${BASE_SHA}'): building everything"
    for i in "${ALL_IMAGES[@]}"; do WANT[$i]=1; NOTE[$i]="no diff base"; done
    return
  fi
  info "diff base ${BASE_SHA} .. ${HEAD_SHA}"
  local i
  for i in "${ALL_IMAGES[@]}"; do
    # shellcheck disable=SC2046
    if changed_any $(inputs "$i"); then WANT[$i]=1; NOTE[$i]="inputs changed"; fi
  done
  # shellcheck disable=SC2086
  if [ "${MODE}" = verify ] && changed_any ${PIPELINE_FILES}; then
    for i in "${ALL_IMAGES[@]}"; do [ -n "${WANT[$i]:-}" ] || { WANT[$i]=1; NOTE[$i]="pipeline changed"; }; done
  fi
 }
 # --- registry helpers (publish mode only) ----------------------------------------------------
 REG_TOKEN=""
 registry_token() {
  [ -n "${REG_TOKEN}" ] && return 0
  REG_TOKEN="$(curl -fsS --config <(printf 'user = "%s:%s"\n' "${REGISTRY_USER}" "${REGISTRY_TOKEN}") \
    "https://${REGISTRY}/v2/token?service=container_registry&scope=*" | jq -r '.token // empty')" \
    || die "could not obtain a registry bearer token from https://${REGISTRY}/v2/token"
  [ -n "${REG_TOKEN}" ] || die "registry returned no bearer token"
 }
 # Prints the manifest digest the REGISTRY serves for <image>:<tag>; nothing if it is absent (404).
 # Any other answer (401, 5xx, no digest header) is fatal: "cannot tell" must never be read as
 # "not published", or the immutable-tag guard would be bypassed by an auth hiccup.
 registry_digest() {
  registry_token
  local hdrs code digest
  hdrs="$(curl -sS -I --config <(printf 'header = "Authorization: Bearer %s"\n' "${REG_TOKEN}") \
    -H 'Accept: application/vnd.oci.image.manifest.v1+json' \
    -H 'Accept: application/vnd.docker.distribution.manifest.v2+json' \
    -H 'Accept: application/vnd.oci.image.index.v1+json' \
    -H 'Accept: application/vnd.docker.distribution.manifest.list.v2+json' \
    "https://${REGISTRY}/v2/${NAMESPACE}/$1/manifests/$2" | tr -d '\r')" \
    || die "registry unreachable while reading $1:$2"
  code="$(awk 'NR==1{print $2}' <<<"${hdrs}")"
  digest="$(awk -F': ' 'tolower($1)=="docker-content-digest"{print $2}' <<<"${hdrs}" | tail -1)"
  case "${code}" in
    200) [[ "${digest}" =~ ^sha256:[0-9a-f]{64}$ ]] || die "registry answered 200 for $1:$2 with no usable digest"
         echo "${digest}" ;;
    404) ;;
    *)   die "registry answered HTTP ${code} for $1:$2 — cannot tell whether it exists" ;;
  esac
 }
 local_ref() { echo "${REGISTRY}/${NAMESPACE}/$1:ci-${HEAD_SHA:0:12}"; }
 build_one() { # <image> [agent base]
  local img="$1" base="${2:-}" t0 tag
  tag="ci-${HEAD_SHA:0:12}"
  t0=$(date +%s)
  log "BUILD + VERIFY ${img}"
  case "${img}" in
    toolbox)
      ENGINE="${ENGINE}" IMAGE="$(local_ref toolbox)" BUILD_REVISION="${HEAD_SHA}" \
        ./infra/devcontainer/build-image.sh ;;
    agent-runner)
      ENGINE="${ENGINE}" IMAGE_TAG="${tag}" AGENT_BASE_IMAGE="${base}" BUILD_REVISION="${HEAD_SHA}" \
        ./infra/agent/build-image.sh ;;
    conformance-runner)
      ENGINE="${ENGINE}" IMAGE_TAG="${tag}" BUILD_REVISION="${HEAD_SHA}" \
        ./conformance/incluster/build-image.sh ;;
  esac
  BUILT[$img]=1
  SECS[$img]=$(( $(date +%s) - t0 ))
  info "${img}: built and verified in ${SECS[$img]}s"
 }
 push_one() { # <image> -> sets DIGEST[image] to the registry-returned digest
  local img="$1" repo src existing pushed got t0
  repo="${REGISTRY}/${NAMESPACE}/${img}"
  src="$(local_ref "${img}")"
  t0=$(date +%s)
  log "PUSH ${img} -> ${repo}:{${SHA_TAG},${MOVING_TAG}}"
  existing="$(registry_digest "${img}" "${SHA_TAG}")"
  if [ -n "${existing}" ]; then
    # Immutable tag: never overwrite. A rebuild is not byte-reproducible, so re-pushing would
    # silently change what `sha-<commit>` means after something may already have pinned it.
    die "${repo}:${SHA_TAG} already exists (${existing}). Tags here are immutable; this run must not
       replace it. (publish skips images that already exist at this commit BEFORE building them,
       so reaching this line means the tag appeared mid-run.)"
  fi
  "${ENGINE}" tag "${src}" "${repo}:${SHA_TAG}"
  "${ENGINE}" tag "${src}" "${repo}:${MOVING_TAG}"
  pushed="$("${ENGINE}" push "${repo}:${SHA_TAG}" | tee /dev/stderr | grep -oE 'digest: sha256:[0-9a-f]{64}' | tail -1 | cut -d' ' -f2)"
  "${ENGINE}" push "${repo}:${MOVING_TAG}" >/dev/null
  # The AUTHORITATIVE digest is what the registry serves, read back — not what the client
  # printed. They should agree; if they do not, nothing downstream may trust either.
  got="$(registry_digest "${img}" "${SHA_TAG}")"
  [ -n "${got}" ] || die "pushed ${repo}:${SHA_TAG} but the registry does not serve it"
  [ -z "${pushed}" ] || [ "${pushed}" = "${got}" ] \
    || die "${img}: client reported ${pushed} but the registry serves ${got}"
  [ "$(registry_digest "${img}" "${MOVING_TAG}")" = "${got}" ] \
    || die "${img}: ${MOVING_TAG} does not point at the digest just pushed (${got})"
  DIGEST[$img]="${got}"
  SECS[$img]=$(( ${SECS[$img]:-0} + $(date +%s) - t0 ))
  info "${img}: registry digest ${got}"
 }
 # --- main ---------------------------------------------------------------------------------
 log "image pipeline: MODE=${MODE} IMAGES=${IMAGES} HEAD=${HEAD_SHA}"
 command -v "${ENGINE}" >/dev/null || die "${ENGINE} is not on PATH in the job container"
 "${ENGINE}" info >/dev/null 2>&1 \
  || die "the ${ENGINE} daemon is not reachable from the job container — the runner did not
       hand it a Docker socket (act_runner container.docker_host). Nothing can be built."
 "${ENGINE}" version 2>/dev/null | sed -n '1,12p' || true
 "${ENGINE}" system df || true
 if [ "${MODE}" = publish ]; then
  : "${REGISTRY_TOKEN:?REGISTRY_TOKEN is required in publish mode}"
  REGISTRY_USER="${REGISTRY_USER:-ci-image-publisher}"
  printf '%s' "${REGISTRY_TOKEN}" | "${ENGINE}" login "${REGISTRY}" -u "${REGISTRY_USER}" --password-stdin >/dev/null \
    || die "docker login ${REGISTRY} failed — check REGISTRY_PUSH_TOKEN (scope write:package, owner has package write on the org)"
 fi
 select_images
 # Publish idempotence: an image already published at THIS commit is not rebuilt. This is what
 # makes a re-run after a half-finished publish safe — and it is the only way a rerun can
 # finish, because the immutable sha tag must never be pushed twice.
 if [ "${MODE}" = publish ]; then
  for i in "${ALL_IMAGES[@]}"; do
    [ -n "${WANT[$i]:-}" ] || continue
    d="$(registry_digest "${i}" "${SHA_TAG}")"
    if [ -n "${d}" ]; then
      info "${i}: already published at ${SHA_TAG} (${d}); not rebuilding"
      unset "WANT[$i]"; DIGEST[$i]="${d}"; NOTE[$i]="already published at this commit"
    fi
  done
 fi
 # Decide the agent-runner base (see header).
 AGENT_BASE=""
 if [ -n "${WANT[agent-runner]:-}" ]; then
  # shellcheck disable=SC1091
  PINNED_BASE="$(. ./infra/agent/versions.env; echo "${AGENT_BASE_IMAGE}")"
  if [ "${MODE}" = verify ] && [ -z "${WANT[toolbox]:-}" ]; then
    WANT[toolbox]=1; NOTE[toolbox]="built as agent-runner's base (verify only)"
  elif [ "${MODE}" = publish ] && [ -z "${WANT[toolbox]:-}" ] && [ -z "${DIGEST[toolbox]:-}" ]; then
    case "${PINNED_BASE}" in
      *@"${ZERO_DIGEST}") WANT[toolbox]=1; NOTE[toolbox]="never published; required as agent-runner's base" ;;
      *@sha256:*)         AGENT_BASE="${PINNED_BASE}" ;;
      *) die "infra/agent/versions.env AGENT_BASE_IMAGE='${PINNED_BASE}' is not digest-pinned; CI will not build on a floating base" ;;
    esac
  fi
 fi
 yn() { if [ -n "$1" ]; then echo "$2"; else echo "$3"; fi; }
 for i in "${ALL_IMAGES[@]}"; do info "plan: ${i}: $(yn "${WANT[$i]:-}" BUILD skip)  (${NOTE[$i]:-inputs unchanged})"; done
 if [ -n "${WANT[toolbox]:-}" ]; then
  build_one toolbox
  if [ "${MODE}" = publish ]; then push_one toolbox; fi
 fi
 if [ -n "${WANT[agent-runner]:-}" ]; then
  if [ -z "${AGENT_BASE}" ]; then
    if [ "${MODE}" = publish ]; then
      AGENT_BASE="${REGISTRY}/${NAMESPACE}/toolbox@${DIGEST[toolbox]:?no toolbox digest to build agent-runner from}"
    else
      AGENT_BASE="$(local_ref toolbox)"
    fi
  fi
  info "agent-runner base: ${AGENT_BASE}"
  build_one agent-runner "${AGENT_BASE}"
  if [ "${MODE}" = publish ]; then push_one agent-runner; fi
 fi
 if [ -n "${WANT[conformance-runner]:-}" ]; then
  build_one conformance-runner
  if [ "${MODE}" = publish ]; then push_one conformance-runner; fi
 fi
 # --- summary ------------------------------------------------------------------------------
 {
  echo "## Images (${MODE})"
  echo
  echo "commit \`${HEAD_SHA}\`"
  echo
  echo "| image | built + verified | seconds | published digest (registry-returned) | note |"
  echo "|---|---|---|---|---|"
  for i in "${ALL_IMAGES[@]}"; do
    d="${DIGEST[$i]:-}"
    echo "| \`${i}\` | $(yn "${BUILT[$i]:-}" yes no) | ${SECS[$i]:--} | $(yn "${d}" "\`${REGISTRY}/${NAMESPACE}/${i}@${d}\`" -) | ${NOTE[$i]:-inputs unchanged} |"
  done
  echo
  if [ "${MODE}" = publish ]; then
    echo "Tags pushed: \`${SHA_TAG}\` (immutable) and \`${MOVING_TAG}\` (moving)."
    echo "Nothing here writes to git. Renovate opens the digest-bump PR on its next run."
  else
    echo "Verify mode: nothing was pushed and no registry credential was present."
  fi
 } >>"${SUMMARY}"
 log "done"
@@ -61,10 +61,13 @@ credentials — the Job's own scoped ServiceAccount does (R-SEC-02).
  > digest-pinned to the conformance-runner image, `command` exactly
  > `/usr/local/bin/conformance-clone`, and **no `args` at all**. `ephemeralContainers` and
  > `env`/`envFrom` were the same shape of hole and are closed in the same policy.
- `build-image.sh` — builds the above, **verifies the toolchain by running it** inside the
+- `build-image.sh` — builds the above and **verifies the toolchain by running it** inside the
-  image as uid 65532 with `--read-only`, and prints the digest. It **never pushes**: it
+  image as uid 65532 with `--read-only`. It **never pushes**, and neither does anyone from a
-  prints the exact `podman push --digestfile …` command for the operator instead. See
+  workstation: CI (`.gitea/workflows/images.yaml`) runs this same script, and on a push to
-  `conformance/versions.env` `CONFORMANCE_RUNNER_IMAGE` for the post-push pin.
+  `main` publishes the verified image under `sha-<commit>` and `main`. Renovate then moves
  `CONFORMANCE_RUNNER_IMAGE_DIGEST` in `conformance/versions.env` onto the registry-returned
  digest in a pull request (that file is a sensitive path, so the bump needs an
  administrator's `/approve-override`).
  `--verify-clone` additionally has `conformance-clone` really clone this repo at a real SHA
  and asserts the checkout matches (opt-in, because it needs the Gitea host reachable — the
  default verify stays offline so an air-gapped build is fully checked, not half-checked).
@@ -1,37 +1,41 @@
 #!/usr/bin/env bash
 ###############################################################################
 # conformance/incluster/build-image.sh — build the pinned `conformance-runner`
-# image and print its digest. Closes docs/plans/l2-l3-conformance-ci-wiring.md §3 item 1.
+# image and prove it works. Closes docs/plans/l2-l3-conformance-ci-wiring.md §3 item 1.
 #
-#   ./build-image.sh                 # build + verify + print the digest and push command
+#   ./build-image.sh                 # build + verify + print the image id
 #   ./build-image.sh --no-verify     # build only (skip the in-image tool proof)
 #   ./build-image.sh --verify-clone  # ALSO really clone this repo with conformance-clone.
 #                                    #   Opt-in because it needs the Gitea host reachable; the
 #                                    #   default verify is offline so an air-gapped build is
 #                                    #   still fully checked rather than half-checked.
 #   ENGINE=docker ./build-image.sh   # build with docker instead of podman
 #   IMAGE_TAG=foo ./build-image.sh   # override the local tag (CI uses this)
 #
-# THIS SCRIPT NEVER PUSHES. Publishing is an operator action: the registry credential is
+# THIS SCRIPT NEVER PUSHES — and nobody pushes this image from a workstation either.
-# not an agent's to hold, and the pushed digest is what a fail-closed admission policy
+# Publishing is done by CI: .gitea/workflows/images.yaml calls this script, and on a push to
-# then trusts. So this builds locally, proves the toolchain works INSIDE the image as the
+# `main` it pushes what this script built AND verified to the Gitea registry, records the
-# non-root uid the policy demands, and prints the exact `podman push` line for a human.
+# registry-returned digest in the job summary, and Renovate then opens the pull request that
 # moves conformance/versions.env onto that digest. So this script's job is to build and to
 # prove the toolchain works INSIDE the image as the non-root uid the policy demands — the same
 # proof locally and in CI. It runs under ENGINE=docker in CI (Docker-in-Docker), so nothing
 # below may be podman-only.
 #
 # VERSIONS ARE NOT DEFINED HERE. They are read from infra/devcontainer/versions.env, the
 # single source of truth (CHAINSAW_VERSION, KUBECTL_VERSION, FEDORA_VERSION) — the same
 # pins the toolbox gate runs with. Nothing in this file invents a version.
 #
-# THE DIGEST CAVEAT — read it before pasting a digest into anything.
+# NO DIGEST IS PRINTED. A manifest digest does not exist until a registry holds the manifest,
-# The digest printed here is the LOCAL manifest digest. It is normally identical to the
+# and the only digest anything may pin is the one the REGISTRY returns. CI captures that at
-# registry digest (podman pushes the manifest it already has rather than recompressing),
+# push time (.gitea/workflows/scripts/images.sh) and Renovate carries it into
-# but that is a property of the push, not a guarantee. So the push command below uses
+# conformance/versions.env.
 # `--digestfile`: the digest the registry actually returns is the authoritative one, and
 # that is the one that goes into conformance/versions.env.
 ###############################################################################
 set -euo pipefail
 HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 REPO_ROOT="$(cd "${HERE}/../.." && pwd)"
 ENGINE="${ENGINE:-podman}"
 TAG_OVERRIDE="${IMAGE_TAG:-}"
 VERIFY=1
 VERIFY_CLONE=0
@@ -64,9 +68,9 @@ source "${TOOLCHAIN_ENV}"
 # gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:*. The repository path
 # is therefore not a preference; a different path is a DENIED Job. Do not "improve" it here.
 IMAGE_REPO="gitea.home.keithjackson.dev/home-network/conformance-runner"
-# The tag is build convenience only (a human-readable handle for the push). Admission
+# The tag is a local handle only. CI re-tags what it publishes (git SHA + `main`), and
-# ignores it: only the digest is ever referenced from a Job.
+# admission ignores tags entirely: only the digest is ever referenced from a Job.
-IMAGE_TAG="chainsaw${CHAINSAW_VERSION}-kubectl${KUBECTL_VERSION}-fc${FEDORA_VERSION}"
+IMAGE_TAG="${TAG_OVERRIDE:-chainsaw${CHAINSAW_VERSION}-kubectl${KUBECTL_VERSION}-fc${FEDORA_VERSION}}"
 IMAGE_REF="${IMAGE_REPO}:${IMAGE_TAG}"
 log "Building ${IMAGE_REF}"
@@ -76,7 +80,11 @@ info "  CHAINSAW        ${CHAINSAW_VERSION}"
 info "  KUBECTL         ${KUBECTL_VERSION}"
 info "  FEDORA          ${FEDORA_VERSION}"
-"${ENGINE}" build \
+# Provenance: CI passes the commit it is building, so a pulled image names its source tree.
 LABEL_ARGS=()
 [ -z "${BUILD_REVISION:-}" ] || LABEL_ARGS+=(--label "org.opencontainers.image.revision=${BUILD_REVISION}")
 "${ENGINE}" build "${LABEL_ARGS[@]}" \
  --file "${HERE}/Containerfile" \
  --tag "${IMAGE_REF}" \
  --build-arg "CHAINSAW_VERSION=${CHAINSAW_VERSION}" \
@@ -184,46 +192,12 @@ if [ "${VERIFY_CLONE}" -eq 1 ]; then
  log "conformance-clone verified against the real repository"
 fi
-# --- digest ------------------------------------------------------------------
+# --- result ------------------------------------------------------------------
 DIGEST="$("${ENGINE}" image inspect --format '{{.Digest}}' "${IMAGE_REF}")"
 IMAGE_ID="$("${ENGINE}" image inspect --format '{{.Id}}' "${IMAGE_REF}")"
 SIZE="$("${ENGINE}" image inspect --format '{{.Size}}' "${IMAGE_REF}")"
 log "Built ${IMAGE_REF}"
-info "local manifest digest  ${DIGEST}"
+info "image id  ${IMAGE_ID}"
-info "image id               ${IMAGE_ID}"
+info "size      $(( SIZE / 1024 / 1024 )) MiB"
-info "size                   $(( SIZE / 1024 / 1024 )) MiB"
+info "Not pushed. CI publishes this image from main (.gitea/workflows/images.yaml) and"
-
+info "Renovate moves conformance/versions.env onto the registry-returned digest."
 cat <<EOF
 ================================================================================
 NOT PUSHED. Publishing is an operator action — this script holds no registry
 credential and will never acquire one.
 OPERATOR: run these two commands, then paste the digest from the digestfile into
 conformance/versions.env (CONFORMANCE_RUNNER_IMAGE) via a PR:
    ${ENGINE} login gitea.home.keithjackson.dev
    ${ENGINE} push --digestfile /tmp/conformance-runner.digest \\
      ${IMAGE_REF}
 Then:
    cat /tmp/conformance-runner.digest
    # -> sha256:....  THIS is the authoritative digest, not the local one above.
    # conformance/versions.env
    CONFORMANCE_RUNNER_IMAGE="${IMAGE_REPO}@\$(cat /tmp/conformance-runner.digest)"
 WHY THE DIGEST AND NOT THE TAG: platform/kyverno/admission/conformance-trigger-job-shape.yaml
 is a fail-closed Enforce policy. It DENIES any conformance-trigger Job whose image is not
 ${IMAGE_REPO}@sha256:<digest>. A tag reference is denied,
 and so is any other repository path.
 TIGHTEN THE BOUNDARY AFTERWARDS: that policy's header asks for the wildcard
 \`${IMAGE_REPO}@sha256:*\` to be replaced with the
 LITERAL digest once the image exists, closing the "anyone who can push to this path" gap.
 That is a separate, human-reviewed PR against the security boundary.
 ================================================================================
 EOF
@@ -29,7 +29,8 @@
 # THEN  (1) promtool is a PINNED member of the toolchain: PROMTOOL_VERSION is declared in BOTH
 #           infra/devcontainer/versions.env and ci/versions.env and the two AGREE (the repo's
 #           lockstep rule), the Containerfile takes it as a build ARG, installs it from the
-#           pinned prometheus release and records it for provenance, dev.sh passes it, and the
+#           pinned prometheus release and records it for provenance, build-image.sh (the one
 #           recipe `dev.sh build` and CI both use) passes it, and the
 #           CI runner installs it for the l0 stage;
 #       (2) the promtool ACTUALLY IN USE reports exactly that pinned version — a unit test is
 #           evidence only if the engine matches the one prod evaluates rules with;
@@ -246,10 +247,12 @@ check('echo "promtool=${PROMTOOL_VERSION}"' in cf_text,
      "infra/devcontainer/Containerfile does not record promtool in /etc/dev-toolchain-versions — "
      "that provenance line is how an operator tells a stale image from a current one")
-DEVSH = ROOT / "infra/devcontainer/dev.sh"
+# The build recipe moved from dev.sh into build-image.sh on 2026-10-29, so that `dev.sh build`
-check(DEVSH.is_file() and "--build-arg PROMTOOL_VERSION=" in DEVSH.read_text(),
+# and the CI image pipeline (.gitea/workflows/images.yaml) share ONE recipe. Check the recipe.
-      "infra/devcontainer/dev.sh does not pass --build-arg PROMTOOL_VERSION — the ARG would be "
+BUILDSH = ROOT / "infra/devcontainer/build-image.sh"
-      "empty and the build would fetch a bogus URL")
+check(BUILDSH.is_file() and "--build-arg PROMTOOL_VERSION=" in BUILDSH.read_text(),
      "infra/devcontainer/build-image.sh does not pass --build-arg PROMTOOL_VERSION — the ARG "
      "would be empty and the build would fetch a bogus URL")
 INSTALL = ROOT / ".gitea/workflows/scripts/install-tools.sh"
 if INSTALL.is_file():
@@ -40,32 +40,29 @@ PYTHON_MIN="3.9"
 #   gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:*
 # Any other path, and any TAG reference, is DENIED at admission. Do not "tidy" these values.
 CONFORMANCE_RUNNER_IMAGE_REPO="gitea.home.keithjackson.dev/home-network/conformance-runner"
-# Build handle only. Admission never sees the tag; a Job must reference the digest.
+# Local build handle only. Admission never sees the tag; a Job must reference the digest.
 # CI publishes under `sha-<commit>` and `main` instead (.gitea/workflows/images.yaml).
 CONFORMANCE_RUNNER_IMAGE_TAG="chainsaw0.2.12-kubectl1.36.4-fc44"
 #
 # ############################################################################
-# # DIGEST DELIBERATELY EMPTY — THE IMAGE HAS NOT BEEN PUBLISHED YET.        #
+# # THE DIGEST IS WRITTEN BY RENOVATE, NOT BY HAND.                          #
 # ############################################################################
-# The image has been BUILT and its toolchain verified locally (non-root uid 65532,
+# CI builds and verifies the image (non-root uid 65532, read-only rootfs, the
-# read-only rootfs: bash 5.3.9, Python 3.14.7, git 2.55.0, kubectl v1.36.4, chainsaw 0.2.12,
+# conformance-clone refusals) and, on a push to main, publishes it and prints the digest the
-# and `./run.sh --layer l2 --no-cluster` completing with rc 0). It has NOT been pushed:
+# REGISTRY returned (.gitea/workflows/images.yaml). Renovate follows the image's `main` tag
-# pushing needs a registry credential, and publishing the artifact a fail-closed security
+# through the marker below and opens the pull request that moves this value. Nobody pushes
-# boundary then trusts is an OPERATOR action, not an agent's.
+# this image from a workstation, and nobody pastes a digest here.
 #
-# OPERATOR, to complete the pin:
+# ALL-ZERO UNTIL THE FIRST PUBLISH. Before 2026-10-29 this was EMPTY, so that a consumer
-#   1. ./conformance/incluster/build-image.sh          # rebuild, verify, print the commands
+# rendering the Job would fail visibly. It is now the same impossible all-zero digest
-#   2. podman login gitea.home.keithjackson.dev
+# platform/pm-agent/ uses, because Renovate can only replace a digest that is there to match.
-#   3. podman push --digestfile /tmp/conformance-runner.digest \
+# A consumer MUST treat the all-zero value as "not published" (no consumer exists yet: the
-#        "${CONFORMANCE_RUNNER_IMAGE_REPO}:${CONFORMANCE_RUNNER_IMAGE_TAG}"
+# L2/L3 trigger is unwired, docs/plans/l2-l3-conformance-ci-wiring.md). Note also that
-#   4. Set CONFORMANCE_RUNNER_IMAGE_DIGEST below to `cat /tmp/conformance-runner.digest`
+# chainsaw-job.yaml's @@CONFORMANCE_RUNNER_IMAGE_DIGEST@@ takes the 64 hex digits WITHOUT the
-#      — the digest the REGISTRY returned, not the local manifest digest the build printed.
+# `sha256:` prefix this value carries — the trigger must strip it.
-#   5. Land steps 4 in a PR (this file is GitOps input, not a live edit).
+# renovate-digest: image=gitea.home.keithjackson.dev/home-network/conformance-runner tag=main
-#
+CONFORMANCE_RUNNER_IMAGE_DIGEST="sha256:0000000000000000000000000000000000000000000000000000000000000000"
-# UNTIL THEN: this is empty on purpose, NOT a placeholder to be guessed at. An empty value
+CONFORMANCE_RUNNER_IMAGE="${CONFORMANCE_RUNNER_IMAGE_REPO}@${CONFORMANCE_RUNNER_IMAGE_DIGEST}"
 # makes any consumer that renders the Job fail visibly; a fake or stale digest would instead
 # produce an ImagePullBackOff that looks like a cluster fault. Do not invent one.
 CONFORMANCE_RUNNER_IMAGE_DIGEST=""
 CONFORMANCE_RUNNER_IMAGE="${CONFORMANCE_RUNNER_IMAGE_DIGEST:+${CONFORMANCE_RUNNER_IMAGE_REPO}@${CONFORMANCE_RUNNER_IMAGE_DIGEST}}"
 #
 # AFTERWARDS, TIGHTEN THE BOUNDARY: the header of conformance-trigger-job-shape.yaml asks for
 # its `@sha256:*` wildcard to be replaced with the LITERAL digest once the image exists —
@@ -0,0 +1,222 @@
 ---
 date: 2026-10-29
 branch: feat/image-pipeline
 title: "Images are built, verified and published by CI; Renovate carries the digests into git"
 ---
 ## What changed
 Operator rule (2026-10-29): container image creation, hosting and lifecycle belong to CI/CD and
 the codebase, never to a one-off workstation build/push. Before this branch the Home-Network org's
 package list was **empty**, `platform/pm-agent/` pinned an all-zero digest (ImagePullBackOff), and
 both `build-image.sh` scripts said "publishing is an operator action".
 - **`.gitea/workflows/images.yaml`** + **`.gitea/workflows/scripts/images.sh`** (the logic).
  - Pull request: build and verify only. No push, and the step references no secret at all.
  - Push to `main`, or `workflow_dispatch` on `main`: build, verify, then push to
    `gitea.home.keithjackson.dev/home-network/{toolbox,agent-runner,conformance-runner}`.
    Tags are `sha-<12 hex>` (immutable: the driver refuses to overwrite one) and `main` (moving).
    The digest is read back from the registry with a HEAD on the manifest, compared with what the
    client printed, and written to the job summary.
  - Only images whose inputs changed are built. A re-run on the same commit skips images that are
    already published.
  - agent-runner builds FROM the toolbox built in the same run (on main, FROM its just-pushed
    digest). Otherwise:
    - in verify mode the toolbox is built anyway as its base, because a PR holds no credential;
    - in publish mode it uses `AGENT_BASE_IMAGE` from `infra/agent/versions.env`;
    - if that is still the zero placeholder, the toolbox is built and published first. This is
      how the first run on main bootstraps.
  - An always-run step prunes the DinD image store.
 - **One build recipe per image, reused by CI.**
  - New `infra/devcontainer/build-image.sh`. `dev.sh build` now delegates to it. It has an
    in-image verification step that the toolbox never had: every tool is on PATH, and
    promtool, kubectl, chainsaw, kyverno and sops report their pins.
  - `infra/agent/build-image.sh` and `conformance/incluster/build-image.sh` were made
    engine-neutral, because CI runs `ENGINE=docker` against DinD:
    - `podman image exists` became `image inspect`, with a pull fallback;
    - `{{.Digest}}` and `--digestfile` are gone. No local digest is printed, since only the
      registry's digest is meaningful;
    - `probe-validation.sh` is fed over **stdin** instead of a bind mount. Under DinD a bind
      source is resolved on the daemon's filesystem, which would silently mount an empty
      directory;
    - `probe-validation.sh` itself gained `${BASH_SOURCE[0]:-$0}` so it survives `bash -s`
      under `set -u`.
  - `AGENT_BASE_IMAGE` and `IMAGE_TAG` can be overridden from the environment, and
    `BUILD_REVISION` adds an `org.opencontainers.image.revision` label.
  - The "publishing is an operator action" text was replaced throughout.
 - **Digest references.** Each now carries a `# renovate-digest: image=<repo> tag=main` marker
  on the line above it:
  - `platform/pm-agent/deployment.yaml` and `configmap.yaml` (`PM_WORKER_IMAGE`), which share
    one dependency, so they bump in one PR;
  - `infra/agent/versions.env` `AGENT_BASE_IMAGE`, which was `home-k3s-dev:local` and is now
    the published toolbox by digest (zero placeholder until the first publish);
  - `conformance/versions.env` `CONFORMANCE_RUNNER_IMAGE_DIGEST`, which went from `""` to the
    zero placeholder because Renovate can only replace a digest it can match.
  The admission policies are **untouched**; the `<repo>@sha256:<digest>` form is preserved.
 - **`renovate.json5`**:
  - a new `custom.regex` manager for the markers;
  - `enabledManagers` `regex` → `custom.regex`;
  - a packageRule that disables the `kubernetes` manager for these three repos. Otherwise it
    would also see the digest-only lines and look them up as `latest`;
  - an `own-image` label;
  - `ignorePaths` narrowed from `conformance/**` to its subdirectories. `ignorePaths` applies
    to custom managers too, so the blanket glob hid `conformance/versions.env`.
 - **`renovate.yaml`**: `renovate-version: 44.167.0` pinned, plus a `RENOVATE_HOST_RULES` docker
  rule for the Gitea registry, using the same `RENOVATE_TOKEN`, which now also needs
  `read:package`. See below for why the version pin is a real fix and not tidiness.
 - **`infra/gitea/registry-retention.sh`**: dry run by default, `--apply` to delete. This is an
  operator action. Per image it keeps:
  - `main` and `latest`;
  - the 10 newest `sha-` tags;
  - every tag whose digest is pinned anywhere in `origin/main`;
  - every digest-named version.
 - **`infra/gitea/conformance-trigger-kubeconfig.sh`**: the "runner image is digest-pinned"
  precondition grepped the literal line `CONFORMANCE_RUNNER_IMAGE="${…_REPO}@${…_DIGEST}"`
  for 64 hex digits, so it **could never pass**. It now sources the file, evaluates the value,
  and rejects the zero placeholder.
 - **`conformance/tests/l0-static/alert-rules-unit-tested/check.sh`**: its "the toolbox build
  passes `--build-arg PROMTOOL_VERSION`" assertion now reads `infra/devcontainer/build-image.sh`
  instead of `dev.sh`. The gate caught this on the first run, because the recipe had moved.
 - **Docs**: `platform/pm-agent/README.md`, `runbooks/pm-agent.md` (two rows),
  `docs/plans/pm-agent-issue-to-pr.md` (follow-up 1), `conformance/incluster/README.md`,
  `infra/devcontainer/README.md`, and a new "How images are built" section in
  `.gitea/workflows/README.md`.
 ### Decisions and pushback
 - **The push credential is a PAT secret, `REGISTRY_PUSH_TOKEN`; the Actions token cannot push.**
  Checked against Gitea **v1.24.3** source:
  - `services/auth/basic.go` `Basic.Verify` turns a task token into the pseudo-user
    `gitea-actions` (ID −2) and sets no token scope;
  - `routers/api/packages/container/container.go` `Authenticate` (the `/v2/token` exchange every
    docker client performs) requires `read:package`/`write:package`/`all` scope, so it answers
    401. The Actions token cannot even *pull*;
  - `services/context/package.go` `determineAccessMode` has `// TODO: ActionUser permission check`
    and no branch for that user. Even past login it would get at most read on a public org;
  - `models/repo/repo_unit.go` `ActionsConfig` has no token-permissions setting in 1.24.3.
  The workflow fails loudly when the secret is absent, copying the `RENOVATE_TOKEN` preflight
  pattern.
 - **`regex` vs `custom.regex`: both work. Switched to the canonical `custom.regex`.** In Renovate:
  - `lib/config/migrations/custom/enabled-managers-migration.ts` rewrites `regex` →
    `custom.regex`;
  - that migration runs on every repository run (`lib/workers/repository/init/merge.ts` →
    `migrateAndValidate`), not only in the validator;
  - `lib/modules/manager/index.ts` `getEnabledManagersList` accepts either spelling;
  - `custom.regex` exists since 37.51.1 (renovatebot/renovate#24079).
 - **The open question hid a bigger bug: the Renovate config on main is invalid for the Renovate
  the workflow actually runs.**
  - `renovatebot/[email protected]` defaults `renovate-version` to `39`
    (`action.yml`, `src/docker.ts`), which is 39.264.0.
  - The validator used in #79 was `ghcr.io/renovatebot/renovate` with no tag, i.e. `latest`,
    44.167.0.
  - **Measured:** `renovate:39.264.0 renovate-config-validator` on main's `renovate.json5` gives
    `ERROR: Found errors in configuration` with five errors, all `managerFilePatterns`.
    `fileMatch` was renamed in 40.2.0.
  - So once #80 fixes the filename, every run would still die at `CONFIG_VALIDATION`.
  - Fixed by pinning `renovate-version: 44.167.0`. This branch's config validates clean on it.
    The only `--strict` complaint is a pending migration of the pre-existing top-level
    `"docker": {"enabled": true}` block, which I left alone.
 - **Unworkable as briefed: tracking the worker Job template's digest.**
  - `infra/agent/worker-job-template.yaml` is `COPY`ed into agent-runner. A Renovate bump there
    changes agent-runner's inputs, CI publishes a new digest, Renovate bumps it again, and so on
    forever.
  - `pm-dispatch` overwrites that field from `PM_WORKER_IMAGE` at runtime
    (`pm-dispatch.py:649`), so nothing is lost by leaving it all-zero. It is commented as
    deliberately untracked.
  - For the same loop reason, `conformance/versions.env` is **not** an input of the
    conformance-runner build, and a pipeline-only change does not republish on main.
 - **Changed from the brief: retention is a script, not a Gitea cleanup rule.**
  - Gitea 1.24.3 has **no API** for cleanup rules (none in `routers/api/v1/api.go`; web
    session+CSRF forms only, in `routers/web/web.go`). So a rule cannot be created, or
    drift-asserted, as code. **`ci/gitea-settings.sh` was therefore not extended.** A cheaper
    proxy (count versions per image) would need `read:package` on `SETTINGS_API_TOKEN`, which
    it lacks.
  - A built-in rule is also **unsafe here**. Gitea stores a manifest as a file of the TAG
    version it was pushed under (`routers/api/packages/container/manifest.go`
    `createPackageAndVersion`), and resolves a by-digest pull from those files
    (`models/packages/container/search.go`). Deleting the tag a deployed digest was pushed under
    breaks the `@sha256:` pull, and the rule's "newest N" knows nothing about what git pins. The
    script keeps every digest git pins.
 - **Auto-merge.** Digest bumps of these images auto-merge on a green gate (the existing
  `digest` rule). The exception is the conformance-runner bump: `conformance/versions.env` is a
  sensitive path, so it waits for an administrator's `/approve-override` every time.
 ## What I verified
 - **Local gate, pinned toolbox, from this worktree, committed HEAD.** Command:
  `ENGINE=<shim> DEV_KUBECONFIG=/dev/null ./infra/devcontainer/dev.sh run bash -c './ci/gate.sh'`.
  The shim adds the `.git` mount. Result: see the PR body for the rc and stage list. They are
  recorded there rather than here because this entry is part of the tree being gated.
 - **Local podman builds through the CI driver**, run as
  `MODE=verify ENGINE=podman BASE_SHA=$(git merge-base origin/main HEAD) .gitea/workflows/scripts/images.sh`.
  It returned rc 0 and built and verified all three images:
  - toolbox: every tool on PATH, and the five pins matched;
  - agent-runner: `PM_SELFTEST=1 pm-dispatch` OK, and the stdin-fed `probe-validation.sh`
    gave "20 refusal(s), 0 failure(s)";
  - conformance-runner: all seven `conformance-clone` refusals.
 - **Other driver paths, run locally:**
  - agent-runner alone builds the toolbox as its base;
  - an empty diff builds nothing;
  - publish mode without a token dies before doing anything;
  - `infra/agent/build-image.sh` with the zero placeholder dies with build-locally
    instructions.
 - **Renovate, with the pinned 44.167.0 image, locally:**
  - `renovate-config-validator` exits 0;
  - a `--platform=local` `extract` dry run finds all four marker references (custom regex manager,
    `currentValue: main`, `currentDigest: sha256:000…`);
  - a `lookup` dry run shows the `kubernetes`-manager duplicates as `skipReason: disabled` and the
    marker deps taking the "unsupported/unversioned value main → getDigest(main)" path. That
    lookup currently 404s because nothing is published yet.
 - **Renovate 39.264.0 rejects main's current `renovate.json5`.** Validator output reproduced
  above.
 - **`registry-retention.sh` dry run against the live Gitea (read-only):** zero versions per image,
  nothing to delete. The token exchange and the package listing both answered.
 - **Live, read-only measurements** (kubectl and the Gitea API):
  - the runner pod is on k3s-prod-1, which had **7 GB** nodefs available;
  - DinD holds only the 1.54 GB act image; buildx is v0.20.1; the daemon is 27.5.1 on overlay2
    with cgroup v2;
  - Gitea `/data` is 9.7G, 1% used;
  - the Gateway's Envoy (Cilium) has `streamIdleTimeout: 300s`, `maxStreamDuration: 0s`, and
    **no route `timeout`** on the gitea route;
  - Gitea's `LIMIT_SIZE_CONTAINER` is unset (default −1 = unlimited, `modules/setting/packages.go`).
 ## What I did NOT verify
 - **Nothing has been pushed to the registry, by me or by CI.** The publish path has never run:
  - the registry HEAD/digest read-back;
  - the immutable-tag guard;
  - "already published at this commit" idempotence;
  - agent-runner `FROM <toolbox>@sha256:`;
  - `docker login` with a PAT.
  The first merge to main is the first test, and it needs `REGISTRY_PUSH_TOKEN` to exist first.
 - **Large-layer push through the Gateway is UNVERIFIED.** Envoy has no request-body cap and no
  stream duration cap here. But with no route `timeout` set, Envoy's **default 15 s upstream
  timeout** applies from the end of the request to the end of the response. That covers Gitea
  finalising a blob: copying the upload into package storage on Longhorn. A ~250 MB compressed
  layer should finish well inside 15 s, but I could not measure it without pushing. If the first
  publish shows 504s on blob PUTs, set `timeouts.request` on the gitea HTTPRoute.
 - **DinD fit is estimated, not measured** (until the PR run reports):
  - memory: 2Gi is shared by the job container and every build container;
  - disk: the toolbox, agent-runner, conformance-runner, Fedora base and build cache should
    total roughly 3 GB on top of the 1.5 GB act image, against 7 GB free on the node, with
    `dind-storage` having no sizeLimit.
 - **That act_runner hands the job container a Docker socket is inferred** from act_runner's
  DOCKER_HOST handling, not measured. The driver fails loudly if it does not.
 - **Renovate has still never completed a run here.** Three things block it:
  - #80, the `configurationFile` name;
  - `RENOVATE_TOKEN` gaining `read:package`;
  - this branch's version pin.
  `renovatebot/[email protected]` running a 44.x image is untested. The action only passes
  the tag to `docker run`.
 - **Pods cannot pull these images yet regardless.** The kubelet pull credential is PR #61
  (`pm-agent-registry` dockerconfigjson, `read:package`), still open. Until it merges, a correct
  digest still ends in ImagePullBackOff (401), as before.
 - **`registry-retention.sh --apply` has never run**, and its keep logic has only run against an
  empty registry.
 - **`git diff` change detection on Gitea `push` events relies on `github.event.before`.** A
  force-push or a first push falls back to building everything, which is intended but not
  observed.
@@ -244,10 +244,12 @@ about runtime behaviour in this document is a design claim, not a measurement.
   (ii) the pull is still unauthenticated and no `imagePullSecret` exists in the tree — with
   the credential decision now being local Gitea accounts with PATs, that means a
   `dockerconfigjson` for an account whose token carries `read:package`. Only then does the
-   digest matter. Operator runs `infra/agent/build-image.sh`, pushes, and one PR replaces both
+   digest matter. **Superseded 2026-10-29:** the build-and-push is no longer an operator step.
-   placeholder digests. Expect the build itself to need iteration too: the nodejs module
+   `.gitea/workflows/images.yaml` builds and verifies the image with
-   stream and the npm install have never run against this base. The same two blockers sit in
+   `infra/agent/build-image.sh`, publishes it on a push to `main`, and Renovate opens the one
-   front of `conformance-runner`, which shares the registry path and has never been pulled.
+   PR that replaces both placeholder digests (`decisions/2026-10-29-image-pipeline.md`). The
   same two blockers sit in front of `conformance-runner`, which shares the registry path and
   has never been pulled.
 2. **Create the credentials** per `pm-agent-credentials.sops.yaml.example`, and **verify the
   bot's permission is `write`, not `admin`** — that measurement is the control that keeps
   the agent out of its own approval gate.
@@ -1,26 +1,36 @@
 #!/usr/bin/env bash
 ###############################################################################
-# infra/agent/build-image.sh — build the pinned `agent-runner` image and print its digest.
+# infra/agent/build-image.sh — build the pinned `agent-runner` image and prove it works.
 #
-#   ./build-image.sh                 # build + verify + print the digest and push command
+#   ./build-image.sh                 # build + verify + print the image id
 #   ./build-image.sh --no-verify     # build only (skip the in-image proof)
 #   ENGINE=docker ./build-image.sh   # build with docker instead of podman
 #   AGENT_BASE_IMAGE=home-k3s-dev:local ./build-image.sh
 #                                    # build on a LOCAL toolbox instead of the pinned one
 #                                    #   (`infra/devcontainer/dev.sh build` makes that tag)
 #   IMAGE_TAG=foo ./build-image.sh   # override the local tag (CI uses this)
 #
-# THIS SCRIPT NEVER PUSHES. Publishing is an operator action: the registry credential is not
+# THIS SCRIPT NEVER PUSHES — and nobody pushes this image from a workstation either.
-# an agent's to hold, and the pushed digest is what a fail-closed admission policy then
+# Publishing is done by CI: .gitea/workflows/images.yaml calls this script, and on a push to
-# trusts. So this builds locally, proves the toolchain and both entrypoints work INSIDE the
+# `main` it pushes what this script built AND verified to the Gitea registry, records the
-# image as the non-root uid with the read-only root filesystem the policy forces, and prints
+# registry-returned digest in the job summary, and Renovate then opens the pull request that
-# the exact push line for a human.
+# moves platform/pm-agent/ onto that digest. So this script's job is to build, and to prove
 # the toolchain and both entrypoints work INSIDE the image as the non-root uid with the
 # read-only root filesystem the admission policy forces — the same proof locally and in CI.
 #
-# VERSIONS ARE NOT DEFINED HERE. infra/agent/versions.env holds the four pins this image
+# VERSIONS ARE NOT DEFINED HERE. infra/agent/versions.env holds the pins this image adds
-# adds (base image, node, Claude Code CLI, model) and infra/devcontainer/versions.env holds
+# (base image, node, Claude Code CLI, model) and infra/devcontainer/versions.env holds the
-# the platform toolchain the base already carries. Nothing in this file invents a version.
+# platform toolchain the base already carries. Nothing in this file invents a version.
 #
-# THE DIGEST CAVEAT — read it before pasting a digest into anything. The digest printed here
+# THE BASE. AGENT_BASE_IMAGE in versions.env is the PUBLISHED toolbox, pinned by digest and
-# is the LOCAL manifest digest. It is normally identical to the registry digest, but that is
+# kept current by Renovate. An AGENT_BASE_IMAGE in the environment overrides it — CI passes
-# a property of the push rather than a guarantee, so the push command below uses
+# the toolbox it built in the same run, and a developer can pass `home-k3s-dev:local`.
-# `--digestfile`: the digest the REGISTRY returns is the authoritative one, and that is the
+#
-# one that goes into platform/pm-agent/configmap.yaml.
+# ENGINE PORTABILITY. CI runs this with ENGINE=docker against a Docker-in-Docker daemon, so
 # nothing below may be podman-only (`image exists`, `--digestfile`, `{{.Digest}}`), and
 # nothing may BIND-MOUNT a file from this checkout into a container: under DinD the daemon
 # resolves bind sources on ITS filesystem, where this checkout does not exist, and silently
 # mounts an empty directory instead. Files go in over stdin.
 ###############################################################################
 set -euo pipefail
@@ -46,8 +56,11 @@ command -v "${ENGINE}" >/dev/null 2>&1 || die "'${ENGINE}' not on PATH (set ENGI
 # --- pins: sourced, never invented -------------------------------------------
 AGENT_ENV="${HERE}/versions.env"
 [ -f "${AGENT_ENV}" ] || die "missing ${AGENT_ENV} — the pins live there, not here"
 BASE_OVERRIDE="${AGENT_BASE_IMAGE:-}"
 TAG_OVERRIDE="${IMAGE_TAG:-}"
 # shellcheck disable=SC1090
 source "${AGENT_ENV}"
 [ -z "${BASE_OVERRIDE}" ] || AGENT_BASE_IMAGE="${BASE_OVERRIDE}"
 : "${AGENT_BASE_IMAGE:?AGENT_BASE_IMAGE not set in ${AGENT_ENV}}"
 : "${NODEJS_MAJOR:?NODEJS_MAJOR not set in ${AGENT_ENV}}"
 : "${CLAUDE_CODE_VERSION:?CLAUDE_CODE_VERSION not set in ${AGENT_ENV}}"
@@ -55,16 +68,28 @@ source "${AGENT_ENV}"
 : "${AGENT_MODEL:?AGENT_MODEL not set in ${AGENT_ENV}}"
 : "${AGENT_IMAGE_REPO:?AGENT_IMAGE_REPO not set in ${AGENT_ENV}}"
-# --- the base must already exist ---------------------------------------------
+# --- the base must exist, locally or in the registry ---------------------------
-# Deliberately NOT built on demand. The toolbox build is its own reviewed artefact with its
+# Deliberately NOT built on demand. The toolbox is its own reviewed artefact with its own
-# own pins, and silently building it here would mean this script could change the toolchain
+# pins, and silently building it here would mean this script could change the toolchain a
-# a worker gates with as a side effect of rebuilding the agent image.
+# worker gates with as a side effect of rebuilding the agent image.
-if ! "${ENGINE}" image exists "${AGENT_BASE_IMAGE}" 2>/dev/null; then
+case "${AGENT_BASE_IMAGE}" in
-  die "base image '${AGENT_BASE_IMAGE}' is not present. Build the pinned toolbox first:
+  *@sha256:0000000000000000000000000000000000000000000000000000000000000000)
-       ${REPO_ROOT}/infra/devcontainer/dev.sh build"
+    die "AGENT_BASE_IMAGE is still the all-zero placeholder in ${AGENT_ENV}: the toolbox has
       not been published yet. Either let CI publish it (.gitea/workflows/images.yaml) and
       merge Renovate's digest bump, or build on a local toolbox:
         ${REPO_ROOT}/infra/devcontainer/dev.sh build
         AGENT_BASE_IMAGE=home-k3s-dev:local $0" ;;
 esac
 if ! "${ENGINE}" image inspect "${AGENT_BASE_IMAGE}" >/dev/null 2>&1; then
  info "base ${AGENT_BASE_IMAGE} is not local; pulling it"
  "${ENGINE}" pull "${AGENT_BASE_IMAGE}" >/dev/null \
    || die "base image '${AGENT_BASE_IMAGE}' is neither local nor pullable (registry login?).
       To build on a local toolbox instead:
         ${REPO_ROOT}/infra/devcontainer/dev.sh build
         AGENT_BASE_IMAGE=home-k3s-dev:local $0"
 fi
-IMAGE_TAG="claude${CLAUDE_CODE_VERSION}-node${NODEJS_MAJOR}"
+IMAGE_TAG="${TAG_OVERRIDE:-claude${CLAUDE_CODE_VERSION}-node${NODEJS_MAJOR}}"
 IMAGE_REF="${AGENT_IMAGE_REPO}:${IMAGE_TAG}"
 log "Building ${IMAGE_REF}"
@@ -75,7 +100,11 @@ info "  node        ${NODEJS_MAJOR}"
 info "  claude      ${CLAUDE_CODE_VERSION} (floor ${CLAUDE_CODE_MIN_VERSION})"
 info "  model       ${AGENT_MODEL}"
-"${ENGINE}" build \
+# Provenance: CI passes the commit it is building, so a pulled image names its source tree.
 LABEL_ARGS=()
 [ -z "${BUILD_REVISION:-}" ] || LABEL_ARGS+=(--label "org.opencontainers.image.revision=${BUILD_REVISION}")
 "${ENGINE}" build "${LABEL_ARGS[@]}" \
  --file "${HERE}/Containerfile" \
  --tag "${IMAGE_REF}" \
  --build-arg "AGENT_BASE_IMAGE=${AGENT_BASE_IMAGE}" \
@@ -138,63 +167,32 @@ if [ "${VERIFY}" -eq 1 ]; then
  # that will actually run in the cluster. The probe is mounted read-only rather than baked
  # in: it is a build-time check, not something a worker Job has any business carrying.
  log "Probing pm-worker's input validation inside the image"
-  "${ENGINE}" run --rm \
+  #
  # The probe goes in over STDIN, not as a bind mount: under Docker-in-Docker (CI) a bind
  # source is resolved on the daemon's filesystem, where this file does not exist, and the
  # container would get an empty directory. PM_WORKER_PATH points it at the baked worker,
  # since the probe's own default (a sibling of the probe file) has no meaning on stdin.
  "${ENGINE}" run --rm -i \
    --user 65532:65532 \
    --read-only \
    --tmpfs /workspace:rw,mode=1777 \
    --tmpfs /tmp:rw,mode=1777 \
    --network none \
-    --volume "${HERE}/probe-validation.sh:/probe-validation.sh:ro,Z" \
+    -e PM_WORKER_PATH=/usr/local/bin/pm-worker \
    --entrypoint "" \
    "${IMAGE_REF}" \
-    /bin/bash /probe-validation.sh
+    /bin/bash -s < "${HERE}/probe-validation.sh"
  log "Image verified"
 fi
-# --- digest ------------------------------------------------------------------
+# --- result ------------------------------------------------------------------
-DIGEST="$("${ENGINE}" image inspect --format '{{.Digest}}' "${IMAGE_REF}")"
+# No digest is printed: a manifest digest does not exist until a registry has the manifest,
 # and the only digest anything may pin is the one the REGISTRY returns. CI captures that at
 # push time and prints it in the job summary (.gitea/workflows/scripts/images.sh).
 SIZE="$("${ENGINE}" image inspect --format '{{.Size}}' "${IMAGE_REF}")"
 log "Built ${IMAGE_REF}"
-info "local manifest digest  ${DIGEST}"
+info "image id  $("${ENGINE}" image inspect --format '{{.Id}}' "${IMAGE_REF}")"
-info "size                   $(( SIZE / 1024 / 1024 )) MiB"
+info "size      $(( SIZE / 1024 / 1024 )) MiB"
-
+info "Not pushed. CI publishes this image from main (.gitea/workflows/images.yaml); see"
-cat <<EOF
+info "platform/pm-agent/README.md for how a new digest reaches the cluster."
 ================================================================================
 NOT PUSHED. Publishing is an operator action — this script holds no registry
 credential and will never acquire one.
 OPERATOR: run these, then put the digest from the digestfile into
 platform/pm-agent/configmap.yaml (PM_WORKER_IMAGE) via a pull request:
    ${ENGINE} login gitea.home.keithjackson.dev
    ${ENGINE} push --digestfile /tmp/agent-runner.digest ${IMAGE_REF}
    cat /tmp/agent-runner.digest
    # -> sha256:....   THIS is the authoritative digest, not the local one above.
    # platform/pm-agent/configmap.yaml
    PM_WORKER_IMAGE: "${AGENT_IMAGE_REPO}@\$(cat /tmp/agent-runner.digest)"
 WHY THE DIGEST AND NOT THE TAG: platform/kyverno/admission/pm-agent-job-shape.yaml is a
 fail-closed Enforce policy. It DENIES any worker Job whose image is not
 ${AGENT_IMAGE_REPO}@sha256:<digest> — a tag reference is denied,
 and so is any other repository path. The dispatcher also refuses to start if
 PM_WORKER_IMAGE carries no digest, so the failure names its own cause.
 THE SAME IMAGE RUNS THE DISPATCHER. platform/pm-agent/deployment.yaml pins it too, so a
 new digest is a two-line change in one pull request: the Deployment's image and
 PM_WORKER_IMAGE. They must be the same digest — the dispatcher and its workers share the
 committed Job template and the env-name contract, and a mismatched pair would disagree
 about it.
 TIGHTEN THE BOUNDARY AFTERWARDS: the policy admits the wildcard
 \`${AGENT_IMAGE_REPO}@sha256:*\`, so anyone who can push to that registry path can publish
 a different digest. Replacing the wildcard with the LITERAL digest closes that gap; it is a
 one-line change, reviewed in a pull request, which is exactly where a change to the
 security boundary belongs.
 ================================================================================
 EOF
@@ -22,7 +22,9 @@
 ###############################################################################
 set -uo pipefail
-HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# `:-$0` because build-image.sh feeds this script over STDIN (`bash -s`), where BASH_SOURCE is
 # empty and `set -u` would abort on it; PM_WORKER_PATH is always set in that case.
 HERE="$(cd "$(dirname "${BASH_SOURCE[0]:-$0}")" && pwd)"
 WORKER="${PM_WORKER_PATH:-${HERE}/pm-worker}"
 [ -r "$WORKER" ] || { echo "cannot read ${WORKER}" >&2; exit 2; }
@@ -9,10 +9,16 @@
 # truth and, the first time the two drifted, the worker's "green" would stop meaning the
 # gate's green. So this file pins ONLY what the toolbox does not carry.
-# The toolbox image the runner is layered on. Built by `infra/devcontainer/dev.sh build`
+# The toolbox image the runner is layered on: the PUBLISHED toolbox, by digest. CI builds and
-# (tag `home-k3s-dev:local`); infra/agent/build-image.sh asserts it exists rather than
+# publishes it (.gitea/workflows/images.yaml) and Renovate moves this line onto each new digest
-# silently building a different base.
+# of its `main` tag via the marker below, in a pull request — nobody edits it by hand.
-AGENT_BASE_IMAGE=home-k3s-dev:local
+#
 # ALL-ZERO UNTIL THE FIRST PUBLISH. The first run of the image pipeline on main sees this
 # placeholder, builds and publishes the toolbox first, and builds agent-runner FROM that fresh
 # digest. For a LOCAL build on a local toolbox, override it:
 #   infra/devcontainer/dev.sh build && AGENT_BASE_IMAGE=home-k3s-dev:local infra/agent/build-image.sh
 # renovate-digest: image=gitea.home.keithjackson.dev/home-network/toolbox tag=main
 AGENT_BASE_IMAGE=gitea.home.keithjackson.dev/home-network/toolbox@sha256:0000000000000000000000000000000000000000000000000000000000000000
 # Node.js — the runtime for the Claude Code CLI, installed from the Fedora base's module
 # stream. A major version, not a patch: the CLI supports the active LTS line and the
@@ -80,6 +80,12 @@ spec:
          # Replaced by the dispatcher with the digest-pinned image. A tag here would be
          # DENIED by admission; the dispatcher refuses a non-digest PM_WORKER_IMAGE at
          # startup so the error names the real cause.
          #
          # DELIBERATELY ALL-ZERO, AND DELIBERATELY NOT TRACKED BY RENOVATE. This file is
          # baked INTO agent-runner, so it cannot carry agent-runner's own digest: bumping it
          # would change the image's inputs, CI would publish a new digest, Renovate would bump
          # it again — forever. pm-dispatch overwrites this field from PM_WORKER_IMAGE
          # (platform/pm-agent/configmap.yaml, which IS tracked) on every Job it creates.
          image: gitea.home.keithjackson.dev/home-network/agent-runner@sha256:0000000000000000000000000000000000000000000000000000000000000000
          imagePullPolicy: IfNotPresent
          # The pinned entrypoint, with NO args. Args on a pinned command are
@@ -34,6 +34,22 @@ cd infra/devcontainer
 ```
 `ENGINE=docker ./dev.sh` to use docker instead of rootless podman.
 ### The published toolbox
 `./dev.sh build` delegates to [`build-image.sh`](build-image.sh), which is the **one** build
 recipe: CI runs the same script (with its in-image verification — every tool executes, and the
 version-sensitive ones report their `versions.env` pins) and, on a push to `main` that changes
 `Containerfile`, `versions.env` or `build-image.sh`, **publishes** the result as
 `gitea.home.keithjackson.dev/home-network/toolbox` (tags `sha-<commit>` and `main`) —
 see [`.gitea/workflows/images.yaml`](../../.gitea/workflows/images.yaml). `agent-runner` is
 built `FROM` that published digest (`infra/agent/versions.env`, kept current by Renovate).
 Nobody pushes this image from a workstation. To use the published one locally instead of
 building, after `podman login gitea.home.keithjackson.dev` (a token with `read:package`):
 ```bash
 IMAGE=gitea.home.keithjackson.dev/home-network/toolbox:main ./dev.sh run make -C ci gate
 ```
 ## Choosing a cluster — `CLUSTER=dev|prod`
 ```bash
@@ -0,0 +1,108 @@
 #!/usr/bin/env bash
 ###############################################################################
 # infra/devcontainer/build-image.sh — build the pinned dev `toolbox` image and prove it works.
 #
 #   ./build-image.sh                 # build + verify (tag: home-k3s-dev:local)
 #   ./build-image.sh --no-verify     # build only
 #   IMAGE=foo:bar ./build-image.sh   # build under another local name
 #   ENGINE=docker ./build-image.sh   # build with docker instead of podman
 #
 # ONE BUILD DEFINITION, TWO CALLERS. `dev.sh build` calls this for local development, and the
 # CI image pipeline (.gitea/workflows/images.yaml -> .gitea/workflows/scripts/images.sh) calls
 # it to build the image it then PUBLISHES as gitea.home.keithjackson.dev/home-network/toolbox.
 # Keeping the build args in exactly one place is what makes "the toolbox CI publishes" and "the
 # toolbox you gate with locally" the same recipe.
 #
 # THIS SCRIPT NEVER PUSHES. Publishing is CI's job (images.yaml, on push to main), not a
 # workstation's: an image that exists only because someone ran `podman push` from a laptop has
 # no build log, no recorded inputs and no verification anyone else can read.
 #
 # VERSIONS ARE NOT DEFINED HERE. infra/devcontainer/versions.env is the single source of truth.
 ###############################################################################
 set -euo pipefail
 HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 ENGINE="${ENGINE:-podman}"
 IMAGE="${IMAGE:-home-k3s-dev:local}"
 VERIFY=1
 for arg in "$@"; do
  case "$arg" in
    --no-verify) VERIFY=0 ;;
    -h|--help) sed -n '2,/^####/p' "${BASH_SOURCE[0]}" | sed '$d' | sed 's/^# \{0,1\}//'; exit 0 ;;
    *) echo "unknown argument: $arg" >&2; exit 2 ;;
  esac
 done
 log()  { printf '\n\033[1m==> %s\033[0m\n' "$*"; }
 die()  { printf '\033[31mERROR: %s\033[0m\n' "$*" >&2; exit 1; }
 command -v "${ENGINE}" >/dev/null 2>&1 || die "'${ENGINE}' not on PATH (set ENGINE=docker to use docker)"
 # shellcheck disable=SC1091
 source "${HERE}/versions.env"
 # Provenance: CI passes the commit it is building, so a pulled image names its source tree.
 LABEL_ARGS=()
 [ -z "${BUILD_REVISION:-}" ] || LABEL_ARGS+=(--label "org.opencontainers.image.revision=${BUILD_REVISION}")
 log "Building ${IMAGE}"
 "$ENGINE" build "${LABEL_ARGS[@]}" \
  --build-arg FEDORA_VERSION="$FEDORA_VERSION" \
  --build-arg OPENTOFU_VERSION="$OPENTOFU_VERSION" \
  --build-arg AGE_VERSION="$AGE_VERSION" \
  --build-arg SOPS_VERSION="$SOPS_VERSION" \
  --build-arg FLUX_VERSION="$FLUX_VERSION" \
  --build-arg KUBECTL_VERSION="$KUBECTL_VERSION" \
  --build-arg HELM_VERSION="$HELM_VERSION" \
  --build-arg KYVERNO_VERSION="$KYVERNO_VERSION" \
  --build-arg KYVERNO_JSON_VERSION="$KYVERNO_JSON_VERSION" \
  --build-arg CHAINSAW_VERSION="$CHAINSAW_VERSION" \
  --build-arg YAMLLINT_VERSION="$YAMLLINT_VERSION" \
  --build-arg YQ_VERSION="$YQ_VERSION" \
  --build-arg CHECK_JSONSCHEMA_VERSION="$CHECK_JSONSCHEMA_VERSION" \
  --build-arg PROMTOOL_VERSION="$PROMTOOL_VERSION" \
  -t "$IMAGE" -f "${HERE}/Containerfile" "${HERE}"
 # --- verify ------------------------------------------------------------------
 # A build that succeeded is not evidence the toolbox can run the gate. Every tool the gate and
 # the runbooks call must EXECUTE, and the version-sensitive ones must report the version
 # versions.env pins — promtool above all, because ci/versions.env asserts a promtool/Prometheus
 # lockstep and a drifted engine evaluates alert-rule tests differently (see versions.env).
 if [ "${VERIFY}" -eq 1 ]; then
  log "Verifying the toolchain inside ${IMAGE}"
  "$ENGINE" run --rm --network none --entrypoint "" \
    -e PROMTOOL_VERSION="${PROMTOOL_VERSION}" \
    -e KUBECTL_VERSION="${KUBECTL_VERSION}" \
    -e CHAINSAW_VERSION="${CHAINSAW_VERSION}" \
    -e KYVERNO_VERSION="${KYVERNO_VERSION}" \
    -e SOPS_VERSION="${SOPS_VERSION}" \
    "$IMAGE" /bin/bash -euo pipefail -c '
      echo "--- baked versions ---"; cat /etc/dev-toolchain-versions
      echo "--- every tool is on PATH ---"
      for t in tofu age age-keygen sops flux kubectl helm kyverno kyverno-json chainsaw yq \
               yamllint check-jsonschema promtool ansible-playbook git make jq diff python3 \
               wg qrencode; do
        if command -v "$t" >/dev/null 2>&1; then printf "    ok      %s\n" "$t";
        else printf "    MISSING %s\n" "$t"; missing=1; fi
      done
      [ -z "${missing:-}" ] || { echo "toolbox is missing tools"; exit 1; }
      echo "--- version-sensitive tools report their pins ---"
      check() { # name pin actual-output
        if printf "%s" "$3" | grep -qF "$2"; then printf "    ok      %-10s %s\n" "$1" "$2";
        else printf "    DRIFT   %-10s want %s, got: %s\n" "$1" "$2" "$(printf "%s" "$3" | head -1)"; drift=1; fi
      }
      check promtool "${PROMTOOL_VERSION}" "$(promtool --version 2>&1)"
      check kubectl  "${KUBECTL_VERSION}"  "$(kubectl version --client 2>&1)"
      check chainsaw "${CHAINSAW_VERSION}" "$(chainsaw version 2>&1)"
      check kyverno  "${KYVERNO_VERSION}"  "$(kyverno version 2>&1)"
      check sops     "${SOPS_VERSION}"     "$(sops --version --disable-version-check 2>&1 || sops --version 2>&1)"
      [ -z "${drift:-}" ] || { echo "a tool does not report its pinned version"; exit 1; }
      echo "--- python can import what the renderer and the gate need ---"
      python3 -c "import yaml; print(\"    PyYAML\", yaml.__version__)"
    '
  log "Toolbox verified"
 fi
 log "Built ${IMAGE}"
 printf '    image id  %s\n' "$("$ENGINE" image inspect --format '{{.Id}}' "$IMAGE")"
@@ -72,22 +72,10 @@ esac
 CLUSTER_KUBECONFIG="${REPO_ROOT}/infra/ansible/.cache/${CLUSTER}.kubeconfig"
 build() {
-  "$ENGINE" build \
+  # The build recipe lives in build-image.sh so that CI (.gitea/workflows/images.yaml), which
-    --build-arg FEDORA_VERSION="$FEDORA_VERSION" \
+  # builds and PUBLISHES this image, and a local `dev.sh build` cannot drift apart. --no-verify
-    --build-arg OPENTOFU_VERSION="$OPENTOFU_VERSION" \
+  # keeps the local loop as fast as it always was; CI runs the verification.
-    --build-arg AGE_VERSION="$AGE_VERSION" \
+  IMAGE="$IMAGE" ENGINE="$ENGINE" "${HERE}/build-image.sh" --no-verify
    --build-arg SOPS_VERSION="$SOPS_VERSION" \
    --build-arg FLUX_VERSION="$FLUX_VERSION" \
    --build-arg KUBECTL_VERSION="$KUBECTL_VERSION" \
    --build-arg HELM_VERSION="$HELM_VERSION" \
    --build-arg KYVERNO_VERSION="$KYVERNO_VERSION" \
    --build-arg KYVERNO_JSON_VERSION="$KYVERNO_JSON_VERSION" \
    --build-arg CHAINSAW_VERSION="$CHAINSAW_VERSION" \
    --build-arg YAMLLINT_VERSION="$YAMLLINT_VERSION" \
    --build-arg YQ_VERSION="$YQ_VERSION" \
    --build-arg CHECK_JSONSCHEMA_VERSION="$CHECK_JSONSCHEMA_VERSION" \
    --build-arg PROMTOOL_VERSION="$PROMTOOL_VERSION" \
    -t "$IMAGE" -f "${HERE}/Containerfile" "${HERE}"
 }
 run() {
@@ -252,11 +252,16 @@ step "preconditions — repo side (these block ONLY arming ${VARIABLE}; the secr
 # R1: the runner image is built and digest-pinned. Without it the Job cannot run run.sh at
 # all (the stock chainsaw image has no bash/python3/kubectl/git) — plan §3 item 1.
 # EVALUATED, not grepped: the line is `CONFORMANCE_RUNNER_IMAGE="${..._REPO}@${..._DIGEST}"`, so
 # its literal text never contains 64 hex digits and the grep this replaced could never pass. The
 # all-zero digest is the "not published yet" placeholder (Renovate replaces it after CI's first
 # publish, .gitea/workflows/images.yaml) and must not count as pinned.
 VERSIONS="${REPO_ROOT}/conformance/versions.env"
-if grep -Eq '^[[:space:]]*CONFORMANCE_RUNNER_IMAGE=.*@sha256:[0-9a-f]{64}' "$VERSIONS" 2>/dev/null; then
+runner_img="$( (set +u; . "$VERSIONS" >/dev/null 2>&1; printf '%s' "${CONFORMANCE_RUNNER_IMAGE:-}") )"
-  pass "CONFORMANCE_RUNNER_IMAGE is digest-pinned in conformance/versions.env"
+if [[ "$runner_img" =~ @sha256:[0-9a-f]{64}$ ]] && [[ ! "$runner_img" =~ @sha256:0{64}$ ]]; then
  pass "CONFORMANCE_RUNNER_IMAGE is digest-pinned in conformance/versions.env (${runner_img##*@})"
 else
-  fail_arm "conformance/versions.env has no digest-pinned CONFORMANCE_RUNNER_IMAGE (plan §3 item 1) — the Job cannot run ./run.sh"
+  fail_arm "conformance/versions.env has no PUBLISHED digest for CONFORMANCE_RUNNER_IMAGE (got '${runner_img}') — the image pipeline has not published it yet, or Renovate's bump is unmerged (plan §3 item 1)"
 fi
 # R2: the Job template is the real thing, not the skeleton: the admitted image AND an
@@ -0,0 +1,130 @@
 #!/usr/bin/env bash
 # Registry retention for this repo's OWN images (toolbox, agent-runner, conformance-runner):
 # keep what is deployed and what is recent, delete the rest. DRY RUN unless --apply.
 #
 #   ./infra/gitea/registry-retention.sh              # show what would be deleted
 #   ./infra/gitea/registry-retention.sh --apply      # delete it
 #   KEEP=15 ./infra/gitea/registry-retention.sh      # keep the 15 newest sha- tags (default 10)
 #
 # RUNNING IT IS AN OPERATOR ACTION. It deletes registry content, so it is not wired into CI and
 # no agent runs it with --apply.
 #
 # WHY A SCRIPT, AND NOT GITEA'S BUILT-IN "CLEANUP RULE" — measured against Gitea 1.24.3's source:
 #
 #   1. There is NO API for cleanup rules. routers/api/v1/api.go has no such route; the rules live
 #      only behind the web UI's session+CSRF forms (routers/web/web.go, org settings
 #      `/packages/rules/add`). A rule therefore cannot be created, read or drift-checked as code —
 #      which is also why ci/gitea-settings.sh cannot assert one exists.
 #   2. A built-in rule would DELETE WHAT IS DEPLOYED. Every reference to these images is
 #      digest-only (`<repo>@sha256:...`, required by admission). Gitea stores a manifest as a
 #      file OF THE TAG VERSION it was pushed under (routers/api/packages/container/manifest.go
 #      createPackageAndVersion), and resolves a by-digest pull by searching those files
 #      (models/packages/container/search.go). So deleting the tag a digest was pushed under makes
 #      that digest unpullable — MANIFEST_UNKNOWN, i.e. ImagePullBackOff on the next reschedule.
 #      The built-in rule keeps "the newest N" and a name pattern, and knows nothing about which
 #      digest git currently pins. Renovate's bump PRs can lag, so "newest N" is not a guarantee.
 #
 # WHAT IS KEPT, per image:
 #   * the `main` and `latest` tags;
 #   * the KEEP newest `sha-<commit>` tags (by creation time);
 #   * EVERY tag whose manifest digest is referenced anywhere in the tracked tree at REF
 #     (default origin/main) — i.e. whatever Flux is deploying, and the toolbox digest
 #     agent-runner builds FROM;
 #   * every version whose NAME is a digest (`sha256:...`): those are index children, and Gitea's
 #     own cleanup handles them (services/packages/container/cleanup.go ShouldBeSkipped).
 # Everything else is deleted. Gitea's `cleanup_packages` cron then reclaims the unreferenced
 # blobs (default: daily, blobs older than 24h) — space comes back the day after, not instantly.
 #
 # THE STORAGE IT PROTECTS. Packages live on the SAME 10Gi Longhorn PVC as every repository and
 # Gitea's database (catalog/gitea.yaml). The toolbox is ~1.4 GB uncompressed and every main push
 # that touches its inputs adds a complete, non-reproducible copy. Unpruned, the registry is what
 # fills the volume the git host lives on.
 #
 # CREDENTIAL: the one `git` has stored for the host (as the other scripts here). It needs
 # `read:package` to list and `write:package` to --apply, and the account needs package write on
 # the org. The secret is never printed.
 set -euo pipefail
 HOST=gitea.home.keithjackson.dev; OWNER=Home-Network; NS=home-network
 IMAGES=(toolbox agent-runner conformance-runner)
 KEEP="${KEEP:-10}"
 REF="${REF:-origin/main}"
 APPLY=0
 for a in "$@"; do
  case "$a" in
    --apply) APPLY=1 ;;
    -h|--help) sed -n '2,/^set -euo/p' "${BASH_SOURCE[0]}" | sed '$d' | sed 's/^# \{0,1\}//'; exit 0 ;;
    *) echo "unknown argument: $a" >&2; exit 2 ;;
  esac
 done
 [[ "$KEEP" =~ ^[0-9]+$ ]] && [ "$KEEP" -ge 1 ] || { echo "KEEP must be a positive integer" >&2; exit 2; }
 command -v jq >/dev/null || { echo "jq is required" >&2; exit 1; }
 REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
 git -C "$REPO_ROOT" fetch -q origin main 2>/dev/null || echo "   (could not fetch origin/main; using the local ${REF})" >&2
 git -C "$REPO_ROOT" rev-parse -q --verify "${REF}^{commit}" >/dev/null || { echo "no such ref: ${REF}" >&2; exit 1; }
 creds="$(printf 'protocol=https\nhost=%s\n\n' "$HOST" | git credential fill)"
 U="$(sed -n 's/^username=//p' <<<"$creds")"; P="$(sed -n 's/^password=//p' <<<"$creds")"
 [ -n "$U" ] && [ -n "$P" ] || { echo "no stored credential for $HOST" >&2; exit 1; }
 echo "authenticating as '${U}' (secret not shown)"
 api() { curl -sS --config <(printf 'user = "%s:%s"\n' "$U" "$P") "$@"; }
 REG_TOKEN="$(api "https://${HOST}/v2/token?service=container_registry" | jq -r '.token // empty')"
 [ -n "$REG_TOKEN" ] || { echo "registry token exchange failed (does the credential have read:package?)" >&2; exit 1; }
 manifest_digest() { # <image> <tag> -> digest, or nothing
  curl -sS -I --config <(printf 'header = "Authorization: Bearer %s"\n' "$REG_TOKEN") \
    -H 'Accept: application/vnd.oci.image.manifest.v1+json' \
    -H 'Accept: application/vnd.docker.distribution.manifest.v2+json' \
    -H 'Accept: application/vnd.oci.image.index.v1+json' \
    -H 'Accept: application/vnd.docker.distribution.manifest.list.v2+json' \
    "https://${HOST}/v2/${NS}/$1/manifests/$2" \
    | tr -d '\r' | awk -F': ' 'tolower($1)=="docker-content-digest"{print $2}' | tail -1
 }
 TOTAL_DEL=0
 for img in "${IMAGES[@]}"; do
  echo
  echo "==> ${NS}/${img}"
  # Digests git pins at REF. The marker convention puts the digest on the line after
  # `# renovate-digest: image=<repo>`; the plain `<repo>@sha256:` grep catches any other use.
  repo="${HOST}/${NS}/${img}"
  pinned="$( { git -C "$REPO_ROOT" grep -h -A1 -F "renovate-digest: image=${repo} " "$REF" -- . 2>/dev/null || true
               git -C "$REPO_ROOT" grep -h -F "${repo}@sha256:" "$REF" -- . 2>/dev/null || true; } \
             | grep -oE 'sha256:[0-9a-f]{64}' | grep -v 'sha256:0\{64\}' | sort -u || true)"
  echo "    pinned at ${REF}: $(wc -w <<<"$pinned") digest(s)"
  versions="[]"; page=1
  while :; do
    chunk="$(api "https://${HOST}/api/v1/packages/${OWNER}?type=container&q=${img}&limit=50&page=${page}")"
    jq -e 'type=="array"' >/dev/null <<<"$chunk" || { echo "    package list failed: $(head -c 200 <<<"$chunk")" >&2; exit 1; }
    [ "$(jq length <<<"$chunk")" -gt 0 ] || break
    versions="$(jq -c --argjson a "$versions" --arg n "$img" '$a + [ .[] | select(.name==$n) | {version, created_at} ]' <<<"$chunk")"
    page=$((page + 1))
  done
  echo "    versions: $(jq length <<<"$versions")"
  keep_sha="$(jq -r '[ .[] | select(.version|startswith("sha-")) ] | sort_by(.created_at) | reverse | .[:'"$KEEP"'][] .version' <<<"$versions")"
  while IFS= read -r v; do
    [ -n "$v" ] || continue
    case "$v" in
      main|latest)  echo "    keep    ${v}  (moving tag)"; continue ;;
      sha256:*)     echo "    keep    ${v}  (digest-named version; Gitea's cleanup owns those)"; continue ;;
    esac
    if grep -qxF "$v" <<<"$keep_sha"; then echo "    keep    ${v}  (among the ${KEEP} newest)"; continue; fi
    d="$(manifest_digest "$img" "$v")"
    if [ -z "$d" ]; then echo "    keep    ${v}  (could not read its digest — never delete what cannot be checked)"; continue; fi
    if grep -qxF "$d" <<<"$pinned"; then echo "    keep    ${v}  (${d} is pinned at ${REF})"; continue; fi
    if [ "$APPLY" -eq 1 ]; then
      code="$(api -o /dev/null -w '%{http_code}' -X DELETE \
        "https://${HOST}/api/v1/packages/${OWNER}/container/${img}/$(jq -rn --arg v "$v" '$v|@uri')")"
      if [ "$code" = 204 ]; then echo "    DELETED ${v}  (${d})"; TOTAL_DEL=$((TOTAL_DEL + 1));
      else echo "    FAILED  ${v}  HTTP ${code}" >&2; fi
    else
      echo "    would delete ${v}  (${d})"; TOTAL_DEL=$((TOTAL_DEL + 1))
    fi
  done < <(jq -r '.[].version' <<<"$versions")
 done
 echo
 if [ "$APPLY" -eq 1 ]; then echo "deleted ${TOTAL_DEL} version(s). Blobs are reclaimed by Gitea's cleanup_packages cron."
 else echo "DRY RUN: ${TOTAL_DEL} version(s) would be deleted. Re-run with --apply to delete them."; fi
@@ -45,11 +45,18 @@ waive the sensitive-path human review on its own pull request.
 Both are deliberate, and both fail **visibly** rather than silently:
-1. **The image does not exist.** `platform/pm-agent/deployment.yaml` and
+1. **The image is published by CI, not by a person.** `platform/pm-agent/deployment.yaml` and
-   `PM_WORKER_IMAGE` in `configmap.yaml` carry an all-zero digest placeholder. Build it with
+   `PM_WORKER_IMAGE` in `configmap.yaml` carry an all-zero digest placeholder until the first
-   `infra/agent/build-image.sh` (which verifies it and refuses to push), push it as the
+   publish. `.gitea/workflows/images.yaml` builds `agent-runner` with
-   operator, then update **both** places with the digest `podman push --digestfile` returned,
+   `infra/agent/build-image.sh` (the same script, and the same in-image verification, you can
-   in one pull request. Until then: `ImagePullBackOff`.
+   run locally), and on a push to `main` publishes it to
   `gitea.home.keithjackson.dev/home-network/agent-runner` under `sha-<commit>` and `main`.
   Renovate then follows the `main` tag through the `# renovate-digest:` marker above **both**
   lines and moves them together in one pull request. Nobody builds-and-pushes this image from
   a workstation, and nobody pastes a digest. Until the first publish, and until that PR
   merges: `ImagePullBackOff`. The one operator prerequisite on the publishing side is the
   `REGISTRY_PUSH_TOKEN` Actions secret (see the header of `.gitea/workflows/images.yaml`);
   without it the publish job fails loudly rather than skipping.
   **Two things have to be true before any digest will pull, and neither is declared in this
   repository.** Both were measured after PR #57 merged — this service is the first workload
@@ -78,8 +85,8 @@ Both are deliberate, and both fail **visibly** rather than silently:
     account whose token carries `read:package`, referenced from the Deployment and the
     committed worker Job template — not built yet, because minting that token is yours.
-   The same two apply to `conformance-runner`: same registry path, same push-then-pin model,
+   The same two apply to `conformance-runner`: same registry path, same publish-then-Renovate
-   never pulled either.
+   model, never pulled either.
 2. **The credentials are placeholders.** `pm-agent-credentials.sops.yaml` is encrypted to the
   repository's age recipient but holds `REPLACE-ME-…` strings. The hand-off —
   which Gitea account to create, which scopes, and why it must *not* be an administrator —
@@ -52,12 +52,13 @@ data:
  # contract. A tag here is refused at startup (and would be DENIED by admission anyway).
  #
  # PLACEHOLDER UNTIL THE IMAGE IS PUBLISHED. The all-zero digest below is deliberately
-  # impossible: `infra/agent/build-image.sh` builds and verifies the image, the OPERATOR
+  # impossible. CI builds, verifies and — on main — publishes agent-runner
-  # pushes it, and the authoritative digest from `--digestfile` replaces this value in a
+  # (.gitea/workflows/images.yaml), and Renovate replaces this value with the digest the
-  # reviewed pull request (the same two-step the conformance-runner image follows, and for
+  # REGISTRY returned for the image's `main` tag, in a pull request that also moves the
-  # the same reason: a registry credential is not an agent's to hold). Until then the
+  # Deployment's image (same marker, same branch). Nobody pushes this image from a workstation.
-  # dispatcher starts, fails to pull, and says so — which is the visible failure, not a
+  # Until the first publish the dispatcher starts, fails to pull, and says so — which is the
-  # silent one.
+  # visible failure, not a silent one.
  # renovate-digest: image=gitea.home.keithjackson.dev/home-network/agent-runner tag=main
  PM_WORKER_IMAGE: "gitea.home.keithjackson.dev/home-network/agent-runner@sha256:0000000000000000000000000000000000000000000000000000000000000000"
  # --- the model -------------------------------------------------------------
@@ -66,9 +66,12 @@ spec:
        - name: dispatch
          # THE SAME IMAGE THE WORKERS RUN, at the same digest. They share the committed Job
          # template and the env-name contract baked into it, so a mismatched pair would
-          # disagree about the shape the admission policy enforces. Replaced together, in one
+          # disagree about the shape the admission policy enforces. Replaced together, in ONE
-          # pull request, with the digest `infra/agent/build-image.sh` printed — the all-zero
+          # Renovate pull request: both lines carry the same marker, so Renovate treats them as
          # one dependency and bumps them in the same branch. The digest comes from CI's publish
          # of agent-runner's `main` tag (.gitea/workflows/images.yaml) — the all-zero
          # placeholder below is impossible on purpose (see configmap.yaml PM_WORKER_IMAGE).
          # renovate-digest: image=gitea.home.keithjackson.dev/home-network/agent-runner tag=main
          image: gitea.home.keithjackson.dev/home-network/agent-runner@sha256:0000000000000000000000000000000000000000000000000000000000000000
          imagePullPolicy: IfNotPresent
          command:
@@ -22,9 +22,28 @@
 //   podman run --rm -v "$PWD/renovate.json5:/tmp/renovate.json5:ro,Z" \
 //     ghcr.io/renovatebot/renovate renovate-config-validator /tmp/renovate.json5
 //
-// NOTE on what was NOT wrong: `enabledManagers` containing bare `regex` was suspected of needing to
+// CORRECTION (2026-10-29): THE VALIDATOR ABOVE WAS NOT THE RENOVATE THE WORKFLOW RUNS. The bare
-// be `custom.regex`. The validator accepts `regex`, so that suspicion was unfounded and nothing
+// `ghcr.io/renovatebot/renovate` tag is `latest` (44.167.0 when this was measured), while
-// here changes it.
+// renovatebot/[email protected] defaults `renovate-version` to `39` (its action.yml), i.e.
 // 39.264.0. Validated against THAT, this file failed with five errors — every
 // `managerFilePatterns` key ("Custom Manager contains disallowed fields", "Invalid configuration
 // option: flux.managerFilePatterns", ...), because 39.x only knows `fileMatch`; the rename landed
 // in 40.2.0. So even after the workflow found this file, every run would have stopped at
 // CONFIG_VALIDATION. renovate.yaml now pins `renovate-version` to the exact release this file is
 // validated against. VALIDATE WITH THAT SAME TAG:
 //   podman run --rm -v "$PWD/renovate.json5:/tmp/renovate.json5:ro,Z" \
 //     ghcr.io/renovatebot/renovate:<renovate-version from renovate.yaml> \
 //     renovate-config-validator /tmp/renovate.json5
 // (Not `--strict`: that also fails on a pending migration of the pre-existing top-level
 // `"docker": {"enabled": true}` block at the bottom, which is a warning, not an error.)
 //
 // `regex` vs `custom.regex` in enabledManagers, RESOLVED FROM SOURCE (Renovate 39.264.0 and
 // later): both work. lib/config/migrations/custom/enabled-managers-migration.ts rewrites
 // `regex` -> `custom.regex`, and that migration runs on every real repository run
 // (lib/workers/repository/init/merge.ts -> migrateAndValidate), not only in the validator;
 // lib/modules/manager/index.ts getEnabledManagersList accepts either spelling. `custom.regex` is
 // the canonical name (introduced in 37.51.1, renovatebot/renovate#24079), so it is used below and
 // nothing depends on the migration any more.
 {  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  // Renovate config — Phase 4 automated updates (ADR-0009, R-UPD-01/02). AS OF 2026-10-17
  // RENOVATE OWNS BOTH HALVES: Helm chart/dependency bumps AND per-service container image tags.
@@ -51,7 +70,7 @@
  "labels": ["dependencies", "automated-update"],
  "ignoreUnstable": true,
  "respectLatest": true,
-  "enabledManagers": ["flux", "kustomize", "helmv3", "kubernetes", "regex", "github-actions"],
+  "enabledManagers": ["flux", "kustomize", "helmv3", "kubernetes", "custom.regex", "github-actions"],
  "flux": {
    "managerFilePatterns": [
      "/(^|/)platform/.+\\.ya?ml$/",
@@ -93,6 +112,42 @@
        "#\\s*renovate:\\s*datasource=(?<datasource>\\S+)\\s+depName=(?<depName>\\S+)(?:\\s+versioning=(?<versioning>\\S+))?\\s*\\n.*?(?<currentValue>v?[0-9]+\\.[0-9]+\\.[0-9]+)"
      ],
      "versioningTemplate": "{{#if versioning}}{{{versioning}}}{{else}}semver{{/if}}"
    },
    // ADDED 2026-10-29 — THIS REPO'S OWN IMAGES (toolbox, agent-runner, conformance-runner).
    // CI publishes them (.gitea/workflows/images.yaml) under an immutable `sha-<commit>` tag and a
    // moving `main` tag, and never writes to git. Every reference to them is DIGEST-ONLY
    // (`<repo>@sha256:<hex>`, no tag), because the admission policies
    // (platform/kyverno/admission/pm-agent-job-shape.yaml, conformance-trigger-job-shape.yaml)
    // admit exactly that form. A digest alone gives Renovate nothing to follow, so the tag to
    // follow rides in a marker comment on the line ABOVE the reference:
    //
    //   # renovate-digest: image=<registry>/<namespace>/<image> tag=main
    //   <anything>sha256:<64 hex>
    //
    // `main` is not a version, so Renovate skips the release lookup and proposes a `digest`
    // update whenever the tag's digest moves (lib/workers/repository/process/lookup/index.ts:
    // an unversioned currentValue WITH a currentDigest yields updateType `digest`). The
    // `sha256:` prefix is INSIDE currentDigest on purpose: Renovate replaces that capture
    // verbatim with the new digest, which carries the prefix.
    //
    // DELIBERATELY NOT TRACKED: infra/agent/worker-job-template.yaml. It is baked INTO
    // agent-runner, so bumping it would rebuild agent-runner and loop forever; pm-dispatch
    // overwrites that field from PM_WORKER_IMAGE at runtime anyway.
    // REGISTRY AUTH: the Gitea registry answers 401 to anonymous reads, and the platform token is
    // scoped to hostType `gitea`, which a docker lookup never matches
    // (lib/util/host-rules.ts find). renovate.yaml supplies a hostType `docker` rule.
    {
      "customType": "regex",
      "description": "This repo's own CI-published images, pinned by digest, following the tag named in a `# renovate-digest:` marker on the previous line.",
      "managerFilePatterns": [
        "/(^|/)versions\\.env$/",
        "/(^|/)platform/pm-agent/.+\\.ya?ml$/"
      ],
      "matchStrings": [
        "#\\s*renovate-digest:\\s*image=(?<depName>\\S+)\\s+tag=(?<currentValue>\\S+)\\s*\\n[^\\n]*?(?<currentDigest>sha256:[a-f0-9]{64})"
      ],
      "datasourceTemplate": "docker",
      "versioningTemplate": "docker"
    }
  ],
  "packageRules": [
@@ -151,6 +206,26 @@
        "needs-review",
        "blast-radius"
      ]
    },
    {
      "description": "ADDED 2026-10-29. This repo's own images are followed ONLY through the `# renovate-digest:` markers above. The `kubernetes` manager also sees platform/pm-agent/deployment.yaml and infra/agent/worker-job-template.yaml, where the image has a digest and no tag; left on, it would look those up as `latest` (a tag CI never pushes) and, for the template, re-open the self-referential loop the custom manager deliberately avoids.",
      "matchManagers": ["kubernetes"],
      "matchPackageNames": [
        "gitea.home.keithjackson.dev/home-network/toolbox",
        "gitea.home.keithjackson.dev/home-network/agent-runner",
        "gitea.home.keithjackson.dev/home-network/conformance-runner"
      ],
      "enabled": false
    },
    {
      "description": "ADDED 2026-10-29. Digest bumps for this repo's own images: the image was built and verified by CI before it was published, so these auto-merge on a green gate like any digest (rule 1). Labelled so they are easy to find. NOTE: the conformance-runner bump edits conformance/versions.env, a sensitive path (infra/gitea/approval-policy.json), so it will wait for an administrator's /approve-override regardless of this rule.",
      "matchManagers": ["custom.regex"],
      "matchPackageNames": [
        "gitea.home.keithjackson.dev/home-network/toolbox",
        "gitea.home.keithjackson.dev/home-network/agent-runner",
        "gitea.home.keithjackson.dev/home-network/conformance-runner"
      ],
      "addLabels": ["own-image"]
    }
  ],
  // WHAT RENOVATE STILL CANNOT FIX, so nobody mistakes a working bot for full coverage. A
@@ -168,10 +243,20 @@
  // shape, not software to keep current. `conformance/**` was already ignored for the same
  // reason; this closes the matching hole that enabling the `kubernetes` manager would otherwise
  // have opened.
  // conformance/** NARROWED 2026-10-29 to its subdirectories: ignorePaths applies to custom
  // managers too (lib/workers/repository/extract/file-match.ts getMatchingFiles ->
  // filterIgnoredFiles), so the blanket glob also hid conformance/versions.env, which now carries
  // the conformance-runner digest Renovate must maintain. Everything else under conformance/
  // stays ignored, for the original reason (test fixtures with deliberately fixed shapes).
  "ignorePaths": [
    "**/*.example",
    "**/*.sops.yaml",
-    "conformance/**",
+    "conformance/config/**",
    "conformance/incluster/**",
    "conformance/lib/**",
    "conformance/requirements/**",
    "conformance/runs/**",
    "conformance/tests/**",
    "platform/image-automation/per-service/**",
    "platform/kyverno/tests/**"
  ],
@@ -67,10 +67,10 @@ Work through these in order; the first three are by far the most likely.
 | Symptom in the log | Cause | Fix |
 |---|---|---|
 | `GET /user returned no login — is GITEA_TOKEN valid?` and a CrashLoop | the credentials are still the committed **placeholders**, or the token was revoked | fill them in per `platform/pm-agent/pm-agent-credentials.sops.yaml.example` |
-| `ImagePullBackOff`, digest `sha256:0000…` | the image has never been pushed; the committed digest is the deliberate placeholder | `infra/agent/build-image.sh`, operator pushes, then one PR updates the digest in **both** `deployment.yaml` and `configmap.yaml` (`PM_WORKER_IMAGE`) — but read the row below first |
+| `ImagePullBackOff`, digest `sha256:0000…` | the image has never been published; the committed digest is the deliberate placeholder | CI publishes it, not a person: check the latest `images` workflow run on `main` (it fails loudly if the `REGISTRY_PUSH_TOKEN` secret is missing; re-run it with `workflow_dispatch`), then merge Renovate's digest-bump PR, which updates **both** `deployment.yaml` and `configmap.yaml` (`PM_WORKER_IMAGE`) together — but read the row below first |
 | `ImagePullBackOff` with `x509: certificate is not valid for any names` | **diagnosed.** containerd pulls as a node process, resolves through the node's public upstreams, and gets the CNAME to the WAN IP — which this router does not hairpin, so it answers :443 with its own `CN=%D` placeholder certificate (no SANs at all). Nothing to do with the digest, the Gateway or the wildcard cert | the `/etc/hosts` pin for `gitea.{{ k3s_base_domain }}` → `192.168.2.20` in `infra/ansible/inventory/group_vars/k3s_prod/vars.yml` fixes it — the same pin the Keycloak issuer already has. Apply it with the project's own tooling, scoped to just that task: `cd infra/ansible && ansible-playbook playbooks/node-baseline.yml -e k3s_hosts=k3s_prod --tags hosts`. Full evidence: `decisions/2026-10-06-pm-agent-node-resolves-gitea-to-wan.md` |
 | `ImagePullBackOff` with `401 Unauthorized` once TLS works | the pull is unauthenticated. An anonymous client gets nothing from this Gitea's registry, and no `imagePullSecret` exists anywhere in the tree | add one (a `dockerconfigjson`, SOPS-encrypted like `pm-agent-credentials`, referenced from the Deployment **and** the committed worker Job template) or allow anonymous reads of the package in Gitea — a posture decision, so yours |
-| `PM_WORKER_IMAGE must be digest-pinned` | someone put a tag in the ConfigMap | use the digest from `podman push --digestfile` |
+| `PM_WORKER_IMAGE must be digest-pinned` | someone put a tag in the ConfigMap | restore the digest form; the digest comes from CI's publish (`.gitea/workflows/images.yaml` job summary) and Renovate's bump PR, never from a hand push |
 | `scope: ONLY issues labelled 'agent'` | `PM_REQUIRE_LABEL` is set; unlabelled issues are out of scope by configuration | label the issue, or clear the knob |
 | `cycle: 2 worker(s) active` and nothing new | at `PM_MAX_ACTIVE_WORKERS` | wait, or raise it (RAM is the binding constraint: each worker's limit is 2Gi) |
 | `at the worker cap` / `worker limit reached` | the per-cycle cap, draining a backlog one issue at a time | expected; it will continue next cycle |