feat(images): build on rootless BuildKit over mTLS; verification moves into Containerfile verify stages

Operator decision on PR #81: no CI job may reach a privileged daemon. The act runner is untouched.

- platform/buildkit (+ clusters/base/buildkit.yaml): moby/buildkit v0.34.1-rootless pinned by
  digest, uid 1000, not privileged; caps drop ALL + SETUID/SETGID, seccomp/AppArmor Unconfined,
  read-only rootfs (each measured under podman). cert-manager CA + server/client certs, ClusterIP
  :1234, default-deny with ingress only from the act-runner pod (DinD masquerades jobs to it).
  No Kyverno exception: every applied policy matches app-namespace=true only.
- Each Containerfile names its shipped stage and ends in a `verify` stage holding every former
  `docker run` assertion (uid 65532, RUN --network=none, writable-path inventory standing in for
  the read-only rootfs). The old `&& echo` checks that could never fail are real assertions now.
- images.sh drives buildkitd with a pinned buildx (sha256-checked) via bake; PRs build verify
  targets with agent-runner FROM the in-run toolbox (target: context); main verifies, then pushes.
  Fails loudly when the builder secrets are absent or buildkitd is unreachable.
- infra/gitea/buildkit-client-secret.sh copies the client cert into Actions secrets.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
jakman85andClaude Opus 5.5 committed 2026-10-29 22:21:03 -05:00
1 parent 65a7769f55
commit 310deb0f7d
28 files changed
+1490 -608

No files matched your search

+28 -26
View File
@@ -41,21 +41,22 @@
# not checked by Gitea (basic.go resolves the owner from the token), so `vars.REGISTRY_PUSH_USER`
# is optional and only labels the login.
#
# ┌─ BLOCKER, MEASURED ON PR #81 (run 325): THIS RUNNER CANNOT BUILD IMAGES YET ───────────────┐
# │ The job container gets no Docker daemon. act_runner 0.2.11 forces container.docker_host to │
# │ "-" whenever DOCKER_HOST is tcp:// (internal/app/cmd/daemon.go), and with valid_volumes: [] │
# │ gitea/act strips every bind including the socket (pkg/container/docker_run.go │
# │ sanitizeConfig). So this job fails at its first check, loudly, until the operator chooses │
# │ how workflow code may reach a daemon — see decisions/2026-10-29-image-pipeline.md. It is │
# │ not a required check, so it does not block merges; it is red, which is the honest state. │
# └──────────────────────────────────────────────────────────────────────────────────────────────┘
# THE BUILDER — rootless BuildKit (operator decision on PR #81, 2026-10-29).
# The act runner's job containers get NO Docker daemon (act_runner 0.2.11 forces docker_host "-"
# for a tcp:// DinD, and gitea/act strips every bind when valid_volumes is empty — measured on
# this PR's first run), and handing them the privileged DinD daemon would be root on the node for
# anyone who can push a branch. So images are built by platform/buildkit/: a ROOTLESS buildkitd
# (not privileged; uid 1000 in a user namespace), reached over mTLS at
# buildkitd.buildkit.svc.cluster.local:1234 with a pinned `buildx` (remote driver). Its client
# certificate reaches jobs as three Actions secrets — BUILDKIT_CA, BUILDKIT_CLIENT_CERT,
# BUILDKIT_CLIENT_KEY — copied from cert-manager by infra/gitea/buildkit-client-secret.sh.
# Pull requests get the client certificate (they must build) but never a registry credential.
#
# UNTIL platform/buildkit IS DEPLOYED (Flux applies it only after merge) THIS JOB FAILS, LOUDLY,
# at its builder preflight. A skipped Gitea job reports `success`, so it is red on purpose.
#
# VERIFICATION is each Containerfile's `verify` stage, built before anything is pushed.
#
# THE RUNNER (apps/wave-1/gitea-actions.yaml): act_runner 0.2.11, one job at a time, Docker
# executor on a privileged `docker:27-dind` sidecar whose image store is an emptyDir on the node
# (no sizeLimit) and whose memory limit is 2Gi — shared by the job container AND every build
# container. The toolbox is ~1.4 GB uncompressed and agent-runner adds ~0.3 GB on top of it, so the
# cleanup step below is not optional: without it each run would leave ~3 GB on a node that had
# 7 GB free when this was written, and the next eviction would take the runner with it.
name: images
on:
@@ -71,6 +72,7 @@ on:
- conformance/incluster/build-image.sh
- .gitea/workflows/images.yaml
- .gitea/workflows/scripts/images.sh
- platform/buildkit/versions.env
push:
branches: [main]
paths:
@@ -83,6 +85,7 @@ on:
- conformance/incluster/build-image.sh
- .gitea/workflows/images.yaml
- .gitea/workflows/scripts/images.sh
- platform/buildkit/versions.env
workflow_dispatch:
inputs:
images:
@@ -115,6 +118,11 @@ jobs:
MODE: verify
IMAGES: auto
PR_BASE_REF: ${{ github.base_ref }}
# The BUILDER's client certificate (platform/buildkit). Not a registry credential:
# a pull request can build, never push.
BUILDKIT_CA: ${{ secrets.BUILDKIT_CA }}
BUILDKIT_CLIENT_CERT: ${{ secrets.BUILDKIT_CLIENT_CERT }}
BUILDKIT_CLIENT_KEY: ${{ secrets.BUILDKIT_CLIENT_KEY }}
run: |
set -euo pipefail
BASE_SHA="$(git merge-base "origin/${PR_BASE_REF}" HEAD)"
@@ -127,6 +135,9 @@ jobs:
env:
MODE: verify
IMAGES: ${{ inputs.images == 'all' && 'toolbox agent-runner conformance-runner' || inputs.images }}
BUILDKIT_CA: ${{ secrets.BUILDKIT_CA }}
BUILDKIT_CLIENT_CERT: ${{ secrets.BUILDKIT_CLIENT_CERT }}
BUILDKIT_CLIENT_KEY: ${{ secrets.BUILDKIT_CLIENT_KEY }}
run: ./.gitea/workflows/scripts/images.sh
# ── main: publish ───────────────────────────────────────────────────────────────────────
@@ -168,16 +179,7 @@ jobs:
BASE_SHA: ${{ github.event.before }}
REGISTRY_TOKEN: ${{ secrets.REGISTRY_PUSH_TOKEN }}
REGISTRY_USER: ${{ vars.REGISTRY_PUSH_USER }}
BUILDKIT_CA: ${{ secrets.BUILDKIT_CA }}
BUILDKIT_CLIENT_CERT: ${{ secrets.BUILDKIT_CLIENT_CERT }}
BUILDKIT_CLIENT_KEY: ${{ secrets.BUILDKIT_CLIENT_KEY }}
run: ./.gitea/workflows/scripts/images.sh
# ── always: give the DinD store back ────────────────────────────────────────────────────
# The runner's image store is a node emptyDir with no sizeLimit (see header). Everything this
# job built or pulled is removed; `prune -a` cannot remove the job's own image, because the
# job container is still running on it.
- name: Clean the DinD image store
if: always()
run: |
docker logout gitea.home.keithjackson.dev >/dev/null 2>&1 || true
docker image prune -af >/dev/null || true
docker builder prune -af >/dev/null || true
docker system df || true