feat(images): build on rootless BuildKit over mTLS; verification moves into Containerfile verify stages
Operator decision on PR #81: no CI job may reach a privileged daemon. The act runner is untouched. - platform/buildkit (+ clusters/base/buildkit.yaml): moby/buildkit v0.34.1-rootless pinned by digest, uid 1000, not privileged; caps drop ALL + SETUID/SETGID, seccomp/AppArmor Unconfined, read-only rootfs (each measured under podman). cert-manager CA + server/client certs, ClusterIP :1234, default-deny with ingress only from the act-runner pod (DinD masquerades jobs to it). No Kyverno exception: every applied policy matches app-namespace=true only. - Each Containerfile names its shipped stage and ends in a `verify` stage holding every former `docker run` assertion (uid 65532, RUN --network=none, writable-path inventory standing in for the read-only rootfs). The old `&& echo` checks that could never fail are real assertions now. - images.sh drives buildkitd with a pinned buildx (sha256-checked) via bake; PRs build verify targets with agent-runner FROM the in-run toolbox (target: context); main verifies, then pushes. Fails loudly when the builder secrets are absent or buildkitd is unreachable. - infra/gitea/buildkit-client-secret.sh copies the client cert into Actions secrets. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
jakman85andClaude Opus 5.5 committed
1 parent
65a7769f55
commit
310deb0f7d
28 files changed
+1490
-608
No files matched your search
@@ -8,7 +8,10 @@
|
||||
# packages are retained as harmless disk-image utilities. Fedora base is pinned for
|
||||
# reproducibility.
|
||||
ARG FEDORA_VERSION=44
|
||||
FROM registry.fedoraproject.org/fedora:${FEDORA_VERSION}
|
||||
# Stage `toolbox` is the image that ships. Stage `verify` (end of file) is a TEST stage: it is
|
||||
# built FROM the finished image and only runs assertions, and it is never pushed. Build with
|
||||
# `--target toolbox` for the image and `--target verify` to prove it (build-image.sh does both).
|
||||
FROM registry.fedoraproject.org/fedora:${FEDORA_VERSION} AS toolbox
|
||||
|
||||
ARG OPENTOFU_VERSION
|
||||
ARG AGE_VERSION
|
||||
@@ -95,3 +98,48 @@ RUN { echo "opentofu=${OPENTOFU_VERSION}"; echo "age=${AGE_VERSION}"; echo "sops
|
||||
|
||||
WORKDIR /work
|
||||
CMD ["/bin/bash"]
|
||||
|
||||
# =============================================================================
|
||||
# verify — a TEST stage, never shipped. It runs every assertion against the finished image AT
|
||||
# BUILD TIME, so CI needs no container runtime to prove the image: the image pipeline builds
|
||||
# this target on rootless BuildKit (platform/buildkit/), where `docker run` does not exist.
|
||||
# `RUN --network=none` reproduces the old `run --network none` exactly, and the first check
|
||||
# below proves it (it fails if any network is reachable).
|
||||
# =============================================================================
|
||||
FROM toolbox AS verify
|
||||
ARG PROMTOOL_VERSION
|
||||
ARG KUBECTL_VERSION
|
||||
ARG CHAINSAW_VERSION
|
||||
ARG KYVERNO_VERSION
|
||||
ARG SOPS_VERSION
|
||||
RUN --network=none /bin/bash -euo pipefail <<'VERIFY'
|
||||
echo "--- no network (the build-time stand-in for run --network none) ---"
|
||||
if getent hosts registry.fedoraproject.org >/dev/null 2>&1 || curl -sS --max-time 3 -o /dev/null https://1.1.1.1 2>/dev/null; then
|
||||
echo "UNEXPECTED: network reachable in the verify stage"; exit 1
|
||||
fi
|
||||
echo " confirmed: no network"
|
||||
echo "--- baked versions ---"; cat /etc/dev-toolchain-versions
|
||||
echo "--- every tool is on PATH ---"
|
||||
for t in tofu age age-keygen sops flux kubectl helm kyverno kyverno-json chainsaw yq \
|
||||
yamllint check-jsonschema promtool ansible-playbook git make jq diff python3 \
|
||||
wg qrencode; do
|
||||
if command -v "$t" >/dev/null 2>&1; then printf " ok %s\n" "$t";
|
||||
else printf " MISSING %s\n" "$t"; missing=1; fi
|
||||
done
|
||||
[ -z "${missing:-}" ] || { echo "toolbox is missing tools"; exit 1; }
|
||||
echo "--- version-sensitive tools report their pins ---"
|
||||
: "${PROMTOOL_VERSION:?}" "${KUBECTL_VERSION:?}" "${CHAINSAW_VERSION:?}" "${KYVERNO_VERSION:?}" "${SOPS_VERSION:?}"
|
||||
check() { # name pin actual-output
|
||||
if printf "%s" "$3" | grep -qF "$2"; then printf " ok %-10s %s\n" "$1" "$2";
|
||||
else printf " DRIFT %-10s want %s, got: %s\n" "$1" "$2" "$(printf "%s" "$3" | head -1)"; drift=1; fi
|
||||
}
|
||||
check promtool "${PROMTOOL_VERSION}" "$(promtool --version 2>&1)"
|
||||
check kubectl "${KUBECTL_VERSION}" "$(kubectl version --client 2>&1)"
|
||||
check chainsaw "${CHAINSAW_VERSION}" "$(chainsaw version 2>&1)"
|
||||
check kyverno "${KYVERNO_VERSION}" "$(kyverno version 2>&1)"
|
||||
check sops "${SOPS_VERSION}" "$(sops --version --disable-version-check 2>&1 || sops --version 2>&1)"
|
||||
[ -z "${drift:-}" ] || { echo "a tool does not report its pinned version"; exit 1; }
|
||||
echo "--- python can import what the renderer and the gate need ---"
|
||||
python3 -c "import yaml; print(' PyYAML', yaml.__version__)"
|
||||
echo "TOOLBOX VERIFIED"
|
||||
VERIFY
|
||||
@@ -36,9 +36,11 @@ cd infra/devcontainer
|
||||
|
||||
### The published toolbox
|
||||
|
||||
`./dev.sh build` delegates to [`build-image.sh`](build-image.sh), which is the **one** build
|
||||
recipe: CI runs the same script (with its in-image verification — every tool executes, and the
|
||||
version-sensitive ones report their `versions.env` pins) and, on a push to `main` that changes
|
||||
`./dev.sh build` delegates to [`build-image.sh`](build-image.sh). The recipe is the
|
||||
`Containerfile` plus that script's build-arg list, and the proof is the Containerfile's `verify`
|
||||
stage (every tool on `PATH`, the version-sensitive ones report their `versions.env` pins, no
|
||||
network). CI builds the same Containerfile on rootless BuildKit with the args from
|
||||
`build-image.sh --print-build-args`, and, on a push to `main` that changes
|
||||
`Containerfile`, `versions.env` or `build-image.sh`, **publishes** the result as
|
||||
`gitea.home.keithjackson.dev/home-network/toolbox` (tags `sha-<commit>` and `main`) —
|
||||
see [`.gitea/workflows/images.yaml`](../../.gitea/workflows/images.yaml). `agent-runner` is
|
||||
|
||||
@@ -6,12 +6,14 @@
|
||||
# ./build-image.sh --no-verify # build only
|
||||
# IMAGE=foo:bar ./build-image.sh # build under another local name
|
||||
# ENGINE=docker ./build-image.sh # build with docker instead of podman
|
||||
# ./build-image.sh --print-build-args # print KEY=VALUE build args (what CI consumes)
|
||||
#
|
||||
# ONE BUILD DEFINITION, TWO CALLERS. `dev.sh build` calls this for local development, and the
|
||||
# CI image pipeline (.gitea/workflows/images.yaml -> .gitea/workflows/scripts/images.sh) calls
|
||||
# it to build the image it then PUBLISHES as gitea.home.keithjackson.dev/home-network/toolbox.
|
||||
# Keeping the build args in exactly one place is what makes "the toolbox CI publishes" and "the
|
||||
# toolbox you gate with locally" the same recipe.
|
||||
# ONE BUILD DEFINITION, TWO CALLERS. `dev.sh build` calls this for local development. The CI
|
||||
# image pipeline (.gitea/workflows/images.yaml -> .gitea/workflows/scripts/images.sh) builds the
|
||||
# SAME Containerfile on rootless BuildKit (platform/buildkit/) — no container engine exists in a
|
||||
# CI job — taking its build args from `--print-build-args`, and PUBLISHES the result as
|
||||
# gitea.home.keithjackson.dev/home-network/toolbox. The Containerfile (including its `verify`
|
||||
# test stage) and the arg list here are the one recipe both use.
|
||||
#
|
||||
# THIS SCRIPT NEVER PUSHES. Publishing is CI's job (images.yaml, on push to main), not a
|
||||
# workstation's: an image that exists only because someone ran `podman push` from a laptop has
|
||||
@@ -26,9 +28,11 @@ ENGINE="${ENGINE:-podman}"
|
||||
IMAGE="${IMAGE:-home-k3s-dev:local}"
|
||||
|
||||
VERIFY=1
|
||||
PRINT_ARGS=0
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
--no-verify) VERIFY=0 ;;
|
||||
--print-build-args) PRINT_ARGS=1 ;;
|
||||
-h|--help) sed -n '2,/^####/p' "${BASH_SOURCE[0]}" | sed '$d' | sed 's/^# \{0,1\}//'; exit 0 ;;
|
||||
*) echo "unknown argument: $arg" >&2; exit 2 ;;
|
||||
esac
|
||||
@@ -37,72 +41,44 @@ done
|
||||
log() { printf '\n\033[1m==> %s\033[0m\n' "$*"; }
|
||||
die() { printf '\033[31mERROR: %s\033[0m\n' "$*" >&2; exit 1; }
|
||||
|
||||
command -v "${ENGINE}" >/dev/null 2>&1 || die "'${ENGINE}' not on PATH (set ENGINE=docker to use docker)"
|
||||
|
||||
# shellcheck disable=SC1091
|
||||
source "${HERE}/versions.env"
|
||||
|
||||
# THE build args, in ONE place. `--print-build-args` hands exactly this list to the CI pipeline
|
||||
# (.gitea/workflows/scripts/images.sh), which builds on rootless BuildKit rather than calling
|
||||
# "$ENGINE build" — so CI and a local build cannot drift apart on a pin.
|
||||
BUILD_ARGS=(
|
||||
"FEDORA_VERSION=$FEDORA_VERSION" "OPENTOFU_VERSION=$OPENTOFU_VERSION" "AGE_VERSION=$AGE_VERSION"
|
||||
"SOPS_VERSION=$SOPS_VERSION" "FLUX_VERSION=$FLUX_VERSION" "KUBECTL_VERSION=$KUBECTL_VERSION"
|
||||
"HELM_VERSION=$HELM_VERSION" "KYVERNO_VERSION=$KYVERNO_VERSION"
|
||||
"KYVERNO_JSON_VERSION=$KYVERNO_JSON_VERSION" "CHAINSAW_VERSION=$CHAINSAW_VERSION"
|
||||
"YAMLLINT_VERSION=$YAMLLINT_VERSION" "YQ_VERSION=$YQ_VERSION"
|
||||
"CHECK_JSONSCHEMA_VERSION=$CHECK_JSONSCHEMA_VERSION" "PROMTOOL_VERSION=$PROMTOOL_VERSION"
|
||||
)
|
||||
if [ "${PRINT_ARGS}" -eq 1 ]; then printf '%s\n' "${BUILD_ARGS[@]}"; exit 0; fi
|
||||
|
||||
command -v "${ENGINE}" >/dev/null 2>&1 || die "'${ENGINE}' not on PATH (set ENGINE=docker to use docker)"
|
||||
|
||||
ARG_FLAGS=()
|
||||
for a in "${BUILD_ARGS[@]}"; do ARG_FLAGS+=(--build-arg "$a"); done
|
||||
# Provenance: CI passes the commit it is building, so a pulled image names its source tree.
|
||||
LABEL_ARGS=()
|
||||
[ -z "${BUILD_REVISION:-}" ] || LABEL_ARGS+=(--label "org.opencontainers.image.revision=${BUILD_REVISION}")
|
||||
|
||||
log "Building ${IMAGE}"
|
||||
"$ENGINE" build "${LABEL_ARGS[@]}" \
|
||||
--build-arg FEDORA_VERSION="$FEDORA_VERSION" \
|
||||
--build-arg OPENTOFU_VERSION="$OPENTOFU_VERSION" \
|
||||
--build-arg AGE_VERSION="$AGE_VERSION" \
|
||||
--build-arg SOPS_VERSION="$SOPS_VERSION" \
|
||||
--build-arg FLUX_VERSION="$FLUX_VERSION" \
|
||||
--build-arg KUBECTL_VERSION="$KUBECTL_VERSION" \
|
||||
--build-arg HELM_VERSION="$HELM_VERSION" \
|
||||
--build-arg KYVERNO_VERSION="$KYVERNO_VERSION" \
|
||||
--build-arg KYVERNO_JSON_VERSION="$KYVERNO_JSON_VERSION" \
|
||||
--build-arg CHAINSAW_VERSION="$CHAINSAW_VERSION" \
|
||||
--build-arg YAMLLINT_VERSION="$YAMLLINT_VERSION" \
|
||||
--build-arg YQ_VERSION="$YQ_VERSION" \
|
||||
--build-arg CHECK_JSONSCHEMA_VERSION="$CHECK_JSONSCHEMA_VERSION" \
|
||||
--build-arg PROMTOOL_VERSION="$PROMTOOL_VERSION" \
|
||||
-t "$IMAGE" -f "${HERE}/Containerfile" "${HERE}"
|
||||
|
||||
# --- verify ------------------------------------------------------------------
|
||||
# A build that succeeded is not evidence the toolbox can run the gate. Every tool the gate and
|
||||
# the runbooks call must EXECUTE, and the version-sensitive ones must report the version
|
||||
# versions.env pins — promtool above all, because ci/versions.env asserts a promtool/Prometheus
|
||||
# lockstep and a drifted engine evaluates alert-rule tests differently (see versions.env).
|
||||
# --- verify --------------------------------------------------------------------
|
||||
# The assertions live in the Containerfile's `verify` stage (every tool on PATH, the
|
||||
# version-sensitive ones report their versions.env pins — promtool above all, because
|
||||
# ci/versions.env asserts a promtool/Prometheus lockstep — and no network), so the SAME proof
|
||||
# runs here under podman and in CI under rootless BuildKit. A failing assertion fails the build.
|
||||
if [ "${VERIFY}" -eq 1 ]; then
|
||||
log "Verifying the toolchain inside ${IMAGE}"
|
||||
"$ENGINE" run --rm --network none --entrypoint "" \
|
||||
-e PROMTOOL_VERSION="${PROMTOOL_VERSION}" \
|
||||
-e KUBECTL_VERSION="${KUBECTL_VERSION}" \
|
||||
-e CHAINSAW_VERSION="${CHAINSAW_VERSION}" \
|
||||
-e KYVERNO_VERSION="${KYVERNO_VERSION}" \
|
||||
-e SOPS_VERSION="${SOPS_VERSION}" \
|
||||
"$IMAGE" /bin/bash -euo pipefail -c '
|
||||
echo "--- baked versions ---"; cat /etc/dev-toolchain-versions
|
||||
echo "--- every tool is on PATH ---"
|
||||
for t in tofu age age-keygen sops flux kubectl helm kyverno kyverno-json chainsaw yq \
|
||||
yamllint check-jsonschema promtool ansible-playbook git make jq diff python3 \
|
||||
wg qrencode; do
|
||||
if command -v "$t" >/dev/null 2>&1; then printf " ok %s\n" "$t";
|
||||
else printf " MISSING %s\n" "$t"; missing=1; fi
|
||||
done
|
||||
[ -z "${missing:-}" ] || { echo "toolbox is missing tools"; exit 1; }
|
||||
echo "--- version-sensitive tools report their pins ---"
|
||||
check() { # name pin actual-output
|
||||
if printf "%s" "$3" | grep -qF "$2"; then printf " ok %-10s %s\n" "$1" "$2";
|
||||
else printf " DRIFT %-10s want %s, got: %s\n" "$1" "$2" "$(printf "%s" "$3" | head -1)"; drift=1; fi
|
||||
}
|
||||
check promtool "${PROMTOOL_VERSION}" "$(promtool --version 2>&1)"
|
||||
check kubectl "${KUBECTL_VERSION}" "$(kubectl version --client 2>&1)"
|
||||
check chainsaw "${CHAINSAW_VERSION}" "$(chainsaw version 2>&1)"
|
||||
check kyverno "${KYVERNO_VERSION}" "$(kyverno version 2>&1)"
|
||||
check sops "${SOPS_VERSION}" "$(sops --version --disable-version-check 2>&1 || sops --version 2>&1)"
|
||||
[ -z "${drift:-}" ] || { echo "a tool does not report its pinned version"; exit 1; }
|
||||
echo "--- python can import what the renderer and the gate need ---"
|
||||
python3 -c "import yaml; print(\" PyYAML\", yaml.__version__)"
|
||||
'
|
||||
log "Verifying (Containerfile target: verify)"
|
||||
"$ENGINE" build "${ARG_FLAGS[@]}" --target verify -f "${HERE}/Containerfile" "${HERE}"
|
||||
log "Toolbox verified"
|
||||
fi
|
||||
|
||||
log "Building ${IMAGE}"
|
||||
"$ENGINE" build "${LABEL_ARGS[@]}" "${ARG_FLAGS[@]}" --target toolbox \
|
||||
-t "$IMAGE" -f "${HERE}/Containerfile" "${HERE}"
|
||||
|
||||
log "Built ${IMAGE}"
|
||||
printf ' image id %s\n' "$("$ENGINE" image inspect --format '{{.Id}}' "$IMAGE")"
|
||||
Reference in new issue
Block a user