jakman85andClaude Opus 5
c8899368e7
The Containerfile said chainsaw "publishes no per-asset checksum file at a stable URL". Literally true, and misleading: it reads as "no integrity option exists". Checked the release assets via the GitHub API -- v0.2.12 ships per-asset cosign `.sig` + `.pem` and an SBOM alongside each tarball. So verification IS available, it just needs cosign in the builder stage and the right keyless identity for kyverno's release workflow. Comment now says what is actually true: kubectl is sha256-verified, chainsaw rests on tag + TLS, and the cosign path is named as a TODO rather than implied away. Deliberately not implemented here -- picking a trusted signer identity is a supply-chain decision that belongs in its own reviewed change, not a silent addition to a runner image. Noted that infra/devcontainer/Containerfile has the same gap for the same binary. Rebuilt after the edit; the in-image verification still passes. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
177 lines
9.9 KiB
Docker
177 lines
9.9 KiB
Docker
# =============================================================================
|
|
# conformance-runner — the image the in-cluster L2/L3 Chainsaw Job actually runs.
|
|
# Closes docs/plans/l2-l3-conformance-ci-wiring.md §3 item 1.
|
|
#
|
|
# WHY THIS EXISTS. chainsaw-job.yaml pins `ghcr.io/kyverno/chainsaw:v0.2.12` and
|
|
# `command: ["./run.sh"]`. That image carries the chainsaw binary and almost nothing else —
|
|
# no bash, no python3, no kubectl, no git. So the committed Job could never have run: it
|
|
# would have failed at exec of `./run.sh`. This image carries exactly what
|
|
# `conformance/run.sh` needs:
|
|
#
|
|
# bash run.sh and every tests/**/check.sh are bash (arrays, mapfile, BASH_SOURCE)
|
|
# python3 lib/{run_l1_offline,gen_coverage,merge_junit,emit_skipped_junit,status}.py
|
|
# (stdlib only — conformance/versions.env PYTHON_MIN=3.9, so no pip layer)
|
|
# chainsaw the L1/L2/L3 runner itself (run_chainsaw / lint_tests)
|
|
# kubectl conf_cluster_reachable()'s `kubectl cluster-info` preflight + probe scripts
|
|
# git the init-container's `clone --depth 1` + `checkout <SHA>` (§3 item 2) and
|
|
# run.sh's `rev-parse --short HEAD` for the run id
|
|
# tar/gzip/coreutils the `=== ARTIFACTS-TGZ-BASE64 ===` results channel (§3 item 3)
|
|
#
|
|
# VERSIONS COME FROM infra/devcontainer/versions.env — NOT from new pins invented here.
|
|
# That file is the single source of truth for CHAINSAW_VERSION / KUBECTL_VERSION /
|
|
# FEDORA_VERSION, and build-image.sh sources it and passes them in as --build-arg. The
|
|
# ARG defaults below are deliberately EMPTY so a build that forgets to pass them fails
|
|
# loudly (see the assertion in the first RUN) instead of silently baking a stale version.
|
|
#
|
|
# KNOWN LIMIT — READ BEFORE CHOOSING THE JOB'S `args`.
|
|
# `run.sh` runs the selected OFFLINE L0/L1 checks (run_offline_checks) before it ever
|
|
# touches the cluster, for ANY selection that is not offline-only. Those checks shell out
|
|
# to flux / promtool / kyverno / kustomize, which this image deliberately does NOT carry —
|
|
# baking the whole devcontainer toolchain into the runner would roughly double its size to
|
|
# re-run work that ci/l0.sh + ci/l1.sh already did offline in the same CI run. So the Job
|
|
# must select the cluster-gated layers only, e.g. `args: ["--layer", "l2"]` /
|
|
# `["--layer", "l3"]`, where offline_selected_dirs() resolves to nothing and
|
|
# run_offline_checks() returns 0. `--layer all` and a bare `--requirements <IDs>` would drag
|
|
# the L0 checks in and fail on a missing tool. Flagged to the plan owner; choosing the Job's
|
|
# args is T-6.1b's call, not this image's.
|
|
#
|
|
# THE ADMISSION CONTRACT THIS IMAGE MUST SATISFY.
|
|
# platform/kyverno/admission/conformance-trigger-job-shape.yaml is a fail-closed Enforce
|
|
# ClusterPolicy (R-SEC-02) and it is the security boundary. It requires the Job's container
|
|
# to be `gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:*` — a DIGEST,
|
|
# never a tag — with `runAsNonRoot: true` and `allowPrivilegeEscalation: false`. Two
|
|
# consequences are baked in here:
|
|
#
|
|
# 1. `USER` is NUMERIC. With `runAsNonRoot: true` and no explicit `runAsUser`, the kubelet
|
|
# REJECTS an image whose USER is a name it cannot resolve to a uid
|
|
# ("container has runAsNonRoot and image has non-numeric user"). So: USER 65532:65532.
|
|
# 2. The pod is `readOnlyRootFilesystem: true` and the policy admits only the `workspace`
|
|
# and `results` volumes — there is no writable /tmp and one cannot be added without
|
|
# changing the boundary. So HOME and every cache path point INTO /workspace (the
|
|
# emptyDir), not at the read-only root. That is why HOME=/workspace below.
|
|
# =============================================================================
|
|
|
|
# --- builder: fetch the pinned release binaries ------------------------------
|
|
# A full Fedora for the download stage (it has curl + tar); none of it ships in the runtime
|
|
# image, so the runner does not carry a downloader it has no business having at runtime.
|
|
ARG FEDORA_VERSION=44
|
|
FROM registry.fedoraproject.org/fedora:${FEDORA_VERSION} AS builder
|
|
|
|
ARG CHAINSAW_VERSION=
|
|
ARG KUBECTL_VERSION=
|
|
ARG TARGETARCH=amd64
|
|
|
|
# Fail the BUILD — not the Job, three weeks later — if a pin was not passed in.
|
|
RUN set -eux; \
|
|
: "${CHAINSAW_VERSION:?pass --build-arg CHAINSAW_VERSION from infra/devcontainer/versions.env}"; \
|
|
: "${KUBECTL_VERSION:?pass --build-arg KUBECTL_VERSION from infra/devcontainer/versions.env}"
|
|
|
|
# Same URLs and the same arch handling as infra/devcontainer/Containerfile, so the chainsaw
|
|
# and kubectl in this image are the same builds the toolbox gate runs with.
|
|
#
|
|
# INTEGRITY, MEASURED NOT ASSUMED. kubectl publishes a plain `<url>.sha256`, so it is verified
|
|
# below. chainsaw publishes NO checksums file — the v0.2.12 release assets are the tarballs plus
|
|
# per-asset cosign `.sig`/`.pem` and an SBOM (listed via the GitHub releases API). So chainsaw's
|
|
# integrity here rests on the tag + TLS, and afterwards on this image's own digest.
|
|
# TODO, worth doing and deliberately not smuggled in here: verify chainsaw with
|
|
# `cosign verify-blob --certificate chainsaw_linux_amd64.tar.gz.pem
|
|
# --signature chainsaw_linux_amd64.tar.gz.sig ...`. That needs cosign in the builder stage and
|
|
# the correct keyless identity/issuer for kyverno's release workflow — a supply-chain decision
|
|
# that belongs in its own reviewed change, not as a silent addition to a runner image.
|
|
# infra/devcontainer/Containerfile has the same gap for the same binary.
|
|
RUN set -eux; \
|
|
mkdir -p /out; \
|
|
curl -fsSL "https://dl.k8s.io/release/v${KUBECTL_VERSION}/bin/linux/${TARGETARCH}/kubectl" \
|
|
-o /out/kubectl; \
|
|
curl -fsSL "https://dl.k8s.io/release/v${KUBECTL_VERSION}/bin/linux/${TARGETARCH}/kubectl.sha256" \
|
|
-o /tmp/kubectl.sha256; \
|
|
echo "$(cat /tmp/kubectl.sha256) /out/kubectl" | sha256sum -c -; \
|
|
curl -fsSL "https://github.com/kyverno/chainsaw/releases/download/v${CHAINSAW_VERSION}/chainsaw_linux_${TARGETARCH}.tar.gz" \
|
|
| tar -xz -C /out chainsaw; \
|
|
chmod 0755 /out/kubectl /out/chainsaw; \
|
|
/out/kubectl version --client=true --output=yaml >/dev/null; \
|
|
/out/chainsaw version >/dev/null
|
|
|
|
# --- runtime -----------------------------------------------------------------
|
|
ARG FEDORA_VERSION=44
|
|
FROM registry.fedoraproject.org/fedora-minimal:${FEDORA_VERSION}
|
|
|
|
ARG CHAINSAW_VERSION=
|
|
ARG KUBECTL_VERSION=
|
|
ARG FEDORA_VERSION=44
|
|
# Nonroot uid/gid. 65532 is the distroless `nonroot` convention; the value is arbitrary but
|
|
# it MUST be numeric and non-zero — see "THE ADMISSION CONTRACT" above.
|
|
ARG RUNNER_UID=65532
|
|
ARG RUNNER_GID=65532
|
|
|
|
LABEL org.opencontainers.image.title="conformance-runner" \
|
|
org.opencontainers.image.description="bash + python3 + chainsaw + kubectl + git for the in-cluster L2/L3 conformance Job (conformance/run.sh)" \
|
|
org.opencontainers.image.source="https://gitea.home.keithjackson.dev/Home-Network/home-network-k3s-impl" \
|
|
org.opencontainers.image.documentation="conformance/incluster/README.md" \
|
|
homelab.keithjackson.dev/requirements="R-TEST-02,R-SEC-02"
|
|
|
|
# bash: run.sh + every check.sh. python3: the stdlib report generators.
|
|
# git-core: the init-container clone (git-core, not git — drops the perl/svn tail).
|
|
# coreutils/findutils/grep/sed/gawk: find, sort, base64, xargs, sed -nE, awk in lib/*.sh.
|
|
# tar+gzip: the base64-tar results channel. diffutils: `diff` in the L1 golden compare.
|
|
# ca-certificates: TLS for the init clone and for kubectl's in-cluster API calls.
|
|
RUN set -eux; \
|
|
microdnf -y install --setopt=install_weak_deps=0 --nodocs \
|
|
bash python3 git-core coreutils findutils grep sed gawk tar gzip diffutils \
|
|
ca-certificates; \
|
|
microdnf clean all; \
|
|
rm -rf /var/cache/* /var/log/dnf*
|
|
|
|
COPY --from=builder /out/kubectl /out/chainsaw /usr/local/bin/
|
|
|
|
# The nonroot account is written directly rather than pulling in shadow-utils to run useradd
|
|
# once. HOME is /workspace because at runtime that emptyDir is the only writable path.
|
|
RUN set -eux; \
|
|
echo "runner:x:${RUNNER_UID}:${RUNNER_GID}:conformance runner:/workspace:/bin/bash" >> /etc/passwd; \
|
|
echo "runner:x:${RUNNER_GID}:" >> /etc/group; \
|
|
install -d -m 0775 -o "${RUNNER_UID}" -g "${RUNNER_GID}" /workspace
|
|
|
|
# SYSTEM-level config, because HOME is an emptyDir that starts empty on every run and the
|
|
# root filesystem is read-only — a runtime `--global` write would fail.
|
|
# safe.directory=*: the init-container clones into the shared emptyDir and the main container
|
|
# then runs `rev-parse` over it; any uid mismatch between the two would otherwise abort with
|
|
# "detected dubious ownership". The repo is fetched fresh per run into an emptyDir, so there
|
|
# is no pre-existing hostile checkout this relaxes a check against.
|
|
RUN set -eux; \
|
|
printf '%s\n' \
|
|
'[safe]' \
|
|
' directory = *' \
|
|
'[advice]' \
|
|
' detachedHead = false' \
|
|
> /etc/gitconfig; \
|
|
chmod 0644 /etc/gitconfig
|
|
|
|
# Provenance: what is actually in this layer, readable from a running pod.
|
|
RUN set -eux; \
|
|
{ echo "fedora=${FEDORA_VERSION}"; \
|
|
echo "chainsaw=${CHAINSAW_VERSION}"; \
|
|
echo "kubectl=${KUBECTL_VERSION}"; \
|
|
echo "bash=$(bash --version | head -1)"; \
|
|
echo "python3=$(python3 --version)"; \
|
|
echo "git=$(git --version)"; \
|
|
} > /etc/conformance-runner-versions; \
|
|
chmod 0444 /etc/conformance-runner-versions
|
|
|
|
# Every writable path points into the `workspace` emptyDir — the root filesystem is
|
|
# read-only and the admission policy forbids adding a third volume for scratch.
|
|
ENV HOME=/workspace \
|
|
XDG_CACHE_HOME=/workspace/.cache \
|
|
XDG_CONFIG_HOME=/workspace/.config \
|
|
TMPDIR=/workspace/.tmp \
|
|
PYTHONDONTWRITEBYTECODE=1 \
|
|
CONF_ROOT=/workspace/conformance
|
|
|
|
USER ${RUNNER_UID}:${RUNNER_GID}
|
|
WORKDIR /workspace
|
|
|
|
# No ENTRYPOINT on purpose. The admission policy pins `command: ["./run.sh"]` with
|
|
# `workingDir: /workspace/conformance`; an ENTRYPOINT here would either be ignored or
|
|
# prepend something the policy cannot see. `bash` is the interactive fallback for a human
|
|
# debugging the image locally.
|
|
CMD ["/bin/bash"]
|