Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b105c764fc | ||
|
|
ab9ba32a76 | ||
|
|
529556d7c7 | ||
|
|
8619c60e6e | ||
|
|
c59fe9f923 |
No files matched your search
@@ -0,0 +1,200 @@
|
||||
# =============================================================================
|
||||
# conformance-runner — the image the in-cluster L2/L3 Chainsaw Job actually runs.
|
||||
# Closes docs/plans/l2-l3-conformance-ci-wiring.md §3 item 1.
|
||||
#
|
||||
# WHY THIS EXISTS. chainsaw-job.yaml pins `ghcr.io/kyverno/chainsaw:v0.2.12` and
|
||||
# `command: ["./run.sh"]`. That image carries the chainsaw binary and almost nothing else —
|
||||
# no bash, no python3, no kubectl, no git. So the committed Job could never have run: it
|
||||
# would have failed at exec of `./run.sh`. This image carries exactly what
|
||||
# `conformance/run.sh` needs:
|
||||
#
|
||||
# bash run.sh and every tests/**/check.sh are bash (arrays, mapfile, BASH_SOURCE)
|
||||
# python3 lib/{run_l1_offline,gen_coverage,merge_junit,emit_skipped_junit,status}.py
|
||||
# (stdlib only — conformance/versions.env PYTHON_MIN=3.9, so no pip layer)
|
||||
# chainsaw the L1/L2/L3 runner itself (run_chainsaw / lint_tests)
|
||||
# kubectl conf_cluster_reachable()'s `kubectl cluster-info` preflight + probe scripts
|
||||
# git the init-container's `clone --depth 1` + `checkout <SHA>` (§3 item 2) and
|
||||
# run.sh's `rev-parse --short HEAD` for the run id
|
||||
# tar/gzip/coreutils the `=== ARTIFACTS-TGZ-BASE64 ===` results channel (§3 item 3)
|
||||
#
|
||||
# It also carries `/usr/local/bin/conformance-clone` (0755 root:root) — the init container's
|
||||
# ONLY entrypoint, a fixed path with no argument surface. That exists to close an admission
|
||||
# hole, not for convenience: see the COPY below and the script's own header.
|
||||
#
|
||||
# VERSIONS COME FROM infra/devcontainer/versions.env — NOT from new pins invented here.
|
||||
# That file is the single source of truth for CHAINSAW_VERSION / KUBECTL_VERSION /
|
||||
# FEDORA_VERSION, and build-image.sh sources it and passes them in as --build-arg. The
|
||||
# ARG defaults below are deliberately EMPTY so a build that forgets to pass them fails
|
||||
# loudly (see the assertion in the first RUN) instead of silently baking a stale version.
|
||||
#
|
||||
# KNOWN LIMIT — READ BEFORE CHOOSING THE JOB'S `args`.
|
||||
# `run.sh` runs the selected OFFLINE L0/L1 checks (run_offline_checks) before it ever
|
||||
# touches the cluster, for ANY selection that is not offline-only. Those checks shell out
|
||||
# to flux / promtool / kyverno / kustomize, which this image deliberately does NOT carry —
|
||||
# baking the whole devcontainer toolchain into the runner would roughly double its size to
|
||||
# re-run work that ci/l0.sh + ci/l1.sh already did offline in the same CI run. So the Job
|
||||
# must select the cluster-gated layers only, e.g. `args: ["--layer", "l2"]` /
|
||||
# `["--layer", "l3"]`, where offline_selected_dirs() resolves to nothing and
|
||||
# run_offline_checks() returns 0. `--layer all` and a bare `--requirements <IDs>` would drag
|
||||
# the L0 checks in and fail on a missing tool. Flagged to the plan owner; choosing the Job's
|
||||
# args is T-6.1b's call, not this image's.
|
||||
#
|
||||
# THE ADMISSION CONTRACT THIS IMAGE MUST SATISFY.
|
||||
# platform/kyverno/admission/conformance-trigger-job-shape.yaml is a fail-closed Enforce
|
||||
# ClusterPolicy (R-SEC-02) and it is the security boundary. It requires the Job's container
|
||||
# to be `gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:*` — a DIGEST,
|
||||
# never a tag — with `runAsNonRoot: true` and `allowPrivilegeEscalation: false`. Two
|
||||
# consequences are baked in here:
|
||||
#
|
||||
# 1. `USER` is NUMERIC. With `runAsNonRoot: true` and no explicit `runAsUser`, the kubelet
|
||||
# REJECTS an image whose USER is a name it cannot resolve to a uid
|
||||
# ("container has runAsNonRoot and image has non-numeric user"). So: USER 65532:65532.
|
||||
# 2. The pod is `readOnlyRootFilesystem: true` and the policy admits only the `workspace`
|
||||
# and `results` volumes — there is no writable /tmp and one cannot be added without
|
||||
# changing the boundary. So HOME and every cache path point INTO /workspace (the
|
||||
# emptyDir), not at the read-only root. That is why HOME=/workspace below.
|
||||
# =============================================================================
|
||||
|
||||
# --- builder: fetch the pinned release binaries ------------------------------
|
||||
# A full Fedora for the download stage (it has curl + tar); none of it ships in the runtime
|
||||
# image, so the runner does not carry a downloader it has no business having at runtime.
|
||||
ARG FEDORA_VERSION=44
|
||||
FROM registry.fedoraproject.org/fedora:${FEDORA_VERSION} AS builder
|
||||
|
||||
ARG CHAINSAW_VERSION=
|
||||
ARG KUBECTL_VERSION=
|
||||
ARG TARGETARCH=amd64
|
||||
|
||||
# Fail the BUILD — not the Job, three weeks later — if a pin was not passed in.
|
||||
RUN set -eux; \
|
||||
: "${CHAINSAW_VERSION:?pass --build-arg CHAINSAW_VERSION from infra/devcontainer/versions.env}"; \
|
||||
: "${KUBECTL_VERSION:?pass --build-arg KUBECTL_VERSION from infra/devcontainer/versions.env}"
|
||||
|
||||
# Same URLs and the same arch handling as infra/devcontainer/Containerfile, so the chainsaw
|
||||
# and kubectl in this image are the same builds the toolbox gate runs with.
|
||||
#
|
||||
# INTEGRITY, MEASURED NOT ASSUMED. kubectl publishes a plain `<url>.sha256`, so it is verified
|
||||
# below. chainsaw publishes NO checksums file — the v0.2.12 release assets are the tarballs plus
|
||||
# per-asset cosign `.sig`/`.pem` and an SBOM (listed via the GitHub releases API). So chainsaw's
|
||||
# integrity here rests on the tag + TLS, and afterwards on this image's own digest.
|
||||
# TODO, worth doing and deliberately not smuggled in here: verify chainsaw with
|
||||
# `cosign verify-blob --certificate chainsaw_linux_amd64.tar.gz.pem
|
||||
# --signature chainsaw_linux_amd64.tar.gz.sig ...`. That needs cosign in the builder stage and
|
||||
# the correct keyless identity/issuer for kyverno's release workflow — a supply-chain decision
|
||||
# that belongs in its own reviewed change, not as a silent addition to a runner image.
|
||||
# infra/devcontainer/Containerfile has the same gap for the same binary.
|
||||
RUN set -eux; \
|
||||
mkdir -p /out; \
|
||||
curl -fsSL "https://dl.k8s.io/release/v${KUBECTL_VERSION}/bin/linux/${TARGETARCH}/kubectl" \
|
||||
-o /out/kubectl; \
|
||||
curl -fsSL "https://dl.k8s.io/release/v${KUBECTL_VERSION}/bin/linux/${TARGETARCH}/kubectl.sha256" \
|
||||
-o /tmp/kubectl.sha256; \
|
||||
echo "$(cat /tmp/kubectl.sha256) /out/kubectl" | sha256sum -c -; \
|
||||
curl -fsSL "https://github.com/kyverno/chainsaw/releases/download/v${CHAINSAW_VERSION}/chainsaw_linux_${TARGETARCH}.tar.gz" \
|
||||
| tar -xz -C /out chainsaw; \
|
||||
chmod 0755 /out/kubectl /out/chainsaw; \
|
||||
/out/kubectl version --client=true --output=yaml >/dev/null; \
|
||||
/out/chainsaw version >/dev/null
|
||||
|
||||
# --- runtime -----------------------------------------------------------------
|
||||
ARG FEDORA_VERSION=44
|
||||
FROM registry.fedoraproject.org/fedora-minimal:${FEDORA_VERSION}
|
||||
|
||||
ARG CHAINSAW_VERSION=
|
||||
ARG KUBECTL_VERSION=
|
||||
ARG FEDORA_VERSION=44
|
||||
# Nonroot uid/gid. 65532 is the distroless `nonroot` convention; the value is arbitrary but
|
||||
# it MUST be numeric and non-zero — see "THE ADMISSION CONTRACT" above.
|
||||
ARG RUNNER_UID=65532
|
||||
ARG RUNNER_GID=65532
|
||||
|
||||
LABEL org.opencontainers.image.title="conformance-runner" \
|
||||
org.opencontainers.image.description="bash + python3 + chainsaw + kubectl + git for the in-cluster L2/L3 conformance Job (conformance/run.sh)" \
|
||||
org.opencontainers.image.source="https://gitea.home.keithjackson.dev/Home-Network/home-network-k3s-impl" \
|
||||
org.opencontainers.image.documentation="conformance/incluster/README.md" \
|
||||
homelab.keithjackson.dev/requirements="R-TEST-02,R-SEC-02"
|
||||
|
||||
# bash: run.sh + every check.sh. python3: the stdlib report generators.
|
||||
# git-core: the init-container clone (git-core, not git — drops the perl/svn tail).
|
||||
# coreutils/findutils/grep/sed/gawk: find, sort, base64, xargs, sed -nE, awk in lib/*.sh.
|
||||
# tar+gzip: the base64-tar results channel. diffutils: `diff` in the L1 golden compare.
|
||||
# ca-certificates: TLS for the init clone and for kubectl's in-cluster API calls.
|
||||
RUN set -eux; \
|
||||
microdnf -y install --setopt=install_weak_deps=0 --nodocs \
|
||||
bash python3 git-core coreutils findutils grep sed gawk tar gzip diffutils \
|
||||
ca-certificates; \
|
||||
microdnf clean all; \
|
||||
rm -rf /var/cache/* /var/log/dnf*
|
||||
|
||||
COPY --from=builder /out/kubectl /out/chainsaw /usr/local/bin/
|
||||
|
||||
# The init container's ONLY entrypoint. Copied as a file with a fixed path and NO argument
|
||||
# surface, because the admission policy pins the main container's `command` but never reads
|
||||
# `initContainers` — so an init container with `command: ["/bin/sh","-c"]` and an arbitrary
|
||||
# script is currently ADMITTED and runs as the privileged `conformance-runner` SA (reproduced
|
||||
# by the policy's author; pass 3, fail 0). A 4th policy rule requiring
|
||||
# `command: ["/usr/local/bin/conformance-clone"]` with no `args` is what makes this file
|
||||
# REQUIRED rather than merely available; that rule belongs in its own reviewed PR against the
|
||||
# boundary, not here. See the script's header for the env-injection half of the same hole.
|
||||
#
|
||||
# 0755 root:root: the container runs as 65532 and MUST NOT be able to replace the one
|
||||
# executable the policy will trust. Ownership is asserted below rather than assumed.
|
||||
COPY conformance-clone /usr/local/bin/conformance-clone
|
||||
RUN set -eux; \
|
||||
chown root:root /usr/local/bin/conformance-clone; \
|
||||
chmod 0755 /usr/local/bin/conformance-clone; \
|
||||
bash -n /usr/local/bin/conformance-clone; \
|
||||
/usr/local/bin/conformance-clone --anything 2>&1 | grep -q 'takes no arguments'; \
|
||||
test "$(stat -c '%U:%G %a' /usr/local/bin/conformance-clone)" = 'root:root 755'
|
||||
|
||||
# The nonroot account is written directly rather than pulling in shadow-utils to run useradd
|
||||
# once. HOME is /workspace because at runtime that emptyDir is the only writable path.
|
||||
RUN set -eux; \
|
||||
echo "runner:x:${RUNNER_UID}:${RUNNER_GID}:conformance runner:/workspace:/bin/bash" >> /etc/passwd; \
|
||||
echo "runner:x:${RUNNER_GID}:" >> /etc/group; \
|
||||
install -d -m 0775 -o "${RUNNER_UID}" -g "${RUNNER_GID}" /workspace
|
||||
|
||||
# SYSTEM-level config, because HOME is an emptyDir that starts empty on every run and the
|
||||
# root filesystem is read-only — a runtime `--global` write would fail.
|
||||
# safe.directory=*: the init-container clones into the shared emptyDir and the main container
|
||||
# then runs `rev-parse` over it; any uid mismatch between the two would otherwise abort with
|
||||
# "detected dubious ownership". The repo is fetched fresh per run into an emptyDir, so there
|
||||
# is no pre-existing hostile checkout this relaxes a check against.
|
||||
RUN set -eux; \
|
||||
printf '%s\n' \
|
||||
'[safe]' \
|
||||
' directory = *' \
|
||||
'[advice]' \
|
||||
' detachedHead = false' \
|
||||
> /etc/gitconfig; \
|
||||
chmod 0644 /etc/gitconfig
|
||||
|
||||
# Provenance: what is actually in this layer, readable from a running pod.
|
||||
RUN set -eux; \
|
||||
{ echo "fedora=${FEDORA_VERSION}"; \
|
||||
echo "chainsaw=${CHAINSAW_VERSION}"; \
|
||||
echo "kubectl=${KUBECTL_VERSION}"; \
|
||||
echo "bash=$(bash --version | head -1)"; \
|
||||
echo "python3=$(python3 --version)"; \
|
||||
echo "git=$(git --version)"; \
|
||||
echo "conformance-clone=$(sha256sum /usr/local/bin/conformance-clone | cut -d' ' -f1)"; \
|
||||
} > /etc/conformance-runner-versions; \
|
||||
chmod 0444 /etc/conformance-runner-versions
|
||||
|
||||
# Every writable path points into the `workspace` emptyDir — the root filesystem is
|
||||
# read-only and the admission policy forbids adding a third volume for scratch.
|
||||
ENV HOME=/workspace \
|
||||
XDG_CACHE_HOME=/workspace/.cache \
|
||||
XDG_CONFIG_HOME=/workspace/.config \
|
||||
TMPDIR=/workspace/.tmp \
|
||||
PYTHONDONTWRITEBYTECODE=1 \
|
||||
CONF_ROOT=/workspace/conformance
|
||||
|
||||
USER ${RUNNER_UID}:${RUNNER_GID}
|
||||
WORKDIR /workspace
|
||||
|
||||
# No ENTRYPOINT on purpose. The admission policy pins `command: ["./run.sh"]` with
|
||||
# `workingDir: /workspace/conformance`; an ENTRYPOINT here would either be ignored or
|
||||
# prepend something the policy cannot see. `bash` is the interactive fallback for a human
|
||||
# debugging the image locally.
|
||||
CMD ["/bin/bash"]
|
||||
@@ -27,7 +27,38 @@ credentials — the Job's own scoped ServiceAccount does (R-SEC-02).
|
||||
audit item.
|
||||
- `chainsaw-job.yaml` — the Job skeleton. Single entrypoint `./run.sh`; CI sets `args` to
|
||||
`--requirements <touched IDs>` (merge gate) or `--layer all --strict-coverage` (phase
|
||||
exit). Image pinned to `versions.env` `CHAINSAW_IMAGE`.
|
||||
exit). **Its `image:` is STALE**: it still names `ghcr.io/kyverno/chainsaw:v0.2.12`, which
|
||||
(a) cannot run `./run.sh` at all and (b) is DENIED at admission. The trigger renders this
|
||||
template per run with `CONFORMANCE_RUNNER_IMAGE` injected (plan §3 item 4, T-6.1b); until
|
||||
that lands, treat the literal `image:` here as a placeholder, not a pin.
|
||||
- `Containerfile` — the **`conformance-runner`** image: `bash` + `python3` + `chainsaw` +
|
||||
`kubectl` + `git`, non-root (numeric uid 65532) and working under a read-only root
|
||||
filesystem. This is what the Job actually needs; the stock chainsaw image carries none of
|
||||
it. Versions are `--build-arg`-injected from `infra/devcontainer/versions.env` — the single
|
||||
source of truth — so the chainsaw/kubectl in the runner are the same pins the toolbox gate
|
||||
uses. Read its header before changing the Job's `args`: `run.sh` runs the selected offline
|
||||
L0/L1 checks for any non-offline-only selection, and those need flux/promtool/kyverno/
|
||||
kustomize, which this image deliberately omits. Use `--layer l2` / `--layer l3`.
|
||||
- `conformance-clone` — the init container's **only** entrypoint, baked into the image at
|
||||
`/usr/local/bin/conformance-clone` (0755 **root:root**, so uid 65532 cannot replace it).
|
||||
Clones the impl repo at the commit under test: the SHA must match `^[0-9a-f]{40}$` and
|
||||
`rev-parse HEAD` is **asserted** equal to it afterwards, so an unrendered placeholder is a
|
||||
hard failure rather than a silent run of the branch tip. It takes **no arguments**, and the
|
||||
repository URL and clone target are **compiled in** — see its header for why the env vars
|
||||
are read but may not override them.
|
||||
> **This file is half a fix.** The admission policy pins the main container's `command` but
|
||||
> **never reads `initContainers`**, so an init container with `command: ["/bin/sh","-c"]` and
|
||||
> an arbitrary script is currently ADMITTED and runs as the privileged `conformance-runner`
|
||||
> SA. What makes this file *required* rather than merely available is a 4th policy rule
|
||||
> demanding `command: ["/usr/local/bin/conformance-clone"]` with no `args`. That rule belongs
|
||||
> in its own reviewed PR against the boundary and is **not** in this branch.
|
||||
- `build-image.sh` — builds the above, **verifies the toolchain by running it** inside the
|
||||
image as uid 65532 with `--read-only`, and prints the digest. It **never pushes**: it
|
||||
prints the exact `podman push --digestfile …` command for the operator instead. See
|
||||
`conformance/versions.env` `CONFORMANCE_RUNNER_IMAGE` for the post-push pin.
|
||||
`--verify-clone` additionally has `conformance-clone` really clone this repo at a real SHA
|
||||
and asserts the checkout matches (opt-in, because it needs the Gitea host reachable — the
|
||||
default verify stays offline so an air-gapped build is fully checked, not half-checked).
|
||||
|
||||
## Flow
|
||||
1. CI (Gitea Actions, delivered in T-6.1b) creates a Job from `chainsaw-job.yaml` (via a
|
||||
|
||||
Executable
+229
@@ -0,0 +1,229 @@
|
||||
#!/usr/bin/env bash
|
||||
###############################################################################
|
||||
# conformance/incluster/build-image.sh — build the pinned `conformance-runner`
|
||||
# image and print its digest. Closes docs/plans/l2-l3-conformance-ci-wiring.md §3 item 1.
|
||||
#
|
||||
# ./build-image.sh # build + verify + print the digest and push command
|
||||
# ./build-image.sh --no-verify # build only (skip the in-image tool proof)
|
||||
# ./build-image.sh --verify-clone # ALSO really clone this repo with conformance-clone.
|
||||
# # Opt-in because it needs the Gitea host reachable; the
|
||||
# # default verify is offline so an air-gapped build is
|
||||
# # still fully checked rather than half-checked.
|
||||
# ENGINE=docker ./build-image.sh # build with docker instead of podman
|
||||
#
|
||||
# THIS SCRIPT NEVER PUSHES. Publishing is an operator action: the registry credential is
|
||||
# not an agent's to hold, and the pushed digest is what a fail-closed admission policy
|
||||
# then trusts. So this builds locally, proves the toolchain works INSIDE the image as the
|
||||
# non-root uid the policy demands, and prints the exact `podman push` line for a human.
|
||||
#
|
||||
# VERSIONS ARE NOT DEFINED HERE. They are read from infra/devcontainer/versions.env, the
|
||||
# single source of truth (CHAINSAW_VERSION, KUBECTL_VERSION, FEDORA_VERSION) — the same
|
||||
# pins the toolbox gate runs with. Nothing in this file invents a version.
|
||||
#
|
||||
# THE DIGEST CAVEAT — read it before pasting a digest into anything.
|
||||
# The digest printed here is the LOCAL manifest digest. It is normally identical to the
|
||||
# registry digest (podman pushes the manifest it already has rather than recompressing),
|
||||
# but that is a property of the push, not a guarantee. So the push command below uses
|
||||
# `--digestfile`: the digest the registry actually returns is the authoritative one, and
|
||||
# that is the one that goes into conformance/versions.env.
|
||||
###############################################################################
|
||||
set -euo pipefail
|
||||
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
REPO_ROOT="$(cd "${HERE}/../.." && pwd)"
|
||||
ENGINE="${ENGINE:-podman}"
|
||||
|
||||
VERIFY=1
|
||||
VERIFY_CLONE=0
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
--no-verify) VERIFY=0 ;;
|
||||
--verify-clone) VERIFY_CLONE=1 ;;
|
||||
-h|--help) sed -n '2,/^####/p' "${BASH_SOURCE[0]}" | sed '$d' | sed 's/^# \{0,1\}//'; exit 0 ;;
|
||||
*) echo "unknown argument: $arg" >&2; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
log() { printf '\n\033[1m==> %s\033[0m\n' "$*"; }
|
||||
info() { printf ' %s\n' "$*"; }
|
||||
die() { printf '\033[31mERROR: %s\033[0m\n' "$*" >&2; exit 1; }
|
||||
|
||||
command -v "${ENGINE}" >/dev/null 2>&1 || die "'${ENGINE}' not on PATH (set ENGINE=docker to use docker)"
|
||||
|
||||
# --- pins: sourced, never invented -------------------------------------------
|
||||
TOOLCHAIN_ENV="${REPO_ROOT}/infra/devcontainer/versions.env"
|
||||
[ -f "${TOOLCHAIN_ENV}" ] || die "missing ${TOOLCHAIN_ENV} — the pins live there, not here"
|
||||
# shellcheck disable=SC1090
|
||||
source "${TOOLCHAIN_ENV}"
|
||||
: "${CHAINSAW_VERSION:?CHAINSAW_VERSION not set in ${TOOLCHAIN_ENV}}"
|
||||
: "${KUBECTL_VERSION:?KUBECTL_VERSION not set in ${TOOLCHAIN_ENV}}"
|
||||
: "${FEDORA_VERSION:?FEDORA_VERSION not set in ${TOOLCHAIN_ENV}}"
|
||||
|
||||
# --- image name: fixed by the admission policy -------------------------------
|
||||
# platform/kyverno/admission/conformance-trigger-job-shape.yaml admits ONLY
|
||||
# gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:*. The repository path
|
||||
# is therefore not a preference; a different path is a DENIED Job. Do not "improve" it here.
|
||||
IMAGE_REPO="gitea.home.keithjackson.dev/home-network/conformance-runner"
|
||||
# The tag is build convenience only (a human-readable handle for the push). Admission
|
||||
# ignores it: only the digest is ever referenced from a Job.
|
||||
IMAGE_TAG="chainsaw${CHAINSAW_VERSION}-kubectl${KUBECTL_VERSION}-fc${FEDORA_VERSION}"
|
||||
IMAGE_REF="${IMAGE_REPO}:${IMAGE_TAG}"
|
||||
|
||||
log "Building ${IMAGE_REF}"
|
||||
info "engine ${ENGINE} ($(${ENGINE} --version 2>/dev/null | head -1))"
|
||||
info "pins from ${TOOLCHAIN_ENV}"
|
||||
info " CHAINSAW ${CHAINSAW_VERSION}"
|
||||
info " KUBECTL ${KUBECTL_VERSION}"
|
||||
info " FEDORA ${FEDORA_VERSION}"
|
||||
|
||||
"${ENGINE}" build \
|
||||
--file "${HERE}/Containerfile" \
|
||||
--tag "${IMAGE_REF}" \
|
||||
--build-arg "CHAINSAW_VERSION=${CHAINSAW_VERSION}" \
|
||||
--build-arg "KUBECTL_VERSION=${KUBECTL_VERSION}" \
|
||||
--build-arg "FEDORA_VERSION=${FEDORA_VERSION}" \
|
||||
"${HERE}"
|
||||
|
||||
# --- verify: the point of the whole exercise ---------------------------------
|
||||
# The stock chainsaw image "had" chainsaw and still could not run ./run.sh. So prove the
|
||||
# tools EXECUTE, and prove they execute as the NON-ROOT uid with a READ-ONLY root
|
||||
# filesystem — the shape the admission policy forces. A pass here is the only evidence
|
||||
# this image is usable; a build that merely succeeded is not.
|
||||
if [ "${VERIFY}" -eq 1 ]; then
|
||||
log "Verifying the toolchain inside the image (non-root, read-only rootfs)"
|
||||
"${ENGINE}" run --rm \
|
||||
--user 65532:65532 \
|
||||
--read-only \
|
||||
--tmpfs /workspace:rw,mode=1777 \
|
||||
--entrypoint "" \
|
||||
"${IMAGE_REF}" \
|
||||
/bin/bash -euo pipefail -c '
|
||||
echo "--- id ---"; id
|
||||
echo "--- HOME ---"; echo "HOME=${HOME}"
|
||||
echo "--- bash ---"; bash --version | head -1
|
||||
echo "--- python3 ---"; python3 --version
|
||||
echo "--- git ---"; git --version
|
||||
echo "--- kubectl ---"; kubectl version --client
|
||||
echo "--- chainsaw ---"; chainsaw version
|
||||
echo "--- baked versions ---"; cat /etc/conformance-runner-versions
|
||||
echo "--- conformance-clone: fixed path, root-owned, NOT writable by the runner ---"
|
||||
stat -c "%n %U:%G %a" /usr/local/bin/conformance-clone
|
||||
if [ -w /usr/local/bin/conformance-clone ]; then
|
||||
echo "UNEXPECTED: uid $(id -u) can write the one executable the policy will trust"; exit 1
|
||||
fi
|
||||
echo "confirmed: not writable by uid $(id -u)"
|
||||
# Each of these MUST fail. Assert the non-zero exit as well as the message: a check
|
||||
# that only greps stderr would pass a build where the guard had stopped rejecting.
|
||||
# (`|| true` inside the capture keeps `set -e`/`pipefail` from aborting on the
|
||||
# expected failure — without it the verify dies at the first negative case.)
|
||||
expect_fail() {
|
||||
local label="$1"; shift
|
||||
local out rc=0
|
||||
out="$(env "$@" 2>&1)" || rc=$?
|
||||
echo "--- ${label} ---"
|
||||
printf "%s\n" "${out}" | head -2
|
||||
if [ "${rc}" -eq 0 ]; then
|
||||
echo "UNEXPECTED: that should have failed, but it exited 0"; exit 1
|
||||
fi
|
||||
echo "(rejected, exit ${rc})"
|
||||
}
|
||||
expect_fail "conformance-clone rejects arguments (that is where an injection would live)" \
|
||||
conformance-clone --layer l2
|
||||
expect_fail "conformance-clone rejects a missing SHA" \
|
||||
conformance-clone
|
||||
expect_fail "conformance-clone rejects a branch name instead of a SHA (no silent run of the tip)" \
|
||||
CONFORMANCE_GIT_SHA=main conformance-clone
|
||||
expect_fail "conformance-clone rejects an UNRENDERED placeholder" \
|
||||
CONFORMANCE_GIT_SHA=@@CONFORMANCE_GIT_SHA@@ conformance-clone
|
||||
expect_fail "conformance-clone rejects a HOME off the workspace volume" \
|
||||
CONFORMANCE_GIT_SHA=0000000000000000000000000000000000000000 \
|
||||
HOME=/etc conformance-clone
|
||||
expect_fail "conformance-clone refuses a REDIRECTED repository URL (the env-injection hole)" \
|
||||
CONFORMANCE_GIT_SHA=0000000000000000000000000000000000000000 \
|
||||
CONFORMANCE_REPO_URL=https://evil.invalid/x.git conformance-clone
|
||||
expect_fail "conformance-clone refuses a clone target off the workspace volume" \
|
||||
CONFORMANCE_GIT_SHA=0000000000000000000000000000000000000000 \
|
||||
CONFORMANCE_SRC=/etc/src conformance-clone
|
||||
echo "--- rootfs is read-only (expected: a failure) ---"
|
||||
if touch /should-fail 2>/dev/null; then echo "UNEXPECTED: rootfs is writable"; exit 1; fi
|
||||
echo "confirmed: / is read-only"
|
||||
echo "--- HOME is writable (the workspace emptyDir) ---"
|
||||
touch "${HOME}/.probe" && echo "confirmed: ${HOME} is writable"
|
||||
echo "--- uid is non-zero ---"
|
||||
[ "$(id -u)" -ne 0 ] && echo "confirmed: running as uid $(id -u), not root"
|
||||
'
|
||||
log "Toolchain verified"
|
||||
fi
|
||||
|
||||
# --- verify the clone helper against the REAL repository (opt-in: needs network) ----
|
||||
# The negative cases above prove conformance-clone refuses bad input. This proves the
|
||||
# positive case: it lands the EXACT commit asked for and can prove it did. Without this, the
|
||||
# rev-parse assertion is only an unexercised line of shell.
|
||||
if [ "${VERIFY_CLONE}" -eq 1 ]; then
|
||||
log "Verifying conformance-clone against the real repository"
|
||||
SHA="$(git -C "${REPO_ROOT}" rev-parse HEAD)"
|
||||
info "requesting commit ${SHA}"
|
||||
"${ENGINE}" run --rm \
|
||||
--user 65532:65532 \
|
||||
--read-only \
|
||||
--tmpfs /workspace:rw,mode=1777 \
|
||||
--network host \
|
||||
--entrypoint "" \
|
||||
-e "CONFORMANCE_GIT_SHA=${SHA}" \
|
||||
"${IMAGE_REF}" \
|
||||
/bin/bash -euo pipefail -c '
|
||||
conformance-clone
|
||||
echo "--- assert rev-parse HEAD == the requested SHA ---"
|
||||
got="$(git -C /workspace/src rev-parse HEAD)"
|
||||
echo "requested: ${CONFORMANCE_GIT_SHA}"
|
||||
echo "HEAD: ${got}"
|
||||
[ "${got}" = "${CONFORMANCE_GIT_SHA}" ] || { echo "MISMATCH"; exit 1; }
|
||||
echo "confirmed: the working tree is at the requested commit"
|
||||
test -x /workspace/src/conformance/run.sh && echo "confirmed: run.sh present and executable"
|
||||
'
|
||||
log "conformance-clone verified against the real repository"
|
||||
fi
|
||||
|
||||
# --- digest ------------------------------------------------------------------
|
||||
DIGEST="$("${ENGINE}" image inspect --format '{{.Digest}}' "${IMAGE_REF}")"
|
||||
IMAGE_ID="$("${ENGINE}" image inspect --format '{{.Id}}' "${IMAGE_REF}")"
|
||||
SIZE="$("${ENGINE}" image inspect --format '{{.Size}}' "${IMAGE_REF}")"
|
||||
|
||||
log "Built ${IMAGE_REF}"
|
||||
info "local manifest digest ${DIGEST}"
|
||||
info "image id ${IMAGE_ID}"
|
||||
info "size $(( SIZE / 1024 / 1024 )) MiB"
|
||||
|
||||
cat <<EOF
|
||||
|
||||
================================================================================
|
||||
NOT PUSHED. Publishing is an operator action — this script holds no registry
|
||||
credential and will never acquire one.
|
||||
|
||||
OPERATOR: run these two commands, then paste the digest from the digestfile into
|
||||
conformance/versions.env (CONFORMANCE_RUNNER_IMAGE) via a PR:
|
||||
|
||||
${ENGINE} login gitea.home.keithjackson.dev
|
||||
|
||||
${ENGINE} push --digestfile /tmp/conformance-runner.digest \\
|
||||
${IMAGE_REF}
|
||||
|
||||
Then:
|
||||
|
||||
cat /tmp/conformance-runner.digest
|
||||
# -> sha256:.... THIS is the authoritative digest, not the local one above.
|
||||
|
||||
# conformance/versions.env
|
||||
CONFORMANCE_RUNNER_IMAGE="${IMAGE_REPO}@\$(cat /tmp/conformance-runner.digest)"
|
||||
|
||||
WHY THE DIGEST AND NOT THE TAG: platform/kyverno/admission/conformance-trigger-job-shape.yaml
|
||||
is a fail-closed Enforce policy. It DENIES any conformance-trigger Job whose image is not
|
||||
${IMAGE_REPO}@sha256:<digest>. A tag reference is denied,
|
||||
and so is any other repository path.
|
||||
|
||||
TIGHTEN THE BOUNDARY AFTERWARDS: that policy's header asks for the wildcard
|
||||
\`${IMAGE_REPO}@sha256:*\` to be replaced with the
|
||||
LITERAL digest once the image exists, closing the "anyone who can push to this path" gap.
|
||||
That is a separate, human-reviewed PR against the security boundary.
|
||||
================================================================================
|
||||
EOF
|
||||
Executable
+138
@@ -0,0 +1,138 @@
|
||||
#!/usr/bin/env bash
|
||||
###############################################################################
|
||||
# conformance-clone — fetch the impl repo at the exact commit under test.
|
||||
#
|
||||
# Baked into the conformance-runner image at /usr/local/bin/conformance-clone, mode 0755
|
||||
# root:root, so the running container (uid 65532) cannot replace it. The Job's init
|
||||
# container runs it as `command: ["/usr/local/bin/conformance-clone"]` with NO args.
|
||||
#
|
||||
# WHY IT IS A FILE IN THE IMAGE AND NOT AN INLINE SCRIPT IN THE JOB
|
||||
# -----------------------------------------------------------------
|
||||
# platform/kyverno/admission/conformance-trigger-job-shape.yaml pins the MAIN container's
|
||||
# `command` to ["./run.sh"] and constrains its `args` — but it never reads `initContainers`.
|
||||
# So a Job from `conformance-trigger` could carry an init container with
|
||||
# `command: ["/bin/sh","-c"]` and an arbitrary script, running as the privileged
|
||||
# `conformance-runner` SA. Reproduced by the policy's author: such a Job is ADMITTED
|
||||
# (pass 3, fail 0).
|
||||
#
|
||||
# The fix has two halves and NEITHER works alone:
|
||||
# 1. this file — a fixed path with fixed behaviour and no argument surface, so there is
|
||||
# nothing for a caller to inject into; and
|
||||
# 2. a 4th policy rule requiring every initContainer's `command` to be exactly
|
||||
# ["/usr/local/bin/conformance-clone"] with no `args`. That rule lives in the policy
|
||||
# and is NOT this branch's to write — a security boundary changes in its own reviewed PR.
|
||||
# Without (2) this file is merely available, not required.
|
||||
#
|
||||
# THE ENV SURFACE IS THE OTHER HALF OF THE SAME HOLE — READ THIS BEFORE "SIMPLIFYING"
|
||||
# -----------------------------------------------------------------------------------
|
||||
# The admission policy says NOTHING about `env`. Verified: no `env`, `valueFrom` or
|
||||
# `secretKeyRef` clause appears in it. So a script that took its repository URL from the
|
||||
# environment would just move the injection one field over — the trigger sets
|
||||
# CONFORMANCE_REPO_URL to a repo it controls, this clones it, and `./run.sh` from THAT repo
|
||||
# executes under the privileged runner SA. Identical shape to the initContainers gap, and
|
||||
# "no args" would not have stopped it.
|
||||
#
|
||||
# Therefore the repository URL and the clone target are COMPILED IN below. The matching env
|
||||
# vars are still read, so the Job template can keep declaring them and stay self-documenting,
|
||||
# but a value that DISAGREES with the baked literal is a hard failure, never a redirect.
|
||||
# The only per-run input that is genuinely trusted is the commit SHA, and it must be 40 hex
|
||||
# characters of a commit that the baked repository actually contains.
|
||||
#
|
||||
# Inputs (environment; no arguments are accepted):
|
||||
# CONFORMANCE_GIT_SHA REQUIRED. 40 lowercase hex. The commit under test.
|
||||
# CONFORMANCE_REPO_URL optional. Must equal the baked URL if set.
|
||||
# CONFORMANCE_SRC optional. Must equal the baked target if set.
|
||||
# GITEA_USERNAME optional, from Secret `conformance-git-readonly`.
|
||||
# GITEA_TOKEN optional, from Secret `conformance-git-readonly`.
|
||||
# TMPDIR / HOME optional; must be under /workspace. Default to /workspace/{tmp,home}.
|
||||
#
|
||||
# Runs under: uid 65532, readOnlyRootFilesystem, only the `workspace` and `results` volumes.
|
||||
# Everything it writes therefore lives under /workspace. There is no writable /tmp.
|
||||
###############################################################################
|
||||
set -euo pipefail
|
||||
|
||||
# --- COMPILED-IN, NOT CONFIGURABLE (see the header) --------------------------
|
||||
readonly BAKED_REPO_URL="https://gitea.home.keithjackson.dev/Home-Network/home-network-k3s-impl.git"
|
||||
readonly BAKED_SRC="/workspace/src"
|
||||
|
||||
die() { printf 'FATAL: %s\n' "$*" >&2; exit 1; }
|
||||
log() { printf '==> %s\n' "$*"; }
|
||||
|
||||
[ "$#" -eq 0 ] || die "takes no arguments (got $#: $*). Arguments are where an injection would live."
|
||||
|
||||
# --- the one genuinely trusted input ----------------------------------------
|
||||
# A bad SHA must never degrade into "ran whatever the branch tip was": an unrendered
|
||||
# @@CONFORMANCE_GIT_SHA@@ placeholder has to be a hard failure, not a silent run of main.
|
||||
: "${CONFORMANCE_GIT_SHA:?CONFORMANCE_GIT_SHA is not set; the CI trigger did not render it}"
|
||||
printf '%s' "${CONFORMANCE_GIT_SHA}" | grep -Eq '^[0-9a-f]{40}$' \
|
||||
|| die "CONFORMANCE_GIT_SHA is not a 40-hex commit id: '${CONFORMANCE_GIT_SHA}'
|
||||
The CI trigger did not substitute it, or substituted junk."
|
||||
|
||||
# --- env that may agree with the baked values, but may not override them -----
|
||||
REPO_URL="${CONFORMANCE_REPO_URL:-${BAKED_REPO_URL}}"
|
||||
[ "${REPO_URL}" = "${BAKED_REPO_URL}" ] || die "CONFORMANCE_REPO_URL does not match the image's baked repository.
|
||||
got: ${REPO_URL}
|
||||
baked: ${BAKED_REPO_URL}
|
||||
Refusing. A caller that can choose the repository can choose the code that runs."
|
||||
|
||||
SRC="${CONFORMANCE_SRC:-${BAKED_SRC}}"
|
||||
[ "${SRC}" = "${BAKED_SRC}" ] || die "CONFORMANCE_SRC does not match the image's baked clone target.
|
||||
got: ${SRC}
|
||||
baked: ${BAKED_SRC}
|
||||
Refusing."
|
||||
|
||||
# Scratch paths must stay on the workspace volume; the root filesystem is read-only.
|
||||
# `/workspace` ITSELF is allowed, not just subdirectories: the image's own ENV sets
|
||||
# HOME=/workspace, and an earlier version of this guard rejected exactly that — it matched
|
||||
# only `/workspace/*` and so refused its own default. Caught by running it, not by reading it.
|
||||
: "${TMPDIR:=/workspace/tmp}"
|
||||
: "${HOME:=/workspace/home}"
|
||||
on_workspace() { case "$1" in /workspace|/workspace/*) return 0 ;; *) return 1 ;; esac; }
|
||||
on_workspace "${TMPDIR}" || die "TMPDIR must be /workspace or under it (got '${TMPDIR}')"
|
||||
on_workspace "${HOME}" || die "HOME must be /workspace or under it (got '${HOME}')"
|
||||
export TMPDIR HOME
|
||||
mkdir -p "${TMPDIR}" "${HOME}"
|
||||
|
||||
# --- credential (optional) ---------------------------------------------------
|
||||
# git's shell credential helper is invoked as `sh -c '<value> get'`, so it reads $1 and
|
||||
# answers from the environment. The token therefore never appears in argv and never on disk.
|
||||
#
|
||||
# ANONYMOUS FALLBACK, DELIBERATE AND ANNOUNCED. Read access to this repository currently
|
||||
# needs no credential (measured: `ls-remote` succeeds unauthenticated from inside this
|
||||
# image), so requiring the Secret would fail runs that would otherwise work — and would make
|
||||
# this script untestable without one. It is not a privilege downgrade: the integrity
|
||||
# guarantee here is the SHA assertion below, not the credential. Which mode was used is
|
||||
# logged, so a run that silently lost its Secret is visible in the log rather than inferred.
|
||||
CRED=''
|
||||
if [ -n "${GITEA_USERNAME:-}" ] && [ -n "${GITEA_TOKEN:-}" ]; then
|
||||
CRED='!f() { if [ "$1" = get ]; then printf "username=%s\npassword=%s\n" "$GITEA_USERNAME" "$GITEA_TOKEN"; fi; }; f'
|
||||
log "authenticating as '${GITEA_USERNAME}' (token from the environment, never argv)"
|
||||
else
|
||||
log "no GITEA_USERNAME/GITEA_TOKEN in the environment — cloning ANONYMOUSLY"
|
||||
fi
|
||||
|
||||
git_c() {
|
||||
if [ -n "${CRED}" ]; then git -c credential.helper="${CRED}" "$@"; else git "$@"; fi
|
||||
}
|
||||
|
||||
# --- clone, land the exact commit, and PROVE it ------------------------------
|
||||
log "shallow clone ${REPO_URL} -> ${SRC}"
|
||||
git_c clone --depth 1 --no-tags --single-branch "${REPO_URL}" "${SRC}"
|
||||
|
||||
if ! git -C "${SRC}" cat-file -e "${CONFORMANCE_GIT_SHA}^{commit}" 2>/dev/null; then
|
||||
log "${CONFORMANCE_GIT_SHA} is not in the shallow tip; fetching that commit"
|
||||
git_c -C "${SRC}" fetch --depth 1 --no-tags origin "${CONFORMANCE_GIT_SHA}"
|
||||
fi
|
||||
|
||||
git -C "${SRC}" checkout --detach "${CONFORMANCE_GIT_SHA}"
|
||||
|
||||
# The assertion is the point of the whole script: everything above is best-effort plumbing,
|
||||
# this is what makes "the suite ran at the commit under test" a fact rather than a hope.
|
||||
head="$(git -C "${SRC}" rev-parse HEAD)"
|
||||
[ "${head}" = "${CONFORMANCE_GIT_SHA}" ] \
|
||||
|| die "checked out ${head}, expected ${CONFORMANCE_GIT_SHA}."
|
||||
|
||||
test -x "${SRC}/conformance/run.sh" \
|
||||
|| die "no executable conformance/run.sh at ${head}."
|
||||
|
||||
log "suite ready at ${head}"
|
||||
@@ -6,6 +6,10 @@
|
||||
|
||||
# Chainsaw — L1 rendering / L2 behavioral / L3 fault-injection runner (ADR-0011).
|
||||
CHAINSAW_VERSION="v0.2.12"
|
||||
# The UPSTREAM chainsaw image. It is NOT what the in-cluster Job runs — see
|
||||
# CONFORMANCE_RUNNER_IMAGE below. It carries the chainsaw binary and essentially nothing
|
||||
# else (no bash, no python3, no kubectl, no git), so `command: ["./run.sh"]` on this image
|
||||
# fails at exec. Kept only as the provenance of the chainsaw binary we rebuild from.
|
||||
CHAINSAW_IMAGE="ghcr.io/kyverno/chainsaw:v0.2.12"
|
||||
|
||||
# kubectl — used by run.sh for cluster reachability preflight and by probe scripts.
|
||||
@@ -18,3 +22,52 @@ KYVERNO_VERSION="v1.19.0"
|
||||
# Python 3 — coverage generator + JUnit emitter use stdlib only (no pip deps), so the
|
||||
# only constraint is a modern interpreter.
|
||||
PYTHON_MIN="3.9"
|
||||
|
||||
# --- In-cluster conformance runner image -------------------------------------
|
||||
# Built from conformance/incluster/Containerfile by conformance/incluster/build-image.sh.
|
||||
# This — not CHAINSAW_IMAGE — is what the L2/L3 Job runs (bash + python3 + chainsaw +
|
||||
# kubectl + git). Closes docs/plans/l2-l3-conformance-ci-wiring.md §3 item 1.
|
||||
#
|
||||
# It ALSO carries /usr/local/bin/conformance-clone (0755 root:root, no argument surface) as the
|
||||
# init container's only entrypoint. That is a security fixture, not a convenience: the
|
||||
# admission policy never reads `initContainers`, so an init container with an inline shell
|
||||
# script is currently admitted with the privileged runner SA. The policy rule that makes the
|
||||
# fixed path REQUIRED is not in this branch — see conformance/incluster/README.md.
|
||||
#
|
||||
# THE REPOSITORY PATH IS NOT A PREFERENCE. platform/kyverno/admission/
|
||||
# conformance-trigger-job-shape.yaml is a fail-closed Enforce ClusterPolicy (R-SEC-02) that
|
||||
# ADMITS a conformance-trigger Job only when its image matches
|
||||
# gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:*
|
||||
# Any other path, and any TAG reference, is DENIED at admission. Do not "tidy" these values.
|
||||
CONFORMANCE_RUNNER_IMAGE_REPO="gitea.home.keithjackson.dev/home-network/conformance-runner"
|
||||
# Build handle only. Admission never sees the tag; a Job must reference the digest.
|
||||
CONFORMANCE_RUNNER_IMAGE_TAG="chainsaw0.2.12-kubectl1.36.4-fc44"
|
||||
#
|
||||
# ############################################################################
|
||||
# # DIGEST DELIBERATELY EMPTY — THE IMAGE HAS NOT BEEN PUBLISHED YET. #
|
||||
# ############################################################################
|
||||
# The image has been BUILT and its toolchain verified locally (non-root uid 65532,
|
||||
# read-only rootfs: bash 5.3.9, Python 3.14.7, git 2.55.0, kubectl v1.36.4, chainsaw 0.2.12,
|
||||
# and `./run.sh --layer l2 --no-cluster` completing with rc 0). It has NOT been pushed:
|
||||
# pushing needs a registry credential, and publishing the artifact a fail-closed security
|
||||
# boundary then trusts is an OPERATOR action, not an agent's.
|
||||
#
|
||||
# OPERATOR, to complete the pin:
|
||||
# 1. ./conformance/incluster/build-image.sh # rebuild, verify, print the commands
|
||||
# 2. podman login gitea.home.keithjackson.dev
|
||||
# 3. podman push --digestfile /tmp/conformance-runner.digest \
|
||||
# "${CONFORMANCE_RUNNER_IMAGE_REPO}:${CONFORMANCE_RUNNER_IMAGE_TAG}"
|
||||
# 4. Set CONFORMANCE_RUNNER_IMAGE_DIGEST below to `cat /tmp/conformance-runner.digest`
|
||||
# — the digest the REGISTRY returned, not the local manifest digest the build printed.
|
||||
# 5. Land steps 4 in a PR (this file is GitOps input, not a live edit).
|
||||
#
|
||||
# UNTIL THEN: this is empty on purpose, NOT a placeholder to be guessed at. An empty value
|
||||
# makes any consumer that renders the Job fail visibly; a fake or stale digest would instead
|
||||
# produce an ImagePullBackOff that looks like a cluster fault. Do not invent one.
|
||||
CONFORMANCE_RUNNER_IMAGE_DIGEST=""
|
||||
CONFORMANCE_RUNNER_IMAGE="${CONFORMANCE_RUNNER_IMAGE_DIGEST:+${CONFORMANCE_RUNNER_IMAGE_REPO}@${CONFORMANCE_RUNNER_IMAGE_DIGEST}}"
|
||||
#
|
||||
# AFTERWARDS, TIGHTEN THE BOUNDARY: the header of conformance-trigger-job-shape.yaml asks for
|
||||
# its `@sha256:*` wildcard to be replaced with the LITERAL digest once the image exists —
|
||||
# that closes the residual "anyone who can push to this path" gap. Separate PR, human
|
||||
# reviewed, because it edits the security boundary itself.
|
||||
@@ -0,0 +1,258 @@
|
||||
---
|
||||
date: 2026-09-29
|
||||
branch: feat/conformance-runner-image
|
||||
title: The in-cluster conformance Job could never have run; built the image it needs (PR #18)
|
||||
---
|
||||
|
||||
## What changed
|
||||
|
||||
Closes `docs/plans/l2-l3-conformance-ci-wiring.md` §3 item 1. PR
|
||||
[#18](https://gitea.home.keithjackson.dev/Home-Network/home-network-k3s-impl/pulls/18). **Not
|
||||
merged** — an agent cannot, and should not.
|
||||
|
||||
### The defect, stated plainly
|
||||
|
||||
`conformance/incluster/chainsaw-job.yaml` pinned `ghcr.io/kyverno/chainsaw:v0.2.12` **and**
|
||||
`command: ["./run.sh"]`. Both lines could not be true. That image carries the chainsaw binary
|
||||
and essentially nothing else — no `bash`, no `python3`, no `kubectl`, no `git`. The Job would
|
||||
have failed at exec of `./run.sh` before a single test ran, and the only place that would have
|
||||
surfaced is a pod log nobody was reading. This is the same failure shape as the entries around
|
||||
it in the archive: a manifest that *described* a working mechanism, with nothing measuring
|
||||
whether the mechanism existed.
|
||||
|
||||
### What landed
|
||||
|
||||
- `conformance/incluster/Containerfile` — the `conformance-runner` image: bash + python3 +
|
||||
chainsaw + kubectl + `git-core` + tar/gzip/coreutils/diffutils. Two-stage (a full Fedora
|
||||
fetches the binaries and verifies kubectl's published `sha256`; the runtime layer is
|
||||
`fedora-minimal` and carries no downloader).
|
||||
- `conformance/incluster/build-image.sh` — builds, **proves the toolchain by executing it**,
|
||||
prints the digest, and **never pushes**.
|
||||
- `conformance/incluster/conformance-clone` — the baked init-container clone helper (see the
|
||||
addendum below).
|
||||
- `conformance/versions.env` — `CONFORMANCE_RUNNER_IMAGE`, digest **empty** pending the push.
|
||||
- `conformance/incluster/README.md` — documents all of it, and flags the Job manifest's stale
|
||||
`image:`.
|
||||
|
||||
`infra/devcontainer/versions.env` stays the single source of truth. `build-image.sh` sources it
|
||||
and injects `CHAINSAW_VERSION` / `KUBECTL_VERSION` / `FEDORA_VERSION` as `--build-arg`; the
|
||||
Containerfile's `ARG` defaults are **empty** and asserted in its first `RUN`, so a build that
|
||||
forgets a pin fails loudly rather than baking a stale version. No new pin was invented.
|
||||
|
||||
Two things the admission boundary forced, both of which would otherwise have been discovered
|
||||
at first run: `USER` must be **numeric** (`runAsNonRoot` with no `runAsUser` makes the kubelet
|
||||
reject a named user), and every writable path must live under `/workspace` (the pod is
|
||||
`readOnlyRootFilesystem` and the policy admits no third volume, so there is no `/tmp`). git's
|
||||
`safe.directory` is therefore written to `/etc/gitconfig` at build time.
|
||||
`platform/kyverno/admission/conformance-trigger-job-shape.yaml` was **not modified**.
|
||||
|
||||
### The chainsaw supply-chain gap, stated accurately
|
||||
|
||||
The Containerfile originally said chainsaw "publishes no per-asset checksum file at a stable
|
||||
URL". Literally true, and misleading: it reads as "no integrity option exists". The release
|
||||
assets say otherwise — v0.2.12 ships per-asset cosign `.sig` + `.pem` and an SBOM alongside
|
||||
each tarball. So verification IS available; it needs cosign in the builder stage and the right
|
||||
keyless identity for kyverno's release workflow. The comment now says what is true: kubectl is
|
||||
sha256-verified, chainsaw rests on tag + TLS, and the cosign path is named as a TODO.
|
||||
Deliberately **not** implemented here — picking a trusted signer identity is a supply-chain
|
||||
decision that belongs in its own reviewed change. `infra/devcontainer/Containerfile` has the
|
||||
same gap for the same binary.
|
||||
|
||||
### Addendum — `/usr/local/bin/conformance-clone`
|
||||
|
||||
Added at the coordinator's request, to close half of the admission gap a sibling agent found:
|
||||
the policy pins the MAIN container's `command` but **never reads `initContainers`**, so a Job
|
||||
from `conformance-trigger` can carry an init container with `command: ["/bin/sh","-c"]` and an
|
||||
arbitrary script, running as the privileged `conformance-runner` SA. Reproduced by the policy's
|
||||
author (`pass: 3, fail: 0` on a privileged `alpine:latest` init container running
|
||||
`kubectl get secrets -A`).
|
||||
|
||||
The image now bakes `/usr/local/bin/conformance-clone`, **0755 root:root**, so uid 65532 cannot
|
||||
replace the one executable the policy will come to trust. Clone logic taken from PR #20's
|
||||
`clone-suite-at-commit`, with both load-bearing properties preserved: the SHA must match
|
||||
`^[0-9a-f]{40}$`, and `rev-parse HEAD` is asserted equal to it after checkout. No arguments are
|
||||
accepted.
|
||||
|
||||
**A DEVIATION FROM THE BRIEF, DELIBERATE — the repo URL and clone target are COMPILED IN, not
|
||||
taken from the environment.** The brief said to take SHA, repo URL and target dir from the
|
||||
environment. The policy contains no `env`, `valueFrom` or `secretKeyRef` clause at all, so a
|
||||
script that trusted `CONFORMANCE_REPO_URL` would have moved the injection one field over — the
|
||||
trigger points it at a repo it controls, `conformance-clone` fetches that, and `./run.sh` from
|
||||
the attacker's tree runs under the privileged SA. "No args" would not have stopped it. The env
|
||||
vars are still read so the Job template stays self-documenting, but a value that disagrees with
|
||||
the baked literal is a hard failure, never a redirect. The only genuinely trusted per-run input
|
||||
is the SHA.
|
||||
|
||||
**This file is half a fix and must not be mistaken for the whole one.** What makes the fixed
|
||||
path *required* rather than merely *available* is a 4th policy rule demanding
|
||||
`command: ["/usr/local/bin/conformance-clone"]` with no `args` on every initContainer. That
|
||||
rule edits the security boundary and belongs in its own reviewed PR — it is **not** in this
|
||||
branch. Until it lands, the hole is open.
|
||||
|
||||
The anonymous-clone fallback is deliberate and announced in the script's log line: read access
|
||||
to this repo needs no credential today, the integrity guarantee here is the SHA assertion
|
||||
rather than the credential, and requiring the Secret would have made the script untestable
|
||||
without one.
|
||||
|
||||
### A REAL CONSTRAINT THE PLAN DOES NOT ACCOUNT FOR — read before wiring T-6.1b
|
||||
|
||||
`run.sh` runs the selected **offline L0/L1 checks** before it touches the cluster, for **any**
|
||||
selection that is not offline-only. Those checks shell out to `flux`, `promtool`, `kyverno` and
|
||||
`kustomize`. This image deliberately omits them — baking the whole devcontainer toolchain into
|
||||
the runner would roughly double its size to re-run work `ci/l0.sh` and `ci/l1.sh` already did
|
||||
offline in the same CI run.
|
||||
|
||||
So **the plan's own invocation shapes do not work on this image.** `--layer all
|
||||
--strict-coverage` (the documented phase-exit shape) and a bare `--requirements <IDs>` (the
|
||||
documented merge-gate shape) both drag the L0 checks in and will fail on a missing tool. The Job
|
||||
must use `--layer l2` / `--layer l3`, which is verified below to resolve the offline selection
|
||||
to nothing.
|
||||
|
||||
**This is a decision for the plan owner, not a detail:** either the Job selects cluster-gated
|
||||
layers only, or the runner grows the full L0 toolchain, or `run.sh` learns to skip offline
|
||||
checks when their tools are absent (which would be a fail-open and is the wrong answer). It is
|
||||
documented in the Containerfile header and the incluster README so it cannot be rediscovered
|
||||
from a pod log.
|
||||
|
||||
### Follow-ups this creates
|
||||
|
||||
1. **Operator:** `podman login` + `podman push --digestfile`, then a PR setting
|
||||
`CONFORMANCE_RUNNER_IMAGE_DIGEST` from the digestfile — the digest the *registry* returned,
|
||||
not the local manifest digest the build printed.
|
||||
2. **Then tighten the boundary**, as that policy's own header asks: replace its `@sha256:*`
|
||||
wildcard with the literal digest, closing the residual "anyone who can push to this path"
|
||||
gap. Separate, human-reviewed PR — it edits the security boundary.
|
||||
3. **T-6.1b:** decide the Job's `args` per the constraint above, and stop `chainsaw-job.yaml`
|
||||
carrying an image reference that admission would deny.
|
||||
4. **The 4th policy rule** pinning every initContainer's `command` to `conformance-clone`.
|
||||
Without it, `conformance-clone` is available but not required, and the injection hole is
|
||||
still open.
|
||||
|
||||
### Rework note — why this entry is a file and not an append
|
||||
|
||||
This branch originally carried its handoff as a ~236-line append to the tail of
|
||||
`DECISIONS-LOG.md`. That append was the **only** conflicting path between this PR and `main`,
|
||||
and it made PR #18 `mergeable: false`. The append has been dropped from the branch entirely and
|
||||
its substance moved here verbatim in meaning; the branch now touches no file outside
|
||||
`conformance/`. Nothing about what the code claims to have proven was changed by that move.
|
||||
|
||||
## What I verified
|
||||
|
||||
### The image, by running it — not by a green build
|
||||
|
||||
As uid 65532 with `--read-only`: bash 5.3.9, Python 3.14.7, git 2.55.0, kubectl v1.36.4
|
||||
(client), chainsaw 0.2.12; `/` confirmed read-only, `/workspace` writable, uid confirmed
|
||||
non-zero. Then the real entrypoint in the same shape against the worktree: `./run.sh --lint`,
|
||||
`--coverage-only`, and `--layer l2 --no-cluster` all **rc 0**, emitting `coverage.md`,
|
||||
`coverage.json` and `report.junit.xml`. 408 MiB. Rebuilt after the chainsaw-comment edit; the
|
||||
in-image verification still passed.
|
||||
|
||||
### The init-container clone path, end to end
|
||||
|
||||
From inside the image, as uid 65532, `python3` completes a verifying TLS handshake to
|
||||
`gitea.home.keithjackson.dev` against the stock `ca-certificates` (issuer `Let's Encrypt / YR2`,
|
||||
notAfter `Dec 12 02:16:57 2026 GMT`), and `git ls-remote https://…/home-network-k3s-impl.git
|
||||
HEAD` returns `20ea70d` with **no credential**. So no internal CA bundle is needed, and
|
||||
anonymous read is currently open — the plan's read-only deploy token buys authorisation
|
||||
hygiene, not reachability.
|
||||
|
||||
`/usr/local/bin/conformance-clone` measured in the image as uid 65532 with `--read-only`:
|
||||
`root:root 755`, confirmed **not writable** by uid 65532. Seven negative cases, each asserted to
|
||||
exit non-zero (not merely to print something): arguments rejected; missing SHA; a branch name
|
||||
instead of a SHA; an unrendered `@@CONFORMANCE_GIT_SHA@@`; `HOME` off the workspace volume; a
|
||||
**redirected** `CONFORMANCE_REPO_URL=https://evil.invalid/x.git`; a `CONFORMANCE_SRC` off the
|
||||
workspace volume. And the positive case: a real anonymous clone of this repo at a real 40-hex
|
||||
SHA, with `rev-parse HEAD` proven equal to the requested SHA.
|
||||
|
||||
Two bugs of my own that only running it could have found, both fixed: the workspace-path guard
|
||||
matched `/workspace/*` and so **rejected the image's own `HOME=/workspace`**; and the negative
|
||||
checks were written as pipelines, which under `pipefail` aborted the verify at the first
|
||||
expected failure and silently skipped the remaining six. A verify that stops early looks
|
||||
identical to a verify that passed.
|
||||
|
||||
### There is no `approval-policy` CI stage, and `conformance/**` is not a protected path
|
||||
|
||||
My brief told me to expect this PR's CI to fail on an `approval-policy` stage because
|
||||
`conformance/**` is a security-relevant path requiring human approval. **I repeated that before
|
||||
checking it. It is not true:**
|
||||
|
||||
| Claim | Measured |
|
||||
|---|---|
|
||||
| an `approval-policy` CI stage exists | **No.** `grep -rn 'approval-policy\|approval_policy' .gitea/ ci/` → nothing. `.gitea/workflows/gate.yml` has jobs `gate` and `conformance`; `merge-gate.yaml` has `spec-immutability, lint, l0, render-golden, l1, conformance, merge-gate`. No approval job in either. |
|
||||
| `conformance/**` is a protected path | **No.** `GET /branch_protections` → `protected_file_patterns: ''`. No path-based rule of any kind, and no `CODEOWNERS` file in the repo. |
|
||||
| required status checks | `status_check_contexts: ['gate / gate (pull_request)']` — one context, and it is not an approval stage. |
|
||||
|
||||
What actually blocks this PR is `required_approvals: 1` with `block_admin_merge_override: true`,
|
||||
and `gitea-admin` authors every agent PR and cannot self-approve. That is a **merge** block, not
|
||||
a CI failure — the distinction matters, because "expect a red X" and "expect a green run that an
|
||||
agent still cannot merge" send a reviewer to different places. Left in as a correction rather
|
||||
than quietly deleted.
|
||||
|
||||
### What PR #18's CI actually did, read out of the job log
|
||||
|
||||
Read from `/Home-Network/home-network-k3s-impl/actions/runs/128/jobs/0/logs`, not off the tick,
|
||||
because a skipped Gitea job reports `success`:
|
||||
|
||||
[skip] stage spec-immutability: SKIPPED (not run)
|
||||
[ ok ] stage lint: pass
|
||||
[ ok ] stage l0: pass
|
||||
[ ok ] stage render-golden: pass
|
||||
[ ok ] stage l1: pass
|
||||
[ ok ] stage gitea-settings: pass
|
||||
[skip] stage conformance: SKIPPED (not run)
|
||||
[fail] MERGE GATE: UNDETERMINED — greenness was NOT ESTABLISHED.
|
||||
|
||||
`gate / gate (pull_request)` = **success in 1m36s**, and that success means **rc 3, tolerated**
|
||||
by `./ci/gate.sh || [ "$?" -eq 3 ]` — not rc 0. `gitea-settings` ran and passed in CI with
|
||||
`REQUIRE_GITEA_SETTINGS=true`, so CI is *more* complete there than the local toolbox run, where
|
||||
that stage can only skip. That measurement predates this rework's force-push; CI has not been
|
||||
re-read since.
|
||||
|
||||
### The gate, on the reworked branch, today
|
||||
|
||||
ENGINE=<worktree shim> DEV_KUBECONFIG=/dev/null ./infra/devcontainer/dev.sh run bash -c './ci/gate.sh'
|
||||
→ rc 3, MERGE GATE: UNDETERMINED. Nothing FAILED.
|
||||
|
||||
[ ok ] spec-immutability [ ok ] decisions-log [ ok ] lint
|
||||
[ ok ] l0 [ ok ] render-golden [ ok ] l1
|
||||
[skip] gitea-settings (SETTINGS_API_TOKEN not set — unwired repo-wide)
|
||||
[skip] conformance (CLUSTER_AVAILABLE != true) — L2/L3 UNVERIFIED
|
||||
Stages that did not run: gitea-settings
|
||||
|
||||
`decisions-log` is the stage PR #24 added; it ran against this branch and passed — "archive
|
||||
frozen, 2 entries well-formed" — so this file's shape is machine-checked, not eyeballed.
|
||||
Diff base `origin/main @ c59fe9f923c4` (this branch was rebased onto main after #24 merged).
|
||||
Touched requirements resolved to
|
||||
`R-SEC-02 R-TEST-01 R-TEST-02 R-TEST-03 R-TEST-04`; all five remain **L2/L3-UNVERIFIED**. This
|
||||
is **not** a green run and is not offered as one.
|
||||
|
||||
### The rework itself
|
||||
|
||||
`git diff origin/main...HEAD --name-only` lists only the five `conformance/` paths plus this
|
||||
file — `DECISIONS-LOG.md` is gone from the branch. `git diff pr18-orig HEAD -- conformance/`
|
||||
is empty, so the code is byte-identical to the pre-rework tip, file modes included
|
||||
(`build-image.sh` and `conformance-clone` both `100755`).
|
||||
|
||||
## What I did NOT verify
|
||||
|
||||
- **The image has never run in the cluster.** Everything above is local podman. Admission has
|
||||
not been exercised against it, because there is no digest to reference yet.
|
||||
- **The image is not pushed anywhere**, so `CONFORMANCE_RUNNER_IMAGE_DIGEST` is empty. Empty on
|
||||
purpose: it fails visibly in whatever renders the Job, whereas a guessed digest would look
|
||||
like an `ImagePullBackOff` cluster fault. Pushing needs a registry credential and is an
|
||||
operator action — `build-image.sh` prints the exact commands.
|
||||
- **The image was not rebuilt during this rework.** Moving where the handoff lives must not
|
||||
change what the branch claims to have proven, so no build was run; the image measurements
|
||||
above are the ones taken when the code was written, against this same code.
|
||||
- **chainsaw's cosign signatures are not verified** by this image. Only kubectl's `sha256` is.
|
||||
The cosign path is named as a TODO, not implemented.
|
||||
- **The 4th policy rule does not exist**, so `conformance-clone` is available but not required.
|
||||
The initContainer injection hole is open until that separate PR lands.
|
||||
- **L2/L3 still do not run in CI** (`vars.CLUSTER_AVAILABLE` unset). This image is a
|
||||
prerequisite for closing that hole, not the closure. R-SEC-02 and R-TEST-01..04 are untested
|
||||
at L2/L3.
|
||||
- **`gitea-settings` skipped in my local gate run**, so branch protection and runner
|
||||
registration are UNVERIFIED by this run. The CI-side claims quoted above were measured
|
||||
earlier, against the pre-rework head.
|
||||
- **The post-force-push CI run has not been read.** The `gate / gate (pull_request)` result
|
||||
quoted above is from run 128 on the old head.
|
||||
Reference in new issue
Block a user