jakman85andClaude Opus 5
ab9ba32a76
Closes half of the admission gap a sibling agent found in PR #13's policy: it pins the MAIN container's `command` but never reads `initContainers`, so a Job from `conformance-trigger` can carry an init container with `command: ["/bin/sh","-c"]` and an arbitrary script, running as the privileged `conformance-runner` SA. Reproduced by the policy's author: pass 3, fail 0 on a privileged alpine:latest init container running `kubectl get secrets -A`. The image now carries /usr/local/bin/conformance-clone, 0755 root:root, so uid 65532 cannot replace the one executable the policy will come to trust. Clone logic taken from PR #20's `clone-suite-at-commit`, with both load-bearing properties preserved: the SHA must match ^[0-9a-f]{40}$, and `rev-parse HEAD` is ASSERTED equal to it after checkout, so an unrendered placeholder is a hard failure instead of a silent run of the branch tip. No arguments are accepted. DEVIATION FROM THE BRIEF, DELIBERATE: the repo URL and clone target are COMPILED IN rather than read from the environment. I was asked to take SHA, repo URL and target dir from env. The policy contains no `env`/`valueFrom`/`secretKeyRef` clause at all -- so trusting CONFORMANCE_REPO_URL would move the injection one field over: the trigger points it at a repo it controls, this fetches that, and ./run.sh from the attacker's tree runs under the privileged SA. "No args" would not have stopped it. The env vars are still read so the Job template stays self-documenting, but a value disagreeing with the baked literal is a hard failure, never a redirect. The SHA is the only genuinely trusted per-run input. THIS IS HALF A FIX. What makes the fixed path REQUIRED rather than merely available is a 4th policy rule demanding command: ["/usr/local/bin/conformance-clone"] with no args on every initContainer. That edits the security boundary and belongs in its own reviewed PR -- it is NOT here. Until it lands the hole is open, and the README, the Containerfile and the script header all say so. VERIFIED by running it (uid 65532, --read-only): root:root 755 and confirmed not writable by the runner; seven negative cases each asserted to exit non-zero (arguments, missing SHA, a branch name, an unrendered @@placeholder@@, HOME off the workspace volume, a REDIRECTED repo URL, a clone target off the workspace volume); and the positive case -- a real clone of this repo at a real 40-hex SHA with rev-parse HEAD proven equal to it. Two bugs of mine that only running it could find, both fixed: the workspace-path guard matched /workspace/* and so rejected the image's own HOME=/workspace; and the negative checks were pipelines, which under pipefail aborted the verify at the first expected failure and silently skipped the other six. A verify that stops early looks exactly like a verify that passed. The anonymous-clone fallback is deliberate and logged: read access to this repo needs no credential today, the integrity guarantee is the SHA assertion rather than the credential, and requiring the Secret would make the script untestable without one. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
230 lines
11 KiB
Bash
Executable File
230 lines
11 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
###############################################################################
|
|
# conformance/incluster/build-image.sh — build the pinned `conformance-runner`
|
|
# image and print its digest. Closes docs/plans/l2-l3-conformance-ci-wiring.md §3 item 1.
|
|
#
|
|
# ./build-image.sh # build + verify + print the digest and push command
|
|
# ./build-image.sh --no-verify # build only (skip the in-image tool proof)
|
|
# ./build-image.sh --verify-clone # ALSO really clone this repo with conformance-clone.
|
|
# # Opt-in because it needs the Gitea host reachable; the
|
|
# # default verify is offline so an air-gapped build is
|
|
# # still fully checked rather than half-checked.
|
|
# ENGINE=docker ./build-image.sh # build with docker instead of podman
|
|
#
|
|
# THIS SCRIPT NEVER PUSHES. Publishing is an operator action: the registry credential is
|
|
# not an agent's to hold, and the pushed digest is what a fail-closed admission policy
|
|
# then trusts. So this builds locally, proves the toolchain works INSIDE the image as the
|
|
# non-root uid the policy demands, and prints the exact `podman push` line for a human.
|
|
#
|
|
# VERSIONS ARE NOT DEFINED HERE. They are read from infra/devcontainer/versions.env, the
|
|
# single source of truth (CHAINSAW_VERSION, KUBECTL_VERSION, FEDORA_VERSION) — the same
|
|
# pins the toolbox gate runs with. Nothing in this file invents a version.
|
|
#
|
|
# THE DIGEST CAVEAT — read it before pasting a digest into anything.
|
|
# The digest printed here is the LOCAL manifest digest. It is normally identical to the
|
|
# registry digest (podman pushes the manifest it already has rather than recompressing),
|
|
# but that is a property of the push, not a guarantee. So the push command below uses
|
|
# `--digestfile`: the digest the registry actually returns is the authoritative one, and
|
|
# that is the one that goes into conformance/versions.env.
|
|
###############################################################################
|
|
set -euo pipefail
|
|
|
|
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
REPO_ROOT="$(cd "${HERE}/../.." && pwd)"
|
|
ENGINE="${ENGINE:-podman}"
|
|
|
|
VERIFY=1
|
|
VERIFY_CLONE=0
|
|
for arg in "$@"; do
|
|
case "$arg" in
|
|
--no-verify) VERIFY=0 ;;
|
|
--verify-clone) VERIFY_CLONE=1 ;;
|
|
-h|--help) sed -n '2,/^####/p' "${BASH_SOURCE[0]}" | sed '$d' | sed 's/^# \{0,1\}//'; exit 0 ;;
|
|
*) echo "unknown argument: $arg" >&2; exit 2 ;;
|
|
esac
|
|
done
|
|
|
|
log() { printf '\n\033[1m==> %s\033[0m\n' "$*"; }
|
|
info() { printf ' %s\n' "$*"; }
|
|
die() { printf '\033[31mERROR: %s\033[0m\n' "$*" >&2; exit 1; }
|
|
|
|
command -v "${ENGINE}" >/dev/null 2>&1 || die "'${ENGINE}' not on PATH (set ENGINE=docker to use docker)"
|
|
|
|
# --- pins: sourced, never invented -------------------------------------------
|
|
TOOLCHAIN_ENV="${REPO_ROOT}/infra/devcontainer/versions.env"
|
|
[ -f "${TOOLCHAIN_ENV}" ] || die "missing ${TOOLCHAIN_ENV} — the pins live there, not here"
|
|
# shellcheck disable=SC1090
|
|
source "${TOOLCHAIN_ENV}"
|
|
: "${CHAINSAW_VERSION:?CHAINSAW_VERSION not set in ${TOOLCHAIN_ENV}}"
|
|
: "${KUBECTL_VERSION:?KUBECTL_VERSION not set in ${TOOLCHAIN_ENV}}"
|
|
: "${FEDORA_VERSION:?FEDORA_VERSION not set in ${TOOLCHAIN_ENV}}"
|
|
|
|
# --- image name: fixed by the admission policy -------------------------------
|
|
# platform/kyverno/admission/conformance-trigger-job-shape.yaml admits ONLY
|
|
# gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:*. The repository path
|
|
# is therefore not a preference; a different path is a DENIED Job. Do not "improve" it here.
|
|
IMAGE_REPO="gitea.home.keithjackson.dev/home-network/conformance-runner"
|
|
# The tag is build convenience only (a human-readable handle for the push). Admission
|
|
# ignores it: only the digest is ever referenced from a Job.
|
|
IMAGE_TAG="chainsaw${CHAINSAW_VERSION}-kubectl${KUBECTL_VERSION}-fc${FEDORA_VERSION}"
|
|
IMAGE_REF="${IMAGE_REPO}:${IMAGE_TAG}"
|
|
|
|
log "Building ${IMAGE_REF}"
|
|
info "engine ${ENGINE} ($(${ENGINE} --version 2>/dev/null | head -1))"
|
|
info "pins from ${TOOLCHAIN_ENV}"
|
|
info " CHAINSAW ${CHAINSAW_VERSION}"
|
|
info " KUBECTL ${KUBECTL_VERSION}"
|
|
info " FEDORA ${FEDORA_VERSION}"
|
|
|
|
"${ENGINE}" build \
|
|
--file "${HERE}/Containerfile" \
|
|
--tag "${IMAGE_REF}" \
|
|
--build-arg "CHAINSAW_VERSION=${CHAINSAW_VERSION}" \
|
|
--build-arg "KUBECTL_VERSION=${KUBECTL_VERSION}" \
|
|
--build-arg "FEDORA_VERSION=${FEDORA_VERSION}" \
|
|
"${HERE}"
|
|
|
|
# --- verify: the point of the whole exercise ---------------------------------
|
|
# The stock chainsaw image "had" chainsaw and still could not run ./run.sh. So prove the
|
|
# tools EXECUTE, and prove they execute as the NON-ROOT uid with a READ-ONLY root
|
|
# filesystem — the shape the admission policy forces. A pass here is the only evidence
|
|
# this image is usable; a build that merely succeeded is not.
|
|
if [ "${VERIFY}" -eq 1 ]; then
|
|
log "Verifying the toolchain inside the image (non-root, read-only rootfs)"
|
|
"${ENGINE}" run --rm \
|
|
--user 65532:65532 \
|
|
--read-only \
|
|
--tmpfs /workspace:rw,mode=1777 \
|
|
--entrypoint "" \
|
|
"${IMAGE_REF}" \
|
|
/bin/bash -euo pipefail -c '
|
|
echo "--- id ---"; id
|
|
echo "--- HOME ---"; echo "HOME=${HOME}"
|
|
echo "--- bash ---"; bash --version | head -1
|
|
echo "--- python3 ---"; python3 --version
|
|
echo "--- git ---"; git --version
|
|
echo "--- kubectl ---"; kubectl version --client
|
|
echo "--- chainsaw ---"; chainsaw version
|
|
echo "--- baked versions ---"; cat /etc/conformance-runner-versions
|
|
echo "--- conformance-clone: fixed path, root-owned, NOT writable by the runner ---"
|
|
stat -c "%n %U:%G %a" /usr/local/bin/conformance-clone
|
|
if [ -w /usr/local/bin/conformance-clone ]; then
|
|
echo "UNEXPECTED: uid $(id -u) can write the one executable the policy will trust"; exit 1
|
|
fi
|
|
echo "confirmed: not writable by uid $(id -u)"
|
|
# Each of these MUST fail. Assert the non-zero exit as well as the message: a check
|
|
# that only greps stderr would pass a build where the guard had stopped rejecting.
|
|
# (`|| true` inside the capture keeps `set -e`/`pipefail` from aborting on the
|
|
# expected failure — without it the verify dies at the first negative case.)
|
|
expect_fail() {
|
|
local label="$1"; shift
|
|
local out rc=0
|
|
out="$(env "$@" 2>&1)" || rc=$?
|
|
echo "--- ${label} ---"
|
|
printf "%s\n" "${out}" | head -2
|
|
if [ "${rc}" -eq 0 ]; then
|
|
echo "UNEXPECTED: that should have failed, but it exited 0"; exit 1
|
|
fi
|
|
echo "(rejected, exit ${rc})"
|
|
}
|
|
expect_fail "conformance-clone rejects arguments (that is where an injection would live)" \
|
|
conformance-clone --layer l2
|
|
expect_fail "conformance-clone rejects a missing SHA" \
|
|
conformance-clone
|
|
expect_fail "conformance-clone rejects a branch name instead of a SHA (no silent run of the tip)" \
|
|
CONFORMANCE_GIT_SHA=main conformance-clone
|
|
expect_fail "conformance-clone rejects an UNRENDERED placeholder" \
|
|
CONFORMANCE_GIT_SHA=@@CONFORMANCE_GIT_SHA@@ conformance-clone
|
|
expect_fail "conformance-clone rejects a HOME off the workspace volume" \
|
|
CONFORMANCE_GIT_SHA=0000000000000000000000000000000000000000 \
|
|
HOME=/etc conformance-clone
|
|
expect_fail "conformance-clone refuses a REDIRECTED repository URL (the env-injection hole)" \
|
|
CONFORMANCE_GIT_SHA=0000000000000000000000000000000000000000 \
|
|
CONFORMANCE_REPO_URL=https://evil.invalid/x.git conformance-clone
|
|
expect_fail "conformance-clone refuses a clone target off the workspace volume" \
|
|
CONFORMANCE_GIT_SHA=0000000000000000000000000000000000000000 \
|
|
CONFORMANCE_SRC=/etc/src conformance-clone
|
|
echo "--- rootfs is read-only (expected: a failure) ---"
|
|
if touch /should-fail 2>/dev/null; then echo "UNEXPECTED: rootfs is writable"; exit 1; fi
|
|
echo "confirmed: / is read-only"
|
|
echo "--- HOME is writable (the workspace emptyDir) ---"
|
|
touch "${HOME}/.probe" && echo "confirmed: ${HOME} is writable"
|
|
echo "--- uid is non-zero ---"
|
|
[ "$(id -u)" -ne 0 ] && echo "confirmed: running as uid $(id -u), not root"
|
|
'
|
|
log "Toolchain verified"
|
|
fi
|
|
|
|
# --- verify the clone helper against the REAL repository (opt-in: needs network) ----
|
|
# The negative cases above prove conformance-clone refuses bad input. This proves the
|
|
# positive case: it lands the EXACT commit asked for and can prove it did. Without this, the
|
|
# rev-parse assertion is only an unexercised line of shell.
|
|
if [ "${VERIFY_CLONE}" -eq 1 ]; then
|
|
log "Verifying conformance-clone against the real repository"
|
|
SHA="$(git -C "${REPO_ROOT}" rev-parse HEAD)"
|
|
info "requesting commit ${SHA}"
|
|
"${ENGINE}" run --rm \
|
|
--user 65532:65532 \
|
|
--read-only \
|
|
--tmpfs /workspace:rw,mode=1777 \
|
|
--network host \
|
|
--entrypoint "" \
|
|
-e "CONFORMANCE_GIT_SHA=${SHA}" \
|
|
"${IMAGE_REF}" \
|
|
/bin/bash -euo pipefail -c '
|
|
conformance-clone
|
|
echo "--- assert rev-parse HEAD == the requested SHA ---"
|
|
got="$(git -C /workspace/src rev-parse HEAD)"
|
|
echo "requested: ${CONFORMANCE_GIT_SHA}"
|
|
echo "HEAD: ${got}"
|
|
[ "${got}" = "${CONFORMANCE_GIT_SHA}" ] || { echo "MISMATCH"; exit 1; }
|
|
echo "confirmed: the working tree is at the requested commit"
|
|
test -x /workspace/src/conformance/run.sh && echo "confirmed: run.sh present and executable"
|
|
'
|
|
log "conformance-clone verified against the real repository"
|
|
fi
|
|
|
|
# --- digest ------------------------------------------------------------------
|
|
DIGEST="$("${ENGINE}" image inspect --format '{{.Digest}}' "${IMAGE_REF}")"
|
|
IMAGE_ID="$("${ENGINE}" image inspect --format '{{.Id}}' "${IMAGE_REF}")"
|
|
SIZE="$("${ENGINE}" image inspect --format '{{.Size}}' "${IMAGE_REF}")"
|
|
|
|
log "Built ${IMAGE_REF}"
|
|
info "local manifest digest ${DIGEST}"
|
|
info "image id ${IMAGE_ID}"
|
|
info "size $(( SIZE / 1024 / 1024 )) MiB"
|
|
|
|
cat <<EOF
|
|
|
|
================================================================================
|
|
NOT PUSHED. Publishing is an operator action — this script holds no registry
|
|
credential and will never acquire one.
|
|
|
|
OPERATOR: run these two commands, then paste the digest from the digestfile into
|
|
conformance/versions.env (CONFORMANCE_RUNNER_IMAGE) via a PR:
|
|
|
|
${ENGINE} login gitea.home.keithjackson.dev
|
|
|
|
${ENGINE} push --digestfile /tmp/conformance-runner.digest \\
|
|
${IMAGE_REF}
|
|
|
|
Then:
|
|
|
|
cat /tmp/conformance-runner.digest
|
|
# -> sha256:.... THIS is the authoritative digest, not the local one above.
|
|
|
|
# conformance/versions.env
|
|
CONFORMANCE_RUNNER_IMAGE="${IMAGE_REPO}@\$(cat /tmp/conformance-runner.digest)"
|
|
|
|
WHY THE DIGEST AND NOT THE TAG: platform/kyverno/admission/conformance-trigger-job-shape.yaml
|
|
is a fail-closed Enforce policy. It DENIES any conformance-trigger Job whose image is not
|
|
${IMAGE_REPO}@sha256:<digest>. A tag reference is denied,
|
|
and so is any other repository path.
|
|
|
|
TIGHTEN THE BOUNDARY AFTERWARDS: that policy's header asks for the wildcard
|
|
\`${IMAGE_REPO}@sha256:*\` to be replaced with the
|
|
LITERAL digest once the image exists, closing the "anyone who can push to this path" gap.
|
|
That is a separate, human-reviewed PR against the security boundary.
|
|
================================================================================
|
|
EOF
|