jakman85andClaude Opus 5
526a38d137
The committed L2/L3 Job pins `ghcr.io/kyverno/chainsaw:v0.2.12` with `command: ["./run.sh"]`. Those two lines cannot both be true. That image carries the chainsaw binary and essentially nothing else -- no bash, no python3, no kubectl, no git -- so the Job would have failed at exec of `./run.sh`, before a single test ran. Nothing in the tree said so. This adds the image that makes the Job runnable (plan docs/plans/l2-l3-conformance-ci-wiring.md section 3 item 1): * conformance/incluster/Containerfile -- bash + python3 + chainsaw + kubectl + git-core + tar/gzip/coreutils (the base64-tar results channel of item 3). Two-stage: a full Fedora fetches the release binaries and verifies kubectl's published sha256; the runtime layer is fedora-minimal and carries no downloader. Versions are --build-arg-injected from infra/devcontainer/versions.env (CHAINSAW_VERSION, KUBECTL_VERSION, FEDORA_VERSION) -- the single source of truth -- so the chainsaw and kubectl in the runner are the same pins the toolbox gate runs with. No new pin is invented here, and the ARG defaults are EMPTY so a build that forgets one fails loudly instead of baking a stale version. * conformance/incluster/build-image.sh -- builds it, PROVES the toolchain by executing it inside the image, and prints the digest. It never pushes: the registry credential is not an agent's to hold and publishing the artifact a fail-closed policy then trusts is an operator action. It prints the exact `podman push --digestfile ...` line instead. * conformance/versions.env -- CONFORMANCE_RUNNER_IMAGE, with the digest deliberately EMPTY and a loud block saying the operator fills it in after the push. Empty is a visible failure in whatever renders the Job; a guessed digest would be an ImagePullBackOff dressed up as a cluster fault. WHAT THE ADMISSION BOUNDARY FORCED, AND WHY THE IMAGE OBEYS IT platform/kyverno/admission/conformance-trigger-job-shape.yaml is Enforce + failurePolicy: Fail and it is the R-SEC-02 boundary. It is NOT modified here. Two of its clauses are load-bearing on the image itself: * `runAsNonRoot: true` with no explicit runAsUser means the kubelet rejects an image whose USER is a name it cannot resolve to a uid. So USER is NUMERIC (65532), not a friendly name. * The pod is readOnlyRootFilesystem and the policy admits only the `workspace` and `results` volumes -- there is no writable /tmp and one cannot be added without editing the boundary. So HOME, XDG_CACHE_HOME, XDG_CONFIG_HOME and TMPDIR all point into /workspace, and git's safe.directory config is written to /etc/gitconfig at BUILD time because a runtime --global write would fail. VERIFIED BY RUNNING IT, not by a successful build As uid 65532 with --read-only: bash 5.3.9, Python 3.14.7, git 2.55.0, kubectl v1.36.4 (client), chainsaw 0.2.12; / confirmed read-only, /workspace writable, uid confirmed non-zero. Then the real thing -- `./run.sh --lint`, `--coverage-only` and `--layer l2 --no-cluster` all rc 0 inside the image against this worktree, emitting coverage.md/.json and report.junit.xml. KNOWN LIMIT, FLAGGED NOT PAPERED OVER run.sh runs the selected offline L0/L1 checks before it touches the cluster for ANY selection that is not offline-only, and those checks shell out to flux, promtool, kyverno and kustomize, which this image does not carry. Baking the whole devcontainer toolchain into the runner would roughly double its size to re-run work ci/l0.sh and ci/l1.sh already did offline in the same CI run. So the Job must select the cluster-gated layers only (`--layer l2` / `--layer l3`); `--layer all` and a bare `--requirements <IDs>` would drag the L0 checks in and fail on a missing tool. `--layer l2` is verified above to resolve the offline selection to nothing. Choosing the Job's args belongs to T-6.1b; the constraint is documented in the Containerfile header and the incluster README rather than left for someone to discover from a pod log. chainsaw-job.yaml's `image:` is left stale ON PURPOSE -- there is no digest to put there yet, and the trigger renders that template per run (plan item 4). The README now says so instead of letting the placeholder read as a pin. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
62 lines
4.1 KiB
Markdown
62 lines
4.1 KiB
Markdown
# In-cluster conformance Job (L2/L3)
|
|
|
|
> **STATUS NOTE (2026-09-29) — read before trusting the "Phase 0" framing below.**
|
|
> This document speaks as though no cluster exists and the platform is scaffolding. **A
|
|
> three-node k3s cluster has existed for weeks** and Flux has been reconciling against it.
|
|
> Gitea is live, the Actions runner `incluster-dind` is online, and `main` is a protected
|
|
> branch. Where the text says "Phase 0", "stub", or "no cluster exists", read it as
|
|
> describing a **live gap** rather than an expected stage — most importantly, L2/L3
|
|
> conformance still does not run in CI (`vars.CLUSTER_AVAILABLE` is unset), which is a real
|
|
> coverage hole, not a phase.
|
|
>
|
|
> This mismatch is not cosmetic. A month of "the gate enforces this" claims rested on
|
|
> documentation rather than measurement, and `main` was unprotected with zero runners
|
|
> registered the whole time. **Query the live API before asserting platform state.**
|
|
|
|
|
|
Per **ADR-0011** and **docs/14 "Where each layer runs"**: L0/L1 run in CI without a
|
|
cluster, but **L2 behavioral and L3 fault-injection run as an in-cluster Chainsaw Job**
|
|
on the dev/test cluster that **CI triggers and waits for**. CI holds **no** cluster
|
|
credentials — the Job's own scoped ServiceAccount does (R-SEC-02).
|
|
|
|
## Files
|
|
- `rbac.yaml` — the `conformance` namespace, the `conformance-runner` ServiceAccount, and
|
|
a **least-privilege ClusterRole skeleton**. Rules start **empty** and are added — each
|
|
commented with the test/requirement that needs it — only as the matching fault-injection
|
|
test lands. This is a **privileged test identity**: never reused by workloads, its own
|
|
audit item.
|
|
- `chainsaw-job.yaml` — the Job skeleton. Single entrypoint `./run.sh`; CI sets `args` to
|
|
`--requirements <touched IDs>` (merge gate) or `--layer all --strict-coverage` (phase
|
|
exit). **Its `image:` is STALE**: it still names `ghcr.io/kyverno/chainsaw:v0.2.12`, which
|
|
(a) cannot run `./run.sh` at all and (b) is DENIED at admission. The trigger renders this
|
|
template per run with `CONFORMANCE_RUNNER_IMAGE` injected (plan §3 item 4, T-6.1b); until
|
|
that lands, treat the literal `image:` here as a placeholder, not a pin.
|
|
- `Containerfile` — the **`conformance-runner`** image: `bash` + `python3` + `chainsaw` +
|
|
`kubectl` + `git`, non-root (numeric uid 65532) and working under a read-only root
|
|
filesystem. This is what the Job actually needs; the stock chainsaw image carries none of
|
|
it. Versions are `--build-arg`-injected from `infra/devcontainer/versions.env` — the single
|
|
source of truth — so the chainsaw/kubectl in the runner are the same pins the toolbox gate
|
|
uses. Read its header before changing the Job's `args`: `run.sh` runs the selected offline
|
|
L0/L1 checks for any non-offline-only selection, and those need flux/promtool/kyverno/
|
|
kustomize, which this image deliberately omits. Use `--layer l2` / `--layer l3`.
|
|
- `build-image.sh` — builds the above, **verifies the toolchain by running it** inside the
|
|
image as uid 65532 with `--read-only`, and prints the digest. It **never pushes**: it
|
|
prints the exact `podman push --digestfile …` command for the operator instead. See
|
|
`conformance/versions.env` `CONFORMANCE_RUNNER_IMAGE` for the post-push pin.
|
|
|
|
## Flow
|
|
1. CI (Gitea Actions, delivered in T-6.1b) creates a Job from `chainsaw-job.yaml` (via a
|
|
trigger with narrow rights: create Job + read its logs/status — *not* a broad
|
|
kubeconfig).
|
|
2. The Job clones/loads the impl repo at the commit under test into `/workspace` and runs
|
|
`./run.sh` under `conformance-runner`.
|
|
3. `run.sh` runs Chainsaw over the selected tests and writes `report.junit.xml` +
|
|
`coverage/` to the results volume (`/workspace/conformance/_artifacts`).
|
|
4. CI collects those artifacts (results volume or the terminating log stream, which echoes
|
|
their paths) and enforces the merge / phase-exit gate on them.
|
|
|
|
## Phase 0
|
|
No cluster exists, so nothing here is applied yet. These manifests are reconciled by Flux
|
|
from Phase 1+ and first exercised when L2/L3 can run (T-3.0 onward). In Phase 0 the same
|
|
`run.sh` runs in CI's no-cluster stage and emits a **skipped** JUnit + the coverage report.
|