# In-cluster conformance Job (L2/L3)

> **STATUS NOTE (2026-09-29) — read before trusting the "Phase 0" framing below.**
> This document speaks as though no cluster exists and the platform is scaffolding. **A
> three-node k3s cluster has existed for weeks** and Flux has been reconciling against it.
> Gitea is live, the Actions runner `incluster-dind` is online, and `main` is a protected
> branch. Where the text says "Phase 0", "stub", or "no cluster exists", read it as
> describing a **live gap** rather than an expected stage — most importantly, L2/L3
> conformance still does not run in CI (`vars.CLUSTER_AVAILABLE` is unset), which is a real
> coverage hole, not a phase.
>
> This mismatch is not cosmetic. A month of "the gate enforces this" claims rested on
> documentation rather than measurement, and `main` was unprotected with zero runners
> registered the whole time. **Query the live API before asserting platform state.**


Per **ADR-0011** and **docs/14 "Where each layer runs"**: L0/L1 run in CI without a
cluster, but **L2 behavioral and L3 fault-injection run as an in-cluster Chainsaw Job**
on the dev/test cluster that **CI triggers and waits for**. CI holds **no** cluster
credentials — the Job's own scoped ServiceAccount does (R-SEC-02).

## Files
- `rbac.yaml` — the `conformance` namespace, the `conformance-runner` ServiceAccount, and
  a **least-privilege ClusterRole skeleton**. Rules start **empty** and are added — each
  commented with the test/requirement that needs it — only as the matching fault-injection
  test lands. This is a **privileged test identity**: never reused by workloads, its own
  audit item.
- `chainsaw-job.yaml` — the **run template** (not reconciled; the trigger renders it per
  run). Single entrypoint `./run.sh`; a git-clone init container delivers the suite at the
  commit under test; artifacts leave over stdout. Its header is the authority on the
  substitution tokens, the admission constraints it satisfies, and **an open security gap
  in the admission policy that it documents and does not exploit** — read it before
  editing either file.
- `Containerfile` — the **`conformance-runner`** image: `bash` + `python3` + `chainsaw` +
  `kubectl` + `git`, non-root (numeric uid 65532) and working under a read-only root
  filesystem. This is what the Job actually needs; the stock chainsaw image carries none of
  it. Versions are `--build-arg`-injected from `infra/devcontainer/versions.env` — the single
  source of truth — so the chainsaw/kubectl in the runner are the same pins the toolbox gate
  uses. Read its header before changing the Job's `args`: `run.sh` runs the selected offline
  L0/L1 checks for any non-offline-only selection, and those need flux/promtool/kyverno/
  kustomize, which this image deliberately omits. Use `--layer l2` / `--layer l3`.
- `conformance-clone` — the init container's **only** entrypoint, baked into the image at
  `/usr/local/bin/conformance-clone` (0755 **root:root**, so uid 65532 cannot replace it).
  Clones the impl repo at the commit under test: the SHA must match `^[0-9a-f]{40}$` and
  `rev-parse HEAD` is **asserted** equal to it afterwards, so an unrendered placeholder is a
  hard failure rather than a silent run of the branch tip. It takes **no arguments**, and the
  repository URL and clone target are **compiled in** — see its header for why the env vars
  are read but may not override them.
  > **This file is half a fix.** The admission policy pins the main container's `command` but
  > **never reads `initContainers`**, so an init container with `command: ["/bin/sh","-c"]` and
  > an arbitrary script is currently ADMITTED and runs as the privileged `conformance-runner`
  > SA. What makes this file *required* rather than merely available is a 4th policy rule
  > demanding `command: ["/usr/local/bin/conformance-clone"]` with no `args`. That rule belongs
  > in its own reviewed PR against the boundary and is **not** in this branch.
- `build-image.sh` — builds the above, **verifies the toolchain by running it** inside the
  image as uid 65532 with `--read-only`, and prints the digest. It **never pushes**: it
  prints the exact `podman push --digestfile …` command for the operator instead. See
  `conformance/versions.env` `CONFORMANCE_RUNNER_IMAGE` for the post-push pin.
  `--verify-clone` additionally has `conformance-clone` really clone this repo at a real SHA
  and asserts the checkout matches (opt-in, because it needs the Gitea host reachable — the
  default verify stays offline so an air-gapped build is fully checked, not half-checked).

## Flow
1. CI (Gitea Actions) renders `chainsaw-job.yaml` — two `@@TOKEN@@` substitutions, the
   runner image digest and the commit SHA — replaces `args` with the invocation shape, and
   `create`s the Job under the narrow `conformance-trigger` identity (create/get jobs, get
   pods + pods/log — *not* a broad kubeconfig, and deliberately **no `pods/exec`**).
2. Admission (`platform/kyverno/admission/conformance-trigger-job-shape.yaml`, Enforce +
   fail-closed) admits only the committed shape. That policy, not the RBAC, is the boundary.
3. The **init container** shallow-clones the repo into `/workspace/src`, fetches the commit
   if the shallow tip does not already contain it, detaches onto it, and **asserts `HEAD`
   equals the requested SHA** — the suite runs at the commit under test or not at all. The
   read-only Gitea credential reaches `git` as env vars from `secretKeyRef` (secret
   `conformance-git-readonly`, keys `username`/`token`), never as a volume: admission allows
   no volume but `workspace` and `results`.
4. The runner executes `./run.sh` under `conformance-runner`, writing `report.junit.xml` +
   `coverage/` to the `results` volume at `/results` (`CONF_ARTIFACTS`).
5. On exit — green, red or aborted — `run.sh` prints a delimited base64 tar of that tree to
   stdout, carrying `bytes`, `lines` and `sha256` in **both** the BEGIN and END markers. CI
   decodes it from `kubectl logs` and must make all five completeness checks listed above
   `conf_emit_artifacts_tgz()` in `../run.sh`; a partial log then fails collection instead
   of decoding to plausible garbage.

## Operator prerequisites (out of GitOps — neither exists yet)
- Secret `conformance-git-readonly` in namespace `conformance`, keys `username` and
  `token`: a **read-only** Gitea token for this repository. Never a write token or a
  push-capable deploy key.
- The `conformance-runner` image built and pushed (plan §5 step 1); its digest is what the
  trigger substitutes. The image must provide bash, git, python3, chainsaw and kubectl, and
  a **uid 65532** non-root user, which the Job's `runAsUser`/`fsGroup` pin.

## Phase 0
No cluster exists, so nothing here is applied yet. These manifests are reconciled by Flux
from Phase 1+ and first exercised when L2/L3 can run (T-3.0 onward). In Phase 0 the same
`run.sh` runs in CI's no-cluster stage and emits a **skipped** JUnit + the coverage report.