Files
home-network-k3s-impl/conformance/incluster
jakman85andClaude Opus 5 c8899368e7
gate / gate (pull_request) Successful in 1m36s
gate / conformance (pull_request) Has been skipped
fix(conformance): state the chainsaw supply-chain gap accurately
The Containerfile said chainsaw "publishes no per-asset checksum file at a stable
URL". Literally true, and misleading: it reads as "no integrity option exists".
Checked the release assets via the GitHub API -- v0.2.12 ships per-asset cosign
`.sig` + `.pem` and an SBOM alongside each tarball. So verification IS available,
it just needs cosign in the builder stage and the right keyless identity for
kyverno's release workflow.

Comment now says what is actually true: kubectl is sha256-verified, chainsaw rests
on tag + TLS, and the cosign path is named as a TODO rather than implied away.
Deliberately not implemented here -- picking a trusted signer identity is a
supply-chain decision that belongs in its own reviewed change, not a silent
addition to a runner image. Noted that infra/devcontainer/Containerfile has the
same gap for the same binary.

Rebuilt after the edit; the in-image verification still passes.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-29 19:22:03 -05:00
..

In-cluster conformance Job (L2/L3)

STATUS NOTE (2026-09-29) — read before trusting the "Phase 0" framing below. This document speaks as though no cluster exists and the platform is scaffolding. A three-node k3s cluster has existed for weeks and Flux has been reconciling against it. Gitea is live, the Actions runner incluster-dind is online, and main is a protected branch. Where the text says "Phase 0", "stub", or "no cluster exists", read it as describing a live gap rather than an expected stage — most importantly, L2/L3 conformance still does not run in CI (vars.CLUSTER_AVAILABLE is unset), which is a real coverage hole, not a phase.

This mismatch is not cosmetic. A month of "the gate enforces this" claims rested on documentation rather than measurement, and main was unprotected with zero runners registered the whole time. Query the live API before asserting platform state.

Per ADR-0011 and docs/14 "Where each layer runs": L0/L1 run in CI without a cluster, but L2 behavioral and L3 fault-injection run as an in-cluster Chainsaw Job on the dev/test cluster that CI triggers and waits for. CI holds no cluster credentials — the Job's own scoped ServiceAccount does (R-SEC-02).

Files

  • rbac.yaml — the conformance namespace, the conformance-runner ServiceAccount, and a least-privilege ClusterRole skeleton. Rules start empty and are added — each commented with the test/requirement that needs it — only as the matching fault-injection test lands. This is a privileged test identity: never reused by workloads, its own audit item.
  • chainsaw-job.yaml — the Job skeleton. Single entrypoint ./run.sh; CI sets args to --requirements <touched IDs> (merge gate) or --layer all --strict-coverage (phase exit). Its image: is STALE: it still names ghcr.io/kyverno/chainsaw:v0.2.12, which (a) cannot run ./run.sh at all and (b) is DENIED at admission. The trigger renders this template per run with CONFORMANCE_RUNNER_IMAGE injected (plan §3 item 4, T-6.1b); until that lands, treat the literal image: here as a placeholder, not a pin.
  • Containerfile — the conformance-runner image: bash + python3 + chainsaw + kubectl + git, non-root (numeric uid 65532) and working under a read-only root filesystem. This is what the Job actually needs; the stock chainsaw image carries none of it. Versions are --build-arg-injected from infra/devcontainer/versions.env — the single source of truth — so the chainsaw/kubectl in the runner are the same pins the toolbox gate uses. Read its header before changing the Job's args: run.sh runs the selected offline L0/L1 checks for any non-offline-only selection, and those need flux/promtool/kyverno/ kustomize, which this image deliberately omits. Use --layer l2 / --layer l3.
  • build-image.sh — builds the above, verifies the toolchain by running it inside the image as uid 65532 with --read-only, and prints the digest. It never pushes: it prints the exact podman push --digestfile … command for the operator instead. See conformance/versions.env CONFORMANCE_RUNNER_IMAGE for the post-push pin.

Flow

  1. CI (Gitea Actions, delivered in T-6.1b) creates a Job from chainsaw-job.yaml (via a trigger with narrow rights: create Job + read its logs/status — not a broad kubeconfig).
  2. The Job clones/loads the impl repo at the commit under test into /workspace and runs ./run.sh under conformance-runner.
  3. run.sh runs Chainsaw over the selected tests and writes report.junit.xml + coverage/ to the results volume (/workspace/conformance/_artifacts).
  4. CI collects those artifacts (results volume or the terminating log stream, which echoes their paths) and enforces the merge / phase-exit gate on them.

Phase 0

No cluster exists, so nothing here is applied yet. These manifests are reconciled by Flux from Phase 1+ and first exercised when L2/L3 can run (T-3.0 onward). In Phase 0 the same run.sh runs in CI's no-cluster stage and emits a skipped JUnit + the coverage report.