Compare commits
12
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9275dae6e0 | ||
|
|
64944aeda0 | ||
|
|
5a006aeb15 | ||
|
|
20ea70d221 | ||
|
|
8c7a770de2 | ||
|
|
1c5974013e | ||
|
|
4791d4a43a | ||
|
|
dfd608ac32 | ||
|
|
1d2d9d6549 | ||
|
|
50a48e6f07 | ||
|
|
8eb5baffa4 | ||
|
|
6586919132 |
No files matched your search
@@ -124,8 +124,13 @@ jobs:
|
||||
- name: Merge gate — ci/gate.sh (what `make -C ci gate` runs)
|
||||
env:
|
||||
GATE_STRICT_SKIPS: "true"
|
||||
GITEA_SETTINGS_TOKEN: ${{ secrets.GITEA_SETTINGS_TOKEN }}
|
||||
# REQUIRE_GITEA_SETTINGS: "true" # <- uncomment AFTER seeing gitea-settings green
|
||||
SETTINGS_API_TOKEN: ${{ secrets.SETTINGS_API_TOKEN }}
|
||||
# ENABLED 2026-09-29. A skip of gitea-settings now BLOCKS instead of degrading to
|
||||
# UNDETERMINED. Safe because THIS PR's own CI run is the proof: if the stage were
|
||||
# skipping, this line turns that into a failure and the PR goes red instead of
|
||||
# merging. Green here means the token resolved and the live Gitea matched the
|
||||
# declaration in infra/gitea/expected-branch-protection.json.
|
||||
REQUIRE_GITEA_SETTINGS: "true"
|
||||
run: ./ci/gate.sh || [ "$?" -eq 3 ]
|
||||
|
||||
# NOTE: no actions/upload-artifact step. Gitea Actions rejects upload-artifact@v4
|
||||
|
||||
@@ -5109,3 +5109,101 @@ next L2/L3 piece on top of unmerged branches would create a rebase tangle for no
|
||||
cannot land any faster. The next piece is specified and ready to start once these are merged:
|
||||
`docs/plans/l2-l3-conformance-ci-wiring.md` §3, with §4's credential model now decided and its
|
||||
boundary landed in #13.
|
||||
|
||||
## 2026-09-29 (agent, branch `feat/conformance-trigger-kubeconfig`) — the trigger credential script: it REFUSES, on purpose
|
||||
|
||||
Delivered `infra/gitea/conformance-trigger-kubeconfig.sh` —
|
||||
`docs/plans/l2-l3-conformance-ci-wiring.md` §3 item 6 — plus its section in
|
||||
`infra/gitea/README.md`. **The script is UNEXECUTED.** Minting a cluster credential and changing
|
||||
Gitea variables are operator actions; nothing was minted, no Gitea object was written, no
|
||||
`kubectl` mutation was made. Everything below is a claim about the *script*, not about the
|
||||
platform.
|
||||
|
||||
### What it does
|
||||
|
||||
Mints a token for `conformance/conformance-trigger` (TokenRequest), wraps **only that token**
|
||||
with the cluster CA and server into a kubeconfig, stores it as the Actions secret
|
||||
`CONFORMANCE_TRIGGER_KUBECONFIG` (legal name — **not** the reserved `GITEA_` prefix), and sets
|
||||
`vars.CLUSTER_AVAILABLE` only as a separately-requested, precondition-gated, TTY-confirmed final
|
||||
step. The token is never printed and never reaches argv: it goes into a 0600 file and into the
|
||||
PUT **body file**. The script refuses to run under `set -x`.
|
||||
|
||||
### The design decision worth reviewing: TWO precondition classes
|
||||
|
||||
- **mint-blocking (cluster-side):** SA/Role/RoleBinding present, and ClusterPolicy
|
||||
`conformance-trigger-job-shape` present + `Enforce` + `failurePolicy: Fail` + Ready. If any
|
||||
fails, **nothing is minted**. Rationale: Kubernetes puts no `use` check on a Job's
|
||||
`serviceAccountName`, so a `create jobs` token in that namespace IS the privileged
|
||||
`conformance-runner` identity unless the admission policy bounds it. RBAC cannot fix that
|
||||
(plan §4); minting before the boundary is live would hand CI the escalation the boundary
|
||||
exists to close.
|
||||
- **arm-blocking (repo-side):** digest-pinned `CONFORMANCE_RUNNER_IMAGE`, an init-container plus
|
||||
the admitted image in `chainsaw-job.yaml`, and `CONFORMANCE_TRIGGER_KUBECONFIG` referenced in
|
||||
`ci/conformance.sh` **code rather than a comment**. These block only the variable, because
|
||||
storing the secret early is harmless and is the plan's own order.
|
||||
|
||||
**Measured in the tree today: all three arm-blocking checks FAIL** (no
|
||||
`CONFORMANCE_RUNNER_IMAGE` in `conformance/versions.env`; `chainsaw-job.yaml` is still the
|
||||
skeleton with an empty `workspace` emptyDir and the stock chainsaw image; `ci/conformance.sh`
|
||||
names the trigger kubeconfig only in its TODO comment). So on a live run today the script would
|
||||
provision the secret and then **refuse to arm `CLUSTER_AVAILABLE`**, naming each missing item —
|
||||
which is the intended behaviour, not a defect. §3 items 1, 2 and 4 are the work that unblocks it.
|
||||
|
||||
### Verification of the credential before it is stored
|
||||
|
||||
`auth whoami` must return `system:serviceaccount:conformance:conformance-trigger`; a real
|
||||
`get jobs` round-trip must succeed; then an `auth can-i` matrix asserts BOTH halves — granted
|
||||
(`create/get jobs`, `get pods`, `get pods/log`) and **denied** (`delete jobs`, `create pods/exec`,
|
||||
`get secrets`, `create jobs` in `default`, `get nodes`, `'*' '*' --all-namespaces`). A mismatch
|
||||
aborts before anything is stored, and the message says explicitly **not** to widen
|
||||
`conformance/incluster/rbac.yaml` to make it pass.
|
||||
|
||||
### Both writes are verified the way `set-ci-token.sh` learned to
|
||||
|
||||
`curl -sS` exits 0 on an HTTP error, so status alone proves nothing: each write checks
|
||||
`%{http_code}` **and** reads the object back. For the secret that is presence-by-name only
|
||||
(Gitea never returns secret values) and the script says so rather than implying the content was
|
||||
checked; for the variable it asserts the value reads back as exactly `true`.
|
||||
|
||||
### UNVERIFIED — state this plainly
|
||||
|
||||
Only `bash -n` was run (clean); `shellcheck` is not on the host and there is no shellcheck stage
|
||||
in the gate. Nothing in the script has *executed*: the Gitea variables API shape (POST-creates /
|
||||
PUT-updates, body field `value` not `data`) is from documentation, not measurement — the script
|
||||
tries POST then falls back to PUT and prints the response body on failure, and the read-back is
|
||||
the authority. The Kyverno readiness field (`.status.ready` vs. a Ready condition) is handled
|
||||
both ways and treats "neither present" as a block rather than assuming ready. Whether the
|
||||
kubeconfig's embedded server URL is reachable from the runner pod cannot be known from here —
|
||||
hence `--server`, and a hard refusal on a loopback address.
|
||||
|
||||
### Noted, not fixed (out of scope for this branch)
|
||||
|
||||
`infra/gitea/README.md`'s detector table still names `GITEA_SETTINGS_TOKEN` — the illegal name
|
||||
PR #15 replaced with `SETTINGS_API_TOKEN`. Harmless prose today, but it is the exact trap that
|
||||
directory documents. Worth a one-line docs PR.
|
||||
|
||||
### Gate (pinned toolbox, from the agent worktree)
|
||||
|
||||
`MERGE GATE: UNDETERMINED`, **rc 3**. `lint`, `l0`, `render-golden`, `l1` all green; touched
|
||||
requirement resolved as `R-ENV-01`. Stages that did not run: `spec-immutability` (sibling spec
|
||||
repo not checked out — the documented tolerated skip) and `gitea-settings` (`SETTINGS_API_TOKEN`
|
||||
is a credential this agent does not hold). Both are the expected local result and `gate.yml`
|
||||
tolerates exit 3; neither was "fixed". Run with `DEV_KUBECONFIG=/dev/null` and an `ENGINE` shim
|
||||
bind-mounting the main checkout's `.git` read-only, per `CLAUDE.md` §3.
|
||||
|
||||
### Measured on PR #19, and it contradicts the brief: there is NO `approval-policy` check
|
||||
|
||||
The brief for this work stated this PR's CI would FAIL `approval-policy` because `infra/gitea/**`
|
||||
is security-relevant. Measured on the PR head commit's `/statuses`:
|
||||
|
||||
gate / gate (pull_request) success Successful in 1m34s
|
||||
gate / conformance (pull_request) pending Blocked by required conditions
|
||||
|
||||
`.gitea/workflows/` on `main` contains only `gate.yml` and the dormant `merge-gate.yaml`. **No
|
||||
path-scoped approval policy exists**, so nothing machine-enforces "security-relevant paths need a
|
||||
human"; that rule rests entirely on `required_approvals: 1` plus a reviewer actually reading the
|
||||
diff. Another instance of the §2b failure mode — a control believed to exist because a brief said
|
||||
so. If the policy is intended, it is missing and needs its own change.
|
||||
|
||||
And per this repo's own rule: the green `gate` tick is **not** evidence this script works. The
|
||||
gate never executes `infra/gitea/**`, and the L2/L3 stage skipped. The script remains UNEXECUTED.
|
||||
@@ -61,5 +61,45 @@ if [ -d "${CONF_ART}/coverage" ]; then
|
||||
cp -r "${CONF_ART}/coverage/." "${COVERAGE_DIR}/"
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------------------
|
||||
# DID IT ACTUALLY RUN? -- the guard without which this stage FAKES A GREEN.
|
||||
#
|
||||
# run.sh exits 0 in "scaffold mode" when it cannot reach a cluster: it emits a JUnit file
|
||||
# whose every testcase is <skipped> and returns success. That is correct FOR run.sh -- an
|
||||
# honest skip is its documented no-cluster behaviour. It is catastrophic HERE, because
|
||||
# reaching this point means the environment DECLARED CLUSTER_AVAILABLE=true, and this stage
|
||||
# would otherwise print "L2/L3 conformance passed" over 54 tests that never executed.
|
||||
#
|
||||
# MEASURED 2026-09-29, before this guard existed:
|
||||
# CLUSTER_AVAILABLE=true ./ci/conformance.sh -> rc=0
|
||||
# "[junit] wrote skipped JUnit for 54 scaffolded test(s)"
|
||||
# "[ ok ] L2/L3 conformance passed for touched subset"
|
||||
# One Gitea variable away from a fake green on an enforcing merge gate. That is exactly the
|
||||
# HARD RULE 4 violation this repo's whole gate design exists to prevent, and nothing caught
|
||||
# it because nobody had ever set the variable.
|
||||
#
|
||||
# A skip while a cluster is DECLARED is a contradiction: either the declaration is wrong or
|
||||
# the cluster is unreachable. The merge-gate decision matrix already says this case is BLOCK
|
||||
# ("conformance skipped while CLUSTER_AVAILABLE=true -> BLOCK (it should have run)"). This
|
||||
# makes the stage itself honour that instead of reporting a pass.
|
||||
JUNIT="${CONF_ART}/report.junit.xml"
|
||||
if [ -f "$JUNIT" ]; then
|
||||
total="$(grep -o '<testcase' "$JUNIT" | wc -l | tr -d ' ')"
|
||||
skipped="$(grep -o '<skipped' "$JUNIT" | wc -l | tr -d ' ')"
|
||||
if [ "${total:-0}" -gt 0 ] && [ "${skipped:-0}" -ge "${total:-0}" ]; then
|
||||
err "CLUSTER_AVAILABLE=true, but the L2/L3 suite SKIPPED every one of its ${total} tests"
|
||||
err " (scaffold mode -- run.sh could not reach a cluster and exited 0)."
|
||||
err " This stage will NOT report that as a pass. Either the cluster is unreachable from"
|
||||
err " this runner, or CLUSTER_AVAILABLE is set when it should not be."
|
||||
err " NOTE: the in-cluster trigger path is still UNIMPLEMENTED -- see"
|
||||
err " docs/plans/l2-l3-conformance-ci-wiring.md. Setting CLUSTER_AVAILABLE=true does not"
|
||||
err " by itself make L2/L3 run; it only makes this stage attempt a LOCAL run that the CI"
|
||||
err " runner has no credential for (automountServiceAccountToken: false, by design)."
|
||||
summary "- conformance-l2-l3: **BLOCKED — declared available but every test skipped** (not a pass)"
|
||||
exit 1
|
||||
fi
|
||||
log "L2/L3 executed: ${total} testcase(s), ${skipped} skipped"
|
||||
fi
|
||||
|
||||
ok "L2/L3 conformance passed for touched subset"
|
||||
summary "- conformance-l2-l3: **passed**"
|
||||
@@ -56,14 +56,20 @@ log "Stage: Gitea platform settings match the declaration"
|
||||
DECL="${REPO_ROOT}/infra/gitea/expected-branch-protection.json"
|
||||
GITEA_URL="${GITEA_URL:-https://gitea.home.keithjackson.dev}"
|
||||
GITEA_REPO="${GITEA_REPO:-Home-Network/home-network-k3s-impl}"
|
||||
TOKEN="${GITEA_SETTINGS_TOKEN:-}"
|
||||
# SETTINGS_API_TOKEN, not GITEA_SETTINGS_TOKEN. Gitea RESERVES the `GITEA_` prefix for
|
||||
# secret names and rejects a PUT of one with HTTP 400 "invalid secret name" — the same way
|
||||
# GitHub reserves `GITHUB_`. The original name could therefore never be stored, and because
|
||||
# infra/gitea/set-ci-token.sh sent that PUT to /dev/null and printed "stored" unconditionally,
|
||||
# it looked like it had worked. A token was minted, no secret existed, and this stage would
|
||||
# have skipped forever. GITEA_SETTINGS_TOKEN is still honoured for a local run.
|
||||
TOKEN="${SETTINGS_API_TOKEN:-${GITEA_SETTINGS_TOKEN:-}}"
|
||||
|
||||
[ -f "$DECL" ] || die "declaration missing: ${DECL}"
|
||||
require_tool jq
|
||||
require_tool curl
|
||||
|
||||
if [ -z "$TOKEN" ]; then
|
||||
skip "GITEA_SETTINGS_TOKEN is not set — cannot read the live Gitea rule."
|
||||
skip "SETTINGS_API_TOKEN is not set — cannot read the live Gitea rule."
|
||||
skip " Branch protection and runner registration are therefore UNVERIFIED. They were"
|
||||
skip " wrong, unnoticed, for a month (2026-09-29). Wire it: infra/gitea/set-ci-token.sh"
|
||||
summary "- gitea-settings: **SKIPPED — no token** (protection + runners UNVERIFIED; not a pass)"
|
||||
|
||||
@@ -79,12 +79,37 @@ else
|
||||
cp -a "${KYVERNO_POLICY_DIR}/." "$l0_render/"
|
||||
find "$l0_render" -type f -name '*.yaml' -print0 \
|
||||
| xargs -0 sed -i "s/\${KYVERNO_VALIDATION_ACTION}/${KYVERNO_VALIDATION_ACTION}/g"
|
||||
if kyverno test "$l0_render"; then
|
||||
ok "Kyverno L0 policy tests passed"
|
||||
summary "- kyverno-l0: **passed**"
|
||||
# DO NOT TRUST `kyverno test`'s EXIT CODE ALONE. A policy that fails to LOAD (schema
|
||||
# error, bad match block, malformed JMESPath) does not fail the run: every result row
|
||||
# becomes `Skip / Invalid Policy` and the CLI still prints "N tests passed" and exits 0.
|
||||
#
|
||||
# MEASURED 2026-09-29 while authoring conformance-trigger-job-shape: the policy had its
|
||||
# `subjects` in a separate `match.all` entry from its `resources` — a schema error. All 9
|
||||
# rows came back `Skip / Invalid Policy`, and the summary read:
|
||||
# Test Summary: 9 tests passed and 0 tests failed
|
||||
# `kyverno apply` on the same file reported `error: 1` and named the path. So the gate's
|
||||
# own policy stage would have certified a SECURITY POLICY THAT DOES NOT LOAD as green.
|
||||
#
|
||||
# This is not specific to that policy: a typo in restrict-host-namespaces or
|
||||
# secret-provenance would silently disable it and the gate would stay green. Scan the
|
||||
# output for the load failure the exit code hides.
|
||||
l0_kout="$(mktemp)"
|
||||
if kyverno test "$l0_render" 2>&1 | tee "$l0_kout"; then
|
||||
if grep -q 'Invalid Policy' "$l0_kout"; then
|
||||
err "Kyverno reported 'Invalid Policy' — one or more policies FAILED TO LOAD and were"
|
||||
err " skipped, so their rules were never evaluated. The CLI exits 0 for this; the gate"
|
||||
err " must not. Run 'kyverno apply <policy> --resource <resources>' for the exact path."
|
||||
grep -n 'Invalid Policy' "$l0_kout" | head -5 >&2
|
||||
summary "- kyverno-l0: **FAILED — policy did not load** (Invalid Policy; rules never evaluated)"
|
||||
rc=1
|
||||
else
|
||||
ok "Kyverno L0 policy tests passed"
|
||||
summary "- kyverno-l0: **passed**"
|
||||
fi
|
||||
else
|
||||
err "Kyverno L0 policy tests failed"; summary "- kyverno-l0: **FAILED**"; rc=1
|
||||
fi
|
||||
rm -f "$l0_kout"
|
||||
rm -rf "$l0_render"
|
||||
fi
|
||||
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
---
|
||||
# Conformance identities (ns + conformance-runner + conformance-trigger RBAC).
|
||||
#
|
||||
# NON-GATING, exactly like apps-vms: `wait: false` and its own Kustomization, so a conformance
|
||||
# identity can never gate the platform or app tiers. A test harness that stalls the estate it is
|
||||
# meant to verify is worse than no harness.
|
||||
#
|
||||
# dependsOn platform-config only — it needs nothing from the app waves, and nothing in the app
|
||||
# waves needs it.
|
||||
#
|
||||
# NOTE: this reconciles RBAC that is INERT until
|
||||
# platform/kyverno/admission/conformance-trigger-job-shape.yaml is also enforcing. That policy
|
||||
# ships in the `platform` tier, which reconciles before this. Do not reorder them so that the
|
||||
# trigger identity can exist while the policy bounding it does not.
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: conformance
|
||||
namespace: flux-system
|
||||
spec:
|
||||
interval: 10m
|
||||
retryInterval: 2m
|
||||
timeout: 5m
|
||||
prune: true
|
||||
wait: false
|
||||
dependsOn:
|
||||
- name: platform-config
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
name: flux-system
|
||||
path: ./conformance/incluster
|
||||
@@ -33,6 +33,7 @@ resources:
|
||||
- apps-wave-2.yaml # Flux Kustomization -> ./apps/wave-2 (dependsOn apps-wave-1)
|
||||
# VM tier (KubeVirt) — non-gating (wait:false), separate lifecycle from the app waves.
|
||||
- apps-vms.yaml # Flux Kustomization -> ./apps/vms (dependsOn platform-config)
|
||||
- conformance.yaml # Flux Kustomization -> ./conformance/incluster (non-gating; R-SEC-02 trigger identity)
|
||||
# oauth2-proxy forward-auth — POST-platform tier (dependsOn platform-config, wait:false). A leaf
|
||||
# service that needs Keycloak + trusted TLS + split-horizon DNS must NOT sit in the gating
|
||||
# `platform` tier, where it self-gated its own dependencies into a cold-start deadlock (twice).
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
# Reconciled by clusters/base/conformance.yaml (non-gating, wait:false).
|
||||
#
|
||||
# RBAC ONLY. chainsaw-job.yaml is deliberately NOT listed: that manifest is a TEMPLATE the CI
|
||||
# trigger renders and submits per run (with the commit SHA and the touched --requirements). Flux
|
||||
# reconciling it would create a permanent Job object and fight the trigger for ownership.
|
||||
#
|
||||
# The identities here are inert without
|
||||
# platform/kyverno/admission/conformance-trigger-job-shape.yaml, which is what actually bounds
|
||||
# the trigger to the committed run shape. RBAC is the necessary half, that policy the sufficient
|
||||
# half — see the header of rbac.yaml.
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
resources:
|
||||
- rbac.yaml
|
||||
@@ -174,3 +174,68 @@ subjects:
|
||||
- kind: ServiceAccount
|
||||
name: conformance-runner
|
||||
namespace: conformance
|
||||
---
|
||||
# =============================================================================
|
||||
# THE TRIGGER IDENTITY — deliberately NOT conformance-runner.
|
||||
#
|
||||
# THE ESCALATION THIS EXISTS TO CLOSE (docs/plans/l2-l3-conformance-ci-wiring.md §4):
|
||||
# Kubernetes places NO `use` check on a Pod/Job's `serviceAccountName`. So any principal
|
||||
# that can `create jobs` in this namespace can create a Job with
|
||||
# `serviceAccountName: conformance-runner` and an ARBITRARY command — i.e. run arbitrary
|
||||
# code with the runner's privileges (nodes/patch, pods/exec, namespaces/patch, jobs).
|
||||
# A CI-held token with plain `create jobs` therefore IS the privileged identity, no matter
|
||||
# how narrow its own RBAC looks. That would break R-SEC-02 ("CI holds no broad cluster
|
||||
# access") while appearing to satisfy it.
|
||||
#
|
||||
# RBAC ALONE CANNOT FIX THIS. The real boundary is the admission policy
|
||||
# platform/kyverno/admission/conformance-trigger-job-shape.yaml, which admits Jobs from
|
||||
# THIS ServiceAccount only when they match the committed conformance shape. This Role is
|
||||
# the necessary half; that policy is the sufficient half. Neither works alone — do not
|
||||
# grant this Role without that policy reconciled and ENFORCING.
|
||||
#
|
||||
# Namespaced Role, bound only in `conformance`. No ClusterRole, deliberately.
|
||||
# =============================================================================
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: conformance-trigger
|
||||
namespace: conformance
|
||||
labels:
|
||||
app.kubernetes.io/part-of: conformance
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
name: conformance-trigger
|
||||
namespace: conformance
|
||||
labels:
|
||||
app.kubernetes.io/part-of: conformance
|
||||
rules:
|
||||
# Create the run, and watch it to completion.
|
||||
- apiGroups: [batch]
|
||||
resources: [jobs]
|
||||
verbs: [create, get, list, watch]
|
||||
# Read the Job's pod and its log — this is how results come back. Deliberately NOT
|
||||
# pods/exec and NOT `kubectl cp`: the run tars its artifacts to stdout and CI decodes
|
||||
# them from the log, so no execution channel into the runner pod is ever granted.
|
||||
- apiGroups: [""]
|
||||
resources: [pods, pods/log]
|
||||
verbs: [get, list, watch]
|
||||
# NOTHING ELSE. In particular NO delete on jobs (a trigger that can delete can hide a
|
||||
# failed run), no secrets, no configmaps, no exec, no create on pods.
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: conformance-trigger
|
||||
namespace: conformance
|
||||
labels:
|
||||
app.kubernetes.io/part-of: conformance
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: Role
|
||||
name: conformance-trigger
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: conformance-trigger
|
||||
namespace: conformance
|
||||
@@ -0,0 +1,42 @@
|
||||
# =============================================================================
|
||||
# L0 STATIC — every authored admission policy is actually APPLIED (R-SEC-01, R-SEC-02)
|
||||
# Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster.
|
||||
#
|
||||
# WHY THIS EXISTS. `kyverno test` proves a policy is CORRECT. Nothing proved it was DEPLOYED.
|
||||
# platform/kyverno/ holds three lists and only platform-config/kustomization.yaml applies
|
||||
# anything; the other two are a reference copy and a namespace+HelmRelease render root. On
|
||||
# 2026-09-29 the R-SEC-02 boundary policy was added to the reference copy: it parsed, its
|
||||
# kyverno test passed, the gate passed, Flux reported the tier Ready at the new revision — and
|
||||
# the cluster ran the conformance-trigger ServiceAccount with nothing bounding it.
|
||||
#
|
||||
# WHOLE-REPO SCOPE, DELIBERATELY. The failure mode is a policy that is present but wired
|
||||
# nowhere, which by definition touches no requirement the diff attributes. A path-scoped run
|
||||
# would have skipped this check on the very commit that introduced the defect.
|
||||
# =============================================================================
|
||||
apiVersion: chainsaw.kyverno.io/v1alpha1
|
||||
kind: Test
|
||||
metadata:
|
||||
name: admission-policies-actually-applied
|
||||
labels:
|
||||
conformance.home.k3s/R-SEC-01: "true"
|
||||
conformance.home.k3s/R-SEC-02: "true"
|
||||
annotations:
|
||||
conformance.home.k3s/requirements: "R-SEC-01,R-SEC-02"
|
||||
conformance.home.k3s/layer: "L0"
|
||||
conformance.home.k3s/scope: "whole-repo"
|
||||
conformance.home.k3s/safety-critical: "false"
|
||||
conformance.home.k3s/executable: "true"
|
||||
conformance.home.k3s/intent: >-
|
||||
Every ClusterPolicy authored under platform/kyverno/ is either referenced by
|
||||
platform-config/kustomization.yaml (so Flux applies it) or is a documented CI-only
|
||||
exemption that is genuinely not applied — so a policy cannot be authored, tested,
|
||||
green, and absent from the cluster.
|
||||
spec:
|
||||
steps:
|
||||
- name: every-authored-policy-is-applied-or-documented-ci-only
|
||||
description: >-
|
||||
Enumerate the ClusterPolicy files under platform/kyverno/ and assert each is wired
|
||||
into the Kustomization that actually applies Kyverno CRs, or exempted with a reason.
|
||||
try:
|
||||
- script:
|
||||
content: bash ./check.sh
|
||||
@@ -0,0 +1,120 @@
|
||||
#!/usr/bin/env bash
|
||||
# =============================================================================
|
||||
# L0 static check (offline, no cluster) — admission-policies-actually-applied
|
||||
# Serves: R-SEC-01, R-SEC-02 (admission enforcement is only real if it is APPLIED)
|
||||
#
|
||||
# WHY THIS EXISTS — a policy that ships green and never reaches the cluster.
|
||||
#
|
||||
# platform/kyverno/ holds THREE lists, and only one of them applies anything:
|
||||
# 1. platform/kyverno/admission/kustomization.yaml — reference only. Its own sibling says
|
||||
# "kept as reference; THIS file is the buildable render root".
|
||||
# 2. platform/kyverno/kustomization.yaml — that render root: namespace +
|
||||
# HelmRelease ONLY. It explicitly notes the ClusterPolicies are applied elsewhere.
|
||||
# 3. platform-config/kustomization.yaml — THE ONE. Kyverno CRs go here because
|
||||
# they need the CRDs from the HelmRelease in (2), so they reconcile a tier later.
|
||||
#
|
||||
# On 2026-09-29 a ClusterPolicy (conformance-trigger-job-shape, the R-SEC-02 boundary bounding
|
||||
# what the conformance CI trigger may create) was added to (1). Everything went green: the file
|
||||
# parsed, `kyverno test` exercised it BY PATH and passed, the gate passed, Flux reported
|
||||
# `platform` Ready at the new revision — and the policy was never in any applied set. The
|
||||
# cluster ended up running the conformance-trigger ServiceAccount with NOTHING bounding it,
|
||||
# which is precisely the state that identity's own RBAC header says must never exist.
|
||||
#
|
||||
# Nothing could have caught it. `kyverno test` proves a policy is CORRECT, not that it is
|
||||
# DEPLOYED. This check asserts the second thing.
|
||||
#
|
||||
# GIVEN the repository as git tracks it,
|
||||
# WHEN every file under platform/kyverno/ that declares a ClusterPolicy is enumerated,
|
||||
# THEN each one is either referenced by platform-config/kustomization.yaml (so Flux applies
|
||||
# it), or named in CI_ONLY below with a reason — and CI_ONLY itself may not name a
|
||||
# policy that IS applied, so the exemption list cannot rot into a lie either way.
|
||||
# =============================================================================
|
||||
set -euo pipefail
|
||||
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ROOT="$(cd "${DIR}/../../../.." && pwd)" # repo root
|
||||
exec python3 - "$ROOT" <<'PY'
|
||||
import sys, subprocess
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(sys.argv[1])
|
||||
FAILURES, CHECKS = [], 0
|
||||
|
||||
def check(cond, desc):
|
||||
global CHECKS
|
||||
CHECKS += 1
|
||||
if not cond:
|
||||
FAILURES.append(desc)
|
||||
|
||||
# Policies authored for the CI L0 gate that are deliberately NOT applied in-cluster. Each needs
|
||||
# a reason, and each is asserted BELOW to really be absent from the applied set — an exemption
|
||||
# that silently starts being applied is as much a lie as one that silently stops.
|
||||
CI_ONLY = {
|
||||
"disallow-plaintext-secrets":
|
||||
"CI-only by design: it requires a top-level `sops` block, which is IMPOSSIBLE at "
|
||||
"admission because Flux decrypts before the Secret reaches the API server. Its "
|
||||
"in-cluster counterpart is admission/secret-provenance.yaml.",
|
||||
}
|
||||
|
||||
tracked = subprocess.run(["git", "ls-files", "-z", "--", "platform/kyverno/**/*.yaml",
|
||||
"platform/kyverno/*.yaml"],
|
||||
cwd=ROOT, capture_output=True, text=True, check=True)
|
||||
files = [ROOT / p for p in tracked.stdout.split("\0") if p]
|
||||
|
||||
# A ClusterPolicy file, identified without a YAML parse of every doc: cheap and sufficient.
|
||||
policies = {}
|
||||
for f in files:
|
||||
try:
|
||||
txt = f.read_text()
|
||||
except Exception:
|
||||
continue
|
||||
if "kind: ClusterPolicy" not in txt:
|
||||
continue
|
||||
for line in txt.splitlines():
|
||||
st = line.strip()
|
||||
if st.startswith("name:") and " name:" in line:
|
||||
policies[f.relative_to(ROOT).as_posix()] = st.split("name:", 1)[1].strip()
|
||||
break
|
||||
|
||||
check(len(policies) >= 6,
|
||||
f"(0) the scan found {len(policies)} ClusterPolicy file(s) under platform/kyverno/ — "
|
||||
f"too few means the scan broke, and every assertion below would be vacuous")
|
||||
|
||||
applied_txt = (ROOT / "platform-config" / "kustomization.yaml").read_text()
|
||||
|
||||
unapplied, wrongly_exempt = [], []
|
||||
for rel, name in sorted(policies.items()):
|
||||
# platform-config references these as ../platform/kyverno/...
|
||||
referenced = rel in applied_txt or rel.replace("platform/", "../platform/", 1) in applied_txt
|
||||
if name in CI_ONLY:
|
||||
if referenced:
|
||||
wrongly_exempt.append(f"{name} ({rel}) is listed CI_ONLY but IS applied by platform-config")
|
||||
elif not referenced:
|
||||
unapplied.append(f"{name} ({rel})")
|
||||
|
||||
check(not unapplied,
|
||||
f"(1) every in-cluster ClusterPolicy is referenced by platform-config/kustomization.yaml "
|
||||
f"— an unreferenced policy is authored, tested, green, and NEVER APPLIED {unapplied if unapplied else ''}")
|
||||
|
||||
check(not wrongly_exempt,
|
||||
f"(2) no CI_ONLY exemption names a policy that IS applied — a stale exemption hides a "
|
||||
f"real deployment {wrongly_exempt if wrongly_exempt else ''}")
|
||||
|
||||
for name, reason in CI_ONLY.items():
|
||||
check(bool(reason.strip()),
|
||||
f"(3) CI_ONLY['{name}'] carries a reason — an unexplained exemption is how this "
|
||||
f"list stops being reviewable")
|
||||
|
||||
check(CHECKS >= 4,
|
||||
f"only {CHECKS} assertions ran — the check degenerated to a near-no-op")
|
||||
|
||||
if FAILURES:
|
||||
print(f"CHECK FAIL (admission-policies-actually-applied): {len(FAILURES)} of {CHECKS} assertions failed",
|
||||
file=sys.stderr)
|
||||
for f in FAILURES:
|
||||
print(f" - {f}", file=sys.stderr)
|
||||
raise SystemExit(1)
|
||||
print(f"check OK: admission-policies-actually-applied — {CHECKS} assertions; "
|
||||
f"{len(policies)} ClusterPolicy file(s), {len(policies) - len(CI_ONLY)} applied by "
|
||||
f"platform-config, {len(CI_ONLY)} CI-only with reasons. Proves policies are DEPLOYED, "
|
||||
f"not that they are correct — `kyverno test` is what proves correctness.")
|
||||
PY
|
||||
@@ -216,6 +216,20 @@ def is_literal_base(host: str) -> bool:
|
||||
DELIVERY_ROOTS = ("apps", "platform", "platform-config", "platform-config-cdi",
|
||||
"runbooks", "clusters", "bootstrap", "catalog")
|
||||
|
||||
# NARROW EXEMPTION to (0), for reconciled trees that deliberately serve NOTHING.
|
||||
# `conformance/incluster` (the L2/L3 runner + CI-trigger identities, R-SEC-02) is reconciled
|
||||
# by clusters/base/conformance.yaml but contains only a Namespace, ServiceAccounts, RBAC and
|
||||
# an admission-bounded Job template. Adding "conformance" to DELIVERY_ROOTS instead would be
|
||||
# wrong for the reason stated above: it would sweep in this harness's synthetic fixture
|
||||
# routes and make (8) demand the operator's dashboard link them.
|
||||
#
|
||||
# THE EXEMPTION POLICES ITSELF. An unchecked allowlist is how (0) would rot into a fail-open —
|
||||
# add a tree here, later put an HTTPRoute in it, and the forward direction goes quietly green
|
||||
# on a hidden service. So each exempted tree is ASSERTED below to declare no route and no
|
||||
# hostname-publishing Service. If that ever stops being true the check fails and the tree must
|
||||
# become a real delivery root.
|
||||
NON_SERVING_RECONCILED = ("conformance/incluster",)
|
||||
|
||||
|
||||
def tracked_yaml() -> list[Path]:
|
||||
out = subprocess.run(["git", "ls-files", "-z", "--", "*.yaml", "*.yml"],
|
||||
@@ -262,12 +276,31 @@ flux_ks = [(p, d) for p, d in docs_all()
|
||||
# a Kustomization declared elsewhere); they name no source and reconcile nothing, so requiring
|
||||
# a path of them would fail on a file that is correct.
|
||||
reconcilers = [(p, d) for p, d in flux_ks if (d.get("spec", {}) or {}).get("sourceRef")]
|
||||
def _reconciled_path(d):
|
||||
return (d.get("spec", {}) or {}).get("path", "").lstrip("./").rstrip("/")
|
||||
|
||||
|
||||
unscanned = sorted({
|
||||
f"{p.relative_to(ROOT)}:{(d.get('spec', {}) or {}).get('path', '') or '(no path)'}"
|
||||
for p, d in reconcilers
|
||||
if (d.get("spec", {}) or {}).get("path", "").lstrip("./").split("/")[0]
|
||||
not in DELIVERY_ROOTS
|
||||
if _reconciled_path(d).split("/")[0] not in DELIVERY_ROOTS
|
||||
and _reconciled_path(d) not in NON_SERVING_RECONCILED
|
||||
})
|
||||
|
||||
# The exemption's own guard: an exempted tree must actually serve nothing.
|
||||
exempt_serving = sorted({
|
||||
f"{p.relative_to(ROOT)} ({d.get('kind')})"
|
||||
for p, d in docs_all()
|
||||
if any(str(p.relative_to(ROOT)).startswith(t + "/") for t in NON_SERVING_RECONCILED)
|
||||
and (d.get("kind") == "HTTPRoute"
|
||||
or (d.get("kind") == "Service"
|
||||
and any("hostname" in str(k).lower()
|
||||
for k in ((d.get("metadata", {}) or {}).get("annotations", {}) or {}))))
|
||||
})
|
||||
check(not exempt_serving,
|
||||
f"(0b) every NON_SERVING_RECONCILED tree really serves nothing — an exempted tree that "
|
||||
f"gains a route would hide a service from this check entirely "
|
||||
f"{exempt_serving if exempt_serving else ''}")
|
||||
check(len(reconcilers) >= 10 and not unscanned,
|
||||
f"(0) all {len(reconcilers)} Flux-reconciled path(s) lie inside a scanned root — a "
|
||||
f"reconciled tree this check does not read is a service it cannot require a link for "
|
||||
|
||||
@@ -38,8 +38,21 @@ agents. So the credential model must be decided by the operator before anything
|
||||
| Runner deployment | `apps/wave-1/gitea-actions.yaml` | act_runner + DinD (Docker executor). **`automountServiceAccountToken: false`** — the runner and its job containers hold NO SA token. Broad egress allowed (`- {}`), so it can *reach* the kube-API VIP `192.168.1.49:6443`, but has no credential. |
|
||||
| Secrets encryption | `.sops.yaml` (repo root) | age recipient `age175l8m…mfzg`; encrypts `data`/`stringData` of `*.sops.yaml` under `platform|apps|catalog/**` and `secrets/**`. Private key is the in-cluster `sops-age` Secret Flux uses. |
|
||||
|
||||
**Cluster reality:** the "Phase 0 / no cluster" language in these files is stale — the dev cluster
|
||||
is LIVE (Phase 6). So `CLUSTER_AVAILABLE` is the only remaining gate, exactly as the blueprint says.
|
||||
**Cluster reality:** the "Phase 0 / no cluster" language in these files is stale — the cluster is
|
||||
LIVE (and it is now PROD; dev was decommissioned 2026-09-12).
|
||||
|
||||
> **CORRECTION (2026-09-29): `CLUSTER_AVAILABLE` is NOT "the only remaining gate", and setting it
|
||||
> today is actively DANGEROUS.** Measured: with `CLUSTER_AVAILABLE=true` and no reachable cluster,
|
||||
> `conformance/run.sh` exits 0 in scaffold mode having marked all 54 tests `<skipped>`, and
|
||||
> `ci/conformance.sh` then printed `[ ok ] L2/L3 conformance passed for touched subset` — **rc 0, a
|
||||
> fake green**, on a merge gate that is now an enforcing required status check. The CI runner has
|
||||
> no cluster credential at all (`automountServiceAccountToken: false`, by design), so the "local
|
||||
> run" this stage attempts can never succeed; it can only skip and, before the guard added on
|
||||
> 2026-09-29, lie about it.
|
||||
>
|
||||
> `ci/conformance.sh` now BLOCKS (exit 1) when a cluster is declared but every test skipped. **Do
|
||||
> not set `CLUSTER_AVAILABLE=true` until §3's trigger path actually exists** — the variable does
|
||||
> not enable L2/L3, it only changes which way this stage fails.
|
||||
|
||||
---
|
||||
|
||||
|
||||
+69
-3
@@ -102,7 +102,19 @@ Behaviour, proven by test rather than assumed:
|
||||
|---|---|---|
|
||||
| live matches the declaration | 0 | green |
|
||||
| any key drifted, or no runner online | 1 | **blocks the merge** |
|
||||
| no `GITEA_SETTINGS_TOKEN`, or Gitea unreachable | 3 | skips **loudly**; never a pass, never a block |
|
||||
| no `SETTINGS_API_TOKEN`, or Gitea unreachable | 3 | skips **loudly**; never a pass, never a block |
|
||||
|
||||
> **The secret is `SETTINGS_API_TOKEN`.** An earlier draft of this table named
|
||||
> `GITEA_SETTINGS_TOKEN`, which **cannot exist as a Gitea secret at all** — Gitea reserves the
|
||||
> `GITEA_` prefix and returns HTTP 400 `invalid secret name`. That is the whole 2026-09-29
|
||||
> inert-detector story, so the wrong name surviving here is worth correcting rather than
|
||||
> reading past. Measured in `ci/gitea-settings.sh`:
|
||||
>
|
||||
> TOKEN="${SETTINGS_API_TOKEN:-${GITEA_SETTINGS_TOKEN:-}}"
|
||||
>
|
||||
> so `GITEA_SETTINGS_TOKEN` is still honoured — but **only as an environment variable for a
|
||||
> LOCAL run**. It can never be the CI wiring, because the secret that would feed it cannot be
|
||||
> created. In CI, only `SETTINGS_API_TOKEN` exists.
|
||||
|
||||
The skip is deliberately *not* a block: an unwired credential blocking every PR is the
|
||||
lock-yourself-out failure this repo keeps hitting. Wire the token with `set-ci-token.sh`, confirm
|
||||
@@ -112,6 +124,59 @@ the stage goes **green on a real PR**, and only then set `REQUIRE_GITEA_SETTINGS
|
||||
**Changing the declaration is how you change policy.** If the live state is what you now want,
|
||||
edit this JSON in the same commit — deliberately, in review — rather than letting the two drift.
|
||||
|
||||
## `conformance-trigger-kubeconfig.sh` — the narrow L2/L3 trigger credential
|
||||
|
||||
Implements `docs/plans/l2-l3-conformance-ci-wiring.md` §3 item 6 (= §5 step 7): mint a
|
||||
ServiceAccount token for `conformance/conformance-trigger`, wrap **only that token** in a
|
||||
kubeconfig, store it as the Actions secret `CONFORMANCE_TRIGGER_KUBECONFIG`, and — separately,
|
||||
and only on request — set `vars.CLUSTER_AVAILABLE=true`.
|
||||
|
||||
./infra/gitea/conformance-trigger-kubeconfig.sh --dry-run # measure only; writes nothing
|
||||
./infra/gitea/conformance-trigger-kubeconfig.sh # mint + store the secret
|
||||
./infra/gitea/conformance-trigger-kubeconfig.sh --arm-cluster-available # + the variable
|
||||
|
||||
Never a cluster-admin kubeconfig, never a copy of the operator's own. The token is never
|
||||
printed: it goes from the TokenRequest API into a 0600 file, into the request **body file**
|
||||
(`--data @file`, so not even into `ps`), and nowhere else. The script refuses to run under
|
||||
`set -x`.
|
||||
|
||||
**Its preconditions are split in two, and the split is the point.**
|
||||
|
||||
| Class | Checks | Blocks |
|
||||
|---|---|---|
|
||||
| **mint-blocking** (cluster) | SA/Role/RoleBinding exist; ClusterPolicy `conformance-trigger-job-shape` is present, `Enforce`, `failurePolicy: Fail`, Ready | the **mint** — nothing is created |
|
||||
| **arm-blocking** (repo) | digest-pinned `CONFORMANCE_RUNNER_IMAGE`; `chainsaw-job.yaml` has an init-container and the admitted image; `ci/conformance.sh` names the trigger kubeconfig in **code, not a comment** | only `CLUSTER_AVAILABLE` |
|
||||
|
||||
Why mint-blocking: Kubernetes puts **no `use` check on a Job's `serviceAccountName`**, so a
|
||||
token that can `create jobs` in `conformance` can run arbitrary code as the privileged
|
||||
`conformance-runner` SA. RBAC cannot close that; the Kyverno policy is the boundary. Minting
|
||||
while it is absent or in `Audit` hands CI the privileged identity, so the script will not do it.
|
||||
|
||||
Why arm-blocking is separate: storing the secret before the runner image / Job template / CI
|
||||
trigger flow exist is harmless (the job is gated off), and is the plan's intended order. So
|
||||
running this today provisions the secret and then **refuses to arm the variable**, naming the
|
||||
missing item.
|
||||
|
||||
**`CLUSTER_AVAILABLE` is not a "turn conformance on" switch.** Measured 2026-09-29: set with no
|
||||
trigger path, `run.sh` scaffold-skipped all 54 tests at rc 0 and `ci/conformance.sh` printed
|
||||
"L2/L3 conformance passed" — a fake green on an enforcing required check. `ci/conformance.sh`
|
||||
now blocks that case, so arming early does not enable L2/L3; it only breaks every PR. Hence:
|
||||
`--arm-cluster-available` plus all preconditions green plus a typed confirmation on a TTY, or
|
||||
nothing happens.
|
||||
|
||||
Both writes are verified the way `set-ci-token.sh` learned to: check `%{http_code}` **and**
|
||||
read the object back. For the secret that proves presence by name only (Gitea never returns
|
||||
secret values); for the variable it proves the value is exactly `true`, which is what
|
||||
`gate.yml` compares against.
|
||||
|
||||
The token **expires** — it is a TokenRequest token, and the script decodes the JWT `exp` claim
|
||||
to report the real expiry rather than trusting `--duration` (the API server caps it). Re-run to
|
||||
rotate. After expiry the trigger gets 401 and the stage fails loudly rather than skipping.
|
||||
|
||||
`--server` exists because the embedded API-server URL must be reachable **from the CI runner
|
||||
pod** — the VIP — which is not necessarily the address your workstation uses. The script
|
||||
refuses a loopback server outright.
|
||||
|
||||
### Why these scripts exist at all, and what is still missing
|
||||
|
||||
Gitea's repo settings live in **its own database**, not in anything Flux reconciles or the
|
||||
@@ -124,8 +189,9 @@ These scripts encode the desired state and can re-assert it; `ci/gitea-settings.
|
||||
**detector** that turns divergence from a silent, invisible failure into a loud one. It closes the
|
||||
root cause of the 2026-09-29 session.
|
||||
|
||||
It is only as good as its wiring: **until `GITEA_SETTINGS_TOKEN` exists it skips**, and a skipped
|
||||
detector detects nothing. That is the remaining step.
|
||||
It is only as good as its wiring: **until `SETTINGS_API_TOKEN` exists it skips**, and a skipped
|
||||
detector detects nothing. (That secret was stored and verified in PR #15; this sentence predates
|
||||
it. The name matters — see the note under the behaviour table above.)
|
||||
|
||||
## Historical sequence (kept for context)
|
||||
|
||||
|
||||
+575
@@ -0,0 +1,575 @@
|
||||
#!/usr/bin/env bash
|
||||
# infra/gitea/conformance-trigger-kubeconfig.sh
|
||||
#
|
||||
# OPERATOR script for docs/plans/l2-l3-conformance-ci-wiring.md §3 item 6 (= §5 step 7):
|
||||
# provision the NARROW CI trigger credential for L2/L3 conformance, and — only when the
|
||||
# rest of the trigger path actually exists — arm `vars.CLUSTER_AVAILABLE`.
|
||||
#
|
||||
# What it does, in order:
|
||||
# 1. measures the preconditions (cluster-side AND repo-side) instead of assuming them
|
||||
# 2. mints a bounded-lifetime ServiceAccount token for `conformance-trigger` in ns
|
||||
# `conformance` (kubectl create token -> TokenRequest API)
|
||||
# 3. assembles a kubeconfig containing ONLY that token + the cluster CA + the API server
|
||||
# 4. PROVES the kubeconfig is the narrow identity (auth whoami + an allow/deny matrix)
|
||||
# 5. stores it as the Gitea Actions repo secret CONFORMANCE_TRIGGER_KUBECONFIG, and
|
||||
# VERIFIES the store by reading the secret list back
|
||||
# 6. sets vars.CLUSTER_AVAILABLE=true ONLY under --arm-cluster-available, only with every
|
||||
# precondition green, and only after a typed confirmation. Otherwise it REFUSES.
|
||||
#
|
||||
# NOTHING SECRET IS EVER PRINTED. The token exists in a shell variable and in a 0600 file
|
||||
# under a 0700 temp dir that is removed on exit; it reaches the Gitea API in a request-body
|
||||
# FILE (`--data @file`), never in argv, never on stdout. This script prints names, byte
|
||||
# lengths, HTTP status codes, the token's expiry timestamp, and yes/no authorization answers.
|
||||
#
|
||||
# ----------------------------------------------------------------------------------------
|
||||
# THREE HARD-WON LESSONS THIS SCRIPT IS BUILT AROUND
|
||||
# ----------------------------------------------------------------------------------------
|
||||
# 1. GITEA RESERVES THE `GITEA_` PREFIX FOR SECRET NAMES.
|
||||
# `PUT .../actions/secrets/GITEA_ANYTHING` returns HTTP 400 {"message":"invalid secret
|
||||
# name"}. That is why set-ci-token.sh stores SETTINGS_API_TOKEN and not
|
||||
# GITEA_SETTINGS_TOKEN. `CONFORMANCE_TRIGGER_KUBECONFIG` is legal, and is the exact name
|
||||
# .gitea/workflows/{gate.yml,merge-gate.yaml} already reference — do not "tidy" it.
|
||||
#
|
||||
# 2. `curl -sS` EXITS 0 ON AN HTTP ERROR. The first version of set-ci-token.sh sent its PUT
|
||||
# to /dev/null and printed "stored" unconditionally; the secret never existed and nobody
|
||||
# knew for hours. So every write here captures `-w '%{http_code}'`, checks it, AND reads
|
||||
# the object back afterwards. A store is not claimed unless the read-back shows it.
|
||||
#
|
||||
# 3. `CLUSTER_AVAILABLE` IS A LOADED GUN — SEE THE 2026-09-29 CORRECTION IN THE PLAN.
|
||||
# `conformance/run.sh` exits 0 in scaffold mode with all 54 tests `<skipped>` when it
|
||||
# cannot reach a cluster. Measured on 2026-09-29: with CLUSTER_AVAILABLE=true and no
|
||||
# trigger path, ci/conformance.sh printed "[ ok ] L2/L3 conformance passed" at rc 0 — a
|
||||
# FAKE GREEN on an enforcing required status check. ci/conformance.sh now BLOCKS that
|
||||
# case (exit 1), so today the variable does not enable L2/L3: it only changes which way
|
||||
# the stage fails. Arming it before §3 items 1-4 land therefore buys nothing and breaks
|
||||
# every PR. This script refuses to do it.
|
||||
#
|
||||
# ----------------------------------------------------------------------------------------
|
||||
# WHY THE PRECONDITIONS ARE SPLIT INTO TWO CLASSES (deliberate, and load-bearing)
|
||||
# ----------------------------------------------------------------------------------------
|
||||
# MINT-BLOCKING (cluster-side). Kubernetes places NO `use` check on a Job's
|
||||
# `serviceAccountName`, so a token with plain `create jobs` in ns `conformance` can create a
|
||||
# Job running as the PRIVILEGED `conformance-runner` SA with an arbitrary command. The
|
||||
# trigger Role is only the necessary half; the sufficient half is the Kyverno ClusterPolicy
|
||||
# `conformance-trigger-job-shape` (platform/kyverno/admission/), which must be present AND
|
||||
# Enforce AND Ready. Minting this token while that policy is absent or in Audit hands CI a
|
||||
# privileged identity. So those checks block the MINT itself — nothing is created if they
|
||||
# fail. (See conformance/incluster/rbac.yaml's trigger header and plan §4.)
|
||||
#
|
||||
# ARM-BLOCKING (repo-side). The runner image, the Job template's init-container, and
|
||||
# ci/conformance.sh's in-cluster trigger flow (plan §3 items 1, 2, 4) are what make a real
|
||||
# L2/L3 run possible. Storing the kubeconfig secret without them is harmless — the
|
||||
# conformance job is gated on the variable and never starts — and the plan explicitly lands
|
||||
# the secret before the switch. So these checks block ONLY the variable.
|
||||
#
|
||||
# Net effect: you can safely run this script today to provision the secret. It will then
|
||||
# REFUSE to arm the variable, and tell you exactly which repo-side item is missing.
|
||||
#
|
||||
# ----------------------------------------------------------------------------------------
|
||||
# USAGE
|
||||
# ----------------------------------------------------------------------------------------
|
||||
# ./infra/gitea/conformance-trigger-kubeconfig.sh --dry-run
|
||||
# Read-only. Measures every precondition, mints nothing, writes nothing.
|
||||
# ./infra/gitea/conformance-trigger-kubeconfig.sh [--server URL]
|
||||
# Mint + store the secret. Does NOT touch CLUSTER_AVAILABLE.
|
||||
# ./infra/gitea/conformance-trigger-kubeconfig.sh --arm-cluster-available
|
||||
# The above, then set vars.CLUSTER_AVAILABLE=true after a typed confirmation —
|
||||
# refused unless every precondition is green.
|
||||
#
|
||||
# Options:
|
||||
# --dry-run measure only; no TokenRequest, no PUT/POST
|
||||
# --arm-cluster-available additionally set vars.CLUSTER_AVAILABLE=true (guarded)
|
||||
# --server URL API server URL to embed (default: the operator context's own)
|
||||
# --context NAME kube context to read the CA/server from (default: k3s-prod)
|
||||
# --duration D requested token lifetime (default: 720h = 30d)
|
||||
# -h | --help this text
|
||||
#
|
||||
# ON --server: the embedded server URL must be reachable FROM THE CI RUNNER POD, which is
|
||||
# not necessarily the address your workstation uses. The runner has broad egress and reaches
|
||||
# the kube-API VIP (per the plan: 192.168.1.49:6443). If your operator kubeconfig points at
|
||||
# a workstation-only name or a localhost tunnel, pass --server explicitly; the script warns
|
||||
# on a loopback server but cannot know your topology for you.
|
||||
#
|
||||
# ON TOKEN LIFETIME: this is a TokenRequest token and it EXPIRES. The API server may cap the
|
||||
# requested duration, so the script decodes the JWT `exp` claim and reports the ACTUAL
|
||||
# expiry rather than trusting --duration. Re-run this script to rotate. After expiry the
|
||||
# trigger gets 401 and the conformance stage FAILS (loudly) rather than skipping — which is
|
||||
# the correct direction, but put the expiry date in your calendar.
|
||||
#
|
||||
# DELIBERATELY NOT DONE: SOPS-sealing the token into secrets/conformance-trigger.sops.yaml
|
||||
# (plan §3 item 6's "for reproducibility/rotation"). A bearer token with a 30-day life
|
||||
# cannot usefully survive the branch -> PR -> review -> merge flow (CLAUDE.md §6), and
|
||||
# committing one buys re-issuance we get for free by re-running this script against the
|
||||
# live SA. If the operator still wants it sealed, that is a separate reviewed change.
|
||||
#
|
||||
# FORBIDDEN HERE, ON PURPOSE: no cluster-admin kubeconfig, no copy of the operator's own
|
||||
# kubeconfig, no widening of conformance/incluster/rbac.yaml, no mutating kubectl beyond the
|
||||
# TokenRequest itself, no `kubectl apply`.
|
||||
set -euo pipefail
|
||||
|
||||
# xtrace would print the token. Refuse rather than leak.
|
||||
case "$-" in
|
||||
*x*) echo "FATAL: refusing to run under 'set -x' — xtrace would print the token" >&2; exit 1 ;;
|
||||
esac
|
||||
umask 077
|
||||
|
||||
HOST=gitea.home.keithjackson.dev
|
||||
OWNER=Home-Network
|
||||
REPO=home-network-k3s-impl
|
||||
# Legal secret name: NOT prefixed GITEA_ (lesson 1), and the name the workflows already use.
|
||||
SECRET=CONFORMANCE_TRIGGER_KUBECONFIG
|
||||
VARIABLE=CLUSTER_AVAILABLE
|
||||
NS=conformance
|
||||
SA=conformance-trigger
|
||||
RUNNER_SA=conformance-runner
|
||||
POLICY=conformance-trigger-job-shape
|
||||
# The image prefix the admission policy admits. Kept in sync with
|
||||
# platform/kyverno/admission/conformance-trigger-job-shape.yaml by the P2 check below.
|
||||
RUNNER_IMAGE_PREFIX='gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:'
|
||||
|
||||
KCTX=k3s-prod
|
||||
DURATION=720h
|
||||
DRY_RUN=false
|
||||
ARM=false
|
||||
SERVER_OVERRIDE=""
|
||||
|
||||
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
|
||||
step() { printf '\n==> %s\n' "$*"; }
|
||||
info() { printf ' %s\n' "$*"; }
|
||||
warn() { printf ' WARN: %s\n' "$*" >&2; }
|
||||
die() { printf 'FATAL: %s\n' "$*" >&2; exit 1; }
|
||||
|
||||
usage() {
|
||||
cat <<'USAGE'
|
||||
conformance-trigger-kubeconfig.sh — provision the narrow L2/L3 conformance CI trigger credential.
|
||||
|
||||
--dry-run measure every precondition; mint nothing, write nothing (read-only)
|
||||
--arm-cluster-available additionally set vars.CLUSTER_AVAILABLE=true (guarded + confirmed)
|
||||
--server URL API server URL to embed (default: the operator context's own;
|
||||
must be reachable FROM THE CI RUNNER POD — the kube-API VIP)
|
||||
--context NAME kube context to read the CA/server from (default: k3s-prod)
|
||||
--duration D requested token lifetime (default: 720h)
|
||||
-h | --help this text
|
||||
|
||||
Read the header of this file before running it: the CLUSTER_AVAILABLE guard, the split between
|
||||
mint-blocking and arm-blocking preconditions, and why nothing secret is ever printed.
|
||||
USAGE
|
||||
}
|
||||
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--dry-run) DRY_RUN=true ;;
|
||||
--arm-cluster-available) ARM=true ;;
|
||||
--server) SERVER_OVERRIDE="${2:?--server needs a URL}"; shift ;;
|
||||
--context) KCTX="${2:?--context needs a name}"; shift ;;
|
||||
--duration) DURATION="${2:?--duration needs a value}"; shift ;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
*) die "unknown argument: $1 (try --help)" ;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
if $DRY_RUN && $ARM; then die "--dry-run and --arm-cluster-available are contradictory"; fi
|
||||
|
||||
for t in kubectl jq curl git base64; do
|
||||
command -v "$t" >/dev/null || die "missing required tool: $t"
|
||||
done
|
||||
|
||||
WORK="$(mktemp -d)"
|
||||
KC="${WORK}/conformance-trigger.kubeconfig"
|
||||
cleanup() { rm -rf "$WORK"; }
|
||||
trap cleanup EXIT
|
||||
trap 'cleanup; exit 130' INT TERM
|
||||
|
||||
# ========================================================================================
|
||||
# 0. Gitea credentials — from the credential helper git already has for this host, exactly
|
||||
# as branch-protection.sh / set-ci-token.sh / require-gate.sh do. Never in argv: curl
|
||||
# reads them from a --config file descriptor.
|
||||
# ========================================================================================
|
||||
creds="$(printf 'protocol=https\nhost=%s\n\n' "$HOST" | git credential fill)"
|
||||
U="$(sed -n 's/^username=//p' <<<"$creds")"
|
||||
P="$(sed -n 's/^password=//p' <<<"$creds")"
|
||||
unset creds
|
||||
[ -n "$U" ] && [ -n "$P" ] || die "no stored credential for ${HOST} (git credential fill returned nothing)"
|
||||
basic() { curl -sS --config <(printf 'user = "%s:%s"\n' "$U" "$P") "$@"; }
|
||||
API="https://${HOST}/api/v1/repos/${OWNER}/${REPO}"
|
||||
|
||||
# ========================================================================================
|
||||
# 1. PRECONDITIONS — measured, never assumed (CLAUDE.md §2b: query the thing itself).
|
||||
# Two independent tallies, per the split documented in the header.
|
||||
# ========================================================================================
|
||||
MINT_FAIL=0
|
||||
ARM_FAIL=0
|
||||
pass() { printf ' [ ok ] %s\n' "$*"; }
|
||||
fail_mint() { printf ' [MINT-BLOCK] %s\n' "$*" >&2; MINT_FAIL=$((MINT_FAIL + 1)); }
|
||||
fail_arm() { printf ' [ARM-BLOCK] %s\n' "$*" >&2; ARM_FAIL=$((ARM_FAIL + 1)); }
|
||||
|
||||
step "preconditions — cluster side (these block the MINT: without them the token IS privileged)"
|
||||
|
||||
kubectl --context "$KCTX" get --raw /version >/dev/null 2>&1 \
|
||||
|| die "cannot reach the API server via context '${KCTX}' — nothing below can be measured"
|
||||
pass "API server reachable via context ${KCTX}"
|
||||
|
||||
# C1: the trigger identity exists (created by Flux from conformance/incluster/rbac.yaml via
|
||||
# clusters/base/conformance.yaml). We print object names only, never contents.
|
||||
if kubectl --context "$KCTX" -n "$NS" get serviceaccount "$SA" -o name >/dev/null 2>&1; then
|
||||
pass "ServiceAccount ${NS}/${SA} exists"
|
||||
else
|
||||
fail_mint "ServiceAccount ${NS}/${SA} is ABSENT — is the 'conformance' Flux Kustomization reconciled?"
|
||||
fi
|
||||
for kind in role rolebinding; do
|
||||
if kubectl --context "$KCTX" -n "$NS" get "$kind" "$SA" -o name >/dev/null 2>&1; then
|
||||
pass "${kind} ${NS}/${SA} exists"
|
||||
else
|
||||
fail_mint "${kind} ${NS}/${SA} is ABSENT — the token would authenticate but authorize nothing"
|
||||
fi
|
||||
done
|
||||
|
||||
# C2: THE SECURITY BOUNDARY. Present, Enforce, and Ready. Anything else and we do not mint.
|
||||
# `validationFailureAction: Audit` is not a boundary; an unready policy is not a boundary.
|
||||
if pol="$(kubectl --context "$KCTX" get clusterpolicy "$POLICY" -o json 2>/dev/null)"; then
|
||||
action="$(jq -r '.spec.validationFailureAction // .spec.rules[0].validate.failureAction // "UNSET"' <<<"$pol")"
|
||||
failpol="$(jq -r '.spec.failurePolicy // "UNSET"' <<<"$pol")"
|
||||
# Kyverno moved readiness from .status.ready to a Ready condition; accept either, and
|
||||
# treat "neither key present" as UNMEASURABLE -> block (fail closed, do not assume ready).
|
||||
ready="$(jq -r '
|
||||
if (.status.ready != null) then (.status.ready | tostring)
|
||||
elif (.status.conditions != null) then
|
||||
((.status.conditions[] | select(.type == "Ready") | .status) // "UNKNOWN")
|
||||
else "UNKNOWN" end' <<<"$pol")"
|
||||
info "ClusterPolicy ${POLICY}: validationFailureAction=${action} failurePolicy=${failpol} ready=${ready}"
|
||||
[ "$action" = "Enforce" ] || fail_mint "ClusterPolicy ${POLICY} is '${action}', not Enforce — the trigger would be unbounded"
|
||||
[ "$failpol" = "Fail" ] || fail_mint "ClusterPolicy ${POLICY} failurePolicy='${failpol}', not Fail — a Kyverno outage would admit arbitrary Jobs"
|
||||
case "$ready" in
|
||||
true|True) pass "ClusterPolicy ${POLICY} reports Ready" ;;
|
||||
*) fail_mint "ClusterPolicy ${POLICY} does not report Ready (got '${ready}') — refusing to assume it is loaded" ;;
|
||||
esac
|
||||
else
|
||||
fail_mint "ClusterPolicy ${POLICY} is ABSENT — plan §4's boundary does not exist; a token with 'create jobs' here can run arbitrary code as ${RUNNER_SA}"
|
||||
fi
|
||||
|
||||
step "preconditions — repo side (these block ONLY arming ${VARIABLE}; the secret may be stored first)"
|
||||
|
||||
# R1: the runner image is built and digest-pinned. Without it the Job cannot run run.sh at
|
||||
# all (the stock chainsaw image has no bash/python3/kubectl/git) — plan §3 item 1.
|
||||
VERSIONS="${REPO_ROOT}/conformance/versions.env"
|
||||
if grep -Eq '^[[:space:]]*CONFORMANCE_RUNNER_IMAGE=.*@sha256:[0-9a-f]{64}' "$VERSIONS" 2>/dev/null; then
|
||||
pass "CONFORMANCE_RUNNER_IMAGE is digest-pinned in conformance/versions.env"
|
||||
else
|
||||
fail_arm "conformance/versions.env has no digest-pinned CONFORMANCE_RUNNER_IMAGE (plan §3 item 1) — the Job cannot run ./run.sh"
|
||||
fi
|
||||
|
||||
# R2: the Job template is the real thing, not the skeleton: the admitted image AND an
|
||||
# init-container that populates the empty `workspace` volume with the suite at the PR SHA
|
||||
# (plan §3 item 2). A template whose image the policy will not admit is a guaranteed denial.
|
||||
JOBTPL="${REPO_ROOT}/conformance/incluster/chainsaw-job.yaml"
|
||||
if grep -q 'initContainers' "$JOBTPL" 2>/dev/null; then
|
||||
pass "chainsaw-job.yaml declares an initContainers block (workspace population)"
|
||||
else
|
||||
fail_arm "chainsaw-job.yaml has no initContainers — 'workspace' is an empty emptyDir and the suite is never delivered (plan §3 item 2)"
|
||||
fi
|
||||
if grep -q "$RUNNER_IMAGE_PREFIX" "$JOBTPL" 2>/dev/null; then
|
||||
pass "chainsaw-job.yaml uses the digest-pinned image the admission policy admits"
|
||||
else
|
||||
fail_arm "chainsaw-job.yaml does not use ${RUNNER_IMAGE_PREFIX}… — every triggered Job would be DENIED by ${POLICY}"
|
||||
fi
|
||||
|
||||
# R3: ci/conformance.sh actually drives the in-cluster Job. Today it mentions
|
||||
# CONFORMANCE_TRIGGER_KUBECONFIG only in a TODO comment and otherwise runs run.sh LOCALLY in
|
||||
# the CI job — where there is no credential at all (the runner sets
|
||||
# automountServiceAccountToken: false by design), so it can only scaffold-skip. Hence the
|
||||
# check must ignore comment lines: a mention in prose is not an implementation.
|
||||
#
|
||||
# ONE awk, NOT `grep -v … | grep -q …`: under `set -o pipefail` a successful `grep -q` exits
|
||||
# early, the upstream grep takes EPIPE, and the pipeline reports non-zero — which would read
|
||||
# here as "unwired" exactly when the seam IS wired. A single process cannot have that bug.
|
||||
CONF_STAGE="${REPO_ROOT}/ci/conformance.sh"
|
||||
if awk '!/^[[:space:]]*#/ && /CONFORMANCE_TRIGGER_KUBECONFIG/ { found = 1 } END { exit !found }' \
|
||||
"$CONF_STAGE" 2>/dev/null; then
|
||||
pass "ci/conformance.sh references CONFORMANCE_TRIGGER_KUBECONFIG in code (in-cluster trigger path wired)"
|
||||
else
|
||||
fail_arm "ci/conformance.sh mentions CONFORMANCE_TRIGGER_KUBECONFIG only in comments — the trigger path is UNIMPLEMENTED (plan §3 item 4); arming ${VARIABLE} now only changes which way the stage fails"
|
||||
fi
|
||||
|
||||
info ""
|
||||
info "precondition tally: mint-blocking failures=${MINT_FAIL} arm-blocking failures=${ARM_FAIL}"
|
||||
|
||||
if [ "$MINT_FAIL" -gt 0 ]; then
|
||||
cat >&2 <<EOF
|
||||
|
||||
REFUSING TO MINT. ${MINT_FAIL} cluster-side precondition(s) failed.
|
||||
|
||||
Minting a token for ${NS}/${SA} while the admission boundary is absent/Audit/unready hands CI
|
||||
an identity that can create a Job running as ${RUNNER_SA} (nodes/patch, pods/exec,
|
||||
namespaces/patch) with an arbitrary command. That is precisely the escalation
|
||||
docs/plans/l2-l3-conformance-ci-wiring.md §4 exists to close, and RBAC cannot close it.
|
||||
|
||||
Fix the cluster side first (reconcile platform/kyverno/admission/ and conformance/incluster/,
|
||||
confirm the ClusterPolicy is Enforce + Ready), then re-run. Nothing was created.
|
||||
EOF
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if $ARM && [ "$ARM_FAIL" -gt 0 ]; then
|
||||
cat >&2 <<EOF
|
||||
|
||||
REFUSING TO ARM ${VARIABLE}. ${ARM_FAIL} repo-side precondition(s) failed.
|
||||
|
||||
Measured 2026-09-29: with ${VARIABLE}=true and no working trigger path, conformance/run.sh
|
||||
scaffold-skips all 54 tests at rc 0 and ci/conformance.sh used to print
|
||||
"[ ok ] L2/L3 conformance passed" — a fake green on an enforcing required status check. The
|
||||
guard now in ci/conformance.sh turns that into exit 1 instead, so arming today does not
|
||||
enable L2/L3: it breaks every PR while proving nothing.
|
||||
|
||||
Land plan §3 items 1, 2 and 4 first. Re-run WITHOUT --arm-cluster-available to provision the
|
||||
secret in the meantime — that part is safe and is step 6's intended order.
|
||||
EOF
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if $DRY_RUN; then
|
||||
step "DRY RUN — stopping here. Nothing was minted, nothing was written."
|
||||
info "secret that WOULD be written: ${SECRET} (repo ${OWNER}/${REPO})"
|
||||
info "variable that WOULD be set: none — ${VARIABLE} is never touched in --dry-run"
|
||||
# Read-only visibility into what is already there. Gitea never returns secret VALUES.
|
||||
names="$(basic "${API}/actions/secrets" | jq -rc '[.. | objects | select(has("name")) | .name] | unique')" || true
|
||||
info "existing Actions secret names: ${names:-<unreadable>}"
|
||||
cur="$(basic -o "${WORK}/var.json" -w '%{http_code}' "${API}/actions/variables/${VARIABLE}")" || true
|
||||
if [ "$cur" = 200 ]; then
|
||||
info "existing ${VARIABLE} = $(jq -r '.data // "<no data field>"' "${WORK}/var.json")"
|
||||
else
|
||||
info "existing ${VARIABLE}: not set (HTTP ${cur})"
|
||||
fi
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# ========================================================================================
|
||||
# 2. MINT the ServiceAccount token (TokenRequest). This is the only write this script makes
|
||||
# against the cluster, and it creates no durable object.
|
||||
# ========================================================================================
|
||||
step "minting a ServiceAccount token for ${NS}/${SA} (requested duration ${DURATION})"
|
||||
TOKEN="$(kubectl --context "$KCTX" -n "$NS" create token "$SA" --duration="$DURATION")" \
|
||||
|| die "kubectl create token failed for ${NS}/${SA}"
|
||||
[ -n "$TOKEN" ] || die "kubectl create token returned an empty token"
|
||||
info "got a ${#TOKEN}-character token (value NOT shown)"
|
||||
|
||||
# Report the ACTUAL expiry from the JWT rather than trusting --duration: the API server caps
|
||||
# TokenRequest lifetimes (--service-account-max-token-expiration) and silently shortens.
|
||||
jwt_exp() {
|
||||
local payload pad b64
|
||||
payload="$(cut -d. -f2 <<<"$1")"
|
||||
pad=$(( (4 - ${#payload} % 4) % 4 ))
|
||||
b64="$(tr '_-' '/+' <<<"$payload")$(printf '%*s' "$pad" '' | tr ' ' '=')"
|
||||
printf '%s' "$b64" | base64 -d 2>/dev/null | jq -r '.exp // empty' 2>/dev/null || true
|
||||
}
|
||||
exp="$(jwt_exp "$TOKEN")"
|
||||
if [ -n "$exp" ]; then
|
||||
info "token expires: $(date -u -d "@${exp}" '+%Y-%m-%dT%H:%M:%SZ' 2>/dev/null || echo "epoch ${exp}") — re-run this script to rotate"
|
||||
else
|
||||
warn "could not decode the token's exp claim; lifetime is UNVERIFIED (requested ${DURATION})"
|
||||
fi
|
||||
|
||||
# ========================================================================================
|
||||
# 3. ASSEMBLE the kubeconfig — SA token + cluster CA + server. Nothing else.
|
||||
#
|
||||
# The CA and the server URL are read from the operator's context, but the operator's
|
||||
# client certificate/key are NEVER copied: only .cluster fields are extracted. `config
|
||||
# view --raw` output is held in a variable and discarded, never printed.
|
||||
# Built with jq (not `kubectl config set-credentials --token=…`) so the token never
|
||||
# appears in argv where `ps` can see it.
|
||||
# ========================================================================================
|
||||
step "assembling the kubeconfig (SA token only — never cluster-admin, never the operator's)"
|
||||
raw="$(kubectl --context "$KCTX" config view --raw --minify -o json)" \
|
||||
|| die "could not read context '${KCTX}' from the operator kubeconfig"
|
||||
SERVER="$(jq -r '.clusters[0].cluster.server // empty' <<<"$raw")"
|
||||
CADATA="$(jq -r '.clusters[0].cluster["certificate-authority-data"] // empty' <<<"$raw")"
|
||||
CAFILE="$(jq -r '.clusters[0].cluster["certificate-authority"] // empty' <<<"$raw")"
|
||||
INSECURE="$(jq -r '.clusters[0].cluster["insecure-skip-tls-verify"] // false | tostring' <<<"$raw")"
|
||||
unset raw
|
||||
|
||||
[ -n "$SERVER" ] || die "context '${KCTX}' has no cluster.server"
|
||||
if [ -n "$SERVER_OVERRIDE" ]; then
|
||||
info "overriding server ${SERVER} -> ${SERVER_OVERRIDE}"
|
||||
SERVER="$SERVER_OVERRIDE"
|
||||
fi
|
||||
case "$SERVER" in
|
||||
*localhost*|*127.0.0.1*|*://0.0.0.0*)
|
||||
die "server '${SERVER}' is loopback — the CI runner pod cannot reach it. Pass --server with the address the runner uses (the kube-API VIP)." ;;
|
||||
esac
|
||||
if [ -z "$CADATA" ] && [ -n "$CAFILE" ]; then
|
||||
[ -r "$CAFILE" ] || die "certificate-authority file '${CAFILE}' is not readable"
|
||||
CADATA="$(base64 -w0 <"$CAFILE")"
|
||||
fi
|
||||
[ -n "$CADATA" ] || die "no cluster CA available for '${KCTX}' (insecure-skip-tls-verify=${INSECURE}); refusing to emit a kubeconfig that trusts anything"
|
||||
[ "$INSECURE" = "false" ] || die "context '${KCTX}' sets insecure-skip-tls-verify; refusing to propagate that into CI"
|
||||
info "server: ${SERVER}"
|
||||
info "CA: ${#CADATA} base64 characters embedded (public certificate, not a secret)"
|
||||
|
||||
jq -n --arg srv "$SERVER" --arg ca "$CADATA" --arg tok "$TOKEN" --arg ns "$NS" --arg sa "$SA" '
|
||||
{
|
||||
apiVersion: "v1",
|
||||
kind: "Config",
|
||||
clusters: [ { name: "conformance", cluster: { server: $srv, "certificate-authority-data": $ca } } ],
|
||||
users: [ { name: $sa, user: { token: $tok } } ],
|
||||
contexts: [ { name: "conformance", context: { cluster: "conformance", user: $sa, namespace: $ns } } ],
|
||||
"current-context": "conformance"
|
||||
}' >"$KC"
|
||||
chmod 600 "$KC"
|
||||
info "kubeconfig written to the temp dir: $(wc -c <"$KC") bytes (content NOT shown)"
|
||||
|
||||
# Prove kubectl can PARSE the file we generated (JSON is valid YAML, but assert it rather
|
||||
# than assume it) — this prints only the server we already printed.
|
||||
parsed="$(kubectl --kubeconfig "$KC" config view -o jsonpath='{.clusters[0].cluster.server}')" \
|
||||
|| die "kubectl cannot parse the generated kubeconfig"
|
||||
[ "$parsed" = "$SERVER" ] || die "generated kubeconfig parses to server '${parsed}', expected '${SERVER}'"
|
||||
pass "kubectl parses the generated kubeconfig"
|
||||
|
||||
# ========================================================================================
|
||||
# 4. PROVE it is the narrow identity. Two independent proofs:
|
||||
# a) who the token authenticates AS (SelfSubjectReview)
|
||||
# b) an allow/deny matrix (SelfSubjectAccessReview) — both halves matter: "can create
|
||||
# jobs" proves it works, "cannot get secrets / delete jobs / exec" proves it is not
|
||||
# something wider that happens to include the trigger rights.
|
||||
# All read-only. `auth can-i` answers from the API server's own authorizer.
|
||||
# ========================================================================================
|
||||
step "verifying the credential is the NARROW trigger identity"
|
||||
EXPECT_USER="system:serviceaccount:${NS}:${SA}"
|
||||
if who="$(kubectl --kubeconfig "$KC" auth whoami -o jsonpath='{.status.userInfo.username}' 2>/dev/null)"; then
|
||||
[ "$who" = "$EXPECT_USER" ] || die "token authenticates as '${who}', expected '${EXPECT_USER}' — WRONG CREDENTIAL, nothing stored"
|
||||
pass "authenticates as ${who}"
|
||||
else
|
||||
warn "'kubectl auth whoami' unavailable (older kubectl / SelfSubjectReview denied) — identity UNVERIFIED by this check; the matrix below still constrains it"
|
||||
fi
|
||||
|
||||
# A real authorization round-trip, not just a review: `list jobs` is granted by the Role.
|
||||
kubectl --kubeconfig "$KC" -n "$NS" get jobs -o name >/dev/null \
|
||||
|| die "the minted credential cannot list Jobs in ns ${NS} — the RoleBinding is not effective; nothing stored"
|
||||
pass "can list Jobs in ns ${NS} (end-to-end authentication + authorization)"
|
||||
|
||||
cani() { # cani <expected yes|no> <verb> <resource...>
|
||||
local want="$1"; shift
|
||||
local got
|
||||
got="$(kubectl --kubeconfig "$KC" auth can-i "$@" 2>/dev/null || true)"
|
||||
case "$got" in
|
||||
yes|no) : ;;
|
||||
*) die "auth can-i '$*' returned '${got:-<nothing>}' — cannot measure the identity's rights; refusing to store an unverified credential" ;;
|
||||
esac
|
||||
if [ "$got" = "$want" ]; then
|
||||
pass "can-i $* -> ${got} (expected)"
|
||||
else
|
||||
die "can-i $* -> ${got}, expected ${want}. This credential is NOT the narrow trigger identity described in conformance/incluster/rbac.yaml. Nothing stored. Do NOT widen that Role to make this pass — investigate what it is actually bound to."
|
||||
fi
|
||||
}
|
||||
# Granted by the trigger Role (create/get/list/watch jobs; get/list/watch pods + pods/log).
|
||||
cani yes create jobs.batch -n "$NS"
|
||||
cani yes get pods -n "$NS"
|
||||
cani yes get pods/log -n "$NS"
|
||||
# Explicitly NOT granted — each is a specific escape route the Role's comments call out.
|
||||
cani no delete jobs.batch -n "$NS" # a trigger that can delete can hide a failed run
|
||||
cani no create pods/exec -n "$NS" # no execution channel into the runner pod
|
||||
cani no get secrets -n "$NS"
|
||||
cani no create jobs.batch -n default # namespaced Role: no reach outside ns conformance
|
||||
cani no get nodes # the RUNNER has nodes/patch; the trigger must not
|
||||
cani no '*' '*' --all-namespaces # not cluster-admin, stated as a measurement
|
||||
|
||||
# ========================================================================================
|
||||
# 5. STORE as the Gitea Actions repo secret, then VERIFY BY READING IT BACK (lesson 2).
|
||||
# The body goes in a 0600 file so the token is not in argv.
|
||||
# ========================================================================================
|
||||
step "storing the kubeconfig as repo Actions secret ${SECRET}"
|
||||
jq -n --rawfile d "$KC" '{data:$d}' >"${WORK}/body.json"
|
||||
chmod 600 "${WORK}/body.json"
|
||||
code="$(basic -o "${WORK}/put.out" -w '%{http_code}' -X PUT \
|
||||
-H 'Content-Type: application/json' --data @"${WORK}/body.json" \
|
||||
"${API}/actions/secrets/${SECRET}")"
|
||||
unset TOKEN
|
||||
rm -f "${WORK}/body.json" "$KC"
|
||||
if [ "$code" != 201 ] && [ "$code" != 204 ]; then
|
||||
# Safe to echo: the response body is Gitea's error message, not our payload. HTTP 400
|
||||
# "invalid secret name" here means the name hit the reserved GITEA_ prefix (lesson 1).
|
||||
die "PUT ${SECRET} failed: HTTP ${code} -- $(head -c 300 "${WORK}/put.out" 2>/dev/null)"
|
||||
fi
|
||||
info "PUT returned HTTP ${code}"
|
||||
|
||||
# curl -sS exits 0 on an HTTP error, so the status check above is necessary but NOT
|
||||
# sufficient: read the object back. Gitea never returns secret VALUES, so this proves
|
||||
# PRESENCE BY NAME only — say so rather than implying the content was checked.
|
||||
names="$(basic "${API}/actions/secrets" | jq -rc '[.. | objects | select(has("name")) | .name] | unique')"
|
||||
grep -q "$SECRET" <<<"$names" \
|
||||
|| die "PUT reported HTTP ${code} but ${SECRET} is NOT in ${names} — refusing to claim success"
|
||||
pass "stored and VERIFIED PRESENT by read-back: ${names}"
|
||||
info "(read-back proves the secret EXISTS; Gitea never returns secret values, so its CONTENT"
|
||||
info " is proven only by the first CI run that uses it — see the NOTE below.)"
|
||||
info ""
|
||||
info "FOR WHOEVER WRITES plan §3 item 4: this secret holds the kubeconfig CONTENT, not a path."
|
||||
info "The plan's sketch writes 'kubectl --kubeconfig=\$CONFORMANCE_TRIGGER_KUBECONFIG', which"
|
||||
info "would treat it as a filename. ci/conformance.sh must write the env var into a 0600 temp"
|
||||
info "file (umask 077, trap-removed) and pass THAT path — an Actions secret is a string, so a"
|
||||
info "kubeconfig cannot reach CI any other way."
|
||||
|
||||
# ========================================================================================
|
||||
# 6. vars.CLUSTER_AVAILABLE — the loaded gun. Only here, only on request, only confirmed.
|
||||
# ========================================================================================
|
||||
if ! $ARM; then
|
||||
cat <<EOF
|
||||
|
||||
==> ${VARIABLE} NOT TOUCHED (no --arm-cluster-available). This is the safe default.
|
||||
|
||||
The secret is in place; the conformance job stays gated off until the variable is set. That
|
||||
is plan §5's order: land items 1-6, prove them, and only then flip step 7's switch.
|
||||
EOF
|
||||
else
|
||||
step "arming vars.${VARIABLE}=true — every precondition above is green"
|
||||
cat <<EOF
|
||||
This makes .gitea/workflows/gate.yml's conformance job RUN on every PR. If the
|
||||
in-cluster trigger path misbehaves, ci/conformance.sh will BLOCK (exit 1) rather than
|
||||
fake a green — so the failure mode is loud PRs, not silent false confidence. Undo with:
|
||||
curl -X DELETE .../repos/${OWNER}/${REPO}/actions/variables/${VARIABLE}
|
||||
EOF
|
||||
[ -t 0 ] || die "refusing to arm ${VARIABLE} non-interactively — this needs a human confirmation"
|
||||
printf ' Type exactly "ARM %s" to proceed: ' "$VARIABLE"
|
||||
read -r confirm
|
||||
[ "$confirm" = "ARM ${VARIABLE}" ] || die "not confirmed (got '${confirm}') — ${VARIABLE} unchanged. The secret IS stored."
|
||||
|
||||
# Gitea's variables API: POST creates, PUT updates, and the body field is `value` — NOT
|
||||
# `data` as in the secrets API above. Try create, fall back to update. The value "true" is
|
||||
# not a secret, so response bodies are printed here to make a failure diagnosable.
|
||||
vcode="$(basic -o "${WORK}/var.out" -w '%{http_code}' -X POST \
|
||||
-H 'Content-Type: application/json' --data '{"value":"true"}' \
|
||||
"${API}/actions/variables/${VARIABLE}")"
|
||||
if [ "$vcode" != 201 ] && [ "$vcode" != 204 ]; then
|
||||
info "POST returned HTTP ${vcode} ($(head -c 200 "${WORK}/var.out" 2>/dev/null)); trying PUT (variable may already exist)"
|
||||
vcode="$(basic -o "${WORK}/var.out" -w '%{http_code}' -X PUT \
|
||||
-H 'Content-Type: application/json' \
|
||||
--data "$(jq -n --arg n "$VARIABLE" '{name:$n,value:"true"}')" \
|
||||
"${API}/actions/variables/${VARIABLE}")"
|
||||
fi
|
||||
[ "$vcode" = 201 ] || [ "$vcode" = 204 ] \
|
||||
|| die "could not set ${VARIABLE}: HTTP ${vcode} -- $(head -c 300 "${WORK}/var.out" 2>/dev/null)"
|
||||
info "write returned HTTP ${vcode}"
|
||||
|
||||
# Unlike a secret, a variable's value IS readable — so verify the VALUE, not just presence.
|
||||
# gate.yml compares `vars.CLUSTER_AVAILABLE == 'true'`, so it must be exactly "true".
|
||||
gcode="$(basic -o "${WORK}/varget.json" -w '%{http_code}' "${API}/actions/variables/${VARIABLE}")"
|
||||
[ "$gcode" = 200 ] || die "stored HTTP ${vcode} but GET ${VARIABLE} returned HTTP ${gcode} — refusing to claim success"
|
||||
val="$(jq -r '.data // .value // empty' "${WORK}/varget.json")"
|
||||
[ "$val" = "true" ] || die "${VARIABLE} reads back as '${val}', not 'true' — gate.yml compares against 'true' exactly"
|
||||
pass "${VARIABLE}=${val} stored and VERIFIED by read-back"
|
||||
fi
|
||||
|
||||
cat <<EOF
|
||||
|
||||
NEXT — and none of it is proven until you watch it happen:
|
||||
1. Open a THROWAWAY PR. Read the conformance job's LOG, not its tick: a skipped Gitea job
|
||||
reports success ("Has been skipped"), and a green tick is not evidence anything ran.
|
||||
2. Confirm a Job appeared in ns ${NS}, ran the touched --requirements subset, and that its
|
||||
artifacts came back. Then break an L3 test deliberately and confirm the gate BLOCKS.
|
||||
3. Only after seeing BOTH directions should the conformance check be added to branch
|
||||
protection (infra/gitea/require-gate.sh's status_check_contexts).
|
||||
4. Diary the token expiry printed above. Re-run this script to rotate; CI fails loudly
|
||||
(401 on the trigger) rather than skipping when it lapses.
|
||||
|
||||
ROLLBACK: DELETE the ${VARIABLE} variable (instant SKIP, gate reverts to L0/L1-only) and/or
|
||||
DELETE the ${SECRET} secret. The token itself is revoked by deleting the ${NS}/${SA}
|
||||
ServiceAccount, which Flux will recreate with a fresh identity.
|
||||
EOF
|
||||
@@ -10,6 +10,10 @@
|
||||
# confirm the check goes green rather than assuming the scope name.
|
||||
set -euo pipefail
|
||||
HOST=gitea.home.keithjackson.dev; OWNER=Home-Network; REPO=home-network-k3s-impl
|
||||
# NOT GITEA_* — Gitea reserves that prefix for secret names and returns HTTP 400
|
||||
# "invalid secret name". The first version of this script used GITEA_SETTINGS_TOKEN, sent the
|
||||
# PUT to /dev/null, and printed "stored" regardless; the secret never existed and nobody knew.
|
||||
SECRET=SETTINGS_API_TOKEN
|
||||
NAME="ci-gitea-settings-$(date +%Y%m%d%H%M%S)"
|
||||
creds="$(printf 'protocol=https\nhost=%s\n\n' "$HOST" | git credential fill)"
|
||||
U="$(sed -n 's/^username=//p' <<<"$creds")"; P="$(sed -n 's/^password=//p' <<<"$creds")"
|
||||
@@ -22,12 +26,24 @@ TOKEN="$(basic -X POST -H 'Content-Type: application/json' \
|
||||
[ -n "$TOKEN" ] || { echo "token creation failed (check scopes for your Gitea version)" >&2; exit 1; }
|
||||
echo "==> got a ${#TOKEN}-character token (value not shown)"
|
||||
|
||||
echo "==> storing it as repo Actions secret GITEA_SETTINGS_TOKEN"
|
||||
basic -X PUT -H 'Content-Type: application/json' \
|
||||
echo "==> storing it as repo Actions secret ${SECRET}"
|
||||
code="$(basic -o /tmp/set-ci-token.out -w '%{http_code}' -X PUT -H 'Content-Type: application/json' \
|
||||
-d "$(jq -n --arg d "$TOKEN" '{data:$d}')" \
|
||||
"https://${HOST}/api/v1/repos/${OWNER}/${REPO}/actions/secrets/GITEA_SETTINGS_TOKEN" >/dev/null
|
||||
"https://${HOST}/api/v1/repos/${OWNER}/${REPO}/actions/secrets/${SECRET}")"
|
||||
unset TOKEN
|
||||
echo "==> stored."
|
||||
if [ "$code" != 201 ] && [ "$code" != 204 ]; then
|
||||
echo " PUT failed: HTTP ${code} -- $(head -c 200 /tmp/set-ci-token.out)" >&2
|
||||
rm -f /tmp/set-ci-token.out; exit 1
|
||||
fi
|
||||
rm -f /tmp/set-ci-token.out
|
||||
|
||||
# VERIFY BY READING IT BACK. curl -sS exits 0 on an HTTP error, so without this the script
|
||||
# reports success for a store that never happened -- which is exactly what it did.
|
||||
names="$(basic "https://${HOST}/api/v1/repos/${OWNER}/${REPO}/actions/secrets" \
|
||||
| jq -rc '[.. | objects | select(has("name")) | .name] | unique')"
|
||||
grep -q "$SECRET" <<<"$names" \
|
||||
|| { echo " stored HTTP ${code} but ${SECRET} is NOT in ${names} -- refusing to claim success" >&2; exit 1; }
|
||||
echo "==> stored and VERIFIED present: ${names}"
|
||||
|
||||
cat <<'NOTE'
|
||||
|
||||
|
||||
@@ -34,6 +34,14 @@ resources:
|
||||
# policies, the next import is blocked in the gap.
|
||||
- ../platform/kyverno/policies/label-cdi-worker-pods.yaml
|
||||
- ../platform/kyverno/admission/secret-provenance.yaml
|
||||
# R-SEC-02 — bounds what the conformance CI trigger may create. MUST be listed HERE: this
|
||||
# Kustomization is what actually applies ClusterPolicies. Adding a policy only to
|
||||
# platform/kyverno/admission/kustomization.yaml is a SILENT NO-OP — that file is reference
|
||||
# only ("kept as reference; THIS file is the buildable render root" in
|
||||
# platform/kyverno/kustomization.yaml, which itself ships only the namespace + HelmRelease).
|
||||
# That is exactly how this policy shipped green and never reached the cluster on 2026-09-29,
|
||||
# leaving the conformance-trigger identity live with nothing bounding it.
|
||||
- ../platform/kyverno/admission/conformance-trigger-job-shape.yaml
|
||||
# cilium: L2 LB IP pool + announcement policy (cilium.io CRDs already registered by Cilium)
|
||||
- cilium/l2-ip-pool.yaml
|
||||
- cilium/l2-announcement-policy.yaml
|
||||
|
||||
@@ -0,0 +1,144 @@
|
||||
# Serves: R-SEC-02 (CI holds no broad cluster access)
|
||||
#
|
||||
# THIS POLICY IS THE SECURITY BOUNDARY. The RBAC in conformance/incluster/rbac.yaml is not.
|
||||
#
|
||||
# Kubernetes places NO `use` check on a Pod/Job's `serviceAccountName`. Any principal that can
|
||||
# `create jobs` in namespace `conformance` can therefore create a Job that runs as
|
||||
# `conformance-runner` — a privileged SA (nodes/patch, pods/exec, namespaces/patch) — with an
|
||||
# arbitrary command. So the CI trigger credential, however narrow its own Role looks, IS the
|
||||
# privileged identity unless something constrains the SHAPE of what it may create.
|
||||
#
|
||||
# That is what this does: for Jobs created by `conformance-trigger` ONLY, admit exactly the
|
||||
# committed conformance run and deny everything else. The operator chose this model
|
||||
# (docs/plans/l2-l3-conformance-ci-wiring.md §4, primary model) over the zero-credential
|
||||
# GitOps-native alternative.
|
||||
#
|
||||
# ENFORCE AND FAIL-CLOSED, NOT SUBSTITUTED. Note this does NOT use
|
||||
# ${KYVERNO_VALIDATION_ACTION} like the other admission policies: a security boundary that an
|
||||
# environment overlay can downgrade to Audit is not a boundary. `failurePolicy: Fail` means a
|
||||
# Kyverno outage blocks conformance Jobs rather than silently admitting unconstrained ones.
|
||||
# Blocking CI is the correct failure direction here; admitting arbitrary privileged code is not.
|
||||
#
|
||||
# SCOPE: it constrains ONLY `conformance-trigger`. Jobs created by humans, by Flux, or by any
|
||||
# other SA are untouched — so a broken policy cannot wedge the rest of the cluster.
|
||||
#
|
||||
# IMAGE PINNING — HONEST LIMIT, READ BEFORE RELYING ON THIS. The plan specifies pinning the
|
||||
# exact `CONFORMANCE_RUNNER_IMAGE` digest. That image does not exist yet, so this enforces the
|
||||
# enforceable half: the image must live under the conformance repository path AND be
|
||||
# digest-pinned (`@sha256:`), never a mutable tag. Residual surface: anyone who can PUSH to that
|
||||
# registry path could publish a different digest. TIGHTEN THIS when the image is built by
|
||||
# replacing the `image` pattern below with the literal digest — a one-line change, reviewed in a
|
||||
# PR, which is exactly where a change to the security boundary belongs.
|
||||
apiVersion: kyverno.io/v1
|
||||
kind: ClusterPolicy
|
||||
metadata:
|
||||
name: conformance-trigger-job-shape
|
||||
annotations:
|
||||
policies.kyverno.io/title: Constrain what the conformance CI trigger may create
|
||||
policies.kyverno.io/category: Supply chain
|
||||
policies.kyverno.io/severity: critical
|
||||
homelab.keithjackson.dev/requirements: R-SEC-02
|
||||
homelab.keithjackson.dev/scope: admission-only
|
||||
spec:
|
||||
validationFailureAction: Enforce
|
||||
failurePolicy: Fail
|
||||
background: false
|
||||
rules:
|
||||
- name: trigger-may-create-only-the-committed-conformance-run
|
||||
# resources AND subjects in the SAME filter entry — that is how Kyverno ANDs them.
|
||||
# As two separate `all` entries it is a schema error ("match resources not specified")
|
||||
# and the whole policy loads as INVALID, which the CLI reports as a SKIP.
|
||||
match:
|
||||
all:
|
||||
- resources:
|
||||
kinds:
|
||||
- Job
|
||||
namespaces:
|
||||
- conformance
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: conformance-trigger
|
||||
namespace: conformance
|
||||
validate:
|
||||
message: >-
|
||||
conformance-trigger may create ONLY the committed conformance run: a single container,
|
||||
digest-pinned conformance-runner image, serviceAccountName conformance-runner,
|
||||
command ["./run.sh"], no host namespaces, non-root with privilege escalation disabled,
|
||||
and only the `workspace` and `results` volumes. See
|
||||
platform/kyverno/admission/conformance-trigger-job-shape.yaml.
|
||||
pattern:
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: conformance-runner
|
||||
=(hostNetwork): "false"
|
||||
=(hostPID): "false"
|
||||
=(hostIPC): "false"
|
||||
containers:
|
||||
- image: "gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:*"
|
||||
command:
|
||||
- "./run.sh"
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
=(privileged): "false"
|
||||
|
||||
- name: trigger-volumes-must-be-workspace-or-results
|
||||
# resources AND subjects in the SAME filter entry — that is how Kyverno ANDs them.
|
||||
# As two separate `all` entries it is a schema error ("match resources not specified")
|
||||
# and the whole policy loads as INVALID, which the CLI reports as a SKIP.
|
||||
match:
|
||||
all:
|
||||
- resources:
|
||||
kinds:
|
||||
- Job
|
||||
namespaces:
|
||||
- conformance
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: conformance-trigger
|
||||
namespace: conformance
|
||||
validate:
|
||||
message: >-
|
||||
conformance-trigger Jobs may mount only the `workspace` and `results` volumes. An extra
|
||||
volume is how a constrained pod reaches a host path, a secret, or another SA's token.
|
||||
# foreach, NOT a `pattern` list: a Kyverno array pattern asserts "at least one element
|
||||
# matches", which would happily admit a Job carrying `workspace` PLUS a hostPath. This
|
||||
# asserts EVERY volume is one of the two allowed names.
|
||||
foreach:
|
||||
- list: "request.object.spec.template.spec.volumes"
|
||||
deny:
|
||||
conditions:
|
||||
all:
|
||||
- key: "{{ element.name }}"
|
||||
operator: AnyNotIn
|
||||
value: ["workspace", "results"]
|
||||
|
||||
- name: trigger-args-must-be-the-committed-flags
|
||||
# resources AND subjects in the SAME filter entry — that is how Kyverno ANDs them.
|
||||
# As two separate `all` entries it is a schema error ("match resources not specified")
|
||||
# and the whole policy loads as INVALID, which the CLI reports as a SKIP.
|
||||
match:
|
||||
all:
|
||||
- resources:
|
||||
kinds:
|
||||
- Job
|
||||
namespaces:
|
||||
- conformance
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: conformance-trigger
|
||||
namespace: conformance
|
||||
validate:
|
||||
message: >-
|
||||
conformance-trigger Job args may contain only the conformance run's own flags
|
||||
(--layer, --requirements, --include-scope, --strict-coverage) and their plain values.
|
||||
Anything else — a shell, a path, a redirect — is denied.
|
||||
foreach:
|
||||
- list: "request.object.spec.template.spec.containers[0].args"
|
||||
deny:
|
||||
conditions:
|
||||
all:
|
||||
- key: "{{ regex_match('^(--layer|--requirements|--include-scope|--strict-coverage|[A-Za-z0-9][A-Za-z0-9,._-]*)$', element) }}"
|
||||
operator: Equals
|
||||
value: false
|
||||
@@ -20,3 +20,7 @@ resources:
|
||||
# --- secrets: the admission variant REPLACES the CI-only sops-block policy ---
|
||||
# (../policies/disallow-plaintext-secrets.yaml is CI-only; see secret-provenance.yaml)
|
||||
- secret-provenance.yaml
|
||||
# --- R-SEC-02: constrain what the conformance CI trigger may create ---
|
||||
# THE security boundary for CI cluster access. Enforce + fail-closed, deliberately NOT
|
||||
# subject to ${KYVERNO_VALIDATION_ACTION}. See the file header.
|
||||
- conformance-trigger-job-shape.yaml
|
||||
@@ -0,0 +1,57 @@
|
||||
# R-SEC-02 — the conformance CI trigger cannot escalate to arbitrary privileged execution.
|
||||
#
|
||||
# Each DENY case below is a real escalation path, not a style violation. The one that matters
|
||||
# most is `evil-command`: correct SA, correct image, arbitrary command. That is the attack the
|
||||
# whole model exists to stop, and RBAC alone cannot — Kubernetes places no `use` check on
|
||||
# serviceAccountName, so `create jobs` in this namespace IS the privileged identity without
|
||||
# this policy.
|
||||
#
|
||||
# user-info.yaml supplies the requesting identity. WITHOUT it the policy's `subjects` match
|
||||
# never fires and every case trivially passes — a test that proves nothing.
|
||||
apiVersion: cli.kyverno.io/v1alpha1
|
||||
kind: Test
|
||||
metadata:
|
||||
name: conformance-trigger-job-shape
|
||||
policies:
|
||||
- ../../admission/conformance-trigger-job-shape.yaml
|
||||
resources:
|
||||
- resources.yaml
|
||||
userinfo: user-info.yaml
|
||||
results:
|
||||
# ---- the committed run is ADMITTED ----
|
||||
- policy: conformance-trigger-job-shape
|
||||
rule: trigger-may-create-only-the-committed-conformance-run
|
||||
kind: Job
|
||||
resources: [good-run]
|
||||
result: pass
|
||||
- policy: conformance-trigger-job-shape
|
||||
rule: trigger-volumes-must-be-workspace-or-results
|
||||
kind: Job
|
||||
resources: [good-run]
|
||||
result: pass
|
||||
- policy: conformance-trigger-job-shape
|
||||
rule: trigger-args-must-be-the-committed-flags
|
||||
kind: Job
|
||||
resources: [good-run]
|
||||
result: pass
|
||||
|
||||
# ---- THE escalation: runner SA + arbitrary command ----
|
||||
- policy: conformance-trigger-job-shape
|
||||
rule: trigger-may-create-only-the-committed-conformance-run
|
||||
kind: Job
|
||||
resources: [evil-command, evil-image, tag-not-digest, host-namespaces]
|
||||
result: fail
|
||||
|
||||
# ---- an extra volume is the way out of a constrained pod ----
|
||||
- policy: conformance-trigger-job-shape
|
||||
rule: trigger-volumes-must-be-workspace-or-results
|
||||
kind: Job
|
||||
resources: [extra-volume]
|
||||
result: fail
|
||||
|
||||
# ---- args beyond the committed flags ----
|
||||
- policy: conformance-trigger-job-shape
|
||||
rule: trigger-args-must-be-the-committed-flags
|
||||
kind: Job
|
||||
resources: [injected-args]
|
||||
result: fail
|
||||
@@ -0,0 +1,154 @@
|
||||
# The committed shape — must be ADMITTED.
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: good-run
|
||||
namespace: conformance
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: conformance-runner
|
||||
containers:
|
||||
- name: runner
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||
command: ["./run.sh"]
|
||||
args: ["--layer", "l2,l3", "--requirements", "R-NET-01"]
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
volumes:
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
- name: results
|
||||
emptyDir: {}
|
||||
---
|
||||
# THE ESCALATION: runner SA, but an arbitrary command. Must be DENIED.
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: evil-command
|
||||
namespace: conformance
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: conformance-runner
|
||||
containers:
|
||||
- name: runner
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||
command: ["/bin/sh", "-c", "kubectl get secrets -A"]
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
volumes:
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
---
|
||||
# An arbitrary image running as the privileged SA. Must be DENIED.
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: evil-image
|
||||
namespace: conformance
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: conformance-runner
|
||||
containers:
|
||||
- name: runner
|
||||
image: docker.io/library/alpine:latest
|
||||
command: ["./run.sh"]
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
volumes:
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
---
|
||||
# A mutable TAG instead of a digest — republishable under the same reference. Must be DENIED.
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: tag-not-digest
|
||||
namespace: conformance
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: conformance-runner
|
||||
containers:
|
||||
- name: runner
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner:latest
|
||||
command: ["./run.sh"]
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
volumes:
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
---
|
||||
# Host namespaces + privilege escalation. Must be DENIED.
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: host-namespaces
|
||||
namespace: conformance
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: conformance-runner
|
||||
hostPID: true
|
||||
containers:
|
||||
- name: runner
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||
command: ["./run.sh"]
|
||||
securityContext:
|
||||
runAsNonRoot: false
|
||||
allowPrivilegeEscalation: true
|
||||
volumes:
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
---
|
||||
# An extra volume — the way out of a constrained pod. Must be DENIED.
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: extra-volume
|
||||
namespace: conformance
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: conformance-runner
|
||||
containers:
|
||||
- name: runner
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||
command: ["./run.sh"]
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
volumes:
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
- name: hostroot
|
||||
hostPath:
|
||||
path: /
|
||||
---
|
||||
# Injected args beyond the committed flags. Must be DENIED.
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: injected-args
|
||||
namespace: conformance
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: conformance-runner
|
||||
containers:
|
||||
- name: runner
|
||||
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||
command: ["./run.sh"]
|
||||
args: ["--layer", "l2", ";", "curl", "http://evil/"]
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
volumes:
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
@@ -0,0 +1,12 @@
|
||||
# The requesting identity the policy keys on. Without this the `subjects` match never fires and
|
||||
# every case below would trivially "pass" — proving nothing.
|
||||
apiVersion: cli.kyverno.io/v1alpha1
|
||||
kind: UserInfo
|
||||
metadata:
|
||||
name: conformance-trigger
|
||||
userInfo:
|
||||
username: system:serviceaccount:conformance:conformance-trigger
|
||||
groups:
|
||||
- system:serviceaccounts
|
||||
- system:serviceaccounts:conformance
|
||||
- system:authenticated
|
||||
Reference in new issue
Block a user