diff --git a/decisions/2026-09-29-conformance-trigger-kubeconfig.md b/decisions/2026-09-29-conformance-trigger-kubeconfig.md new file mode 100644 index 0000000..45a9671 --- /dev/null +++ b/decisions/2026-09-29-conformance-trigger-kubeconfig.md @@ -0,0 +1,125 @@ +--- +date: 2026-09-29 +branch: feat/conformance-trigger-kubeconfig +title: The trigger credential script — it REFUSES, on purpose, and it is UNEXECUTED +--- + +## What changed + +Delivered `infra/gitea/conformance-trigger-kubeconfig.sh` — +`docs/plans/l2-l3-conformance-ci-wiring.md` §3 item 6 — plus its section in +`infra/gitea/README.md`. **The script is UNEXECUTED.** Minting a cluster credential and +changing Gitea variables are operator actions; nothing was minted, no Gitea object was +written, no `kubectl` mutation was made. Everything below is a claim about the *script*, +not about the platform. + +*What it does.* Mints a token for `conformance/conformance-trigger` (TokenRequest), wraps +**only that token** with the cluster CA and server into a kubeconfig, stores it as the +Actions secret `CONFORMANCE_TRIGGER_KUBECONFIG` (legal name — **not** the reserved `GITEA_` +prefix), and sets `vars.CLUSTER_AVAILABLE` only as a separately-requested, +precondition-gated, TTY-confirmed final step. The token is never printed and never reaches +argv: it goes into a 0600 file and into the PUT **body file**. The script refuses to run +under `set -x`. + +*The design decision worth reviewing: TWO precondition classes.* + +- **mint-blocking (cluster-side):** SA/Role/RoleBinding present, and ClusterPolicy + `conformance-trigger-job-shape` present + `Enforce` + `failurePolicy: Fail` + Ready. If any + fails, **nothing is minted**. Rationale: Kubernetes puts no `use` check on a Job's + `serviceAccountName`, so a `create jobs` token in that namespace IS the privileged + `conformance-runner` identity unless the admission policy bounds it. RBAC cannot fix that + (plan §4); minting before the boundary is live would hand CI the escalation the boundary + exists to close. +- **arm-blocking (repo-side):** digest-pinned `CONFORMANCE_RUNNER_IMAGE`, an init-container + plus the admitted image in `chainsaw-job.yaml`, and `CONFORMANCE_TRIGGER_KUBECONFIG` + referenced in `ci/conformance.sh` **code rather than a comment**. These block only the + variable, because storing the secret early is harmless and is the plan's own order. + +**Measured in the tree at the time of writing: all three arm-blocking checks FAIL** (no +`CONFORMANCE_RUNNER_IMAGE` in `conformance/versions.env`; `chainsaw-job.yaml` is still the +skeleton with an empty `workspace` emptyDir and the stock chainsaw image; `ci/conformance.sh` +names the trigger kubeconfig only in its TODO comment). So on a live run today the script +would provision the secret and then **refuse to arm `CLUSTER_AVAILABLE`**, naming each +missing item — which is the intended behaviour, not a defect. §3 items 1, 2 and 4 are the +work that unblocks it. + +*Both writes are verified the way `set-ci-token.sh` learned to.* `curl -sS` exits 0 on an +HTTP error, so status alone proves nothing: each write checks `%{http_code}` **and** reads +the object back. For the secret that is presence-by-name only (Gitea never returns secret +values) and the script says so rather than implying the content was checked; for the variable +it asserts the value reads back as exactly `true`. + +*Noted, not fixed at the time (since fixed on this branch).* `infra/gitea/README.md`'s +detector table still named `GITEA_SETTINGS_TOKEN` — the illegal name PR #15 replaced with +`SETTINGS_API_TOKEN`. The second commit on this branch corrects the README's table and its +closing "remaining step" line, and records what `ci/gitea-settings.sh` actually does +(`TOKEN="${SETTINGS_API_TOKEN:-${GITEA_SETTINGS_TOKEN:-}}"` — the old name survives only as a +local-run environment variable, never as CI wiring, because the secret that would feed it +cannot be created at all). Documentation only, no behaviour change. Still NOT fixed, because +it is outside this PR's diff: `ci/gitea-settings.sh:45` instructs the reader to create a repo +secret `GITEA_SETTINGS_TOKEN`, which is impossible. + +*Rework, 2026-09-29.* This branch originally appended ~99 lines to the tail of +`DECISIONS-LOG.md`. That is the hunk every concurrent branch collided on and the sole reason +PR #19 was `mergeable: false`. The branch was rebuilt on today's `main` with the +`DECISIONS-LOG.md` change dropped entirely, and this file is that handoff moved into +`decisions/`. The changes to `infra/gitea/**` are byte-identical to the original three +commits; the commit that contained only the log append was dropped and its substance folded +in below. + +## What I verified + +*Of the script, before it stores anything:* it PROVES the credential narrow — `auth whoami` +must return `system:serviceaccount:conformance:conformance-trigger`; a real `get jobs` +round-trip must succeed; then an `auth can-i` matrix asserts BOTH halves — granted +(`create/get jobs`, `get pods`, `get pods/log`) and **denied** (`delete jobs`, `create +pods/exec`, `get secrets`, `create jobs` in `default`, `get nodes`, +`'*' '*' --all-namespaces`). A mismatch aborts before anything is stored, and the message +says explicitly **not** to widen `conformance/incluster/rbac.yaml` to make it pass. That is a +property of the code as read, not of an execution. + +*Of PR #19's CI, measured rather than assumed — and it contradicts the brief.* The brief for +this work stated this PR's CI would FAIL `approval-policy` because `infra/gitea/**` is +security-relevant. Measured on the PR head commit's `/statuses`: + + gate / gate (pull_request) success Successful in 1m34s + gate / conformance (pull_request) pending Blocked by required conditions + +`.gitea/workflows/` on `main` contains only `gate.yml` and the dormant `merge-gate.yaml`. +**No path-scoped approval policy exists**, so nothing machine-enforces "security-relevant +paths need a human"; that rule rests entirely on `required_approvals: 1` plus a reviewer +actually reading the diff. Another instance of the CLAUDE.md §2b failure mode — a control +believed to exist because a brief said so. If the policy is intended, it is missing and needs +its own change. And per this repo's own rule, that green `gate` tick is **not** evidence this +script works: the gate never executes `infra/gitea/**`, and the L2/L3 stage skipped. + +*Gate on the reworked branch (pinned toolbox, from the agent worktree, `DEV_KUBECONFIG=/dev/null` +plus the `ENGINE` bind-mount shim per CLAUDE.md §3):* `MERGE GATE: UNDETERMINED`, **rc 3**. +Diff base `origin/main @ c59fe9f923c4` ... this branch's head; touched requirement resolved as +`R-ENV-01`. `spec-immutability`, `decisions-log`, `lint`, `l0`, `render-golden` and `l1` all +green; the gate decision names exactly one stage that did not run — `gitea-settings` +(`SETTINGS_API_TOKEN` is a credential this agent does not hold, unwired repo-wide). The +`conformance` stage also reported SKIPPED (`CLUSTER_AVAILABLE!=true`, Phase 0). The new +`decisions-log` stage — merged to `main` as PR #24 after this rework began — validated this very +file: "archive frozen, 2 entries well-formed", with `DECISIONS-LOG.md` unchanged and matching +`decisions/ARCHIVE.sha256`. rc 3 is UNDETERMINED, not green, and is not reported as green here. + +## What I did NOT verify + +**The script has never executed.** Not one line of it has run against Gitea or against the +cluster. Only `bash -n` was run (clean); `shellcheck` is not on the host and there is no +shellcheck stage in the gate. + +- The Gitea variables API shape (POST-creates / PUT-updates, body field `value` not `data`) + is from **documentation, not measurement**. The script tries POST then falls back to PUT, + prints the response body on failure, and treats the read-back as the authority. +- The Kyverno readiness field (`.status.ready` vs. a Ready condition) is handled both ways + and treats "neither present" as a block rather than assuming ready — but which form this + cluster actually reports is **unmeasured**. +- Whether the kubeconfig's embedded server URL is reachable from the runner pod **cannot be + known from here** — hence `--server`, and a hard refusal on a loopback address. +- No credential was minted, no Actions secret was created, `vars.CLUSTER_AVAILABLE` was not + set, and no `can-i` matrix was ever run against a real token. The narrowness argument above + is read from the RBAC and the script, not observed. +- The gate proves only its L0/L1 offline subset. L2/L3 conformance did not run, so nothing + here is evidence about runtime behaviour. diff --git a/infra/gitea/README.md b/infra/gitea/README.md index 0b8c2da..caf48ad 100644 --- a/infra/gitea/README.md +++ b/infra/gitea/README.md @@ -102,7 +102,19 @@ Behaviour, proven by test rather than assumed: |---|---|---| | live matches the declaration | 0 | green | | any key drifted, or no runner online | 1 | **blocks the merge** | -| no `GITEA_SETTINGS_TOKEN`, or Gitea unreachable | 3 | skips **loudly**; never a pass, never a block | +| no `SETTINGS_API_TOKEN`, or Gitea unreachable | 3 | skips **loudly**; never a pass, never a block | + +> **The secret is `SETTINGS_API_TOKEN`.** An earlier draft of this table named +> `GITEA_SETTINGS_TOKEN`, which **cannot exist as a Gitea secret at all** — Gitea reserves the +> `GITEA_` prefix and returns HTTP 400 `invalid secret name`. That is the whole 2026-09-29 +> inert-detector story, so the wrong name surviving here is worth correcting rather than +> reading past. Measured in `ci/gitea-settings.sh`: +> +> TOKEN="${SETTINGS_API_TOKEN:-${GITEA_SETTINGS_TOKEN:-}}" +> +> so `GITEA_SETTINGS_TOKEN` is still honoured — but **only as an environment variable for a +> LOCAL run**. It can never be the CI wiring, because the secret that would feed it cannot be +> created. In CI, only `SETTINGS_API_TOKEN` exists. The skip is deliberately *not* a block: an unwired credential blocking every PR is the lock-yourself-out failure this repo keeps hitting. Wire the token with `set-ci-token.sh`, confirm @@ -112,6 +124,59 @@ the stage goes **green on a real PR**, and only then set `REQUIRE_GITEA_SETTINGS **Changing the declaration is how you change policy.** If the live state is what you now want, edit this JSON in the same commit — deliberately, in review — rather than letting the two drift. +## `conformance-trigger-kubeconfig.sh` — the narrow L2/L3 trigger credential + +Implements `docs/plans/l2-l3-conformance-ci-wiring.md` §3 item 6 (= §5 step 7): mint a +ServiceAccount token for `conformance/conformance-trigger`, wrap **only that token** in a +kubeconfig, store it as the Actions secret `CONFORMANCE_TRIGGER_KUBECONFIG`, and — separately, +and only on request — set `vars.CLUSTER_AVAILABLE=true`. + + ./infra/gitea/conformance-trigger-kubeconfig.sh --dry-run # measure only; writes nothing + ./infra/gitea/conformance-trigger-kubeconfig.sh # mint + store the secret + ./infra/gitea/conformance-trigger-kubeconfig.sh --arm-cluster-available # + the variable + +Never a cluster-admin kubeconfig, never a copy of the operator's own. The token is never +printed: it goes from the TokenRequest API into a 0600 file, into the request **body file** +(`--data @file`, so not even into `ps`), and nowhere else. The script refuses to run under +`set -x`. + +**Its preconditions are split in two, and the split is the point.** + +| Class | Checks | Blocks | +|---|---|---| +| **mint-blocking** (cluster) | SA/Role/RoleBinding exist; ClusterPolicy `conformance-trigger-job-shape` is present, `Enforce`, `failurePolicy: Fail`, Ready | the **mint** — nothing is created | +| **arm-blocking** (repo) | digest-pinned `CONFORMANCE_RUNNER_IMAGE`; `chainsaw-job.yaml` has an init-container and the admitted image; `ci/conformance.sh` names the trigger kubeconfig in **code, not a comment** | only `CLUSTER_AVAILABLE` | + +Why mint-blocking: Kubernetes puts **no `use` check on a Job's `serviceAccountName`**, so a +token that can `create jobs` in `conformance` can run arbitrary code as the privileged +`conformance-runner` SA. RBAC cannot close that; the Kyverno policy is the boundary. Minting +while it is absent or in `Audit` hands CI the privileged identity, so the script will not do it. + +Why arm-blocking is separate: storing the secret before the runner image / Job template / CI +trigger flow exist is harmless (the job is gated off), and is the plan's intended order. So +running this today provisions the secret and then **refuses to arm the variable**, naming the +missing item. + +**`CLUSTER_AVAILABLE` is not a "turn conformance on" switch.** Measured 2026-09-29: set with no +trigger path, `run.sh` scaffold-skipped all 54 tests at rc 0 and `ci/conformance.sh` printed +"L2/L3 conformance passed" — a fake green on an enforcing required check. `ci/conformance.sh` +now blocks that case, so arming early does not enable L2/L3; it only breaks every PR. Hence: +`--arm-cluster-available` plus all preconditions green plus a typed confirmation on a TTY, or +nothing happens. + +Both writes are verified the way `set-ci-token.sh` learned to: check `%{http_code}` **and** +read the object back. For the secret that proves presence by name only (Gitea never returns +secret values); for the variable it proves the value is exactly `true`, which is what +`gate.yml` compares against. + +The token **expires** — it is a TokenRequest token, and the script decodes the JWT `exp` claim +to report the real expiry rather than trusting `--duration` (the API server caps it). Re-run to +rotate. After expiry the trigger gets 401 and the stage fails loudly rather than skipping. + +`--server` exists because the embedded API-server URL must be reachable **from the CI runner +pod** — the VIP — which is not necessarily the address your workstation uses. The script +refuses a loopback server outright. + ### Why these scripts exist at all, and what is still missing Gitea's repo settings live in **its own database**, not in anything Flux reconciles or the @@ -124,8 +189,9 @@ These scripts encode the desired state and can re-assert it; `ci/gitea-settings. **detector** that turns divergence from a silent, invisible failure into a loud one. It closes the root cause of the 2026-09-29 session. -It is only as good as its wiring: **until `GITEA_SETTINGS_TOKEN` exists it skips**, and a skipped -detector detects nothing. That is the remaining step. +It is only as good as its wiring: **until `SETTINGS_API_TOKEN` exists it skips**, and a skipped +detector detects nothing. (That secret was stored and verified in PR #15; this sentence predates +it. The name matters — see the note under the behaviour table above.) ## Historical sequence (kept for context) diff --git a/infra/gitea/conformance-trigger-kubeconfig.sh b/infra/gitea/conformance-trigger-kubeconfig.sh new file mode 100755 index 0000000..4b7e30d --- /dev/null +++ b/infra/gitea/conformance-trigger-kubeconfig.sh @@ -0,0 +1,575 @@ +#!/usr/bin/env bash +# infra/gitea/conformance-trigger-kubeconfig.sh +# +# OPERATOR script for docs/plans/l2-l3-conformance-ci-wiring.md §3 item 6 (= §5 step 7): +# provision the NARROW CI trigger credential for L2/L3 conformance, and — only when the +# rest of the trigger path actually exists — arm `vars.CLUSTER_AVAILABLE`. +# +# What it does, in order: +# 1. measures the preconditions (cluster-side AND repo-side) instead of assuming them +# 2. mints a bounded-lifetime ServiceAccount token for `conformance-trigger` in ns +# `conformance` (kubectl create token -> TokenRequest API) +# 3. assembles a kubeconfig containing ONLY that token + the cluster CA + the API server +# 4. PROVES the kubeconfig is the narrow identity (auth whoami + an allow/deny matrix) +# 5. stores it as the Gitea Actions repo secret CONFORMANCE_TRIGGER_KUBECONFIG, and +# VERIFIES the store by reading the secret list back +# 6. sets vars.CLUSTER_AVAILABLE=true ONLY under --arm-cluster-available, only with every +# precondition green, and only after a typed confirmation. Otherwise it REFUSES. +# +# NOTHING SECRET IS EVER PRINTED. The token exists in a shell variable and in a 0600 file +# under a 0700 temp dir that is removed on exit; it reaches the Gitea API in a request-body +# FILE (`--data @file`), never in argv, never on stdout. This script prints names, byte +# lengths, HTTP status codes, the token's expiry timestamp, and yes/no authorization answers. +# +# ---------------------------------------------------------------------------------------- +# THREE HARD-WON LESSONS THIS SCRIPT IS BUILT AROUND +# ---------------------------------------------------------------------------------------- +# 1. GITEA RESERVES THE `GITEA_` PREFIX FOR SECRET NAMES. +# `PUT .../actions/secrets/GITEA_ANYTHING` returns HTTP 400 {"message":"invalid secret +# name"}. That is why set-ci-token.sh stores SETTINGS_API_TOKEN and not +# GITEA_SETTINGS_TOKEN. `CONFORMANCE_TRIGGER_KUBECONFIG` is legal, and is the exact name +# .gitea/workflows/{gate.yml,merge-gate.yaml} already reference — do not "tidy" it. +# +# 2. `curl -sS` EXITS 0 ON AN HTTP ERROR. The first version of set-ci-token.sh sent its PUT +# to /dev/null and printed "stored" unconditionally; the secret never existed and nobody +# knew for hours. So every write here captures `-w '%{http_code}'`, checks it, AND reads +# the object back afterwards. A store is not claimed unless the read-back shows it. +# +# 3. `CLUSTER_AVAILABLE` IS A LOADED GUN — SEE THE 2026-09-29 CORRECTION IN THE PLAN. +# `conformance/run.sh` exits 0 in scaffold mode with all 54 tests `` when it +# cannot reach a cluster. Measured on 2026-09-29: with CLUSTER_AVAILABLE=true and no +# trigger path, ci/conformance.sh printed "[ ok ] L2/L3 conformance passed" at rc 0 — a +# FAKE GREEN on an enforcing required status check. ci/conformance.sh now BLOCKS that +# case (exit 1), so today the variable does not enable L2/L3: it only changes which way +# the stage fails. Arming it before §3 items 1-4 land therefore buys nothing and breaks +# every PR. This script refuses to do it. +# +# ---------------------------------------------------------------------------------------- +# WHY THE PRECONDITIONS ARE SPLIT INTO TWO CLASSES (deliberate, and load-bearing) +# ---------------------------------------------------------------------------------------- +# MINT-BLOCKING (cluster-side). Kubernetes places NO `use` check on a Job's +# `serviceAccountName`, so a token with plain `create jobs` in ns `conformance` can create a +# Job running as the PRIVILEGED `conformance-runner` SA with an arbitrary command. The +# trigger Role is only the necessary half; the sufficient half is the Kyverno ClusterPolicy +# `conformance-trigger-job-shape` (platform/kyverno/admission/), which must be present AND +# Enforce AND Ready. Minting this token while that policy is absent or in Audit hands CI a +# privileged identity. So those checks block the MINT itself — nothing is created if they +# fail. (See conformance/incluster/rbac.yaml's trigger header and plan §4.) +# +# ARM-BLOCKING (repo-side). The runner image, the Job template's init-container, and +# ci/conformance.sh's in-cluster trigger flow (plan §3 items 1, 2, 4) are what make a real +# L2/L3 run possible. Storing the kubeconfig secret without them is harmless — the +# conformance job is gated on the variable and never starts — and the plan explicitly lands +# the secret before the switch. So these checks block ONLY the variable. +# +# Net effect: you can safely run this script today to provision the secret. It will then +# REFUSE to arm the variable, and tell you exactly which repo-side item is missing. +# +# ---------------------------------------------------------------------------------------- +# USAGE +# ---------------------------------------------------------------------------------------- +# ./infra/gitea/conformance-trigger-kubeconfig.sh --dry-run +# Read-only. Measures every precondition, mints nothing, writes nothing. +# ./infra/gitea/conformance-trigger-kubeconfig.sh [--server URL] +# Mint + store the secret. Does NOT touch CLUSTER_AVAILABLE. +# ./infra/gitea/conformance-trigger-kubeconfig.sh --arm-cluster-available +# The above, then set vars.CLUSTER_AVAILABLE=true after a typed confirmation — +# refused unless every precondition is green. +# +# Options: +# --dry-run measure only; no TokenRequest, no PUT/POST +# --arm-cluster-available additionally set vars.CLUSTER_AVAILABLE=true (guarded) +# --server URL API server URL to embed (default: the operator context's own) +# --context NAME kube context to read the CA/server from (default: k3s-prod) +# --duration D requested token lifetime (default: 720h = 30d) +# -h | --help this text +# +# ON --server: the embedded server URL must be reachable FROM THE CI RUNNER POD, which is +# not necessarily the address your workstation uses. The runner has broad egress and reaches +# the kube-API VIP (per the plan: 192.168.1.49:6443). If your operator kubeconfig points at +# a workstation-only name or a localhost tunnel, pass --server explicitly; the script warns +# on a loopback server but cannot know your topology for you. +# +# ON TOKEN LIFETIME: this is a TokenRequest token and it EXPIRES. The API server may cap the +# requested duration, so the script decodes the JWT `exp` claim and reports the ACTUAL +# expiry rather than trusting --duration. Re-run this script to rotate. After expiry the +# trigger gets 401 and the conformance stage FAILS (loudly) rather than skipping — which is +# the correct direction, but put the expiry date in your calendar. +# +# DELIBERATELY NOT DONE: SOPS-sealing the token into secrets/conformance-trigger.sops.yaml +# (plan §3 item 6's "for reproducibility/rotation"). A bearer token with a 30-day life +# cannot usefully survive the branch -> PR -> review -> merge flow (CLAUDE.md §6), and +# committing one buys re-issuance we get for free by re-running this script against the +# live SA. If the operator still wants it sealed, that is a separate reviewed change. +# +# FORBIDDEN HERE, ON PURPOSE: no cluster-admin kubeconfig, no copy of the operator's own +# kubeconfig, no widening of conformance/incluster/rbac.yaml, no mutating kubectl beyond the +# TokenRequest itself, no `kubectl apply`. +set -euo pipefail + +# xtrace would print the token. Refuse rather than leak. +case "$-" in + *x*) echo "FATAL: refusing to run under 'set -x' — xtrace would print the token" >&2; exit 1 ;; +esac +umask 077 + +HOST=gitea.home.keithjackson.dev +OWNER=Home-Network +REPO=home-network-k3s-impl +# Legal secret name: NOT prefixed GITEA_ (lesson 1), and the name the workflows already use. +SECRET=CONFORMANCE_TRIGGER_KUBECONFIG +VARIABLE=CLUSTER_AVAILABLE +NS=conformance +SA=conformance-trigger +RUNNER_SA=conformance-runner +POLICY=conformance-trigger-job-shape +# The image prefix the admission policy admits. Kept in sync with +# platform/kyverno/admission/conformance-trigger-job-shape.yaml by the P2 check below. +RUNNER_IMAGE_PREFIX='gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:' + +KCTX=k3s-prod +DURATION=720h +DRY_RUN=false +ARM=false +SERVER_OVERRIDE="" + +REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" + +step() { printf '\n==> %s\n' "$*"; } +info() { printf ' %s\n' "$*"; } +warn() { printf ' WARN: %s\n' "$*" >&2; } +die() { printf 'FATAL: %s\n' "$*" >&2; exit 1; } + +usage() { + cat <<'USAGE' +conformance-trigger-kubeconfig.sh — provision the narrow L2/L3 conformance CI trigger credential. + + --dry-run measure every precondition; mint nothing, write nothing (read-only) + --arm-cluster-available additionally set vars.CLUSTER_AVAILABLE=true (guarded + confirmed) + --server URL API server URL to embed (default: the operator context's own; + must be reachable FROM THE CI RUNNER POD — the kube-API VIP) + --context NAME kube context to read the CA/server from (default: k3s-prod) + --duration D requested token lifetime (default: 720h) + -h | --help this text + +Read the header of this file before running it: the CLUSTER_AVAILABLE guard, the split between +mint-blocking and arm-blocking preconditions, and why nothing secret is ever printed. +USAGE +} + +while [ $# -gt 0 ]; do + case "$1" in + --dry-run) DRY_RUN=true ;; + --arm-cluster-available) ARM=true ;; + --server) SERVER_OVERRIDE="${2:?--server needs a URL}"; shift ;; + --context) KCTX="${2:?--context needs a name}"; shift ;; + --duration) DURATION="${2:?--duration needs a value}"; shift ;; + -h|--help) usage; exit 0 ;; + *) die "unknown argument: $1 (try --help)" ;; + esac + shift +done +if $DRY_RUN && $ARM; then die "--dry-run and --arm-cluster-available are contradictory"; fi + +for t in kubectl jq curl git base64; do + command -v "$t" >/dev/null || die "missing required tool: $t" +done + +WORK="$(mktemp -d)" +KC="${WORK}/conformance-trigger.kubeconfig" +cleanup() { rm -rf "$WORK"; } +trap cleanup EXIT +trap 'cleanup; exit 130' INT TERM + +# ======================================================================================== +# 0. Gitea credentials — from the credential helper git already has for this host, exactly +# as branch-protection.sh / set-ci-token.sh / require-gate.sh do. Never in argv: curl +# reads them from a --config file descriptor. +# ======================================================================================== +creds="$(printf 'protocol=https\nhost=%s\n\n' "$HOST" | git credential fill)" +U="$(sed -n 's/^username=//p' <<<"$creds")" +P="$(sed -n 's/^password=//p' <<<"$creds")" +unset creds +[ -n "$U" ] && [ -n "$P" ] || die "no stored credential for ${HOST} (git credential fill returned nothing)" +basic() { curl -sS --config <(printf 'user = "%s:%s"\n' "$U" "$P") "$@"; } +API="https://${HOST}/api/v1/repos/${OWNER}/${REPO}" + +# ======================================================================================== +# 1. PRECONDITIONS — measured, never assumed (CLAUDE.md §2b: query the thing itself). +# Two independent tallies, per the split documented in the header. +# ======================================================================================== +MINT_FAIL=0 +ARM_FAIL=0 +pass() { printf ' [ ok ] %s\n' "$*"; } +fail_mint() { printf ' [MINT-BLOCK] %s\n' "$*" >&2; MINT_FAIL=$((MINT_FAIL + 1)); } +fail_arm() { printf ' [ARM-BLOCK] %s\n' "$*" >&2; ARM_FAIL=$((ARM_FAIL + 1)); } + +step "preconditions — cluster side (these block the MINT: without them the token IS privileged)" + +kubectl --context "$KCTX" get --raw /version >/dev/null 2>&1 \ + || die "cannot reach the API server via context '${KCTX}' — nothing below can be measured" +pass "API server reachable via context ${KCTX}" + +# C1: the trigger identity exists (created by Flux from conformance/incluster/rbac.yaml via +# clusters/base/conformance.yaml). We print object names only, never contents. +if kubectl --context "$KCTX" -n "$NS" get serviceaccount "$SA" -o name >/dev/null 2>&1; then + pass "ServiceAccount ${NS}/${SA} exists" +else + fail_mint "ServiceAccount ${NS}/${SA} is ABSENT — is the 'conformance' Flux Kustomization reconciled?" +fi +for kind in role rolebinding; do + if kubectl --context "$KCTX" -n "$NS" get "$kind" "$SA" -o name >/dev/null 2>&1; then + pass "${kind} ${NS}/${SA} exists" + else + fail_mint "${kind} ${NS}/${SA} is ABSENT — the token would authenticate but authorize nothing" + fi +done + +# C2: THE SECURITY BOUNDARY. Present, Enforce, and Ready. Anything else and we do not mint. +# `validationFailureAction: Audit` is not a boundary; an unready policy is not a boundary. +if pol="$(kubectl --context "$KCTX" get clusterpolicy "$POLICY" -o json 2>/dev/null)"; then + action="$(jq -r '.spec.validationFailureAction // .spec.rules[0].validate.failureAction // "UNSET"' <<<"$pol")" + failpol="$(jq -r '.spec.failurePolicy // "UNSET"' <<<"$pol")" + # Kyverno moved readiness from .status.ready to a Ready condition; accept either, and + # treat "neither key present" as UNMEASURABLE -> block (fail closed, do not assume ready). + ready="$(jq -r ' + if (.status.ready != null) then (.status.ready | tostring) + elif (.status.conditions != null) then + ((.status.conditions[] | select(.type == "Ready") | .status) // "UNKNOWN") + else "UNKNOWN" end' <<<"$pol")" + info "ClusterPolicy ${POLICY}: validationFailureAction=${action} failurePolicy=${failpol} ready=${ready}" + [ "$action" = "Enforce" ] || fail_mint "ClusterPolicy ${POLICY} is '${action}', not Enforce — the trigger would be unbounded" + [ "$failpol" = "Fail" ] || fail_mint "ClusterPolicy ${POLICY} failurePolicy='${failpol}', not Fail — a Kyverno outage would admit arbitrary Jobs" + case "$ready" in + true|True) pass "ClusterPolicy ${POLICY} reports Ready" ;; + *) fail_mint "ClusterPolicy ${POLICY} does not report Ready (got '${ready}') — refusing to assume it is loaded" ;; + esac +else + fail_mint "ClusterPolicy ${POLICY} is ABSENT — plan §4's boundary does not exist; a token with 'create jobs' here can run arbitrary code as ${RUNNER_SA}" +fi + +step "preconditions — repo side (these block ONLY arming ${VARIABLE}; the secret may be stored first)" + +# R1: the runner image is built and digest-pinned. Without it the Job cannot run run.sh at +# all (the stock chainsaw image has no bash/python3/kubectl/git) — plan §3 item 1. +VERSIONS="${REPO_ROOT}/conformance/versions.env" +if grep -Eq '^[[:space:]]*CONFORMANCE_RUNNER_IMAGE=.*@sha256:[0-9a-f]{64}' "$VERSIONS" 2>/dev/null; then + pass "CONFORMANCE_RUNNER_IMAGE is digest-pinned in conformance/versions.env" +else + fail_arm "conformance/versions.env has no digest-pinned CONFORMANCE_RUNNER_IMAGE (plan §3 item 1) — the Job cannot run ./run.sh" +fi + +# R2: the Job template is the real thing, not the skeleton: the admitted image AND an +# init-container that populates the empty `workspace` volume with the suite at the PR SHA +# (plan §3 item 2). A template whose image the policy will not admit is a guaranteed denial. +JOBTPL="${REPO_ROOT}/conformance/incluster/chainsaw-job.yaml" +if grep -q 'initContainers' "$JOBTPL" 2>/dev/null; then + pass "chainsaw-job.yaml declares an initContainers block (workspace population)" +else + fail_arm "chainsaw-job.yaml has no initContainers — 'workspace' is an empty emptyDir and the suite is never delivered (plan §3 item 2)" +fi +if grep -q "$RUNNER_IMAGE_PREFIX" "$JOBTPL" 2>/dev/null; then + pass "chainsaw-job.yaml uses the digest-pinned image the admission policy admits" +else + fail_arm "chainsaw-job.yaml does not use ${RUNNER_IMAGE_PREFIX}… — every triggered Job would be DENIED by ${POLICY}" +fi + +# R3: ci/conformance.sh actually drives the in-cluster Job. Today it mentions +# CONFORMANCE_TRIGGER_KUBECONFIG only in a TODO comment and otherwise runs run.sh LOCALLY in +# the CI job — where there is no credential at all (the runner sets +# automountServiceAccountToken: false by design), so it can only scaffold-skip. Hence the +# check must ignore comment lines: a mention in prose is not an implementation. +# +# ONE awk, NOT `grep -v … | grep -q …`: under `set -o pipefail` a successful `grep -q` exits +# early, the upstream grep takes EPIPE, and the pipeline reports non-zero — which would read +# here as "unwired" exactly when the seam IS wired. A single process cannot have that bug. +CONF_STAGE="${REPO_ROOT}/ci/conformance.sh" +if awk '!/^[[:space:]]*#/ && /CONFORMANCE_TRIGGER_KUBECONFIG/ { found = 1 } END { exit !found }' \ + "$CONF_STAGE" 2>/dev/null; then + pass "ci/conformance.sh references CONFORMANCE_TRIGGER_KUBECONFIG in code (in-cluster trigger path wired)" +else + fail_arm "ci/conformance.sh mentions CONFORMANCE_TRIGGER_KUBECONFIG only in comments — the trigger path is UNIMPLEMENTED (plan §3 item 4); arming ${VARIABLE} now only changes which way the stage fails" +fi + +info "" +info "precondition tally: mint-blocking failures=${MINT_FAIL} arm-blocking failures=${ARM_FAIL}" + +if [ "$MINT_FAIL" -gt 0 ]; then + cat >&2 <&2 <}" + cur="$(basic -o "${WORK}/var.json" -w '%{http_code}' "${API}/actions/variables/${VARIABLE}")" || true + if [ "$cur" = 200 ]; then + info "existing ${VARIABLE} = $(jq -r '.data // ""' "${WORK}/var.json")" + else + info "existing ${VARIABLE}: not set (HTTP ${cur})" + fi + exit 0 +fi + +# ======================================================================================== +# 2. MINT the ServiceAccount token (TokenRequest). This is the only write this script makes +# against the cluster, and it creates no durable object. +# ======================================================================================== +step "minting a ServiceAccount token for ${NS}/${SA} (requested duration ${DURATION})" +TOKEN="$(kubectl --context "$KCTX" -n "$NS" create token "$SA" --duration="$DURATION")" \ + || die "kubectl create token failed for ${NS}/${SA}" +[ -n "$TOKEN" ] || die "kubectl create token returned an empty token" +info "got a ${#TOKEN}-character token (value NOT shown)" + +# Report the ACTUAL expiry from the JWT rather than trusting --duration: the API server caps +# TokenRequest lifetimes (--service-account-max-token-expiration) and silently shortens. +jwt_exp() { + local payload pad b64 + payload="$(cut -d. -f2 <<<"$1")" + pad=$(( (4 - ${#payload} % 4) % 4 )) + b64="$(tr '_-' '/+' <<<"$payload")$(printf '%*s' "$pad" '' | tr ' ' '=')" + printf '%s' "$b64" | base64 -d 2>/dev/null | jq -r '.exp // empty' 2>/dev/null || true +} +exp="$(jwt_exp "$TOKEN")" +if [ -n "$exp" ]; then + info "token expires: $(date -u -d "@${exp}" '+%Y-%m-%dT%H:%M:%SZ' 2>/dev/null || echo "epoch ${exp}") — re-run this script to rotate" +else + warn "could not decode the token's exp claim; lifetime is UNVERIFIED (requested ${DURATION})" +fi + +# ======================================================================================== +# 3. ASSEMBLE the kubeconfig — SA token + cluster CA + server. Nothing else. +# +# The CA and the server URL are read from the operator's context, but the operator's +# client certificate/key are NEVER copied: only .cluster fields are extracted. `config +# view --raw` output is held in a variable and discarded, never printed. +# Built with jq (not `kubectl config set-credentials --token=…`) so the token never +# appears in argv where `ps` can see it. +# ======================================================================================== +step "assembling the kubeconfig (SA token only — never cluster-admin, never the operator's)" +raw="$(kubectl --context "$KCTX" config view --raw --minify -o json)" \ + || die "could not read context '${KCTX}' from the operator kubeconfig" +SERVER="$(jq -r '.clusters[0].cluster.server // empty' <<<"$raw")" +CADATA="$(jq -r '.clusters[0].cluster["certificate-authority-data"] // empty' <<<"$raw")" +CAFILE="$(jq -r '.clusters[0].cluster["certificate-authority"] // empty' <<<"$raw")" +INSECURE="$(jq -r '.clusters[0].cluster["insecure-skip-tls-verify"] // false | tostring' <<<"$raw")" +unset raw + +[ -n "$SERVER" ] || die "context '${KCTX}' has no cluster.server" +if [ -n "$SERVER_OVERRIDE" ]; then + info "overriding server ${SERVER} -> ${SERVER_OVERRIDE}" + SERVER="$SERVER_OVERRIDE" +fi +case "$SERVER" in + *localhost*|*127.0.0.1*|*://0.0.0.0*) + die "server '${SERVER}' is loopback — the CI runner pod cannot reach it. Pass --server with the address the runner uses (the kube-API VIP)." ;; +esac +if [ -z "$CADATA" ] && [ -n "$CAFILE" ]; then + [ -r "$CAFILE" ] || die "certificate-authority file '${CAFILE}' is not readable" + CADATA="$(base64 -w0 <"$CAFILE")" +fi +[ -n "$CADATA" ] || die "no cluster CA available for '${KCTX}' (insecure-skip-tls-verify=${INSECURE}); refusing to emit a kubeconfig that trusts anything" +[ "$INSECURE" = "false" ] || die "context '${KCTX}' sets insecure-skip-tls-verify; refusing to propagate that into CI" +info "server: ${SERVER}" +info "CA: ${#CADATA} base64 characters embedded (public certificate, not a secret)" + +jq -n --arg srv "$SERVER" --arg ca "$CADATA" --arg tok "$TOKEN" --arg ns "$NS" --arg sa "$SA" ' +{ + apiVersion: "v1", + kind: "Config", + clusters: [ { name: "conformance", cluster: { server: $srv, "certificate-authority-data": $ca } } ], + users: [ { name: $sa, user: { token: $tok } } ], + contexts: [ { name: "conformance", context: { cluster: "conformance", user: $sa, namespace: $ns } } ], + "current-context": "conformance" +}' >"$KC" +chmod 600 "$KC" +info "kubeconfig written to the temp dir: $(wc -c <"$KC") bytes (content NOT shown)" + +# Prove kubectl can PARSE the file we generated (JSON is valid YAML, but assert it rather +# than assume it) — this prints only the server we already printed. +parsed="$(kubectl --kubeconfig "$KC" config view -o jsonpath='{.clusters[0].cluster.server}')" \ + || die "kubectl cannot parse the generated kubeconfig" +[ "$parsed" = "$SERVER" ] || die "generated kubeconfig parses to server '${parsed}', expected '${SERVER}'" +pass "kubectl parses the generated kubeconfig" + +# ======================================================================================== +# 4. PROVE it is the narrow identity. Two independent proofs: +# a) who the token authenticates AS (SelfSubjectReview) +# b) an allow/deny matrix (SelfSubjectAccessReview) — both halves matter: "can create +# jobs" proves it works, "cannot get secrets / delete jobs / exec" proves it is not +# something wider that happens to include the trigger rights. +# All read-only. `auth can-i` answers from the API server's own authorizer. +# ======================================================================================== +step "verifying the credential is the NARROW trigger identity" +EXPECT_USER="system:serviceaccount:${NS}:${SA}" +if who="$(kubectl --kubeconfig "$KC" auth whoami -o jsonpath='{.status.userInfo.username}' 2>/dev/null)"; then + [ "$who" = "$EXPECT_USER" ] || die "token authenticates as '${who}', expected '${EXPECT_USER}' — WRONG CREDENTIAL, nothing stored" + pass "authenticates as ${who}" +else + warn "'kubectl auth whoami' unavailable (older kubectl / SelfSubjectReview denied) — identity UNVERIFIED by this check; the matrix below still constrains it" +fi + +# A real authorization round-trip, not just a review: `list jobs` is granted by the Role. +kubectl --kubeconfig "$KC" -n "$NS" get jobs -o name >/dev/null \ + || die "the minted credential cannot list Jobs in ns ${NS} — the RoleBinding is not effective; nothing stored" +pass "can list Jobs in ns ${NS} (end-to-end authentication + authorization)" + +cani() { # cani + local want="$1"; shift + local got + got="$(kubectl --kubeconfig "$KC" auth can-i "$@" 2>/dev/null || true)" + case "$got" in + yes|no) : ;; + *) die "auth can-i '$*' returned '${got:-}' — cannot measure the identity's rights; refusing to store an unverified credential" ;; + esac + if [ "$got" = "$want" ]; then + pass "can-i $* -> ${got} (expected)" + else + die "can-i $* -> ${got}, expected ${want}. This credential is NOT the narrow trigger identity described in conformance/incluster/rbac.yaml. Nothing stored. Do NOT widen that Role to make this pass — investigate what it is actually bound to." + fi +} +# Granted by the trigger Role (create/get/list/watch jobs; get/list/watch pods + pods/log). +cani yes create jobs.batch -n "$NS" +cani yes get pods -n "$NS" +cani yes get pods/log -n "$NS" +# Explicitly NOT granted — each is a specific escape route the Role's comments call out. +cani no delete jobs.batch -n "$NS" # a trigger that can delete can hide a failed run +cani no create pods/exec -n "$NS" # no execution channel into the runner pod +cani no get secrets -n "$NS" +cani no create jobs.batch -n default # namespaced Role: no reach outside ns conformance +cani no get nodes # the RUNNER has nodes/patch; the trigger must not +cani no '*' '*' --all-namespaces # not cluster-admin, stated as a measurement + +# ======================================================================================== +# 5. STORE as the Gitea Actions repo secret, then VERIFY BY READING IT BACK (lesson 2). +# The body goes in a 0600 file so the token is not in argv. +# ======================================================================================== +step "storing the kubeconfig as repo Actions secret ${SECRET}" +jq -n --rawfile d "$KC" '{data:$d}' >"${WORK}/body.json" +chmod 600 "${WORK}/body.json" +code="$(basic -o "${WORK}/put.out" -w '%{http_code}' -X PUT \ + -H 'Content-Type: application/json' --data @"${WORK}/body.json" \ + "${API}/actions/secrets/${SECRET}")" +unset TOKEN +rm -f "${WORK}/body.json" "$KC" +if [ "$code" != 201 ] && [ "$code" != 204 ]; then + # Safe to echo: the response body is Gitea's error message, not our payload. HTTP 400 + # "invalid secret name" here means the name hit the reserved GITEA_ prefix (lesson 1). + die "PUT ${SECRET} failed: HTTP ${code} -- $(head -c 300 "${WORK}/put.out" 2>/dev/null)" +fi +info "PUT returned HTTP ${code}" + +# curl -sS exits 0 on an HTTP error, so the status check above is necessary but NOT +# sufficient: read the object back. Gitea never returns secret VALUES, so this proves +# PRESENCE BY NAME only — say so rather than implying the content was checked. +names="$(basic "${API}/actions/secrets" | jq -rc '[.. | objects | select(has("name")) | .name] | unique')" +grep -q "$SECRET" <<<"$names" \ + || die "PUT reported HTTP ${code} but ${SECRET} is NOT in ${names} — refusing to claim success" +pass "stored and VERIFIED PRESENT by read-back: ${names}" +info "(read-back proves the secret EXISTS; Gitea never returns secret values, so its CONTENT" +info " is proven only by the first CI run that uses it — see the NOTE below.)" +info "" +info "FOR WHOEVER WRITES plan §3 item 4: this secret holds the kubeconfig CONTENT, not a path." +info "The plan's sketch writes 'kubectl --kubeconfig=\$CONFORMANCE_TRIGGER_KUBECONFIG', which" +info "would treat it as a filename. ci/conformance.sh must write the env var into a 0600 temp" +info "file (umask 077, trap-removed) and pass THAT path — an Actions secret is a string, so a" +info "kubeconfig cannot reach CI any other way." + +# ======================================================================================== +# 6. vars.CLUSTER_AVAILABLE — the loaded gun. Only here, only on request, only confirmed. +# ======================================================================================== +if ! $ARM; then + cat < ${VARIABLE} NOT TOUCHED (no --arm-cluster-available). This is the safe default. + +The secret is in place; the conformance job stays gated off until the variable is set. That +is plan §5's order: land items 1-6, prove them, and only then flip step 7's switch. +EOF +else + step "arming vars.${VARIABLE}=true — every precondition above is green" + cat </dev/null)); trying PUT (variable may already exist)" + vcode="$(basic -o "${WORK}/var.out" -w '%{http_code}' -X PUT \ + -H 'Content-Type: application/json' \ + --data "$(jq -n --arg n "$VARIABLE" '{name:$n,value:"true"}')" \ + "${API}/actions/variables/${VARIABLE}")" + fi + [ "$vcode" = 201 ] || [ "$vcode" = 204 ] \ + || die "could not set ${VARIABLE}: HTTP ${vcode} -- $(head -c 300 "${WORK}/var.out" 2>/dev/null)" + info "write returned HTTP ${vcode}" + + # Unlike a secret, a variable's value IS readable — so verify the VALUE, not just presence. + # gate.yml compares `vars.CLUSTER_AVAILABLE == 'true'`, so it must be exactly "true". + gcode="$(basic -o "${WORK}/varget.json" -w '%{http_code}' "${API}/actions/variables/${VARIABLE}")" + [ "$gcode" = 200 ] || die "stored HTTP ${vcode} but GET ${VARIABLE} returned HTTP ${gcode} — refusing to claim success" + val="$(jq -r '.data // .value // empty' "${WORK}/varget.json")" + [ "$val" = "true" ] || die "${VARIABLE} reads back as '${val}', not 'true' — gate.yml compares against 'true' exactly" + pass "${VARIABLE}=${val} stored and VERIFIED by read-back" +fi + +cat <