Compare commits

...
Author SHA1 Message Date
jakman85andClaude Opus 5 e993d0f7d1 docs(ci): the wiring comment named a secret Gitea cannot accept
gate / gate (pull_request) Successful in 1m36s
gate / conformance (pull_request) Has been skipped
ci/gitea-settings.sh told the reader to "create a Gitea repo secret
GITEA_SETTINGS_TOKEN". That instruction is IMPOSSIBLE to follow: Gitea reserves the
GITEA_ prefix for secret names and rejects the PUT with HTTP 400 "invalid secret name".

Same defect as the one fixed in PR #15 and in the README in PR #19 -- this was the third
file carrying it, found by an agent reading the file rather than trusting my description
of it. A comment that instructs an impossible action is not cosmetic: this exact wrong
name is why the drift detector sat inert while a token existed and no secret did.

Now names SETTINGS_API_TOKEN, and states explicitly that GITEA_SETTINGS_TOKEN remains
honoured ONLY as a local-run env var -- it can never be the CI wiring, because the secret
that would feed it cannot exist.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-29 19:30:35 -05:00
jakman85 20ea70d221 Merge pull request 'fix(sec): the R-SEC-02 admission policy was never applied' (#16) from fix/kyverno-policy-was-never-applied into main
gate / gate (push) Successful in 1m36s
gate / conformance (push) Has been skipped
Reviewed-on: #16
Reviewed-by: Keith Jackson <[email protected]>
2026-09-29 18:28:54 +00:00
jakman85andClaude Opus 5 8c7a770de2 fix(sec): the R-SEC-02 admission policy was never applied -- and nothing could tell
gate / gate (pull_request) Successful in 1m37s
gate / conformance (pull_request) Has been skipped
PR #13 added conformance-trigger-job-shape.yaml to platform/kyverno/admission/
kustomization.yaml. That file applies NOTHING. Its own sibling says so: "kept as
reference; THIS file is the buildable render root" -- and that render root
(platform/kyverno/kustomization.yaml) ships only the namespace and the HelmRelease,
noting that ClusterPolicies are applied a tier later, by platform-config, because they
need the Kyverno CRDs first.

So three lists exist and only platform-config/kustomization.yaml applies anything. The
policy went into the wrong one and EVERYTHING WENT GREEN: the file parsed, kyverno test
exercised it by path and passed, the merge gate passed, and Flux reported `platform`
Ready at the new revision. Measured on prod afterwards:

    kubectl get clusterpolicy conformance-trigger-job-shape
    Error from server (NotFound)

while ns conformance, the conformance-trigger ServiceAccount, its Role and its
RoleBinding were all live. That is exactly the state that identity's own RBAC header
says must never exist -- the trigger identity present with nothing bounding it. Blast
radius was small only because no trigger kubeconfig exists yet, which is luck, not design.

Wires the policy into platform-config (verified: the render goes from 7 ClusterPolicies to
8 and contains conformance-trigger-job-shape).

CLOSES THE CLASS, which matters more than the instance. Adds the L0 check
admission-policies-actually-applied: every ClusterPolicy file under platform/kyverno/ is
either referenced by platform-config/kustomization.yaml or named in a CI_ONLY list with a
reason -- and CI_ONLY may not name a policy that IS applied, so the exemption cannot rot in
either direction. Declared scope whole-repo deliberately: a policy wired nowhere touches no
requirement the diff attributes, so a path-scoped run would have skipped it on the exact
commit that introduced the defect.

Proven both ways, not assumed:
  with the fix reverted -> FAIL, naming conformance-trigger-job-shape as never applied
  with the fix          -> PASS, 9 policy files, 8 applied, 1 CI-only with a reason

`kyverno test` proves a policy is CORRECT. This proves it is DEPLOYED. Nothing did before.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-29 13:23:09 -05:00
jakman85 1c5974013e Merge pull request 'docs(decisions-log): end-of-session PM handoff' (#14) from docs/handoff-2026-09-29 into main
gate / gate (push) Successful in 1m35s
gate / conformance (push) Has been skipped
Reviewed-on: #14
Reviewed-by: Keith Jackson <[email protected]>
2026-09-29 18:16:22 +00:00
jakman85 4791d4a43a Merge pull request 'feat(sec): constrain the conformance CI trigger at admission' (#13) from feat/conformance-trigger-admission-boundary into main
gate / conformance (push) Has been cancelled
gate / gate (push) Has been cancelled
Reviewed-on: #13
Reviewed-by: Keith Jackson <[email protected]>
2026-09-29 18:16:00 +00:00
jakman85 dfd608ac32 Merge pull request 'fix(ci): CLUSTER_AVAILABLE=true would have faked a green' (#12) from fix/cluster-available-would-fake-a-green into main
gate / conformance (push) Has been cancelled
gate / gate (push) Has been cancelled
Reviewed-on: #12
Reviewed-by: Keith Jackson <[email protected]>
2026-09-29 18:14:55 +00:00
jakman85 1d2d9d6549 Merge pull request 'fix(ci): the settings token could never have been stored' (#15) from fix/secret-name-and-require-gitea-settings into main
gate / conformance (push) Has been cancelled
gate / gate (push) Has been cancelled
Reviewed-on: #15
Reviewed-by: Keith Jackson <[email protected]>
2026-09-29 18:14:30 +00:00
jakman85andClaude Opus 5 4b31f97486 docs(decisions-log): addendum — detector armed, and why an agent cannot merge here
gate / gate (pull_request) Successful in 1m34s
gate / conformance (pull_request) Has been skipped
Records two findings from the unattended session.

The drift detector was INERT: set-ci-token.sh stored the secret under a name Gitea
reserves (GITEA_ prefix, HTTP 400 "invalid secret name"), sent the PUT to /dev/null,
and printed success anyway. A token existed; no secret did. Enabling
REQUIRE_GITEA_SETTINGS on top of that would have blocked every PR including its own
fix. PR #15 fixes it and its CI is green WITH require active, which is the only
proof that counts.

And: merging as gitea-admin returns HTTP 405 "Does not have enough approvals".
required_approvals:1 + block_admin_merge_override:true + an author who cannot
self-approve means "merge on green CI" is not executable by an agent. Recorded as
the control working rather than a fault, and deliberately not circumvented --
self-approval would hollow out the boundary just built. The choice of what to do
about it is the operator s, and whatever is chosen must also change the declaration
in expected-branch-protection.json or the detector will correctly fail on the drift.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-29 12:12:33 -05:00
jakman85andClaude Opus 5 50a48e6f07 fix(ci): the settings token could NEVER have been stored -- Gitea reserves the GITEA_ prefix
gate / gate (pull_request) Successful in 1m36s
gate / conformance (pull_request) Has been skipped
The drift detector merged in #11 was inert and would have stayed inert silently.

WHAT HAPPENED. infra/gitea/set-ci-token.sh minted a token and PUT it to the repo Actions
secret GITEA_SETTINGS_TOKEN. Gitea RESERVES the `GITEA_` prefix for secret names -- exactly
as GitHub reserves `GITHUB_` -- and rejects that PUT with:

    HTTP 400  {"message":"invalid secret name"}

The script sent the PUT to /dev/null and printed "==> stored." unconditionally, and
`curl -sS` exits 0 on an HTTP error, so it reported success for a store that never happened.
Measured afterwards: the token existed in the user's token list, and the repo's secret list
was EMPTY. ci/gitea-settings.sh would have skipped on every CI run forever, and with
REQUIRE_GITEA_SETTINGS=true it would instead have BLOCKED EVERY PR -- including the one
fixing it.

Three changes:
  * the secret is SETTINGS_API_TOKEN; ci/gitea-settings.sh reads it, keeping
    GITEA_SETTINGS_TOKEN as a local-run fallback;
  * set-ci-token.sh checks the PUT's HTTP status AND reads the secret back, refusing to claim
    success otherwise. That missing verification is the actual defect -- the wrong name was
    just what it hid;
  * gate.yml maps the secret and ENABLES REQUIRE_GITEA_SETTINGS.

WHY ENABLING IT HERE IS SAFE: this PR's own CI run is the proof. If the stage were still
skipping, REQUIRE turns that into a failure and this PR goes RED rather than merging. Green
means the token resolved and live Gitea matched infra/gitea/expected-branch-protection.json.

Verified locally before pushing:
  REQUIRE=true, token present -> MERGE GATE: PASS
  REQUIRE=true, no token      -> gitea-settings SKIPPED -> BLOCKED (not a silent skip)

The secret is already stored and read back as present, so CI has what it needs.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-29 12:08:12 -05:00
jakman85andClaude Opus 5 911ead5fbd docs(decisions-log): end-of-session PM handoff — six PRs open, and what a new machine needs
gate / gate (pull_request) Successful in 1m33s
gate / conformance (pull_request) Has been skipped
Written for a different session on a different computer. Lists the three open gated
PRs with their risk, the one command that must follow #11 (the drift detector skips
until its token exists, and a skipped detector detects nothing), and the three
credentials that are deliberately NOT in git and without which the toolbox and
cluster are unreachable: the age private key, prod.kubeconfig, and a Gitea
credential.

Records the measured platform state while telling the reader to re-measure it rather
than trust this log -- which is the whole point of CLAUDE.md section 2b, and which
this log earned by being confidently wrong for a month.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-29 03:28:33 -05:00
jakman85 4634b44cc7 Merge pull request 'feat(ci): detect Gitea settings drift' (#11) from feat/gitea-settings-drift-check into main
gate / gate (push) Successful in 1m34s
gate / conformance (push) Has been skipped
Reviewed-on: #11
Reviewed-by: Keith Jackson <[email protected]>
2026-09-29 07:59:29 +00:00
jakman85andClaude Opus 5 6586919132 fix(ci): CLUSTER_AVAILABLE=true would have FAKED A GREEN over 54 skipped tests
gate / gate (pull_request) Successful in 1m36s
gate / conformance (pull_request) Has been skipped
Wiring L2/L3 into CI was a tracked next step, described in the plan as needing only
vars.CLUSTER_AVAILABLE. Attempting it surfaced that the variable does not enable L2/L3 at
all -- it only changes which way the stage fails, and before this commit one of those ways
was a false pass.

MEASURED, before the guard:
  CLUSTER_AVAILABLE=true ./ci/conformance.sh   ->  rc=0
  "[junit] wrote skipped JUnit for 54 scaffolded test(s)"
  "[ ok ] L2/L3 conformance passed for touched subset"

run.sh exits 0 in scaffold mode when it cannot reach a cluster, marking every test
<skipped>. That is correct and honest FOR run.sh. It is catastrophic in ci/conformance.sh,
which reaches that code only because the environment DECLARED a cluster available -- so it
reported a pass over 54 tests that never executed. One Gitea variable away from a fake green
on a gate that is now an ENFORCING required status check, and nothing caught it because
nobody had ever set the variable.

The CI runner holds no cluster credential by design (automountServiceAccountToken: false),
so the local run this stage attempts can never succeed. The in-cluster trigger path that
would make L2/L3 actually run is still unimplemented, and is blocked on a privilege-
escalation design decision that is the operator's to make (plan section 4: any principal
that can `create jobs` in ns conformance can run arbitrary code as the privileged
conformance-runner SA, because Kubernetes places no `use` check on serviceAccountName).

The stage now BLOCKS (exit 1) when a cluster is declared but every test skipped, which is
what the merge-gate decision matrix already said should happen ("conformance skipped while
CLUSTER_AVAILABLE=true -> BLOCK"). Verified: rc=1, naming the count and the reason.

The plan document claimed CLUSTER_AVAILABLE was "the only remaining gate". Corrected in
place with the measurement and an explicit do-not-set-this-yet.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-29 02:58:26 -05:00
jakman85andClaude Opus 5 e6f6e4febb feat(ci): detect Gitea settings drift -- the blind spot that hid a month of outage
gate / gate (pull_request) Successful in 1m34s
gate / conformance (pull_request) Has been skipped
Every check in this repo reads the repository, the rendered manifests, or the cluster.
NONE could see Gitea's own database, and that is where branch protection and runner
registration live. Three failures came out of that blind spot on 2026-09-29, all the same
shape -- a setting that READ as enforced while not being, with nothing able to notice:

  * main had NO branch protection for ~a month while DECISIONS-LOG recorded it as a
    required status check;
  * ZERO Actions runners were registered, so PR #1 -- the first PR this repo ever had --
    sat pending for 30.6 minutes with nothing to execute it;
  * after enable_status_check and required_approvals were turned on they STILL did not
    bite, because block_admin_merge_override was false: PRs #5-#9 merged with zero
    approvals recorded against required_approvals: 1.

infra/gitea/*.sh ASSERT the desired state. Nothing DETECTED drift from it. This is the
detector.

ci/gitea-settings.sh runs as a merge-gate stage: it diffs the live rule at
/branch_protections/main against infra/gitea/expected-branch-protection.json and asserts at
least one runner is online. It reads the RULE object, not /branches/main -- the branch view
does not expose block_admin_merge_override and showed all-green while the rule was
overridable by the only account that merges.

Verified by test, not assumed:
  no token            -> exit 3, skips loudly, does NOT block
  live == declaration -> exit 0
  injected drift      -> exit 1, blocks, naming both the drifted key and the runner count

The skip is DELIBERATELY not in gate.sh's strict-skips loop. Making an absent credential
block every PR is precisely the lock-yourself-out failure this repo keeps walking into, so
a skip denies a PASS (exit 3) and says loudly what is unverified. REQUIRE_GITEA_SETTINGS=true
upgrades it to a block once the token is wired.

gate.yml's `|| [ "$?" -eq 3 ]` tolerance now covers a second stage. That widening is
declared in the file rather than left to be discovered, and it narrows back the moment
REQUIRE_GITEA_SETTINGS is set.

infra/gitea/set-ci-token.sh mints the token and stores it as the repo Actions secret without
ever printing the value.

UNTIL THAT TOKEN EXISTS THIS STAGE SKIPS, AND A SKIPPED DETECTOR DETECTS NOTHING.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-29 02:52:28 -05:00
13 changed files with 658 additions and 6 deletions

No files matched your search

+18
View File
@@ -110,9 +110,27 @@ jobs:
# UNVERIFIED and NAMED in the PASS headline, never faked green. GATE_STRICT_SKIPS + the
# tolerated exit 3 are explained in the file header; the tolerance covers exactly one
# thing, the unverifiable spec repo, and nothing else can produce a 3 here.
# GITEA_SETTINGS_TOKEN feeds ci/gitea-settings.sh, the only check that can see Gitea's
# OWN database (branch protection + runner registration) — the blind spot that let `main`
# sit unprotected with zero runners for a month while this repo recorded both as working.
# Referenced BY NAME only; create it with infra/gitea/set-ci-token.sh.
#
# WIDENED TOLERANCE, DECLARED: `|| [ "$?" -eq 3 ]` used to cover exactly one skip
# (spec-immutability). It now also covers gitea-settings skipping when that secret is
# absent. That is a deliberate, temporary weakening: making an unwired credential BLOCK
# every PR is the lock-yourself-out failure this repo keeps walking into. Once the secret
# exists and the stage has been SEEN green, set REQUIRE_GITEA_SETTINGS=true below and the
# tolerance narrows back to spec-immutability alone.
- name: Merge gate — ci/gate.sh (what `make -C ci gate` runs)
env:
GATE_STRICT_SKIPS: "true"
SETTINGS_API_TOKEN: ${{ secrets.SETTINGS_API_TOKEN }}
# ENABLED 2026-09-29. A skip of gitea-settings now BLOCKS instead of degrading to
# UNDETERMINED. Safe because THIS PR's own CI run is the proof: if the stage were
# skipping, this line turns that into a failure and the PR goes red instead of
# merging. Green here means the token resolved and the live Gitea matched the
# declaration in infra/gitea/expected-branch-protection.json.
REQUIRE_GITEA_SETTINGS: "true"
run: ./ci/gate.sh || [ "$?" -eq 3 ]
# NOTE: no actions/upload-artifact step. Gitea Actions rejects upload-artifact@v4
+145
View File
@@ -4964,3 +4964,148 @@ account rubber-stamping the human. That is ceremony, not review. Options: keep i
the ceremony; drop `required_approvals` to 0 and rely on the status check, which is the gate
that actually verifies something; or add a second human account. **This is a decision for the
operator, not a default to be chosen silently.**
## 2026-09-29 (end of session) — **PM HANDOFF.** Six PRs open; read this before touching anything.
The session that began "set a SOPS secret" ended having rebuilt the delivery pipeline. Plex is
claimed and scanning. `main` is genuinely protected and the gate genuinely enforces. What is
open is listed below with the reason each is not merged.
### MERGE THESE — both gated green
| PR | What | Risk |
|---|---|---|
| #12 | `CLUSTER_AVAILABLE=true` would have FAKED A GREEN over 54 skipped tests — now blocks | low; makes a wrong state loud |
| #13 | Admission boundary constraining the conformance CI trigger (R-SEC-02) + a fail-open fix in `ci/l0.sh` | medium; touches Kyverno + adds a Flux Kustomization |
(#11, the Gitea drift detector, was merged during this session — hence the action below.)
Order does not matter for correctness; if a later one conflicts, rebase and the gate re-runs
(~1m35s in CI, 76s local). **Each was gated in the pinned toolbox, not on a host.**
### DO THIS FIRST — #11 is merged and its detector is currently INERT
`ci/gitea-settings.sh` SKIPS until its token exists, and **a skipped detector detects nothing**:
./infra/gitea/set-ci-token.sh # mints + stores GITEA_SETTINGS_TOKEN, never prints it
Then open any PR and confirm the gate prints `gitea-settings: green`. ONLY THEN uncomment
`REQUIRE_GITEA_SETTINGS: "true"` in `.gitea/workflows/gate.yml`, so a future missing token blocks
instead of skipping. Doing that before seeing it green locks the repo.
Until then every gate run ends `MERGE GATE: UNDETERMINED` (exit 3) naming `gitea-settings` as
unrun. That is BY DESIGN and `gate.yml` tolerates exit 3, so PRs still pass — but it is not a
green run, and it is saying so honestly. The token is what turns it green.
### A DIFFERENT COMPUTER NEEDS THREE THINGS THAT ARE NOT IN GIT
This is the part that will silently waste an hour otherwise. None of these are in the repo, by
design, and without them the toolbox and the cluster are unreachable:
1. **`~/.config/sops/age/keys.txt`** — the age PRIVATE key. Without it NO `*.sops.yaml` can be
read or edited, and `sops` will fail in a way that looks like a broken file. Copy it from this
workstation; the public recipient in `.sops.yaml` is only half the pair.
2. **`infra/ansible/.cache/prod.kubeconfig`** — untracked, so it exists only in the main checkout
on this machine. Without it `dev.sh` refuses to start. In a git WORKTREE it is absent even
here: use `DEV_KUBECONFIG=/dev/null` for anything that needs no cluster (see `CLAUDE.md` §3).
3. **A Gitea credential** for `gitea.home.keithjackson.dev` (this session used the stored
`~/.git-credentials`). `git push` and every `infra/gitea/*.sh` script needs it.
### STATE, MEASURED — do not re-derive it from this log, re-measure it
`CLAUDE.md` §2b exists because this log was confidently wrong for a month. Verify before relying:
protected:true user_can_push:false enable_status_check:true
status_check_contexts:["gate / gate (pull_request)"] required_approvals:1
block_admin_merge_override:true runner incluster-dind: online gate ~1m35s
### NEXT WORK, in priority order
1. **Finish the L2/L3 wiring** now that the credential model is decided and #13 lands the
boundary. Still missing (`docs/plans/l2-l3-conformance-ci-wiring.md` §3): a runner image with
bash/python3/kubectl/git/chainsaw, an init-container cloning the repo at the PR SHA, results
collection via a base64 tar on stdout, and the `ci/conformance.sh` rewrite to drive the Job.
**Tighten the admission policy's image pattern to the literal digest when that image exists.**
2. **Do NOT set `vars.CLUSTER_AVAILABLE=true`** until step 1 is done. It does not enable L2/L3; it
only changes which way the stage fails (see #12).
3. The `gitea-settings` detector covers branch protection + runners. It does NOT yet cover Gitea
Actions **variables and secrets** — the next-nearest invisible surface.
4. `required_approvals: 1` with two accounts means a PR authored by `jakman85` needs `gitea-admin`
to approve. The operator holds ultimate authority and does not require that review; revisit if
it becomes friction.
### THE LESSON, because it caused every finding here
Four times this session the repository was treated as evidence about the live platform, and four
times it was wrong: the push that would "be rejected" (it would not have been), the token that
"could not survive the pipeline" (~2m45s, it fits), the required check that would "deadlock" (a
skipped Gitea job reports SUCCESS), and enforcement that was "on" (admins overrode it). Two
fail-opens in the gate itself were found the same way — `kyverno test` certifying policies that do
not load, and `CLUSTER_AVAILABLE` certifying 54 skipped tests as passed.
**Measure, then assert. `CLAUDE.md` §2b is the rule; this entry is the evidence for it.**
## 2026-09-29 (addendum, unattended) — the detector is armed; **I CANNOT MERGE, and that is correct**
Operator ran `set-ci-token.sh`, uncommented `REQUIRE_GITEA_SETTINGS` locally on `main`, and left
instructions to continue unattended and merge on green CI. Two things came out of that.
### 1. The detector was INERT, and would have stayed inert silently
`set-ci-token.sh` PUT the secret as `GITEA_SETTINGS_TOKEN`. **Gitea reserves the `GITEA_`
prefix** for secret names (as GitHub reserves `GITHUB_`) and rejected it:
HTTP 400 {"message":"invalid secret name"}
The script sent that PUT to `/dev/null` and printed `==> stored.` unconditionally; `curl -sS`
exits 0 on an HTTP error. **A token existed; no secret did.** So `ci/gitea-settings.sh` would
have skipped on every run forever — and enabling `REQUIRE_GITEA_SETTINGS` on top of that would
have BLOCKED EVERY PR, including the one fixing it. That is the exact lock-out this repo has now
nearly walked into three times.
Fixed in **PR #15**: the secret is `SETTINGS_API_TOKEN`, and `set-ci-token.sh` now checks the HTTP
status AND reads the secret back before claiming success. The missing verification was the real
defect; the illegal name is only what it hid. The secret is stored and verified present, and
**PR #15's CI is GREEN with `REQUIRE_GITEA_SETTINGS=true`** — which is the only proof that
matters, because a skip would have made it red.
The operator's local edit could not be pushed to `main` (protected). It is carried in PR #15.
### 2. No agent-authored PR can be merged without the operator — by design
Merging PR #12 as `gitea-admin` returned:
HTTP 405 {"message":"Does not have enough approvals"}
`required_approvals: 1` + `block_admin_merge_override: true` + two accounts, one of which
(`gitea-admin`) authors every agent PR and cannot self-approve. **So "merge on green CI" is not
something an agent can execute here.** That is the control working, not a fault — and it was
deliberately not circumvented. Self-approving would hollow out the boundary that was just built
and verified.
**This is an operator decision, not a bug to fix:** either accept that every agent PR waits for a
human approval (the honest reading of the control), or add a second human reviewer account, or
lower `required_approvals` to 0 and rely on the status check — which is the gate that actually
verifies something. Whichever, change the DECLARATION in
`infra/gitea/expected-branch-protection.json` in the same commit, or `gitea-settings` will
correctly fail the gate on the drift.
### Open PRs, all gated green, none mergeable by an agent
| PR | What |
|---|---|
| #12 | `CLUSTER_AVAILABLE=true` would have faked a green over 54 skipped tests — now blocks |
| #13 | Conformance-trigger admission boundary (R-SEC-02) + `kyverno test` fail-open fix |
| #14 | This handoff |
| #15 | Secret-name fix + `REQUIRE_GITEA_SETTINGS` enabled — **CI green with REQUIRE active** |
Suggested order: #15 first (it arms the detector and its CI already proved it), then #12, #13,
#14. If a later one conflicts, rebase; the gate re-runs in ~1m36s.
### Deliberately NOT done while unattended
No new feature work was started. Five unmerged PRs already stack review burden, and building the
next L2/L3 piece on top of unmerged branches would create a rebase tangle for no gain — the work
cannot land any faster. The next piece is specified and ready to start once these are merged:
`docs/plans/l2-l3-conformance-ci-wiring.md` §3, with §4's credential model now decided and its
boundary landed in #13.
+3
View File
@@ -32,6 +32,9 @@ render-golden: ## render==golden: canonical render reproduces committed apps/ (n
l1: ## L1 rendering checks (no cluster; live now)
@"$(CI_DIR)l1.sh"
gitea-settings: ## Gitea branch protection + runners match infra/gitea/expected-branch-protection.json (SKIPS w/o GITEA_SETTINGS_TOKEN)
@./gitea-settings.sh
conformance: ## L2/L3 in-cluster Chainsaw Job (SKIPS w/o CLUSTER_AVAILABLE=true)
@"$(CI_DIR)conformance.sh"
+40
View File
@@ -61,5 +61,45 @@ if [ -d "${CONF_ART}/coverage" ]; then
cp -r "${CONF_ART}/coverage/." "${COVERAGE_DIR}/"
fi
# ---------------------------------------------------------------------------------------
# DID IT ACTUALLY RUN? -- the guard without which this stage FAKES A GREEN.
#
# run.sh exits 0 in "scaffold mode" when it cannot reach a cluster: it emits a JUnit file
# whose every testcase is <skipped> and returns success. That is correct FOR run.sh -- an
# honest skip is its documented no-cluster behaviour. It is catastrophic HERE, because
# reaching this point means the environment DECLARED CLUSTER_AVAILABLE=true, and this stage
# would otherwise print "L2/L3 conformance passed" over 54 tests that never executed.
#
# MEASURED 2026-09-29, before this guard existed:
# CLUSTER_AVAILABLE=true ./ci/conformance.sh -> rc=0
# "[junit] wrote skipped JUnit for 54 scaffolded test(s)"
# "[ ok ] L2/L3 conformance passed for touched subset"
# One Gitea variable away from a fake green on an enforcing merge gate. That is exactly the
# HARD RULE 4 violation this repo's whole gate design exists to prevent, and nothing caught
# it because nobody had ever set the variable.
#
# A skip while a cluster is DECLARED is a contradiction: either the declaration is wrong or
# the cluster is unreachable. The merge-gate decision matrix already says this case is BLOCK
# ("conformance skipped while CLUSTER_AVAILABLE=true -> BLOCK (it should have run)"). This
# makes the stage itself honour that instead of reporting a pass.
JUNIT="${CONF_ART}/report.junit.xml"
if [ -f "$JUNIT" ]; then
total="$(grep -o '<testcase' "$JUNIT" | wc -l | tr -d ' ')"
skipped="$(grep -o '<skipped' "$JUNIT" | wc -l | tr -d ' ')"
if [ "${total:-0}" -gt 0 ] && [ "${skipped:-0}" -ge "${total:-0}" ]; then
err "CLUSTER_AVAILABLE=true, but the L2/L3 suite SKIPPED every one of its ${total} tests"
err " (scaffold mode -- run.sh could not reach a cluster and exited 0)."
err " This stage will NOT report that as a pass. Either the cluster is unreachable from"
err " this runner, or CLUSTER_AVAILABLE is set when it should not be."
err " NOTE: the in-cluster trigger path is still UNIMPLEMENTED -- see"
err " docs/plans/l2-l3-conformance-ci-wiring.md. Setting CLUSTER_AVAILABLE=true does not"
err " by itself make L2/L3 run; it only makes this stage attempt a LOCAL run that the CI"
err " runner has no credential for (automountServiceAccountToken: false, by design)."
summary "- conformance-l2-l3: **BLOCKED — declared available but every test skipped** (not a pass)"
exit 1
fi
log "L2/L3 executed: ${total} testcase(s), ${skipped} skipped"
fi
ok "L2/L3 conformance passed for touched subset"
summary "- conformance-l2-l3: **passed**"
+26
View File
@@ -112,6 +112,7 @@ run_stage lint "${CI_DIR}/lint.sh"
run_stage l0 "${CI_DIR}/l0.sh"
run_stage render-golden "${CI_DIR}/render-golden.sh"
run_stage l1 "${CI_DIR}/l1.sh"
run_stage gitea-settings "${CI_DIR}/gitea-settings.sh"
run_stage conformance "${CI_DIR}/conformance.sh"
echo >&2
@@ -188,6 +189,31 @@ case "${OUTCOME[spec-immutability]}" in
pass) ok "spec-immutability: green" ;;
esac
# gitea-settings: the ONLY check that can see Gitea's own database (branch protection +
# runner registration). A FAIL is real drift and always blocks -- that is the whole point.
# A SKIP means no token or unreachable Gitea, which is an UNWIRED CREDENTIAL, not a defect
# in the change under test. It is deliberately NOT in the strict-skips loop below: making an
# absent secret block every PR is precisely the lock-yourself-out failure this repo keeps
# walking into (a required check with nothing able to satisfy it). So a skip denies a PASS
# (exit 3) and says loudly what is unverified. Set REQUIRE_GITEA_SETTINGS=true once the
# token is wired, and the skip becomes a block as it should.
case "${OUTCOME[gitea-settings]}" in
fail) err "gitea-settings: Gitea DRIFTED from infra/gitea/expected-branch-protection.json -> blocks merge"; BLOCK=1 ;;
skip)
if truthy "${REQUIRE_GITEA_SETTINGS:-false}"; then
err "gitea-settings: skipped and REQUIRE_GITEA_SETTINGS=true -> blocks merge"
summary "- gitea-settings: **SKIPPED** (no token) — blocks merge (REQUIRE_GITEA_SETTINGS=true)"
BLOCK=1
else
warn "gitea-settings: skipped (no GITEA_SETTINGS_TOKEN) — branch protection and runner"
warn " registration are UNVERIFIED. Both were silently wrong for a month. Wire the token"
warn " (infra/gitea/set-ci-token.sh), then set REQUIRE_GITEA_SETTINGS=true."
summary "- gitea-settings: **SKIPPED** (no token — protection/runners unverified; not a pass)"
NOT_ESTABLISHED+=("gitea-settings")
fi ;;
pass) ok "gitea-settings: green" ;;
esac
# L0/L1 stages: any failure blocks. A SKIP here means a pinned tool is missing or a seam is
# unwired — NEVER green. In CI (strict) that is a defect and blocks; on a workstation it
# degrades to UNDETERMINED, which is still non-zero and still not a pass. render-golden is a
+133
View File
@@ -0,0 +1,133 @@
#!/usr/bin/env bash
# ci/gitea-settings.sh — Stage: the Gitea platform settings that gate this repo still
# match what git declares.
#
# WHY THIS STAGE EXISTS AT ALL
# ----------------------------
# Every other check in this repo reads the repository, the rendered manifests, or the
# cluster. NONE of them can see Gitea's own database — and that is where branch
# protection and runner registration live. On 2026-09-29 that blind spot was measured:
#
# * `main` had NO branch protection, and
# * ZERO Actions runners were registered,
#
# for roughly a month, while DECISIONS-LOG.md recorded both as working and every agent
# and human in the loop repeated that claim because the repository said so. PR #1 — the
# first pull request this repo ever had — sat `pending` for 30.6 minutes with nothing to
# execute it. Later the same day `enable_status_check` and `required_approvals` were turned
# on and STILL did not bite, because `block_admin_merge_override` was false; PRs #5-#9
# merged with zero approvals recorded against `required_approvals: 1`.
#
# Three separate failures, one shape: a setting that READ as enforced while not being, with
# no mechanism anywhere that could notice. Scripts that ASSERT the desired state
# (infra/gitea/*.sh) do not detect drift from it. This stage is the detector.
#
# WHAT IT ASSERTS
# 1. every key in infra/gitea/expected-branch-protection.json `.protection` matches the
# LIVE rule at /repos/{repo}/branch_protections/{branch}, exactly;
# 2. at least `.runners_online_min` Actions runners report status "online".
#
# (2) is not decoration. A required status check with no runner is the deadlock this repo
# nearly walked into: every PR pends forever, including the one that would fix it.
#
# READ THE RULE OBJECT, NOT THE BRANCH VIEW. /repos/{repo}/branches/{branch} reports
# `protected`, `enable_status_check` and `required_approvals` but NOT
# `block_admin_merge_override` — it showed all-green while the rule was overridable by the
# only account that merges. This stage queries /branch_protections/ deliberately.
#
# EXIT CODES (the gate contract)
# 0 live Gitea matches the declaration
# 1 DIVERGENCE — something drifted, or was never applied. Blocks the merge.
# 3 SKIP — no token, or Gitea unreachable. Never a pass; see gate.sh, which treats this
# skip like spec-immutability's (UNDETERMINED, exit 3) rather than a hard block, so
# that an unwired credential cannot lock the repository.
#
# TO WIRE IT: create a Gitea repo secret SETTINGS_API_TOKEN (a token that can read repo
# settings) — infra/gitea/set-ci-token.sh does it. Until then this stage skips LOUDLY and
# the drift it exists to catch is still undetected.
#
# NOT `GITEA_SETTINGS_TOKEN`. This comment named that until 2026-09-29 and it instructed the
# reader to do something IMPOSSIBLE: Gitea reserves the `GITEA_` prefix for secret names and
# rejects the PUT with HTTP 400 "invalid secret name". That is not a style rule — it is why the
# detector sat inert for hours while a token existed and no secret did. The env var below still
# accepts GITEA_SETTINGS_TOKEN, but only for a LOCAL run: it can never be the CI wiring, because
# the secret that would feed it cannot be created at all.
set -euo pipefail
CI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "${CI_DIR}/.." && pwd)"
# shellcheck disable=SC1091
source "${CI_DIR}/lib.sh"
log "Stage: Gitea platform settings match the declaration"
DECL="${REPO_ROOT}/infra/gitea/expected-branch-protection.json"
GITEA_URL="${GITEA_URL:-https://gitea.home.keithjackson.dev}"
GITEA_REPO="${GITEA_REPO:-Home-Network/home-network-k3s-impl}"
# SETTINGS_API_TOKEN, not GITEA_SETTINGS_TOKEN. Gitea RESERVES the `GITEA_` prefix for
# secret names and rejects a PUT of one with HTTP 400 "invalid secret name" — the same way
# GitHub reserves `GITHUB_`. The original name could therefore never be stored, and because
# infra/gitea/set-ci-token.sh sent that PUT to /dev/null and printed "stored" unconditionally,
# it looked like it had worked. A token was minted, no secret existed, and this stage would
# have skipped forever. GITEA_SETTINGS_TOKEN is still honoured for a local run.
TOKEN="${SETTINGS_API_TOKEN:-${GITEA_SETTINGS_TOKEN:-}}"
[ -f "$DECL" ] || die "declaration missing: ${DECL}"
require_tool jq
require_tool curl
if [ -z "$TOKEN" ]; then
skip "SETTINGS_API_TOKEN is not set — cannot read the live Gitea rule."
skip " Branch protection and runner registration are therefore UNVERIFIED. They were"
skip " wrong, unnoticed, for a month (2026-09-29). Wire it: infra/gitea/set-ci-token.sh"
summary "- gitea-settings: **SKIPPED — no token** (protection + runners UNVERIFIED; not a pass)"
exit 3
fi
api() { curl -sS --max-time 20 -H "Authorization: token ${TOKEN}" "$@"; }
BRANCH="$(jq -r '.branch' "$DECL")"
LIVE="$(api "${GITEA_URL}/api/v1/repos/${GITEA_REPO}/branch_protections/${BRANCH}" || true)"
if [ -z "$LIVE" ] || ! jq -e 'type == "object" and has("branch_name")' >/dev/null 2>&1 <<<"$LIVE"; then
skip "could not read the branch-protection rule from ${GITEA_URL} (unreachable, or the token"
skip " lacks permission). Not treating this as a pass."
summary "- gitea-settings: **SKIPPED — Gitea unreadable** (not a pass)"
exit 3
fi
RC=0
# --- 1. every declared protection key must match the live rule, exactly ---------------
while IFS= read -r key; do
want="$(jq -Sc --arg k "$key" '.protection[$k]' "$DECL")"
got="$(jq -Sc --arg k "$key" 'if has($k) then .[$k] else "<<ABSENT>>" end' <<<"$LIVE")"
if [ "$want" = "$got" ]; then
ok " ${key} = ${got}"
else
err " ${key}: DECLARED ${want} but LIVE is ${got}"
RC=1
fi
done < <(jq -r '.protection | keys[]' "$DECL")
# --- 2. a runner must actually be online ----------------------------------------------
MIN="$(jq -r '.runners_online_min // 1' "$DECL")"
RUNNERS="$(api "${GITEA_URL}/api/v1/admin/actions/runners" || true)"
N="$(jq -r '[(.runners//[])[]|select(.status=="online")]|length' <<<"${RUNNERS:-{\}}" 2>/dev/null || echo 0)"
if [ "${N:-0}" -ge "$MIN" ]; then
ok " runners online: ${N} (>= ${MIN})"
else
err " runners online: ${N}, DECLARED minimum ${MIN}."
err " A required status check with no runner leaves every PR pending forever —"
err " including the PR that would fix it. This is the 2026-09-29 outage condition."
RC=1
fi
if [ "$RC" -eq 0 ]; then
ok "Gitea settings match the declaration."
summary "- gitea-settings: **green** (branch protection + runners match \`infra/gitea/expected-branch-protection.json\`)"
else
err "Gitea has DRIFTED from infra/gitea/expected-branch-protection.json."
err " Re-assert it (infra/gitea/require-gate.sh, block-admin-override.sh), or if the live"
err " state is what you now want, change the DECLARATION in the same commit — deliberately."
summary "- gitea-settings: **DRIFT** — live Gitea does not match the declaration; blocks merge"
fi
exit "$RC"
@@ -0,0 +1,42 @@
# =============================================================================
# L0 STATIC — every authored admission policy is actually APPLIED (R-SEC-01, R-SEC-02)
# Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster.
#
# WHY THIS EXISTS. `kyverno test` proves a policy is CORRECT. Nothing proved it was DEPLOYED.
# platform/kyverno/ holds three lists and only platform-config/kustomization.yaml applies
# anything; the other two are a reference copy and a namespace+HelmRelease render root. On
# 2026-09-29 the R-SEC-02 boundary policy was added to the reference copy: it parsed, its
# kyverno test passed, the gate passed, Flux reported the tier Ready at the new revision — and
# the cluster ran the conformance-trigger ServiceAccount with nothing bounding it.
#
# WHOLE-REPO SCOPE, DELIBERATELY. The failure mode is a policy that is present but wired
# nowhere, which by definition touches no requirement the diff attributes. A path-scoped run
# would have skipped this check on the very commit that introduced the defect.
# =============================================================================
apiVersion: chainsaw.kyverno.io/v1alpha1
kind: Test
metadata:
name: admission-policies-actually-applied
labels:
conformance.home.k3s/R-SEC-01: "true"
conformance.home.k3s/R-SEC-02: "true"
annotations:
conformance.home.k3s/requirements: "R-SEC-01,R-SEC-02"
conformance.home.k3s/layer: "L0"
conformance.home.k3s/scope: "whole-repo"
conformance.home.k3s/safety-critical: "false"
conformance.home.k3s/executable: "true"
conformance.home.k3s/intent: >-
Every ClusterPolicy authored under platform/kyverno/ is either referenced by
platform-config/kustomization.yaml (so Flux applies it) or is a documented CI-only
exemption that is genuinely not applied — so a policy cannot be authored, tested,
green, and absent from the cluster.
spec:
steps:
- name: every-authored-policy-is-applied-or-documented-ci-only
description: >-
Enumerate the ClusterPolicy files under platform/kyverno/ and assert each is wired
into the Kustomization that actually applies Kyverno CRs, or exempted with a reason.
try:
- script:
content: bash ./check.sh
@@ -0,0 +1,120 @@
#!/usr/bin/env bash
# =============================================================================
# L0 static check (offline, no cluster) — admission-policies-actually-applied
# Serves: R-SEC-01, R-SEC-02 (admission enforcement is only real if it is APPLIED)
#
# WHY THIS EXISTS — a policy that ships green and never reaches the cluster.
#
# platform/kyverno/ holds THREE lists, and only one of them applies anything:
# 1. platform/kyverno/admission/kustomization.yaml — reference only. Its own sibling says
# "kept as reference; THIS file is the buildable render root".
# 2. platform/kyverno/kustomization.yaml — that render root: namespace +
# HelmRelease ONLY. It explicitly notes the ClusterPolicies are applied elsewhere.
# 3. platform-config/kustomization.yaml — THE ONE. Kyverno CRs go here because
# they need the CRDs from the HelmRelease in (2), so they reconcile a tier later.
#
# On 2026-09-29 a ClusterPolicy (conformance-trigger-job-shape, the R-SEC-02 boundary bounding
# what the conformance CI trigger may create) was added to (1). Everything went green: the file
# parsed, `kyverno test` exercised it BY PATH and passed, the gate passed, Flux reported
# `platform` Ready at the new revision — and the policy was never in any applied set. The
# cluster ended up running the conformance-trigger ServiceAccount with NOTHING bounding it,
# which is precisely the state that identity's own RBAC header says must never exist.
#
# Nothing could have caught it. `kyverno test` proves a policy is CORRECT, not that it is
# DEPLOYED. This check asserts the second thing.
#
# GIVEN the repository as git tracks it,
# WHEN every file under platform/kyverno/ that declares a ClusterPolicy is enumerated,
# THEN each one is either referenced by platform-config/kustomization.yaml (so Flux applies
# it), or named in CI_ONLY below with a reason — and CI_ONLY itself may not name a
# policy that IS applied, so the exemption list cannot rot into a lie either way.
# =============================================================================
set -euo pipefail
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT="$(cd "${DIR}/../../../.." && pwd)" # repo root
exec python3 - "$ROOT" <<'PY'
import sys, subprocess
from pathlib import Path
ROOT = Path(sys.argv[1])
FAILURES, CHECKS = [], 0
def check(cond, desc):
global CHECKS
CHECKS += 1
if not cond:
FAILURES.append(desc)
# Policies authored for the CI L0 gate that are deliberately NOT applied in-cluster. Each needs
# a reason, and each is asserted BELOW to really be absent from the applied set — an exemption
# that silently starts being applied is as much a lie as one that silently stops.
CI_ONLY = {
"disallow-plaintext-secrets":
"CI-only by design: it requires a top-level `sops` block, which is IMPOSSIBLE at "
"admission because Flux decrypts before the Secret reaches the API server. Its "
"in-cluster counterpart is admission/secret-provenance.yaml.",
}
tracked = subprocess.run(["git", "ls-files", "-z", "--", "platform/kyverno/**/*.yaml",
"platform/kyverno/*.yaml"],
cwd=ROOT, capture_output=True, text=True, check=True)
files = [ROOT / p for p in tracked.stdout.split("\0") if p]
# A ClusterPolicy file, identified without a YAML parse of every doc: cheap and sufficient.
policies = {}
for f in files:
try:
txt = f.read_text()
except Exception:
continue
if "kind: ClusterPolicy" not in txt:
continue
for line in txt.splitlines():
st = line.strip()
if st.startswith("name:") and " name:" in line:
policies[f.relative_to(ROOT).as_posix()] = st.split("name:", 1)[1].strip()
break
check(len(policies) >= 6,
f"(0) the scan found {len(policies)} ClusterPolicy file(s) under platform/kyverno/ — "
f"too few means the scan broke, and every assertion below would be vacuous")
applied_txt = (ROOT / "platform-config" / "kustomization.yaml").read_text()
unapplied, wrongly_exempt = [], []
for rel, name in sorted(policies.items()):
# platform-config references these as ../platform/kyverno/...
referenced = rel in applied_txt or rel.replace("platform/", "../platform/", 1) in applied_txt
if name in CI_ONLY:
if referenced:
wrongly_exempt.append(f"{name} ({rel}) is listed CI_ONLY but IS applied by platform-config")
elif not referenced:
unapplied.append(f"{name} ({rel})")
check(not unapplied,
f"(1) every in-cluster ClusterPolicy is referenced by platform-config/kustomization.yaml "
f"— an unreferenced policy is authored, tested, green, and NEVER APPLIED {unapplied if unapplied else ''}")
check(not wrongly_exempt,
f"(2) no CI_ONLY exemption names a policy that IS applied — a stale exemption hides a "
f"real deployment {wrongly_exempt if wrongly_exempt else ''}")
for name, reason in CI_ONLY.items():
check(bool(reason.strip()),
f"(3) CI_ONLY['{name}'] carries a reason — an unexplained exemption is how this "
f"list stops being reviewable")
check(CHECKS >= 4,
f"only {CHECKS} assertions ran — the check degenerated to a near-no-op")
if FAILURES:
print(f"CHECK FAIL (admission-policies-actually-applied): {len(FAILURES)} of {CHECKS} assertions failed",
file=sys.stderr)
for f in FAILURES:
print(f" - {f}", file=sys.stderr)
raise SystemExit(1)
print(f"check OK: admission-policies-actually-applied — {CHECKS} assertions; "
f"{len(policies)} ClusterPolicy file(s), {len(policies) - len(CI_ONLY)} applied by "
f"platform-config, {len(CI_ONLY)} CI-only with reasons. Proves policies are DEPLOYED, "
f"not that they are correct — `kyverno test` is what proves correctness.")
PY
+15 -2
View File
@@ -38,8 +38,21 @@ agents. So the credential model must be decided by the operator before anything
| Runner deployment | `apps/wave-1/gitea-actions.yaml` | act_runner + DinD (Docker executor). **`automountServiceAccountToken: false`** — the runner and its job containers hold NO SA token. Broad egress allowed (`- {}`), so it can *reach* the kube-API VIP `192.168.1.49:6443`, but has no credential. |
| Secrets encryption | `.sops.yaml` (repo root) | age recipient `age175l8m…mfzg`; encrypts `data`/`stringData` of `*.sops.yaml` under `platform|apps|catalog/**` and `secrets/**`. Private key is the in-cluster `sops-age` Secret Flux uses. |
**Cluster reality:** the "Phase 0 / no cluster" language in these files is stale — the dev cluster
is LIVE (Phase 6). So `CLUSTER_AVAILABLE` is the only remaining gate, exactly as the blueprint says.
**Cluster reality:** the "Phase 0 / no cluster" language in these files is stale — the cluster is
LIVE (and it is now PROD; dev was decommissioned 2026-09-12).
> **CORRECTION (2026-09-29): `CLUSTER_AVAILABLE` is NOT "the only remaining gate", and setting it
> today is actively DANGEROUS.** Measured: with `CLUSTER_AVAILABLE=true` and no reachable cluster,
> `conformance/run.sh` exits 0 in scaffold mode having marked all 54 tests `<skipped>`, and
> `ci/conformance.sh` then printed `[ ok ] L2/L3 conformance passed for touched subset` — **rc 0, a
> fake green**, on a merge gate that is now an enforcing required status check. The CI runner has
> no cluster credential at all (`automountServiceAccountToken: false`, by design), so the "local
> run" this stage attempts can never succeed; it can only skip and, before the guard added on
> 2026-09-29, lie about it.
>
> `ci/conformance.sh` now BLOCKS (exit 1) when a cluster is declared but every test skipped. **Do
> not set `CLUSTER_AVAILABLE=true` until §3's trigger path actually exists** — the variable does
> not enable L2/L3, it only changes which way this stage fails.
---
+32 -4
View File
@@ -86,6 +86,32 @@ enabling enforcement and probing the push. `require-gate.sh` is the enforcement
Re-run `verify-then-enforce.sh` after any future change to the act-runner Deployment.
### `expected-branch-protection.json` + `ci/gitea-settings.sh` — the detector
`expected-branch-protection.json` **declares** the desired rule. `ci/gitea-settings.sh` runs as
a merge-gate stage, reads the LIVE rule, and **fails the gate on any divergence** — plus asserts
at least one runner is online. It is the only thing in this repository that can see Gitea's own
database.
It reads `/branch_protections/main`, not `/branches/main`: the branch view does **not** expose
`block_admin_merge_override`, and it showed all-green for hours while the rule was overridable.
Behaviour, proven by test rather than assumed:
| Situation | Exit | Effect |
|---|---|---|
| live matches the declaration | 0 | green |
| any key drifted, or no runner online | 1 | **blocks the merge** |
| no `GITEA_SETTINGS_TOKEN`, or Gitea unreachable | 3 | skips **loudly**; never a pass, never a block |
The skip is deliberately *not* a block: an unwired credential blocking every PR is the
lock-yourself-out failure this repo keeps hitting. Wire the token with `set-ci-token.sh`, confirm
the stage goes **green on a real PR**, and only then set `REQUIRE_GITEA_SETTINGS=true` in
`gate.yml` so a future missing token blocks instead of skipping.
**Changing the declaration is how you change policy.** If the live state is what you now want,
edit this JSON in the same commit — deliberately, in review — rather than letting the two drift.
### Why these scripts exist at all, and what is still missing
Gitea's repo settings live in **its own database**, not in anything Flux reconciles or the
@@ -94,10 +120,12 @@ the root cause of the entire 2026-09-29 session: branch protection was off and t
gone for a month while the log recorded both as working, and **no check existed that could
have caught it.**
These scripts encode the desired state and can re-assert it, but they are not a detector. The
durable fix is a conformance check that reads the live Gitea API and fails when it diverges
from what is declared here — turning a class of silent, invisible failure into a loud one.
**That check does not exist yet and is the highest-value remaining work in this directory.**
These scripts encode the desired state and can re-assert it; `ci/gitea-settings.sh` above is the
**detector** that turns divergence from a silent, invisible failure into a loud one. It closes the
root cause of the 2026-09-29 session.
It is only as good as its wiring: **until `GITEA_SETTINGS_TOKEN` exists it skips**, and a skipped
detector detects nothing. That is the remaining step.
## Historical sequence (kept for context)
@@ -0,0 +1,20 @@
{
"_comment": "DECLARED desired state of Gitea branch protection on main. ci/gitea-settings.sh fails the merge gate when the LIVE Gitea diverges from this. Gitea settings live in Gitea database - invisible to Flux, unreadable by every other check - so this file plus that check are the only drift detection that exists for them.",
"_generated": "2026-09-29 from the live rule, after the session that found protection OFF and zero runners for a month",
"branch": "main",
"runners_online_min": 1,
"protection": {
"enable_push": false,
"enable_push_whitelist": false,
"enable_merge_whitelist": false,
"enable_status_check": true,
"status_check_contexts": [
"gate / gate (pull_request)"
],
"required_approvals": 1,
"enable_approvals_whitelist": false,
"block_admin_merge_override": true,
"block_on_rejected_reviews": true,
"dismiss_stale_approvals": true
}
}
+56
View File
@@ -0,0 +1,56 @@
#!/usr/bin/env bash
# Mint a Gitea API token and store it as the repo Actions secret GITEA_SETTINGS_TOKEN,
# so ci/gitea-settings.sh can read the live branch-protection rule during the gate.
#
# The token value is NEVER printed. It goes from the token API into the secrets API and
# nowhere else -- not to stdout, not to a file, not into an agent transcript.
#
# Scope: `read:repository` + `read:admin` are what the check needs (the rule object, and the
# runner list). Gitea token scopes are coarse; grant the least that works on your version and
# confirm the check goes green rather than assuming the scope name.
set -euo pipefail
HOST=gitea.home.keithjackson.dev; OWNER=Home-Network; REPO=home-network-k3s-impl
# NOT GITEA_* — Gitea reserves that prefix for secret names and returns HTTP 400
# "invalid secret name". The first version of this script used GITEA_SETTINGS_TOKEN, sent the
# PUT to /dev/null, and printed "stored" regardless; the secret never existed and nobody knew.
SECRET=SETTINGS_API_TOKEN
NAME="ci-gitea-settings-$(date +%Y%m%d%H%M%S)"
creds="$(printf 'protocol=https\nhost=%s\n\n' "$HOST" | git credential fill)"
U="$(sed -n 's/^username=//p' <<<"$creds")"; P="$(sed -n 's/^password=//p' <<<"$creds")"
basic() { curl -sS --config <(printf 'user = "%s:%s"\n' "$U" "$P") "$@"; }
echo "==> minting token '${NAME}' as ${U}"
TOKEN="$(basic -X POST -H 'Content-Type: application/json' \
-d "{\"name\":\"${NAME}\",\"scopes\":[\"read:repository\",\"read:admin\"]}" \
"https://${HOST}/api/v1/users/${U}/tokens" | jq -r '.sha1 // empty')"
[ -n "$TOKEN" ] || { echo "token creation failed (check scopes for your Gitea version)" >&2; exit 1; }
echo "==> got a ${#TOKEN}-character token (value not shown)"
echo "==> storing it as repo Actions secret ${SECRET}"
code="$(basic -o /tmp/set-ci-token.out -w '%{http_code}' -X PUT -H 'Content-Type: application/json' \
-d "$(jq -n --arg d "$TOKEN" '{data:$d}')" \
"https://${HOST}/api/v1/repos/${OWNER}/${REPO}/actions/secrets/${SECRET}")"
unset TOKEN
if [ "$code" != 201 ] && [ "$code" != 204 ]; then
echo " PUT failed: HTTP ${code} -- $(head -c 200 /tmp/set-ci-token.out)" >&2
rm -f /tmp/set-ci-token.out; exit 1
fi
rm -f /tmp/set-ci-token.out
# VERIFY BY READING IT BACK. curl -sS exits 0 on an HTTP error, so without this the script
# reports success for a store that never happened -- which is exactly what it did.
names="$(basic "https://${HOST}/api/v1/repos/${OWNER}/${REPO}/actions/secrets" \
| jq -rc '[.. | objects | select(has("name")) | .name] | unique')"
grep -q "$SECRET" <<<"$names" \
|| { echo " stored HTTP ${code} but ${SECRET} is NOT in ${names} -- refusing to claim success" >&2; exit 1; }
echo "==> stored and VERIFIED present: ${names}"
cat <<'NOTE'
Next:
1. .gitea/workflows/gate.yml already passes this secret to the gate step.
2. Open any PR and confirm the gate prints "gitea-settings: green" -- it SKIPS silently
without the token, so a green run is the only proof the wiring works.
3. THEN set REQUIRE_GITEA_SETTINGS=true in gate.yml, so a future missing token BLOCKS
instead of skipping. Do that only after you have seen it green, or you lock the repo.
NOTE
+8
View File
@@ -34,6 +34,14 @@ resources:
# policies, the next import is blocked in the gap.
- ../platform/kyverno/policies/label-cdi-worker-pods.yaml
- ../platform/kyverno/admission/secret-provenance.yaml
# R-SEC-02 — bounds what the conformance CI trigger may create. MUST be listed HERE: this
# Kustomization is what actually applies ClusterPolicies. Adding a policy only to
# platform/kyverno/admission/kustomization.yaml is a SILENT NO-OP — that file is reference
# only ("kept as reference; THIS file is the buildable render root" in
# platform/kyverno/kustomization.yaml, which itself ships only the namespace + HelmRelease).
# That is exactly how this policy shipped green and never reached the cluster on 2026-09-29,
# leaving the conformance-trigger identity live with nothing bounding it.
- ../platform/kyverno/admission/conformance-trigger-job-shape.yaml
# cilium: L2 LB IP pool + announcement policy (cilium.io CRDs already registered by Cilium)
- cilium/l2-ip-pool.yaml
- cilium/l2-announcement-policy.yaml