Merge remote-tracking branch 'origin/main' into sync-39-onto-main

This commit is contained in:
jakman85 committed 2026-10-02 01:31:02 -05:00
commit 61ea71cc35
12 files changed
+1112 -20

No files matched your search

@@ -0,0 +1,256 @@
---
date: 2026-10-01
branch: fix/admin-approval-override
title: Administrators can merge their own green PR — required_approvals 1 → 0, plus an attributed /approve-override for sensitive paths
---
## What changed
**The problem, measured first.** `required_approvals: 1` on `main` was a lock-out, not a
protection. Gitea refuses to let a pull request's author approve their own PR, and `jakman85` is
the **only** collaborator (permission `owner`). So no account in existence could approve a PR the
operator opened. Measured on the live API, 2026-10-01:
| | |
|---|---|
| PR #35 (`bugfix/arr-to-plex`, author `jakman85`) | `gate / gate (pull_request)` = **success, 1m41s** |
| its reviews | `[]` |
| live `required_approvals` | `1` |
| live `block_admin_merge_override` | `true` |
Green gate, zero reviews, no path to a merge — and the same was true of any PR that would have
fixed it.
It was also never the design this repo wrote down. `ci/approval-policy.sh`'s own header and
`ci/gate.sh:194` both already described the model as *"`required_approvals` drops to 0 and the
GATE decides"*, and `infra/gitea/require-gate.sh`'s closing note named this exact failure and
prescribed this exact remedy. `decisions/2026-09-30-flux-deploy-notification-inhibited.md` had
flagged the 1-vs-0 disagreement as unresolved and not that session's to settle. This settles it
at 0.
**Two layers were blocking, and both are addressed.**
*Layer 1 — Gitea branch protection.* `required_approvals: 1 → 0` in
`infra/gitea/expected-branch-protection.json`, with the rationale in the file, plus
`infra/gitea/allow-admin-self-merge.sh` to apply it to the live rule (it reads the target **from
the declaration**, so the script cannot drift from what the gate checks). `require-gate.sh` was
changed to 0 as well — left at 1 it would silently re-impose the lock-out on its next run.
The narrow choice matters: the *other* way to unblock an owner is
`block_admin_merge_override: false`, and that would have re-opened precisely what
`infra/gitea/block-admin-override.sh` was written to close (PRs #5–#9 merged with zero approvals
while the settings read as enforced). This change drops the approval **count** only.
`block_admin_merge_override` and `enable_status_check` both stay `true`, and the script asserts
both afterwards and fails loudly if the PATCH disturbed either. **A red gate remains unmergeable
by everyone, administrators included.**
*Layer 2 — `ci/approval-policy.sh`.* Sensitive paths (`ci/**`, `infra/gitea/**`, `CLAUDE.md`,
kyverno, rbac, sops, Flux wiring) still require a non-author `APPROVED` review, which the sole
administrator still cannot obtain. An administrator may now waive it by commenting on the PR:
/approve-override <reason>
- the directive must **start a line** (a quoted or mid-sentence mention does not count, or
repeating someone else's override would re-arm it);
- a **reason is mandatory** — a bare directive records that review was waived but not why;
- the commenter's permission is **re-checked live** against
`/collaborators/{login}/permission` when the gate runs, so an old comment from a demoted
account carries nothing;
- it is **invalidated by a new commit** (the comment must post-date the head commit), mirroring
Gitea's own `dismiss_stale_approvals`;
- the overrider, timestamp and reason are printed in the gate log and the job summary, and a
self-waiver by the PR's own author is called out as such.
Selection of the candidate comment is `ci/approval-override.py` — pure, offline, no authority of
its own — so it could be unit-tested; authorisation stays in the fail-closed stage.
`ci/tests/approval-override.test.sh` adds **23 assertions**, all passing, each constructing a
case the parser must refuse. Every new failure mode blocks: unreadable head-commit timestamp,
unreadable permission, parser error.
**The cost, stated rather than buried.** For an administrator, *"a change to the gate cannot be
gated by the gate"* is now **advisory, not structural**. That is unavoidable with one human — the
alternative is a repository its owner cannot change — and what is bought back is an attributed,
reasoned, commit-scoped record of each waiver instead of a silent bypass.
**The uncomfortable part, accepted with eyes open (operator decision).** Authorisation is by live
permission **alone**, with no allowlist of human accounts. The **agent account `gitea-admin` also
reports permission `owner`** on this repo (measured), so an agent *can* clear the sensitive-path
review on its own PR — and agents author most PRs here, so that is the common case, not an edge
case. I raised this before implementing; the operator chose permission-only. It is documented in
three places rather than left to be discovered: `approval-policy.json`
(`_known_consequence_agent_accounts`), the stage's header, and `CLAUDE.md`, which now tells
agents plainly **not** to use it on their own work. The narrowing path needs no code change —
add an `override_admins` list to `approval-policy.json` and the stage already requires membership
in addition to the permission check.
Also updated: `CLAUDE.md` §1 and the §2 enforcement table (the "approval required" row is now
explicitly ⬜ with the reason, rather than a stale ✅), and `infra/gitea/README.md` gains §6 and
marks its historical step 4 as reversed — do not re-apply.
**Second pass, 2026-10-02 — the override did not work the first time, and why.** The operator ran
the live PATCH (`required_approvals: 0` confirmed live, with `block_admin_merge_override` and
`enable_status_check` still `true`), merged PR #35, and posted `/approve-override Need to ensure
approval override work as expected` on PR #38. The gate re-ran (run 203) and **still blocked.**
The comment was not at fault: posted by `jakman85`, directive at the start of the line, reason
present, and 29 minutes **newer** than head commit `616a497`, so it cleared the staleness rule.
What failed was that **CI could not read it.** Run 203's log:
approval-override: COMMENTS_JSON is not a list
`SETTINGS_API_TOKEN` is minted `["read:repository", "read:admin"]` by
`infra/gitea/set-ci-token.sh`, and Gitea puts `/issues/{n}/comments` behind the **issue** scope. So
Gitea returned an error *object*, `jq -e .` accepted it as valid JSON, and the stage fell through
to "no approving review" — **a permission failure wearing the costume of a missing override.**
`/pulls/{n}/files` and `/pulls/{n}/reviews` are under `read:repository`, which is exactly why every
other part of the same stage worked and this one did not.
Three fixes, and the first is the one that mattered most:
1. **The diagnosis is no longer misleading.** `COMMENTS_JSON` is out of the must-parse-or-block
loop and is explicitly checked for being an *array*; when it is not, the stage prints Gitea's
own `.message`, names the missing scope, and says where to post the override instead. An
unreadable source can only cause a false BLOCK, never a false pass, so degrading it to a loud
warning is safe — and blocking on it was never the honest behaviour.
2. **The override is accepted from a PR REVIEW body as well as a comment**, so the hatch is not
hostage to one API scope. A review of type "Comment" works on the token CI has *today*. The
gate names which source it used. `PENDING` (unsubmitted, author-only draft) reviews are dropped.
3. **`set-ci-token.sh` now requests `read:issue`**, which makes the ordinary comment box work —
the thing anybody reaches for first. It takes effect only on a **re-run**: an existing token's
scopes cannot be widened in place.
`ci/tests/approval-override.test.sh` grows from 23 to **38 assertions**, covering the reviews
source, `PENDING` reviews, an error *object* from the comments API, "an unreadable comments source
must not suppress a valid review override", latest-wins across both sources, and that the
diagnostic actually quotes Gitea's message.
**One adjacent fix, deliberately in scope: `ci/tests/gate-decision.test.sh` was already broken.**
It hand-maintains a list of stub stage scripts that "must name EVERY stage `gate.sh` runs", and it
did not name `approval-policy` — added to `gate.sh` by PR #17 and never added here. `run_stage`
therefore got exit 127 from a missing file and recorded a FAILED stage in every case, so **15 of
50 assertions failed**, including every one expecting a PASS/UNDETERMINED headline. Measured on
pristine `origin/main` (`0c7e8dd`) before touching it, so the attribution is not a guess.
I fixed it because `make -C ci test` is the *only* runner for the suite this change adds — leaving
it red would have made the new tests unverifiable in practice. The file's own comment already
narrated this exact failure for `gitea-settings` and did not prevent the repeat, which says the
comment was not the fix. So the stub list is now **derived from `gate.sh`'s own `run_stage` lines**
and asserts it parsed at least 8 of them: a stage added to the gate can no longer be silently
missing from its test. `make -C ci test` goes from **FAILED to all suites passed**.
## What I verified
- **The lock-out is real, and PR #35 is the proof.** Queried live: head sha
`8a094c46`, statuses `gate / gate (pull_request) = success :: Successful in 1m41s`,
`/pulls/35/reviews` → `[]`, `/branch_protections/main` → `required_approvals: 1`. I read the
**rule object**, not `/branches/main`, per §2b.
- **`jakman85` is the only collaborator**: `/collaborators` → one entry. Permission `owner`.
- **`gitea-admin` also reports `owner`** — this is why the agent-account consequence above is a
statement of fact and not a hypothetical.
- **PR #35 touches no sensitive paths** (`apps/wave-0/plex.yaml`, `apps/wave-1/{radarr,sonarr}.yaml`,
`catalog/{radarr,sonarr}.yaml`, its decisions entry) — so `ci/approval-policy.sh` was never its
blocker. Layer 1 alone unblocks it.
- **API shapes verified before depending on them**, rather than assumed: `/git/commits/{sha}`
returns `.commit.committer.date` as RFC3339 **with an offset** (`2026-10-01T23:09:08-05:00`),
not a trailing `Z` — a parser handling only `Z` would have silently dropped every real
override, so there is a test for the offset form. PR conversation comments are under
`/issues/{n}/comments`; `/pulls/{n}/comments` returns diff comments only and would never have
seen an override.
- **`ci/tests/approval-override.test.sh`: 23 passed, 0 failed.** Covers the happy path, bare
directive, mid-sentence and `>`-quoted directives, stale and equal-timestamp overrides,
unparseable timestamps, missing login, invalid JSON, non-list JSON, latest-wins, offset
timestamps, and a configurable directive.
- `bash -n` clean on `ci/approval-policy.sh`, `infra/gitea/allow-admin-self-merge.sh`,
`infra/gitea/require-gate.sh`; `jq -e .` clean on both JSON declarations.
- **`gate-decision.test.sh`'s 15 failures are pre-existing, measured not assumed.** Extracted
`origin/main`'s `ci/` at `0c7e8dd` into a throwaway directory and ran the suite from there:
**FAIL, 15 failed / 35 passed**. After deriving the stub list from `gate.sh`: **PASS, 50
assertions**.
- **`make -C ci test` in the pinned toolbox: `ci/tests: all suites passed`, rc 0** — approval-override
**38**, gate-decision 50, touched-requirements-base 31, test-attribution 15, touched-requirements 26,
decisions-log pass. It was rc 2 (`ci/tests: FAILED`) before the stub-list fix.
- **The live flip worked, and the drift detector proved it.** Run 202 (before): `gitea-settings:
FAIL`, `required_approvals: DECLARED 0 but LIVE is 1`. Run 203 (after the operator ran the
PATCH): `gitea-settings: pass`, with `block_admin_merge_override = true` among the keys it
checked. So the narrow change held — the approval count moved and the gate's teeth did not.
- **The first override attempt failed for a measured reason, not a guessed one.** Run 203 log:
`approval-override: COMMENTS_JSON is not a list`. Token scopes read from `set-ci-token.sh`:
`[read:repository, read:admin]`. The same run successfully read `/pulls/38` and
`/pulls/38/files` (author and sensitive paths both resolved) and got past the reviews
validation, which is how I know `/pulls/{n}/reviews` IS readable with that token — from
behaviour, not from documentation.
- **The comment itself was valid.** `created_at` 2026-10-02T05:31:24Z vs head commit `616a497`
committer date 2026-10-02T00:02:20-05:00 → the comment is 29m04s newer, so it passed the
staleness rule. Ruling the comment out is what left the token scope as the only candidate.
- **`?limit=1000` is not the culprit.** Queried `/issues/38/comments` with `limit=1000`, `limit=50`
and no limit using my own credentials: `type=array` in all three. So the difference is the
credential, not the query.
- **The root cause is confirmed by Gitea, in its own words.** Run 206 (the fix in place):
`[warn] cannot read PR COMMENTS from Gitea — token does not have at least one of required
scope(s), required=[read:issue], token scope=read:admin,read:repository.` followed by the
instruction to post the override as a review. That is the diagnosis the first attempt should
have produced, and `gitea-settings: pass` in the same run. The stage still ends `FAIL`, correctly
— there is no valid override on the PR yet.
- **Toolbox merge gate** (`./infra/devcontainer/dev.sh run bash -c './ci/gate.sh'`), run on the
rebased branch: **rc 3, `MERGE GATE: UNDETERMINED`**. Green stages: spec-immutability,
decisions-log, lint, l0 (23 offline cases, 0 failures), render-golden including the new
`catalog/render.py --selftest`, l1 (3 cases), approval-policy (N/A offline). Skipped:
`gitea-settings` (no `SETTINGS_API_TOKEN` on this host) and `conformance` (no cluster).
**This is NOT a green run and I am not reporting it as one** — rc 3 is UNDETERMINED per the
CLAUDE.md §3 contract.
## What I did NOT verify
- **The override has still never been ACCEPTED by a real run.** This is the second attempt at
this bullet, which is itself the point: the first pass called the hatch unverified, and the
first real test found a defect that no amount of local testing would have (an API scope CI has
and I did not). What is now measured is that the parse → *reject* path works end to end and
diagnoses itself. The **accepting** branch — directive found → live permission lookup → exit 0 —
has never executed. Posting the override as a **review** on PR #38 is its first real test, and
I would not be surprised by a third round.
- **`read:issue` is the right scope name — Gitea said so itself, so this is no longer a guess.**
Run 206 printed Gitea's own message: `token does not have at least one of required scope(s),
required=[read:issue], token scope=read:admin,read:repository`. That both confirms the root
cause and names the string, which matters because this repo has been burned by a token *name*
that could not exist (`GITEA_SETTINGS_TOKEN`). What is **still** unverified is that the token
*mint* accepts it — `POST /users/{u}/tokens` with that scope has not been run. The review route
does not depend on it.
- **I did not verify that Gitea lets a PR's own author submit a `COMMENT`-type review.** Gitea
blocks author self-*approval*; "Comment" is expected to be allowed, and that expectation is now
load-bearing for the recommended route. It is one click to find out, and I deliberately did not
test it by posting an override myself: doing so as `gitea-admin` is precisely the agent
self-override that `CLAUDE.md` now forbids.
- **The whole re-run mechanism is manual.** `gate.yml` triggers on `pull_request` and `push: main`
only — there is no `issue_comment` trigger, so posting an override does **not** re-run the gate.
Re-run the job from the Actions UI, or close and reopen the PR (which fires `pull_request` and
leaves the head SHA alone). **Do not push an empty commit to re-trigger it:** that moves the head
commit and makes the override stale by the rule above, so it would have to be posted again. A
comment trigger was considered and not built — it would need to check out PR head code while the
job holds `SETTINGS_API_TOKEN`, which is the classic pwn-request shape and not worth it here.
- **I have not seen the gate still REFUSE a merge** now that approvals are 0. The claim that
`block_admin_merge_override: true` keeps a red gate unmergeable rests on Gitea's documented
behaviour and on the 2026-09-29 session, **not** on a provoked refusal in this one. That probe
is written out at the foot of `allow-admin-self-merge.sh` and should be done, exactly as the
push rejection was provoked rather than assumed.
- **Clock skew in the staleness check is unhandled.** Staleness compares the comment's
server-side `created_at` against the head commit's *committer* date, which comes from the
committer's machine. A commit dated in the future would reject legitimate overrides until real
time catches up (fail-closed, so loud, but annoying). Gitea's own `dismiss_stale_approvals`
compares SHAs instead; a comment cannot. Not hit here — PR #35's commit date is ~21 min behind
server time — and not fixed.
- **`ci/tests/approval-override.test.sh` does not run in the merge gate.** `ci/gate.sh` runs no
`ci/tests/*`; they run only via `make -C ci test`. So a regression in the parser would not be
caught by CI. PR #33 (merged while this branch was being written, and rebased onto here) fixed
the *same class* of gap for `catalog/render.py --selftest` by invoking it from
`ci/render-golden.sh` — it did **not** wire `ci/tests/*` in, which I confirmed after rebasing
rather than assuming. Wiring them is a one-line stage and I did not widen scope to it.
- **Authorisation itself is untested code.** `ci/approval-override.py` is unit-tested; the live
permission lookup, the `override_admins` allowlist branch, and every new BLOCK path in
`ci/approval-policy.sh` need Gitea and have never been exercised. The allowlist branch in
particular is implemented but **unused** — `override_admins` is absent by design.
- **The other open PRs will go red** once the live value changes, until they are rebased onto the
merged declaration: as of writing those are **#29** and **#37** (both `gitea-admin`), plus
**#35**. I did not rebase them. Merging them first avoids it entirely; the script's header gives
the order. PRs #33, #34 and #36 merged while this branch was being written — this branch is
rebased onto `0c7e8dd`, so the list was re-measured rather than carried forward.
- No cluster state was read or changed; nothing here touches Flux, Kyverno or any workload.