The override did not work on its first real test, and the reason was invisible from the outside. Run 203 on PR #38: a well-formed `/approve-override` posted by jakman85 in the PR comment box, 29 minutes newer than the head commit, and the stage still blocked with: approval-override: COMMENTS_JSON is not a list SETTINGS_API_TOKEN is minted [read:repository, read:admin], and Gitea puts /issues/{n}/comments behind the *issue* scope — so Gitea answered with an error OBJECT, `jq -e .` accepted it as valid JSON, and the stage fell through to 'no approving review'. A permission failure wearing the costume of a missing override. /pulls/{n}/files and /pulls/{n}/reviews are under read:repository, which is why every other part of the same stage worked. 1. The diagnosis no longer lies. COMMENTS_JSON leaves the must-parse-or-block loop and is checked for being an ARRAY; when it is not, the stage prints Gitea's own .message, names the missing scope and says where to post instead. An unreadable source can only cause a false BLOCK, never a false pass, so a loud warning is the honest behaviour and blocking was not. 2. An override is accepted from a PR REVIEW body as well as a comment, so the hatch is not hostage to one API scope — a review of type 'Comment' works on the token CI has today. The gate names which source it used; PENDING (unsubmitted, author-only) reviews are dropped. 3. set-ci-token.sh requests read:issue, making the ordinary comment box work — on a RE-RUN only, since an existing token's scopes cannot be widened. ci/tests/approval-override.test.sh: 23 -> 38 assertions, covering the reviews source, PENDING reviews, an error object from the comments API, 'an unreadable comments source must not suppress a valid review override', latest-wins across both sources, and that the diagnostic quotes Gitea's message. Also measured and recorded: run 203 shows gitea-settings PASS after the operator ran the live PATCH, with block_admin_merge_override still true — the narrow change held, the approval count moved and the gate's teeth did not. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
257 lines
18 KiB
Markdown
257 lines
18 KiB
Markdown
---
|
||
date: 2026-10-01
|
||
branch: fix/admin-approval-override
|
||
title: Administrators can merge their own green PR — required_approvals 1 → 0, plus an attributed /approve-override for sensitive paths
|
||
---
|
||
|
||
## What changed
|
||
|
||
**The problem, measured first.** `required_approvals: 1` on `main` was a lock-out, not a
|
||
protection. Gitea refuses to let a pull request's author approve their own PR, and `jakman85` is
|
||
the **only** collaborator (permission `owner`). So no account in existence could approve a PR the
|
||
operator opened. Measured on the live API, 2026-10-01:
|
||
|
||
| | |
|
||
|---|---|
|
||
| PR #35 (`bugfix/arr-to-plex`, author `jakman85`) | `gate / gate (pull_request)` = **success, 1m41s** |
|
||
| its reviews | `[]` |
|
||
| live `required_approvals` | `1` |
|
||
| live `block_admin_merge_override` | `true` |
|
||
|
||
Green gate, zero reviews, no path to a merge — and the same was true of any PR that would have
|
||
fixed it.
|
||
|
||
It was also never the design this repo wrote down. `ci/approval-policy.sh`'s own header and
|
||
`ci/gate.sh:194` both already described the model as *"`required_approvals` drops to 0 and the
|
||
GATE decides"*, and `infra/gitea/require-gate.sh`'s closing note named this exact failure and
|
||
prescribed this exact remedy. `decisions/2026-09-30-flux-deploy-notification-inhibited.md` had
|
||
flagged the 1-vs-0 disagreement as unresolved and not that session's to settle. This settles it
|
||
at 0.
|
||
|
||
**Two layers were blocking, and both are addressed.**
|
||
|
||
*Layer 1 — Gitea branch protection.* `required_approvals: 1 → 0` in
|
||
`infra/gitea/expected-branch-protection.json`, with the rationale in the file, plus
|
||
`infra/gitea/allow-admin-self-merge.sh` to apply it to the live rule (it reads the target **from
|
||
the declaration**, so the script cannot drift from what the gate checks). `require-gate.sh` was
|
||
changed to 0 as well — left at 1 it would silently re-impose the lock-out on its next run.
|
||
|
||
The narrow choice matters: the *other* way to unblock an owner is
|
||
`block_admin_merge_override: false`, and that would have re-opened precisely what
|
||
`infra/gitea/block-admin-override.sh` was written to close (PRs #5–#9 merged with zero approvals
|
||
while the settings read as enforced). This change drops the approval **count** only.
|
||
`block_admin_merge_override` and `enable_status_check` both stay `true`, and the script asserts
|
||
both afterwards and fails loudly if the PATCH disturbed either. **A red gate remains unmergeable
|
||
by everyone, administrators included.**
|
||
|
||
*Layer 2 — `ci/approval-policy.sh`.* Sensitive paths (`ci/**`, `infra/gitea/**`, `CLAUDE.md`,
|
||
kyverno, rbac, sops, Flux wiring) still require a non-author `APPROVED` review, which the sole
|
||
administrator still cannot obtain. An administrator may now waive it by commenting on the PR:
|
||
|
||
/approve-override <reason>
|
||
|
||
- the directive must **start a line** (a quoted or mid-sentence mention does not count, or
|
||
repeating someone else's override would re-arm it);
|
||
- a **reason is mandatory** — a bare directive records that review was waived but not why;
|
||
- the commenter's permission is **re-checked live** against
|
||
`/collaborators/{login}/permission` when the gate runs, so an old comment from a demoted
|
||
account carries nothing;
|
||
- it is **invalidated by a new commit** (the comment must post-date the head commit), mirroring
|
||
Gitea's own `dismiss_stale_approvals`;
|
||
- the overrider, timestamp and reason are printed in the gate log and the job summary, and a
|
||
self-waiver by the PR's own author is called out as such.
|
||
|
||
Selection of the candidate comment is `ci/approval-override.py` — pure, offline, no authority of
|
||
its own — so it could be unit-tested; authorisation stays in the fail-closed stage.
|
||
`ci/tests/approval-override.test.sh` adds **23 assertions**, all passing, each constructing a
|
||
case the parser must refuse. Every new failure mode blocks: unreadable head-commit timestamp,
|
||
unreadable permission, parser error.
|
||
|
||
**The cost, stated rather than buried.** For an administrator, *"a change to the gate cannot be
|
||
gated by the gate"* is now **advisory, not structural**. That is unavoidable with one human — the
|
||
alternative is a repository its owner cannot change — and what is bought back is an attributed,
|
||
reasoned, commit-scoped record of each waiver instead of a silent bypass.
|
||
|
||
**The uncomfortable part, accepted with eyes open (operator decision).** Authorisation is by live
|
||
permission **alone**, with no allowlist of human accounts. The **agent account `gitea-admin` also
|
||
reports permission `owner`** on this repo (measured), so an agent *can* clear the sensitive-path
|
||
review on its own PR — and agents author most PRs here, so that is the common case, not an edge
|
||
case. I raised this before implementing; the operator chose permission-only. It is documented in
|
||
three places rather than left to be discovered: `approval-policy.json`
|
||
(`_known_consequence_agent_accounts`), the stage's header, and `CLAUDE.md`, which now tells
|
||
agents plainly **not** to use it on their own work. The narrowing path needs no code change —
|
||
add an `override_admins` list to `approval-policy.json` and the stage already requires membership
|
||
in addition to the permission check.
|
||
|
||
Also updated: `CLAUDE.md` §1 and the §2 enforcement table (the "approval required" row is now
|
||
explicitly ⬜ with the reason, rather than a stale ✅), and `infra/gitea/README.md` gains §6 and
|
||
marks its historical step 4 as reversed — do not re-apply.
|
||
|
||
**Second pass, 2026-10-02 — the override did not work the first time, and why.** The operator ran
|
||
the live PATCH (`required_approvals: 0` confirmed live, with `block_admin_merge_override` and
|
||
`enable_status_check` still `true`), merged PR #35, and posted `/approve-override Need to ensure
|
||
approval override work as expected` on PR #38. The gate re-ran (run 203) and **still blocked.**
|
||
|
||
The comment was not at fault: posted by `jakman85`, directive at the start of the line, reason
|
||
present, and 29 minutes **newer** than head commit `616a497`, so it cleared the staleness rule.
|
||
What failed was that **CI could not read it.** Run 203's log:
|
||
|
||
approval-override: COMMENTS_JSON is not a list
|
||
|
||
`SETTINGS_API_TOKEN` is minted `["read:repository", "read:admin"]` by
|
||
`infra/gitea/set-ci-token.sh`, and Gitea puts `/issues/{n}/comments` behind the **issue** scope. So
|
||
Gitea returned an error *object*, `jq -e .` accepted it as valid JSON, and the stage fell through
|
||
to "no approving review" — **a permission failure wearing the costume of a missing override.**
|
||
`/pulls/{n}/files` and `/pulls/{n}/reviews` are under `read:repository`, which is exactly why every
|
||
other part of the same stage worked and this one did not.
|
||
|
||
Three fixes, and the first is the one that mattered most:
|
||
|
||
1. **The diagnosis is no longer misleading.** `COMMENTS_JSON` is out of the must-parse-or-block
|
||
loop and is explicitly checked for being an *array*; when it is not, the stage prints Gitea's
|
||
own `.message`, names the missing scope, and says where to post the override instead. An
|
||
unreadable source can only cause a false BLOCK, never a false pass, so degrading it to a loud
|
||
warning is safe — and blocking on it was never the honest behaviour.
|
||
2. **The override is accepted from a PR REVIEW body as well as a comment**, so the hatch is not
|
||
hostage to one API scope. A review of type "Comment" works on the token CI has *today*. The
|
||
gate names which source it used. `PENDING` (unsubmitted, author-only draft) reviews are dropped.
|
||
3. **`set-ci-token.sh` now requests `read:issue`**, which makes the ordinary comment box work —
|
||
the thing anybody reaches for first. It takes effect only on a **re-run**: an existing token's
|
||
scopes cannot be widened in place.
|
||
|
||
`ci/tests/approval-override.test.sh` grows from 23 to **38 assertions**, covering the reviews
|
||
source, `PENDING` reviews, an error *object* from the comments API, "an unreadable comments source
|
||
must not suppress a valid review override", latest-wins across both sources, and that the
|
||
diagnostic actually quotes Gitea's message.
|
||
|
||
**One adjacent fix, deliberately in scope: `ci/tests/gate-decision.test.sh` was already broken.**
|
||
It hand-maintains a list of stub stage scripts that "must name EVERY stage `gate.sh` runs", and it
|
||
did not name `approval-policy` — added to `gate.sh` by PR #17 and never added here. `run_stage`
|
||
therefore got exit 127 from a missing file and recorded a FAILED stage in every case, so **15 of
|
||
50 assertions failed**, including every one expecting a PASS/UNDETERMINED headline. Measured on
|
||
pristine `origin/main` (`0c7e8dd`) before touching it, so the attribution is not a guess.
|
||
|
||
I fixed it because `make -C ci test` is the *only* runner for the suite this change adds — leaving
|
||
it red would have made the new tests unverifiable in practice. The file's own comment already
|
||
narrated this exact failure for `gitea-settings` and did not prevent the repeat, which says the
|
||
comment was not the fix. So the stub list is now **derived from `gate.sh`'s own `run_stage` lines**
|
||
and asserts it parsed at least 8 of them: a stage added to the gate can no longer be silently
|
||
missing from its test. `make -C ci test` goes from **FAILED to all suites passed**.
|
||
|
||
## What I verified
|
||
|
||
- **The lock-out is real, and PR #35 is the proof.** Queried live: head sha
|
||
`8a094c46`, statuses `gate / gate (pull_request) = success :: Successful in 1m41s`,
|
||
`/pulls/35/reviews` → `[]`, `/branch_protections/main` → `required_approvals: 1`. I read the
|
||
**rule object**, not `/branches/main`, per §2b.
|
||
- **`jakman85` is the only collaborator**: `/collaborators` → one entry. Permission `owner`.
|
||
- **`gitea-admin` also reports `owner`** — this is why the agent-account consequence above is a
|
||
statement of fact and not a hypothetical.
|
||
- **PR #35 touches no sensitive paths** (`apps/wave-0/plex.yaml`, `apps/wave-1/{radarr,sonarr}.yaml`,
|
||
`catalog/{radarr,sonarr}.yaml`, its decisions entry) — so `ci/approval-policy.sh` was never its
|
||
blocker. Layer 1 alone unblocks it.
|
||
- **API shapes verified before depending on them**, rather than assumed: `/git/commits/{sha}`
|
||
returns `.commit.committer.date` as RFC3339 **with an offset** (`2026-10-01T23:09:08-05:00`),
|
||
not a trailing `Z` — a parser handling only `Z` would have silently dropped every real
|
||
override, so there is a test for the offset form. PR conversation comments are under
|
||
`/issues/{n}/comments`; `/pulls/{n}/comments` returns diff comments only and would never have
|
||
seen an override.
|
||
- **`ci/tests/approval-override.test.sh`: 23 passed, 0 failed.** Covers the happy path, bare
|
||
directive, mid-sentence and `>`-quoted directives, stale and equal-timestamp overrides,
|
||
unparseable timestamps, missing login, invalid JSON, non-list JSON, latest-wins, offset
|
||
timestamps, and a configurable directive.
|
||
- `bash -n` clean on `ci/approval-policy.sh`, `infra/gitea/allow-admin-self-merge.sh`,
|
||
`infra/gitea/require-gate.sh`; `jq -e .` clean on both JSON declarations.
|
||
- **`gate-decision.test.sh`'s 15 failures are pre-existing, measured not assumed.** Extracted
|
||
`origin/main`'s `ci/` at `0c7e8dd` into a throwaway directory and ran the suite from there:
|
||
**FAIL, 15 failed / 35 passed**. After deriving the stub list from `gate.sh`: **PASS, 50
|
||
assertions**.
|
||
- **`make -C ci test` in the pinned toolbox: `ci/tests: all suites passed`, rc 0** — approval-override
|
||
**38**, gate-decision 50, touched-requirements-base 31, test-attribution 15, touched-requirements 26,
|
||
decisions-log pass. It was rc 2 (`ci/tests: FAILED`) before the stub-list fix.
|
||
- **The live flip worked, and the drift detector proved it.** Run 202 (before): `gitea-settings:
|
||
FAIL`, `required_approvals: DECLARED 0 but LIVE is 1`. Run 203 (after the operator ran the
|
||
PATCH): `gitea-settings: pass`, with `block_admin_merge_override = true` among the keys it
|
||
checked. So the narrow change held — the approval count moved and the gate's teeth did not.
|
||
- **The first override attempt failed for a measured reason, not a guessed one.** Run 203 log:
|
||
`approval-override: COMMENTS_JSON is not a list`. Token scopes read from `set-ci-token.sh`:
|
||
`[read:repository, read:admin]`. The same run successfully read `/pulls/38` and
|
||
`/pulls/38/files` (author and sensitive paths both resolved) and got past the reviews
|
||
validation, which is how I know `/pulls/{n}/reviews` IS readable with that token — from
|
||
behaviour, not from documentation.
|
||
- **The comment itself was valid.** `created_at` 2026-10-02T05:31:24Z vs head commit `616a497`
|
||
committer date 2026-10-02T00:02:20-05:00 → the comment is 29m04s newer, so it passed the
|
||
staleness rule. Ruling the comment out is what left the token scope as the only candidate.
|
||
- **`?limit=1000` is not the culprit.** Queried `/issues/38/comments` with `limit=1000`, `limit=50`
|
||
and no limit using my own credentials: `type=array` in all three. So the difference is the
|
||
credential, not the query.
|
||
- **The root cause is confirmed by Gitea, in its own words.** Run 206 (the fix in place):
|
||
`[warn] cannot read PR COMMENTS from Gitea — token does not have at least one of required
|
||
scope(s), required=[read:issue], token scope=read:admin,read:repository.` followed by the
|
||
instruction to post the override as a review. That is the diagnosis the first attempt should
|
||
have produced, and `gitea-settings: pass` in the same run. The stage still ends `FAIL`, correctly
|
||
— there is no valid override on the PR yet.
|
||
- **Toolbox merge gate** (`./infra/devcontainer/dev.sh run bash -c './ci/gate.sh'`), run on the
|
||
rebased branch: **rc 3, `MERGE GATE: UNDETERMINED`**. Green stages: spec-immutability,
|
||
decisions-log, lint, l0 (23 offline cases, 0 failures), render-golden including the new
|
||
`catalog/render.py --selftest`, l1 (3 cases), approval-policy (N/A offline). Skipped:
|
||
`gitea-settings` (no `SETTINGS_API_TOKEN` on this host) and `conformance` (no cluster).
|
||
**This is NOT a green run and I am not reporting it as one** — rc 3 is UNDETERMINED per the
|
||
CLAUDE.md §3 contract.
|
||
|
||
## What I did NOT verify
|
||
|
||
- **The override has still never been ACCEPTED by a real run.** This is the second attempt at
|
||
this bullet, which is itself the point: the first pass called the hatch unverified, and the
|
||
first real test found a defect that no amount of local testing would have (an API scope CI has
|
||
and I did not). What is now measured is that the parse → *reject* path works end to end and
|
||
diagnoses itself. The **accepting** branch — directive found → live permission lookup → exit 0 —
|
||
has never executed. Posting the override as a **review** on PR #38 is its first real test, and
|
||
I would not be surprised by a third round.
|
||
- **`read:issue` is the right scope name — Gitea said so itself, so this is no longer a guess.**
|
||
Run 206 printed Gitea's own message: `token does not have at least one of required scope(s),
|
||
required=[read:issue], token scope=read:admin,read:repository`. That both confirms the root
|
||
cause and names the string, which matters because this repo has been burned by a token *name*
|
||
that could not exist (`GITEA_SETTINGS_TOKEN`). What is **still** unverified is that the token
|
||
*mint* accepts it — `POST /users/{u}/tokens` with that scope has not been run. The review route
|
||
does not depend on it.
|
||
- **I did not verify that Gitea lets a PR's own author submit a `COMMENT`-type review.** Gitea
|
||
blocks author self-*approval*; "Comment" is expected to be allowed, and that expectation is now
|
||
load-bearing for the recommended route. It is one click to find out, and I deliberately did not
|
||
test it by posting an override myself: doing so as `gitea-admin` is precisely the agent
|
||
self-override that `CLAUDE.md` now forbids.
|
||
- **The whole re-run mechanism is manual.** `gate.yml` triggers on `pull_request` and `push: main`
|
||
only — there is no `issue_comment` trigger, so posting an override does **not** re-run the gate.
|
||
Re-run the job from the Actions UI, or close and reopen the PR (which fires `pull_request` and
|
||
leaves the head SHA alone). **Do not push an empty commit to re-trigger it:** that moves the head
|
||
commit and makes the override stale by the rule above, so it would have to be posted again. A
|
||
comment trigger was considered and not built — it would need to check out PR head code while the
|
||
job holds `SETTINGS_API_TOKEN`, which is the classic pwn-request shape and not worth it here.
|
||
- **I have not seen the gate still REFUSE a merge** now that approvals are 0. The claim that
|
||
`block_admin_merge_override: true` keeps a red gate unmergeable rests on Gitea's documented
|
||
behaviour and on the 2026-09-29 session, **not** on a provoked refusal in this one. That probe
|
||
is written out at the foot of `allow-admin-self-merge.sh` and should be done, exactly as the
|
||
push rejection was provoked rather than assumed.
|
||
- **Clock skew in the staleness check is unhandled.** Staleness compares the comment's
|
||
server-side `created_at` against the head commit's *committer* date, which comes from the
|
||
committer's machine. A commit dated in the future would reject legitimate overrides until real
|
||
time catches up (fail-closed, so loud, but annoying). Gitea's own `dismiss_stale_approvals`
|
||
compares SHAs instead; a comment cannot. Not hit here — PR #35's commit date is ~21 min behind
|
||
server time — and not fixed.
|
||
- **`ci/tests/approval-override.test.sh` does not run in the merge gate.** `ci/gate.sh` runs no
|
||
`ci/tests/*`; they run only via `make -C ci test`. So a regression in the parser would not be
|
||
caught by CI. PR #33 (merged while this branch was being written, and rebased onto here) fixed
|
||
the *same class* of gap for `catalog/render.py --selftest` by invoking it from
|
||
`ci/render-golden.sh` — it did **not** wire `ci/tests/*` in, which I confirmed after rebasing
|
||
rather than assuming. Wiring them is a one-line stage and I did not widen scope to it.
|
||
- **Authorisation itself is untested code.** `ci/approval-override.py` is unit-tested; the live
|
||
permission lookup, the `override_admins` allowlist branch, and every new BLOCK path in
|
||
`ci/approval-policy.sh` need Gitea and have never been exercised. The allowlist branch in
|
||
particular is implemented but **unused** — `override_admins` is absent by design.
|
||
- **The other open PRs will go red** once the live value changes, until they are rebased onto the
|
||
merged declaration: as of writing those are **#29** and **#37** (both `gitea-admin`), plus
|
||
**#35**. I did not rebase them. Merging them first avoids it entirely; the script's header gives
|
||
the order. PRs #33, #34 and #36 merged while this branch was being written — this branch is
|
||
rebased onto `0c7e8dd`, so the list was re-measured rather than carried forward.
|
||
- No cluster state was read or changed; nothing here touches Flux, Kyverno or any workload.
|