Files
home-network-k3s-impl/decisions/2026-10-01-admin-approval-override.md
T
jakman85andClaude Opus 5 b52c8d1af6
gate / gate (pull_request) Successful in 1m46s
gate / conformance (pull_request) Has been skipped
fix(ci): accept the override from a PR review, and stop misdiagnosing an unreadable comments API
The override did not work on its first real test, and the reason was invisible
from the outside. Run 203 on PR #38: a well-formed `/approve-override` posted by
jakman85 in the PR comment box, 29 minutes newer than the head commit, and the
stage still blocked with:

    approval-override: COMMENTS_JSON is not a list

SETTINGS_API_TOKEN is minted [read:repository, read:admin], and Gitea puts
/issues/{n}/comments behind the *issue* scope — so Gitea answered with an error
OBJECT, `jq -e .` accepted it as valid JSON, and the stage fell through to 'no
approving review'. A permission failure wearing the costume of a missing
override. /pulls/{n}/files and /pulls/{n}/reviews are under read:repository,
which is why every other part of the same stage worked.

1. The diagnosis no longer lies. COMMENTS_JSON leaves the must-parse-or-block
   loop and is checked for being an ARRAY; when it is not, the stage prints
   Gitea's own .message, names the missing scope and says where to post instead.
   An unreadable source can only cause a false BLOCK, never a false pass, so a
   loud warning is the honest behaviour and blocking was not.
2. An override is accepted from a PR REVIEW body as well as a comment, so the
   hatch is not hostage to one API scope — a review of type 'Comment' works on
   the token CI has today. The gate names which source it used; PENDING
   (unsubmitted, author-only) reviews are dropped.
3. set-ci-token.sh requests read:issue, making the ordinary comment box work —
   on a RE-RUN only, since an existing token's scopes cannot be widened.

ci/tests/approval-override.test.sh: 23 -> 38 assertions, covering the reviews
source, PENDING reviews, an error object from the comments API, 'an unreadable
comments source must not suppress a valid review override', latest-wins across
both sources, and that the diagnostic quotes Gitea's message.

Also measured and recorded: run 203 shows gitea-settings PASS after the operator
ran the live PATCH, with block_admin_merge_override still true — the narrow
change held, the approval count moved and the gate's teeth did not.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-10-02 00:50:20 -05:00

257 lines
18 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
date: 2026-10-01
branch: fix/admin-approval-override
title: Administrators can merge their own green PR — required_approvals 1 → 0, plus an attributed /approve-override for sensitive paths
---
## What changed
**The problem, measured first.** `required_approvals: 1` on `main` was a lock-out, not a
protection. Gitea refuses to let a pull request's author approve their own PR, and `jakman85` is
the **only** collaborator (permission `owner`). So no account in existence could approve a PR the
operator opened. Measured on the live API, 2026-10-01:
| | |
|---|---|
| PR #35 (`bugfix/arr-to-plex`, author `jakman85`) | `gate / gate (pull_request)` = **success, 1m41s** |
| its reviews | `[]` |
| live `required_approvals` | `1` |
| live `block_admin_merge_override` | `true` |
Green gate, zero reviews, no path to a merge — and the same was true of any PR that would have
fixed it.
It was also never the design this repo wrote down. `ci/approval-policy.sh`'s own header and
`ci/gate.sh:194` both already described the model as *"`required_approvals` drops to 0 and the
GATE decides"*, and `infra/gitea/require-gate.sh`'s closing note named this exact failure and
prescribed this exact remedy. `decisions/2026-09-30-flux-deploy-notification-inhibited.md` had
flagged the 1-vs-0 disagreement as unresolved and not that session's to settle. This settles it
at 0.
**Two layers were blocking, and both are addressed.**
*Layer 1 — Gitea branch protection.* `required_approvals: 1 → 0` in
`infra/gitea/expected-branch-protection.json`, with the rationale in the file, plus
`infra/gitea/allow-admin-self-merge.sh` to apply it to the live rule (it reads the target **from
the declaration**, so the script cannot drift from what the gate checks). `require-gate.sh` was
changed to 0 as well — left at 1 it would silently re-impose the lock-out on its next run.
The narrow choice matters: the *other* way to unblock an owner is
`block_admin_merge_override: false`, and that would have re-opened precisely what
`infra/gitea/block-admin-override.sh` was written to close (PRs #5–#9 merged with zero approvals
while the settings read as enforced). This change drops the approval **count** only.
`block_admin_merge_override` and `enable_status_check` both stay `true`, and the script asserts
both afterwards and fails loudly if the PATCH disturbed either. **A red gate remains unmergeable
by everyone, administrators included.**
*Layer 2 — `ci/approval-policy.sh`.* Sensitive paths (`ci/**`, `infra/gitea/**`, `CLAUDE.md`,
kyverno, rbac, sops, Flux wiring) still require a non-author `APPROVED` review, which the sole
administrator still cannot obtain. An administrator may now waive it by commenting on the PR:
/approve-override <reason>
- the directive must **start a line** (a quoted or mid-sentence mention does not count, or
repeating someone else's override would re-arm it);
- a **reason is mandatory** — a bare directive records that review was waived but not why;
- the commenter's permission is **re-checked live** against
`/collaborators/{login}/permission` when the gate runs, so an old comment from a demoted
account carries nothing;
- it is **invalidated by a new commit** (the comment must post-date the head commit), mirroring
Gitea's own `dismiss_stale_approvals`;
- the overrider, timestamp and reason are printed in the gate log and the job summary, and a
self-waiver by the PR's own author is called out as such.
Selection of the candidate comment is `ci/approval-override.py` — pure, offline, no authority of
its own — so it could be unit-tested; authorisation stays in the fail-closed stage.
`ci/tests/approval-override.test.sh` adds **23 assertions**, all passing, each constructing a
case the parser must refuse. Every new failure mode blocks: unreadable head-commit timestamp,
unreadable permission, parser error.
**The cost, stated rather than buried.** For an administrator, *"a change to the gate cannot be
gated by the gate"* is now **advisory, not structural**. That is unavoidable with one human — the
alternative is a repository its owner cannot change — and what is bought back is an attributed,
reasoned, commit-scoped record of each waiver instead of a silent bypass.
**The uncomfortable part, accepted with eyes open (operator decision).** Authorisation is by live
permission **alone**, with no allowlist of human accounts. The **agent account `gitea-admin` also
reports permission `owner`** on this repo (measured), so an agent *can* clear the sensitive-path
review on its own PR — and agents author most PRs here, so that is the common case, not an edge
case. I raised this before implementing; the operator chose permission-only. It is documented in
three places rather than left to be discovered: `approval-policy.json`
(`_known_consequence_agent_accounts`), the stage's header, and `CLAUDE.md`, which now tells
agents plainly **not** to use it on their own work. The narrowing path needs no code change —
add an `override_admins` list to `approval-policy.json` and the stage already requires membership
in addition to the permission check.
Also updated: `CLAUDE.md` §1 and the §2 enforcement table (the "approval required" row is now
explicitly ⬜ with the reason, rather than a stale ✅), and `infra/gitea/README.md` gains §6 and
marks its historical step 4 as reversed — do not re-apply.
**Second pass, 2026-10-02 — the override did not work the first time, and why.** The operator ran
the live PATCH (`required_approvals: 0` confirmed live, with `block_admin_merge_override` and
`enable_status_check` still `true`), merged PR #35, and posted `/approve-override Need to ensure
approval override work as expected` on PR #38. The gate re-ran (run 203) and **still blocked.**
The comment was not at fault: posted by `jakman85`, directive at the start of the line, reason
present, and 29 minutes **newer** than head commit `616a497`, so it cleared the staleness rule.
What failed was that **CI could not read it.** Run 203's log:
approval-override: COMMENTS_JSON is not a list
`SETTINGS_API_TOKEN` is minted `["read:repository", "read:admin"]` by
`infra/gitea/set-ci-token.sh`, and Gitea puts `/issues/{n}/comments` behind the **issue** scope. So
Gitea returned an error *object*, `jq -e .` accepted it as valid JSON, and the stage fell through
to "no approving review" — **a permission failure wearing the costume of a missing override.**
`/pulls/{n}/files` and `/pulls/{n}/reviews` are under `read:repository`, which is exactly why every
other part of the same stage worked and this one did not.
Three fixes, and the first is the one that mattered most:
1. **The diagnosis is no longer misleading.** `COMMENTS_JSON` is out of the must-parse-or-block
loop and is explicitly checked for being an *array*; when it is not, the stage prints Gitea's
own `.message`, names the missing scope, and says where to post the override instead. An
unreadable source can only cause a false BLOCK, never a false pass, so degrading it to a loud
warning is safe — and blocking on it was never the honest behaviour.
2. **The override is accepted from a PR REVIEW body as well as a comment**, so the hatch is not
hostage to one API scope. A review of type "Comment" works on the token CI has *today*. The
gate names which source it used. `PENDING` (unsubmitted, author-only draft) reviews are dropped.
3. **`set-ci-token.sh` now requests `read:issue`**, which makes the ordinary comment box work —
the thing anybody reaches for first. It takes effect only on a **re-run**: an existing token's
scopes cannot be widened in place.
`ci/tests/approval-override.test.sh` grows from 23 to **38 assertions**, covering the reviews
source, `PENDING` reviews, an error *object* from the comments API, "an unreadable comments source
must not suppress a valid review override", latest-wins across both sources, and that the
diagnostic actually quotes Gitea's message.
**One adjacent fix, deliberately in scope: `ci/tests/gate-decision.test.sh` was already broken.**
It hand-maintains a list of stub stage scripts that "must name EVERY stage `gate.sh` runs", and it
did not name `approval-policy` — added to `gate.sh` by PR #17 and never added here. `run_stage`
therefore got exit 127 from a missing file and recorded a FAILED stage in every case, so **15 of
50 assertions failed**, including every one expecting a PASS/UNDETERMINED headline. Measured on
pristine `origin/main` (`0c7e8dd`) before touching it, so the attribution is not a guess.
I fixed it because `make -C ci test` is the *only* runner for the suite this change adds — leaving
it red would have made the new tests unverifiable in practice. The file's own comment already
narrated this exact failure for `gitea-settings` and did not prevent the repeat, which says the
comment was not the fix. So the stub list is now **derived from `gate.sh`'s own `run_stage` lines**
and asserts it parsed at least 8 of them: a stage added to the gate can no longer be silently
missing from its test. `make -C ci test` goes from **FAILED to all suites passed**.
## What I verified
- **The lock-out is real, and PR #35 is the proof.** Queried live: head sha
`8a094c46`, statuses `gate / gate (pull_request) = success :: Successful in 1m41s`,
`/pulls/35/reviews` → `[]`, `/branch_protections/main` → `required_approvals: 1`. I read the
**rule object**, not `/branches/main`, per §2b.
- **`jakman85` is the only collaborator**: `/collaborators` → one entry. Permission `owner`.
- **`gitea-admin` also reports `owner`** — this is why the agent-account consequence above is a
statement of fact and not a hypothetical.
- **PR #35 touches no sensitive paths** (`apps/wave-0/plex.yaml`, `apps/wave-1/{radarr,sonarr}.yaml`,
`catalog/{radarr,sonarr}.yaml`, its decisions entry) — so `ci/approval-policy.sh` was never its
blocker. Layer 1 alone unblocks it.
- **API shapes verified before depending on them**, rather than assumed: `/git/commits/{sha}`
returns `.commit.committer.date` as RFC3339 **with an offset** (`2026-10-01T23:09:08-05:00`),
not a trailing `Z` — a parser handling only `Z` would have silently dropped every real
override, so there is a test for the offset form. PR conversation comments are under
`/issues/{n}/comments`; `/pulls/{n}/comments` returns diff comments only and would never have
seen an override.
- **`ci/tests/approval-override.test.sh`: 23 passed, 0 failed.** Covers the happy path, bare
directive, mid-sentence and `>`-quoted directives, stale and equal-timestamp overrides,
unparseable timestamps, missing login, invalid JSON, non-list JSON, latest-wins, offset
timestamps, and a configurable directive.
- `bash -n` clean on `ci/approval-policy.sh`, `infra/gitea/allow-admin-self-merge.sh`,
`infra/gitea/require-gate.sh`; `jq -e .` clean on both JSON declarations.
- **`gate-decision.test.sh`'s 15 failures are pre-existing, measured not assumed.** Extracted
`origin/main`'s `ci/` at `0c7e8dd` into a throwaway directory and ran the suite from there:
**FAIL, 15 failed / 35 passed**. After deriving the stub list from `gate.sh`: **PASS, 50
assertions**.
- **`make -C ci test` in the pinned toolbox: `ci/tests: all suites passed`, rc 0** — approval-override
**38**, gate-decision 50, touched-requirements-base 31, test-attribution 15, touched-requirements 26,
decisions-log pass. It was rc 2 (`ci/tests: FAILED`) before the stub-list fix.
- **The live flip worked, and the drift detector proved it.** Run 202 (before): `gitea-settings:
FAIL`, `required_approvals: DECLARED 0 but LIVE is 1`. Run 203 (after the operator ran the
PATCH): `gitea-settings: pass`, with `block_admin_merge_override = true` among the keys it
checked. So the narrow change held — the approval count moved and the gate's teeth did not.
- **The first override attempt failed for a measured reason, not a guessed one.** Run 203 log:
`approval-override: COMMENTS_JSON is not a list`. Token scopes read from `set-ci-token.sh`:
`[read:repository, read:admin]`. The same run successfully read `/pulls/38` and
`/pulls/38/files` (author and sensitive paths both resolved) and got past the reviews
validation, which is how I know `/pulls/{n}/reviews` IS readable with that token — from
behaviour, not from documentation.
- **The comment itself was valid.** `created_at` 2026-10-02T05:31:24Z vs head commit `616a497`
committer date 2026-10-02T00:02:20-05:00 → the comment is 29m04s newer, so it passed the
staleness rule. Ruling the comment out is what left the token scope as the only candidate.
- **`?limit=1000` is not the culprit.** Queried `/issues/38/comments` with `limit=1000`, `limit=50`
and no limit using my own credentials: `type=array` in all three. So the difference is the
credential, not the query.
- **The root cause is confirmed by Gitea, in its own words.** Run 206 (the fix in place):
`[warn] cannot read PR COMMENTS from Gitea — token does not have at least one of required
scope(s), required=[read:issue], token scope=read:admin,read:repository.` followed by the
instruction to post the override as a review. That is the diagnosis the first attempt should
have produced, and `gitea-settings: pass` in the same run. The stage still ends `FAIL`, correctly
— there is no valid override on the PR yet.
- **Toolbox merge gate** (`./infra/devcontainer/dev.sh run bash -c './ci/gate.sh'`), run on the
rebased branch: **rc 3, `MERGE GATE: UNDETERMINED`**. Green stages: spec-immutability,
decisions-log, lint, l0 (23 offline cases, 0 failures), render-golden including the new
`catalog/render.py --selftest`, l1 (3 cases), approval-policy (N/A offline). Skipped:
`gitea-settings` (no `SETTINGS_API_TOKEN` on this host) and `conformance` (no cluster).
**This is NOT a green run and I am not reporting it as one** — rc 3 is UNDETERMINED per the
CLAUDE.md §3 contract.
## What I did NOT verify
- **The override has still never been ACCEPTED by a real run.** This is the second attempt at
this bullet, which is itself the point: the first pass called the hatch unverified, and the
first real test found a defect that no amount of local testing would have (an API scope CI has
and I did not). What is now measured is that the parse → *reject* path works end to end and
diagnoses itself. The **accepting** branch — directive found → live permission lookup → exit 0 —
has never executed. Posting the override as a **review** on PR #38 is its first real test, and
I would not be surprised by a third round.
- **`read:issue` is the right scope name — Gitea said so itself, so this is no longer a guess.**
Run 206 printed Gitea's own message: `token does not have at least one of required scope(s),
required=[read:issue], token scope=read:admin,read:repository`. That both confirms the root
cause and names the string, which matters because this repo has been burned by a token *name*
that could not exist (`GITEA_SETTINGS_TOKEN`). What is **still** unverified is that the token
*mint* accepts it — `POST /users/{u}/tokens` with that scope has not been run. The review route
does not depend on it.
- **I did not verify that Gitea lets a PR's own author submit a `COMMENT`-type review.** Gitea
blocks author self-*approval*; "Comment" is expected to be allowed, and that expectation is now
load-bearing for the recommended route. It is one click to find out, and I deliberately did not
test it by posting an override myself: doing so as `gitea-admin` is precisely the agent
self-override that `CLAUDE.md` now forbids.
- **The whole re-run mechanism is manual.** `gate.yml` triggers on `pull_request` and `push: main`
only — there is no `issue_comment` trigger, so posting an override does **not** re-run the gate.
Re-run the job from the Actions UI, or close and reopen the PR (which fires `pull_request` and
leaves the head SHA alone). **Do not push an empty commit to re-trigger it:** that moves the head
commit and makes the override stale by the rule above, so it would have to be posted again. A
comment trigger was considered and not built — it would need to check out PR head code while the
job holds `SETTINGS_API_TOKEN`, which is the classic pwn-request shape and not worth it here.
- **I have not seen the gate still REFUSE a merge** now that approvals are 0. The claim that
`block_admin_merge_override: true` keeps a red gate unmergeable rests on Gitea's documented
behaviour and on the 2026-09-29 session, **not** on a provoked refusal in this one. That probe
is written out at the foot of `allow-admin-self-merge.sh` and should be done, exactly as the
push rejection was provoked rather than assumed.
- **Clock skew in the staleness check is unhandled.** Staleness compares the comment's
server-side `created_at` against the head commit's *committer* date, which comes from the
committer's machine. A commit dated in the future would reject legitimate overrides until real
time catches up (fail-closed, so loud, but annoying). Gitea's own `dismiss_stale_approvals`
compares SHAs instead; a comment cannot. Not hit here — PR #35's commit date is ~21 min behind
server time — and not fixed.
- **`ci/tests/approval-override.test.sh` does not run in the merge gate.** `ci/gate.sh` runs no
`ci/tests/*`; they run only via `make -C ci test`. So a regression in the parser would not be
caught by CI. PR #33 (merged while this branch was being written, and rebased onto here) fixed
the *same class* of gap for `catalog/render.py --selftest` by invoking it from
`ci/render-golden.sh` — it did **not** wire `ci/tests/*` in, which I confirmed after rebasing
rather than assuming. Wiring them is a one-line stage and I did not widen scope to it.
- **Authorisation itself is untested code.** `ci/approval-override.py` is unit-tested; the live
permission lookup, the `override_admins` allowlist branch, and every new BLOCK path in
`ci/approval-policy.sh` need Gitea and have never been exercised. The allowlist branch in
particular is implemented but **unused** — `override_admins` is absent by design.
- **The other open PRs will go red** once the live value changes, until they are rebased onto the
merged declaration: as of writing those are **#29** and **#37** (both `gitea-admin`), plus
**#35**. I did not rebase them. Merging them first avoids it entirely; the script's header gives
the order. PRs #33, #34 and #36 merged while this branch was being written — this branch is
rebased onto `0c7e8dd`, so the list was re-measured rather than carried forward.
- No cluster state was read or changed; nothing here touches Flux, Kyverno or any workload.