jakman85andClaude Opus 5
8ccfdab180
`required_approvals: 1` on `main` was a lock-out, not a protection. Gitea
refuses to let a PR's author approve their own PR, and `jakman85` is the only
collaborator — so no account in existence could approve a PR the operator
opened, including the PR that would fix that. Measured 2026-10-01 on PR #35:
gate green in 1m41s, reviews `[]`, `required_approvals: 1`, unmergeable.
It was never the documented design either: `ci/approval-policy.sh`'s header and
`ci/gate.sh:194` both already describe the model as "required_approvals drops
to 0 and the GATE decides", and `require-gate.sh`'s own closing note named this
failure and prescribed this remedy. 2026-09-30's entry flagged the 1-vs-0
disagreement as unresolved; this settles it at 0.
Layer 1 — branch protection. required_approvals 1 -> 0 in the declaration, plus
infra/gitea/allow-admin-self-merge.sh to apply it live (target read FROM the
declaration, so the script cannot drift from what the gate checks).
require-gate.sh moves to 0 too; left at 1 it would re-impose the lock-out on its
next run. The narrow choice is deliberate: the other way to unblock an owner is
block_admin_merge_override: false, which would re-open what
block-admin-override.sh was written to close (PRs #5-#9, zero approvals). This
drops the approval COUNT only — block_admin_merge_override and
enable_status_check stay true and are asserted after the PATCH, so a red gate
stays unmergeable by everyone, administrators included.
Layer 2 — ci/approval-policy.sh. Sensitive paths still need a non-author
approval the sole admin cannot get, so an administrator may waive it with an
`/approve-override <reason>` comment: directive must start a line, reason
mandatory, permission re-checked LIVE, invalidated by a new commit, and the
overrider + reason printed in the gate log and job summary. Candidate selection
is the pure, offline ci/approval-override.py (23 unit assertions, all passing);
authorisation stays in the fail-closed stage. Every new failure mode blocks.
The cost, stated not buried: for an administrator, "a change to the gate cannot
be gated by the gate" is now advisory rather than structural. Unavoidable with
one human; what is bought back is an attributed, commit-scoped record instead of
a silent bypass. And the uncomfortable part, accepted as an operator decision:
authorisation is by live permission alone, and the AGENT account `gitea-admin`
also holds `owner` — so an agent can clear its own PR this way. Documented in
approval-policy.json, the stage header, and CLAUDE.md (which tells agents not
to). Narrowing needs no code change: add `override_admins` and the stage already
requires membership.
Adjacent fix, in scope because `make -C ci test` is the only runner for the
suite added here: ci/tests/gate-decision.test.sh hand-maintained a stub list
that "must name EVERY stage gate.sh runs" and omitted `approval-policy` (added
to gate.sh by PR #17), so run_stage hit exit 127 and 15 of 50 assertions failed.
Verified pre-existing on pristine origin/main @ 0c7e8dd. The list is now derived
from gate.sh's own run_stage lines, so the omission cannot recur; the suite goes
15-failed -> 50 passing and `make -C ci test` goes FAILED -> all suites passed.
Toolbox gate: rc 3 UNDETERMINED — every stage green except `gitea-settings`
(no SETTINGS_API_TOKEN locally) and `conformance` (no cluster). NOT a green run.
The live rule is UNCHANGED and still reads 1 — mutating it is an operator
action. Until allow-admin-self-merge.sh is run, ci/gitea-settings.sh will
correctly report DRIFT and this PR will be red.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
146 lines
7.1 KiB
Bash
Executable File
146 lines
7.1 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# ci/tests/approval-override.test.sh — proves ci/approval-override.py selects the override it
|
|
# claims to select, and REJECTS the ones it claims to reject.
|
|
#
|
|
# This parser decides whether a human review requirement gets waived. Every rule in it exists
|
|
# because the obvious implementation ("does the body contain the directive?") is wrong in a way
|
|
# that is invisible until abused: a quoted directive would re-arm an override, a bare directive
|
|
# would waive review with no stated reason, and an override posted against commit A would go on
|
|
# covering commit B forever. A guard is worth nothing until somebody has watched it refuse, which
|
|
# is the same reasoning as ci/tests/decisions-log.test.sh one layer up.
|
|
#
|
|
# The parser is pure and offline BY DESIGN — authorisation (the live permission lookup) lives in
|
|
# ci/approval-policy.sh, which cannot be tested without Gitea. That split is what makes this file
|
|
# possible, and it is also why a green run here does NOT prove the stage authorises correctly.
|
|
set -uo pipefail
|
|
CI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
|
SUT="${CI_DIR}/approval-override.py"
|
|
|
|
PASS=0 FAIL=0
|
|
ok() { printf ' ok %s\n' "$*"; PASS=$((PASS+1)); }
|
|
bad() { printf ' FAIL %s\n' "$*"; FAIL=$((FAIL+1)); }
|
|
|
|
HEAD_TIME="2026-10-01T12:00:00Z"
|
|
AFTER="2026-10-01T12:30:00Z"
|
|
BEFORE="2026-10-01T11:30:00Z"
|
|
|
|
# run <comments-json> -> stdout of the parser (stderr is diagnostic only)
|
|
run() {
|
|
COMMENTS_JSON="$1" HEAD_TIME="${2:-$HEAD_TIME}" DIRECTIVE="${3:-/approve-override}" \
|
|
python3 "$SUT" 2>/dev/null
|
|
}
|
|
|
|
# comment <login> <created_at> <body>
|
|
comment() {
|
|
python3 -c '
|
|
import json, sys
|
|
print(json.dumps([{"user": {"login": sys.argv[1]}, "created_at": sys.argv[2], "body": sys.argv[3]}]))
|
|
' "$1" "$2" "$3"
|
|
}
|
|
|
|
expect_none() {
|
|
local label="$1" out
|
|
out="$(run "$2")"
|
|
if [ -z "$out" ]; then ok "$label"; else bad "$label (got: ${out})"; fi
|
|
}
|
|
|
|
expect_user() {
|
|
local label="$1" json="$2" want="$3" out got
|
|
out="$(run "$json")"
|
|
got="$(cut -f1 <<<"$out")"
|
|
if [ "$got" = "$want" ]; then ok "$label"; else bad "$label (wanted ${want}, got '${got}')"; fi
|
|
}
|
|
|
|
expect_reason() {
|
|
local label="$1" json="$2" want="$3" out got
|
|
out="$(run "$json")"
|
|
got="$(cut -f3- <<<"$out")"
|
|
if [ "$got" = "$want" ]; then ok "$label"; else bad "$label (wanted '${want}', got '${got}')"; fi
|
|
}
|
|
|
|
echo "== ci/approval-override.py"
|
|
|
|
# --- the happy path -------------------------------------------------------------------
|
|
expect_user "a directive with a reason, posted after the head commit, is selected" \
|
|
"$(comment jakman85 "$AFTER" '/approve-override sole admin, gate is green')" jakman85
|
|
|
|
expect_reason "the reason is returned verbatim" \
|
|
"$(comment jakman85 "$AFTER" '/approve-override sole admin, gate is green')" \
|
|
'sole admin, gate is green'
|
|
|
|
# --- absence is a normal answer, not an error -----------------------------------------
|
|
expect_none "no comments at all" '[]'
|
|
expect_none "an ordinary comment" "$(comment jakman85 "$AFTER" 'looks good to me')"
|
|
|
|
# --- a reason is MANDATORY ------------------------------------------------------------
|
|
expect_none "a bare directive (no reason) is refused" \
|
|
"$(comment jakman85 "$AFTER" '/approve-override')"
|
|
expect_none "a directive followed by only whitespace is refused" \
|
|
"$(comment jakman85 "$AFTER" '/approve-override ')"
|
|
|
|
# --- the directive must START a line --------------------------------------------------
|
|
expect_none "the directive mid-sentence does not count" \
|
|
"$(comment jakman85 "$AFTER" 'you could just /approve-override this one')"
|
|
expect_none "a QUOTED directive does not re-arm an override" \
|
|
"$(comment jakman85 "$AFTER" '> /approve-override quoting someone else')"
|
|
expect_user "leading whitespace before the directive is fine" \
|
|
"$(comment jakman85 "$AFTER" ' /approve-override indented but mine')" jakman85
|
|
expect_user "the directive on a later line of a longer comment counts" \
|
|
"$(comment jakman85 "$AFTER" 'some preamble
|
|
/approve-override the real reason here')" jakman85
|
|
|
|
# --- staleness: an override must post-date the commit it waives review on -------------
|
|
expect_none "an override OLDER than the head commit is dropped as stale" \
|
|
"$(comment jakman85 "$BEFORE" '/approve-override stale, predates the push')"
|
|
expect_none "an override exactly AT the head commit time is dropped (strictly newer)" \
|
|
"$(comment jakman85 "$HEAD_TIME" '/approve-override same instant')"
|
|
|
|
# --- malformed input is never a free pass --------------------------------------------
|
|
expect_none "an unreadable timestamp drops the candidate" \
|
|
"$(comment jakman85 'not-a-date' '/approve-override bad clock')"
|
|
expect_none "a comment with no author is dropped" \
|
|
'[{"user": {}, "created_at": "2026-10-01T12:30:00Z", "body": "/approve-override no login"}]'
|
|
expect_none "invalid JSON yields no override (and never a traceback into stdout)" \
|
|
'this is not json'
|
|
expect_none "a JSON object instead of a list yields no override" \
|
|
'{"user": "nope"}'
|
|
|
|
# --- latest wins ----------------------------------------------------------------------
|
|
TWO='[
|
|
{"user": {"login": "early"}, "created_at": "2026-10-01T12:10:00Z", "body": "/approve-override first"},
|
|
{"user": {"login": "latest"}, "created_at": "2026-10-01T12:40:00Z", "body": "/approve-override second"}
|
|
]'
|
|
expect_user "the LATEST override supersedes an earlier one" "$TWO" latest
|
|
expect_reason "and brings its own reason" "$TWO" second
|
|
|
|
# --- offsets, not just Z --------------------------------------------------------------
|
|
# Gitea emits RFC3339 WITH AN OFFSET (measured: 2026-10-01T23:09:08-05:00), not a trailing Z.
|
|
# A parser that only handles 'Z' silently drops every real override, so this case is load-bearing.
|
|
expect_user "an RFC3339 offset timestamp (-05:00), as Gitea actually emits, parses" \
|
|
'[{"user": {"login": "jakman85"}, "created_at": "2026-10-01T08:30:00-05:00", "body": "/approve-override offset form"}]' \
|
|
jakman85
|
|
expect_none "and an offset timestamp that is genuinely earlier is still stale" \
|
|
'[{"user": {"login": "jakman85"}, "created_at": "2026-10-01T06:00:00-05:00", "body": "/approve-override offset, stale"}]'
|
|
|
|
# --- the directive is configurable ----------------------------------------------------
|
|
out="$(run "$(comment jakman85 "$AFTER" '/admin-waive because policy says so')" "$HEAD_TIME" '/admin-waive')"
|
|
if [ "$(cut -f1 <<<"$out")" = "jakman85" ]; then
|
|
ok "a custom directive from the declaration is honoured"
|
|
else
|
|
bad "a custom directive from the declaration is honoured (got '${out}')"
|
|
fi
|
|
expect_none "the default directive does not match a custom-directive comment" \
|
|
"$(comment jakman85 "$AFTER" '/admin-waive because policy says so')"
|
|
|
|
# --- no staleness anchor: do not invent one -------------------------------------------
|
|
out="$(COMMENTS_JSON="$(comment jakman85 "$BEFORE" '/approve-override no anchor')" \
|
|
HEAD_TIME="" DIRECTIVE=/approve-override python3 "$SUT" 2>/dev/null)"
|
|
if [ "$(cut -f1 <<<"$out")" = "jakman85" ]; then
|
|
ok "with no HEAD_TIME the staleness test is skipped, not faked (caller blocks instead)"
|
|
else
|
|
bad "with no HEAD_TIME the candidate should still be returned (got '${out}')"
|
|
fi
|
|
|
|
printf '\n %d passed, %d failed\n' "$PASS" "$FAIL"
|
|
[ "$FAIL" -eq 0 ] || exit 1
|