fix(ci): accept the override from a PR review, and stop misdiagnosing an unreadable comments API
gate / gate (pull_request) Successful in 1m46s
gate / conformance (pull_request) Has been skipped

The override did not work on its first real test, and the reason was invisible
from the outside. Run 203 on PR #38: a well-formed `/approve-override` posted by
jakman85 in the PR comment box, 29 minutes newer than the head commit, and the
stage still blocked with:

    approval-override: COMMENTS_JSON is not a list

SETTINGS_API_TOKEN is minted [read:repository, read:admin], and Gitea puts
/issues/{n}/comments behind the *issue* scope — so Gitea answered with an error
OBJECT, `jq -e .` accepted it as valid JSON, and the stage fell through to 'no
approving review'. A permission failure wearing the costume of a missing
override. /pulls/{n}/files and /pulls/{n}/reviews are under read:repository,
which is why every other part of the same stage worked.

1. The diagnosis no longer lies. COMMENTS_JSON leaves the must-parse-or-block
   loop and is checked for being an ARRAY; when it is not, the stage prints
   Gitea's own .message, names the missing scope and says where to post instead.
   An unreadable source can only cause a false BLOCK, never a false pass, so a
   loud warning is the honest behaviour and blocking was not.
2. An override is accepted from a PR REVIEW body as well as a comment, so the
   hatch is not hostage to one API scope — a review of type 'Comment' works on
   the token CI has today. The gate names which source it used; PENDING
   (unsubmitted, author-only) reviews are dropped.
3. set-ci-token.sh requests read:issue, making the ordinary comment box work —
   on a RE-RUN only, since an existing token's scopes cannot be widened.

ci/tests/approval-override.test.sh: 23 -> 38 assertions, covering the reviews
source, PENDING reviews, an error object from the comments API, 'an unreadable
comments source must not suppress a valid review override', latest-wins across
both sources, and that the diagnostic quotes Gitea's message.

Also measured and recorded: run 203 shows gitea-settings PASS after the operator
ran the live PATCH, with block_admin_merge_override still true — the narrow
change held, the approval count moved and the gate's teeth did not.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
jakman85andClaude Opus 5 committed 2026-10-02 00:50:20 -05:00
1 parent 8ccfdab180
commit b52c8d1af6
8 files changed
+357 -84

No files matched your search

+89 -2
View File
@@ -26,7 +26,13 @@ BEFORE="2026-10-01T11:30:00Z"
# run <comments-json> -> stdout of the parser (stderr is diagnostic only)
run() {
COMMENTS_JSON="$1" HEAD_TIME="${2:-$HEAD_TIME}" DIRECTIVE="${3:-/approve-override}" \
COMMENTS_JSON="$1" REVIEWS_JSON="" HEAD_TIME="${2:-$HEAD_TIME}" DIRECTIVE="${3:-/approve-override}" \
python3 "$SUT" 2>/dev/null
}
# run_rev <reviews-json> — the same, but through the REVIEWS source.
run_rev() {
COMMENTS_JSON="" REVIEWS_JSON="$1" HEAD_TIME="${2:-$HEAD_TIME}" DIRECTIVE="${3:-/approve-override}" \
python3 "$SUT" 2>/dev/null
}
@@ -54,10 +60,24 @@ expect_user() {
expect_reason() {
local label="$1" json="$2" want="$3" out got
out="$(run "$json")"
got="$(cut -f3- <<<"$out")"
got="$(cut -f4- <<<"$out")"
if [ "$got" = "$want" ]; then ok "$label"; else bad "$label (wanted '${want}', got '${got}')"; fi
}
# check <label> <actual> <wanted>
check() {
if [ "$2" = "$3" ]; then ok "$1"; else bad "$1 (wanted '${3}', got '${2}')"; fi
}
# review <login> <submitted_at> <state> <body>
review() {
python3 -c '
import json, sys
print(json.dumps([{"user": {"login": sys.argv[1]}, "submitted_at": sys.argv[2],
"state": sys.argv[3], "body": sys.argv[4]}]))
' "$1" "$2" "$3" "$4"
}
echo "== ci/approval-override.py"
# --- the happy path -------------------------------------------------------------------
@@ -141,5 +161,72 @@ else
bad "with no HEAD_TIME the candidate should still be returned (got '${out}')"
fi
# --- the REVIEWS source ---------------------------------------------------------------
# Load-bearing, not redundant. Measured on PR #38: SETTINGS_API_TOKEN is minted
# [read:repository, read:admin], /issues/{n}/comments needs the ISSUE scope, so a well-formed
# override in the comment box came back as an error object and was invisible. /pulls/{n}/reviews
# IS readable with that token, so a review body is the route that works today.
check "a review body carries an override" \
"$(cut -f1 <<<"$(run_rev "$(review jakman85 "$AFTER" COMMENT '/approve-override via a review')")")" \
jakman85
check "the source is reported as 'review'" \
"$(cut -f3 <<<"$(run_rev "$(review jakman85 "$AFTER" COMMENT '/approve-override via a review')")")" \
review
check "the source is reported as 'comment' for a comment" \
"$(cut -f3 <<<"$(run "$(comment jakman85 "$AFTER" '/approve-override via a comment')")")" \
comment
out="$(run_rev "$(review jakman85 "$BEFORE" COMMENT '/approve-override stale review')")"
check "a stale review override is dropped too" "${out:-<none>}" "<none>"
out="$(run_rev "$(review jakman85 "$AFTER" PENDING '/approve-override unsubmitted draft')")"
check "a PENDING (unsubmitted) review is NOT an act of anything" "${out:-<none>}" "<none>"
check "an APPROVED review carrying the directive still counts as an override" \
"$(cut -f1 <<<"$(run_rev "$(review jakman85 "$AFTER" APPROVED '/approve-override belt and braces')")")" \
jakman85
out="$(run_rev "$(review jakman85 "$AFTER" COMMENT 'ordinary review prose, no directive')")"
check "an ordinary review body is not an override" "${out:-<none>}" "<none>"
# --- an unreadable source degrades to 'no candidates', never to a pass ----------------
# The real-world shape: Gitea answers with {"message": "token does not have ... scope"}.
ERR_OBJ='{"message":"token does not have at least one of required scope(s): [read:issue]"}'
out="$(COMMENTS_JSON="$ERR_OBJ" REVIEWS_JSON="" HEAD_TIME="$HEAD_TIME" \
DIRECTIVE=/approve-override python3 "$SUT" 2>/dev/null)"
check "an error OBJECT from the comments API yields no override" "${out:-<none>}" "<none>"
out="$(COMMENTS_JSON="$ERR_OBJ" \
REVIEWS_JSON="$(review jakman85 "$AFTER" COMMENT '/approve-override reviews still work')" \
HEAD_TIME="$HEAD_TIME" DIRECTIVE=/approve-override python3 "$SUT" 2>/dev/null)"
check "an unreadable COMMENTS source does not suppress a valid REVIEW override" \
"$(cut -f1 <<<"$out")" jakman85
# and it must SAY so, or a scope problem reads as 'you never posted one' (it did, on PR #38)
errout="$(COMMENTS_JSON="$ERR_OBJ" REVIEWS_JSON="" HEAD_TIME="$HEAD_TIME" \
DIRECTIVE=/approve-override python3 "$SUT" 2>&1 >/dev/null)"
case "$errout" in
*"not a list"*read:issue*) ok "the diagnostic names the scope Gitea complained about" ;;
*) bad "the diagnostic should quote Gitea's message (got: ${errout})" ;;
esac
# --- latest wins ACROSS the two sources ----------------------------------------------
out="$(COMMENTS_JSON="$(comment early "2026-10-01T12:10:00Z" '/approve-override from a comment')" \
REVIEWS_JSON="$(review later "2026-10-01T12:50:00Z" COMMENT '/approve-override from a review')" \
HEAD_TIME="$HEAD_TIME" DIRECTIVE=/approve-override python3 "$SUT" 2>/dev/null)"
check "the latest override wins across both sources (review newer)" "$(cut -f1 <<<"$out")" later
check "and its source is labelled" "$(cut -f3 <<<"$out")" review
out="$(COMMENTS_JSON="$(comment later "2026-10-01T12:50:00Z" '/approve-override from a comment')" \
REVIEWS_JSON="$(review early "2026-10-01T12:10:00Z" COMMENT '/approve-override from a review')" \
HEAD_TIME="$HEAD_TIME" DIRECTIVE=/approve-override python3 "$SUT" 2>/dev/null)"
check "the latest override wins across both sources (comment newer)" "$(cut -f1 <<<"$out")" later
check "and that source is labelled too" "$(cut -f3 <<<"$out")" comment
# --- both sources absent entirely -----------------------------------------------------
out="$(COMMENTS_JSON="" REVIEWS_JSON="" HEAD_TIME="$HEAD_TIME" \
DIRECTIVE=/approve-override python3 "$SUT" 2>/dev/null)"
check "no sources at all yields no override, and no crash" "${out:-<none>}" "<none>"
printf '\n %d passed, %d failed\n' "$PASS" "$FAIL"
[ "$FAIL" -eq 0 ] || exit 1