Files
home-network-k3s-impl/infra/gitea/verify-then-enforce.sh
T
jakman85andClaude Opus 5 b2d9883958
gate / gate (pull_request) Successful in 2m2s
gate / conformance (pull_request) Has been skipped
docs(ci): close out the hardening -- land the scripts, correct a claim that shipped wrong
The branch-protection + PR-gate + sidecar sequence is complete and verified. This records
the finished state and fixes what the sequence itself proved wrong.

CORRECTS A FALSE CLAIM I SHIPPED. PR #7 said the sidecar conversion would remove the
one-per-start dockerd line "http: TLS handshake error from 127.0.0.1: EOF". It does NOT.
Exactly one still appears, immediately after "API listen on [::]:2376" -- it is now the
startupProbe's FIRST attempt, landing microseconds after dockerd begins listening and cut
off by timeoutSeconds: 1; the next poll succeeds. Cosmetic, and deliberately not chased:
raising the timeout to hide one benign startup line would slow every subsequent poll. The
manifest comment now states this instead of the wrong prediction.

Lands infra/gitea/require-gate.sh and verify-then-enforce.sh, which drove the rollout from
a session temp dir. verify-then-enforce.sh merges nothing -- merging is a human's reviewed
act -- but forces the reconcile and ABORTS unless the pod is healthy, a runner is online,
and a real workflow goes green on it, before enabling enforcement and probing the push.
Re-run it after any future act-runner change.

Trims the now-obsolete feat/ rationale from claim-ship.sh: per-branch push triggers were
deleted, so no prefix produces a duplicate run any more.

CLAUDE.md's enforcement table flips from "weaker than this rule" to machine-enforced, with
all four rows marked verified and the push rejection noted as PROVOKED rather than assumed.
It also now states what a green PR does NOT prove: CLUSTER_AVAILABLE is unset, so L2/L3
never runs in CI and only the L0/L1 offline subset is covered.

infra/gitea/README.md marks the sequence done and names the root cause that remains
unfixed: Gitea repo settings live in its own database, invisible to Flux and unreadable by
the conformance gate, so NOTHING detects drift from what this directory declares. These
scripts assert desired state; they are not a detector. The check that reads the live API
and fails on divergence is the highest-value work left, and it is the difference between
this being fixed and this being unable to recur.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-29 02:28:55 -05:00

84 lines
4.1 KiB
Bash
Executable File

#!/usr/bin/env bash
# verify-then-enforce.sh — run this AFTER you have merged PRs 8, 5, 6, 7 in that order.
#
# It does NOT merge anything. Merging is your reviewed act (CLAUDE.md: a human approves and
# merges). This script does the mechanical half: prove the converted runner actually works,
# and only then turn on enforcement.
#
# WHY THIS ORDER. The act-runner Deployment IS the merge gate for this repository. Enabling a
# required status check while the runner is broken means NO PR can merge -- including the
# revert. So PR #7 lands while the gate is still advisory, is PROVEN green on a real run, and
# only then does enforcement go on.
#
# Aborts at the first checkpoint that does not pass. Safe to re-run.
set -euo pipefail
HOST=gitea.home.keithjackson.dev; OWNER=Home-Network; REPO=home-network-k3s-impl
REPO_ROOT=/var/home/keith/source/home-network-k3s-impl
D="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$REPO_ROOT"
creds="$(printf 'protocol=https\nhost=%s\n\n' "$HOST" | git credential fill)"
U="$(sed -n 's/^username=//p' <<<"$creds")"; P="$(sed -n 's/^password=//p' <<<"$creds")"
api() { curl -sS --config <(printf 'user = "%s:%s"\n' "$U" "$P") "$@"; }
A="https://${HOST}/api/v1/repos/${OWNER}/${REPO}"
say() { printf '\n\033[1m==> %s\033[0m\n' "$1"; }
die() { printf '\n\033[31mABORT: %s\033[0m\n' "$1" >&2; exit 1; }
say "PRECHECK — all four PRs must be merged first"
for n in 8 5 6 7; do
m=$(api "$A/pulls/$n" | jq -r .merged)
printf ' PR #%s merged=%s\n' "$n" "$m"
[ "$m" = true ] || die "PR #$n is not merged yet. Merge 8, 5, 6, 7 in that order, then re-run."
done
say "STEP 1 — force Flux to apply the sidecar now, not on the 10m interval"
./infra/devcontainer/dev.sh run bash -c \
'flux --context k3s-prod -n flux-system reconcile kustomization apps-wave-1 --with-source'
say "STEP 2 — CHECKPOINT: the converted pod must come up healthy"
./infra/devcontainer/dev.sh run bash -c '
kubectl --context k3s-prod -n gitea-actions rollout status deploy/act-runner --timeout=180s
echo "--- sidecar (initContainers) ---"
kubectl --context k3s-prod -n gitea-actions get deploy act-runner \
-o jsonpath="{range .spec.template.spec.initContainers[*]}{.name} restartPolicy={.restartPolicy}{\"\n\"}{end}"
kubectl --context k3s-prod -n gitea-actions get pod -l app.kubernetes.io/name=act-runner' \
|| die "converted act-runner pod is not healthy -- REVERT PR #7 before anything else"
say "STEP 3 — CHECKPOINT: a runner must be ONLINE again"
n=0
for i in $(seq 1 40); do
n=$(api "https://${HOST}/api/v1/admin/actions/runners" | jq -r '[(.runners//[])[]|select(.status=="online")]|length')
printf '\r online runners: %s (%ss) ' "$n" "$((i*5))"
[ "${n:-0}" -ge 1 ] && { echo; break; }
sleep 5
done
[ "${n:-0}" -ge 1 ] || die "no runner online after the conversion -- REVERT PR #7"
say "STEP 4 — CHECKPOINT: a REAL workflow must go green ON THE CONVERTED POD"
MAIN_SHA=$(api "$A/branches/main" | jq -r .commit.id)
echo " watching the post-merge push run on main (${MAIN_SHA:0:8})..."
st=""
for i in $(seq 1 60); do
st=$(api "$A/commits/$MAIN_SHA/status" | jq -r .state)
printf '\r main gate: %-8s (%ss) ' "$st" "$((i*10))"
case "$st" in success) echo; break;; failure|error) die "gate FAILED on the converted pod -- REVERT PR #7";; esac
sleep 10
done
[ "$st" = success ] || die "gate never completed on the converted pod -- REVERT PR #7; do NOT enable enforcement"
echo " the converted sidecar ran a real job green. Enforcement is now safe."
say "STEP 5 — require the gate + one approval"
"$D/require-gate.sh"
say "STEP 6 — prove a direct push is rejected (do not trust the API's word)"
git fetch -q origin && git checkout -q main && git reset -q --hard origin/main
git commit -q --allow-empty -m 'protection probe'
if git push origin main 2>&1 | grep -q 'protected branch'; then
echo " CONFIRMED: push to main rejected by the pre-receive hook."
else
echo " WARNING: the push was NOT rejected -- protection is not doing what it claims."
fi
git reset -q --hard origin/main
say "FINAL STATE"
api "$A/branches/main" | jq -r '{protected,user_can_push,enable_status_check,status_check_contexts,required_approvals}'