Files
home-network-k3s-impl/decisions/2026-10-01-oauth2-proxy-api-route-per-app.md
jakman85andClaude Opus 5 2983378496
gate / gate (pull_request) Successful in 1m44s
gate / conformance (pull_request) Has been skipped
chore(decisions): record the CI reading, and why the metrics policy is NOT here
Follow-up to the branch's one substantive commit; no code change.

CI on 99560c2d, read from the commit's /statuses rather than from a tick:
  gate / gate (pull_request)         state=failure   desc='Failing after 1m44s'
  gate / conformance (pull_request)  state=pending   desc='Blocked by required conditions'
The second line is PENDING, not the `success` + "Has been skipped" trap -- the
conformance job is gated on the gate job, which failed, so it never ran.

MY LOCAL `approval-policy: green` WAS MEANINGLESS AND I NEARLY REPORTED IT AS
EVIDENCE. ci/approval-policy.sh:47 reads GITHUB_EVENT_NAME, unset outside CI, so
it short-circuits to "N/A": a clean local gate STRUCTURALLY cannot catch a
sensitive-path problem. Verified by re-running with
GITHUB_EVENT_NAME=pull_request PR_NUMBER=34, where the stage evaluates and fails
CLOSED on the missing token while every other stage stays green -- which bounds
the CI failure to that one stage. I could NOT read CI's job log to confirm it
from CI's own output: this Gitea answers 404 on /actions/runs and on
/actions/jobs/{task}/logs, so only /actions/tasks is available. Attribution
therefore rests on the local reproduction plus the sensitive-path rule, not on
CI's log -- weaker evidence than PR #28 had, named rather than papered over.

Branch protection and the runner re-measured against the live API (CLAUDE.md
2b), not inherited from the CLAUDE.md table: protected=True,
user_can_push=False, enable_status_check=True, contexts=['gate / gate
(pull_request)'], required_approvals=1; runner id=1 incluster-dind online. So a
human is required on every PR here regardless of path. I did NOT provoke a
rejected push to main, so user_can_push is the API's word only.

DECLINED, with reasons: bundling Ines Dubois's metrics-ingress NetworkPolicy.
The coordinator asked for my judgement and the answer is to queue it as the next
PR, not to drop it. Principally because it is a MODEL question, not a policy to
bolt on: schema 1.6.0's observability.metricsEndpoint already renders the
monitoring-namespace ingress allow, and what the model cannot express is a
SECOND metrics endpoint belonging to a sidecar the RENDERER injected. Keying a
bespoke policy off the pod label auth-mode=forward-auth hardcodes "forward-auth
implies 44180" instead of deriving it -- the "list, don't derive" anti-pattern
oauth2-proxy-cookie-names-distinct assertion (5) forbids, one directory from the
check that forbids it. Also: it is a trust-boundary ingress allow and a
different requirement set (R-NET-01/02, R-OBS-01 vs R-IDP-02), so it should not
be reviewed in the shadow of an --api-route headline; the conflict argument does
not apply to my diff, which regenerates TWO goldens not five, and a
CiliumNetworkPolicy is a different document in files git merges independently;
every fact in the finding is hers and I have measured none of it; and the
PodMonitor it serves is in HER PR #29, so shipping the policy alone would open
an ingress allow for a scraper that selects nothing.

Also records that the sidecars' metrics remain UNSCRAPABLE and that this branch
does not change that.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-10-01 22:37:06 -05:00

36 KiB
Raw Permalink Blame History

date, branch, title
date branch title
2026-10-01 feat/oauth2-proxy-api-route-per-app The config model had no way to say "this app's API lives here" — so --api-route could not ship. Now it can, per app.

What changed

Second half of PR #28 (decisions/2026-09-30-oauth2-proxy-cookie-collision-fleet.md), which fixed the cookie collision, the 5-minute session and the missing metrics listener in one renderer function and deliberately declined --api-route for a reason I have preserved rather than worked around: that flag was measured on one app, and PiHole serves no /api/ at all, so a renderer-wide constant would apply one app's path shape to five. The auth block is additionalProperties: false, so there was no per-app override hook. Creating the hook was the work; setting the flag is the easy part.

Three surfaces, in order of what they protect:

Surface Change What it guards that the others cannot
schemas/servicedefinition.schema.json (1.8.0 → 1.9.0) new optional auth.apiRoutes: array of anchored path regexes, uniqueItems, default [] rejects a too-broad regex where a human types it — ci/l0.sh validates catalog/*.yaml against this schema on every gate run
catalog/render.py emits one --api-route=<r> per declared entry, in order; raises on an unanchored entry ci/render-golden.sh runs on every gate invocation even when the schema stage skips
conformance/tests/l0-static/oauth2-proxy-api-routes-per-app/ (new) cross-checks each rendered sidecar against its own ServiceDefinition catches a renderer constant, which neither of the above can see

The auth block stays additionalProperties: false. One named, documented key was added; the door is not opened. That property is the reason the gap existed, and it is worth keeping — it is what made "there is no override hook" a fact rather than a guess for the two engineers who hit it before me.

Rebased mid-flight. origin/main advanced from 08deb20 to cdf3e14 while I was working (PR #27 and PR #32, the hardware.gpu: intel-quicksync capability). That change also took schema 1.8.0, so mine is renumbered to 1.9.0 and chained behind it; both version entries are present in the description, and catalog/render.py had a two-sided conflict in the constants block which is resolved by keeping both (the DRI_* constants and OAUTH2_API_ROUTE_PATTERN are unrelated). Re-rendering after the rebase still changes only jackett and radarr — plex's new golden reproduces byte-identically, so the two changes do not interact.

Shape: a list of regexes, not a single string, because Radarr needs two (^/api/ and ^/signalr/) and --api-route is natively repeatable. A single string would have forced alternation into the value, and alternation is exactly how breadth gets smuggled past a prefix-only reading of a regex.

The anchoring ratchet. Each entry must match ^\^/[A-Za-z0-9._~/-]+(\.\*|\$)?$ — anchored at ^/, then literal path characters, with an optional trailing .* or $. The same pattern is in the schema, in the renderer, and in the new check, and assertion (5) of the check fails if they drift apart, because if they do the loosest one wins silently. Rejected shapes, all five provoked: ^/, ^/.*, .*, /api/ (unanchored), ^/api/|^/ (alternation). Alternation is refused rather than supported: the flag is repeatable, so one entry per route costs nothing.

Why breadth is the dangerous direction, and it is not symmetric. A missing --api-route is a silent 302 — bad, invisible, recoverable. A too broad one answers a plain page load with 401: a logged-out operator gets a bare error page and is never offered a sign-in, so the service looks permanently broken and no reload recovers it. That is a worse outcome than the defect the flag fixes, which is why it is gated on three surfaces rather than documented in one.

Also folded in: the selftest could exit 1 with ZERO diagnostics

Not my change originally — Desmond Oyelaran found it while wiring catalog/render.py --selftest into the render-golden stage (his PR #33, which touches only ci/render-golden.sh and is not in my base; origin/main at cdf3e14 does not yet drive the selftest from any gate stage). The coordinator asked me to fold the fix into this PR because the defect is in the file I own.

_selftest() reached _cookie_name() through a bare next(genexpr) with no default. Delete the --cookie-name emission — exactly the mutation that assertion exists to catch — and it raises StopIteration, which escapes _selftest() before its collected SELFTEST FAIL: lines are printed. Right verdict, no diagnostics, and it masks every other assertion the same mutation broke. Fixed with next(..., None) plus an explicit check() so a missing flag reports as a named assertion.

Provoking it turned up three more instances of the same failure class in the same block, which is why this is not literally a one-line change: side = fa_containers[1], side["env"][0][...], and the fa_svc lookup all raise rather than assert when the sidecar is absent entirely. All four are now None-tolerant, and the two fixtures I added for --api-route were written the same way from the start. Measured, on throwaway copies:

Mutation Before After
--cookie-name emission deleted (origin/main's renderer) rc 1, 0 SELFTEST FAIL lines, 1 StopIteration rc 1, 3 SELFTEST FAIL lines, 0 tracebacks
the oauth2-proxy sidecar not appended at all rc 1, 0 lines, IndexError rc 1, 20 SELFTEST FAIL lines, 0 tracebacks

Note the consequence of his stage change for anyone reading this later: once PR #33 lands, .gitea/workflows/gate.yml:138 runs ./ci/gate.sh || [ "$?" -eq 3 ], so rc 1 is not tolerated in CI the way rc 3 is — a selftest failure will block. Which is the right outcome, and it is why a selftest that prints nothing was worth fixing before that lands rather than after.

Declined, with reasons: the metrics-ingress NetworkPolicy

The coordinator asked for my judgement on bundling Ines Dubois's finding — that #28's metrics listener genuinely serves (she measured http=200, 10,208 bytes, oauth2_proxy_requests_total{code="302"} 2 via port-forward) but the scrape will be dropped, because namespace jackett's default-deny has no ingress key at all and jackett-allow-gateway-ingress admits fromEntities: [ingress] only. The fix belongs in each app's own rendered policy, i.e. in my file and my goldens. I declined to bundle it, and asked for it as the immediate next PR. Five reasons, in the order that decided it:

  1. The real reason is that it is a MODEL question, not a policy to bolt on. Schema 1.6.0 already owns this: observability.metricsEndpoint renders the Service port and the monitoring-namespace ingress allow. What the model cannot currently express is a second metrics endpoint on the same pod belonging to a sidecar the renderer injected, rather than to the workload the operator declared. Keying a bespoke policy off the pod label homelab.keithjackson.dev/auth-mode: forward-auth works today, but it hardcodes "forward-auth implies a metrics port on 44180" into a policy instead of deriving it from the sidecar declaration that is already the single source of that port — the exact "list, don't derive" anti-pattern oauth2-proxy-cookie-names-distinct assertion (5) exists to forbid, and which I would be introducing one directory away from the check that forbids it. That deserves its own review, and it is squarely the kind of decision I was brought in to make deliberately.
  2. Different subject, and it is a trust-boundary change. Mine is the auth response code on API paths (R-IDP-02). That is an ingress allow (R-NET-01/02, R-OBS-01) admitting a new namespace to a new port. Bundling a security-relevant ingress rule into a PR whose headline is --api-route is how such a rule gets waved through on the strength of the other half — and this PR already needs a human for a different sensitive-path reason, so the second subject would be reviewed in the shadow of the first.
  3. The conflict argument does not actually apply to my diff. It assumes I am regenerating five goldens; I am regenerating two (jackett, radarr), because the flag renders only where declared. A metrics-ingress policy renders a different document — a CiliumNetworkPolicy — in all five. Different hunks, in files git merges independently; and goldens regenerate deterministically from the renderer, so the follow-up rebases onto main cleanly. The N-1 rebase problem decisions/README.md records was two branches appending to the same hunk, which this is not. I checked my own diff before answering rather than accepting the premise.
  4. Every fact in the finding is hers and I have measured none of it — not the 200/10,208 bytes, not the wget: download timed out on :44180 and :4180, not the Flux sibling ordering that makes oauth2-proxy and apps-wave-0 unordered. Shipping a security-adjacent rule on wholly inherited evidence, as a footnote in someone else's PR, is precisely what CLAUDE.md §2b exists to stop. In its own PR I would measure it first.
  5. Sequencing: the PodMonitor is in HER PR #29, not mine. If I ship the policy and her PodMonitor is reworked or delayed, I have opened an ingress allow for a scraper that selects nothing — a hole with no consumer. The policy is only useful paired with the PodMonitor, which argues for it landing adjacent to that, not here. It also drags in a KNOWN_EXTERNAL_MONITORS entry in conformance/tests/l0-static/servicemonitor-ports-resolve/, a second conformance interaction on top of the new check this PR already adds.

Diff size is the tiebreaker rather than the argument: 1,245 insertions across 13 files, including a ~400-line new conformance check. I would rather hand a reviewer one subject they can actually falsify. Queued, not dropped — and if the next person takes it, the model note in (1) is the part worth arguing about, not the YAML.

Which apps got which routes, and the measurement behind each

App auth.apiRoutes Evidence
jackett ^/api/ 2026-09-30, its own access log: 33 × 302 against 2 × 401 on /api/ paths, and 4 of 6 POSTs answered by the proxy and never reaching Jackett (upstream field -, not /), including both /api/v2.0/server/update calls
radarr ^/api/, ^/signalr/ ^/signalr/: measured by me, on the live post-fix pod — see below. ^/api/: measured on radarr by the PR #30 author, on the pre-fix pod
pihole (none — deliberate) serves no /api/; 0 unauthenticated requests in its current pod's 23 h
sonarr (none — not measured) 13,643 log lines, 0 unauthenticated requests. No traffic, not a negative result
qbittorrent (none — not measured) 13,649 log lines, 0 unauthenticated requests. Same

^/signalr/ on radarr is the live proof this change was needed, and it is the strongest evidence in this entry. 23 h after PR #28 deployed, on radarr-7d55cbffb9-42hmh (0 restarts), the sidecar logged 232 unauthenticated requests across the pod's entire lifetime, and all 232 are the same call: POST /signalr/messages/negotiate, browser UA, 302, upstream - (Radarr never saw one of them). That is a left-open browser tab whose session the cookie rename orphaned, polling SignalR forever, silently redirected every time, and never re-prompting the operator. It is the reported symptom — "reloading never fixes it and I am never re-prompted" — still happening in production, on a service nobody was looking at. With 401 the UI's connection breaks loudly; the operator reloads; / matches neither regex, so the reload signs them back in.

^/api/ on radarr: included, and here is the honest basis. The current pod has logged zero /api/ traffic, so I did not re-measure it. It rests on a measurement of radarr recorded in the merged PR #30 handoff — its 156 unauthenticated requests were /api/v3/command (62), /api/v3/queue/details (46) and /signalr/messages/negotiate (43), i.e. 108 of 156 were /api/v3 XHRs. That is this app's own evidence, by another engineer, not an analogy to another app. I include it because /api/v3/ is a pure JSON namespace a browser never navigates to (Radarr serves its UI at /, assets under /Content/ and /MediaCover/), so the breadth risk is nil and the measured benefit is 70% of that app's rejection volume.

Sonarr and qBittorrent were measured and deliberately left alone. Sonarr is nearly the same codebase as Radarr, which is exactly what makes ^/api/ + ^/signalr/ a tempting guess and still a guess. I read both pods' logs: zero unauthenticated requests, nothing but health probes, because nobody has opened either UI since the cookie fix rolled. That is evidence of no traffic, not evidence of no need. Each catalog file now carries a comment saying so, with the numbers and what to measure before adding the routes.

Goldens and fixtures regenerated

Rendered with the canonical command (python3 catalog/render.py --all catalog -o apps --wave-layout --no-image-automation). Two goldens changed, not five — and that is the per-app property proving itself on the real tree rather than in a fixture:

apps/wave-0/jackett.yaml   + --api-route=^/api/
apps/wave-1/radarr.yaml    + --api-route=^/api/
                           + --api-route=^/signalr/

apps/wave-0/pihole.yaml, apps/wave-0/qbittorrent.yaml and apps/wave-1/sonarr.yaml are byte-identical to main. No L1 fixture needed regenerating: all three conformance/tests/l1-rendering/*/fixtures/svcdef*.yaml declare auth.mode: forward-auth with no apiRoutes, so they render no flag and their goldens are unchanged. PR #28 touched 5 goldens and 3 L1 fixtures because it changed flags on every sidecar; this one changes flags only where a measurement says so, which is the whole point.

The L0 decision (and it is a decision, not a default)

Not in oauth2-proxy-cookie-names-distinct. That check's subject is cookie identity — its inputs are names and domains, its failure mode is mutual session eviction, its register is the shared proxy's claim on the default name. --api-route shares none of those. Bundling them would make that check's name a lie and couple two unrelated ratchets, so a future api-route decision could only be expressed by weakening a cookie check. That is PR #28's own stated discipline, applied to my change rather than quietly dropped.

A sibling check, oauth2-proxy-api-routes-per-app (whole-repo scope, so it runs on every gate invocation via ci/l0.sh's --include-scope union — no bespoke stage, which whole-repo-invariants-wired would fail). Nowhere was the wrong answer for exactly one reason: the existing gates cannot see a renderer constant. render-golden compares the render to the goldens, so a constant plus regenerated goldens agree with each other while both being wrong — I provoked that and it regenerates with rc 0. The new check's assertion (2) compares each rendered sidecar against its own ServiceDefinition, which is the only reading that fails it.

What I verified

  • --api-route is SINGULAR, read off the PINNED binary rather than taken from the brief. podman run --rm quay.io/oauth2-proxy/oauth2-proxy:v7.8.1 --help: --api-route strings return HTTP 401 instead of redirecting to authentication server if token is not valid. Format: path_regex. There is no --api-routes in the help output. The PM asked me to verify rather than trust, and that is the result — the singular form is confirmed and the help text also confirms the semantics I am relying on (401 instead of a redirect).

  • The premise, in-cluster, read-only, through the pinned toolbox, --context k3s-prod named explicitly (CLAUDE.md §2b and §5). PR #28 is deployed: all five sidecars carry --cookie-name=_oauth2_proxy_<app>, --cookie-domain=<app>.home.keithjackson.dev, --cookie-refresh=2m30s and --metrics-address=0.0.0.0:44180. Pods, all 0 restarts: jackett-6877fcd9f5-ht9d9, pihole-…, qbittorrent-5547bdb9bd-zq557, radarr-7d55cbffb9-42hmh, sonarr-ddf485775-8ckwt (started 2026-09-30T21:41–21:42Z).

  • The full post-fix log census, parsed rather than grepped (oauth2-proxy's access-log format, with /ping health probes excluded), over each pod's whole life:

    app            lines   unauth  AuthSuccess  removing session  cookie sig invalid
    jackett       13,649        0            0                 0                   0
    pihole        13,649        0            0                 0                   0
    qbittorrent   13,649        0            0                 0                   0
    radarr        14,099      232            0                 0                   0
    sonarr        13,643        0            0                 0                   0
    

    Three findings, kept separate:

    1. The collision fix works. Jackett was 39 unauthenticated before; it is 0 in 13,612 non-probe lines. The eviction fingerprint (removing session / cookie signature not valid) is 0 on all five, where radarr previously had 8 and 8.
    2. Radarr's 232 are one request. 100% POST /signalr/messages/negotiate, 100% 302, 100% upstream -, browser UA (Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537…). Zero /api/ traffic of any kind on this pod.
    3. 0 AuthSuccess anywhere, so nobody has signed in since the restart and --cookie-refresh is still unexercised — unchanged from PR #28's own caveat.
  • Only two goldens change, and three are byte-identical to main. git status after the canonical render lists exactly apps/wave-0/jackett.yaml and apps/wave-1/radarr.yaml. render-golden is green.

  • catalog/render.py --selftest: PASS, with new assertions that (a) the fa fixture, which declares nothing, renders no --api-route; (b) a fixture declaring two routes renders exactly those two, in order; (c) the renderer refuses all five breadth shapes; (d) four legitimate routes (^/api/, ^/signalr/, ^/api/v3/.*, ^/api/v2.0/server/update$) are accepted — a ratchet that blocks real API prefixes is a ratchet someone eventually deletes.

  • The new check: rc 0, 43 assertions, 6 sidecars audited, with its inventory printed so the reviewer can see what it read: jackett -> ['^/api/'] | pihole -> (none) | qbittorrent -> (none) | radarr -> ['^/api/', '^/signalr/'] | sonarr -> (none) | crafty -> (none).

  • The check was tested in BOTH directions, on throwaway copies outside the repo, never by mutating the branch. Each mutation fired its own assertion class and nothing else:

    • Pre-change tree (git archive origin/main + the test dropped in): rc 1, 11 failures — no service declares routes, the renderer emits none, the renderer does not read auth.apiRoutes, the pattern is absent from both renderer and schema, and the shipped renderer accepts all five breadth shapes.
    • pihole's golden hand-edited to carry a route it never declared: rc 1, 2 failures (the catalog↔golden mismatch, and "declares nothing must render nothing").
    • radarr's golden widened to --api-route=^/: rc 1, 2 failures (unanchored, plus the mismatch).
    • Renderer turned into a constant and EVERY golden regenerated to match: the regen exits 0 — i.e. render-golden is happy — and the new check exits 1 with 15 failures, naming pihole, qbittorrent, radarr and sonarr individually. This is the case that justifies the check's existence and I provoked it rather than argued it.
    • Schema items.pattern deleted: rc 1, 1 failure — the three-surface drift guard.
    • Unmutated control: rc 0.
  • The selftest diagnostics fix was PROVOKED, not reasoned. Both mutations above were run against throwaway copies of the renderer outside the repo (origin/main's render.py for the "before" column, mine for the "after"), never by mutating the branch. The pre-fix StopIteration and the post-fix 3-and-20 named assertions are quoted from those runs. --selftest on the unmutated branch: PASS. Goldens after the fix: unchanged (it touches only _selftest()).

  • The check is selected by the whole-repo union. ./conformance/run.sh --layer l0 --requirements r-nosuchreq-99 --include-scope whole-repo --print-selection lists tests/l0-static/oauth2-proxy-api-routes-per-app among 21 tests (it was 20), i.e. it runs even when the touched-requirement filter matches nothing.

  • THE SCHEMA GUARD IS PROVEN WITH THE PINNED VALIDATOR ci/l0.sh ACTUALLY DRIVES, not by reading the pattern. ./schemas/validate.sh (check-jsonschema, version 0.29.4) against seven throwaway ServiceDefinitions written outside the repo:

    good            ["^/api/", "^/signalr/"]   rc 0
    bad-root        ["^/"]                     rc 1  apiRoutes ... does not match
    bad-dotstar     ["^/.*"]                   rc 1  apiRoutes ... does not match
    bad-unanchored  ["/api/"]                  rc 1  apiRoutes ... does not match
    bad-alt         ["^/api/|^/"]              rc 1  apiRoutes ... does not match
    bad-everything  [".*"]                     rc 1  apiRoutes ... does not match
    bad-mode        ["^/api/"] on mode: oidc   rc 1  const 'forward-auth'
    

    So a too-broad regex is rejected where a human types it, on every gate run, by the real tool — and the shape invariant fires too.

  • THE RENDERER'S REFUSAL REACHES render-golden AS A READABLE FAILURE, NOT A TRACEBACK. With ^/signalr/ swapped for ^/ in a copy of the catalog (outside the repo), both invocation shapes exit rc 1 with 0 tracebacks — the single-SD form and --all <dir> -o <dir> --wave-layout --no-image-automation, which is the one ci/render-golden.sh drives:

    render.py: radarr: auth.apiRoutes entry '^/' is not an anchored path regex. Each entry
    must start with '^/' and continue with literal path characters (e.g. '^/api/',
    '^/signalr/', '^/api/v3/.*'). ... A too-broad --api-route answers a PAGE LOAD with 401, so
    a logged-out operator sees a bare error page and is never offered a sign-in: the service
    looks permanently broken rather than merely unauthenticated.
    

    So the breadth guard is gated by render-golden even on a run where the schema stage skipped, and the operator is told why rather than handed a stack trace.

  • The schema's new shape invariant. A non-empty auth.apiRoutes requires auth.mode: forward-auth, because the flag renders onto the per-app sidecar and no sidecar exists for oidc/native/none — declaring it elsewhere would be a silent no-op, which is the failure class this repo keeps paying for.

  • Gate: toolbox ./ci/gate.sh with the two-mount ENGINE shim — exit code and stage list in the next section, because it is not a pass and that belongs there.

  • No mutating cluster access. kubectl get/logs/jsonpath only, --context k3s-prod named on every call. No apply/edit/patch, no flux reconcile. No Secret .data or .stringData was read or printed; I did not need to touch a Secret at all this session. The one container I ran locally was the pinned oauth2-proxy image, to read its --help.

  • ~/.git-credentials was never read. PR creation used git credential fill, which does not echo the value.

What I did NOT verify

  • NO BROWSER ROUND-TRIP. BY ANYONE. FOR THIS FLAG. This is the most important line here. The 302/401 split, the dropped POST bodies and the orphaned SignalR poll are all read from access logs; that a 401 repairs the operator's experience is reasoned from HTTP semantics, not watched. Nobody has added a Jackett indexer or recovered a Radarr tab through this configuration. The human test below is the first real evidence this change will ever have.

  • Local gate: rc 3 (UNDETERMINED), not 0. Green: spec-immutability, decisions-log, lint, l0, render-golden, l1. Skipped: gitea-settings (no SETTINGS_API_TOKEN locally) and L2/L3 conformance (CLUSTER_AVAILABLE unset). Nothing FAILED; nothing runtime was certified either. rc 3 is UNDETERMINED, never green.

  • MY LOCAL approval-policy: green WAS MEANINGLESS, AND I ALMOST REPORTED IT AS EVIDENCE. ci/approval-policy.sh:47 reads GITHUB_EVENT_NAME, which is unset outside CI, so the stage short-circuits at line 50: ok "not a pull_request event (event='none') — approval policy N/A". A clean local gate therefore CANNOT catch a sensitive-path problem — structurally, not by accident. Flagged to me by the coordinator from Ines Dubois's round, and I verified it by re-running the gate with GITHUB_EVENT_NAME=pull_request PR_NUMBER=34: the stage then evaluates and fails closed on the missing token —

    [fail] no SETTINGS_API_TOKEN — cannot read this PR's changed files or its reviews.
    [fail] stage approval-policy: FAIL
    [fail] approval-policy: a sensitive change lacks a non-author approval -> blocks merge
    [fail] MERGE GATE: BLOCKED        (rc 1, not 3)
    

    — while every other stage stays green (spec-immutability, decisions-log, lint, l0, render-golden, l1). That is the useful result: it bounds the CI failure to this one stage rather than leaving "something is red" as the whole report. Fail-closed is correct here and deliberate: unlike gitea-settings, where a missing credential is a lock-out, a skip of this stage would be a bypass (CLAUDE.md §1).

  • CI, read from each commit's /statuses rather than from a tick. Both contexts report, and only one of them runs. Identical on 99560c2d, df1b25b6 and f1d16f44:

    gate / gate (pull_request)         state=failure   desc='Failing after ~1m43-1m45s'
    gate / conformance (pull_request)  state=success   desc='Has been skipped'
    

    That second line is §2b's trap in the wild: a skipped Gitea job reports state=success. The conformance job did not execute, so its green tick certifies nothing — read the description, not the colour. The duration is given as a range on purpose: it differs by a second or two per run, and pinning one number inside the file whose commit triggers the next run is a self-reference, not a measurement. The load-bearing fields are the state and the description, and those are stable across all three.

    Worth recording that the same context changed its answer over time: on the first push it briefly read state=pending desc='Blocked by required conditions' — still waiting on the gate job it depends on — and resolved to success only once that job settled. So sampling a status once can show either a pending or a green tick for a job that never executed. I could NOT read the CI job log to confirm the failing stage from CI's own output: this Gitea's /api/v1/repos/{owner}/{repo}/actions/runs endpoint answers 404 (only /actions/tasks exists, which lists tasks without logs), and /actions/jobs/{task_id}/logs answers 404 Not found for the task id. So my attribution of the CI failure to approval-policy rests on the local event-set reproduction above plus the sensitive-path rule, not on CI's own log. That is weaker evidence than PR #28 had and I am naming it rather than borrowing its wording.

  • Branch protection and the runner, RE-MEASURED this session against the live API (§2b), not inherited from CLAUDE.md §2's table:

    GET /repos/Home-Network/home-network-k3s-impl/branches/main
      protected              True
      user_can_push          False
      enable_status_check    True
      status_check_contexts  ['gate / gate (pull_request)']
      required_approvals     1
    GET /admin/actions/runners
      id=1 name=incluster-dind status=online
    

    So required_approvals=1 applies to every PR here regardless of path — this needs a human even without the sensitive-path rule. I did not provoke a rejected push to main this session; user_can_push: False is the API's word, which is exactly the weaker form of evidence CLAUDE.md §2 warns about, and I did not strengthen it.

  • ^/api/ ON RADARR IS NOT MY MEASUREMENT. It is PR #30's, taken on the pre-fix pod. The current pod has logged zero /api/ traffic in 23 h, so I could not reproduce it. I judge the breadth risk nil (/api/v3/ is pure JSON; the UI is at /), but if someone wants a strictly self-measured configuration, drop ^/api/ and keep ^/signalr/ — the 232 are entirely SignalR.

  • SONARR AND QBITTORRENT ARE UNMEASURED, NOT CLEARED. Zero traffic in 23 h is zero evidence. qBittorrent's WebUI drives everything through /api/v2/ and is plausibly affected the same way Jackett is; Sonarr is nearly Radarr's codebase. Both are left alone on purpose — plausibility is what put a renderer constant on the table in the first place. Expect to add ^/api/ to qBittorrent and ^/api/ + ^/signalr/ to Sonarr once someone opens those UIs and records the split. This PR does not fix them.

  • PiHole is asserted to serve no /api/ on inherited evidence, not mine. Its log shows 0 unauthenticated requests, which says nothing about what paths it serves. The claim comes from PR #28/#30. Note the pinned update.range is >=2024.0.0, and PiHole v6 does introduce a REST API at /api/ — so if PiHole is ever upgraded across that line, this needs re-measuring. It is still correct to declare nothing today, because declaring an unmeasured route is the mistake.

  • --cookie-refresh=2m30s has STILL never been exercised — 0 AuthSuccess on all five pods. PR #28 named this as the fix most likely to need a second pass and it remains unobserved. My change does not touch it, but a sign-in during the human test below will exercise both at once, so a failure there must be attributed carefully.

  • The regexes are not validated against the apps' real route tables. I assert /api/v3/ and /signalr/ are JSON/RPC namespaces a browser never navigates to from Radarr's and Jackett's observed traffic and from how those UIs are built — not from reading their routing code or enumerating their endpoints. A path I have not thought of that matches ^/api/ and is browser-navigated would 401.

  • OPTIONS preflight and non-browser clients are unconsidered. --skip-auth-preflight exists and is not set; nothing in these logs shows a CORS preflight, and Sonarr/Radarr talk to Jackett directly on :9117, bypassing the proxy entirely — so I do not expect impact. Unverified.

  • The live pods do not carry this change and nothing re-measures that they will. git is not the cluster (§2b). Merging is what deploys, and it will roll jackett and radarr only (strategy: Recreate on RWO Longhorn volumes, so a brief gap per app, and radarr/sonarr lose direct :9117 access to Jackett while its pod restarts). Existing sessions survive — no cookie flag changed — so unlike PR #28 nobody is signed out.

  • No L2/L3 assertion reads any of this. There is no runtime check that a /api/ request to a gated app answers 401, and I did not write one; CLUSTER_AVAILABLE is unset so it could not run in CI anyway. The whole guard is static.

  • catalog/render.py --selftest is UNGATED IN MY BASE, and I did not wire it. Desmond's PR #33 does that (ci/render-golden.sh) and is in review, not merged — origin/main at cdf3e14 contains no stage that drives it, which I checked rather than assumed (grep -n selftest ci/render-golden.sh → no match). So until #33 lands, my new selftest assertions are dev-time only, and the gated half of my breadth ratchet is the schema pattern (via ci/l0.sh's schema stage), the renderer's own raise (via render-golden), and the new L0 check. I deliberately did not wire the selftest myself: #33 already does it, and duplicating it here would collide with his branch in the one file he owns. Note the two PRs are independent but complementary — his makes the stage real, mine makes its output readable when it fires.

  • I did NOT verify PR #33's own claims. The 106-assertion count, the selftest→stage→gate.sh→BLOCKED propagation chain, and the --cookie-refresh 2m30s → 10m mutation being caught by the selftest but not by render-golden are all his measurements, relayed to me by the coordinator. I reproduced only the StopIteration defect itself, because that is the part I was asked to fix. If his branch is reworked, nothing in mine depends on it.

  • The new check does not read the spec repo and asserts nothing about ../home-network-k3s (R-TEST-06, read-only). schemas/renderer-contract.md is updated in this repo only; the canonical spec docs/13-service-catalog.md is not updated, because it is immutable from here. Someone with spec write access should mirror auth.apiRoutes into docs/13 and docs/15.

  • THE OAUTH2-PROXY SIDECARS' METRICS ARE STILL UNSCRAPABLE, AND THIS PR DOES NOT FIX IT. See the declined-bundle section above. #28 shipped --metrics-address as a prerequisite and said plainly that it was not scraped; Ines has now measured that the listener serves and that the scrape is dropped by each app namespace's default-deny. Nothing in my change touches that. Anyone reading this entry as "the fleet gates are observable now" is wrong, and the next PR is where that changes.

  • Alternation is refused, and that is a judgement I may have to revisit. No app in this catalog needs it, and the flag is repeatable — but if some future app genuinely needs a regex the ratchet rejects, the fix is to widen the pattern with a reason on all three surfaces at once, not to delete the ratchet on one.

  • CI status: two contexts, and only one of them ran. Quoted in the PR body and in the report to the PM. approval-policy fails by design until a non-author approving review exists, because the diff touches conformance/**, which infra/gitea/approval-policy.json lists as sensitive ("the tests and the harness; changing them changes what green MEANS"). That is CLAUDE.md §1 working, not a defect. required_approvals=1 applies to every PR here regardless of path, so this needs a human either way.

The human test — and how to tell a too-broad regex from a working one

After merge, wait for new pod names at 2/2 in jackett and radarr.

  1. The breadth test FIRST, before anything else, from a signed-out browser (private window). Open https://radarr.home.keithjackson.dev/ and https://jackett.home.keithjackson.dev/UI/Dashboard. Each must bounce through Keycloak and land on the app's UI. If either shows a bare 401 Unauthorized instead of a sign-in, the regex is too broad — stop and revert. That is the one failure this change can introduce, and it is the only test that can see it. A working regex is invisible on a page load; a broken one is total and immediate.
  2. Jackett, the original report: signed in, + Add indexer → pick one → Configure → Okay. It must save. Previously 4 of 6 POSTs never reached Jackett at all.
  3. Radarr, the live symptom: open Radarr and leave the tab open. DevTools → Network, filter signalr. Today those negotiate calls are 302 and the UI never complains. They must now be 200 while signed in — and if the session is ever lost, 401, with the UI showing a visible connection error instead of polling silently forever.
  4. The loud-failure test: in DevTools, delete the _oauth2_proxy_radarr cookie and let the tab poll. The negotiate call must answer 401 and Radarr must show a problem. Then reload — / matches neither regex, so the reload must sign you back in silently. That round trip is the entire fix: fail loudly on the API, recover on the page load.
  5. The six-minute test (PR #28's open item, still open): stay on a dashboard for >6 minutes, then save something. That exercises --cookie-refresh=2m30s past the 300 s access-token cliff. If it fails, suspect the realm's refresh/SSO-idle timeouts — not this change.
  6. If something still fails, it must now fail LOUDLY. A silent success-looking no-op means the change did not take: check the running pod's args for --api-route before assuming anything else.