docs(auth): root-cause jackett's session loss; add the oauth2-proxy-session runbook

Reported as "GET works, PUT/POST to add indexers fail, reloading never helps,
and I am never re-prompted". Diagnosis lands here; the MANIFEST FIX DOES NOT,
deliberately -- see the blocker below.

ROOT CAUSE, measured from two pods' logs interleaved to the second:

  15:31:06  jackett  [AuthSuccess]
  15:31:29  radarr   "cookie signature not valid, removing session"
  15:31:32  jackett  "No valid authentication in request"   <- 3s later

Six oauth2-proxy instances (seven pods) emit the DEFAULT cookie name
`_oauth2_proxy` on `.home.keithjackson.dev` holding seven DIFFERENT
cookie-secrets (compared in-cluster by sha256; all distinct). `removing
session` is ClearSession, which expires the cookie on radarr's own zone-wide
--cookie-domain, so jackett's session is DELETED by a process in another
namespace, 26s after a clean login whose `expires` was 5 minutes out. 8 such
lines in radarr's log, 1 in jackett's, 0 elsewhere. It was never expiry and it
was never Jackett -- and it is why reloading does not help: the operator's
Radarr tab re-clears the cookie seconds after each new login.

Two contributing mechanisms, also measured:
  * session lifetime == Keycloak access-token lifetime, 300.000s exactly, with
    refresh_token:true on the same line and --cookie-refresh unset (default 0).
  * XHRs answered 302 not 401: oauth2-proxy's AJAX guard needs an exact
    `Accept: application/json`, which Jackett's jQuery does not send. On /api/
    paths: 33 x 302 vs 2 x 401. A browser following a 302 re-issues a POST as a
    GET and DROPS THE BODY -- 4 of 6 POSTs never reached Jackett at all.

WHY NO MANIFEST FIX HERE. apps/wave-0/jackett.yaml is a GENERATED GOLDEN. The
sidecar args come from one shared function, _oauth2_sidecar_container() in
catalog/render.py; catalog/jackett.yaml declares only auth.mode/auth.groups and
the auth schema is additionalProperties:false, so there is NO per-app override
hook. Hand-editing the rendered file is rejected by the gate exactly as
designed ("GOLDEN DRIFT ... render-golden: FAIL"). Changing the renderer
instead rewrites all five rendered apps -- measured:

  CHANGED: apps/wave-0/{jackett,pihole,qbittorrent}.yaml
  CHANGED: apps/wave-1/{radarr,sonarr}.yaml

and render-golden requires all five be regenerated in the same commit. Four of
them are owned by fix/oauth2-proxy-cookie-collision-fleet right now. The cookie
fix is ONE change, not five; it needs re-cutting as a single renderer commit
rather than two branches rewriting the same generated block.

Validated the intended argument list against the pinned image and the real
issuer before stopping: it printed back `name:_oauth2_proxy_jackett ...
domains:jackett.home.keithjackson.dev ... refresh:after 2m30s` and
`API route - Path: ^/api/`. Two corrections fell out of that: --api-route is
SINGULAR (--api-routes does not exist and the proxy refuses to start), and
--session-cookie-minimal is mutually exclusive with --cookie-refresh
("cookie_refresh > 0 requires oauth tokens in sessions"), so dropping
--pass-access-token does NOT shrink the cookie -- it is a least-privilege
change, not a size one.

Adds runbooks/oauth2-proxy-session.md: the four sidecar log lines that tell the
auth layers apart (including the upstream `-` that means the proxy answered a
request the app never saw), the cross-log collision diagnosis, and the
generated-golden warning with its blast radius. Registered in the
configMapGenerator AND runbooks/README.md so the URL resolves.

Jackett is NOT fixed by this commit and will keep losing its session to Radarr
until the renderer change ships.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
jakman85andClaude Opus 5 committed 2026-09-30 12:47:42 -05:00
1 parent 0fbf544d5e
commit 6a6fea8151
4 files changed
+380

No files matched your search

+5
View File
@@ -85,6 +85,11 @@ configMapGenerator:
- node-io=node-io.md
- node-memory=node-memory.md
- node-memory-swap=node-memory-swap.md
# Operator procedure, not alert-linked. Reading the per-app oauth2-proxy sidecar log to tell
# Keycloak, the sidecar and the app's own login apart. Cookie collision between two sidecars
# presents as random session loss in ONE app's log and is only legible with both side by side,
# so the page has to be reachable at 3am without going looking for it in git.
- oauth2-proxy-session=oauth2-proxy-session.md
# Operator procedure, not an alert page (no alert links to it — the
# alert-runbooks-resolvable check treats that as advisory). It is served
# because the plex.tv claim window is short and is not a thing to go looking