jakman85andClaude Opus 5
99560c2d22
PR #28 fixed the cookie collision, the 5-minute session and the missing metrics
listener in one renderer function, and deliberately declined `--api-route`
because the config model could not express the difference between two apps.
This adds the hook it was missing and sets the flag only where a measurement
says to.
THE DEFECT. oauth2-proxy answers an unauthenticated request with a 302 to
Keycloak. Its AJAX guard would send 401 instead, but it fires only on an EXACT
`Accept: application/json`, which these apps' JavaScript does not consistently
send. So a browser XHR gets a 302, FOLLOWS IT INVISIBLY, has it satisfied
silently by the live SSO session, and resolves 200 carrying the app's HTML where
the JavaScript expected JSON. On a write it is worse: a browser following a 302
re-issues a POST as a GET and DROPS THE BODY, so the operator's change is
discarded and the UI reports success.
MEASURED, on two apps and two different paths -- which is the whole argument:
jackett 2026-09-30: 33 x 302 vs 2 x 401 on /api/ paths, and 4 of 6 POSTs
answered by the proxy and never reaching Jackett (upstream `-`).
radarr 2026-10-01, 23h after PR #28 deployed (pod radarr-7d55cbffb9-42hmh,
0 restarts): 232 unauthenticated requests in the pod's whole life and
ALL 232 are the same call -- POST /signalr/messages/negotiate from a
browser, 302, upstream `-`. A left-open tab whose session the cookie
rename orphaned, polling forever, silently redirected every time and
NEVER re-prompting. The reported symptom, still in production.
So radarr's broken path is /signalr/ and jackett's is /api/, and PIHOLE SERVES
NO /api/ AT ALL. One function renders all five sidecars, so the cheap fix is a
constant -- and a constant applies one app's measurement to four apps nobody
measured. `auth` was `additionalProperties: false`, so there was no per-app
hook. Adding one named, documented key is the change; the block stays closed.
schemas/servicedefinition.schema.json 1.8.0 -> 1.9.0: optional
`auth.apiRoutes`, a list (radarr needs two, and --api-route is natively
repeatable) of anchored path regexes. Plus the shape invariant that a
non-empty list requires mode: forward-auth, since no sidecar is rendered
for oidc/native/none and declaring it there would be a SILENT NO-OP.
catalog/render.py emits one --api-route per entry, in order, and RAISES on
an unanchored entry. The flag is SINGULAR -- verified against the pinned
v7.8.1 binary's own --help; --api-routes does not exist and the proxy
refuses to start with it.
catalog/{jackett,radarr}.yaml the two measured apps.
catalog/{pihole,sonarr,qbittorrent}.yaml comments recording that they get
NOTHING, and why. Sonarr and qBittorrent were measured (0 unauthenticated
requests in 23h) and left alone: that is evidence of no TRAFFIC, not of no
need, and sonarr being nearly radarr's codebase is what makes the guess
tempting and still a guess.
apps/wave-0/jackett.yaml, apps/wave-1/radarr.yaml regenerated. Only TWO
goldens change; pihole/qbittorrent/sonarr are byte-identical to main,
which is the per-app property proving itself on the real tree.
BREADTH IS THE DANGEROUS DIRECTION AND IT IS NOT SYMMETRIC. A missing
--api-route is a silent 302. A too-broad one answers a plain PAGE LOAD with
401, so a logged-out operator gets a bare error page and is never offered a
sign-in -- the service looks permanently broken. Each entry must therefore
match `^\^/[A-Za-z0-9._~/-]+(\.\*|\$)?$`: anchored at ^/ then literal path
characters. `^/`, `^/.*`, `.*`, `/api/` and `^/api/|^/` are all rejected, all
five provoked. Alternation is refused rather than supported because the flag is
repeatable, so one entry per route costs nothing.
NEW GATE CHECK: conformance/tests/l0-static/oauth2-proxy-api-routes-per-app
(whole-repo scope, so ci/l0.sh's --include-scope union runs it on every
invocation; no bespoke stage, which whole-repo-invariants-wired would fail).
It is a SIBLING of oauth2-proxy-cookie-names-distinct, not an extension of it:
that check's subject is cookie IDENTITY and shares neither inputs nor failure
mode with this, so bundling them would make its name a lie and let a future
api-route decision be expressed only by weakening a COOKIE check.
The check exists for one reason the other gates cannot cover: render-golden
compares the render to the goldens, so a RENDERER CONSTANT plus regenerated
goldens AGREE WITH EACH OTHER while both being wrong. Provoked: the regen exits
0 -- render-golden is happy -- and this check exits 1 with 15 failures naming
pihole, qbittorrent, radarr and sonarr individually. It also asserts the
anchoring pattern is the SAME in the schema and the renderer, because three
surfaces guard this and if they drift the loosest wins silently.
Tested in both directions on throwaway trees outside the repo: compliant tree
rc 0 / 43 assertions / 6 sidecars; pre-change tree rc 1 / 11 failures; and four
targeted mutations (hand-edited golden, widened regex, renderer constant,
deleted schema pattern) each firing their own assertion class and nothing else.
ALSO FOLDED IN (coordinator's request; defect is in the file I own). _selftest()
reached _cookie_name() through a bare `next(genexpr)` with NO default, so
deleting the --cookie-name emission -- the exact mutation that assertion exists
to catch -- raised StopIteration, which escaped _selftest() BEFORE its collected
`SELFTEST FAIL:` lines were printed. Right verdict, zero diagnostics, and it
masked every other assertion the same mutation broke (found by Desmond Oyelaran
while wiring the selftest into the render-golden stage; his PR #33 is not in this
base -- origin/main @ cdf3e14 drives the selftest from no stage, checked). Fixed
with `next(..., None)` plus an explicit check(), so a missing flag reports as a
NAMED assertion. Provoking it exposed three more instances of the same class in
the same block -- `side = fa_containers[1]`, `side["env"][0]`, and the fa_svc
lookup all raise rather than assert when the sidecar is absent -- so all four are
now None-tolerant, and the two fixtures added for --api-route were written that
way from the start. Measured on throwaway copies:
--cookie-name deleted: before rc 1 / 0 FAIL lines / StopIteration
after rc 1 / 3 FAIL lines / no traceback
sidecar not appended: before rc 1 / 0 FAIL lines / IndexError
after rc 1 / 20 FAIL lines / no traceback
Goldens are unchanged by this part; it touches _selftest() only.
NOT verified, in full in the handoff entry: no browser round-trip by anyone for
this flag -- that a 401 REPAIRS the operator's experience is reasoned from HTTP
semantics, not watched. `^/api/` on radarr is PR #30's measurement of radarr's
pre-fix pod, not mine; the current pod has zero /api/ traffic. Sonarr and
qBittorrent are UNMEASURED, not cleared. --cookie-refresh is still unexercised
(0 AuthSuccess on all five pods). Nothing was applied; the live pods are still
the pre-change spec.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
134 lines
9.5 KiB
YAML
134 lines
9.5 KiB
YAML
# ServiceDefinition: sonarr
|
|
# Source of truth: spec docs/13-service-catalog.md row 2; docs/15-identity-and-access.md; docs/03.
|
|
# internal | egress VPN (+ kill-switch) | config on Longhorn | auth forward-auth | arr→downloaders.
|
|
# Sonarr (TV) scrapes indexers and hands releases to the download client; both of those hops leave
|
|
# the cluster over the shared PIA/WireGuard egress gateway (egress: vpn, ADR-0015; R-NET-05/06), and
|
|
# the intra-cluster hops to qbittorrent/jackett are EXPLICIT east-west allows on top of default-deny
|
|
# (R-NET-01/02). Validates against schemas/servicedefinition.schema.json.
|
|
name: sonarr
|
|
kind: container
|
|
source:
|
|
image: lscr.io/linuxserver/sonarr:4.0.15 # pinned stable v4 tag (R-UPD-01); update.range governs advancement
|
|
# LinuxServer images run their service as the user `abc`, whose uid/gid these two set. They are
|
|
# LOAD-BEARING for the bulk mount, not cosmetic: see the storage note below. The default is 911,
|
|
# which cannot write a 1000:3000-owned 0755 tree.
|
|
env:
|
|
- name: PUID
|
|
value: "1000" # owner uid of every directory under /mnt/Storage/plex
|
|
- name: PGID
|
|
value: "3000" # dominant group there; keeps files this writes readable to the
|
|
# EXISTING non-k3s consumers of the same export, which stay live
|
|
- name: TZ
|
|
value: Etc/UTC # same safe default plex uses; the arrs schedule by local time
|
|
podSecurityContext:
|
|
# For the LONGHORN config PVC only. fsGroup does nothing on the NFS volume (the in-tree plugin
|
|
# does no ownership management), but on a Longhorn filesystem the kubelet applies it, which is
|
|
# what makes /config writable by the now-uid-1000 process WITHOUT relying on the image chowning
|
|
# /config for itself when PUID changes. Matches PGID so the primary group owns the config too.
|
|
fsGroup: 3000
|
|
exposure: internal # LAN-only web UI at sonarr.${BASE_DOMAIN} (docs/13 row 2; R-DNS/R-NET)
|
|
egress: vpn # indexer scrape + downloader handoff steered via the shared VPN egress gateway,
|
|
# kill-switch enforced OUTSIDE the workload (docs/03; ADR-0015; R-NET-05/06)
|
|
storage:
|
|
- name: config
|
|
size: 5Gi
|
|
class: longhorn # app DB / config on the 3-replica tier; DR-critical (docs/04; R-STATE-02)
|
|
mountPath: /config # THE APP WRITES /config, NOT the renderer default /data/config.
|
|
# LSIO images run with `-data=/config` (Jackett/qBittorrent likewise use
|
|
# /config), so without this the Longhorn PVC mounted at /data/config sat
|
|
# holding nothing but lost+found while the app wrote its entire
|
|
# configuration to the container's EPHEMERAL writable layer — destroyed on
|
|
# every pod recreation: restart, reschedule, image bump. Measured in-cluster
|
|
# 2026-09-12: /data/config was dev=2080 with [lost+found], /config was
|
|
# dev=2053 on the node's own disk with [Sentry asp config.xml logs].
|
|
# This is the SAME failure catalog/plex.yaml:54 already records fixing.
|
|
# The live /config contents were copied onto each PVC before this landed,
|
|
# so API keys and credentials other apps reference were preserved.
|
|
# THE MEDIA TREE (docs/04 bulk tier) — ONE read-WRITE mount of the export ROOT at /media, giving
|
|
# /media/tv (the library) and /media/downloads (the import source) UNDER THE SAME MOUNT.
|
|
#
|
|
# *** THE ROOT MOUNT IS THE POINT, NOT A CONVENIENCE. *** Sonarr imports by HARDLINKING or
|
|
# atomically renaming a finished download into the library. Both are same-FILESYSTEM operations,
|
|
# and to the kernel "same filesystem" means SAME MOUNT. The layout on the host is:
|
|
# /mnt/Storage/plex/tv/ the library
|
|
# /mnt/Storage/plex/downloads/complete/tv-sonarr/ where finished releases land
|
|
# — siblings under one dataset, so a hardlink between them is legal on the server. But mounting
|
|
# them as two subPath volumes (subPath: tv at /media/tv, subPath: downloads elsewhere) creates
|
|
# TWO mounts, and link() across mounts fails EXDEV no matter what the server thinks. Sonarr then
|
|
# silently degrades to copy-then-delete: every import rewrites the whole file over NFS and the
|
|
# seeding copy is destroyed. Their smallest common ancestor IS the export root, so the root mount
|
|
# is the MINIMAL mount that preserves hardlinks — least privilege bounded below by a hard
|
|
# requirement, not a shortcut. (The cost is stated plainly: Sonarr can also write /media/movies,
|
|
# which is radarr's. Narrowing that would cost hardlinks, which is a worse trade.)
|
|
#
|
|
# *** /media, NOT /data — AND THAT IS NOT A STYLE CHOICE. *** The config PVC above renders at the
|
|
# renderer default /data/config. A media mount at /data would be mounted OVER it and MASK the
|
|
# config volume entirely; the pod would come up with its app database shadowed by an 8 T media
|
|
# tree. /media collides with nothing in a LinuxServer image and matches the path plex uses for
|
|
# the same export, so when qbittorrent is eventually given this mount too, the arrs and the
|
|
# download client will already agree on absolute paths — which is the other precondition for
|
|
# hardlinked imports (otherwise every import needs a Remote Path Mapping).
|
|
#
|
|
# SEPARATE LATENT BUG, NOT FIXED HERE (deliberately — it changes live on-disk state): /data/config
|
|
# is almost certainly the wrong place for this PVC in the first place. LinuxServer's arr images run
|
|
# with `-data=/config`, so the app writes /config — the container's EPHEMERAL layer — while the
|
|
# Longhorn PVC at /data/config sits unused. That is the EXACT failure plex's own mountPath comment
|
|
# documents having already been bitten by. Fixing it means starting from an empty config volume, so
|
|
# it belongs in its own change with the operator present.
|
|
#
|
|
# UID/GID — THE FAILURE THIS AVOIDS IS "MOUNTS FINE, CANNOT WRITE". The export is no_root_squash
|
|
# + sec=sys, so the client's numeric uid is passed through verbatim and the server's ownership is
|
|
# the whole story. The tree is 0755 owned 1000:3000 (some dirs 1000:1000), so ONLY uid 1000 can
|
|
# write it. LinuxServer images default to PUID/PGID 911 — which would mount cleanly and then fail
|
|
# every import with EACCES. The PUID/PGID set above make the process uid 1000, matching the owner.
|
|
#
|
|
# AND NOTE fsGroup CANNOT SUBSTITUTE FOR THAT. The in-tree NFS volume plugin performs no
|
|
# ownership management, so podSecurityContext.fsGroup is a NO-OP on this volume — it is not the
|
|
# lever here, and reaching for it is how this failure normally gets misdiagnosed. The fsGroup
|
|
# that IS set above exists for the LONGHORN config PVC, so that changing PUID does not depend on
|
|
# the image re-chowning /config for itself.
|
|
- name: media
|
|
class: bulk
|
|
export: /mnt/Storage/plex # the ZFS dataset Storage/plex, on ${BULK_NFS_SERVER}
|
|
mountPath: /media # => /media/tv (Root Folder) and /media/downloads (import source)
|
|
size: 9Ti # ADVISORY ONLY on a static NFS PV: the dataset's 8.33 T used,
|
|
# rounded up, so `kubectl get pv` reports something true.
|
|
dependencies:
|
|
- to: qbittorrent # arr → download client (docs/13 dependency graph; R-NET-02 explicit allow)
|
|
port: 8080
|
|
protocol: tcp
|
|
- to: jackett # arr → indexer proxy (docs/13 dependency graph; R-NET-02 explicit allow)
|
|
port: 9117
|
|
protocol: tcp
|
|
ingressRoutes:
|
|
- host: sonarr.${BASE_DOMAIN}
|
|
port: 8989 # Sonarr web/API; TLS terminates at the shared Gateway (docs/08, docs/09)
|
|
auth:
|
|
mode: forward-auth # bare UI, no usable OIDC: oauth2-proxy → Keycloak at the Gateway (docs/15; R-IDP-02)
|
|
groups:
|
|
- media-users # authenticated AND in the media group (R-IDP-06)
|
|
# See catalog/jackett.yaml for the full rationale: --allowed-group is repeatable and OR-ed, so
|
|
# platform-admins here is a superset, not a widening (that group already carries cluster-admin
|
|
# via k8s-oidc-rbac.yaml). Without it, an empty media-users locks every operator out with an
|
|
# immediate 403 at the oauth2-proxy callback — observed on prod 2026-09-12.
|
|
- platform-admins # operators of these services must not be locked out of them
|
|
# NO `apiRoutes` HERE, AND NOT BECAUSE IT NEEDS NONE — because nothing measures that it does.
|
|
# Sonarr is nearly the same codebase as Radarr, which is exactly what makes `^/api/` +
|
|
# `^/signalr/` a tempting guess and still a guess. MEASURED 2026-10-01 on the live pod
|
|
# (sonarr-ddf485775-8ckwt, 0 restarts, 23h): its oauth2-proxy log contains 13,643 lines and
|
|
# ZERO unauthenticated requests — no /api/, no /signalr/, nothing but health probes. Nobody
|
|
# has opened this UI since the cookie fix rolled, so the measurement is empty rather than
|
|
# negative: it is evidence of no TRAFFIC, not evidence of no need. Add the routes when someone
|
|
# reproduces the 302-on-XHR here, with the numbers.
|
|
update:
|
|
channel: stable # R-UPD-01
|
|
range: ">=4.0.0 <5.0.0" # stay on the Sonarr v4 stable line
|
|
cadence: "0 4 * * *"
|
|
autoRollbackOnFailedHealth: true # R-UPD-03
|
|
observability:
|
|
metrics: true
|
|
slis:
|
|
- up # arr down => alert (docs/10; R-OBS-01)
|
|
backup:
|
|
policy: daily-offcluster # Sonarr config/DB is irreplaceable, backed up off-cluster (docs/04; R-STATE-03)
|