jakman85andClaude Opus 5
82aff4d053
plex, sonarr, radarr and nextcloud each gain one bulk volume. Authored in the catalog and re-rendered, never hand-edited into apps/ (render-golden: 10/10). Layout, the export change the operator must apply, the uid analysis and the validation probes are in docs/BULK-NFS-STORAGE.md; everything there was measured on the host over read-only SSH, not assumed. WHAT THE LAYOUT TURNED OUT TO BE, AND WHY IT DECIDED THE DESIGN /mnt/Storage/plex holds movies/ tv/ photos/ AND downloads/ as SIBLINGS in one dataset — so the arrs' import source and their libraries are one filesystem on the server, and the download client's completed folders are already named radarr/ and tv-sonarr/. The k3s arrs are joining a tree an arr stack has been driving for a long time. That sibling fact is the whole argument for the arrs mounting the export ROOT rather than a subPath per directory. Sonarr and Radarr import by hardlinking or atomically renaming, both same-filesystem operations, and to the kernel "same filesystem" means SAME MOUNT. Two subPath mounts of one export are two mounts: link() across them returns EXDEV regardless of what the server knows, and the arrs do not fail loudly — they degrade to copy-then-delete, rewriting every file over NFS and destroying the seeding copy. The smallest subtree containing both tv/ and downloads/ IS the export root, so the root mount is the MINIMAL mount that preserves hardlinks. Least privilege bounded below by a hard requirement. The cost is stated, not hidden: sonarr can also write /media/movies. plex /mnt/Storage/plex -> /media ro ReadOnlyMany sonarr /mnt/Storage/plex -> /media rw ReadWriteMany radarr /mnt/Storage/plex -> /media rw ReadWriteMany nextcloud /mnt/Storage/nextcloud -> /mnt/legacy-nextcloud ro ReadOnlyMany (subPath: nextcloud) Plex is read-only because it indexes and streams; every write on that tree belongs to the arrs, and saying so removes Plex from the set of things that can damage 8.33 T. All three media consumers use the SAME mountPath deliberately: when qbittorrent is eventually given this export, the arrs and the download client will already agree on absolute paths, which is the other precondition for hardlinked imports. *** /media AND NOT /data — A COLLISION CAUGHT WHILE RENDERING. *** The arrs' Longhorn config PVC renders at the renderer default /data/config. A media mount at /data is mounted OVER it and MASKS the config volume: the pod comes up with its app database shadowed by an 8 T media tree. That discovery also exposed a SEPARATE PRE-EXISTING BUG, reported and deliberately NOT fixed here: LinuxServer arr images run with -data=/config, so /data/config is very likely the wrong home for that PVC and the app has been writing its EPHEMERAL layer — the exact failure catalog/plex.yaml's mountPath comment records already having been bitten by. Plex was fixed; sonarr, radarr, qbittorrent and jackett were not. Fixing it starts those apps from an empty config volume, so it needs its own change with the operator present. This change does not make it worse. UID/GID — THE FAILURE THIS TIER ALWAYS HAS The export is sec=sys + no_root_squash, so the client's numeric uid passes through verbatim and the server's ownership is the entire access story. The tree is 0755 owned 1000:3000, so ONLY uid 1000 can write it. LinuxServer images default to 911. Without intervention sonarr and radarr would have mounted cleanly, gone Ready, and failed EVERY import with EACCES. And fsGroup cannot fix that — the in-tree NFS plugin does NO ownership management, so podSecurityContext.fsGroup is a no-op on an NFS volume. Reaching for it is how this gets misdiagnosed. So PUID=1000/PGID=3000 are added to both arrs (PGID 3000 keeps what they write group-readable to the live non-k3s consumers of the same export), and fsGroup: 3000 is added for the LONGHORN config PVC only, so changing PUID does not depend on the image re-chowning /config. plex already runs as 1000 and reads a 0755 tree; nextcloud reads as www-data. Neither needs work. NEXTCLOUD IS MOUNTED READ-ONLY, WHICH IS A DEVIATION FROM THE BRIEF — ON EVIDENCE The task said nextcloud writes. The dataset says otherwise: of 270 G, 267 G is config_backup/, the rest is config/ (103 M) and config_old/ (3.4 G). There is NO data/ directory. This is the OLD non-k3s instance's configuration plus a backup blob — an archive — and that instance is STILL LIVE on it (exported rw to 192.168.1.6 and .5, which the operator has said must stay active). Two Nextclouds writing one instance's config directory, each with its own database and therefore its own oc_filecache, is unrecoverable corruption. "Do not grant more than needed" lands on ro, and ro is the trivially reversible choice. If the intent is in fact a MIGRATION, that is a larger and more deliberate change than a mount; the one-line flip is documented. Read-only for nextcloud can additionally be enforced SERVER-side (only nextcloud consumes that dataset from k3s), and the sharenfs value in the doc uses ro= for the k3s clients. The media tree cannot: plex and the arrs may land on any node, so the server sees the same addresses for reader and writers. Plex's ro is therefore a client-side guarantee — stated as a limitation, not left implied. Co-Authored-By: Claude Opus 5 <[email protected]>
110 lines
7.2 KiB
YAML
110 lines
7.2 KiB
YAML
# ServiceDefinition: sonarr
|
|
# Source of truth: spec docs/13-service-catalog.md row 2; docs/15-identity-and-access.md; docs/03.
|
|
# internal | egress VPN (+ kill-switch) | config on Longhorn | auth forward-auth | arr→downloaders.
|
|
# Sonarr (TV) scrapes indexers and hands releases to the download client; both of those hops leave
|
|
# the cluster over the shared PIA/WireGuard egress gateway (egress: vpn, ADR-0015; R-NET-05/06), and
|
|
# the intra-cluster hops to qbittorrent/jackett are EXPLICIT east-west allows on top of default-deny
|
|
# (R-NET-01/02). Validates against schemas/servicedefinition.schema.json.
|
|
name: sonarr
|
|
kind: container
|
|
source:
|
|
image: lscr.io/linuxserver/sonarr:4.0.15 # pinned stable v4 tag (R-UPD-01); update.range governs advancement
|
|
# LinuxServer images run their service as the user `abc`, whose uid/gid these two set. They are
|
|
# LOAD-BEARING for the bulk mount, not cosmetic: see the storage note below. The default is 911,
|
|
# which cannot write a 1000:3000-owned 0755 tree.
|
|
env:
|
|
- name: PUID
|
|
value: "1000" # owner uid of every directory under /mnt/Storage/plex
|
|
- name: PGID
|
|
value: "3000" # dominant group there; keeps files this writes readable to the
|
|
# EXISTING non-k3s consumers of the same export, which stay live
|
|
- name: TZ
|
|
value: Etc/UTC # same safe default plex uses; the arrs schedule by local time
|
|
podSecurityContext:
|
|
# For the LONGHORN config PVC only. fsGroup does nothing on the NFS volume (the in-tree plugin
|
|
# does no ownership management), but on a Longhorn filesystem the kubelet applies it, which is
|
|
# what makes /config writable by the now-uid-1000 process WITHOUT relying on the image chowning
|
|
# /config for itself when PUID changes. Matches PGID so the primary group owns the config too.
|
|
fsGroup: 3000
|
|
exposure: internal # LAN-only web UI at sonarr.${BASE_DOMAIN} (docs/13 row 2; R-DNS/R-NET)
|
|
egress: vpn # indexer scrape + downloader handoff steered via the shared VPN egress gateway,
|
|
# kill-switch enforced OUTSIDE the workload (docs/03; ADR-0015; R-NET-05/06)
|
|
storage:
|
|
- name: config
|
|
size: 5Gi
|
|
class: longhorn # app DB / config on the 3-replica tier; DR-critical (docs/04; R-STATE-02)
|
|
# THE MEDIA TREE (docs/04 bulk tier) — ONE read-WRITE mount of the export ROOT at /media, giving
|
|
# /media/tv (the library) and /media/downloads (the import source) UNDER THE SAME MOUNT.
|
|
#
|
|
# *** THE ROOT MOUNT IS THE POINT, NOT A CONVENIENCE. *** Sonarr imports by HARDLINKING or
|
|
# atomically renaming a finished download into the library. Both are same-FILESYSTEM operations,
|
|
# and to the kernel "same filesystem" means SAME MOUNT. The layout on the host is:
|
|
# /mnt/Storage/plex/tv/ the library
|
|
# /mnt/Storage/plex/downloads/complete/tv-sonarr/ where finished releases land
|
|
# — siblings under one dataset, so a hardlink between them is legal on the server. But mounting
|
|
# them as two subPath volumes (subPath: tv at /media/tv, subPath: downloads elsewhere) creates
|
|
# TWO mounts, and link() across mounts fails EXDEV no matter what the server thinks. Sonarr then
|
|
# silently degrades to copy-then-delete: every import rewrites the whole file over NFS and the
|
|
# seeding copy is destroyed. Their smallest common ancestor IS the export root, so the root mount
|
|
# is the MINIMAL mount that preserves hardlinks — least privilege bounded below by a hard
|
|
# requirement, not a shortcut. (The cost is stated plainly: Sonarr can also write /media/movies,
|
|
# which is radarr's. Narrowing that would cost hardlinks, which is a worse trade.)
|
|
#
|
|
# *** /media, NOT /data — AND THAT IS NOT A STYLE CHOICE. *** The config PVC above renders at the
|
|
# renderer default /data/config. A media mount at /data would be mounted OVER it and MASK the
|
|
# config volume entirely; the pod would come up with its app database shadowed by an 8 T media
|
|
# tree. /media collides with nothing in a LinuxServer image and matches the path plex uses for
|
|
# the same export, so when qbittorrent is eventually given this mount too, the arrs and the
|
|
# download client will already agree on absolute paths — which is the other precondition for
|
|
# hardlinked imports (otherwise every import needs a Remote Path Mapping).
|
|
#
|
|
# SEPARATE LATENT BUG, NOT FIXED HERE (deliberately — it changes live on-disk state): /data/config
|
|
# is almost certainly the wrong place for this PVC in the first place. LinuxServer's arr images run
|
|
# with `-data=/config`, so the app writes /config — the container's EPHEMERAL layer — while the
|
|
# Longhorn PVC at /data/config sits unused. That is the EXACT failure plex's own mountPath comment
|
|
# documents having already been bitten by. Fixing it means starting from an empty config volume, so
|
|
# it belongs in its own change with the operator present.
|
|
#
|
|
# UID/GID — THE FAILURE THIS AVOIDS IS "MOUNTS FINE, CANNOT WRITE". The export is no_root_squash
|
|
# + sec=sys, so the client's numeric uid is passed through verbatim and the server's ownership is
|
|
# the whole story. The tree is 0755 owned 1000:3000 (some dirs 1000:1000), so ONLY uid 1000 can
|
|
# write it. LinuxServer images default to PUID/PGID 911 — which would mount cleanly and then fail
|
|
# every import with EACCES. The PUID/PGID set above make the process uid 1000, matching the owner.
|
|
#
|
|
# AND NOTE fsGroup CANNOT SUBSTITUTE FOR THAT. The in-tree NFS volume plugin performs no
|
|
# ownership management, so podSecurityContext.fsGroup is a NO-OP on this volume — it is not the
|
|
# lever here, and reaching for it is how this failure normally gets misdiagnosed. The fsGroup
|
|
# that IS set above exists for the LONGHORN config PVC, so that changing PUID does not depend on
|
|
# the image re-chowning /config for itself.
|
|
- name: media
|
|
class: bulk
|
|
export: /mnt/Storage/plex # the ZFS dataset Storage/plex, on ${BULK_NFS_SERVER}
|
|
mountPath: /media # => /media/tv (Root Folder) and /media/downloads (import source)
|
|
size: 9Ti # ADVISORY ONLY on a static NFS PV: the dataset's 8.33 T used,
|
|
# rounded up, so `kubectl get pv` reports something true.
|
|
dependencies:
|
|
- to: qbittorrent # arr → download client (docs/13 dependency graph; R-NET-02 explicit allow)
|
|
port: 8080
|
|
protocol: tcp
|
|
- to: jackett # arr → indexer proxy (docs/13 dependency graph; R-NET-02 explicit allow)
|
|
port: 9117
|
|
protocol: tcp
|
|
ingressRoutes:
|
|
- host: sonarr.${BASE_DOMAIN}
|
|
port: 8989 # Sonarr web/API; TLS terminates at the shared Gateway (docs/08, docs/09)
|
|
auth:
|
|
mode: forward-auth # bare UI, no usable OIDC: oauth2-proxy → Keycloak at the Gateway (docs/15; R-IDP-02)
|
|
groups:
|
|
- media-users # authenticated AND in the media group (R-IDP-06)
|
|
update:
|
|
channel: stable # R-UPD-01
|
|
range: ">=4.0.0 <5.0.0" # stay on the Sonarr v4 stable line
|
|
cadence: "0 4 * * *"
|
|
autoRollbackOnFailedHealth: true # R-UPD-03
|
|
observability:
|
|
metrics: true
|
|
slis:
|
|
- up # arr down => alert (docs/10; R-OBS-01)
|
|
backup:
|
|
policy: daily-offcluster # Sonarr config/DB is irreplaceable, backed up off-cluster (docs/04; R-STATE-03)
|