jakman85andClaude Opus 5
82aff4d053
plex, sonarr, radarr and nextcloud each gain one bulk volume. Authored in the catalog and re-rendered, never hand-edited into apps/ (render-golden: 10/10). Layout, the export change the operator must apply, the uid analysis and the validation probes are in docs/BULK-NFS-STORAGE.md; everything there was measured on the host over read-only SSH, not assumed. WHAT THE LAYOUT TURNED OUT TO BE, AND WHY IT DECIDED THE DESIGN /mnt/Storage/plex holds movies/ tv/ photos/ AND downloads/ as SIBLINGS in one dataset — so the arrs' import source and their libraries are one filesystem on the server, and the download client's completed folders are already named radarr/ and tv-sonarr/. The k3s arrs are joining a tree an arr stack has been driving for a long time. That sibling fact is the whole argument for the arrs mounting the export ROOT rather than a subPath per directory. Sonarr and Radarr import by hardlinking or atomically renaming, both same-filesystem operations, and to the kernel "same filesystem" means SAME MOUNT. Two subPath mounts of one export are two mounts: link() across them returns EXDEV regardless of what the server knows, and the arrs do not fail loudly — they degrade to copy-then-delete, rewriting every file over NFS and destroying the seeding copy. The smallest subtree containing both tv/ and downloads/ IS the export root, so the root mount is the MINIMAL mount that preserves hardlinks. Least privilege bounded below by a hard requirement. The cost is stated, not hidden: sonarr can also write /media/movies. plex /mnt/Storage/plex -> /media ro ReadOnlyMany sonarr /mnt/Storage/plex -> /media rw ReadWriteMany radarr /mnt/Storage/plex -> /media rw ReadWriteMany nextcloud /mnt/Storage/nextcloud -> /mnt/legacy-nextcloud ro ReadOnlyMany (subPath: nextcloud) Plex is read-only because it indexes and streams; every write on that tree belongs to the arrs, and saying so removes Plex from the set of things that can damage 8.33 T. All three media consumers use the SAME mountPath deliberately: when qbittorrent is eventually given this export, the arrs and the download client will already agree on absolute paths, which is the other precondition for hardlinked imports. *** /media AND NOT /data — A COLLISION CAUGHT WHILE RENDERING. *** The arrs' Longhorn config PVC renders at the renderer default /data/config. A media mount at /data is mounted OVER it and MASKS the config volume: the pod comes up with its app database shadowed by an 8 T media tree. That discovery also exposed a SEPARATE PRE-EXISTING BUG, reported and deliberately NOT fixed here: LinuxServer arr images run with -data=/config, so /data/config is very likely the wrong home for that PVC and the app has been writing its EPHEMERAL layer — the exact failure catalog/plex.yaml's mountPath comment records already having been bitten by. Plex was fixed; sonarr, radarr, qbittorrent and jackett were not. Fixing it starts those apps from an empty config volume, so it needs its own change with the operator present. This change does not make it worse. UID/GID — THE FAILURE THIS TIER ALWAYS HAS The export is sec=sys + no_root_squash, so the client's numeric uid passes through verbatim and the server's ownership is the entire access story. The tree is 0755 owned 1000:3000, so ONLY uid 1000 can write it. LinuxServer images default to 911. Without intervention sonarr and radarr would have mounted cleanly, gone Ready, and failed EVERY import with EACCES. And fsGroup cannot fix that — the in-tree NFS plugin does NO ownership management, so podSecurityContext.fsGroup is a no-op on an NFS volume. Reaching for it is how this gets misdiagnosed. So PUID=1000/PGID=3000 are added to both arrs (PGID 3000 keeps what they write group-readable to the live non-k3s consumers of the same export), and fsGroup: 3000 is added for the LONGHORN config PVC only, so changing PUID does not depend on the image re-chowning /config. plex already runs as 1000 and reads a 0755 tree; nextcloud reads as www-data. Neither needs work. NEXTCLOUD IS MOUNTED READ-ONLY, WHICH IS A DEVIATION FROM THE BRIEF — ON EVIDENCE The task said nextcloud writes. The dataset says otherwise: of 270 G, 267 G is config_backup/, the rest is config/ (103 M) and config_old/ (3.4 G). There is NO data/ directory. This is the OLD non-k3s instance's configuration plus a backup blob — an archive — and that instance is STILL LIVE on it (exported rw to 192.168.1.6 and .5, which the operator has said must stay active). Two Nextclouds writing one instance's config directory, each with its own database and therefore its own oc_filecache, is unrecoverable corruption. "Do not grant more than needed" lands on ro, and ro is the trivially reversible choice. If the intent is in fact a MIGRATION, that is a larger and more deliberate change than a mount; the one-line flip is documented. Read-only for nextcloud can additionally be enforced SERVER-side (only nextcloud consumes that dataset from k3s), and the sharenfs value in the doc uses ro= for the k3s clients. The media tree cannot: plex and the arrs may land on any node, so the server sees the same addresses for reader and writers. Plex's ro is therefore a client-side guarantee — stated as a limitation, not left implied. Co-Authored-By: Claude Opus 5 <[email protected]>
408 lines
9.2 KiB
YAML
408 lines
9.2 KiB
YAML
---
|
|
# Namespace (trust zone; default-deny + policies apply)
|
|
apiVersion: v1
|
|
kind: Namespace
|
|
metadata:
|
|
name: sonarr
|
|
labels:
|
|
homelab.keithjackson.dev/app-namespace: 'true'
|
|
gateway-access: shared
|
|
annotations:
|
|
homelab.keithjackson.dev/backup-policy: daily-offcluster
|
|
---
|
|
# Default-deny baseline (R-NET-01): deny all, allow only DNS
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: NetworkPolicy
|
|
metadata:
|
|
name: default-deny
|
|
namespace: sonarr
|
|
labels:
|
|
homelab.keithjackson.dev/baseline: default-deny
|
|
spec:
|
|
podSelector: {}
|
|
policyTypes:
|
|
- Ingress
|
|
- Egress
|
|
ingress: []
|
|
egress:
|
|
- to:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: kube-system
|
|
ports:
|
|
- protocol: UDP
|
|
port: 53
|
|
- protocol: TCP
|
|
port: 53
|
|
---
|
|
# PersistentVolume sonarr-media - static external NFS (bulk tier)
|
|
apiVersion: v1
|
|
kind: PersistentVolume
|
|
metadata:
|
|
name: sonarr-media
|
|
labels:
|
|
app.kubernetes.io/name: sonarr
|
|
spec:
|
|
capacity:
|
|
storage: 9Ti
|
|
accessModes:
|
|
- ReadWriteMany
|
|
persistentVolumeReclaimPolicy: Retain
|
|
storageClassName: ''
|
|
mountOptions:
|
|
- nfsvers=4.1
|
|
- hard
|
|
- noatime
|
|
- timeo=600
|
|
- retrans=2
|
|
nfs:
|
|
server: ${BULK_NFS_SERVER}
|
|
path: /mnt/Storage/plex
|
|
claimRef:
|
|
namespace: sonarr
|
|
name: sonarr-media
|
|
---
|
|
# PVC sonarr-config (R-STATE)
|
|
apiVersion: v1
|
|
kind: PersistentVolumeClaim
|
|
metadata:
|
|
name: sonarr-config
|
|
namespace: sonarr
|
|
labels:
|
|
app.kubernetes.io/name: sonarr
|
|
spec:
|
|
accessModes:
|
|
- ReadWriteOnce
|
|
storageClassName: longhorn
|
|
resources:
|
|
requests:
|
|
storage: 5Gi
|
|
---
|
|
# PVC sonarr-media (R-STATE)
|
|
apiVersion: v1
|
|
kind: PersistentVolumeClaim
|
|
metadata:
|
|
name: sonarr-media
|
|
namespace: sonarr
|
|
labels:
|
|
app.kubernetes.io/name: sonarr
|
|
spec:
|
|
accessModes:
|
|
- ReadWriteMany
|
|
storageClassName: ''
|
|
volumeName: sonarr-media
|
|
resources:
|
|
requests:
|
|
storage: 9Ti
|
|
---
|
|
# Deployment (R-PLAT-02)
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata:
|
|
name: sonarr
|
|
namespace: sonarr
|
|
labels:
|
|
app.kubernetes.io/name: sonarr
|
|
spec:
|
|
replicas: 1
|
|
strategy:
|
|
type: Recreate
|
|
selector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: sonarr
|
|
template:
|
|
metadata:
|
|
labels:
|
|
app.kubernetes.io/name: sonarr
|
|
homelab.keithjackson.dev/egress-class: vpn
|
|
homelab.keithjackson.dev/auth-mode: forward-auth
|
|
annotations:
|
|
homelab.keithjackson.dev/auth-mode: forward-auth
|
|
homelab.keithjackson.dev/auth-groups: media-users
|
|
spec:
|
|
containers:
|
|
- name: sonarr
|
|
image: lscr.io/linuxserver/sonarr:4.0.15
|
|
ports:
|
|
- containerPort: 8989
|
|
env:
|
|
- name: PUID
|
|
value: '1000'
|
|
- name: PGID
|
|
value: '3000'
|
|
- name: TZ
|
|
value: Etc/UTC
|
|
volumeMounts:
|
|
- name: config
|
|
mountPath: /data/config
|
|
- name: media
|
|
mountPath: /media
|
|
- name: oauth2-proxy
|
|
image: quay.io/oauth2-proxy/oauth2-proxy:v7.8.1
|
|
imagePullPolicy: IfNotPresent
|
|
args:
|
|
- --http-address=0.0.0.0:4180
|
|
- --reverse-proxy=true
|
|
- --provider=oidc
|
|
- --oidc-issuer-url=https://keycloak.${BASE_DOMAIN}/realms/homelab
|
|
- --upstream=http://127.0.0.1:8989
|
|
- --email-domain=*
|
|
- --scope=openid email profile
|
|
- --oidc-groups-claim=groups
|
|
- --allowed-group=media-users
|
|
- --skip-provider-button=true
|
|
- --redirect-url=https://sonarr.${BASE_DOMAIN}/oauth2/callback
|
|
- --cookie-domain=.${BASE_DOMAIN}
|
|
- --whitelist-domain=.${BASE_DOMAIN}
|
|
- --cookie-secure=true
|
|
- --pass-access-token=true
|
|
- --set-xauthrequest=true
|
|
env:
|
|
- name: OAUTH2_PROXY_CLIENT_ID
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: sonarr-oauth2-proxy
|
|
key: client-id
|
|
- name: OAUTH2_PROXY_CLIENT_SECRET
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: sonarr-oauth2-proxy
|
|
key: client-secret
|
|
- name: OAUTH2_PROXY_COOKIE_SECRET
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: sonarr-oauth2-proxy
|
|
key: cookie-secret
|
|
ports:
|
|
- name: http
|
|
containerPort: 4180
|
|
protocol: TCP
|
|
resources:
|
|
requests:
|
|
cpu: 25m
|
|
memory: 64Mi
|
|
limits:
|
|
memory: 128Mi
|
|
securityContext:
|
|
allowPrivilegeEscalation: false
|
|
readOnlyRootFilesystem: true
|
|
capabilities:
|
|
drop:
|
|
- ALL
|
|
livenessProbe:
|
|
httpGet:
|
|
path: /ping
|
|
port: http
|
|
initialDelaySeconds: 5
|
|
periodSeconds: 15
|
|
readinessProbe:
|
|
httpGet:
|
|
path: /ping
|
|
port: http
|
|
initialDelaySeconds: 5
|
|
periodSeconds: 15
|
|
securityContext:
|
|
fsGroup: 3000
|
|
volumes:
|
|
- name: config
|
|
persistentVolumeClaim:
|
|
claimName: sonarr-config
|
|
- name: media
|
|
persistentVolumeClaim:
|
|
claimName: sonarr-media
|
|
---
|
|
# Service
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: sonarr
|
|
namespace: sonarr
|
|
labels:
|
|
app.kubernetes.io/name: sonarr
|
|
spec:
|
|
selector:
|
|
app.kubernetes.io/name: sonarr
|
|
ports:
|
|
- name: port-8989
|
|
port: 8989
|
|
targetPort: 8989
|
|
- name: authproxy
|
|
port: 4180
|
|
targetPort: 4180
|
|
---
|
|
# CiliumNetworkPolicy egress allow (R-NET-02): sonarr-allow-qbittorrent-8080
|
|
apiVersion: cilium.io/v2
|
|
kind: CiliumNetworkPolicy
|
|
metadata:
|
|
name: sonarr-allow-qbittorrent-8080
|
|
namespace: sonarr
|
|
labels:
|
|
app.kubernetes.io/name: sonarr
|
|
spec:
|
|
endpointSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: sonarr
|
|
egress:
|
|
- toEndpoints:
|
|
- matchLabels:
|
|
app.kubernetes.io/name: qbittorrent
|
|
k8s:io.kubernetes.pod.namespace: qbittorrent
|
|
toPorts:
|
|
- ports:
|
|
- port: '8080'
|
|
protocol: TCP
|
|
---
|
|
# CiliumNetworkPolicy egress allow (R-NET-02): sonarr-allow-jackett-9117
|
|
apiVersion: cilium.io/v2
|
|
kind: CiliumNetworkPolicy
|
|
metadata:
|
|
name: sonarr-allow-jackett-9117
|
|
namespace: sonarr
|
|
labels:
|
|
app.kubernetes.io/name: sonarr
|
|
spec:
|
|
endpointSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: sonarr
|
|
egress:
|
|
- toEndpoints:
|
|
- matchLabels:
|
|
app.kubernetes.io/name: jackett
|
|
k8s:io.kubernetes.pod.namespace: jackett
|
|
toPorts:
|
|
- ports:
|
|
- port: '9117'
|
|
protocol: TCP
|
|
---
|
|
# CiliumNetworkPolicy reciprocal ingress allow (R-NET-02): sonarr-allow-seerr-8989
|
|
apiVersion: cilium.io/v2
|
|
kind: CiliumNetworkPolicy
|
|
metadata:
|
|
name: sonarr-allow-seerr-8989
|
|
namespace: sonarr
|
|
labels:
|
|
app.kubernetes.io/name: sonarr
|
|
spec:
|
|
endpointSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: sonarr
|
|
ingress:
|
|
- fromEndpoints:
|
|
- matchLabels:
|
|
app.kubernetes.io/name: seerr
|
|
k8s:io.kubernetes.pod.namespace: seerr
|
|
toPorts:
|
|
- ports:
|
|
- port: '8989'
|
|
protocol: TCP
|
|
---
|
|
# CiliumNetworkPolicy gateway-ingress allow (R-NET-01/02): sonarr-allow-gateway-ingress
|
|
apiVersion: cilium.io/v2
|
|
kind: CiliumNetworkPolicy
|
|
metadata:
|
|
name: sonarr-allow-gateway-ingress
|
|
namespace: sonarr
|
|
labels:
|
|
app.kubernetes.io/name: sonarr
|
|
spec:
|
|
endpointSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: sonarr
|
|
ingress:
|
|
- fromEntities:
|
|
- ingress
|
|
toPorts:
|
|
- ports:
|
|
- port: '4180'
|
|
protocol: TCP
|
|
---
|
|
# CiliumNetworkPolicy world egress allow (egress class governs the path; R-NET-05): sonarr-allow-internet-egress
|
|
apiVersion: cilium.io/v2
|
|
kind: CiliumNetworkPolicy
|
|
metadata:
|
|
name: sonarr-allow-internet-egress
|
|
namespace: sonarr
|
|
labels:
|
|
app.kubernetes.io/name: sonarr
|
|
spec:
|
|
endpointSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: sonarr
|
|
egress:
|
|
- toEntities:
|
|
- world
|
|
- toEndpoints:
|
|
- matchLabels:
|
|
io.kubernetes.pod.namespace: keycloak
|
|
app: keycloak
|
|
---
|
|
# HTTPRoute sonarr-8989 (R-TLS/R-DNS)
|
|
apiVersion: gateway.networking.k8s.io/v1
|
|
kind: HTTPRoute
|
|
metadata:
|
|
name: sonarr-8989
|
|
namespace: sonarr
|
|
labels:
|
|
app.kubernetes.io/name: sonarr
|
|
annotations:
|
|
homelab.keithjackson.dev/auth-mode: forward-auth
|
|
homelab.keithjackson.dev/auth-groups: media-users
|
|
spec:
|
|
parentRefs:
|
|
- name: shared
|
|
namespace: gateway
|
|
sectionName: https
|
|
hostnames:
|
|
- sonarr.${BASE_DOMAIN}
|
|
rules:
|
|
- matches:
|
|
- path:
|
|
type: PathPrefix
|
|
value: /
|
|
backendRefs:
|
|
- name: sonarr
|
|
port: 4180
|
|
---
|
|
# ServiceMonitor (R-OBS-01)
|
|
apiVersion: monitoring.coreos.com/v1
|
|
kind: ServiceMonitor
|
|
metadata:
|
|
name: sonarr
|
|
namespace: sonarr
|
|
labels:
|
|
release: kube-prometheus-stack
|
|
app.kubernetes.io/name: sonarr
|
|
spec:
|
|
selector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: sonarr
|
|
namespaceSelector:
|
|
matchNames:
|
|
- sonarr
|
|
endpoints:
|
|
- port: metrics
|
|
interval: 30s
|
|
---
|
|
# PrometheusRule SLI alerts (R-OBS-01)
|
|
apiVersion: monitoring.coreos.com/v1
|
|
kind: PrometheusRule
|
|
metadata:
|
|
name: sonarr-slis
|
|
namespace: sonarr
|
|
labels:
|
|
release: kube-prometheus-stack
|
|
app.kubernetes.io/name: sonarr
|
|
spec:
|
|
groups:
|
|
- name: sonarr.slis
|
|
rules:
|
|
- alert: sonarr-down
|
|
expr: up{job="sonarr"} == 0
|
|
for: 5m
|
|
labels:
|
|
severity: critical
|
|
service: sonarr
|
|
annotations:
|
|
summary: sonarr is down
|
|
description: 'sonarr has been unreachable for 5m (SLI: up).'
|