The hostPath mechanism was MEASURED NOT TO WORK. In the live Plex pod on
k3s-prod-2, 2026-10-01: the device node was VISIBLE in the container
(`crw-rw---- root video2 226,128 renderD128`) and could not be opened --
`uid 0 O_RDWR -> EPERM` and `uid 1000 O_RDWR -> EPERM`. EPERM from the DEVICE
CGROUP, not EACCES from file permissions, proven three ways: root on the HOST
opens the same device rc=0; the node is unified cgroup2fs; and the container's
runc config.json read `linux.resources.devices: [{allow: False, access: 'rwm'}]`
with `linux.devices: None` -- one deny-all rule, no allow entry for major 226.
A hostPath is a filesystem operation: it binds the device node in but adds no
cgroup allow rule, because Kubernetes cannot infer one from a volume.
The Intel GPU device plugin produces that allow rule
(`createDeviceSpecsFromDrmFiles` emits `DeviceSpec{..., Permissions: "rw"}`).
It shipped as #36 and is live: all three prod nodes advertise
`gpu.intel.com/i915: 1`, DaemonSet 3/3/3 (measured 2026-10-01).
ZERO ServiceDefinition field changes, which was the entire point of making the
field an enum of capability classes rather than a device path. `catalog/plex.yaml`'s
`hardware:` block comes through byte-identical; only its comment changes, because
it restated the old mechanism. Contract invariant 8 holds, now demonstrated rather
than claimed.
- catalog/render.py: `DRI_VOLUME_NAME`/`DRI_DEVICE_DIR` retired for
`QUICKSYNC_RESOURCE`; the hardware branch now sets
`resources.limits["gpu.intel.com/i915"] = 1` on the APP container only (never a
sidecar) and emits no volume, no mount, no securityContext, no nodeSelector.
`limits` not `requests` is mandatory: Kubernetes mirrors requests from limits,
and a requests-only extended resource is invalid. This is the only app-container
`resources` render path and is deliberately narrow -- no general CPU/memory
feature was invented.
- selftests: resources dict asserted EXACTLY; hostPath/mount asserted ABSENT; the
no-`hardware` negative extended to `resources`; the podSecurityContext
pass-through and the no-synthesised-securityContext assertions kept; one new
assertion that the limit never reaches the oauth2-proxy sidecar. Mutation-tested:
8/8 mutants caught (requests-instead-of-limits, emission dropped, hostPath
re-added, emitted unconditionally, supplementalGroups synthesised, wrong resource
name, quantity as a string, limit leaked onto the sidecar).
- apps/wave-0/plex.yaml: six hostPath lines removed, a three-line resources block
added. Only golden touched; keycloak/vm-demo unchanged.
THE REGRESSION, STATED RATHER THAN SOLD. Before: device missing ->
`DirectoryOrCreate` mounted an empty dir -> Plex ran and SILENTLY
software-transcoded (invisible, harmless to everything else). After: resource
unavailable (plugin broken, removed, or not yet reconciled) -> Plex is
UNSCHEDULABLE, and `apps-wave-0` is `wait: true` with `apps-wave-1` dependsOn it,
so that STALLS THE APP TIER. Unavoidable: an extended resource is a hard
scheduling constraint, and it is also the only thing that produces the cgroup
allow rule. There is no graceful-degradation variant. Plex's schedulability is now
coupled to the intel-gpu-plugin DaemonSet. Escape hatch, one field and no code
change: delete the `hardware:` block from catalog/plex.yaml and re-render.
Documented in the contract, the runbook (its own top-of-page section and a
"Plex will not schedule" procedure) and the handoff.
HARDWARE TRANSCODING IS STILL NOT OBSERVED WORKING. The decisive open() test
cannot be re-run until this merges and Flux reconciles, and the Plex-side
`HardwareAcceleratedCodecs` toggle is still unset. A green gate proves the
manifest renders.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
277 lines
5.9 KiB
YAML
277 lines
5.9 KiB
YAML
---
|
|
# Namespace (trust zone; default-deny + policies apply)
|
|
apiVersion: v1
|
|
kind: Namespace
|
|
metadata:
|
|
name: plex
|
|
labels:
|
|
homelab.keithjackson.dev/app-namespace: 'true'
|
|
gateway-access: shared
|
|
annotations:
|
|
homelab.keithjackson.dev/backup-policy: daily-offcluster
|
|
---
|
|
# Default-deny baseline (R-NET-01): deny all, allow only DNS
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: NetworkPolicy
|
|
metadata:
|
|
name: default-deny
|
|
namespace: plex
|
|
labels:
|
|
homelab.keithjackson.dev/baseline: default-deny
|
|
spec:
|
|
podSelector: {}
|
|
policyTypes:
|
|
- Ingress
|
|
- Egress
|
|
ingress: []
|
|
egress:
|
|
- to:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: kube-system
|
|
ports:
|
|
- protocol: UDP
|
|
port: 53
|
|
- protocol: TCP
|
|
port: 53
|
|
---
|
|
# PersistentVolume plex-media - static external NFS (bulk tier)
|
|
apiVersion: v1
|
|
kind: PersistentVolume
|
|
metadata:
|
|
name: plex-media
|
|
labels:
|
|
app.kubernetes.io/name: plex
|
|
spec:
|
|
capacity:
|
|
storage: 9Ti
|
|
accessModes:
|
|
- ReadOnlyMany
|
|
persistentVolumeReclaimPolicy: Retain
|
|
storageClassName: ''
|
|
mountOptions:
|
|
- nfsvers=4.1
|
|
- hard
|
|
- noatime
|
|
- timeo=600
|
|
- retrans=2
|
|
nfs:
|
|
server: ${BULK_NFS_SERVER}
|
|
path: /mnt/Storage/plex
|
|
readOnly: true
|
|
claimRef:
|
|
namespace: plex
|
|
name: plex-media
|
|
---
|
|
# PVC plex-config (R-STATE)
|
|
apiVersion: v1
|
|
kind: PersistentVolumeClaim
|
|
metadata:
|
|
name: plex-config
|
|
namespace: plex
|
|
labels:
|
|
app.kubernetes.io/name: plex
|
|
spec:
|
|
accessModes:
|
|
- ReadWriteOnce
|
|
storageClassName: longhorn
|
|
resources:
|
|
requests:
|
|
storage: 8Gi
|
|
---
|
|
# PVC plex-media (R-STATE)
|
|
apiVersion: v1
|
|
kind: PersistentVolumeClaim
|
|
metadata:
|
|
name: plex-media
|
|
namespace: plex
|
|
labels:
|
|
app.kubernetes.io/name: plex
|
|
spec:
|
|
accessModes:
|
|
- ReadOnlyMany
|
|
storageClassName: ''
|
|
volumeName: plex-media
|
|
resources:
|
|
requests:
|
|
storage: 9Ti
|
|
---
|
|
# Deployment (R-PLAT-02)
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata:
|
|
name: plex
|
|
namespace: plex
|
|
labels:
|
|
app.kubernetes.io/name: plex
|
|
spec:
|
|
replicas: 1
|
|
strategy:
|
|
type: Recreate
|
|
selector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: plex
|
|
template:
|
|
metadata:
|
|
labels:
|
|
app.kubernetes.io/name: plex
|
|
homelab.keithjackson.dev/egress-class: internet
|
|
homelab.keithjackson.dev/auth-mode: native
|
|
spec:
|
|
containers:
|
|
- name: plex
|
|
image: plexinc/pms-docker:latest
|
|
ports:
|
|
- containerPort: 32400
|
|
env:
|
|
- name: PLEX_UID
|
|
value: '1000'
|
|
- name: PLEX_GID
|
|
value: '1000'
|
|
- name: TZ
|
|
value: Etc/UTC
|
|
- name: ALLOWED_NETWORKS
|
|
value: 10.0.0.0/8,192.168.0.0/16
|
|
- name: PLEX_CLAIM
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: plex-claim
|
|
key: PLEX_CLAIM
|
|
resources:
|
|
limits:
|
|
gpu.intel.com/i915: 1
|
|
volumeMounts:
|
|
- name: config
|
|
mountPath: /config
|
|
- name: media
|
|
mountPath: /media
|
|
readOnly: true
|
|
volumes:
|
|
- name: config
|
|
persistentVolumeClaim:
|
|
claimName: plex-config
|
|
- name: media
|
|
persistentVolumeClaim:
|
|
claimName: plex-media
|
|
---
|
|
# Service
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: plex
|
|
namespace: plex
|
|
labels:
|
|
app.kubernetes.io/name: plex
|
|
spec:
|
|
selector:
|
|
app.kubernetes.io/name: plex
|
|
ports:
|
|
- name: port-32400
|
|
port: 32400
|
|
targetPort: 32400
|
|
---
|
|
# CiliumNetworkPolicy reciprocal ingress allow (R-NET-02): plex-allow-seerr-32400
|
|
apiVersion: cilium.io/v2
|
|
kind: CiliumNetworkPolicy
|
|
metadata:
|
|
name: plex-allow-seerr-32400
|
|
namespace: plex
|
|
labels:
|
|
app.kubernetes.io/name: plex
|
|
spec:
|
|
endpointSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: plex
|
|
ingress:
|
|
- fromEndpoints:
|
|
- matchLabels:
|
|
app.kubernetes.io/name: seerr
|
|
k8s:io.kubernetes.pod.namespace: seerr
|
|
toPorts:
|
|
- ports:
|
|
- port: '32400'
|
|
protocol: TCP
|
|
---
|
|
# CiliumNetworkPolicy gateway-ingress allow (R-NET-01/02): plex-allow-gateway-ingress
|
|
apiVersion: cilium.io/v2
|
|
kind: CiliumNetworkPolicy
|
|
metadata:
|
|
name: plex-allow-gateway-ingress
|
|
namespace: plex
|
|
labels:
|
|
app.kubernetes.io/name: plex
|
|
spec:
|
|
endpointSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: plex
|
|
ingress:
|
|
- fromEntities:
|
|
- ingress
|
|
toPorts:
|
|
- ports:
|
|
- port: '32400'
|
|
protocol: TCP
|
|
---
|
|
# CiliumNetworkPolicy world egress allow (egress class governs the path; R-NET-05): plex-allow-internet-egress
|
|
apiVersion: cilium.io/v2
|
|
kind: CiliumNetworkPolicy
|
|
metadata:
|
|
name: plex-allow-internet-egress
|
|
namespace: plex
|
|
labels:
|
|
app.kubernetes.io/name: plex
|
|
spec:
|
|
endpointSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: plex
|
|
egress:
|
|
- toEntities:
|
|
- world
|
|
---
|
|
# HTTPRoute plex-32400 (R-TLS/R-DNS)
|
|
apiVersion: gateway.networking.k8s.io/v1
|
|
kind: HTTPRoute
|
|
metadata:
|
|
name: plex-32400
|
|
namespace: plex
|
|
labels:
|
|
app.kubernetes.io/name: plex
|
|
spec:
|
|
parentRefs:
|
|
- name: shared
|
|
namespace: gateway
|
|
sectionName: https
|
|
hostnames:
|
|
- plex.${BASE_DOMAIN}
|
|
rules:
|
|
- matches:
|
|
- path:
|
|
type: PathPrefix
|
|
value: /
|
|
backendRefs:
|
|
- name: plex
|
|
port: 32400
|
|
---
|
|
# PrometheusRule SLI alerts (R-OBS-01)
|
|
apiVersion: monitoring.coreos.com/v1
|
|
kind: PrometheusRule
|
|
metadata:
|
|
name: plex-slis
|
|
namespace: plex
|
|
labels:
|
|
release: kube-prometheus-stack
|
|
app.kubernetes.io/name: plex
|
|
spec:
|
|
groups:
|
|
- name: plex.slis
|
|
rules:
|
|
- alert: plex-down
|
|
expr: kube_deployment_status_replicas_available{namespace="plex", deployment="plex"} == 0 or absent(kube_deployment_status_replicas_available{namespace="plex", deployment="plex"})
|
|
for: 5m
|
|
labels:
|
|
severity: critical
|
|
service: plex
|
|
annotations:
|
|
summary: plex is down
|
|
description: 'plex: its Deployment has no available/ready replica for 5m, or the signal is gone entirely (SLI: up, via kube_deployment_status_replicas_available).'
|