Files
home-network-k3s-impl/apps/wave-0/plex.yaml
T
jakman85andClaude Opus 5 78bf68493d feat(gpu): satisfy hardware.gpu: intel-quicksync with gpu.intel.com/i915, not a /dev/dri hostPath
The hostPath mechanism was MEASURED NOT TO WORK. In the live Plex pod on
k3s-prod-2, 2026-10-01: the device node was VISIBLE in the container
(`crw-rw---- root video2 226,128 renderD128`) and could not be opened --
`uid 0 O_RDWR -> EPERM` and `uid 1000 O_RDWR -> EPERM`. EPERM from the DEVICE
CGROUP, not EACCES from file permissions, proven three ways: root on the HOST
opens the same device rc=0; the node is unified cgroup2fs; and the container's
runc config.json read `linux.resources.devices: [{allow: False, access: 'rwm'}]`
with `linux.devices: None` -- one deny-all rule, no allow entry for major 226.
A hostPath is a filesystem operation: it binds the device node in but adds no
cgroup allow rule, because Kubernetes cannot infer one from a volume.

The Intel GPU device plugin produces that allow rule
(`createDeviceSpecsFromDrmFiles` emits `DeviceSpec{..., Permissions: "rw"}`).
It shipped as #36 and is live: all three prod nodes advertise
`gpu.intel.com/i915: 1`, DaemonSet 3/3/3 (measured 2026-10-01).

ZERO ServiceDefinition field changes, which was the entire point of making the
field an enum of capability classes rather than a device path. `catalog/plex.yaml`'s
`hardware:` block comes through byte-identical; only its comment changes, because
it restated the old mechanism. Contract invariant 8 holds, now demonstrated rather
than claimed.

- catalog/render.py: `DRI_VOLUME_NAME`/`DRI_DEVICE_DIR` retired for
  `QUICKSYNC_RESOURCE`; the hardware branch now sets
  `resources.limits["gpu.intel.com/i915"] = 1` on the APP container only (never a
  sidecar) and emits no volume, no mount, no securityContext, no nodeSelector.
  `limits` not `requests` is mandatory: Kubernetes mirrors requests from limits,
  and a requests-only extended resource is invalid. This is the only app-container
  `resources` render path and is deliberately narrow -- no general CPU/memory
  feature was invented.
- selftests: resources dict asserted EXACTLY; hostPath/mount asserted ABSENT; the
  no-`hardware` negative extended to `resources`; the podSecurityContext
  pass-through and the no-synthesised-securityContext assertions kept; one new
  assertion that the limit never reaches the oauth2-proxy sidecar. Mutation-tested:
  8/8 mutants caught (requests-instead-of-limits, emission dropped, hostPath
  re-added, emitted unconditionally, supplementalGroups synthesised, wrong resource
  name, quantity as a string, limit leaked onto the sidecar).
- apps/wave-0/plex.yaml: six hostPath lines removed, a three-line resources block
  added. Only golden touched; keycloak/vm-demo unchanged.

THE REGRESSION, STATED RATHER THAN SOLD. Before: device missing ->
`DirectoryOrCreate` mounted an empty dir -> Plex ran and SILENTLY
software-transcoded (invisible, harmless to everything else). After: resource
unavailable (plugin broken, removed, or not yet reconciled) -> Plex is
UNSCHEDULABLE, and `apps-wave-0` is `wait: true` with `apps-wave-1` dependsOn it,
so that STALLS THE APP TIER. Unavoidable: an extended resource is a hard
scheduling constraint, and it is also the only thing that produces the cgroup
allow rule. There is no graceful-degradation variant. Plex's schedulability is now
coupled to the intel-gpu-plugin DaemonSet. Escape hatch, one field and no code
change: delete the `hardware:` block from catalog/plex.yaml and re-render.
Documented in the contract, the runbook (its own top-of-page section and a
"Plex will not schedule" procedure) and the handoff.

HARDWARE TRANSCODING IS STILL NOT OBSERVED WORKING. The decisive open() test
cannot be re-run until this merges and Flux reconciles, and the Plex-side
`HardwareAcceleratedCodecs` toggle is still unset. A green gate proves the
manifest renders.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-10-01 23:21:28 -05:00

277 lines
5.9 KiB
YAML

---
# Namespace (trust zone; default-deny + policies apply)
apiVersion: v1
kind: Namespace
metadata:
name: plex
labels:
homelab.keithjackson.dev/app-namespace: 'true'
gateway-access: shared
annotations:
homelab.keithjackson.dev/backup-policy: daily-offcluster
---
# Default-deny baseline (R-NET-01): deny all, allow only DNS
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny
namespace: plex
labels:
homelab.keithjackson.dev/baseline: default-deny
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
ingress: []
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
ports:
- protocol: UDP
port: 53
- protocol: TCP
port: 53
---
# PersistentVolume plex-media - static external NFS (bulk tier)
apiVersion: v1
kind: PersistentVolume
metadata:
name: plex-media
labels:
app.kubernetes.io/name: plex
spec:
capacity:
storage: 9Ti
accessModes:
- ReadOnlyMany
persistentVolumeReclaimPolicy: Retain
storageClassName: ''
mountOptions:
- nfsvers=4.1
- hard
- noatime
- timeo=600
- retrans=2
nfs:
server: ${BULK_NFS_SERVER}
path: /mnt/Storage/plex
readOnly: true
claimRef:
namespace: plex
name: plex-media
---
# PVC plex-config (R-STATE)
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: plex-config
namespace: plex
labels:
app.kubernetes.io/name: plex
spec:
accessModes:
- ReadWriteOnce
storageClassName: longhorn
resources:
requests:
storage: 8Gi
---
# PVC plex-media (R-STATE)
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: plex-media
namespace: plex
labels:
app.kubernetes.io/name: plex
spec:
accessModes:
- ReadOnlyMany
storageClassName: ''
volumeName: plex-media
resources:
requests:
storage: 9Ti
---
# Deployment (R-PLAT-02)
apiVersion: apps/v1
kind: Deployment
metadata:
name: plex
namespace: plex
labels:
app.kubernetes.io/name: plex
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app.kubernetes.io/name: plex
template:
metadata:
labels:
app.kubernetes.io/name: plex
homelab.keithjackson.dev/egress-class: internet
homelab.keithjackson.dev/auth-mode: native
spec:
containers:
- name: plex
image: plexinc/pms-docker:latest
ports:
- containerPort: 32400
env:
- name: PLEX_UID
value: '1000'
- name: PLEX_GID
value: '1000'
- name: TZ
value: Etc/UTC
- name: ALLOWED_NETWORKS
value: 10.0.0.0/8,192.168.0.0/16
- name: PLEX_CLAIM
valueFrom:
secretKeyRef:
name: plex-claim
key: PLEX_CLAIM
resources:
limits:
gpu.intel.com/i915: 1
volumeMounts:
- name: config
mountPath: /config
- name: media
mountPath: /media
readOnly: true
volumes:
- name: config
persistentVolumeClaim:
claimName: plex-config
- name: media
persistentVolumeClaim:
claimName: plex-media
---
# Service
apiVersion: v1
kind: Service
metadata:
name: plex
namespace: plex
labels:
app.kubernetes.io/name: plex
spec:
selector:
app.kubernetes.io/name: plex
ports:
- name: port-32400
port: 32400
targetPort: 32400
---
# CiliumNetworkPolicy reciprocal ingress allow (R-NET-02): plex-allow-seerr-32400
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: plex-allow-seerr-32400
namespace: plex
labels:
app.kubernetes.io/name: plex
spec:
endpointSelector:
matchLabels:
app.kubernetes.io/name: plex
ingress:
- fromEndpoints:
- matchLabels:
app.kubernetes.io/name: seerr
k8s:io.kubernetes.pod.namespace: seerr
toPorts:
- ports:
- port: '32400'
protocol: TCP
---
# CiliumNetworkPolicy gateway-ingress allow (R-NET-01/02): plex-allow-gateway-ingress
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: plex-allow-gateway-ingress
namespace: plex
labels:
app.kubernetes.io/name: plex
spec:
endpointSelector:
matchLabels:
app.kubernetes.io/name: plex
ingress:
- fromEntities:
- ingress
toPorts:
- ports:
- port: '32400'
protocol: TCP
---
# CiliumNetworkPolicy world egress allow (egress class governs the path; R-NET-05): plex-allow-internet-egress
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: plex-allow-internet-egress
namespace: plex
labels:
app.kubernetes.io/name: plex
spec:
endpointSelector:
matchLabels:
app.kubernetes.io/name: plex
egress:
- toEntities:
- world
---
# HTTPRoute plex-32400 (R-TLS/R-DNS)
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: plex-32400
namespace: plex
labels:
app.kubernetes.io/name: plex
spec:
parentRefs:
- name: shared
namespace: gateway
sectionName: https
hostnames:
- plex.${BASE_DOMAIN}
rules:
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: plex
port: 32400
---
# PrometheusRule SLI alerts (R-OBS-01)
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
name: plex-slis
namespace: plex
labels:
release: kube-prometheus-stack
app.kubernetes.io/name: plex
spec:
groups:
- name: plex.slis
rules:
- alert: plex-down
expr: kube_deployment_status_replicas_available{namespace="plex", deployment="plex"} == 0 or absent(kube_deployment_status_replicas_available{namespace="plex", deployment="plex"})
for: 5m
labels:
severity: critical
service: plex
annotations:
summary: plex is down
description: 'plex: its Deployment has no available/ready replica for 5m, or the signal is gone entirely (SLI: up, via kube_deployment_status_replicas_available).'