Files
home-network-k3s-impl/decisions/2026-09-30-plex-quicksync.md
T
jakman85andClaude Opus 5 7a9d37e601
gate / gate (pull_request) Successful in 1m42s
gate / conformance (pull_request) Has been skipped
feat(plex): opt into hardware.gpu: intel-quicksync, and write the runbook for the step GitOps cannot reach
Plex has been transcoding in software on hardware that can do it in the iGPU. This
opts its ServiceDefinition into the `hardware` capability from #27 and re-renders.

catalog/plex.yaml gains four lines of YAML and the comment that explains them:

    hardware:
      gpu: intel-quicksync

which renders exactly two things into apps/wave-0/plex.yaml and nothing else — a
`dri` hostPath volume (/dev/dri, type: DirectoryOrCreate) and a read-write /dev/dri
volumeMount on the app container. Whole-tree render diff: 6 added lines, one file.

The comment records what is not obvious from the four lines: the device exists on
ALL THREE prod nodes with i915 bound (so no nodeSelector is rendered and none is
wanted); why `supplementalGroups` would be INERT rather than a hardening (the
image's cont-init hook adds the plex user to the device's group inside the
container, and s6-setuidgid then REPLACES the process group list from /etc/group,
discarding anything Kubernetes supplied); and why `DirectoryOrCreate` is
load-bearing (apps-wave-0 is wait:true with apps-wave-1 depending on it, so a
failed mount would stall the whole app tier for a feature that is not).

THE FEATURE IS INERT UNTIL A HUMAN CLICKS A BOX, and that is the part this commit
is really about. Mounting the device does not enable hardware transcoding; Plex's
`HardwareAcceleratedCodecs` preference does, and it is ABSENT from Preferences.xml
— the toggle has never been on. It cannot be automated: a bounded grep across the
running image's entire init surface (/etc/cont-init.d, /etc/services.d,
/etc/fix-attrs.d, /plex-common.sh, /usr/local/bin) found ZERO PLEX_PREFERENCE*
support, and the six setPref keys the image does write are all unrelated. So
runbooks/plex-quicksync.md is the procedure, registered in the configMapGenerator
(a runbook in git with no generator entry is reviewed, merged, and still a 404) and
indexed in runbooks/README.md. It covers the GitOps/manual boundary, the toggle and
the real preference keys (including the widely-cited one that does not exist and
the upstream misspelling nobody should "fix"), verification via `Transcode (hw)` on
the Plex dashboard — stated plainly as the ONLY proof, because Plex renders no
ServiceMonitor and an alert could never fire — the device-cgroup failure to expect
first with its Intel-device-plugin fallback, and the floating-`:latest` caveat.

Stacked on #27 (feat/sd-hardware-capability), which is open and unmerged; that
branch's commit is in this range by design and this must merge after it.

NOT VERIFIED, and the handoff says so at length: hardware transcoding has not been
observed working. Whether containerd's device cgroup lets a non-privileged pod
open() a bind-mounted char device is an open question that only deploying settles,
and the Plex-side toggle was still unset at the time of writing. A green gate
proves the manifest renders; it proves nothing about transcoding.

Gate (pinned toolbox, worktree): rc 3 — UNDETERMINED, not green. Passed:
decisions-log, lint, l0, render-golden, l1, approval-policy. Skipped (the known
local ceiling): spec-immutability (no sibling spec repo), gitea-settings (no
SETTINGS_API_TOKEN), conformance (CLUSTER_AVAILABLE unset, so L2/L3 did not run).
No fourth skip, nothing failed.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-30 13:09:20 -05:00

10 KiB


date: 2026-09-30 branch: feat/plex-quicksync title: Plex opts into hardware.gpu: intel-quicksync — the device mount is GitOps, the Plex-side toggle is a documented manual step, and hardware transcoding has NOT been observed working

What changed

Three things, and the third is the point of the branch.

  1. catalog/plex.yaml — added the four lines the whole feature reduces to:

    hardware:
      gpu: intel-quicksync
    

    placed between env and exposure to match the schema's own property order, with ~48 lines of comment in the file's established style. The comment records: the field is intent, not mechanism (a later move to the Intel device plugin is a render.py change and this block does not move); that /dev/dri/renderD128 is present on all three prod nodes with i915 bound; why a supplementalGroups entry would be inert (the image's cont-init hook does the group work inside the container and s6-setuidgid replaces the process group list from /etc/group); why DirectoryOrCreate is the load-bearing safe failure mode (apps-wave-0 is wait: true with apps-wave-1 depending on it, so a failed mount would stall the whole app tier for a feature that is not load-bearing); and that the Plex-side toggle is a separate manual operator step, pointing at the new runbook.

  2. apps/wave-0/plex.yaml — re-rendered. Six added lines, nothing else, in the whole apps/ tree:

            - name: dri
              mountPath: /dev/dri
          - name: dri
            hostPath:
              path: /dev/dri
              type: DirectoryOrCreate
    
  3. runbooks/plex-quicksync.md (new), registered in runbooks/kustomization.yaml's configMapGenerator as - plex-quicksync=plex-quicksync.md and indexed in runbooks/README.md under "Operator procedures". The file alone would have been a 404 — the pod serves a ConfigMap, not the git tree, and conformance/tests/l0-static/alert-runbooks-resolvable exists precisely because "the file is in git" and "the URL resolves" are different claims. The page covers: the GitOps/manual boundary as a table; the toggle (Settings → Transcoder → "Use hardware acceleration when available", preference key HardwareAcceleratedCodecs); the real neighbouring keys and the widely-cited one that does not exist; that the only proof is Transcode (hw) on the Plex dashboard and why no alert is possible; the device-cgroup failure to expect first with a concrete open() diagnostic and the Intel-device-plugin fallback; and the floating-:latest caveat with a digest check.

Stacked on PR #27 (feat/sd-hardware-capability), which is open and unmerged. This branch was cut from it rather than from main, so its one commit (d9baa2a, the schema + renderer + contract) is in this PR's range. That is deliberate: merging #27 is not an agent's action (CLAUDE.md §1), and duplicating its changes here would have been worse than showing them. This must merge after #27.

Not done, deliberately: no --emit-waves (it writes into clusters/base/, a sensitive path that would force a human reviewer, and it is a no-op unless the set of files under apps/ changes — it does not here). No podSecurityContext. No nodeSelector. No alert rule. No kubectl apply.

Two follow-ups left unclaimed, with the reasoning

  • Transcode scratch is on the wrong medium, and it is invisible. The image sets TranscoderTempDirectory=/transcode, which is not a volume — it is the container's writable overlay, i.e. the node's /data. That is NVMe on prod-2 but SATA on prod-1 and prod-3, so transcode scratch performance changes silently on reschedule, with nothing reporting it. Both candidate fixes need a schema change and so neither belongs in this PR: pin Plex to prod-2 (needs a nodeSelector field the schema does not have), or give /transcode a medium: Memory emptyDir and transcode in RAM (~31 GiB per node; needs an emptyDir field the schema does not have). The RAM option is the better one — it removes the node asymmetry outright instead of working around it — which is exactly why it deserves its own PR and its own review rather than being smuggled in behind a four-line catalog change. Documented in the runbook's "Known gaps".
  • GuC/HuC firmware is not loaded (i915.enable_guc=0). The cost is low-power H.264 encode quality only; decode and this feature are unaffected. Changing it needs a kernel parameter plus a serialized rolling reboot — an operator action under CLAUDE.md §5, on a cluster with a documented history of that going badly (a prod node hung on a warm reboot on 2026-09-03). Kept well clear of this PR.

What I verified

  • The render is exactly the intended change, and nothing else moved. After python3 catalog/render.py --all catalog -o apps --wave-layout --no-image-automation (exit 0, silent): git diff --stat apps/ = apps/wave-0/plex.yaml | 6 ++++++, and git status --porcelain apps/ = a single M apps/wave-0/plex.yaml. Read line by line, the diff is the dri volumeMount and the dri hostPath volume, and nothing more. Specifically checked: the same invocation also rewrites apps/wave-1/keycloak.yaml and apps/wave-0/vm-demo.yaml, which are git-tracked but excluded from the golden comparison by RENDER_GOLDEN_NOT_ONBOARDED in ci/render-golden.sh — a spurious diff there would have landed ungated. Neither file shows a diff; whole-repo git status --porcelain lists only apps/wave-0/plex.yaml and catalog/plex.yaml plus my new files.
  • The rendered output matches what the contract promises — schemas/renderer-contract.md ### hardware says exactly two things (a /dev/dri hostPath with type: DirectoryOrCreate and a read-write mount on the app container) and exactly two things appeared. No securityContext, no nodeSelector, no resources in the diff.
  • The runbook is projected into the server, not merely committed. runbooks/kustomization.yaml carries - plex-quicksync=plex-quicksync.md, whose shape matches the generator-entry regex the L0 check parses; the extensionless key is what makes /runbooks/plex-quicksync a plain static lookup. Also indexed in runbooks/README.md.
  • The runbook clears the L0 quality floor that alert-runbooks-resolvable applies to alert-linked pages, even though nothing links to this one (it is advisory for an unreferenced page): all four required sections (## What this means here, ## First commands, ## Likely causes, most likely first, ## When to escalate or stop), well over the 1500-byte floor, dev.sh used throughout, and --context k3s-prod on every command.
  • The gate. See the run recorded in the PR description for the per-stage table and the real rc.
  • No secret was printed. Every Preferences.xml read in the runbook greps one named key, and the page states explicitly why: the file holds PlexOnlineToken. Nothing was sops -d'd.

What I did NOT verify

Hardware transcoding has NOT been observed working. Nothing in this branch is evidence that it does. A green gate here proves the manifest renders and the golden matches — it proves nothing about runtime behaviour, and vars.CLUSTER_AVAILABLE is unset so L2/L3 conformance did not run at all.

  • The device-cgroup question is OPEN, and it is the largest remaining technical risk. Whether containerd on this k3s build permits a non-privileged pod to open() a bind-mounted character device is UNVERIFIED. It can only be settled by deploying and trying. The container inits as uid 0, which settles file permissions, but uid 0 does not bypass the cgroup device gate — and the two failures look identical from the outside. If it is refused, the fallback is the Intel Device Plugin (gpu.intel.com/i915), which is a catalog/render.py change plus a re-render with no ServiceDefinition edit; privileged: true is not available (Kyverno restrict-host-namespaces is Enforce). The runbook has the diagnostic sequence.
  • The Plex-side toggle was still unset at the time of writing. HardwareAcceleratedCodecs was absent from Preferences.xml — the toggle has never been enabled on this server. Until an operator ticks it in the UI, this branch changes the pod spec and nothing about transcoding. That is the whole reason the runbook exists.
  • No metric, so no alert, so no automated detection — permanently. catalog/plex.yaml declares no metricsEndpoint, so Plex renders no ServiceMonitor; there is no transcode-mode signal for a rule to evaluate. A silent degradation to software transcoding (a node losing its iGPU, the DirectoryOrCreate path) will not be reported by anything. I deliberately did not add an alert: with no metric behind it, it could never fire, and this repo has been burned at scale by exactly that. Closing the gap honestly means an exporter first.
  • I did not re-derive this session's node or in-container measurements. The three-node renderD128/i915/GID-992 facts, the cont-init hook behaviour, the s6-setuidgid group replacement, the PLEX_PREFERENCE* absence, the six setPref keys and the HardwareAcceleratedCodecs string were established earlier in this session and are cited, not re-measured, by me. Per CLAUDE.md §2b, treat this file as a record of belief and re-measure against the cluster before relying on any of it.
  • The image is a floating tag. Everything known about the hook came from digest sha256:e0ab27395614… (plexmediaserver 1.43.4.10903), while the Deployment runs plexinc/pms-docker:latest with imagePullPolicy: Always. A silent pull can invalidate the "no supplementalGroups needed" conclusion with no commit in this repo. Not guarded against — the runbook tells the reader to re-check the hook after an image bump, which is documentation, not enforcement.
  • I did not verify the live Gitea gate state (branch protection, required contexts, runner online) for this PR. Per §2b that must be queried, not read out of CLAUDE.md's table; I did not query it.
  • /runbooks/plex-quicksync was not fetched. The L0 check asserts every static precondition for an HTTP 200; it does not make the request, and I did not either.