home-network-k3s-impl/runbooks/plex-quicksync.md

# plex-quicksync — making Plex actually use Intel Quick Sync, and proving it did

**Alert:** none, and none is possible — see "How to verify" below. This is an **operator procedure**,
not an incident page. It is indexed here because the feature it completes is **inert until a human
clicks one button**, and a switched-off feature that looks shipped is exactly what this page exists
to prevent.
**Files:** `catalog/plex.yaml` (`hardware.gpu: intel-quicksync`), `apps/wave-0/plex.yaml` (the
rendered golden — the `dri` hostPath volume and the `/dev/dri` mount), `catalog/render.py` (the
mechanism), `schemas/renderer-contract.md` `### hardware` (the contract).
**Related:** `runbooks/plex-claim.md` — the other Plex procedure with a manual last mile.

## What this means here

Plex transcodes in **software** by default. On the UHD Graphics 630 iGPU of an i7-9700T, Quick Sync
moves that work off the CPU — which matters on a three-node cluster where the CPU is also running
everything else. Two independent things have to be true before that happens, and **only one of them
is in git**:

| Half | Who does it | State |
|---|---|---|
| The render device is inside the container at `/dev/dri` | **GitOps.** `hardware.gpu: intel-quicksync` in `catalog/plex.yaml` renders a `dri` hostPath volume (`/dev/dri`, `type: DirectoryOrCreate`) and a read-write `/dev/dri` volumeMount on the app container. Flux applies it. | Done by merging that change. |
| Plex is *told* to use it | **The operator, by hand, in the Plex web UI.** | **NOT done.** `HardwareAcceleratedCodecs` was **absent** from `Preferences.xml` when this page was written — the toggle has never been on. |

**There is no GitOps path to the second half, and that is verified rather than assumed.** A bounded
grep across the running image's entire init surface — `/etc/cont-init.d`, `/etc/services.d`,
`/etc/fix-attrs.d`, `/plex-common.sh`, `/usr/local/bin` — found **zero** `PLEX_PREFERENCE*` support.
Every `setPref` the image performs is one of six hardcoded keys (`MachineIdentifier`,
`ProcessedMachineIdentifier`, `PlexOnlineToken`, `customConnections`, `allowedNetworks`,
`TranscoderTempDirectory`), none of them the transcode toggle. No environment variable, no ConfigMap
and no `postBuild` substitution can set it. So do not go looking for the clever way to automate this;
there isn't one, and the search has already been done.

What the device mount does **not** do, so nobody reads more into a green gate than is there: it does
not enable hardware transcoding, it does not prove the device can be *opened* from inside a
non-privileged pod (see "Likely causes" #2), and it renders no `nodeSelector` — Plex can land on any
of the three nodes and all three carry the device.

## Running the commands in this runbook

`kubectl` does not work on the operator workstation; the toolchain lives only in the pinned dev
container. Every command below runs as:

    infra/devcontainer/dev.sh run bash -c 'kubectl --context k3s-prod get nodes'

Always pass `--context k3s-prod` explicitly. `dev.sh` merges both clusters into one kubeconfig and
picks a current-context **per invocation**, and that default has already changed once. Do not run two
`dev.sh` invocations at once.

**Never `cat` `Preferences.xml`.** It contains `PlexOnlineToken`, a plex.tv account credential. Every
read of that file below greps a single named key — do not widen one.

## First commands

**1. Is the device mounted into the pod at all?** (Proves Flux applied the change, not that it works.)

```
infra/devcontainer/dev.sh run bash -c \
  'kubectl --context k3s-prod -n plex get pod -l app.kubernetes.io/name=plex \
     -o jsonpath="{.items[0].spec.containers[0].volumeMounts[*].mountPath}"; echo'
```

`/dev/dri` in that list ⇒ the GitOps half landed. Absent ⇒ the running pod predates the merge; compare

```
infra/devcontainer/dev.sh run bash -c \
  'kubectl --context k3s-prod -n flux-system get kustomization apps-wave-0 \
     -o jsonpath="{.status.lastAppliedRevision}"; echo'
```

against the merge commit before blaming anything else.

**2. Is the device actually visible inside the container?**

```
infra/devcontainer/dev.sh run bash -c \
  'kubectl --context k3s-prod -n plex exec deploy/plex -- ls -l /dev/dri'
```

Expect `renderD128` (usually `card0` too), mode `crw-rw----`, group `render`. An **empty directory**
here is the `DirectoryOrCreate` degradation: the kubelet created an empty `/dev/dri` because the node
had none. That is the designed-safe outcome — the app tier still reconciles — but it means software
transcoding, silently. Find which node the pod is on and ask why that node lost its iGPU:

```
infra/devcontainer/dev.sh run bash -c \
  'kubectl --context k3s-prod -n plex get pod -l app.kubernetes.io/name=plex -o wide'
```

**3. Can Plex *open* it?** This is the question the manifest cannot answer — see "Likely causes" #2.

```
infra/devcontainer/dev.sh run bash -c \
  'kubectl --context k3s-prod -n plex exec deploy/plex -- \
     bash -c "exec 3<>/dev/dri/renderD128 && echo RW-OPEN-OK || echo RW-OPEN-DENIED"'
```

`RW-OPEN-DENIED` (or `Operation not permitted` / `Permission denied`) with the device plainly visible
in step 2 is the **device-cgroup** failure, not a file-permission failure. The two look identical from
the outside and are not the same problem.

**4. Is the toggle on?** Named key only — never the whole file:

```
infra/devcontainer/dev.sh run bash -c \
  'kubectl --context k3s-prod -n plex exec deploy/plex -- \
     grep -o "HardwareAcceleratedCodecs=\"[01]\"" \
     "/config/Library/Application Support/Plex Media Server/Preferences.xml" || echo ABSENT'
```

`ABSENT` ⇒ never enabled (the state this page was written in). `="0"` ⇒ explicitly off. `="1"` ⇒ on,
and the remaining question is whether it *works* — go to "How to verify".

## The one manual step

Plex web UI → **Settings → Transcoder → "Use hardware acceleration when available"** → tick → Save.
(It sits under *Show Advanced*.) A transcode already in flight is unaffected; the next one picks it up.

The underlying preference key is **`HardwareAcceleratedCodecs`**. That name was confirmed by
extracting strings from the 23 MB `Plex Media Server` binary in the running container — **not** from
recall and not from a forum post. Real neighbouring keys from the same binary, so you can recognise
them and not invent them:

| Key | What it is |
|---|---|
| `HardwareAcceleratedCodecs` | the toggle this page is about |
| `HardwareAcceleratedEncoders` | hardware **encode**, as distinct from decode |
| `HardwareDevicePath` | how Plex is pointed at a *specific* render node — relevant only if a host ever exposes more than one |
| `TranscoderToneMapping`, `TranscoderToneMappingAgorithm` | HDR→SDR tone mapping. **That misspelling of "Algorithm" is upstream's, in the binary. It is not a typo in this page — do not "fix" it** here or anywhere else; the corrected spelling is not a key Plex reads. |
| `TranscoderHEVCEncoding` | HEVC output |

**`HardwareAcceleratedCodecsEnableHEVC` does not exist.** It is widely cited and it is not in the
binary. Recorded explicitly so nobody spends an evening hunting for it or, worse, writes it into
`Preferences.xml` and concludes hardware transcoding is broken.

Editing `Preferences.xml` by hand is **not** the procedure. It is app state on the replicated Longhorn
tier, it is in no git commit, and the UI writes the key correctly. Use the UI.

## How to verify — and why there is no alert

Start a playback that **forces a transcode** (a client that cannot direct-play the source, or pick a
lower quality explicitly), then read Plex's own **Dashboard → Activity**:

- **`Transcode (hw)`** on the session ⇒ hardware transcoding. **This is the proof.**
- plain **`Transcode`** ⇒ software. The feature is not working, whatever the manifest says.

**That dashboard is the *only* proof available, and nothing else will ever tell you.**
`catalog/plex.yaml` declares no `metricsEndpoint`, so Plex renders **no ServiceMonitor at all** —
there is no scrape, no transcode-mode metric, and therefore no alert that could fire. Do not open work
to "alert on software-transcode fallback": with no metric behind it such a rule would be permanently
inert, and this repo has already been burned at scale by rules that evaluated against nothing while
looking like coverage (`runbooks/README.md`, "Alerts you should not trust the silence of"). If that
gap is ever worth closing, the honest order is a real exporter first and a rule second.

Corroborating, not a substitute — the server log on the config PVC:

```
infra/devcontainer/dev.sh run bash -c \
  'kubectl --context k3s-prod -n plex exec deploy/plex -- \
     sh -c "grep -i -m20 -e hardware -e vaapi -e hwdevice \
     \"/config/Library/Application Support/Plex Media Server/Logs/Plex Media Server.log\""'
```

Treat those strings as advisory: they are Plex's, not this repo's, and they move between versions.
Absence of a match is **not** evidence of software transcoding; a VA-API device open is good evidence
of hardware. The dashboard remains the verdict.

## Likely causes, most likely first

When the dashboard still says plain `Transcode`:

1. **The toggle is off.** Overwhelmingly the most likely answer, and the reason this page exists.
   First-command #4 returns `ABSENT` or `="0"`. Remedy: the manual step above. Check this **before**
   investigating anything technical.

2. **containerd's device cgroup is refusing the `open()` — UNVERIFIED, and the largest remaining
   technical risk in this feature.** Whether a non-privileged pod may `open()` a bind-mounted
   character device on this k3s/containerd build **has not been established**; it can only be settled
   by deploying and trying. The container inits as uid 0, which settles *file* permissions, but
   **uid 0 does not bypass the cgroup device gate.** Symptom: step 2 shows `renderD128` present and
   correctly owned, step 3 returns `RW-OPEN-DENIED`.
   **The documented fallback is the Intel Device Plugin for Kubernetes** (`gpu.intel.com/i915` as a
   resource request, no hostPath). Because the ServiceDefinition field expresses *intent* rather than
   a device path, that migration is a **`catalog/render.py` change plus a re-render — with no
   ServiceDefinition edit anywhere**; `catalog/plex.yaml` keeps saying exactly what it says today.
   **`privileged: true` is not an available escape hatch:** Kyverno `restrict-host-namespaces` is in
   **Enforce** mode and rejects it in app namespaces. Do not reach for it.

3. **The pod landed on a node without the device.** Step 2 shows an empty `/dev/dri`. All three prod
   nodes carried `renderD128` with `i915` bound when this was measured (2026-09-30), so this means a
   node changed — a rebuild, a BIOS reset, or a driver regression. `DirectoryOrCreate` is why the app
   tier still came up; the cost is that the loss is silent. **Fix the node, do not tighten the mount:**
   `Directory` would stall `apps-wave-0` (`wait: true`, with `apps-wave-1` depending on it) for a
   feature that is not load-bearing.

4. **The image changed under you.** See "The floating-image caveat". A `:latest` pull can remove the
   mechanism this feature leans on with no commit in this repo.

5. **The session was never transcoding.** Direct Play / Direct Stream never touches the transcoder, so
   `Transcode (hw)` cannot appear. Confirm the session actually says `Transcode` before treating it as
   a hardware failure.

6. **The source needs a codec Quick Sync on this generation cannot handle** — e.g. AV1 decode, which
   Coffee Lake's UHD 630 has no engine for. Plex falls back to software *per stream*. Symptom: some
   sessions show `(hw)` and others do not. Not a fault.

## The floating-image caveat

Everything this page states about the image's behaviour — the `cont-init.d` hook that adds the app
user to the device's group, the absence of `PLEX_PREFERENCE*`, the six `setPref` keys, the
`HardwareAcceleratedCodecs` string — was read from digest **`sha256:e0ab27395614…`**
(`plexmediaserver 1.43.4.10903`).

The deployment runs **`plexinc/pms-docker:latest` with `imagePullPolicy: Always`**. A silent pull can
therefore change the behaviour this feature depends on **with no commit in this repo**, while the
manifest still looks identical. Symptom to watch for: hardware transcoding that worked and quietly
stopped, with no git change to point at.

```
infra/devcontainer/dev.sh run bash -c \
  'kubectl --context k3s-prod -n plex get pod -l app.kubernetes.io/name=plex \
     -o jsonpath="{.items[0].status.containerStatuses[0].imageID}"; echo'
```

If that digest is not the one above, **re-verify the hook before trusting this page**:

```
infra/devcontainer/dev.sh run bash -c \
  'kubectl --context k3s-prod -n plex exec deploy/plex -- ls /etc/cont-init.d/'
```

`45-plex-hw-transcode-and-connected-tuner` still present ⇒ the group mechanism is still the one
described here. Gone or renamed ⇒ stop and re-measure. In particular the "no `supplementalGroups`
needed" conclusion in `catalog/plex.yaml` and `schemas/renderer-contract.md` is only true while that
hook exists.

## When to escalate or stop

- **Do not add `supplementalGroups` or a `podSecurityContext` to `catalog/plex.yaml`.** It would be
  **inert**, and it would mislead the next person. Measured in-container 2026-09-30: the image's
  `cont-init.d` hook stats each `/dev/dri` char device and `usermod -a -G`s the `plex` user into its
  group *inside* the container, and the service then execs via `s6-setuidgid plex`, which **replaces**
  the process group list from `/etc/group` rather than appending — `Plex Media Server` runs with
  `Groups: 100 1000`, root's gid `0` (which PID 1 does hold) gone. Anything Kubernetes supplies is
  discarded before Plex starts. `render.py --selftest` asserts that absence deliberately.
- **Do not reach for `privileged: true`.** Kyverno `restrict-host-namespaces` is Enforce and will
  reject it. If the cgroup gate is the blocker, the answer is the Intel device plugin (cause #2).
- **Do not `kubectl edit`/`patch` the Deployment to add or change the device mount.** Flux reconciles
  `main`; a hand-applied change is drift that the next reconcile reverts or, worse, adopts. Mutating
  `kubectl` and `flux reconcile` are **operator** actions, and the durable change is a git change
  through a branch, a PR and a green gate.
- **Do not edit `Preferences.xml` on the PVC**, never `cat` it, and never `sops -d` anything into a
  terminal. It holds `PlexOnlineToken`.
- **Do not delete the `plex-config` PVC to "start clean".** It is the library database, metadata and
  watch state (`policy: daily-offcluster`). The transcode toggle is one preference, not a reason to
  lose the DB.
- **Stop and get the operator** if the device cannot be opened from the pod (cause #2 confirmed). The
  next step is a platform change — the Intel device plugin — with its own PR, not more poking at this
  workload.

## Known gaps, deliberately not fixed here

Recorded so they are findable, and so nobody assumes they were overlooked:

- **Transcode scratch is on the wrong medium, and it is invisible.** The image sets
  `TranscoderTempDirectory=/transcode`, which is **not a volume** — it is the container's writable
  overlay, i.e. the node's `/data`. That is NVMe on prod-2 but **SATA on prod-1 and prod-3**, so
  transcode scratch performance changes silently whenever the pod is rescheduled. Both candidate fixes
  need a **schema** change and are therefore out of scope here: pin Plex to prod-2 (needs a
  `nodeSelector` field the schema does not have), or give `/transcode` a `medium: Memory` emptyDir and
  transcode in RAM (~31 GiB per node; needs an emptyDir field the schema does not have). The RAM option
  is the better one, because it removes the node asymmetry outright instead of working around it.
- **GuC/HuC firmware is not loaded** (`i915.enable_guc=0`). The cost is low-power H.264 **encode**
  quality only — decode, and the feature on this page, are unaffected. Changing it needs a kernel
  parameter and a **serialized rolling reboot**, which is an operator action (CLAUDE.md §5), and this
  cluster has a documented history of that going badly (a prod node hung on a warm reboot on
  2026-09-03). Not worth coupling to this change.

---
Source of truth: `runbooks/plex-quicksync.md`. Index: `/runbooks/`
The node measurements (all three prod nodes) and the in-container image measurements cited here were
taken **2026-09-30** against `plexinc/pms-docker` digest `sha256:e0ab27395614…`
(`plexmediaserver 1.43.4.10903`). Hardware transcoding had **not** been observed working at the time
of writing, and the Plex-side toggle was still unset.