jakman85andClaude Opus 5
df1fd1b3d1
The runbooks/README.md conflict was two index entries appended at the same point, not a semantic one. Both kept, in the file's grouped order. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
175 lines
12 KiB
Markdown
175 lines
12 KiB
Markdown
---
|
|
date: 2026-09-30
|
|
branch: feat/plex-quicksync
|
|
title: Plex opts into `hardware.gpu: intel-quicksync` — the device mount is GitOps, the Plex-side toggle is a documented manual step, and hardware transcoding has NOT been observed working
|
|
---
|
|
|
|
## What changed
|
|
|
|
Three things, and the third is the point of the branch.
|
|
|
|
1. **`catalog/plex.yaml`** — added the four lines the whole feature reduces to:
|
|
|
|
```yaml
|
|
hardware:
|
|
gpu: intel-quicksync
|
|
```
|
|
|
|
placed between `env` and `exposure` to match the schema's own property order, with ~48 lines of
|
|
comment in the file's established style. The comment records: the field is **intent, not
|
|
mechanism** (a later move to the Intel device plugin is a `render.py` change and this block does
|
|
not move); that `/dev/dri/renderD128` is present on **all three** prod nodes with `i915` bound;
|
|
**why a `supplementalGroups` entry would be inert** (the image's cont-init hook does the group
|
|
work inside the container and `s6-setuidgid` replaces the process group list from `/etc/group`);
|
|
why `DirectoryOrCreate` is the load-bearing safe failure mode (`apps-wave-0` is `wait: true` with
|
|
`apps-wave-1` depending on it, so a failed mount would stall the whole app tier for a feature
|
|
that is not load-bearing); and that the Plex-side toggle is a **separate manual operator step**,
|
|
pointing at the new runbook.
|
|
|
|
2. **`apps/wave-0/plex.yaml`** — re-rendered. Six added lines, nothing else, in the whole `apps/`
|
|
tree:
|
|
|
|
```
|
|
- name: dri
|
|
mountPath: /dev/dri
|
|
- name: dri
|
|
hostPath:
|
|
path: /dev/dri
|
|
type: DirectoryOrCreate
|
|
```
|
|
|
|
3. **`runbooks/plex-quicksync.md`** (new), registered in `runbooks/kustomization.yaml`'s
|
|
`configMapGenerator` as `- plex-quicksync=plex-quicksync.md` and indexed in `runbooks/README.md`
|
|
under "Operator procedures". **The file alone would have been a 404** — the pod serves a
|
|
ConfigMap, not the git tree, and `conformance/tests/l0-static/alert-runbooks-resolvable` exists
|
|
precisely because "the file is in git" and "the URL resolves" are different claims. The page
|
|
covers: the GitOps/manual boundary as a table; the toggle (Settings → Transcoder → "Use hardware
|
|
acceleration when available", preference key `HardwareAcceleratedCodecs`); the real neighbouring
|
|
keys and the widely-cited one that **does not exist**; that the only proof is `Transcode (hw)` on
|
|
the Plex dashboard **and why no alert is possible**; the device-cgroup failure to expect first
|
|
with a concrete open() diagnostic and the Intel-device-plugin fallback; and the floating-`:latest`
|
|
caveat with a digest check.
|
|
|
|
**Stacked on PR #27 (`feat/sd-hardware-capability`), which is open and unmerged.** This branch was
|
|
cut from it rather than from `main`, so its one commit (`d9baa2a`, the schema + renderer + contract)
|
|
is in this PR's range. That is deliberate: merging #27 is not an agent's action (CLAUDE.md §1), and
|
|
duplicating its changes here would have been worse than showing them. **This must merge after #27.**
|
|
|
|
Not done, deliberately: no `--emit-waves` (it writes into `clusters/base/`, a sensitive path that
|
|
would force a human reviewer, and it is a no-op unless the *set* of files under `apps/` changes — it
|
|
does not here). No `podSecurityContext`. No `nodeSelector`. No alert rule. No `kubectl apply`.
|
|
|
|
### Two follow-ups left unclaimed, with the reasoning
|
|
|
|
- **Transcode scratch is on the wrong medium, and it is invisible.** The image sets
|
|
`TranscoderTempDirectory=/transcode`, which is **not a volume** — it is the container's writable
|
|
overlay, i.e. the node's `/data`. That is NVMe on prod-2 but **SATA on prod-1 and prod-3**, so
|
|
transcode scratch performance changes silently on reschedule, with nothing reporting it. Both
|
|
candidate fixes need a **schema** change and so neither belongs in this PR: pin Plex to prod-2
|
|
(needs a `nodeSelector` field the schema does not have), or give `/transcode` a `medium: Memory`
|
|
emptyDir and transcode in RAM (~31 GiB per node; needs an emptyDir field the schema does not
|
|
have). **The RAM option is the better one** — it removes the node asymmetry outright instead of
|
|
working around it — which is exactly why it deserves its own PR and its own review rather than
|
|
being smuggled in behind a four-line catalog change. Documented in the runbook's "Known gaps".
|
|
- **GuC/HuC firmware is not loaded** (`i915.enable_guc=0`). The cost is low-power H.264 **encode**
|
|
quality only; decode and this feature are unaffected. Changing it needs a kernel parameter plus a
|
|
**serialized rolling reboot** — an operator action under CLAUDE.md §5, on a cluster with a
|
|
documented history of that going badly (a prod node hung on a warm reboot on 2026-09-03). Kept
|
|
well clear of this PR.
|
|
|
|
### Rebase note (2026-10-01, Ruth Okonkwo) — resolution only, no redesign
|
|
|
|
Rebased onto `main` @ `08deb20` (was cut from `0fbf544`), restacked on the rebased
|
|
`feat/sd-hardware-capability` (PR #27) so the stack stays coherent — this branch still carries
|
|
exactly PR #27's commit plus its own, with no duplicate of anything already in `main`. The branch
|
|
ref was force-updated with `--force-with-lease`, so PR #32 is the same PR with a new head.
|
|
|
|
**The `catalog/render.py` conflict disappeared** once PR #27 was rebased; it was never this
|
|
branch's own conflict. **One conflict remained, in `runbooks/README.md`, purely textual.** PR #30
|
|
(`fix/jackett-oauth2-proxy-session`, merged as `08deb20`) appended an `oauth2-proxy-session` index
|
|
entry under *Operator procedures* at the same point this branch appended its `plex-quicksync` entry.
|
|
**Both entries were kept, in full.** Ordering follows this file's existing convention, which is
|
|
grouped/narrative rather than alphabetical (the *Nodes* section runs down/io/memory/disk/cpu):
|
|
`plex-quicksync` sits next to `plex-claim` as the other Plex page, leaving `oauth2-proxy-session`
|
|
adjacent to `crafty-admin-credential`, which cross-references the same sidecar gate. The change to
|
|
`runbooks/README.md` is an 8-line pure insertion, so PR #30's entry is untouched.
|
|
`runbooks/kustomization.yaml` auto-merged and **both** `configMapGenerator` entries are present.
|
|
|
|
Re-verified after the resolution, on the rebased stack: `--selftest` PASS (rc 0); a full re-render
|
|
reproduces every committed golden byte-identically; and the **only** `apps/` diff against the new
|
|
`main` is `apps/wave-0/plex.yaml`, confined to the expected six lines (the `dri` mount and its
|
|
`DirectoryOrCreate` hostPath volume). No other service's golden moved — PR #28 added ~176 lines of
|
|
oauth2-proxy sidecar rendering, and this confirms by measurement rather than assumption that Plex
|
|
(`auth: native`, no sidecar) is unaffected by it.
|
|
|
|
## What I verified
|
|
|
|
- **The render is exactly the intended change, and nothing else moved.** After
|
|
`python3 catalog/render.py --all catalog -o apps --wave-layout --no-image-automation` (exit 0,
|
|
silent): `git diff --stat apps/` = `apps/wave-0/plex.yaml | 6 ++++++`, and
|
|
`git status --porcelain apps/` = a single ` M apps/wave-0/plex.yaml`. Read line by line, the diff
|
|
is the `dri` volumeMount and the `dri` hostPath volume, and nothing more. **Specifically checked:**
|
|
the same invocation also rewrites `apps/wave-1/keycloak.yaml` and `apps/wave-0/vm-demo.yaml`, which
|
|
are git-tracked but excluded from the golden comparison by `RENDER_GOLDEN_NOT_ONBOARDED` in
|
|
`ci/render-golden.sh` — a spurious diff there would have landed **ungated**. Neither file shows a
|
|
diff; whole-repo `git status --porcelain` lists only `apps/wave-0/plex.yaml` and `catalog/plex.yaml`
|
|
plus my new files.
|
|
- **The rendered output matches what the contract promises** — `schemas/renderer-contract.md`
|
|
`### hardware` says exactly two things (a `/dev/dri` hostPath with `type: DirectoryOrCreate` and a
|
|
read-write mount on the app container) and exactly two things appeared. No `securityContext`, no
|
|
`nodeSelector`, no resources in the diff.
|
|
- **The runbook is projected into the server, not merely committed.** `runbooks/kustomization.yaml`
|
|
carries `- plex-quicksync=plex-quicksync.md`, whose shape matches the generator-entry regex the
|
|
L0 check parses; the extensionless key is what makes `/runbooks/plex-quicksync` a plain static
|
|
lookup. Also indexed in `runbooks/README.md`.
|
|
- **The runbook clears the L0 quality floor** that `alert-runbooks-resolvable` applies to
|
|
alert-linked pages, even though nothing links to this one (it is advisory for an unreferenced
|
|
page): all four required sections (`## What this means here`, `## First commands`,
|
|
`## Likely causes, most likely first`, `## When to escalate or stop`), well over the 1500-byte
|
|
floor, `dev.sh` used throughout, and `--context k3s-prod` on every command.
|
|
- **The gate.** See the run recorded in the PR description for the per-stage table and the real rc.
|
|
- **No secret was printed.** Every `Preferences.xml` read in the runbook greps one named key, and the
|
|
page states explicitly why: the file holds `PlexOnlineToken`. Nothing was `sops -d`'d.
|
|
|
|
## What I did NOT verify
|
|
|
|
**Hardware transcoding has NOT been observed working.** Nothing in this branch is evidence that it
|
|
does. A green gate here proves the manifest renders and the golden matches — it proves nothing about
|
|
runtime behaviour, and `vars.CLUSTER_AVAILABLE` is unset so L2/L3 conformance did not run at all.
|
|
|
|
- **The device-cgroup question is OPEN, and it is the largest remaining technical risk.** Whether
|
|
containerd on this k3s build permits a **non-privileged** pod to `open()` a bind-mounted character
|
|
device is **UNVERIFIED**. It can only be settled by deploying and trying. The container inits as
|
|
uid 0, which settles *file* permissions, but uid 0 does **not** bypass the cgroup device gate — and
|
|
the two failures look identical from the outside. If it is refused, the fallback is the Intel
|
|
Device Plugin (`gpu.intel.com/i915`), which is a `catalog/render.py` change plus a re-render with
|
|
**no ServiceDefinition edit**; `privileged: true` is not available (Kyverno
|
|
`restrict-host-namespaces` is Enforce). The runbook has the diagnostic sequence.
|
|
- **The Plex-side toggle was still unset at the time of writing.** `HardwareAcceleratedCodecs` was
|
|
**absent** from `Preferences.xml` — the toggle has never been enabled on this server. Until an
|
|
operator ticks it in the UI, this branch changes the pod spec and **nothing about transcoding**.
|
|
That is the whole reason the runbook exists.
|
|
- **No metric, so no alert, so no automated detection — permanently.** `catalog/plex.yaml` declares
|
|
no `metricsEndpoint`, so Plex renders no ServiceMonitor; there is no transcode-mode signal for a
|
|
rule to evaluate. A silent degradation to software transcoding (a node losing its iGPU, the
|
|
`DirectoryOrCreate` path) will **not** be reported by anything. I deliberately did not add an
|
|
alert: with no metric behind it, it could never fire, and this repo has been burned at scale by
|
|
exactly that. Closing the gap honestly means an exporter first.
|
|
- **I did not re-derive this session's node or in-container measurements.** The three-node
|
|
`renderD128`/`i915`/GID-992 facts, the cont-init hook behaviour, the `s6-setuidgid` group
|
|
replacement, the `PLEX_PREFERENCE*` absence, the six `setPref` keys and the
|
|
`HardwareAcceleratedCodecs` string were established earlier in this session and are cited, not
|
|
re-measured, by me. Per CLAUDE.md §2b, treat this file as a record of belief and re-measure against
|
|
the cluster before relying on any of it.
|
|
- **The image is a floating tag.** Everything known about the hook came from digest
|
|
`sha256:e0ab27395614…` (`plexmediaserver 1.43.4.10903`), while the Deployment runs
|
|
`plexinc/pms-docker:latest` with `imagePullPolicy: Always`. A silent pull can invalidate the
|
|
"no `supplementalGroups` needed" conclusion **with no commit in this repo**. Not guarded against —
|
|
the runbook tells the reader to re-check the hook after an image bump, which is documentation, not
|
|
enforcement.
|
|
- **I did not verify the live Gitea gate state** (branch protection, required contexts, runner
|
|
online) for this PR. Per §2b that must be queried, not read out of `CLAUDE.md`'s table; I did not
|
|
query it.
|
|
- **`/runbooks/plex-quicksync` was not fetched.** The L0 check asserts every static precondition for
|
|
an HTTP 200; it does not make the request, and I did not either.
|