feat(schema): declare a host hardware capability, hardware.gpu: intel-quicksync
A ServiceDefinition could not say it needs the host's Intel QuickSync render
device. The schema is `additionalProperties: false`, so there was no authorable
form for it at all, and `apps/` is generated — a hand-edit would be reverted by
the next render.
Adds optional top-level `hardware`, with one optional member `gpu`, an enum
whose only current value is `intel-quicksync` (schema 1.7.0 -> 1.8.0).
The enum is the point. The SD declares WHAT CLASS of device it needs; the
hostPath, the mount and the GID live only in catalog/render.py. Migrating to
the Intel GPU device plugin (`gpu.intel.com/i915` resource requests, no
hostPath) is then a renderer change with zero catalog edits. Letting an SD name
`/dev/dri` directly would look simpler and make that migration a catalog-wide
edit, so both the schema description and the renderer contract say so
explicitly.
When `hardware.gpu == intel-quicksync`, r_deployment emits, on the app
container / pod spec only:
- a hostPath volume `dri` -> /dev/dri, type DirectoryOrCreate;
- a read-write volumeMount of it at /dev/dri (VA-API opens the render node
read-write, so deliberately not readOnly);
- INTEL_RENDER_GID MERGED into podSecurityContext.supplementalGroups, never
clobbering what the SD already declares.
`DirectoryOrCreate` is load-bearing. A node that comes back without the render
device (rebuild, BIOS change, driver regression, hardware swap) must degrade
the workload to software transcoding, not fail the mount: apps-wave-0 is
`wait: true` and apps-wave-1 dependsOn it, so a failed hostPath mount stalls
the whole app tier for a feature that is not load-bearing. /dev is tmpfs, so
the empty directory the kubelet creates leaves nothing durable behind. Its cost
— a missing device is silent, and no metric exists to alert on — is recorded in
the contract rather than papered over with an alert that could not fire.
Deliberately NOT emitted: no `privileged: true` (Kyverno restrict-host-
namespaces is Enforce and it is not needed), no nodeSelector (placement is a
separate concern with its own justification), no resource requests (there is no
app-container resource render path).
INTEL_RENDER_GID = 992 is a single named constant carrying its provenance:
measured on all three prod nodes on 2026-09-30, and a *systemd dynamic*
allocation rather than a static GID, so a node rebuild can move it.
No catalog entry uses the field. Re-rendering `--all catalog -o apps` is a zero
diff, which is the proof the change is inert until opted into.
Three self-test cases in the existing check(...) style: the positive, the
negative (an SD without `hardware` gains no volume, no mount, no
securityContext — mirroring the podSecurityContext negative) and the merge case
([44] + 992 -> [44, 992], fsGroup preserved). Note that NO gate stage runs
`render.py --selftest`; they were run by hand and the gap is recorded in the
handoff.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
1 parent
0fbf544d5e
commit
2279d14ec5
5 files changed
+343
-4
No files matched your search
@@ -0,0 +1,147 @@
|
||||
---
|
||||
date: 2026-09-30
|
||||
branch: feat/sd-hardware-capability
|
||||
title: ServiceDefinition gains a `hardware` capability field — `gpu: intel-quicksync` renders the /dev/dri wiring, and the enum is what keeps the mechanism swappable
|
||||
---
|
||||
|
||||
## What changed
|
||||
|
||||
A new optional top-level ServiceDefinition property, `hardware`, with one optional member,
|
||||
`gpu`, whose enum today has exactly one value: `intel-quicksync`. **No catalog entry uses it.**
|
||||
This PR delivers the capability; consuming it (Plex) is separate work on top.
|
||||
|
||||
- `schemas/servicedefinition.schema.json` — adds `hardware` (object, `additionalProperties: false`,
|
||||
`minProperties: 1`) with `gpu` (enum `["intel-quicksync"]`). Schema version bumped **1.7.0 →
|
||||
1.8.0**, changelog entry prepended to the schema `description` per that file's own convention.
|
||||
- `catalog/render.py` — three module-level constants (`DRI_VOLUME_NAME`, `DRI_DEVICE_DIR`,
|
||||
`INTEL_RENDER_GID = 992`) and the emission in `r_deployment`.
|
||||
- `schemas/renderer-contract.md` — new `### hardware` field-mapping section, plus cross-field
|
||||
invariant **8** in that file's numbered list.
|
||||
- `schemas/README.md` — its `**Schema version:**` line said `1.0.0`; it had been stale since 1.1.0.
|
||||
Synced to 1.8.0 and annotated that the schema `description` is the changelog of record. Added one
|
||||
bullet to "Cross-field rules" for the new enum. That was the whole of the staleness; nothing
|
||||
ballooned.
|
||||
|
||||
**What the renderer emits** for `hardware.gpu: intel-quicksync`, on the app container / pod spec
|
||||
only (never a sidecar):
|
||||
|
||||
1. a hostPath volume `dri` → `/dev/dri`, `type: DirectoryOrCreate`;
|
||||
2. a read-write `volumeMount` of `dri` at `/dev/dri` on the app container (deliberately not
|
||||
`readOnly` — VA-API opens the render node read-write);
|
||||
3. `INTEL_RENDER_GID` **merged into** `podSecurityContext.supplementalGroups`, never clobbering
|
||||
what the SD already declares.
|
||||
|
||||
It emits **no** `privileged: true` (Kyverno `restrict-host-namespaces` is Enforce on app namespaces
|
||||
and it is not needed), **no** `nodeSelector` (placement is a separate concern with its own
|
||||
justification) and **no** resource requests (there is no app-container resource render path).
|
||||
|
||||
**The design decision worth defending: the field is an enum of intent, not a device path.** The SD
|
||||
says *what class of host device the workload needs*; the hostPath, the mount and the GID exist only
|
||||
in `catalog/render.py`. Migrating to the Intel GPU device plugin (`gpu.intel.com/i915` resource
|
||||
requests, no hostPath at all) is then a renderer change with zero catalog edits. Both the schema
|
||||
`description` and the contract section say this explicitly, because the obvious "simplification" —
|
||||
letting an SD name `/dev/dri` — would push a platform implementation detail into the operator-facing
|
||||
contract and make that migration a catalog-wide edit.
|
||||
|
||||
**`DirectoryOrCreate` is load-bearing.** A node that comes back without the render device (rebuild,
|
||||
BIOS change, driver regression, hardware swap) must degrade the workload to software transcoding,
|
||||
not fail the mount: `apps-wave-0` is `wait: true` and `apps-wave-1` `dependsOn` it, so a failed
|
||||
hostPath mount stalls the entire app tier for the sake of a feature that is not load-bearing.
|
||||
`/dev` is tmpfs, so the empty directory the kubelet creates on such a node leaves nothing durable
|
||||
behind. Do not "tighten" it to `Directory`. The reasoning is recorded at the emission site.
|
||||
|
||||
## What I verified
|
||||
|
||||
Measured, in this worktree, on this branch:
|
||||
|
||||
- **`python3 catalog/render.py --selftest` → `render.py selftest: PASS`, rc 0.** Three new cases
|
||||
were added in the existing `check(...)` style: the positive (volume + mount + GID, and *no*
|
||||
privileged/nodeSelector), the negative (an SD without `hardware` gains no volume, no volumeMount
|
||||
and no `securityContext`), and the merge case (`supplementalGroups: [44]` + the field →
|
||||
`[44, 992]`, with `fsGroup` preserved).
|
||||
- **The self-tests actually bite.** I mutated the guard (`if hw_gpu == "intel-quicksync":` → `if
|
||||
True:`) in a scratch copy and re-ran: rc 1 with three named failures, including "an SD without
|
||||
`hardware` must gain no dri volume, no mount and no securityContext". A test that cannot fail is
|
||||
not a test; this one can.
|
||||
- **Re-render is a zero diff.**
|
||||
`python3 catalog/render.py --all catalog -o apps --wave-layout --no-image-automation` then
|
||||
`git diff --stat apps/` → **empty**, and `git status --porcelain apps/` → **empty** (so the two
|
||||
not-onboarded by-products, `apps/wave-1/keycloak.yaml` and `apps/wave-0/vm-demo.yaml`, were
|
||||
rewritten byte-identically and needed no revert). The change is inert until a service opts in.
|
||||
- **Schema validity and catalog conformance.** `Draft202012Validator.check_schema` passes; all
|
||||
**12** `catalog/*.yaml` validate with **0** errors. On the new field specifically: `gpu:
|
||||
intel-quicksync` accepted; `gpu: /dev/dri` rejected (enum); `hardware: {gpu: …, path: …}` rejected
|
||||
(`additionalProperties: false`); `hardware: {}` rejected (`minProperties: 1`).
|
||||
- **Hand-checked the emitted YAML** by rendering a throwaway SD carrying `hardware`, a Longhorn
|
||||
volume and its own `podSecurityContext: {runAsUser: 1000, supplementalGroups: [44]}`. Result:
|
||||
`volumeMounts` = config then `{name: dri, mountPath: /dev/dri}`; `securityContext` =
|
||||
`{runAsUser: 1000, supplementalGroups: [44, 992]}`; `volumes` = the PVC then
|
||||
`{name: dri, hostPath: {path: /dev/dri, type: DirectoryOrCreate}}`. That toy SD was never written
|
||||
into `catalog/`.
|
||||
- **GATE: `rc 3` — UNDETERMINED, NOT green.** Run in the pinned toolbox from this worktree with
|
||||
both CLAUDE.md §3 workarounds (`DEV_KUBECONFIG=/dev/null`, and an `ENGINE` shim bind-mounting the
|
||||
main checkout's `.git` at its own absolute path). Diff base `origin/main @ 0fbf544d5e2f` … `HEAD @
|
||||
9f0db1495cfd`; touched requirements `R-DNS-03 R-GITOPS-02 R-IDP-02`. Per stage:
|
||||
|
||||
| stage | result |
|
||||
|---|---|
|
||||
| spec-immutability | **SKIPPED** — sibling spec repo not at `/work/../home-network-k3s`. Expected from a worktree; tolerated by §3. |
|
||||
| decisions-log | pass |
|
||||
| lint | pass |
|
||||
| l0 | pass |
|
||||
| render-golden | pass — *"all 10 onboarded service golden(s) reproduce from the canonical render"* |
|
||||
| l1 | pass — 3 offline checks, `0 failure(s), 0 skip(s), 0 unwired`; coverage 51/51 requirements scaffolded |
|
||||
| approval-policy | pass (`event='none'`, not a pull_request event — N/A locally; it will evaluate for real on the PR) |
|
||||
| gitea-settings | **SKIPPED** — `SETTINGS_API_TOKEN` unset. Expected; branch protection and runner registration are therefore UNVERIFIED by this run. |
|
||||
| conformance (L2/L3) | **SKIPPED** — `CLUSTER_AVAILABLE!=true`. Phase-0 standing skip. |
|
||||
|
||||
The gate's own verdict: *"Stages that did not run: spec-immutability gitea-settings"*. **No stage
|
||||
failed, and no stage skipped for a reason caused by this change.** rc 3 is reported as rc 3; I am
|
||||
not calling this green.
|
||||
- **Blast radius of the schema change:** `hardware` is absent from every catalog entry, so
|
||||
`apply_defaults` never materializes it and `s.get("hardware")` is `None` on all 12 — which is what
|
||||
the zero diff above demonstrates rather than asserts.
|
||||
- **Touched no sensitive path.** Changes are confined to `schemas/`, `catalog/`, `decisions/`; per
|
||||
`infra/gitea/approval-policy.json` none of those are sensitive. `conformance/**` and `clusters/**`
|
||||
are untouched. I did not run `--emit-waves` (it writes `clusters/base/`, and the set of files
|
||||
under `apps/` did not change).
|
||||
|
||||
## What I did NOT verify
|
||||
|
||||
- **Whether a non-privileged pod can actually `open()` this hostPath char device at all. This is
|
||||
the largest technical risk in the workstream and nobody has tested it.** `supplementalGroups` gets
|
||||
the DAC permission bits right, but containerd installs a **device cgroup** allow-list per
|
||||
container, and whether `/dev/dri/renderD128` reaching the container as a bind-mounted device node
|
||||
is also permitted by that cgroup on this k3s/containerd version is **unmeasured**. If it is not,
|
||||
the mount will succeed, the group will be present, and the `open()` will still fail `EPERM` — and
|
||||
the design would need the device plugin (which sets the cgroup entry) rather than a hostPath. The
|
||||
first real consumer proves or disproves this; nothing in this PR does.
|
||||
- **That a missing device degrades silently — and nothing will tell you.** This is the accepted cost
|
||||
of `DirectoryOrCreate`. The workload will software-transcode and **no signal reports it**: media
|
||||
apps in this catalog declare no `metricsEndpoint`, so no ServiceMonitor exists, and there is no
|
||||
metric to alert on. I deliberately did **not** propose an alert, because an alert that cannot fire
|
||||
is worse than none — see the 2026-09-13 correction in `renderer-contract.md` §`observability`,
|
||||
where twelve monitors discovered zero targets. The only proof of hardware transcode is an operator
|
||||
seeing it in the application's own dashboard.
|
||||
- **Whether GID 992 survives a node rebuild.** It was measured on all three prod nodes on
|
||||
2026-09-30 and was identical on each, but `render` is a *systemd dynamic* group allocation, not a
|
||||
Debian/LSB static GID. A reinstall that creates system groups in a different order can move it,
|
||||
and nothing in this repo would notice. Recorded in the constant's comment; re-measure `getent
|
||||
group render` on every node after any rebuild.
|
||||
- **Whether `supplementalGroups` is the load-bearing mechanism or belt-and-braces.** A concurrent
|
||||
agent was checking whether the pms-docker image's `45-plex-hw-transcode-and-connected-tuner` init
|
||||
hook resolves the render group dynamically at startup. **No finding reached me**, so the constant's
|
||||
comment describes the GID and its provenance without claiming which mechanism does the work.
|
||||
Update that comment if the answer arrives.
|
||||
- **Anything on the live cluster.** I ran no `kubectl`. The node facts above (device present,
|
||||
`root:render` 0660, GID 992, `i915` bound, no device plugin, no `gpu.intel.com/*` advertised) and
|
||||
the Kyverno posture (`restrict-host-namespaces` Enforce; no hostPath or `supplementalGroups`
|
||||
restriction; no PSA labels on app namespaces) are **prior recon handed to me, not re-measured by
|
||||
me**. Per CLAUDE.md §2b, treat them as second-hand until re-queried.
|
||||
- **That CI exercises the self-tests. It does not.** `git grep -- --selftest ci/` returns nothing —
|
||||
no gate stage invokes the renderer's self-test. They are the executable spec and house style, and
|
||||
I ran them by hand, but a future regression in this code path **will not be caught by the gate**.
|
||||
Wiring `--selftest` into a gate stage is a real gap and is not part of this PR (`ci/` is a
|
||||
sensitive path and would drag in a human reviewer for a change unrelated to this one).
|
||||
- **Runtime behaviour of any kind.** The gate proves the L0/L1 offline subset only;
|
||||
`vars.CLUSTER_AVAILABLE` is unset, so L2/L3 conformance does not run in CI.
|
||||
Reference in new issue
Block a user