feat(schema): declare a host hardware capability, hardware.gpu: intel-quicksync
A ServiceDefinition could not say it needs the host's Intel QuickSync render
device. The schema is `additionalProperties: false`, so there was no authorable
form for it at all, and `apps/` is generated — a hand-edit would be reverted by
the next render.
Adds optional top-level `hardware`, with one optional member `gpu`, an enum
whose only current value is `intel-quicksync` (schema 1.7.0 -> 1.8.0).
The enum is the point. The SD declares WHAT CLASS of device it needs; the
hostPath, the mount and the GID live only in catalog/render.py. Migrating to
the Intel GPU device plugin (`gpu.intel.com/i915` resource requests, no
hostPath) is then a renderer change with zero catalog edits. Letting an SD name
`/dev/dri` directly would look simpler and make that migration a catalog-wide
edit, so both the schema description and the renderer contract say so
explicitly.
When `hardware.gpu == intel-quicksync`, r_deployment emits, on the app
container / pod spec only:
- a hostPath volume `dri` -> /dev/dri, type DirectoryOrCreate;
- a read-write volumeMount of it at /dev/dri (VA-API opens the render node
read-write, so deliberately not readOnly);
- INTEL_RENDER_GID MERGED into podSecurityContext.supplementalGroups, never
clobbering what the SD already declares.
`DirectoryOrCreate` is load-bearing. A node that comes back without the render
device (rebuild, BIOS change, driver regression, hardware swap) must degrade
the workload to software transcoding, not fail the mount: apps-wave-0 is
`wait: true` and apps-wave-1 dependsOn it, so a failed hostPath mount stalls
the whole app tier for a feature that is not load-bearing. /dev is tmpfs, so
the empty directory the kubelet creates leaves nothing durable behind. Its cost
— a missing device is silent, and no metric exists to alert on — is recorded in
the contract rather than papered over with an alert that could not fire.
Deliberately NOT emitted: no `privileged: true` (Kyverno restrict-host-
namespaces is Enforce and it is not needed), no nodeSelector (placement is a
separate concern with its own justification), no resource requests (there is no
app-container resource render path).
INTEL_RENDER_GID = 992 is a single named constant carrying its provenance:
measured on all three prod nodes on 2026-09-30, and a *systemd dynamic*
allocation rather than a static GID, so a node rebuild can move it.
No catalog entry uses the field. Re-rendering `--all catalog -o apps` is a zero
diff, which is the proof the change is inert until opted into.
Three self-test cases in the existing check(...) style: the positive, the
negative (an SD without `hardware` gains no volume, no mount, no
securityContext — mirroring the podSecurityContext negative) and the merge case
([44] + 992 -> [44, 992], fsGroup preserved). Note that NO gate stage runs
`render.py --selftest`; they were run by hand and the gap is recorded in the
handoff.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
1 parent
0fbf544d5e
commit
2279d14ec5
5 files changed
+343
-4
No files matched your search
+134
-2
@@ -133,6 +133,24 @@ OAUTH2_PROXY_IMAGE = "quay.io/oauth2-proxy/oauth2-proxy:v7.8.1"
|
||||
OAUTH2_PROXY_PORT = 4180
|
||||
KEYCLOAK_ISSUER = f"https://keycloak.{ZONE}/realms/homelab"
|
||||
|
||||
# --- Hardware capability: `hardware.gpu: intel-quicksync` ------------------------------
|
||||
# The ServiceDefinition declares the CAPABILITY (schemas/servicedefinition.schema.json
|
||||
# `hardware.gpu`); everything below is the MECHANISM that currently satisfies it and is
|
||||
# deliberately confined to this file. Swapping to the Intel GPU device plugin
|
||||
# (`gpu.intel.com/i915` resource requests instead of a hostPath) is a change here and
|
||||
# nowhere else -- no catalog entry moves.
|
||||
DRI_VOLUME_NAME = "dri"
|
||||
DRI_DEVICE_DIR = "/dev/dri"
|
||||
# GID of the host `render` group, which owns /dev/dri/renderD128 (mode 0660, root:render).
|
||||
# MEASURED 2026-09-30 on ALL THREE prod nodes (HP EliteDesk 800 G5 Mini, i7-9700T, UHD
|
||||
# Graphics 630, i915 bound -- not xe): GID 992, group name `render`, identical on each.
|
||||
# This is a *systemd dynamic* allocation (systemd-udev creates `render` out of the dynamic
|
||||
# system range at first boot), NOT a Debian/LSB static GID, so it is NOT guaranteed stable:
|
||||
# a node rebuild, or a reinstall that creates system groups in a different order, can move
|
||||
# it. If hardware transcode stops working after a node rebuild, re-measure
|
||||
# `getent group render` on every node before assuming anything else.
|
||||
INTEL_RENDER_GID = 992
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Defaults (renderer-contract.md "Default posture") — materialize BEFORE mapping.
|
||||
@@ -483,6 +501,30 @@ def r_deployment(s: dict) -> dict:
|
||||
"persistentVolumeClaim": {"claimName": f"{s['name']}-{vol['name']}"},
|
||||
}
|
||||
)
|
||||
# hardware capability (schema-optional): `hardware.gpu: intel-quicksync` means this
|
||||
# workload needs the host iGPU's render node for hardware video transcode. The SD says
|
||||
# only that; the wiring below is the platform's current mechanism for satisfying it.
|
||||
#
|
||||
# `type: DirectoryOrCreate` is DELIBERATE and load-bearing -- do NOT "tighten" it to
|
||||
# `Directory`. A node that comes back without the render device (rebuild, BIOS change,
|
||||
# driver regression, hardware swap) must degrade the workload to SOFTWARE transcoding,
|
||||
# not fail the mount: `apps-wave-0` is `wait: true` and `apps-wave-1` dependsOn it, so a
|
||||
# failed hostPath mount stalls the entire app tier for the sake of a feature that is not
|
||||
# load-bearing. `DirectoryOrCreate` turns that failure mode into "transcodes in
|
||||
# software" instead of "the app tier does not reconcile". /dev is tmpfs, so the empty
|
||||
# directory the kubelet creates on such a node leaves nothing durable behind.
|
||||
#
|
||||
# The mount is deliberately NOT readOnly: VA-API opens the render node read-write.
|
||||
hw_gpu = (s.get("hardware") or {}).get("gpu")
|
||||
if hw_gpu == "intel-quicksync":
|
||||
volume_mounts.append({"name": DRI_VOLUME_NAME, "mountPath": DRI_DEVICE_DIR})
|
||||
volumes.append(
|
||||
{
|
||||
"name": DRI_VOLUME_NAME,
|
||||
"hostPath": {"path": DRI_DEVICE_DIR, "type": "DirectoryOrCreate"},
|
||||
}
|
||||
)
|
||||
|
||||
if volume_mounts:
|
||||
container["volumeMounts"] = volume_mounts
|
||||
|
||||
@@ -505,8 +547,19 @@ def r_deployment(s: dict) -> dict:
|
||||
# pod spec. Its primary use is `fsGroup`, which makes a mounted Longhorn PVC group-writable by a
|
||||
# non-root image (e.g. semaphore runs as UID 1001 and cannot otherwise write its config PVC).
|
||||
# Absent => the key is never set, so apps that declare none render byte-identical.
|
||||
if s.get("podSecurityContext"):
|
||||
pod_spec["securityContext"] = s["podSecurityContext"]
|
||||
#
|
||||
# `hardware.gpu: intel-quicksync` additionally needs the host `render` GID in
|
||||
# supplementalGroups, because /dev/dri/renderD128 is root:render 0660 and the container
|
||||
# does not run as root. It is MERGED into whatever the SD already declares rather than
|
||||
# replacing it -- an SD that sets its own supplementalGroups keeps every one of them.
|
||||
psc = dict(s.get("podSecurityContext") or {})
|
||||
if hw_gpu == "intel-quicksync":
|
||||
groups = list(psc.get("supplementalGroups") or [])
|
||||
if INTEL_RENDER_GID not in groups:
|
||||
groups.append(INTEL_RENDER_GID)
|
||||
psc["supplementalGroups"] = groups
|
||||
if psc:
|
||||
pod_spec["securityContext"] = psc
|
||||
|
||||
if volumes:
|
||||
pod_spec["volumes"] = volumes
|
||||
@@ -2440,6 +2493,85 @@ def _selftest() -> int:
|
||||
"an SD without podSecurityContext must not gain a pod securityContext",
|
||||
)
|
||||
|
||||
# 6b) hardware.gpu: intel-quicksync -> /dev/dri hostPath volume + RW mount + the host
|
||||
# `render` GID. The SD declares the CAPABILITY; these assertions pin the MECHANISM
|
||||
# that currently satisfies it, so a change to it is a deliberate, visible edit.
|
||||
hw_docs = render(
|
||||
{
|
||||
"name": "hw",
|
||||
"source": {"image": "a:1"},
|
||||
"hardware": {"gpu": "intel-quicksync"},
|
||||
}
|
||||
)
|
||||
hw_pod = next(
|
||||
d[1] for d in hw_docs if d[1]["kind"] == "Deployment"
|
||||
)["spec"]["template"]["spec"]
|
||||
check(
|
||||
{
|
||||
"name": "dri",
|
||||
"hostPath": {"path": "/dev/dri", "type": "DirectoryOrCreate"},
|
||||
}
|
||||
in hw_pod.get("volumes", []),
|
||||
"hardware.gpu=intel-quicksync must render a /dev/dri hostPath volume "
|
||||
"with type DirectoryOrCreate (a GPU-less node must degrade, not fail the mount)",
|
||||
)
|
||||
hw_mounts = hw_pod["containers"][0].get("volumeMounts", [])
|
||||
check(
|
||||
{"name": "dri", "mountPath": "/dev/dri"} in hw_mounts,
|
||||
"hardware.gpu=intel-quicksync must mount /dev/dri READ-WRITE on the app container",
|
||||
)
|
||||
check(
|
||||
hw_pod.get("securityContext", {}).get("supplementalGroups")
|
||||
== [INTEL_RENDER_GID],
|
||||
"hardware.gpu=intel-quicksync must add the host render GID to supplementalGroups",
|
||||
)
|
||||
# Nothing privileged, and no node pinning: Kyverno restrict-host-namespaces (Enforce)
|
||||
# rejects `privileged: true` in an app namespace, and placement is a separate concern
|
||||
# with its own justification -- this field is about the DEVICE only.
|
||||
check(
|
||||
"privileged" not in hw_pod.get("securityContext", {})
|
||||
and "securityContext" not in hw_pod["containers"][0]
|
||||
and "nodeSelector" not in hw_pod,
|
||||
"hardware.gpu must emit no privileged securityContext and no nodeSelector",
|
||||
)
|
||||
# The regression that matters: absence renders NOTHING. Mirrors the podSecurityContext
|
||||
# negative above -- an SD that does not opt in must be byte-identical to before.
|
||||
nohw_pod = next(
|
||||
d[1] for d in render({"name": "nohw", "source": {"image": "a:1"}})
|
||||
if d[1]["kind"] == "Deployment"
|
||||
)["spec"]["template"]["spec"]
|
||||
check(
|
||||
"volumes" not in nohw_pod
|
||||
and "volumeMounts" not in nohw_pod["containers"][0]
|
||||
and "securityContext" not in nohw_pod,
|
||||
"an SD without `hardware` must gain no dri volume, no mount and no securityContext",
|
||||
)
|
||||
# Merge, not clobber: an SD that declares its own supplementalGroups keeps them AND
|
||||
# gains the render GID, and its other podSecurityContext keys survive untouched.
|
||||
hwm_pod = next(
|
||||
d[1]
|
||||
for d in render(
|
||||
{
|
||||
"name": "hwm",
|
||||
"source": {"image": "a:1"},
|
||||
"hardware": {"gpu": "intel-quicksync"},
|
||||
"podSecurityContext": {
|
||||
"fsGroup": 1001,
|
||||
"supplementalGroups": [44],
|
||||
},
|
||||
}
|
||||
)
|
||||
if d[1]["kind"] == "Deployment"
|
||||
)["spec"]["template"]["spec"]
|
||||
check(
|
||||
hwm_pod["securityContext"] == {
|
||||
"fsGroup": 1001,
|
||||
"supplementalGroups": [44, INTEL_RENDER_GID],
|
||||
},
|
||||
"hardware.gpu must MERGE the render GID into the SD's own supplementalGroups "
|
||||
"(both GIDs, fsGroup preserved), never clobber them",
|
||||
)
|
||||
|
||||
# 6c) forward-auth (Option B): an exposed forward-auth app gets an oauth2-proxy sidecar that
|
||||
# fronts the Gateway path (fail-closed), while oidc/native/none apps are unchanged.
|
||||
fa_docs = render(
|
||||
|
||||
Reference in new issue
Block a user