100 lines
7.2 KiB
YAML
100 lines
7.2 KiB
YAML
---
|
|
# Intel GPU device plugin for Kubernetes — makes the nodes advertise `gpu.intel.com/i915`.
|
|
#
|
|
# WHAT THIS FIXES, and why nothing cheaper works. Plex already has a `/dev/dri` hostPath mount
|
|
# (merged on main @ cdf3e14). The device node is VISIBLE in the container and cannot be OPENED.
|
|
# Measured in the running pod on k3s-prod-2:
|
|
# uid 0, O_RDWR -> EPERM "Operation not permitted"
|
|
# uid 1000, O_RDWR -> EPERM "Operation not permitted"
|
|
# EPERM from the DEVICE CGROUP, not EACCES from file permissions — proven three ways: root on the
|
|
# HOST opens the same device rc=0; the node is unified cgroup2fs; and the container's runc
|
|
# config.json carries one deny-all rule and nothing else:
|
|
# linux.resources.devices: [{'allow': False, 'access': 'rwm'}]
|
|
# linux.devices: None
|
|
# No allow entry for major 226, and the runtime was never asked to grant a device. A hostPath is a
|
|
# FILESYSTEM operation: it makes the device node appear, and adds no cgroup allow rule, because
|
|
# Kubernetes cannot infer one from a volume.
|
|
#
|
|
# A device plugin is the mechanism that closes exactly that gap. Verified by reading the v0.37.0
|
|
# source (cmd/gpu_plugin/gpu_plugin.go, createDeviceSpecsFromDrmFiles): the Allocate response
|
|
# carries `pluginapi.DeviceSpec{HostPath, ContainerPath, Permissions: "rw"}` per DRM node, and it
|
|
# is that DeviceSpec which makes kubelet populate `linux.devices` AND the matching cgroup allow
|
|
# rule. It is the only NON-PRIVILEGED way to get there. `privileged: true` would also work and is
|
|
# not available: platform/kyverno/policies/restrict-host-namespaces.yaml blocks it under Enforce in
|
|
# app namespaces, and `plex` is labelled app-namespace=true (measured 2026-10-01). Correctly so.
|
|
#
|
|
# STANDALONE DAEMONSET, NOT THE OPERATOR. Upstream ships both. The operator adds a CRD
|
|
# (GpuDevicePlugin), a controller-manager Deployment, cert-manager-backed admission webhooks and a
|
|
# second namespace — i.e. three more things that can be unhealthy — to buy declarative management
|
|
# of a DaemonSet we are declaring in git anyway. For three identical nodes and one consumer that is
|
|
# strictly more blast radius for no gain. The DaemonSet has no CRDs, no controller, no webhook, and
|
|
# no API-server access at all: a device plugin talks to the LOCAL kubelet over a unix socket, which
|
|
# is why there is no ServiceAccount or RBAC here (and therefore no `**/rbac.yaml` sensitive path).
|
|
#
|
|
# LISTING, NOT OPENING — the thing worth being sure about, since opening is precisely what fails
|
|
# today. Verified in v0.37.0 source: the scan path uses only os.ReadDir / os.ReadFile / os.Stat /
|
|
# os.Readlink over /sys/class/drm and /dev/dri. There is no os.Open, no OpenFile, no ioctl and no
|
|
# syscall of any kind against a DRM node anywhere in cmd/gpu_plugin/gpu_plugin.go. Listing and
|
|
# stat() demonstrably work today (that is how the Plex probe found the device before failing to
|
|
# open it), so the plugin needs no cgroup grant of its own and no privilege.
|
|
# * `--health-management` is left OFF (upstream default). It is the one flag that WOULD need to
|
|
# open a device: it starts a Level-Zero sidecar (setupLevelZeroService returns early unless the
|
|
# flag is set). Turning it on would reintroduce the exact EPERM this component exists to fix.
|
|
# * /dev/dri and /sys/class/drm are mounted readOnly, consistent with list-and-stat only.
|
|
# * /dev/dri/by-path absence is non-fatal (newDevicePlugin warns and sets bypathFound=false).
|
|
#
|
|
# RESOURCE NAME: `gpu.intel.com/i915`. Not a guess — `scan()` does
|
|
# `devTree.AddDevice(devProps.driver(), ...)`, so the suffix is the KERNEL DRIVER name. These nodes
|
|
# have i915 bound (not xe) on UHD Graphics 630 / PCI 8086:3e98, so the resource is i915. A future
|
|
# kernel moving CoffeeLake to xe would silently rename it to `gpu.intel.com/xe` and the consumer's
|
|
# request would stop matching — worth knowing before that kernel arrives.
|
|
#
|
|
# `--shared-dev-num` LEFT AT THE UPSTREAM DEFAULT OF 1 (so: not set). The flag multiplies each
|
|
# card into N schedulable device IDs (`for i := 0; i < sharedDevNum; i++`), letting N containers
|
|
# share one GPU. Deliberately not raised:
|
|
# * Plex is the only consumer and wants one device.
|
|
# * The count is a LIE the scheduler believes. There is no GPU-side partitioning or quota behind
|
|
# it — N containers get the same renderD128 and contend for the same two media engines
|
|
# (vcs0/vecs0). Raising it converts "Unschedulable", which is loud and obvious, into
|
|
# "transcodes are mysteriously slow", which is neither.
|
|
# * It is a one-line, zero-migration change on the day a second consumer actually exists, and at
|
|
# that point the right number will be known rather than guessed.
|
|
# No NFD overlay either: all three nodes are identical (established cluster fact), so a nodeSelector
|
|
# on a GPU feature label would gate on something that is true everywhere. If a GPU-less node ever
|
|
# joins, add upstream's nfd overlay THEN rather than relying on the plugin's behaviour there.
|
|
#
|
|
# NO supplementalGroups, anywhere, on purpose. The pms-docker image's own cont-init hook stats each
|
|
# /dev/dri char device and adds the `plex` user to its group — measured live as `Groups: 44 100 992
|
|
# 1000` on the running Plex process, where GID 992 is the host `render` group. That mechanism is
|
|
# image-side and works identically on a plugin-injected device, so adding a supplementalGroup here
|
|
# or on the consumer would duplicate a thing that already works.
|
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
kind: Kustomization
|
|
namespace: intel-gpu-plugin
|
|
resources:
|
|
- namespace.yaml
|
|
- intel-gpu-plugin.yaml
|
|
images:
|
|
# PINNED, never `latest`. Upstream release v0.37.0 (2026-09-16); verified present on Docker Hub
|
|
# 2026-10-01, manifest digest sha256:7001200071db10587fc31184eedda4f5cf291efba737adf85a8c80539f7fa5e1
|
|
# (single amd64 manifest, not an index). Recorded here rather than pinned by digest because this
|
|
# repo pins upstream platform components by VERSION TAG (platform/kubevirt, external-snapshotter);
|
|
# digest pinning is reserved for the conformance runner, where it is a security boundary.
|
|
# `newName` fully qualifies the registry: upstream writes the bare `intel/intel-gpu-plugin`, which
|
|
# relies on containerd's docker.io default rather than saying so.
|
|
#
|
|
# RENOVATE: the inline-marker idiom (customManagers in renovate.json), not the docker datasource —
|
|
# renovate.json sets `"docker": {"enabled": false}` and `"kubernetes": {"enabled": false}`, so an
|
|
# image reference in a plain manifest is tracked by NOTHING. The marker must sit on the line
|
|
# IMMEDIATELY above the version literal for the regex to match, which is why it is down here on
|
|
# `newTag` and not up in this header.
|
|
# CAVEAT, stated because it will bite someone: the GitHub release tags carry a `v`
|
|
# (`v0.37.0`) and the IMAGE tags do not (`0.37.0` — confirmed against the Docker Hub tag list).
|
|
# Whether Renovate normalises that for this dep is UNVERIFIED here (no Renovate run available).
|
|
# If it ever proposes `newTag: v0.38.0`, the fix is an `extractVersion` packageRule in
|
|
# renovate.json, not a hand-edit of this line.
|
|
- name: intel/intel-gpu-plugin
|
|
newName: docker.io/intel/intel-gpu-plugin
|
|
# renovate: datasource=github-releases depName=intel/intel-device-plugins-for-kubernetes
|
|
newTag: "0.37.1"
|