Compare commits

..
Author SHA1 Message Date
jakman85andClaude Opus 5 f716e9d167 chore(decisions): record the rebase onto main @ 08deb20
gate / gate (pull_request) Successful in 1m41s
gate / conformance (pull_request) Has been skipped
The render.py conflict was textual adjacency with PR #28's constants
block, not a semantic one. Both sides kept.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-10-01 21:01:16 -05:00
jakman85andClaude Opus 5 fa4765bb3e feat(schema): declare a host hardware capability, hardware.gpu: intel-quicksync
A ServiceDefinition could not say it needs the host's Intel QuickSync render
device. The schema is `additionalProperties: false`, so there was no authorable
form for it at all, and `apps/` is generated — a hand-edit would be reverted by
the next render.

Adds optional top-level `hardware`, with one optional member `gpu`, an enum
whose only current value is `intel-quicksync` (schema 1.7.0 -> 1.8.0).

The enum is the point. The SD declares WHAT CLASS of device it needs; the
hostPath and the mount live only in catalog/render.py. Migrating to the Intel
GPU device plugin (`gpu.intel.com/i915` resource requests, no hostPath) is then
a renderer change with zero catalog edits. Letting an SD name `/dev/dri`
directly would look simpler and make that migration a catalog-wide edit, so
both the schema description and the renderer contract say so explicitly.

When `hardware.gpu == intel-quicksync`, r_deployment emits exactly two things,
on the app container only:

  - a hostPath volume `dri` -> /dev/dri, type DirectoryOrCreate;
  - a read-write volumeMount of it at /dev/dri (VA-API opens the render node
    read-write, so deliberately not readOnly).

Mounting the device is sufficient. In particular NO
podSecurityContext.supplementalGroups is emitted, and adding one would be a
regression rather than a hardening. Measured in the running container on
prod-2, 2026-09-30:

  PID 1   s6-svscan            Uid: 0     Gid: 0     Groups: 0
  pid 246 Plex Media Server    Uid: 1000  Gid: 1000  Groups: 100 1000

The service is started with `exec s6-setuidgid plex ...`, and the child's group
list is exactly the /etc/group account-database list — root's gid 0, which
PID 1 holds, is gone. That is a setgroups() that REPLACES rather than appends,
so anything Kubernetes supplies is discarded before the app starts. What works
is the image's own /etc/cont-init.d/45-plex-hw-transcode-and-connected-tuner,
which stats each /dev/dri char device, groupadds a matching group inside the
container and usermods the app user into it, every start, nothing hardcoded.
An inert supplementalGroups would have read as the mechanism to whoever
debugged this next, so the absence is documented at the emission site and
asserted by --selftest. Corollary: the hook re-stats on every start, so host
GID drift is self-healing and no GID is pinned anywhere in this repo.

`DirectoryOrCreate` is load-bearing. A node that comes back without the render
device (rebuild, BIOS change, driver regression, hardware swap) must degrade
the workload to software transcoding, not fail the mount: apps-wave-0 is
`wait: true` and apps-wave-1 dependsOn it, so a failed hostPath mount stalls
the whole app tier for a feature that is not load-bearing. /dev is tmpfs, so
the empty directory the kubelet creates leaves nothing durable behind. Its cost
— a missing device is silent, and no metric exists to alert on — is recorded in
the contract rather than papered over with an alert that could not fire.

Also deliberately NOT emitted: no `privileged: true` (Kyverno restrict-host-
namespaces is Enforce and it is not needed), no nodeSelector (placement is a
separate concern with its own justification), no resource requests (there is no
app-container resource render path).

No catalog entry uses the field. Re-rendering `--all catalog -o apps` is a zero
diff, which is the proof the change is inert until opted into.

Self-tests in the existing check(...) style: the positive (volume + mount, and
no securityContext / privileged / nodeSelector), the negative (an SD without
`hardware` gains no volume, no mount, no securityContext — mirroring the
podSecurityContext negative) and a pass-through case (an SD's own
podSecurityContext survives byte-for-byte). Note that NO gate stage runs
`render.py --selftest`; they were run by hand and the gap is recorded in the
handoff.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-10-01 20:54:35 -05:00
7 changed files with 412 additions and 261 deletions

No files matched your search

+126
View File
@@ -163,6 +163,15 @@ OAUTH2_COOKIE_REFRESH = "2m30s"
# are added. Do not read this flag as "metrics are working".
OAUTH2_PROXY_METRICS_PORT = 44180
# --- Hardware capability: `hardware.gpu: intel-quicksync` ------------------------------
# The ServiceDefinition declares the CAPABILITY (schemas/servicedefinition.schema.json
# `hardware.gpu`); everything below is the MECHANISM that currently satisfies it and is
# deliberately confined to this file. Swapping to the Intel GPU device plugin
# (`gpu.intel.com/i915` resource requests instead of a hostPath) is a change here and
# nowhere else -- no catalog entry moves.
DRI_VOLUME_NAME = "dri"
DRI_DEVICE_DIR = "/dev/dri"
# =============================================================================
# Defaults (renderer-contract.md "Default posture") — materialize BEFORE mapping.
@@ -513,6 +522,46 @@ def r_deployment(s: dict) -> dict:
"persistentVolumeClaim": {"claimName": f"{s['name']}-{vol['name']}"},
}
)
# hardware capability (schema-optional): `hardware.gpu: intel-quicksync` means this
# workload needs the host iGPU's render node for hardware video transcode. The SD says
# only that; the wiring below is the platform's current mechanism for satisfying it.
#
# `type: DirectoryOrCreate` is DELIBERATE and load-bearing -- do NOT "tighten" it to
# `Directory`. A node that comes back without the render device (rebuild, BIOS change,
# driver regression, hardware swap) must degrade the workload to SOFTWARE transcoding,
# not fail the mount: `apps-wave-0` is `wait: true` and `apps-wave-1` dependsOn it, so a
# failed hostPath mount stalls the entire app tier for the sake of a feature that is not
# load-bearing. `DirectoryOrCreate` turns that failure mode into "transcodes in
# software" instead of "the app tier does not reconcile". /dev is tmpfs, so the empty
# directory the kubelet creates on such a node leaves nothing durable behind.
#
# The mount is deliberately NOT readOnly: VA-API opens the render node read-write.
#
# `podSecurityContext.supplementalGroups` is DELIBERATELY NOT SET here, and adding it
# would be a regression, not a hardening. MEASURED in the running container on prod-2,
# 2026-09-30: the image's own cont-init hook
# `/etc/cont-init.d/45-plex-hw-transcode-and-connected-tuner` `find`s every char device
# under /dev/dri, `stat -c '%g'` each one's GID, `groupadd`s a matching group INSIDE the
# container when none exists, and `usermod -a -G` the app user into it -- every start,
# nothing hardcoded. cont-init.d completes before services.d, so the account database
# s6 reads at exec time already carries the group. The service is then started with
# `exec s6-setuidgid plex ...`, and `s6-setuidgid` REPLACES the process group list from
# /etc/group rather than appending to it: `Plex Media Server` runs with
# `Uid: 1000 Gid: 1000 Groups: 100 1000` -- root's gid 0, which PID 1 (s6-svscan) does
# have, is GONE. Anything Kubernetes puts in supplementalGroups reaches s6-svscan and is
# then discarded before the app ever starts. MOUNTING THE DEVICE IS SUFFICIENT; the
# group handling is the image's job and it does it correctly. A supplementalGroups entry
# here would be inert AND would mislead whoever debugs this next.
hw_gpu = (s.get("hardware") or {}).get("gpu")
if hw_gpu == "intel-quicksync":
volume_mounts.append({"name": DRI_VOLUME_NAME, "mountPath": DRI_DEVICE_DIR})
volumes.append(
{
"name": DRI_VOLUME_NAME,
"hostPath": {"path": DRI_DEVICE_DIR, "type": "DirectoryOrCreate"},
}
)
if volume_mounts:
container["volumeMounts"] = volume_mounts
@@ -2530,6 +2579,83 @@ def _selftest() -> int:
"an SD without podSecurityContext must not gain a pod securityContext",
)
# 6b) hardware.gpu: intel-quicksync -> a /dev/dri hostPath volume and a read-write mount
# on the app container, and NOTHING ELSE. The SD declares the CAPABILITY; these
# assertions pin the MECHANISM that currently satisfies it -- including what it
# deliberately does NOT emit -- so a change to it is a deliberate, visible edit.
hw_docs = render(
{
"name": "hw",
"source": {"image": "a:1"},
"hardware": {"gpu": "intel-quicksync"},
}
)
hw_pod = next(
d[1] for d in hw_docs if d[1]["kind"] == "Deployment"
)["spec"]["template"]["spec"]
check(
{
"name": "dri",
"hostPath": {"path": "/dev/dri", "type": "DirectoryOrCreate"},
}
in hw_pod.get("volumes", []),
"hardware.gpu=intel-quicksync must render a /dev/dri hostPath volume "
"with type DirectoryOrCreate (a GPU-less node must degrade, not fail the mount)",
)
hw_mounts = hw_pod["containers"][0].get("volumeMounts", [])
check(
{"name": "dri", "mountPath": "/dev/dri"} in hw_mounts,
"hardware.gpu=intel-quicksync must mount /dev/dri READ-WRITE on the app container",
)
# The device mount is the WHOLE feature: no pod securityContext is invented. The image's
# own cont-init hook adds the app user to the device's group inside the container, and
# s6-setuidgid discards any Kubernetes-supplied supplementalGroups anyway (see the
# emission site). Asserting the absence is what stops someone adding an inert one back.
check(
"securityContext" not in hw_pod,
"hardware.gpu must NOT invent a pod securityContext (supplementalGroups is inert "
"under s6-setuidgid; the image's cont-init hook handles the device group)",
)
# Nothing privileged, and no node pinning: Kyverno restrict-host-namespaces (Enforce)
# rejects `privileged: true` in an app namespace, and placement is a separate concern
# with its own justification -- this field is about the DEVICE only.
check(
"privileged" not in hw_pod.get("securityContext", {})
and "securityContext" not in hw_pod["containers"][0]
and "nodeSelector" not in hw_pod,
"hardware.gpu must emit no privileged securityContext and no nodeSelector",
)
# The regression that matters: absence renders NOTHING. Mirrors the podSecurityContext
# negative above -- an SD that does not opt in must be byte-identical to before.
nohw_pod = next(
d[1] for d in render({"name": "nohw", "source": {"image": "a:1"}})
if d[1]["kind"] == "Deployment"
)["spec"]["template"]["spec"]
check(
"volumes" not in nohw_pod
and "volumeMounts" not in nohw_pod["containers"][0]
and "securityContext" not in nohw_pod,
"an SD without `hardware` must gain no dri volume, no mount and no securityContext",
)
# An SD that declares BOTH keeps its own podSecurityContext exactly as written -- the
# hardware field neither adds to it nor rewrites it.
hwm_pod = next(
d[1]
for d in render(
{
"name": "hwm",
"source": {"image": "a:1"},
"hardware": {"gpu": "intel-quicksync"},
"podSecurityContext": {"fsGroup": 1001, "supplementalGroups": [44]},
}
)
if d[1]["kind"] == "Deployment"
)["spec"]["template"]["spec"]
check(
hwm_pod["securityContext"] == {"fsGroup": 1001, "supplementalGroups": [44]},
"hardware.gpu must leave the SD's own podSecurityContext untouched",
)
# 6c) forward-auth (Option B): an exposed forward-auth app gets an oauth2-proxy sidecar that
# fronts the Gateway path (fail-closed), while oidc/native/none apps are unchanged.
fa_docs = render(
+1 -54
View File
@@ -1,10 +1,5 @@
#!/usr/bin/env bash
# ci/render-golden.sh — Stage: the renderer is trustworthy.
# (1) catalog/render.py passes its OWN output-contract self-test, and
# (2) every committed apps/ golden reproduces byte-for-byte from the canonical render.
# Two independent assertions about one subject, catalog/render.py. (1) was added
# 2026-10-01 because nothing in ci/ or .gitea/ had ever invoked `--selftest`; see the
# block above the invocation for why it lives here and not in a conformance check.
# ci/render-golden.sh — Stage: render==golden reproducibility gate.
#
# WHY: the committed apps/ tree is a set of GOLDENS produced by ONE canonical
# invocation of catalog/render.py. If a golden is hand-edited, or a catalog
@@ -59,54 +54,6 @@ if [ ! -f "$RENDER_PY" ]; then
summary "- render-golden: **SKIPPED** (renderer missing)"
exit 3
fi
# --- The renderer's OWN self-test, before anything is compared ---------------
# `catalog/render.py --selftest` is ~106 executed assertions on the renderer's output
# contract (default-deny covers both directions, the Kyverno namespace hook label, a valid
# auth-mode, the oauth2-proxy sidecar's cookie-name/-domain/-refresh flags, the
# placeholder-substitution guard, the observability refusals...). Until now NOTHING in ci/
# or .gitea/ invoked it — measured by grep on 2026-09-30 and recorded as the top follow-up
# of decisions/2026-09-30-oauth2-proxy-cookie-collision-fleet.md — so every one of those
# assertions was dev-time only and a merge gate could not tell a sound renderer from a
# gutted one.
#
# WHY HERE, AND NOT IN A CONFORMANCE CHECK. The lead was
# conformance/tests/l0-static/auditability-proofs-producible/check.sh, which drives
# `conformance/lib/{record_run,status}.py --selftest` "because nothing else drives them".
# That loop is NOT the right hook: that check serves R-SEC-03 and its subject is the
# EVIDENCE CHAIN (can a forged or stale run become a proof?). The renderer is not part of
# that chain, and hanging it there would make the check's name and its requirement mapping
# a lie. This stage's subject already IS catalog/render.py — it exists to assert that the
# renderer's output is trustworthy — so the renderer's own contract test belongs in it.
#
# It runs BEFORE the catalog/empty-catalog exit-3 below on purpose: the self-test is
# deterministic and catalog-independent (it renders synthetic ServiceDefinitions in
# memory), so an empty catalog must not be able to skip it.
#
# render-golden and the self-test are INDEPENDENT assertions, which is why both are needed:
# render-golden catches golden-vs-renderer drift but is satisfied by ANY renderer whose
# output was re-rendered, including one that just lost a contract invariant and had all ten
# goldens regenerated in the same commit. The self-test catches exactly that.
#
# FAILURE IS A HARD FAIL, NOT A SKIP: the renderer is source in this repo, so "could not
# run its self-test" has no legitimate environmental cause once python3 and render.py are
# both present (both asserted above). ci/gate.sh maps this stage's rc 1 to BLOCKED.
log "Stage: catalog/render.py --selftest (the renderer's own output-contract assertions)"
selftest_out="$(mktemp)"
if "$PYTHON" "$RENDER_PY" --selftest >"$selftest_out" 2>&1; then
ok "render.py --selftest passed"
summary "- render-selftest: **passed** (\`catalog/render.py --selftest\`)"
else
err "catalog/render.py --selftest FAILED — the renderer violates its own output contract."
err " Every golden under apps/ is produced by this renderer, so the goldens below cannot"
err " be trusted even if they reproduce byte-for-byte. Reproduce with:"
err " ${PYTHON} catalog/render.py --selftest"
sed 's/^/ /' "$selftest_out" >&2 || true
summary "- render-selftest: **FAILED** (\`catalog/render.py --selftest\` — renderer contract violated)"
rm -f "$selftest_out"
exit 1
fi
rm -f "$selftest_out"
if [ ! -d "$CATALOG_DIR" ] || ! compgen -G "${CATALOG_DIR}/*.yaml" >/dev/null 2>&1; then
skip "no catalog ServiceDefinitions under '${CATALOG_DIR}' — nothing to render."
summary "- render-golden: **SKIPPED** (empty catalog)"
@@ -0,0 +1,204 @@
---
date: 2026-09-30
branch: feat/sd-hardware-capability
title: ServiceDefinition gains a `hardware` capability field — `gpu: intel-quicksync` mounts /dev/dri and nothing else; the enum is what keeps the mechanism swappable
---
## What changed
A new optional top-level ServiceDefinition property, `hardware`, with one optional member,
`gpu`, whose enum today has exactly one value: `intel-quicksync`. **No catalog entry uses it.**
This PR delivers the capability; consuming it (Plex) is separate work on top.
- `schemas/servicedefinition.schema.json` — adds `hardware` (object, `additionalProperties: false`,
`minProperties: 1`) with `gpu` (enum `["intel-quicksync"]`). Schema version bumped **1.7.0 →
1.8.0**, changelog entry prepended to the schema `description` per that file's own convention.
- `catalog/render.py` — two module-level constants (`DRI_VOLUME_NAME`, `DRI_DEVICE_DIR`) and the
emission in `r_deployment`.
- `schemas/renderer-contract.md` — new `### hardware` field-mapping section, plus cross-field
invariant **8** in that file's numbered list.
- `schemas/README.md` — its `**Schema version:**` line said `1.0.0`; it had been stale since 1.1.0.
Synced to 1.8.0 and annotated that the schema `description` is the changelog of record. Added one
bullet to "Cross-field rules" for the new enum. That was the whole of the staleness; nothing
ballooned.
**What the renderer emits** for `hardware.gpu: intel-quicksync`, on the app container only (never a
sidecar), and it is exactly two things:
1. a hostPath volume `dri` → `/dev/dri`, `type: DirectoryOrCreate`;
2. a read-write `volumeMount` of `dri` at `/dev/dri` on the app container (deliberately not
`readOnly` — VA-API opens the render node read-write).
**Mounting the device is sufficient.** It emits **no** `privileged: true` (Kyverno
`restrict-host-namespaces` is Enforce on app namespaces and it is not needed), **no** `nodeSelector`
(placement is a separate concern with its own justification), **no** resource requests (there is no
app-container resource render path), and — the one that is worth spelling out — **no
`podSecurityContext.supplementalGroups`**.
**Why no `supplementalGroups`, when `/dev/dri/renderD128` is `root:render 0660`.** An earlier
revision of this branch merged the host `render` GID into `supplementalGroups`. **It was measured to
be a no-op and has been removed.** In the running container on prod-2, 2026-09-30:
PID 1 s6-svscan Uid: 0 Gid: 0 Groups: 0
pid 246 Plex Media Server Uid: 1000 Gid: 1000 Groups: 100 1000
`/etc/services.d/plex/run` ends in `exec s6-setuidgid plex …`, and the child's group list is exactly
the `/etc/group` account-database list — root's gid `0`, which PID 1 holds, has **vanished**. That is
a `setgroups()` that *replaces* rather than appends, so anything Kubernetes puts in
`supplementalGroups` reaches `s6-svscan` and is discarded before the app ever starts. What actually
works is the image's own `/etc/cont-init.d/45-plex-hw-transcode-and-connected-tuner`, which `find`s
every char device under `/dev/dri`, `stat -c '%g'`s its GID, `groupadd`s a matching group inside the
container when none exists, and `usermod -a -G`s the app user into it — every start, nothing
hardcoded; `cont-init.d` completes before `services.d`, so the database is correct at exec time.
Shipping an inert `supplementalGroups` would have been worse than shipping nothing: it would read as
the mechanism to whoever debugs this next. The reasoning lives in a comment **at the emission site**
and in `renderer-contract.md`, and `render.py --selftest` now asserts the *absence* of a
`securityContext`, so it cannot be "helpfully" added back without a test failing. A pleasant
corollary: because the hook re-`stat`s the device every start, host GID drift is **self-healing**,
and no GID is pinned anywhere in this repo.
**The design decision worth defending: the field is an enum of intent, not a device path.** The SD
says *what class of host device the workload needs*; the hostPath and the mount exist only in
`catalog/render.py`. Migrating to the Intel GPU device plugin (`gpu.intel.com/i915` resource
requests, no hostPath at all) is then a renderer change with zero catalog edits. Both the schema
`description` and the contract section say this explicitly, because the obvious "simplification" —
letting an SD name `/dev/dri` — would push a platform implementation detail into the operator-facing
contract and make that migration a catalog-wide edit.
**`DirectoryOrCreate` is load-bearing.** A node that comes back without the render device (rebuild,
BIOS change, driver regression, hardware swap) must degrade the workload to software transcoding,
not fail the mount: `apps-wave-0` is `wait: true` and `apps-wave-1` `dependsOn` it, so a failed
hostPath mount stalls the entire app tier for the sake of a feature that is not load-bearing.
`/dev` is tmpfs, so the empty directory the kubelet creates on such a node leaves nothing durable
behind. Do not "tighten" it to `Directory`. The reasoning is recorded at the emission site.
### Rebase note (2026-10-01, Ruth Okonkwo) — resolution only, no redesign
Rebased onto `main` @ `08deb20` (was cut from `0fbf544`). Re-landed as a single cherry-pick onto
current `main`; the branch ref was force-updated with `--force-with-lease`, so PR #27 is the same
PR with a new head.
**One conflict, in `catalog/render.py`, purely textual.** PR #28
(`fix/oauth2-proxy-cookie-collision-fleet`, merged as `6caa5f2`) inserted its own constants blocks
(`OAUTH2_ACCESS_TOKEN_LIFETIME_S` / `OAUTH2_COOKIE_REFRESH`, and `OAUTH2_PROXY_METRICS_PORT`) at the
same insertion point immediately after `KEYCLOAK_ISSUER` that this branch used for
`DRI_VOLUME_NAME` / `DRI_DEVICE_DIR`. The two blocks are unrelated and do not interact, so **both
sides were kept in full**, oauth2-proxy's first and the hardware block after it. Nothing was
dropped and nothing was rewritten: the rebased commit adds the **same 126 `render.py` lines** as the
pre-rebase commit (verified by comparing the added-line sets) and deletes none. In particular the
`supplementalGroups`-is-deliberately-not-emitted and `DirectoryOrCreate`-is-load-bearing comments
survive verbatim.
Re-verified after the resolution: `--selftest` PASS (rc 0), and a full re-render still produces a
**zero** diff in `apps/` — this PR remains inert, as it must be while no ServiceDefinition opts in.
## What I verified
Measured, in this worktree, on this branch:
- **`python3 catalog/render.py --selftest` → `render.py selftest: PASS`, rc 0.** New cases in the
existing `check(...)` style: the positive (volume + mount, and *no* `securityContext`, *no*
privileged, *no* nodeSelector), the negative that matters (an SD without `hardware` gains no
volume, no volumeMount and no `securityContext` — mirroring the existing `podSecurityContext`
negative), and a pass-through case (an SD declaring both `hardware` and its own
`podSecurityContext: {fsGroup: 1001, supplementalGroups: [44]}` keeps it **exactly** as written).
- **The self-tests actually bite.** I mutated the guard (`if hw_gpu == "intel-quicksync":` → `if
True:`) in a scratch copy and re-ran: **rc 1**, failing on "an SD without `hardware` must gain no
dri volume, no mount and no securityContext". A test that cannot fail is not a test; this one can.
- **Re-render is a zero diff.**
`python3 catalog/render.py --all catalog -o apps --wave-layout --no-image-automation` then
`git diff --stat apps/` → **empty**, and `git status --porcelain apps/` → **empty** (so the two
not-onboarded by-products, `apps/wave-1/keycloak.yaml` and `apps/wave-0/vm-demo.yaml`, were
rewritten byte-identically and needed no revert). The change is inert until a service opts in.
- **Schema validity and catalog conformance.** `Draft202012Validator.check_schema` passes; all
**12** `catalog/*.yaml` validate with **0** errors. On the new field specifically: `gpu:
intel-quicksync` accepted; `gpu: /dev/dri` rejected (enum); `hardware: {gpu: …, path: …}` rejected
(`additionalProperties: false`); `hardware: {}` rejected (`minProperties: 1`).
- **Hand-checked the emitted YAML** by rendering a throwaway SD carrying `hardware`, a Longhorn
volume and its own `podSecurityContext: {runAsUser: 1000, supplementalGroups: [44]}`. Result:
`volumeMounts` = config then `{name: dri, mountPath: /dev/dri}`; `volumes` = the PVC then
`{name: dri, hostPath: {path: /dev/dri, type: DirectoryOrCreate}}`; `securityContext` =
`{runAsUser: 1000, supplementalGroups: [44]}` — i.e. **unchanged**, nothing appended. That toy SD
was never written into `catalog/`.
- **GATE: `rc 3` — UNDETERMINED, NOT green.** Run in the pinned toolbox from this worktree with
both CLAUDE.md §3 workarounds (`DEV_KUBECONFIG=/dev/null`, and an `ENGINE` shim bind-mounting the
main checkout's `.git` at its own absolute path). Diff base `origin/main @ 0fbf544d5e2f`; touched
requirements `R-DNS-03 R-GITOPS-02 R-IDP-02`. Run **four** times on this branch with an identical
result each time, most recently against tree `c132893045b1` — the final commit differs from that
run only by this paragraph. Per stage:
| stage | result |
|---|---|
| spec-immutability | **SKIPPED** — sibling spec repo not at `/work/../home-network-k3s`. Expected from a worktree; tolerated by §3. |
| decisions-log | pass |
| lint | pass |
| l0 | pass — 19 offline cases, `0 failure(s), 0 skip(s), 0 unwired` |
| render-golden | pass — *"all 10 onboarded service golden(s) reproduce from the canonical render"* |
| l1 | pass — 3 offline checks, `0 failure(s), 0 skip(s), 0 unwired`; coverage 51/51 requirements scaffolded |
| approval-policy | pass (`event='none'`, not a pull_request event — N/A locally; it will evaluate for real on the PR) |
| gitea-settings | **SKIPPED** — `SETTINGS_API_TOKEN` unset. Expected; branch protection and runner registration are therefore UNVERIFIED by this run. |
| conformance (L2/L3) | **SKIPPED** — `CLUSTER_AVAILABLE!=true`. Phase-0 standing skip. |
The gate's own verdict: *"Stages that did not run: spec-immutability gitea-settings"*. **No stage
failed, and no stage skipped for a reason caused by this change.** rc 3 is reported as rc 3; I am
not calling this green.
- **Blast radius of the schema change:** `hardware` is absent from every catalog entry, so
`apply_defaults` never materializes it and `s.get("hardware")` is `None` on all 12 — which is what
the zero diff above demonstrates rather than asserts.
- **Touched no sensitive path.** Changes are confined to `schemas/`, `catalog/`, `decisions/`; per
`infra/gitea/approval-policy.json` none of those are sensitive. `conformance/**` and `clusters/**`
are untouched. I did not run `--emit-waves` (it writes `clusters/base/`, and the set of files
under `apps/` did not change).
## What I did NOT verify
- **Whether containerd's device cgroup permits a non-privileged pod to `open()` a bind-mounted char
device. Still unverified, and it is the single largest technical risk in this workstream.**
containerd installs a per-container **device cgroup** allow-list, and whether
`/dev/dri/renderD128` arriving as a bind-mounted device node is permitted by it on this
k3s/containerd version is **unmeasured**. Settling it requires applying a pod, which is an
operator action, not an agent one. The container's PID 1 running as uid 0 helps on the DAC side
but does **not** bypass the cgroup gate. If it blocks, the mount will succeed, the image's group
hook will do its job, and the `open()` will still fail — and the fallback is the Intel GPU device
plugin, which *sets* the cgroup entry. Note that that fallback is a **renderer-only change**: no
ServiceDefinition moves, because the authoring surface is an intent enum. That is the design
justifying itself, and it is the reason to keep the enum even if the hostPath mechanism loses.
- **That a missing device degrades silently — and nothing will tell you.** This is the accepted cost
of `DirectoryOrCreate`. The workload will software-transcode and **no signal reports it**: media
apps in this catalog declare no `metricsEndpoint`, so no ServiceMonitor exists, and there is no
metric to alert on. I deliberately did **not** propose an alert, because an alert that cannot fire
is worse than none — see the 2026-09-13 correction in `renderer-contract.md` §`observability`,
where twelve monitors discovered zero targets. The only proof of hardware transcode is an operator
seeing it in the application's own dashboard.
- **That "mounting the device is sufficient" generalizes beyond this one image.** It does not, and
I have not verified that it does. The claim was measured for an image that resolves the device
group itself in `cont-init.d`. A future consumer of `hardware.gpu` that runs non-root with no
equivalent hook would still need the group solved — and the fix for that case is a **renderer**
change (per-image handling, or the device plugin), never a new SD field. I scoped the claim in
`renderer-contract.md` rather than letting it read as general; the first such consumer must
re-measure.
- **The feature now depends on a hook inside an image that floats.** The `cont-init.d` behaviour
above was read from digest `sha256:e0ab27395614…` (`plexmediaserver 1.43.4.10903`), but the
deployment runs `plexinc/pms-docker:latest` with `imagePullPolicy: Always`. **A silent pull can
change the behaviour this feature depends on, with no commit anywhere in this repo to show for
it** — and the failure would be the silent software-transcode described above. I have not fixed
that here (it is the Plex wiring's tag/digest decision, not the capability's), but it should not
be forgotten.
- **GID drift is NOT on this list any more, deliberately.** An earlier revision pinned
`INTEL_RENDER_GID = 992` and listed "a node rebuild could move it" as an open risk. The hook
re-`stat`s the device on every start, so drift is self-healing and no GID is pinned anywhere. The
risk is retired, not overlooked.
- **Anything on the live cluster.** I ran no `kubectl` and I opened no shell in any container. The
node facts (device present, `root:render` 0660, `i915` bound, no device plugin, no
`gpu.intel.com/*` advertised), the in-container measurements quoted above, and the Kyverno
posture (`restrict-host-namespaces` Enforce; no hostPath or `supplementalGroups`
restriction; no PSA labels on app namespaces) are **prior recon handed to me, not re-measured by
me**. Per CLAUDE.md §2b, treat them as second-hand until re-queried.
- **That CI exercises the self-tests. It does not.** `git grep -- --selftest ci/` returns nothing —
no gate stage invokes the renderer's self-test. They are the executable spec and house style, and
I ran them by hand, but a future regression in this code path **will not be caught by the gate**.
Wiring `--selftest` into a gate stage is a real gap and is not part of this PR (`ci/` is a
sensitive path and would drag in a human reviewer for a change unrelated to this one).
- **Runtime behaviour of any kind.** The gate proves the L0/L1 offline subset only;
`vars.CLUSTER_AVAILABLE` is unset, so L2/L3 conformance does not run in CI.
@@ -1,205 +0,0 @@
---
date: 2026-10-01
branch: fix/render-selftest-runs-in-gate
title: catalog/render.py's 106 assertions were dev-time only; the gate now runs them and blocks on them
---
## What changed
Single subject, one file: **`ci/render-golden.sh` now invokes `catalog/render.py --selftest`**
and exits 1 when it fails. Nothing else. No renderer change, no new conformance test, no new
gate stage.
This closes the top follow-up of `decisions/2026-09-30-oauth2-proxy-cookie-collision-fleet.md`:
> Nothing in `ci/` or `.gitea/` runs `catalog/render.py --selftest` — I grepped for it. So the
> twenty-odd assertions in that selftest, including the six I just added, are dev-time only: a
> future edit that drops either flag *and* re-renders the goldens would pass the entire merge
> gate.
Re-measured here and confirmed: before this commit, `grep -rl -- --selftest` over `ci/` and
`.gitea/` returned **nothing** for the renderer. The selftest executes **106** assertions
(`check()` invocations counted at runtime via `sys.settrace`, without editing `render.py`; 109
static call sites, three of them on failure-only branches). All 106 were unexercised by the
merge gate.
### The lead was WRONG, and here is why
The brief named `conformance/tests/l0-static/auditability-proofs-producible/check.sh:291` as the
one-line fix. That line is real and it does drive selftests:
for tool in ("record_run.py", "status.py"):
r = run([sys.executable, str(LIB / tool), "--selftest"])
But it is **the wrong hook**. That loop is section (d) of a check that serves **R-SEC-03**, and
its subject is the *evidence chain* — can a self-skipped run, or a stale ledger row, become an
audit proof? `record_run.py` and `status.py` are **in** that chain; `catalog/render.py` is not.
Hanging the renderer there would have made the check's name, its `intent` annotation and its
requirement mapping (`R-SEC-03`) all untrue, and a future decision about the renderer could then
only be expressed by weakening a *security-auditability* check. That is the precise coupling the
cookie-collision entry refused for its own check, and it applies here.
The second option that entry offered — a new scope-declaring L0 test — is the supported mechanism
for a **whole-repo invariant** (`ci/l0.sh` Stage 3b's `--include-scope whole-repo` union, ratcheted
by `tests/l0-static/whole-repo-invariants-wired`). It is the right shape for an invariant
quantified over the tracked tree. It is the wrong shape for *"a script in this repo must pass its
own unit test"*: it would need to declare requirements it does not serve, and it would be two new
files of several hundred lines by this tree's conventions for what is a six-line invocation.
**The honest home is `ci/render-golden.sh`** — the gate stage whose subject *already is*
`catalog/render.py`. It already resolves `$PYTHON` and `$RENDER_PY`, already reports rc 3 when
either is missing, already exits 1 on renderer error, and `ci/gate.sh` already maps it through
`for s in lint l0 render-golden l1` where a FAIL is BLOCKED and a SKIP is UNDETERMINED. The
stage's two assertions are independent and both are needed:
- `render-golden` catches golden-vs-renderer drift, but is satisfied by **any** renderer whose
output was re-rendered — including one that just lost a contract invariant and had all ten
goldens regenerated in the same commit. That is exactly the hole named in the quote above.
- `--selftest` catches that, and is blind to golden drift.
The file header and the stage name were updated so neither is a lie: the stage is now "the
renderer is trustworthy", with two numbered assertions, and the step summary emits its own
`- render-selftest:` line so the gate output still NAMES what ran.
The selftest runs **before** the empty-catalog `exit 3` on purpose: it renders synthetic
ServiceDefinitions in memory and is catalog-independent, so an empty catalog must not be able to
skip it.
`catalog/render.py` itself was **not touched** (owned by a concurrent branch this session), and
neither was any file under `conformance/`.
## What I verified
- **The selftest runs and passes: rc 0, `render.py selftest: PASS`, 106 assertions executed.**
Counted at runtime, not from the source: a `sys.settrace` hook counting `check()` calls, in a
throwaway script outside the repo, with no edit to `render.py`.
- **The stage passes with it wired in**, host run of the single stage:
[gate] Stage: render==golden reproducibility (canonical render vs committed apps/)
[gate] Stage: catalog/render.py --selftest (the renderer's own output-contract assertions)
[ ok ] render.py --selftest passed
[ ok ] render==golden: all 10 onboarded service golden(s) reproduce from the canonical render
rc **0**. The golden comparison still reports all 10 goldens, so the new invocation did not
displace what the stage already asserted.
- **IT CAN FAIL — provoked twice, against a mutated COPY of the renderer, never against the
tree.** `ci/render-golden.sh` already honours `RENDER_PY`, so each mutant lived in
`/tmp/claude-1000/desmond-selftest/` and the repo's `catalog/render.py` was never modified —
which matters, because that file belongs to a concurrent branch this session.
- **Mutation 1 — the regression this selftest was written for.** Deleted the
`f"--cookie-name=_oauth2_proxy_{s['name']}"` emission from `_oauth2_sidecar_container()`,
i.e. reverted the fleet-wide cookie-collision fix of PR #28. Stage rc **1**. Note the shape
of the failure, because it is a real (minor) weakness and is listed as a follow-up: the
selftest died with an uncaught `StopIteration` out of `next(a for a in container["args"] if
a.startswith("--cookie-name="))` at `render.py:2641`, rather than printing its collected
`SELFTEST FAIL:` lines. It is correctly non-zero and the stage correctly blocks, but the
operator gets a traceback instead of the assertion message.
- **Mutation 2 — a clean, named assertion failure.** `OAUTH2_COOKIE_REFRESH = "2m30s"` ->
`"10m"`, widening the refresh past the measured 300s access-token lifetime. Stage rc **1**:
[fail] catalog/render.py --selftest FAILED — the renderer violates its own output contract.
[fail] Every golden under apps/ is produced by this renderer, so the goldens below cannot
[fail] be trusted even if they reproduce byte-for-byte. Reproduce with:
[fail] python3 catalog/render.py --selftest
SELFTEST FAIL: --cookie-refresh=10m parses to 600.0s, which is not inside (0, 300s). A
refresh at or past the access-token lifetime never renews in time, which is the same
logout it was added to stop
This is the mutation that proves the wiring's value most directly: mutation 2 would **not**
be caught by `render-golden`'s diff if the goldens were regenerated in the same commit, and
it is one of the two flags the cookie-collision entry explicitly recorded as **ungated**.
- **THE NON-ZERO PROPAGATES ALL THE WAY TO `gate.sh`'s rc — provoked, in the pinned toolbox, not
reasoned about.** `RENDER_PY=/desmond-selftest/render-mutant2.py ./ci/gate.sh`:
[ ok ] stage spec-immutability: pass
[ ok ] stage decisions-log: pass
[ ok ] stage lint: pass
[ ok ] stage l0: pass
[gate] Stage: catalog/render.py --selftest (the renderer's own output-contract assertions)
[fail] catalog/render.py --selftest FAILED — the renderer violates its own output contract.
SELFTEST FAIL: --cookie-refresh=10m parses to 600.0s, which is not inside (0, 300s). ...
[fail] stage render-golden: FAIL
[ ok ] stage l1: pass
...
[fail] render-golden: FAILED -> blocks merge
[fail] MERGE GATE: BLOCKED
`gate.sh` rc **1**. Three links in the chain, each measured: selftest rc 1 -> stage rc 1 ->
gate rc 1 / `MERGE GATE: BLOCKED`. This mattered enough to provoke rather than assume: this
repo has a documented habit of exit codes getting flattened, and
`.gitea/workflows/gate.yml:138` runs `./ci/gate.sh || [ "$?" -eq 3 ]`, which tolerates rc 3 —
so a stage that degraded a failure to a skip would be swallowed by CI. rc 1 is not tolerated
there, and rc 1 is what this produces.
- **`spec-immutability` genuinely RAN and passed, it did not skip.** The two-mount worktree shim
plus `-e SPEC_REPO=/home-network-k3s` (the fix recorded in the 2026-09-30 cookie entry) works:
`stage spec-immutability: pass`, not SKIPPED. R-TEST-06 is therefore actually covered by these
runs.
- **No collision with PR #29.** The brief warned that `feat/oauth2-proxy-auth-failure-visible`
had added checks under `conformance/tests/l0-static/`. Measured —
`git diff origin/main...origin/feat/oauth2-proxy-auth-failure-visible --name-only` — it touches
**no `conformance/` path at all**: `decisions/`, `platform/oauth2-proxy/` (a promtool test, an
alert rule, its kustomization), `runbooks/`. Disjoint from this branch either way, since this
branch touches only `ci/render-golden.sh` and this file.
- **Priya's files were not touched.** `catalog/render.py`, `schemas/servicedefinition.schema.json`,
`apps/wave-*` and `conformance/tests/l1-rendering/` are all untouched by this branch — confirmed
by `git diff --name-only`, which lists exactly two paths.
- **`catalog/render.py --selftest` was the LAST unwired selftest in the tree.** Measured:
`grep -rl -- --selftest` across the repo finds selftest-capable artifacts in exactly four
places, and after this commit every one is driven by the gate — `conformance/lib/record_run.py`
and `conformance/lib/status.py` by `auditability-proofs-producible/check.sh:291`, the Keycloak
client-secret reconciler (embedded in `platform/keycloak/client-secret-reconciler.yaml`) by
`keycloak-client-secret-reconciler/check.sh:633`, and `catalog/render.py` by this change.
## What I did NOT verify
- **Local gate: rc 3 (UNDETERMINED), NOT 0. This is not green and must not be reported as
green.** Measured in the pinned toolbox on `eeb6211`:
[ ok ] spec-immutability: green
[ ok ] approval-policy: green
[ ok ] decisions-log: green
[ ok ] lint: green
[ ok ] l0: green
[ ok ] render-golden: green
[ ok ] l1: green
[fail] MERGE GATE: UNDETERMINED — greenness was NOT ESTABLISHED.
[fail] Stages that did not run: gitea-settings
Nothing FAILED. The single cause is `gitea-settings` **SKIPPED** — `SETTINGS_API_TOKEN` is not
set in my environment, so branch protection and runner registration were not read; CI has that
token and the stage is expected to run there. **L2/L3 `conformance`** also skipped
(`CLUSTER_AVAILABLE` unset) but is a *declared* gap that `gate.sh` treats as not-applicable
rather than undetermined, so it is not what produced the 3. Touched requirements
**R-GITOPS-01 R-TEST-02 R-TEST-03 R-TEST-06** are therefore **UNVERIFIED at L2/L3**, all four.
The stage this branch changes, `render-golden`, is green **and** was independently proven able
to turn the whole gate red (above).
- **`approval-policy` passed LOCALLY only because there is no pull_request event**
(`not a pull_request event (event='none') — approval policy N/A`). That is not evidence that it
passes. This diff touches `ci/**`, a sensitive path, so in CI the stage is expected to **FAIL**
until a non-author approving review exists. That is §1 working, fail-closed, not a defect.
- **The selftest's own quality is unchanged and ungated.** I wired its *invocation*; I did not
review its 106 assertions, and nothing ratchets them. Deleting the body of `_selftest()` would
still pass this stage, because an empty selftest returns 0. The repo already has the right
answer for that — `conformance/tests/l0-static/keycloak-client-secret-reconciler/mutations.py`
injects regressions one at a time and requires `--selftest` to catch every one — and applying
that pattern to `catalog/render.py` is the follow-up, deliberately not done here: it is a
second subject, it touches `conformance/`, and it would need to live beside the renderer whose
file another branch owns this session. **Until it exists, "the selftest runs" is proven and
"the selftest is falsifiable" rests on my two hand-run mutations, which are not in the gate.**
- **Mutation 1's traceback is not fixed.** `render.py:2641`'s `next(...)` has no default, so the
absence of `--cookie-name` raises `StopIteration` and the selftest's collected `SELFTEST FAIL:`
lines are never printed. Correct verdict, poor diagnostics, and it suppresses the other
assertions that mutation also broke. The one-line fix (`next(..., None)` plus a `check()`) is in
`catalog/render.py`, which is another branch's file this session, so it is a follow-up and not
a thing I touched.
- **Nothing about the live cluster.** No `kubectl` of any kind was run on this branch — not even
read-only; this change is offline CI wiring and needed none. No `apply`/`edit`/`patch`, no
`flux reconcile`. Merging this deploys nothing: it changes only what the gate refuses.
- **CI's own run of this branch.** Quoted in the PR body from the commit's `/statuses`, with
descriptions read rather than colours. A *skipped* Gitea job reports `success` with
"Has been skipped" and certifies nothing.
- **`vars.CLUSTER_AVAILABLE` is still unset in CI**, so even a fully green PR proves the L0/L1
offline subset and nothing about runtime behaviour.
- **I did not re-measure branch protection, required approvals, or the runner.** Those are
CLAUDE.md §2 claims and §2b says re-measure before relying on them; this branch relies on none
of them beyond the PR flow itself, and I read the gate's own output rather than the API.
+4 -1
View File
@@ -15,7 +15,7 @@ docs/13 exactly. Do not diverge from it here — escalate spec problems, don't f
| `servicedefinition.schema.json` | Formal JSON Schema (draft 2020-12) for a ServiceDefinition. Enforces enums, required fields, patterns, and cross-field invariants. |
| `renderer-contract.md` | Authoritative field → rendered-resource mapping, with requirement IDs. The contract other workstreams consume. |
**Schema version:** 1.0.0 (in the schema `description`). Treat any change as an interface change:
**Schema version:** 1.8.0 (in the schema `description`, which carries the full per-version changelog — this line had said `1.0.0` since 1.1.0 and is now synced). Treat any change as an interface change:
propose on the task board, update dependent renderers/tests in the same change, never silently break a
consumer (coordination-protocol §2).
@@ -35,6 +35,9 @@ consumer (coordination-protocol §2).
- `ingressRoutes[].host` must match `<label>.home.keithjackson.dev`.
- `storage[].class ∈ {longhorn, bulk}`; `exposure ∈ {internal, public}`; `egress ∈ {internet, vpn}`;
`auth.mode ∈ {oidc, forward-auth, native, none}`.
- `hardware.gpu ∈ {intel-quicksync}` — a host **capability class**, never a device path. The enum
is the contract: the mechanism that satisfies it lives only in `catalog/render.py` (see
`renderer-contract.md` §`hardware`).
- `additionalProperties: false` everywhere — unknown keys are rejected (catches typos in review/CI).
## Validate examples against the schema — L1, no cluster (R-TEST-02)
+63
View File
@@ -160,6 +160,64 @@ platform must render, and cites the requirement each mapping satisfies. Changing
- **Requirements:** **R-STATE-01** (rebuild survival), **R-STATE-02** (Longhorn replication),
**R-STATE-04** (bulk tier). (docs/04)
### `hardware` (`gpu`) — host device capability
- **The field is INTENT, never MECHANISM.** `hardware.gpu` is an **enum of capability classes**
(today: `intel-quicksync`), not a device path, not a hostPath, not a resource request. The
ServiceDefinition says *what class of host device this workload needs*; how the platform hands it
over is `catalog/render.py`'s business and lives nowhere else. **Do not "simplify" this into a raw
`/dev/...` path.** The entire payoff is that migrating from the hostPath mechanism below to the
Intel GPU device plugin (`resources.limits: gpu.intel.com/i915`) is a renderer change with **zero**
catalog edits and zero re-review of per-service device wiring. Extend the enum for a genuinely
different capability class (e.g. `nvidia-nvenc`); never to name a second route to the same one.
- `gpu: intel-quicksync` → on the **app container only** (never a sidecar), exactly two things:
1. a **hostPath volume** `dri` → `/dev/dri`, **`type: DirectoryOrCreate`**;
2. a **read-write volumeMount** of `dri` at `/dev/dri` on the app container (VA-API opens the
render node read-write, so it is deliberately not `readOnly`).
**Mounting the device is sufficient. That is the whole mapping.**
- **`podSecurityContext.supplementalGroups` is deliberately NOT rendered, and adding it would be a
regression rather than a hardening.** `/dev/dri/renderD128` is `root:render` `0660`, so the obvious
move is to push the host `render` GID in — and it does not work. **Measured in the running
container on prod-2, 2026-09-30:** the image's own
`/etc/cont-init.d/45-plex-hw-transcode-and-connected-tuner` hook `find`s every char device under
`/dev/dri`, `stat -c '%g'` each one's GID, `groupadd`s a matching group *inside the container* when
none exists, and `usermod -a -G` the app user into it — every start, nothing hardcoded.
`cont-init.d` completes before `services.d`, so the account database is already correct when the
service execs. The service then starts via `exec s6-setuidgid plex …`, and `s6-setuidgid`
**replaces** the process group list from `/etc/group` rather than appending to it:
`Plex Media Server` runs as `Uid: 1000 Gid: 1000 Groups: 100 1000`, with root's gid `0` — which
PID 1 (`s6-svscan`) does hold — **gone**. A Kubernetes `supplementalGroups` therefore reaches
`s6-svscan` and is discarded before the app ever starts. It would be inert *and* it would mislead
whoever debugs this next. The renderer asserts its own absence in `render.py --selftest`.
- A corollary worth having: because the hook re-`stat`s the device on every start, the host GID
**drifting** (a node rebuild renumbering the *systemd dynamic* `render` group) is self-healing.
No GID is pinned anywhere in this repo.
- **Scope of that measurement, stated so nobody over-reads it.** "Mounting the device is
sufficient" was established for an image that resolves the device group itself. A FUTURE
consumer that runs non-root and has no equivalent init hook would still need the group solved
somehow — and the answer for that case is a **renderer** change (per-image handling, or the
device plugin), not a new ServiceDefinition field. The enum is what keeps that option open. The
first such consumer must re-measure rather than assume this bullet covers it.
- **`DirectoryOrCreate` is load-bearing — do not tighten it to `Directory`.** A node that comes back
without the render device (rebuild, BIOS change, driver regression, hardware swap) must degrade the
workload to **software transcoding**, not fail the mount. `apps-wave-0` is `wait: true` and
`apps-wave-1` `dependsOn` it, so a failed hostPath mount stalls the **whole app tier** for the sake
of a feature that is not load-bearing. `/dev` is tmpfs, so the empty directory the kubelet creates
on such a node leaves nothing durable behind.
- **The cost of that choice, stated plainly:** a missing device is **silent**. The workload
software-transcodes and nothing reports it. There is no metric to alert on — media apps here
render no `metricsEndpoint`, so no ServiceMonitor exists — and an alert that cannot fire is worse
than none (see §`observability`'s 2026-09-13 correction). The only proof of hardware transcode is
an operator observing it in the application's own dashboard.
- **Deliberately NOT rendered** from this field: no `privileged: true` (Kyverno
`restrict-host-namespaces` is Enforce on app namespaces and would reject it, and the render node
does not need it); no `nodeSelector` (placement is a separate concern with its own justification —
this field is about the device, not about where the pod lands); no resource requests/limits (there
is no app-container resource render path, and inventing one here would be scope creep).
- Absent ⇒ **nothing at all** is rendered; every service that does not opt in stays byte-identical.
Ignored for `kind: vm` — a VM receives devices through KubeVirt's own `domain.devices`.
- **Requirements:** **R-PLAT-02** (the PaaS provides the host capability; the app declares need).
### `ingressRoutes[]` (`host` / `port`) — HTTPRoute + Certificate + DNS
- Each entry renders an **HTTPRoute** attaching `host` → the service `port` on the shared Gateway.
- TLS: the Gateway terminates with the cert-manager **wildcard `*.home.keithjackson.dev`** certificate
@@ -286,6 +344,11 @@ platform must render, and cites the requirement each mapping satisfies. Changing
`external-dns.alpha.kubernetes.io/controller: none`, and no `public` service's route carries it.
Derived from `exposure` alone; enforced on every gate run by
`conformance/tests/l0-static/internal-services-publish-no-public-dns` (R-DNS-03).
8. **`hardware` is inert until opted into, and its mechanism is renderer-local.** An SD with no
`hardware` key renders no volume and no volumeMount; an SD *with* it gains no `securityContext`
either. Both directions are proven by `render.py --selftest`. The `/dev/dri` hostPath and the
`DirectoryOrCreate` type appear in `catalog/render.py` and nowhere else, so the capability's
implementation can be replaced without touching a single ServiceDefinition.
## Requirement traceability (contract-level)
Primary **R-GITOPS-02** (single reviewable object wires every requirement). Field-level: R-NET-01/02
+14 -1
View File
@@ -2,7 +2,7 @@
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://home.keithjackson.dev/schemas/servicedefinition.schema.json",
"title": "ServiceDefinition",
"description": "Operator-facing onboarding contract for the home k3s + Longhorn platform. One ServiceDefinition per file under catalog/. Captures INTENT; the platform compiles it into concrete resources (see schemas/renderer-contract.md). Canonical source of truth: spec docs/13-service-catalog.md. Serves R-GITOPS-02; ties into R-NET-*, R-STATE-*, R-TLS-*, R-DNS-*, R-OBS-*, R-UPD-*, R-IDP-*. Schema version: 1.7.0 (no field changes; a MAPPING change to `exposure`, which had never had an effect on DNS. `exposure: internal` now renders ExternalDNS's own controller annotation onto every HTTPRoute it emits, set to a value other than `dns-controller`, so external-dns (v0.21.0, source/gateway.go -> annotations.IsControllerMismatch) skips the route before resolving any hostname and the service is INCAPABLE of producing a public record. Both docs/09 R-DNS-03 and schemas/renderer-contract.md already said `internal` renders no public DNS record; nothing enforced it, and six internal services -- jackett, pihole, qbittorrent, radarr, semaphore, sonarr -- were publishing `<name>.home.keithjackson.dev` into the PUBLIC Cloudflare zone answering with the RFC1918 Gateway VIP, which published the internal subnet and a full service inventory and made each name a DNS-rebind primitive. Enforced on every gate run by conformance/tests/l0-static/internal-services-publish-no-public-dns). 1.6.0 (splits `observability.metrics` from WHERE metrics live: adds optional `metricsEndpoint` (port/name/path/interval) and optional `livenessWorkload` (kind/name). `metrics: true` previously rendered a ServiceMonitor hardcoded to a port named `metrics` that no rendered Service ever defined, so all twelve app monitors discovered ZERO targets and every `up{job=...}` SLI was structurally inert on prod; a monitor is now rendered only from a declared `metricsEndpoint`, which also renders the Service port and the monitoring-namespace ingress allow, and the `up` SLI is expressed against kube-state-metrics -- a signal that exists for every workload without a per-app exporter. `livenessWorkload` names the object whose readiness IS the service, for the case where the running workload is not the one this SD renders). 1.5.0 (adds optional `inClusterClients` — additional in-cluster clients, named by NAMESPACE, admitted to a declared port of this service DIRECTLY, bypassing the shared Gateway. `ingressRoutes` admits only the Gateway proxy identity and `dependencies` only links catalog services, so a PLATFORM controller dialling a Service ClusterIP had no authorable form at all; that gap is what deadlocked prod's GitOps when its GitRepository moved to the in-cluster Gitea Service. Renders one least-privilege ingress CiliumNetworkPolicy per entry — the named namespace and the declared port, nothing wider). 1.4.0 (makes the `bulk` tier REAL: a bulk storage entry now carries `export` (the absolute NFS export path on the ${BULK_NFS_SERVER} host) and renders a STATIC, pre-bound NFS PersistentVolume + PVC over data that already exists outside the cluster, plus optional `subPath` and `readOnly` on any entry. Resolves the OQ-2 bulk-backing stub for STATIC pre-existing datasets; dynamic bulk provisioning remains unimplemented and unneeded). 1.3.0 added optional pod-level podSecurityContext for the container kind — k8s PodSecurityContext emitted verbatim onto the pod spec; its fsGroup unblocks non-root images e.g. semaphore/UID-1001 writing a mounted Longhorn PVC). 1.2.0 added optional per-container env/envFrom/lifecycle for the container kind — k8s EnvVar/EnvFromSource/Lifecycle shapes emitted verbatim onto the app container; unblocks headless-install/DB-backed apps e.g. gitea/nextcloud/semaphore/unifi/minecraft. 1.1.0 added the shape invariant exposure:public => auth.mode!=none (docs/13 v1.1 / docs/15 / R-IDP-02).",
"description": "Operator-facing onboarding contract for the home k3s + Longhorn platform. One ServiceDefinition per file under catalog/. Captures INTENT; the platform compiles it into concrete resources (see schemas/renderer-contract.md). Canonical source of truth: spec docs/13-service-catalog.md. Serves R-GITOPS-02; ties into R-NET-*, R-STATE-*, R-TLS-*, R-DNS-*, R-OBS-*, R-UPD-*, R-IDP-*. Schema version: 1.8.0 (adds optional `hardware` - a host HARDWARE CAPABILITY declaration for the container kind, currently one member: `hardware.gpu: intel-quicksync`, meaning the workload needs the host iGPU render device for hardware video transcode. It is an ENUM OF INTENT, not a device path: the renderer satisfies it today with just two things -- a /dev/dri hostPath volume (`DirectoryOrCreate`) and a read-write /dev/dri volumeMount on the app container -- and a later move to the Intel GPU device plugin is then a render.py change with no catalog edit. It sets NO podSecurityContext.supplementalGroups: measured in-container on prod 2026-09-30, the pms-docker cont-init hook adds the app user to the device's group inside the container and `s6-setuidgid` then REPLACES the process group list from /etc/group, discarding anything Kubernetes supplied, so such an entry would be inert and misleading. Renders NOTHING when absent, so every existing app stays byte-identical. Emits no privileged securityContext (Kyverno restrict-host-namespaces would reject it and it is not needed), no nodeSelector (node placement is a separate concern with its own justification) and no resource requests). 1.7.0 (no field changes; a MAPPING change to `exposure`, which had never had an effect on DNS. `exposure: internal` now renders ExternalDNS's own controller annotation onto every HTTPRoute it emits, set to a value other than `dns-controller`, so external-dns (v0.21.0, source/gateway.go -> annotations.IsControllerMismatch) skips the route before resolving any hostname and the service is INCAPABLE of producing a public record. Both docs/09 R-DNS-03 and schemas/renderer-contract.md already said `internal` renders no public DNS record; nothing enforced it, and six internal services -- jackett, pihole, qbittorrent, radarr, semaphore, sonarr -- were publishing `<name>.home.keithjackson.dev` into the PUBLIC Cloudflare zone answering with the RFC1918 Gateway VIP, which published the internal subnet and a full service inventory and made each name a DNS-rebind primitive. Enforced on every gate run by conformance/tests/l0-static/internal-services-publish-no-public-dns). 1.6.0 (splits `observability.metrics` from WHERE metrics live: adds optional `metricsEndpoint` (port/name/path/interval) and optional `livenessWorkload` (kind/name). `metrics: true` previously rendered a ServiceMonitor hardcoded to a port named `metrics` that no rendered Service ever defined, so all twelve app monitors discovered ZERO targets and every `up{job=...}` SLI was structurally inert on prod; a monitor is now rendered only from a declared `metricsEndpoint`, which also renders the Service port and the monitoring-namespace ingress allow, and the `up` SLI is expressed against kube-state-metrics -- a signal that exists for every workload without a per-app exporter. `livenessWorkload` names the object whose readiness IS the service, for the case where the running workload is not the one this SD renders). 1.5.0 (adds optional `inClusterClients` — additional in-cluster clients, named by NAMESPACE, admitted to a declared port of this service DIRECTLY, bypassing the shared Gateway. `ingressRoutes` admits only the Gateway proxy identity and `dependencies` only links catalog services, so a PLATFORM controller dialling a Service ClusterIP had no authorable form at all; that gap is what deadlocked prod's GitOps when its GitRepository moved to the in-cluster Gitea Service. Renders one least-privilege ingress CiliumNetworkPolicy per entry — the named namespace and the declared port, nothing wider). 1.4.0 (makes the `bulk` tier REAL: a bulk storage entry now carries `export` (the absolute NFS export path on the ${BULK_NFS_SERVER} host) and renders a STATIC, pre-bound NFS PersistentVolume + PVC over data that already exists outside the cluster, plus optional `subPath` and `readOnly` on any entry. Resolves the OQ-2 bulk-backing stub for STATIC pre-existing datasets; dynamic bulk provisioning remains unimplemented and unneeded). 1.3.0 added optional pod-level podSecurityContext for the container kind — k8s PodSecurityContext emitted verbatim onto the pod spec; its fsGroup unblocks non-root images e.g. semaphore/UID-1001 writing a mounted Longhorn PVC). 1.2.0 added optional per-container env/envFrom/lifecycle for the container kind — k8s EnvVar/EnvFromSource/Lifecycle shapes emitted verbatim onto the app container; unblocks headless-install/DB-backed apps e.g. gitea/nextcloud/semaphore/unifi/minecraft. 1.1.0 added the shape invariant exposure:public => auth.mode!=none (docs/13 v1.1 / docs/15 Line truncated
"type": "object",
"additionalProperties": false,
"required": ["name", "source"],
@@ -139,6 +139,19 @@
"supplementalGroups": { "type": "array", "items": { "type": "integer" }, "description": "Additional GIDs added to the first process in each container." }
}
},
"hardware": {
"type": "object",
"description": "Optional host HARDWARE CAPABILITY the workload needs, for the container kind (docs/13). This field declares INTENT (`what device class this service needs`), never MECHANISM (`which host path to bind-mount`, `which extended resource to request`). That separation is the whole point of the enum: the platform currently satisfies `intel-quicksync` with a /dev/dri hostPath volume and a matching mount on the app container, but a later migration to the Intel GPU device plugin (`gpu.intel.com/i915` resource requests, no hostPath) must be a catalog/render.py change and NOTHING else - no ServiceDefinition may need editing, and no reviewer may have to re-derive per-service device wiring. Do NOT 'simplify' this into a raw device path, a hostPath, or a resource request; that would push a platform implementation detail into the operator-facing contract and make the migration a catalog-wide edit. Absent => no device wiring at all (existing apps render byte-identical). Ignored for kind=vm (a VM gets devices through KubeVirt's own domain.devices).",
"additionalProperties": false,
"minProperties": 1,
"properties": {
"gpu": {
"type": "string",
"description": "GPU capability class this service needs. `intel-quicksync` = the host's Intel Quick Sync Video render device (the iGPU's /dev/dri/renderD* node) for hardware video transcode/decode. Enum, not a free-form device path, so the satisfying mechanism stays swappable (see the parent description). Extend the enum when a genuinely different capability class appears (e.g. nvidia-nvenc), never to name a different path to the same one.",
"enum": ["intel-quicksync"]
}
}
},
"exposure": {
"type": "string",
"description": "internal = LAN-only (internal DNS + Gateway on LAN); public = additionally edge port-forwarded 80/443 (docs/09; R-DNS/R-NET).",