home-network-k3s-impl/schemas/renderer-contract.md

# Renderer Contract — ServiceDefinition → rendered resources

**Status:** authoritative Phase-0 interface contract. **Consumed by** the manifest renderer
(WS-GITOPS), the network/egress renderer (WS-NET), storage (WS-STATE), TLS/DNS (WS-EDGE), identity
(WS-SEC), observability (WS-OBS), updates (WS-UPD), and the conformance/policy/CI agents (WS-TEST).

**Source of truth:** spec [`docs/13-service-catalog.md`](../../home-network-k3s/docs/13-service-catalog.md)
(the ServiceDefinition schema and default posture). This document maps each field of
[`servicedefinition.schema.json`](./servicedefinition.schema.json) to the concrete resources the
platform must render, and cites the requirement each mapping satisfies. Changing this contract is an
**interface change** — coordinate per the spec's `agents/coordination-protocol.md` (§2).

> The renderer MUST first materialize the schema **defaults** (below), then apply this mapping. A
> field absent from a ServiceDefinition renders from its default posture, not "not at all."

## Default posture (applied before mapping) — docs/13 §"Default posture"
| Field | Default | Requirement |
|---|---|---|
| `kind` | `container` | R-PLAT-02/03 |
| `exposure` | `internal` | R-DNS/R-NET |
| `egress` | `internet` | R-NET-05 |
| `storage` | `[]` (config PVC still expected for stateful svc) | R-STATE-02 |
| `dependencies` | `[]` → reaches **only DNS** (default-deny) | R-NET-01 |
| `inClusterClients` | `[]` → no direct east-west client; only the Gateway may reach an exposed port | R-NET-01/02 |
| `auth.mode` | `forward-auth` for any exposed UI | R-IDP-02 |
| `auth.apiRoutes` | `[]` → the sidecar renders **no** `--api-route`; every unauthenticated request is 302'd to Keycloak | R-IDP-02 |
| `update.channel` | `stable` | R-UPD-01 |
| `update.autoRollbackOnFailedHealth` | `true` | R-UPD-03 |
| `observability.metrics` | `true` | R-OBS-01 |
| `observability.metricsEndpoint` | **absent** → the workload serves no Prometheus endpoint; no ServiceMonitor | R-OBS-01 |
| `observability.livenessWorkload` | the object this SD renders (`Deployment` named after the service; `VirtualMachineInstance` for `kind: vm`) | R-OBS-01 |
| backup of irreplaceable state | off-cluster (never `reacquirable`) | R-STATE-03 |

---

## Field → resource mapping

### `name`
- **Renders:** the service Namespace (trust zone), Deployment/VM name, Service name, and the DNS label
  `<name>.home.keithjackson.dev`.
- **Requirements:** R-DNS-01 (naming model, docs/09), R-NET (namespace = isolation boundary, docs/03).

### `kind` + `source`
- `container` + `source.image` → **Deployment** (+ Service) using the image ref.
- `vm` + `source.vmTemplate` → **KubeVirt VirtualMachine** with a **CDI**-imported root disk.
- The schema enforces exactly-one-of (`image` for container, `vmTemplate` for vm).
- **Requirements:** R-PLAT-02/03 (PaaS/IaaS, docs/13); VM disks inherit Longhorn durability (docs/04).

### `exposure` (`internal` | `public`)
- Both classes render an **HTTPRoute** on the shared Gateway and are reachable on the LAN via internal
  split-horizon DNS.
- `public` **additionally** signals the documented **manual edge port-forward** of 80/443 (and any
  service-specific port, e.g. Plex 32400, Minecraft 25565) to the ingress VIP, and a **public**
  ExternalDNS record for `<name>.home.keithjackson.dev`.
- `internal` renders **no public DNS record** and is never port-forwarded (PiHole resolves it to the
  internal Gateway VIP).
- **HOW "no public DNS record" IS MADE TRUE, since it was prose only until 2026-09-14.** Every
  HTTPRoute rendered for an `internal` service carries ExternalDNS's own controller annotation with
  a value other than the one ExternalDNS claims:

      external-dns.alpha.kubernetes.io/controller: none

  ExternalDNS v0.21.0 (`source/annotations/processors.go`) skips any source object whose
  `controller` annotation is present and `!= "dns-controller"`, and `source/gateway.go` applies that
  check **inside `Endpoints()` before any hostname is resolved** — so the route never becomes an
  endpoint for any name it declares. `source/service.go` applies the identical check, which covers
  the second configured source. It is derived **from `exposure` and nothing else**
  (`catalog/render.py` `_externaldns_route_annotations`), so the operator's one declaration decides,
  a service that OMITS `exposure` inherits the `internal` default and is likewise unpublishable, and
  there is no per-service list to forget to extend. `ci/render-golden.sh` proves `apps/` reproduces
  from `catalog/`, so an `internal` service cannot exist in the applied tree without it, and
  `conformance/tests/l0-static/internal-services-publish-no-public-dns` re-proves both directions
  (present on internal, **absent** on public) on every gate run.
  - The route itself, the wildcard TLS and LAN reachability are **unchanged** — only the public
    record is withdrawn. The internal split-horizon answer is Pi-hole's job (R-DNS-03), which is why
    withdrawing these records requires Pi-hole to be answering the LAN authoritatively first: the
    zone has no `*.home.keithjackson.dev` wildcard, and `*.keithjackson.dev` does not match two
    labels, so a withdrawn name is a public NXDOMAIN with no fallback.
  - Why not a cluster-wide ExternalDNS opt-in filter (`labelFilter`, which v0.21.0 supports for
    both configured sources): seven published names come from hand-authored `platform/`/`apps/`
    manifests that carry no `exposure` field for a renderer to read, and an opt-in filter deletes
    all seven the moment it is set. Per-resource exclusion has exactly the blast radius of the
    defect. The filter stays available as hardening once those seven are adjudicated — see
    `platform/externaldns/helmrelease.yaml` and the register in the L0 check.
- `public` renders the record but the renderer does **not** set a target: external-dns reads
  `external-dns.alpha.kubernetes.io/target` only from the **Gateway** (and a ListenerSet), never from
  an HTTPRoute, so the target is set once, on the `shared` Gateway — since 2026-11-08 the Cloudflare
  Tunnel, proxied (`platform/gateway/gateway.yaml`, `platform/externaldns/README.md`).
- Exposure does **not** relax east-west default-deny (docs/09 §"Interaction with default-deny").
- **Requirements:** R-DNS-02/03 (docs/09), R-NET-04 (docs/03).

### `egress` (`internet` | `vpn`) — egress steering
- `internet` → pod egress allow to the default WAN route (node → SOHO edge → internet).
- `vpn` → the pod is **steered through the shared in-cluster Gluetun/PIA (WireGuard) egress gateway**
  (Cilium Egress Gateway / routing). The pod gets **no independent egress path**; the
  **kill-switch (fail-closed)** and allow rules are enforced at the gateway + namespace/policy level,
  **outside** the workload.
- **Requirements:** **R-NET-05** (egress class), **R-NET-06** (VPN kill-switch, mandatory & central).
  VPN-class set is normative: sonarr, radarr, jackett, qbittorrent (docs/03/13).

### `dependencies[]` (`to` / `port` / `protocol`) — east-west policy
- Each namespace first gets the **default-deny baseline** (deny all ingress + egress) applied by
  GitOps at namespace creation, with a standing allow only for **DNS to the in-cluster resolver**.
  **Observability scraping is NOT in the baseline** — it is an additive allow rendered per declared
  `observability.metricsEndpoint` (see below), scoped to that one port. This paragraph used to claim
  the baseline carried a standing monitoring allow; it never did, and an app metrics port would have
  been dropped by policy even with the Service port and monitor endpoint in agreement.
- Each `dependencies[]` entry renders one **CiliumNetworkPolicy** egress allow: this service → `to`
  service on `port`/`protocol`. Everything undeclared stays denied.
- Empty `dependencies` ⇒ the service can reach **only DNS**.
- **Requirements:** **R-NET-01** (default-deny baseline), **R-NET-02** (explicit allow / deny
  undeclared). Auditable via Hubble (R-SEC-03, R-OBS-04).

### `storage[]` (`name` / `size` / `class` / `mountPath` / `export` / `subPath` / `readOnly`) — PVs + PVCs
- `class: longhorn` → a **PVC** on the 3-replica Longhorn StorageClass sized by `size` (replicated
  config/DB/app state), `ReadWriteOnce`, dynamically provisioned. For a `vm`, the disk is a Longhorn
  volume via CDI.
- `class: bulk` → a **static, pre-bound external NFS PersistentVolume _and_ its PVC**. The entry's
  `export` is the absolute path on the bulk server; the SERVER is never named in a ServiceDefinition
  — it is the `${BULK_NFS_SERVER}` cluster-vars knob, so one authored definition promotes across
  environments that reach the same storage by different addresses (**R-ENV-02**). Not 3× replicated:
  durability is the storage host's ZFS pool, not Longhorn.
  - `storageClassName: ""` on **both** PV and PVC (explicitly empty ⇒ statically provisioned; no
    provisioner is ever asked to create anything, and the default-StorageClass annotation is
    suppressed). Deliberately NOT the `bulk` StorageClass: a StorageClass `provisioner` is
    **immutable**, so re-pointing that stub would make Flux fail permanently on it.
  - Binding is pinned from both ends — `claimRef` on the PV, `volumeName` on the PVC — so no other
    claim of matching size/mode can win a race for the volume.
  - `persistentVolumeReclaimPolicy: Retain`, always. Bulk data pre-exists the cluster, is far larger
    than the cluster, and may have consumers outside it; nothing here may reclaim it.
  - Access mode follows the read/write posture and is always `…Many`, because one export is
    consumed by several namespaces and several nodes at once: `ReadOnlyMany` when `readOnly`,
    `ReadWriteMany` otherwise. `ReadWriteOnce` cannot express either.
  - `mountOptions` are a **platform** decision held in the renderer, not a per-service field:
    `nfsvers=4.1,hard,noatime,timeo=600,retrans=2`. NFSv4 carries mount and lock in-protocol, so
    the only port that must be reachable is **tcp/2049** — no rpcbind/111, mountd or statd, which is
    what keeps the firewall surface to a single rule. `hard` is chosen over `soft` because a soft
    mount converts a server hiccup into a short read handed to the application.
  - `size` on a bulk entry is the PV's **advertised capacity and nothing more** — no component
    enforces it (the ZFS pool is the real limit), so it should state the dataset's actual size so
    `kubectl get pv` reports something true.
  - **fsGroup does not apply.** The in-tree NFS volume plugin performs no ownership management, so
    `podSecurityContext.fsGroup` is a **no-op** on a bulk volume. A bulk mount that must be written
    therefore requires the container's own uid/gid to be one the export's ownership permits (e.g.
    Plex's `PLEX_UID`/`PLEX_GID`, LinuxServer images' `PUID`/`PGID`) — or a server-side squash. A
    mount that succeeds but cannot write is the characteristic failure of this tier.
- `mountPath` (any class) overrides the `/data/<name>` default verbatim.
- `subPath` (any class) mounts a subtree instead of the volume root. **Two `subPath` mounts of one
  export are two separate mounts**, so `link()` between them fails `EXDEV`; a workload that needs
  hardlinks or atomic moves between two directories (Sonarr/Radarr importing from a download path
  into a library) must mount their common ancestor **once** instead.
- `readOnly` (any class) marks the volumeMount read-only, and on a bulk entry also sets the PV's
  `nfs.readOnly` so the kernel mount really is `ro`. Note this is a **client-side** guarantee: one
  export serving both a reader and a writer that may land on any node cannot distinguish them
  server-side.
- State separation rule: anything that must survive a rebuild lives on a named volume here, never on
  ephemeral fs/hostPath — so the workload can be destroyed and re-applied from git with data intact.
- **Requirements:** **R-STATE-01** (rebuild survival), **R-STATE-02** (Longhorn replication),
  **R-STATE-04** (bulk tier). (docs/04)

### `availability` (`replicas`) — workload kind, per-replica volumes, spreading, budget

**Schema 1.10.0.** Availability INTENT. It names no Kubernetes object, no topology key and no
budget — exactly like `hardware.gpu`, and for the same reason: the mechanism below should be
replaceable without a catalog edit.

- **Absent, or `replicas: 1`** → a **Deployment**, standalone PVC(s), `strategy: Recreate` when there
  is storage, no PodDisruptionBudget. Byte-identical to pre-1.10.0 output, so every existing app is
  untouched.
- **`replicas: N > 1`** → all of the following, together:
  | Rendered | Why |
  |---|---|
  | **StatefulSet** instead of Deployment | A longhorn PVC is **ReadWriteOnce**. One RWO claim cannot attach on two nodes, so a multi-replica Deployment does not degrade — the second pod never schedules. |
  | longhorn storage becomes **`volumeClaimTemplates`** | Gives each replica its own claim (`<vol>-<svc>-<ordinal>`, e.g. `config-pihole-0`). The template's `metadata.name` **must** equal the pod volume name the container mounts. The standalone PVC is then **not** emitted — it would be an orphan nothing mounts. |
  | `bulk` storage stays a **standalone PVC** | A bulk entry is a static, pre-bound NFS PV over data that already exists outside the cluster and is shared read-only. Per-replica copies are neither possible nor wanted. |
  | **required** `podAntiAffinity` on `kubernetes.io/hostname` | `preferred` would let every replica land on one node and deliver none of the intended node-failure tolerance, while every object still reported healthy. |
  | `podManagementPolicy: Parallel` | `OrderedReady` serialises first-boot work across replicas; `apps-wave-0` has `timeout: 5m`. There is no ordering requirement between replicas to protect. |
  | **PodDisruptionBudget**, `minAvailable: 1` | Stops a *voluntary* disruption (node drain, upgrade) taking the last replica. It does nothing about a node dying — that is what the replica count and anti-affinity are for. Never rendered at `replicas: 1`, where `minAvailable: 1` would block every drain of that node forever. |
  | `observability.livenessWorkload.kind` defaults to **`StatefulSet`** | The `up` SLI reads kube-state-metrics by workload kind. A Deployment-shaped rule against a StatefulSet `absent()`s forever — an alert that can never fire. Deriving the default means bumping the replica count cannot leave a stale rule behind. |

**Two costs, both deliberate, neither hidden:**

1. **N replicas need N schedulable nodes.** On a 3-node cluster, losing two nodes leaves a replica
   `Pending`; since `apps-wave-0` is `wait: true` and `apps-wave-1` `dependsOn` it, that **stalls the
   app tier**. That coincides with having lost etcd quorum on a 3-node control plane, so nothing is
   reconciling at that point anyway — the stall is a symptom, not a new failure mode.
2. **Only declare `> 1` for a service whose on-disk state is reproducible from git.** A replica with
   a fresh volume comes up serving whatever it can bootstrap itself and nothing else. For pihole that
   prerequisite was done first — the admin password and the adlists were declared in
   `decisions/2026-10-04-pihole-declare-state.md` — because otherwise a second replica would have
   served DNS while blocking nothing, with a different password.

**Requirements:** R-PLAT-02 (the workload), R-STATE-01/02 (per-replica replicated volumes),
R-OBS-01 (the SLI must follow the rendered kind).

### `hardware` (`gpu`) — host device capability
- **The field is INTENT, never MECHANISM.** `hardware.gpu` is an **enum of capability classes**
  (today: `intel-quicksync`), not a device path, not a hostPath, not a resource request. The
  ServiceDefinition says *what class of host device this workload needs*; how the platform hands it
  over is `catalog/render.py`'s business and lives nowhere else. **Do not "simplify" this into a raw
  `/dev/...` path.** The payoff is no longer hypothetical: on **2026-10-01** the mechanism was
  swapped outright — from a `/dev/dri` hostPath to the Intel GPU device plugin's extended resource —
  as a **`catalog/render.py` change plus a re-render, with ZERO catalog edits** and no re-review of
  per-service device wiring. `catalog/plex.yaml`'s `hardware:` block came through byte-identical.
  Extend the enum for a genuinely different capability class (e.g. `nvidia-nvenc`); never to name a
  second route to the same one.
- `gpu: intel-quicksync` → on the **app container only** (never a sidecar), **exactly one thing**:

      resources:
        limits:
          gpu.intel.com/i915: 1

  **That is the whole mapping.** No volume, no volumeMount, no `securityContext`, no `nodeSelector`,
  no privileged anything.
  - **`limits`, not `requests`, and that is not a style choice.** An extended resource must be
    expressed in `limits`; Kubernetes mirrors `requests` from `limits` automatically, and a
    requests-only extended resource is rejected as invalid.
  - The resource is advertised by the **`intel-gpu-plugin` DaemonSet** (`platform/intel-gpu-plugin/`,
    its own Flux Kustomization at `clusters/base/intel-gpu-plugin.yaml`, pinned
    `intel/intel-gpu-plugin:0.37.0`). **Measured 2026-10-01:** all three prod nodes report
    `.status.allocatable["gpu.intel.com/i915"] == 1`; DaemonSet 3 desired / 3 ready / 3 available.
  - This is the **only** app-container `resources` render path in the file, and it is deliberately
    narrow. **Do not generalise it into a CPU/memory resources feature** while passing through — that
    is a separate field with a separate justification, and `render.py --selftest` asserts the emitted
    dict *exactly* so the attempt fails loudly.
- **WHY THE hostPath MECHANISM WAS ABANDONED — read this before "simplifying" it back.** The first
  implementation (merged 2026-09-30) rendered a `dri` hostPath volume at `/dev/dri` plus a read-write
  mount. It was **measured not to work** in the live Plex pod on k3s-prod-2 on **2026-10-01**. The
  device node was **visible** in the container — `crw-rw---- root video2 226,128 renderD128` — and
  **could not be opened**:

  | attempt | result |
  |---|---|
  | `uid 0`, `O_RDWR` | **`EPERM`** "Operation not permitted" |
  | `uid 1000`, `O_RDWR` | **`EPERM`** "Operation not permitted" |

  `EPERM` is the **device cgroup** refusing, not `EACCES` from file permissions — the two look
  identical from the outside and are not the same problem. Proven three ways: root on the **host**
  opens the same device `rc=0`; the node is unified `cgroup2fs`; and the container's runc
  `config.json` reads `linux.resources.devices: [{allow: False, access: 'rwm'}]` with
  `linux.devices: None` — **one deny-all rule, no allow entry for major 226**. A hostPath is a
  *filesystem* operation: it binds the device node in but adds **no cgroup allow rule**, because
  Kubernetes cannot infer one from a volume. **Note that `uid 0` failed too** — root does not bypass
  the cgroup device gate, so "run it as root" is not a workaround. Neither is `privileged: true`:
  Kyverno `restrict-host-namespaces` is **Enforce** on app namespaces and rejects it.
  The device plugin is what produces the allow rule — confirmed in the v0.37.0 source,
  `createDeviceSpecsFromDrmFiles` emits `DeviceSpec{…, Permissions: "rw"}`, which the kubelet turns
  into both the device injection and the matching cgroup allow.
- **THE FAILURE MODE THIS TRADED INTO IS WORSE IN BLAST RADIUS, AND IT IS UNAVOIDABLE.** Stated
  plainly rather than sold:

  | | device/resource unavailable ⇒ |
  |---|---|
  | **Before** (hostPath, `DirectoryOrCreate`) | kubelet mounts an empty dir; Plex runs and **silently software-transcodes**. Invisible, but harmless to everything else. |
  | **Now** (extended resource) | the pod is **Unschedulable**. `apps-wave-0` is `wait: true` and `apps-wave-1` `dependsOn` it, so this **stalls the app tier**. |

  There is **no graceful-degradation variant**: an extended resource is a hard scheduling
  constraint, and it is also the only thing that produces the cgroup allow rule. The workload's
  schedulability is now **coupled to the `intel-gpu-plugin` DaemonSet** — a plugin that is broken,
  removed, or merely not yet reconciled takes the app tier with it. That is the price of hardware
  transcoding working at all.
  - **Escape hatch, one field and no code change:** delete the `hardware:` block from the
    ServiceDefinition (`catalog/plex.yaml`) and re-render. The workload returns to software
    transcoding and the tier unblocks. `runbooks/plex-quicksync.md` carries it as a first-class
    procedure, because it is what someone will need at 2am.
  - The old mechanism's cost is **gone**, and that is the one improvement: a missing device is no
    longer silent. There is still no metric and no alert (media apps here render no
    `metricsEndpoint`, so no ServiceMonitor exists), but "Plex is Unschedulable" is loud, whereas
    "Plex is quietly software-transcoding" never was.
- **`podSecurityContext.supplementalGroups` is deliberately NOT rendered, and adding it would be a
  regression rather than a hardening.** This survives the mechanism swap unchanged, and it was
  re-measured against the plugin's behaviour. `renderD128` is `root:render` `0660`, so the obvious
  move is to push the host `render` GID in — and it does not work. **Measured in the running
  container on prod-2 (2026-09-30, re-measured 2026-10-01):** the image's own
  `/etc/cont-init.d/45-plex-hw-transcode-and-connected-tuner` hook `find`s every char device under
  `/dev/dri`, `stat -c '%g'` each one's GID, `groupadd`s a matching group *inside the container* when
  none exists, and `usermod -a -G` the app user into it — every start, nothing hardcoded. It does
  exactly the same thing to a **plugin-injected** device as to a bind-mounted one, which is why this
  swap needed no group work. `cont-init.d` completes before `services.d`, so the account database is
  already correct when the service execs. The service then starts via `exec s6-setuidgid plex …`, and
  `s6-setuidgid` **replaces** the process group list from `/etc/group` rather than appending to it —
  measured live, `Plex Media Server` ran with `Groups: 44 100 992 1000`, where **992 is the host
  `render` GID**, while root's gid `0` (which PID 1, `s6-svscan`, does hold) was **gone**. A
  Kubernetes `supplementalGroups` therefore reaches `s6-svscan` and is discarded before the app ever
  starts. It would be inert *and* it would mislead whoever debugs this next. The renderer asserts its
  own absence in `render.py --selftest`.
  - A corollary worth having: because the hook re-`stat`s the device on every start, the host GID
    **drifting** (a node rebuild renumbering the *systemd dynamic* `render` group) is self-healing.
    No GID is pinned anywhere in this repo.
  - **Scope of that measurement, stated so nobody over-reads it.** "The group is the image's job" was
    established for an image that resolves the device group itself. A FUTURE consumer that runs
    non-root and has no equivalent init hook would still need the group solved somehow — and the
    answer for that case is a **renderer** change (per-image handling), not a new ServiceDefinition
    field. The enum is what keeps that option open, and the 2026-10-01 swap is the proof it works.
    The first such consumer must re-measure rather than assume this bullet covers it.
- **Deliberately NOT rendered** from this field: no `privileged: true` (Kyverno
  `restrict-host-namespaces` is Enforce on app namespaces and would reject it, and it would not have
  helped anyway — see the `EPERM` table); no `nodeSelector` (placement is a separate concern with its
  own justification — this field is about the device, not about where the pod lands; the extended
  resource does constrain placement to nodes the plugin advertises, but that is the **scheduler's**
  doing, not a selector this renderer writes); no CPU/memory requests or limits (see above — the
  `resources` block carries the extended resource and nothing else).
- **Historical, kept so a reader recognises it rather than reintroduces it:**
  `type: DirectoryOrCreate` on the retired hostPath was load-bearing *for that mechanism* — it is
  what made a GPU-less node degrade to software transcoding instead of failing the mount and
  stalling the app tier. That rationale died with the hostPath; it is **not** an argument for bringing it back, since
  the mechanism it protected never granted access in the first place.
- Absent ⇒ **nothing at all** is rendered; every service that does not opt in stays byte-identical
  (no `resources` key, no volume, no mount, no `securityContext`).
  Ignored for `kind: vm` — a VM receives devices through KubeVirt's own `domain.devices`.
- **Requirements:** **R-PLAT-02** (the PaaS provides the host capability; the app declares need).

### `ingressRoutes[]` (`host` / `port`) — HTTPRoute + Certificate + DNS
- Each entry renders an **HTTPRoute** attaching `host` → the service `port` on the shared Gateway.
- TLS: the Gateway terminates with the cert-manager **wildcard `*.home.keithjackson.dev`** certificate
  (Let's Encrypt via **DNS-01**, scoped Cloudflare token), so no per-service cert step — trusted,
  non-self-signed, auto-renewed, including for `internal` services. Per-service `Certificate` only
  where isolation warrants.
- DNS: **ExternalDNS** watches the HTTPRoute and manages the `<host>` record — but **only for a
  `public` service**. An `internal` service's route is rendered with ExternalDNS's controller
  annotation set to `none`, which removes it from ExternalDNS's view entirely (see §`exposure`), so
  its `<host>` exists **only** in the internal split-horizon answer PiHole serves, never in the
  public zone. A `public` record's TARGET is the Cloudflare Tunnel (a proxied CNAME, from the
  shared Gateway's target annotation; `platform/externaldns/README.md`), and a `public` name is
  not listed in PiHole unless it must stay on the LAN path.
- The `host` pattern is enforced to `<label>.home.keithjackson.dev` by the schema.
- **Requirements:** **R-TLS-01/02/03** (docs/08), **R-DNS-01/02/03** (docs/09).

### `inClusterClients[]` (`namespace` / `port` / `protocol` / `name`) — direct east-west clients
- Each entry renders ONE **CiliumNetworkPolicy** ingress allow on this service:
  `endpointSelector` = this service, `fromEndpoints` matching **only** the reserved
  `k8s:io.kubernetes.pod.namespace: <namespace>` label, `toPorts` carrying **only** the declared
  `port`/`protocol`. Name: `<service>-allow-<name>-ingress`, `name` defaulting to `namespace`.
- **Why the field exists.** `ingressRoutes` opens the **north-south** path only (the gateway allow
  admits `fromEntities: [ingress]`, the Gateway proxy identity, and nothing else), and
  `dependencies` compiles edges **only between catalog services** — both halves come from a catalog
  entry's own declaration. A **platform controller** that dials this service's ClusterIP directly
  was therefore unauthorable, and under default-deny it is dropped. Prod hit exactly that: its
  `GitRepository` points at the in-cluster Gitea Service
  (`http://gitea.gitea.svc.cluster.local:3000/...`, clusters/prod/kustomization.yaml) so GitOps
  depends on no public DNS/Cloudflare/Gateway/`${BASE_DOMAIN}`, but `flux-system`'s
  source-controller was denied — and the HTTPRoute that would have served a public name lives in a
  **downstream** tier that cannot apply while the source is down (source down → routes never rename
  → source stays down).
- **Least privilege is part of the contract:** a namespace and a port, never `fromEntities: all`,
  never a port-less allow. The client is selected by **namespace** rather than pod labels because an
  upstream controller's labels are its chart's to change, not this platform's contract — so name a
  platform-controller namespace whose entire membership may hold that reach, never a shared one.
- **Additive, and not a dependency:** the namespace default-deny baseline is unchanged
  (`ingress: []`), every undeclared source/port stays denied, and the entry creates **no wave
  ordering edge** — the client is a platform component, not a catalog service.
- **Requirements:** **R-NET-01** (default-deny baseline stands), **R-NET-02** (explicit allow / deny
  undeclared), **R-GITOPS-02** (the allowance is authored in the one reviewable object rather than
  surviving as a live-applied cluster object). Auditable via Hubble (R-SEC-03, R-OBS-04).

### `auth` (`mode` / `groups[]` / `apiRoutes[]`) — identity integration (Keycloak / oauth2-proxy)
- `oidc` → register the app as a **Keycloak OIDC client** (declarative realm/client config, GitOps;
  client secret SOPS-encrypted). App does its own login redirect.
- `forward-auth` → protect the service's `ingressRoutes` at the **Gateway** with **oauth2-proxy →
  Keycloak** so auth happens before traffic reaches the app (bare UIs without usable OIDC).
- `native` → no platform SSO wiring; app owns its auth (documented exception, e.g. Plex/Seerr; and
  Keycloak **itself** as the IdP, admin login local + MFA).
- `none` → no auth wiring (internal-only, discouraged).
- `groups[]` → the Keycloak groups permitted access, enforced by the OIDC client mappers /
  oauth2-proxy `allowed_groups`. **Schema requires ≥1 group for `oidc`/`forward-auth`** — access is
  "in an authorized group," not merely "logged in."
- `apiRoutes[]` (**`forward-auth` only**) → one repeatable **`--api-route=<regex>`** per entry on
  the per-app oauth2-proxy sidecar, making matching paths answer **401** instead of a **302** to
  Keycloak. Default `[]` renders **no flag at all**, which is the common case. It exists because
  oauth2-proxy's built-in AJAX guard fires only on an exact `Accept: application/json`, which
  these apps' JavaScript does not consistently send: the XHR is 302'd, the browser follows it
  invisibly, the live SSO session satisfies it silently, and the call resolves 200 carrying the
  app's **HTML** where the JavaScript expected JSON — and a POST following a 302 is re-issued as
  a GET with its **body dropped**. It is **per-service and must never become a renderer
  constant**: the path shape belongs to the app (radarr `/signalr/`, jackett `/api/`, pihole
  none), and a wrong constant does not merely fail to help — a path a **browser navigates to**,
  answered 401, shows a logged-out operator a bare error page and never offers a sign-in. Each
  entry must be **anchored at `^/`** and continue with literal path characters; `^/`, `^/.*`,
  `.*`, an unanchored `/api/` and any alternation are **rejected** by the schema and by the
  renderer (alternation is refused rather than supported because the flag is repeatable).
- Keycloak, oauth2-proxy, and app→Keycloak / proxy→Keycloak / Keycloak→DB flows are ordinary
  workloads under default-deny with explicit allows (ties back to `dependencies`).
- **Requirements:** **R-IDP-02** (protect OIDC-capable and bare UIs), **R-IDP-06** (group-based
  authorization). (docs/15, ADR-0014)

### `update` (`channel` / `range` / `cadence` / `autoRollbackOnFailedHealth` / `pin`)
- container `kind` → a Flux **ImagePolicy** (+ ImageRepository) encoding `channel`/`range`; Flux
  image-automation advances the running version on its scan interval (`cadence`). `pin` freezes the
  service (stops automation). No human/AI edits a tag.
- `vm` `kind` → in-guest unattended package updates on `cadence` (cloud-init), per docs/11 §3.
- `autoRollbackOnFailedHealth: true` → health-gated rollout that reverts on failed health and alerts.
- **Requirements:** **R-UPD-01** (stable only), **R-UPD-02** (no human/AI authoring), **R-UPD-03**
  (health-gated auto-rollback). (docs/11, ADR-0009)

### `observability` (`metrics` / `slis[]` / `metricsEndpoint` / `livenessWorkload`) — ServiceMonitor + alerts
- `metrics: true` → the platform **collects signals** for this service: logs to Loki via the node
  agent, and kube-state-metrics readiness for the `up` SLI below. Both are platform-wide scrapes;
  neither needs a per-service CR. **It does not assert that the workload serves a Prometheus
  endpoint.** *(Corrected 2026-09-13. It used to render a ServiceMonitor unconditionally, hardcoded
  to a Service port named `metrics`. No rendered Service ever defined such a port, so all twelve app
  monitors discovered ZERO targets — verified on prod: 24 jobs in `up`, not one of the twelve — and
  every `up{job=...}` SLI built on them was structurally inert. `metrics: true` could not have been
  right: Plex, Pi-hole, Jackett, qBittorrent, Sonarr, Radarr, Seerr, Semaphore and Nextcloud serve no
  Prometheus endpoint at all without a separate exporter.)*
- `metricsEndpoint: {port, name?, path?, interval?}` → **the workload does serve metrics, here.**
  Renders all three halves of a working scrape from the one declaration, because any one of them
  missing makes the scrape silently yield nothing:
  1. a **named Service port** (`name`, default `metrics`) on that `port`;
  2. a **ServiceMonitor** endpoint selecting that port name, on `path` at `interval`;
  3. a **CiliumNetworkPolicy** admitting the `monitoring` namespace to that port — the namespace
     default-deny baseline is `ingress: []`, so without it the scrape is dropped by policy.
  The metrics port is deliberately **not** added to the Gateway ingress allow: it is east-west for
  Prometheus, never north-south.
- each entry in `slis[]` → at least one **alert rule** routed by Alertmanager to email; a per-service
  Grafana dashboard row. `up` → a `<name>-down` **critical** alert reading the readiness of
  `livenessWorkload` from **kube-state-metrics** (`kube_deployment_status_replicas_available`,
  `kube_statefulset_status_replicas_ready`, `kube_daemonset_status_number_available`) or, for a
  `VirtualMachineInstance`, the existence of `kubevirt_vmi_info{phase="running"}`. Each rule carries
  an `or absent(...)` half, so the signal DISAPPEARING fires the alert rather than silencing it.
- `livenessWorkload: {kind, name}` → which object's readiness **is** this service. Defaults to the
  object this SD renders; override it when the running workload is a different one (keycloak's server
  is a StatefulSet owned by the Keycloak operator — a Deployment-shaped rule would `absent()` and
  fire forever).
- **Requirements:** **R-OBS-01** (per-service SLIs/alerts). Platform-health alerts (R-OBS-04) are
  rendered platform-wide, not per ServiceDefinition. (docs/10)

### `backup` (`policy`)
- a schedule ref (e.g. `daily-offcluster`) → **Velero + Longhorn** off-cluster backup schedule for
  this service's volumes/namespace.
- literal `reacquirable` → **no** 1:1 backup (bulk media only). **Irreplaceable** state (configs, DBs,
  Nextcloud, Gitea, PiHole config, Keycloak DB) MUST NOT be `reacquirable`.
- **Requirements:** **R-STATE-03** (off-cluster DR backup). (docs/04)

---

## Cross-field invariants the renderer / L0-L1 tests rely on
1. **kind ↔ source:** container⇒`image`, vm⇒`vmTemplate` (schema-enforced).
2. **auth ↔ groups:** `oidc`/`forward-auth` ⇒ non-empty `groups` (schema-enforced; R-IDP-06).
2a. **auth ↔ apiRoutes:** a non-empty `auth.apiRoutes` ⇒ `auth.mode: forward-auth`
   (schema-enforced). It renders onto the per-app oauth2-proxy sidecar, and no sidecar exists for
   `oidc`/`native`/`none`, so declaring it elsewhere would be a **silent no-op** — the operator
   believes the app's XHRs now answer 401 and nothing changed. Every entry must be anchored at
   `^/` (schema-enforced **and** enforced in `catalog/render.py`, so a breadth mistake is caught by
   `render-golden` even on a run where the schema stage did not execute).
3. **default-deny is unconditional:** every namespace gets the deny baseline regardless of
   `dependencies`; `dependencies` only *adds* allows (R-NET-01/02). The same holds for
   `inClusterClients`: it adds one namespace+port ingress allow and never relaxes the baseline.
4. **egress is enforced outside the workload:** `vpn` renders gateway steering + kill-switch, never a
   per-service sidecar the app could bypass (R-NET-05/06).
5. **TLS is never self-signed and needs no per-service step:** the wildcard Gateway cert covers every
   `ingressRoutes.host`, `internal` included (R-TLS-01/03).
6. **irreplaceable ⇒ off-cluster backup:** DR-critical services must not carry `backup.policy:
   reacquirable` (R-STATE-03). *(Advisory: not enforced by the schema — an L0 policy check should
   assert it; see "Ambiguities / escalations" in schemas/README.md.)*
7. **`exposure: internal` ⇒ no public DNS record, structurally.** Every HTTPRoute rendered for an
   `internal` service (including one that omits `exposure` and takes the default) carries
   `external-dns.alpha.kubernetes.io/controller: none`, and no `public` service's route carries it.
   Derived from `exposure` alone; enforced on every gate run by
   `conformance/tests/l0-static/internal-services-publish-no-public-dns` (R-DNS-03).
8. **`hardware` is inert until opted into, and its mechanism is renderer-local — now DEMONSTRATED,
   not merely claimed.** An SD with no `hardware` key renders no `resources` block, no volume and no
   volumeMount; an SD *with* it gains a `gpu.intel.com/i915: 1` **limit on the app container only**
   (never a sidecar) and no `securityContext` either. All directions are proven by
   `render.py --selftest`, and each assertion is mutation-tested. The resource name
   `gpu.intel.com/i915` appears in `catalog/render.py` and nowhere else in the catalog path, so the
   capability's implementation can be replaced without touching a single ServiceDefinition — which
   is exactly what happened on **2026-10-01**, when the mechanism moved from a `/dev/dri` hostPath
   (measured non-functional: device-cgroup `EPERM`) to the device plugin's extended resource with
   **zero** catalog edits. The cost of that mechanism is recorded in §`hardware`: the workload is now
   **Unschedulable** when the resource is absent, which stalls the app tier, and the escape hatch is
   to delete the `hardware:` block and re-render.

## Requirement traceability (contract-level)
Primary **R-GITOPS-02** (single reviewable object wires every requirement). Field-level: R-NET-01/02
(dependencies, inClusterClients), R-NET-05/06 (egress), R-STATE-01/02/04 (storage), R-STATE-03 (backup),
R-TLS-01/02/03 + R-DNS-01/02/03 (ingressRoutes + exposure), R-OBS-01 (observability),
R-UPD-01/02/03 (update), R-IDP-02/06 (auth), R-PLAT-02/03 (kind/source).