A ServiceDefinition could not say it needs the host's Intel QuickSync render
device. The schema is `additionalProperties: false`, so there was no authorable
form for it at all, and `apps/` is generated — a hand-edit would be reverted by
the next render.
Adds optional top-level `hardware`, with one optional member `gpu`, an enum
whose only current value is `intel-quicksync` (schema 1.7.0 -> 1.8.0).
The enum is the point. The SD declares WHAT CLASS of device it needs; the
hostPath, the mount and the GID live only in catalog/render.py. Migrating to
the Intel GPU device plugin (`gpu.intel.com/i915` resource requests, no
hostPath) is then a renderer change with zero catalog edits. Letting an SD name
`/dev/dri` directly would look simpler and make that migration a catalog-wide
edit, so both the schema description and the renderer contract say so
explicitly.
When `hardware.gpu == intel-quicksync`, r_deployment emits, on the app
container / pod spec only:
- a hostPath volume `dri` -> /dev/dri, type DirectoryOrCreate;
- a read-write volumeMount of it at /dev/dri (VA-API opens the render node
read-write, so deliberately not readOnly);
- INTEL_RENDER_GID MERGED into podSecurityContext.supplementalGroups, never
clobbering what the SD already declares.
`DirectoryOrCreate` is load-bearing. A node that comes back without the render
device (rebuild, BIOS change, driver regression, hardware swap) must degrade
the workload to software transcoding, not fail the mount: apps-wave-0 is
`wait: true` and apps-wave-1 dependsOn it, so a failed hostPath mount stalls
the whole app tier for a feature that is not load-bearing. /dev is tmpfs, so
the empty directory the kubelet creates leaves nothing durable behind. Its cost
— a missing device is silent, and no metric exists to alert on — is recorded in
the contract rather than papered over with an alert that could not fire.
Deliberately NOT emitted: no `privileged: true` (Kyverno restrict-host-
namespaces is Enforce and it is not needed), no nodeSelector (placement is a
separate concern with its own justification), no resource requests (there is no
app-container resource render path).
INTEL_RENDER_GID = 992 is a single named constant carrying its provenance:
measured on all three prod nodes on 2026-09-30, and a *systemd dynamic*
allocation rather than a static GID, so a node rebuild can move it.
No catalog entry uses the field. Re-rendering `--all catalog -o apps` is a zero
diff, which is the proof the change is inert until opted into.
Three self-test cases in the existing check(...) style: the positive, the
negative (an SD without `hardware` gains no volume, no mount, no
securityContext — mirroring the podSecurityContext negative) and the merge case
([44] + 992 -> [44, 992], fsGroup preserved). Note that NO gate stage runs
`render.py --selftest`; they were run by hand and the gap is recorded in the
handoff.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
339 lines
26 KiB
Markdown
339 lines
26 KiB
Markdown
# Renderer Contract — ServiceDefinition → rendered resources
|
||
|
||
**Status:** authoritative Phase-0 interface contract. **Consumed by** the manifest renderer
|
||
(WS-GITOPS), the network/egress renderer (WS-NET), storage (WS-STATE), TLS/DNS (WS-EDGE), identity
|
||
(WS-SEC), observability (WS-OBS), updates (WS-UPD), and the conformance/policy/CI agents (WS-TEST).
|
||
|
||
**Source of truth:** spec [`docs/13-service-catalog.md`](../../home-network-k3s/docs/13-service-catalog.md)
|
||
(the ServiceDefinition schema and default posture). This document maps each field of
|
||
[`servicedefinition.schema.json`](./servicedefinition.schema.json) to the concrete resources the
|
||
platform must render, and cites the requirement each mapping satisfies. Changing this contract is an
|
||
**interface change** — coordinate per the spec's `agents/coordination-protocol.md` (§2).
|
||
|
||
> The renderer MUST first materialize the schema **defaults** (below), then apply this mapping. A
|
||
> field absent from a ServiceDefinition renders from its default posture, not "not at all."
|
||
|
||
## Default posture (applied before mapping) — docs/13 §"Default posture"
|
||
| Field | Default | Requirement |
|
||
|---|---|---|
|
||
| `kind` | `container` | R-PLAT-02/03 |
|
||
| `exposure` | `internal` | R-DNS/R-NET |
|
||
| `egress` | `internet` | R-NET-05 |
|
||
| `storage` | `[]` (config PVC still expected for stateful svc) | R-STATE-02 |
|
||
| `dependencies` | `[]` → reaches **only DNS** (default-deny) | R-NET-01 |
|
||
| `inClusterClients` | `[]` → no direct east-west client; only the Gateway may reach an exposed port | R-NET-01/02 |
|
||
| `auth.mode` | `forward-auth` for any exposed UI | R-IDP-02 |
|
||
| `update.channel` | `stable` | R-UPD-01 |
|
||
| `update.autoRollbackOnFailedHealth` | `true` | R-UPD-03 |
|
||
| `observability.metrics` | `true` | R-OBS-01 |
|
||
| `observability.metricsEndpoint` | **absent** → the workload serves no Prometheus endpoint; no ServiceMonitor | R-OBS-01 |
|
||
| `observability.livenessWorkload` | the object this SD renders (`Deployment` named after the service; `VirtualMachineInstance` for `kind: vm`) | R-OBS-01 |
|
||
| backup of irreplaceable state | off-cluster (never `reacquirable`) | R-STATE-03 |
|
||
|
||
---
|
||
|
||
## Field → resource mapping
|
||
|
||
### `name`
|
||
- **Renders:** the service Namespace (trust zone), Deployment/VM name, Service name, and the DNS label
|
||
`<name>.home.keithjackson.dev`.
|
||
- **Requirements:** R-DNS-01 (naming model, docs/09), R-NET (namespace = isolation boundary, docs/03).
|
||
|
||
### `kind` + `source`
|
||
- `container` + `source.image` → **Deployment** (+ Service) using the image ref.
|
||
- `vm` + `source.vmTemplate` → **KubeVirt VirtualMachine** with a **CDI**-imported root disk.
|
||
- The schema enforces exactly-one-of (`image` for container, `vmTemplate` for vm).
|
||
- **Requirements:** R-PLAT-02/03 (PaaS/IaaS, docs/13); VM disks inherit Longhorn durability (docs/04).
|
||
|
||
### `exposure` (`internal` | `public`)
|
||
- Both classes render an **HTTPRoute** on the shared Gateway and are reachable on the LAN via internal
|
||
split-horizon DNS.
|
||
- `public` **additionally** signals the documented **manual edge port-forward** of 80/443 (and any
|
||
service-specific port, e.g. Plex 32400, Minecraft 25565) to the ingress VIP, and a **public**
|
||
ExternalDNS record for `<name>.home.keithjackson.dev`.
|
||
- `internal` renders **no public DNS record** and is never port-forwarded (PiHole resolves it to the
|
||
internal Gateway VIP).
|
||
- **HOW "no public DNS record" IS MADE TRUE, since it was prose only until 2026-09-14.** Every
|
||
HTTPRoute rendered for an `internal` service carries ExternalDNS's own controller annotation with
|
||
a value other than the one ExternalDNS claims:
|
||
|
||
external-dns.alpha.kubernetes.io/controller: none
|
||
|
||
ExternalDNS v0.21.0 (`source/annotations/processors.go`) skips any source object whose
|
||
`controller` annotation is present and `!= "dns-controller"`, and `source/gateway.go` applies that
|
||
check **inside `Endpoints()` before any hostname is resolved** — so the route never becomes an
|
||
endpoint for any name it declares. `source/service.go` applies the identical check, which covers
|
||
the second configured source. It is derived **from `exposure` and nothing else**
|
||
(`catalog/render.py` `_externaldns_route_annotations`), so the operator's one declaration decides,
|
||
a service that OMITS `exposure` inherits the `internal` default and is likewise unpublishable, and
|
||
there is no per-service list to forget to extend. `ci/render-golden.sh` proves `apps/` reproduces
|
||
from `catalog/`, so an `internal` service cannot exist in the applied tree without it, and
|
||
`conformance/tests/l0-static/internal-services-publish-no-public-dns` re-proves both directions
|
||
(present on internal, **absent** on public) on every gate run.
|
||
- The route itself, the wildcard TLS and LAN reachability are **unchanged** — only the public
|
||
record is withdrawn. The internal split-horizon answer is Pi-hole's job (R-DNS-03), which is why
|
||
withdrawing these records requires Pi-hole to be answering the LAN authoritatively first: the
|
||
zone has no `*.home.keithjackson.dev` wildcard, and `*.keithjackson.dev` does not match two
|
||
labels, so a withdrawn name is a public NXDOMAIN with no fallback.
|
||
- Why not a cluster-wide ExternalDNS opt-in filter (`labelFilter`, which v0.21.0 supports for
|
||
both configured sources): seven published names come from hand-authored `platform/`/`apps/`
|
||
manifests that carry no `exposure` field for a renderer to read, and an opt-in filter deletes
|
||
all seven the moment it is set. Per-resource exclusion has exactly the blast radius of the
|
||
defect. The filter stays available as hardening once those seven are adjudicated — see
|
||
`platform/externaldns/helmrelease.yaml` and the register in the L0 check.
|
||
- `public` renders the record but the renderer does **not** yet set a target: the record still
|
||
resolves to the shared Gateway VIP, i.e. an RFC1918 address, so a `public` service is not in fact
|
||
publicly reachable. Fixing that is **not** a renderer change on v0.21.0 —
|
||
`source/gateway.go` reads `external-dns.alpha.kubernetes.io/target` only from the **Gateway** (and
|
||
a ListenerSet), never from an HTTPRoute, so a per-service target override is not expressible while
|
||
the public services share the `shared` Gateway. The open decision is recorded in
|
||
`platform/externaldns/helmrelease.yaml`.
|
||
- Exposure does **not** relax east-west default-deny (docs/09 §"Interaction with default-deny").
|
||
- **Requirements:** R-DNS-02/03 (docs/09), R-NET-04 (docs/03).
|
||
|
||
### `egress` (`internet` | `vpn`) — egress steering
|
||
- `internet` → pod egress allow to the default WAN route (node → SOHO edge → internet).
|
||
- `vpn` → the pod is **steered through the shared in-cluster Gluetun/PIA (WireGuard) egress gateway**
|
||
(Cilium Egress Gateway / routing). The pod gets **no independent egress path**; the
|
||
**kill-switch (fail-closed)** and allow rules are enforced at the gateway + namespace/policy level,
|
||
**outside** the workload.
|
||
- **Requirements:** **R-NET-05** (egress class), **R-NET-06** (VPN kill-switch, mandatory & central).
|
||
VPN-class set is normative: sonarr, radarr, jackett, qbittorrent (docs/03/13).
|
||
|
||
### `dependencies[]` (`to` / `port` / `protocol`) — east-west policy
|
||
- Each namespace first gets the **default-deny baseline** (deny all ingress + egress) applied by
|
||
GitOps at namespace creation, with a standing allow only for **DNS to the in-cluster resolver**.
|
||
**Observability scraping is NOT in the baseline** — it is an additive allow rendered per declared
|
||
`observability.metricsEndpoint` (see below), scoped to that one port. This paragraph used to claim
|
||
the baseline carried a standing monitoring allow; it never did, and an app metrics port would have
|
||
been dropped by policy even with the Service port and monitor endpoint in agreement.
|
||
- Each `dependencies[]` entry renders one **CiliumNetworkPolicy** egress allow: this service → `to`
|
||
service on `port`/`protocol`. Everything undeclared stays denied.
|
||
- Empty `dependencies` ⇒ the service can reach **only DNS**.
|
||
- **Requirements:** **R-NET-01** (default-deny baseline), **R-NET-02** (explicit allow / deny
|
||
undeclared). Auditable via Hubble (R-SEC-03, R-OBS-04).
|
||
|
||
### `storage[]` (`name` / `size` / `class` / `mountPath` / `export` / `subPath` / `readOnly`) — PVs + PVCs
|
||
- `class: longhorn` → a **PVC** on the 3-replica Longhorn StorageClass sized by `size` (replicated
|
||
config/DB/app state), `ReadWriteOnce`, dynamically provisioned. For a `vm`, the disk is a Longhorn
|
||
volume via CDI.
|
||
- `class: bulk` → a **static, pre-bound external NFS PersistentVolume _and_ its PVC**. The entry's
|
||
`export` is the absolute path on the bulk server; the SERVER is never named in a ServiceDefinition
|
||
— it is the `${BULK_NFS_SERVER}` cluster-vars knob, so one authored definition promotes across
|
||
environments that reach the same storage by different addresses (**R-ENV-02**). Not 3× replicated:
|
||
durability is the storage host's ZFS pool, not Longhorn.
|
||
- `storageClassName: ""` on **both** PV and PVC (explicitly empty ⇒ statically provisioned; no
|
||
provisioner is ever asked to create anything, and the default-StorageClass annotation is
|
||
suppressed). Deliberately NOT the `bulk` StorageClass: a StorageClass `provisioner` is
|
||
**immutable**, so re-pointing that stub would make Flux fail permanently on it.
|
||
- Binding is pinned from both ends — `claimRef` on the PV, `volumeName` on the PVC — so no other
|
||
claim of matching size/mode can win a race for the volume.
|
||
- `persistentVolumeReclaimPolicy: Retain`, always. Bulk data pre-exists the cluster, is far larger
|
||
than the cluster, and may have consumers outside it; nothing here may reclaim it.
|
||
- Access mode follows the read/write posture and is always `…Many`, because one export is
|
||
consumed by several namespaces and several nodes at once: `ReadOnlyMany` when `readOnly`,
|
||
`ReadWriteMany` otherwise. `ReadWriteOnce` cannot express either.
|
||
- `mountOptions` are a **platform** decision held in the renderer, not a per-service field:
|
||
`nfsvers=4.1,hard,noatime,timeo=600,retrans=2`. NFSv4 carries mount and lock in-protocol, so
|
||
the only port that must be reachable is **tcp/2049** — no rpcbind/111, mountd or statd, which is
|
||
what keeps the firewall surface to a single rule. `hard` is chosen over `soft` because a soft
|
||
mount converts a server hiccup into a short read handed to the application.
|
||
- `size` on a bulk entry is the PV's **advertised capacity and nothing more** — no component
|
||
enforces it (the ZFS pool is the real limit), so it should state the dataset's actual size so
|
||
`kubectl get pv` reports something true.
|
||
- **fsGroup does not apply.** The in-tree NFS volume plugin performs no ownership management, so
|
||
`podSecurityContext.fsGroup` is a **no-op** on a bulk volume. A bulk mount that must be written
|
||
therefore requires the container's own uid/gid to be one the export's ownership permits (e.g.
|
||
Plex's `PLEX_UID`/`PLEX_GID`, LinuxServer images' `PUID`/`PGID`) — or a server-side squash. A
|
||
mount that succeeds but cannot write is the characteristic failure of this tier.
|
||
- `mountPath` (any class) overrides the `/data/<name>` default verbatim.
|
||
- `subPath` (any class) mounts a subtree instead of the volume root. **Two `subPath` mounts of one
|
||
export are two separate mounts**, so `link()` between them fails `EXDEV`; a workload that needs
|
||
hardlinks or atomic moves between two directories (Sonarr/Radarr importing from a download path
|
||
into a library) must mount their common ancestor **once** instead.
|
||
- `readOnly` (any class) marks the volumeMount read-only, and on a bulk entry also sets the PV's
|
||
`nfs.readOnly` so the kernel mount really is `ro`. Note this is a **client-side** guarantee: one
|
||
export serving both a reader and a writer that may land on any node cannot distinguish them
|
||
server-side.
|
||
- State separation rule: anything that must survive a rebuild lives on a named volume here, never on
|
||
ephemeral fs/hostPath — so the workload can be destroyed and re-applied from git with data intact.
|
||
- **Requirements:** **R-STATE-01** (rebuild survival), **R-STATE-02** (Longhorn replication),
|
||
**R-STATE-04** (bulk tier). (docs/04)
|
||
|
||
### `hardware` (`gpu`) — host device capability
|
||
- **The field is INTENT, never MECHANISM.** `hardware.gpu` is an **enum of capability classes**
|
||
(today: `intel-quicksync`), not a device path, not a hostPath, not a resource request. The
|
||
ServiceDefinition says *what class of host device this workload needs*; how the platform hands it
|
||
over is `catalog/render.py`'s business and lives nowhere else. **Do not "simplify" this into a raw
|
||
`/dev/...` path.** The entire payoff is that migrating from the hostPath mechanism below to the
|
||
Intel GPU device plugin (`resources.limits: gpu.intel.com/i915`) is a renderer change with **zero**
|
||
catalog edits and zero re-review of per-service device wiring. Extend the enum for a genuinely
|
||
different capability class (e.g. `nvidia-nvenc`); never to name a second route to the same one.
|
||
- `gpu: intel-quicksync` → on the **app container / pod spec only** (never a sidecar):
|
||
1. a **hostPath volume** `dri` → `/dev/dri`, **`type: DirectoryOrCreate`**;
|
||
2. a **read-write volumeMount** of `dri` at `/dev/dri` on the app container (VA-API opens the
|
||
render node read-write, so it is deliberately not `readOnly`);
|
||
3. the host `render` **GID merged into `podSecurityContext.supplementalGroups`** — merged, never
|
||
clobbered, so an SD that declares its own supplemental groups keeps all of them and gains one.
|
||
`/dev/dri/renderD128` is `root:render` mode `0660`, so a non-root container needs the group.
|
||
The GID is the single constant `INTEL_RENDER_GID` in `catalog/render.py` (992, **measured
|
||
2026-09-30 on all three prod nodes**; it is a *systemd dynamic* allocation, not a static GID,
|
||
so a node rebuild can move it — re-measure `getent group render` before assuming).
|
||
- **`DirectoryOrCreate` is load-bearing — do not tighten it to `Directory`.** A node that comes back
|
||
without the render device (rebuild, BIOS change, driver regression, hardware swap) must degrade the
|
||
workload to **software transcoding**, not fail the mount. `apps-wave-0` is `wait: true` and
|
||
`apps-wave-1` `dependsOn` it, so a failed hostPath mount stalls the **whole app tier** for the sake
|
||
of a feature that is not load-bearing. `/dev` is tmpfs, so the empty directory the kubelet creates
|
||
on such a node leaves nothing durable behind.
|
||
- **The cost of that choice, stated plainly:** a missing device is **silent**. The workload
|
||
software-transcodes and nothing reports it. There is no metric to alert on — media apps here
|
||
render no `metricsEndpoint`, so no ServiceMonitor exists — and an alert that cannot fire is worse
|
||
than none (see §`observability`'s 2026-09-13 correction). The only proof of hardware transcode is
|
||
an operator observing it in the application's own dashboard.
|
||
- **Deliberately NOT rendered** from this field: no `privileged: true` (Kyverno
|
||
`restrict-host-namespaces` is Enforce on app namespaces and would reject it, and the render node
|
||
does not need it); no `nodeSelector` (placement is a separate concern with its own justification —
|
||
this field is about the device, not about where the pod lands); no resource requests/limits (there
|
||
is no app-container resource render path, and inventing one here would be scope creep).
|
||
- Absent ⇒ **nothing at all** is rendered; every service that does not opt in stays byte-identical.
|
||
Ignored for `kind: vm` — a VM receives devices through KubeVirt's own `domain.devices`.
|
||
- **Requirements:** **R-PLAT-02** (the PaaS provides the host capability; the app declares need).
|
||
|
||
### `ingressRoutes[]` (`host` / `port`) — HTTPRoute + Certificate + DNS
|
||
- Each entry renders an **HTTPRoute** attaching `host` → the service `port` on the shared Gateway.
|
||
- TLS: the Gateway terminates with the cert-manager **wildcard `*.home.keithjackson.dev`** certificate
|
||
(Let's Encrypt via **DNS-01**, scoped Cloudflare token), so no per-service cert step — trusted,
|
||
non-self-signed, auto-renewed, including for `internal` services. Per-service `Certificate` only
|
||
where isolation warrants.
|
||
- DNS: **ExternalDNS** watches the HTTPRoute and manages the `<host>` record — but **only for a
|
||
`public` service**. An `internal` service's route is rendered with ExternalDNS's controller
|
||
annotation set to `none`, which removes it from ExternalDNS's view entirely (see §`exposure`), so
|
||
its `<host>` exists **only** in the internal split-horizon answer PiHole serves, never in the
|
||
public zone. A `public` record's TARGET is still the Gateway VIP, not a routable address — open
|
||
operator decision, see §`exposure`.
|
||
- The `host` pattern is enforced to `<label>.home.keithjackson.dev` by the schema.
|
||
- **Requirements:** **R-TLS-01/02/03** (docs/08), **R-DNS-01/02/03** (docs/09).
|
||
|
||
### `inClusterClients[]` (`namespace` / `port` / `protocol` / `name`) — direct east-west clients
|
||
- Each entry renders ONE **CiliumNetworkPolicy** ingress allow on this service:
|
||
`endpointSelector` = this service, `fromEndpoints` matching **only** the reserved
|
||
`k8s:io.kubernetes.pod.namespace: <namespace>` label, `toPorts` carrying **only** the declared
|
||
`port`/`protocol`. Name: `<service>-allow-<name>-ingress`, `name` defaulting to `namespace`.
|
||
- **Why the field exists.** `ingressRoutes` opens the **north-south** path only (the gateway allow
|
||
admits `fromEntities: [ingress]`, the Gateway proxy identity, and nothing else), and
|
||
`dependencies` compiles edges **only between catalog services** — both halves come from a catalog
|
||
entry's own declaration. A **platform controller** that dials this service's ClusterIP directly
|
||
was therefore unauthorable, and under default-deny it is dropped. Prod hit exactly that: its
|
||
`GitRepository` points at the in-cluster Gitea Service
|
||
(`http://gitea.gitea.svc.cluster.local:3000/...`, clusters/prod/kustomization.yaml) so GitOps
|
||
depends on no public DNS/Cloudflare/Gateway/`${BASE_DOMAIN}`, but `flux-system`'s
|
||
source-controller was denied — and the HTTPRoute that would have served a public name lives in a
|
||
**downstream** tier that cannot apply while the source is down (source down → routes never rename
|
||
→ source stays down).
|
||
- **Least privilege is part of the contract:** a namespace and a port, never `fromEntities: all`,
|
||
never a port-less allow. The client is selected by **namespace** rather than pod labels because an
|
||
upstream controller's labels are its chart's to change, not this platform's contract — so name a
|
||
platform-controller namespace whose entire membership may hold that reach, never a shared one.
|
||
- **Additive, and not a dependency:** the namespace default-deny baseline is unchanged
|
||
(`ingress: []`), every undeclared source/port stays denied, and the entry creates **no wave
|
||
ordering edge** — the client is a platform component, not a catalog service.
|
||
- **Requirements:** **R-NET-01** (default-deny baseline stands), **R-NET-02** (explicit allow / deny
|
||
undeclared), **R-GITOPS-02** (the allowance is authored in the one reviewable object rather than
|
||
surviving as a live-applied cluster object). Auditable via Hubble (R-SEC-03, R-OBS-04).
|
||
|
||
### `auth` (`mode` / `groups[]`) — identity integration (Keycloak / oauth2-proxy)
|
||
- `oidc` → register the app as a **Keycloak OIDC client** (declarative realm/client config, GitOps;
|
||
client secret SOPS-encrypted). App does its own login redirect.
|
||
- `forward-auth` → protect the service's `ingressRoutes` at the **Gateway** with **oauth2-proxy →
|
||
Keycloak** so auth happens before traffic reaches the app (bare UIs without usable OIDC).
|
||
- `native` → no platform SSO wiring; app owns its auth (documented exception, e.g. Plex/Seerr; and
|
||
Keycloak **itself** as the IdP, admin login local + MFA).
|
||
- `none` → no auth wiring (internal-only, discouraged).
|
||
- `groups[]` → the Keycloak groups permitted access, enforced by the OIDC client mappers /
|
||
oauth2-proxy `allowed_groups`. **Schema requires ≥1 group for `oidc`/`forward-auth`** — access is
|
||
"in an authorized group," not merely "logged in."
|
||
- Keycloak, oauth2-proxy, and app→Keycloak / proxy→Keycloak / Keycloak→DB flows are ordinary
|
||
workloads under default-deny with explicit allows (ties back to `dependencies`).
|
||
- **Requirements:** **R-IDP-02** (protect OIDC-capable and bare UIs), **R-IDP-06** (group-based
|
||
authorization). (docs/15, ADR-0014)
|
||
|
||
### `update` (`channel` / `range` / `cadence` / `autoRollbackOnFailedHealth` / `pin`)
|
||
- container `kind` → a Flux **ImagePolicy** (+ ImageRepository) encoding `channel`/`range`; Flux
|
||
image-automation advances the running version on its scan interval (`cadence`). `pin` freezes the
|
||
service (stops automation). No human/AI edits a tag.
|
||
- `vm` `kind` → in-guest unattended package updates on `cadence` (cloud-init), per docs/11 §3.
|
||
- `autoRollbackOnFailedHealth: true` → health-gated rollout that reverts on failed health and alerts.
|
||
- **Requirements:** **R-UPD-01** (stable only), **R-UPD-02** (no human/AI authoring), **R-UPD-03**
|
||
(health-gated auto-rollback). (docs/11, ADR-0009)
|
||
|
||
### `observability` (`metrics` / `slis[]` / `metricsEndpoint` / `livenessWorkload`) — ServiceMonitor + alerts
|
||
- `metrics: true` → the platform **collects signals** for this service: logs to Loki via the node
|
||
agent, and kube-state-metrics readiness for the `up` SLI below. Both are platform-wide scrapes;
|
||
neither needs a per-service CR. **It does not assert that the workload serves a Prometheus
|
||
endpoint.** *(Corrected 2026-09-13. It used to render a ServiceMonitor unconditionally, hardcoded
|
||
to a Service port named `metrics`. No rendered Service ever defined such a port, so all twelve app
|
||
monitors discovered ZERO targets — verified on prod: 24 jobs in `up`, not one of the twelve — and
|
||
every `up{job=...}` SLI built on them was structurally inert. `metrics: true` could not have been
|
||
right: Plex, Pi-hole, Jackett, qBittorrent, Sonarr, Radarr, Seerr, Semaphore and Nextcloud serve no
|
||
Prometheus endpoint at all without a separate exporter.)*
|
||
- `metricsEndpoint: {port, name?, path?, interval?}` → **the workload does serve metrics, here.**
|
||
Renders all three halves of a working scrape from the one declaration, because any one of them
|
||
missing makes the scrape silently yield nothing:
|
||
1. a **named Service port** (`name`, default `metrics`) on that `port`;
|
||
2. a **ServiceMonitor** endpoint selecting that port name, on `path` at `interval`;
|
||
3. a **CiliumNetworkPolicy** admitting the `monitoring` namespace to that port — the namespace
|
||
default-deny baseline is `ingress: []`, so without it the scrape is dropped by policy.
|
||
The metrics port is deliberately **not** added to the Gateway ingress allow: it is east-west for
|
||
Prometheus, never north-south.
|
||
- each entry in `slis[]` → at least one **alert rule** routed by Alertmanager to email; a per-service
|
||
Grafana dashboard row. `up` → a `<name>-down` **critical** alert reading the readiness of
|
||
`livenessWorkload` from **kube-state-metrics** (`kube_deployment_status_replicas_available`,
|
||
`kube_statefulset_status_replicas_ready`, `kube_daemonset_status_number_available`) or, for a
|
||
`VirtualMachineInstance`, the existence of `kubevirt_vmi_info{phase="running"}`. Each rule carries
|
||
an `or absent(...)` half, so the signal DISAPPEARING fires the alert rather than silencing it.
|
||
- `livenessWorkload: {kind, name}` → which object's readiness **is** this service. Defaults to the
|
||
object this SD renders; override it when the running workload is a different one (keycloak's server
|
||
is a StatefulSet owned by the Keycloak operator — a Deployment-shaped rule would `absent()` and
|
||
fire forever).
|
||
- **Requirements:** **R-OBS-01** (per-service SLIs/alerts). Platform-health alerts (R-OBS-04) are
|
||
rendered platform-wide, not per ServiceDefinition. (docs/10)
|
||
|
||
### `backup` (`policy`)
|
||
- a schedule ref (e.g. `daily-offcluster`) → **Velero + Longhorn** off-cluster backup schedule for
|
||
this service's volumes/namespace.
|
||
- literal `reacquirable` → **no** 1:1 backup (bulk media only). **Irreplaceable** state (configs, DBs,
|
||
Nextcloud, Gitea, PiHole config, Keycloak DB) MUST NOT be `reacquirable`.
|
||
- **Requirements:** **R-STATE-03** (off-cluster DR backup). (docs/04)
|
||
|
||
---
|
||
|
||
## Cross-field invariants the renderer / L0-L1 tests rely on
|
||
1. **kind ↔ source:** container⇒`image`, vm⇒`vmTemplate` (schema-enforced).
|
||
2. **auth ↔ groups:** `oidc`/`forward-auth` ⇒ non-empty `groups` (schema-enforced; R-IDP-06).
|
||
3. **default-deny is unconditional:** every namespace gets the deny baseline regardless of
|
||
`dependencies`; `dependencies` only *adds* allows (R-NET-01/02). The same holds for
|
||
`inClusterClients`: it adds one namespace+port ingress allow and never relaxes the baseline.
|
||
4. **egress is enforced outside the workload:** `vpn` renders gateway steering + kill-switch, never a
|
||
per-service sidecar the app could bypass (R-NET-05/06).
|
||
5. **TLS is never self-signed and needs no per-service step:** the wildcard Gateway cert covers every
|
||
`ingressRoutes.host`, `internal` included (R-TLS-01/03).
|
||
6. **irreplaceable ⇒ off-cluster backup:** DR-critical services must not carry `backup.policy:
|
||
reacquirable` (R-STATE-03). *(Advisory: not enforced by the schema — an L0 policy check should
|
||
assert it; see "Ambiguities / escalations" in schemas/README.md.)*
|
||
7. **`exposure: internal` ⇒ no public DNS record, structurally.** Every HTTPRoute rendered for an
|
||
`internal` service (including one that omits `exposure` and takes the default) carries
|
||
`external-dns.alpha.kubernetes.io/controller: none`, and no `public` service's route carries it.
|
||
Derived from `exposure` alone; enforced on every gate run by
|
||
`conformance/tests/l0-static/internal-services-publish-no-public-dns` (R-DNS-03).
|
||
8. **`hardware` is inert until opted into, and its mechanism is renderer-local.** An SD with no
|
||
`hardware` key renders no volume, no volumeMount and no `securityContext` — proven by
|
||
`render.py --selftest`. The `/dev/dri` hostPath, the `DirectoryOrCreate` type and
|
||
`INTEL_RENDER_GID` appear in `catalog/render.py` and nowhere else, so the capability's
|
||
implementation can be replaced without touching a single ServiceDefinition.
|
||
|
||
## Requirement traceability (contract-level)
|
||
Primary **R-GITOPS-02** (single reviewable object wires every requirement). Field-level: R-NET-01/02
|
||
(dependencies, inClusterClients), R-NET-05/06 (egress), R-STATE-01/02/04 (storage), R-STATE-03 (backup),
|
||
R-TLS-01/02/03 + R-DNS-01/02/03 (ingressRoutes + exposure), R-OBS-01 (observability),
|
||
R-UPD-01/02/03 (update), R-IDP-02/06 (auth), R-PLAT-02/03 (kind/source).
|