A platform controller that dials a service's ClusterIP directly had NO authorable
form in the ServiceDefinition contract:
* ingressRoutes renders the north-south allow only — gateway-ingress admits
`fromEntities: [ingress]`, the Gateway's Envoy identity, and nothing else;
* dependencies compiles edges only BETWEEN CATALOG SERVICES, from the source
entry's own declaration.
Under the default-deny baseline (R-NET-01) such a client is therefore DROPPED, and
the only way to unblock it was a live `kubectl apply` of a hand-written CNP — a
cluster object outside git, one `kubectl delete` or one rebuild from coming back.
`inClusterClients: [{namespace, port, protocol?, name?}]` closes that. Each entry
renders ONE least-privilege ingress CiliumNetworkPolicy: this service's
endpointSelector, `fromEndpoints` matching ONLY `k8s:io.kubernetes.pod.namespace`,
`toPorts` carrying ONLY the declared port/protocol — never `fromEntities: all`.
Additive: the namespace default-deny keeps denying everything undeclared (R-NET-02),
and the entry is NOT a dependency edge, so it creates no wave ordering.
Shape notes:
* selected by NAMESPACE, not pod labels: an upstream controller's labels belong to
its chart, not to this platform's contract. The cost is explicit — every pod in
that namespace is admitted on that port, so the field is for platform-controller
namespaces (flux-system, monitoring), never a shared one. The schema description
says so.
* optional `name` (default: the namespace) only names the rendered policy,
`<service>-allow-<name>-ingress`. That name is operator-facing (kubectl get cnp,
a Hubble drop), and pinning it lets an allowance that already had to be applied
live be authored under its EXISTING name, so Flux adopts the live object in place
instead of creating a near-duplicate and leaving the original unmanaged.
* one field, not a policy DSL: no arbitrary selectors, entities or CIDRs.
Covered by render.py --selftest (4c): name/namespace/selector/port shape, no
fromEntities widening, default-deny still `ingress: []`, the gateway allow not
displaced, `name` defaulting to the namespace, and nothing rendered when the field is
absent. No golden changes here — no catalog entry uses the field yet.
Co-Authored-By: Claude Opus 5 <[email protected]>
16 KiB
Renderer Contract — ServiceDefinition → rendered resources
Status: authoritative Phase-0 interface contract. Consumed by the manifest renderer (WS-GITOPS), the network/egress renderer (WS-NET), storage (WS-STATE), TLS/DNS (WS-EDGE), identity (WS-SEC), observability (WS-OBS), updates (WS-UPD), and the conformance/policy/CI agents (WS-TEST).
Source of truth: spec docs/13-service-catalog.md
(the ServiceDefinition schema and default posture). This document maps each field of
servicedefinition.schema.json to the concrete resources the
platform must render, and cites the requirement each mapping satisfies. Changing this contract is an
interface change — coordinate per the spec's agents/coordination-protocol.md (§2).
The renderer MUST first materialize the schema defaults (below), then apply this mapping. A field absent from a ServiceDefinition renders from its default posture, not "not at all."
Default posture (applied before mapping) — docs/13 §"Default posture"
| Field | Default | Requirement |
|---|---|---|
kind |
container |
R-PLAT-02/03 |
exposure |
internal |
R-DNS/R-NET |
egress |
internet |
R-NET-05 |
storage |
[] (config PVC still expected for stateful svc) |
R-STATE-02 |
dependencies |
[] → reaches only DNS (default-deny) |
R-NET-01 |
inClusterClients |
[] → no direct east-west client; only the Gateway may reach an exposed port |
R-NET-01/02 |
auth.mode |
forward-auth for any exposed UI |
R-IDP-02 |
update.channel |
stable |
R-UPD-01 |
update.autoRollbackOnFailedHealth |
true |
R-UPD-03 |
observability.metrics |
true |
R-OBS-01 |
| backup of irreplaceable state | off-cluster (never reacquirable) |
R-STATE-03 |
Field → resource mapping
name
- Renders: the service Namespace (trust zone), Deployment/VM name, Service name, and the DNS label
<name>.home.keithjackson.dev. - Requirements: R-DNS-01 (naming model, docs/09), R-NET (namespace = isolation boundary, docs/03).
kind + source
container+source.image→ Deployment (+ Service) using the image ref.vm+source.vmTemplate→ KubeVirt VirtualMachine with a CDI-imported root disk.- The schema enforces exactly-one-of (
imagefor container,vmTemplatefor vm). - Requirements: R-PLAT-02/03 (PaaS/IaaS, docs/13); VM disks inherit Longhorn durability (docs/04).
exposure (internal | public)
- Both classes render an HTTPRoute on the shared Gateway and are reachable on the LAN via internal split-horizon DNS.
publicadditionally signals the documented manual edge port-forward of 80/443 (and any service-specific port, e.g. Plex 32400, Minecraft 25565) to the ingress VIP, and a public ExternalDNS record for<name>.home.keithjackson.dev.internalrenders no public DNS record and is never port-forwarded (PiHole resolves it to the internal Gateway VIP).- Exposure does not relax east-west default-deny (docs/09 §"Interaction with default-deny").
- Requirements: R-DNS-02/03 (docs/09), R-NET-04 (docs/03).
egress (internet | vpn) — egress steering
internet→ pod egress allow to the default WAN route (node → SOHO edge → internet).vpn→ the pod is steered through the shared in-cluster Gluetun/PIA (WireGuard) egress gateway (Cilium Egress Gateway / routing). The pod gets no independent egress path; the kill-switch (fail-closed) and allow rules are enforced at the gateway + namespace/policy level, outside the workload.- Requirements: R-NET-05 (egress class), R-NET-06 (VPN kill-switch, mandatory & central). VPN-class set is normative: sonarr, radarr, jackett, qbittorrent (docs/03/13).
dependencies[] (to / port / protocol) — east-west policy
- Each namespace first gets the default-deny baseline (deny all ingress + egress) applied by GitOps at namespace creation, with a standing allow only for DNS to the in-cluster resolver and for observability scraping from the monitoring namespace.
- Each
dependencies[]entry renders one CiliumNetworkPolicy egress allow: this service →toservice onport/protocol. Everything undeclared stays denied. - Empty
dependencies⇒ the service can reach only DNS. - Requirements: R-NET-01 (default-deny baseline), R-NET-02 (explicit allow / deny undeclared). Auditable via Hubble (R-SEC-03, R-OBS-04).
storage[] (name / size / class / mountPath / export / subPath / readOnly) — PVs + PVCs
class: longhorn→ a PVC on the 3-replica Longhorn StorageClass sized bysize(replicated config/DB/app state),ReadWriteOnce, dynamically provisioned. For avm, the disk is a Longhorn volume via CDI.class: bulk→ a static, pre-bound external NFS PersistentVolume and its PVC. The entry'sexportis the absolute path on the bulk server; the SERVER is never named in a ServiceDefinition — it is the${BULK_NFS_SERVER}cluster-vars knob, so one authored definition promotes across environments that reach the same storage by different addresses (R-ENV-02). Not 3× replicated: durability is the storage host's ZFS pool, not Longhorn.storageClassName: ""on both PV and PVC (explicitly empty ⇒ statically provisioned; no provisioner is ever asked to create anything, and the default-StorageClass annotation is suppressed). Deliberately NOT thebulkStorageClass: a StorageClassprovisioneris immutable, so re-pointing that stub would make Flux fail permanently on it.- Binding is pinned from both ends —
claimRefon the PV,volumeNameon the PVC — so no other claim of matching size/mode can win a race for the volume. persistentVolumeReclaimPolicy: Retain, always. Bulk data pre-exists the cluster, is far larger than the cluster, and may have consumers outside it; nothing here may reclaim it.- Access mode follows the read/write posture and is always
…Many, because one export is consumed by several namespaces and several nodes at once:ReadOnlyManywhenreadOnly,ReadWriteManyotherwise.ReadWriteOncecannot express either. mountOptionsare a platform decision held in the renderer, not a per-service field:nfsvers=4.1,hard,noatime,timeo=600,retrans=2. NFSv4 carries mount and lock in-protocol, so the only port that must be reachable is tcp/2049 — no rpcbind/111, mountd or statd, which is what keeps the firewall surface to a single rule.hardis chosen oversoftbecause a soft mount converts a server hiccup into a short read handed to the application.sizeon a bulk entry is the PV's advertised capacity and nothing more — no component enforces it (the ZFS pool is the real limit), so it should state the dataset's actual size sokubectl get pvreports something true.- fsGroup does not apply. The in-tree NFS volume plugin performs no ownership management, so
podSecurityContext.fsGroupis a no-op on a bulk volume. A bulk mount that must be written therefore requires the container's own uid/gid to be one the export's ownership permits (e.g. Plex'sPLEX_UID/PLEX_GID, LinuxServer images'PUID/PGID) — or a server-side squash. A mount that succeeds but cannot write is the characteristic failure of this tier.
mountPath(any class) overrides the/data/<name>default verbatim.subPath(any class) mounts a subtree instead of the volume root. TwosubPathmounts of one export are two separate mounts, solink()between them failsEXDEV; a workload that needs hardlinks or atomic moves between two directories (Sonarr/Radarr importing from a download path into a library) must mount their common ancestor once instead.readOnly(any class) marks the volumeMount read-only, and on a bulk entry also sets the PV'snfs.readOnlyso the kernel mount really isro. Note this is a client-side guarantee: one export serving both a reader and a writer that may land on any node cannot distinguish them server-side.- State separation rule: anything that must survive a rebuild lives on a named volume here, never on ephemeral fs/hostPath — so the workload can be destroyed and re-applied from git with data intact.
- Requirements: R-STATE-01 (rebuild survival), R-STATE-02 (Longhorn replication), R-STATE-04 (bulk tier). (docs/04)
ingressRoutes[] (host / port) — HTTPRoute + Certificate + DNS
- Each entry renders an HTTPRoute attaching
host→ the serviceporton the shared Gateway. - TLS: the Gateway terminates with the cert-manager wildcard
*.home.keithjackson.devcertificate (Let's Encrypt via DNS-01, scoped Cloudflare token), so no per-service cert step — trusted, non-self-signed, auto-renewed, including forinternalservices. Per-serviceCertificateonly where isolation warrants. - DNS: ExternalDNS watches the HTTPRoute and manages the
<host>record (public IP forpublic, internal Gateway VIP via PiHole split-horizon forinternal). - The
hostpattern is enforced to<label>.home.keithjackson.devby the schema. - Requirements: R-TLS-01/02/03 (docs/08), R-DNS-01/02/03 (docs/09).
inClusterClients[] (namespace / port / protocol / name) — direct east-west clients
- Each entry renders ONE CiliumNetworkPolicy ingress allow on this service:
endpointSelector= this service,fromEndpointsmatching only the reservedk8s:io.kubernetes.pod.namespace: <namespace>label,toPortscarrying only the declaredport/protocol. Name:<service>-allow-<name>-ingress,namedefaulting tonamespace. - Why the field exists.
ingressRoutesopens the north-south path only (the gateway allow admitsfromEntities: [ingress], the Gateway proxy identity, and nothing else), anddependenciescompiles edges only between catalog services — both halves come from a catalog entry's own declaration. A platform controller that dials this service's ClusterIP directly was therefore unauthorable, and under default-deny it is dropped. Prod hit exactly that: itsGitRepositorypoints at the in-cluster Gitea Service (http://gitea.gitea.svc.cluster.local:3000/..., clusters/prod/kustomization.yaml) so GitOps depends on no public DNS/Cloudflare/Gateway/${BASE_DOMAIN}, butflux-system's source-controller was denied — and the HTTPRoute that would have served a public name lives in a downstream tier that cannot apply while the source is down (source down → routes never rename → source stays down). - Least privilege is part of the contract: a namespace and a port, never
fromEntities: all, never a port-less allow. The client is selected by namespace rather than pod labels because an upstream controller's labels are its chart's to change, not this platform's contract — so name a platform-controller namespace whose entire membership may hold that reach, never a shared one. - Additive, and not a dependency: the namespace default-deny baseline is unchanged
(
ingress: []), every undeclared source/port stays denied, and the entry creates no wave ordering edge — the client is a platform component, not a catalog service. - Requirements: R-NET-01 (default-deny baseline stands), R-NET-02 (explicit allow / deny undeclared), R-GITOPS-02 (the allowance is authored in the one reviewable object rather than surviving as a live-applied cluster object). Auditable via Hubble (R-SEC-03, R-OBS-04).
auth (mode / groups[]) — identity integration (Keycloak / oauth2-proxy)
oidc→ register the app as a Keycloak OIDC client (declarative realm/client config, GitOps; client secret SOPS-encrypted). App does its own login redirect.forward-auth→ protect the service'singressRoutesat the Gateway with oauth2-proxy → Keycloak so auth happens before traffic reaches the app (bare UIs without usable OIDC).native→ no platform SSO wiring; app owns its auth (documented exception, e.g. Plex/Seerr; and Keycloak itself as the IdP, admin login local + MFA).none→ no auth wiring (internal-only, discouraged).groups[]→ the Keycloak groups permitted access, enforced by the OIDC client mappers / oauth2-proxyallowed_groups. Schema requires ≥1 group foroidc/forward-auth— access is "in an authorized group," not merely "logged in."- Keycloak, oauth2-proxy, and app→Keycloak / proxy→Keycloak / Keycloak→DB flows are ordinary
workloads under default-deny with explicit allows (ties back to
dependencies). - Requirements: R-IDP-02 (protect OIDC-capable and bare UIs), R-IDP-06 (group-based authorization). (docs/15, ADR-0014)
update (channel / range / cadence / autoRollbackOnFailedHealth / pin)
- container
kind→ a Flux ImagePolicy (+ ImageRepository) encodingchannel/range; Flux image-automation advances the running version on its scan interval (cadence).pinfreezes the service (stops automation). No human/AI edits a tag. vmkind→ in-guest unattended package updates oncadence(cloud-init), per docs/11 §3.autoRollbackOnFailedHealth: true→ health-gated rollout that reverts on failed health and alerts.- Requirements: R-UPD-01 (stable only), R-UPD-02 (no human/AI authoring), R-UPD-03 (health-gated auto-rollback). (docs/11, ADR-0009)
observability (metrics / slis[]) — ServiceMonitor + alerts
metrics: true→ a ServiceMonitor/PodMonitor so Prometheus scrapes the service; logs are shipped to Loki by the node/pod agent by default.- each entry in
slis[]→ at least one alert rule (e.g.up→ service-down alert) routed by Alertmanager to email; a per-service Grafana dashboard row. - Requirements: R-OBS-01 (per-service SLIs/alerts). Platform-health alerts (R-OBS-04) are rendered platform-wide, not per ServiceDefinition. (docs/10)
backup (policy)
- a schedule ref (e.g.
daily-offcluster) → Velero + Longhorn off-cluster backup schedule for this service's volumes/namespace. - literal
reacquirable→ no 1:1 backup (bulk media only). Irreplaceable state (configs, DBs, Nextcloud, Gitea, PiHole config, Keycloak DB) MUST NOT bereacquirable. - Requirements: R-STATE-03 (off-cluster DR backup). (docs/04)
Cross-field invariants the renderer / L0-L1 tests rely on
- kind ↔ source: container⇒
image, vm⇒vmTemplate(schema-enforced). - auth ↔ groups:
oidc/forward-auth⇒ non-emptygroups(schema-enforced; R-IDP-06). - default-deny is unconditional: every namespace gets the deny baseline regardless of
dependencies;dependenciesonly adds allows (R-NET-01/02). The same holds forinClusterClients: it adds one namespace+port ingress allow and never relaxes the baseline. - egress is enforced outside the workload:
vpnrenders gateway steering + kill-switch, never a per-service sidecar the app could bypass (R-NET-05/06). - TLS is never self-signed and needs no per-service step: the wildcard Gateway cert covers every
ingressRoutes.host,internalincluded (R-TLS-01/03). - irreplaceable ⇒ off-cluster backup: DR-critical services must not carry
backup.policy: reacquirable(R-STATE-03). (Advisory: not enforced by the schema — an L0 policy check should assert it; see "Ambiguities / escalations" in schemas/README.md.)
Requirement traceability (contract-level)
Primary R-GITOPS-02 (single reviewable object wires every requirement). Field-level: R-NET-01/02 (dependencies, inClusterClients), R-NET-05/06 (egress), R-STATE-01/02/04 (storage), R-STATE-03 (backup), R-TLS-01/02/03 + R-DNS-01/02/03 (ingressRoutes + exposure), R-OBS-01 (observability), R-UPD-01/02/03 (update), R-IDP-02/06 (auth), R-PLAT-02/03 (kind/source).