Files
home-network-k3s-impl/schemas/servicedefinition.schema.json
T
jakman85andClaude Opus 5 cec550045b feat(storage): the bulk tier becomes real — static NFS PVs, authored once per environment
The `bulk` class has existed since Phase 0 as a deliberate stub: the StorageClass named an NFS
CSI driver that is not installed and a server/share of REPLACE-ME, so plex, sonarr, radarr and
qbittorrent all carry a comment saying "attach the library once OQ-2 is resolved". This is that
resolution — for the case that actually exists here, which is NOT the case the stub anticipated.

The stub assumed DYNAMIC provisioning: ask a CSI driver for a bulk volume and get a fresh empty
subdirectory. That is the wrong shape for this platform's bulk data. The media library is 8.33 TiB
that already exists, on ZFS datasets on the Proxmox host, with live non-k3s consumers. Nothing
needs provisioning; existing datasets need ATTACHING. So a bulk storage entry now renders a
STATIC, pre-bound NFS PersistentVolume plus its PVC, and needs no CSI driver at all (the in-tree
NFS volume source is not deprecated and has no CSI migration).

Schema 1.4.0 adds, on a storage entry:
  export     absolute NFS export path; REQUIRED when class is bulk, FORBIDDEN otherwise (an `if`
             in the schema, so a bulk entry cannot be authored without saying what it attaches).
  subPath    mount a subtree instead of the volume root — any class.
  readOnly   any class; on bulk it also sets the PV's nfs.readOnly, so the mount really is `ro`.

The SERVER is deliberately NOT a schema field. It is the `${BULK_NFS_SERVER}` cluster-vars knob,
because dev and prod reach the SAME storage host by different addresses and R-ENV-02 says the
service definition is authored once. A ServiceDefinition says WHICH dataset; the overlay says
where the host is.

Three decisions worth the reader's time, argued in full in the r_bulk_pvs docstring:

  * PV/PVC, not an inline `nfs:` pod volume. Inline is fewer objects and would work. It cannot
    express `persistentVolumeReclaimPolicy: Retain`, which is the one property most worth having
    in git for 8.33 TiB of data with consumers outside this cluster; it puts read-only enforcement
    one Deployment edit away from silent removal; and it is invisible to `kubectl get pv,pvc`,
    which is where an operator looks for attached external storage.
  * storageClassName: "" on both ends, NOT the `bulk` StorageClass. Explicitly empty is the
    canonical marker for a statically provisioned volume. It also sidesteps a trap: a
    StorageClass's `provisioner` is IMMUTABLE, so re-pointing the deployed `bulk` stub would make
    Flux fail on that object forever.
  * Access mode is always `…Many` and follows the read/write posture — ReadOnlyMany for a reader,
    ReadWriteMany for a writer. One export is consumed by several namespaces AND several nodes at
    once, which is exactly what ReadWriteOnce cannot express; which `Many` it is then makes the
    intent legible in `kubectl get pv`.

mountOptions are a platform decision held in the renderer rather than a per-service field:
nfsvers=4.1,hard,noatime,timeo=600,retrans=2. NFSv4 carries mount and lock in-protocol, so the
only port that must be reachable is tcp/2049 — no rpcbind/111, mountd or statd. That is a
firewall decision as much as a storage one. `hard` over `soft` because a soft mount converts a
server hiccup into a short read handed to the application.

The contract doc records the trap that will otherwise be discovered in production: fsGroup is a
NO-OP on an NFS volume (the in-tree plugin does no ownership management), so a writable bulk mount
requires the container's own uid/gid to suit the export's ownership. A mount that succeeds and
then cannot write is this tier's characteristic failure.

Every pre-existing golden renders byte-identical: the new keys are emitted only when declared, and
`readOnly` defaults to False. render-golden: 10/10. selftest: PASS.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-09-07 23:45:09 -05:00

415 lines
20 KiB
JSON

{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://home.keithjackson.dev/schemas/servicedefinition.schema.json",
"title": "ServiceDefinition",
"description": "Operator-facing onboarding contract for the home k3s + Longhorn platform. One ServiceDefinition per file under catalog/. Captures INTENT; the platform compiles it into concrete resources (see schemas/renderer-contract.md). Canonical source of truth: spec docs/13-service-catalog.md. Serves R-GITOPS-02; ties into R-NET-*, R-STATE-*, R-TLS-*, R-DNS-*, R-OBS-*, R-UPD-*, R-IDP-*. Schema version: 1.4.0 (makes the `bulk` tier REAL: a bulk storage entry now carries `export` (the absolute NFS export path on the ${BULK_NFS_SERVER} host) and renders a STATIC, pre-bound NFS PersistentVolume + PVC over data that already exists outside the cluster, plus optional `subPath` and `readOnly` on any entry. Resolves the OQ-2 bulk-backing stub for STATIC pre-existing datasets; dynamic bulk provisioning remains unimplemented and unneeded). 1.3.0 added optional pod-level podSecurityContext for the container kind — k8s PodSecurityContext emitted verbatim onto the pod spec; its fsGroup unblocks non-root images e.g. semaphore/UID-1001 writing a mounted Longhorn PVC). 1.2.0 added optional per-container env/envFrom/lifecycle for the container kind — k8s EnvVar/EnvFromSource/Lifecycle shapes emitted verbatim onto the app container; unblocks headless-install/DB-backed apps e.g. gitea/nextcloud/semaphore/unifi/minecraft. 1.1.0 added the shape invariant exposure:public => auth.mode!=none (docs/13 v1.1 / docs/15 / R-IDP-02).",
"type": "object",
"additionalProperties": false,
"required": ["name", "source"],
"properties": {
"name": {
"type": "string",
"description": "DNS label -> <name>.${BASE_DOMAIN}, i.e. <name>.home.keithjackson.dev on dev and <name>.prod.home.keithjackson.dev on prod (docs/13, docs/09). Must be a valid RFC 1123 DNS label.",
"pattern": "^[a-z0-9]([-a-z0-9]*[a-z0-9])?$",
"minLength": 1,
"maxLength": 63
},
"kind": {
"type": "string",
"description": "PaaS container (default) or IaaS VM via KubeVirt (docs/13; R-PLAT-02/03).",
"enum": ["container", "vm"],
"default": "container"
},
"source": {
"type": "object",
"description": "Workload source. container => image; vm => vmTemplate (CDI source). Exactly one, matching kind.",
"additionalProperties": false,
"minProperties": 1,
"properties": {
"image": {
"type": "string",
"description": "Container image reference (repo/image[:tag]). Required when kind=container.",
"minLength": 1
},
"vmTemplate": {
"type": "string",
"description": "KubeVirt/CDI source image reference. Required when kind=vm.",
"minLength": 1
}
}
},
"env": {
"type": "array",
"description": "Optional per-container environment variables for the container kind (docs/13). Mirrors the k8s EnvVar shape and is emitted verbatim onto the app container: each entry is either a literal {name, value} or a {name, valueFrom: {secretKeyRef|configMapKeyRef}} sourced from a Secret/ConfigMap key. Absent => no env (existing apps render byte-identical). Ignored for kind=vm (VMs configure via cloud-init).",
"default": [],
"items": {
"type": "object",
"additionalProperties": false,
"required": ["name"],
"properties": {
"name": {
"type": "string",
"description": "Environment variable name (C_IDENTIFIER).",
"pattern": "^[A-Za-z_][A-Za-z0-9_]*$"
},
"value": {
"type": "string",
"description": "Literal value. Mutually exclusive with valueFrom."
},
"valueFrom": {
"type": "object",
"description": "Source the value from a Secret or ConfigMap key (exactly one of secretKeyRef/configMapKeyRef).",
"additionalProperties": false,
"minProperties": 1,
"maxProperties": 1,
"properties": {
"secretKeyRef": {
"type": "object",
"additionalProperties": false,
"required": ["name", "key"],
"properties": {
"name": { "type": "string", "minLength": 1 },
"key": { "type": "string", "minLength": 1 }
}
},
"configMapKeyRef": {
"type": "object",
"additionalProperties": false,
"required": ["name", "key"],
"properties": {
"name": { "type": "string", "minLength": 1 },
"key": { "type": "string", "minLength": 1 }
}
}
}
}
},
"oneOf": [
{ "required": ["value"], "not": { "required": ["valueFrom"] } },
{ "required": ["valueFrom"], "not": { "required": ["value"] } }
]
}
},
"envFrom": {
"type": "array",
"description": "Optional bulk env sources for the container kind. Mirrors the k8s EnvFromSource shape and is emitted verbatim onto the app container: each entry pulls ALL keys from a Secret or ConfigMap (exactly one of secretRef/configMapRef). Absent => no envFrom (existing apps render byte-identical). Ignored for kind=vm.",
"default": [],
"items": {
"type": "object",
"additionalProperties": false,
"minProperties": 1,
"maxProperties": 1,
"properties": {
"secretRef": {
"type": "object",
"additionalProperties": false,
"required": ["name"],
"properties": { "name": { "type": "string", "minLength": 1 } }
},
"configMapRef": {
"type": "object",
"additionalProperties": false,
"required": ["name"],
"properties": { "name": { "type": "string", "minLength": 1 } }
}
}
}
},
"lifecycle": {
"type": "object",
"description": "Optional container lifecycle hooks for the container kind (docs/13). Mirrors the k8s Lifecycle shape and is emitted verbatim onto the app container. Used for idempotent break-glass/bootstrap steps (e.g. gitea's postStart exec that seeds a local admin). Absent => no lifecycle (existing apps render byte-identical). Ignored for kind=vm.",
"additionalProperties": false,
"minProperties": 1,
"properties": {
"postStart": { "$ref": "#/$defs/execHandler" },
"preStop": { "$ref": "#/$defs/execHandler" }
}
},
"podSecurityContext": {
"type": "object",
"description": "Optional pod-level SecurityContext for the container kind (docs/13). Mirrors the k8s PodSecurityContext shape and is emitted verbatim onto the pod spec (spec.template.spec.securityContext). Its primary use is `fsGroup`, which makes a mounted Longhorn PVC group-writable by a non-root image (e.g. semaphore runs as UID 1001 and cannot otherwise write its config PVC). Absent => no pod securityContext (existing apps render byte-identical). Ignored for kind=vm.",
"additionalProperties": false,
"minProperties": 1,
"properties": {
"runAsUser": { "type": "integer", "description": "UID the container processes run as." },
"runAsGroup": { "type": "integer", "description": "Primary GID the container processes run as." },
"runAsNonRoot": { "type": "boolean", "description": "Require the container to run as a non-root UID (validated at admission)." },
"fsGroup": { "type": "integer", "description": "Supplemental group applied to mounted volumes so they are group-owned + group-writable (chowned by the kubelet), letting a non-root image write a PVC." },
"fsGroupChangePolicy": { "type": "string", "enum": ["Always", "OnRootMismatch"], "description": "When the kubelet recursively applies fsGroup ownership to a volume." },
"supplementalGroups": { "type": "array", "items": { "type": "integer" }, "description": "Additional GIDs added to the first process in each container." }
}
},
"exposure": {
"type": "string",
"description": "internal = LAN-only (internal DNS + Gateway on LAN); public = additionally edge port-forwarded 80/443 (docs/09; R-DNS/R-NET).",
"enum": ["internal", "public"],
"default": "internal"
},
"egress": {
"type": "string",
"description": "Egress class (docs/03; R-NET-05). internet = default WAN route; vpn = steered by Cilium Egress Gateway out the egress-gateway node's host PIA VPN tunnel, with a mandatory host kill-switch (ADR-0015; R-NET-06).",
"enum": ["internet", "vpn"],
"default": "internet"
},
"storage": {
"type": "array",
"description": "Persistent volumes. If it must survive a rebuild it lives here, never on ephemeral/hostPath (docs/04; R-STATE-01/02/04). A `longhorn` entry renders a dynamically provisioned PVC; a `bulk` entry renders a STATIC, pre-bound NFS PersistentVolume + its PVC over data that already exists outside the cluster.",
"default": [],
"items": {
"type": "object",
"additionalProperties": false,
"required": ["name", "class"],
"properties": {
"name": {
"type": "string",
"description": "Volume identifier within the service (e.g. config, media, db).",
"pattern": "^[a-z0-9]([-a-z0-9]*[a-z0-9])?$"
},
"size": {
"type": "string",
"description": "PVC request in Kubernetes quantity form (e.g. 5Gi). Required for the longhorn tier. For the bulk tier it is the STATIC PV's advertised capacity: NOTHING enforces it (the ZFS pool is the real limit), so state the dataset's current size so `kubectl get pv` reports something true. Omitted => the bulk PV/PVC carry no capacity, which the API rejects, so bulk in practice sets it.",
"pattern": "^[1-9][0-9]*(Mi|Gi|Ti)$"
},
"mountPath": {
"type": "string",
"description": "Optional explicit absolute container mount path for this volume. Default is /data/<name>. Set it when the app expects a FIXED on-disk layout the default cannot express (e.g. Gitea keeps all state directly under /data). Renderer honors this verbatim; omitting it preserves the /data/<name> default for back-compat (R-STATE-01).",
"pattern": "^/.*[^/]$"
},
"class": {
"type": "string",
"description": "longhorn = 3-replica block storage for app/config/DB state (R-STATE-02); bulk = large external media tier, NOT 3x replicated, backed by a STATIC NFS PersistentVolume the renderer emits (requires `export`).",
"enum": ["longhorn", "bulk"],
"default": "longhorn"
},
"export": {
"type": "string",
"description": "bulk ONLY: the absolute NFS export path on the bulk server (e.g. /mnt/Storage/plex). The SERVER is NOT named here - it is the ${BULK_NFS_SERVER} cluster-vars knob, so the same ServiceDefinition promotes across environments (R-ENV-02). Required whenever class is bulk; forbidden otherwise.",
"pattern": "^/.*[^/]$"
},
"subPath": {
"type": "string",
"description": "Optional relative path WITHIN the volume to mount instead of its root. NOTE for hardlink/atomic-move workloads (Sonarr/Radarr): two subPath mounts of the same export are two separate mounts, so link() across them fails EXDEV. Mount the common ancestor once instead.",
"pattern": "^[^/].*[^/]$"
},
"readOnly": {
"type": "boolean",
"description": "Mount this volume read-only. For a bulk volume this also sets the PersistentVolume's nfs.readOnly (the mount really is `ro`) and makes its access mode ReadOnlyMany instead of ReadWriteMany.",
"default": false
}
},
"allOf": [
{
"if": { "properties": { "class": { "const": "bulk" } }, "required": ["class"] },
"then": { "required": ["export"] },
"else": { "not": { "required": ["export"] } }
}
]
}
},
"dependencies": {
"type": "array",
"description": "Explicit east-west allow rules; everything else is denied (docs/03; R-NET-01/02). Empty => the service can reach only DNS.",
"default": [],
"items": {
"type": "object",
"additionalProperties": false,
"required": ["to", "port"],
"properties": {
"to": {
"type": "string",
"description": "Destination service name (another catalog entry or a co-located component such as this service's DB).",
"minLength": 1
},
"port": {
"type": "integer",
"description": "Destination TCP/UDP port.",
"minimum": 1,
"maximum": 65535
},
"protocol": {
"type": "string",
"enum": ["tcp", "udp"],
"default": "tcp"
}
}
}
},
"ingressRoutes": {
"type": "array",
"description": "Hostnames/ports to expose north-south; rendered to HTTPRoute + Certificate + DNS (docs/08, docs/09; R-DNS/R-TLS).",
"default": [],
"items": {
"type": "object",
"additionalProperties": false,
"required": ["host", "port"],
"properties": {
"host": {
"type": "string",
"description": "Hostname under the platform zone, as a single DNS label plus the ${BASE_DOMAIN} token — e.g. 'gitea.${BASE_DOMAIN}' (docs/09). ${BASE_DOMAIN} is the per-environment hostname suffix from clusters/base/cluster-vars.yaml, resolved by Flux postBuild substitution at reconcile time (dev: home.keithjackson.dev, prod: prod.home.keithjackson.dev). AUTHOR THE TOKEN, NOT A LITERAL: a literal suffix is what let prod be authored to serve dev's public hostnames out of the shared Cloudflare zone, and prod's external-dns then deleted dev's records (infra/PROD-PROVISIONING.md §10). The literal 'home.keithjackson.dev' form stays accepted so the L1 render fixtures — which are renderer INPUTS, not reconciled manifests — keep validating unchanged.",
"pattern": "^[a-z0-9]([-a-z0-9]*[a-z0-9])?\\.(\\$\\{BASE_DOMAIN\\}|home\\.keithjackson\\.dev)$"
},
"port": {
"type": "integer",
"description": "Backing Service port the route targets.",
"minimum": 1,
"maximum": 65535
}
}
}
},
"auth": {
"type": "object",
"description": "SSO integration (docs/15, ADR-0014; R-IDP-02/06). Default posture for any exposed UI is forward-auth unless it declares native OIDC (oidc) or a documented exception (native).",
"additionalProperties": false,
"required": ["mode"],
"properties": {
"mode": {
"type": "string",
"description": "oidc = app is a Keycloak OIDC client (own login); forward-auth = oauth2-proxy -> Keycloak at the Gateway; native = app manages its own auth (documented exception, e.g. Plex/Seerr, or Keycloak itself as the IdP); none = no auth (internal-only, discouraged).",
"enum": ["oidc", "forward-auth", "native", "none"],
"default": "forward-auth"
},
"groups": {
"type": "array",
"description": "Keycloak groups permitted access (R-IDP-06). Required and non-empty for oidc/forward-auth (least privilege: authenticated AND in an authorized group). N/A for native/none.",
"items": { "type": "string", "minLength": 1 },
"uniqueItems": true,
"default": []
}
}
},
"update": {
"type": "object",
"description": "Per-service automated-update policy (docs/11; R-UPD-01/02/03). The operator declares the policy once; an automation controller resolves it - no human/AI edits a tag.",
"additionalProperties": false,
"properties": {
"channel": {
"type": "string",
"description": "Release channel. stable only unless explicitly opted in (R-UPD-01).",
"default": "stable"
},
"range": {
"type": "string",
"description": "Acceptable versions: semver range or tag regex (feeds the Flux ImagePolicy)."
},
"cadence": {
"type": "string",
"description": "How often to check/apply (the schedule; scan interval)."
},
"autoRollbackOnFailedHealth": {
"type": "boolean",
"description": "Revert automatically on failed health gates (R-UPD-03).",
"default": true
},
"pin": {
"type": "string",
"description": "Operator freeze escape hatch: a fixed version that stops automation for this service."
}
}
},
"observability": {
"type": "object",
"description": "Observability intent; rendered to ServiceMonitor/PodMonitor + log scraping + dashboards + alerts (docs/10; R-OBS-01).",
"additionalProperties": false,
"properties": {
"metrics": {
"type": "boolean",
"description": "Whether the platform scrapes metrics for this service.",
"default": true
},
"slis": {
"type": "array",
"description": "Per-service alert intents (e.g. up). Each becomes at least one alert rule (R-OBS-01).",
"items": { "type": "string", "minLength": 1 },
"default": []
}
}
},
"backup": {
"type": "object",
"description": "Backup posture (docs/04; R-STATE-03). Irreplaceable state MUST be backed up off-cluster; bulk media may be 'reacquirable'.",
"additionalProperties": false,
"required": ["policy"],
"properties": {
"policy": {
"type": "string",
"description": "Reference to an off-cluster backup schedule, or the literal 'reacquirable' for bulk media that is not backed up 1:1.",
"minLength": 1
}
}
}
},
"allOf": [
{
"$comment": "kind determines which source field is required (docs/13). Absent kind defaults to container -> image.",
"if": {
"required": ["kind"],
"properties": { "kind": { "const": "vm" } }
},
"then": {
"properties": { "source": { "required": ["vmTemplate"] } }
},
"else": {
"properties": { "source": { "required": ["image"] } }
}
},
{
"$comment": "oidc/forward-auth require at least one authorized group (R-IDP-06; docs/15 acceptance gate: restricted to the allowed group).",
"if": {
"required": ["auth"],
"properties": {
"auth": {
"required": ["mode"],
"properties": { "mode": { "enum": ["oidc", "forward-auth"] } }
}
}
},
"then": {
"properties": {
"auth": {
"required": ["groups"],
"properties": { "groups": { "minItems": 1 } }
}
}
}
},
{
"$comment": "public exposure must not be unauthenticated: an explicit auth.mode 'none' is disallowed on public services (docs/13 v1.1, docs/15, R-IDP-02). Absent auth defaults to forward-auth at render time, so it is permitted here.",
"if": {
"required": ["exposure"],
"properties": { "exposure": { "const": "public" } }
},
"then": {
"properties": {
"auth": {
"properties": { "mode": { "not": { "const": "none" } } }
}
}
}
}
],
"$defs": {
"execHandler": {
"type": "object",
"description": "A container lifecycle hook. Only the exec form is supported (a command run inside the container); mirrors k8s LifecycleHandler.exec.",
"additionalProperties": false,
"required": ["exec"],
"properties": {
"exec": {
"type": "object",
"additionalProperties": false,
"required": ["command"],
"properties": {
"command": {
"type": "array",
"description": "Command line to execute inside the container. Not run in a shell unless the command itself invokes one.",
"minItems": 1,
"items": { "type": "string" }
}
}
}
}
}
}
}