home-network-k3s/docs/13-service-catalog.md

# 13 — Service Catalog & the ServiceDefinition

Serves: **R-GITOPS-02**, ties together **R-NET-***, **R-STATE-***, **R-TLS-***, **R-DNS-***,
**R-OBS-***, **R-UPD-***. The catalog is the operator-facing onboarding surface.

## The ServiceDefinition abstraction

Onboarding a service is adding/editing one **ServiceDefinition** under `catalog/` in the
implementation repo. It captures *intent*; the platform compiles it into concrete resources
(Deployment or KubeVirt VM, PVCs, CiliumNetworkPolicies, egress routing, HTTPRoute, Certificate, DNS
record, ServiceMonitor + dashboards + alerts, and update policy). Operators declare **what**;
platform provides the **how**. This keeps every requirement wired from a single reviewable object.

**Platform-managed services** are the deliberate exception. A component with its own operator,
lifecycle or install flow is declared directly in git, under the same controls (default-deny, trusted
TLS, SSO, GitOps delivery through the merge gate): keycloak, Crafty, the command-center UIs, and VMs
with a custom install flow such as UniFi. `kind: vm` remains the catalog path for VMs.

### Fields (canonical schema — implementers render manifests from these)

```yaml
name: <string>                     # DNS label → <name>.home.keithjackson.dev
kind: container | vm               # PaaS (default) or IaaS via KubeVirt (R-PLAT-02/03)
source:
  image: <repo/image>              # container: image ref
  # or vmTemplate: <image/source>  # vm: CDI source image
exposure: internal | public        # LAN/VPN-only vs Cloudflare Tunnel (R-DNS-04/05, ADR-0016)
egress: internet | vpn             # egress class (R-NET-05); vpn ⇒ kill-switch (R-NET-06)
storage:
  - name: config
    size: <e.g. 5Gi>
    class: longhorn                # replicated app/config state (R-STATE-02)
  - name: media                    # optional bulk tier (R-STATE-04)
    class: bulk
    export: <path>                 # bulk only: the data's location on the bulk NFS server
dependencies:                      # explicit allow rules; everything else denied (R-NET-01/02)
  - to: <service>
    port: <n>
    protocol: tcp|udp
ingressRoutes:                     # hostnames/paths to expose (rendered to HTTPRoute + TLS + DNS)
  - host: <name>.home.keithjackson.dev
    port: <servicePort>
auth:                              # SSO integration (R-IDP-02; see 15-identity-and-access.md)
  mode: oidc | forward-auth | native | none
  # oidc         → app configured as a Keycloak OIDC client (does its own login)
  # forward-auth → per-app oauth2-proxy sidecar → Keycloak (UIs w/o OIDC; ADR-0022)
  # native       → app manages its own auth (e.g. Plex/Seerr via Plex) — documented exception
  # none         → no auth (internal-only, discouraged)
  groups: [ <keycloak-group>, ... ]  # groups allowed access (R-IDP-06)
update:
  channel: stable                  # never pre-release unless opted in (R-UPD-01)
  range: <semver range | tag regex># acceptable versions
  cadence: <schedule>              # how often to check/apply
  autoRollbackOnFailedHealth: true # R-UPD-03
  pin: <optional version>          # operator freeze escape hatch
observability:
  metrics: true
  slis: [ up, ... ]                # per-service alert intents (R-OBS-01)
backup:
  policy: <ref to backup schedule> # off-cluster (R-STATE-03); or 'reacquirable' for bulk media
```

> The schema is the **contract** between the operator surface and the implementation. Changing it is
> an interface change — coordinate per [`agents/coordination-protocol.md`](../agents/coordination-protocol.md).

The fields above are the core of the contract. The formal schema,
`schemas/servicedefinition.schema.json` in the
[implementation repo](https://gitea.home.keithjackson.dev/Home-Network/home-network-k3s-impl), is
canonical and adds workload-shaping fields for what the core cannot express: `env`, `envFrom`,
`lifecycle`, `podSecurityContext`, `availability`, `hardware` (a host capability class such as a
GPU, never a device path), `inClusterClients`, `auth.apiRoutes`, `observability.livenessWorkload`
and `storage[].{mountPath,subPath,readOnly}`.

## Default posture for every service (unless overridden)
- `kind: container`, `exposure: internal`, `egress: internet`.
- **Default-deny**: no `dependencies` ⇒ can reach only DNS ([03](03-networking-isolation-and-egress.md)).
- **Auth default `forward-auth`**: any exposed UI is protected by Keycloak SSO unless it declares
  native OIDC (`oidc`) or a documented exception (`native`) ([15](15-identity-and-access.md), R-IDP-02).
- Longhorn-backed config PVC; off-cluster backup; trusted TLS; auto-DNS; monitored; auto-updated on
  `stable`.

## The catalog (14 services)

Egress and dependencies below are **normative intent**; implementers render the exact policies.
"Bulk" = large media tier ([04](04-storage-and-state.md)). All get trusted TLS + auto-DNS + monitoring
+ auto-update unless noted. **Status** gives each service's delivery path: `catalog` (rendered from
its ServiceDefinition), `platform-managed` (declared directly in git), or `not onboarded`.

| # | Service | Status | kind | exposure | egress | Storage | Key dependencies (explicit allows) | Notes |
|---|---|---|---|---|---|---|---|---|
| 0 | **keycloak** | platform-managed | container | public | internet | db (PostgreSQL, ADR-0019): longhorn (irreplaceable → backed up) | → its DB; ← all SSO clients + oauth2-proxy | **IdP/SSO** ([15](15-identity-and-access.md)); **foundational**, onboarded **early**; DR-critical + break-glass path; hardened public exposure. Run by its operator; its catalog entry is declaration only and deploys nothing |
| 1 | **plex** | catalog | container | internal | **internet** | config: longhorn (small, replicated); media: **bulk** | ← seerr, ← sonarr, ← radarr; clients reach it | Largest service; media on bulk tier, metadata DB on Longhorn ([04](04-storage-and-state.md)); auth: **native** (Plex); web UI LAN/VPN-only, remote streaming per Cross-cutting notes |
| 2 | **sonarr** | catalog | container | internal | **vpn** | config: longhorn; media/downloads: bulk | → qbittorrent, → jackett, → plex:32400; ← seerr | VPN + kill-switch (R-NET-06) |
| 3 | **radarr** | catalog | container | internal | **vpn** | config: longhorn; media/downloads: bulk | → qbittorrent, → jackett, → plex:32400; ← seerr | VPN + kill-switch |
| 4 | **jackett** | catalog | container | internal | **vpn** | config: longhorn | ← sonarr, ← radarr | VPN + kill-switch (indexer) |
| 5 | **seerr** | catalog | container | public | internet | config: longhorn | → sonarr:8989, → radarr:7878, → plex:32400 | Request UI; talks to arr + Plex |
| 6 | **pihole** | catalog | container | internal | internet | config: longhorn | (provides DNS to cluster/LAN) | Also a platform DNS dependency ([09](09-dns-and-ingress.md)) |
| 7 | **nextcloud** | catalog | container | public | internet | data: longhorn (irreplaceable → backed up); legacy archive: bulk, read-only | → its DB | File sync; DR-critical data |
| 8 | **minecraft** (Crafty) | platform-managed | container | internal (game ports); public (web UI) | internet | world: longhorn (backed up) | — | Delivered as the Crafty controller, which runs the game servers; game ports on a LAN/VPN L4 address |
| 9 | **unifi** | platform-managed | vm | internal | internet | VM disk: longhorn (replicated, live-migratable) | — | UniFi OS Server in a KubeVirt VM, declared directly in git (custom install flow); browser and device traffic on a LAN/VPN L4 address, no Gateway route |
| 10 | **semaphore** | catalog | container | internal | internet | config/db: longhorn | → its DB | Ansible UI |
| 11 | **kasm** | not onboarded | container | internal | internet | config: longhorn | — | Kasm Workspaces (may need more resources; container-first, VM if required) |
| 12 | **gitea** | catalog | container | public | internet | repos/db: longhorn (irreplaceable → backed up) | — | Git hosting; **bootstrapping-critical** — self-hosts the platform's GitOps source + CI/CD; onboarded **early** (see below); DR-critical (off-cluster backup + mirror) |
| 13 | **qbittorrent** | catalog | container | internal | **vpn** | config: longhorn; downloads: **bulk** | ← sonarr, ← radarr | VPN + kill-switch; download client |

**VPN-class set (R-NET-05/06):** sonarr, radarr, jackett, qbittorrent — all egress via VPN with an
enforced kill-switch. Their inter-dependencies (arr → qbittorrent/jackett) are **east-west allows**
within the deny-by-default fabric; being VPN-class governs *egress*, not intra-cluster reachability.

**Dependency graph (explicit allows only; all else denied):**
```
seerr → sonarr, radarr, plex
sonarr → qbittorrent, jackett, plex
radarr → qbittorrent, jackett, plex
(everything → DNS only, by default)
```

The arr → plex allows exist because the bulk tier is NFS, which delivers no change notifications:
the arr stack tells Plex when new media lands.

**Delivery order (waves).** Catalog services reconcile in dependency-ordered waves computed from the
`dependencies` graph: a service's wave is its longest dependency path, so a database always lands
below its consumer. Each wave health-gates only itself and what depends on it, so a crashloop cannot
stall an earlier wave. VMs and leaf platform components sit on their own non-gating tiers: a slow
image import or a broken UI never holds up the app tier.

**SSO integration (`auth.mode`, R-IDP-02; see [15](15-identity-and-access.md)):**
- **oidc** (native Keycloak client): gitea, nextcloud, semaphore, kasm, grafana*, headlamp*
  (*platform UIs, not in the numbered app list).
- **forward-auth** (per-app oauth2-proxy sidecar → Keycloak, ADR-0022): pihole, sonarr, radarr,
  jackett, qbittorrent, Homepage*, and the Crafty web UI* (which adds its own local login; it has no
  OIDC).
- **native** (own auth, documented exceptions): plex, seerr (Plex accounts); minecraft game ports
  (Mojang/Microsoft); unifi (its own login, on its own L4 address).
- **none** (documented exception, R-IDP-02): the read-only runbooks at `/runbooks`, because gating
  them on Keycloak would lock the operator out of the Keycloak recovery procedures.
- **keycloak** itself is the IdP — its own admin login is local + MFA (the root of trust).
- Prometheus, Alertmanager and Hubble UI are not exposed; any future exposure MUST be gated.

**"…but not limited to":** the catalog is open-ended. New services follow the same ServiceDefinition
and inherit all guarantees automatically — that is the point of the abstraction.

## Contract clarifications (v1.1 — resolved 2026-08-16)

Points the schema left ambiguous, now pinned so the JSON Schema, Kyverno policies, and renderers agree:
- **`source` is exactly-one-of** — `image` for `kind: container`, `vmTemplate` for `kind: vm`
  (schema-enforced conditional).
- **`storage[].class: longhorn ⇒ size` is required** — enforced by an **L0 Kyverno policy** (a
  replicated volume with no size is a defect). `storage[].class: bulk ⇒ export` is required: a bulk
  volume is statically bound to data that already exists, and its `size` is advertised capacity only.
- **`backup` is a reference** — `backup: {policy: <ref>}` where `policy` is either the literal
  `reacquirable` (bulk/re-acquirable data) or a named off-cluster backup schedule. Cadence/retention
  live in the referenced schedule object, **not** inline in the ServiceDefinition.
- **`update.channel` is a free string defaulting to `stable`** — not a closed enum, so the sanctioned
  "opt-in to a non-stable channel" (R-UPD-01) is expressible per service.
- **`auth.groups` is required for `auth.mode ∈ {oidc, forward-auth}`** — access must be scoped to at
  least one Keycloak group (R-IDP-06); optional for `native`/`none`.
- **`exposure: public ⇒ auth.mode ≠ none`** — a public service may not be unauthenticated
  (docs/15, R-IDP-02); **schema-enforced** invariant.
- **Irreplaceable ⇒ off-cluster backup** — DR-critical services MUST NOT be `reacquirable`. The JSON
  Schema can't see criticality, so an **L0 Kyverno policy** applies the conservative rule: a service
  with any `longhorn` volume MUST NOT declare `reacquirable`.
- **`update` declares policy; it does not run updates.** Renovate acts on it through machine-authored
  changes at the merge gate ([11](11-automated-updates.md), ADR-0023).

## Cross-cutting notes
- **Plex media** is the R-STATE-04 case: keep the metadata DB on replicated Longhorn; keep the large
  library on the bulk tier; downloads stage on bulk and are handed off by the arr stack.
- **DR-critical data** (Nextcloud, Gitea, Minecraft worlds, PiHole config, service DBs) MUST have
  off-cluster backups ([04](04-storage-and-state.md)). The bulk tier is outside platform backup.
- **Public exposure** is through the **Cloudflare Tunnel** ([09](09-dns-and-ingress.md), ADR-0016).
  A service is public by catalog `exposure: public` (Keycloak, Seerr, Nextcloud, Gitea) or as a
  registered platform route (the Homepage apex, `/runbooks`, Grafana, the Crafty web UI, and the
  oauth2-proxy host). Everything else stays `internal`. **Plex** remote
  streaming uses plex.tv + an edge forward of tcp/32400 (not the tunnel), and its web UI is LAN-only.
  **Minecraft (crafty-mc)** and other L4 services are LAN/VPN-only.
- **Kasm** is resource-heavy and may be a candidate for `kind: vm` if container mode proves
  insufficient — the ServiceDefinition makes that a one-field change.
- **Gitea is bootstrapping-critical.** The platform self-hosts its git and CI/CD, so Gitea is brought
  online **early** (the first wave, with PiHole). Gitea + Gitea Actions are the GitOps source and CI
  ([05](05-gitops-and-delivery.md), [`../CONTRIBUTING.md`](../CONTRIBUTING.md)). Because a cluster
  loss would take its own git host with it, Gitea's repos/DB are backed up **off-cluster** and
  mirrored, so the platform can always be rebuilt from a source that outlives the cluster.

## Acceptance gates
- Each `catalog` service is fully wired from its ServiceDefinition alone (compute, storage, policy,
  egress, TLS, DNS, monitoring, updates) — no manual steps beyond merge. Each `platform-managed`
  service meets the same guarantees from its declaration in git.
- Default-deny holds; only listed dependencies work.
- VPN-class services egress via VPN with a proven kill-switch.
- Stateful services survive destroy/rebuild; DR-critical data restores from off-cluster backup.