jakman85andClaude Opus 4.8
3586769460
Adds the VM/CDI storage tier approved by the PM (RWX for live-migration): - platform/external-snapshotter (NEW, controllers pass): the cluster-singleton snapshot.storage.k8s.io control plane (CRDs + snapshot-controller in kube-system), pinned v8.2.0 (Kubernetes deps aligned to 1.31 since v8.1.0; this cluster is k3s v1.31.5). These CRDs were absent — only snapshots.longhorn.io and kubevirt.io snapshot CRDs existed. - platform/longhorn/volumesnapshotclass.yaml (NEW, platform-config): the driver.longhorn.io VolumeSnapshotClass (type: snap, in-cluster). - platform/kubevirt/cdi-storageprofiles.yaml (NEW, platform-config): StorageProfile longhorn -> cloneStrategy: snapshot; StorageProfile longhorn-vm -> snapshot clone + claimPropertySets RWX/Block. Non-degraded default virt class clears CDIDefaultStorageClassDegraded. - platform/longhorn/storageclasses.yaml: NEW longhorn-vm StorageClass — migratable: "true" (Longhorn's KubeVirt-live-migration knob; RWX/Block-only on 1.7.3), is-default-virt-class (NOT the k8s default; longhorn stays RWO default). - platform/kubevirt/kubevirt-cr.yaml: comment only — LiveMigration is GA on v1.9 (no feature gate); no cluster-wide evictionStrategy (would wedge RWO-disk drains); live-migration stays per-VM opt-in on a longhorn-vm disk. Co-Authored-By: Claude Opus 4.8 <[email protected]>
platform/longhorn
Longhorn — distributed replicated block storage across the 3 nodes.
- Governing ADR: ADR-0003 — Distributed storage: Longhorn (+ bulk tier + Velero)
- Spec: docs/04 — Storage & State
- Serves: R-STATE-01, R-STATE-02, R-STATE-04 (R-STATE-03 is Velero's, Phase 4)
- Owner: WS-STORAGE
What this is
A real, dev-appropriate Flux HelmRelease for Longhorn plus two StorageClasses:
longhorn— the cluster's only default StorageClass, 3 replicas with hard node anti-affinity (one replica per node). Durable home for app/config state, databases, and KubeVirt VM disks (via CDI). Replica count is the${LONGHORN_DEFAULT_REPLICA_COUNT}overlay knob (cluster-vars, =3 in dev).bulk— thin indirection for the external (outside-cluster) media tier: Plex library, download staging, arr handoff. STUB placeholder modelled as an external NFS mount; its backing is OQ-2 (unresolved) and will not provision until answered. Plex's metadata DB stays on the 3-replicalonghornclass regardless (docs/04, ADR-0003).
Requirement mapping
| Requirement | Summary | How this component serves it |
|---|---|---|
| R-STATE-01 | Stateful data survives destroy/rebuild of the workload | Volumes are Longhorn PVCs, independent of pod/VM lifecycle; reclaimPolicy: Delete on the class ties data to the PVC, not the workload. Destroy the pod/Deployment/VM → PVC + data persist → rebuild re-attaches. (Per-service rebuild test is the Phase-2 exit gate; T-1.5 proves node-level rebuild.) |
| R-STATE-02 | Persistent storage replicated across the 3 nodes | numberOfReplicas=3 on the default class + replicaSoftAntiAffinity: false (hard) → exactly one replica per node, so any single node loss keeps volumes available. Replicas live on the dedicated /var/lib/longhorn data disk. replicaAutoBalance: best-effort re-spreads on node return. |
| R-STATE-04 | Bulk media has a capacity strategy distinct from the replicated tier | Separate bulk StorageClass (NON-replicated, external NFS mount) keeps multi-TB media off the 3x tier; only Plex's small metadata DB uses longhorn. |
Dev-scoped choices (change for prod)
reclaimPolicy: Deleteonlonghorn— convenient in dev; prod may preferRetainfor accidental-delete protection (backups still cover DR — R-STATE-03).storageOverProvisioningPercentage: 100— no thin over-provisioning in dev (small disks); prod may raise it.storageReservedPercentageForDefaultDisk: 20/storageMinimalAvailablePercentage: 25— headroom guardrails tuned for small dev disks; revisit for prod disk sizes.- Replica count = 3 comes from
cluster-vars.longhorn_default_replica_count; both dev and prod are 3 today (3 nodes), so the value is identical — the knob exists so it is never forked per env.
Guardrails baked in (docs/04)
- Reserved disk headroom (never fill a node to 100%).
- Hard replica anti-affinity across nodes (no two replicas of a volume on one node).
- Single-zone anti-affinity relaxed (3 mini-PCs = one failure zone).
- Volume/disk health + degraded-replica alerting is WS-OBS's job (docs/10) — not configured here.
Node prerequisites (already handled, informational)
Each node presents a dedicated Longhorn data disk (/dev/vdb → ext4) mounted at
/var/lib/longhorn, and installs open-iscsi + nfs-common + kernel modules — all via
infra/cloud-init/user-data.yaml.tftpl. This component assumes that floor.
Contract (consumed by everyone with state)
Per roadmap/workstreams.md, WS-STORAGE exposes the longhorn (replicated) and bulk (media)
StorageClasses; WS-APPS/WS-VM/WS-OBS reference them from ServiceDefinitions. Do not place bulk
media on the replicated tier, or app DBs on the bulk tier.
Open decisions / TODO (later phases)
- OQ-2 (human): decide the
bulk-tier backing (NAS/NFS host + share, vs dedicated large disk). Until answered, thebulkclass is a non-functional placeholder (server/share= REPLACE-ME). Blocksbulkfinalize; does not block the core platform (docs/04). - Version pin (WS-STORAGE):
>=1.7.0 <1.8.0is a range; pin exactly at bring-up, Renovate maintains. - Phase 4 (WS-STORAGE): off-cluster backup target via
defaultSettings.backupTarget+ SOPS-encrypted credential secret, wired alongsideplatform/velero/(R-STATE-03).
Build check
kubectl kustomize platform/longhorn renders the namespace, HelmRepository, HelmRelease, and both
StorageClasses. (Structural only — not applied to a cluster here; the PM validates + deploys.)