jakman85andClaude Opus 4.8
cbc2301b30
Text-only cleanup after the upgrade campaign. Updates stale version strings, prose, and comments to the verified current facts; no Kubernetes behavior or version pins changed except the kubectl toolchain bump (below). Verified facts applied: k3s v1.36.4+k3s1, Debian 13 trixie, Cilium 1.20.1 / Gateway API v1.6.1, Longhorn 1.12.1, node RAM 24 GiB, dedicated virtio2 SSD etcd disk per node, Velero chart 12.x / v1.18, Grafana 13.2.0. - Version strings across infra/ and platform/ READMEs + comments - kubectl toolchain 1.31.4 -> 1.36.4 (devcontainer/ci/conformance versions.env; 1.31 vs a 1.36 cluster exceeded the +-1 skew) - Retired-single-apps-Kustomization refs -> apps-wave-0/1/2 + apps-vms with per-wave SOPS decryption (flux/clusters docs + bootstrap.sh) - COMPLETED/historical banners on the three executed plan docs - Resolved TODO(WS-SEC) markers -> Renovate-maintained ranges - Appended dated DECISIONS-LOG entry (2026-08-30) superseding the 2026-08-26 LIVE STATE line Left as revalidation items (not blind-edited): external-snapshotter v8.2.0 vs 1.36, Cilium ext_authz/forward-auth relevance under 1.20.1, conformance ImageVolume feature-gate assertions under 1.36. Co-Authored-By: Claude Opus 4.8 <[email protected]>
75 lines
4.7 KiB
Markdown
75 lines
4.7 KiB
Markdown
# platform/longhorn
|
|
|
|
**Longhorn** — distributed replicated block storage across the 3 nodes.
|
|
|
|
- **Governing ADR:** [ADR-0003 — Distributed storage: Longhorn (+ bulk tier + Velero)](../../../home-network-k3s/decisions/adr-0003-storage-longhorn.md)
|
|
- **Spec:** [docs/04 — Storage & State](../../../home-network-k3s/docs/04-storage-and-state.md)
|
|
- **Serves:** R-STATE-01, R-STATE-02, R-STATE-04 (R-STATE-03 is Velero's, Phase 4)
|
|
- **Owner:** WS-STORAGE
|
|
|
|
## What this is
|
|
|
|
A real, dev-appropriate Flux `HelmRelease` for Longhorn plus two `StorageClass`es:
|
|
|
|
- **`longhorn`** — the cluster's **only default** StorageClass, **3 replicas** with **hard node
|
|
anti-affinity** (one replica per node). Durable home for app/config state, databases, and
|
|
KubeVirt VM disks (via CDI). Replica count is the `${LONGHORN_DEFAULT_REPLICA_COUNT}` overlay
|
|
knob (cluster-vars, =3 in dev).
|
|
- **`bulk`** — thin indirection for the **external** (outside-cluster) media tier: Plex library,
|
|
download staging, arr handoff. **STUB placeholder** modelled as an external NFS mount; its backing
|
|
is **OQ-2 (unresolved)** and will not provision until answered. Plex's metadata DB stays on the
|
|
3-replica `longhorn` class regardless (docs/04, ADR-0003).
|
|
|
|
## Requirement mapping
|
|
|
|
| Requirement | Summary | How this component serves it |
|
|
|---|---|---|
|
|
| **R-STATE-01** | Stateful data survives destroy/rebuild of the workload | Volumes are Longhorn PVCs, independent of pod/VM lifecycle; `reclaimPolicy: Delete` on the class ties data to the PVC, not the workload. Destroy the pod/Deployment/VM → PVC + data persist → rebuild re-attaches. (Per-service rebuild test is the Phase-2 exit gate; T-1.5 proves node-level rebuild.) |
|
|
| **R-STATE-02** | Persistent storage replicated across the 3 nodes | `numberOfReplicas=3` on the default class + `replicaSoftAntiAffinity: false` (hard) → exactly one replica per node, so any single node loss keeps volumes available. Replicas live on the dedicated `/var/lib/longhorn` data disk. `replicaAutoBalance: best-effort` re-spreads on node return. |
|
|
| **R-STATE-04** | Bulk media has a capacity strategy distinct from the replicated tier | Separate `bulk` StorageClass (NON-replicated, external NFS mount) keeps multi-TB media off the 3x tier; only Plex's small metadata DB uses `longhorn`. |
|
|
|
|
## Dev-scoped choices (change for prod)
|
|
|
|
- **`reclaimPolicy: Delete`** on `longhorn` — convenient in dev; prod may prefer `Retain` for
|
|
accidental-delete protection (backups still cover DR — R-STATE-03).
|
|
- **`storageOverProvisioningPercentage: 100`** — no thin over-provisioning in dev (small disks);
|
|
prod may raise it.
|
|
- **`storageReservedPercentageForDefaultDisk: 20` / `storageMinimalAvailablePercentage: 25`** —
|
|
headroom guardrails tuned for small dev disks; revisit for prod disk sizes.
|
|
- **Replica count = 3** comes from `cluster-vars.longhorn_default_replica_count`; both dev and prod
|
|
are 3 today (3 nodes), so the value is identical — the knob exists so it is never forked per env.
|
|
|
|
## Guardrails baked in (docs/04)
|
|
|
|
- Reserved disk headroom (never fill a node to 100%).
|
|
- Hard replica anti-affinity across nodes (no two replicas of a volume on one node).
|
|
- Single-zone anti-affinity relaxed (3 mini-PCs = one failure zone).
|
|
- Volume/disk health + degraded-replica alerting is WS-OBS's job (docs/10) — not configured here.
|
|
|
|
## Node prerequisites (already handled, informational)
|
|
|
|
Each node presents a dedicated Longhorn data disk (`/dev/vdb` → ext4) mounted at
|
|
`/var/lib/longhorn`, and installs `open-iscsi` + `nfs-common` + kernel modules — all via
|
|
`infra/cloud-init/user-data.yaml.tftpl`. This component assumes that floor.
|
|
|
|
## Contract (consumed by everyone with state)
|
|
|
|
Per `roadmap/workstreams.md`, WS-STORAGE exposes the `longhorn` (replicated) and `bulk` (media)
|
|
StorageClasses; WS-APPS/WS-VM/WS-OBS reference them from ServiceDefinitions. **Do not place bulk
|
|
media on the replicated tier, or app DBs on the bulk tier.**
|
|
|
|
## Open decisions / TODO (later phases)
|
|
|
|
- **OQ-2 (human):** decide the `bulk`-tier backing (NAS/NFS host + share, vs dedicated large disk).
|
|
Until answered, the `bulk` class is a non-functional placeholder (`server`/`share` = REPLACE-ME).
|
|
Blocks `bulk` finalize; does **not** block the core platform (docs/04).
|
|
- **Version pin (WS-STORAGE):** `>=1.12.0 <1.13.0` is a range; pin exactly at bring-up, Renovate
|
|
maintains.
|
|
- **Phase 4 (WS-STORAGE):** off-cluster backup target via `defaultSettings.backupTarget` +
|
|
SOPS-encrypted credential secret, wired alongside `platform/velero/` (R-STATE-03).
|
|
|
|
## Build check
|
|
|
|
`kubectl kustomize platform/longhorn` renders the namespace, HelmRepository, HelmRelease, and both
|
|
StorageClasses. (Structural only — not applied to a cluster here; the PM validates + deploys.)
|