jakman85andClaude Opus 4.8
e8c555cfeb
Greenfield spec set for a GitOps-driven PaaS+IaaS home platform: vision & requirements (R-* IDs), architecture, 14 domain specs, 11 ADRs, roadmap (phases + workstreams), and AI-agent coordination (roles, protocol, definition-of-done, task board). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
39 lines
2.2 KiB
Markdown
39 lines
2.2 KiB
Markdown
# ADR-0003 — Distributed storage: Longhorn (+ bulk tier + Velero)
|
||
- Status: Accepted
|
||
- Date: 2026-08-15
|
||
- Serves: R-STATE-01, R-STATE-02, R-STATE-03, R-STATE-04
|
||
- Supersedes: —
|
||
|
||
## Context
|
||
Three nodes with **local disks** and no SAN must provide durable, replicated persistent storage so
|
||
any workload can be destroyed and rebuilt without data loss, tolerate a single node failure, and
|
||
support off-cluster backup/DR. Plex's multi-TB media library makes uniform 3× replication
|
||
impractical, requiring a distinct bulk tier.
|
||
|
||
## Decision
|
||
- Use **Longhorn** for distributed replicated block storage across the 3 nodes; default StorageClass
|
||
is **3 replicas**. This is the durable home for app/config state and KubeVirt VM disks (via CDI).
|
||
- Define a separate **bulk media tier** for large, non-replicated data (Plex library, download
|
||
staging). Its concrete backing (NAS/NFS vs. dedicated large disk vs. mixed) is decided under OQ-2;
|
||
**Plex's metadata DB stays on 3-replica Longhorn** regardless.
|
||
- Use **Velero** (with Longhorn backups) to push volume + resource-state backups to an **off-cluster**
|
||
S3-compatible/NFS target on a schedule, with a **tested restore** path (DR drill).
|
||
|
||
## Alternatives considered
|
||
- **Rook/Ceph** — more powerful but far heavier/operationally complex for 3 small nodes.
|
||
- **OpenEBS (Mayastor/other)** — viable; Longhorn chosen for simplicity, strong UI, snapshot/backup
|
||
ergonomics, and fit with small clusters.
|
||
- **Uniform 3× replication of everything including media** — rejected: capacity-prohibitive on
|
||
mini-PC local disks (R-STATE-04).
|
||
- **hostPath/local-only PVs for stateful apps** — rejected: node-bound, no replication, breaks the
|
||
rebuild guarantee.
|
||
|
||
## Consequences
|
||
- Single-node loss keeps replicated data available; workloads rebuild by re-attaching PVCs
|
||
(R-STATE-01/02).
|
||
- A two-tier storage story must be clearly documented so implementers don't accidentally place bulk
|
||
media on the replicated tier or app DBs on the bulk tier ([04](../docs/04-storage-and-state.md)).
|
||
- Off-cluster backup target + its credentials/resilience are prerequisites (OQ-2); DR drill is part
|
||
of acceptance.
|
||
- Longhorn's own capacity/health must be monitored and alerted ([10](../docs/10-observability-and-alerting.md)).
|