# ADR-0003 — Distributed storage: Longhorn (+ bulk tier + Velero)
- Status: Accepted
- Date: 2026-08-15
- Serves: R-STATE-01, R-STATE-02, R-STATE-03, R-STATE-04
- Supersedes: —

## Context
Three nodes with **local disks** and no SAN must provide durable, replicated persistent storage so
any workload can be destroyed and rebuilt without data loss, tolerate a single node failure, and
support off-cluster backup/DR. Plex's multi-TB media library makes uniform 3× replication
impractical, requiring a distinct bulk tier.

## Decision
- Use **Longhorn** for distributed replicated block storage across the 3 nodes; default StorageClass
  is **3 replicas**. This is the durable home for app/config state and KubeVirt VM disks (via CDI).
- Define a separate **bulk media tier** for large, non-replicated data (Plex library, download
  staging). Its concrete backing (NAS/NFS vs. dedicated large disk vs. mixed) is decided under OQ-2;
  **Plex's metadata DB stays on 3-replica Longhorn** regardless.
- Use **Velero** (with Longhorn backups) to push volume + resource-state backups to an **off-cluster**
  S3-compatible/NFS target on a schedule, with a **tested restore** path (DR drill).

## Alternatives considered
- **Rook/Ceph** — more powerful but far heavier/operationally complex for 3 small nodes.
- **OpenEBS (Mayastor/other)** — viable; Longhorn chosen for simplicity, strong UI, snapshot/backup
  ergonomics, and fit with small clusters.
- **Uniform 3× replication of everything including media** — rejected: capacity-prohibitive on
  mini-PC local disks (R-STATE-04).
- **hostPath/local-only PVs for stateful apps** — rejected: node-bound, no replication, breaks the
  rebuild guarantee.

## Consequences
- Single-node loss keeps replicated data available; workloads rebuild by re-attaching PVCs
  (R-STATE-01/02).
- A two-tier storage story must be clearly documented so implementers don't accidentally place bulk
  media on the replicated tier or app DBs on the bulk tier ([04](../docs/04-storage-and-state.md)).
- Off-cluster backup target + its credentials/resilience are prerequisites (OQ-2); DR drill is part
  of acceptance.
- Longhorn's own capacity/health must be monitored and alerted ([10](../docs/10-observability-and-alerting.md)).