Files
home-network-k3s/decisions/adr-0003-storage-longhorn.md
jakman85andClaude Opus 5.5 250fb54154 docs: bring the specification in line with the accepted implementation
The operator ruled the implementation repo's main (50c8719) the latest
and accepted state. Sync the spec to it.

Accepted changes are reflected: one bare-metal environment, TrueNAS bulk
storage, R2 off-cluster backup, CloudNativePG, single-node VPN gateway
with Cilium fail-closed, inbound WireGuard, per-app auth sidecars,
Renovate, the CI gate as approver, and the pm-agent. New ADR-0017..0025;
ADR-0016 corrected and accepted; R-* IDs unchanged, eight amended.

Where the implementation has simply not met a requirement, the
requirement stands and the shortfall is recorded in docs/16-known-gaps.md.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-08 22:46:00 -06:00

39 lines
2.3 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# ADR-0003 — Distributed storage: Longhorn (+ bulk tier + Velero)
- Status: Accepted — Longhorn stands (backup and bulk-backing clauses superseded by [ADR-0018](adr-0018-storage-backup-truenas-r2.md))
- Date: 2026-08-15
- Serves: R-STATE-01, R-STATE-02, R-STATE-03, R-STATE-04
- Supersedes: —
## Context
Three nodes with **local disks** and no SAN must provide durable, replicated persistent storage so
any workload can be destroyed and rebuilt without data loss, tolerate a single node failure, and
support off-cluster backup/DR. Plex's multi-TB media library makes uniform 3× replication
impractical, requiring a distinct bulk tier.
## Decision
- Use **Longhorn** for distributed replicated block storage across the 3 nodes; default StorageClass
is **3 replicas**. This is the durable home for app/config state and KubeVirt VM disks (via CDI).
- Define a separate **bulk media tier** for large, non-replicated data (Plex library, download
staging). Its concrete backing (NAS/NFS vs. dedicated large disk vs. mixed) is decided under OQ-2;
**Plex's metadata DB stays on 3-replica Longhorn** regardless.
- Use **Velero** (with Longhorn backups) to push volume + resource-state backups to an **off-cluster**
S3-compatible/NFS target on a schedule, with a **tested restore** path (DR drill).
## Alternatives considered
- **Rook/Ceph** — more powerful but far heavier/operationally complex for 3 small nodes.
- **OpenEBS (Mayastor/other)** — viable; Longhorn chosen for simplicity, strong UI, snapshot/backup
ergonomics, and fit with small clusters.
- **Uniform 3× replication of everything including media** — rejected: capacity-prohibitive on
mini-PC local disks (R-STATE-04).
- **hostPath/local-only PVs for stateful apps** — rejected: node-bound, no replication, breaks the
rebuild guarantee.
## Consequences
- Single-node loss keeps replicated data available; workloads rebuild by re-attaching PVCs
(R-STATE-01/02).
- A two-tier storage story must be clearly documented so implementers don't accidentally place bulk
media on the replicated tier or app DBs on the bulk tier ([04](../docs/04-storage-and-state.md)).
- Off-cluster backup target + its credentials/resilience are prerequisites (OQ-2); DR drill is part
of acceptance.
- Longhorn's own capacity/health must be monitored and alerted ([10](../docs/10-observability-and-alerting.md)).