jakman85andClaude Opus 4.8
d1248e6778
Author the GitOps side of off-cluster backups against the on-Proxmox MinIO (http://192.168.1.9:9000, path-style; buckets velero + cnpg). Velero (R-STATE-03): - HelmRelease (chart 8.x / Velero 1.15) + velero-plugin-for-aws init container + node-agent kopia File System Backup (deployNodeAgent, uploaderType=kopia, snapshotsEnabled=false). BSL default -> bucket velero, s3ForcePathStyle. - Two daily Schedules: `daily` backs up the whole cluster's resources+PVC data (defaultVolumesToFsBackup opt-out) EXCEPT monitoring+unifi; `daily-resources-only` captures monitoring+unifi RESOURCES only (Prometheus TSDB, Loki 72h logs, and the ~7.4GiB UniFi VM disk are reconstructable — excluded from volume backup). - velero-cloud-credentials SOPS Secret (cloud INI key, ns velero). - Own non-gating Flux tier clusters/base/backup.yaml (dependsOn platform-config, wait:false) + clusters/dev SOPS decryption patch — mirrors headlamp. CNPG native Barman PITR (R-IDP-05 / R-STATE-03): - keycloak-db, nextcloud-db, semaphore-db gain spec.backup.barmanObjectStore (s3://cnpg/<cluster>, endpointURL MinIO, gzip WAL+data, 30d retention) + a daily ScheduledBackup (6-field cron, immediate) each. - Per-namespace scoped-key SOPS Secrets (<db>-backup-credentials), same-ns only. - render.py: emit-waves now recognizes <svc>-db-backup-credentials.sops.yaml as a DB-wave sidecar (+ self-test); wave-0 kustomization regenerated. Creds mirror platform/backup/minio-credentials.sops.yaml (scoped key k3s-backup), SOPS round-trip verified with the cluster age key. gate: green. Co-Authored-By: Claude Opus 4.8 <[email protected]>
53 lines
2.8 KiB
Markdown
53 lines
2.8 KiB
Markdown
# platform/velero
|
|
|
|
**Velero** — off-cluster backup and DR of Kubernetes resource state + PVC data.
|
|
|
|
- **Governing ADR:** [ADR-0003 — Longhorn (+ bulk tier + Velero)](../../../home-network-k3s/decisions/adr-0003-storage-longhorn.md)
|
|
- **Spec:** [docs/04 — Storage & State](../../../home-network-k3s/docs/04-storage-and-state.md)
|
|
- **Serves:** R-STATE-03
|
|
- **Owner:** WS-STORAGE
|
|
|
|
## What this is (LIVE)
|
|
|
|
Flux `HelmRelease` (Velero chart 8.x / Velero 1.15.x) wired to the on-Proxmox **MinIO** S3 target:
|
|
|
|
- **Endpoint:** `http://192.168.1.9:9000` (plain HTTP, **path-style**, region `us-east-1` placeholder)
|
|
- **BucketStorageLocation `default`:** provider `aws`, bucket `velero`, `s3ForcePathStyle=true`.
|
|
- **AWS plugin** (`velero/velero-plugin-for-aws`) loaded as an init container.
|
|
- **File System Backup (kopia):** `deployNodeAgent=true`, `uploaderType=kopia`, `snapshotsEnabled=false`
|
|
— storage-agnostic PVC backup, **no** CSI snapshot / data-mover.
|
|
- **Credentials:** `velero-cloud-credentials` — a SOPS Secret (`velero-cloud.sops.yaml`) holding the
|
|
`cloud` INI key, the **scoped** MinIO key limited to the `velero`+`cnpg` buckets. Kept in sync with
|
|
the staging source of truth `platform/backup/minio-credentials.sops.yaml` and the Proxmox host
|
|
ansible-vault.
|
|
|
|
Reconciled by its **own non-gating** Flux Kustomization `clusters/base/backup.yaml`
|
|
(`dependsOn: platform-config`, `wait:false`) — a backup controller must never gate the platform graph.
|
|
|
|
## Backup scope (what's included vs excluded)
|
|
|
|
Two daily `Schedule`s (cadence `${VELERO_BACKUP_SCHEDULE}`, retention `${VELERO_RETENTION}`):
|
|
|
|
| Schedule | Namespaces | Volumes (kopia FSB) |
|
|
|---|---|---|
|
|
| `daily` | ALL **except** `monitoring`, `unifi` | **all** PVC data (`defaultVolumesToFsBackup:true`, opt-out) |
|
|
| `daily-resources-only` | `monitoring`, `unifi` | **none** (`defaultVolumesToFsBackup:false`) — resources only |
|
|
|
|
**Excluded volume data (reconstructable, per the sizing decision):**
|
|
- `monitoring` — Prometheus TSDB PVC (ephemeral) + Loki 72h-log PVC. Grafana/Alertmanager are
|
|
`emptyDir` (no PVC); Grafana dashboards are GitOps-provisioned.
|
|
- `unifi` — the ~7.4 GiB KubeVirt VM disk (huge; UniFi is protected by its own config export).
|
|
|
|
Their **Kubernetes resources** are still captured (by `daily-resources-only`) to aid DR.
|
|
|
|
**Included (the irreplaceable ~6 GiB):** every other namespace's PVC data via kopia — Nextcloud data,
|
|
Gitea, Plex/*arr config, PiHole, and the CNPG Postgres PVCs. (CNPG DBs additionally have native Barman
|
|
PITR to the `cnpg` bucket — see the DB manifests.)
|
|
|
|
## Post-deploy (PM)
|
|
|
|
- `velero backup-location get` → `default` Available.
|
|
- `velero backup create --from-schedule daily` → objects land under `s3://velero/`.
|
|
- DR drill (R-STATE-03, Phase-4 gate): `velero restore create --from-backup <name>` into a scratch
|
|
namespace and verify PVC data restores via the node-agent.
|