0 Fork 0

Files

T

jakman85andClaude Opus 4.8 d1248e6778 feat(backup): wire Velero + CNPG Barman to the live MinIO S3 target

Author the GitOps side of off-cluster backups against the on-Proxmox MinIO
(http://192.168.1.9:9000, path-style; buckets velero + cnpg).

Velero (R-STATE-03):
- HelmRelease (chart 8.x / Velero 1.15) + velero-plugin-for-aws init container +
  node-agent kopia File System Backup (deployNodeAgent, uploaderType=kopia,
  snapshotsEnabled=false). BSL default -> bucket velero, s3ForcePathStyle.
- Two daily Schedules: `daily` backs up the whole cluster's resources+PVC data
  (defaultVolumesToFsBackup opt-out) EXCEPT monitoring+unifi; `daily-resources-only`
  captures monitoring+unifi RESOURCES only (Prometheus TSDB, Loki 72h logs, and the
  ~7.4GiB UniFi VM disk are reconstructable — excluded from volume backup).
- velero-cloud-credentials SOPS Secret (cloud INI key, ns velero).
- Own non-gating Flux tier clusters/base/backup.yaml (dependsOn platform-config,
  wait:false) + clusters/dev SOPS decryption patch — mirrors headlamp.

CNPG native Barman PITR (R-IDP-05 / R-STATE-03):
- keycloak-db, nextcloud-db, semaphore-db gain spec.backup.barmanObjectStore
  (s3://cnpg/<cluster>, endpointURL MinIO, gzip WAL+data, 30d retention) + a daily
  ScheduledBackup (6-field cron, immediate) each.
- Per-namespace scoped-key SOPS Secrets (<db>-backup-credentials), same-ns only.
- render.py: emit-waves now recognizes <svc>-db-backup-credentials.sops.yaml as a
  DB-wave sidecar (+ self-test); wave-0 kustomization regenerated.

Creds mirror platform/backup/minio-credentials.sops.yaml (scoped key k3s-backup),
SOPS round-trip verified with the cluster age key. gate: green.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

2026-08-28 22:37:17 -05:00

2.8 KiB

Raw Blame History

platform/velero

Velero — off-cluster backup and DR of Kubernetes resource state + PVC data.

Governing ADR: ADR-0003 — Longhorn (+ bulk tier + Velero)
Spec: docs/04 — Storage & State
Serves: R-STATE-03
Owner: WS-STORAGE

What this is (LIVE)

Flux HelmRelease (Velero chart 8.x / Velero 1.15.x) wired to the on-Proxmox MinIO S3 target:

Endpoint: http://192.168.1.9:9000 (plain HTTP, path-style, region us-east-1 placeholder)
BucketStorageLocation default: provider aws, bucket velero, s3ForcePathStyle=true.
AWS plugin (velero/velero-plugin-for-aws) loaded as an init container.
File System Backup (kopia): deployNodeAgent=true, uploaderType=kopia, snapshotsEnabled=false — storage-agnostic PVC backup, no CSI snapshot / data-mover.
Credentials: velero-cloud-credentials — a SOPS Secret (velero-cloud.sops.yaml) holding the cloud INI key, the scoped MinIO key limited to the velero+cnpg buckets. Kept in sync with the staging source of truth platform/backup/minio-credentials.sops.yaml and the Proxmox host ansible-vault.

Reconciled by its own non-gating Flux Kustomization clusters/base/backup.yaml (dependsOn: platform-config, wait:false) — a backup controller must never gate the platform graph.

Backup scope (what's included vs excluded)

Two daily Schedules (cadence ${VELERO_BACKUP_SCHEDULE}, retention ${VELERO_RETENTION}):

Schedule	Namespaces	Volumes (kopia FSB)
`daily`	ALL except `monitoring`, `unifi`	all PVC data (`defaultVolumesToFsBackup:true`, opt-out)
`daily-resources-only`	`monitoring`, `unifi`	none (`defaultVolumesToFsBackup:false`) — resources only

Excluded volume data (reconstructable, per the sizing decision):

monitoring — Prometheus TSDB PVC (ephemeral) + Loki 72h-log PVC. Grafana/Alertmanager are emptyDir (no PVC); Grafana dashboards are GitOps-provisioned.
unifi — the ~7.4 GiB KubeVirt VM disk (huge; UniFi is protected by its own config export).

Their Kubernetes resources are still captured (by daily-resources-only) to aid DR.

Included (the irreplaceable ~6 GiB): every other namespace's PVC data via kopia — Nextcloud data, Gitea, Plex/*arr config, PiHole, and the CNPG Postgres PVCs. (CNPG DBs additionally have native Barman PITR to the cnpg bucket — see the DB manifests.)

Post-deploy (PM)

velero backup-location get → default Available.
velero backup create --from-schedule daily → objects land under s3://velero/.
DR drill (R-STATE-03, Phase-4 gate): velero restore create --from-backup <name> into a scratch namespace and verify PVC data restores via the node-agent.

2.8 KiB Raw Blame History

platform/velero

What this is (LIVE)

Backup scope (what's included vs excluded)

Post-deploy (PM)

2.8 KiB

Raw Blame History