jakman85andClaude Opus 5
161430d915
The previous commit created the two halves. This one wires the choice, so dev renders MinIO
credentials and prod renders R2 credentials from the SAME base manifests (R-ENV-02: the
overlay patch is the entire difference between environments; no base manifest is forked).
cluster-vars BACKUP_CREDENTIALS_SUFFIX base "" | dev "" | prod "-prod"
velero HelmRelease credentials.existingSecret: velero-cloud-credentials${BACKUP_CREDENTIALS_SUFFIX}
3x CNPG Cluster s3Credentials.*.name: <db>-backup-credentials${BACKUP_CREDENTIALS_SUFFIX}
ONE SUFFIX, NOT FOUR SECRET-NAME KNOBS. Four independent knobs can be updated three-quarters
of the way, leaving one consumer reading the OTHER environment's key with nothing rendering
wrong — and StrictPostBuildSubstitutions cannot catch that: strict mode fails on an UNDEFINED
variable, never on a defined-but-stale one. With one suffix the four move together or not at
all. The cost is that the rendered Secret name is not greppable from the consumer, so both
kustomizations spell the two names out in full beside the resources.
THE BASE DEFAULT IS "" AND THAT IS LOAD-BEARING. It renders the UNSUFFIXED names — objects
that exist in every render, and that dev is running today. The alternative failure mode is
strictly worse than the shared-Secret state this replaces: velero's pod hangs in
ContainerCreating forever when `existingSecret` names a missing Secret (the volume never
mounts), and a CNPG Cluster whose s3Credentials Secret is absent stops archiving WAL — which
under archive_mode=on is a filling pgdata PVC, not a missing backup. So an unresolvable
default breaks the referencing cluster HARDER than a wrong key does. Both overlays state the
value explicitly anyway, so neither inherits it silently.
Prod's overlay change is a cluster-vars op only. The velero Flux Kustomization patch is
untouched (it stays decryption-only, and velero stays suspended on prod).
VERIFIED — `kubectl kustomize --load-restrictor LoadRestrictionsNone` on clusters/dev and
clusters/prod for the cluster-vars values, then on each of the four affected tiers (platform,
platform-config, platform/velero, apps/wave-0) piped through `flux envsubst --strict` with
that environment's variables:
strict envsubst PASSES for all 8 (env x tier) combinations
dev velero existingSecret velero-cloud-credentials
dev keycloak/nextcloud/semaphore-db <db>-backup-credentials (both key refs each)
prod velero existingSecret velero-cloud-credentials-prod
prod keycloak/nextcloud/semaphore-db <db>-backup-credentials-prod (both key refs each)
both all EIGHT Secret objects present in the rendered output (dev and prod alike)
no `${...}` survives in any rendered tier except gitea's three DELIBERATE `$${...}` shell
escapes in apps/wave-0, which are pre-existing and unrelated.
Read alongside the endpoints the same overlays already set, the two environments are now
isolated on credential AND target:
dev velero s3://velero (root) + cnpg s3://cnpg @ http://192.168.1.9:9000 region us-east-1
prod velero k3s-prod-backup/velero + barman prefix @ https://<acct>.r2.cloudflarestorage.com region auto
Co-Authored-By: Claude Opus 5 <[email protected]>
124 lines
6.1 KiB
YAML
124 lines
6.1 KiB
YAML
---
|
|
# Nextcloud's PostgreSQL — a CloudNativePG Cluster (CR of the CNPG CRDs from platform/cnpg), modeled
|
|
# on platform/keycloak/postgres-cluster.yaml. This is the first CNPG-DB in apps/ and establishes the
|
|
# reusable pattern for later DB-backed services (semaphore, unifi, ...).
|
|
#
|
|
# WHY A DEDICATED `nextcloud-db` NAMESPACE (not co-located in `nextcloud`):
|
|
# The app->db edge in catalog/nextcloud.yaml (`dependencies: to: nextcloud-db`) compiles to a
|
|
# CiliumNetworkPolicy whose egress selector pins BOTH the destination pod label
|
|
# (app.kubernetes.io/name=nextcloud-db) AND its namespace (k8s:io.kubernetes.pod.namespace=
|
|
# nextcloud-db) — cross-namespace Cilium selectors must name the namespace (catalog/render.py
|
|
# r_dependency_policies). So the DB MUST live in a namespace literally named `nextcloud-db` and its
|
|
# pods MUST carry app.kubernetes.io/name=nextcloud-db, injected below via inheritedMetadata.
|
|
#
|
|
# DEFAULT-DENY POSTURE — mirrors keycloak (platform/keycloak/namespace.yaml): this namespace is
|
|
# INTENTIONALLY not homelab.keithjackson.dev/app-namespace=true, so Kyverno does NOT generate the app
|
|
# default-deny baseline here and CNPG runs under Cilium's cluster-wide default-allow — the SAME posture
|
|
# the verified keycloak-db uses. The nextcloud SIDE keeps its default-deny; its rendered egress-allow
|
|
# opens ONLY nextcloud -> nextcloud-db:5432, and the DB side's default-allow accepts that reciprocal
|
|
# ingress. No app-namespace policy (auth-mode/host-namespace/secret-provenance) is weakened.
|
|
apiVersion: v1
|
|
kind: Namespace
|
|
metadata:
|
|
name: nextcloud-db
|
|
labels:
|
|
# Match the Cilium egress selector's namespace pin; NOT an app-namespace (see header).
|
|
app.kubernetes.io/name: nextcloud-db
|
|
---
|
|
# Owner credentials for the CNPG cluster. SOPS-encrypted (age); type basic-auth so CNPG's initdb
|
|
# consumes it as the `nextcloud` role's password instead of generating a random one — this lets the
|
|
# SAME password be mirrored into ns nextcloud (nextcloud-bootstrap/db-password) for the app's env,
|
|
# since a Pod cannot read a Secret in another namespace. Provided as apps/nextcloud-db-app.sops.yaml.
|
|
#
|
|
# CNPG Cluster (CR of postgresql.cnpg.io installed by platform/cnpg).
|
|
apiVersion: postgresql.cnpg.io/v1
|
|
kind: Cluster
|
|
metadata:
|
|
name: nextcloud-db
|
|
namespace: nextcloud-db
|
|
spec:
|
|
# Single Postgres instance on 3-replica Longhorn is acceptable at home scale (ADR-0014, same as
|
|
# keycloak-db). The storage is already 3-way replicated by Longhorn; raise to 3 for Postgres-level
|
|
# HA as a future hardening.
|
|
instances: 1
|
|
imageName: ghcr.io/cloudnative-pg/postgresql:16.4
|
|
# Stamp app.kubernetes.io/name=nextcloud-db onto every generated pod/service so the rendered
|
|
# nextcloud egress CiliumNetworkPolicy's toEndpoints selector matches the primary pod (R-NET-02).
|
|
inheritedMetadata:
|
|
labels:
|
|
app.kubernetes.io/name: nextcloud-db
|
|
storage:
|
|
storageClass: longhorn
|
|
size: 5Gi
|
|
bootstrap:
|
|
initdb:
|
|
database: nextcloud
|
|
owner: nextcloud
|
|
# Use the pre-provisioned owner secret (basic-auth) so its password is known + reusable by the
|
|
# app in the other namespace, instead of the random CNPG-generated one.
|
|
secret:
|
|
name: nextcloud-db-app
|
|
# Emit Postgres metrics so Prometheus scrapes DB health (R-OBS-01). The CNPG operator creates the
|
|
# PodMonitor at runtime (CRD present from Phase-4 monitoring).
|
|
monitoring:
|
|
enablePodMonitor: true
|
|
resources:
|
|
requests:
|
|
cpu: 100m
|
|
memory: 256Mi
|
|
limits:
|
|
memory: 512Mi
|
|
# --- Off-cluster DR backup (R-STATE-03) — CNPG-native Barman → object storage ---
|
|
# Continuous WAL archiving + periodic base backups = proper Postgres PITR (realises the SD's
|
|
# backup.policy daily-offcluster). CNPG appends serverName (defaults to the Cluster name) to
|
|
# destinationPath, so objects land under ${CNPG_BACKUP_PATH}/nextcloud-db/ — dev
|
|
# s3://cnpg/nextcloud-db/ on the on-Proxmox MinIO, prod
|
|
# s3://k3s-prod-backup/barman/nextcloud-db/ on Cloudflare R2. Both endpointURL and
|
|
# destinationPath are cluster-vars knobs (R-ENV-02), substituted by `apps-wave-0`.
|
|
# Credentials: PER-ENVIRONMENT (2026-09-07). dev reads nextcloud-db-backup-credentials (the
|
|
# scoped MinIO key), prod reads nextcloud-db-backup-credentials-prod (the Cloudflare R2
|
|
# token). BOTH Secrets are applied to BOTH clusters by apps/wave-0/kustomization.yaml (a
|
|
# generated golden — render.py places the -prod sidecar in this DB's wave); the
|
|
# cluster-vars knob ${BACKUP_CREDENTIALS_SUFFIX} (dev "" | prod "-prod") chooses which
|
|
# one this Cluster reads. Both are in-namespace, which is all a pod can read. They were
|
|
# ONE shared Secret until the split, so giving prod its R2 token would have left dev
|
|
# signing MinIO requests with an R2 key id — 403 on every archive command, i.e. the
|
|
# disk-full clock below rather than a merely missing backup.
|
|
#
|
|
# archive_mode is ON, so a FAILING archive command is a disk-full clock, not a degraded backup:
|
|
# Postgres refuses to recycle a WAL segment it has not archived, and the pgdata PVC fills until
|
|
# the primary PANICs. Prod demonstrated this from bootstrap to 2026-09-07 with a placeholder
|
|
# endpoint. Treat ContinuousArchiving=False here as an availability incident.
|
|
backup:
|
|
retentionPolicy: "30d"
|
|
barmanObjectStore:
|
|
destinationPath: ${CNPG_BACKUP_PATH}
|
|
endpointURL: ${S3_BACKUP_ENDPOINT}
|
|
s3Credentials:
|
|
accessKeyId:
|
|
name: nextcloud-db-backup-credentials${BACKUP_CREDENTIALS_SUFFIX}
|
|
key: ACCESS_KEY_ID
|
|
secretAccessKey:
|
|
name: nextcloud-db-backup-credentials${BACKUP_CREDENTIALS_SUFFIX}
|
|
key: ACCESS_SECRET_KEY
|
|
wal:
|
|
compression: gzip
|
|
maxParallel: 2
|
|
data:
|
|
compression: gzip
|
|
---
|
|
# Daily base backup of nextcloud-db (WAL archived continuously by the barmanObjectStore above). CNPG
|
|
# cron is 6-field (sec min hour dom mon dow); immediate:true seeds the first base backup. 02:00 slot.
|
|
apiVersion: postgresql.cnpg.io/v1
|
|
kind: ScheduledBackup
|
|
metadata:
|
|
name: nextcloud-db-daily
|
|
namespace: nextcloud-db
|
|
spec:
|
|
immediate: true
|
|
schedule: "0 0 2 * * *"
|
|
backupOwnerReference: self
|
|
cluster:
|
|
name: nextcloud-db
|
|
method: barmanObjectStore
|