Operator ruling: "the S3 backup location is the same as Dev." Prod's
S3_BACKUP_ENDPOINT is now http://192.168.1.9:9000 and the TBD-PROD-* placeholder
is retired. Set EXPLICITLY rather than by omitting the op — an omitted op would
inherit the same value silently, and a shared backup target must be visible in
the overlay that owns it.
SHARING THE ENDPOINT WAS NOT SAFE AS AUTHORED. Nothing about the backup PATHS
was per-environment, and enabling prod against dev's MinIO would have been the
2026-09-04 external-dns incident again, in a worse place:
VELERO — bucket `velero`, NO prefix, and both clusters run schedules named
`daily` / `daily-resources-only`. Velero SYNCS backups it finds in its BSL
into itself as Backup CRs and GARBAGE-COLLECTS them at TTL, so prod would
have adopted dev's backups and then deleted them on schedule.
CNPG — the worse one. destinationPath `s3://cnpg` with CNPG appending
serverName, which defaults to the CLUSTER NAME. Both environments have
keycloak-db, nextcloud-db and semaphore-db, so prod's Postgres would archive
WAL into the EXACT prefix dev's does. Two servers interleaving WAL into one
Barman stream CORRUPTS THE CHAIN and destroys PITR — and dev's restore is
PROVEN (R-STATE-03). Sharing an endpoint is fine; sharing a path is not.
Two knobs, authored once, environments differing only by scoped overrides
(R-ENV-02):
VELERO_BACKUP_PREFIX dev "" | prod "prod"
CNPG_BACKUP_PATH dev "s3://cnpg" | prod "s3://cnpg/prod"
THE BASE VALUES ARE DEV'S CURRENT LAYOUT ON PURPOSE. Dev's existing backups live
at the bucket root and at s3://cnpg; giving dev a prefix would ORPHAN the very
backups its proven restore depends on — the same reasoning that keeps dev's
EXTERNALDNS_TXT_OWNER_ID at `default`. Prod has no history to preserve.
*** PROD STILL HAS NO BACKUP COVERAGE — the endpoint is not reachable. ***
Measured from a throwaway pod INSIDE the prod cluster (the workstation proves
nothing about pod egress, and a workstation-side reading has already misled this
project once): curl http://192.168.1.9:9000/minio/health/live → connection timed
out after 15s. pfSense does not route prod's pod network to the dev LAN's MinIO.
OPERATOR ACTION REQUIRED. velero stays suspended; note apps-wave-0 applies GREEN
regardless — the token IS defined so strict substitution succeeds — while every
write fails at RUNTIME. A green apps-wave-0 does not mean backups work.
New `standup.sh s3check` step probes the endpoint from a throwaway in-cluster
pod and fails closed, with `--s3-endpoint <url>` to test a CANDIDATE before
authoring it into cluster-vars. That is how the unreachability above was
established rather than assumed.
VERIFIED: full CI gate PASS. flux-postbuild-substitution 254 -> 260 assertions,
and neither new knob appears in the dead-knob advisory list, so both are really
consumed. Rendered both overlays: all three CNPG destinationPaths and Velero's
prefix carry the tokens; dev resolves to its existing layout, prod to the nested
paths.
DELIVERY: prod reconciles the bootstrap Gitea's main at c8c1d29. This branch also
carries the DNS inversion, which must NOT reach prod. Delivering needs a
c8c1d29-based branch with only these commits — no urgency while the routing is
still closed.
Refs R-STATE-03, R-ENV-02, R-IDP-05.
Co-Authored-By: Claude Opus 5 <[email protected]>
115 lines
5.5 KiB
YAML
115 lines
5.5 KiB
YAML
---
|
|
# Nextcloud's PostgreSQL — a CloudNativePG Cluster (CR of the CNPG CRDs from platform/cnpg), modeled
|
|
# on platform/keycloak/postgres-cluster.yaml. This is the first CNPG-DB in apps/ and establishes the
|
|
# reusable pattern for later DB-backed services (semaphore, unifi, ...).
|
|
#
|
|
# WHY A DEDICATED `nextcloud-db` NAMESPACE (not co-located in `nextcloud`):
|
|
# The app->db edge in catalog/nextcloud.yaml (`dependencies: to: nextcloud-db`) compiles to a
|
|
# CiliumNetworkPolicy whose egress selector pins BOTH the destination pod label
|
|
# (app.kubernetes.io/name=nextcloud-db) AND its namespace (k8s:io.kubernetes.pod.namespace=
|
|
# nextcloud-db) — cross-namespace Cilium selectors must name the namespace (catalog/render.py
|
|
# r_dependency_policies). So the DB MUST live in a namespace literally named `nextcloud-db` and its
|
|
# pods MUST carry app.kubernetes.io/name=nextcloud-db, injected below via inheritedMetadata.
|
|
#
|
|
# DEFAULT-DENY POSTURE — mirrors keycloak (platform/keycloak/namespace.yaml): this namespace is
|
|
# INTENTIONALLY not homelab.keithjackson.dev/app-namespace=true, so Kyverno does NOT generate the app
|
|
# default-deny baseline here and CNPG runs under Cilium's cluster-wide default-allow — the SAME posture
|
|
# the verified keycloak-db uses. The nextcloud SIDE keeps its default-deny; its rendered egress-allow
|
|
# opens ONLY nextcloud -> nextcloud-db:5432, and the DB side's default-allow accepts that reciprocal
|
|
# ingress. No app-namespace policy (auth-mode/host-namespace/secret-provenance) is weakened.
|
|
apiVersion: v1
|
|
kind: Namespace
|
|
metadata:
|
|
name: nextcloud-db
|
|
labels:
|
|
# Match the Cilium egress selector's namespace pin; NOT an app-namespace (see header).
|
|
app.kubernetes.io/name: nextcloud-db
|
|
---
|
|
# Owner credentials for the CNPG cluster. SOPS-encrypted (age); type basic-auth so CNPG's initdb
|
|
# consumes it as the `nextcloud` role's password instead of generating a random one — this lets the
|
|
# SAME password be mirrored into ns nextcloud (nextcloud-bootstrap/db-password) for the app's env,
|
|
# since a Pod cannot read a Secret in another namespace. Provided as apps/nextcloud-db-app.sops.yaml.
|
|
#
|
|
# CNPG Cluster (CR of postgresql.cnpg.io installed by platform/cnpg).
|
|
apiVersion: postgresql.cnpg.io/v1
|
|
kind: Cluster
|
|
metadata:
|
|
name: nextcloud-db
|
|
namespace: nextcloud-db
|
|
spec:
|
|
# Single Postgres instance on 3-replica Longhorn is acceptable at home scale (ADR-0014, same as
|
|
# keycloak-db). The storage is already 3-way replicated by Longhorn; raise to 3 for Postgres-level
|
|
# HA as a future hardening.
|
|
instances: 1
|
|
imageName: ghcr.io/cloudnative-pg/postgresql:16.4
|
|
# Stamp app.kubernetes.io/name=nextcloud-db onto every generated pod/service so the rendered
|
|
# nextcloud egress CiliumNetworkPolicy's toEndpoints selector matches the primary pod (R-NET-02).
|
|
inheritedMetadata:
|
|
labels:
|
|
app.kubernetes.io/name: nextcloud-db
|
|
storage:
|
|
storageClass: longhorn
|
|
size: 5Gi
|
|
bootstrap:
|
|
initdb:
|
|
database: nextcloud
|
|
owner: nextcloud
|
|
# Use the pre-provisioned owner secret (basic-auth) so its password is known + reusable by the
|
|
# app in the other namespace, instead of the random CNPG-generated one.
|
|
secret:
|
|
name: nextcloud-db-app
|
|
# Emit Postgres metrics so Prometheus scrapes DB health (R-OBS-01). The CNPG operator creates the
|
|
# PodMonitor at runtime (CRD present from Phase-4 monitoring).
|
|
monitoring:
|
|
enablePodMonitor: true
|
|
resources:
|
|
requests:
|
|
cpu: 100m
|
|
memory: 256Mi
|
|
limits:
|
|
memory: 512Mi
|
|
# --- Off-cluster DR backup (R-STATE-03) — CNPG-native Barman → the on-Proxmox MinIO `cnpg` bucket ---
|
|
# Continuous WAL archiving + periodic base backups = proper Postgres PITR (realises the SD's
|
|
# backup.policy daily-offcluster). destinationPath is the bucket ROOT; CNPG appends serverName
|
|
# (defaults to the Cluster name), so objects land under s3://cnpg/nextcloud-db/. Path-style HTTP.
|
|
# Credentials: the scoped key from the in-namespace SOPS Secret nextcloud-db-backup-credentials
|
|
# (same-ns only), kept in sync with platform/backup/minio-credentials.sops.yaml.
|
|
# endpointURL is ${S3_BACKUP_ENDPOINT} (cluster-vars, R-ENV-02; dev http://192.168.1.9:9000),
|
|
# substituted by the `apps-wave-0` Flux Kustomization. Prod supplies its own off-cluster target.
|
|
backup:
|
|
retentionPolicy: "30d"
|
|
barmanObjectStore:
|
|
# PER-ENVIRONMENT PATH (cluster-vars). CNPG appends serverName, which defaults to the CLUSTER
|
|
# NAME — and both environments have a `nextcloud-db`. Sharing one MinIO with a shared path
|
|
# would interleave two Postgres servers' WAL into ONE Barman stream, corrupting the chain and
|
|
# destroying PITR for both. Dev keeps `s3://cnpg` (its existing, PROVEN backups); prod nests.
|
|
destinationPath: ${CNPG_BACKUP_PATH}
|
|
endpointURL: ${S3_BACKUP_ENDPOINT}
|
|
s3Credentials:
|
|
accessKeyId:
|
|
name: nextcloud-db-backup-credentials
|
|
key: ACCESS_KEY_ID
|
|
secretAccessKey:
|
|
name: nextcloud-db-backup-credentials
|
|
key: ACCESS_SECRET_KEY
|
|
wal:
|
|
compression: gzip
|
|
maxParallel: 2
|
|
data:
|
|
compression: gzip
|
|
---
|
|
# Daily base backup of nextcloud-db (WAL archived continuously by the barmanObjectStore above). CNPG
|
|
# cron is 6-field (sec min hour dom mon dow); immediate:true seeds the first base backup. 02:00 slot.
|
|
apiVersion: postgresql.cnpg.io/v1
|
|
kind: ScheduledBackup
|
|
metadata:
|
|
name: nextcloud-db-daily
|
|
namespace: nextcloud-db
|
|
spec:
|
|
immediate: true
|
|
schedule: "0 0 2 * * *"
|
|
backupOwnerReference: self
|
|
cluster:
|
|
name: nextcloud-db
|
|
method: barmanObjectStore
|