421 lines
25 KiB
YAML
421 lines
25 KiB
YAML
---
|
|
# clusters/prod — DESIGN-ONLY overlay. NOT a build target until roadmap Phase 7 opens prod
|
|
# (spec roadmap/phases.md, AGENTS.md §6: "Never build directly on production").
|
|
#
|
|
# When Phase 7 is opened, `flux bootstrap ... --path=clusters/prod` points the prod cluster's Flux
|
|
# here. Like dev, this overlay adds NO resources — it ONLY patches cluster-vars with prod-scoped
|
|
# values (R-ENV-02). Promotion to prod is a change to THESE variables, never a base edit.
|
|
#
|
|
# *** PHASE-7 BLOCKER — READ BEFORE BOOTSTRAPPING PROD ***
|
|
# `clusters/base/platform.yaml` now carries `dependsOn: [flux-system]` (the 2026-09-01 substitution-
|
|
# race fix). That names the ROOT Kustomization, which clusters/dev supplies as `dev/sync.yaml`
|
|
# (path ./clusters/dev). THIS overlay has no sync.yaml yet, so as written a prod bootstrap would
|
|
# leave `platform` stuck on "dependency 'flux-system/flux-system' is not ready" FOREVER and nothing
|
|
# downstream would reconcile. Phase 7 MUST add `clusters/prod/sync.yaml` — a copy of dev/sync.yaml
|
|
# with `path: ./clusters/prod` — as part of opening this overlay. (Adding it now is deliberately NOT
|
|
# done here: prod is design-only and its root object is the prod bring-up owner's call.)
|
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
kind: Kustomization
|
|
resources:
|
|
- ../base
|
|
- sync.yaml # self-managing root Kustomization -> ./clusters/prod (REQUIRED: see below)
|
|
patches:
|
|
# --- SOPS decryption (operator decision 2026-09-01: prod uses the SAME age key as dev) ---
|
|
# Mirrors clusters/dev exactly, one stanza per Kustomization that carries encrypted Secrets.
|
|
# The `sops-age` Secret is created OUT OF BAND on the prod cluster
|
|
# (bootstrap/sops/install-age-key.sh) — referenced by NAME only, never inlined.
|
|
# Add a stanza here whenever `render.py --emit-waves` introduces a new deepest wave.
|
|
- patch: |-
|
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
|
kind: Kustomization
|
|
metadata:
|
|
name: platform
|
|
namespace: flux-system
|
|
spec:
|
|
decryption:
|
|
provider: sops
|
|
secretRef:
|
|
name: sops-age
|
|
- patch: |-
|
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
|
kind: Kustomization
|
|
metadata:
|
|
name: apps-wave-0
|
|
namespace: flux-system
|
|
spec:
|
|
decryption:
|
|
provider: sops
|
|
secretRef:
|
|
name: sops-age
|
|
- patch: |-
|
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
|
kind: Kustomization
|
|
metadata:
|
|
name: apps-wave-1
|
|
namespace: flux-system
|
|
spec:
|
|
decryption:
|
|
provider: sops
|
|
secretRef:
|
|
name: sops-age
|
|
- patch: |-
|
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
|
kind: Kustomization
|
|
metadata:
|
|
name: apps-wave-2
|
|
namespace: flux-system
|
|
spec:
|
|
decryption:
|
|
provider: sops
|
|
secretRef:
|
|
name: sops-age
|
|
- patch: |-
|
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
|
kind: Kustomization
|
|
metadata:
|
|
name: oauth2-proxy
|
|
namespace: flux-system
|
|
spec:
|
|
decryption:
|
|
provider: sops
|
|
secretRef:
|
|
name: sops-age
|
|
- patch: |-
|
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
|
kind: Kustomization
|
|
metadata:
|
|
name: headlamp
|
|
namespace: flux-system
|
|
spec:
|
|
decryption:
|
|
provider: sops
|
|
secretRef:
|
|
name: sops-age
|
|
- patch: |-
|
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
|
kind: Kustomization
|
|
metadata:
|
|
name: crafty
|
|
namespace: flux-system
|
|
spec:
|
|
decryption:
|
|
provider: sops
|
|
secretRef:
|
|
name: sops-age
|
|
- patch: |-
|
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
|
kind: Kustomization
|
|
metadata:
|
|
name: velero
|
|
namespace: flux-system
|
|
spec:
|
|
decryption:
|
|
provider: sops
|
|
secretRef:
|
|
name: sops-age
|
|
# --- GitOps source (operator decision 2026-09-01: prod tracks the SAME BRANCH as dev) ---
|
|
#
|
|
# ############################################################################################
|
|
# ### ###
|
|
# ### *** T E M P O R A R Y B O O T S T R A P S O U R C E — NOT PERMANENT *** ###
|
|
# ### ###
|
|
# ### The url below is a Gitea container running ON THE OPERATOR'S WORKSTATION, outside ###
|
|
# ### any cluster. It exists ONLY to get prod's first reconcile off the ground. It MUST ###
|
|
# ### be cut over to the in-cluster URL once Gitea runs on prod. If you are reading this ###
|
|
# ### and prod has been serving traffic for months, THE CUTOVER WAS NEVER DONE — that is ###
|
|
# ### a live single point of failure, not a settled design. See "CUTOVER" below. ###
|
|
# ### ###
|
|
# ############################################################################################
|
|
#
|
|
# WHY THE IN-CLUSTER URL CANNOT BE USED YET (blocker found 2026-09-01, still true; the
|
|
# unusable value was `https://gitea.home.keithjackson.dev/Home-Network/home-network-k3s-impl.git`).
|
|
# Two independent reasons, both consequences of the split-horizon design:
|
|
# 1. SELF-HOSTING CIRCULARITY. Gitea runs ON THE DEV CLUSTER. Prod syncing from it means prod's
|
|
# GitOps source lives inside the very cluster slated for decommission. Gitea must move to
|
|
# prod (or off-cluster) BEFORE dev is retired, or prod loses its source of truth.
|
|
# 2. SPLIT-HORIZON DNS SENDS IT TO THE WRONG PLACE. platform/coredns/coredns-custom.yaml
|
|
# answers the ENTIRE home.keithjackson.dev zone with ${GATEWAY_VIP}. On prod that is
|
|
# 192.168.2.20 — PROD's own gateway, where no Gitea is listening. Prod's Flux would resolve
|
|
# gitea.home.keithjackson.dev to itself and never reach a repository.
|
|
#
|
|
# WHAT THE TEMPORARY SOURCE IS (stood up 2026-09-04 on explicit operator direction):
|
|
# bootstrap/gitea/bootstrap-gitea.sh with HOST_NAT_IP=192.168.2.132 — a podman container on the
|
|
# operator's workstation, Gitea 1.22.6, listening on 192.168.2.132:3000. All three prod nodes
|
|
# reach it DIRECTLY on the LAN: no libvirt NAT hop (that network is retired) and no firewall
|
|
# change was needed. This is the same "local bootstrap source" pattern dev itself used before
|
|
# its Phase-6 cutover (docs/05 "Source-of-truth").
|
|
#
|
|
# PLAIN HTTP, AND WHY THAT IS ACCEPTED HERE: the bootstrap Gitea has no TLS, so there is no cert
|
|
# for Flux to trust and no caCertSecretRef to wire. Traffic is LAN-local between prod nodes and
|
|
# the workstation, the repo carries NO plaintext secrets (everything sensitive is SOPS-encrypted,
|
|
# .sops.yaml), and the window is the bring-up only. Do NOT extend this posture past cutover.
|
|
#
|
|
# NO secretRef, DELIBERATELY: the repo is PUBLIC on the bootstrap Gitea, so anonymous clone
|
|
# works and Flux needs no git credentials. VERIFIED 2026-09-04 by unauthenticated request:
|
|
# curl http://192.168.2.132:3000/api/v1/repos/Home-Network/home-network-k3s-impl -> "private":false
|
|
# git ls-remote http://192.168.2.132:3000/Home-Network/home-network-k3s-impl.git -> refs/heads/main
|
|
# If the GitRepository ever reports an auth failure that looks like a network problem, RE-CHECK
|
|
# THAT FIRST: a private repo here fails exactly that way.
|
|
#
|
|
# *** CUTOVER — the exit condition, mirroring dev's Phase-6 cutover (docs/05) ***
|
|
# WHEN: Gitea runs on prod (reconciled in-cluster) and answers gitea.home.keithjackson.dev
|
|
# through prod's own Gateway VIP with a publicly-trusted cert.
|
|
# DO: 1. push/mirror this repo into the prod in-cluster Gitea;
|
|
# 2. change the url below to
|
|
# https://gitea.home.keithjackson.dev/Home-Network/home-network-k3s-impl.git
|
|
# and DELETE this whole TEMPORARY block (dev's equivalent stanza, ~15 lines above in
|
|
# clusters/dev/kustomization.yaml, is what "done" looks like);
|
|
# 3. confirm `flux get sources git -n flux-system` reconciles the new revision;
|
|
# 4. only THEN stop the workstation container. Keep it stopped-not-deleted as recovery,
|
|
# exactly as dev retained its bootstrap Gitea.
|
|
# UNTIL THEN: prod's reconciliation depends on a container on a workstation that may be
|
|
# powered off. Flux keeps the last-reconciled state running, but NO new revision lands while
|
|
# it is down. Tracked as an open blocker in infra/PROD-PROVISIONING.md §7.
|
|
#
|
|
# The BRANCH below is settled and correct: prod tracks `main`, the same branch as dev.
|
|
# CONSEQUENCE, recorded deliberately: once prod is live, EVERY MERGE TO main RECONCILES BOTH
|
|
# CLUSTERS. There is no promotion step and no soak window — dev and prod move together.
|
|
# NOTE for bring-up: bootstrap-gitea.sh pushes the operator's CURRENT HEAD to the bootstrap
|
|
# Gitea's `main`, whatever branch that is. Before bootstrapping prod, confirm that remote `main`
|
|
# is the revision you intend prod to run (`git ls-remote` above, against local `main`).
|
|
- patch: |-
|
|
apiVersion: source.toolkit.fluxcd.io/v1
|
|
kind: GitRepository
|
|
metadata:
|
|
name: flux-system
|
|
namespace: flux-system
|
|
spec:
|
|
# TEMPORARY bootstrap source — workstation container, NOT in any cluster. See the block
|
|
# above for the cutover to https://gitea.home.keithjackson.dev/... . Do not treat as final.
|
|
url: http://192.168.2.132:3000/Home-Network/home-network-k3s-impl.git
|
|
ref:
|
|
branch: main
|
|
- target:
|
|
kind: ConfigMap
|
|
name: cluster-vars
|
|
patch: |-
|
|
- op: replace
|
|
path: /data/CLUSTER_ENV
|
|
value: "prod"
|
|
# trusted, rate-limited real certs in prod — ADR-0006.
|
|
- op: replace
|
|
path: /data/ACME_CLUSTER_ISSUER
|
|
value: "letsencrypt-prod"
|
|
# ==========================================================================================
|
|
# PROD DNS IDENTITY (R-DNS-01/02/03) — the fix for the 2026-09-04 incident.
|
|
# ==========================================================================================
|
|
# PROD SERVES ITS OWN SUBDOMAIN. Until 2026-09-04 every hostname was HARDCODED to
|
|
# home.keithjackson.dev in the base manifests, so prod was authored to serve the EXACT SAME
|
|
# public names as the live dev cluster out of the SAME Cloudflare zone. Two consequences, one
|
|
# of which already happened:
|
|
# 1. HAPPENED: prod's external-dns (policy: sync, owner id `default` — the same implicit
|
|
# default dev had) concluded it owned dev's records and DELETED EVERY ONE of them.
|
|
# 2. QUEUED BEHIND IT: platform-config would have requested a `*.home.keithjackson.dev`
|
|
# wildcard from letsencrypt-prod for a cert dev already holds, and both clusters would
|
|
# have raced each other's `_acme-challenge` TXT records on that one zone.
|
|
# Moving prod to its own suffix dissolves both: the two clusters can no longer name the same
|
|
# record. This is what makes ACME_CLUSTER_ISSUER: letsencrypt-prod above safe here.
|
|
#
|
|
# dev -> home.keithjackson.dev (UNCHANGED — dev is live; phase 2 moves it)
|
|
# prod -> prod.home.keithjackson.dev (this knob)
|
|
#
|
|
# CONSEQUENCE — prod's hostnames are NOT the apex names. Prod's Gitea is
|
|
# gitea.prod.home.keithjackson.dev, its Keycloak keycloak.prod.home.keithjackson.dev, and so
|
|
# on; the wildcard cert becomes `*.prod.home.keithjackson.dev`. The apex CNAMEs that make
|
|
# `<svc>.home.keithjackson.dev` point at prod are PHASE 2, deliberately not done here — they
|
|
# would take traffic off the live dev cluster. infra/PROD-PROVISIONING.md §10.
|
|
#
|
|
# THE APEX INVERSION IS DELIBERATELY NOT ON THIS BRANCH. Moving prod to the bare apex (and dev
|
|
# to dev.home.) is a LIVE MIGRATION of the dev cluster, not a variable change — see
|
|
# infra/PROD-PROVISIONING.md §10a. It is kept as a single commit on branch
|
|
# `migration/dns-apex-inversion` (9137bbd) and is applied as step D3 of that runbook, with the
|
|
# operator present. Everything else on this branch is independent of it, which is the whole
|
|
# reason it was split out.
|
|
#
|
|
# [parity] MUST equal k3s_base_domain in infra/ansible/inventory/group_vars/k3s_prod/vars.yml.
|
|
# The kube-apiserver's oidc-issuer-url is an INSTALL-TIME k3s flag, not a manifest, so it is
|
|
# set Ansible-side from that variable and cannot read this one. If they disagree, kubectl SSO
|
|
# trusts a different Keycloak than the one prod actually runs.
|
|
- op: replace
|
|
path: /data/BASE_DOMAIN
|
|
value: "prod.home.keithjackson.dev"
|
|
# SAFETY NET #2 (independent of the owner id below): narrow ExternalDNS's name scope to prod's
|
|
# OWN subtree, so prod cannot create, update or delete any record outside it no matter what the
|
|
# TXT registry says. VERIFY ON FIRST RECONCILE — the Cloudflare provider uses this value for
|
|
# ZONE selection too, and on the version in use in 2026-08 a strict-subdomain filter dropped
|
|
# the parent zone ("no hosted zone matching record DNS Name"). If prod's external-dns logs no
|
|
# matching zone, it will create NOTHING — fail-closed, and it can still never delete dev's
|
|
# records. Fallback in that case: set this to "keithjackson.dev" (dev's value) and rely on
|
|
# EXTERNALDNS_TXT_OWNER_ID below, which is sufficient on its own.
|
|
- op: replace
|
|
path: /data/EXTERNALDNS_DOMAIN_FILTER
|
|
# MUST be the Cloudflare ZONE, not prod's subdomain. FINDING F-DNS1 (DECISIONS-LOG,
|
|
# 2026-08-17) proved a subdomain filter DROPS THE PARENT ZONE: ExternalDNS generated the
|
|
# record then skipped it with "no hosted zone matching record DNS Name", so prod would
|
|
# publish NOTHING. Isolation from dev does NOT come from this filter — it comes from
|
|
# (1) prod only requesting *.prod.home.keithjackson.dev hostnames, and
|
|
# (2) txtOwnerId: prod, so prod's registry never claims dev's `default`-owned records.
|
|
# That pairing is what the 2026-09-04 incident needed and this filter cannot provide.
|
|
value: "keithjackson.dev"
|
|
# SAFETY NET #1 — THE ACTUAL ROOT CAUSE OF THE INCIDENT. Both clusters left this unset and so
|
|
# both inherited the chart default `default`; prod then treated dev's TXT-registry entries as
|
|
# its own and, under policy: sync, deleted the records behind them. This value MUST differ from
|
|
# dev's (`default`, pinned in clusters/dev — dev's live records are registered under it and
|
|
# cannot be renamed without orphaning them). `prod` is distinct, which is the entire fix.
|
|
- op: replace
|
|
path: /data/EXTERNALDNS_TXT_OWNER_ID
|
|
value: "prod"
|
|
- op: replace
|
|
path: /data/LONGHORN_DEFAULT_REPLICA_COUNT
|
|
value: "3"
|
|
# Longhorn's on-node replica directory: a SUBDIRECTORY of the claimed data partition, not
|
|
# the mount root. Prod's ~424 GB partition (LABEL=k3s-data) is mounted at the neutral /data
|
|
# and serves TWO tenants — /data/longhorn here, and /data/k3s for k3s's entire data dir
|
|
# (embedded etcd + the containerd image store), which had to move off the 11.8 GiB /var:
|
|
# measured on dev, containerd alone is 21-25 GB per node, so prod would have filled /var on
|
|
# its first image pulls and etcd could not have written its WAL. infra/PROD-PROVISIONING.md §2.
|
|
#
|
|
# [parity] MUST equal node_baseline_longhorn_data_path in
|
|
# infra/ansible/inventory/group_vars/k3s_prod/vars.yml — Ansible creates and mounts the
|
|
# directory, this tells Longhorn to use it. If they disagree, Longhorn writes replicas to a
|
|
# path on the OS filesystem instead of the data partition and /var fills up.
|
|
- op: replace
|
|
path: /data/LONGHORN_DATA_PATH
|
|
value: "/data/longhorn"
|
|
# longer retention / larger volumes on prod NVMe.
|
|
- op: replace
|
|
path: /data/MONITORING_PROMETHEUS_RETENTION
|
|
value: "30d"
|
|
- op: replace
|
|
path: /data/MONITORING_PROMETHEUS_PVC_SIZE
|
|
value: "50Gi"
|
|
# Size-cap below the 50Gi PVC (headroom for WAL + head + compaction scratch): prod may hit the
|
|
# size cap before 30d if series volume is high, keeping the disk from filling either way.
|
|
- op: replace
|
|
path: /data/MONITORING_PROMETHEUS_RETENTION_SIZE
|
|
value: "45GiB"
|
|
- op: replace
|
|
path: /data/MONITORING_LOKI_PVC_SIZE
|
|
value: "50Gi"
|
|
# HA control-plane addons in prod.
|
|
- op: replace
|
|
path: /data/KEYCLOAK_REPLICAS
|
|
value: "2"
|
|
- op: replace
|
|
path: /data/OAUTH2_PROXY_REPLICAS
|
|
value: "2"
|
|
- op: replace
|
|
path: /data/VELERO_RETENTION
|
|
value: "720h"
|
|
# ==========================================================================================
|
|
# PROD LAN ADDRESSING (192.168.2.0/24). Nodes: k3s-prod-1/2/3 = .10 / .11 / .12.
|
|
# ==========================================================================================
|
|
# These knobs were HARDCODED to the dev LAN inside base manifests until 2026-08-31; prod would
|
|
# have come up ANNOUNCING DEV ADDRESSES (192.168.1.x) on the prod LAN. They are knobs now, and
|
|
# this overlay supplies every one of them. Anything the operator has NOT yet decided stays an
|
|
# INVALID `TBD-PROD-*` placeholder on purpose: base defaults are the DEV values, so an OMITTED
|
|
# op means prod silently inherits a dev address — far worse than a failed apply. `TBD-PROD-*`
|
|
# is not a valid IP/URL, so Cilium CRD validation / LB-IPAM / CNPG / Velero reject it LOUDLY.
|
|
#
|
|
# ------------------------------------------------------------------------------------------
|
|
# ADDRESS PLAN (operator, 2026-08-31) — see infra/PROD-PROVISIONING.md §4 for the full table.
|
|
# ------------------------------------------------------------------------------------------
|
|
# nodes 192.168.2.10 / .11 / .12 (Ansible inventory)
|
|
# control-plane VIP 192.168.2.9 (Ansible group_vars/k3s_prod — NOT here)
|
|
# ingress Gateway VIP 192.168.2.20 (GATEWAY_VIP, below)
|
|
# Cilium LB pool 192.168.2.20 - .30 (CILIUM_LB_POOL_*, below)
|
|
#
|
|
# CONTAINMENT INVARIANT SATISFIED: GATEWAY_VIP (.20) IS CILIUM_LB_POOL_START, mirroring dev
|
|
# (.53 of .53-.60). This matters because the `shared` Gateway (platform/gateway/gateway.yaml)
|
|
# pins NOTHING — no `lbipam.cilium.io/ips`, no spec.addresses (unlike platform/crafty and
|
|
# apps/vms/unifi, which DO pin) — so Cilium LB-IPAM assigns it the FIRST FREE address of a
|
|
# matching pool block. Had GATEWAY_VIP stayed outside every pool (the original .21-.30
|
|
# proposal), the Gateway would have come up on .21 while split-horizon coredns-custom kept
|
|
# answering *.home.keithjackson.dev with .20 — an address nothing announces — blackholing
|
|
# in-cluster hairpin traffic (OIDC clients -> Keycloak/Grafana) while looking healthy.
|
|
# KEEP GATEWAY_VIP == CILIUM_LB_POOL_START when changing either.
|
|
# The control-plane VIP (.9) is deliberately OUTSIDE this pool: kube-vip floats it, and
|
|
# LB-IPAM must never be able to hand it to a Service.
|
|
- op: replace
|
|
path: /data/GATEWAY_VIP
|
|
value: "192.168.2.20"
|
|
# Cilium LB IP pool, operator-supplied: 192.168.2.20-.30 (START == GATEWAY_VIP, see above).
|
|
# Mirror this range into the prod tofu reservation (cilium_l2_vip_range) if prod ever gets one.
|
|
- op: replace
|
|
path: /data/CILIUM_LB_POOL_START
|
|
value: "192.168.2.20"
|
|
- op: replace
|
|
path: /data/CILIUM_LB_POOL_STOP
|
|
value: "192.168.2.30"
|
|
# k3s API endpoint Cilium's kube-proxy replacement dials. This is the prod CONTROL-PLANE VIP
|
|
# (kube-vip), NOT .20 (the ingress VIP) and NOT the single node address .10.
|
|
#
|
|
# *** DECIDED 2026-09-06 by the operator: the VIP, as intended. *** (This block previously
|
|
# read "Not yet decided" while shipping .10 — the founder's own address — and that is what
|
|
# the LIVE prod cluster is running: all three cilium-agents carry
|
|
# KUBERNETES_SERVICE_HOST=192.168.2.10 while kube-vip holds .9, verified 2026-09-06. On a
|
|
# 3-node HA cluster that makes the CNI's control-plane path depend on k3s-prod-1: lose it and
|
|
# the datapath keeps running off Cilium's cache while no service or policy update lands, with
|
|
# nothing looking broken. The VIP is a virtual address that reaches the cluster wherever the
|
|
# control plane is, which is the whole reason kube-vip exists here.)
|
|
#
|
|
# It is owned Ansible-side by **k3s_apiserver_endpoint** (group_vars/all.yml, derived from
|
|
# this environment's kube_vip_address = 192.168.2.9 in group_vars/k3s_prod/vars.yml) and MUST
|
|
# equal that value — [parity] contract in platform/cilium/helmrelease.yaml. NOT k3s_api_host:
|
|
# that variable still exists and still holds the FOUNDER's .10, because the cold-bootstrap
|
|
# `--server` join needs a host that is up before any VIP is floated. Two addresses, two
|
|
# variables, deliberately.
|
|
#
|
|
# .9 sits OUTSIDE the Cilium LB pool (.20-.30) and is not a node address, so kube-vip and
|
|
# LB-IPAM can never be handed the same IP — asserted by
|
|
# conformance/tests/l0-static/ha-control-plane-as-code.
|
|
- op: replace
|
|
path: /data/K3S_API_HOST
|
|
value: "192.168.2.9"
|
|
# --- still UNDECIDED -> loud placeholders (must be real before `flux bootstrap`) ------------
|
|
# Pinned L4 VIPs, both carved out of the pool by lbipam.cilium.io/ips. Not yet assigned for
|
|
# prod. Leaving the dev literals in place would hand a prod LoadBalancer a dev-LAN address, so
|
|
# they stay placeholders; whatever is chosen MUST fall inside [POOL_START, POOL_STOP] and not
|
|
# collide with the Gateway VIP or the node IPs (.10/.11/.12).
|
|
- op: replace
|
|
path: /data/UNIFI_L4_VIP
|
|
value: "192.168.2.21"
|
|
- op: replace
|
|
path: /data/CRAFTY_L4_VIP
|
|
value: "192.168.2.22"
|
|
# ------------------------------------------------------------------------------------------
|
|
# OFF-CLUSTER DR TARGET — Cloudflare R2 (operator, 2026-09-05).
|
|
#
|
|
# THERE IS DELIBERATELY NO `S3_BACKUP_ENDPOINT` OP HERE, and that is a reversal of the rule
|
|
# stated at the top of this overlay ("an OMITTED op means prod silently inherits a dev
|
|
# address"). That rule exists because base defaults are dev's LAN addresses. R2 is not one: it
|
|
# is a single shared, environment-independent endpoint, so under R-ENV-02 it belongs in base and
|
|
# overriding it with an identical value would only create two places to change on a rotation —
|
|
# the exact drift this repo has been bitten by before. Isolation does NOT come from the
|
|
# endpoint; it comes from the two path knobs below.
|
|
#
|
|
# WHY R2 AND NOT THE MinIO the operator first named: prod is on the SOHO network
|
|
# 192.168.2.0/24, which is pfSense's WAN, while the MinIO sits inside pfSense's LAN
|
|
# 192.168.1.0/24. Measured from a pod INSIDE prod (the workstation proves nothing about pod
|
|
# egress): `curl http://192.168.1.9:9000/minio/health/live` → connection timed out after 15s.
|
|
# Reaching it would need a WAN→LAN rule that does not exist. R2 needs no firewall change, and
|
|
# unlike that MinIO — which runs on the same Proxmox host as dev's VMs — it is genuinely
|
|
# off-site. Verify prod can reach the new endpoint before resuming velero:
|
|
# bootstrap/standup.sh ... --s3-endpoint <the R2 endpoint> s3check
|
|
#
|
|
# THE PATH KNOBS ARE WHAT MAKE ONE SHARED BUCKET SAFE. Nothing about the paths was
|
|
# per-environment before 2026-09-05: Velero used one unprefixed bucket, and every CNPG Cluster
|
|
# used `s3://cnpg` with serverName defaulting to the CLUSTER NAME — and both environments have
|
|
# keycloak-db / nextcloud-db / semaphore-db. Without these knobs prod would (a) adopt and
|
|
# TTL-delete dev's Velero backups and (b) interleave two Postgres servers' WAL into ONE Barman
|
|
# stream, corrupting the chain and destroying PITR for both. See clusters/base/cluster-vars.yaml.
|
|
# ------------------------------------------------------------------------------------------
|
|
- op: replace
|
|
path: /data/VELERO_BACKUP_PREFIX
|
|
value: "prod"
|
|
# Barman objects land at s3://cnpg/prod/<clusterName>/; dev's at s3://cnpg/dev/<clusterName>/.
|
|
- op: replace
|
|
path: /data/CNPG_BACKUP_PATH
|
|
value: "s3://cnpg/prod"
|
|
# TODO(WS-INFRA, Phase 7): pin prod node placement once the bare-metal role labels exist.
|