jakman85andClaude Opus 5
6f61c9622d
OPERATOR DECISION 2026-09-06: "use the VIP as intended — a virtual address to reach
the cluster and provide fault tolerant HA. Without a change it will not work in HA."
THE DEFECT. Cilium's kube-proxy replacement dialled a SINGLE NODE. Re-confirmed live
before changing anything: all three prod cilium-agent pods carried
KUBERNETES_SERVICE_HOST=192.168.2.10 — k3s-prod-1's own address — while the kube-vip
VIP 192.168.2.9 was Running on all three nodes with a held lease. Lose prod-1 and the
CNI's control-plane path is dead on an "HA" cluster. clusters/prod's own comment said
the value must be the VIP and NOT .10, then called it "Not yet decided" — I treated
that stale comment as an open question when the evidence already settled it.
SPLIT, NOT REPOINT, because k3s_api_host had two consumers with opposite needs:
k3s_api_host unchanged — the FOUNDER's IP: the cold-bootstrap join
target and a tls-san. infra/tofu/outputs.tf GENERATES this
name, so renaming would ripple into tofu and the runbooks.
k3s_apiserver_endpoint NEW = {{ kube_vip_address }} — the HA address; what Cilium
consumes. Derived, not restated, so an environment overrides
ONE line and the VIP, the SAN and the CNI endpoint move
together. Prod needed no new override.
THE SECOND HAZARD, WHICH MY BRIEF DID NOT NAME and the agent found: pointing Cilium at
the VIP breaks a COLD bootstrap for the same reason the join would — site.yml ran
node-baseline -> k3s -> cilium, so Cilium would install against an address nothing
announced yet. That is the 2026-09-01 deadlock shape. Fixed by SEQUENCE, not value:
site.yml now imports kube-vip.yml BETWEEN k3s and cilium. It cannot go earlier
(kube-vip is a k3s auto-deploy manifest, so it needs a cluster) and it works there
because the DaemonSet is hostNetwork with KUBERNETES_SERVICE_HOST=127.0.0.1 — needing
neither CNI nor kube-proxy. roles/kube_vip gained a read-only `wait_for` on the VIP so
a cold run stops THERE with a named cause instead of surfacing as unreachable agents.
Also fixed a pre-existing bug that only mattered once this play joined site.yml: the
role's read-only interface probe ran without `check_mode: false`, so under --check it
was skipped and the assert fired with a misleading "set kube_vip_interface explicitly".
VERIFIED BY ME, not just reported:
renders dev K3S_API_HOST=192.168.1.49 prod=192.168.2.9 (each environment's VIP)
cilium template consumes k3s_apiserver_endpoint, not k3s_api_host
cold join still `--server https://{{ k3s_api_host }}:6443` — the founder
order site.yml: node-baseline -> k3s -> kube-vip -> cilium -> verify -> flux
gate full CI gate PASS
Agent-side, additionally: in-memory render of cilium-values.yaml.j2 per host showing
k8sServiceHost=the VIP with the join still on the founder; k3s_tls_sans renders
['.10','.9','127.0.0.1','.11','.12'] so NO cert reissue and NO k3s restart is involved
— .9 already answers under full CA verification; and --check --diff of kube-vip.yml
against prod reported 6 ok / 0 changed, i.e. the role edits change nothing about the
kube-vip prod already runs.
ha-control-plane-as-code 92 -> 112 assertions, turning two escalations into machine
checks: the Cilium template must use the endpoint var; the endpoint must derive from
kube_vip_address and k3s_api_host must not; per environment K3S_API_HOST must equal
that environment's VIP and be none of its node IPs (the [parity] contract, now
machine-checked rather than a comment); site.yml must order k3s -> kube-vip -> cilium.
Every new assertion mutation-tested and bites. One honest negative recorded: reverting
only the BASE default is not caught, because both overlays patch the key — the check
reads effective per-environment values, so the base default is unguarded.
APPLY ORDER, and this is the operator's step — nothing was applied:
PROD FIRST, and it is the low-risk direction (prod is the broken one). Push and let
Flux reconcile platform/cilium: the value change rolls the cilium-agent DaemonSet
ONE NODE AT A TIME (maxUnavailable: 1 — the knob that bounded the 2026-09-01
incident), it will not proceed past an unready pod, existing datapath survives an
agent restart, and .9 is already live before the change lands.
DEV SEPARATELY, operator present. Same mechanics but live dev agents dial
192.168.1.50, and dev is mid-DNS-migration on dev-live. Do not fold it into that.
ANSIBLE: nothing to run for either live cluster; the Ansible half matters only for a
cold build. Remove the now-redundant standalone kube-vip.yml step from any runbook.
NEW NAMED GAP, unwritten: the L2 test cannot assert the running agents' live
KUBERNETES_SERVICE_HOST, which is the drift the L0 check cannot see. Recorded in that
test's gap list.
Refs R-PLAT-01, R-NET-01, R-ENV-02.
Co-Authored-By: Claude Opus 5 <[email protected]>
354 lines
20 KiB
YAML
354 lines
20 KiB
YAML
---
|
|
# clusters/dev — the ACTIVE build target overlay.
|
|
#
|
|
# `flux bootstrap ... --path=clusters/dev` points the dev cluster's Flux here. It reuses the base
|
|
# reconciliation graph and patches the shared cluster-vars ConfigMap with dev-scoped values (R-ENV-02).
|
|
# It also adds a self-managing ROOT Kustomization (sync.yaml) so the whole graph reconciles from git
|
|
# on every push — no more manual `kubectl apply -k` for cluster-vars/overlay changes (F-OPS1).
|
|
#
|
|
# ##############################################################################################
|
|
# ### *** DO NOT MERGE THIS FILE TO `main` UNATTENDED (as of 2026-09-04) *** ###
|
|
# ### ###
|
|
# ### This overlay now carries dev's DNS MIGRATION target: BASE_DOMAIN moves from the apex ###
|
|
# ### `home.keithjackson.dev` to `dev.home.keithjackson.dev`, and the GitRepository url ###
|
|
# ### moves with it. sync.yaml reconciles ./clusters/dev from `main` EVERY 10 MINUTES, so a ###
|
|
# ### merge to main IS the cutover — it executes on the live dev cluster with nobody present. ###
|
|
# ### Dev's Gitea is both its GitOps source and the push target: if the new host is not up ###
|
|
# ### with a Ready cert first, dev stops reconciling and pushing cannot repair it. ###
|
|
# ### ###
|
|
# ### Run the ordered runbook: infra/PROD-PROVISIONING.md §10 (steps D1-D8), operator ###
|
|
# ### present. Merging is step D3, not step one. ###
|
|
# ##############################################################################################
|
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
kind: Kustomization
|
|
resources:
|
|
- ../base
|
|
- sync.yaml # self-managing root Kustomization → ./clusters/dev
|
|
patches:
|
|
# --- SOPS decryption + bootstrap source (PM-inlined from bootstrap/flux/patches/, 2026-08-16) ---
|
|
# Inlined (not cross-dir file refs) to stay within kustomize's default load restrictions.
|
|
# Realises the commented decryption TODOs in clusters/base and sets the real source URL.
|
|
# The `sops-age` Secret is created out-of-band (bootstrap/sops/install-age-key.sh) — name only.
|
|
- patch: |-
|
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
|
kind: Kustomization
|
|
metadata:
|
|
name: platform
|
|
namespace: flux-system
|
|
spec:
|
|
decryption:
|
|
provider: sops
|
|
secretRef:
|
|
name: sops-age
|
|
# SOPS decryption on EACH app wave (the app tier is now apps-wave-0..N, not a single `apps`
|
|
# Kustomization). Same stanza the retired `apps` Kustomization received — one per wave, by
|
|
# name. Add a patch here whenever `render.py --emit-waves` introduces a new deepest wave
|
|
# (DESIGN-waves.md cutover note). The `sops-age` Secret is created out-of-band (name only).
|
|
- patch: |-
|
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
|
kind: Kustomization
|
|
metadata:
|
|
name: apps-wave-0
|
|
namespace: flux-system
|
|
spec:
|
|
decryption:
|
|
provider: sops
|
|
secretRef:
|
|
name: sops-age
|
|
- patch: |-
|
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
|
kind: Kustomization
|
|
metadata:
|
|
name: apps-wave-1
|
|
namespace: flux-system
|
|
spec:
|
|
decryption:
|
|
provider: sops
|
|
secretRef:
|
|
name: sops-age
|
|
- patch: |-
|
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
|
kind: Kustomization
|
|
metadata:
|
|
name: apps-wave-2
|
|
namespace: flux-system
|
|
spec:
|
|
decryption:
|
|
provider: sops
|
|
secretRef:
|
|
name: sops-age
|
|
# oauth2-proxy moved to its own post-platform Flux Kustomization (clusters/base/oauth2-proxy.yaml).
|
|
# It carries oauth2-proxy-credentials.sops.yaml, previously decrypted via the `platform`
|
|
# Kustomization; now that this tier owns the encrypted Secret it needs its own decryption stanza.
|
|
- patch: |-
|
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
|
kind: Kustomization
|
|
metadata:
|
|
name: oauth2-proxy
|
|
namespace: flux-system
|
|
spec:
|
|
decryption:
|
|
provider: sops
|
|
secretRef:
|
|
name: sops-age
|
|
# headlamp moved to its own post-platform Flux Kustomization (clusters/base/headlamp.yaml).
|
|
# It carries headlamp-oidc-credentials.sops.yaml, previously decrypted via the `platform`
|
|
# Kustomization; now that this tier owns the encrypted Secret it needs its own decryption stanza.
|
|
- patch: |-
|
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
|
kind: Kustomization
|
|
metadata:
|
|
name: headlamp
|
|
namespace: flux-system
|
|
spec:
|
|
decryption:
|
|
provider: sops
|
|
secretRef:
|
|
name: sops-age
|
|
# crafty has its own post-platform Flux Kustomization (clusters/base/crafty.yaml). It carries
|
|
# crafty-oauth2-proxy-credentials.sops.yaml, so it needs its own SOPS decryption stanza (same as
|
|
# headlamp/oauth2-proxy). The `sops-age` Secret is created out-of-band (name only).
|
|
- patch: |-
|
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
|
kind: Kustomization
|
|
metadata:
|
|
name: crafty
|
|
namespace: flux-system
|
|
spec:
|
|
decryption:
|
|
provider: sops
|
|
secretRef:
|
|
name: sops-age
|
|
# velero (backup controller) has its own post-platform Flux Kustomization (clusters/base/backup.yaml).
|
|
# It carries velero-cloud.sops.yaml (the scoped MinIO `cloud` credential), so it needs its own SOPS
|
|
# decryption stanza (same as headlamp/oauth2-proxy/crafty). The `sops-age` Secret is out-of-band (name only).
|
|
- patch: |-
|
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
|
kind: Kustomization
|
|
metadata:
|
|
name: velero
|
|
namespace: flux-system
|
|
spec:
|
|
decryption:
|
|
provider: sops
|
|
secretRef:
|
|
name: sops-age
|
|
- patch: |-
|
|
apiVersion: source.toolkit.fluxcd.io/v1
|
|
kind: GitRepository
|
|
metadata:
|
|
name: flux-system
|
|
namespace: flux-system
|
|
spec:
|
|
# Phase-6 cutover done: source is now the in-cluster Gitea; bootstrap host-Gitea retained
|
|
# stopped as recovery. PUBLIC repo => Flux needs no creds; Flux resolves the host in-cluster
|
|
# via coredns -> Gateway VIP, and the LE-prod wildcard cert is publicly trusted so NO
|
|
# secretRef/caCert is needed. Recovery: restart bootstrap Gitea (192.168.2.103:3000) and
|
|
# revert this url (docs/05).
|
|
#
|
|
# *** THE STRAND. THIS URL IS THE MOST DANGEROUS LINE IN THE REPOSITORY. ***
|
|
# It is BOTH dev's GitOps source AND the operator's push target, and it is NOT tokenized —
|
|
# it cannot be: the `flux-system` root Kustomization that renders ./clusters/dev declares NO
|
|
# postBuild.substituteFrom, so `gitea.${BASE_DOMAIN}` here would ship the LITERAL token and
|
|
# break the source outright (the L0 flux-postbuild-substitution check asserts exactly this).
|
|
# So it is a hand-maintained literal that must be moved in lockstep with BASE_DOMAIN below.
|
|
#
|
|
# It has been moved to dev's NEW suffix as part of the 2026-09-04 migration authoring. That
|
|
# makes this file the TARGET state, NOT a safe unattended merge: dev's root Kustomization
|
|
# (sync.yaml) reconciles ./clusters/dev from `main` every 10m, so MERGING THIS TO main WHILE
|
|
# DEV IS LIVE MIGRATES DEV UNATTENDED. If gitea.dev.home.keithjackson.dev does not already
|
|
# resolve and serve a valid cert at that instant, dev stops reconciling and CANNOT BE FIXED
|
|
# BY PUSHING — the broken thing is the path Flux uses to read pushes.
|
|
# ORDERING THAT AVOIDS IT (infra/PROD-PROVISIONING.md §10, steps D1-D3): suspend dev's root
|
|
# Kustomization or park this url on the bootstrap Gitea (http://192.168.2.132:3000/...) for
|
|
# the migration window, and only restore the in-cluster url after the new host is proven.
|
|
url: https://gitea.dev.home.keithjackson.dev/Home-Network/home-network-k3s-impl.git
|
|
ref:
|
|
branch: main
|
|
# --- dev-scoped cluster-vars (R-ENV-02) ---
|
|
- target:
|
|
kind: ConfigMap
|
|
name: cluster-vars
|
|
patch: |-
|
|
- op: replace
|
|
path: /data/CLUSTER_ENV
|
|
value: "dev"
|
|
# DNS-01 via LE PRODUCTION: in-cluster OIDC clients (oauth2-proxy, Grafana) must trust the
|
|
# wildcard cert for SSO to work; the staging root isn't in any trust store. Prod issuer is
|
|
# registered + Ready; one wildcard cert is well within LE rate limits — ADR-0006, R-TLS-01/03.
|
|
- op: replace
|
|
path: /data/ACME_CLUSTER_ISSUER
|
|
value: "letsencrypt-prod"
|
|
# ==========================================================================================
|
|
# DEV DNS IDENTITY (R-DNS-01/02/03) — DEV MOVES OFF THE APEX. **NOT YET APPLIED.**
|
|
# ==========================================================================================
|
|
# OPERATOR DECISION 2026-09-04, superseding the interim split committed earlier the same day:
|
|
#
|
|
# dev -> dev.home.keithjackson.dev (this knob; = the base default, restated)
|
|
# prod -> home.keithjackson.dev (THE APEX — prod IS the apex, no `prod.` label,
|
|
# and therefore no apex CNAME layer at all)
|
|
#
|
|
# *** THIS VALUE IS AHEAD OF THE LIVE CLUSTER. *** The dev cluster is serving the APEX right
|
|
# now. Merging this file does not perform the migration — it DEFINES the target. The cutover is
|
|
# an ORDERED, OPERATOR-PRESENT runbook: infra/PROD-PROVISIONING.md §10.
|
|
#
|
|
# WHY IT CANNOT BE A PLAIN VARIABLE FLIP — THE STRAND. `spec.url` of the GitRepository ~20
|
|
# lines above is https://gitea.home.keithjackson.dev/... . Dev's Gitea is BOTH dev's GitOps
|
|
# source AND the operator's push target. The moment BASE_DOMAIN moves, dev's Gitea HTTPRoute,
|
|
# DNS record and wildcard cert all move to gitea.dev.home.keithjackson.dev. If that host does
|
|
# not ALREADY resolve with a valid, Ready cert, dev stops reconciling — and it CANNOT BE FIXED
|
|
# BY PUSHING, because the thing that is broken is the path Flux uses to read pushes. Recovery
|
|
# is then hands-on (point the source at the bootstrap Gitea on 192.168.2.132:3000, or repair
|
|
# in-cluster by hand). §10 states the ordering that avoids it: publish and verify the new host
|
|
# FIRST, or park the source on the bootstrap Gitea for the window.
|
|
#
|
|
# ALSO MOVED BY THIS ONE LINE, all on a live cluster: the `*.` wildcard Certificate (a fresh
|
|
# ACME order), Keycloak's issuer URL and ~12 OIDC clients' redirect URIs, kubectl SSO (a k3s
|
|
# server flag => a rolling restart + every user re-issuing their kubeconfig), coredns-custom's
|
|
# split-horizon zone, Homepage's links, and 6 L2 chainsaw tests that hardcode the apex.
|
|
- op: replace
|
|
path: /data/BASE_DOMAIN
|
|
value: "dev.home.keithjackson.dev"
|
|
# ExternalDNS scope. MUST stay the Cloudflare ZONE — NOT dev.home.keithjackson.dev. FINDING
|
|
# F-DNS1 (DECISIONS-LOG 2026-08-17): the Cloudflare provider uses this for ZONE SELECTION, so a
|
|
# strict-subdomain filter matches no zone and publishes NOTHING ("no hosted zone matching
|
|
# record DNS Name"). Dev moving to a subdomain does NOT make narrowing safe — it makes it
|
|
# newly tempting, which is why this comment is here. Same value the retired DNS_ZONE knob held.
|
|
- op: replace
|
|
path: /data/EXTERNALDNS_DOMAIN_FILTER
|
|
value: "keithjackson.dev"
|
|
# OWNER ID: `dev` (was `default` until 2026-09-04). The old value was FORCED — dev's live apex
|
|
# records were already in Cloudflare's TXT registry under owner `default`, so renaming would
|
|
# have orphaned every one of them. THE MIGRATION DISSOLVES THAT CONSTRAINT: on the new suffix
|
|
# dev publishes a brand-new record set with no prior registry entries to recognise, so the id
|
|
# is chosen on its merits, and `default` has none — it is the value external-dns uses when you
|
|
# FORGET the flag, which is exactly how both clusters came to share it. Consequences, both
|
|
# deliberate:
|
|
# - dev's OLD apex records remain owned by `default`, which no environment now claims, so
|
|
# nothing deletes them implicitly. Retiring them is an explicit §10 step, and prod's first
|
|
# publish is gated on that retirement being CONFIRMED — not on two reconcile loops racing.
|
|
# - prod stays `prod` and must NEVER be given `default`, or prod would own the live dev
|
|
# cluster's apex records and sync-delete them. Prod inherits the apex NAMES, not its OWNER.
|
|
# dev != prod is the load-bearing invariant; the base default stays a non-colliding sentinel.
|
|
- op: replace
|
|
path: /data/EXTERNALDNS_TXT_OWNER_ID
|
|
value: "dev"
|
|
# --- dev LAN addressing (192.168.1.0/24 on the Proxmox LAN) -----------------------------
|
|
# EVERY knob in cluster-vars' "LAN addressing" block is restated here EXPLICITLY, even where it
|
|
# equals the base default. These values are what the LIVE dev cluster is running right now, so
|
|
# the substituted output is byte-identical to the pre-parameterization hardcoded manifests
|
|
# (verified 2026-08-31). Never let this block drift from the live cluster.
|
|
#
|
|
# Split-horizon DNS target: the Cilium-L2 VIP of the `shared` Gateway on the Proxmox LAN
|
|
# (= CILIUM_LB_POOL_START, the pool's first address). coredns-custom answers
|
|
# *.dev.home.keithjackson.dev with this so in-cluster OIDC clients reach Keycloak via the Gateway.
|
|
- op: replace
|
|
path: /data/GATEWAY_VIP
|
|
value: "192.168.1.53"
|
|
# k3s API endpoint Cilium's kube-proxy replacement dials — DEV'S CONTROL-PLANE VIP,
|
|
# the address kube-vip floats, NOT the primary node .50.
|
|
# [parity] equals the Ansible inventory's k3s_apiserver_endpoint for dev
|
|
# (= kube_vip_address, group_vars/all.yml). See clusters/base/cluster-vars.yaml for why.
|
|
#
|
|
# *** THE ONE EXCEPTION TO "NEVER LET THIS BLOCK DRIFT FROM THE LIVE CLUSTER" ***
|
|
# Every other value in this block is what dev is running right now. This one is not:
|
|
# the live dev cilium-agents dial 192.168.1.50 (k3s-1). Changing it here is a
|
|
# DELIBERATE, OPERATOR-APPROVED move (2026-09-06) and reconciling it ROLLS THE CILIUM
|
|
# AGENT DAEMONSET on dev — one node at a time (updateStrategy.maxUnavailable: 1, kept
|
|
# in parity for exactly this reason). Do not push this to dev-live casually, and not
|
|
# while dev's DNS migration is mid-flight; the L2 datapath survives an agent restart
|
|
# but a wrong value here is the 2026-09-01 incident. .49 is in dev's serving-cert SANs
|
|
# already (k3s_tls_sans) and dev's own kubeconfig validates against it via
|
|
# tls-server-name, so TLS is not the risk — the risk is reachability, and .49 is on the
|
|
# same LAN as the nodes and ARP-announced by kube-vip on every server.
|
|
- op: replace
|
|
path: /data/K3S_API_HOST
|
|
value: "192.168.1.49"
|
|
# Cilium LB IP pool = the tofu reservation cilium_l2_vip_range .53-.60, outside the node IPs
|
|
# (.50/.51/.52) and outside the DHCP scope. .53 Gateway, .54 unifi, .55 crafty; .56-.60 free.
|
|
- op: replace
|
|
path: /data/CILIUM_LB_POOL_START
|
|
value: "192.168.1.53"
|
|
- op: replace
|
|
path: /data/CILIUM_LB_POOL_STOP
|
|
value: "192.168.1.60"
|
|
# Pinned L4 VIPs out of that pool (lbipam.cilium.io/ips). Changing either MOVES a live device-
|
|
# facing endpoint: unifi is the inform/STUN target every adopted UniFi device holds.
|
|
- op: replace
|
|
path: /data/UNIFI_L4_VIP
|
|
value: "192.168.1.54"
|
|
- op: replace
|
|
path: /data/CRAFTY_L4_VIP
|
|
value: "192.168.1.55"
|
|
# OFF-CLUSTER BACKUP TARGET — no op here any more, deliberately.
|
|
#
|
|
# This used to pin the on-Proxmox MinIO (http://192.168.1.9:9000). As of 2026-09-05 the operator
|
|
# moved backups to Cloudflare R2 "for the time being until a better on-prem backup solution is
|
|
# created", and R2 is a single environment-INDEPENDENT endpoint: both clusters write to the same
|
|
# buckets and are isolated by VELERO_BACKUP_PREFIX / CNPG_BACKUP_PATH (dev vs prod), not by URL.
|
|
# Under R-ENV-02 a value that does not differ per environment belongs in base and must NOT be
|
|
# restated here — two copies of one endpoint is just a rotation waiting to drift.
|
|
#
|
|
# *** MERGING THIS RESETS DEV'S PITR WINDOW. *** Dev's R2 prefix starts EMPTY, so Velero writes a
|
|
# fresh kopia repository and CNPG takes new base backups; the 30-day Barman window rebuilds from
|
|
# zero. The existing MinIO copy is NOT deleted and stays restorable — point S3_BACKUP_ENDPOINT
|
|
# back at http://192.168.1.9:9000 (and VELERO_BACKUP_PREFIX back to "") to reach it. Do not merge
|
|
# this while dev's proven restore is the only recovery path you are relying on.
|
|
# dev mirrors prod's 3-node topology, so the replicated tier keeps 3 replicas.
|
|
- op: replace
|
|
path: /data/LONGHORN_DEFAULT_REPLICA_COUNT
|
|
value: "3"
|
|
# DO NOT CHANGE THIS VALUE. The LIVE dev cluster has 18 Longhorn volumes whose replicas are
|
|
# on disk under /var/lib/longhorn — the dedicated /dev/vdb ext4 disk roles/node_baseline
|
|
# mounts there. Longhorn addresses a replica by dataPath; pointing defaultDataPath anywhere
|
|
# else would orphan every one of them (they are not recoverable from the old path by
|
|
# rescheduling). Restated explicitly even though it equals the base default, exactly like the
|
|
# LAN block above: this is what the live cluster is running. Prod uses /data/longhorn because
|
|
# its data partition also carries k3s's data dir — see clusters/prod and PROD-PROVISIONING §2.
|
|
- op: replace
|
|
path: /data/LONGHORN_DATA_PATH
|
|
value: "/var/lib/longhorn"
|
|
# smaller retention / volumes to keep the dev workstation footprint light.
|
|
- op: replace
|
|
path: /data/MONITORING_PROMETHEUS_RETENTION
|
|
value: "3d"
|
|
# 5Gi was too tight: 3d of this cluster's series volume (~1.2Gi/day of blocks) plus a WAL that
|
|
# ballooned past 1Gi overran the disk, wedging WAL commits and firing every absent()/==0 alert
|
|
# falsely. Bumped to 8Gi (still lighter than base 10Gi) so the size-cap below has real headroom.
|
|
- op: replace
|
|
path: /data/MONITORING_PROMETHEUS_PVC_SIZE
|
|
value: "8Gi"
|
|
# Size-cap kept well below the 8Gi PVC (headroom for WAL + head + compaction scratch) so
|
|
# Prometheus prunes oldest blocks by size and never fills the disk on this young cluster.
|
|
- op: replace
|
|
path: /data/MONITORING_PROMETHEUS_RETENTION_SIZE
|
|
value: "5GiB"
|
|
- op: replace
|
|
path: /data/MONITORING_LOKI_PVC_SIZE
|
|
value: "5Gi"
|
|
# single-replica control-plane addons in dev (no HA needed to iterate).
|
|
- op: replace
|
|
path: /data/KEYCLOAK_REPLICAS
|
|
value: "1"
|
|
- op: replace
|
|
path: /data/OAUTH2_PROXY_REPLICAS
|
|
value: "1"
|
|
- op: replace
|
|
path: /data/VELERO_RETENTION
|
|
value: "72h"
|
|
# dev-softened Kyverno for cold-start (base defaults Enforce/Fail/3 = prod posture).
|
|
# Not credential-driven — bootstrap-safety knobs so a cold cluster comes up before
|
|
# admission is proven/HA; flip dev to the strict posture once green.
|
|
- op: replace
|
|
path: /data/KYVERNO_VALIDATION_ACTION
|
|
value: "Audit"
|
|
- op: replace
|
|
path: /data/KYVERNO_FORCE_FAILURE_POLICY_IGNORE
|
|
value: "true"
|
|
# 2 (not base-default 1): the webhook is fail-closed, so a single-pod restart 502s every
|
|
# admission-gated write cluster-wide. Two replicas give a fail-closed webhook availability
|
|
# headroom during a restart/upgrade while staying dev-modest (prod runs 3).
|
|
- op: replace
|
|
path: /data/KYVERNO_ADMISSION_REPLICAS
|
|
value: "2"
|