Files
home-network-k3s-impl/clusters/dev/kustomization.yaml
T
jakman85andClaude Opus 4.8 960cab937b fix(flux): move oauth2-proxy to a post-platform tier (dependsOn platform-config)
oauth2-proxy is a leaf forward-auth service that depends on Keycloak +
trusted wildcard TLS + split-horizon DNS. It lived in the gating `platform`
Flux Kustomization (wait:true), so on a cold start it Failed until those
dependencies existed, keeping the whole `platform` tier not-Ready. That in
turn blocked `platform-config` — which brings up the very Keycloak instance,
wildcard Certificate and L2 VIP oauth2-proxy needs (and previously carried
oauth2-proxy's own httproute/servicemonitor) — from ever reconciling. A
self-gating cold-start deadlock, twice-proven (DECISIONS-LOG sessions 7 & 9).

Break the cycle by moving the whole oauth2-proxy component out of the
`platform` tier into its own non-gating post-platform Flux Kustomization:

- clusters/base/oauth2-proxy.yaml: new Kustomization -> ./platform/oauth2-proxy,
  dependsOn platform-config, wait:false, substituteFrom cluster-vars.
- clusters/base/kustomization.yaml: reference the new Kustomization.
- platform/kustomization.yaml: drop the oauth2-proxy component (no longer in
  the gating tier).
- platform/oauth2-proxy/kustomization.yaml: pull in httproute + servicemonitor
  so the full component travels together.
- platform-config/kustomization.yaml: drop the in-place oauth2-proxy
  httproute/servicemonitor refs (moved with the controller).
- clusters/dev/kustomization.yaml: add the SOPS decryption patch for the new
  oauth2-proxy Kustomization (it now owns the encrypted credentials Secret that
  the `platform` Kustomization used to decrypt).

Now `platform` reaches Ready without oauth2-proxy; platform-config brings up
Keycloak/TLS/DNS; then oauth2-proxy reconciles against dependencies that
already exist. wait:false guarantees its health can never gate anything above
it. No config of oauth2-proxy itself changed — only its reconciliation tier.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-08-27 18:12:43 -05:00

154 lines
6.2 KiB
YAML

---
# clusters/dev — the ACTIVE build target overlay.
#
# `flux bootstrap ... --path=clusters/dev` points the dev cluster's Flux here. It reuses the base
# reconciliation graph and patches the shared cluster-vars ConfigMap with dev-scoped values (R-ENV-02).
# It also adds a self-managing ROOT Kustomization (sync.yaml) so the whole graph reconciles from git
# on every push — no more manual `kubectl apply -k` for cluster-vars/overlay changes (F-OPS1).
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../base
- sync.yaml # self-managing root Kustomization → ./clusters/dev
patches:
# --- SOPS decryption + bootstrap source (PM-inlined from bootstrap/flux/patches/, 2026-08-16) ---
# Inlined (not cross-dir file refs) to stay within kustomize's default load restrictions.
# Realises the commented decryption TODOs in clusters/base and sets the real source URL.
# The `sops-age` Secret is created out-of-band (bootstrap/sops/install-age-key.sh) — name only.
- patch: |-
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: platform
namespace: flux-system
spec:
decryption:
provider: sops
secretRef:
name: sops-age
# SOPS decryption on EACH app wave (the app tier is now apps-wave-0..N, not a single `apps`
# Kustomization). Same stanza the retired `apps` Kustomization received — one per wave, by
# name. Add a patch here whenever `render.py --emit-waves` introduces a new deepest wave
# (DESIGN-waves.md cutover note). The `sops-age` Secret is created out-of-band (name only).
- patch: |-
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: apps-wave-0
namespace: flux-system
spec:
decryption:
provider: sops
secretRef:
name: sops-age
- patch: |-
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: apps-wave-1
namespace: flux-system
spec:
decryption:
provider: sops
secretRef:
name: sops-age
- patch: |-
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: apps-wave-2
namespace: flux-system
spec:
decryption:
provider: sops
secretRef:
name: sops-age
# oauth2-proxy moved to its own post-platform Flux Kustomization (clusters/base/oauth2-proxy.yaml).
# It carries oauth2-proxy-credentials.sops.yaml, previously decrypted via the `platform`
# Kustomization; now that this tier owns the encrypted Secret it needs its own decryption stanza.
- patch: |-
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: oauth2-proxy
namespace: flux-system
spec:
decryption:
provider: sops
secretRef:
name: sops-age
- patch: |-
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
name: flux-system
namespace: flux-system
spec:
# Phase-6 cutover done: source is now the in-cluster Gitea; bootstrap host-Gitea retained
# stopped as recovery. PUBLIC repo => Flux needs no creds; Flux resolves the host in-cluster
# via coredns -> Gateway VIP, and the LE-prod wildcard cert is publicly trusted so NO
# secretRef/caCert is needed. Recovery: restart bootstrap Gitea (192.168.2.103:3000) and
# revert this url (docs/05).
url: https://gitea.home.keithjackson.dev/Home-Network/home-network-k3s-impl.git
ref:
branch: main
# --- dev-scoped cluster-vars (R-ENV-02) ---
- target:
kind: ConfigMap
name: cluster-vars
patch: |-
- op: replace
path: /data/CLUSTER_ENV
value: "dev"
# DNS-01 via LE PRODUCTION: in-cluster OIDC clients (oauth2-proxy, Grafana) must trust the
# wildcard cert for SSO to work; the staging root isn't in any trust store. Prod issuer is
# registered + Ready; one wildcard cert is well within LE rate limits — ADR-0006, R-TLS-01/03.
- op: replace
path: /data/ACME_CLUSTER_ISSUER
value: "letsencrypt-prod"
# Split-horizon DNS target: the Cilium-L2 VIP of the `shared` Gateway on the Proxmox LAN
# (base default 10.73.0.240 was the old libvirt VIP). coredns-custom answers
# *.home.keithjackson.dev with this so in-cluster OIDC clients reach Keycloak via the Gateway.
- op: replace
path: /data/GATEWAY_VIP
value: "192.168.1.53"
# dev mirrors prod's 3-node topology, so the replicated tier keeps 3 replicas.
- op: replace
path: /data/LONGHORN_DEFAULT_REPLICA_COUNT
value: "3"
# smaller retention / volumes to keep the dev workstation footprint light.
- op: replace
path: /data/MONITORING_PROMETHEUS_RETENTION
value: "3d"
- op: replace
path: /data/MONITORING_PROMETHEUS_PVC_SIZE
value: "5Gi"
- op: replace
path: /data/MONITORING_LOKI_PVC_SIZE
value: "5Gi"
# single-replica control-plane addons in dev (no HA needed to iterate).
- op: replace
path: /data/KEYCLOAK_REPLICAS
value: "1"
- op: replace
path: /data/OAUTH2_PROXY_REPLICAS
value: "1"
- op: replace
path: /data/VELERO_RETENTION
value: "72h"
# dev-softened Kyverno for cold-start (base defaults Enforce/Fail/3 = prod posture).
# Not credential-driven — bootstrap-safety knobs so a cold cluster comes up before
# admission is proven/HA; flip dev to the strict posture once green.
- op: replace
path: /data/KYVERNO_VALIDATION_ACTION
value: "Audit"
- op: replace
path: /data/KYVERNO_FORCE_FAILURE_POLICY_IGNORE
value: "true"
# 2 (not base-default 1): the webhook is fail-closed, so a single-pod restart 502s every
# admission-gated write cluster-wide. Two replicas give a fail-closed webhook availability
# headroom during a restart/upgrade while staying dev-modest (prod runs 3).
- op: replace
path: /data/KYVERNO_ADMISSION_REPLICAS
value: "2"