Add Crafty Controller 4 (registry.gitlab.com/crafty-controller/crafty-4:4.10.8) as a platform component: a web panel that runs Java + Bedrock Minecraft servers as subprocesses in one pod + a 20Gi Longhorn PVC (mounted 5x by subPath). Auth = Option B (per-app oauth2-proxy reverse-proxy sidecar), since Cilium 1.16.5 cannot do ext_authz on the shared Gateway: - oauth2-proxy v7.8.1 sidecar (:4180) reverse-proxies to Crafty's self-signed HTTPS UI (127.0.0.1:8443), gated to Keycloak homelab realm group platform-admins. - Confidential `crafty` client provisioned live via kcadm + captured in the realm import; client secret + cookie secret via SOPS/age (mirrored in realm-clients). - Web UI: HTTPRoute crafty.home.keithjackson.dev -> Service :80 -> sidecar :4180. - Game ports: Cilium L2 LoadBalancer on VIP 192.168.1.55 (next free after .53/.54), starter set TCP 25565-25567 (Java) + UDP 19132-19134 (Bedrock), NOT auth-gated. - fsGroup:0 / runAsGroup:0 / runAsUser:1000 for the image's crafty:root identity on the root:root Longhorn PVC (the seerr fsGroup lesson). - Own non-gating Flux tier clusters/base/crafty.yaml (dependsOn platform-config, wait:false) + sops-decryption patch in clusters/dev — a leaf UI must never gate the platform. - Explicit default-deny NetworkPolicy + CiliumNetworkPolicies (platform namespace, not app-namespace, so Kyverno does not auto-generate the baseline). make -C ci gate green; kustomize build of platform/crafty, clusters/base, clusters/dev clean. Co-Authored-By: Claude Opus 4.8 <[email protected]>
190 lines
7.8 KiB
YAML
190 lines
7.8 KiB
YAML
---
|
|
# clusters/dev — the ACTIVE build target overlay.
|
|
#
|
|
# `flux bootstrap ... --path=clusters/dev` points the dev cluster's Flux here. It reuses the base
|
|
# reconciliation graph and patches the shared cluster-vars ConfigMap with dev-scoped values (R-ENV-02).
|
|
# It also adds a self-managing ROOT Kustomization (sync.yaml) so the whole graph reconciles from git
|
|
# on every push — no more manual `kubectl apply -k` for cluster-vars/overlay changes (F-OPS1).
|
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
kind: Kustomization
|
|
resources:
|
|
- ../base
|
|
- sync.yaml # self-managing root Kustomization → ./clusters/dev
|
|
patches:
|
|
# --- SOPS decryption + bootstrap source (PM-inlined from bootstrap/flux/patches/, 2026-08-16) ---
|
|
# Inlined (not cross-dir file refs) to stay within kustomize's default load restrictions.
|
|
# Realises the commented decryption TODOs in clusters/base and sets the real source URL.
|
|
# The `sops-age` Secret is created out-of-band (bootstrap/sops/install-age-key.sh) — name only.
|
|
- patch: |-
|
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
|
kind: Kustomization
|
|
metadata:
|
|
name: platform
|
|
namespace: flux-system
|
|
spec:
|
|
decryption:
|
|
provider: sops
|
|
secretRef:
|
|
name: sops-age
|
|
# SOPS decryption on EACH app wave (the app tier is now apps-wave-0..N, not a single `apps`
|
|
# Kustomization). Same stanza the retired `apps` Kustomization received — one per wave, by
|
|
# name. Add a patch here whenever `render.py --emit-waves` introduces a new deepest wave
|
|
# (DESIGN-waves.md cutover note). The `sops-age` Secret is created out-of-band (name only).
|
|
- patch: |-
|
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
|
kind: Kustomization
|
|
metadata:
|
|
name: apps-wave-0
|
|
namespace: flux-system
|
|
spec:
|
|
decryption:
|
|
provider: sops
|
|
secretRef:
|
|
name: sops-age
|
|
- patch: |-
|
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
|
kind: Kustomization
|
|
metadata:
|
|
name: apps-wave-1
|
|
namespace: flux-system
|
|
spec:
|
|
decryption:
|
|
provider: sops
|
|
secretRef:
|
|
name: sops-age
|
|
- patch: |-
|
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
|
kind: Kustomization
|
|
metadata:
|
|
name: apps-wave-2
|
|
namespace: flux-system
|
|
spec:
|
|
decryption:
|
|
provider: sops
|
|
secretRef:
|
|
name: sops-age
|
|
# oauth2-proxy moved to its own post-platform Flux Kustomization (clusters/base/oauth2-proxy.yaml).
|
|
# It carries oauth2-proxy-credentials.sops.yaml, previously decrypted via the `platform`
|
|
# Kustomization; now that this tier owns the encrypted Secret it needs its own decryption stanza.
|
|
- patch: |-
|
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
|
kind: Kustomization
|
|
metadata:
|
|
name: oauth2-proxy
|
|
namespace: flux-system
|
|
spec:
|
|
decryption:
|
|
provider: sops
|
|
secretRef:
|
|
name: sops-age
|
|
# headlamp moved to its own post-platform Flux Kustomization (clusters/base/headlamp.yaml).
|
|
# It carries headlamp-oidc-credentials.sops.yaml, previously decrypted via the `platform`
|
|
# Kustomization; now that this tier owns the encrypted Secret it needs its own decryption stanza.
|
|
- patch: |-
|
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
|
kind: Kustomization
|
|
metadata:
|
|
name: headlamp
|
|
namespace: flux-system
|
|
spec:
|
|
decryption:
|
|
provider: sops
|
|
secretRef:
|
|
name: sops-age
|
|
# crafty has its own post-platform Flux Kustomization (clusters/base/crafty.yaml). It carries
|
|
# crafty-oauth2-proxy-credentials.sops.yaml, so it needs its own SOPS decryption stanza (same as
|
|
# headlamp/oauth2-proxy). The `sops-age` Secret is created out-of-band (name only).
|
|
- patch: |-
|
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
|
kind: Kustomization
|
|
metadata:
|
|
name: crafty
|
|
namespace: flux-system
|
|
spec:
|
|
decryption:
|
|
provider: sops
|
|
secretRef:
|
|
name: sops-age
|
|
- patch: |-
|
|
apiVersion: source.toolkit.fluxcd.io/v1
|
|
kind: GitRepository
|
|
metadata:
|
|
name: flux-system
|
|
namespace: flux-system
|
|
spec:
|
|
# Phase-6 cutover done: source is now the in-cluster Gitea; bootstrap host-Gitea retained
|
|
# stopped as recovery. PUBLIC repo => Flux needs no creds; Flux resolves the host in-cluster
|
|
# via coredns -> Gateway VIP, and the LE-prod wildcard cert is publicly trusted so NO
|
|
# secretRef/caCert is needed. Recovery: restart bootstrap Gitea (192.168.2.103:3000) and
|
|
# revert this url (docs/05).
|
|
url: https://gitea.home.keithjackson.dev/Home-Network/home-network-k3s-impl.git
|
|
ref:
|
|
branch: main
|
|
# --- dev-scoped cluster-vars (R-ENV-02) ---
|
|
- target:
|
|
kind: ConfigMap
|
|
name: cluster-vars
|
|
patch: |-
|
|
- op: replace
|
|
path: /data/CLUSTER_ENV
|
|
value: "dev"
|
|
# DNS-01 via LE PRODUCTION: in-cluster OIDC clients (oauth2-proxy, Grafana) must trust the
|
|
# wildcard cert for SSO to work; the staging root isn't in any trust store. Prod issuer is
|
|
# registered + Ready; one wildcard cert is well within LE rate limits — ADR-0006, R-TLS-01/03.
|
|
- op: replace
|
|
path: /data/ACME_CLUSTER_ISSUER
|
|
value: "letsencrypt-prod"
|
|
# Split-horizon DNS target: the Cilium-L2 VIP of the `shared` Gateway on the Proxmox LAN
|
|
# (base default 10.73.0.240 was the old libvirt VIP). coredns-custom answers
|
|
# *.home.keithjackson.dev with this so in-cluster OIDC clients reach Keycloak via the Gateway.
|
|
- op: replace
|
|
path: /data/GATEWAY_VIP
|
|
value: "192.168.1.53"
|
|
# dev mirrors prod's 3-node topology, so the replicated tier keeps 3 replicas.
|
|
- op: replace
|
|
path: /data/LONGHORN_DEFAULT_REPLICA_COUNT
|
|
value: "3"
|
|
# smaller retention / volumes to keep the dev workstation footprint light.
|
|
- op: replace
|
|
path: /data/MONITORING_PROMETHEUS_RETENTION
|
|
value: "3d"
|
|
# 5Gi was too tight: 3d of this cluster's series volume (~1.2Gi/day of blocks) plus a WAL that
|
|
# ballooned past 1Gi overran the disk, wedging WAL commits and firing every absent()/==0 alert
|
|
# falsely. Bumped to 8Gi (still lighter than base 10Gi) so the size-cap below has real headroom.
|
|
- op: replace
|
|
path: /data/MONITORING_PROMETHEUS_PVC_SIZE
|
|
value: "8Gi"
|
|
# Size-cap kept well below the 8Gi PVC (headroom for WAL + head + compaction scratch) so
|
|
# Prometheus prunes oldest blocks by size and never fills the disk on this young cluster.
|
|
- op: replace
|
|
path: /data/MONITORING_PROMETHEUS_RETENTION_SIZE
|
|
value: "5GiB"
|
|
- op: replace
|
|
path: /data/MONITORING_LOKI_PVC_SIZE
|
|
value: "5Gi"
|
|
# single-replica control-plane addons in dev (no HA needed to iterate).
|
|
- op: replace
|
|
path: /data/KEYCLOAK_REPLICAS
|
|
value: "1"
|
|
- op: replace
|
|
path: /data/OAUTH2_PROXY_REPLICAS
|
|
value: "1"
|
|
- op: replace
|
|
path: /data/VELERO_RETENTION
|
|
value: "72h"
|
|
# dev-softened Kyverno for cold-start (base defaults Enforce/Fail/3 = prod posture).
|
|
# Not credential-driven — bootstrap-safety knobs so a cold cluster comes up before
|
|
# admission is proven/HA; flip dev to the strict posture once green.
|
|
- op: replace
|
|
path: /data/KYVERNO_VALIDATION_ACTION
|
|
value: "Audit"
|
|
- op: replace
|
|
path: /data/KYVERNO_FORCE_FAILURE_POLICY_IGNORE
|
|
value: "true"
|
|
# 2 (not base-default 1): the webhook is fail-closed, so a single-pod restart 502s every
|
|
# admission-gated write cluster-wide. Two replicas give a fail-closed webhook availability
|
|
# headroom during a restart/upgrade while staying dev-modest (prod runs 3).
|
|
- op: replace
|
|
path: /data/KYVERNO_ADMISSION_REPLICAS
|
|
value: "2"
|