jakman85andClaude Opus 4.8
960cab937b
oauth2-proxy is a leaf forward-auth service that depends on Keycloak + trusted wildcard TLS + split-horizon DNS. It lived in the gating `platform` Flux Kustomization (wait:true), so on a cold start it Failed until those dependencies existed, keeping the whole `platform` tier not-Ready. That in turn blocked `platform-config` — which brings up the very Keycloak instance, wildcard Certificate and L2 VIP oauth2-proxy needs (and previously carried oauth2-proxy's own httproute/servicemonitor) — from ever reconciling. A self-gating cold-start deadlock, twice-proven (DECISIONS-LOG sessions 7 & 9). Break the cycle by moving the whole oauth2-proxy component out of the `platform` tier into its own non-gating post-platform Flux Kustomization: - clusters/base/oauth2-proxy.yaml: new Kustomization -> ./platform/oauth2-proxy, dependsOn platform-config, wait:false, substituteFrom cluster-vars. - clusters/base/kustomization.yaml: reference the new Kustomization. - platform/kustomization.yaml: drop the oauth2-proxy component (no longer in the gating tier). - platform/oauth2-proxy/kustomization.yaml: pull in httproute + servicemonitor so the full component travels together. - platform-config/kustomization.yaml: drop the in-place oauth2-proxy httproute/servicemonitor refs (moved with the controller). - clusters/dev/kustomization.yaml: add the SOPS decryption patch for the new oauth2-proxy Kustomization (it now owns the encrypted credentials Secret that the `platform` Kustomization used to decrypt). Now `platform` reaches Ready without oauth2-proxy; platform-config brings up Keycloak/TLS/DNS; then oauth2-proxy reconciles against dependencies that already exist. wait:false guarantees its health can never gate anything above it. No config of oauth2-proxy itself changed — only its reconciliation tier. Co-Authored-By: Claude Opus 4.8 <[email protected]>
67 lines
4.4 KiB
YAML
67 lines
4.4 KiB
YAML
---
|
|
# platform-config — the CONFIGS half of the controllers/configs split (flux2 idiomatic).
|
|
#
|
|
# WHY: the `platform` Flux Kustomization installs the CONTROLLERS (HelmReleases) that register CRDs
|
|
# (cert-manager.io, kyverno.io, …). The custom resources BELOW use those CRDs, so applying them in
|
|
# the SAME pass fails atomically ("no matches for kind ClusterIssuer") and blocks the whole platform
|
|
# apply. This Kustomization carries the CRs and is gated `dependsOn: platform` (wait) so the CRDs
|
|
# exist first. See clusters/base/platform-config.yaml + DECISIONS-LOG.
|
|
#
|
|
# Files are referenced IN PLACE via ../platform/... (Flux builds with LoadRestrictionsNone, and the
|
|
# Kyverno policies must stay under platform/kyverno/ for the CI L0 `kyverno test` gate). Build/verify
|
|
# locally with: kubectl kustomize --load-restrictor LoadRestrictionsNone platform-config
|
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
kind: Kustomization
|
|
resources:
|
|
# cert-manager: DNS-01 issuers (need cert-manager.io CRDs from the cert-manager HelmRelease)
|
|
- ../platform/cert-manager/clusterissuers.yaml
|
|
# gateway: the Gateway + wildcard Certificate (Certificate needs cert-manager.io; Gateway needs the
|
|
# GatewayClass, which stays in platform/gateway as a controller resource)
|
|
- ../platform/gateway/gateway.yaml
|
|
- ../platform/gateway/wildcard-certificate.yaml
|
|
# kyverno: ClusterPolicies (need kyverno.io CRDs from the Kyverno HelmRelease)
|
|
- ../platform/kyverno/policies/require-default-deny-baseline.yaml
|
|
- ../platform/kyverno/policies/validate-default-deny-shape.yaml
|
|
- ../platform/kyverno/policies/restrict-host-namespaces.yaml
|
|
- ../platform/kyverno/policies/require-egress-class-label.yaml
|
|
- ../platform/kyverno/policies/require-auth-mode-label.yaml
|
|
- ../platform/kyverno/admission/secret-provenance.yaml
|
|
# cilium: L2 LB IP pool + announcement policy (cilium.io CRDs already registered by Cilium)
|
|
- cilium/l2-ip-pool.yaml
|
|
- cilium/l2-announcement-policy.yaml
|
|
# longhorn/external-snapshotter: the driver.longhorn.io VolumeSnapshotClass (needs the
|
|
# snapshot.storage.k8s.io CRDs from platform/external-snapshotter). Enables CSI snapshots +
|
|
# CDI snapshot smart-clone.
|
|
- ../platform/longhorn/volumesnapshotclass.yaml
|
|
# monitoring: PrometheusRule alert rules (need monitoring.coreos.com CRDs from kube-prometheus-stack)
|
|
- ../platform/monitoring/alerts.yaml
|
|
# kubevirt/cdi (Phase 5): the KubeVirt + CDI CRs consume the kubevirt.io / cdi.kubevirt.io CRDs the
|
|
# operators register; the ServiceMonitors/alerts consume monitoring.coreos.com. All CRs of platform-
|
|
# installed CRDs -> applied here, gated dependsOn platform (F-MON1). The dashboard is a plain
|
|
# ConfigMap kept beside them for locality.
|
|
- ../platform/kubevirt/kubevirt-cr.yaml
|
|
- ../platform/kubevirt/cdi-cr.yaml
|
|
# CDI StorageProfiles: snapshot smart-clone for `longhorn`, and RWX/Block + snapshot clone for the
|
|
# `longhorn-vm` virt-default class (consume cdi.kubevirt.io; clears CDIDefaultStorageClassDegraded).
|
|
- ../platform/kubevirt/cdi-storageprofiles.yaml
|
|
- ../platform/kubevirt/servicemonitor.yaml
|
|
- ../platform/kubevirt/alerts.yaml
|
|
- ../platform/kubevirt/dashboard.yaml
|
|
# --- Phase 6 — Identity / SSO (ADR-0014) ---
|
|
# keycloak: the CNPG DB Cluster + Keycloak instance + realm import (CRs of the CNPG / Keycloak
|
|
# operator CRDs installed by platform/cnpg + platform/keycloak) + its ingress + ServiceMonitor.
|
|
# ${KEYCLOAK_REPLICAS} in keycloak-instance.yaml is resolved by this Kustomization's substituteFrom.
|
|
- ../platform/keycloak/postgres-cluster.yaml
|
|
- ../platform/keycloak/keycloak-instance.yaml
|
|
- ../platform/keycloak/keycloak-realm-import.yaml
|
|
- ../platform/keycloak/k8s-oidc-rbac.yaml # maps OIDC groups -> cluster roles (kubectl SSO RBAC)
|
|
- ../platform/keycloak/httproute.yaml
|
|
- ../platform/keycloak/servicemonitor.yaml
|
|
# oauth2-proxy: forward-auth httproute + ServiceMonitor MOVED OUT of platform-config. They now
|
|
# travel with the oauth2-proxy controller in its own post-platform Flux Kustomization
|
|
# (clusters/base/oauth2-proxy.yaml, dependsOn platform-config). Keeping them here — while the
|
|
# controller sat in the gating `platform` tier — was one half of the twice-proven cold-start
|
|
# deadlock (platform never Ready -> platform-config never reconciles -> these never apply).
|
|
# monitoring SSO proof: Grafana native-OIDC login ingress (Gateway API CR)
|
|
- ../platform/monitoring/grafana-httproute.yaml
|