--- # platform-config — the CONFIGS half of the controllers/configs split (flux2 idiomatic). # # WHY: the `platform` Flux Kustomization installs the CONTROLLERS (HelmReleases) that register CRDs # (cert-manager.io, kyverno.io, …). The custom resources BELOW use those CRDs, so applying them in # the SAME pass fails atomically ("no matches for kind ClusterIssuer") and blocks the whole platform # apply. This Kustomization carries the CRs and is gated `dependsOn: platform` (wait) so the CRDs # exist first. See clusters/base/platform-config.yaml + DECISIONS-LOG. # # Files are referenced IN PLACE via ../platform/... (Flux builds with LoadRestrictionsNone, and the # Kyverno policies must stay under platform/kyverno/ for the CI L0 `kyverno test` gate). Build/verify # locally with: kubectl kustomize --load-restrictor LoadRestrictionsNone platform-config apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: # cert-manager: DNS-01 issuers (need cert-manager.io CRDs from the cert-manager HelmRelease) - ../platform/cert-manager/clusterissuers.yaml # gateway: the Gateway + wildcard Certificate (Certificate needs cert-manager.io; Gateway needs the # GatewayClass, which stays in platform/gateway as a controller resource) - ../platform/gateway/gateway.yaml - ../platform/gateway/wildcard-certificate.yaml # kyverno: ClusterPolicies (need kyverno.io CRDs from the Kyverno HelmRelease) - ../platform/kyverno/policies/require-default-deny-baseline.yaml - ../platform/kyverno/policies/validate-default-deny-shape.yaml - ../platform/kyverno/policies/restrict-host-namespaces.yaml - ../platform/kyverno/policies/require-egress-class-label.yaml - ../platform/kyverno/policies/require-auth-mode-label.yaml # COLD-START FIX (2026-09-06, prod): a MUTATE policy that must travel WITH the two label # policies above. CDI generates its own short-lived worker pods (importer-*, uploader-*, # cloner-*) inside app namespaces; nothing in git can label them, so both policies DENIED them # and `DataVolume unifi/unifi-rootdisk` sat in ImportScheduled with no disk. This policy adds # the two labels to pods created by ServiceAccount cdi/cdi-sa, so the requirement is SATISFIED # rather than bypassed. Applying it here (not later) matters: if it lands after the validating # policies, the next import is blocked in the gap. - ../platform/kyverno/policies/label-cdi-worker-pods.yaml - ../platform/kyverno/admission/secret-provenance.yaml # R-SEC-02 — bounds what the conformance CI trigger may create. MUST be listed HERE: this # Kustomization is what actually applies ClusterPolicies. Adding a policy only to # platform/kyverno/admission/kustomization.yaml is a SILENT NO-OP — that file is reference # only ("kept as reference; THIS file is the buildable render root" in # platform/kyverno/kustomization.yaml, which itself ships only the namespace + HelmRelease). # That is exactly how this policy shipped green and never reached the cluster on 2026-09-29, # leaving the conformance-trigger identity live with nothing bounding it. - ../platform/kyverno/admission/conformance-trigger-job-shape.yaml # cilium: L2 LB IP pool + announcement policy (cilium.io CRDs already registered by Cilium) - cilium/l2-ip-pool.yaml - cilium/l2-announcement-policy.yaml # longhorn/external-snapshotter: the driver.longhorn.io VolumeSnapshotClass (needs the # snapshot.storage.k8s.io CRDs from platform/external-snapshotter). Enables CSI snapshots + # CDI snapshot smart-clone. - ../platform/longhorn/volumesnapshotclass.yaml # monitoring: PrometheusRule alert rules (need monitoring.coreos.com CRDs from kube-prometheus-stack) - ../platform/monitoring/alerts.yaml # Cluster-wide PVC capacity rules. A SEPARATE FILE from alerts.yaml, whose only PVC-fill rule # (MonitoringVolumeFillingUp) is scoped namespace="monitoring" — so before this, no database PVC # was covered by any fill alert. Partitioned by namespace so the two never double-fire; see the # file header. (2026-09-06, from the CNPG WAL/archiving incident.) - ../platform/monitoring/pvc-capacity-alerts.yaml # PodMonitor for the six Flux controllers — the ONLY thing that puts `gotk_*` in the TSDB. # kube-prometheus-stack does not scrape flux-system (the controllers are bootstrapped, expose no # Service, and no chart value covers them), so until this landed the five gotk_* rules in # alerts.yaml above were structurally inert: verified on prod 2026-09-13, # `count({__name__=~"gotk.*"})` returned nothing. A CR of monitoring.coreos.com -> this tier, same # as alerts.yaml. (2026-09-13, from the inert-alert audit.) - ../platform/monitoring/flux-podmonitor.yaml # kubevirt/cdi (Phase 5): the KubeVirt + CDI CRs consume the kubevirt.io / cdi.kubevirt.io CRDs the # operators register; the ServiceMonitors/alerts consume monitoring.coreos.com. All CRs of platform- # installed CRDs -> applied here, gated dependsOn platform (F-MON1). The dashboard is a plain # ConfigMap kept beside them for locality. - ../platform/kubevirt/kubevirt-cr.yaml - ../platform/kubevirt/cdi-cr.yaml # *** DO NOT ADD ../platform/kubevirt/cdi-storageprofiles.yaml BACK HERE. IT DEADLOCKS THIS TIER. *** # The CDI StorageProfiles were MOVED OUT on 2026-09-05 into their own Flux Kustomization # (clusters/base/platform-config-cdi.yaml -> ./platform-config-cdi, dependsOn platform-config). # # `StorageProfile`'s CRD is NOT registered by the CDI operator INSTALL — cdi-operator.yaml ships only # `cdis.cdi.kubevirt.io`. cdi-operator creates `storageprofiles` (and datavolumes, datasources, …) # only AFTER it has reconciled the `CDI` CR two lines above. Flux server-side DRY-RUNS EVERY object # in a Kustomization before applying ANY of them, so having both in this list meant: unknown kind # `StorageProfile` -> the whole pass aborts -> the `CDI` CR is never applied -> the CRD is never # created -> THIS TIER CAN NEVER CONVERGE, and every tier that dependsOn it is blocked forever. # A true deadlock, not a race. Observed on prod 2026-09-05 (cdi-operator Available 12h, no CR): # StorageProfile/longhorn dry-run failed: no matches for kind "StorageProfile" in version # "cdi.kubevirt.io/v1beta1" # Invisible on dev, whose CDI CRDs have existed for months. See DECISIONS-LOG 2026-09-05. # # THE RULE: a CR whose CRD is created by a controller AT RUNTIME (rather than by a manifest in git) # is SECOND-ORDER and belongs in ./platform-config-cdi, not here. - ../platform/kubevirt/servicemonitor.yaml - ../platform/kubevirt/alerts.yaml - ../platform/kubevirt/dashboard.yaml # --- Phase 6 — Identity / SSO (ADR-0014) --- # keycloak: the CNPG DB Cluster + Keycloak instance + realm import (CRs of the CNPG / Keycloak # operator CRDs installed by platform/cnpg + platform/keycloak) + its ingress + ServiceMonitor. # ${KEYCLOAK_REPLICAS} in keycloak-instance.yaml is resolved by this Kustomization's substituteFrom. # cnpg: CloudNativePG platform-health alerts (WAL archiving / recoverability). PrometheusRule is a # CR of monitoring.coreos.com, so it belongs to this pass, not the platform controllers pass — # identical wiring to platform/kubevirt/alerts.yaml. Added 2026-09-06 after all three prod CNPG # clusters ran 38h+ with ContinuousArchiving=False and firstRecoverabilityPoint=null, unalerted. - ../platform/cnpg/alerts.yaml - ../platform/keycloak/postgres-cluster.yaml - ../platform/keycloak/keycloak-instance.yaml - ../platform/keycloak/keycloak-realm-import.yaml # The realm import is a ONE-SHOT that fails OPEN: an unresolved secret placeholder is # stored LITERALLY and the import still reports success, then rests at Done=True and # never re-runs. This CronJob is the level-triggered loop that corrects that drift # (2026-09-06 incident; see the file header). It belongs in THIS tier, not ./platform, # so it applies after the keycloak-realm-clients Secret (rendered by ./platform) and # alongside the import whose failure mode it covers. - ../platform/keycloak/client-secret-reconciler.yaml # The DESIRED STATE for every environment-derived client URL (redirectUris, webOrigins, # rootUrl, baseUrl, adminUrl), GENERATED from keycloak-realm-import.yaml by # ci/render-keycloak-client-urls.py and gated by the L0 check. The reconciler pod has no # Kubernetes API access (automountServiceAccountToken: false), so it cannot read the # KeycloakRealmImport CR — its desired state must be MOUNTED, hence this ConfigMap. # # IT MUST BE RENDERED BY THIS SAME KUSTOMIZATION as the realm import above: both are then # substituted in one pass by this Kustomization's postBuild.substituteFrom (cluster-vars), # so a ${BASE_DOMAIN} token in either resolves to the same value and the two declarations # cannot disagree about the domain. - ../platform/keycloak/client-urls.yaml - ../platform/keycloak/k8s-oidc-rbac.yaml # maps OIDC groups -> cluster roles (kubectl SSO RBAC) - ../platform/keycloak/httproute.yaml - ../platform/keycloak/servicemonitor.yaml # oauth2-proxy: forward-auth httproute + ServiceMonitor MOVED OUT of platform-config. They now # travel with the oauth2-proxy controller in its own post-platform Flux Kustomization # (clusters/base/oauth2-proxy.yaml, dependsOn platform-config). Keeping them here — while the # controller sat in the gating `platform` tier — was one half of the twice-proven cold-start # deadlock (platform never Ready -> platform-config never reconciles -> these never apply). # monitoring SSO proof: Grafana native-OIDC login ingress (Gateway API CR) - ../platform/monitoring/grafana-httproute.yaml # headlamp: httproute MOVED OUT of platform-config. It now travels with the headlamp component in # its own post-platform Flux Kustomization (clusters/base/headlamp.yaml, dependsOn platform-config). # Keeping it here — while the HelmRelease sat in the gating `platform` tier — meant a headlamp # crashloop stalled platform-config (and everything below it). See DECISIONS-LOG "Headlamp gating stall".