jakman85andClaude Opus 5.5
17eed6e8ae
Review findings on the passkey reconciler, fixed:
1. HIGH - the bind was automatic and rollback did not stick. A git-declared switch,
`bind-browser-flow` in passkey-desired-state.yaml, now decides whether pass 3 may
PUT browserFlow. Default "false": build + verify the candidate (or reuse a verified
one), log WOULD BIND <alias>, never touch the binding - including after an earlier
bind, so an operator's rollback sticks, and a hand-edited bound candidate is left
alone rather than shadowed by an -r2 build. "true" binds as before. The renderer
carries the one hand-edited line through and --check refuses anything but the
quoted strings "true"/"false". --no-bind can only narrow it.
2. HIGH - the protected-subtree guard read `alias` on sub-flow rows. Keycloak 26.0.6
AuthenticationManagementResource.recurseExecutions sets displayName =
flowRef.getAlias() and flowId on a sub-flow row and never sets alias (alias is the
CONFIG alias on authenticator rows), so the protected set was only the top-level
alias. Sub-flows are now named by displayName (getFlows() lists top-level flows
only, so flowId cannot be resolved through it); a sub-flow row without one leaves
the subtree undetermined and the flow pass fail-closed. The selftest stub now
returns 26.0.6-shaped rows, and a real run is asserted to refuse writes into
browser-mfa-forms and browser-mfa-conditional-otp.
3. MEDIUM - PUT /flows/{alias}/executions was screened on the path alias only, but
updateExecutions resolves the execution realm-wide by body id. body.id is now
screened against the protected ids AND required to be in the tree this run is
building (recorded from the build's own readback); the config POST is held to the
same build scope; and the body may carry only id/requirement/priority (the same
handler renames any flow named by flowId).
4. Stale "audit-only" docs rewritten (platform-config kustomization, chainsaw test,
desired-state header and its renderer template, README) to describe what ships.
5. backoffLimit 0 (a retried deterministic build failure burns the candidate-alias
budget); the stale "Renovate does not manage image: tags" comment corrected and the
image pinned by index digest.
6. The ~160-line header duplicated in the YAML and the script is now kept once, in the
script, with a short pointer above it.
Selftest: 259 assertions (was 216). mutations.py: 40 injected regressions (was 30),
including one per finding above, all caught.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
168 lines
12 KiB
YAML
168 lines
12 KiB
YAML
---
|
|
# platform-config — the CONFIGS half of the controllers/configs split (flux2 idiomatic).
|
|
#
|
|
# WHY: the `platform` Flux Kustomization installs the CONTROLLERS (HelmReleases) that register CRDs
|
|
# (cert-manager.io, kyverno.io, …). The custom resources BELOW use those CRDs, so applying them in
|
|
# the SAME pass fails atomically ("no matches for kind ClusterIssuer") and blocks the whole platform
|
|
# apply. This Kustomization carries the CRs and is gated `dependsOn: platform` (wait) so the CRDs
|
|
# exist first. See clusters/base/platform-config.yaml + DECISIONS-LOG.
|
|
#
|
|
# Files are referenced IN PLACE via ../platform/... (Flux builds with LoadRestrictionsNone, and the
|
|
# Kyverno policies must stay under platform/kyverno/ for the CI L0 `kyverno test` gate). Build/verify
|
|
# locally with: kubectl kustomize --load-restrictor LoadRestrictionsNone platform-config
|
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
kind: Kustomization
|
|
resources:
|
|
# cert-manager: DNS-01 issuers (need cert-manager.io CRDs from the cert-manager HelmRelease)
|
|
- ../platform/cert-manager/clusterissuers.yaml
|
|
# gateway: the Gateway + wildcard Certificate (Certificate needs cert-manager.io; Gateway needs the
|
|
# GatewayClass, which stays in platform/gateway as a controller resource)
|
|
- ../platform/gateway/gateway.yaml
|
|
- ../platform/gateway/wildcard-certificate.yaml
|
|
# kyverno: ClusterPolicies (need kyverno.io CRDs from the Kyverno HelmRelease)
|
|
- ../platform/kyverno/policies/require-default-deny-baseline.yaml
|
|
- ../platform/kyverno/policies/validate-default-deny-shape.yaml
|
|
- ../platform/kyverno/policies/restrict-host-namespaces.yaml
|
|
- ../platform/kyverno/policies/require-egress-class-label.yaml
|
|
- ../platform/kyverno/policies/require-auth-mode-label.yaml
|
|
# COLD-START FIX (2026-09-06, prod): a MUTATE policy that must travel WITH the two label
|
|
# policies above. CDI generates its own short-lived worker pods (importer-*, uploader-*,
|
|
# cloner-*) inside app namespaces; nothing in git can label them, so both policies DENIED them
|
|
# and `DataVolume unifi/unifi-rootdisk` sat in ImportScheduled with no disk. This policy adds
|
|
# the two labels to pods created by ServiceAccount cdi/cdi-sa, so the requirement is SATISFIED
|
|
# rather than bypassed. Applying it here (not later) matters: if it lands after the validating
|
|
# policies, the next import is blocked in the gap.
|
|
- ../platform/kyverno/policies/label-cdi-worker-pods.yaml
|
|
- ../platform/kyverno/admission/secret-provenance.yaml
|
|
# R-SEC-02 — bounds what the conformance CI trigger may create. MUST be listed HERE: this
|
|
# Kustomization is what actually applies ClusterPolicies. Adding a policy only to
|
|
# platform/kyverno/admission/kustomization.yaml is a SILENT NO-OP — that file is reference
|
|
# only ("kept as reference; THIS file is the buildable render root" in
|
|
# platform/kyverno/kustomization.yaml, which itself ships only the namespace + HelmRelease).
|
|
# That is exactly how this policy shipped green and never reached the cluster on 2026-09-29,
|
|
# leaving the conformance-trigger identity live with nothing bounding it.
|
|
- ../platform/kyverno/admission/conformance-trigger-job-shape.yaml
|
|
# R-SEC-02 — bounds what the PM agent dispatcher may create (platform/pm-agent/). SAME
|
|
# REASON THIS LINE EXISTS AS THE ONE ABOVE, and the same consequence if it is omitted: this
|
|
# Kustomization is the only list that applies ClusterPolicies, so adding a policy to
|
|
# platform/kyverno/admission/kustomization.yaml alone is a SILENT NO-OP. The policy would
|
|
# pass `kyverno test`, pass the gate, and never reach the cluster — leaving a
|
|
# ServiceAccount that can create Jobs in a namespace, driven by a model reading untrusted
|
|
# issue text, with nothing bounding the shape of what it creates.
|
|
- ../platform/kyverno/admission/pm-agent-job-shape.yaml
|
|
# cilium: L2 LB IP pool + announcement policy (cilium.io CRDs already registered by Cilium)
|
|
- cilium/l2-ip-pool.yaml
|
|
- cilium/l2-announcement-policy.yaml
|
|
# longhorn/external-snapshotter: the driver.longhorn.io VolumeSnapshotClass (needs the
|
|
# snapshot.storage.k8s.io CRDs from platform/external-snapshotter). Enables CSI snapshots +
|
|
# CDI snapshot smart-clone.
|
|
- ../platform/longhorn/volumesnapshotclass.yaml
|
|
# monitoring: PrometheusRule alert rules (need monitoring.coreos.com CRDs from kube-prometheus-stack)
|
|
- ../platform/monitoring/alerts.yaml
|
|
# Cluster-wide PVC capacity rules. A SEPARATE FILE from alerts.yaml, whose only PVC-fill rule
|
|
# (MonitoringVolumeFillingUp) is scoped namespace="monitoring" — so before this, no database PVC
|
|
# was covered by any fill alert. Partitioned by namespace so the two never double-fire; see the
|
|
# file header. (2026-09-06, from the CNPG WAL/archiving incident.)
|
|
- ../platform/monitoring/pvc-capacity-alerts.yaml
|
|
# PodMonitor for the six Flux controllers — the ONLY thing that puts `gotk_*` in the TSDB.
|
|
# kube-prometheus-stack does not scrape flux-system (the controllers are bootstrapped, expose no
|
|
# Service, and no chart value covers them), so until this landed the five gotk_* rules in
|
|
# alerts.yaml above were structurally inert: verified on prod 2026-09-13,
|
|
# `count({__name__=~"gotk.*"})` returned nothing. A CR of monitoring.coreos.com -> this tier, same
|
|
# as alerts.yaml. (2026-09-13, from the inert-alert audit.)
|
|
- ../platform/monitoring/flux-podmonitor.yaml
|
|
# kubevirt/cdi (Phase 5): the KubeVirt + CDI CRs consume the kubevirt.io / cdi.kubevirt.io CRDs the
|
|
# operators register; the ServiceMonitors/alerts consume monitoring.coreos.com. All CRs of platform-
|
|
# installed CRDs -> applied here, gated dependsOn platform (F-MON1). The dashboard is a plain
|
|
# ConfigMap kept beside them for locality.
|
|
- ../platform/kubevirt/kubevirt-cr.yaml
|
|
- ../platform/kubevirt/cdi-cr.yaml
|
|
# *** DO NOT ADD ../platform/kubevirt/cdi-storageprofiles.yaml BACK HERE. IT DEADLOCKS THIS TIER. ***
|
|
# The CDI StorageProfiles were MOVED OUT on 2026-09-05 into their own Flux Kustomization
|
|
# (clusters/base/platform-config-cdi.yaml -> ./platform-config-cdi, dependsOn platform-config).
|
|
#
|
|
# `StorageProfile`'s CRD is NOT registered by the CDI operator INSTALL — cdi-operator.yaml ships only
|
|
# `cdis.cdi.kubevirt.io`. cdi-operator creates `storageprofiles` (and datavolumes, datasources, …)
|
|
# only AFTER it has reconciled the `CDI` CR two lines above. Flux server-side DRY-RUNS EVERY object
|
|
# in a Kustomization before applying ANY of them, so having both in this list meant: unknown kind
|
|
# `StorageProfile` -> the whole pass aborts -> the `CDI` CR is never applied -> the CRD is never
|
|
# created -> THIS TIER CAN NEVER CONVERGE, and every tier that dependsOn it is blocked forever.
|
|
# A true deadlock, not a race. Observed on prod 2026-09-05 (cdi-operator Available 12h, no CR):
|
|
# StorageProfile/longhorn dry-run failed: no matches for kind "StorageProfile" in version
|
|
# "cdi.kubevirt.io/v1beta1"
|
|
# Invisible on dev, whose CDI CRDs have existed for months. See DECISIONS-LOG 2026-09-05.
|
|
#
|
|
# THE RULE: a CR whose CRD is created by a controller AT RUNTIME (rather than by a manifest in git)
|
|
# is SECOND-ORDER and belongs in ./platform-config-cdi, not here.
|
|
- ../platform/kubevirt/servicemonitor.yaml
|
|
- ../platform/kubevirt/alerts.yaml
|
|
- ../platform/kubevirt/dashboard.yaml
|
|
# --- Phase 6 — Identity / SSO (ADR-0014) ---
|
|
# keycloak: the CNPG DB Cluster + Keycloak instance + realm import (CRs of the CNPG / Keycloak
|
|
# operator CRDs installed by platform/cnpg + platform/keycloak) + its ingress + ServiceMonitor.
|
|
# ${KEYCLOAK_REPLICAS} in keycloak-instance.yaml is resolved by this Kustomization's substituteFrom.
|
|
# cnpg: CloudNativePG platform-health alerts (WAL archiving / recoverability). PrometheusRule is a
|
|
# CR of monitoring.coreos.com, so it belongs to this pass, not the platform controllers pass —
|
|
# identical wiring to platform/kubevirt/alerts.yaml. Added 2026-09-06 after all three prod CNPG
|
|
# clusters ran 38h+ with ContinuousArchiving=False and firstRecoverabilityPoint=null, unalerted.
|
|
- ../platform/cnpg/alerts.yaml
|
|
- ../platform/keycloak/postgres-cluster.yaml
|
|
- ../platform/keycloak/keycloak-instance.yaml
|
|
- ../platform/keycloak/keycloak-realm-import.yaml
|
|
# The realm import is a ONE-SHOT that fails OPEN: an unresolved secret placeholder is
|
|
# stored LITERALLY and the import still reports success, then rests at Done=True and
|
|
# never re-runs. This CronJob is the level-triggered loop that corrects that drift
|
|
# (2026-09-06 incident; see the file header). It belongs in THIS tier, not ./platform,
|
|
# so it applies after the keycloak-realm-clients Secret (rendered by ./platform) and
|
|
# alongside the import whose failure mode it covers.
|
|
- ../platform/keycloak/client-secret-reconciler.yaml
|
|
# The DESIRED STATE for every environment-derived client URL (redirectUris, webOrigins,
|
|
# rootUrl, baseUrl, adminUrl), GENERATED from keycloak-realm-import.yaml by
|
|
# ci/render-keycloak-client-urls.py and gated by the L0 check. The reconciler pod has no
|
|
# Kubernetes API access (automountServiceAccountToken: false), so it cannot read the
|
|
# KeycloakRealmImport CR — its desired state must be MOUNTED, hence this ConfigMap.
|
|
#
|
|
# IT MUST BE RENDERED BY THIS SAME KUSTOMIZATION as the realm import above: both are then
|
|
# substituted in one pass by this Kustomization's postBuild.substituteFrom (cluster-vars),
|
|
# so a ${BASE_DOMAIN} token in either resolves to the same value and the two declarations
|
|
# cannot disagree about the domain.
|
|
- ../platform/keycloak/client-urls.yaml
|
|
# PASSKEYS — the same one-shot failure, for the WebAuthn passwordless policy and the
|
|
# browser login flow. Measured on prod 2026-10-18, hours after the passkey declaration
|
|
# merged: the KeycloakRealmImport read Done=True / Started=False with the whole
|
|
# webAuthnPolicyPasswordless* block and the restructured browser-mfa flow INERT. This
|
|
# CronJob converges the policy and the enrolment required action, and builds the
|
|
# declared browser flow BLUE/GREEN: under a new alias, verified by readback, and
|
|
# structurally unable to write the flow that is currently bound (a half-rewritten bound
|
|
# flow is a total SSO outage, and this repo has had one). It BINDS the verified flow
|
|
# (one atomic PUT of browserFlow) ONLY when passkey-desired-state.yaml sets
|
|
# bind-browser-flow: "true"; the default "false" leaves the realm's binding untouched.
|
|
# See the script header in that file for the full rationale.
|
|
- ../platform/keycloak/passkey-reconciler.yaml
|
|
# Its DESIRED STATE, GENERATED from keycloak-realm-import.yaml by
|
|
# ci/render-keycloak-passkey-policy.py and gated by the L0 check. Same reason as
|
|
# client-urls.yaml above: the reconciler pod has no Kubernetes API access
|
|
# (automountServiceAccountToken: false), so it cannot read the KeycloakRealmImport CR
|
|
# and its input must be MOUNTED.
|
|
#
|
|
# IT MUST BE RENDERED BY THIS SAME KUSTOMIZATION as the realm import above, for exactly
|
|
# the reason given for client-urls.yaml: webAuthnPolicyPasswordlessRpId is the
|
|
# ${BASE_DOMAIN} token, and a split across Kustomizations would let one token resolve to
|
|
# two different values. An RP ID is baked into every passkey at enrolment and can never
|
|
# be re-keyed, so a domain the two files disagree about is an unrecoverable credential.
|
|
- ../platform/keycloak/passkey-desired-state.yaml
|
|
- ../platform/keycloak/k8s-oidc-rbac.yaml # maps OIDC groups -> cluster roles (kubectl SSO RBAC)
|
|
- ../platform/keycloak/httproute.yaml
|
|
- ../platform/keycloak/servicemonitor.yaml
|
|
# oauth2-proxy: forward-auth httproute + ServiceMonitor MOVED OUT of platform-config. They now
|
|
# travel with the oauth2-proxy controller in its own post-platform Flux Kustomization
|
|
# (clusters/base/oauth2-proxy.yaml, dependsOn platform-config). Keeping them here — while the
|
|
# controller sat in the gating `platform` tier — was one half of the twice-proven cold-start
|
|
# deadlock (platform never Ready -> platform-config never reconciles -> these never apply).
|
|
# monitoring SSO proof: Grafana native-OIDC login ingress (Gateway API CR)
|
|
- ../platform/monitoring/grafana-httproute.yaml
|
|
# headlamp: httproute MOVED OUT of platform-config. It now travels with the headlamp component in
|
|
# its own post-platform Flux Kustomization (clusters/base/headlamp.yaml, dependsOn platform-config).
|
|
# Keeping it here — while the HelmRelease sat in the gating `platform` tier — meant a headlamp
|
|
# crashloop stalled platform-config (and everything below it). See DECISIONS-LOG "Headlamp gating stall".
|