keycloak: README — operator+CNPG design, runbook, hand-offs

platform/keycloak/README.md

@@ -1,30 +1,65 @@
 # platform/keycloak
 
-**Keycloak** — the platform Identity Provider (SSO via OIDC + SAML).
+**Keycloak** — the platform Identity Provider (SSO via OIDC + SAML), onboarded early (Phase 6 IdP).
 
 - **Governing ADR:** [ADR-0014 — IdP: Keycloak](../../../home-network-k3s/decisions/adr-0014-idp-keycloak.md)
 - **Spec:** [docs/15 — Identity & Access](../../../home-network-k3s/docs/15-identity-and-access.md)
 - **Serves:** R-IDP-01..06
 - **Owner:** WS-SEC
 
-## What this stub is
+## Image / tooling strategy (F-KY1 landmine avoided)
 
-Flux `HelmRelease` for the Keycloak instance with **PostgreSQL on Longhorn** (DR-critical). Replica
-count is an overlay knob (`${KEYCLOAK_REPLICAS}`, R-ENV-02). Admin/DB credentials referenced **by
-name**, **SOPS-encrypted** (WS-SEC). Realms/clients/groups are config-as-code (Phase 6).
+The original stub used the **Bitnami Keycloak chart + bundled Bitnami PostgreSQL**. Bitnami's free
+Docker Hub images were purged (DECISIONS-LOG **F-KY1**) → that path ImagePullBackOffs. Replaced with:
 
-> The ADR allows the **Keycloak Operator OR Helm** for the instance; this stub uses Helm — swap to
-> the Operator if WS-SEC prefers CR-driven config.
+- **Keycloak: the official Keycloak Operator** (`quay.io/keycloak/keycloak-operator` +
+  `quay.io/keycloak/keycloak`, v26.0.6) driving a **`Keycloak` CR** + a **`KeycloakRealmImport` CR** for
+  config-as-code (ADR-0014 permits Operator OR Helm; Operator gives clean CR-driven realm reconcile).
+- **Database: CloudNativePG** (`ghcr.io/cloudnative-pg`, see [`../cnpg`](../cnpg)) — a `Cluster` on the
+  3-replica **`longhorn`** StorageClass (DR-critical, R-IDP-05), with declarative backup hooks.
 
-## Contract (SSO/auth)
+quay.io + ghcr.io are unaffected by the Bitnami purge. Operator manifests are pinned upstream release
+files pulled as remote kustomize bases (the kubevirt-stub pattern); Renovate maintains the version.
 
-The "SSO/auth contract" (`roadmap/workstreams.md`, WS-SEC): a ServiceDefinition's `auth` block maps to
-a Keycloak client (native OIDC) or oauth2-proxy protection (forward-auth — see
-[`../oauth2-proxy`](../oauth2-proxy)). Consumed by WS-APPS/WS-C2.
+## Layout (controllers / configs split — F-MON1)
 
-## TODO (later phases — onboarded EARLY in Phase 6)
+| File | Applied by | What |
+|---|---|---|
+| `kustomization.yaml` (namespace, operator CRDs+Deployment, admin secret, dashboard) | **platform** | controllers |
+| `keycloak-instance.yaml` (`Keycloak` CR, `${KEYCLOAK_REPLICAS}`) | **platform-config** | config |
+| `postgres-cluster.yaml` (CNPG `Cluster` on longhorn) | **platform-config** | config |
+| `keycloak-realm-import.yaml` (`KeycloakRealmImport`: realm/groups/clients/MFA) | **platform-config** | config |
+| `httproute.yaml` (keycloak.home.keithjackson.dev) | **platform-config** | config |
+| `servicemonitor.yaml` (Prometheus scrape) | **platform-config** | config |
 
-- Realms/clients/groups as code (keycloak-config-cli / realm-import); MFA for privileged groups.
-- **DR-critical:** off-cluster DB backup (velero/) + documented **break-glass** local-admin path.
-- Exceptions that keep native auth (do not force SSO): Plex/Seerr (Plex auth), Minecraft
-  (Mojang/Microsoft), UniFi (native) — ADR-0014.
+Realm config (groups `platform-admins`/`media-users`/`viewers`, OIDC clients `grafana`+`oauth2-proxy`,
+brute-force ON, and a **conditional MFA browser flow that REQUIRES OTP for platform-admins**, R-IDP-04)
+reconciles purely from git — no console clicking (R-IDP-03).
+
+## Admin credential + break-glass (R-IDP-04 / R-IDP-05)
+
+The provisioned `keycloak-admin-credentials.sops.yaml` (keys `admin-user`/`admin-password`) is injected
+as the temporary **bootstrap/break-glass admin** via `KC_BOOTSTRAP_ADMIN_*` env (podTemplate escape
+hatch) — this consumes the secret **as provisioned, without re-keying it** (the operator's native
+`bootstrapAdmin` wants `username`/`password` keys, which would require editing the encrypted file).
+The Keycloak bootstrap admin is the documented break-glass path so an IdP/SSO outage does not lock the
+operator out (ADR-0014); Grafana likewise keeps its local admin login until SSO is proven.
+
+## Operator hand-offs (fill before / during deploy)
+
+1. **`keycloak-realm-clients.sops.yaml`** (keycloak ns) — from `.sops.yaml.example`. Two client
+   secrets (`grafana-client-secret`, `oauth2-proxy-client-secret`), `openssl rand -hex 32` each. The
+   realm import injects them into the Keycloak clients via `spec.placeholders`. Add to
+   `kustomization.yaml` once filled. The realm import stays PENDING until this exists.
+2. **`../oauth2-proxy/oauth2-proxy-credentials.sops.yaml`** (oauth2-proxy ns) — `client-id: oauth2-proxy`,
+   `client-secret` = the SAME `oauth2-proxy-client-secret` above, `cookie-secret` = `openssl rand -base64 32`.
+3. **`../monitoring/grafana-oidc-credentials.sops.yaml`** (monitoring ns) — `client-secret` = the SAME
+   `grafana-client-secret` above.
+
+(Cross-namespace Secrets can't be shared, so each client secret is stored twice — once for the realm
+import in `keycloak`, once for the consuming app.) See the two `.sops.yaml.example` templates for the
+exact keys + steps. No plaintext secret is ever committed.
+
+## Exceptions that keep native auth (do not force SSO) — ADR-0014
+
+Plex/Seerr (Plex auth), Minecraft (Mojang/Microsoft), UniFi (native).