home-network-k3s-impl/platform/keycloak/README.md

# platform/keycloak

**Keycloak** — the platform Identity Provider (SSO via OIDC + SAML), onboarded early (Phase 6 IdP).

- **Governing ADR:** [ADR-0014 — IdP: Keycloak](../../../home-network-k3s/decisions/adr-0014-idp-keycloak.md)
- **Spec:** [docs/15 — Identity & Access](../../../home-network-k3s/docs/15-identity-and-access.md)
- **Serves:** R-IDP-01..06
- **Owner:** WS-SEC

## Image / tooling strategy (F-KY1 landmine avoided)

The original stub used the **Bitnami Keycloak chart + bundled Bitnami PostgreSQL**. Bitnami's free
Docker Hub images were purged (DECISIONS-LOG **F-KY1**) → that path ImagePullBackOffs. Replaced with:

- **Keycloak: the official Keycloak Operator** (`quay.io/keycloak/keycloak-operator` +
  `quay.io/keycloak/keycloak`, v26.0.6) driving a **`Keycloak` CR** + a **`KeycloakRealmImport` CR** for
  config-as-code (ADR-0014 permits Operator OR Helm; Operator gives clean CR-driven realm reconcile).
- **Database: CloudNativePG** (`ghcr.io/cloudnative-pg`, see [`../cnpg`](../cnpg)) — a `Cluster` on the
  3-replica **`longhorn`** StorageClass (DR-critical, R-IDP-05), with declarative backup hooks.

quay.io + ghcr.io are unaffected by the Bitnami purge. Operator manifests are pinned upstream release
files pulled as remote kustomize bases (the kubevirt-stub pattern). Renovate tracks the version via
`# renovate:` markers on those URLs (since 2026-10-30 — before that it could not see them) and never
auto-merges it; see [Upgrading Keycloak](#upgrading-keycloak-the-operator-is-the-version).

## Layout (controllers / configs split — F-MON1)

| File | Applied by | What |
|---|---|---|
| `kustomization.yaml` (namespace, operator CRDs+Deployment, admin secret, dashboard) | **platform** | controllers |
| `keycloak-instance.yaml` (`Keycloak` CR, `${KEYCLOAK_REPLICAS}`) | **platform-config** | config |
| `postgres-cluster.yaml` (CNPG `Cluster` on longhorn) | **platform-config** | config |
| `keycloak-realm-import.yaml` (`KeycloakRealmImport`: realm/groups/clients/MFA) | **platform-config** | config |
| `client-secret-reconciler.yaml` (CronJob: client secrets **and** client URLs ← git, every 15m) | **platform-config** | config |
| `client-urls.yaml` (**GENERATED** ConfigMap: the client-URL desired state) | **platform-config** | config |
| `passkey-reconciler.yaml` (CronJob: WebAuthn passwordless policy + enrolment action + the **browser flow** ← git every 15m; the flow blue/green, never mutating the bound one, **bound only when git says so**) | **platform-config** | config |
| `passkey-desired-state.yaml` (**GENERATED** ConfigMap: the passkey desired state — plus the one hand-edited line, `bind-browser-flow`) | **platform-config** | config |
| `httproute.yaml` (keycloak.home.keithjackson.dev) | **platform-config** | config |
| `servicemonitor.yaml` (Prometheus scrape) | **platform-config** | config |

Realm config (groups `platform-admins`/`media-users`/`viewers`, OIDC clients `grafana`+`oauth2-proxy`,
brute-force ON, and a **conditional MFA browser flow that REQUIRES OTP for platform-admins**, R-IDP-04)
reconciles purely from git — no console clicking (R-IDP-03).

## Admin credential + break-glass (R-IDP-04 / R-IDP-05)

The provisioned `keycloak-admin-credentials.sops.yaml` (keys `admin-user`/`admin-password`) is injected
as the temporary **bootstrap/break-glass admin** via `KC_BOOTSTRAP_ADMIN_*` env (podTemplate escape
hatch) — this consumes the secret **as provisioned, without re-keying it** (the operator's native
`bootstrapAdmin` wants `username`/`password` keys, which would require editing the encrypted file).
The Keycloak bootstrap admin is the documented break-glass path so an IdP/SSO outage does not lock the
operator out (ADR-0014); Grafana likewise keeps its local admin login until SSO is proven.

## Operator hand-offs (fill before / during deploy)

1. **`keycloak-realm-clients.sops.yaml`** (keycloak ns) — from `.sops.yaml.example`. One
   `<clientId>-client-secret` key per confidential client, `openssl rand -hex 32` each. The realm
   import injects them into the Keycloak clients via `spec.placeholders`. Add to
   `kustomization.yaml` once filled.
   **The key name MUST be `<clientId>-client-secret`** — the reconciler below derives the clientId
   from it, and CI rejects any other name.
   *(This step used to read "the realm import stays PENDING until this exists". It does not — see
   the next section. That false belief is what allowed the 2026-09-06 outage.)*
2. **`../oauth2-proxy/oauth2-proxy-credentials.sops.yaml`** (oauth2-proxy ns) — `client-id: oauth2-proxy`,
   `client-secret` = the SAME `oauth2-proxy-client-secret` above, `cookie-secret` = `openssl rand -base64 32`.
3. **`../monitoring/grafana-oidc-credentials.sops.yaml`** (monitoring ns) — `client-secret` = the SAME
   `grafana-client-secret` above.

(Cross-namespace Secrets can't be shared, so each client secret is stored twice — once for the realm
import in `keycloak`, once for the consuming app.) See the two `.sops.yaml.example` templates for the
exact keys + steps. No plaintext secret is ever committed.

## Client secrets: the realm import fails OPEN, so a reconciler owns them (2026-09-06)

**The realm import is a one-shot that fails open.** Measured on both clusters:

* When the operator **cannot resolve** a `$(env:...)` placeholder it stores **the literal placeholder
  text** as the client secret — and the import Job still exits 0. There is no PENDING state, no
  error condition, and `HasErrors` stays `False`.
* Once complete the CR rests at **`Done=True` / `Started=False` and never re-runs**, so a Secret key
  added later is never picked up.

Together those produced a total SSO outage on prod (all 12 clients holding the literal placeholder;
every login rejected with *"Invalid client or Invalid client credentials"*) and a single broken client
on dev (`oauth2-proxy`, whose key was added after dev's import had already completed). Every cluster
signal was green throughout, including the conformance test that asserts `Done=True`.

Three layers now cover it:

| Layer | Where | What it does |
|---|---|---|
| **Prevent** | `conformance/tests/l0-static/keycloak-client-secret-reconciler` (runs on every merge, unconditionally, from `ci/l0.sh`) | Every placeholder is defined, reads `keycloak-realm-clients`, and names an **existing** key called `<clientId>-client-secret`; public clients declare no secret; and the Flux Kustomization applying the Secret is a real transitive `dependsOn` ancestor (`wait: true`) of the ones applying the import and the reconciler. A missing SOPS key is now unmergeable. |
| **Heal** | `client-secret-reconciler.yaml` — CronJob, every 15m, two passes | Compares each client's **stored** secret with the mounted `keycloak-realm-clients` value and PUTs the **minimal** `{clientId, secret}` representation only where they differ; a second, independent pass does the same for the client URLs against the generated `client-urls.yaml` (see below). Skips correct clients, never logs a secret, holds a hard endpoint allow-list so users/groups/roles/flows are unreachable, and needs no Kubernetes API access (`automountServiceAccountToken: false`). |
| **Detect** | `conformance/tests/l2-behavioral/keycloak-client-secrets-match-git` + alerts `KeycloakClientSecretReconcilerFailing` / `…Stale` | Runs the same script in read-only `--audit` mode against the live realm and fails on any mismatch, literal placeholder **or drifted client URL**; the alerts make the reconciler's silence observable in prod. |

The reconciler's own `--selftest` (run by the L0 gate, which extracts the script from the shipped
ConfigMap) proves compare-then-write, **idempotence** (zero writes on a converged realm),
no-secret-logging, the endpoint allow-list, and fail-closed behaviour.

**Manual repair / one-off run** (the CronJob normally does this):

```sh
kubectl -n keycloak create job --from=cronjob/keycloak-client-secret-reconciler kc-fix-$(date +%s)
kubectl -n keycloak logs -l app.kubernetes.io/name=keycloak-client-secret-reconciler --tail=-1
```

## Client URLs: the same one-shot failure, one field over

Moving an environment's `BASE_DOMAIN` re-renders every declared `redirectUris` / `webOrigins` /
`rootUrl` / `baseUrl` / `adminUrl` in git — and reaches the realm **never**, because a completed
import does not re-run. Measured on prod: stored redirect URIs still on the pre-migration hostname
while git declared the new one. Every OIDC login then fails with a `redirect_uri` mismatch, the
`KeycloakRealmImport` still reads `Done=True`, and nothing alerts.

**Pass 2 of the same CronJob** closes it. It was made a second pass of the existing reconciler rather
than a sibling CronJob for one decisive reason: **a URL write needs no new endpoint.** It is the same
minimal `PUT /admin/realms/<realm>/clients/<uuid>`, and the current URLs already come back in the
allow-listed client lookup — so the hard allow-list that makes `/users`, `/groups`, `/roles` and
`/authentication` unreachable is widened by *nothing*, and is preserved rather than re-declared in a
second copy. (The ConfigMap also carries exactly one script key by L0 contract, so a sibling would
have had to fork ~300 lines of safety-critical code the header explicitly forbids forking.)

**The desired state is generated, not authored.** `client-urls.yaml` is rendered from
`keycloak-realm-import.yaml` by `ci/render-keycloak-client-urls.py`, and the L0 gate runs that
renderer with `--check` on every merge, so the committed file cannot disagree with the import. URL
strings are copied verbatim — a `${BASE_DOMAIN}` token passes through unchanged, and both files are
rendered by the **same** Flux Kustomization, so both are substituted in one pass with one value. The
renderer never parses a hostname and never reads `cluster-vars`: there is no second place the domain
can be decided.

```sh
# regenerate after editing a client URL in the realm import (the gate will demand it)
python3 ci/render-keycloak-client-urls.py --write
```

Properties, each proved by `--selftest` and each shown to be *falsifiable* by
`conformance/tests/l0-static/keycloak-client-secret-reconciler/mutations.py`:

* **Converged ⇒ zero writes.** List fields compare order-insensitively (`sorted`, not `set` —
  duplicates are still drift), because Keycloak does not preserve the authored order of
  `redirectUris`; an order-sensitive compare would rewrite every client forever while reporting
  success.
* **Gaining and losing a URI both work.** The declared list is exact, so a URI removed from git is
  removed from the realm — a stale wildcard left behind quietly widens what the realm will redirect
  to.
* **Undeclared ⇒ unmanaged.** A field the desired state omits is left exactly as it is. Absence never
  means "force to empty".
* **Public clients' URLs *are* managed** (their secrets are not): `kubernetes` is PKCE-public and its
  localhost callbacks are what `kubectl` logins land on.
* **The two passes are independent.** Each reads its own desired state and fails closed for its own
  fields only. A malformed URL ConfigMap cannot stop client-*secret* repair, and an unmounted Secret
  cannot stop URL repair.
* URLs are logged in full (they are public hostnames, and the before/after is what makes a migration
  auditable); secrets are still never logged.

**What the reconciler does NOT fix:** anything outside client secrets and those five URL fields. A
NEW client, a new protocol mapper or a changed scope still needs the realm import to actually re-run
(it will not on its own) or a deliberate admin-API/`kcadm` change. `Done=True` is never evidence that
a realm edit landed.

## Passkeys (WebAuthn Passwordless) — DECLARED IN GIT, RECONCILED INTO THE REALM (2026-10-18)

`keycloak-realm-import.yaml` now declares passkey login: the realm's **WebAuthn Passwordless**
policy (`webAuthnPolicyPasswordless*`), and a `browser-mfa` flow that offers a passkey as an
**ALTERNATIVE** beside the password form. Design decisions and their reasoning are in the manifest
comments, not repeated here. The headline choices:

| Decision | Value | One-line reason |
|---|---|---|
| Additive or replacing? | **Additive** — passkey *or* password | A misfiring mandatory flow locks the single operator out of every SSO app *and* kubectl |
| Enrolment | **Optional**, never a default action | Same reason; `webauthn-register-passwordless` is already enabled-not-forced by default |
| Relying-party ID | `${BASE_DOMAIN}` (not `keycloak.${BASE_DOMAIN}`) | An unre-keyable credential must not be bound to a hostname that can move |
| User verification | `required` | Without it, mere possession of an unlocked device is a complete login |
| Resident key | `Yes` | A non-discoverable credential is a second factor, not a passkey |
| OTP after a passkey? | **No** — conditional OTP stays on the password branch | A UV-required passkey is already two factors, and a phishing-resistant pair at that |

### The declaration did not reach the realm — a reconciler now carries all of it (2026-10-18)

Same one-shot failure this file already documents twice above, third field over. Measured on prod
2026-10-17:

```
$ kubectl --context k3s-prod -n keycloak get keycloakrealmimport homelab -o jsonpath='{.status}'
{"conditions":[{"status":"True","type":"Done"},{"status":"False","type":"Started"},
               {"status":"False","type":"HasErrors"}]}
```

`Done=True` / `Started=False` — the import has completed and **will not re-run**, so a policy block
and three new sub-flows added to this file change nothing about how anyone logs in. The
client-secret/client-URL reconciler does **not** close this gap and was never meant to: its hard
endpoint allow-list makes `/authentication` unreachable **by design** (see the table above), and
realm-level policy fields are not client fields. This is exactly the "a NEW client, a new protocol
mapper or a changed scope still needs the realm import to actually re-run" case named above.

`passkey-reconciler.yaml` closes that gap the same way the client reconciler closed the
secret and URL halves: a CronJob on `*/15` that authenticates to the admin API and converges
declared state. It is a **sibling**, not a second pass of the client reconciler, precisely so that
reconciler's allow-list — and its guarantee that prod's real user and the `media-users` /
`platform-admins` / `viewers` groups are **structurally unreachable** — stays literally true.

Its desired state is `passkey-desired-state.yaml`, **generated** from the realm import by
`ci/render-keycloak-passkey-policy.py` and gated by
`conformance/tests/l0-static/keycloak-passkey-reconciler`, so the RP ID is declared exactly once.
That matters more here than for client URLs: an RP ID is baked into every credential at enrolment
and **can never be re-keyed**, so two declarations free to skew about the domain would be an
unrecoverable-credential bug, not a cosmetic one.

| Declared thing | Who applies it | Why |
|---|---|---|
| `webAuthnPolicyPasswordless*` (realm policy) | **The reconciler. GitOps.** | Realm top-level attributes; strict read-modify-write of `GET`/`PUT /admin/realms/homelab` |
| `webauthn-register-passwordless` required action | **The reconciler. GitOps.** | Per-alias `PUT`, one literally-named alias; enabled, `defaultAction: false` |
| The `browser-mfa` flow restructure | **The reconciler builds and verifies it. Git decides whether it is bound.** | Blue/green: build under a new alias, verify, and flip `browserFlow` in one atomic PUT **only when `bind-browser-flow: "true"`** — see below |

#### The flow converges blue/green — it never mutates the flow you log in through

Converging a five-flow / ten-execution restructure needs create-subflow,
create-execution, set-requirement and bind-config as **separate** admin API calls, with no
transaction. Doing that to the flow bound as `browserFlow` would mean a run dying part-way
leaves a **half-built login path** for Grafana, Gitea, Nextcloud, Headlamp, Semaphore,
PiHole, Crafty, the *arr stack *and* kubectl. This file already documents a total SSO
outage from a realm change, and the conditional-OTP sub-flow — which the declared
structure moves **one level deeper** than it sits today — once 403'd every platform-admin.

That risk is real but it is **not inherent**: it follows only from mutating the *bound*
flow. So the reconciler does not.

1. **Build** the declared tree under a **new, unused alias** — the declared alias plus a
   hash of the desired shape, applied to every flow in the tree. Every multi-call write
   lands on a flow nothing is using. A failure at any point leaves an orphaned, unbound
   flow and **zero user-visible effect**.
2. **Verify** by readback — nesting levels, ordering, authenticators, requirements, and
   the conditional-OTP config *contents*. Nothing is bound unless this passes, so the
   regression that once 403'd every platform-admin is caught **before** it can bite.
3. **Bind** with **one atomic PUT** of the realm, setting `browserFlow` — **only if git says
   so** (next section). Otherwise the run stops here and logs `WOULD BIND browser-mfa-<hash>`.
4. **Rollback is a PR plus one write** (below), and nothing is ever deleted, so the old flow
   stays usable.

#### The bind switch: `bind-browser-flow` in `passkey-desired-state.yaml`

That generated file carries exactly one hand-edited line, `bind-browser-flow`, and it is
`"false"` by default. The renderer carries it through on `--write`, and `--check` refuses
anything but the quoted strings `"true"` / `"false"` (an unquoted `true` is a YAML boolean,
which a ConfigMap rejects at apply time and which would fail the whole `platform-config`
Kustomization). The CronJob reads it from the same mount as the desired state; no flag can
widen it.

| `bind-browser-flow` | What the reconciler does to the realm's `browserFlow` |
|---|---|
| `"false"` (default) | **Never writes it.** It builds and verifies the candidate (or reuses an already verified one), logs `WOULD BIND <alias>`, and exits 0. Whatever the realm is bound to — including your rollback — is left alone on every run. A hand-edited bound flow is left alone too (no rebuild beside it). |
| `"true"` | Binds the verified candidate in one atomic PUT, **and keeps asserting it**: a binding changed by hand is changed back within 15 minutes, and a hand-edit *inside* the bound flow makes the next run build and bind a fresh copy. That is GitOps doing its job — so never hotfix the flow while this is `"true"`. |

**Merging with `"false"` changes nothing about how anyone logs in.** Passes 1 and 2 do write
the realm (the passwordless policy and the enrolment action, neither of which any login
step uses until a flow offers passkeys), and pass 3 creates the unbound candidate flow, but
`browserFlow` is untouched.

**Going live** is a later one-line PR — `bind-browser-flow: "true"` — made **after** the
candidate has been tested in a browser (see "Testing the candidate flow before binding it").

**Rolling back**, in this order:

1. A PR setting `bind-browser-flow: "false"`, merged and applied by Flux. **First** — while it
   is `"true"`, the next run rebinds the candidate over your rollback.
2. One write: bind the previous flow again. The Job log of the bind printed it
   (`ROLLBACK is one write: set browserFlow back to ...`); it is `browser-mfa` for the first
   go-live. Admin console: *Authentication → Flows → <previous flow> → Action → Bind flow →
   Browser flow*, or `PUT /admin/realms/homelab` as a read-modify-write of the full realm
   representation (a partial body drops the passkey policy).

If login is broken *right now*, do step 2 immediately and step 1 straight after: the worst
case of that order is that one scheduled run (≤15 min) rebinds the candidate before Flux
applies the PR, and you repeat step 2.

**The safety property is structural, not intentional.** The reconciler is *incapable* of
writing to the bound flow or anything beneath it: the live `browserFlow` alias and its
whole subtree of sub-flow aliases and execution ids are read first and screened against
every write before a socket opens. It is **fail-closed** — until that binding has been
read, every flow write is refused, and if the bound flow's executions cannot be read the
flow pass writes nothing at all. There is **no `DELETE`** anywhere in its allow-list.

**Idempotence is load-bearing**, not a nicety: a rebuild-and-reflip every 15 minutes
would churn the login path on a schedule. The bound flow is compared structurally against
the declared shape and, when they match, the run issues **zero writes and builds
nothing**. Sub-flow and config *aliases* are deliberately **not** compared — a flow the
reconciler built carries a hash suffix, so comparing labels would make it permanently
unrecognisable as converged.

**Recovery.** A crashed run can leave a partial flow under the primary candidate alias. A
later run reads it: if it matches desired it is a completed-but-unbound build and is
simply **bound**; if not, it is left alone and the next candidate alias is used. The
candidate count is **bounded**, so a persistently failing build fails loudly naming the
flows to clean up rather than creating unbounded sprawl.

```sh
# What the reconciler currently says. A normal run prints the full comparison.
kubectl --context k3s-prod -n keycloak get jobs -l app.kubernetes.io/name=keycloak-passkey-reconciler
kubectl --context k3s-prod -n keycloak logs -l app.kubernetes.io/name=keycloak-passkey-reconciler --tail=200

# Which candidate it would bind (bind-browser-flow "false"):
kubectl --context k3s-prod -n keycloak logs -l app.kubernetes.io/name=keycloak-passkey-reconciler --tail=300 \
  | grep -E 'WOULD BIND|BOUND browserFlow|PROBLEM'

# Force a run now rather than wait for the tick. It does exactly what the CronJob does,
# including honouring bind-browser-flow. `kubectl create job --from=cronjob/...` CANNOT
# override the container command, so --audit / --no-bind need a one-off Job with the flag
# appended (--no-bind can only NARROW git's switch, never widen it).
kubectl --context k3s-prod -n keycloak create job --from=cronjob/keycloak-passkey-reconciler \
  passkey-run-once
kubectl --context k3s-prod -n keycloak logs job/passkey-run-once

# ROLLBACK — see "The bind switch" above: PR bind-browser-flow "false" FIRST, then rebind
# the previous flow (Authentication -> Flows -> <previous> -> Action -> Bind flow ->
# Browser flow). The old flow is never deleted, so this always works.

# Regenerate the desired state after editing the passkey policy or the flow in the realm
# import (the merge gate will demand it).
python3 ci/render-keycloak-passkey-policy.py --write
```

### Testing the candidate flow before binding it

With `bind-browser-flow: "false"` the candidate exists but nothing logs in through it. Test it
on **one client** without touching the realm binding, using a per-client flow override:

1. Find the candidate: the Job log line `WOULD BIND browser-mfa-<hash>`.
2. Admin console → *Clients → `account-console` → Advanced → Authentication flow overrides →
   Browser Flow* = `browser-mfa-<hash>`. This is a temporary, hand-made client change; remove it
   in step 7.
3. **Password fallback.** Private window → `https://keycloak.<base domain>/realms/homelab/account`.
   Expect a **username** screen, then a choice: passkey, or the password form. As a user with no
   passkey, choose password and sign in.
4. **Platform-admin TOTP on the password branch.** Repeat step 3 as a `platform-admins` member:
   after the password, a TOTP prompt **must** appear. If it does not — or you get a 403 — the
   conditional-OTP step is wrong: stop, do not bind.
5. **Passkey enrolment.** Signed in to the account console → *Signing in → Passkey → Set up*. A
   browser/OS passkey prompt must appear; if it does not, the RP ID is wrong for the origin and
   nothing server-side will say so.
6. **Passkey login.** Sign out; private window again; username, then choose passkey. Expect to be
   signed in with **no** TOTP prompt, even as a platform-admin.
7. Remove the override from step 2.
8. Only now: the one-line PR `bind-browser-flow: "true"`. The next run binds the candidate you
   tested (same alias — the hash is of the declared shape).

### Verifying it actually works (nobody has, as of 2026-10-29)

No passkey has been enrolled and no browser has been pointed at this. The checks that would prove
it on the BOUND flow, after go-live, in order:

```sh
# 1. the policy is live (RP ID must read as the base domain, NOT the keycloak host, NOT a URL)
#    Authentication -> Policies -> WebAuthn Passwordless Policy, in the admin console.
# 2. enrol: account console (https://keycloak.${BASE_DOMAIN}/realms/homelab/account) -> Signing in
#    -> Passwordless -> set up. A browser/OS passkey prompt must appear; if it does not, the RP ID
#    is wrong for the origin and NOTHING will say so.
# 3. log in: a fresh private window to any SSO-gated app. Expect username screen, then a passkey
#    prompt. A platform-admin must NOT be asked for TOTP on this path.
# 4. regression (the important one): a user with NO passkey must still get the password screen,
#    and a platform-admin must still be asked for TOTP there.
```

A failed passkey ceremony is usually silent in the server log — the browser refuses before Keycloak
is involved — so step 2 failing with no log entry is the expected signature of a wrong RP ID.

## Upgrading Keycloak: the operator IS the version

The `Keycloak` CR has no `spec.image`, so the server image is whatever the operator's
`RELATED_IMAGE_KEYCLOAK` names — the same tag as the operator (prod, 2026-10-30: operator
`26.0.6`, `keycloak-0/1` on `quay.io/keycloak/keycloak:26.0.6`). Operator, CRDs and server are
released in lockstep from `keycloak/keycloak-k8s-resources`, and the three URLs in
`kustomization.yaml` are the one knob. Renovate raises them as ONE PR (`keycloak-operator` group,
labels `needs-review` + `blast-radius`), **never auto-merged**; majors wait for a dashboard tick.
`catalog/keycloak.yaml`'s image tag is documentation only (its render deploys nothing) and
Renovate is disabled for it — update it by hand to match after an operator bump merges.

**An operator bump requires re-checking the passkey reconciler's assumptions.**
`passkey-reconciler.yaml` (and `ci/render-keycloak-passkey-policy.py`, which generates its desired
state) is written against **Keycloak 26.0.6 Admin API behaviour read from that tag's source** —
grep it for `26.0.6`. The gate cannot catch a break: it is static and never runs the reconciler
against a server. Before merging a bump, diff the new tag against 26.0.6 for at least:

- `AuthenticationManagementResource.recurseExecutions` — does a sub-flow row still carry its name
  in `displayName` (not `alias`)? The reconciler's flow matching depends on it.
- `updateExecutions` / execution-requirement updates — same request shape and id resolution?
- the WebAuthn Passwordless policy — is it still gated all-or-nothing on a non-empty
  `webAuthnPolicyPasswordlessRpEntityName`, and does a realm `PUT` still REPLACE rather than merge
  `requiredActions`? (The strict read-modify-write exists because of both.)
- authenticator-config alias handling for copied flows.

Then, after the merge reconciles (operator actions): watch the `Keycloak` CR reach Ready and both
pods roll; run the reconciler once **with `--audit`** (see "Passkeys" above for the one-off Job)
and confirm it reports convergence with no write; confirm the client-secret and client-URL
reconcilers' next runs succeed; log in to one SSO app and run one `kubectl --context k3s-prod`
OIDC command. Roll back by reverting the PR — but a server minor may migrate the database
schema on start, so take/confirm a fresh CNPG backup before merging.

## Exceptions that keep native auth (do not force SSO) — ADR-0014

Plex/Seerr (Plex auth), Minecraft (Mojang/Microsoft), UniFi (native).