fix(keycloak-passkey): bind only on git's say-so; protect real sub-flows; screen the requirement PUT by body id

Review findings on the passkey reconciler, fixed:

1. HIGH - the bind was automatic and rollback did not stick. A git-declared switch,
   `bind-browser-flow` in passkey-desired-state.yaml, now decides whether pass 3 may
   PUT browserFlow. Default "false": build + verify the candidate (or reuse a verified
   one), log WOULD BIND <alias>, never touch the binding - including after an earlier
   bind, so an operator's rollback sticks, and a hand-edited bound candidate is left
   alone rather than shadowed by an -r2 build. "true" binds as before. The renderer
   carries the one hand-edited line through and --check refuses anything but the
   quoted strings "true"/"false". --no-bind can only narrow it.

2. HIGH - the protected-subtree guard read `alias` on sub-flow rows. Keycloak 26.0.6
   AuthenticationManagementResource.recurseExecutions sets displayName =
   flowRef.getAlias() and flowId on a sub-flow row and never sets alias (alias is the
   CONFIG alias on authenticator rows), so the protected set was only the top-level
   alias. Sub-flows are now named by displayName (getFlows() lists top-level flows
   only, so flowId cannot be resolved through it); a sub-flow row without one leaves
   the subtree undetermined and the flow pass fail-closed. The selftest stub now
   returns 26.0.6-shaped rows, and a real run is asserted to refuse writes into
   browser-mfa-forms and browser-mfa-conditional-otp.

3. MEDIUM - PUT /flows/{alias}/executions was screened on the path alias only, but
   updateExecutions resolves the execution realm-wide by body id. body.id is now
   screened against the protected ids AND required to be in the tree this run is
   building (recorded from the build's own readback); the config POST is held to the
   same build scope; and the body may carry only id/requirement/priority (the same
   handler renames any flow named by flowId).

4. Stale "audit-only" docs rewritten (platform-config kustomization, chainsaw test,
   desired-state header and its renderer template, README) to describe what ships.

5. backoffLimit 0 (a retried deterministic build failure burns the candidate-alias
   budget); the stale "Renovate does not manage image: tags" comment corrected and the
   image pinned by index digest.

6. The ~160-line header duplicated in the YAML and the script is now kept once, in the
   script, with a short pointer above it.

Selftest: 259 assertions (was 216). mutations.py: 40 injected regressions (was 30),
including one per finding above, all caught.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
jakman85andClaude Opus 5.5 committed 2026-10-29 21:41:14 -05:00
1 parent f5931944fa
commit 17eed6e8ae
8 files changed
+893 -310

No files matched your search

+71 -9
View File
@@ -18,7 +18,18 @@
# keycloak-realm-import.yaml declares are INERT. They are in git, Flux applied # keycloak-realm-import.yaml declares are INERT. They are in git, Flux applied
# them, and they reach the realm never. platform/keycloak/passkey-reconciler.yaml # them, and they reach the realm never. platform/keycloak/passkey-reconciler.yaml
# is the level-triggered loop that closes that; this renderer supplies the DESIRED # is the level-triggered loop that closes that; this renderer supplies the DESIRED
# STATE it converges towards (and, for the flow, audits against). # STATE it converges towards.
#
# THE ONE VALUE THAT IS NOT RENDERED: `bind-browser-flow`, git's switch for whether the
# reconciler may BIND its verified candidate browser flow (one atomic PUT of the
# realm's browserFlow). It is an operator DECISION -- "I have tested the candidate
# flow in a browser; make it the login path" -- not a fact derivable from the realm
# import, so it cannot be generated. It is instead CARRIED THROUGH from the committed
# file: --write preserves it, --check accepts any value of it that is exactly the
# string "true" or "false" and rejects everything else (an unquoted YAML true is a
# bool, which a ConfigMap rejects at apply time -- failing the whole platform-config
# Kustomization -- so it is refused HERE, at merge time). An absent file or key
# renders as "false". Going live is therefore a one-line PR flipping that line.
# #
# THE CONSTRAINT THAT SHAPES IT: the reconciler pod runs with # THE CONSTRAINT THAT SHAPES IT: the reconciler pod runs with
# automountServiceAccountToken: false, exactly like its client-secret sibling. It # automountServiceAccountToken: false, exactly like its client-secret sibling. It
@@ -148,6 +159,33 @@ PASSKEY_AUTHENTICATOR = "webauthn-authenticator-passwordless"
ENROL_ACTION = "webauthn-register-passwordless" ENROL_ACTION = "webauthn-register-passwordless"
BIND_KEY = "bind-browser-flow"
BIND_VALUES = ("true", "false")
def committed_bind(root: Path) -> str:
"""The operator's bind-browser-flow value from the COMMITTED file, or "false".
This is the only input read from the output file, and it is validated, not trusted:
anything other than the exact strings "true" / "false" is a hard failure."""
target = root / OUT_FILE
if not target.is_file():
return "false"
docs = [d for d in yaml.safe_load_all(target.read_text(encoding="utf-8")) if d]
cm = next((d for d in docs if d.get("kind") == "ConfigMap"), None) or {}
data = cm.get("data") or {}
if BIND_KEY not in data:
return "false"
v = data[BIND_KEY]
if not isinstance(v, str) or v not in BIND_VALUES:
raise SystemExit(
"FATAL: %s data.%s is %r. It must be exactly the QUOTED string \"true\" or "
"\"false\": an unquoted true/false is a YAML boolean, which a ConfigMap rejects "
"at apply time and which would fail the whole platform-config Kustomization"
% (OUT_FILE, BIND_KEY, v))
return v
def render(root: Path) -> str: def render(root: Path) -> str:
"""Return the full text of passkey-desired-state.yaml as it should be committed.""" """Return the full text of passkey-desired-state.yaml as it should be committed."""
docs = list(yaml.safe_load_all((root / IMPORT_FILE).read_text())) docs = list(yaml.safe_load_all((root / IMPORT_FILE).read_text()))
@@ -323,12 +361,13 @@ def render(root: Path) -> str:
body = json.dumps(want, indent=2, sort_keys=True, ensure_ascii=False) body = json.dumps(want, indent=2, sort_keys=True, ensure_ascii=False)
indented = "\n".join((" " + ln) if ln else "" for ln in body.splitlines()) indented = "\n".join((" " + ln) if ln else "" for ln in body.splitlines())
return HEADER + indented + "\n" return (HEADER + BIND_BLOCK % (BIND_KEY, committed_bind(root))
+ " %s: |\n" % DATA_KEY + indented + "\n")
HEADER = """--- HEADER = """---
# ============================================================================= # =============================================================================
# GENERATED FILE — DO NOT EDIT BY HAND. # GENERATED FILE — DO NOT EDIT BY HAND, except the one bind-browser-flow line below.
# #
# source: platform/keycloak/keycloak-realm-import.yaml (spec.realm) # source: platform/keycloak/keycloak-realm-import.yaml (spec.realm)
# regenerate: python3 ci/render-keycloak-passkey-policy.py --write # regenerate: python3 ci/render-keycloak-passkey-policy.py --write
@@ -338,8 +377,14 @@ HEADER = """---
# WHAT THIS IS: the DESIRED STATE for the `homelab` realm's PASSKEY configuration — # WHAT THIS IS: the DESIRED STATE for the `homelab` realm's PASSKEY configuration —
# the WebAuthn passwordless policy, the enrolment required action, and the declared # the WebAuthn passwordless policy, the enrolment required action, and the declared
# browser authentication flow tree. platform/keycloak/passkey-reconciler.yaml mounts # browser authentication flow tree. platform/keycloak/passkey-reconciler.yaml mounts
# it and (a) converges the policy and the required action onto it, (b) AUDITS the # it and (a) converges the policy and the required action onto it, (b) builds the
# live browser flow against it and alarms on divergence without rewriting it. # declared flow tree BLUE/GREEN under a new alias and verifies it by readback, never
# writing the flow users are logging in through, and (c) BINDS that verified flow as
# the realm's browserFlow ONLY when `bind-browser-flow` below is "true".
#
# EVERYTHING HERE IS GENERATED EXCEPT ONE LINE: `bind-browser-flow`. That is an
# operator decision, carried through by the renderer from this committed file (see
# ci/render-keycloak-passkey-policy.py). It defaults to "false".
# #
# WHY IT IS A SEPARATE OBJECT: the realm import is a ONE-SHOT that never re-runs # WHY IT IS A SEPARATE OBJECT: the realm import is a ONE-SHOT that never re-runs
# once Done=True, so a realm-level field or a flow edited in git never reaches the # once Done=True, so a realm-level field or a flow edited in git never reaches the
@@ -379,8 +424,25 @@ metadata:
app.kubernetes.io/name: %s app.kubernetes.io/name: %s
app.kubernetes.io/part-of: keycloak app.kubernetes.io/part-of: keycloak
data: data:
%s: | """ % (CONFIGMAP_NAME, NAMESPACE, CONFIGMAP_NAME)
""" % (CONFIGMAP_NAME, NAMESPACE, CONFIGMAP_NAME, DATA_KEY)
BIND_BLOCK = """ # ---------------------------------------------------------------------------
# THE BIND SWITCH -- the ONLY hand-edited line in this file. Exactly "true" or
# "false", QUOTED.
#
# "false" the reconciler builds and verifies the candidate browser flow and
# logs `WOULD BIND <alias>`, but NEVER changes the realm's browserFlow.
# Whatever the realm is bound to -- including a rollback -- is left alone.
# "true" it binds the verified candidate in one atomic PUT, and keeps the
# realm bound to it: a binding changed by hand is changed back.
#
# GO LIVE: test the candidate in a browser first (platform/keycloak/README.md,
# "Passkeys"), then a PR setting this to "true".
# ROLL BACK: a PR setting this to "false" FIRST (or the next run rebinds), then
# one write setting browserFlow back to the previous alias from the Job log.
# ---------------------------------------------------------------------------
%s: "%s"
"""
def main() -> int: def main() -> int:
@@ -410,8 +472,8 @@ def main() -> int:
print("passkey-desired-state.yaml is in sync with %s" % IMPORT_FILE) print("passkey-desired-state.yaml is in sync with %s" % IMPORT_FILE)
return 0 return 0
print("CHECK FAIL: %s has DRIFTED from %s — the reconciler would converge the " print("CHECK FAIL: %s has DRIFTED from %s — the reconciler would converge the "
"realm on a passkey policy the realm import does not declare, and audit " "realm on a passkey policy the realm import does not declare, and build "
"the login flow against a tree nobody authored.\n" "the login flow from a tree nobody authored.\n"
"Regenerate with: python3 ci/render-keycloak-passkey-policy.py --write\n" "Regenerate with: python3 ci/render-keycloak-passkey-policy.py --write\n"
% (OUT_FILE, IMPORT_FILE), file=sys.stderr) % (OUT_FILE, IMPORT_FILE), file=sys.stderr)
for line in difflib.unified_diff(have.splitlines(True), want.splitlines(True), for line in difflib.unified_diff(have.splitlines(True), want.splitlines(True),
@@ -1,6 +1,7 @@
# ============================================================================= # =============================================================================
# L0 STATIC — the passkey configuration is derived from one declaration, continuously # L0 STATIC — the passkey configuration is derived from one declaration, continuously
# reconciled, and the browser login flow is auditable but UNWRITABLE # reconciled, and the browser login flow converges BLUE/GREEN: the BOUND flow is
# unwritable, and binding a new one is git's explicit decision (bind-browser-flow)
# Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster. # Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster.
# #
# R-IDP-03 (identity config is declarative) and R-IDP-04 (MFA available) are both # R-IDP-03 (identity config is declarative) and R-IDP-04 (MFA available) are both
@@ -25,25 +26,30 @@
# import declares NO requiredActions block (26.0.6 treats it as REPLACE, which would # import declares NO requiredActions block (26.0.6 treats it as REPLACE, which would
# delete CONFIGURE_TOTP and lock out every platform-admin); the enrolment action is # delete CONFIGURE_TOTP and lock out every platform-admin); the enrolment action is
# enabled but NOT a defaultAction, so enrolment stays opt-in; the reconciler carries # enabled but NOT a defaultAction, so enrolment stays opt-in; the reconciler carries
# no ServiceAccount token, mounts only what it needs, and contains no dollar sign; # no ServiceAccount token, mounts only what it needs, passes no mode flag, and
# the endpoint allow-list contains NO write for any authentication flow, execution # contains no dollar sign; the endpoint allow-list contains NO delete, NO
# or authenticator config, and NO required-action write for any alias other than the # single-execution or config-edit write, and NO required-action write for any alias
# one named literally; the SIBLING client-secret reconciler's allow-list is NOT # other than the one named literally; the protected-subtree guard exists, is wired
# widened; and the reconciler's own `--selftest` (extracted from the shipped # into the single request choke point and is fail-closed; the SIBLING client-secret
# ConfigMap and executed) proves compare-then-write, idempotence, strict # reconciler's allow-list is NOT widened; and the reconciler's own `--selftest`
# read-modify-write, the RpEntityName gate, flow-is-unwritable, the conditional-OTP # (extracted from the shipped ConfigMap and executed) proves compare-then-write,
# regression and fail-closed — with mutations.py proving that selftest can FAIL. # idempotence, strict read-modify-write, the RpEntityName gate, the bound flow and
# its sub-flows unwritable, the git bind switch and a rollback that sticks, the
# conditional-OTP regression and fail-closed — with mutations.py proving that
# selftest can FAIL.
# #
# WHY THE FLOW IS AUDITED AND NOT CONVERGED, asserted here rather than only argued: # WHY THE FLOW IS BUILT BLUE/GREEN AND BOUND ONLY ON GIT'S SAY-SO, asserted here rather
# converging a five-flow, ten-execution restructure needs create-subflow, # than only argued: converging a five-flow, ten-execution restructure needs
# create-execution, set-requirement, set-priority and bind-config as SEPARATE admin API # create-subflow, create-execution, set-requirement and bind-config as SEPARATE admin API
# calls with no transaction. A run that dies between them leaves a HALF-BUILT FLOW BOUND # calls with no transaction. Doing that to the BOUND flow means a run that dies between
# AS THE LOGIN PATH for Grafana, Gitea, Nextcloud, Headlamp, Semaphore, PiHole, Crafty, # them leaves a HALF-BUILT FLOW AS THE LOGIN PATH for Grafana, Gitea, Nextcloud, Headlamp,
# the *arr stack AND kubectl. This repo has already had a total SSO outage from a realm # Semaphore, PiHole, Crafty, the *arr stack AND kubectl — this repo has had a total SSO
# change (2026-09-06), and the conditional-OTP sub-flow — which the declared structure # outage from a realm change (2026-09-06), and the conditional-OTP sub-flow once 403'd
# moves one level deeper than it sits today — once 403'd every platform-admin. A # every platform-admin. So every multi-call write lands on a NEW, unused candidate flow,
# reconciler that half-rewrites the bound flow every 15 minutes is a worse failure than # which is verified by readback before anything is bound. Binding it is then ONE atomic
# the one it replaces, because it is a loop rather than a single bad change. # PUT of browserFlow, made only when passkey-desired-state.yaml sets
# bind-browser-flow: "true" (default "false"): an untested login flow never goes live on
# a schedule tick, and with the switch off an operator's rollback is never undone.
# #
# STATUS: EXECUTABLE. `check.sh` is a pure offline assertion script (git + PyYAML + # STATUS: EXECUTABLE. `check.sh` is a pure offline assertion script (git + PyYAML +
# stdlib); conformance/run.sh --layer l0 and ci/l0.sh run it directly, and this Chainsaw # stdlib); conformance/run.sh --layer l0 and ci/l0.sh run it directly, and this Chainsaw
@@ -93,9 +99,10 @@ spec:
- name: audit-passkey-declaration-renderer-and-reconciler - name: audit-passkey-declaration-renderer-and-reconciler
description: >- description: >-
Assert the desired state is derived from the realm import and shares its Assert the desired state is derived from the realm import and shares its
substitution pass, that the reconciler's write surface excludes every flow substitution pass, that the reconciler cannot write the bound flow or delete
mutation, that the sibling reconciler is unwidened, and run the reconciler's own anything and binds only on git's bind-browser-flow, that the sibling reconciler
selftest plus the mutation suite that proves it can fail. is unwidened, and run the reconciler's own selftest plus the mutation suite that
proves it can fail.
try: try:
- script: - script:
content: bash ./check.sh content: bash ./check.sh
@@ -62,6 +62,11 @@
# been read. That last one is what the whole design rests on: a reconciler that # been read. That last one is what the whole design rests on: a reconciler that
# half-rewrites the BOUND browser flow is a total SSO outage on a loop, and # half-rewrites the BOUND browser flow is a total SSO outage on a loop, and
# this repo has had one (2026-09-06). # this repo has had one (2026-09-06).
# (6b) THE BIND IS GIT'S DECISION: the desired-state ConfigMap carries
# bind-browser-flow as exactly "true" or "false", the script reads it from the
# SAME mount, and the CronJob passes no flag that could widen it. With "false"
# (the default) the reconciler builds and verifies but never PUTs browserFlow,
# so an operator's rollback sticks; the selftest proves that end to end.
# (7) THE SIBLING RECONCILER IS UNTOUCHED: client-secret-reconciler.yaml's # (7) THE SIBLING RECONCILER IS UNTOUCHED: client-secret-reconciler.yaml's
# allow-list is NOT widened by this work. Its --selftest asserts it refuses # allow-list is NOT widened by this work. Its --selftest asserts it refuses
# /users, /groups, /roles, /authentication and the whole-realm endpoint, and # /users, /groups, /roles, /authentication and the whole-realm endpoint, and
@@ -152,6 +157,7 @@ RENDERER = "ci/render-keycloak-passkey-policy.py"
SIBLING_FILE = "platform/keycloak/client-secret-reconciler.yaml" SIBLING_FILE = "platform/keycloak/client-secret-reconciler.yaml"
STATE_CM_NAME = "keycloak-passkey-desired-state" STATE_CM_NAME = "keycloak-passkey-desired-state"
STATE_DATA_KEY = "passkey-desired-state.json" STATE_DATA_KEY = "passkey-desired-state.json"
BIND_KEY = "bind-browser-flow"
SCRIPT_KEY = "reconcile-keycloak-passkeys.py" SCRIPT_KEY = "reconcile-keycloak-passkeys.py"
ENROL_ACTION = "webauthn-register-passwordless" ENROL_ACTION = "webauthn-register-passwordless"
DOLLAR = chr(36) DOLLAR = chr(36)
@@ -225,10 +231,16 @@ if state_cm is not None:
f"{STATE_FILE}: the ConfigMap is named " f"{STATE_FILE}: the ConfigMap is named "
f"{state_cm['metadata']['name']!r}, but the CronJob mounts {STATE_CM_NAME!r}") f"{state_cm['metadata']['name']!r}, but the CronJob mounts {STATE_CM_NAME!r}")
data = state_cm.get("data") or {} data = state_cm.get("data") or {}
check(list(data) == [STATE_DATA_KEY], check(sorted(data) == sorted([STATE_DATA_KEY, BIND_KEY]),
f"{STATE_FILE}: expected exactly the data key {STATE_DATA_KEY!r}, got " f"{STATE_FILE}: expected exactly the data keys {STATE_DATA_KEY!r} and "
f"{sorted(data)} — the reconciler's PASSKEY_STATE_FILE default would not " f"{BIND_KEY!r}, got {sorted(data)} — the reconciler's PASSKEY_STATE_FILE / "
f"resolve") f"PASSKEY_BIND_FILE defaults would not resolve")
# THE BIND SWITCH. An unquoted true/false is a YAML BOOLEAN, which a ConfigMap rejects
# at apply time -- failing the whole platform-config Kustomization, realm import
# included. The renderer refuses it too; asserted here so the reason is on record.
check(data.get(BIND_KEY) in ("true", "false"),
f"{STATE_FILE}: data.{BIND_KEY} is {data.get(BIND_KEY)!r}; it must be the QUOTED "
f"string \"true\" or \"false\"")
raw = data.get(STATE_DATA_KEY, "") raw = data.get(STATE_DATA_KEY, "")
try: try:
desired = json.loads(raw) desired = json.loads(raw)
@@ -457,6 +469,25 @@ if cj is not None:
f"the mount path — they are a pair and must move together") f"the mount path — they are a pair and must move together")
check((mounts.get(state_vol) or {}).get("readOnly") is True, check((mounts.get(state_vol) or {}).get("readOnly") is True,
f"{RECON_FILE}: the desired-state mount must be readOnly") f"{RECON_FILE}: the desired-state mount must be readOnly")
check(f'"/etc/keycloak-passkey/{BIND_KEY}"' in script,
f"{RECON_FILE}: the script's PASSKEY_BIND_FILE default is not "
f"/etc/keycloak-passkey/{BIND_KEY}. An unresolved bind file reads as \"false\" "
f"(safe, but git's \"true\" would then silently never take effect)")
check(not (state_vol and (cm_vols[state_vol]["configMap"] or {}).get("items")),
f"{RECON_FILE}: the desired-state volume selects `items`, so one of its two "
f"keys may not be projected")
# The bind decision belongs to git. A container arg could only NARROW it today
# (--no-bind), but nothing in the CronJob may carry a mode flag at all: the shipped
# behaviour must be readable from passkey-desired-state.yaml alone.
cmd = [str(x) for x in (ctr.get("command") or []) + (ctr.get("args") or [])]
check(not [x for x in cmd if x.startswith("--")],
f"{RECON_FILE}: the CronJob passes {[x for x in cmd if x.startswith('--')]} to "
f"the reconciler. Its mode is decided by git (bind-browser-flow), not by flags")
# Retries burn the bounded candidate-alias budget on a deterministic failure.
check(cj["spec"]["jobTemplate"]["spec"].get("backoffLimit") == 0,
f"{RECON_FILE}: backoffLimit must be 0 — every failure here is deterministic "
f"within a tick, and each retried build consumes another candidate alias")
# The admin credential, referenced BY NAME. Never decrypted, never a literal. # The admin credential, referenced BY NAME. Never decrypted, never a literal.
for key in ("KEYCLOAK_ADMIN_USER", "KEYCLOAK_ADMIN_PASSWORD"): for key in ("KEYCLOAK_ADMIN_USER", "KEYCLOAK_ADMIN_PASSWORD"):
@@ -591,6 +622,21 @@ check("_FLOW_CREATE_PATH" in script,
f"altogether") f"altogether")
# And the one function allowed to change the live binding must be distinct from the # And the one function allowed to change the live binding must be distinct from the
# policy write, which must refuse to. # policy write, which must refuse to.
# FINDING 2026-10-29: sub-flows are named by displayName in 26.0.6 (recurseExecutions
# never sets `alias` on a sub-flow row). Reading `alias` shrank the protected subtree to
# the top-level flow alone while every selftest passed against a stub that returned it.
check(re.search(r'^def subflow_alias\(ex\):\n(?:.*\n)*? name = ex\.get\("displayName"\)',
script, re.M) is not None,
f"{RECON_FILE}: subflow_alias() no longer reads displayName. Keycloak 26.0.6's "
f"executions listing names a sub-flow ONLY by displayName; reading `alias` yields "
f"None and the protected subtree silently shrinks to the top-level flow")
# FINDING 2026-10-29: the requirement PUT's real target is body["id"].
check("_EXEC_UPDATE_PATH" in script and "_BUILD_EXECUTION_IDS" in script,
f"{RECON_FILE}: the requirement PUT (PUT /flows/<alias>/executions) is no longer "
f"screened on its body id. updateExecutions resolves the execution realm-wide by id "
f"and ignores the path alias, so a path-only screen lets it write the bound flow")
check("def declared_bind(" in script and "BIND_FILE" in script,
f"{RECON_FILE}: the reconciler no longer reads git's bind-browser-flow switch")
check("def bind_browser_flow(" in script, check("def bind_browser_flow(" in script,
f"{RECON_FILE}: there is no bind_browser_flow() function. The atomic flip must be " f"{RECON_FILE}: there is no bind_browser_flow() function. The atomic flip must be "
f"one named, auditable operation, not an incidental field in some other write") f"one named, auditable operation, not an incidental field in some other write")
@@ -78,10 +78,10 @@ MUTATIONS: list[tuple[str, str, str, str]] = [
"module-level initial value is deliberately NOT the anchor here: run() resets " "module-level initial value is deliberately NOT the anchor here: run() resets "
"these at the start of every run, so the initialiser is unreachable and mutating " "these at the start of every run, so the initialiser is unreachable and mutating "
"it would prove nothing.)", "it would prove nothing.)",
" global _PROTECTED_FLOW_ALIASES, _PROTECTED_EXECUTION_IDS\n" " global _PROTECTED_FLOW_ALIASES, _PROTECTED_EXECUTION_IDS, _BUILD_EXECUTION_IDS\n"
" _PROTECTED_FLOW_ALIASES = None\n" " _PROTECTED_FLOW_ALIASES = None\n"
" _PROTECTED_EXECUTION_IDS = None", " _PROTECTED_EXECUTION_IDS = None",
" global _PROTECTED_FLOW_ALIASES, _PROTECTED_EXECUTION_IDS\n" " global _PROTECTED_FLOW_ALIASES, _PROTECTED_EXECUTION_IDS, _BUILD_EXECUTION_IDS\n"
" _PROTECTED_FLOW_ALIASES = frozenset()\n" " _PROTECTED_FLOW_ALIASES = frozenset()\n"
" _PROTECTED_EXECUTION_IDS = frozenset()"), " _PROTECTED_EXECUTION_IDS = frozenset()"),
@@ -117,6 +117,84 @@ MUTATIONS: list[tuple[str, str, str, str]] = [
" \"the existing flow %r\" % candidate):", " \"the existing flow %r\" % candidate):",
" if cand_shape is not None:"), " if cand_shape is not None:"),
# ------------------------------------- 2026-10-29 review findings, each made falsifiable
("sub-flows identified by `alias` again",
"read `alias` off a sub-flow row, as the first version did -- Keycloak 26.0.6's "
"recurseExecutions sets only displayName (and flowId) on a sub-flow row, so every "
"sub-flow resolves to nothing and the protected subtree is the top-level flow alone: "
"a write into browser-mfa-forms would NOT be refused",
" name = ex.get(\"displayName\")\n"
" return name if isinstance(name, str) and name else None",
" name = ex.get(\"alias\")\n"
" return name if isinstance(name, str) and name else None"),
("an unnameable sub-flow is tolerated",
"carry on when a sub-flow row has no displayName -- the protected subtree is then "
"SMALLER than the live tree, which is the fail-open the guard exists to prevent",
" if name is None:\n"
" return None, None",
" if name is None:\n"
" name = \"\""),
("the requirement PUT is screened on its path only",
"trust the alias in PUT /flows/<alias>/executions -- updateExecutions resolves "
"body.id realm-wide and never checks it belongs to <alias>, so a candidate alias in "
"the path can carry a BOUND execution's id in the body",
" targets.append((\"execution\", (body or {}).get(\"id\")))",
" pass"),
("execution-addressed writes are not held to the build tree",
"'outside the bound subtree is enough' -- it is not: every OTHER flow in the realm is "
"outside it too, so a stray id would edit an unrelated flow",
" if what == \"execution\" and (_BUILD_EXECUTION_IDS is None\n"
" or target not in _BUILD_EXECUTION_IDS):",
" if False:"),
("the requirement PUT may carry any key",
"let the body carry flowId / displayName -- the same handler then RENAMES whatever "
"flow flowId names, bound or not",
" if extra:\n"
" raise Refused(",
" if False:\n"
" raise Refused("),
("git's bind switch is ignored",
"bind whenever verification passes, as the first version did -- the first live use of "
"an untested login flow becomes a schedule tick, and an operator's rollback is "
"undone within 15 minutes, forever",
" bind = git_bind and not no_bind",
" bind = not no_bind"),
("bind=false still binds inside the flow pass",
"keep the flag but forget to honour it where the bind happens",
" if not bind:\n"
" # THE BIND SWITCH IS OFF.",
" if False:\n"
" # THE BIND SWITCH IS OFF."),
("an absent bind switch means bind",
"default to true when the key is missing -- a ConfigMap predating the switch, or a "
"deleted line, would silently put an untested flow live",
" except FileNotFoundError:\n"
" return False",
" except FileNotFoundError:\n"
" return True"),
("an invalid bind switch is guessed at",
"read anything that is not \"true\" as false -- a typo then passes silently instead of "
"stopping the run, and the operator believes the flow is live",
" if raw == \"false\":\n"
" return False\n"
" raise ValueError(",
" return False\n"
" raise ValueError("),
("bind=false rebuilds beside a hand-edited bound flow",
"treat the hand-edited bound candidate as a 'half-built leftover' -- with the switch "
"off this builds an -r2 next to the operator's hotfix on every tick",
" if candidate in have and candidate == bound and not bind:",
" if False:"),
# ------------------------------------------------------------------- idempotence # ------------------------------------------------------------------- idempotence
("sub-flow aliases are compared as names", ("sub-flow aliases are compared as names",
"compare the sub-flow alias like any other field -- the flows this program builds " "compare the sub-flow alias like any other field -- the flows this program builds "
+7 -4
View File
@@ -131,10 +131,13 @@ resources:
# browser login flow. Measured on prod 2026-10-18, hours after the passkey declaration # browser login flow. Measured on prod 2026-10-18, hours after the passkey declaration
# merged: the KeycloakRealmImport read Done=True / Started=False with the whole # merged: the KeycloakRealmImport read Done=True / Started=False with the whole
# webAuthnPolicyPasswordless* block and the restructured browser-mfa flow INERT. This # webAuthnPolicyPasswordless* block and the restructured browser-mfa flow INERT. This
# CronJob converges the policy and the enrolment required action, and AUDITS the browser # CronJob converges the policy and the enrolment required action, and builds the
# flow without the ability to rewrite it (no flow-mutating endpoint is on its # declared browser flow BLUE/GREEN: under a new alias, verified by readback, and
# allow-list) — a half-rewritten flow bound as the realm's browserFlow is a total SSO # structurally unable to write the flow that is currently bound (a half-rewritten bound
# outage, and this repo has had one. See the file header for the full rationale. # flow is a total SSO outage, and this repo has had one). It BINDS the verified flow
# (one atomic PUT of browserFlow) ONLY when passkey-desired-state.yaml sets
# bind-browser-flow: "true"; the default "false" leaves the realm's binding untouched.
# See the script header in that file for the full rationale.
- ../platform/keycloak/passkey-reconciler.yaml - ../platform/keycloak/passkey-reconciler.yaml
# Its DESIRED STATE, GENERATED from keycloak-realm-import.yaml by # Its DESIRED STATE, GENERATED from keycloak-realm-import.yaml by
# ci/render-keycloak-passkey-policy.py and gated by the L0 check. Same reason as # ci/render-keycloak-passkey-policy.py and gated by the L0 check. Same reason as
+79 -22
View File
@@ -31,8 +31,8 @@ files pulled as remote kustomize bases (the kubevirt-stub pattern); Renovate mai
| `keycloak-realm-import.yaml` (`KeycloakRealmImport`: realm/groups/clients/MFA) | **platform-config** | config | | `keycloak-realm-import.yaml` (`KeycloakRealmImport`: realm/groups/clients/MFA) | **platform-config** | config |
| `client-secret-reconciler.yaml` (CronJob: client secrets **and** client URLs ← git, every 15m) | **platform-config** | config | | `client-secret-reconciler.yaml` (CronJob: client secrets **and** client URLs ← git, every 15m) | **platform-config** | config |
| `client-urls.yaml` (**GENERATED** ConfigMap: the client-URL desired state) | **platform-config** | config | | `client-urls.yaml` (**GENERATED** ConfigMap: the client-URL desired state) | **platform-config** | config |
| `passkey-reconciler.yaml` (CronJob: WebAuthn passwordless policy + enrolment action + the **browser flow** ← git every 15m; the flow blue/green, never mutating the bound one) | **platform-config** | config | | `passkey-reconciler.yaml` (CronJob: WebAuthn passwordless policy + enrolment action + the **browser flow** ← git every 15m; the flow blue/green, never mutating the bound one, **bound only when git says so**) | **platform-config** | config |
| `passkey-desired-state.yaml` (**GENERATED** ConfigMap: the passkey desired state) | **platform-config** | config | | `passkey-desired-state.yaml` (**GENERATED** ConfigMap: the passkey desired state — plus the one hand-edited line, `bind-browser-flow`) | **platform-config** | config |
| `httproute.yaml` (keycloak.home.keithjackson.dev) | **platform-config** | config | | `httproute.yaml` (keycloak.home.keithjackson.dev) | **platform-config** | config |
| `servicemonitor.yaml` (Prometheus scrape) | **platform-config** | config | | `servicemonitor.yaml` (Prometheus scrape) | **platform-config** | config |
@@ -207,7 +207,7 @@ unrecoverable-credential bug, not a cosmetic one.
|---|---|---| |---|---|---|
| `webAuthnPolicyPasswordless*` (realm policy) | **The reconciler. GitOps.** | Realm top-level attributes; strict read-modify-write of `GET`/`PUT /admin/realms/homelab` | | `webAuthnPolicyPasswordless*` (realm policy) | **The reconciler. GitOps.** | Realm top-level attributes; strict read-modify-write of `GET`/`PUT /admin/realms/homelab` |
| `webauthn-register-passwordless` required action | **The reconciler. GitOps.** | Per-alias `PUT`, one literally-named alias; enabled, `defaultAction: false` | | `webauthn-register-passwordless` required action | **The reconciler. GitOps.** | Per-alias `PUT`, one literally-named alias; enabled, `defaultAction: false` |
| The `browser-mfa` flow restructure | **The reconciler. GitOps.** | Blue/green: build under a new alias, verify, flip `browserFlow` in one atomic PUT — see below | | The `browser-mfa` flow restructure | **The reconciler builds and verifies it. Git decides whether it is bound.** | Blue/green: build under a new alias, verify, and flip `browserFlow` in one atomic PUT **only when `bind-browser-flow: "true"`** — see below |
#### The flow converges blue/green — it never mutates the flow you log in through #### The flow converges blue/green — it never mutates the flow you log in through
@@ -229,8 +229,46 @@ flow. So the reconciler does not.
2. **Verify** by readback — nesting levels, ordering, authenticators, requirements, and 2. **Verify** by readback — nesting levels, ordering, authenticators, requirements, and
the conditional-OTP config *contents*. Nothing is bound unless this passes, so the the conditional-OTP config *contents*. Nothing is bound unless this passes, so the
regression that once 403'd every platform-admin is caught **before** it can bite. regression that once 403'd every platform-admin is caught **before** it can bite.
3. **Bind** with **one atomic PUT** of the realm, setting `browserFlow`. 3. **Bind** with **one atomic PUT** of the realm, setting `browserFlow` — **only if git says
4. **Rollback is one write**, and nothing is ever deleted, so the old flow stays usable. so** (next section). Otherwise the run stops here and logs `WOULD BIND browser-mfa-<hash>`.
4. **Rollback is a PR plus one write** (below), and nothing is ever deleted, so the old flow
stays usable.
#### The bind switch: `bind-browser-flow` in `passkey-desired-state.yaml`
That generated file carries exactly one hand-edited line, `bind-browser-flow`, and it is
`"false"` by default. The renderer carries it through on `--write`, and `--check` refuses
anything but the quoted strings `"true"` / `"false"` (an unquoted `true` is a YAML boolean,
which a ConfigMap rejects at apply time and which would fail the whole `platform-config`
Kustomization). The CronJob reads it from the same mount as the desired state; no flag can
widen it.
| `bind-browser-flow` | What the reconciler does to the realm's `browserFlow` |
|---|---|
| `"false"` (default) | **Never writes it.** It builds and verifies the candidate (or reuses an already verified one), logs `WOULD BIND <alias>`, and exits 0. Whatever the realm is bound to — including your rollback — is left alone on every run. A hand-edited bound flow is left alone too (no rebuild beside it). |
| `"true"` | Binds the verified candidate in one atomic PUT, **and keeps asserting it**: a binding changed by hand is changed back within 15 minutes, and a hand-edit *inside* the bound flow makes the next run build and bind a fresh copy. That is GitOps doing its job — so never hotfix the flow while this is `"true"`. |
**Merging with `"false"` changes nothing about how anyone logs in.** Passes 1 and 2 do write
the realm (the passwordless policy and the enrolment action, neither of which any login
step uses until a flow offers passkeys), and pass 3 creates the unbound candidate flow, but
`browserFlow` is untouched.
**Going live** is a later one-line PR — `bind-browser-flow: "true"` — made **after** the
candidate has been tested in a browser (see "Testing the candidate flow before binding it").
**Rolling back**, in this order:
1. A PR setting `bind-browser-flow: "false"`, merged and applied by Flux. **First** — while it
is `"true"`, the next run rebinds the candidate over your rollback.
2. One write: bind the previous flow again. The Job log of the bind printed it
(`ROLLBACK is one write: set browserFlow back to ...`); it is `browser-mfa` for the first
go-live. Admin console: *Authentication → Flows → <previous flow> → Action → Bind flow →
Browser flow*, or `PUT /admin/realms/homelab` as a read-modify-write of the full realm
representation (a partial body drops the passkey policy).
If login is broken *right now*, do step 2 immediately and step 1 straight after: the worst
case of that order is that one scheduled run (≤15 min) rebinds the candidate before Flux
applies the PR, and you repeat step 2.
**The safety property is structural, not intentional.** The reconciler is *incapable* of **The safety property is structural, not intentional.** The reconciler is *incapable* of
writing to the bound flow or anything beneath it: the live `browserFlow` alias and its writing to the bound flow or anything beneath it: the live `browserFlow` alias and its
@@ -257,36 +295,55 @@ flows to clean up rather than creating unbounded sprawl.
kubectl --context k3s-prod -n keycloak get jobs -l app.kubernetes.io/name=keycloak-passkey-reconciler kubectl --context k3s-prod -n keycloak get jobs -l app.kubernetes.io/name=keycloak-passkey-reconciler
kubectl --context k3s-prod -n keycloak logs -l app.kubernetes.io/name=keycloak-passkey-reconciler --tail=200 kubectl --context k3s-prod -n keycloak logs -l app.kubernetes.io/name=keycloak-passkey-reconciler --tail=200
# Force a run now rather than wait for the tick. NOTE: this runs the DEFAULT mode, which # Which candidate it would bind (bind-browser-flow "false"):
# converges (including flipping browserFlow once a new flow verifies). `kubectl create job kubectl --context k3s-prod -n keycloak logs -l app.kubernetes.io/name=keycloak-passkey-reconciler --tail=300 \
# --from=cronjob/...` CANNOT override the container command, so there is no way to get | grep -E 'WOULD BIND|BOUND browserFlow|PROBLEM'
# --audit or --no-bind out of it — for those, copy the CronJob's pod spec into a one-off
# Job and append the flag. # Force a run now rather than wait for the tick. It does exactly what the CronJob does,
# including honouring bind-browser-flow. `kubectl create job --from=cronjob/...` CANNOT
# override the container command, so --audit / --no-bind need a one-off Job with the flag
# appended (--no-bind can only NARROW git's switch, never widen it).
kubectl --context k3s-prod -n keycloak create job --from=cronjob/keycloak-passkey-reconciler \ kubectl --context k3s-prod -n keycloak create job --from=cronjob/keycloak-passkey-reconciler \
passkey-run-once passkey-run-once
kubectl --context k3s-prod -n keycloak logs job/passkey-run-once kubectl --context k3s-prod -n keycloak logs job/passkey-run-once
# ROLLBACK — one write. The run log names the previous alias ("ROLLBACK is one write: # ROLLBACK — see "The bind switch" above: PR bind-browser-flow "false" FIRST, then rebind
# set browserFlow back to ..."). The old flow is never deleted, so this always works. # the previous flow (Authentication -> Flows -> <previous> -> Action -> Bind flow ->
# browserFlow is a realm setting: Realm settings -> ... or # Browser flow). The old flow is never deleted, so this always works.
# PUT /admin/realms/homelab with browserFlow set to the previous alias
# (read-modify-write the realm representation; a partial body drops the passkey policy).
# The admin console path is Authentication -> Flows -> <old flow> -> Action -> Bind flow
# -> Browser flow. The console does this as one atomic change too.
# Regenerate the desired state after editing the passkey policy or the flow in the realm # Regenerate the desired state after editing the passkey policy or the flow in the realm
# import (the merge gate will demand it). # import (the merge gate will demand it).
python3 ci/render-keycloak-passkey-policy.py --write python3 ci/render-keycloak-passkey-policy.py --write
``` ```
**If you would rather the go-live be a human act,** the reconciler takes `--no-bind`: it ### Testing the candidate flow before binding it
builds and verifies the flow and stops, printing the one write needed to bind it. That
automates everything except one atomic operator action.
### Verifying it actually works (nobody has, as of 2026-10-18) With `bind-browser-flow: "false"` the candidate exists but nothing logs in through it. Test it
on **one client** without touching the realm binding, using a per-client flow override:
1. Find the candidate: the Job log line `WOULD BIND browser-mfa-<hash>`.
2. Admin console → *Clients → `account-console` → Advanced → Authentication flow overrides →
Browser Flow* = `browser-mfa-<hash>`. This is a temporary, hand-made client change; remove it
in step 7.
3. **Password fallback.** Private window → `https://keycloak.<base domain>/realms/homelab/account`.
Expect a **username** screen, then a choice: passkey, or the password form. As a user with no
passkey, choose password and sign in.
4. **Platform-admin TOTP on the password branch.** Repeat step 3 as a `platform-admins` member:
after the password, a TOTP prompt **must** appear. If it does not — or you get a 403 — the
conditional-OTP step is wrong: stop, do not bind.
5. **Passkey enrolment.** Signed in to the account console → *Signing in → Passkey → Set up*. A
browser/OS passkey prompt must appear; if it does not, the RP ID is wrong for the origin and
nothing server-side will say so.
6. **Passkey login.** Sign out; private window again; username, then choose passkey. Expect to be
signed in with **no** TOTP prompt, even as a platform-admin.
7. Remove the override from step 2.
8. Only now: the one-line PR `bind-browser-flow: "true"`. The next run binds the candidate you
tested (same alias — the hash is of the declared shape).
### Verifying it actually works (nobody has, as of 2026-10-29)
No passkey has been enrolled and no browser has been pointed at this. The checks that would prove No passkey has been enrolled and no browser has been pointed at this. The checks that would prove
it, in order: it on the BOUND flow, after go-live, in order:
```sh ```sh
# 1. the policy is live (RP ID must read as the base domain, NOT the keycloak host, NOT a URL) # 1. the policy is live (RP ID must read as the base domain, NOT the keycloak host, NOT a URL)
+25 -3
View File
@@ -1,6 +1,6 @@
--- ---
# ============================================================================= # =============================================================================
# GENERATED FILE — DO NOT EDIT BY HAND. # GENERATED FILE — DO NOT EDIT BY HAND, except the one bind-browser-flow line below.
# #
# source: platform/keycloak/keycloak-realm-import.yaml (spec.realm) # source: platform/keycloak/keycloak-realm-import.yaml (spec.realm)
# regenerate: python3 ci/render-keycloak-passkey-policy.py --write # regenerate: python3 ci/render-keycloak-passkey-policy.py --write
@@ -10,8 +10,14 @@
# WHAT THIS IS: the DESIRED STATE for the `homelab` realm's PASSKEY configuration — # WHAT THIS IS: the DESIRED STATE for the `homelab` realm's PASSKEY configuration —
# the WebAuthn passwordless policy, the enrolment required action, and the declared # the WebAuthn passwordless policy, the enrolment required action, and the declared
# browser authentication flow tree. platform/keycloak/passkey-reconciler.yaml mounts # browser authentication flow tree. platform/keycloak/passkey-reconciler.yaml mounts
# it and (a) converges the policy and the required action onto it, (b) AUDITS the # it and (a) converges the policy and the required action onto it, (b) builds the
# live browser flow against it and alarms on divergence without rewriting it. # declared flow tree BLUE/GREEN under a new alias and verifies it by readback, never
# writing the flow users are logging in through, and (c) BINDS that verified flow as
# the realm's browserFlow ONLY when `bind-browser-flow` below is "true".
#
# EVERYTHING HERE IS GENERATED EXCEPT ONE LINE: `bind-browser-flow`. That is an
# operator decision, carried through by the renderer from this committed file (see
# ci/render-keycloak-passkey-policy.py). It defaults to "false".
# #
# WHY IT IS A SEPARATE OBJECT: the realm import is a ONE-SHOT that never re-runs # WHY IT IS A SEPARATE OBJECT: the realm import is a ONE-SHOT that never re-runs
# once Done=True, so a realm-level field or a flow edited in git never reaches the # once Done=True, so a realm-level field or a flow edited in git never reaches the
@@ -51,6 +57,22 @@ metadata:
app.kubernetes.io/name: keycloak-passkey-desired-state app.kubernetes.io/name: keycloak-passkey-desired-state
app.kubernetes.io/part-of: keycloak app.kubernetes.io/part-of: keycloak
data: data:
# ---------------------------------------------------------------------------
# THE BIND SWITCH -- the ONLY hand-edited line in this file. Exactly "true" or
# "false", QUOTED.
#
# "false" the reconciler builds and verifies the candidate browser flow and
# logs `WOULD BIND <alias>`, but NEVER changes the realm's browserFlow.
# Whatever the realm is bound to -- including a rollback -- is left alone.
# "true" it binds the verified candidate in one atomic PUT, and keeps the
# realm bound to it: a binding changed by hand is changed back.
#
# GO LIVE: test the candidate in a browser first (platform/keycloak/README.md,
# "Passkeys"), then a PR setting this to "true".
# ROLL BACK: a PR setting this to "false" FIRST (or the next run rebinds), then
# one write setting browserFlow back to the previous alias from the Job log.
# ---------------------------------------------------------------------------
bind-browser-flow: "false"
passkey-desired-state.json: | passkey-desired-state.json: |
{ {
"authenticatorConfig": { "authenticatorConfig": {
File diff suppressed because it is too large. Load diff