feat(keycloak): reconcile the passkey realm config via GitOps, the browser flow blue/green
PR #72 declared the WebAuthn passwordless policy and a restructured browser-mfa flow in the realm import. That declaration could not take effect: a KeycloakRealmImport is a ONE-SHOT, and prod's reads Done=True / Started=False, so every realm-level field and flow declared after the first import is inert -- in git, applied by Flux, reported Ready, reaching the realm never. Merging it changed nothing about how anyone logs in, and nothing anywhere said so. Same failure class as the 2026-09-06 client-secret incident, third field over. A SIBLING reconciler converges all three declared things every 15 minutes. It is a sibling, not a second pass of client-secret-reconciler.yaml, so that reconciler's allow-list -- and its guarantee that prod's real user and the media-users / platform-admins / viewers groups are structurally unreachable -- stays literally true. Its selftest still passes unchanged. pass 1 webAuthnPolicyPasswordless* -- strict read-modify-write of GET/PUT /admin/realms/homelab, readback-verified. pass 2 webauthn-register-passwordless -- strict read-modify-write of ONE literally-named alias; enabled, defaultAction false so enrolment stays opt-in. pass 3 the browser flow -- BLUE/GREEN. PASS 3 NEVER MUTATES THE FLOW USERS ARE LOGGING IN THROUGH. Converging a five-flow, ten-execution tree needs create-subflow, create-execution, set-requirement and bind-config as SEPARATE untransacted calls; doing that to the BOUND flow means a run dying between them leaves a half-built login path for every SSO app and kubectl, and this repo has had a total SSO outage from a realm change. So instead: 1. BUILD under a NEW, unused alias (declared alias + a hash of the desired shape, applied to EVERY alias in the tree -- reusing browser-mfa-forms would mean writing INTO the bound tree). A failure leaves an orphaned, unbound flow and zero user-visible effect. 2. VERIFY by readback -- levels, ordering, authenticators, requirements, and the conditional-OTP config CONTENTS -- before anything is bound, so the regression that once 403'd every platform-admin is caught BEFORE it bites. 3. BIND with ONE atomic PUT setting browserFlow. 4. ROLLBACK is one write; no DELETE exists anywhere in the allow-list, so the old flow survives as the rollback target. The widened allow-list is made safe structurally. It cannot say "but not the bound flow" -- its write entries are alias-parameterised -- so a second guard screens every write against the live browserFlow's WHOLE subtree (sub-flow aliases and execution ids) before a socket opens, FAIL-CLOSED until that binding has been read. The create-flow endpoint carries no alias in its path, so it is screened on its request BODY; the selftest caught that it otherwise escaped the guard. Idempotence is load-bearing: a rebuild-and-reflip every 15 minutes would churn the login path on a schedule. The bound flow is compared STRUCTURALLY and a match produces zero writes and no new flow. Sub-flow and config ALIASES are deliberately not compared -- a built flow carries a hash suffix, so comparing labels would make it permanently unrecognisable as converged. Single source of truth: the desired state is GENERATED from the realm import by ci/render-keycloak-passkey-policy.py and gated with --check, so the RP ID -- baked into every credential at enrolment and never re-keyable -- is declared once. The pod keeps automountServiceAccountToken: false and reads a mounted ConfigMap, so no rbac.yaml is added anywhere. Read from Keycloak 26.0.6 source before being relied on: updateRealm (DefaultExportImportManager 725-894) makes no getUsers/getClients/getGroups/ getRoles/getRequiredActions/getAuthenticationFlows call; it DOES read getBrowserFlow(), which is what makes the atomic bind possible and why the policy write pins it; the passwordless policy is applied all-or-nothing only when RpEntityName is non-empty, with defaults substituted for omitted keys; and updateExecutions compares rep.getPriority(), so a body omitting `priority` would silently reshuffle the whole execution order. Gate coverage: a new whole-repo L0 check extracts the script and runs its --selftest (216 assertions, including a build killed at four different points each asserting the old flow is still bound and the bound subtree was never written), plus mutations.py (30 injected regressions, all caught) proving the selftest can fail. An earlier iteration of this work only AUDITED the flow and left the restructure manual; decisions/2026-10-18-keycloak-passkey-reconciler.md records that design and why it was reconsidered. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
jakman85andClaude Opus 5 committed
1 parent
1301af9a4c
commit
f5931944fa
9 files changed
+5200
-14
No files matched your search
@@ -0,0 +1,101 @@
|
||||
# =============================================================================
|
||||
# L0 STATIC — the passkey configuration is derived from one declaration, continuously
|
||||
# reconciled, and the browser login flow is auditable but UNWRITABLE
|
||||
# Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster.
|
||||
#
|
||||
# R-IDP-03 (identity config is declarative) and R-IDP-04 (MFA available) are both
|
||||
# implemented through a mechanism that FAILS OPEN: the KeycloakRealmImport is a ONE-SHOT.
|
||||
# Measured on prod 2026-10-18, hours after the passkey declaration merged:
|
||||
#
|
||||
# kubectl -n keycloak get keycloakrealmimport homelab -o jsonpath='{.status}'
|
||||
# Done=True Started=False HasErrors=False
|
||||
#
|
||||
# Once it rests at Done=True the operator never re-runs it, so the WebAuthn passwordless
|
||||
# policy and the restructured `browser-mfa` flow that keycloak-realm-import.yaml declares
|
||||
# are INERT — in git, applied by Flux, reported Ready, reaching the realm never. Merging
|
||||
# them changed nothing about how anyone logs in, and nothing anywhere said so.
|
||||
# Declarative config that silently does not apply is not declarative config.
|
||||
#
|
||||
# GIVEN the repository as authored,
|
||||
# THEN the reconciler's desired state is GENERATED from the realm import (so the RP ID —
|
||||
# which is baked into every passkey at enrolment and can NEVER be re-keyed — is
|
||||
# declared exactly once); both files are rendered by the SAME Flux Kustomization, so
|
||||
# their ${BASE_DOMAIN} token cannot resolve to two values; the renderer's exported
|
||||
# policy keys and the reconciler's writable keys are the SAME tuple; the realm
|
||||
# import declares NO requiredActions block (26.0.6 treats it as REPLACE, which would
|
||||
# delete CONFIGURE_TOTP and lock out every platform-admin); the enrolment action is
|
||||
# enabled but NOT a defaultAction, so enrolment stays opt-in; the reconciler carries
|
||||
# no ServiceAccount token, mounts only what it needs, and contains no dollar sign;
|
||||
# the endpoint allow-list contains NO write for any authentication flow, execution
|
||||
# or authenticator config, and NO required-action write for any alias other than the
|
||||
# one named literally; the SIBLING client-secret reconciler's allow-list is NOT
|
||||
# widened; and the reconciler's own `--selftest` (extracted from the shipped
|
||||
# ConfigMap and executed) proves compare-then-write, idempotence, strict
|
||||
# read-modify-write, the RpEntityName gate, flow-is-unwritable, the conditional-OTP
|
||||
# regression and fail-closed — with mutations.py proving that selftest can FAIL.
|
||||
#
|
||||
# WHY THE FLOW IS AUDITED AND NOT CONVERGED, asserted here rather than only argued:
|
||||
# converging a five-flow, ten-execution restructure needs create-subflow,
|
||||
# create-execution, set-requirement, set-priority and bind-config as SEPARATE admin API
|
||||
# calls with no transaction. A run that dies between them leaves a HALF-BUILT FLOW BOUND
|
||||
# AS THE LOGIN PATH for Grafana, Gitea, Nextcloud, Headlamp, Semaphore, PiHole, Crafty,
|
||||
# the *arr stack AND kubectl. This repo has already had a total SSO outage from a realm
|
||||
# change (2026-09-06), and the conditional-OTP sub-flow — which the declared structure
|
||||
# moves one level deeper than it sits today — once 403'd every platform-admin. A
|
||||
# reconciler that half-rewrites the bound flow every 15 minutes is a worse failure than
|
||||
# the one it replaces, because it is a loop rather than a single bad change.
|
||||
#
|
||||
# STATUS: EXECUTABLE. `check.sh` is a pure offline assertion script (git + PyYAML +
|
||||
# stdlib); conformance/run.sh --layer l0 and ci/l0.sh run it directly, and this Chainsaw
|
||||
# Test wraps the SAME script.
|
||||
#
|
||||
# NOT proven here: that the LIVE realm's policy, enrolment action or browser flow match
|
||||
# git (that is cluster state — the reconciler's `--audit` mode is the read-only probe for
|
||||
# it, and NO L2 test is added by this change, so the live realm is UNVERIFIED by CI);
|
||||
# that a passkey ceremony actually completes or that the ALTERNATIVE fork degrades to the
|
||||
# password form (needs a real browser against a real realm); that the five flow-building
|
||||
# admin-API calls behave against a live Keycloak the way the 26.0.6 source says they do
|
||||
# (every proof here is against an in-process stub); or that the CronJob's schedule fires.
|
||||
# =============================================================================
|
||||
apiVersion: chainsaw.kyverno.io/v1alpha1
|
||||
kind: Test
|
||||
metadata:
|
||||
name: keycloak-passkey-reconciler
|
||||
labels:
|
||||
conformance.home.k3s/R-IDP-03: "true"
|
||||
conformance.home.k3s/R-IDP-04: "true"
|
||||
annotations:
|
||||
conformance.home.k3s/requirements: "R-IDP-03, R-IDP-04"
|
||||
conformance.home.k3s/layer: "L0"
|
||||
# WHOLE-REPO SCOPE, for the same reason the sibling keycloak-client-secret-reconciler
|
||||
# check declares it: this invariant spans platform/ (the realm import, the reconciler,
|
||||
# the generated desired state), platform-config/ (which Flux Kustomization renders
|
||||
# them, and therefore which substitution pass they share), ci/ (the renderer) and
|
||||
# clusters/ (that both are reachable from both entrypoints). A change that breaks it
|
||||
# need not attribute R-IDP-03/04 at all, and the requirement-scoped selection would
|
||||
# then filter this check out of exactly the change that introduces the bug — which is
|
||||
# how 12 unusable prod OIDC clients sat behind an entirely green cluster on
|
||||
# 2026-09-06.
|
||||
conformance.home.k3s/scope: "whole-repo"
|
||||
# Not safety-critical in the harness's sense (it mutates nothing and needs no
|
||||
# cluster), but it guards a component that holds WRITE access to the identity
|
||||
# provider's realm settings.
|
||||
conformance.home.k3s/safety-critical: "false"
|
||||
conformance.home.k3s/executable: "true"
|
||||
conformance.home.k3s/intent: >-
|
||||
The realm's passkey configuration — policy, enrolment action AND browser login
|
||||
flow — is declared once in the realm import, derived from it by a gated renderer,
|
||||
and continuously reconciled; the flow converges blue/green, so a one-shot import
|
||||
that never re-runs can neither leave the configuration inert nor let a reconciler
|
||||
half-rewrite the login path of the whole platform.
|
||||
spec:
|
||||
steps:
|
||||
- name: audit-passkey-declaration-renderer-and-reconciler
|
||||
description: >-
|
||||
Assert the desired state is derived from the realm import and shares its
|
||||
substitution pass, that the reconciler's write surface excludes every flow
|
||||
mutation, that the sibling reconciler is unwidened, and run the reconciler's own
|
||||
selftest plus the mutation suite that proves it can fail.
|
||||
try:
|
||||
- script:
|
||||
content: bash ./check.sh
|
||||
@@ -0,0 +1,708 @@
|
||||
#!/usr/bin/env bash
|
||||
# =============================================================================
|
||||
# L0 static check (offline, no cluster) — keycloak-passkey-reconciler
|
||||
# Serves: R-IDP-03 ("Identity config (realms, clients, groups, roles) is declarative."
|
||||
# — spec docs/15-identity-and-access.md)
|
||||
# R-IDP-04 ("IdP secrets SOPS-encrypted; MFA available."
|
||||
# — spec docs/15-identity-and-access.md)
|
||||
#
|
||||
# WHY THIS EXISTS — THE REALM IMPORT IS A ONE-SHOT THAT FAILS OPEN
|
||||
# ---------------------------------------------------------------
|
||||
# platform/keycloak/keycloak-realm-import.yaml declares the WebAuthn PASSWORDLESS
|
||||
# policy and a restructured `browser-mfa` browser flow. Measured on prod 2026-10-18,
|
||||
# hours after that declaration merged:
|
||||
#
|
||||
# kubectl -n keycloak get keycloakrealmimport homelab -o jsonpath='{.status}'
|
||||
# Done=True Started=False HasErrors=False
|
||||
#
|
||||
# A KeycloakRealmImport is a ONE-SHOT. Once it rests at Done=True the operator never
|
||||
# re-runs it, so every realm-level field and every flow declared after the FIRST
|
||||
# import is inert — in git, applied by Flux, reported Ready, reaching the realm
|
||||
# never. The same shape as the 2026-09-06 client-secret incident (12 prod clients
|
||||
# holding the literal `(env:...)` placeholder behind an entirely green cluster), and
|
||||
# the same fix: a level-triggered reconciler. This check is the static half.
|
||||
#
|
||||
# THE RECONCILER HOLDS WRITE ACCESS TO THE IDENTITY PROVIDER. That is the reason this
|
||||
# check is unusually thorough about its BLAST RADIUS rather than only its behaviour.
|
||||
#
|
||||
# GIVEN the repository as authored,
|
||||
# THEN (1) THE DESIRED STATE IS DERIVED, NOT DECLARED TWICE: the reconciler's input
|
||||
# (platform/keycloak/passkey-desired-state.yaml) is GENERATED from the realm
|
||||
# import by ci/render-keycloak-passkey-policy.py, and this check RUNS the
|
||||
# renderer with --check, so a hand edit that makes the two disagree is
|
||||
# unmergeable. The RP ID is the sharpest case: it is baked into every passkey
|
||||
# at enrolment and can NEVER be re-keyed, so two declarations that can skew
|
||||
# about the domain is an unrecoverable-credential bug, not a cosmetic one.
|
||||
# (2) BOTH FILES ARE RENDERED BY THE SAME FLUX KUSTOMIZATION.
|
||||
# postBuild.substituteFrom is per-Kustomization, so a split would let the
|
||||
# ${BASE_DOMAIN} token in the realm import and the one in the desired state
|
||||
# resolve to two different values — and an RP ID mismatch invalidates every
|
||||
# enrolled passkey with no error the user can act on.
|
||||
# (3) THE DESIRED STATE USES NO SUBSTITUTION TOKEN THE REALM IMPORT DOES NOT, and
|
||||
# declares no realm key outside the reconciler's write surface.
|
||||
# (4) THE TWO WRITE SURFACES AGREE: the renderer's exported policy keys and the
|
||||
# reconciler's POLICY_FIELDS are the SAME tuple. If they drifted, the renderer
|
||||
# could emit a key the reconciler refuses (fail-closed forever) or the
|
||||
# reconciler could accept a key the renderer never emits (dead surface).
|
||||
# (5) THE RECONCILER MANIFEST IS SAFE AND WIRED: reconciled from BOTH cluster
|
||||
# entrypoints, mounts exactly the ConfigMaps its code defaults to, carries NO
|
||||
# ServiceAccount token, and contains no dollar sign (it ships inside a
|
||||
# SUBSTITUTED Flux path, where an unescaped dollar-brace expansion is
|
||||
# rewritten before any container runs).
|
||||
# (6) THE CURRENTLY BOUND FLOW IS STRUCTURALLY UNWRITABLE — asserted against the
|
||||
# SOURCE, not the comments. The flow converges BLUE/GREEN (build under a new
|
||||
# alias, verify, flip browserFlow in one atomic PUT), so flow-create and
|
||||
# execution-add endpoints ARE on the allow-list and their presence is not the
|
||||
# hazard. What is asserted instead: nothing is DELETABLE (the superseded flow
|
||||
# is the rollback target); the execution- and config-id-scoped write endpoints
|
||||
# stay off the list because they carry no flow alias and so cannot be screened;
|
||||
# no required-action write reaches any alias but the one named literally; and
|
||||
# the guard the allow-list CANNOT express exists in the source and is wired
|
||||
# into the single request choke point, FAIL-CLOSED until the live binding has
|
||||
# been read. That last one is what the whole design rests on: a reconciler that
|
||||
# half-rewrites the BOUND browser flow is a total SSO outage on a loop, and
|
||||
# this repo has had one (2026-09-06).
|
||||
# (7) THE SIBLING RECONCILER IS UNTOUCHED: client-secret-reconciler.yaml's
|
||||
# allow-list is NOT widened by this work. Its --selftest asserts it refuses
|
||||
# /users, /groups, /roles, /authentication and the whole-realm endpoint, and
|
||||
# its guarantee that prod's real user and the media-users / platform-admins /
|
||||
# viewers groups survive every run must stay LITERALLY true.
|
||||
# (8) THE RECONCILER'S BEHAVIOUR IS PROVEN, NOT ASSERTED: this check EXTRACTS the
|
||||
# script from the ConfigMap and executes its `--selftest`, which drives the
|
||||
# real code paths against an in-process stub Keycloak that MODELS the three
|
||||
# Keycloak 26.0.6 behaviours that bite — the RpEntityName gate on the
|
||||
# passwordless policy, the all-or-nothing policy rebuild that defaults every
|
||||
# omitted key, and the required-action representation replace.
|
||||
# (8b) AND THAT PROOF IS FALSIFIABLE: mutations.py injects plausible regressions
|
||||
# one at a time and requires --selftest to CATCH every one. A passing suite
|
||||
# proves nothing if it cannot fail, and "reported success while the realm was
|
||||
# broken" is the exact failure this component exists to end.
|
||||
# AND (9) THIS CHECK IS LOAD-BEARING: it DECLARES
|
||||
# `conformance.home.k3s/scope: whole-repo` so ci/l0.sh runs it unconditionally
|
||||
# via --include-scope, ci/gate.sh blocks the merge when the l0 stage fails, and
|
||||
# ci/touched-requirements.sh maps platform/keycloak/ to these requirement IDs.
|
||||
# An unreferenced test is precisely the "looks applied and isn't" failure this
|
||||
# repo keeps hitting.
|
||||
#
|
||||
# NOT proven here (named honestly, in the style of the sibling L0 checks):
|
||||
# - that the LIVE realm's policy, enrolment action or browser flow match git. That is
|
||||
# cluster state. The reconciler's own `--audit` mode is the read-only probe for it
|
||||
# and is what an L2 test should run; no such L2 test is added by this change, so the
|
||||
# live realm is UNVERIFIED by CI.
|
||||
# - that the declared browser flow actually WORKS as a login path. Nothing offline can
|
||||
# prove that a passkey ceremony completes, that the ALTERNATIVE fork degrades to the
|
||||
# password form, or that a platform-admin is still asked for TOTP. Those need a real
|
||||
# browser against a real realm.
|
||||
# - that the reconciler's first real run against prod will succeed. The flow is built
|
||||
# through five admin-API endpoints whose request shapes were read from the 26.0.6
|
||||
# source but never exercised against a live Keycloak.
|
||||
# - that the admin credential decrypts to a usable password. Key PRESENCE is checkable
|
||||
# offline; the plaintext is not, and must not be.
|
||||
# - that the CronJob's image can be pulled or its schedule fires.
|
||||
# =============================================================================
|
||||
set -euo pipefail
|
||||
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ROOT="$(cd "${DIR}/../../../.." && pwd)" # repo root
|
||||
PYTHON="${PYTHON:-python3}"
|
||||
|
||||
IMPORT_FILE="platform/keycloak/keycloak-realm-import.yaml"
|
||||
RECON_FILE="platform/keycloak/passkey-reconciler.yaml"
|
||||
STATE_FILE="platform/keycloak/passkey-desired-state.yaml"
|
||||
RENDERER="ci/render-keycloak-passkey-policy.py"
|
||||
SIBLING_FILE="platform/keycloak/client-secret-reconciler.yaml"
|
||||
|
||||
# Honest SKIP (never green) if the thing under test is absent entirely.
|
||||
for f in "$IMPORT_FILE" "$RECON_FILE" "$STATE_FILE" "$RENDERER"; do
|
||||
if [ ! -f "${ROOT}/${f}" ]; then
|
||||
echo "check SKIPPED: ${f} absent — nothing to audit" >&2
|
||||
exit 3
|
||||
fi
|
||||
done
|
||||
|
||||
cd "${ROOT}"
|
||||
"${PYTHON}" - "${ROOT}" <<'PY'
|
||||
"""Static audit of the Keycloak passkey declaration, the renderer that derives the
|
||||
reconciler's desired state from it, and the reconciler's blast radius. PyYAML parses the
|
||||
manifests; lib/status.py is IMPORTED for the Flux graph walk rather than forked, so there
|
||||
is one definition of "what a Flux path renders" in the harness."""
|
||||
from __future__ import annotations
|
||||
|
||||
import ast
|
||||
import json
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
ROOT = Path(sys.argv[1])
|
||||
# This body is fed to `python3 -` on stdin, so there is no __file__ to derive from; the
|
||||
# test directory is named relative to the repo root the wrapper passes in.
|
||||
DIR_PATH = (ROOT / "conformance" / "tests" / "l0-static"
|
||||
/ "keycloak-passkey-reconciler")
|
||||
sys.path.insert(0, str(ROOT / "conformance" / "lib"))
|
||||
import status as st # noqa: E402
|
||||
|
||||
IMPORT_FILE = "platform/keycloak/keycloak-realm-import.yaml"
|
||||
RECON_FILE = "platform/keycloak/passkey-reconciler.yaml"
|
||||
STATE_FILE = "platform/keycloak/passkey-desired-state.yaml"
|
||||
RENDERER = "ci/render-keycloak-passkey-policy.py"
|
||||
SIBLING_FILE = "platform/keycloak/client-secret-reconciler.yaml"
|
||||
STATE_CM_NAME = "keycloak-passkey-desired-state"
|
||||
STATE_DATA_KEY = "passkey-desired-state.json"
|
||||
SCRIPT_KEY = "reconcile-keycloak-passkeys.py"
|
||||
ENROL_ACTION = "webauthn-register-passwordless"
|
||||
DOLLAR = chr(36)
|
||||
|
||||
failures: list[str] = []
|
||||
|
||||
|
||||
def check(cond, msg):
|
||||
if not cond:
|
||||
failures.append(msg)
|
||||
|
||||
|
||||
def load(rel):
|
||||
return list(yaml.safe_load_all((ROOT / rel).read_text()))
|
||||
|
||||
|
||||
# ================================================== (1) the desired state is DERIVED
|
||||
proc = subprocess.run([sys.executable, str(ROOT / RENDERER), "--check"],
|
||||
capture_output=True, text=True, timeout=120)
|
||||
check(proc.returncode == 0,
|
||||
f"{STATE_FILE} has DRIFTED from {IMPORT_FILE} (ci/render-keycloak-passkey-policy"
|
||||
f".py --check failed). The reconciler would converge the realm on a passkey policy "
|
||||
f"the realm import does not declare, and audit the login flow against a tree "
|
||||
f"nobody authored. The RP ID is the sharpest case: it is baked into every passkey "
|
||||
f"at enrolment and can NEVER be re-keyed, so two declarations that skew about the "
|
||||
f"domain is an unrecoverable-credential bug.\n "
|
||||
+ "\n ".join((proc.stdout + proc.stderr).strip().splitlines()[-12:]))
|
||||
|
||||
# The realm import must actually declare the things the renderer derives from. If it
|
||||
# stops, the renderer fails -- but say so HERE too, because a SKIP-shaped failure in the
|
||||
# renderer would otherwise be the only signal.
|
||||
imp_docs = [d for d in load(IMPORT_FILE) if d]
|
||||
imp = next((d for d in imp_docs if d.get("kind") == "KeycloakRealmImport"), None)
|
||||
check(imp is not None, f"{IMPORT_FILE}: no KeycloakRealmImport document")
|
||||
realm = ((imp or {}).get("spec") or {}).get("realm") or {}
|
||||
policy_keys_in_import = sorted(k for k in realm
|
||||
if k.startswith("webAuthnPolicyPasswordless"))
|
||||
check(policy_keys_in_import,
|
||||
f"{IMPORT_FILE} declares no webAuthnPolicyPasswordless* field, so there is no "
|
||||
f"passkey policy for the reconciler to converge")
|
||||
check(realm.get("browserFlow"),
|
||||
f"{IMPORT_FILE} declares no spec.realm.browserFlow — a perfect flow that is not "
|
||||
f"BOUND changes nothing about how anyone logs in")
|
||||
|
||||
# THE requiredActions TRAP, enforced rather than only commented. The realm import's own
|
||||
# header explains it at length: Keycloak 26.0.6 treats requiredActions in a realm
|
||||
# representation as REPLACE, not merge, so declaring ONE action imports exactly that
|
||||
# action plus delete_account and skips DefaultRequiredActions.addActions entirely --
|
||||
# deleting CONFIGURE_TOTP, UPDATE_PASSWORD, VERIFY_EMAIL and seven more. Losing
|
||||
# CONFIGURE_TOTP is a hard lock-out of every platform-admin, because auth-otp-form in the
|
||||
# browser flow still demands a TOTP they could then never enrol. The comment saying "DO
|
||||
# NOT ADD ONE" is advice; this is the gate.
|
||||
check("requiredActions" not in realm,
|
||||
f"{IMPORT_FILE} declares spec.realm.requiredActions. Keycloak 26.0.6 treats that "
|
||||
f"as REPLACE, not merge: declaring any action imports exactly those plus "
|
||||
f"delete_account and NEVER runs DefaultRequiredActions.addActions, so "
|
||||
f"CONFIGURE_TOTP, UPDATE_PASSWORD, VERIFY_EMAIL and seven more are DELETED. That "
|
||||
f"is a permanent lock-out of every platform-admin, because auth-otp-form in this "
|
||||
f"realm's browser flow still demands a TOTP they could no longer enrol. The "
|
||||
f"enrolment action is already registered by default (priority 80, enabled, "
|
||||
f"defaultAction false) and the passkey reconciler converges it through the "
|
||||
f"per-alias admin API, which has no such hazard")
|
||||
|
||||
# ============================== (2)(3) the two files share one substitution pass
|
||||
state_docs = [d for d in load(STATE_FILE) if d]
|
||||
state_cm = next((d for d in state_docs if d.get("kind") == "ConfigMap"), None)
|
||||
check(state_cm is not None, f"{STATE_FILE}: no ConfigMap")
|
||||
desired = {}
|
||||
if state_cm is not None:
|
||||
check(state_cm["metadata"]["name"] == STATE_CM_NAME,
|
||||
f"{STATE_FILE}: the ConfigMap is named "
|
||||
f"{state_cm['metadata']['name']!r}, but the CronJob mounts {STATE_CM_NAME!r}")
|
||||
data = state_cm.get("data") or {}
|
||||
check(list(data) == [STATE_DATA_KEY],
|
||||
f"{STATE_FILE}: expected exactly the data key {STATE_DATA_KEY!r}, got "
|
||||
f"{sorted(data)} — the reconciler's PASSKEY_STATE_FILE default would not "
|
||||
f"resolve")
|
||||
raw = data.get(STATE_DATA_KEY, "")
|
||||
try:
|
||||
desired = json.loads(raw)
|
||||
except ValueError as e:
|
||||
check(False, f"{STATE_FILE}: the {STATE_DATA_KEY} payload is not valid JSON "
|
||||
f"({e}) — the reconciler would fail closed on every run")
|
||||
|
||||
# Both files must be rendered by ONE Flux Kustomization. substituteFrom is
|
||||
# per-Kustomization, so a split lets the same ${BASE_DOMAIN} token resolve to two
|
||||
# different values -- and an RP ID mismatch invalidates every enrolled passkey silently.
|
||||
import_owners = set()
|
||||
state_owners = set()
|
||||
recon_owners = set()
|
||||
for env in ("dev", "prod"):
|
||||
entry = f"clusters/{env}"
|
||||
if not (ROOT / entry).is_dir():
|
||||
continue
|
||||
applied, _ = st.reconciled_files(ROOT, entry)
|
||||
check(RECON_FILE in applied,
|
||||
f"{RECON_FILE} is not reachable from the {entry} entrypoint — the passkey "
|
||||
f"healer would exist on one cluster only")
|
||||
check(STATE_FILE in applied,
|
||||
f"{STATE_FILE} is not reachable from the {entry} entrypoint — the reconciler "
|
||||
f"would fail closed on that cluster forever")
|
||||
check(IMPORT_FILE in applied, f"{IMPORT_FILE} is not reachable from {entry}")
|
||||
|
||||
# Which Kustomization DIRECTORY lists each file. Asserted structurally: both must be
|
||||
# listed as resources of the same kustomization.yaml.
|
||||
for kpath in ROOT.glob("*/kustomization.yaml"):
|
||||
try:
|
||||
k = yaml.safe_load(kpath.read_text()) or {}
|
||||
except Exception:
|
||||
continue
|
||||
res = [str(r) for r in (k.get("resources") or [])]
|
||||
base = kpath.parent.name
|
||||
if any(r.endswith("keycloak-realm-import.yaml") for r in res):
|
||||
import_owners.add(base)
|
||||
if any(r.endswith("passkey-desired-state.yaml") for r in res):
|
||||
state_owners.add(base)
|
||||
if any(r.endswith("passkey-reconciler.yaml") for r in res):
|
||||
recon_owners.add(base)
|
||||
check(import_owners and import_owners == state_owners,
|
||||
f"{IMPORT_FILE} is rendered by {sorted(import_owners)} but {STATE_FILE} by "
|
||||
f"{sorted(state_owners)}. They MUST be the same Flux Kustomization: "
|
||||
f"postBuild.substituteFrom is per-Kustomization, so a split would let the "
|
||||
f"{DOLLAR}{{BASE_DOMAIN}} token in each resolve to a different value. An RP ID is "
|
||||
f"baked into every passkey at enrolment and can never be re-keyed, so a domain the "
|
||||
f"two files disagree about is an unrecoverable credential, not a cosmetic drift")
|
||||
check(recon_owners == state_owners,
|
||||
f"{RECON_FILE} is rendered by {sorted(recon_owners)} but its desired state by "
|
||||
f"{sorted(state_owners)} — the reconciler could be applied without its input")
|
||||
|
||||
# (3) no substitution token the realm definition does not also use.
|
||||
import_text = (ROOT / IMPORT_FILE).read_text()
|
||||
TOKEN = re.compile(re.escape(DOLLAR) + r"\{([A-Za-z_][A-Za-z0-9_]*)")
|
||||
state_tokens = set(TOKEN.findall((ROOT / STATE_FILE).read_text()))
|
||||
import_tokens = set(TOKEN.findall(import_text))
|
||||
# The renderer's own HEADER mentions ${BASE_DOMAIN} in prose; that is the same token the
|
||||
# policy uses, so a subset check is the right assertion rather than an exact match.
|
||||
check(state_tokens <= import_tokens,
|
||||
f"{STATE_FILE} uses substitution token(s) {sorted(state_tokens - import_tokens)} "
|
||||
f"that {IMPORT_FILE} does not. A token the realm definition never uses is either a "
|
||||
f"hand edit or a renderer bug, and an UNDEFINED one renders as the EMPTY STRING "
|
||||
f"silently — which for an RP ID means every passkey ceremony fails")
|
||||
|
||||
# ======================================= (4) the two write surfaces are the SAME tuple
|
||||
recon_docs = [d for d in load(RECON_FILE) if d]
|
||||
cm = next((d for d in recon_docs if d.get("kind") == "ConfigMap"), None)
|
||||
cj = next((d for d in recon_docs if d.get("kind") == "CronJob"), None)
|
||||
check(cm is not None, f"{RECON_FILE}: no ConfigMap carrying the reconciler script")
|
||||
check(cj is not None, f"{RECON_FILE}: no CronJob running the reconciler")
|
||||
|
||||
script = ""
|
||||
if cm is not None:
|
||||
data = cm.get("data") or {}
|
||||
check(len(data) == 1,
|
||||
f"{RECON_FILE}: the reconciler ConfigMap must carry exactly one script key "
|
||||
f"(found {sorted(data)}) — a second copy would be untested by this gate")
|
||||
check(SCRIPT_KEY in data,
|
||||
f"{RECON_FILE}: expected the script under {SCRIPT_KEY!r}, got {sorted(data)}")
|
||||
script = next(iter(data.values()), "")
|
||||
check(len(script.splitlines()) > 200,
|
||||
f"{RECON_FILE}: the extracted script is implausibly short "
|
||||
f"({len(script.splitlines())} lines) — the ConfigMap shape changed and the "
|
||||
f"selftest below would prove nothing")
|
||||
# THE nextcloud TRAP. This file is rendered by a Flux Kustomization with
|
||||
# postBuild.substituteFrom, so envsubst rewrites it before any container runs.
|
||||
check(DOLLAR not in script,
|
||||
f"{RECON_FILE}: the reconciler script contains a dollar sign. It ships inside "
|
||||
f"a SUBSTITUTED Flux path, where a dollar-brace expansion is rewritten at "
|
||||
f"reconcile time (see apps/wave-1/nextcloud.yaml). Rewrite without it — Python "
|
||||
f"never needs one; chr(36) spells it where a literal is genuinely required")
|
||||
|
||||
|
||||
def tuple_literal(src, name):
|
||||
"""The value of a module-level tuple assignment, via ast. Parsing the SOURCE rather
|
||||
than importing it keeps this check offline and side-effect free, and means the thing
|
||||
asserted is the text that ships."""
|
||||
try:
|
||||
tree = ast.parse(src)
|
||||
except SyntaxError:
|
||||
return None
|
||||
for node in tree.body:
|
||||
if isinstance(node, ast.Assign):
|
||||
for t in node.targets:
|
||||
if isinstance(t, ast.Name) and t.id == name:
|
||||
try:
|
||||
return ast.literal_eval(node.value)
|
||||
except ValueError:
|
||||
return None
|
||||
return None
|
||||
|
||||
|
||||
renderer_src = (ROOT / RENDERER).read_text()
|
||||
|
||||
|
||||
# POLICY_FIELDS is a CONCATENATION of other tuples in both files, so literal_eval of the
|
||||
# assignment itself cannot evaluate it. Resolve it from its parts instead. Both files name
|
||||
# their parts from the same vocabulary; a part present in one and not the other simply
|
||||
# contributes nothing, which is why the comparison below is of the resolved SETS.
|
||||
def resolve_policy(src):
|
||||
parts = ("POLICY_STR_FIELDS", "POLICY_ORDERED_LIST_FIELDS",
|
||||
"POLICY_UNORDERED_LIST_FIELDS", "POLICY_LIST_FIELDS", "POLICY_INT_FIELDS",
|
||||
"POLICY_BOOL_FIELDS")
|
||||
out = []
|
||||
for p in parts:
|
||||
v = tuple_literal(src, p)
|
||||
if v:
|
||||
out.extend(v)
|
||||
return sorted(set(out))
|
||||
|
||||
|
||||
recon_keys = resolve_policy(script)
|
||||
rend_keys = resolve_policy(renderer_src)
|
||||
check(recon_keys and recon_keys == rend_keys,
|
||||
f"the reconciler's writable policy keys {recon_keys} and the renderer's exported "
|
||||
f"keys {rend_keys} DISAGREE. If the renderer emits a key the reconciler refuses, "
|
||||
f"every run fails closed forever; if the reconciler accepts a key the renderer "
|
||||
f"never emits, that is dead write surface on the identity provider")
|
||||
# Every exported key must actually be a passwordless key. The two-factor
|
||||
# webAuthnPolicy* family is a silent no-op for passkeys (the passwordless authenticator
|
||||
# reads getWebAuthnPolicyPasswordless() only), so exporting one would be a write surface
|
||||
# with no effect.
|
||||
check(all(k.startswith("webAuthnPolicyPasswordless") for k in recon_keys),
|
||||
f"the reconciler's writable keys include a non-passwordless field: "
|
||||
f"{[k for k in recon_keys if not k.startswith('webAuthnPolicyPasswordless')]}. "
|
||||
f"The two-factor webAuthnPolicy* family is never consulted by a passkey login, so "
|
||||
f"writing it is a silent no-op")
|
||||
# And the desired state must declare only those.
|
||||
if isinstance(desired.get("webAuthnPolicyPasswordless"), dict):
|
||||
extra = sorted(set(desired["webAuthnPolicyPasswordless"]) - set(recon_keys))
|
||||
check(not extra,
|
||||
f"{STATE_FILE} declares policy key(s) {extra} outside the reconciler's write "
|
||||
f"surface — it would fail closed on every run")
|
||||
# The 26.0.6 gate: the policy is applied ONLY when RpEntityName is non-empty.
|
||||
check(desired["webAuthnPolicyPasswordless"].get(
|
||||
"webAuthnPolicyPasswordlessRpEntityName"),
|
||||
f"{STATE_FILE} carries no non-empty "
|
||||
f"webAuthnPolicyPasswordlessRpEntityName. Keycloak 26.0.6 applies the "
|
||||
f"passwordless policy only when that field is set "
|
||||
f"(DefaultExportImportManager.updateRealm), so every realm PUT would be "
|
||||
f"accepted with 204 and change nothing")
|
||||
|
||||
# THE ENROLMENT POSTURE. defaultAction true would force every user to enrol a passkey at
|
||||
# next login -- a platform-wide lock-out risk, and the exact opposite of the realm
|
||||
# import's stated "additive and optional" decision. It is one boolean, so it is gated.
|
||||
ra = desired.get("requiredAction") or {}
|
||||
check(ra.get("alias") == ENROL_ACTION,
|
||||
f"{STATE_FILE}: requiredAction.alias is {ra.get('alias')!r}, expected "
|
||||
f"{ENROL_ACTION!r} — the reconciler's allow-list names that alias literally, so "
|
||||
f"any other value is unreachable and the pass would fail closed")
|
||||
check(ra.get("enabled") is True,
|
||||
f"{STATE_FILE}: requiredAction.enabled must be true, else passkey ENROLMENT is "
|
||||
f"impossible while the login flow offers passkeys")
|
||||
check(ra.get("defaultAction") is False,
|
||||
f"{STATE_FILE}: requiredAction.defaultAction is {ra.get('defaultAction')!r}, which "
|
||||
f"must be false. True forces EVERY user to enrol a passkey at next login — this "
|
||||
f"IdP gates Grafana, Gitea, Nextcloud, Headlamp, Semaphore, PiHole, Crafty, the "
|
||||
f"*arr stack AND kubectl, so a misfiring mandatory enrolment is a total platform "
|
||||
f"lock-out. The realm import's whole stated posture is additive and OPTIONAL")
|
||||
|
||||
# ========================================= (5) the reconciler manifest is safe + wired
|
||||
if cj is not None:
|
||||
pod = cj["spec"]["jobTemplate"]["spec"]["template"]["spec"]
|
||||
ctr = pod["containers"][0]
|
||||
vols = {v["name"]: v for v in pod.get("volumes", [])}
|
||||
mounts = {m["name"]: m for m in ctr.get("volumeMounts", [])}
|
||||
env = {e["name"]: e for e in ctr.get("env", [])}
|
||||
|
||||
# NO KUBERNETES API ACCESS. This is what makes the mounted-ConfigMap design strictly
|
||||
# narrower than reading the KeycloakRealmImport CR through the API: no Role, no
|
||||
# RoleBinding, no token. It is also why this change adds no **/rbac.yaml.
|
||||
check(pod.get("automountServiceAccountToken") is False,
|
||||
f"{RECON_FILE}: the reconciler must set automountServiceAccountToken: false. "
|
||||
f"It talks only to the Keycloak admin API and reads a mounted ConfigMap, so a "
|
||||
f"ServiceAccount token is pure blast radius (R-SEC-02)")
|
||||
check("serviceAccountName" not in pod,
|
||||
f"{RECON_FILE}: the reconciler must not name a ServiceAccount")
|
||||
|
||||
cm_vols = {n: v for n, v in vols.items() if "configMap" in v}
|
||||
names = {(v["configMap"] or {}).get("name") for v in cm_vols.values()}
|
||||
check(STATE_CM_NAME in names,
|
||||
f"{RECON_FILE}: the CronJob does not mount the {STATE_CM_NAME} ConfigMap, so "
|
||||
f"the reconciler would fail closed on every run. Mounted: {sorted(names)}")
|
||||
check(cm["metadata"]["name"] in names,
|
||||
f"{RECON_FILE}: the CronJob does not mount its own script ConfigMap")
|
||||
# NO SECRET IS MOUNTED AS A VOLUME. The admin credential is consumed via
|
||||
# secretKeyRef only; projecting it as a file would widen what a readable mount leaks.
|
||||
check(not [v for v in vols.values() if "secret" in v],
|
||||
f"{RECON_FILE}: the reconciler mounts a Secret as a volume. The admin "
|
||||
f"credential is referenced by secretKeyRef only; it must never be projected "
|
||||
f"as a file here")
|
||||
|
||||
# The mount paths must be the ones the code DEFAULTS to, or the container starts and
|
||||
# fails closed forever while every manifest looks correct.
|
||||
state_vol = next((n for n, v in cm_vols.items()
|
||||
if (v["configMap"] or {}).get("name") == STATE_CM_NAME), None)
|
||||
if state_vol:
|
||||
mp = (mounts.get(state_vol) or {}).get("mountPath")
|
||||
check(mp == "/etc/keycloak-passkey",
|
||||
f"{RECON_FILE}: the desired-state ConfigMap is mounted at {mp!r}, but the "
|
||||
f"script's PASSKEY_STATE_FILE default is "
|
||||
f"/etc/keycloak-passkey/{STATE_DATA_KEY}")
|
||||
check(f'"/etc/keycloak-passkey/{STATE_DATA_KEY}"' in script,
|
||||
f"{RECON_FILE}: the script's PASSKEY_STATE_FILE default no longer matches "
|
||||
f"the mount path — they are a pair and must move together")
|
||||
check((mounts.get(state_vol) or {}).get("readOnly") is True,
|
||||
f"{RECON_FILE}: the desired-state mount must be readOnly")
|
||||
|
||||
# The admin credential, referenced BY NAME. Never decrypted, never a literal.
|
||||
for key in ("KEYCLOAK_ADMIN_USER", "KEYCLOAK_ADMIN_PASSWORD"):
|
||||
e = env.get(key) or {}
|
||||
ref = ((e.get("valueFrom") or {}).get("secretKeyRef") or {})
|
||||
check(ref.get("name") == "keycloak-admin-credentials",
|
||||
f"{RECON_FILE}: {key} must come from the keycloak-admin-credentials "
|
||||
f"Secret by secretKeyRef, got {e!r}")
|
||||
check("value" not in e,
|
||||
f"{RECON_FILE}: {key} carries a literal `value`. A credential in a "
|
||||
f"manifest is a plaintext secret in git")
|
||||
check((env.get("KEYCLOAK_REALM") or {}).get("value") == desired.get("realm"),
|
||||
f"{RECON_FILE}: KEYCLOAK_REALM is "
|
||||
f"{(env.get('KEYCLOAK_REALM') or {}).get('value')!r} but the desired state "
|
||||
f"declares realm {desired.get('realm')!r}. The reconciler would converge the "
|
||||
f"wrong realm, or none")
|
||||
|
||||
check(ctr["securityContext"]["readOnlyRootFilesystem"] is True,
|
||||
f"{RECON_FILE}: readOnlyRootFilesystem must be true")
|
||||
check(ctr["securityContext"]["allowPrivilegeEscalation"] is False,
|
||||
f"{RECON_FILE}: allowPrivilegeEscalation must be false")
|
||||
check(pod["securityContext"].get("runAsNonRoot") is True,
|
||||
f"{RECON_FILE}: runAsNonRoot must be true")
|
||||
check(cj["spec"].get("concurrencyPolicy") == "Forbid",
|
||||
f"{RECON_FILE}: concurrencyPolicy must be Forbid — two concurrent reconcilers "
|
||||
f"would double the write attempts against the IdP during an outage recovery")
|
||||
check(re.fullmatch(r"\S+ \S+ \S+ \S+ \S+", cj["spec"].get("schedule", "")),
|
||||
f"{RECON_FILE}: schedule {cj['spec'].get('schedule')!r} is not a 5-field cron")
|
||||
|
||||
# ===== (6) THE BOUND FLOW IS STRUCTURALLY UNWRITABLE — asserted against the SOURCE
|
||||
#
|
||||
# This component converges the browser flow BLUE/GREEN: it builds the target structure
|
||||
# under a NEW, unused alias, verifies it, then flips the realm's browserFlow in ONE
|
||||
# atomic PUT. It therefore NEEDS flow-create and execution-add endpoints, so unlike the
|
||||
# first design of this check the mere presence of a flow write is no longer the thing to
|
||||
# forbid. What must hold instead is narrower and stronger:
|
||||
#
|
||||
# (a) nothing may be DELETED, ever — the superseded flow is the rollback target;
|
||||
# (b) the single-execution and config-edit endpoints stay off the list — requirements
|
||||
# are set through the flow-scoped endpoint, which the guard in (d) screens;
|
||||
# (c) no required-action write may reach any alias but the one named LITERALLY;
|
||||
# (d) and the guard the allow-list CANNOT express must exist in the source: a
|
||||
# method-aware screen that refuses any flow write targeting the CURRENTLY BOUND
|
||||
# flow's subtree, FAIL-CLOSED until that binding has been read.
|
||||
#
|
||||
# (d) is the load-bearing one. Without it the reconciler could mutate the flow users are
|
||||
# logging in through, and a run dying mid-restructure would leave a HALF-BUILT FLOW BOUND
|
||||
# AS THE LOGIN PATH for Grafana, Gitea, Nextcloud, Headlamp, Semaphore, PiHole, Crafty,
|
||||
# the *arr stack AND kubectl. This repo has already had a total SSO outage from a realm
|
||||
# change (2026-09-06).
|
||||
allow = tuple_literal(script, "_ALLOWED") or ()
|
||||
check(bool(allow),
|
||||
f"{RECON_FILE}: could not parse the _ALLOWED endpoint allow-list out of the "
|
||||
f"script. This check's central assertions about the write surface cannot be made, "
|
||||
f"so it must FAIL rather than pass silently")
|
||||
if allow:
|
||||
check(all(isinstance(e, tuple) and len(e) == 2 for e in allow),
|
||||
f"{RECON_FILE}: _ALLOWED is not a tuple of (method, path) pairs. A path-only "
|
||||
f"allow-list cannot distinguish reading a flow from writing one, which this "
|
||||
f"component's whole safety argument rests on")
|
||||
writes = [(m, p) for m, p in allow if m in ("PUT", "POST", "PATCH", "DELETE")]
|
||||
# (a) nothing is ever deleted.
|
||||
check(not [(m, p) for m, p in allow if m == "DELETE"],
|
||||
f"{RECON_FILE}: the allow-list contains a DELETE method. Nothing this program "
|
||||
f"does requires deleting anything: a superseded flow is left unbound as the "
|
||||
f"ROLLBACK TARGET, and DELETE /admin/realms/<realm> deletes the realm")
|
||||
# (b) the endpoints that would let it edit live objects in place.
|
||||
banned = [(m, p) for m, p in writes
|
||||
if re.fullmatch(r"/admin/realms/\[\^/\]\+/authentication/executions/"
|
||||
r"\[\^/\]\+", p)
|
||||
or re.fullmatch(r"/admin/realms/\[\^/\]\+/authentication/config/\[\^/\]\+",
|
||||
p)
|
||||
or p.endswith("/authentication/register-required-action")]
|
||||
check(not banned,
|
||||
f"{RECON_FILE}: the allow-list permits {banned}. The single-execution update "
|
||||
f"and config-edit endpoints must stay off it: requirements are set through the "
|
||||
f"FLOW-SCOPED endpoint, which carries the flow alias in its path and is "
|
||||
f"therefore screenable against the bound subtree. An execution- or "
|
||||
f"config-id-scoped write is not, so it would be an unscreened hole")
|
||||
# (c) no required-action write may reach any other alias.
|
||||
ra_writes = [(m, p) for m, p in writes if "required-actions" in p]
|
||||
check(all(p.endswith("required-actions/" + ENROL_ACTION) for _, p in ra_writes),
|
||||
f"{RECON_FILE}: the allow-list permits a required-action write to an alias "
|
||||
f"other than {ENROL_ACTION!r}: {ra_writes}. Disabling CONFIGURE_TOTP is a "
|
||||
f"permanent lock-out of every platform-admin, because auth-otp-form in the "
|
||||
f"browser flow still demands a TOTP that could no longer be enrolled")
|
||||
check(not [p for _, p in ra_writes if "[^/]+" in p.split("required-actions/")[-1]],
|
||||
f"{RECON_FILE}: a required-action write path uses a WILDCARD alias. The alias "
|
||||
f"must be named LITERALLY so no other action is writable by construction")
|
||||
# Nothing outside the realm + authentication surface at all.
|
||||
stray = [(m, p) for m, p in allow
|
||||
if "/users" in p or "/groups" in p or "/roles" in p or "/clients" in p]
|
||||
check(not stray,
|
||||
f"{RECON_FILE}: the allow-list reaches {stray}. Users, groups and roles must "
|
||||
f"be unreachable (prod's real user and the media-users / platform-admins / "
|
||||
f"viewers groups must survive every run); clients are the SIBLING "
|
||||
f"reconciler's surface and the two must not overlap")
|
||||
|
||||
# ---- (d) THE PROTECTED-SUBTREE GUARD EXISTS, IS WIRED, AND IS FAIL-CLOSED ----------
|
||||
# Asserted against the SOURCE, because this is the property the blue/green design rests
|
||||
# on and a comment claiming it is worth nothing. Three things must all be true.
|
||||
check(re.search(r"^def _screen_write\(", script, re.M) is not None,
|
||||
f"{RECON_FILE}: the script has no _screen_write() function. That is the guard "
|
||||
f"that refuses any flow write targeting the CURRENTLY BOUND flow's subtree — the "
|
||||
f"one thing the allow-list cannot express, because its write entries are "
|
||||
f"alias-parameterised. Without it this component can mutate the live login path")
|
||||
# It must be CALLED from the single choke point, not merely defined.
|
||||
check(re.search(r"_screen_write\(method, path", script) is not None,
|
||||
f"{RECON_FILE}: _screen_write() is defined but never called from _http() with the "
|
||||
f"request's method and path. A guard that is not on the request path is decoration")
|
||||
# FAIL-CLOSED: the protected sets must start undetermined, and the reset that runs at
|
||||
# the top of every run must put them back there.
|
||||
reset_fn = re.search(r"def _reset_protected\(\):(.*?)(?=\n\S)", script, re.S)
|
||||
check(reset_fn is not None
|
||||
and "_PROTECTED_FLOW_ALIASES = None" in reset_fn.group(1)
|
||||
and "_PROTECTED_EXECUTION_IDS = None" in reset_fn.group(1),
|
||||
f"{RECON_FILE}: _reset_protected() does not set both protected sets to None. They "
|
||||
f"must start each run UNDETERMINED, because while they are, every flow write is "
|
||||
f"refused — that is the fail-closed half of the guard, and it is what stops a "
|
||||
f"build running before the live binding has been read")
|
||||
check(re.search(r"_reset_protected\(\)", script.split("def _selftest")[0]) is not None,
|
||||
f"{RECON_FILE}: _reset_protected() is never called outside the selftest, so the "
|
||||
f"protected sets are not reset per run and a previous run's binding could be "
|
||||
f"trusted for this one")
|
||||
# The create-flow endpoint carries NO alias in its path, so it must be screened on its
|
||||
# BODY or it escapes the guard entirely. (Found by the selftest; recorded here so a
|
||||
# future edit cannot silently undo it.)
|
||||
check("_FLOW_CREATE_PATH" in script,
|
||||
f"{RECON_FILE}: there is no separate screen for POST .../authentication/flows. "
|
||||
f"That endpoint carries no alias in its path — the alias is in the request BODY — "
|
||||
f"so without a body-based screen it bypasses the protected-subtree guard "
|
||||
f"altogether")
|
||||
# And the one function allowed to change the live binding must be distinct from the
|
||||
# policy write, which must refuse to.
|
||||
check("def bind_browser_flow(" in script,
|
||||
f"{RECON_FILE}: there is no bind_browser_flow() function. The atomic flip must be "
|
||||
f"one named, auditable operation, not an incidental field in some other write")
|
||||
check(re.search(r"never rebinds the browser flow", script) is not None,
|
||||
f"{RECON_FILE}: the policy write no longer refuses to change browserFlow. "
|
||||
f"PUT /admin/realms/<realm> DOES read rep.getBrowserFlow(), so the policy pass "
|
||||
f"can rebind the login path by accident — to an UNVERIFIED flow")
|
||||
|
||||
# ======================================== (7) the SIBLING reconciler is NOT widened
|
||||
if (ROOT / SIBLING_FILE).is_file():
|
||||
sib_docs = [d for d in load(SIBLING_FILE) if d]
|
||||
sib_cm = next((d for d in sib_docs if d.get("kind") == "ConfigMap"), None)
|
||||
sib_script = next(iter((sib_cm.get("data") or {}).values()), "") if sib_cm else ""
|
||||
sib_allow = tuple_literal(sib_script, "_ALLOWED") or ()
|
||||
check(bool(sib_allow),
|
||||
f"{SIBLING_FILE}: could not parse its _ALLOWED list, so this check cannot "
|
||||
f"prove the sibling was not widened")
|
||||
if sib_allow:
|
||||
# The sibling's list is PATH-ONLY by design. Its selftest asserts it refuses the
|
||||
# whole-realm endpoint and the /authentication tree; assert the same here so a
|
||||
# future edit that "just adds the realm endpoint there instead" is caught.
|
||||
bad = [p for p in sib_allow
|
||||
if isinstance(p, str)
|
||||
and (re.fullmatch(r"/admin/realms/\[\^/\]\+", p)
|
||||
or "authentication" in p or "/users" in p or "/groups" in p
|
||||
or "/roles" in p)]
|
||||
check(not bad,
|
||||
f"{SIBLING_FILE}: its allow-list was WIDENED to {bad}. That reconciler's "
|
||||
f"stated guarantee — prod's real user and the media-users / "
|
||||
f"platform-admins / viewers groups survive every run — is expressed as "
|
||||
f"endpoints it CANNOT reach, and its own --selftest asserts the refusal of "
|
||||
f"/users, /groups, /roles, /authentication and the whole-realm endpoint. "
|
||||
f"The passkey work is a SIBLING program with its own narrow list precisely "
|
||||
f"so that guarantee stays literally true. Revert the widening")
|
||||
|
||||
# ============================== (8) the reconciler's BEHAVIOUR, proven by its selftest
|
||||
if script:
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
p = Path(td) / "reconcile.py"
|
||||
p.write_text(script)
|
||||
proc = subprocess.run([sys.executable, str(p), "--selftest"],
|
||||
capture_output=True, text=True, timeout=300)
|
||||
tail = (proc.stdout + proc.stderr).strip().splitlines()
|
||||
check(proc.returncode == 0,
|
||||
"the passkey reconciler's --selftest FAILED (compare-then-write / "
|
||||
"idempotence / strict read-modify-write / the RpEntityName gate / "
|
||||
"flow-is-unwritable / the conditional-OTP regression / fail-closed):\n"
|
||||
" " + "\n ".join(tail[-14:]))
|
||||
if proc.returncode == 0:
|
||||
print(" passkey reconciler --selftest: " + (tail[-1] if tail else "(none)"))
|
||||
|
||||
# ---- (8b) THE SELFTEST IS FALSIFIABLE, not decorative ----------------------
|
||||
# A passing suite proves nothing unless it can be shown to FAIL. mutations.py
|
||||
# injects plausible regressions one at a time -- a partial policy PUT, a
|
||||
# rebind of browserFlow, a widened allow-list that makes the flow writable, a
|
||||
# fail-open on an unreadable input -- and requires --selftest to catch every
|
||||
# one. This matters more than usual here: the whole incident class is a
|
||||
# component that REPORTED SUCCESS while the realm was broken, and a green test
|
||||
# that cannot fail is the same lie one level up.
|
||||
mut = DIR_PATH / "mutations.py"
|
||||
if mut.is_file():
|
||||
mproc = subprocess.run([sys.executable, str(mut), str(p)],
|
||||
capture_output=True, text=True, timeout=900)
|
||||
mtail = (mproc.stdout + mproc.stderr).strip().splitlines()
|
||||
check(mproc.returncode == 0,
|
||||
"the passkey reconciler's --selftest is NOT FALSIFIABLE — at least "
|
||||
"one injected regression survived it, so the suite does not prove "
|
||||
"what it claims:\n " + "\n ".join(mtail[-16:]))
|
||||
if mproc.returncode == 0:
|
||||
print(" passkey reconciler mutations: "
|
||||
+ (mtail[-1] if mtail else "(no output)"))
|
||||
else:
|
||||
check(False,
|
||||
"conformance/tests/l0-static/keycloak-passkey-reconciler/mutations.py "
|
||||
"is missing — nothing proves the --selftest can fail, and an "
|
||||
"unfalsifiable proof is the failure mode this component exists to end")
|
||||
|
||||
# ======================================================= (9) this check is load-bearing
|
||||
L0 = (ROOT / "ci" / "l0.sh").read_text() if (ROOT / "ci" / "l0.sh").is_file() else ""
|
||||
GATE = (ROOT / "ci" / "gate.sh").read_text() if (ROOT / "ci" / "gate.sh").is_file() else ""
|
||||
TOUCHED = ((ROOT / "ci" / "touched-requirements.sh").read_text()
|
||||
if (ROOT / "ci" / "touched-requirements.sh").is_file() else "")
|
||||
SELF_YAML = DIR_PATH / "chainsaw-test.yaml"
|
||||
SELF = SELF_YAML.read_text() if SELF_YAML.is_file() else ""
|
||||
check(re.search(r'^\s*conformance\.home\.k3s/scope:\s*["\']?whole-repo', SELF, re.M)
|
||||
is not None,
|
||||
"keycloak-passkey-reconciler/chainsaw-test.yaml does not declare "
|
||||
'conformance.home.k3s/scope: "whole-repo", so the merge gate selects this check '
|
||||
"only when a change happens to attribute one of the requirements it is labelled "
|
||||
"with. This invariant spans platform/, platform-config/, ci/ and clusters/, so a "
|
||||
"change that breaks it need not attribute R-IDP-03/04 — which is how 12 unusable "
|
||||
"prod OIDC clients sat behind an entirely green cluster on 2026-09-06")
|
||||
check("--include-scope" in L0,
|
||||
"ci/l0.sh no longer passes --include-scope to the conformance runner, so the L0 "
|
||||
"stage is a purely requirement-scoped run again and the scope annotation is inert "
|
||||
"decoration — this check would then run only by coincidence")
|
||||
check(re.search(r"^for s in .*\bl0\b.*; do", GATE, re.M)
|
||||
and re.search(r"fail\).*BLOCK=1", GATE),
|
||||
"ci/gate.sh does not set the merge block when the l0 stage fails")
|
||||
check(re.search(r"^\s*platform/keycloak/\*\)", TOUCHED, re.M),
|
||||
"ci/touched-requirements.sh has no platform/keycloak/* arm, so editing the realm "
|
||||
"import or the reconciler would attribute NO requirement and the keycloak tests "
|
||||
"would be filtered out of the merge gate's touched subset")
|
||||
|
||||
# =============================================================================
|
||||
if failures:
|
||||
print("CHECK FAIL: %d assertion(s) failed" % len(failures), file=sys.stderr)
|
||||
for f in failures:
|
||||
print(" FAIL %s" % f, file=sys.stderr)
|
||||
raise SystemExit(1)
|
||||
print("keycloak-passkey-reconciler: all assertions passed "
|
||||
"(desired state derived from the realm import, one substitution pass, write "
|
||||
"surfaces agree, the BOUND flow is unwritable and fail-closed, nothing is "
|
||||
"deletable, sibling allow-list unwidened, selftest green and falsifiable)")
|
||||
PY
|
||||
@@ -0,0 +1,378 @@
|
||||
#!/usr/bin/env python3
|
||||
# =============================================================================
|
||||
# mutations.py -- prove the passkey reconciler's --selftest IS FALSIFIABLE.
|
||||
#
|
||||
# WHY THIS EXISTS
|
||||
# ---------------
|
||||
# A selftest that passes proves nothing on its own: a selftest asserting `True ==
|
||||
# True` also passes. This reconciler exists because a one-shot realm import
|
||||
# REPORTED SUCCESS while the realm was never converged (Done=True / Started=False
|
||||
# with the whole passkey block inert, measured on prod 2026-10-18), and the same
|
||||
# trap applies one level up -- a green test suite that cannot fail is the same lie
|
||||
# in a different place.
|
||||
#
|
||||
# It matters more here than for the sibling client reconciler, because this program
|
||||
# CONSTRUCTS AUTHENTICATION FLOWS and then REBINDS THE REALM'S LOGIN PATH. An
|
||||
# untested safety property on that surface is not a documentation gap, it is an
|
||||
# outage waiting for a schedule tick.
|
||||
#
|
||||
# So this injects a series of PLAUSIBLE regressions into the shipped script, one at
|
||||
# a time, and requires that --selftest FAILS for every one. A mutation that
|
||||
# SURVIVES means the suite does not actually check that property, and the check
|
||||
# fails, naming it.
|
||||
#
|
||||
# THE FIRST SIX ARE THE BLUE/GREEN SAFETY ARGUMENT ITSELF. The design builds a new
|
||||
# flow under an unused alias, verifies it, and flips `browserFlow` in one atomic
|
||||
# write -- never mutating the flow users are logging in through. Each of those
|
||||
# properties is enforced in code; these mutations are the proof that the enforcement
|
||||
# is load-bearing rather than commentary. Several model the obvious "simplification"
|
||||
# someone will propose:
|
||||
# * "just build it under the declared alias, the suffix is ugly"
|
||||
# * "bind it first, then check -- the check is just belt and braces"
|
||||
# * "the protected-alias screen is redundant, the allow-list covers it"
|
||||
# Every one of those would be a total SSO outage waiting to happen.
|
||||
#
|
||||
# ANCHORS ARE AUTHORED AGAINST THE EXTRACTED SCRIPT, not against the YAML manifest.
|
||||
# The sibling mutations.py dedents by four because its anchors are copied out of the
|
||||
# ConfigMap block scalar; these are copied out of the script as `python3 -` sees it,
|
||||
# which is one less transformation to get wrong. check.sh passes the extracted file,
|
||||
# so what is proven falsifiable is byte-for-byte what runs on the cluster either way.
|
||||
#
|
||||
# python3 mutations.py <path-to-extracted-reconciler.py>
|
||||
#
|
||||
# Runtime is ~60s (one subprocess per mutation, each running the full selftest
|
||||
# against an in-process stub Keycloak; no network, no cluster).
|
||||
# =============================================================================
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
# (name, the mistake being modelled, anchor, replacement)
|
||||
MUTATIONS: list[tuple[str, str, str, str]] = [
|
||||
# ---------------------------------------------------------------- blue/green safety
|
||||
("build INTO the currently bound flow",
|
||||
"'the suffix is ugly, just build under the declared alias' -- every "
|
||||
"create-execution and set-requirement call would then land on the flow users are "
|
||||
"logging in through, and a run dying part-way leaves a HALF-BUILT LOGIN PATH for "
|
||||
"every SSO app and kubectl. This is the single regression the whole design exists "
|
||||
"to make impossible",
|
||||
" def mapped(alias):\n"
|
||||
" return \"%s-%s\" % (alias, suffix)",
|
||||
" def mapped(alias):\n return alias"),
|
||||
|
||||
("the protected-subtree screen is never called",
|
||||
"'the allow-list already covers it' -- it does not: the allow-list's write entries "
|
||||
"are alias-parameterised, so only this screen can say 'but not the flow that is "
|
||||
"bound'. Without it the reconciler may write anywhere in the authentication tree",
|
||||
" _screen_write(method, path, body if isinstance(body, dict) else None)",
|
||||
" pass"),
|
||||
|
||||
("the per-run reset makes the protected sets EMPTY, not UNDETERMINED",
|
||||
"reset them to frozenset() so the code reads more simply -- it silently removes "
|
||||
"the fail-closed property, so a build that ran before the live binding had been "
|
||||
"read would be PERMITTED, and could write straight into the live flow. (The "
|
||||
"module-level initial value is deliberately NOT the anchor here: run() resets "
|
||||
"these at the start of every run, so the initialiser is unreachable and mutating "
|
||||
"it would prove nothing.)",
|
||||
" global _PROTECTED_FLOW_ALIASES, _PROTECTED_EXECUTION_IDS\n"
|
||||
" _PROTECTED_FLOW_ALIASES = None\n"
|
||||
" _PROTECTED_EXECUTION_IDS = None",
|
||||
" global _PROTECTED_FLOW_ALIASES, _PROTECTED_EXECUTION_IDS\n"
|
||||
" _PROTECTED_FLOW_ALIASES = frozenset()\n"
|
||||
" _PROTECTED_EXECUTION_IDS = frozenset()"),
|
||||
|
||||
("bind BEFORE verifying the build",
|
||||
"'bind it first, the readback is just belt and braces' -- it is not: the readback "
|
||||
"is what catches a build that was accepted but came out wrong, including the "
|
||||
"conditional-OTP config landing under the wrong key. Binding first makes that the "
|
||||
"live login path before anyone notices",
|
||||
" verify = compare_shape(final_shape, expected, base, realm, token, configs,\n"
|
||||
" \"the newly built flow %r\" % target)",
|
||||
" verify = []"),
|
||||
|
||||
("a failed build is reported as SUCCESS",
|
||||
"swallow the build error and return a converged result -- the realm is then left "
|
||||
"on the old flow (which blue/green makes safe) but the run says it converged, so "
|
||||
"nobody ever learns the passkey flow was never built. That is the one-shot "
|
||||
"import's fail-open, reintroduced. NOTE: simply removing the early return here "
|
||||
"is NOT a regression -- verify-before-bind catches a half-built flow anyway, which "
|
||||
"is defence in depth working as intended -- so the mutation targets the reporting",
|
||||
" problems.append(\n"
|
||||
" \"flow: building %r FAILED (%s). Nothing was bound, so login is \"\n"
|
||||
" \"UNAFFECTED -- the old flow is still live. The partial flow is left in \"\n"
|
||||
" \"place, unbound and harmless\" % (candidate, e))\n"
|
||||
" return 0, problems",
|
||||
" return 1, []"),
|
||||
|
||||
("an existing candidate flow is reused WITHOUT verifying it",
|
||||
"'if the alias exists we already built it' -- a half-built leftover from a crashed "
|
||||
"run lives under exactly that alias, so this binds a partially constructed login "
|
||||
"path",
|
||||
" if cand_shape is not None and not compare_shape(\n"
|
||||
" cand_shape, expected, base, realm, token, configs,\n"
|
||||
" \"the existing flow %r\" % candidate):",
|
||||
" if cand_shape is not None:"),
|
||||
|
||||
# ------------------------------------------------------------------- idempotence
|
||||
("sub-flow aliases are compared as names",
|
||||
"compare the sub-flow alias like any other field -- the flows this program builds "
|
||||
"carry a hash suffix on every alias, so a built flow could NEVER be recognised as "
|
||||
"converged: it would be rebuilt and the login path reflipped every fifteen "
|
||||
"minutes, forever",
|
||||
" out.append((level, ex[\"kind\"], \"\" if is_flow else ex[\"name\"],\n"
|
||||
" ex[\"requirement\"], ex.get(\"configAlias\") or \"\", ex[\"name\"]))",
|
||||
" out.append((level, ex[\"kind\"], ex[\"name\"],\n"
|
||||
" ex[\"requirement\"], ex.get(\"configAlias\") or \"\", ex[\"name\"]))"),
|
||||
|
||||
("the converged short-circuit is removed",
|
||||
"drop the 'bound flow already matches git' early return -- every run rebuilds and "
|
||||
"reflips, which is a permanent write loop against the IdP and churns the realm's "
|
||||
"login path on a schedule",
|
||||
" if not drift:\n"
|
||||
" log(\"flow ok (the bound flow %r matches git: %d execution(s), no \"\n"
|
||||
" \"write)\" % (bound, len(expected)))\n"
|
||||
" return 0, problems",
|
||||
" if False:\n return 0, problems"),
|
||||
|
||||
("the execution priority is dropped from the requirement write",
|
||||
"omit `priority` from the body -- Keycloak's updateExecutions deserialises a "
|
||||
"missing int as 0 and compares it against the stored priority, so this RESHUFFLES "
|
||||
"the execution order while only a requirement change was intended. The passkey "
|
||||
"branch would end up in the wrong place in the tree",
|
||||
" body={\"id\": ex.get(\"id\"), \"requirement\": want_req,\n"
|
||||
" \"priority\": ex.get(\"priority\") or 0})",
|
||||
" body={\"id\": ex.get(\"id\"), \"requirement\": want_req})"),
|
||||
|
||||
("the alias budget is unbounded",
|
||||
"loop forever looking for a free alias -- a persistently failing build then creates "
|
||||
"unbounded flow sprawl in the realm, and nothing here can delete any of it",
|
||||
" for attempt in range(1, MAX_BUILD_ATTEMPTS + 1):",
|
||||
" for attempt in range(1, 10000):"),
|
||||
|
||||
("fail-open when the bound flow cannot be read",
|
||||
"carry on when the live flow's executions are unreadable -- the program then cannot "
|
||||
"know which flows are live, so it cannot prove a build would miss them, and it "
|
||||
"would build with the protected sets still undetermined",
|
||||
" return 0, [\n"
|
||||
" \"flow: the realm binds browserFlow to %r but its executions could not \"\n"
|
||||
" \"be read, so this program cannot establish which flows are live. Flow \"\n"
|
||||
" \"writes stay FAIL-CLOSED for this run -- nothing was built or bound\"\n"
|
||||
" % bound]",
|
||||
" _set_protected(set(), set())\n bound_shape, bound_raw = None, None"),
|
||||
|
||||
# ------------------------------------------------------- the conditional-OTP regression
|
||||
("authenticator config contents no longer compared",
|
||||
"check only that a config is BOUND, not what it says -- this is THE conditional-OTP "
|
||||
"regression: the original bug wrote the role under "
|
||||
"`condition.evaluated.user.role` instead of `condUserRole`, so the role resolved "
|
||||
"to null, the REQUIRED conditional step threw 'Invalid role name submitted: null' "
|
||||
"and every platform-admin got a 403 from every SSO-gated app",
|
||||
" if str(have_cfg.get(key)) != str(want_cfg[key]):",
|
||||
" if False:"),
|
||||
|
||||
("undeclared config keys are ignored",
|
||||
"stop reporting keys the realm holds and git does not -- the stale "
|
||||
"`condition.evaluated.user.role` key is the signature of the original null-role "
|
||||
"bug, and a flow carrying both keys would look correct",
|
||||
" extra = sorted(k for k in have_cfg if k not in want_cfg)",
|
||||
" extra = []"),
|
||||
|
||||
("flow structure divergence detected but not reported",
|
||||
"leave the per-step comparison out -- the realm's login flow could be the old "
|
||||
"pre-passkey shape forever and every run would report the realm converged, which "
|
||||
"is the exact fail-open of the one-shot import this program replaces",
|
||||
" if observed[idx][:4] != expected[idx][:4]:",
|
||||
" if False:"),
|
||||
|
||||
("the browserFlow BINDING is no longer compared",
|
||||
"stop noticing that a correct flow is not the one bound -- a perfect, fully-built "
|
||||
"passkey flow that nothing logs in through changes nothing at all",
|
||||
" if bound_shape is not None:\n"
|
||||
" drift = compare_shape(bound_shape, expected, base, realm, token, configs,",
|
||||
" if False:\n"
|
||||
" drift = compare_shape(bound_shape, expected, base, realm, token, configs,"),
|
||||
|
||||
# ------------------------------------------------------------- realm / policy writes
|
||||
("partial policy PUT instead of read-modify-write",
|
||||
"send only the drifted policy keys -- Keycloak 26.0.6 rebuilds the WHOLE "
|
||||
"passwordless policy from the representation and substitutes its own DEFAULT for "
|
||||
"every key the body omits, so a partial PUT silently resets everything it did not "
|
||||
"mention while returning 204",
|
||||
" body = dict(realm_rep)\n"
|
||||
" for key in drift:\n"
|
||||
" body[key] = wanted[key]",
|
||||
" body = {RP_ENTITY_NAME: realm_rep.get(RP_ENTITY_NAME),\n"
|
||||
" BROWSER_FLOW: realm_rep.get(BROWSER_FLOW)}\n"
|
||||
" for key in drift:\n"
|
||||
" body[key] = wanted[key]"),
|
||||
|
||||
("the policy pass rebinds browserFlow as a side effect",
|
||||
"'while we are here, let the policy write fix the flow binding too' -- the realm "
|
||||
"PUT does read rep.getBrowserFlow(), so this is genuinely in reach, and it would "
|
||||
"rebind the login path to an UNVERIFIED flow from a pass whose stated surface is "
|
||||
"the passkey policy",
|
||||
" if body.get(BROWSER_FLOW) != realm_rep.get(BROWSER_FLOW):\n"
|
||||
" raise RuntimeError(",
|
||||
" body[BROWSER_FLOW] = \"browser\"\n"
|
||||
" if False:\n"
|
||||
" raise RuntimeError("),
|
||||
|
||||
("the RpEntityName gate removed from desired-state validation",
|
||||
"accept a desired state with an empty RpEntityName -- Keycloak 26.0.6 applies the "
|
||||
"passwordless policy ONLY when that field is non-empty, so every realm PUT would "
|
||||
"be accepted with 204 and change nothing: reported success, zero convergence",
|
||||
" if not policy.get(RP_ENTITY_NAME):\n"
|
||||
" raise ValueError(",
|
||||
" if False:\n raise ValueError("),
|
||||
|
||||
("the RpEntityName gate removed from the write path",
|
||||
"drop the second, defence-in-depth guard -- it is unreachable while the "
|
||||
"desired-state validation holds, which is exactly why deleting it would otherwise "
|
||||
"be an invisible regression",
|
||||
" if not body.get(RP_ENTITY_NAME):\n"
|
||||
" raise RuntimeError(",
|
||||
" if False:\n raise RuntimeError("),
|
||||
|
||||
("no readback verification on the policy write",
|
||||
"trust the 204 -- a server that accepts the PUT and does not apply it is reported "
|
||||
"as REPAIRED, which is 'reports success while the realm never converged' again, "
|
||||
"and it is precisely what the RpEntityName gate does when it is missed",
|
||||
" still = policy_drift(after, wanted)",
|
||||
" still = {}"),
|
||||
|
||||
("AAGUID comparison made order-sensitive",
|
||||
"compare the AAGUID allow-list in the order Keycloak returns it -- an allow-list "
|
||||
"is a set membership test, so an order-sensitive comparison rewrites a CONVERGED "
|
||||
"realm on every run forever while every run still reports success",
|
||||
" if stored != sorted(declared_value):\n"
|
||||
" drift[key] = (stored, sorted(declared_value))",
|
||||
" if stored != declared_value:\n"
|
||||
" drift[key] = (stored, declared_value)"),
|
||||
|
||||
("signature-algorithm order treated as insignificant",
|
||||
"sort the signature algorithms before comparing -- it is a PREFERENCE LIST handed "
|
||||
"to the authenticator, so ES256-then-RS256 is not the same declaration as "
|
||||
"RS256-then-ES256, and a reorder in the realm would never be corrected",
|
||||
" if stored != list(declared_value):",
|
||||
" if sorted(stored) != sorted(declared_value):"),
|
||||
|
||||
("unmanaged policy keys forced to a value",
|
||||
"iterate every writable key instead of only the declared ones -- a "
|
||||
"webAuthnPolicyPasswordless field the realm holds and git does not declare would "
|
||||
"be overwritten, turning 'not declared' into 'must equal my default'",
|
||||
" for key in sorted(wanted):\n declared_value = wanted[key]",
|
||||
" for key in POLICY_FIELDS:\n declared_value = wanted.get(key, \"\")"),
|
||||
|
||||
# ------------------------------------------------------------------ required actions
|
||||
("required-action PUT sends only the two drifted fields",
|
||||
"a minimal body looks safer -- but updateRequiredAction REPLACES the provider from "
|
||||
"the representation, so name, priority and config are all blanked: the enrolment "
|
||||
"action keeps working but loses its priority and display name",
|
||||
" body = dict(rep)\n for key in drift:\n body[key] = wanted[key]",
|
||||
" body = {}\n for key in drift:\n body[key] = wanted[key]"),
|
||||
|
||||
("the guarded default required actions are no longer audited",
|
||||
"stop checking that CONFIGURE_TOTP / UPDATE_PASSWORD / delete_credential still "
|
||||
"exist -- their disappearance is the signature of a requiredActions block in the "
|
||||
"realm import, which 26.0.6 treats as REPLACE, and it is a silent permanent "
|
||||
"lock-out of every platform-admin",
|
||||
" missing = [a for a in GUARDED_ACTIONS if a not in aliases]",
|
||||
" missing = []"),
|
||||
|
||||
# ----------------------------------------------------------------- inputs / coupling
|
||||
("the three passes are coupled",
|
||||
"abort the run at the first failing pass -- a policy write that fails for an "
|
||||
"unrelated reason would then suppress the browser-flow pass, which is the most "
|
||||
"safety-critical thing here",
|
||||
" problems.extend(found)\n if found:\n rc = 1",
|
||||
" problems.extend(found)\n if found:\n rc = 1\n"
|
||||
" break"),
|
||||
|
||||
("policy-key validation removed from the desired state",
|
||||
"accept any realm key from the ConfigMap -- an arbitrary realm attribute becomes "
|
||||
"writable through a file edit, so a hand-edited ConfigMap could set sslRequired, "
|
||||
"bruteForceProtected or any other top-level realm field on the live IdP",
|
||||
" if key not in POLICY_FIELDS:",
|
||||
" if False:"),
|
||||
|
||||
("fail-open on an unreadable desired state",
|
||||
"treat a missing ConfigMap as 'nothing declared' -- the run reports success while "
|
||||
"reconciling nothing, which is precisely the fail-open of the realm import this "
|
||||
"whole component exists to replace",
|
||||
" if not isinstance(raw, dict) or not raw:\n"
|
||||
" raise ValueError(\"%s does not hold a non-empty JSON object\" % path)",
|
||||
" if not isinstance(raw, dict) or not raw:\n return {}"),
|
||||
|
||||
("an undeclared authenticatorConfig reference is tolerated",
|
||||
"stop validating that every configAlias a flow references is actually declared -- "
|
||||
"the build then fails at the config-binding step, AFTER flows and executions have "
|
||||
"already been created, leaving leftovers on every run",
|
||||
" if ca and ca not in configs:\n"
|
||||
" raise ValueError(",
|
||||
" if False:\n raise ValueError("),
|
||||
|
||||
("the flow-cycle guard is removed",
|
||||
"drop the `seen` check -- a cyclic flow declaration then recurses until the "
|
||||
"interpreter dies, in a CronJob that retries",
|
||||
" if alias in seen:\n"
|
||||
" raise ValueError(\"flow %r is cyclic in the desired state\" % alias)",
|
||||
" if False:\n"
|
||||
" raise ValueError(\"flow %r is cyclic in the desired state\" % alias)"),
|
||||
]
|
||||
|
||||
|
||||
def main() -> int:
|
||||
if len(sys.argv) != 2:
|
||||
print("usage: mutations.py <extracted-reconciler.py>", file=sys.stderr)
|
||||
return 2
|
||||
src = Path(sys.argv[1]).read_text()
|
||||
|
||||
problems: list[str] = []
|
||||
killed = 0
|
||||
for name, why, old, new in MUTATIONS:
|
||||
# An anchor that no longer matches is a FAILURE, not a skip: it means the code
|
||||
# moved and this mutation silently stopped proving anything.
|
||||
n = src.count(old)
|
||||
if n != 1:
|
||||
problems.append("%s: anchor matches %d times (expected exactly 1) -- the "
|
||||
"code moved and this mutation no longer proves anything. "
|
||||
"Re-anchor it against the current source." % (name, n))
|
||||
continue
|
||||
with tempfile.NamedTemporaryFile("w", suffix=".py", delete=False) as fh:
|
||||
fh.write(src.replace(old, new, 1))
|
||||
path = fh.name
|
||||
try:
|
||||
proc = subprocess.run([sys.executable, path, "--selftest"],
|
||||
capture_output=True, text=True, timeout=300)
|
||||
finally:
|
||||
Path(path).unlink(missing_ok=True)
|
||||
if proc.returncode == 0:
|
||||
problems.append("%s SURVIVED -- --selftest still passed with this regression "
|
||||
"applied, so nothing in the suite actually checks it. The "
|
||||
"mistake it models: %s" % (name, why))
|
||||
continue
|
||||
killed += 1
|
||||
out = proc.stdout + proc.stderr
|
||||
m = re.search(r"SELFTEST FAIL: (\d+) of \d+", out)
|
||||
how = ("%s assertion(s) caught it" % m.group(1) if m
|
||||
else "the selftest aborted (caught, less cleanly)")
|
||||
print(" killed %-54s %s" % (name, how))
|
||||
|
||||
print()
|
||||
if problems:
|
||||
print("MUTATION CHECK FAIL: %d of %d mutation(s) not proven"
|
||||
% (len(problems), len(MUTATIONS)), file=sys.stderr)
|
||||
for p in problems:
|
||||
print(" FAIL %s" % p, file=sys.stderr)
|
||||
return 1
|
||||
print("MUTATION CHECK OK: all %d injected regressions were caught by --selftest "
|
||||
"(the suite is falsifiable, not decorative)" % killed)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Reference in new issue
Block a user