Files
home-network-k3s-impl/decisions
jakman85andClaude Opus 5 f5931944fa feat(keycloak): reconcile the passkey realm config via GitOps, the browser flow blue/green
PR #72 declared the WebAuthn passwordless policy and a restructured browser-mfa
flow in the realm import. That declaration could not take effect: a
KeycloakRealmImport is a ONE-SHOT, and prod's reads Done=True / Started=False, so
every realm-level field and flow declared after the first import is inert -- in
git, applied by Flux, reported Ready, reaching the realm never. Merging it changed
nothing about how anyone logs in, and nothing anywhere said so. Same failure class
as the 2026-09-06 client-secret incident, third field over.

A SIBLING reconciler converges all three declared things every 15 minutes. It is a
sibling, not a second pass of client-secret-reconciler.yaml, so that reconciler's
allow-list -- and its guarantee that prod's real user and the media-users /
platform-admins / viewers groups are structurally unreachable -- stays literally
true. Its selftest still passes unchanged.

  pass 1  webAuthnPolicyPasswordless* -- strict read-modify-write of
          GET/PUT /admin/realms/homelab, readback-verified.
  pass 2  webauthn-register-passwordless -- strict read-modify-write of ONE
          literally-named alias; enabled, defaultAction false so enrolment stays
          opt-in.
  pass 3  the browser flow -- BLUE/GREEN.

PASS 3 NEVER MUTATES THE FLOW USERS ARE LOGGING IN THROUGH. Converging a
five-flow, ten-execution tree needs create-subflow, create-execution,
set-requirement and bind-config as SEPARATE untransacted calls; doing that to the
BOUND flow means a run dying between them leaves a half-built login path for every
SSO app and kubectl, and this repo has had a total SSO outage from a realm change.
So instead:

  1. BUILD under a NEW, unused alias (declared alias + a hash of the desired
     shape, applied to EVERY alias in the tree -- reusing browser-mfa-forms would
     mean writing INTO the bound tree). A failure leaves an orphaned, unbound flow
     and zero user-visible effect.
  2. VERIFY by readback -- levels, ordering, authenticators, requirements, and the
     conditional-OTP config CONTENTS -- before anything is bound, so the
     regression that once 403'd every platform-admin is caught BEFORE it bites.
  3. BIND with ONE atomic PUT setting browserFlow.
  4. ROLLBACK is one write; no DELETE exists anywhere in the allow-list, so the old
     flow survives as the rollback target.

The widened allow-list is made safe structurally. It cannot say "but not the bound
flow" -- its write entries are alias-parameterised -- so a second guard screens
every write against the live browserFlow's WHOLE subtree (sub-flow aliases and
execution ids) before a socket opens, FAIL-CLOSED until that binding has been
read. The create-flow endpoint carries no alias in its path, so it is screened on
its request BODY; the selftest caught that it otherwise escaped the guard.

Idempotence is load-bearing: a rebuild-and-reflip every 15 minutes would churn the
login path on a schedule. The bound flow is compared STRUCTURALLY and a match
produces zero writes and no new flow. Sub-flow and config ALIASES are deliberately
not compared -- a built flow carries a hash suffix, so comparing labels would make
it permanently unrecognisable as converged.

Single source of truth: the desired state is GENERATED from the realm import by
ci/render-keycloak-passkey-policy.py and gated with --check, so the RP ID -- baked
into every credential at enrolment and never re-keyable -- is declared once. The
pod keeps automountServiceAccountToken: false and reads a mounted ConfigMap, so no
rbac.yaml is added anywhere.

Read from Keycloak 26.0.6 source before being relied on: updateRealm
(DefaultExportImportManager 725-894) makes no getUsers/getClients/getGroups/
getRoles/getRequiredActions/getAuthenticationFlows call; it DOES read
getBrowserFlow(), which is what makes the atomic bind possible and why the policy
write pins it; the passwordless policy is applied all-or-nothing only when
RpEntityName is non-empty, with defaults substituted for omitted keys; and
updateExecutions compares rep.getPriority(), so a body omitting `priority` would
silently reshuffle the whole execution order.

Gate coverage: a new whole-repo L0 check extracts the script and runs its
--selftest (216 assertions, including a build killed at four different points each
asserting the old flow is still bound and the bound subtree was never written),
plus mutations.py (30 injected regressions, all caught) proving the selftest can
fail. An earlier iteration of this work only AUDITED the flow and left the
restructure manual; decisions/2026-10-18-keycloak-passkey-reconciler.md records
that design and why it was reconsidered.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-10-29 21:22:08 -05:00
..

decisions/ — the handoff log, one file per entry

This directory is the decision log. ../DECISIONS-LOG.md is the frozen archive of everything written before 2026-09-29 and is never edited again (see "The archive is frozen" below).

Why this exists

DECISIONS-LOG.md was append-only in a single file, and every agent appended to its tail. Two branches that both append hit the same last hunk, so git cannot auto-merge them: the second one to be reviewed conflicts, and so does the third, and the fourth. Measured on 2026-09-29: four of the five open PRs were unmergeable, and DECISIONS-LOG.md was the only conflicting file in all four:

#22 fix/crafty-oauth2-proxy-signin      CONFLICT (content): DECISIONS-LOG.md
#20 feat/conformance-job-template       CONFLICT (content): DECISIONS-LOG.md
#19 feat/conformance-trigger-kubeconfig CONFLICT (content): DECISIONS-LOG.md
#18 feat/conformance-runner-image       CONFLICT (content): DECISIONS-LOG.md
#17 feat/ci-is-the-approver             clean  (it touched no log entry)

Nothing was wrong with any of those changes. The log's storage format serialised work that was otherwise parallel, and it got worse with every concurrent branch: N branches produce N-1 rebases, each of which invalidates any gate run that preceded it.

The fix is the obvious one for concurrent appenders: stop sharing a file. Each handoff is its own file. Two agents writing handoffs touch disjoint paths, so there is nothing for git to merge and nothing to conflict. This scales to any number of parallel branches with zero coordination — which is the point, since the agents are deliberately isolated from each other in separate worktrees (required by CLAUDE.md §0) and cannot coordinate.

There is deliberately no index file and no counter. An index would be a shared mutable file — exactly the thing that just failed — and a sequence number would have two agents racing for 0043. The directory listing is the index.

Writing an entry

One file per handoff, named:

decisions/YYYY-MM-DD-<slug>.md

<slug> is lowercase [a-z0-9-]. Use your branch name's last segment — it is already unique per unit of work, which is precisely the property the filename needs. A branch fix/crafty-oauth2-proxy-signin handed off on 2026-09-29 writes:

decisions/2026-09-29-crafty-oauth2-proxy-signin.md

The file is YAML frontmatter followed by three required sections:

---
date: 2026-09-29
branch: fix/crafty-oauth2-proxy-signin
title: The Keycloak gate works — Crafty's own login was the wall
---

## What changed

Prose. What you actually did, and why.

## What I verified

The measurements, with their numbers. Quote the field you read, not the prose you
believed. Name the gate result and its exit code.

## What I did NOT verify

Everything else. An honest "UNVERIFIED" is worth more than an optimistic "green" —
this repo has been burned by the difference more than once.

All three headings are required and enforced (ci/decisions-log.sh). They are CLAUDE.md §7's contract; making them a checked structure is what stops an entry degrading into "done ✅".

Reading the log

The whole log, oldest first:

ls decisions/[0-9]*.md | sort

The current PM handoff — the newest entry. Ties within a day are broken by what landed on main last, which is the honest answer, so ask git rather than the filename:

git log --diff-filter=A --format='%H' --name-only -- 'decisions/[0-9]*.md' \
  | grep '^decisions/' | head -1

For a quick skim of who said what:

grep -H '^title:' decisions/[0-9]*.md | sort

The archive is frozen

DECISIONS-LOG.md is read-only history. Its SHA-256 is pinned in decisions/ARCHIVE.sha256 and ci/decisions-log.sh fails the merge gate if the file changes. That is not ceremony: without it, the next agent to read runbooks/ or an old CLAUDE.md habit appends to the tail again and re-creates the conflict for everybody else, silently, one PR at a time.

If the archive genuinely must change (a leaked credential to redact, say), that is a deliberate, reviewed act: change it, run sha256sum DECISIONS-LOG.md > decisions/ARCHIVE.sha256, and say in your handoff entry why. Re-pinning is visible in the diff. Appending is not.