test(conformance): gate the URL desired state, and prove the selftest can fail

L0 (212 assertions, was 175):
  * runs ci/render-keycloak-client-urls.py --check, so client-urls.yaml must be exactly
    what the realm import renders. A hand edit that makes the two disagree is
    unmergeable -- this is what keeps the realm import the single source of truth for
    hostnames rather than just the place they were first typed.
  * asserts client-urls.yaml and the realm import are rendered by the SAME Flux
    Kustomization. postBuild.substituteFrom is per-Kustomization, so a split would let
    one ${BASE_DOMAIN} resolve to two different values and the realm would converge on
    a hostname the HTTPRoutes do not serve.
  * asserts the desired state uses no substitution token the realm definition lacks,
    names only clients the import declares, and declares no field outside the
    five-field write surface; that the CronJob mounts it readOnly at the path the
    script defaults to; and that the script, renderer and check agree on which fields
    are writable.
  * NEW mutations.py: injects 12 plausible regressions into the EXTRACTED script and
    requires --selftest to catch every one. A passing suite proves nothing if it cannot
    fail, and "reported success while the realm was broken" is precisely what this
    component exists to end -- so the same standard applies to its own test. ~20s.

L2: the audit Job now mounts keycloak-client-urls and asserts
KEYCLOAK_CLIENT_URLS_AUDIT_OK. Without that it would pass on a realm still holding
pre-migration redirect URIs -- the exact state prod was measured in.

Alerts: the existing Failing/Stale pair is EXTENDED to name redirect-URI drift, not
duplicated. One CronJob gets one pair; a second pair on the same job_name regex would
fire twice for one cause. No new critical, no new permanently-firing rule.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
jakman85andClaude Opus 5 committed 2026-09-12 21:26:12 -05:00
1 parent 23ea72d01e
commit 6202d92c27
4 files changed
+465 -20

No files matched your search

@@ -1,6 +1,6 @@
# =============================================================================
# L2 BEHAVIORAL — every OIDC client's STORED secret equals the one git declares
# (R-IDP-02, R-IDP-04) [RUNNABLE]
# L2 BEHAVIORAL — every OIDC client's STORED secret AND redirect URIs equal what git
# declares (R-IDP-02, R-IDP-04) [RUNNABLE]
# Spec: docs/15 identity-and-access; ADR-0014.
# Impl: platform/keycloak/client-secret-reconciler.yaml (the CronJob + its script).
#
@@ -27,14 +27,26 @@
# its endpoint allow-list makes users/groups/roles unreachable),
# -> THEN every declared client's stored secret matches git, and none holds an
# unresolved import placeholder. Emits KEYCLOAK_CLIENT_SECRETS_AUDIT_OK.
# -> AND every declared client URL — redirectUris, webOrigins, rootUrl, baseUrl,
# adminUrl — matches git. Emits KEYCLOAK_CLIENT_URLS_AUDIT_OK.
#
# THE URL HALF EXISTS FOR THE SAME REASON, ONE FIELD OVER. The realm import is templated
# per environment, so moving prod's BASE_DOMAIN re-renders every declared redirect URI in
# git — and reaches the realm never, because a completed import does not re-run. Measured
# on prod: stored redirect URIs still on the PRE-migration hostname while git declared the
# new one. Every OIDC login would fail with a redirect_uri mismatch, and once again every
# signal would be green: the CR reads Done=True, the pods are healthy, and the
# secret-only version of this very test would have PASSED. Asserting only the secret
# sentinel is what would have let that ship.
#
# WHY IT RUNS THE SHIPPED SCRIPT rather than re-implementing the comparison: a detector
# that duplicates the healer's logic drifts from it. This audits with the same code, in
# the same mode the L0 gate selftests, so a passing audit is evidence about the healer
# too. It runs as a Job IN the keycloak namespace (not in the ephemeral chainsaw
# namespace) because the desired state is the mounted keycloak-realm-clients Secret and
# the admin credential is keycloak-admin-credentials — both namespace-local, which is
# also why neither this Job nor the CronJob needs any Kubernetes API access.
# namespace) because the desired state is the mounted keycloak-realm-clients Secret plus
# the keycloak-client-urls ConfigMap, and the admin credential is
# keycloak-admin-credentials — all namespace-local, which is also why neither this Job nor
# the CronJob needs any Kubernetes API access.
#
# chainsaw v0.2.12: Job + status assert; the leftover-Job delete keeps a re-run from
# colliding with an immutable Job left by a previous failure.
@@ -149,6 +161,12 @@ spec:
- name: client-secrets
mountPath: /etc/keycloak-realm-clients
readOnly: true
# The URL pass's desired state. Without this mount the audit
# fails closed on pass 2 and this test would report a false
# failure rather than auditing the realm's redirect URIs.
- name: client-urls
mountPath: /etc/keycloak-client-urls
readOnly: true
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
@@ -170,6 +188,10 @@ spec:
secret:
secretName: keycloak-realm-clients
defaultMode: 0400
- name: client-urls
configMap:
name: keycloak-client-urls
defaultMode: 0444
- assert:
resource:
apiVersion: batch/v1
@@ -192,6 +214,14 @@ spec:
|| { echo "FAIL: the audit did not report every client secret matching git"; exit 1; }
echo "$out" | grep -q 'UNRESOLVED IMPORT PLACEHOLDER' \
&& { echo "FAIL: a client still holds the literal import placeholder"; exit 1; }
# The URL half. A realm still holding PRE-migration redirect URIs is the
# state prod was measured in: every OIDC login fails with a redirect_uri
# mismatch while the KeycloakRealmImport CR still reads Done=True. Asserting
# only the secret sentinel would pass on exactly that realm.
echo "$out" | grep -q 'KEYCLOAK_CLIENT_URLS_AUDIT_OK' \
|| { echo "FAIL: the audit did not report every client URL matching git"; exit 1; }
echo "$out" | grep -q 'DRIFT -- realm has' \
&& { echo "FAIL: a client URL still differs from the declared one"; exit 1; }
echo KEYCLOAK_CLIENT_SECRET_DRIFT_NONE_OK
check:
(contains($stdout, 'KEYCLOAK_CLIENT_SECRET_DRIFT_NONE_OK')): true