The Keycloak CR has no spec.image, so the operator's RELATED_IMAGE_KEYCLOAK picks the
server image; the three keycloak-k8s-resources release URLs in
platform/keycloak/kustomization.yaml are therefore THE Keycloak version, and Renovate
could not see them (no `# renovate:` marker). They now carry markers under one depName
(one branch, groupName keycloak-operator), with a packageRule: automerge false, labels
needs-review + blast-radius; majors still wait on the dashboard.
The untagged quay.io/keycloak/keycloak in catalog/keycloak.yaml renders
apps/wave-1/keycloak.yaml, which no kustomization lists. It is pinned to the operator's
26.0.6 as documentation, re-rendered, and Renovate is disabled for that image so it
cannot raise PRs against a file nothing deploys. The render is kept (not deleted): it is
the fixture for apps/wave-1/tests/app-slis.test.yaml's keycloak SLI rule tests.
README: an "Upgrading Keycloak" section — an operator bump must be checked against
passkey-reconciler.yaml's 26.0.6 Admin API assumptions.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The four backup-credential SOPS Secrets are SHARED BASE manifests: platform/ and apps/ are
rendered by the same Flux tiers on both clusters, and the ONLY per-environment difference in
this repo is cluster-vars substitution (R-ENV-02). So writing the R2 token into all four
(0a6a862) did not "give prod credentials" — it REPLACED DEV'S. Merging that as-is would have
handed dev's Velero and all three dev CNPG Clusters an R2 key id to sign MinIO requests with:
403 on every WAL segment, and archive_mode=on turns that into a disk-full clock on the pgdata
PVC, not a degraded backup. Dev's PROVEN DR restore (R-STATE-03) is the thing that would have
broken.
This commit splits the material. Each credential is now TWO Secret objects, both applied to
BOTH clusters, differing only in name suffix:
dev -> velero-cloud-credentials keycloak/nextcloud/semaphore-db-backup-credentials
prod -> velero-cloud-credentials-prod ...-db-backup-credentials-prod
The consumer picks one by name; that wiring is the NEXT commit, so this one changes no
rendered reference and is a no-op for both live clusters.
WHY DEV KEEPS THE UNSUFFIXED NAME. Dev is live. Renaming its Secret would change
`s3Credentials.accessKeyId.name` on three running CNPG Clusters — which projects into the
instance pods' environment and therefore ROLLS keycloak-db, nextcloud-db and semaphore-db —
and would roll the velero Deployment, all to rename an object whose contents do not change.
This repo already makes exactly this call for dev's live identifiers (EXTERNALDNS_TXT_OWNER_ID
"default", VELERO_BACKUP_PREFIX ""): the live cluster keeps the name it is running, and the
NEW environment takes the new one. The asymmetry is deliberate and documented at each
kustomization.
WHERE THE MATERIAL CAME FROM. Neither side is invented:
- the four unsuffixed files are restored byte-for-byte from `main` (the MinIO material as
it stood before 0a6a862), so dev's Secrets are IDENTICAL to what dev is running;
- the four `-prod` files carry the R2 token from 0a6a862 (already on this branch as
9739e9b), re-encrypted only because metadata.name changed.
RE-ENCRYPTION, AND WHY A COPY+SED IS NOT ENOUGH. `.sops.yaml` sets
`encrypted_regex: ^(data|stringData)$`, so metadata.name stays cleartext — but SOPS MACs the
WHOLE document (there is no `mac_only_encrypted` here), so editing that cleartext line
invalidates the MAC and `sops -d` then refuses the file. The `-prod` files were therefore
produced inside the dev toolbox (infra/devcontainer, which has sops 3.9.4 and the age key
mounted) by decrypt -> rename -> re-encrypt, with the plaintext never leaving a container
tmpfile. Note for the next person: `sops -e --output NEW OLD` matches creation rules against
the INPUT path, so encrypting a tmpfile into a repo path needs `--filename-override NEW` or
it dies with "no matching creation rules found".
Verified: all eight files `sops -d` cleanly; the four dev files hash to dev's material and the
four prod files to R2's, and the two sets differ. `.sops.yaml` needed no new rule — the
existing `(platform|apps|catalog)/.*(secret|sops)\.ya?ml$` rule already matches
`*-prod.sops.yaml`, so the new files encrypt to the same age recipient.
apps/wave-0/kustomization.yaml is GENERATED, so render.py learns the `-prod` sidecar rather
than the golden being hand-edited: `<db>-backup-credentials-prod.sops.yaml` now places into
the DB's wave alongside its unsuffixed twin (both must exist before the Cluster that names
one), and the selftest asserts it.
Co-Authored-By: Claude Opus 5 <[email protected]>
Generate the 3 OIDC client secrets (SOPS-encrypted, minimal manifests) so the realm
import can inject the grafana + oauth2-proxy client secrets and both consumers match:
- platform/keycloak/keycloak-realm-clients.sops.yaml (grafana + oauth2-proxy client secrets)
- platform/oauth2-proxy/oauth2-proxy-credentials.sops.yaml (client-id/secret/cookie)
- platform/monitoring/grafana-oidc-credentials.sops.yaml (grafana client secret)
Wire each into its component kustomization and re-enable oauth2-proxy in platform +
platform-config (reverses the temp deferral now that its secret exists).
Values generated by PM per operator instruction (random; recorded out-of-repo at
~/.config/home-k3s/keycloak-sso-secrets.txt for later rotation). Encrypted via sops in the
dev container to the age recipient in .sops.yaml; plaintext never committed.
Serves: R-IDP-01, R-IDP-02, R-IDP-04, R-SEC-01.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
renovate-bot
jakman85andClaude Opus 5.5