jakman85andClaude Opus 4.8
5fb097d019
Add Headlamp (kubernetes-sigs/headlamp Helm chart) at headlamp.home.keithjackson.dev with native Keycloak OIDC login and PER-USER RBAC: Headlamp forwards each user's id_token to the k3s apiserver, so the existing oidc:* group bindings (platform-admins->cluster-admin, viewers->view) apply per user. The ServiceAccount is deliberately NOT bound to cluster-admin (clusterRoleBinding.create=false), since user requests ride the forwarded token. Key integration nuance: the apiserver is configured oidc-client-id=kubernetes, so it only accepts tokens whose aud contains `kubernetes`. The new `headlamp` Keycloak client carries an oidc-audience-mapper (included.client.audience= kubernetes) so tokens carry aud=[headlamp, kubernetes] -- accepted by both Headlamp's own verifier and the apiserver, leaving the apiserver config untouched. A groups mapper emits the `groups` claim the apiserver maps to RBAC. - platform/headlamp: namespace, HelmRepository/HelmRelease, SOPS OIDC secret (externalSecret envFrom), default-deny NetworkPolicy + Cilium ingress (gateway)/egress (apiserver+Keycloak) policies, README. Wired into platform/kustomization.yaml (platform tier). - platform/headlamp/httproute.yaml wired into platform-config (Gateway API CR, dependsOn platform), modeled on grafana-httproute. - keycloak-realm-import: add `headlamp` confidential client + groups/audience mappers + HEADLAMP_CLIENT_SECRET placeholder. Client also created LIVE via kcadm (realm import no-ops on the existing realm). - keycloak-realm-clients.sops: add headlamp-client-secret (mirrored into headlamp-oidc-credentials clientSecret). Both encrypted to the repo age key. Co-Authored-By: Claude Opus 4.8 <[email protected]>
platform/keycloak
Keycloak — the platform Identity Provider (SSO via OIDC + SAML), onboarded early (Phase 6 IdP).
- Governing ADR: ADR-0014 — IdP: Keycloak
- Spec: docs/15 — Identity & Access
- Serves: R-IDP-01..06
- Owner: WS-SEC
Image / tooling strategy (F-KY1 landmine avoided)
The original stub used the Bitnami Keycloak chart + bundled Bitnami PostgreSQL. Bitnami's free Docker Hub images were purged (DECISIONS-LOG F-KY1) → that path ImagePullBackOffs. Replaced with:
- Keycloak: the official Keycloak Operator (
quay.io/keycloak/keycloak-operator+quay.io/keycloak/keycloak, v26.0.6) driving aKeycloakCR + aKeycloakRealmImportCR for config-as-code (ADR-0014 permits Operator OR Helm; Operator gives clean CR-driven realm reconcile). - Database: CloudNativePG (
ghcr.io/cloudnative-pg, see../cnpg) — aClusteron the 3-replicalonghornStorageClass (DR-critical, R-IDP-05), with declarative backup hooks.
quay.io + ghcr.io are unaffected by the Bitnami purge. Operator manifests are pinned upstream release files pulled as remote kustomize bases (the kubevirt-stub pattern); Renovate maintains the version.
Layout (controllers / configs split — F-MON1)
| File | Applied by | What |
|---|---|---|
kustomization.yaml (namespace, operator CRDs+Deployment, admin secret, dashboard) |
platform | controllers |
keycloak-instance.yaml (Keycloak CR, ${KEYCLOAK_REPLICAS}) |
platform-config | config |
postgres-cluster.yaml (CNPG Cluster on longhorn) |
platform-config | config |
keycloak-realm-import.yaml (KeycloakRealmImport: realm/groups/clients/MFA) |
platform-config | config |
httproute.yaml (keycloak.home.keithjackson.dev) |
platform-config | config |
servicemonitor.yaml (Prometheus scrape) |
platform-config | config |
Realm config (groups platform-admins/media-users/viewers, OIDC clients grafana+oauth2-proxy,
brute-force ON, and a conditional MFA browser flow that REQUIRES OTP for platform-admins, R-IDP-04)
reconciles purely from git — no console clicking (R-IDP-03).
Admin credential + break-glass (R-IDP-04 / R-IDP-05)
The provisioned keycloak-admin-credentials.sops.yaml (keys admin-user/admin-password) is injected
as the temporary bootstrap/break-glass admin via KC_BOOTSTRAP_ADMIN_* env (podTemplate escape
hatch) — this consumes the secret as provisioned, without re-keying it (the operator's native
bootstrapAdmin wants username/password keys, which would require editing the encrypted file).
The Keycloak bootstrap admin is the documented break-glass path so an IdP/SSO outage does not lock the
operator out (ADR-0014); Grafana likewise keeps its local admin login until SSO is proven.
Operator hand-offs (fill before / during deploy)
keycloak-realm-clients.sops.yaml(keycloak ns) — from.sops.yaml.example. Two client secrets (grafana-client-secret,oauth2-proxy-client-secret),openssl rand -hex 32each. The realm import injects them into the Keycloak clients viaspec.placeholders. Add tokustomization.yamlonce filled. The realm import stays PENDING until this exists.../oauth2-proxy/oauth2-proxy-credentials.sops.yaml(oauth2-proxy ns) —client-id: oauth2-proxy,client-secret= the SAMEoauth2-proxy-client-secretabove,cookie-secret=openssl rand -base64 32.../monitoring/grafana-oidc-credentials.sops.yaml(monitoring ns) —client-secret= the SAMEgrafana-client-secretabove.
(Cross-namespace Secrets can't be shared, so each client secret is stored twice — once for the realm
import in keycloak, once for the consuming app.) See the two .sops.yaml.example templates for the
exact keys + steps. No plaintext secret is ever committed.
Exceptions that keep native auth (do not force SSO) — ADR-0014
Plex/Seerr (Plex auth), Minecraft (Mojang/Microsoft), UniFi (native).