feat(auth): Keycloak OIDC login + RBAC for kubectl (cert kubeconfig stays break-glass)

Lets users kubectl into the cluster via Keycloak SSO, mapping realm groups to
cluster RBAC. The cert-based admin kubeconfig is untouched and remains a fully
independent break-glass path (works even if Keycloak/OIDC is down).

- keycloak-realm-import.yaml: new `kubernetes` PUBLIC client + PKCE (S256) — no
  client secret in any kubeconfig; kubelogin runs auth-code+PKCE to localhost.
  groups mapper + email/profile/roles scopes.
- k8s-oidc-rbac.yaml (NEW, platform-config): ClusterRoleBindings
  oidc:platform-admins -> cluster-admin, oidc:viewers -> view.
- k3s_server role: kube-apiserver-arg OIDC flags (issuer=homelab realm,
  client-id=kubernetes, username-claim=email + oidc: prefix, groups-claim=groups
  + oidc: prefix). No oidc-ca-file (nodes reach the issuer via the gateway VIP
  and trust the public LE cert). Applied via the rolling Restart k3s handler.

Keycloak/RBAC land via Flux now; the API-server flags take effect on the next
`ansible-playbook playbooks/k3s.yml` rolling restart (deferred until cluster load
from the VM-import incident settles).

Co-Authored-By: Claude Opus 4.8 <[email protected]>

infra/ansible/roles/k3s_server/tasks/main.yml

@@ -30,7 +30,7 @@
 # its containerd config.toml (so nonroot-devices takes effect). Apply the restart
 # ROLLING, one node at a time (kube-vip keeps the API reachable; etcd quorum holds
 # at 2/3). See playbooks note: run with `serial: 1`.
-- name: Write /etc/rancher/k3s/config.yaml (tls-san + nonroot-devices)
+- name: Write /etc/rancher/k3s/config.yaml (tls-san + nonroot-devices + oidc)
   ansible.builtin.copy:
     dest: /etc/rancher/k3s/config.yaml
     owner: root
@@ -62,6 +62,22 @@
       # pod stays non-root, only the device node it already mounts follows its UID/GID.
       # Preserves the longhorn-vm RWX + Block + migratable disk contract unchanged.
       nonroot-devices: true
+
+      # OIDC (Keycloak SSO for kubectl). The API server trusts ID tokens from the homelab realm;
+      # the `kubernetes` PUBLIC client (PKCE) + kubelogin mint them, and RBAC binds the `groups`
+      # claim (platform/keycloak/k8s-oidc-rbac.yaml: oidc:platform-admins->cluster-admin,
+      # oidc:viewers->view). Nodes resolve keycloak.home.keithjackson.dev to the Cilium gateway VIP
+      # (192.168.1.53) and trust its public LE-prod cert, so NO oidc-ca-file is needed. The `oidc:`
+      # prefixes namespace federated identities away from cert / service-account subjects.
+      # ADDITIVE + break-glass-safe: the cert-based admin kubeconfig keeps full access with or
+      # without Keycloak, so a bad realm/OIDC state can never lock you out of the cluster.
+      kube-apiserver-arg:
+        - "oidc-issuer-url=https://keycloak.home.keithjackson.dev/realms/homelab"
+        - "oidc-client-id=kubernetes"
+        - "oidc-username-claim=email"
+        - "oidc-username-prefix=oidc:"
+        - "oidc-groups-claim=groups"
+        - "oidc-groups-prefix=oidc:"
   notify: Restart k3s
 
 # --- primary: found the cluster --------------------------------------------

platform-config/kustomization.yaml

@@ -54,6 +54,7 @@ resources:
   - ../platform/keycloak/postgres-cluster.yaml
   - ../platform/keycloak/keycloak-instance.yaml
   - ../platform/keycloak/keycloak-realm-import.yaml
+  - ../platform/keycloak/k8s-oidc-rbac.yaml   # maps OIDC groups -> cluster roles (kubectl SSO RBAC)
   - ../platform/keycloak/httproute.yaml
   - ../platform/keycloak/servicemonitor.yaml
   # oauth2-proxy: forward-auth sign-in/callback ingress + ServiceMonitor (Gateway API / monitoring CRs).

platform/keycloak/k8s-oidc-rbac.yaml

@@ -0,0 +1,46 @@
+---
+# Kubernetes RBAC for Keycloak-federated (OIDC) users. The k3s API server is configured
+# (infra/ansible roles/k3s_server) with:
+#   oidc-issuer-url=https://keycloak.home.keithjackson.dev/realms/homelab
+#   oidc-client-id=kubernetes   oidc-username-claim=email   oidc-username-prefix=oidc:
+#   oidc-groups-claim=groups    oidc-groups-prefix=oidc:
+# so a Keycloak group `platform-admins` appears to RBAC as the group `oidc:platform-admins`, and a
+# user with email [email protected] as `oidc:[email protected]`. These bindings map the Keycloak group
+# claim to cluster roles. Group-based (not user-based) so access follows Keycloak group membership
+# with no per-user k8s change. Applied by platform-config (dependsOn platform).
+#
+# BREAK-GLASS: the cert-based admin in the bootstrap kubeconfig is unaffected by any of this and
+# keeps full access even if Keycloak/OIDC is down — do not remove it.
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: oidc-platform-admins
+  labels:
+    app.kubernetes.io/part-of: keycloak
+    homelab.keithjackson.dev/oidc-rbac: "true"
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: cluster-admin
+subjects:
+  - apiGroup: rbac.authorization.k8s.io
+    kind: Group
+    name: "oidc:platform-admins"
+---
+# Read-only tier: the Keycloak `viewers` group gets the built-in `view` ClusterRole (read most
+# resources cluster-wide, excluding Secrets and RBAC escalation paths).
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: oidc-viewers
+  labels:
+    app.kubernetes.io/part-of: keycloak
+    homelab.keithjackson.dev/oidc-rbac: "true"
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: view
+subjects:
+  - apiGroup: rbac.authorization.k8s.io
+    kind: Group
+    name: "oidc:viewers"

platform/keycloak/keycloak-realm-import.yaml

@@ -103,6 +103,48 @@ spec:
               "access.token.claim": "true"
               "userinfo.token.claim": "true"
               "claim.name": groups
+      # kubernetes — kubectl OIDC login via kubelogin (int128/kubelogin `oidc-login`). PUBLIC client +
+      # PKCE (S256) so NO client secret lives in any user's kubeconfig; kubelogin runs the auth-code
+      # +PKCE flow against a localhost callback and caches the ID token. The k3s API server validates
+      # that ID token (kube-apiserver oidc-issuer-url=this realm, oidc-client-id=kubernetes,
+      # oidc-username-claim=email, oidc-groups-claim=groups); RBAC binds the `groups` claim — see
+      # k8s-oidc-rbac.yaml (oidc:platform-admins -> cluster-admin, oidc:viewers -> view). The cert
+      # kubeconfig remains the break-glass admin, independent of Keycloak/OIDC availability.
+      - clientId: kubernetes
+        name: Kubernetes (kubectl)
+        protocol: openid-connect
+        enabled: true
+        publicClient: true
+        standardFlowEnabled: true
+        directAccessGrantsEnabled: false
+        attributes:
+          "pkce.code.challenge.method": "S256"
+        redirectUris:
+          - http://localhost:8000
+          - http://localhost:8000/
+          - http://localhost:18000
+          - http://localhost:18000/
+          - http://127.0.0.1:8000
+          - http://127.0.0.1:8000/
+          - http://127.0.0.1:18000
+          - http://127.0.0.1:18000/
+        webOrigins:
+          - http://localhost
+          - http://127.0.0.1
+        defaultClientScopes:
+          - email
+          - profile
+          - roles
+        protocolMappers:
+          - name: groups
+            protocol: openid-connect
+            protocolMapper: oidc-group-membership-mapper
+            config:
+              "full.path": "false"
+              "id.token.claim": "true"
+              "access.token.claim": "true"
+              "userinfo.token.claim": "true"
+              "claim.name": groups
     # --- MFA for platform-admins (R-IDP-04): a browser flow whose OTP step is CONDITIONAL on the
     #     platform-admin role. Non-admins log in without OTP; platform-admins must enroll/present TOTP.
     #     Set as the realm browserFlow below. VERIFY at bring-up (see runbook): a malformed flow fails