From 5fb097d019d402220a3aa1d39c6f203742dad297 Mon Sep 17 00:00:00 2001 From: Keith Jackson Date: Fri, 28 Aug 2026 11:43:48 -0500 Subject: [PATCH] platform: deploy Headlamp dashboard with Keycloak OIDC + per-user RBAC Add Headlamp (kubernetes-sigs/headlamp Helm chart) at headlamp.home.keithjackson.dev with native Keycloak OIDC login and PER-USER RBAC: Headlamp forwards each user's id_token to the k3s apiserver, so the existing oidc:* group bindings (platform-admins->cluster-admin, viewers->view) apply per user. The ServiceAccount is deliberately NOT bound to cluster-admin (clusterRoleBinding.create=false), since user requests ride the forwarded token. Key integration nuance: the apiserver is configured oidc-client-id=kubernetes, so it only accepts tokens whose aud contains `kubernetes`. The new `headlamp` Keycloak client carries an oidc-audience-mapper (included.client.audience= kubernetes) so tokens carry aud=[headlamp, kubernetes] -- accepted by both Headlamp's own verifier and the apiserver, leaving the apiserver config untouched. A groups mapper emits the `groups` claim the apiserver maps to RBAC. - platform/headlamp: namespace, HelmRepository/HelmRelease, SOPS OIDC secret (externalSecret envFrom), default-deny NetworkPolicy + Cilium ingress (gateway)/egress (apiserver+Keycloak) policies, README. Wired into platform/kustomization.yaml (platform tier). - platform/headlamp/httproute.yaml wired into platform-config (Gateway API CR, dependsOn platform), modeled on grafana-httproute. - keycloak-realm-import: add `headlamp` confidential client + groups/audience mappers + HEADLAMP_CLIENT_SECRET placeholder. Client also created LIVE via kcadm (realm import no-ops on the existing realm). - keycloak-realm-clients.sops: add headlamp-client-secret (mirrored into headlamp-oidc-credentials clientSecret). Both encrypted to the repo age key. Co-Authored-By: Claude Opus 4.8 --- platform-config/kustomization.yaml | 2 + platform/headlamp/README.md | 58 +++++++++++++ .../headlamp-oidc-credentials.sops.yaml | 32 +++++++ ...eadlamp-oidc-credentials.sops.yaml.example | 36 ++++++++ platform/headlamp/helmrelease.yaml | 87 +++++++++++++++++++ platform/headlamp/helmrepository.yaml | 11 +++ platform/headlamp/httproute.yaml | 35 ++++++++ platform/headlamp/kustomization.yaml | 18 ++++ platform/headlamp/namespace.yaml | 16 ++++ platform/headlamp/networkpolicy.yaml | 73 ++++++++++++++++ .../keycloak/keycloak-realm-clients.sops.yaml | 5 +- .../keycloak-realm-clients.sops.yaml.example | 3 + platform/keycloak/keycloak-realm-import.yaml | 53 +++++++++++ platform/kustomization.yaml | 3 + 14 files changed, 430 insertions(+), 2 deletions(-) create mode 100644 platform/headlamp/README.md create mode 100644 platform/headlamp/headlamp-oidc-credentials.sops.yaml create mode 100644 platform/headlamp/headlamp-oidc-credentials.sops.yaml.example create mode 100644 platform/headlamp/helmrelease.yaml create mode 100644 platform/headlamp/helmrepository.yaml create mode 100644 platform/headlamp/httproute.yaml create mode 100644 platform/headlamp/kustomization.yaml create mode 100644 platform/headlamp/namespace.yaml create mode 100644 platform/headlamp/networkpolicy.yaml diff --git a/platform-config/kustomization.yaml b/platform-config/kustomization.yaml index c543c13..58810f3 100644 --- a/platform-config/kustomization.yaml +++ b/platform-config/kustomization.yaml @@ -64,3 +64,5 @@ resources: # deadlock (platform never Ready -> platform-config never reconciles -> these never apply). # monitoring SSO proof: Grafana native-OIDC login ingress (Gateway API CR) - ../platform/monitoring/grafana-httproute.yaml + # headlamp: Kubernetes dashboard ingress (Gateway API CR; native-OIDC + per-user RBAC, ADR-0014) + - ../platform/headlamp/httproute.yaml diff --git a/platform/headlamp/README.md b/platform/headlamp/README.md new file mode 100644 index 0000000..8fc0012 --- /dev/null +++ b/platform/headlamp/README.md @@ -0,0 +1,58 @@ +# Headlamp — Kubernetes dashboard (Keycloak OIDC + per-user RBAC) + +Headlamp is served at `https://headlamp.home.keithjackson.dev` behind the shared Gateway +(wildcard TLS), with **native Keycloak OIDC login** and **per-user RBAC** (ADR-0014, docs/15). + +## How per-user RBAC works (end to end) + +1. A user logs in at Keycloak (`homelab` realm, `headlamp` confidential client, auth-code flow). +2. Headlamp's backend keeps the user's **`id_token`** and forwards it as the **bearer** on every + request it proxies to the k3s API server (this is Headlamp's default OIDC behaviour). +3. The apiserver validates that token — it is configured (`infra/ansible` `roles/k3s_server`) with + `oidc-issuer-url` = this realm, `oidc-client-id=kubernetes`, `oidc-username-claim=email`, + `oidc-groups-claim=groups`, prefixes `oidc:`. +4. RBAC is applied **per user** via the existing group bindings in + [`../keycloak/k8s-oidc-rbac.yaml`](../keycloak/k8s-oidc-rbac.yaml): + `oidc:platform-admins` → `cluster-admin`, `oidc:viewers` → `view`. + +So each user gets exactly their own rights — **not** a shared Headlamp identity. That is why the +Headlamp ServiceAccount is **not** bound to cluster-admin: the chart default +`clusterRoleBinding.create: true` (→ `cluster-admin`) is overridden to **`false`** in +`helmrelease.yaml`. User requests ride the forwarded user token; the SA needs no cluster rights. + +## The audience nuance (why the client has an audience mapper) + +The apiserver only accepts ID tokens whose `aud` **contains `kubernetes`** (`oidc-client-id= +kubernetes`). A plain `headlamp` client issues `aud=[headlamp]`, which the apiserver would reject. +The `headlamp` client therefore carries an **`oidc-audience-mapper`** (see +`../keycloak/keycloak-realm-import.yaml`) with `included.client.audience: kubernetes`, so tokens +carry `aud=[headlamp, kubernetes]`: + +- Headlamp's own verifier accepts it (aud contains `headlamp`), and +- the apiserver accepts it (aud contains `kubernetes`). + +This keeps the apiserver OIDC config untouched (preferred over widening `--api-audiences`). + +## Secrets + +- The confidential client secret lives SOPS-encrypted in + [`../keycloak/keycloak-realm-clients.sops.yaml`](../keycloak/keycloak-realm-clients.sops.yaml) + (`headlamp-client-secret`, injected into the realm import via the `HEADLAMP_CLIENT_SECRET` + placeholder) and mirrored in this namespace's `headlamp-oidc-credentials.sops.yaml` + (`clientSecret`), which the chart `config.oidc.externalSecret` envFrom's. Same cross-namespace + mirror pattern as grafana/gitea. + +## Break-glass + +The cert-based admin in the bootstrap kubeconfig is unaffected by any of this and retains full +access if Keycloak/OIDC is down — the standard break-glass path. + +## Files + +| File | Tier | Purpose | +|---|---|---| +| `namespace.yaml` | platform | `headlamp` namespace (`gateway-access=shared`; platform, not app-namespace) | +| `helmrepository.yaml` / `helmrelease.yaml` | platform | Official `kubernetes-sigs/headlamp` chart + OIDC/RBAC values | +| `headlamp-oidc-credentials.sops.yaml` | platform | SOPS OIDC config the chart envFrom's (decrypted by the `platform` Flux Kustomization) | +| `networkpolicy.yaml` | platform | default-deny baseline + Cilium ingress(gateway)/egress(apiserver+Keycloak) allows | +| `httproute.yaml` | platform-config | Gateway API HTTPRoute for `headlamp.home.keithjackson.dev` (needs Gateway CRDs) | diff --git a/platform/headlamp/headlamp-oidc-credentials.sops.yaml b/platform/headlamp/headlamp-oidc-credentials.sops.yaml new file mode 100644 index 0000000..88e99bd --- /dev/null +++ b/platform/headlamp/headlamp-oidc-credentials.sops.yaml @@ -0,0 +1,32 @@ +apiVersion: v1 +kind: Secret +metadata: + name: headlamp-oidc + namespace: headlamp +type: Opaque +stringData: + clientID: ENC[AES256_GCM,data:PXrq7Qc830U=,iv:t/cXsXYyQGT7zDUL2LT0cJX/B0a69Efr9wz//wacEzw=,tag:Pnqx2MQvebI6mHXTfd4IvQ==,type:str] + issuerURL: ENC[AES256_GCM,data:A6YtkQCpqwEeDInEKZgW0E/7WiNiVHxR2rvcv0ZWuuy25N1v9xOMAiaELS55DvJF+JvMiyc=,iv:9VVQoAz27X8eW1XsSUdZAY9tdlSnCfXHnYqYH5/N5gs=,tag:gLZGYm4n1pxAT0EezWY0sg==,type:str] + scopes: ENC[AES256_GCM,data:bps1T/YTCRN44u+ERVP2vJ2r1PE=,iv:4+6KdXbfVQUp4NYgAkHFRyIpcr9YoQw91G8o/UmEDTA=,tag:bs0JKsveYekxHB45d0Qk0g==,type:str] + callbackURL: ENC[AES256_GCM,data:ViCO0/9pAxzIN3wzQ7MeOJaZEc3yddQ56SD7/ie3IaL0LP7zkjSIlkZ2hP9wJMqokmETCA==,iv:1X+DbHxnN4yEFzQBjIymOfGPGRbFA+OlJ+bUZ1ZLD0Q=,tag:3UAcnbXdoN0Fed74zNg5bg==,type:str] + clientSecret: ENC[AES256_GCM,data:SXzue9IzhyNsr2MFxDfEBNCgOTL6XLYqv0E0Rl6YOO62+B0b0V8dVmLKjBsK7VS+P6pXJQNZNDfFfRFC/3f9aA==,iv:sChj2ZDG2XapK7G9xIZ4qH8PvJbSjWVQqy3CEoD8LDU=,tag:3y4EbrlFt1EmzAadMTBN4g==,type:str] +sops: + kms: [] + gcp_kms: [] + azure_kv: [] + hc_vault: [] + age: + - recipient: age175l8mdpdfu7hq0aml3exs3gykarwjmuvr23sq56s3wjejnyk44wsp4mfzg + enc: | + -----BEGIN AGE ENCRYPTED FILE----- + YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA3RjJ5MGJ1cEJxMXRXTkZ5 + YURYaU5LTk1rT0kvOE5xMTZMcDVmVjA0TTFVCm1rTzBYaktSVE02TGdKZjBCaEda + cFQzZjZUTmZEWUs1cTljZk5NUDdRSjgKLS0tIFBjNXlNbFhSMy9yRm1TOE0yeUxr + dXlZUHdoVWpDS2I3TFBsOVdxc0NadDAKqemweg0FDcUxc93I8t6+o708TWjseh6f + W0r+NjeEYwcb9iJBGqkO04jP6Z0+QGFdAuQZB8w8AsPYMNTRNTwDwg== + -----END AGE ENCRYPTED FILE----- + lastmodified: "2026-08-28T16:39:25Z" + mac: ENC[AES256_GCM,data:QtrqHMgaldW05Zag5AuH44+zBcyDJXTWhnuDVoV6UveMGSep/u4L/paMvA6ubuyZN07qfEv0SZVgybFh2EWKw/7dhE4Wbp0sttnrZ62J3TTnE3U6y9MbkLIuaM5+19Wfi7xVQ9Ovb9c1gELgEpCIo5KDSquA3yEvu1JBwHwmA2s=,iv:CAR7LqRItL5czUeCmr1QtTKlEqAFoiHBtHq24e6zNZs=,tag:YAciIDYGryK8kXQpay+etg==,type:str] + pgp: [] + encrypted_regex: ^(data|stringData)$ + version: 3.9.4 diff --git a/platform/headlamp/headlamp-oidc-credentials.sops.yaml.example b/platform/headlamp/headlamp-oidc-credentials.sops.yaml.example new file mode 100644 index 0000000..b0f2d18 --- /dev/null +++ b/platform/headlamp/headlamp-oidc-credentials.sops.yaml.example @@ -0,0 +1,36 @@ +# ============================================================================= +# TEMPLATE — NOT APPLIED, NOT ENCRYPTED (.yaml.example is ignored by kustomize and never +# auto-encrypted by SOPS). Reference scaffold + OPERATOR HAND-OFF. +# +# WHAT THIS IS: the OIDC config Headlamp uses to authenticate against the Keycloak `homelab` realm's +# `headlamp` client. The Headlamp chart's `config.oidc.externalSecret` (helmrelease.yaml) envFrom's +# THIS Secret, so ALL OIDC env (clientID/clientSecret/issuerURL/scopes/callbackURL) live here, +# SOPS-encrypted, referenced by name only (R-IDP-04). The chart maps these keys to the container's +# OIDC_CLIENT_ID / OIDC_CLIENT_SECRET / OIDC_ISSUER_URL / OIDC_SCOPES / OIDC_CALLBACK_URL env vars. +# +# The `clientSecret` value MUST equal the `headlamp-client-secret` placed in +# platform/keycloak/keycloak-realm-clients.sops.yaml (which the realm import injects into the +# Keycloak `headlamp` client) — same cross-namespace mirror pattern as grafana/gitea (a Secret +# cannot be shared across namespaces, so the one value is stored in two encrypted files). +# +# HAND-OFF STEPS (operator): +# (a) Use the SAME headlamp-client-secret generated for keycloak-realm-clients (openssl rand -hex 32). +# (b) Copy to: platform/headlamp/headlamp-oidc-credentials.sops.yaml +# (c) Fill `clientSecret`; encrypt: +# sops -e -i platform/headlamp/headlamp-oidc-credentials.sops.yaml +# (d) It is already listed under `resources:` in platform/headlamp/kustomization.yaml. +# ============================================================================= +apiVersion: v1 +kind: Secret +metadata: + name: headlamp-oidc + namespace: headlamp +type: Opaque +stringData: + # Non-secret OIDC config (kept in the encrypted Secret because the chart envFrom's a single Secret). + clientID: headlamp + issuerURL: https://keycloak.home.keithjackson.dev/realms/homelab + scopes: "openid email profile" + callbackURL: https://headlamp.home.keithjackson.dev/oidc-callback + # The confidential client secret — MUST match keycloak-realm-clients headlamp-client-secret. + clientSecret: REPLACE-ME diff --git a/platform/headlamp/helmrelease.yaml b/platform/headlamp/helmrelease.yaml new file mode 100644 index 0000000..e155373 --- /dev/null +++ b/platform/headlamp/helmrelease.yaml @@ -0,0 +1,87 @@ +--- +# Headlamp — the Kubernetes web dashboard, served at headlamp.home.keithjackson.dev with Keycloak +# native-OIDC login and PER-USER RBAC (ADR-0014, docs/15). Official kubernetes-sigs/headlamp Helm +# chart via a Flux HelmRelease (same pattern as platform/monitoring's kube-prometheus-stack). +# +# ── PER-USER RBAC (the whole point) ──────────────────────────────────────────────────────────── +# Headlamp is configured for OIDC, so its backend forwards the LOGGED-IN USER's OIDC id_token as the +# bearer to the k3s API server. The apiserver validates that token (oidc-issuer-url = this realm, +# oidc-client-id=kubernetes, oidc-username-claim=email, oidc-groups-claim=groups; see +# infra/ansible roles/k3s_server) and RBAC is applied per user via the EXISTING group bindings +# (platform/keycloak/k8s-oidc-rbac.yaml: oidc:platform-admins -> cluster-admin, oidc:viewers -> view). +# So every user gets exactly their own rights — NOT a shared identity. +# +# clusterRoleBinding.create=false ← DELIBERATE. The default chart binds Headlamp's ServiceAccount +# to cluster-admin; that would let the backend act as cluster-admin. We do NOT want that: user +# requests ride the user's forwarded id_token, so the SA needs no cluster rights. Removing the +# binding means an unauthenticated backend can do nothing — the secure posture for per-user RBAC. +# +# ── AUDIENCE NUANCE ───────────────────────────────────────────────────────────────────────────── +# The apiserver only accepts ID tokens whose `aud` CONTAINS `kubernetes` (oidc-client-id=kubernetes). +# A `headlamp` client would issue aud=[headlamp]. The Keycloak `headlamp` client therefore carries an +# AUDIENCE protocol mapper that ADDS `kubernetes` to the token audience (keycloak-realm-import.yaml), +# giving aud=[headlamp, kubernetes]: Headlamp's own verifier accepts it (aud contains headlamp) AND +# the apiserver accepts it (aud contains kubernetes). This keeps the apiserver config untouched. +apiVersion: helm.toolkit.fluxcd.io/v2 +kind: HelmRelease +metadata: + name: headlamp + namespace: headlamp +spec: + interval: 30m + chart: + spec: + chart: headlamp + version: ">=0.40.0 <0.41.0" # Renovate maintains; chart appVersion pins the Headlamp image + sourceRef: + kind: HelmRepository + name: headlamp + namespace: headlamp + install: + remediation: + retries: 3 + upgrade: + remediation: + retries: 3 + values: + replicaCount: 1 + # In-cluster mode: Headlamp discovers the apiserver from the pod's in-cluster config, then + # OVERRIDES the bearer with the user's OIDC id_token for authenticated requests. + config: + inCluster: true + oidc: + # OIDC config (clientID / clientSecret / issuerURL / scopes / callbackURL) is supplied via an + # EXISTING SOPS-encrypted Secret consumed by name only (R-IDP-04) — the chart envFrom's it. + # (The chart's `externalSecret` path reads OIDC_* env from this Secret; nothing sensitive is + # rendered into the HelmRelease.) + externalSecret: + enabled: true + name: headlamp-oidc + hasScopes: true + # PER-USER RBAC: do NOT grant Headlamp's ServiceAccount cluster-admin. User requests are + # authorized by their forwarded OIDC token against the existing oidc:* group bindings. + serviceAccount: + create: true + clusterRoleBinding: + create: false + # Hardened pod defaults from the chart (runAsNonRoot, uid 100). Kept explicit for clarity. + securityContext: + runAsNonRoot: true + runAsUser: 100 + runAsGroup: 101 + privileged: false + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL + # NOTE: readOnlyRootFilesystem intentionally left unset (chart default). Enabling it needs a + # writable /tmp (+ plugins dir) emptyDir; deferred as a runtime-verified follow-up hardening. + resources: + requests: + cpu: 50m + memory: 128Mi + limits: + memory: 256Mi + service: + type: ClusterIP + port: 80 diff --git a/platform/headlamp/helmrepository.yaml b/platform/headlamp/helmrepository.yaml new file mode 100644 index 0000000..74e37f9 --- /dev/null +++ b/platform/headlamp/helmrepository.yaml @@ -0,0 +1,11 @@ +--- +# HelmRepository for the official Headlamp chart (kubernetes-sigs/headlamp). Renovate maintains the +# pinned chart version in helmrelease.yaml. Same Flux source pattern as platform/monitoring. +apiVersion: source.toolkit.fluxcd.io/v1 +kind: HelmRepository +metadata: + name: headlamp + namespace: headlamp +spec: + interval: 1h + url: https://kubernetes-sigs.github.io/headlamp/ diff --git a/platform/headlamp/httproute.yaml b/platform/headlamp/httproute.yaml new file mode 100644 index 0000000..433e6bd --- /dev/null +++ b/platform/headlamp/httproute.yaml @@ -0,0 +1,35 @@ +--- +# North-south ingress for Headlamp at headlamp.home.keithjackson.dev. Attaches to the shared Gateway +# (docs/09); TLS terminated at the Gateway by the *.home.keithjackson.dev wildcard cert (already +# covers this host). Same-namespace backend, so no ReferenceGrant is needed; the parentRef is +# permitted because the headlamp namespace carries gateway-access=shared. CR of the Gateway API CRDs +# — CARRIED BY platform-config (dependsOn platform) so the CRDs exist first (like grafana-httproute). +# +# auth-mode annotation records that Headlamp authenticates via native Keycloak OIDC (R-IDP-02); this +# route is NOT protected by oauth2-proxy forward-auth (Headlamp runs its own OIDC login + per-user +# token forwarding), so it must NOT carry an oauth2-proxy ExtAuth filter. +apiVersion: gateway.networking.k8s.io/v1 +kind: HTTPRoute +metadata: + name: headlamp + namespace: headlamp + labels: + app.kubernetes.io/name: headlamp + annotations: + homelab.keithjackson.dev/auth-mode: oidc + homelab.keithjackson.dev/auth-groups: platform-admins +spec: + parentRefs: + - name: shared + namespace: gateway + sectionName: https + hostnames: + - headlamp.home.keithjackson.dev + rules: + - matches: + - path: + type: PathPrefix + value: / + backendRefs: + - name: headlamp + port: 80 diff --git a/platform/headlamp/kustomization.yaml b/platform/headlamp/kustomization.yaml new file mode 100644 index 0000000..a7cba5e --- /dev/null +++ b/platform/headlamp/kustomization.yaml @@ -0,0 +1,18 @@ +--- +# Headlamp — Kubernetes dashboard with Keycloak native-OIDC login + per-user RBAC (ADR-0014). +# CONTROLLERS half: the HelmRepository + HelmRelease (chart installs the Deployment/Service/SA), the +# SOPS-encrypted OIDC credential (decrypted by the `platform` Flux Kustomization), and the +# default-deny + Cilium egress/ingress policies. The HTTPRoute is a Gateway API CR and lives in +# platform-config (dependsOn platform) so the Gateway CRDs exist first — same split as grafana. +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +namespace: headlamp +resources: + - namespace.yaml + - helmrepository.yaml + - helmrelease.yaml + # OIDC client credential (envFrom'd by the chart via config.oidc.externalSecret) — SOPS-encrypted, + # referenced by name (R-IDP-04). The matching value is mirrored in keycloak-realm-clients. + - headlamp-oidc-credentials.sops.yaml + # default-deny baseline + Cilium ingress(gateway)/egress(apiserver+Keycloak) allows. + - networkpolicy.yaml diff --git a/platform/headlamp/namespace.yaml b/platform/headlamp/namespace.yaml new file mode 100644 index 0000000..f1bee88 --- /dev/null +++ b/platform/headlamp/namespace.yaml @@ -0,0 +1,16 @@ +--- +apiVersion: v1 +kind: Namespace +metadata: + name: headlamp + labels: + # Expose Headlamp at headlamp.home.keithjackson.dev via the shared Gateway (docs/09). The + # *.home.keithjackson.dev wildcard cert already covers this host (httproute.yaml). + gateway-access: "shared" + # NOTE: intentionally NOT homelab.keithjackson.dev/app-namespace=true. Headlamp is a PLATFORM + # admin tool (like Grafana in `monitoring`), so Kyverno does not generate the app default-deny + # baseline here and the app-scoped auth-mode / egress-class / host-namespace policies do not + # apply. We still ship an explicit default-deny NetworkPolicy + CiliumNetworkPolicies + # (networkpolicy.yaml) as defense-in-depth: this pod holds the kube-apiserver egress path, so it + # is locked down to exactly DNS + apiserver + Keycloak (via the Gateway VIP) egress and + # Gateway-only ingress. diff --git a/platform/headlamp/networkpolicy.yaml b/platform/headlamp/networkpolicy.yaml new file mode 100644 index 0000000..2908fc3 --- /dev/null +++ b/platform/headlamp/networkpolicy.yaml @@ -0,0 +1,73 @@ +--- +# Default-deny baseline (R-NET-01): deny all ingress + egress, allow only DNS. `headlamp` is a +# PLATFORM namespace so Kyverno does not auto-generate this — we ship it explicitly (defense in +# depth; this pod holds the kube-apiserver egress path, so it is worth locking down). Same shape as +# the Kyverno-generated app baseline. Cilium enforces the UNION of this NetworkPolicy and the +# CiliumNetworkPolicies below (allow if ANY policy allows), so these add the apiserver/Keycloak/ +# Gateway paths on top of the deny-all floor. +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: default-deny + namespace: headlamp + labels: + homelab.keithjackson.dev/baseline: default-deny +spec: + podSelector: {} + policyTypes: + - Ingress + - Egress + ingress: [] + egress: + - to: + - namespaceSelector: + matchLabels: + kubernetes.io/metadata.name: kube-system + ports: + - protocol: UDP + port: 53 + - protocol: TCP + port: 53 +--- +# Gateway ingress allow (R-NET-01/02): only the shared Gateway (the Cilium `ingress` entity) may +# reach Headlamp, on its container port 4466 (Service headlamp:80 -> targetPort 4466). +apiVersion: cilium.io/v2 +kind: CiliumNetworkPolicy +metadata: + name: headlamp-allow-gateway-ingress + namespace: headlamp + labels: + app.kubernetes.io/name: headlamp +spec: + endpointSelector: + matchLabels: + app.kubernetes.io/name: headlamp + ingress: + - fromEntities: + - ingress + toPorts: + - ports: + - port: "4466" + protocol: TCP +--- +# Egress allow: Headlamp's backend must reach (a) the kube-apiserver — it forwards each user's OIDC +# id_token there for per-user RBAC — and (b) Keycloak for the OIDC auth-code/token/JWKS exchange. +# Keycloak is reached at keycloak.home.keithjackson.dev, which coredns split-horizon resolves to the +# Gateway VIP (an L2-announced LoadBalancer IP that Cilium classifies as `world`), so `world` egress +# covers the Keycloak path (same as the gitea OIDC bootstrap). DNS is already allowed by the +# default-deny baseline above. +apiVersion: cilium.io/v2 +kind: CiliumNetworkPolicy +metadata: + name: headlamp-allow-egress + namespace: headlamp + labels: + app.kubernetes.io/name: headlamp +spec: + endpointSelector: + matchLabels: + app.kubernetes.io/name: headlamp + egress: + - toEntities: + - kube-apiserver + - world diff --git a/platform/keycloak/keycloak-realm-clients.sops.yaml b/platform/keycloak/keycloak-realm-clients.sops.yaml index d25b4e1..5bd23f9 100644 --- a/platform/keycloak/keycloak-realm-clients.sops.yaml +++ b/platform/keycloak/keycloak-realm-clients.sops.yaml @@ -10,6 +10,7 @@ stringData: gitea-client-secret: ENC[AES256_GCM,data:ObU5sWSq2xEZKgqwccFo/kzs/nfhOVJ0NceBlgxgy6a6iOWeUN079iQpJ0ADoJyjsoQk1IQediFD06rwVSvNIQ==,iv:Bc4L8MngA6YKNRQRwEQojnUo5ixdS2t1AocvGMwE0oQ=,tag:YsozaKtL6CR68sPmGiL5wA==,type:str] nextcloud-client-secret: ENC[AES256_GCM,data:Uh3+EqIr88CNNt3KfDYP8lbNIhcB9iQrxJvbaxek40hD+JYAJmUP31WQypyDY3BiSvkjE6urr9PJmYoDgz7tkw==,iv:Tg4WAgyydIUBB1mqdOZPt0HsCkWSGNPvbI4WtFKj9V8=,tag:1LgZYdDPVEM+9UROanBllw==,type:str] semaphore-client-secret: ENC[AES256_GCM,data:BX5lXhFNX6vQh67G2+XM2vt9wymhL8Psx+NqQJneFup7gdZRjhk8paEU5dOfugZNwooieara7UFyAFm5+1BJFg==,iv:kmmz0ZiCZgsSowns3hjDTQzoD2lPCr9Oz1iS5WgbhEE=,tag:BbesQX6lxAhaGiqPqMQ9vA==,type:str] + headlamp-client-secret: ENC[AES256_GCM,data:hlQ5m6FeqwOqX9HTT8jC6NZmhQH0kMgUveMZ2Cfen1zLLxWfYXXC1QMBWSqrchY2EpyIr7vNPbFjbqffwiSDHQ==,iv:WS1F3gkWn18STl4c38AlCe7TU6jvZpl/eULf2TnjIZ4=,tag:n2BnQYrm4gZbLeemDBykjA==,type:str] sops: kms: [] gcp_kms: [] @@ -25,8 +26,8 @@ sops: YXJGUUJiTnFndFhDZDU1bmxXTENCbmsKuhb3RQ3LZmuTPAXEck8b6iuZ5vxLb9fV +fQELmU4kOTbZrPSxy+kpttLG5Cy5d6X2ieZXBf6opzpLTUhlBOkHA== -----END AGE ENCRYPTED FILE----- - lastmodified: "2026-08-27T22:12:32Z" - mac: ENC[AES256_GCM,data:nruWApjP/BK694O9/BtSsQExmjtWTBNCfq+FyXDtockjRdDHRmyxuePVdfvm2F/OO7AuxI2EPYhr+a1SiB+RYGWfz7GCtbKW1iDPPmgiAnGxFH/HiuEGpa7S4+CVkNaD1j/z6jBkYe6oPko2IO/ezVtcTG/x7Wzst8vRDeid5Tw=,iv:wzuEl5cnycPMpoBGWrx1F2YoIgFufZ9se0yzVcQznEQ=,tag:B5pSL2IEC9T5maa7UZhYJw==,type:str] + lastmodified: "2026-08-28T16:39:25Z" + mac: ENC[AES256_GCM,data:HRQ1SISIr9DrfCz96vEDGd/vP0o6pkVy5iMAUS+310oWAWhTxeXFvROJAhfdL1Pq8GzgFnA8s/Z6mSwr2E/D3Sjp679dYrZJeuKqt7A/TFiIsTt2yka+xpH9uCtoX8XpQDXFgq2bpAeOYoUVIahHCm9/P6mRT+D0tOkUNCcgOPE=,iv:WNR1+aVE/Ru3UmNiTRwDVN3KshIJvCdlvo3TIJJQiLA=,tag:J9r9x2yeRhiTFCraW0XH9g==,type:str] pgp: [] encrypted_regex: ^(data|stringData)$ version: 3.9.4 diff --git a/platform/keycloak/keycloak-realm-clients.sops.yaml.example b/platform/keycloak/keycloak-realm-clients.sops.yaml.example index ff0bbc5..b0f40a9 100644 --- a/platform/keycloak/keycloak-realm-clients.sops.yaml.example +++ b/platform/keycloak/keycloak-realm-clients.sops.yaml.example @@ -21,6 +21,8 @@ # (key: client-secret) [oauth2-proxy namespace] # - grafana-client-secret -> platform/monitoring/grafana-oidc-credentials.sops.yaml # (key: client-secret) [monitoring namespace] +# - headlamp-client-secret -> platform/headlamp/headlamp-oidc-credentials.sops.yaml +# (key: clientSecret) [headlamp namespace] # (Cross-namespace Secrets cannot be shared, hence one value stored in two encrypted files.) # # Until this file is filled + added to the kustomization, the KeycloakRealmImport stays PENDING @@ -35,3 +37,4 @@ type: Opaque stringData: grafana-client-secret: REPLACE-ME oauth2-proxy-client-secret: REPLACE-ME + headlamp-client-secret: REPLACE-ME diff --git a/platform/keycloak/keycloak-realm-import.yaml b/platform/keycloak/keycloak-realm-import.yaml index 0bfad6e..9cd920d 100644 --- a/platform/keycloak/keycloak-realm-import.yaml +++ b/platform/keycloak/keycloak-realm-import.yaml @@ -35,6 +35,10 @@ spec: secret: name: keycloak-realm-clients key: semaphore-client-secret + HEADLAMP_CLIENT_SECRET: + secret: + name: keycloak-realm-clients + key: headlamp-client-secret realm: realm: homelab enabled: true @@ -260,6 +264,55 @@ spec: "access.token.claim": "true" "userinfo.token.claim": "true" "claim.name": groups + # headlamp — the Kubernetes dashboard (platform/headlamp). Confidential auth-code client modeled + # on grafana. Redirect URI is Headlamp's OIDC callback (/oidc-callback). Headlamp forwards + # the logged-in user's id_token to the k3s apiserver as the bearer, so RBAC is applied PER USER + # via the existing oidc:* group bindings (k8s-oidc-rbac.yaml) — NOT via Headlamp's ServiceAccount. + # + # AUDIENCE MAPPER (the key integration nuance): the apiserver is configured oidc-client-id= + # kubernetes, so it only accepts ID tokens whose `aud` CONTAINS `kubernetes`. A plain `headlamp` + # client issues aud=[headlamp] -> the apiserver would REJECT it. The oidc-audience-mapper below + # ADDS the `kubernetes` client to the audience, giving aud=[headlamp, kubernetes]: Headlamp's own + # verifier accepts it (aud contains headlamp) AND the apiserver accepts it (aud contains + # kubernetes). This keeps the apiserver's oidc config untouched (preferred over widening + # api-audiences). The `groups` mapper (same as the other clients) carries the group claim the + # apiserver maps to oidc: for the RBAC bindings. + - clientId: headlamp + name: Headlamp (Kubernetes dashboard) + protocol: openid-connect + enabled: true + publicClient: false + standardFlowEnabled: true + directAccessGrantsEnabled: false + secret: "$(env:HEADLAMP_CLIENT_SECRET)" + redirectUris: + - https://headlamp.home.keithjackson.dev/oidc-callback + webOrigins: + - https://headlamp.home.keithjackson.dev + defaultClientScopes: + - email + - profile + - roles + protocolMappers: + - name: groups + protocol: openid-connect + protocolMapper: oidc-group-membership-mapper + config: + "full.path": "false" + "id.token.claim": "true" + "access.token.claim": "true" + "userinfo.token.claim": "true" + "claim.name": groups + # Add `kubernetes` to the token audience so the k3s apiserver (oidc-client-id=kubernetes) + # accepts the forwarded id_token. `included.client.audience` names the existing `kubernetes` + # client; the mapper appends it to `aud` in the ID token (and access token, harmless). + - name: audience-kubernetes + protocol: openid-connect + protocolMapper: oidc-audience-mapper + config: + "included.client.audience": kubernetes + "id.token.claim": "true" + "access.token.claim": "true" # --- MFA for platform-admins (R-IDP-04): a browser flow whose OTP step is CONDITIONAL on the # platform-admin role. Non-admins log in without OTP; platform-admins must enroll/present TOTP. # Set as the realm browserFlow below. VERIFY at bring-up (see runbook): a malformed flow fails diff --git a/platform/kustomization.yaml b/platform/kustomization.yaml index efb4817..416a0ff 100644 --- a/platform/kustomization.yaml +++ b/platform/kustomization.yaml @@ -34,6 +34,9 @@ resources: # (clusters/base/oauth2-proxy.yaml, dependsOn platform-config). See DECISIONS-LOG. - cnpg # CloudNativePG operator (DB backend for Keycloak) — ghcr.io images (not Bitnami) - keycloak # Keycloak Operator (quay.io images) + SOPS admin credential + - headlamp # Kubernetes dashboard — Keycloak native-OIDC login + PER-USER RBAC (ADR-0014). + # HelmRelease + SOPS OIDC credential + default-deny/Cilium policies; its + # HTTPRoute is a Gateway API CR in platform-config (dependsOn platform). # ---- Later phases: enabled when their phase opens ---- # - velero # Phase 4 (Backup/DR): BLOCKED on OQ-BKP (off-cluster backup target decision) # - image-automation # Phase 4 (Automated updates): Flux image-automation + Renovate