platform: move Headlamp out of the gating platform tier + fix -session-ttl crashloop
Headlamp's HelmRelease sat in the wait:true `platform` Flux tier. Its pod crashlooped (`flag provided but not defined: -session-ttl` — chart 0.40.1 renders a container arg the appVersion v0.40.1 backend lacks; upstream kubernetes-sigs/headlamp#4883), and because it gated `platform`, the whole chain stalled: platform Ready=Unknown -> platform-config, oauth2-proxy, apps-wave-0/1/2 and apps-vms all Ready=False "dependency not ready". Part A (unblock + durable lesson): a leaf UI must never gate the chain. Move all Headlamp resources into a new non-gating Flux Kustomization clusters/base/headlamp.yaml (dependsOn platform-config, wait:false, SOPS decryption patched per-env in clusters/dev) — mirroring oauth2-proxy exactly. The new Kustomization owns the whole platform/headlamp component; its HTTPRoute moves from platform-config into platform/headlamp/kustomization.yaml. Removed headlamp from platform/kustomization.yaml and its httproute from platform-config/kustomization.yaml. Now a crashlooping Headlamp can never stall platform/platform-config/apps. Part B (make it start): pin the chart to 0.41.0 (`>=0.41.0 <0.42.0`). The backend added `f.Int("session-ttl", …)` in v0.41.0 and chart 0.41.0 pins that same image (appVersion == chart version), so chart+image realign; every arg the chart renders for our values is supported. OIDC config + per-user RBAC (clusterRoleBinding.create:false) unchanged. Gate: `make -C ci gate` PASS. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
jakman85andClaude Opus 4.8 committed
1 parent
d8883e4481
commit
ebe0371b1b
7 files changed
+92
-10
No files matched your search
@@ -0,0 +1,45 @@
|
||||
---
|
||||
# Flux Kustomization: headlamp dashboard — reconciles ./platform/headlamp.
|
||||
#
|
||||
# POST-PLATFORM TIER (dependsOn platform-config). Headlamp is a LEAF Kubernetes-dashboard UI that
|
||||
# depends on Keycloak (native-OIDC login) + the trusted wildcard TLS cert + split-horizon DNS + the
|
||||
# Gateway API CRDs — all brought up by `platform` (controllers) and `platform-config` (Keycloak
|
||||
# instance/realm CRs, the wildcard Certificate, cilium L2 VIP, Gateway). It therefore MUST NOT live
|
||||
# inside the gating `platform` tier: a UI HelmRelease whose image/chart can crashloop (e.g. the
|
||||
# upstream headlamp-0.40.1 `-session-ttl` chart/image desync, issue #4883) would keep `platform`
|
||||
# not-Ready, and while `platform` is not-Ready NOTHING downstream reconciles — platform-config,
|
||||
# oauth2-proxy, every apps-wave and apps-vms all stall on "dependency not ready". That is exactly
|
||||
# how a single leaf UI stalled the whole chain (DECISIONS-LOG: "Headlamp gating stall"). This is the
|
||||
# same lesson oauth2-proxy already encodes: a leaf that can be legitimately unhealthy must reconcile
|
||||
# in its OWN post-platform Kustomization so its health can never gate the graph.
|
||||
#
|
||||
# NON-GATING (wait:false): nothing in the graph dependsOn headlamp (apps waves gate on
|
||||
# platform-config, not on this), and a dashboard pod can be long-not-Ready during a cold start while
|
||||
# it waits on Keycloak/DNS/TLS — same rationale as oauth2-proxy and apps-vms. wait:false guarantees
|
||||
# its health can NEVER gate anything, even if a future resource is (mistakenly) pointed dependsOn it.
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: headlamp
|
||||
namespace: flux-system
|
||||
spec:
|
||||
interval: 10m
|
||||
retryInterval: 2m
|
||||
timeout: 5m
|
||||
prune: true
|
||||
wait: false
|
||||
dependsOn:
|
||||
- name: platform-config # Keycloak instance/realm + wildcard TLS + split-horizon VIP + Gateway first
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
name: flux-system
|
||||
path: ./platform/headlamp
|
||||
# Same cluster-vars ConfigMap the platform tier fed it before the move (headlamp itself references
|
||||
# no ${VARS} today, but the substitution is retained for parity with the other tiers/future knobs).
|
||||
postBuild:
|
||||
substituteFrom:
|
||||
- kind: ConfigMap
|
||||
name: cluster-vars
|
||||
# SOPS decryption for headlamp-oidc-credentials.sops.yaml is layered per-environment by the overlay
|
||||
# (clusters/dev patches THIS Kustomization by name), identical to how it was decrypted via the
|
||||
# `platform` Kustomization before the move. See clusters/dev/kustomization.yaml.
|
||||
@@ -21,4 +21,9 @@ resources:
|
||||
# service that needs Keycloak + trusted TLS + split-horizon DNS must NOT sit in the gating
|
||||
# `platform` tier, where it self-gated its own dependencies into a cold-start deadlock (twice).
|
||||
- oauth2-proxy.yaml # Flux Kustomization -> ./platform/oauth2-proxy (dependsOn platform-config)
|
||||
# headlamp dashboard — POST-platform tier (dependsOn platform-config, wait:false). A leaf UI whose
|
||||
# HelmRelease can crashloop (upstream headlamp-0.40.1 `-session-ttl` chart/image desync) must NOT
|
||||
# sit in the gating `platform` tier, where its unhealth stalled platform-config + oauth2-proxy +
|
||||
# every apps wave/vm on "dependency not ready" (DECISIONS-LOG: "Headlamp gating stall").
|
||||
- headlamp.yaml # Flux Kustomization -> ./platform/headlamp (dependsOn platform-config)
|
||||
- cluster-vars.yaml # overlay knob surface (base defaults)
|
||||
@@ -77,6 +77,20 @@ patches:
|
||||
provider: sops
|
||||
secretRef:
|
||||
name: sops-age
|
||||
# headlamp moved to its own post-platform Flux Kustomization (clusters/base/headlamp.yaml).
|
||||
# It carries headlamp-oidc-credentials.sops.yaml, previously decrypted via the `platform`
|
||||
# Kustomization; now that this tier owns the encrypted Secret it needs its own decryption stanza.
|
||||
- patch: |-
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: headlamp
|
||||
namespace: flux-system
|
||||
spec:
|
||||
decryption:
|
||||
provider: sops
|
||||
secretRef:
|
||||
name: sops-age
|
||||
- patch: |-
|
||||
apiVersion: source.toolkit.fluxcd.io/v1
|
||||
kind: GitRepository
|
||||
|
||||
@@ -64,5 +64,7 @@ resources:
|
||||
# deadlock (platform never Ready -> platform-config never reconciles -> these never apply).
|
||||
# monitoring SSO proof: Grafana native-OIDC login ingress (Gateway API CR)
|
||||
- ../platform/monitoring/grafana-httproute.yaml
|
||||
# headlamp: Kubernetes dashboard ingress (Gateway API CR; native-OIDC + per-user RBAC, ADR-0014)
|
||||
- ../platform/headlamp/httproute.yaml
|
||||
# headlamp: httproute MOVED OUT of platform-config. It now travels with the headlamp component in
|
||||
# its own post-platform Flux Kustomization (clusters/base/headlamp.yaml, dependsOn platform-config).
|
||||
# Keeping it here — while the HelmRelease sat in the gating `platform` tier — meant a headlamp
|
||||
# crashloop stalled platform-config (and everything below it). See DECISIONS-LOG "Headlamp gating stall".
|
||||
@@ -32,7 +32,15 @@ spec:
|
||||
chart:
|
||||
spec:
|
||||
chart: headlamp
|
||||
version: ">=0.40.0 <0.41.0" # Renovate maintains; chart appVersion pins the Headlamp image
|
||||
# 0.41.x, NOT 0.40.x. Chart 0.40.1 shipped the `-session-ttl=86400` container arg BEFORE the
|
||||
# backend flag existed: its image (appVersion v0.40.1) has no `session-ttl` flag, so the pod
|
||||
# crashlooped with `flag provided but not defined: -session-ttl` (upstream issue #4883). The
|
||||
# backend gained `f.Int("session-ttl", …)` in v0.41.0, and chart 0.41.0 pins that same image
|
||||
# (appVersion == chart version), so chart+image realign: every arg the chart renders for our
|
||||
# values (-session-ttl + the -oidc-* args from the externalSecret; useCookie stays default-false
|
||||
# so `-oidc-use-cookie` is NOT emitted) is supported by the v0.41.0 backend. Renovate maintains
|
||||
# within 0.41.x; the appVersion continues to pin the Headlamp image.
|
||||
version: ">=0.41.0 <0.42.0"
|
||||
sourceRef:
|
||||
kind: HelmRepository
|
||||
name: headlamp
|
||||
|
||||
@@ -1,9 +1,14 @@
|
||||
---
|
||||
# Headlamp — Kubernetes dashboard with Keycloak native-OIDC login + per-user RBAC (ADR-0014).
|
||||
# CONTROLLERS half: the HelmRepository + HelmRelease (chart installs the Deployment/Service/SA), the
|
||||
# SOPS-encrypted OIDC credential (decrypted by the `platform` Flux Kustomization), and the
|
||||
# default-deny + Cilium egress/ingress policies. The HTTPRoute is a Gateway API CR and lives in
|
||||
# platform-config (dependsOn platform) so the Gateway CRDs exist first — same split as grafana.
|
||||
# Rendered by its OWN post-platform Flux Kustomization (clusters/base/headlamp.yaml, dependsOn
|
||||
# platform-config, wait:false) — NOT by the gating `platform` tier, where a crashlooping UI
|
||||
# HelmRelease stalled the whole chain (DECISIONS-LOG "Headlamp gating stall"). This component owns
|
||||
# the HelmRepository + HelmRelease (chart installs the Deployment/Service/SA), the SOPS-encrypted
|
||||
# OIDC credential (decrypted per-env by the headlamp Kustomization), the default-deny + Cilium
|
||||
# policies, AND the HTTPRoute. The httproute (a Gateway API CR) now travels WITH the component here
|
||||
# rather than being referenced in-place from platform-config: the CRDs it needs
|
||||
# (gateway.networking.k8s.io) are registered by platform, and platform is Ready before this tier's
|
||||
# platform-config dependency completes, so ordering holds — same move oauth2-proxy already made.
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
namespace: headlamp
|
||||
@@ -16,3 +21,4 @@ resources:
|
||||
- headlamp-oidc-credentials.sops.yaml
|
||||
# default-deny baseline + Cilium ingress(gateway)/egress(apiserver+Keycloak) allows.
|
||||
- networkpolicy.yaml
|
||||
- httproute.yaml # dashboard ingress (was referenced in-place by platform-config)
|
||||
@@ -34,9 +34,11 @@ resources:
|
||||
# (clusters/base/oauth2-proxy.yaml, dependsOn platform-config). See DECISIONS-LOG.
|
||||
- cnpg # CloudNativePG operator (DB backend for Keycloak) — ghcr.io images (not Bitnami)
|
||||
- keycloak # Keycloak Operator (quay.io images) + SOPS admin credential
|
||||
- headlamp # Kubernetes dashboard — Keycloak native-OIDC login + PER-USER RBAC (ADR-0014).
|
||||
# HelmRelease + SOPS OIDC credential + default-deny/Cilium policies; its
|
||||
# HTTPRoute is a Gateway API CR in platform-config (dependsOn platform).
|
||||
# headlamp (Kubernetes dashboard) is DELIBERATELY NOT here: a leaf UI whose HelmRelease can
|
||||
# crashloop (upstream headlamp-0.40.1 `-session-ttl` chart/image desync, issue #4883) must not
|
||||
# sit in this wait:true tier, where its unhealth stalled platform-config + oauth2-proxy + every
|
||||
# apps wave/vm on "dependency not ready". It now reconciles in its own post-platform Flux
|
||||
# Kustomization (clusters/base/headlamp.yaml, dependsOn platform-config). See DECISIONS-LOG.
|
||||
# ---- Later phases: enabled when their phase opens ----
|
||||
# - velero # Phase 4 (Backup/DR): BLOCKED on OQ-BKP (off-cluster backup target decision)
|
||||
# - image-automation # Phase 4 (Automated updates): Flux image-automation + Renovate
|
||||
|
||||
Reference in new issue
Block a user