jakman85andClaude Opus 5
6548f9a89c
Semaphore's whole SEMAPHORE_OIDC_PROVIDERS JSON blob lived inside the SOPS-encrypted
semaphore-bootstrap Secret (key `oidc-providers-json`) because it CONTAINS the confidential
client secret. That buried two environment-specific, NON-SECRET URLs in ciphertext —
provider_url (the realm issuer) and redirect_url — where Flux's ${BASE_DOMAIN} postBuild
substitution cannot reach. Both were frozen at `home.keithjackson.dev`, so prod's Semaphore
was pointed at dev's Keycloak and dev's own redirect URI: identical root cause to the
Headlamp login timeout, not yet reported only because nobody had tried prod SSO.
FIX: split the blob. The URLs are public and now live as a plaintext literal in
catalog/semaphore.yaml (rendered to the apps/wave-1 golden), ${BASE_DOMAIN}-substituted like
every other hostname. Only the client secret stays encrypted, in its own key
`oidc-client-secret`, and the KUBELET splices it back via dependent-env-var expansion:
$(SEMAPHORE_OIDC_CLIENT_SECRET) resolves against the secretKeyRef env var declared
immediately above it (order is load-bearing — the kubelet resolves env in list order and
leaves a forward reference as literal text).
Verified: the new key holds the same value as keycloak-realm-clients/semaphore-client-secret
(the mirror invariant), Semaphore v2.19.10's util.OidcProvider consumes client_secret as
before, and `flux envsubst --strict` leaves $(...) untouched while resolving ${BASE_DOMAIN} —
dev renders keycloak.home.keithjackson.dev, prod keycloak.prod.home.keithjackson.dev.
apps/wave-1/semaphore.yaml is a render golden; regenerated with the canonical
`catalog/render.py --all catalog -o apps --wave-layout --no-image-automation`.
Co-Authored-By: Claude Opus 5 <[email protected]>
52 lines
3.6 KiB
YAML
52 lines
3.6 KiB
YAML
# Semaphore break-glass admin + DB password + access-key encryption key (WS-APPS, Phase 6).
|
|
# SOPS-encrypted (age).
|
|
# admin-*: break-glass local admin (docs/15 availability caveat), created on first-boot.
|
|
# db-password: MIRRORS semaphore-db-app/password (ns semaphore-db) so the app can read the DB
|
|
# credential from its own namespace (a Pod cannot read a Secret in another namespace).
|
|
# access-key-encryption: FIXED base64 32-byte AES key Semaphore uses to encrypt stored SSH keys /
|
|
# credentials; MUST stay stable across restarts or stored secrets become undecryptable.
|
|
# oidc-client-secret: the Keycloak `semaphore` CONFIDENTIAL client secret, and ONLY that. It MIRRORS
|
|
# `semaphore-client-secret` in platform/keycloak/keycloak-realm-clients.sops.yaml (the realm import
|
|
# injects that copy into the client), same cross-namespace mirror pattern as db-password above.
|
|
# It replaced the former `oidc-providers-json` key (2026-09-07): that key held the WHOLE
|
|
# SEMAPHORE_OIDC_PROVIDERS blob, which buried the environment-specific realm-issuer and redirect
|
|
# URLs inside ciphertext where Flux's ${BASE_DOMAIN} postBuild substitution could not reach them,
|
|
# so prod ran with dev's URLs. Those URLs are PUBLIC and now live in plaintext in
|
|
# catalog/semaphore.yaml (rendered to apps/wave-1/semaphore.yaml); the kubelet splices THIS value
|
|
# back into the blob via $(SEMAPHORE_OIDC_CLIENT_SECRET). Only the secret half is encrypted.
|
|
# Recover: infra/devcontainer/dev.sh run sops -d apps/semaphore-bootstrap-secret.sops.yaml
|
|
apiVersion: v1
|
|
kind: Secret
|
|
metadata:
|
|
name: semaphore-bootstrap
|
|
namespace: semaphore
|
|
labels:
|
|
app.kubernetes.io/name: semaphore
|
|
type: Opaque
|
|
stringData:
|
|
admin-username: ENC[AES256_GCM,data:ju3wVdjNzRUP/w==,iv:d/5ZxgQZ5bympu3gT2ck3DJy9iUPPBAtZYsJvD3To7I=,tag:0UtBaDVqNSK4RCJrQicchw==,type:str]
|
|
admin-password: ENC[AES256_GCM,data:pxIwpgh3BoND2TI0mPTixOTJoictzT77dqtuRdMKgY/2WaW4wM1Ra6H6n/dxnyR2,iv:eYbFQGFYVJsnYGwn6cOI3BgAdzMcin9YggAhiuL+xqM=,tag:/sPWGv4gue0vjc2nMk7wWA==,type:str]
|
|
db-password: ENC[AES256_GCM,data:hBFpWXdpk1R5a23Pntup03t56NBRPS+SZU8CrHVb/ypIaIpgf7kjYtVy6nhTF6u0,iv:eBzoQgH5ZhWWpfKZ6AQmwaSal5QaiuN9gfexQ7GsVFs=,tag:j2k8EwhKDlj/Ud6O39ioVQ==,type:str]
|
|
access-key-encryption: ENC[AES256_GCM,data:SKtAV4HxJ9pNYDqUK/1uOdI1rtMji3pTfprI2rhQxFQyhc6dFlhqI+OUEcY=,iv:rKCP+FLidcOltOwmMMcO5M8tz4qiifIMI+W5soGO2M8=,tag:HaHzkX/JI+9wmcg2SD9GAQ==,type:str]
|
|
oidc-client-secret: ENC[AES256_GCM,data:uCo7TNzbM2QKw9ie+xzyUG9MFKco1OAlwr5qPn/AGpNvo1YyXSfbZyyM0CLJ1NS821hGg5Zr/m3R/xpy84s3FQ==,iv:O8a4haskufpmorPaxOkJ1Q99Y5GTlzmDFJAsHS+irLk=,tag:VojJpA9jNMbNQmIe1Qr44w==,type:str]
|
|
sops:
|
|
kms: []
|
|
gcp_kms: []
|
|
azure_kv: []
|
|
hc_vault: []
|
|
age:
|
|
- recipient: age175l8mdpdfu7hq0aml3exs3gykarwjmuvr23sq56s3wjejnyk44wsp4mfzg
|
|
enc: |
|
|
-----BEGIN AGE ENCRYPTED FILE-----
|
|
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAyTzF6NTBtcFVWK1Vtc0I2
|
|
VGdySHBsRGgvc1h0VzgvQ01xYVBJV3g2Q1NRCjJyZHVYZGtuamo1MFJHWlBwWkxt
|
|
cHA4VkY1TkRpN3M4UTdMOWZTbFVGemMKLS0tIGVNdDBrWVJveU56VjVBU2JWdHF4
|
|
RGtaYStXYUpONTZjZCtmM0NyZ3Rka1UKIQpmQai8dz+uRAsXxyeuGaRlRwsFqIPQ
|
|
9AfrZJcslYwlaK3ptm1oqaCdrlxcMar2nI7thSqzbLe+e1nzA2z7vQ==
|
|
-----END AGE ENCRYPTED FILE-----
|
|
lastmodified: "2026-09-07T07:01:50Z"
|
|
mac: ENC[AES256_GCM,data:Xutm1rQt57J4RILElD8FkNOsQT1HK5jZlPtqqLyZ/VqOoDwo+IS1wp7hROprlUvQC0bUXy79tRRIl5Q0CCOD+fvAGIdmade+vB7qXlYN6Q1441dGhuSTPvckso6FaB2ddRjKhKGLc/Z4bJoz9VaRNNxJNpWXIrENTg+I+QhNhlo=,iv:klJOrEQ6QN5dTGkQfGnbMbgCWsgWwZPb5YXDMeWeE34=,tag:fuJr4t3Tt3dT9u3/0SFo8Q==,type:str]
|
|
pgp: []
|
|
encrypted_regex: ^(data|stringData)$
|
|
version: 3.9.4
|