#51 - fix(plex): unblock the app tier — drop the Certificate that stalled apps-wave-0, advertise http - home-network-k3s-impl

Conversation 0 Commits 1 Files Changed 6

+162 -86

1 Commits

Author	SHA1	Message	Date
jakman85andClaude Opus 5	a0e6ef145f	fix(plex): unblock the app tier — drop the Certificate that stalled apps-wave-0, advertise http gate / gate (pull_request) Successful in 1m50s Details gate / conformance (pull_request) Has been skipped Details I broke the app tier with #48. Two faults, both mine, both measured after merge: 1. KYVERNO BLOCKS cert-manager IN APP NAMESPACES. The temporary private-key Secrets (plex-vip-<random>) were denied by secret-provenance/secrets-must-be-flux-managed, which matches every Secret in a namespace labelled homelab.keithjackson.dev/app-namespace=true and excludes only those labelled secret-managed-by=controller or carrying a Flux label. `plex` HAS that label; `gateway` -- where the working wildcard cert lives -- does NOT. That asymmetry is why the existing Certificate was never affected and mine was, and I did not check it. secretTemplate cannot fix it: those Secrets come from the keymanager controller, not the Certificate template. So no CertificateRequest, Order or Challenge was ever created. 2. FLUX HEALTH-CHECKS THE CERTIFICATE. `optional: true` did its job -- the pod rolled and ran fine without the Secret -- but apps-wave-0 is wait:true, so Flux waits on the Certificate OBJECT: timeout waiting for: [Certificate/plex/plex-vip status: 'InProgress'] apps-wave-0 never went Ready; wave-1/-2 and everything downstream of platform-config sat in DependencyNotReady behind it. I guarded the pod and missed the object. That distinction is the whole bug. Third, independent: the advertised URL was https with no servable certificate -- Plex answers 32400 with its *.plex.direct cert, which cannot cover plex-vip -- the exact failure mode of the outage earlier the same day. So: delete the Certificate (pruning it clears the health gate) and advertise http://plex-vip:32400, which is MEASURED to work today -- PiHole answers the name with the Plex VIP (verified against pihole-0) and Plex serves 32400 there. http is explicitly temporary and only tolerable because remote access does not work yet; the restore condition is written at the call site. DELETED, not merely unregistered: my first attempt kept the file as "tracked but not onboarded" and the gate rejected it -- tls-lifecycle-automated (R-TLS-02) fails any Certificate no Flux path reconciles. Legal for a Deployment, not for a TLS object. The re-enable recipe (Kyverno exemption + a wait:false Kustomization + the https flip, as ONE change) is recorded in catalog/plex.yaml. Kept and verified working from #48: the hostname design, PiHole's split-horizon answer, and the postStart hook -- LanNetworksBandwidth is now set live. Gate: rc 3, all substantive stages pass (first attempt was rc 1 BLOCKED). Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>	2026-10-04 17:10:25 -05:00

Author

SHA1

Message

Date

jakman85andClaude Opus 5

a0e6ef145f

fix(plex): unblock the app tier — drop the Certificate that stalled apps-wave-0, advertise http

gate / gate (pull_request) Successful in 1m50s

Details

gate / conformance (pull_request) Has been skipped

Details

I broke the app tier with #48. Two faults, both mine, both measured after merge:

1. KYVERNO BLOCKS cert-manager IN APP NAMESPACES. The temporary private-key
   Secrets (plex-vip-<random>) were denied by
   secret-provenance/secrets-must-be-flux-managed, which matches every Secret in
   a namespace labelled homelab.keithjackson.dev/app-namespace=true and excludes
   only those labelled secret-managed-by=controller or carrying a Flux label.
   `plex` HAS that label; `gateway` -- where the working wildcard cert lives --
   does NOT. That asymmetry is why the existing Certificate was never affected
   and mine was, and I did not check it. secretTemplate cannot fix it: those
   Secrets come from the keymanager controller, not the Certificate template. So
   no CertificateRequest, Order or Challenge was ever created.

2. FLUX HEALTH-CHECKS THE CERTIFICATE. `optional: true` did its job -- the pod
   rolled and ran fine without the Secret -- but apps-wave-0 is wait:true, so
   Flux waits on the Certificate OBJECT:
     timeout waiting for: [Certificate/plex/plex-vip status: 'InProgress']
   apps-wave-0 never went Ready; wave-1/-2 and everything downstream of
   platform-config sat in DependencyNotReady behind it. I guarded the pod and
   missed the object. That distinction is the whole bug.

Third, independent: the advertised URL was https with no servable certificate --
Plex answers 32400 with its *.plex.direct cert, which cannot cover plex-vip --
the exact failure mode of the outage earlier the same day.

So: delete the Certificate (pruning it clears the health gate) and advertise
http://plex-vip:32400, which is MEASURED to work today -- PiHole answers the
name with the Plex VIP (verified against pihole-0) and Plex serves 32400 there.
http is explicitly temporary and only tolerable because remote access does not
work yet; the restore condition is written at the call site.

DELETED, not merely unregistered: my first attempt kept the file as "tracked but
not onboarded" and the gate rejected it -- tls-lifecycle-automated (R-TLS-02)
fails any Certificate no Flux path reconciles. Legal for a Deployment, not for a
TLS object. The re-enable recipe (Kyverno exemption + a wait:false Kustomization
+ the https flip, as ONE change) is recorded in catalog/plex.yaml.

Kept and verified working from #48: the hostname design, PiHole's split-horizon
answer, and the postStart hook -- LanNetworksBandwidth is now set live.

Gate: rc 3, all substantive stages pass (first attempt was rc 1 BLOCKED).

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

2026-10-04 17:10:25 -05:00

fix(plex): unblock the app tier — drop the Certificate that stalled apps-wave-0, advertise http #51