fix(plex): unblock the app tier — drop the Certificate that stalled apps-wave-0, advertise http

I broke the app tier with #48. Two faults, both mine, both measured after merge:

1. KYVERNO BLOCKS cert-manager IN APP NAMESPACES. The temporary private-key
   Secrets (plex-vip-<random>) were denied by
   secret-provenance/secrets-must-be-flux-managed, which matches every Secret in
   a namespace labelled homelab.keithjackson.dev/app-namespace=true and excludes
   only those labelled secret-managed-by=controller or carrying a Flux label.
   `plex` HAS that label; `gateway` -- where the working wildcard cert lives --
   does NOT. That asymmetry is why the existing Certificate was never affected
   and mine was, and I did not check it. secretTemplate cannot fix it: those
   Secrets come from the keymanager controller, not the Certificate template. So
   no CertificateRequest, Order or Challenge was ever created.

2. FLUX HEALTH-CHECKS THE CERTIFICATE. `optional: true` did its job -- the pod
   rolled and ran fine without the Secret -- but apps-wave-0 is wait:true, so
   Flux waits on the Certificate OBJECT:
     timeout waiting for: [Certificate/plex/plex-vip status: 'InProgress']
   apps-wave-0 never went Ready; wave-1/-2 and everything downstream of
   platform-config sat in DependencyNotReady behind it. I guarded the pod and
   missed the object. That distinction is the whole bug.

Third, independent: the advertised URL was https with no servable certificate --
Plex answers 32400 with its *.plex.direct cert, which cannot cover plex-vip --
the exact failure mode of the outage earlier the same day.

So: delete the Certificate (pruning it clears the health gate) and advertise
http://plex-vip:32400, which is MEASURED to work today -- PiHole answers the
name with the Plex VIP (verified against pihole-0) and Plex serves 32400 there.
http is explicitly temporary and only tolerable because remote access does not
work yet; the restore condition is written at the call site.

DELETED, not merely unregistered: my first attempt kept the file as "tracked but
not onboarded" and the gate rejected it -- tls-lifecycle-automated (R-TLS-02)
fails any Certificate no Flux path reconciles. Legal for a Deployment, not for a
TLS object. The re-enable recipe (Kyverno exemption + a wait:false Kustomization
+ the https flip, as ONE change) is recorded in catalog/plex.yaml.

Kept and verified working from #48: the hostname design, PiHole's split-horizon
answer, and the postStart hook -- LanNetworksBandwidth is now set live.

Gate: rc 3, all substantive stages pass (first attempt was rc 1 BLOCKED).

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

apps/wave-0/kustomization.yaml

@@ -18,7 +18,6 @@ resources:
 - pihole-dns.yaml
 - pihole-oauth2-proxy-credentials.sops.yaml
 - pihole.yaml
-- plex-cert.yaml
 - plex-claim-secret.sops.yaml
 - plex-remote.yaml
 - plex.yaml

apps/wave-0/plex-cert.yaml

@@ -1,66 +0,0 @@
----
-# =================================================================================================
-# PLEX TLS — a Let's Encrypt certificate for the ONE hostname Plex advertises to plex.tv.
-# =================================================================================================
-# HAND-WRITTEN, NOT RENDERED. A non-catalog app-tier unit anchored to the `plex` ServiceDefinition
-# and placed into wave 0 by `catalog/render.py --emit-waves` via its EXTRA_UNITS table — the same
-# mechanism gitea-actions, pihole-dns and plex-remote use. It is hand-written for the same reason:
-# `schemas/servicedefinition.schema.json` is `additionalProperties: false` and has NO field for a
-# cert-manager Certificate. (`ingressRoutes` gets TLS for free from the Gateway's wildcard; this is
-# the opposite case — a certificate for a workload that terminates its OWN TLS on a raw L4 port.)
-#
-# WHY PLEX NEEDS ITS OWN CERTIFICATE AT ALL. Plex serves tcp/32400 directly (not through the L7
-# Gateway — the operator chose the native path 2026-10-03, apps/wave-0/plex-remote.yaml) and it
-# presents its own `*.<serverhash>.plex.direct` certificate there. That name cannot cover
-# `plex-vip.${BASE_DOMAIN}`, the hostname we advertise, so without this the advertised `https://`
-# URL fails hostname verification for every client. That is not hypothetical: it is the shape of
-# the 2026-10-04 outage, where Plex's log carried a continuous storm of
-# `CERT: incomplete TLS handshake ... sslv3 alert certificate unknown` from real LAN clients while
-# plex.tv could publish nothing and the server vanished from the account.
-#
-# A SAN match is not what makes Plex FUNCTION — `httpsRequired=0` and `secureConnections` default
-# to Preferred, so clients can fall back to cleartext. It is what keeps the REMOTE leg encrypted
-# rather than silently downgrading traffic that crosses the internet.
-#
-# HOW IT REACHES PLEX. catalog/plex.yaml takes `tls.crt`/`tls.key` from this Secret as env
-# (`optional: true` — see below), and its `lifecycle.postStart` hook converts the PEM into the
-# PKCS#12 bundle Plex's `customCertificatePath` requires and sets the three `customCertificate*`
-# prefs. The conversion is redone on EVERY pod start, so a cert-manager RENEWAL is picked up with
-# no extra moving part and nothing to remember.
-#
-# ORDERING, AND WHY THE ENV REFS ARE OPTIONAL. This Certificate must live in the `plex` namespace
-# for the Deployment to read its Secret, and that namespace is created by the rendered
-# apps/wave-0/plex.yaml ITSELF — so this cannot be issued in an earlier tier (platform-config runs
-# before the namespace exists) and lands in the SAME reconcile as the workload. On a cold bootstrap
-# the Secret is therefore absent for as long as a DNS01 challenge takes. With hard env refs Plex
-# would sit in CreateContainerConfigError until then, and because `apps-wave-0` is `wait: true`
-# with `apps-wave-1` dependsOn it, a failed or slow issuance would STALL THE WHOLE APP TIER. Hence
-# `optional: true` on both refs (schema 1.11.0, added for exactly this): no certificate means Plex
-# still starts and serves on its own plex.direct cert, the hook logs that TLS was not installed,
-# and the next start after issuance converges.
-#
-# ISSUER: ClusterIssuer `letsencrypt-prod` (platform/cert-manager/clusterissuers.yaml), the same
-# issuer behind the Gateway's wildcard (platform/gateway/wildcard-certificate.yaml, Ready 28d on
-# prod as measured 2026-10-04). A SINGLE name is requested rather than reusing the wildcard,
-# because the wildcard's Secret lives in the `gateway` namespace and a Secret cannot be read across
-# namespaces; requesting `*.${BASE_DOMAIN}` again here would also burn a duplicate-certificate slot
-# against Let's Encrypt's weekly limit for no benefit. A single name is the least-privilege shape:
-# this key signs for `plex-vip.${BASE_DOMAIN}` and nothing else.
-apiVersion: cert-manager.io/v1
-kind: Certificate
-metadata:
-  name: plex-vip
-  namespace: plex
-  labels:
-    app.kubernetes.io/name: plex
-spec:
-  # The name Plex ADVERTISES and the name on this certificate must stay identical. They are two
-  # halves of one decision: catalog/plex.yaml's ADVERTISE_IP, the postStart hook's
-  # customCertificateDomain, this dnsNames entry, and catalog/pihole.yaml's split-horizon answer.
-  # Nothing enforces that they agree.
-  dnsNames:
-    - plex-vip.${BASE_DOMAIN}
-  secretName: plex-vip-tls
-  issuerRef:
-    kind: ClusterIssuer
-    name: letsencrypt-prod

apps/wave-0/plex.yaml

@@ -138,7 +138,7 @@ spec:
               name: plex-claim
               key: PLEX_CLAIM
         - name: ADVERTISE_IP
-          value: https://plex-vip.${BASE_DOMAIN}:32400
+          value: http://plex-vip.${BASE_DOMAIN}:32400
         - name: PLEX_TLS_CRT
           valueFrom:
             secretKeyRef: