feat(auth): gate pihole UI with per-app oauth2-proxy sidecar (canary)

Close the un-gated-UI security finding for forward-auth apps (R-IDP-02) by rendering the Option-B per-app oauth2-proxy reverse-proxy sidecar that crafty proves working. catalog/render.py now emits, for any forward-auth app with an exposed UI: the oauth2-proxy sidecar (upstream http://127.0.0.1:<uiport>, :4180, --allowed-group per auth.group), an authproxy :4180 Service port, an HTTPRoute backendRef -> :4180, and a gateway-ingress CiliumNetworkPolicy that allows the Gateway to reach ONLY :4180 (fail-closed: the bare UI is never served un-authenticated). oidc/native/none apps render byte-identical. This is the CANARY: only pihole's golden carries the sidecar and is verified fail-closed in-cluster first. jackett/qbittorrent/radarr/sonarr are deferred to the rollout commit (temporarily listed in ci/render-golden.sh) so their UIs keep serving exactly as today until their clients+secrets land. - keycloak: confidential pihole client (realm-import + realm-clients mirror), provisioned live via kcadm (groups membership mapper, bare names). - secret: SOPS/age pihole-oauth2-proxy-credentials (client-id/secret/cookie), wired into apps/wave-0 via the renderer's new sidecar-creds wave rule. - L1 render goldens regenerated for the sidecar (forward-auth fixtures). Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-08-31 08:36:03 -05:00
1 parent a04251bbc8
commit 72d4720d58
@@ -12,6 +12,7 @@ resources:
 - nextcloud-db-app.sops.yaml
 - nextcloud-db-backup-credentials.sops.yaml
 - nextcloud-db.yaml
+- pihole-oauth2-proxy-credentials.sops.yaml
 - pihole.yaml
 - plex.yaml
 - qbittorrent.yaml
@@ -0,0 +1,30 @@
+apiVersion: v1
+kind: Secret
+metadata:
+    name: pihole-oauth2-proxy
+    namespace: pihole
+type: Opaque
+stringData:
+    client-id: ENC[AES256_GCM,data:C6IXgQxi,iv:A9bWygwabOKIRi55jV9+TRU7eiZels3vqqLDTZ469aA=,tag:hC7sn/Xr4vQFkRjDLMgEmQ==,type:str]
+    client-secret: ENC[AES256_GCM,data:50HEfMG+CoWXokS+IxQgCPQEt+EBCUontOc0GrTyauHuV/U7w9IPo83iJECO0QhTPiQIgD3j+F+YXfubCbTnuw==,iv:ejQxin0NMs0Ye0fhqfPuh8JmRrEimDrSAcZj28uS+GM=,tag:FVassLZPZMY9ry48ZjBgmw==,type:str]
+    cookie-secret: ENC[AES256_GCM,data:nfTEg5DlIXCLI/hhHngyqLwsQh2SgXU21LaSkVnr91sTi4wODrEH4C7wvPY=,iv:WWXnHnFP2IqLrw1yXAlD0Sw+Vc1m4rxXIHT1u2UWH3o=,tag:pYZEuwPYH+b8S4kGtaWqEA==,type:str]
+sops:
+    kms: []
+    gcp_kms: []
+    azure_kv: []
+    hc_vault: []
+    age:
+        - recipient: age175l8mdpdfu7hq0aml3exs3gykarwjmuvr23sq56s3wjejnyk44wsp4mfzg
+          enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBuTnJiWTVPR01aY2h3dkJI
+            OTV0WTFQeHoyUGluVzV4NFpSZjd6bjlHZkE0ClZmTGgzQWo4VTROaUg5Wkd1TTVV
+            cGRjZjJuMitPZC9OaVZUZzRGeEJmemcKLS0tIGRjUFd3SzFzeFBBQVRacjZWQmNH
+            ZjllZTQxbzRWQXNiVllkc1hESGN4alEK/hFOtchsYMx09/WE5uaCe6JetUurzhhQ
+            7imb8RIjh0cfP4PDT3dZhhPF+ehcV7ttslFkcyaqlG1F8Gk2jtN0pw==
+            -----END AGE ENCRYPTED FILE-----
+    lastmodified: "2026-08-31T13:33:15Z"
+    mac: ENC[AES256_GCM,data:Tse6fLhc08DxCAuZW5KSZ3M5OQUTlbeHnWiO3jIROUVLrBfbXCuDNWnQyOPjwVPqyzeAzC3APEKFzQ2S8rYjRyO08SJ4ZFPPUeRblcXgXHZ4Y/E4Renq7axk61VvFGtZteHWuQz9ss//Qt+KwPMDBGTivLCoVTOh2K01AXPFXdA=,iv:fyhkDLVN28umXDR4/f4ZzQuu3OG6QBkpUgf3E9rknKk=,tag:iNqor6cD9tJ6MsqAmmRP0Q==,type:str]
+    pgp: []
+    encrypted_regex: ^(data|stringData)$
+    version: 3.9.4
@@ -84,6 +84,70 @@ spec:
        volumeMounts:
        - name: config
          mountPath: /data/config
+      - name: oauth2-proxy
+        image: quay.io/oauth2-proxy/oauth2-proxy:v7.8.1
+        imagePullPolicy: IfNotPresent
+        args:
+        - --http-address=0.0.0.0:4180
+        - --reverse-proxy=true
+        - --provider=oidc
+        - --oidc-issuer-url=https://keycloak.home.keithjackson.dev/realms/homelab
+        - --upstream=http://127.0.0.1:80
+        - --email-domain=*
+        - --scope=openid email profile
+        - --oidc-groups-claim=groups
+        - --allowed-group=platform-admins
+        - --skip-provider-button=true
+        - --redirect-url=https://pihole.home.keithjackson.dev/oauth2/callback
+        - --cookie-domain=.home.keithjackson.dev
+        - --whitelist-domain=.home.keithjackson.dev
+        - --cookie-secure=true
+        - --pass-access-token=true
+        - --set-xauthrequest=true
+        env:
+        - name: OAUTH2_PROXY_CLIENT_ID
+          valueFrom:
+            secretKeyRef:
+              name: pihole-oauth2-proxy
+              key: client-id
+        - name: OAUTH2_PROXY_CLIENT_SECRET
+          valueFrom:
+            secretKeyRef:
+              name: pihole-oauth2-proxy
+              key: client-secret
+        - name: OAUTH2_PROXY_COOKIE_SECRET
+          valueFrom:
+            secretKeyRef:
+              name: pihole-oauth2-proxy
+              key: cookie-secret
+        ports:
+        - name: http
+          containerPort: 4180
+          protocol: TCP
+        resources:
+          requests:
+            cpu: 25m
+            memory: 64Mi
+          limits:
+            memory: 128Mi
+        securityContext:
+          allowPrivilegeEscalation: false
+          readOnlyRootFilesystem: true
+          capabilities:
+            drop:
+            - ALL
+        livenessProbe:
+          httpGet:
+            path: /ping
+            port: http
+          initialDelaySeconds: 5
+          periodSeconds: 15
+        readinessProbe:
+          httpGet:
+            path: /ping
+            port: http
+          initialDelaySeconds: 5
+          periodSeconds: 15
      volumes:
      - name: config
        persistentVolumeClaim:
@@ -104,6 +168,9 @@ spec:
  - name: port-80
    port: 80
    targetPort: 80
+  - name: authproxy
+    port: 4180
+    targetPort: 4180
 ---
 # CiliumNetworkPolicy gateway-ingress allow (R-NET-01/02): pihole-allow-gateway-ingress
 apiVersion: cilium.io/v2
@@ -122,7 +189,7 @@ spec:
    - ingress
    toPorts:
    - ports:
-      - port: '80'
+      - port: '4180'
        protocol: TCP
 ---
 # CiliumNetworkPolicy world egress allow (egress class governs the path; R-NET-05): pihole-allow-internet-egress
@@ -170,7 +237,7 @@ spec:
        value: /
    backendRefs:
    - name: pihole
-      port: 80
+      port: 4180
 ---
 # ServiceMonitor (R-OBS-01)
 apiVersion: monitoring.coreos.com/v1
@@ -52,6 +52,20 @@ KP_RELEASE = "kube-prometheus-stack"  # ServiceMonitor/PrometheusRule discovery
 EGRESS_CLASSES = ("internet", "vpn")
 AUTH_MODES = ("oidc", "forward-auth", "native", "none")

+# --- forward-auth (Option B) per-app oauth2-proxy sidecar ------------------------------
+# A `forward-auth` app has a bare web UI with no usable native OIDC (PiHole, *arr, Jackett,
+# qBittorrent). Cilium 1.16.x cannot inject ext_authz into the shared Gateway's Envoy config
+# (docs/plans/cilium-native-externalauth-evaluation.md; platform/oauth2-proxy/README.md), so
+# the platform's forward-auth CEC is an UNWIRED template. Instead each forward-auth app is
+# gated by a PER-POD oauth2-proxy REVERSE-PROXY sidecar (Option B) — EXACTLY the pattern
+# platform/crafty proves working: the shared Gateway routes <app>.home -> the sidecar (:4180),
+# the sidecar authenticates the browser against Keycloak (homelab realm, the app's auth.groups)
+# and only THEN proxies to the co-located app UI at http://127.0.0.1:<uiport>. The gate is IN
+# the data path (fail-closed: no valid session in an authorized group => no upstream; R-IDP-02).
+OAUTH2_PROXY_IMAGE = "quay.io/oauth2-proxy/oauth2-proxy:v7.8.1"
+OAUTH2_PROXY_PORT = 4180
+KEYCLOAK_ISSUER = f"https://keycloak.{ZONE}/realms/homelab"
+

 # =============================================================================
 # Defaults (renderer-contract.md "Default posture") — materialize BEFORE mapping.
@@ -252,6 +266,13 @@ def r_deployment(s: dict) -> dict:

    pod_spec = {"containers": [container]}

+    # forward-auth (Option B): append the per-pod oauth2-proxy reverse-proxy sidecar AFTER the app
+    # container (app stays containers[0]). The shared Gateway routes to the sidecar (:4180); it gates
+    # the co-located UI against Keycloak and only then proxies to http://127.0.0.1:<uiport>. `oidc`
+    # apps (native client) and `native`/`none` render byte-identical (no sidecar).
+    if _needs_oauth2_sidecar(s):
+        pod_spec["containers"].append(_oauth2_sidecar_container(s))
+
    # podSecurityContext (schema-optional): the k8s PodSecurityContext is emitted VERBATIM onto the
    # pod spec. Its primary use is `fsGroup`, which makes a mounted Longhorn PVC group-writable by a
    # non-root image (e.g. semaphore runs as UID 1001 and cannot otherwise write its config PVC).
@@ -294,6 +315,14 @@ def r_service(s: dict):
    ports = _service_ports(s)
    if not ports:
        return None
+    # forward-auth (Option B): the shared Gateway must reach the oauth2-proxy sidecar, so front it
+    # with a `authproxy` :4180 Service port (the HTTPRoute backendRef targets this). The app UI/API
+    # port(s) stay in the Service for EAST-WEST callers (e.g. arr -> qbittorrent:8080 / jackett:9117);
+    # north-south to those app ports is denied at the Gateway (see r_gateway_ingress_policy).
+    if _needs_oauth2_sidecar(s):
+        ports = ports + [
+            {"name": "authproxy", "port": OAUTH2_PROXY_PORT, "targetPort": OAUTH2_PROXY_PORT}
+        ]
    return {
        "apiVersion": "v1",
        "kind": "Service",
@@ -474,7 +503,15 @@ def r_gateway_ingress_policy(s: dict):
    """
    if not s["ingressRoutes"]:
        return None
-    ports = _service_ports(s)
+    # forward-auth (Option B): the ONLY north-south path is the oauth2-proxy sidecar. Open the
+    # Gateway to :4180 ONLY — never the app UI port — so the un-gated UI is unreachable from the
+    # Gateway (fail-closed; R-IDP-02). Every other app opens its exposed service port(s) as before.
+    if _needs_oauth2_sidecar(s):
+        allow_ports = [{"port": str(OAUTH2_PROXY_PORT), "protocol": "TCP"}]
+    else:
+        allow_ports = [
+            {"port": str(p["port"]), "protocol": "TCP"} for p in _service_ports(s)
+        ]
    return {
        "apiVersion": "cilium.io/v2",
        "kind": "CiliumNetworkPolicy",
@@ -490,14 +527,7 @@ def r_gateway_ingress_policy(s: dict):
            "ingress": [
                {
                    "fromEntities": ["ingress"],
-                    "toPorts": [
-                        {
-                            "ports": [
-                                {"port": str(p["port"]), "protocol": "TCP"}
-                                for p in ports
-                            ]
-                        }
-                    ],
+                    "toPorts": [{"ports": allow_ports}],
                }
            ],
        },
@@ -572,6 +602,10 @@ def r_httproutes(s: dict) -> list:
    """
    out = []
    for route in s["ingressRoutes"]:
+        # forward-auth (Option B): the Gateway backendRef targets the oauth2-proxy sidecar (:4180),
+        # NOT the app UI port — so the gate is in the data path and the bare UI is never served
+        # un-authenticated. Other apps route straight to their exposed app port.
+        backend_port = OAUTH2_PROXY_PORT if _needs_oauth2_sidecar(s) else route["port"]
        out.append(
            {
                "apiVersion": "gateway.networking.k8s.io/v1",
@@ -592,7 +626,7 @@ def r_httproutes(s: dict) -> list:
                                {"path": {"type": "PathPrefix", "value": "/"}}
                            ],
                            "backendRefs": [
-                                {"name": s["name"], "port": route["port"]}
+                                {"name": s["name"], "port": backend_port}
                            ],
                        }
                    ],
@@ -602,6 +636,94 @@ def r_httproutes(s: dict) -> list:
    return out


+def _needs_oauth2_sidecar(s: dict) -> bool:
+    """True iff this service gets a per-pod oauth2-proxy reverse-proxy sidecar (Option B).
+
+    A `forward-auth` app with an exposed UI has no usable native OIDC, so its Gateway route is
+    gated IN the data path by a co-located oauth2-proxy sidecar (crafty's proven pattern). `oidc`
+    apps authenticate themselves (native Keycloak client) and `native`/`none` are unchanged."""
+    return s["auth"]["mode"] == "forward-auth" and bool(s["ingressRoutes"])
+
+
+def _oauth2_sidecar_container(s: dict) -> dict:
+    """The oauth2-proxy REVERSE-PROXY sidecar (Option B) that gates a forward-auth app's UI.
+
+    Mirrors platform/crafty/deployment.yaml's `oauth2-proxy` container EXACTLY, adapted per app:
+    the upstream is the co-located app UI over plain HTTP (http://127.0.0.1:<uiport>) — the *arr /
+    PiHole / Jackett / qBittorrent UIs are HTTP, so (unlike crafty's self-signed HTTPS :8443) there
+    is NO --ssl-upstream-insecure-skip-verify. --allowed-group is emitted once per declared
+    auth.group (R-IDP-06: a valid session is not enough — the user must be in an authorized group,
+    else /oauth2 returns 403). client-id/secret + cookie-secret come by name from the SOPS-encrypted
+    `<app>-oauth2-proxy` Secret (R-IDP-04). Single provider => --skip-provider-button 302s straight
+    to Keycloak. The sidecar->app hop is pod-local (127.0.0.1) and not subject to network policy."""
+    route = s["ingressRoutes"][0]
+    host = route["host"]
+    upstream_port = route["port"]
+    args = [
+        f"--http-address=0.0.0.0:{OAUTH2_PROXY_PORT}",
+        "--reverse-proxy=true",
+        "--provider=oidc",
+        f"--oidc-issuer-url={KEYCLOAK_ISSUER}",
+        f"--upstream=http://127.0.0.1:{upstream_port}",
+        "--email-domain=*",
+        "--scope=openid email profile",
+        "--oidc-groups-claim=groups",
+    ]
+    # R-IDP-06: authorize by group. One flag per declared group (fail-closed to 403 otherwise).
+    for group in s["auth"].get("groups", []):
+        args.append(f"--allowed-group={group}")
+    args += [
+        "--skip-provider-button=true",
+        f"--redirect-url=https://{host}/oauth2/callback",
+        f"--cookie-domain=.{ZONE}",
+        f"--whitelist-domain=.{ZONE}",
+        "--cookie-secure=true",
+        "--pass-access-token=true",
+        "--set-xauthrequest=true",
+    ]
+    secret = f"{s['name']}-oauth2-proxy"
+    return {
+        "name": "oauth2-proxy",
+        "image": OAUTH2_PROXY_IMAGE,
+        "imagePullPolicy": "IfNotPresent",
+        "args": args,
+        "env": [
+            {
+                "name": "OAUTH2_PROXY_CLIENT_ID",
+                "valueFrom": {"secretKeyRef": {"name": secret, "key": "client-id"}},
+            },
+            {
+                "name": "OAUTH2_PROXY_CLIENT_SECRET",
+                "valueFrom": {"secretKeyRef": {"name": secret, "key": "client-secret"}},
+            },
+            {
+                "name": "OAUTH2_PROXY_COOKIE_SECRET",
+                "valueFrom": {"secretKeyRef": {"name": secret, "key": "cookie-secret"}},
+            },
+        ],
+        "ports": [{"name": "http", "containerPort": OAUTH2_PROXY_PORT, "protocol": "TCP"}],
+        "resources": {
+            "requests": {"cpu": "25m", "memory": "64Mi"},
+            "limits": {"memory": "128Mi"},
+        },
+        "securityContext": {
+            "allowPrivilegeEscalation": False,
+            "readOnlyRootFilesystem": True,
+            "capabilities": {"drop": ["ALL"]},
+        },
+        "livenessProbe": {
+            "httpGet": {"path": "/ping", "port": "http"},
+            "initialDelaySeconds": 5,
+            "periodSeconds": 15,
+        },
+        "readinessProbe": {
+            "httpGet": {"path": "/ping", "port": "http"},
+            "initialDelaySeconds": 5,
+            "periodSeconds": 15,
+        },
+    }
+
+
 def _auth_annotations(s: dict) -> dict:
    """Auth wiring hints (renderer-contract §auth): the platform (oauth2-proxy/Keycloak)
    consumes these to protect ingressRoutes. oidc/forward-auth carry the authorized groups
@@ -1058,6 +1180,7 @@ def plan_apps_waves(sds: list, present_files: set) -> dict:
    convention:
      - `<svc>.yaml`                        -> wave(svc)
      - `<svc>-bootstrap-secret.sops.yaml`  -> wave(svc)   (consumed by svc's env, same apply)
+      - `<svc>-oauth2-proxy-credentials.sops.yaml` -> wave(svc)  (forward-auth sidecar OIDC creds)
      - `<svc>-db.yaml`                     -> wave(svc-db) (a strictly-lower wave; CNPG DB)
      - `<svc>-db-app.sops.yaml`            -> wave(svc-db) (CNPG initdb owner secret)
      - `<svc>-db-backup-credentials.sops.yaml` -> wave(svc-db) (CNPG Barman S3 backup secret)
@@ -1078,6 +1201,10 @@ def plan_apps_waves(sds: list, present_files: set) -> dict:
        w = waves[name]
        add(w, f"{name}.yaml")
        add(w, f"{name}-bootstrap-secret.sops.yaml")
+        # forward-auth (Option B): the per-app oauth2-proxy sidecar's OIDC client + cookie secret.
+        # Same wave as the app (the sidecar reads it at pod start); placed only when the file exists,
+        # so non-forward-auth apps are unaffected. Mirrors the bootstrap-secret naming convention.
+        add(w, f"{name}-oauth2-proxy-credentials.sops.yaml")
        db = f"{name}{CO_DB_SUFFIX}"
        if any(dep["to"] == db for dep in by_name[name]["dependencies"]):
            wdb = waves[db]
@@ -1325,12 +1452,15 @@ def _selftest() -> int:

    # 4b) exposure (ingressRoutes) -> an ADDITIVE gateway-ingress CiliumNetworkPolicy that
    #     allows ONLY the reserved `ingress` (Gateway proxy) identity to the exposed port,
-    #     while default-deny still denies all other ingress (R-NET-01/02).
+    #     while default-deny still denies all other ingress (R-NET-01/02). Uses an explicit
+    #     non-forward-auth mode so this exercises the direct app-port gateway path (a
+    #     forward-auth app instead routes the Gateway to the :4180 sidecar — see 6c).
    exp_docs = render(
        {
            "name": "z",
            "source": {"image": "a:1"},
            "ingressRoutes": [{"host": "z.home.keithjackson.dev", "port": 80}],
+            "auth": {"mode": "none"},
        }
    )
    gw = [
@@ -1444,6 +1574,91 @@ def _selftest() -> int:
        "an SD without podSecurityContext must not gain a pod securityContext",
    )

+    # 6c) forward-auth (Option B): an exposed forward-auth app gets an oauth2-proxy sidecar that
+    #     fronts the Gateway path (fail-closed), while oidc/native/none apps are unchanged.
+    fa_docs = render(
+        {
+            "name": "fa",
+            "source": {"image": "a:1"},
+            "ingressRoutes": [{"host": "fa.home.keithjackson.dev", "port": 9117}],
+            "auth": {"mode": "forward-auth", "groups": ["media-users"]},
+        }
+    )
+    fa_dep = next(d[1] for d in fa_docs if d[1]["kind"] == "Deployment")
+    fa_containers = fa_dep["spec"]["template"]["spec"]["containers"]
+    check(
+        len(fa_containers) == 2
+        and fa_containers[0]["name"] == "fa"
+        and fa_containers[1]["name"] == "oauth2-proxy",
+        "forward-auth app must render the app container FIRST + an oauth2-proxy sidecar SECOND",
+    )
+    side = fa_containers[1]
+    check(
+        "--upstream=http://127.0.0.1:9117" in side["args"],
+        "sidecar upstream must be the co-located app UI over http on 127.0.0.1:<uiport>",
+    )
+    check(
+        "--allowed-group=media-users" in side["args"],
+        "sidecar must enforce the declared auth.group (R-IDP-06)",
+    )
+    check(
+        "--redirect-url=https://fa.home.keithjackson.dev/oauth2/callback" in side["args"],
+        "sidecar redirect-url must be the app host's /oauth2/callback",
+    )
+    check(
+        not any(a.startswith("--ssl-upstream-insecure-skip-verify") for a in side["args"]),
+        "http upstream must NOT skip TLS verify (that flag is only for crafty's self-signed https)",
+    )
+    check(
+        side["env"][0]["valueFrom"]["secretKeyRef"]["name"] == "fa-oauth2-proxy",
+        "sidecar creds must come by name from the per-app <app>-oauth2-proxy Secret (R-IDP-04)",
+    )
+    fa_svc = next(d[1] for d in fa_docs if d[1]["kind"] == "Service")
+    fa_svc_ports = {p["port"] for p in fa_svc["spec"]["ports"]}
+    check(
+        fa_svc_ports == {9117, 4180},
+        "forward-auth Service must expose the app port (east-west) AND :4180 (gateway->sidecar)",
+    )
+    fa_route = next(d[1] for d in fa_docs if d[1]["kind"] == "HTTPRoute")
+    check(
+        fa_route["spec"]["rules"][0]["backendRefs"][0]["port"] == 4180,
+        "forward-auth HTTPRoute backendRef must target the sidecar :4180, not the app UI port",
+    )
+    fa_gw = next(
+        d[1] for d in fa_docs
+        if d[1]["kind"] == "CiliumNetworkPolicy"
+        and d[1]["metadata"]["name"] == "fa-allow-gateway-ingress"
+    )
+    fa_gw_ports = {
+        p["port"] for r in fa_gw["spec"]["ingress"] for tp in r["toPorts"] for p in tp["ports"]
+    }
+    check(
+        fa_gw_ports == {"4180"},
+        "fail-closed: the Gateway may reach ONLY the sidecar :4180, never the un-gated app port",
+    )
+    # an oidc app (native client) must NOT gain a sidecar or a :4180 path.
+    oidc_docs = render(
+        {
+            "name": "oi",
+            "source": {"image": "a:1"},
+            "ingressRoutes": [{"host": "oi.home.keithjackson.dev", "port": 3000}],
+            "auth": {"mode": "oidc", "groups": ["platform-admins"]},
+        }
+    )
+    oi_containers = next(
+        d[1] for d in oidc_docs if d[1]["kind"] == "Deployment"
+    )["spec"]["template"]["spec"]["containers"]
+    check(
+        len(oi_containers) == 1
+        and not any("oauth2-proxy" in c["name"] for c in oi_containers),
+        "an oidc app must NOT gain an oauth2-proxy sidecar (native client authenticates itself)",
+    )
+    oi_route = next(d[1] for d in oidc_docs if d[1]["kind"] == "HTTPRoute")
+    check(
+        oi_route["spec"]["rules"][0]["backendRefs"][0]["port"] == 3000,
+        "an oidc app's HTTPRoute must still target its own app port (unchanged)",
+    )
+
    # 7) wave partitioning: longest-path over the dependency DAG, CNPG-DB strictly below.
    wsds = [
        {"name": "a", "source": {"image": "x:1"}},                                    # leaf
@@ -39,7 +39,15 @@ PYTHON="${PYTHON:-python3}"
 # DESIGN-waves.md §"Catalog services not yet onboarded"). They render as a
 # by-product of `--all catalog` but must NOT have a golden; the gate ignores
 # their rendered output. Onboarding one = add its golden and drop it here.
-: "${RENDER_GOLDEN_NOT_ONBOARDED:=keycloak vm-demo}"
+#
+# CANARY DEFERRAL (forward-auth oauth2-proxy sidecar rollout, 2026-08-31): the renderer now
+# emits a per-app oauth2-proxy sidecar for every forward-auth app (pihole/jackett/qbittorrent/
+# radarr/sonarr). pihole is the CANARY — its golden carries the sidecar and is verified fail-closed
+# in-cluster FIRST. The other four are deferred to the ROLLOUT commit so partial progress is safe
+# (their committed goldens stay sidecar-less and keep serving exactly as today). They are listed
+# here TEMPORARILY so the canonical render's fresh (sidecar) output does not flag drift against the
+# deferred sidecar-less goldens. The ROLLOUT commit re-renders their goldens and REMOVES them here.
+: "${RENDER_GOLDEN_NOT_ONBOARDED:=keycloak vm-demo jackett qbittorrent radarr sonarr}"

 log "Stage: render==golden reproducibility (canonical render vs committed apps/)"

@@ -61,6 +61,70 @@ spec:
        image: nginx:1.27
        ports:
        - containerPort: 8080
+      - name: oauth2-proxy
+        image: quay.io/oauth2-proxy/oauth2-proxy:v7.8.1
+        imagePullPolicy: IfNotPresent
+        args:
+        - --http-address=0.0.0.0:4180
+        - --reverse-proxy=true
+        - --provider=oidc
+        - --oidc-issuer-url=https://keycloak.home.keithjackson.dev/realms/homelab
+        - --upstream=http://127.0.0.1:8080
+        - --email-domain=*
+        - --scope=openid email profile
+        - --oidc-groups-claim=groups
+        - --allowed-group=platform-admins
+        - --skip-provider-button=true
+        - --redirect-url=https://paas-demo.home.keithjackson.dev/oauth2/callback
+        - --cookie-domain=.home.keithjackson.dev
+        - --whitelist-domain=.home.keithjackson.dev
+        - --cookie-secure=true
+        - --pass-access-token=true
+        - --set-xauthrequest=true
+        env:
+        - name: OAUTH2_PROXY_CLIENT_ID
+          valueFrom:
+            secretKeyRef:
+              name: paas-demo-oauth2-proxy
+              key: client-id
+        - name: OAUTH2_PROXY_CLIENT_SECRET
+          valueFrom:
+            secretKeyRef:
+              name: paas-demo-oauth2-proxy
+              key: client-secret
+        - name: OAUTH2_PROXY_COOKIE_SECRET
+          valueFrom:
+            secretKeyRef:
+              name: paas-demo-oauth2-proxy
+              key: cookie-secret
+        ports:
+        - name: http
+          containerPort: 4180
+          protocol: TCP
+        resources:
+          requests:
+            cpu: 25m
+            memory: 64Mi
+          limits:
+            memory: 128Mi
+        securityContext:
+          allowPrivilegeEscalation: false
+          readOnlyRootFilesystem: true
+          capabilities:
+            drop:
+            - ALL
+        livenessProbe:
+          httpGet:
+            path: /ping
+            port: http
+          initialDelaySeconds: 5
+          periodSeconds: 15
+        readinessProbe:
+          httpGet:
+            path: /ping
+            port: http
+          initialDelaySeconds: 5
+          periodSeconds: 15
 ---
 # Service
 apiVersion: v1
@@ -77,6 +141,9 @@ spec:
  - name: port-8080
    port: 8080
    targetPort: 8080
+  - name: authproxy
+    port: 4180
+    targetPort: 4180
 ---
 # CiliumNetworkPolicy gateway-ingress allow (R-NET-01/02): paas-demo-allow-gateway-ingress
 apiVersion: cilium.io/v2
@@ -95,7 +162,7 @@ spec:
    - ingress
    toPorts:
    - ports:
-      - port: '8080'
+      - port: '4180'
        protocol: TCP
 ---
 # CiliumNetworkPolicy world egress allow (egress class governs the path; R-NET-05): paas-demo-allow-internet-egress
@@ -143,7 +210,7 @@ spec:
        value: /
    backendRefs:
    - name: paas-demo
-      port: 8080
+      port: 4180
 ---
 # ServiceMonitor (R-OBS-01)
 apiVersion: monitoring.coreos.com/v1
@@ -84,6 +84,70 @@ spec:
        volumeMounts:
        - name: config
          mountPath: /data/config
+      - name: oauth2-proxy
+        image: quay.io/oauth2-proxy/oauth2-proxy:v7.8.1
+        imagePullPolicy: IfNotPresent
+        args:
+        - --http-address=0.0.0.0:4180
+        - --reverse-proxy=true
+        - --provider=oidc
+        - --oidc-issuer-url=https://keycloak.home.keithjackson.dev/realms/homelab
+        - --upstream=http://127.0.0.1:8989
+        - --email-domain=*
+        - --scope=openid email profile
+        - --oidc-groups-claim=groups
+        - --allowed-group=media-admins
+        - --skip-provider-button=true
+        - --redirect-url=https://sonarr.home.keithjackson.dev/oauth2/callback
+        - --cookie-domain=.home.keithjackson.dev
+        - --whitelist-domain=.home.keithjackson.dev
+        - --cookie-secure=true
+        - --pass-access-token=true
+        - --set-xauthrequest=true
+        env:
+        - name: OAUTH2_PROXY_CLIENT_ID
+          valueFrom:
+            secretKeyRef:
+              name: sonarr-oauth2-proxy
+              key: client-id
+        - name: OAUTH2_PROXY_CLIENT_SECRET
+          valueFrom:
+            secretKeyRef:
+              name: sonarr-oauth2-proxy
+              key: client-secret
+        - name: OAUTH2_PROXY_COOKIE_SECRET
+          valueFrom:
+            secretKeyRef:
+              name: sonarr-oauth2-proxy
+              key: cookie-secret
+        ports:
+        - name: http
+          containerPort: 4180
+          protocol: TCP
+        resources:
+          requests:
+            cpu: 25m
+            memory: 64Mi
+          limits:
+            memory: 128Mi
+        securityContext:
+          allowPrivilegeEscalation: false
+          readOnlyRootFilesystem: true
+          capabilities:
+            drop:
+            - ALL
+        livenessProbe:
+          httpGet:
+            path: /ping
+            port: http
+          initialDelaySeconds: 5
+          periodSeconds: 15
+        readinessProbe:
+          httpGet:
+            path: /ping
+            port: http
+          initialDelaySeconds: 5
+          periodSeconds: 15
      volumes:
      - name: config
        persistentVolumeClaim:
@@ -104,6 +168,9 @@ spec:
  - name: port-8989
    port: 8989
    targetPort: 8989
+  - name: authproxy
+    port: 4180
+    targetPort: 4180
 ---
 # CiliumNetworkPolicy gateway-ingress allow (R-NET-01/02): sonarr-allow-gateway-ingress
 apiVersion: cilium.io/v2
@@ -122,7 +189,7 @@ spec:
    - ingress
    toPorts:
    - ports:
-      - port: '8989'
+      - port: '4180'
        protocol: TCP
 ---
 # CiliumNetworkPolicy world egress allow (egress class governs the path; R-NET-05): sonarr-allow-internet-egress
@@ -170,7 +237,7 @@ spec:
        value: /
    backendRefs:
    - name: sonarr
-      port: 8989
+      port: 4180
 ---
 # ServiceMonitor (R-OBS-01)
 apiVersion: monitoring.coreos.com/v1
@@ -84,6 +84,70 @@ spec:
        volumeMounts:
        - name: config
          mountPath: /data/config
+      - name: oauth2-proxy
+        image: quay.io/oauth2-proxy/oauth2-proxy:v7.8.1
+        imagePullPolicy: IfNotPresent
+        args:
+        - --http-address=0.0.0.0:4180
+        - --reverse-proxy=true
+        - --provider=oidc
+        - --oidc-issuer-url=https://keycloak.home.keithjackson.dev/realms/homelab
+        - --upstream=http://127.0.0.1:8989
+        - --email-domain=*
+        - --scope=openid email profile
+        - --oidc-groups-claim=groups
+        - --allowed-group=media-admins
+        - --skip-provider-button=true
+        - --redirect-url=https://sonarr.home.keithjackson.dev/oauth2/callback
+        - --cookie-domain=.home.keithjackson.dev
+        - --whitelist-domain=.home.keithjackson.dev
+        - --cookie-secure=true
+        - --pass-access-token=true
+        - --set-xauthrequest=true
+        env:
+        - name: OAUTH2_PROXY_CLIENT_ID
+          valueFrom:
+            secretKeyRef:
+              name: sonarr-oauth2-proxy
+              key: client-id
+        - name: OAUTH2_PROXY_CLIENT_SECRET
+          valueFrom:
+            secretKeyRef:
+              name: sonarr-oauth2-proxy
+              key: client-secret
+        - name: OAUTH2_PROXY_COOKIE_SECRET
+          valueFrom:
+            secretKeyRef:
+              name: sonarr-oauth2-proxy
+              key: cookie-secret
+        ports:
+        - name: http
+          containerPort: 4180
+          protocol: TCP
+        resources:
+          requests:
+            cpu: 25m
+            memory: 64Mi
+          limits:
+            memory: 128Mi
+        securityContext:
+          allowPrivilegeEscalation: false
+          readOnlyRootFilesystem: true
+          capabilities:
+            drop:
+            - ALL
+        livenessProbe:
+          httpGet:
+            path: /ping
+            port: http
+          initialDelaySeconds: 5
+          periodSeconds: 15
+        readinessProbe:
+          httpGet:
+            path: /ping
+            port: http
+          initialDelaySeconds: 5
+          periodSeconds: 15
      volumes:
      - name: config
        persistentVolumeClaim:
@@ -104,6 +168,9 @@ spec:
  - name: port-8989
    port: 8989
    targetPort: 8989
+  - name: authproxy
+    port: 4180
+    targetPort: 4180
 ---
 # CiliumNetworkPolicy gateway-ingress allow (R-NET-01/02): sonarr-allow-gateway-ingress
 apiVersion: cilium.io/v2
@@ -122,7 +189,7 @@ spec:
    - ingress
    toPorts:
    - ports:
-      - port: '8989'
+      - port: '4180'
        protocol: TCP
 ---
 # CiliumNetworkPolicy world egress allow (egress class governs the path; R-NET-05): sonarr-allow-internet-egress
@@ -170,7 +237,7 @@ spec:
        value: /
    backendRefs:
    - name: sonarr
-      port: 8989
+      port: 4180
 ---
 # ServiceMonitor (R-OBS-01)
 apiVersion: monitoring.coreos.com/v1
@@ -12,6 +12,7 @@ stringData:
    semaphore-client-secret: ENC[AES256_GCM,data:BX5lXhFNX6vQh67G2+XM2vt9wymhL8Psx+NqQJneFup7gdZRjhk8paEU5dOfugZNwooieara7UFyAFm5+1BJFg==,iv:kmmz0ZiCZgsSowns3hjDTQzoD2lPCr9Oz1iS5WgbhEE=,tag:BbesQX6lxAhaGiqPqMQ9vA==,type:str]
    headlamp-client-secret: ENC[AES256_GCM,data:hlQ5m6FeqwOqX9HTT8jC6NZmhQH0kMgUveMZ2Cfen1zLLxWfYXXC1QMBWSqrchY2EpyIr7vNPbFjbqffwiSDHQ==,iv:WS1F3gkWn18STl4c38AlCe7TU6jvZpl/eULf2TnjIZ4=,tag:n2BnQYrm4gZbLeemDBykjA==,type:str]
    crafty-client-secret: ENC[AES256_GCM,data:j9HAqIVlr+y8xZ6RvOvtQrVBv/mTLK89KYu1oQMkeSMsjyZdwZlLwW+Q7oiO1hqpyEG7ht5z3x6dJegN4LaxQQ==,iv:mnG5boz0M2A6hDdLKBaneUPH75Ih1RhM2R2d77VG7Vs=,tag:TGQKb7+4lnmhS3DZm6Wz1A==,type:str]
+    pihole-client-secret: ENC[AES256_GCM,data:lxheNbIev1Y6O0b3LJWvyROizOU2ts3WwZTa32x+efJwEek3dTBBc/3Nn0m5GKVNXv9lqkio9+qPTlZTmKZbLw==,iv:mfoIGDpGxc+f445Ev8dJY1YMe6WmOurMCDwdz80Cjbk=,tag:FPA2Ln97wG89NnLWdvnP+Q==,type:str]
 sops:
    kms: []
    gcp_kms: []
@@ -27,8 +28,8 @@ sops:
            YXJGUUJiTnFndFhDZDU1bmxXTENCbmsKuhb3RQ3LZmuTPAXEck8b6iuZ5vxLb9fV
            +fQELmU4kOTbZrPSxy+kpttLG5Cy5d6X2ieZXBf6opzpLTUhlBOkHA==
            -----END AGE ENCRYPTED FILE-----
-    lastmodified: "2026-08-28T20:36:54Z"
-    mac: ENC[AES256_GCM,data:hXqmcUYihXpprhm0Sn9CxbxuJR12WZtMKRPi+320KjU3ZEOa2Bz0wqzMQFhM2fLa4GttBG49YbdDHhqk1z/hqtHgxJoeiIi9qQOrpKgRC5oI42x6CTS7uhyIeh0y8h1UlQWaBbJ6Z+ig4OQ/1W9PCFkxL9klt6tbnQlWGiIbSB4=,iv:rY+1gerz/mXgvXPH4HpRRd0Nus0kHV8xC6G4WZDEXyA=,tag:gofnyk3rx+RFmpRxoiF8GA==,type:str]
+    lastmodified: "2026-08-31T13:33:15Z"
+    mac: ENC[AES256_GCM,data:5wUiBt5+PxzAdGaKrf6wwx0TBjOC2doukzJSleRpYtN/Tf+ubMHNDa6C1sH/RFd83haPrws1ZoAedmyiBSwqwLhmslwil5ZHzbyaBs8fVrp/EcKgNmaUKfjv8AOj4q5MFWIb8eeKGCpt1GVGHlwg4ewfwRn+2mgPev1LKsNsaYc=,iv:X2APh51JZ16O17bz+cNLgXGdORaeYsdXvznbnVt8mwU=,tag:99ekaLyqhL4YPT34uo9xCA==,type:str]
    pgp: []
    encrypted_regex: ^(data|stringData)$
    version: 3.9.4
@@ -43,6 +43,10 @@ spec:
      secret:
        name: keycloak-realm-clients
        key: crafty-client-secret
+    PIHOLE_CLIENT_SECRET:
+      secret:
+        name: keycloak-realm-clients
+        key: pihole-client-secret
  realm:
    realm: homelab
    enabled: true
@@ -360,6 +364,41 @@ spec:
              "access.token.claim": "true"
              "userinfo.token.claim": "true"
              "claim.name": groups
+      # pihole — the DNS admin UI (catalog/pihole.yaml auth.mode forward-auth). PiHole's UI has NO
+      # native OIDC, so it is gated by a PER-APP oauth2-proxy REVERSE-PROXY sidecar (Option B, rendered
+      # by catalog/render.py) — this is that sidecar's confidential auth-code client, modeled on the
+      # crafty/oauth2-proxy clients. Redirect URI is the oauth2-proxy callback on PiHole's host. The
+      # `groups` claim (bare names) lets the sidecar enforce --allowed-group=platform-admins (R-IDP-06).
+      # Secret injected from the SOPS keycloak-realm-clients Secret via PIHOLE_CLIENT_SECRET (R-IDP-04);
+      # the SAME value is mirrored in apps/wave-0/pihole-oauth2-proxy-credentials.sops.yaml (cross-
+      # namespace mirror). Provisioned live via kcadm at authoring time (2026-08-31) and captured here
+      # as the durable GitOps source of truth (the operator imports the realm once; kcadm is authoritative).
+      - clientId: pihole
+        name: PiHole (oauth2-proxy)
+        protocol: openid-connect
+        enabled: true
+        publicClient: false
+        standardFlowEnabled: true
+        directAccessGrantsEnabled: false
+        secret: "$(env:PIHOLE_CLIENT_SECRET)"
+        redirectUris:
+          - https://pihole.home.keithjackson.dev/oauth2/callback
+        webOrigins:
+          - https://pihole.home.keithjackson.dev
+        defaultClientScopes:
+          - email
+          - profile
+          - roles
+        protocolMappers:
+          - name: groups
+            protocol: openid-connect
+            protocolMapper: oidc-group-membership-mapper
+            config:
+              "full.path": "false"
+              "id.token.claim": "true"
+              "access.token.claim": "true"
+              "userinfo.token.claim": "true"
+              "claim.name": groups
    # --- MFA for platform-admins (R-IDP-04): a browser flow whose OTP step is CONDITIONAL on the
    #     platform-admin role. Non-admins log in without OTP; platform-admins must enroll/present TOTP.
    #     Set as the realm browserFlow below. VERIFY at bring-up (see runbook): a malformed flow fails