Home-Network/home-network-k3s-impl
2
0 Fork 0
Code Issues 2 Pull Requests 5 Actions Packages Projects Releases Wiki Activity

feat(auth): gate pihole UI with per-app oauth2-proxy sidecar (canary)

Browse Source 
Close the un-gated-UI security finding for forward-auth apps (R-IDP-02) by
rendering the Option-B per-app oauth2-proxy reverse-proxy sidecar that crafty
proves working. catalog/render.py now emits, for any forward-auth app with an
exposed UI: the oauth2-proxy sidecar (upstream http://127.0.0.1:<uiport>,
:4180, --allowed-group per auth.group), an authproxy :4180 Service port, an
HTTPRoute backendRef -> :4180, and a gateway-ingress CiliumNetworkPolicy that
allows the Gateway to reach ONLY :4180 (fail-closed: the bare UI is never
served un-authenticated). oidc/native/none apps render byte-identical.

This is the CANARY: only pihole's golden carries the sidecar and is verified
fail-closed in-cluster first. jackett/qbittorrent/radarr/sonarr are deferred to
the rollout commit (temporarily listed in ci/render-golden.sh) so their UIs
keep serving exactly as today until their clients+secrets land.

- keycloak: confidential pihole client (realm-import + realm-clients mirror),
  provisioned live via kcadm (groups membership mapper, bare names).
- secret: SOPS/age pihole-oauth2-proxy-credentials (client-id/secret/cookie),
  wired into apps/wave-0 via the renderer's new sidecar-creds wave rule.
- L1 render goldens regenerated for the sidecar (forward-auth fixtures).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
jakman85andClaude Opus 4.8 committed 2026-08-31 08:36:03 -05:00
1 parent a04251bbc8
commit 72d4720d58
10 files changed
+584 -22
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

catalog/render.py
+226 -11
View File
@@ -52,6 +52,20 @@ KP_RELEASE = "kube-prometheus-stack" # ServiceMonitor/PrometheusRule discovery
EGRESS_CLASSES = ("internet", "vpn")
AUTH_MODES = ("oidc", "forward-auth", "native", "none")
# --- forward-auth (Option B) per-app oauth2-proxy sidecar ------------------------------
# A `forward-auth` app has a bare web UI with no usable native OIDC (PiHole, *arr, Jackett,
# qBittorrent). Cilium 1.16.x cannot inject ext_authz into the shared Gateway's Envoy config
# (docs/plans/cilium-native-externalauth-evaluation.md; platform/oauth2-proxy/README.md), so
# the platform's forward-auth CEC is an UNWIRED template. Instead each forward-auth app is
# gated by a PER-POD oauth2-proxy REVERSE-PROXY sidecar (Option B) — EXACTLY the pattern
# platform/crafty proves working: the shared Gateway routes <app>.home -> the sidecar (:4180),
# the sidecar authenticates the browser against Keycloak (homelab realm, the app's auth.groups)
# and only THEN proxies to the co-located app UI at http://127.0.0.1:<uiport>. The gate is IN
# the data path (fail-closed: no valid session in an authorized group => no upstream; R-IDP-02).
OAUTH2_PROXY_IMAGE = "quay.io/oauth2-proxy/oauth2-proxy:v7.8.1"
OAUTH2_PROXY_PORT = 4180
KEYCLOAK_ISSUER = f"https://keycloak.{ZONE}/realms/homelab"
# =============================================================================
# Defaults (renderer-contract.md "Default posture") — materialize BEFORE mapping.
@@ -252,6 +266,13 @@ def r_deployment(s: dict) -> dict:
pod_spec = {"containers": [container]}
# forward-auth (Option B): append the per-pod oauth2-proxy reverse-proxy sidecar AFTER the app
# container (app stays containers[0]). The shared Gateway routes to the sidecar (:4180); it gates
# the co-located UI against Keycloak and only then proxies to http://127.0.0.1:<uiport>. `oidc`
# apps (native client) and `native`/`none` render byte-identical (no sidecar).
if _needs_oauth2_sidecar(s):
pod_spec["containers"].append(_oauth2_sidecar_container(s))
# podSecurityContext (schema-optional): the k8s PodSecurityContext is emitted VERBATIM onto the
# pod spec. Its primary use is `fsGroup`, which makes a mounted Longhorn PVC group-writable by a
# non-root image (e.g. semaphore runs as UID 1001 and cannot otherwise write its config PVC).
@@ -294,6 +315,14 @@ def r_service(s: dict):
ports = _service_ports(s)
if not ports:
return None
# forward-auth (Option B): the shared Gateway must reach the oauth2-proxy sidecar, so front it
# with a `authproxy` :4180 Service port (the HTTPRoute backendRef targets this). The app UI/API
# port(s) stay in the Service for EAST-WEST callers (e.g. arr -> qbittorrent:8080 / jackett:9117);
# north-south to those app ports is denied at the Gateway (see r_gateway_ingress_policy).
if _needs_oauth2_sidecar(s):
ports = ports + [
{"name": "authproxy", "port": OAUTH2_PROXY_PORT, "targetPort": OAUTH2_PROXY_PORT}
]
return {
"apiVersion": "v1",
"kind": "Service",
@@ -474,7 +503,15 @@ def r_gateway_ingress_policy(s: dict):
"""
if not s["ingressRoutes"]:
return None
ports = _service_ports(s)
# forward-auth (Option B): the ONLY north-south path is the oauth2-proxy sidecar. Open the
# Gateway to :4180 ONLY — never the app UI port — so the un-gated UI is unreachable from the
# Gateway (fail-closed; R-IDP-02). Every other app opens its exposed service port(s) as before.
if _needs_oauth2_sidecar(s):
allow_ports = [{"port": str(OAUTH2_PROXY_PORT), "protocol": "TCP"}]
else:
allow_ports = [
{"port": str(p["port"]), "protocol": "TCP"} for p in _service_ports(s)
]
return {
"apiVersion": "cilium.io/v2",
"kind": "CiliumNetworkPolicy",
@@ -490,14 +527,7 @@ def r_gateway_ingress_policy(s: dict):
"ingress": [
{
"fromEntities": ["ingress"],
"toPorts": [
{
"ports": [
{"port": str(p["port"]), "protocol": "TCP"}
for p in ports
]
}
],
"toPorts": [{"ports": allow_ports}],
}
],
},
@@ -572,6 +602,10 @@ def r_httproutes(s: dict) -> list:
"""
out = []
for route in s["ingressRoutes"]:
# forward-auth (Option B): the Gateway backendRef targets the oauth2-proxy sidecar (:4180),
# NOT the app UI port — so the gate is in the data path and the bare UI is never served
# un-authenticated. Other apps route straight to their exposed app port.
backend_port = OAUTH2_PROXY_PORT if _needs_oauth2_sidecar(s) else route["port"]
out.append(
{
"apiVersion": "gateway.networking.k8s.io/v1",
@@ -592,7 +626,7 @@ def r_httproutes(s: dict) -> list:
{"path": {"type": "PathPrefix", "value": "/"}}
],
"backendRefs": [
{"name": s["name"], "port": route["port"]}
{"name": s["name"], "port": backend_port}
],
}
],
@@ -602,6 +636,94 @@ def r_httproutes(s: dict) -> list:
return out
def _needs_oauth2_sidecar(s: dict) -> bool:
"""True iff this service gets a per-pod oauth2-proxy reverse-proxy sidecar (Option B).
A `forward-auth` app with an exposed UI has no usable native OIDC, so its Gateway route is
gated IN the data path by a co-located oauth2-proxy sidecar (crafty's proven pattern). `oidc`
apps authenticate themselves (native Keycloak client) and `native`/`none` are unchanged."""
return s["auth"]["mode"] == "forward-auth" and bool(s["ingressRoutes"])
def _oauth2_sidecar_container(s: dict) -> dict:
"""The oauth2-proxy REVERSE-PROXY sidecar (Option B) that gates a forward-auth app's UI.
Mirrors platform/crafty/deployment.yaml's `oauth2-proxy` container EXACTLY, adapted per app:
the upstream is the co-located app UI over plain HTTP (http://127.0.0.1:<uiport>) — the *arr /
PiHole / Jackett / qBittorrent UIs are HTTP, so (unlike crafty's self-signed HTTPS :8443) there
is NO --ssl-upstream-insecure-skip-verify. --allowed-group is emitted once per declared
auth.group (R-IDP-06: a valid session is not enough — the user must be in an authorized group,
else /oauth2 returns 403). client-id/secret + cookie-secret come by name from the SOPS-encrypted
`<app>-oauth2-proxy` Secret (R-IDP-04). Single provider => --skip-provider-button 302s straight
to Keycloak. The sidecar->app hop is pod-local (127.0.0.1) and not subject to network policy."""
route = s["ingressRoutes"][0]
host = route["host"]
upstream_port = route["port"]
args = [
f"--http-address=0.0.0.0:{OAUTH2_PROXY_PORT}",
"--reverse-proxy=true",
"--provider=oidc",
f"--oidc-issuer-url={KEYCLOAK_ISSUER}",
f"--upstream=http://127.0.0.1:{upstream_port}",
"--email-domain=*",
"--scope=openid email profile",
"--oidc-groups-claim=groups",
]
# R-IDP-06: authorize by group. One flag per declared group (fail-closed to 403 otherwise).
for group in s["auth"].get("groups", []):
args.append(f"--allowed-group={group}")
args += [
"--skip-provider-button=true",
f"--redirect-url=https://{host}/oauth2/callback",
f"--cookie-domain=.{ZONE}",
f"--whitelist-domain=.{ZONE}",
"--cookie-secure=true",
"--pass-access-token=true",
"--set-xauthrequest=true",
]
secret = f"{s['name']}-oauth2-proxy"
return {
"name": "oauth2-proxy",
"image": OAUTH2_PROXY_IMAGE,
"imagePullPolicy": "IfNotPresent",
"args": args,
"env": [
{
"name": "OAUTH2_PROXY_CLIENT_ID",
"valueFrom": {"secretKeyRef": {"name": secret, "key": "client-id"}},
},
{
"name": "OAUTH2_PROXY_CLIENT_SECRET",
"valueFrom": {"secretKeyRef": {"name": secret, "key": "client-secret"}},
},
{
"name": "OAUTH2_PROXY_COOKIE_SECRET",
"valueFrom": {"secretKeyRef": {"name": secret, "key": "cookie-secret"}},
},
],
"ports": [{"name": "http", "containerPort": OAUTH2_PROXY_PORT, "protocol": "TCP"}],
"resources": {
"requests": {"cpu": "25m", "memory": "64Mi"},
"limits": {"memory": "128Mi"},
},
"securityContext": {
"allowPrivilegeEscalation": False,
"readOnlyRootFilesystem": True,
"capabilities": {"drop": ["ALL"]},
},
"livenessProbe": {
"httpGet": {"path": "/ping", "port": "http"},
"initialDelaySeconds": 5,
"periodSeconds": 15,
},
"readinessProbe": {
"httpGet": {"path": "/ping", "port": "http"},
"initialDelaySeconds": 5,
"periodSeconds": 15,
},
}
def _auth_annotations(s: dict) -> dict:
"""Auth wiring hints (renderer-contract §auth): the platform (oauth2-proxy/Keycloak)
consumes these to protect ingressRoutes. oidc/forward-auth carry the authorized groups
@@ -1058,6 +1180,7 @@ def plan_apps_waves(sds: list, present_files: set) -> dict:
convention:
- `<svc>.yaml` -> wave(svc)
- `<svc>-bootstrap-secret.sops.yaml` -> wave(svc) (consumed by svc's env, same apply)
- `<svc>-oauth2-proxy-credentials.sops.yaml` -> wave(svc) (forward-auth sidecar OIDC creds)
- `<svc>-db.yaml` -> wave(svc-db) (a strictly-lower wave; CNPG DB)
- `<svc>-db-app.sops.yaml` -> wave(svc-db) (CNPG initdb owner secret)
- `<svc>-db-backup-credentials.sops.yaml` -> wave(svc-db) (CNPG Barman S3 backup secret)
@@ -1078,6 +1201,10 @@ def plan_apps_waves(sds: list, present_files: set) -> dict:
w = waves[name]
add(w, f"{name}.yaml")
add(w, f"{name}-bootstrap-secret.sops.yaml")
# forward-auth (Option B): the per-app oauth2-proxy sidecar's OIDC client + cookie secret.
# Same wave as the app (the sidecar reads it at pod start); placed only when the file exists,
# so non-forward-auth apps are unaffected. Mirrors the bootstrap-secret naming convention.
add(w, f"{name}-oauth2-proxy-credentials.sops.yaml")
db = f"{name}{CO_DB_SUFFIX}"
if any(dep["to"] == db for dep in by_name[name]["dependencies"]):
wdb = waves[db]
@@ -1325,12 +1452,15 @@ def _selftest() -> int:
# 4b) exposure (ingressRoutes) -> an ADDITIVE gateway-ingress CiliumNetworkPolicy that
# allows ONLY the reserved `ingress` (Gateway proxy) identity to the exposed port,
# while default-deny still denies all other ingress (R-NET-01/02).
# while default-deny still denies all other ingress (R-NET-01/02). Uses an explicit
# non-forward-auth mode so this exercises the direct app-port gateway path (a
# forward-auth app instead routes the Gateway to the :4180 sidecar — see 6c).
exp_docs = render(
{
"name": "z",
"source": {"image": "a:1"},
"ingressRoutes": [{"host": "z.home.keithjackson.dev", "port": 80}],
"auth": {"mode": "none"},
}
)
gw = [
@@ -1444,6 +1574,91 @@ def _selftest() -> int:
"an SD without podSecurityContext must not gain a pod securityContext",
)
# 6c) forward-auth (Option B): an exposed forward-auth app gets an oauth2-proxy sidecar that
# fronts the Gateway path (fail-closed), while oidc/native/none apps are unchanged.
fa_docs = render(
{
"name": "fa",
"source": {"image": "a:1"},
"ingressRoutes": [{"host": "fa.home.keithjackson.dev", "port": 9117}],
"auth": {"mode": "forward-auth", "groups": ["media-users"]},
}
)
fa_dep = next(d[1] for d in fa_docs if d[1]["kind"] == "Deployment")
fa_containers = fa_dep["spec"]["template"]["spec"]["containers"]
check(
len(fa_containers) == 2
and fa_containers[0]["name"] == "fa"
and fa_containers[1]["name"] == "oauth2-proxy",
"forward-auth app must render the app container FIRST + an oauth2-proxy sidecar SECOND",
)
side = fa_containers[1]
check(
"--upstream=http://127.0.0.1:9117" in side["args"],
"sidecar upstream must be the co-located app UI over http on 127.0.0.1:<uiport>",
)
check(
"--allowed-group=media-users" in side["args"],
"sidecar must enforce the declared auth.group (R-IDP-06)",
)
check(
"--redirect-url=https://fa.home.keithjackson.dev/oauth2/callback" in side["args"],
"sidecar redirect-url must be the app host's /oauth2/callback",
)
check(
not any(a.startswith("--ssl-upstream-insecure-skip-verify") for a in side["args"]),
"http upstream must NOT skip TLS verify (that flag is only for crafty's self-signed https)",
)
check(
side["env"][0]["valueFrom"]["secretKeyRef"]["name"] == "fa-oauth2-proxy",
"sidecar creds must come by name from the per-app <app>-oauth2-proxy Secret (R-IDP-04)",
)
fa_svc = next(d[1] for d in fa_docs if d[1]["kind"] == "Service")
fa_svc_ports = {p["port"] for p in fa_svc["spec"]["ports"]}
check(
fa_svc_ports == {9117, 4180},
"forward-auth Service must expose the app port (east-west) AND :4180 (gateway->sidecar)",
)
fa_route = next(d[1] for d in fa_docs if d[1]["kind"] == "HTTPRoute")
check(
fa_route["spec"]["rules"][0]["backendRefs"][0]["port"] == 4180,
"forward-auth HTTPRoute backendRef must target the sidecar :4180, not the app UI port",
)
fa_gw = next(
d[1] for d in fa_docs
if d[1]["kind"] == "CiliumNetworkPolicy"
and d[1]["metadata"]["name"] == "fa-allow-gateway-ingress"
)
fa_gw_ports = {
p["port"] for r in fa_gw["spec"]["ingress"] for tp in r["toPorts"] for p in tp["ports"]
}
check(
fa_gw_ports == {"4180"},
"fail-closed: the Gateway may reach ONLY the sidecar :4180, never the un-gated app port",
)
# an oidc app (native client) must NOT gain a sidecar or a :4180 path.
oidc_docs = render(
{
"name": "oi",
"source": {"image": "a:1"},
"ingressRoutes": [{"host": "oi.home.keithjackson.dev", "port": 3000}],
"auth": {"mode": "oidc", "groups": ["platform-admins"]},
}
)
oi_containers = next(
d[1] for d in oidc_docs if d[1]["kind"] == "Deployment"
)["spec"]["template"]["spec"]["containers"]
check(
len(oi_containers) == 1
and not any("oauth2-proxy" in c["name"] for c in oi_containers),
"an oidc app must NOT gain an oauth2-proxy sidecar (native client authenticates itself)",
)
oi_route = next(d[1] for d in oidc_docs if d[1]["kind"] == "HTTPRoute")
check(
oi_route["spec"]["rules"][0]["backendRefs"][0]["port"] == 3000,
"an oidc app's HTTPRoute must still target its own app port (unchanged)",
)
# 7) wave partitioning: longest-path over the dependency DAG, CNPG-DB strictly below.
wsds = [
{"name": "a", "source": {"image": "x:1"}}, # leaf