fix(net): allow SSO apps egress to keycloak backend (Cilium 1.20 CVE-2025-30162)
App SSO broke cluster-wide after the Cilium 1.16.5->1.20.1 upgrade: the CVE-2025-30162 fix (in
>=1.16.8/1.17.2) makes the shared Gateway's L7 proxy re-evaluate a pod's egress policy against
Keycloak's REAL backend cluster identity (enforce_policy_on_l7lb) when it hairpins to keycloak.home.
'toEntities: world' matches the Gateway VIP at L3/L4 but not the resolved keycloak identity at L7, so
OIDC discovery/token 403'd ('Access denied') for every app with a NetworkPolicy (gitea/nextcloud/
semaphore/headlamp/crafty...); Grafana survived only because monitoring has no NetworkPolicy.
Fix: explicit toEndpoints egress allow to the keycloak backend for SSO apps.
- catalog/render.py: emit it for auth.mode in {oidc,forward-auth}; re-rendered 8 app goldens.
- platform/headlamp + platform/crafty: hand-written policies get the same allow.
Split-horizon design unchanged. Verified live: headlamp-ns -> keycloak discovery 403 -> 200.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
1 parent
45deac9d50
commit
68690e5019
11 files changed
+73
-5
No files matched your search
@@ -185,6 +185,10 @@ spec:
|
|||||||
egress:
|
egress:
|
||||||
- toEntities:
|
- toEntities:
|
||||||
- world
|
- world
|
||||||
|
- toEndpoints:
|
||||||
|
- matchLabels:
|
||||||
|
io.kubernetes.pod.namespace: keycloak
|
||||||
|
app: keycloak
|
||||||
---
|
---
|
||||||
# HTTPRoute gitea-3000 (R-TLS/R-DNS)
|
# HTTPRoute gitea-3000 (R-TLS/R-DNS)
|
||||||
apiVersion: gateway.networking.k8s.io/v1
|
apiVersion: gateway.networking.k8s.io/v1
|
||||||
|
|||||||
@@ -184,6 +184,10 @@ spec:
|
|||||||
egress:
|
egress:
|
||||||
- toEntities:
|
- toEntities:
|
||||||
- world
|
- world
|
||||||
|
- toEndpoints:
|
||||||
|
- matchLabels:
|
||||||
|
io.kubernetes.pod.namespace: keycloak
|
||||||
|
app: keycloak
|
||||||
---
|
---
|
||||||
# HTTPRoute jackett-9117 (R-TLS/R-DNS)
|
# HTTPRoute jackett-9117 (R-TLS/R-DNS)
|
||||||
apiVersion: gateway.networking.k8s.io/v1
|
apiVersion: gateway.networking.k8s.io/v1
|
||||||
|
|||||||
@@ -140,6 +140,10 @@ spec:
|
|||||||
egress:
|
egress:
|
||||||
- toEntities:
|
- toEntities:
|
||||||
- world
|
- world
|
||||||
|
- toEndpoints:
|
||||||
|
- matchLabels:
|
||||||
|
io.kubernetes.pod.namespace: keycloak
|
||||||
|
app: keycloak
|
||||||
---
|
---
|
||||||
# HTTPRoute pihole-80 (R-TLS/R-DNS)
|
# HTTPRoute pihole-80 (R-TLS/R-DNS)
|
||||||
apiVersion: gateway.networking.k8s.io/v1
|
apiVersion: gateway.networking.k8s.io/v1
|
||||||
|
|||||||
@@ -184,6 +184,10 @@ spec:
|
|||||||
egress:
|
egress:
|
||||||
- toEntities:
|
- toEntities:
|
||||||
- world
|
- world
|
||||||
|
- toEndpoints:
|
||||||
|
- matchLabels:
|
||||||
|
io.kubernetes.pod.namespace: keycloak
|
||||||
|
app: keycloak
|
||||||
---
|
---
|
||||||
# HTTPRoute qbittorrent-8080 (R-TLS/R-DNS)
|
# HTTPRoute qbittorrent-8080 (R-TLS/R-DNS)
|
||||||
apiVersion: gateway.networking.k8s.io/v1
|
apiVersion: gateway.networking.k8s.io/v1
|
||||||
|
|||||||
@@ -205,6 +205,10 @@ spec:
|
|||||||
egress:
|
egress:
|
||||||
- toEntities:
|
- toEntities:
|
||||||
- world
|
- world
|
||||||
|
- toEndpoints:
|
||||||
|
- matchLabels:
|
||||||
|
io.kubernetes.pod.namespace: keycloak
|
||||||
|
app: keycloak
|
||||||
---
|
---
|
||||||
# HTTPRoute nextcloud-80 (R-TLS/R-DNS)
|
# HTTPRoute nextcloud-80 (R-TLS/R-DNS)
|
||||||
apiVersion: gateway.networking.k8s.io/v1
|
apiVersion: gateway.networking.k8s.io/v1
|
||||||
|
|||||||
@@ -206,6 +206,10 @@ spec:
|
|||||||
egress:
|
egress:
|
||||||
- toEntities:
|
- toEntities:
|
||||||
- world
|
- world
|
||||||
|
- toEndpoints:
|
||||||
|
- matchLabels:
|
||||||
|
io.kubernetes.pod.namespace: keycloak
|
||||||
|
app: keycloak
|
||||||
---
|
---
|
||||||
# HTTPRoute radarr-7878 (R-TLS/R-DNS)
|
# HTTPRoute radarr-7878 (R-TLS/R-DNS)
|
||||||
apiVersion: gateway.networking.k8s.io/v1
|
apiVersion: gateway.networking.k8s.io/v1
|
||||||
|
|||||||
@@ -209,6 +209,10 @@ spec:
|
|||||||
egress:
|
egress:
|
||||||
- toEntities:
|
- toEntities:
|
||||||
- world
|
- world
|
||||||
|
- toEndpoints:
|
||||||
|
- matchLabels:
|
||||||
|
io.kubernetes.pod.namespace: keycloak
|
||||||
|
app: keycloak
|
||||||
---
|
---
|
||||||
# HTTPRoute semaphore-3000 (R-TLS/R-DNS)
|
# HTTPRoute semaphore-3000 (R-TLS/R-DNS)
|
||||||
apiVersion: gateway.networking.k8s.io/v1
|
apiVersion: gateway.networking.k8s.io/v1
|
||||||
|
|||||||
@@ -206,6 +206,10 @@ spec:
|
|||||||
egress:
|
egress:
|
||||||
- toEntities:
|
- toEntities:
|
||||||
- world
|
- world
|
||||||
|
- toEndpoints:
|
||||||
|
- matchLabels:
|
||||||
|
io.kubernetes.pod.namespace: keycloak
|
||||||
|
app: keycloak
|
||||||
---
|
---
|
||||||
# HTTPRoute sonarr-8989 (R-TLS/R-DNS)
|
# HTTPRoute sonarr-8989 (R-TLS/R-DNS)
|
||||||
apiVersion: gateway.networking.k8s.io/v1
|
apiVersion: gateway.networking.k8s.io/v1
|
||||||
|
|||||||
+23
-1
@@ -527,6 +527,28 @@ def r_internet_egress(s: dict) -> dict:
|
|||||||
switch (a cluster-wide singleton keyed on the egress-class pod label) decide the PATH; this
|
switch (a cluster-wide singleton keyed on the egress-class pod label) decide the PATH; this
|
||||||
policy decides that north-south egress is ALLOWED at all.
|
policy decides that north-south egress is ALLOWED at all.
|
||||||
"""
|
"""
|
||||||
|
egress = [{"toEntities": ["world"]}]
|
||||||
|
# SSO apps (oidc/forward-auth) reach Keycloak at keycloak.home, which coredns split-horizon
|
||||||
|
# resolves to the shared Gateway VIP (an L2-announced LoadBalancer IP Cilium classifies as
|
||||||
|
# `world`), so the pod hairpins THROUGH the Gateway. Under Cilium >=1.16.8/1.17.2 (the
|
||||||
|
# CVE-2025-30162 fix, present in 1.20), the Gateway's L7 proxy re-evaluates THIS pod's egress
|
||||||
|
# policy against Keycloak's REAL backend cluster identity (enforce_policy_on_l7lb) — `world`
|
||||||
|
# matches the VIP at L3/L4 but NOT the resolved keycloak identity at L7, so OIDC discovery/
|
||||||
|
# token 403s ("Access denied") without an explicit allow to the keycloak backend. Scope it to
|
||||||
|
# the apps that actually talk to Keycloak (auth mode oidc or forward-auth).
|
||||||
|
if s.get("auth", {}).get("mode") in ("oidc", "forward-auth"):
|
||||||
|
egress.append(
|
||||||
|
{
|
||||||
|
"toEndpoints": [
|
||||||
|
{
|
||||||
|
"matchLabels": {
|
||||||
|
"io.kubernetes.pod.namespace": "keycloak",
|
||||||
|
"app": "keycloak",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
)
|
||||||
return {
|
return {
|
||||||
"apiVersion": "cilium.io/v2",
|
"apiVersion": "cilium.io/v2",
|
||||||
"kind": "CiliumNetworkPolicy",
|
"kind": "CiliumNetworkPolicy",
|
||||||
@@ -537,7 +559,7 @@ def r_internet_egress(s: dict) -> dict:
|
|||||||
},
|
},
|
||||||
"spec": {
|
"spec": {
|
||||||
"endpointSelector": {"matchLabels": {"app.kubernetes.io/name": s["name"]}},
|
"endpointSelector": {"matchLabels": {"app.kubernetes.io/name": s["name"]}},
|
||||||
"egress": [{"toEntities": ["world"]}],
|
"egress": egress,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -80,7 +80,10 @@ spec:
|
|||||||
# Bedrock binaries) when the operator creates/updates a server.
|
# Bedrock binaries) when the operator creates/updates a server.
|
||||||
# - The oauth2-proxy sidecar must reach Keycloak at keycloak.home.keithjackson.dev, which coredns
|
# - The oauth2-proxy sidecar must reach Keycloak at keycloak.home.keithjackson.dev, which coredns
|
||||||
# split-horizon resolves to the Gateway VIP (an L2-announced LoadBalancer IP Cilium classifies as
|
# split-horizon resolves to the Gateway VIP (an L2-announced LoadBalancer IP Cilium classifies as
|
||||||
# `world`) — so `world` egress covers both the jar downloads AND the OIDC path (same as headlamp).
|
# `world`); the sidecar hairpins THROUGH the shared Gateway. `world` covers the jar downloads and
|
||||||
|
# the VIP at L3/L4, but under Cilium >=1.16.8/1.17.2 (CVE-2025-30162 fix, in 1.20) the Gateway's
|
||||||
|
# L7 proxy re-checks this egress against Keycloak's REAL backend identity (enforce_policy_on_l7lb),
|
||||||
|
# so the OIDC path 403s ("Access denied") without the explicit keycloak toEndpoints allow below.
|
||||||
# - The sidecar -> Crafty hop is pod-local (127.0.0.1:8443) and is NOT subject to network policy.
|
# - The sidecar -> Crafty hop is pod-local (127.0.0.1:8443) and is NOT subject to network policy.
|
||||||
apiVersion: cilium.io/v2
|
apiVersion: cilium.io/v2
|
||||||
kind: CiliumNetworkPolicy
|
kind: CiliumNetworkPolicy
|
||||||
@@ -96,3 +99,7 @@ spec:
|
|||||||
egress:
|
egress:
|
||||||
- toEntities:
|
- toEntities:
|
||||||
- world
|
- world
|
||||||
|
- toEndpoints:
|
||||||
|
- matchLabels:
|
||||||
|
io.kubernetes.pod.namespace: keycloak
|
||||||
|
app: keycloak
|
||||||
@@ -53,9 +53,12 @@ spec:
|
|||||||
# Egress allow: Headlamp's backend must reach (a) the kube-apiserver — it forwards each user's OIDC
|
# Egress allow: Headlamp's backend must reach (a) the kube-apiserver — it forwards each user's OIDC
|
||||||
# id_token there for per-user RBAC — and (b) Keycloak for the OIDC auth-code/token/JWKS exchange.
|
# id_token there for per-user RBAC — and (b) Keycloak for the OIDC auth-code/token/JWKS exchange.
|
||||||
# Keycloak is reached at keycloak.home.keithjackson.dev, which coredns split-horizon resolves to the
|
# Keycloak is reached at keycloak.home.keithjackson.dev, which coredns split-horizon resolves to the
|
||||||
# Gateway VIP (an L2-announced LoadBalancer IP that Cilium classifies as `world`), so `world` egress
|
# Gateway VIP (an L2-announced LoadBalancer IP that Cilium classifies as `world`); the pod hairpins
|
||||||
# covers the Keycloak path (same as the gitea OIDC bootstrap). DNS is already allowed by the
|
# THROUGH the shared Gateway. `world` matches the VIP at L3/L4, but under Cilium >=1.16.8/1.17.2 (the
|
||||||
# default-deny baseline above.
|
# CVE-2025-30162 fix, present in 1.20) the Gateway's L7 proxy re-evaluates this egress against
|
||||||
|
# Keycloak's REAL backend cluster identity (enforce_policy_on_l7lb) — so `world` alone 403s the OIDC
|
||||||
|
# path ("Access denied"). The explicit toEndpoints allow to the keycloak backend below is required.
|
||||||
|
# DNS is already allowed by the default-deny baseline above.
|
||||||
apiVersion: cilium.io/v2
|
apiVersion: cilium.io/v2
|
||||||
kind: CiliumNetworkPolicy
|
kind: CiliumNetworkPolicy
|
||||||
metadata:
|
metadata:
|
||||||
@@ -71,3 +74,7 @@ spec:
|
|||||||
- toEntities:
|
- toEntities:
|
||||||
- kube-apiserver
|
- kube-apiserver
|
||||||
- world
|
- world
|
||||||
|
- toEndpoints:
|
||||||
|
- matchLabels:
|
||||||
|
io.kubernetes.pod.namespace: keycloak
|
||||||
|
app: keycloak
|
||||||
Reference in new issue
Block a user