jakman85andClaude Opus 4.8
68690e5019
App SSO broke cluster-wide after the Cilium 1.16.5->1.20.1 upgrade: the CVE-2025-30162 fix (in
>=1.16.8/1.17.2) makes the shared Gateway's L7 proxy re-evaluate a pod's egress policy against
Keycloak's REAL backend cluster identity (enforce_policy_on_l7lb) when it hairpins to keycloak.home.
'toEntities: world' matches the Gateway VIP at L3/L4 but not the resolved keycloak identity at L7, so
OIDC discovery/token 403'd ('Access denied') for every app with a NetworkPolicy (gitea/nextcloud/
semaphore/headlamp/crafty...); Grafana survived only because monitoring has no NetworkPolicy.
Fix: explicit toEndpoints egress allow to the keycloak backend for SSO apps.
- catalog/render.py: emit it for auth.mode in {oidc,forward-auth}; re-rendered 8 app goldens.
- platform/headlamp + platform/crafty: hand-written policies get the same allow.
Split-horizon design unchanged. Verified live: headlamp-ns -> keycloak discovery 403 -> 200.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
106 lines
4.1 KiB
YAML
106 lines
4.1 KiB
YAML
---
|
|
# Default-deny baseline (R-NET-01): deny all ingress + egress, allow only DNS. `crafty` is a PLATFORM
|
|
# namespace so Kyverno does NOT auto-generate this (its policies match app-namespace=true) — we ship it
|
|
# explicitly (defense in depth), same shape as the Kyverno-generated app baseline and as
|
|
# headlamp/homepage. Cilium enforces the UNION of this NetworkPolicy and the CiliumNetworkPolicies
|
|
# below (allow if ANY policy allows), so those add the Gateway/game/world paths on top of deny-all.
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: NetworkPolicy
|
|
metadata:
|
|
name: default-deny
|
|
namespace: crafty
|
|
labels:
|
|
homelab.keithjackson.dev/baseline: default-deny
|
|
spec:
|
|
podSelector: {}
|
|
policyTypes:
|
|
- Ingress
|
|
- Egress
|
|
ingress: []
|
|
egress:
|
|
- to:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: kube-system
|
|
ports:
|
|
- protocol: UDP
|
|
port: 53
|
|
- protocol: TCP
|
|
port: 53
|
|
---
|
|
# INGRESS allows. The crafty pod runs BOTH containers (oauth2-proxy sidecar :4180 + Crafty :8443 + the
|
|
# MC game-port subprocesses), all selected by app.kubernetes.io/name=crafty.
|
|
# - Gateway (Cilium `ingress` entity) -> oauth2-proxy sidecar :4180 ONLY. The Gateway can reach the
|
|
# GATE, never Crafty's :8443 directly (there is no Service for :8443 either).
|
|
# - LAN players + admins (`world` = off-cluster to Cilium) -> the raw game ports. NOT auth-gated:
|
|
# Minecraft protocols authenticate the player. `host`/`cluster` cover node-local health probes and
|
|
# any in-cluster caller.
|
|
# - `host` (the kubelet) -> :8443 and :4180 for the liveness/readiness probes.
|
|
apiVersion: cilium.io/v2
|
|
kind: CiliumNetworkPolicy
|
|
metadata:
|
|
name: crafty-allow-ingress
|
|
namespace: crafty
|
|
labels:
|
|
app.kubernetes.io/name: crafty
|
|
spec:
|
|
endpointSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: crafty
|
|
ingress:
|
|
# Gateway -> the oauth2-proxy gate. Kubelet (host) probes the gate too.
|
|
- fromEntities:
|
|
- ingress
|
|
- host
|
|
toPorts:
|
|
- ports:
|
|
- { port: "4180", protocol: TCP }
|
|
# LAN/world players -> raw game ports (starter set). Add ports here when new servers are created.
|
|
- fromEntities:
|
|
- world
|
|
- cluster
|
|
- host
|
|
toPorts:
|
|
- ports:
|
|
- { port: "25565", protocol: TCP } # Java slot 0
|
|
- { port: "25566", protocol: TCP } # Java slot 1
|
|
- { port: "25567", protocol: TCP } # Java slot 2
|
|
- { port: "19132", protocol: UDP } # Bedrock slot 0
|
|
- { port: "19133", protocol: UDP } # Bedrock slot 1
|
|
- { port: "19134", protocol: UDP } # Bedrock slot 2
|
|
# Kubelet (host) -> Crafty's HTTPS UI port for its TCP health probe.
|
|
- fromEntities:
|
|
- host
|
|
toPorts:
|
|
- ports:
|
|
- { port: "8443", protocol: TCP }
|
|
---
|
|
# EGRESS allows (DNS already permitted by the default-deny baseline above).
|
|
# - Crafty must reach the INTERNET to download Minecraft server jars (Mojang/PaperMC/Purpur/Fabric,
|
|
# Bedrock binaries) when the operator creates/updates a server.
|
|
# - The oauth2-proxy sidecar must reach Keycloak at keycloak.home.keithjackson.dev, which coredns
|
|
# split-horizon resolves to the Gateway VIP (an L2-announced LoadBalancer IP Cilium classifies as
|
|
# `world`); the sidecar hairpins THROUGH the shared Gateway. `world` covers the jar downloads and
|
|
# the VIP at L3/L4, but under Cilium >=1.16.8/1.17.2 (CVE-2025-30162 fix, in 1.20) the Gateway's
|
|
# L7 proxy re-checks this egress against Keycloak's REAL backend identity (enforce_policy_on_l7lb),
|
|
# so the OIDC path 403s ("Access denied") without the explicit keycloak toEndpoints allow below.
|
|
# - The sidecar -> Crafty hop is pod-local (127.0.0.1:8443) and is NOT subject to network policy.
|
|
apiVersion: cilium.io/v2
|
|
kind: CiliumNetworkPolicy
|
|
metadata:
|
|
name: crafty-allow-egress
|
|
namespace: crafty
|
|
labels:
|
|
app.kubernetes.io/name: crafty
|
|
spec:
|
|
endpointSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: crafty
|
|
egress:
|
|
- toEntities:
|
|
- world
|
|
- toEndpoints:
|
|
- matchLabels:
|
|
io.kubernetes.pod.namespace: keycloak
|
|
app: keycloak
|