Home-Network/home-network-k3s-impl
2
0 Fork 0
Code Issues 2 Pull Requests 5 Actions Packages Projects Releases Wiki Activity

Merge: oauth2-proxy forward-auth hardening (platform-admins floor, skip-button) + ext_authz mechanism research/template

Browse Source
This commit is contained in:
jakman85 committed 2026-08-28 15:20:38 -05:00
2 parents 3c7d7e0a1b 3e7e9386a0
commit a207103c7b
3 files changed
+257 -9
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

platform/oauth2-proxy/README.md
+66 -4
View File
@@ -21,8 +21,70 @@ Pairs with [`../keycloak`](../keycloak) to realize the "SSO/auth contract": a Se
forward-auth` service is protected by oauth2-proxy at the Gateway (ext-auth); `auth: oidc` services go
native to Keycloak. Gateway ext-auth wiring is a **WS-NET** contract.
## TODO (later phases — Phase 6)
## Enforcement (ext_authz forward-auth)
- OIDC issuer URL to the Keycloak realm; client registration.
- Gateway forward-auth / ext-authz wiring (WS-NET); group-based authorization; MFA enforced in
Keycloak upstream.
The **authenticator** (oauth2-proxy) is deployed and wired to Keycloak: `provider=oidc`,
`oidc-issuer-url=.../realms/homelab`, `--reverse-proxy`, `--upstream=static://202`,
`--set-xauthrequest`, cookie/redirect domain `.home.keithjackson.dev`, and — added here —
`--allowed-groups=platform-admins` (R-IDP-06 floor) + `--skip-provider-button` (302 straight to
Keycloak). The Keycloak `oauth2-proxy` client already carries the `auth.home…/oauth2/callback`
redirect URI and a `groups` claim with `full.path=false` (bare `platform-admins`) — no realm change
needed.
**Mechanism.** Forward-auth is an Envoy `ext_authz` HTTP filter that calls oauth2-proxy's
`/oauth2/auth` decision endpoint: **202** for a valid session in `platform-admins` (request passes to
the upstream, `X-Auth-Request-*` identity headers forwarded), **401/403** otherwise. With
`failure_mode_allow: false` the gateway **fails closed** — a denied or errored check never reaches the
upstream UI (R-IDP-02). The filter + oauth2-proxy cluster live in
[`forward-auth-cec.yaml`](./forward-auth-cec.yaml).
### Why this is NOT wired into the shared Gateway (Cilium 1.16.5 constraint)
The shared Gateway's Envoy program is a **machine-owned, constantly-regenerated**
`CiliumEnvoyConfig` (`gateway/cilium-gateway-shared`, `ownerReferences->Gateway`, regenerated on every
HTTPRoute change). Cilium 1.16.5 exposes **no supported hook** to inject an `ext_authz` filter into
that generated listener, and native Gateway-API **ExternalAuth (GEP-1494) ships in no released Cilium
version yet** (PoC only). So there is **no clean per-route ext_authz on the shared Gateway today**.
**Options (PM decision):**
| # | Approach | Pros | Cons |
|---|---|---|---|
| A | **Interim self-authored CEC** — a dedicated ext_authz HTTPS listener on its own L2 VIP (`forward-auth-cec.yaml`), protected hosts moved off the shared Gateway and DNS repointed. | Works on 1.16.5 **today**; per-route opt-in; reuses the wildcard TLS via SDS; **verified below**. | Hand-authored TLS+routes; a 2nd ingress VIP + DNS; per-app maintenance; not Gateway-API-native. |
| B | **oauth2-proxy in reverse-proxy mode per app** — app HTTPRoute `backendRef` -> a per-app oauth2-proxy (`--upstream=http://app.svc`). | Fully Gateway-API-native (just a backendRef swap); robust; no CEC. | One oauth2-proxy instance/config per app; more pods; not the single-proxy ext_authz model. |
| C | **Upgrade Cilium** to a release with native `ExternalAuth` (GEP-1494) and gate routes with an HTTPRoute filter. | Cleanest, truly per-route, Crafty-friendly. | Not released yet; large jump from 1.16.5; must keep the Ansible-bootstrap/Flux **parity contract** in lock-step. |
**Recommendation:** ship the oauth2-proxy hardening now (done); adopt **A** as the interim enforcement
for the sensitive UIs when the Phase-2 L2 VIP pool lands; track **C** as the long-term target and drop
A once released.
### Opt-in (per route)
A route is protected **only** by being listed as a virtualHost on the ext_authz listener — apps not
listed keep hitting the shared Gateway. To protect an app (pihole today; **Crafty next**): add its
Service + EDS cluster + virtualHost to `forward-auth-cec.yaml`, **remove its HTTPRoute from the shared
Gateway**, and repoint its DNS at the ext_authz VIP. The ServiceDefinition already declares
`auth.mode: forward-auth` + `groups: [platform-admins]` (rendered onto the HTTPRoute annotations) — the
opt-in signal. Ready to extend to Crafty and the other non-OIDC UIs (PiHole/*arr/qBittorrent/Homepage).
### Proof (in-cluster, cilium-envoy v1.30.8, 2026-08-28)
A throwaway standalone `cilium-envoy` pod ran the exact `ext_authz` stanza from `forward-auth-cec.yaml`
against the live oauth2-proxy Service, with a no-filter **control** route on the same backend:
- **Control** (`:8081`, no ext_authz) -> **200** `CONTROL-OK`.
- **Protected** (`:8080`, ext_authz), unauthenticated -> **403**; body is oauth2-proxy's sign-in page,
**not** the upstream (`UPSTREAM-OK` never served). oauth2-proxy's own access log recorded the
`/oauth2/auth` call from the envoy pod. `failure_mode_allow: false` confirmed fail-closed.
This proves the R-IDP-02 property (unauthenticated cannot reach the upstream) on this exact Envoy. The
**allow path** (valid `platform-admins` session -> 202 -> upstream) needs a real browser round-trip.
### Browser test (for the human, once `forward-auth-cec.yaml` is deployed)
1. Incognito -> `https://pihole.home.keithjackson.dev` -> expect a 302 to
`https://keycloak.home.keithjackson.dev/realms/homelab/...` (not the PiHole UI).
2. Log in as a **platform-admins** user (MFA/OTP enforced) -> land back on the PiHole UI.
3. Log in as a **non-platform-admins** user -> expect **403** (authenticated but not authorized).
4. Confirm the session cookie is `.home.keithjackson.dev`-scoped (single sign-on across the other
protected UIs).
platform/oauth2-proxy/forward-auth-cec.yaml
+170
View File
@@ -0,0 +1,170 @@
---
# =============================================================================
# forward-auth-cec.yaml — INTERIM ext_authz enforcement for Cilium 1.16.5
# Serves: R-IDP-02 (no management UI unauthenticated), R-IDP-06 (group authz).
# Owner: WS-NET (+ WS-SEC). Status: TEMPLATE — NOT wired into kustomization.yaml.
# =============================================================================
#
# WHY THIS IS A STANDALONE CEC (and not an edit to the shared Gateway):
# On Cilium 1.16.5 the shared Gateway's Envoy program is a MACHINE-OWNED,
# constantly-regenerated CiliumEnvoyConfig (`gateway/cilium-gateway-shared`,
# ownerReferences->Gateway, regenerated on EVERY HTTPRoute change). There is
# no supported hook to inject an `ext_authz` HTTP filter into that generated
# listener, and native Gateway-API ExternalAuth (GEP-1494) ships in NO
# released Cilium version yet (PoC only). So ext_authz forward-auth on 1.16.5
# must be a SEPARATELY-authored Envoy listener. See README.md "Enforcement".
#
# WHAT IT DOES (VERIFIED in-cluster against cilium-envoy v1.30.8, 2026-08-28):
# An `envoy.filters.http.ext_authz` HTTP filter calls oauth2-proxy's
# /oauth2/auth decision endpoint. oauth2-proxy returns 202 for a valid
# Keycloak session in `platform-admins` (request passes to the upstream) or
# 401/403 otherwise (`failure_mode_allow: false` => Envoy denies and returns
# oauth2-proxy's sign-in/redirect to the browser — the request NEVER reaches
# the upstream). Proven: unauthenticated => oauth2-proxy 403 sign-in returned,
# upstream body ("UPSTREAM-OK") never served; a no-filter control route on the
# same backend returned 200.
#
# OPT-IN MODEL (per-route, low blast radius):
# A route is protected ONLY by being listed as a virtualHost on THIS listener.
# Apps NOT listed here keep hitting the shared Gateway unprotected. To protect
# an app (e.g. pihole below; next: Crafty):
# 1. Add its Service to `spec.backendServices` and a matching EDS Cluster.
# 2. Add its virtualHost to the `forward-auth` RouteConfiguration.
# 3. REMOVE its HTTPRoute from the shared Gateway (else it stays reachable
# unauthenticated there) and point its DNS at THIS listener's VIP.
# The app's ServiceDefinition already declares `auth.mode: forward-auth` +
# `groups: [platform-admins]`; the HTTPRoute carries those as annotations
# (homelab.keithjackson.dev/auth-mode,/auth-groups) — the opt-in signal.
#
# DEPLOY (PM step — NOT automatic on merge):
# * Allocate a dedicated L2 VIP (CiliumLoadBalancerIPPool / the Phase-2 pool)
# and a LoadBalancer Service that this CEC binds via `spec.services`
# (`oauth2-protected-ingress` below). TLS reuses the SAME wildcard cert the
# shared Gateway already syncs to SDS: `cilium-secrets/gateway-wildcard-home-tls`.
# * Repoint the protected hostnames' DNS (ExternalDNS/pihole) at that VIP.
# * `kubectl apply -f forward-auth-cec.yaml` (or add to kustomization.yaml).
# -----------------------------------------------------------------------------
apiVersion: cilium.io/v2
kind: CiliumClusterwideEnvoyConfig
metadata:
name: oauth2-forward-auth
labels:
app.kubernetes.io/name: oauth2-proxy
app.kubernetes.io/component: forward-auth-enforcement
spec:
# Services Cilium must resolve to EDS endpoints for the clusters below.
# oauth2-proxy = the ext_authz backend; the rest = protected upstreams.
backendServices:
- name: oauth2-proxy
namespace: oauth2-proxy
number: ["80"]
- name: pihole # PROOF target (docs/13 row 6, auth.mode=forward-auth)
namespace: pihole
number: ["80"]
# Bind this listener to a dedicated LoadBalancer Service (its own L2 VIP).
# The Service is a PM deploy step (see header) so this CEC stays inert on merge.
services:
- name: oauth2-protected-ingress
namespace: gateway
listener: oauth2-forward-auth-listener
ports: [443]
resources:
# --- Listener: TLS-terminated HTTPS with the ext_authz HTTP filter ---------
- "@type": type.googleapis.com/envoy.config.listener.v3.Listener
name: oauth2-forward-auth-listener
listenerFilters:
- name: envoy.filters.listener.tls_inspector
typedConfig:
"@type": type.googleapis.com/envoy.extensions.filters.listener.tls_inspector.v3.TlsInspector
filterChains:
- filterChainMatch:
transportProtocol: tls
serverNames:
- "*.home.keithjackson.dev"
transportSocket:
name: envoy.transport_sockets.tls
typedConfig:
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
commonTlsContext:
tlsCertificateSdsSecretConfigs:
# SAME wildcard cert the shared Gateway already syncs to SDS.
- name: cilium-secrets/gateway-wildcard-home-tls
filters:
- name: envoy.filters.network.http_connection_manager
typedConfig:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
statPrefix: forward-auth
useRemoteAddress: true
upgradeConfigs:
- upgradeType: websocket
rds:
routeConfigName: forward-auth
httpFilters:
# ---- THE ENFORCEMENT FILTER (verified stanza) --------------
- name: envoy.filters.http.ext_authz
typedConfig:
"@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthz
transportApiVersion: V3
# HARD FAIL CLOSED: if oauth2-proxy is down or errors, DENY
# (never fail open onto an unauthenticated UI — R-IDP-02).
failureModeAllow: false
httpService:
serverUri:
uri: http://oauth2-proxy.oauth2-proxy.svc.cluster.local
cluster: oauth2-proxy:oauth2-proxy:80
timeout: 5s
# Prepended to the request path -> /oauth2/auth<path>.
# Per-route group differentiation: rely on oauth2-proxy's
# global `--allowed-groups=platform-admins` floor
# (helmrelease.yaml). A future non-admin group would use a
# dedicated listener rather than a query on path_prefix.
pathPrefix: /oauth2/auth
authorizationRequest:
allowedHeaders:
patterns:
- exact: cookie # the oauth2-proxy session
- exact: authorization
- exact: x-forwarded-uri
- exact: x-forwarded-host
- exact: x-forwarded-proto
authorizationResponse:
# On ALLOW (202): forward identity to the upstream app.
allowedUpstreamHeaders:
patterns:
- prefix: x-auth-request-
- exact: authorization
# On DENY: return the sign-in redirect/cookie to browser.
allowedClientHeaders:
patterns:
- exact: set-cookie
- exact: location
- exact: www-authenticate
- name: envoy.filters.http.router
typedConfig:
"@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
# --- Routes: one virtualHost per PROTECTED host (add Crafty etc. here) ------
- "@type": type.googleapis.com/envoy.config.route.v3.RouteConfiguration
name: forward-auth
virtualHosts:
- name: pihole
domains:
- pihole.home.keithjackson.dev
- pihole.home.keithjackson.dev:*
routes:
- match:
prefix: /
route:
cluster: pihole:pihole:80
# --- Clusters ---------------------------------------------------------------
- "@type": type.googleapis.com/envoy.config.cluster.v3.Cluster
name: oauth2-proxy:oauth2-proxy:80 # the ext_authz backend
type: EDS
connectTimeout: 5s
edsClusterConfig:
serviceName: oauth2-proxy/oauth2-proxy:80
- "@type": type.googleapis.com/envoy.config.cluster.v3.Cluster
name: pihole:pihole:80 # the protected upstream
type: EDS
connectTimeout: 5s
edsClusterConfig:
serviceName: pihole/pihole:80
platform/oauth2-proxy/helmrelease.yaml
+21 -5
View File
@@ -46,13 +46,29 @@ spec:
cookie-domain: ".home.keithjackson.dev"
cookie-secure: "true"
# Expose group membership so per-service group authorization (R-IDP-06) can be enforced. The
# baseline here is "authenticated"; per-service `--allowed-groups` is layered on at the Gateway
# ext-auth attachment (WS-NET contract) or via a dedicated oauth2-proxy instance per group.
# Keycloak `oauth2-proxy` client emits a `groups` claim (full.path=false => bare names like
# `platform-admins`, keycloak-realm-import.yaml). `--allowed-groups` below is the PLATFORM FLOOR:
# a valid Keycloak session is NOT enough — the user must ALSO be in `platform-admins`, or
# /oauth2/auth returns 403 and ext_authz denies (R-IDP-06). Every currently-onboarded
# forward-auth app (pihole, *arr, qbittorrent, homepage, prometheus, alertmanager, hubble)
# declares `groups: [platform-admins]` in its ServiceDefinition, so this floor matches them all.
# Per-ROUTE group differentiation (a future app allowing a different group) does NOT need a
# second proxy: the ext_authz filter appends `?allowed_groups=<group>` to /oauth2/auth from the
# HTTPRoute's `homelab.keithjackson.dev/auth-groups` annotation — oauth2-proxy honours the
# per-request query on top of this floor. See forward-auth-cec.yaml + README "opt-in".
oidc-groups-claim: groups
allowed-groups: platform-admins # R-IDP-06 floor: authenticated AND in platform-admins
# Single Keycloak provider => skip the oauth2-proxy interstitial "Sign In" button and 302
# straight to Keycloak. Verified in-cluster: WITHOUT this, an ext_authz-denied browser request
# gets oauth2-proxy's HTML sign-in page (HTTP 403) instead of an immediate redirect (docs/15 UX).
skip-provider-button: "true"
scope: "openid email profile"
# Metrics for the ServiceMonitor (../oauth2-proxy servicemonitor in platform-config).
metrics:
enabled: true
# TODO(WS-SEC/WS-NET, Phase 6): the Gateway ext-authz ATTACHMENT (which HTTPRoutes require this
# proxy, and each route's allowed groups) is a WS-NET Gateway contract — Cilium Gateway API
# ext-auth wiring. oauth2-proxy is the authenticator; the per-route enforcement is Gateway-side.
# ENFORCEMENT (Phase 6): oauth2-proxy is the AUTHENTICATOR; the per-route ext_authz ENFORCEMENT is
# Gateway-side. On Cilium 1.16.5 the shared Gateway's Envoy config is a machine-owned, constantly
# regenerated CiliumEnvoyConfig (`gateway/cilium-gateway-shared`) that CANNOT be injected into,
# and native Gateway-API ExternalAuth (GEP-1494) ships in NO released Cilium version yet. The
# enforcement mechanism + the three wiring options are documented in README.md; the interim
# self-authored ext_authz listener is forward-auth-cec.yaml (a template, not yet Flux-wired).