jakman85andClaude Opus 5
9a4489b733
Every oauth2-proxy on this platform is an independent instance with its OWN
cookie-secret. Measured on prod 2026-09-30: seven `*-oauth2-proxy` Secrets, seven
DISTINCT cookie-secret values (compared in-cluster, never printed). So no instance
can DECRYPT another's cookie -- it can only OVERWRITE it.
All of them were emitting oauth2-proxy's DEFAULT names (`_oauth2_proxy`,
`_oauth2_proxy_csrf`) scoped to `.${BASE_DOMAIN}`, so the browser kept ONE cookie
pair for SIX live instances (jackett, pihole, qbittorrent, radarr, sonarr, and the
shared forward-auth proxy). Whichever authenticated last clobbered the rest; the
loser logged `No valid authentication in request`, bounced to Keycloak, and -- the
SSO session still being live -- was redirected straight back and clobbered it again.
The symptom is "I signed into Keycloak but I'm still not signed in", which sends you
hunting in Keycloak, where nothing is wrong. Crafty was fixed on its own side on
2026-09-29 (#22); this is that fix generalized to the pattern.
THE DEFECT WAS NOT IN THE MANIFESTS. apps/wave-*/{jackett,pihole,qbittorrent,radarr,
sonarr}.yaml are GENERATED goldens; one function emits all five. So the fix is in
catalog/render.py::_oauth2_sidecar_container(), and both flags are now DERIVED from
the ServiceDefinition rather than defaulted:
--cookie-name=_oauth2_proxy_{s['name']} unique by construction
--cookie-domain={host} the app's own host, not the zone
A per-service table was rejected for the reason twelve ServiceMonitors once acquired
one identical wrong port name in a single edit: it would leave the NEXT app onboarded
defaulting straight back into the collision. --whitelist-domain stays zone-wide; it
governs post-login REDIRECT targets, not cookie scope.
Host-scoping is defence in depth and is asserted deliberately: a zone-wide cookie is
transmitted by the browser to every host in the zone, so each app's session token was
being sent to every sibling -- and it is what would make a FUTURE name collision
platform-wide instead of contained.
New gate check (whole-repo scope, so it runs on EVERY gate invocation, not only when
a touched path happens to attribute an identity requirement):
conformance/tests/l0-static/oauth2-proxy-cookie-names-distinct/
It resolves each instance's EFFECTIVE cookie name -- declared, or oauth2-proxy's
built-in default when omitted, because an omitted flag is not an absence but a CLAIM
on a name, and reading it as an absence is exactly why this survived five reviews.
Then: no two collide; none is another's name + `_csrf` (oauth2-proxy derives the CSRF
cookie that way); the default has exactly ONE owner, registered with its reason in a
shrink-only register -- the SHARED proxy, which MUST keep `_oauth2_proxy` because
renaming it signs every other gated surface out at once; every other instance is
host-scoped; render.py derives rather than lists; and the SHIPPED renderer is proven
end-to-end on two synthetic ServiceDefinitions, so the check stays falsifiable once
the tree is clean.
Tested in both directions: rc 0 on this tree (39 assertions, 7 instances), rc 1 with
26 failures against the pre-fix tree, plus one targeted mutation per assertion class
(duplicate name, CSRF derivation, zone-wide domain, unregistered default heir, stale
register entry) -- each fired, and nothing else did.
Also regenerates the three L1 rendering goldens, which are expected-output fixtures
of the same renderer.
UNVERIFIED: no browser round-trip. The collision has still never been REPRODUCED in a
browser -- doing so needs two gated apps visited in one session -- so this is reasoned
from measured configuration and from the mechanism, not from a failure anyone watched.
Nothing was applied; merging rolls five Deployments and invalidates every existing
session on them once. See decisions/2026-09-30-oauth2-proxy-cookie-collision-fleet.md.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
153 lines
11 KiB
YAML
153 lines
11 KiB
YAML
# =============================================================================
|
|
# L0 STATIC — no two oauth2-proxy instances can share a browser cookie
|
|
# Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster.
|
|
#
|
|
# THE DEFECT THIS CLOSES. Every oauth2-proxy on this platform is an INDEPENDENT instance with
|
|
# its OWN `cookie-secret`: five per-app reverse-proxy sidecars (jackett, pihole, qbittorrent,
|
|
# radarr, sonarr), crafty's, and the SHARED forward-auth proxy. Measured on prod 2026-09-30:
|
|
# seven `*-oauth2-proxy` Secrets, SEVEN DISTINCT cookie-secret values — so no instance can
|
|
# decrypt any other's cookie.
|
|
#
|
|
# They were all emitting oauth2-proxy's DEFAULT cookie names (`_oauth2_proxy` and
|
|
# `_oauth2_proxy_csrf`), scoped to `.${BASE_DOMAIN}`. A cookie is keyed by (name, domain, path),
|
|
# so the browser kept ONE pair for ALL of them. Whichever proxy authenticated last OVERWROTE
|
|
# every other's session; the loser logged `No valid authentication in request`, bounced to
|
|
# Keycloak, and — the SSO session still being live — was redirected straight back and overwrote
|
|
# the cookie again. Six live instances, one cookie, mutual eviction.
|
|
#
|
|
# WHY IT IS THE WORST KIND OF BUG. The symptom is "I signed into Keycloak but I'm still not
|
|
# signed in", which points the investigator at Keycloak, where nothing is wrong. It is
|
|
# INTERMITTENT — it only appears once a browser has visited a SECOND gated app, so the obvious
|
|
# single-app reproduction succeeds — and it is SILENT: every Deployment stays Ready, every
|
|
# Kustomization stays Ready, the proxy's own log line names an authentication failure rather
|
|
# than a collision. It went unnoticed across five services for the whole life of the pattern,
|
|
# and was found only as a side effect of investigating something else (2026-09-29, crafty).
|
|
#
|
|
# WHY A COMMIT IS NOT ENOUGH, AND THIS IS A GATE STAGE. The collision is a property of the
|
|
# PATTERN, not of any one app: the next forward-auth service onboarded inherits the defect by
|
|
# doing nothing at all, because a missing flag IS the bug. Nothing about its absence is loud.
|
|
# This is the same shape as the twelve inert ServiceMonitors in this tree — one omission,
|
|
# copied, invisible for weeks — and the same remedy: make the omission fail the merge.
|
|
#
|
|
# GIVEN every oauth2-proxy instance this repo authors — a container in any manifest (the
|
|
# rendered sidecars in apps/, crafty's hand-authored one in platform/) and the shared
|
|
# proxy's HelmRelease values,
|
|
# WHEN each instance's EFFECTIVE cookie name is resolved (its declared `--cookie-name` /
|
|
# `OAUTH2_PROXY_COOKIE_NAME`, or oauth2-proxy's built-in default `_oauth2_proxy` when it
|
|
# declares none — an omission is a NAME, not an absence, and that is the whole bug),
|
|
# THEN (1) no two instances share an effective cookie name;
|
|
# (2) no instance's effective name is another's name + `_csrf` — oauth2-proxy derives the
|
|
# CSRF cookie that way, so `_oauth2_proxy` and `_oauth2_proxy_csrf` as two SESSION
|
|
# names would collide on the second cookie while looking distinct on the first;
|
|
# (3) the DEFAULT name has exactly ONE owner and it is the instance declared in
|
|
# SHARED_DEFAULT_COOKIE_OWNER. This is the deliberate ASYMMETRY. The shared
|
|
# forward-auth proxy MUST keep `_oauth2_proxy`: it is the single session every
|
|
# OTHER forward-auth-gated surface on the platform rides on, so renaming it signs
|
|
# every one of them out at once. The rule is therefore NOT "every instance must
|
|
# declare a name" — it is "the default is a NAME LIKE ANY OTHER, held by exactly one
|
|
# REGISTERED owner". That formulation needs no exemption from (1), and it closes the
|
|
# hole an exemption would leave: if the shared proxy is ever removed, an app silently
|
|
# inheriting the default would still FAIL here instead of quietly becoming its heir;
|
|
# (4) every instance that is NOT that owner is HOST-SCOPED: its `--cookie-domain` equals
|
|
# the host of its own `--redirect-url`. See the argument below;
|
|
# (5) catalog/render.py DERIVES both flags from the ServiceDefinition (the service name
|
|
# and the route host) and does not carry a per-service table. A list is how twelve
|
|
# ServiceMonitors acquired one identical wrong port name in a single edit, and a list
|
|
# leaves the NEXT app defaulting into the collision;
|
|
# (6) the SHIPPED renderer is proven END-TO-END on synthetic ServiceDefinitions, run as a
|
|
# subprocess: two forward-auth fixtures must yield two different, non-default cookie
|
|
# names, each scoped to its own host. Without this, once the tree is compliant (1)-(4)
|
|
# are satisfied by a tree with nothing left to catch, and a guard whose only evidence
|
|
# is "nothing failed" cannot be told apart from one that stopped working;
|
|
# AND (7) the check is LOAD-BEARING: this Test DECLARES `conformance.home.k3s/scope:
|
|
# whole-repo`, so ci/l0.sh Stage 3b's `--include-scope whole-repo` unions it into
|
|
# EVERY gate run, and ci/gate.sh blocks the merge when the l0 stage fails. It
|
|
# deliberately adds NO stage to ci/l0.sh — tests/l0-static/whole-repo-invariants-wired
|
|
# fails the gate if anyone adds one.
|
|
#
|
|
# WHY SCOPE, NOT REQUIREMENTS: ci/touched-requirements.sh maps `apps/*` to R-IDP-02/R-IDP-06
|
|
# today, but the instances live in THREE unrelated subtrees (apps/, platform/crafty/,
|
|
# platform/oauth2-proxy/) and the file most likely to reintroduce the defect is
|
|
# catalog/render.py, which maps to R-GITOPS-02 and to no identity requirement at all. A
|
|
# requirement-scoped selection would filter this check out of exactly the change that breaks it.
|
|
#
|
|
# ON ASSERTION (4), HOST-SCOPED COOKIE-DOMAIN — the argument, and the decision.
|
|
# AGAINST asserting it (overreach): distinct NAMES already close the eviction bug completely.
|
|
# Two cookies with different names do not overwrite each other however wide their domains are,
|
|
# so (4) fixes nothing that (1) has not already fixed. `--cookie-domain` is also a legitimate
|
|
# knob — a future proxy fronting two hostnames of one app would need a wider value — and a
|
|
# check that fails correct configurations is a check someone eventually disables.
|
|
# FOR asserting it (and this wins): a zone-wide cookie is TRANSMITTED BY THE BROWSER TO EVERY
|
|
# HOST IN THE ZONE. qBittorrent's session token is therefore sent to pihole, to radarr, to
|
|
# sonarr, and to every other name under `.${BASE_DOMAIN}` — including hosts behind no proxy at
|
|
# all. That is a session-credential disclosure across trust boundaries that no one needs and
|
|
# nobody asked for; the sidecar's session is only ever meaningful to the one app it fronts.
|
|
# And it is what makes a FUTURE name collision catastrophic rather than contained: with
|
|
# host-scoped cookies, two instances that accidentally share a name cannot even see each
|
|
# other's cookie, so the failure this whole test exists to prevent degrades from platform-wide
|
|
# to nothing. Defence in depth on the exact axis that already failed once.
|
|
# DECISION: assert it, for every instance except the registered shared proxy — whose whole
|
|
# job IS cross-host forward-auth, and which is exempted by name and with a reason, not by
|
|
# loosening the rule. `--whitelist-domain` is deliberately NOT touched: it governs permitted
|
|
# post-login REDIRECT targets, not cookie scope, and narrowing it breaks the sign-in bounce.
|
|
#
|
|
# STATUS: EXECUTABLE. `check.sh` is an offline assertion script (PyYAML, as catalog/render.py
|
|
# and the sibling l0-static checks use); conformance/run.sh --layer l0 and ci/l0.sh run it
|
|
# directly, and this Chainsaw Test wraps the SAME script.
|
|
#
|
|
# NOT proven (honest gaps; this is STATIC, with no cluster and no browser):
|
|
# - That anyone can actually sign in. No browser round-trip is performed, and none can be
|
|
# from L0. The 2026-09-29 crafty entry recorded the same gap and it is still open: the
|
|
# symptom has never been REPRODUCED in a browser, so this is a latent-bug fix reasoned from
|
|
# measured configuration, not a fix watched to work.
|
|
# - That the LIVE pods carry what the tree says. git is not the cluster (CLAUDE.md §2b). The
|
|
# live flags were measured separately, once, by hand; nothing here re-measures them, and a
|
|
# hand-applied drift would not be visible to this check.
|
|
# - That the cookie-secrets really differ. That is a Secret comparison and belongs in-cluster;
|
|
# it was measured out of band (seven distinct values, compared without printing any of
|
|
# them). If they were ever made IDENTICAL the collision would become harmless — but sharing
|
|
# one session-encryption key across trust zones is a worse idea than the bug it would mask,
|
|
# so this check does not offer it as an alternative and does not look for it.
|
|
# - Anything about the CONTENT of a session, group authorization, or the CSRF cookie's own
|
|
# behaviour. Names and domains only.
|
|
# - It does not read ../home-network-k3s. The spec is read-only (R-TEST-06).
|
|
# =============================================================================
|
|
apiVersion: chainsaw.kyverno.io/v1alpha1
|
|
kind: Test
|
|
metadata:
|
|
name: oauth2-proxy-cookie-names-distinct
|
|
labels:
|
|
conformance.home.k3s/R-IDP-02: "true"
|
|
conformance.home.k3s/R-IDP-06: "true"
|
|
annotations:
|
|
conformance.home.k3s/requirements: "R-IDP-02, R-IDP-06"
|
|
conformance.home.k3s/layer: "L0"
|
|
# WHOLE-REPO SCOPE — the instances span apps/wave-0, apps/wave-1, platform/crafty and
|
|
# platform/oauth2-proxy, and the generator that emits five of them is catalog/render.py.
|
|
# No single requirement path-maps to that set, and the renderer maps to no identity
|
|
# requirement at all, so a requirement-scoped selection would skip this check on precisely
|
|
# the change that can reintroduce the defect fleet-wide in one edit.
|
|
conformance.home.k3s/scope: "whole-repo"
|
|
conformance.home.k3s/safety-critical: "false"
|
|
conformance.home.k3s/executable: "true"
|
|
conformance.home.k3s/intent: >-
|
|
No two oauth2-proxy instances can share a browser cookie: every instance's effective
|
|
cookie name is unique, the built-in default belongs to exactly one registered owner (the
|
|
shared forward-auth proxy, which cannot be renamed without signing every gated app out),
|
|
every other instance is host-scoped, and catalog/render.py derives both flags from the
|
|
ServiceDefinition rather than from a per-service list — proven end-to-end on synthetic
|
|
fixtures so the guard stays falsifiable once the tree is clean.
|
|
spec:
|
|
steps:
|
|
- name: assert-no-two-oauth2-proxy-instances-share-a-cookie
|
|
description: >-
|
|
Inventory every oauth2-proxy container and HelmRelease in the tree, resolve each one's
|
|
effective cookie name (declared, or oauth2-proxy's default when omitted), assert
|
|
pairwise distinctness including the session/CSRF derivation, ratchet the single
|
|
registered owner of the default name, assert host-scoped cookie-domain on every other
|
|
instance, assert catalog/render.py derives rather than lists, and prove the shipped
|
|
renderer on two synthetic forward-auth ServiceDefinitions.
|
|
try:
|
|
- script:
|
|
content: bash ./check.sh
|