fix(dns): add resilient fallback resolver to k3s nodes (PiHole primary + pfSense fallback)

Root cause: the k3s nodes shipped with a SINGLE upstream resolver, PiHole
(192.168.1.18), set by cloud-init/netplan. PiHole answers the local jackson.lan
zone instantly but its external/recursive resolution flaps — verified 0/25 for
public names (registry-1.docker.io, api.cloudflare.com, quay.io, ghcr.io,
google.com) while gitea.jackson.lan resolved 5/5. A single upstream is a hard
SPOF: both containerd image pulls ("lookup registry-1.docker.io: Try again")
and CoreDNS ("forward . /etc/resolv.conf" -> pods' external DNS, e.g.
external-dns -> Cloudflare) fail whenever PiHole recursion is down. CoreDNS
logs were flooded with "read udp ...->192.168.1.18:53: i/o timeout".

Fix (additive, no policy bypass): keep PiHole PRIMARY (preserves jackson.lan +
network ad-blocking) and ADD the pfSense gateway (192.168.1.1) as a FALLBACK,
which resolves the public zones reliably (verified). The resolver fails over to
the fallback when PiHole times out.

- node_baseline: netplan drop-in (99-k3s-dns.yaml) with both nameservers +
  "Apply netplan" handler; new node_baseline_dns_* defaults; tagged `dns`.
  Applied to all 3 nodes via playbooks/node-baseline.yml --tags dns.
- tofu: initialization.dns.servers now lists [dns_server, network_gateway] so a
  fresh cloud-init/netplan build stays in parity with the drop-in.

Verified: all external names resolve cluster-wide (10/10, zero timeouts) from
multiple pods after a CoreDNS restart; CoreDNS i/o-timeout floods stopped;
split-horizon (keycloak.home.keithjackson.dev -> Gateway VIP) intact.

NOTE: a SEPARATE, pre-existing WAN-egress outage (ICMP + TCP 443 to the internet
time out from every node while the LAN/pfSense resolver work) still blocks image
pulls, external-dns->Cloudflare, and Flux remote bases. That is a pfSense
NAT/firewall/WAN problem outside this repo, flagged for PM follow-up.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

.gitea/workflows/gate.yml

@@ -1,27 +1,19 @@
 # .gitea/workflows/gate.yml — the ENFORCED PR merge gate (Phase 6, WS-GITOSS).
 #
 # This is the LIVE gate wired to the in-cluster Gitea Actions runner (ns gitea-actions,
-# Docker executor + DinD). It invokes the AGGREGATE merge gate — `make -C ci gate` — so CI
+# Docker executor + DinD). It runs the STATIC layers of the conformance gate — the ones
-# runs BYTE-FOR-BYTE the same command a human runs in the devcontainer (R-TEST-02): the
+# docs/14 says run "without a cluster": spec-immutability (best-effort), lint, Kyverno L0,
-# runner just installs the pinned tools from ci/versions.env first. `make -C ci gate`
+# the render==golden reproducibility check, and L1 offline render checks. It is the single
-# (ci/gate.sh) runs every stage in order — spec-immutability, lint, Kyverno L0,
+# required status check on `main`
-# render==golden, L1 offline render — and applies the R-TEST-03 decision, printing
+# (see branch protection). `make -C ci lint l0 render-golden l1` here is byte-identical to a laptop run
-# "MERGE GATE: PASS" / "MERGE GATE: BLOCKED". It is the single required status check on
+# (R-TEST-02) — the runner just installs the pinned tools from ci/versions.env first.
-# `main` (see branch protection). A non-zero exit fails the job, so a BLOCKED gate blocks
-# the merge.
 #
-# The cluster-gated L2/L3 conformance stage SKIPS honestly here (exit 3, surfaced not faked
+# The L2/L3 CONFORMANCE layer (behavioural + fault-injection, needs a live cluster) is
-# — HARD RULE 4) because this runner declares no cluster: `make -C ci gate` does not set
+# NOT wired yet — it is a deliberate design decision (a CI toolchain image + a scoped
-# CLUSTER_AVAILABLE, so gate.sh reports the touched requirements UNVERIFIED without blocking
+# in-cluster kubeconfig/RBAC for the runner). It is present below as an `if: false`
-# Phase-0/6 static work. L2/L3 run as an IN-CLUSTER Chainsaw Job (its own scoped SA; CI holds
+# placeholder so the shape is visible and the wiring is a one-line flip once decided.
-# no broad kubeconfig) driven by the separate `conformance` job below.
+# The full pipeline blueprint (spec-checkout, per-layer jobs, decision job) is retained
-#
+# dormant in merge-gate.yaml.
-# NOTE (R-TEST-06 / gate-semantics): spec-immutability is ALSO surfaced as its own explicit
-# early step (below) so its result is a distinct, tolerant status in the Gitea UI; gate.sh
-# re-runs it as its first stage, which is the authoritative one. `make -C ci gate` here is
-# the byte-identical laptop gate; the extra pre-step is a read-only git-diff check and is
-# purely additive. The full pipeline blueprint (spec-checkout, per-layer jobs, decision job)
-# is retained dormant in merge-gate.yaml.
 ---
 name: gate
 
@@ -55,27 +47,19 @@ jobs:
           ./.gitea/workflows/scripts/install-tools.sh l0
           ./.gitea/workflows/scripts/install-tools.sh l1
 
-      # R-TEST-06: the spec is read-only to agents. An EXPLICIT early surfacing of the
+      # R-TEST-06: the spec is read-only to agents. The spec repo is NOT checked out into
-      # spec-immutability check, so its result is a distinct, tolerant status line in the
+      # CI here (it is a separate repo; SPEC_REPO_SLUG is not wired into this runner), so
-      # Gitea UI before the (longer) aggregate gate log. The spec repo is NOT checked out
+      # the guard honestly SKIPS (exit 3) rather than faking green. A real drift/edit of a
-      # into CI here (separate repo; SPEC_REPO_SLUG not wired into this runner), so the guard
+      # co-located spec would exit 1 and fail. Tolerated: exit 3 only.
-      # honestly SKIPS (exit 3) rather than faking green; a real drift/edit of a co-located
+      # Call the script DIRECTLY, not via `make`: GNU make masks a recipe's exit code as
-      # spec exits 1 and fails. Tolerated: exit 3 only. Call the script DIRECTLY, not via
+      # its own 2 on failure, so `make spec-immutability || [ $? -eq 3 ]` would see 2 and
-      # `make spec-immutability`: GNU make masks a recipe's exit code as its own 2, so
+      # fail. The script itself exits 3 on an honest skip (spec repo not checked out here),
-      # `make spec-immutability || [ $? -eq 3 ]` would see 2 and fail. The AUTHORITATIVE
+      # which we tolerate; a real drift/edit exits 1 and fails the gate.
-      # spec-immutability run is the FIRST stage of `make -C ci gate` below (gate.sh calls
-      # the script directly, so it sees exit 3 → SKIP, not a mask); this pre-step is a cheap,
-      # read-only git-diff duplicate that only adds early UI signal.
       - name: Spec immutability (R-TEST-06, skips if spec repo absent)
         run: ./ci/spec-immutability.sh || [ "$?" -eq 3 ]
 
-      # Aggregate merge gate: byte-for-byte the command a human runs in the devcontainer
+      - name: Static gate — lint + Kyverno L0 + render==golden + L1 render
-      # (`make -C ci gate` -> ci/gate.sh). Runs spec-immutability, lint, Kyverno L0,
+        run: make -C ci lint l0 render-golden l1
-      # render==golden and L1 render, then the R-TEST-03 decision. Exits non-zero on a
-      # BLOCKED gate, which fails this job. The cluster-gated L2/L3 stage SKIPS here (no
-      # CLUSTER_AVAILABLE) — surfaced as UNVERIFIED, never faked green (see header).
-      - name: Merge gate — make -C ci gate
-        run: make -C ci gate
 
       # NOTE: no actions/upload-artifact step. Gitea Actions rejects upload-artifact@v4
       # ("not supported on GHES" — Gitea does not implement the artifact-v2 backend), which
@@ -84,16 +68,15 @@ jobs:
       # Gitea-compatible @v3 action or Gitea's own artifact backend if retention is wanted.
 
   # ───────────────────────────────────────────────────────────────────────────
-  # PLACEHOLDER — NOT WIRED (deferred; concrete plan written, awaiting operator decision).
+  # PLACEHOLDER — NOT WIRED (STOP-and-report item, WS-GITOSS Phase 6).
-  # L2/L3 conformance (Chainsaw behavioural + fault-injection) needs a LIVE cluster and is
+  # L2/L3 conformance (Chainsaw behavioural + fault-injection) needs a LIVE cluster.
-  # gated behind vars.CLUSTER_AVAILABLE + a narrow secrets.CONFORMANCE_TRIGGER_KUBECONFIG.
+  # Turning this on is a design decision with two open sub-decisions:
-  # The intended model: an in-cluster Chainsaw Job (conformance/incluster/) under its OWN
+  #   1. Toolchain: build+push a pinned CI image to the Gitea container registry and
-  # scoped ServiceAccount — CI holds NO broad kubeconfig, only a trigger credential. The
+  #      reference it as the job image, vs. install-in-job from ci/versions.env (slower).
-  # blocker is a real privilege-escalation design question (a `create jobs` credential can
+  #   2. Cluster creds: how the runner reaches the cluster — the intended model is an
-  # create a Job running as the privileged runner SA), plus a runner image + repo-clone +
+  #      in-cluster Chainsaw Job driven by its OWN scoped ServiceAccount (CI holds only a
-  # results-collection that don't exist yet. Full analysis + step sequence:
+  #      narrow trigger credential; the runner mounts NO SA token today). That SA + RBAC
-  #   docs/plans/l2-l3-conformance-ci-wiring.md
+  #      + the trigger secret do not exist yet.
-  # The blueprint job shape is retained dormant in merge-gate.yaml.
   # Kept `if: false` so it never runs (shows skipped, never fakes green — HARD RULE 4).
   # ───────────────────────────────────────────────────────────────────────────
   conformance:

.gitignore

@@ -41,6 +41,3 @@ infra/tofu/.r2.env
 # ignoring it keeps `git add -A` from staging embedded worktree repos and keeps the lint
 # stage (which scans untracked files) from tripping on .claude/worktrees/.
 .claude/
-
-# Vendored ansible collections (installed by ansible-galaxy for node re-provision; not tracked or linted)
-infra/ansible/collections/

DECISIONS-LOG.md

@@ -1150,349 +1150,3 @@ gitignored (agent worktrees were tripping the lint stage's untracked-file scan).
    Plex uses `/config` → Plex writes the ephemeral layer, is unclaimed, loses config on restart
    (seerr↔plex connectivity is actually fine — 200 on `/identity`). FIX PENDING operator go (resets
    Plex, needs re-claim): mount PVC at `/config` + wire `PLEX_CLAIM`.
-
----
-
-## 2026-08-30 — Upgrade campaign landed: cluster now k3s v1.36.4 / Debian 13 / Cilium 1.20.1 / Longhorn 1.12.1
-
-The platform-wide upgrade campaign completed and is live. Verified current state of the dev cluster:
-
-- **k3s v1.36.4+k3s1** (was v1.31.5+k3s1), 3-server embedded-etcd HA.
-- **Debian 13 (trixie)** node OS (was Debian 12 bookworm).
-- **Cilium 1.20.1** (was 1.16.5); **Gateway API v1.6.1** (was v1.2.x).
-- **Longhorn 1.12.1** (was 1.7.3).
-- cert-manager >=1.21; Velero chart 12.x / Velero v1.18; kube-prometheus-stack 88 (operator
-  v0.93.1, Grafana 13.2.0); kube-vip v1.2.3.
-- Node RAM **24 GiB / 24576 MB** (tofu default; earlier docs said 16 GiB).
-- Each node now carries a **dedicated virtio2 SSD etcd disk** (`/dev/vdc`, `local-zfs`) isolating
-  etcd WAL fsync from HDD Longhorn I/O — see `docs/plans/ssd-etcd-isolation.md`.
-
-**The "LIVE STATE VERIFIED 2026-08-26" line above (k3s v1.31.5) is SUPERSEDED by this entry.**
-That older line is retained as historical record — this append is the current source of truth.
-Documentation and comments across the repo were reconciled to these facts in the same change;
-toolchain kubectl bumped 1.31.4 → 1.36.4 (the 1.31 pin exceeded the ±1 version skew against a 1.36
-cluster). Revalidation items deliberately left open: external-snapshotter v8.2.0 pin vs 1.36, the
-Cilium ext_authz/forward-auth workaround relevance under 1.20.1, and the conformance ImageVolume
-feature-gate assertions under 1.36.
-
-## 2026-08-31 — Post-upgrade stabilization: SSO fixed, all UIs gated, 8 vCPU, CI live. **PM HANDOFF.**
-
-Everything the 08-30 campaign deferred is resolved; the platform is fully green. **This entry is the
-current source of truth and the handoff for the next PM.** (Session ran on a different workstation; a
-fresh agent should read this entry + `infra/PHASE1-STATUS.md` + `docs/plans/` and start at Next Steps.)
-
-### LIVE STATE (verified 2026-08-31, all green)
-- 3× k3s **v1.36.4+k3s1**, Debian 13, **8 vCPU / 24 GiB** each (raised from 6 — see #5). Proxmox host 24 cores, ~idle.
-- Cilium 1.20.1 (GW-API v1.6.1), Longhorn 1.12.1, cert-manager 1.21, Velero chart 12/v1.18, kube-prometheus-stack 88
-  (Grafana 13.2), kube-vip v1.2.3, **Loki chart 7.x / v3.6.11**, **Kyverno chart 3.9 / v1.19** (the last two were the
-  08-30 "deferred" items — now done). etcd on the dedicated SSD virtio2 disk per node.
-- All Flux kustomizations + HelmReleases Ready; 0 not-ready pods.
-- **SSO end-to-end**: native OIDC for grafana/gitea/nextcloud/semaphore/headlamp; the 5 forward-auth UIs
-  (pihole/jackett/qbittorrent/radarr/sonarr) gated by per-app oauth2-proxy sidecars (fail-closed, 302→Keycloak).
-  Platform-admins require TOTP MFA.
-- **Gitea Actions CI is LIVE + enforcing**: `.gitea/workflows/gate.yml` runs `make -C ci gate` on push/PR to main;
-  required status check; runner `incluster-dind` online (ns gitea-actions).
-
-### What changed this session (durable)
-1. **SSO cluster-wide break→fix — root cause Cilium CVE-2025-30162.** The 1.16→1.20 security fix makes the shared
-   Gateway L7 proxy (`enforce_policy_on_l7lb`) re-check a pod's egress policy against Keycloak's REAL backend
-   identity on the split-horizon hairpin; `toEntities: world` no longer covers it → all app SSO 403'd
-   ("Access denied", `server: envoy`). Fix: explicit `toEndpoints` egress allow to the keycloak backend —
-   `catalog/render.py` emits it for `auth.mode ∈ {oidc,forward-auth}`; headlamp/crafty hand-written. (68690e5, 8a424bc)
-2. **Keycloak MFA config bug**: browser-mfa conditional-OTP used a non-existent config key → "Invalid role name: null"
-   → 403 for every platform-admin login. Fixed to `condUserRole: platform-admin` in git + live via kcadm. (45deac9)
-3. **VPN egress restored**: the Debian 12→13 re-provision wiped k3s-dev-3's node-host OpenVPN (PIA); re-ran
-   `vpn.egress.gateway.yml`. Re-provision runbook (infra/ansible/README.md) updated — node-host roles (k3s, kube-vip,
-   vpn.egress.gateway) must be re-run after any reprovision.
-4. **Grafana OOM** (kps 65→88 ships Grafana 13) — the hidden blocker that stalled the whole kps upgrade + platform
-   tier. limits 256→512→768Mi; Prometheus 2→3Gi. (b2b1aa5 and follow-ups)
-5. **Node vCPU 6→8** — steady CPU PSI was 15-23% / load at-or-over core count (no spike headroom); now PSI ~4.5-7.7%.
-   bpg reboots the VM on a cores change → rolled ONE node at a time, waiting Longhorn healthy between. (676c37a)
-   + node CPU PSI/load alerts + a "Nodes — CPU/Memory Pressure" Grafana dashboard. (a720c73)
-6. **Security: 5 forward-auth UIs gated** with per-app oauth2-proxy sidecars (Option B, crafty's proven pattern) +
-   per-app Keycloak clients + SOPS creds. Closes the un-gated-admin-UI exposure. (72d4720, b2752ae)
-7. **Gitea Actions CI finished**: `ci/touched-requirements.sh` now excludes comment-only R-ID prose, so the workflow
-   runs the exact aggregate `make -C ci gate`. (27bc024)
-8. **external-snapshotter vendored** — was remote github kustomize bases; the kustomize-controller's per-reconcile
-   `git fetch github.com` flaked past its 27s timeout → `BuildFailed` stalled the platform tier. Now local
-   `crds.yaml` + `snapshot-controller.yaml` (offline, deterministic build). (7a79cea)
-9. **08-30 revalidations closed**: ImageVolume gate is now ON in 1.36 → conformance rationale comments corrected
-   (d60e553); external-snapshotter v8.2.0 pin kept (CRDs served at v1, works on 1.36); Cilium native ExternalAuth
-   evaluated → **DEFER** (see plan).
-
-### Operational gotchas that STILL matter (for the next PM)
-- **Access**: `./infra/devcontainer/dev.sh run <cmd>` (kubectl/flux/helm/tofu/ansible/kcadm). Do NOT run `dev.sh`/`make
-  gate` concurrently — execs into the one container interfere → transient failures, just retry. Deploy = `git push
-  gitea main` → Flux. Gate before push: `dev.sh run make -C ci gate` → "MERGE GATE: PASS".
-- **NAT topology** (see PHASE1-STATUS): API via `192.168.2.50:6443`→VIP .1.49; node SSH `192.168.2.102:225X`;
-  Proxmox API `192.168.2.102:8006`. A single node reboot can briefly drop the kubeconfig NAT — wait ~3 min for boot.
-- **Keycloak realm import is ONE-SHOT**: the operator logs "Job already executed - not recreating" — a git change to
-  `keycloak-realm-import.yaml` does NOT re-import. Edit the live realm via `kcadm.sh` in keycloak-0 (admin creds:
-  secret `keycloak-admin-credentials`; kcadm present, curl/python3 are NOT), AND fix git for reproducibility.
-- **Cilium world-hairpin (CVE-2025-30162)**: any in-cluster→`*.home` call hairpins through the Gateway and gets its
-  egress re-checked against the DESTINATION's real cluster identity — `toEntities: world` won't cover it. render.py
-  handles the keycloak case for SSO apps; a NEW app→app-by-hostname dependency needs an explicit `toEndpoints` allow.
-- **Remote github kustomize bases flake** (27s git-fetch timeout). external-snapshotter is now vendored;
-  **platform/kubevirt still uses remote bases** — vendor it the same way if it `BuildFail`s.
-- **Longhorn on the HDD**: every node reboot degrades all 18 volumes → full rebuild (slow; 30%+ Proxmox IO-delay).
-  Roll reboots one node at a time; wait `robustness=healthy` before the next. Fix = `docs/plans/longhorn-ssd-tiering.md`.
-- **Gitea** can go read-only (ext4 remount-ro) during node churn → `kubectl -n gitea rollout restart deploy/gitea`.
-
-### NEXT STEPS (prioritized) — where to start
-1. **Longhorn → SSD hot-tier move** — `docs/plans/longhorn-ssd-tiering.md`. The "won't fit" blocker was a measurement
-   artifact (working set ~79 GB; SSD ~323 GB free). Moving the hot tier (Prometheus/CNPG DBs/Loki/Gitea ≈ 51 GB) off
-   the HDD removes the reboot-rebuild tax on every future maintenance op. **Highest-leverage infra item.** Pre-flight:
-   re-verify live SSD free + read HDD SMART.
-2. **L2/L3 conformance in CI** — `docs/plans/l2-l3-conformance-ci-wiring.md`. Needs the trigger-credential decision
-   (scoped in-cluster SA + Kyverno-pinned Job shape; never a broad kubeconfig in CI). The gate runs no-cluster stages today.
-3. **Forward-auth long-term** — `docs/plans/cilium-native-externalauth-evaluation.md` (DEFER now: experimental CRD
-   channel + fail-open history). The 5 UIs are safely sidecar-gated in the meantime.
-4. **Remaining Phase-6 onboarding** — see DESIGN-waves.md / `catalog/` (services rendered-but-not-onboarded).
-5. **Vendor platform/kubevirt** remote bases proactively (same flake class as #8).
-6. **Human action**: finish Headlamp login = complete TOTP enrollment in the browser.
-
----
-
-## 2026-08-31 — **PHASE 7 OPENED (prep only)**: first prod node on the bench
-
-The operator added the first **production** node and opened roadmap Phase 7. This entry records the
-opening, the state of the hardware, the operator's three decisions, and the blockers that stand
-between "a node exists" and "prod reconciles from `clusters/prod`".
-
-### Governance — Phase 7 entry gate, recorded not waived
-
-`roadmap/phases.md` gates Phase 7 on **two** conditions: *"Phases 1–6 proven in dev/test; operator
-declares prod hardware ready."* The operator has declared hardware ready. **The first condition is
-not met** — conformance is 37/51, and the uncovered set includes **R-ENV-01/02/03**, the exact
-requirements prod provisioning serves. We proceed with *prep only* on that basis, with no prod
-install until the §7 repo blockers close and the gate is green. This is an accepted, explicit risk,
-not an oversight.
-
-### Node 1 as-found (`192.168.2.10`, verified by SSH 2026-08-31)
-
-Not blank — imaged and reachable. **HP EliteDesk 800 G5 Desktop Mini** (matches the spec's `3× mini-PC
-bare metal`), i7-9700T 8c/8t, **31 GiB RAM**, Debian 13 trixie / kernel 6.12, hostname already
-`k3s-prod-1`, `k3s` user reachable with the `home_network` key, VT-x + `/dev/kvm` present
-(KubeVirt-capable), k3s not installed. Firmware HP R21 Ver. 02.26.00.
-
-**One 477 GB SATA SSD** (`C400-MTFDDAC512M` — a ~2011-era Micron/Crucial consumer drive) carrying
-everything: `/` 30G, `/var` 12G, swap, and `/srv` 433G. No second device. Dev, by contrast, runs two
-dedicated data disks per node (Longhorn + a dedicated SSD for etcd, added after HDD contention
-starved etcd WAL fsync).
-
-### Operator decisions (2026-08-31)
-
-1. **Phase 7 opens, prep work only** — repo work and node baselining now; no k3s install until the
-   parameterization blockers land and the gate is green.
-2. **Wait for all three nodes before `cluster-init`** — prod comes up as spec-conformant 3-node HA
-   embedded etcd on day one; no single-server-etcd window, no later member-add.
-3. **Check drive health before choosing the disk layout** — a 15-year-old consumer SSD is not an
-   obvious home for prod state.
-
-### Findings — prod would build WRONG today
-
-1. **R-ENV-02 is violated in the base (the headline blocker).** The overlay model promises dev and
-   prod differ *only* by `cluster-vars` values. In fact dev LAN addresses are hardcoded into base
-   manifests: `k8sServiceHost: 192.168.1.50` (`platform/cilium/helmrelease.yaml:50` + the
-   `values.yaml` mirror), the Cilium LB pool `192.168.1.53-.60`
-   (`platform-config/cilium/l2-ip-pool.yaml`), crafty `.55`, the UniFi VM `.54`, and three S3 backup
-   endpoints at `192.168.1.9:9000`. **Prod is on `192.168.2.x` — none of these follow.** Fix is to
-   tokenize each into a `cluster-vars` knob and set it in both overlays; the `postBuild.substituteFrom`
-   seam already exists on every relevant Kustomization. Note `platform/cilium/values.yaml` is only a
-   *reference mirror* — the authoritative pair is the HelmRelease (Flux) and the Ansible j2 template
-   (already parameterized), so the Ansible side needs no change.
-2. **`clusters/prod` never sets `GATEWAY_VIP`** → it would inherit the base default `10.73.0.240`,
-   an address on the **retired libvirt network**. `cluster-vars`' own comment flags this as owed
-   "when hardware is built".
-3. **Sudo on node 1 requires a password** — every Ansible playbook fails at its first `become`.
-   **Operator ruling: prod does NOT get NOPASSWD sudo** (my initial recommendation to mirror dev was
-   rejected, correctly — dev's passwordless sudo is a convenience that should not propagate to
-   prod). Instead prod uses Ansible `become` with an **ansible-vault-encrypted**
-   `ansible_become_password` in `inventory/group_vars/k3s_prod/vault.yml`, mirroring the existing
-   `hypervisor` and `proxmox_hosts` groups; the decrypting passphrase stays outside the repo at
-   `~/.config/home-k3s/ansible-vault-pass`. Plays still run unattended, no plaintext secret in git,
-   and prod keeps a real sudo boundary.
-4. **`node_baseline` device defaults are virtio** (`/dev/vdb`, `/dev/vdc`) and do not exist on bare
-   metal; the role makes a filesystem on a *block device*, so a single-disk node needs either a
-   second drive, `/dev/sda5`, or a repartition. Prod `group_vars` must override.
-5. **Single upstream resolver on node 1** (`192.168.2.1`) — the same SPOF class that broke
-   containerd pulls and CoreDNS forwarding on dev. The fix is **stranded unmerged** on
-   `fix/node-dns-resilient-upstream`.
-
-### RESOLVED BY OPERATOR — status comes from the repository, not the task board
-
-I raised an escalation here: `AGENTS.md` §11 designates `agents/task-board.md` as the one surface
-agents may edit, yet `ci/spec-immutability.sh` fails the merge gate on **any** spec-repo dirt with no
-path exception, and committing the board there moves spec `HEAD` off `ci/SPEC_PIN` — which also
-fails. Editing the board as instructed blocks every merge. That is why the board still reads
-all-`todo` after six delivered phases.
-
-**Operator ruling (2026-08-31): the task board is not the status surface. The repository is.**
-Status of each requirement/spec is to be *derived from the state of the repo* — what is authored,
-what is reconciled, what has a green requirement-tagged test — not from a hand-maintained board.
-
-Consequences, adopted:
-- The spec board is left alone. It is a historical seed, not a live surface. No agent edits it, and
-  no `SPEC_PIN` churn is spent keeping it current.
-- `ci/spec-immutability.sh` keeps its full strictness — **no path exception was added**. Weakening
-  an R-TEST-06 gate to make a bookkeeping file writable would have been a defect, not a fix.
-- The **repo-derived spec status report** becomes the artifact of record, alongside the existing
-  requirement-coverage report and this log. Coverage already answers "does a tagged test exist";
-  the status view must also answer "is this requirement actually *implemented and reconciled* in
-  the repo as it stands".
-
-### Next steps
-
-1. **Operator:** configure NOPASSWD sudo on `k3s-prod-1`, then capture `smartctl -a /dev/sda`
-   (commands in `infra/PROD-PROVISIONING.md` §5). The disk decision waits on that output.
-2. **Land the R-ENV-02 tokenization** — the critical path for prod reconciling at all. Dev overlay
-   must set every new knob to today's exact value so live dev sees zero drift (`make -C ci render-golden`).
-3. **Complete `clusters/prod`** — `GATEWAY_VIP`, the L2 pool, and the prod backup target.
-4. **Land the two stranded fix branches** (`fix/node-dns-resilient-upstream`,
-   `fix/act-runner-dind-readiness`) — the DNS one now matters for prod too.
-5. Resolve the task-board escalation above.
-6. Nodes 2 and 3 through `infra/PROD-PROVISIONING.md` §1–§5 as they arrive.
-
----
-
-## 2026-08-31 — CI is the enforcement point; the LOCAL gate produced a FALSE GREEN
-
-**Operator directive: "Conformance and gates are enforced at the gitea runner and CI/CD level.
-Use it."** Adopted. The authority on whether a change is mergeable is the Gitea Actions run, not a
-local `make -C ci gate`. Local runs are a fast pre-check only.
-
-This was proven immediately and expensively. Branch `docs/phase-7-open-prod-node-1`:
-
-- **Local** `make -C ci gate` → `no changed files vs base; empty touched-requirement set` →
-  **MERGE GATE: PASS**.
-- **CI**, same commit (`a1fb31a`, task 200) → 17 touched requirements correctly computed →
-  `REQUIRE: FAILED — required requirement(s) uncovered: ['R-SEC-01','R-SEC-02','R-TEST-06']` →
-  **MERGE GATE: BLOCKED**.
-
-The local run resolved an empty diff base and therefore checked nothing, reporting green. **A local
-false green is the most dangerous failure mode this project has** — it hid real coverage debt from
-the PM and would have hidden it from any future agent. `ci/touched-requirements.sh` base-ref
-resolution is being fixed so local agrees with CI, or announces loudly that it cannot determine a
-base. An honest noisy skip beats a quiet false pass (`ci/README.md` exit-code contract).
-
-The gate itself was RIGHT: we authored a vaulted-secret flow (R-SEC-01) and status tooling that
-reasons about spec immutability (R-TEST-06) while those requirements had **no covering test at
-all**. Tests are being authored rather than the rule being narrowed.
-
-**How to read CI from a workstation** (the API paths are non-obvious, so recording them):
-```bash
-CRED=$(printf 'protocol=https\nhost=gitea.home.keithjackson.dev\n\n' | git credential fill)
-# list recent runs (id | workflow | branch | status | sha):
-curl -sk -u "$U:$P" "$B/api/v1/repos/Home-Network/home-network-k3s-impl/actions/tasks?limit=10"
-# fetch a task's log — NOTE the job-id/task-id OFFSET: task N is at jobs/(N+5)/logs on this instance
-curl -sk -u "$U:$P" "$B/api/v1/repos/Home-Network/home-network-k3s-impl/actions/jobs/205/logs"
-```
-`.../actions/runs/<run>/jobs` is 404 on this Gitea version; use `actions/tasks` + `actions/jobs/<id>/logs`.
-
-### OPERATOR ATTESTATIONS (2026-08-31) — capability vs. automated proof
-
-The operator states, and this log records as fact:
-
-1. **DR is PROVEN** — Velero backups to the Proxmox server, restore demonstrated. R-STATE-03's
-   capability is delivered and in operational use.
-2. **Cluster resilience is PROVEN at the dev-cluster level** — node loss / rebuild demonstrated.
-   It is slow because Longhorn storage is on HDD (etcd is the exception, on its dedicated SSD),
-   which is a performance property, not a correctness gap.
-
-**Distinguish capability from automated proof.** The new repo-derived status report flags
-`R-STATE-03` as a safety-critical requirement whose L3 test body asserts nothing. That finding
-stands and is NOT contradicted by the attestation above — it is **test debt, not capability debt**.
-The system does the thing; the suite does not yet assert that it does. Per the project's own
-definition of done, only a green requirement-tagged test is acceptance — so the remaining work is
-to make `dr-restore/chainsaw-test.yaml` and the node-loss test actually assert the behaviour the
-operator has already demonstrated, and let the runner enforce it. Recording the attestation here
-does not close the requirement, and the gate must not be softened to make it appear closed.
-
----
-
-## 2026-09-01 — Phase-7 prep session complete. **PM HANDOFF.**
-
-Everything below is on branch `docs/phase-7-open-prod-node-1` / **PR #4**, 13 commits, the
-enforced Gitea gate **green**. Nothing has been installed on prod hardware. `main` is untouched
-until the operator merges.
-
-### Delivered
-1. **Phase 7 opened (prep only)** with the entry-gate shortfall recorded, not waived.
-2. **T-7.1 checklist** (`infra/PROD-PROVISIONING.md`) written against the real hardware.
-3. **R-ENV-02 tokenization** — 6 `cluster-vars` knobs; prod would otherwise have come up
-   announcing DEV addresses. Dev proven byte-identical.
-4. **Prod Ansible**: inventory, vaulted `become` (prod gets NO passwordless sudo), and
-   **target-group parameterisation** so prod reuses the roles unchanged.
-5. **`node_health` role** — read-only, portable dev+prod, run against both.
-6. **Repo-derived spec status report** (`run.sh --status`).
-7. **CI false-green killed** + L0 static tests for R-SEC-01/02 and R-TEST-06.
-8. **dind readiness fix**; **DNS resilience re-authored** for the current stack.
-9. **`dr-restore` now asserts** — safety-critical scaffolds 0.
-
-### Numbers
-Coverage 37/51 → **40/51**. Executable tests 32 → **36**. Scaffold-only 5 → **4**.
-Safety-critical without an executable L3 1 → **0**. **PROVEN: 0 — still not derivable**, by
-design; proof-of-pass is not recoverable from repo state, so a green-run ledger is the only
-thing that ever closes a requirement.
-
-### Corrections made during the session (recorded so they are not re-litigated)
-- **NOPASSWD sudo on prod: REJECTED** by the operator. Prod uses vaulted `become`. My original
-  checklist copied dev's convenience onto prod; that was wrong.
-- **The "`$i` envsubst bug" was FALSE.** An assessment claimed Flux envsubst ate bare `$i`,
-  making the dind cert-wait bound a no-op. Running the real `flux envsubst` disproved it: bare
-  `$VAR` passes through; only `${VAR}` expands — matching this repo's own contract in
-  `clusters/base/apps-vms.yaml`, itself written after empirical probing. **No escaping was
-  applied**; `$${i}` would have been an active regression (correct only on the envsubst path —
-  on a plain `kustomize build`, `sh` reads `$$` as its PID). Agent reports are not evidence;
-  the tool is.
-- **The stranded DNS branch was NOT landed.** Its netplan mechanism no longer exists on Debian
-  13; landing it would have broken `node-baseline.yml` on every node.
-
-### Near-misses worth remembering
-- A YAML **comment** nearly broke every merge in the repo: Flux envsubst runs over comments too,
-  so a braced name with punctuation is a fatal parse error that fails the whole `apps-wave-1`
-  Kustomization.
-- `roles/cilium`, `cluster_verify`, `flux_bootstrap` all hardcoded `.cache/dev.kubeconfig` — the
-  file `dev.sh` exports as `$KUBECONFIG`. The moment prod became targetable, a prod run would
-  have silently overwritten it and left the operator's `kubectl` answering from **prod**.
-- Prod's `GATEWAY_VIP` outside the LB pool would have brought the Gateway up on `.21` while
-  split-horizon DNS answered `.20` — blackholing in-cluster SSO while reporting healthy.
-
-### BLOCKED ON THE OPERATOR — nothing else moves without these
-1. **Create `inventory/group_vars/k3s_prod/vault.yml`** (`ansible_become_password`). The
-   passphrase file already exists. This gates SMART, the packages-only baseline, and every
-   future prod play.
-2. **Drive health decision** — run `smartctl -a /dev/sda` once `smartmontools` lands via
-   `node-baseline.yml -e k3s_hosts=k3s_prod --tags packages`. Prod state on an unmeasured
-   ~2011 consumer SSD is the open risk.
-3. **Prod's second DNS upstream** — `192.168.2.1` is both gateway and only IPv4 resolver.
-4. **Prod backup target** — `S3_BACKUP_ENDPOINT` is still `TBD-PROD-*`; dev's versitygw is on
-   the far side of the pfSense boundary.
-5. **`UNIFI_L4_VIP` / `CRAFTY_L4_VIP`** — allocatable from `.20-.30`, not invented here.
-6. **Merge PR #4.**
-
-### Known, documented, NOT fixed
-- `roles/flux_bootstrap` shells to `bootstrap/flux/bootstrap.sh` with a hardcoded
-  `--path=clusters/dev` — retargeting prod needs a `bootstrap/` change.
-- Prod has no VPN egress group (needs its own vaulted PIA credentials).
-- **`dr-restore` is AUTHORED, NEVER EXECUTED** (`conformance.home.k3s/never-executed`). It is
-  destructive; converting it to proven needs an operator maintenance-window run. The log must
-  show `DR_RESTORED_FROM_OFF_CLUSTER_BACKUP_OK`, **not** `DR_RESTORE_SKIPPED_OK` — green but
-  proving nothing.
-- `dind` → native sidecar `initContainer` with a `docker info` startupProbe (NOT a bare
-  tcpSocket probe) is the right shape on k3s 1.36; deferred so a merge-gate failure is
-  unambiguous.
-- **Non-ECC RAM — RISK ACCEPTED by the operator (2026-09-01), not an open item.** `node_health`
-  correctly reports ECC status as unknowable (no EDAC controller bound). Operator rationale:
-  this is a 3-node k3s homelab, not enterprise k8s; corruption on one node is outweighed by the
-  other two and results in node ejection. That holds for the dominant failure mode — etcd raft
-  (per-entry CRC + boltdb page checksums, quorum of 2 survives) and Longhorn (3 replicas, hard
-  anti-affinity, rebuild from healthy peers) both detect a *divergent* node and eject/rebuild it.
-  The residual it does not cover is corruption introduced *upstream* of replication: bytes
-  corrupted before the checksum is computed replicate faithfully to all three replicas, so
-  quorum sees no disagreement. Low probability, and the mitigation (ECC hardware) is out of
-  scope for this platform. **Do not re-litigate.** The one thing that operationalises the
-  operator's own argument is Longhorn's `snapshotDataIntegrity`, which is currently left at the
-  chart default rather than set explicitly — see the follow-up in `platform/longhorn/`.
-- L2/L3 conformance still SKIPs in CI (`docs/plans/l2-l3-conformance-ci-wiring.md`); it needs
-  the trigger-credential decision — a scoped in-cluster SA, never a broad kubeconfig in CI.

apps/vms/unifi/30-virtualmachine.yaml

@@ -89,13 +89,7 @@ spec:
             # userDataSecretRef only if needed). qemu-guest-agent gives the KubeVirt AgentConnected
             # signal (proves cloud-init ran); install-uos resolves the latest UOS linux-x64 build from
             # Ubiquiti's PUBLIC firmware API (no auth) and installs it non-interactively (UI on :11443).
-            # NOTE: apps-vms NOW carries postBuild.substituteFrom cluster-vars (needed for
+            # NOTE: apps-vms has NO postBuild.substituteFrom, so shell ${...} below is NOT envsubst'd.
-            # ${UNIFI_L4_VIP} in 50-service-l4.yaml), so Flux envsubst DOES run over this userData.
-            # Every shell brace-expansion below is therefore escaped as $${...}, which envsubst emits
-            # as a literal ${...} — verified with `flux envsubst`. Bare $VAR and $(...) are left alone
-            # by envsubst and need no escaping. NEVER write an unescaped ${...} here, and never write
-            # ${ARR[0]} even escaped-adjacent: envsubst hard-fails on `[` ("missing closing brace")
-            # and would break the whole apps-vms apply.
             userData: |-
               #cloud-config
               hostname: unifi
@@ -109,11 +103,11 @@ spec:
                   content: |
                     #!/usr/bin/env bash
                     set -euo pipefail
-                    if [ -z "$${UOS_URL:-}" ]; then
+                    if [ -z "${UOS_URL:-}" ]; then
                       UOS_URL="$(curl -fsSL https://fw-update.ui.com/api/firmware-latest | jq -r '._embedded.firmware|map(select(.product=="unifi-os-server" and .platform=="linux-x64"))|sort_by(.version_major,.version_minor,.version_patch)|last._links.data.href')"
                     fi
-                    [ -n "$${UOS_URL}" ] && [ "$${UOS_URL}" != "null" ] || { echo "no UOS URL"; exit 1; }
+                    [ -n "${UOS_URL}" ] && [ "${UOS_URL}" != "null" ] || { echo "no UOS URL"; exit 1; }
-                    t="$(mktemp)"; echo "downloading $${UOS_URL}"; curl -fL --retry 5 --retry-delay 10 -o "$t" "$UOS_URL"; chmod +x "$t"; echo "installing"; yes | "$t"; echo "done"
+                    t="$(mktemp)"; echo "downloading ${UOS_URL}"; curl -fL --retry 5 --retry-delay 10 -o "$t" "$UOS_URL"; chmod +x "$t"; echo "installing"; yes | "$t"; echo "done"
               runcmd:
                 - [ systemctl, enable, --now, qemu-guest-agent ]
                 - [ systemctl, enable, --now, podman.socket ]

apps/vms/unifi/50-service-l4.yaml

@@ -8,9 +8,8 @@
 # CANNOT express: raw TCP on non-HTTP ports (8080 inform, 6789), UDP at all (3478 STUN, 10001
 # discovery), or the device-facing protocols that never speak HTTP. UniFi device ADOPTION rides
 # exactly those L4 ports. So the controller is exposed as a type=LoadBalancer Service whose VIP
-# is pulled from the CiliumLoadBalancerIPPool `default-pool` (dev 192.168.1.53-.60 — the
+# is pulled from the CiliumLoadBalancerIPPool `default-pool` (192.168.1.53-.60) and ARP-announced
-# CILIUM_LB_POOL_START/STOP cluster-vars knobs) and ARP-announced on the flat LAN by the existing
+# on the flat LAN by the existing CiliumL2AnnouncementPolicy `default-l2` — the SAME L2/LB-IPAM
-# CiliumL2AnnouncementPolicy `default-l2` — the SAME L2/LB-IPAM
 # machinery the Gateway ingress VIP already uses (platform-config/cilium/*). No new Cilium config
 # is required; these Services just consume the pool.
 #
@@ -24,11 +23,8 @@
 # into the guest. externalTrafficPolicy: Local preserves the LAN client source IP and keeps the
 # VIP pinned to the node running the VM (also what L2 announcement does).
 #
-# VIP: ${UNIFI_L4_VIP} (cluster-vars, R-ENV-02; dev 192.168.1.54, confirmed free, inside
+# FLAG: confirm 192.168.1.54 is free, inside default-pool, and reserved in tofu (the Gateway
-# default-pool and reserved in tofu — the Gateway already holds the pool's first IP, dev .53).
+# already holds one pool IP, typically .53). Adjust if it collides.
-# Substituted by the `apps-vms` Flux Kustomization, which carries postBuild.substituteFrom for
-# exactly this token; every shell ${...} in 30-virtualmachine.yaml's cloud-init is escaped as $${...}
-# accordingly. A new environment MUST set UNIFI_L4_VIP to a free address in ITS pool.
 #
 # REUSABLE PATTERN (the deferred `minecraft` :25565/L4 problem is the same shape): expose an
 # L4-only workload as a Cilium LoadBalancer on a pinned, L2-announced VIP with a per-app sharing
@@ -43,8 +39,8 @@ metadata:
     app.kubernetes.io/name: unifi
   annotations:
     lbipam.cilium.io/sharing-key: "unifi"
-    lbipam.cilium.io/ips: "${UNIFI_L4_VIP}"
+    lbipam.cilium.io/ips: "192.168.1.54"
-    # external-dns (source: service) publishes A unifi.home.keithjackson.dev -> the pinned VIP.
+    # external-dns (source: service) publishes A unifi.home.keithjackson.dev -> 192.168.1.54.
     # This is the browser + device target; UI is https://unifi.home.keithjackson.dev:11443.
     external-dns.alpha.kubernetes.io/hostname: "unifi.home.keithjackson.dev"
 spec:
@@ -69,7 +65,7 @@ metadata:
     app.kubernetes.io/name: unifi
   annotations:
     lbipam.cilium.io/sharing-key: "unifi"     # same key + same IP => shares the VIP with -tcp
-    lbipam.cilium.io/ips: "${UNIFI_L4_VIP}"
+    lbipam.cilium.io/ips: "192.168.1.54"
 spec:
   type: LoadBalancer
   externalTrafficPolicy: Local

apps/wave-0/gitea.yaml

@@ -185,10 +185,6 @@ spec:
   egress:
   - toEntities:
     - world
-  - toEndpoints:
-    - matchLabels:
-        io.kubernetes.pod.namespace: keycloak
-        app: keycloak
 ---
 # HTTPRoute gitea-3000 (R-TLS/R-DNS)
 apiVersion: gateway.networking.k8s.io/v1

apps/wave-0/jackett-oauth2-proxy-credentials.sops.yaml

@@ -1,30 +0,0 @@
-apiVersion: v1
-kind: Secret
-metadata:
-    name: jackett-oauth2-proxy
-    namespace: jackett
-type: Opaque
-stringData:
-    client-id: ENC[AES256_GCM,data:n5h6NG0emA==,iv:d6lvZfgRq6+IKNpsSoRa5a38R2f2+tWBO+j9QKDThMY=,tag:P+cvQXlWR4aZs7a9/lVuMQ==,type:str]
-    client-secret: ENC[AES256_GCM,data:WL5G7PuVTdHQoe7Zp2eiE9ppxSGE0+W2RMfISuwqD6JA9auG97ZYnPUjG8HNx7sosOvsmqeE9kONK23IqIeaQw==,iv:kSP10UKeDv/8BsgX36+uKwEY+LBovcicce1RkD14M4g=,tag:fEUsVuajWPxAmAo8Nnrrrg==,type:str]
-    cookie-secret: ENC[AES256_GCM,data:WChW9YLhzG9Nwp8dCWTFtirufXxGk3Ep9HXpfqcNz69PXQW3RaMC4EJpw2U=,iv:f6obIerkog8Q2gfj1tVL259LOl5jj59VE29KnJwmCXQ=,tag:s0FwNtU6vffKgDcGUNUhCg==,type:str]
-sops:
-    kms: []
-    gcp_kms: []
-    azure_kv: []
-    hc_vault: []
-    age:
-        - recipient: age175l8mdpdfu7hq0aml3exs3gykarwjmuvr23sq56s3wjejnyk44wsp4mfzg
-          enc: |
-            -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBLWVptVHc4aFl0MXRWTHVP
-            cC9qcVoyZUV2TGZnTEpHY3RLOU8vQnNNNm00ClZRejE2bzFaWXNSUmQ5MEZtV3VK
-            djB2TXJqQjVUOE93QjN6U0lMVDBEbm8KLS0tIFFTZnB0c2pCZWZMZ084WEIxT2tW
-            WUxxemJHdWRZbnBpL280dzlpMmhwSW8Kra0bkx3iJYQi+3rnpU/89fJbJHzQjKi2
-            ZWNwYhLpULw1xNo3xbCm+PUp5f30kpyMKQld2qaLOZAqWI5vNl9Byw==
-            -----END AGE ENCRYPTED FILE-----
-    lastmodified: "2026-08-31T13:41:37Z"
-    mac: ENC[AES256_GCM,data:zU9nWCrSomAXAZ/Ve8U3yLHnoxhu8nkQ7523neJTGv810ONzAAZ7Gr8nFHR7bTqz2QwuGYD3P1ypdqYnKLM8ri2Zf1LDo/Su9ptnvQhUbfXKlRorcW6uAWsViCwvTM05wZNkdzZfza7tGwJQVTD6hc0cP+/7Y7JnpT5FEstK254=,iv:oggTRjcWqAJizA/a22+mtslkEWka8hUG0wudM8obO4g=,tag:i0TtNmqGe+b5t4iq8LpsqA==,type:str]
-    pgp: []
-    encrypted_regex: ^(data|stringData)$
-    version: 3.9.4

apps/wave-0/jackett.yaml

@@ -84,70 +84,6 @@ spec:
         volumeMounts:
         - name: config
           mountPath: /data/config
-      - name: oauth2-proxy
-        image: quay.io/oauth2-proxy/oauth2-proxy:v7.8.1
-        imagePullPolicy: IfNotPresent
-        args:
-        - --http-address=0.0.0.0:4180
-        - --reverse-proxy=true
-        - --provider=oidc
-        - --oidc-issuer-url=https://keycloak.home.keithjackson.dev/realms/homelab
-        - --upstream=http://127.0.0.1:9117
-        - --email-domain=*
-        - --scope=openid email profile
-        - --oidc-groups-claim=groups
-        - --allowed-group=media-users
-        - --skip-provider-button=true
-        - --redirect-url=https://jackett.home.keithjackson.dev/oauth2/callback
-        - --cookie-domain=.home.keithjackson.dev
-        - --whitelist-domain=.home.keithjackson.dev
-        - --cookie-secure=true
-        - --pass-access-token=true
-        - --set-xauthrequest=true
-        env:
-        - name: OAUTH2_PROXY_CLIENT_ID
-          valueFrom:
-            secretKeyRef:
-              name: jackett-oauth2-proxy
-              key: client-id
-        - name: OAUTH2_PROXY_CLIENT_SECRET
-          valueFrom:
-            secretKeyRef:
-              name: jackett-oauth2-proxy
-              key: client-secret
-        - name: OAUTH2_PROXY_COOKIE_SECRET
-          valueFrom:
-            secretKeyRef:
-              name: jackett-oauth2-proxy
-              key: cookie-secret
-        ports:
-        - name: http
-          containerPort: 4180
-          protocol: TCP
-        resources:
-          requests:
-            cpu: 25m
-            memory: 64Mi
-          limits:
-            memory: 128Mi
-        securityContext:
-          allowPrivilegeEscalation: false
-          readOnlyRootFilesystem: true
-          capabilities:
-            drop:
-            - ALL
-        livenessProbe:
-          httpGet:
-            path: /ping
-            port: http
-          initialDelaySeconds: 5
-          periodSeconds: 15
-        readinessProbe:
-          httpGet:
-            path: /ping
-            port: http
-          initialDelaySeconds: 5
-          periodSeconds: 15
       volumes:
       - name: config
         persistentVolumeClaim:
@@ -168,9 +104,6 @@ spec:
   - name: port-9117
     port: 9117
     targetPort: 9117
-  - name: authproxy
-    port: 4180
-    targetPort: 4180
 ---
 # CiliumNetworkPolicy reciprocal ingress allow (R-NET-02): jackett-allow-radarr-9117
 apiVersion: cilium.io/v2
@@ -233,7 +166,7 @@ spec:
     - ingress
     toPorts:
     - ports:
-      - port: '4180'
+      - port: '9117'
         protocol: TCP
 ---
 # CiliumNetworkPolicy world egress allow (egress class governs the path; R-NET-05): jackett-allow-internet-egress
@@ -251,10 +184,6 @@ spec:
   egress:
   - toEntities:
     - world
-  - toEndpoints:
-    - matchLabels:
-        io.kubernetes.pod.namespace: keycloak
-        app: keycloak
 ---
 # HTTPRoute jackett-9117 (R-TLS/R-DNS)
 apiVersion: gateway.networking.k8s.io/v1
@@ -281,7 +210,7 @@ spec:
         value: /
     backendRefs:
     - name: jackett
-      port: 4180
+      port: 9117
 ---
 # ServiceMonitor (R-OBS-01)
 apiVersion: monitoring.coreos.com/v1

apps/wave-0/kustomization.yaml

@@ -8,15 +8,12 @@ kind: Kustomization
 resources:
 - gitea-bootstrap-secret.sops.yaml
 - gitea.yaml
-- jackett-oauth2-proxy-credentials.sops.yaml
 - jackett.yaml
 - nextcloud-db-app.sops.yaml
 - nextcloud-db-backup-credentials.sops.yaml
 - nextcloud-db.yaml
-- pihole-oauth2-proxy-credentials.sops.yaml
 - pihole.yaml
 - plex.yaml
-- qbittorrent-oauth2-proxy-credentials.sops.yaml
 - qbittorrent.yaml
 - semaphore-db-app.sops.yaml
 - semaphore-db-backup-credentials.sops.yaml

apps/wave-0/nextcloud-db.yaml

@@ -74,13 +74,11 @@ spec:
   # (defaults to the Cluster name), so objects land under s3://cnpg/nextcloud-db/. Path-style HTTP.
   # Credentials: the scoped key from the in-namespace SOPS Secret nextcloud-db-backup-credentials
   # (same-ns only), kept in sync with platform/backup/minio-credentials.sops.yaml.
-  # endpointURL is ${S3_BACKUP_ENDPOINT} (cluster-vars, R-ENV-02; dev http://192.168.1.9:9000),
-  # substituted by the `apps-wave-0` Flux Kustomization. Prod supplies its own off-cluster target.
   backup:
     retentionPolicy: "30d"
     barmanObjectStore:
       destinationPath: s3://cnpg
-      endpointURL: ${S3_BACKUP_ENDPOINT}
+      endpointURL: http://192.168.1.9:9000
       s3Credentials:
         accessKeyId:
           name: nextcloud-db-backup-credentials

apps/wave-0/pihole-oauth2-proxy-credentials.sops.yaml

@@ -1,30 +0,0 @@
-apiVersion: v1
-kind: Secret
-metadata:
-    name: pihole-oauth2-proxy
-    namespace: pihole
-type: Opaque
-stringData:
-    client-id: ENC[AES256_GCM,data:C6IXgQxi,iv:A9bWygwabOKIRi55jV9+TRU7eiZels3vqqLDTZ469aA=,tag:hC7sn/Xr4vQFkRjDLMgEmQ==,type:str]
-    client-secret: ENC[AES256_GCM,data:50HEfMG+CoWXokS+IxQgCPQEt+EBCUontOc0GrTyauHuV/U7w9IPo83iJECO0QhTPiQIgD3j+F+YXfubCbTnuw==,iv:ejQxin0NMs0Ye0fhqfPuh8JmRrEimDrSAcZj28uS+GM=,tag:FVassLZPZMY9ry48ZjBgmw==,type:str]
-    cookie-secret: ENC[AES256_GCM,data:nfTEg5DlIXCLI/hhHngyqLwsQh2SgXU21LaSkVnr91sTi4wODrEH4C7wvPY=,iv:WWXnHnFP2IqLrw1yXAlD0Sw+Vc1m4rxXIHT1u2UWH3o=,tag:pYZEuwPYH+b8S4kGtaWqEA==,type:str]
-sops:
-    kms: []
-    gcp_kms: []
-    azure_kv: []
-    hc_vault: []
-    age:
-        - recipient: age175l8mdpdfu7hq0aml3exs3gykarwjmuvr23sq56s3wjejnyk44wsp4mfzg
-          enc: |
-            -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBuTnJiWTVPR01aY2h3dkJI
-            OTV0WTFQeHoyUGluVzV4NFpSZjd6bjlHZkE0ClZmTGgzQWo4VTROaUg5Wkd1TTVV
-            cGRjZjJuMitPZC9OaVZUZzRGeEJmemcKLS0tIGRjUFd3SzFzeFBBQVRacjZWQmNH
-            ZjllZTQxbzRWQXNiVllkc1hESGN4alEK/hFOtchsYMx09/WE5uaCe6JetUurzhhQ
-            7imb8RIjh0cfP4PDT3dZhhPF+ehcV7ttslFkcyaqlG1F8Gk2jtN0pw==
-            -----END AGE ENCRYPTED FILE-----
-    lastmodified: "2026-08-31T13:33:15Z"
-    mac: ENC[AES256_GCM,data:Tse6fLhc08DxCAuZW5KSZ3M5OQUTlbeHnWiO3jIROUVLrBfbXCuDNWnQyOPjwVPqyzeAzC3APEKFzQ2S8rYjRyO08SJ4ZFPPUeRblcXgXHZ4Y/E4Renq7axk61VvFGtZteHWuQz9ss//Qt+KwPMDBGTivLCoVTOh2K01AXPFXdA=,iv:fyhkDLVN28umXDR4/f4ZzQuu3OG6QBkpUgf3E9rknKk=,tag:iNqor6cD9tJ6MsqAmmRP0Q==,type:str]
-    pgp: []
-    encrypted_regex: ^(data|stringData)$
-    version: 3.9.4

apps/wave-0/pihole.yaml

@@ -84,70 +84,6 @@ spec:
         volumeMounts:
         - name: config
           mountPath: /data/config
-      - name: oauth2-proxy
-        image: quay.io/oauth2-proxy/oauth2-proxy:v7.8.1
-        imagePullPolicy: IfNotPresent
-        args:
-        - --http-address=0.0.0.0:4180
-        - --reverse-proxy=true
-        - --provider=oidc
-        - --oidc-issuer-url=https://keycloak.home.keithjackson.dev/realms/homelab
-        - --upstream=http://127.0.0.1:80
-        - --email-domain=*
-        - --scope=openid email profile
-        - --oidc-groups-claim=groups
-        - --allowed-group=platform-admins
-        - --skip-provider-button=true
-        - --redirect-url=https://pihole.home.keithjackson.dev/oauth2/callback
-        - --cookie-domain=.home.keithjackson.dev
-        - --whitelist-domain=.home.keithjackson.dev
-        - --cookie-secure=true
-        - --pass-access-token=true
-        - --set-xauthrequest=true
-        env:
-        - name: OAUTH2_PROXY_CLIENT_ID
-          valueFrom:
-            secretKeyRef:
-              name: pihole-oauth2-proxy
-              key: client-id
-        - name: OAUTH2_PROXY_CLIENT_SECRET
-          valueFrom:
-            secretKeyRef:
-              name: pihole-oauth2-proxy
-              key: client-secret
-        - name: OAUTH2_PROXY_COOKIE_SECRET
-          valueFrom:
-            secretKeyRef:
-              name: pihole-oauth2-proxy
-              key: cookie-secret
-        ports:
-        - name: http
-          containerPort: 4180
-          protocol: TCP
-        resources:
-          requests:
-            cpu: 25m
-            memory: 64Mi
-          limits:
-            memory: 128Mi
-        securityContext:
-          allowPrivilegeEscalation: false
-          readOnlyRootFilesystem: true
-          capabilities:
-            drop:
-            - ALL
-        livenessProbe:
-          httpGet:
-            path: /ping
-            port: http
-          initialDelaySeconds: 5
-          periodSeconds: 15
-        readinessProbe:
-          httpGet:
-            path: /ping
-            port: http
-          initialDelaySeconds: 5
-          periodSeconds: 15
       volumes:
       - name: config
         persistentVolumeClaim:
@@ -168,9 +104,6 @@ spec:
   - name: port-80
     port: 80
     targetPort: 80
-  - name: authproxy
-    port: 4180
-    targetPort: 4180
 ---
 # CiliumNetworkPolicy gateway-ingress allow (R-NET-01/02): pihole-allow-gateway-ingress
 apiVersion: cilium.io/v2
@@ -189,7 +122,7 @@ spec:
     - ingress
     toPorts:
     - ports:
-      - port: '4180'
+      - port: '80'
         protocol: TCP
 ---
 # CiliumNetworkPolicy world egress allow (egress class governs the path; R-NET-05): pihole-allow-internet-egress
@@ -207,10 +140,6 @@ spec:
   egress:
   - toEntities:
     - world
-  - toEndpoints:
-    - matchLabels:
-        io.kubernetes.pod.namespace: keycloak
-        app: keycloak
 ---
 # HTTPRoute pihole-80 (R-TLS/R-DNS)
 apiVersion: gateway.networking.k8s.io/v1
@@ -237,7 +166,7 @@ spec:
         value: /
     backendRefs:
     - name: pihole
-      port: 4180
+      port: 80
 ---
 # ServiceMonitor (R-OBS-01)
 apiVersion: monitoring.coreos.com/v1

apps/wave-0/qbittorrent-oauth2-proxy-credentials.sops.yaml

@@ -1,30 +0,0 @@
-apiVersion: v1
-kind: Secret
-metadata:
-    name: qbittorrent-oauth2-proxy
-    namespace: qbittorrent
-type: Opaque
-stringData:
-    client-id: ENC[AES256_GCM,data:ytfVRDabEfRM8NI=,iv:ERtFy5iss10id/cGLMZDmIAazkYtFG/spkxRYkoT2pw=,tag:FzOepKeYT05BBrz6RlnACQ==,type:str]
-    client-secret: ENC[AES256_GCM,data:klZzTdyAqQgG+1s074/0Bh5ybuHehiEo/RYceRGAGwuUrgCMb0AmYHCrE9gvtnPkSOBqWvFnPnorWkERc+V73g==,iv:nOUO/p+WnI5657I3Lm6zQAsumkscjF7J0fK0EVFHGo0=,tag:dF3AHYP4sxDChroRer9kMQ==,type:str]
-    cookie-secret: ENC[AES256_GCM,data:jVpsZjYHRrPvblfsb2f9OuneqoZ0cwcTLKrPb8XJ5oves1PmHtN5njrkk4o=,iv:bNjUK/ar4nniR7AjJ9fsGVkKd8nxwphkptmhFaYh9Ek=,tag:2axf+AshSdIvvqwyIfhq6g==,type:str]
-sops:
-    kms: []
-    gcp_kms: []
-    azure_kv: []
-    hc_vault: []
-    age:
-        - recipient: age175l8mdpdfu7hq0aml3exs3gykarwjmuvr23sq56s3wjejnyk44wsp4mfzg
-          enc: |
-            -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAreWdPdUxiYXBxYWxZNlhy
-            WHFTZmo1RW1McXJJTjlWeTBCd2Y4WVU2TVRNCnExbEQrQXZCS2RRaGxJQlQ5eHpY
-            Z1UrZWlubUJpRC9YeEhJZW8zNVdLUVUKLS0tIGtmblFDRzk3ZC8wWVo2ZHJHWmEx
-            OW9DRUpaSlFrbHNDTGYvQUdyamh2bWsK5VvV4cl1iNXYECnSk3fi1ZuAPKGN+n8B
-            g51lT4gi03KA67ARzwVGVBfP5E1Cdmx4rJhi305tG+Ug99VyLBCUfQ==
-            -----END AGE ENCRYPTED FILE-----
-    lastmodified: "2026-08-31T13:42:02Z"
-    mac: ENC[AES256_GCM,data:7GdlUKnZ/ST7EOhfPB6I2e2wyVHWmZ2G554yBcZQzqoZZWZ1hn+lkosNJxWD7E0hRId6P320EKj+Sf0g2ur3Mna5zii1GKFJHGtDEoccKvJQRx2JMkZlEC/x6xwc4BeJNtvidECVRvKC2WdrByRMr9rrI8NfXpmWH4e7iz1Xac0=,iv:UqsNOPRwzT+Dt043ldxIubkAAkPr0/nvjxlgVPiXIbw=,tag:7Wex5Rng3Nwtr8wxkFJD9Q==,type:str]
-    pgp: []
-    encrypted_regex: ^(data|stringData)$
-    version: 3.9.4

apps/wave-0/qbittorrent.yaml

@@ -84,70 +84,6 @@ spec:
         volumeMounts:
         - name: config
           mountPath: /data/config
-      - name: oauth2-proxy
-        image: quay.io/oauth2-proxy/oauth2-proxy:v7.8.1
-        imagePullPolicy: IfNotPresent
-        args:
-        - --http-address=0.0.0.0:4180
-        - --reverse-proxy=true
-        - --provider=oidc
-        - --oidc-issuer-url=https://keycloak.home.keithjackson.dev/realms/homelab
-        - --upstream=http://127.0.0.1:8080
-        - --email-domain=*
-        - --scope=openid email profile
-        - --oidc-groups-claim=groups
-        - --allowed-group=media-users
-        - --skip-provider-button=true
-        - --redirect-url=https://qbittorrent.home.keithjackson.dev/oauth2/callback
-        - --cookie-domain=.home.keithjackson.dev
-        - --whitelist-domain=.home.keithjackson.dev
-        - --cookie-secure=true
-        - --pass-access-token=true
-        - --set-xauthrequest=true
-        env:
-        - name: OAUTH2_PROXY_CLIENT_ID
-          valueFrom:
-            secretKeyRef:
-              name: qbittorrent-oauth2-proxy
-              key: client-id
-        - name: OAUTH2_PROXY_CLIENT_SECRET
-          valueFrom:
-            secretKeyRef:
-              name: qbittorrent-oauth2-proxy
-              key: client-secret
-        - name: OAUTH2_PROXY_COOKIE_SECRET
-          valueFrom:
-            secretKeyRef:
-              name: qbittorrent-oauth2-proxy
-              key: cookie-secret
-        ports:
-        - name: http
-          containerPort: 4180
-          protocol: TCP
-        resources:
-          requests:
-            cpu: 25m
-            memory: 64Mi
-          limits:
-            memory: 128Mi
-        securityContext:
-          allowPrivilegeEscalation: false
-          readOnlyRootFilesystem: true
-          capabilities:
-            drop:
-            - ALL
-        livenessProbe:
-          httpGet:
-            path: /ping
-            port: http
-          initialDelaySeconds: 5
-          periodSeconds: 15
-        readinessProbe:
-          httpGet:
-            path: /ping
-            port: http
-          initialDelaySeconds: 5
-          periodSeconds: 15
       volumes:
       - name: config
         persistentVolumeClaim:
@@ -168,9 +104,6 @@ spec:
   - name: port-8080
     port: 8080
     targetPort: 8080
-  - name: authproxy
-    port: 4180
-    targetPort: 4180
 ---
 # CiliumNetworkPolicy reciprocal ingress allow (R-NET-02): qbittorrent-allow-radarr-8080
 apiVersion: cilium.io/v2
@@ -233,7 +166,7 @@ spec:
     - ingress
     toPorts:
     - ports:
-      - port: '4180'
+      - port: '8080'
         protocol: TCP
 ---
 # CiliumNetworkPolicy world egress allow (egress class governs the path; R-NET-05): qbittorrent-allow-internet-egress
@@ -251,10 +184,6 @@ spec:
   egress:
   - toEntities:
     - world
-  - toEndpoints:
-    - matchLabels:
-        io.kubernetes.pod.namespace: keycloak
-        app: keycloak
 ---
 # HTTPRoute qbittorrent-8080 (R-TLS/R-DNS)
 apiVersion: gateway.networking.k8s.io/v1
@@ -281,7 +210,7 @@ spec:
         value: /
     backendRefs:
     - name: qbittorrent
-      port: 4180
+      port: 8080
 ---
 # ServiceMonitor (R-OBS-01)
 apiVersion: monitoring.coreos.com/v1

apps/wave-0/semaphore-db.yaml

@@ -75,13 +75,11 @@ spec:
   # (defaults to the Cluster name), so objects land under s3://cnpg/semaphore-db/. Path-style HTTP.
   # Credentials: the scoped key from the in-namespace SOPS Secret semaphore-db-backup-credentials
   # (same-ns only), kept in sync with platform/backup/minio-credentials.sops.yaml.
-  # endpointURL is ${S3_BACKUP_ENDPOINT} (cluster-vars, R-ENV-02; dev http://192.168.1.9:9000),
-  # substituted by the `apps-wave-0` Flux Kustomization. Prod supplies its own off-cluster target.
   backup:
     retentionPolicy: "30d"
     barmanObjectStore:
       destinationPath: s3://cnpg
-      endpointURL: ${S3_BACKUP_ENDPOINT}
+      endpointURL: http://192.168.1.9:9000
       s3Credentials:
         accessKeyId:
           name: semaphore-db-backup-credentials

apps/wave-1/gitea-actions.yaml

@@ -103,65 +103,21 @@ spec:
         # ---- act_runner: registers with Gitea, dispatches jobs to the DinD daemon ----
         - name: runner
           image: gitea/act_runner:0.2.11
-          # TWO-STAGE BOUNDED WAIT on the DinD sidecar, then exec the image's normal
+          # Wait for the DinD sidecar to publish its TLS client certs into the shared
-          # entrypoint (/sbin/tini -- /opt/act/run.sh), which registers-from-env on first
+          # emptyDir before starting the daemon — act_runner reads /certs/client/ca.pem at
-          # boot and runs `act_runner daemon`.
+          # startup and hard-exits if it is missing (a startup race against the sidecar).
-          #
+          # Then exec the image's normal entrypoint (/sbin/tini -- /opt/act/run.sh), which
-          # STAGE 1 — the certs must EXIST: act_runner reads /certs/client/ca.pem at startup
+          # registers-from-env on first boot and runs `act_runner daemon`.
-          #   and hard-exits if it is missing (a startup race against the sidecar).
-          # STAGE 2 — the daemon must actually ACCEPT a connection: the cert FILE appearing is
-          #   necessary but NOT sufficient. dockerd writes the certs and only THEN begins
-          #   listening on :2376, so a runner that proceeds on the file alone still hard-exits
-          #   with "cannot ping the docker daemon at tcp://localhost:2376" — observed
-          #   crashlooping 11 times in ~8 minutes on a fresh pod after a node bounce.
-          # Both loops are bounded (60 x 1s) and exit 1 on timeout, so a genuinely dead sidecar
-          # surfaces as a CrashLoopBackOff with a clear message instead of hanging forever.
-          #
-          # ENVSUBST SAFETY — the BARE $i / $j below are CORRECT. Do not "tidy" them into the
-          #   braced form or the double-dollar form; both are wrong here, for different reasons.
-          #   clusters/base/apps-wave-1.yaml carries postBuild.substituteFrom cluster-vars, so
-          #   Flux envsubst runs over this ENTIRE manifest — YAML comments very much included.
-          #   Its contract (drone/envsubst; the same one spelled out in clusters/base/apps-vms.yaml,
-          #   and re-verified against this exact file with `flux envsubst`, flux v2.4.0) is:
-          #     - A bare  $name  is NOT a reference to envsubst: it passes through untouched, and
-          #       that holds even when the name IS set in cluster-vars. So $i and $j reach the
-          #       cluster verbatim and both 60s bounds below are real, enforced bounds.
-          #     - $((i + 1))  is likewise untouched: "(" cannot begin a variable name.
-          #     - A BRACED dollar-name IS a reference and gets replaced — with the empty string
-          #       when the name is unset. One of those here would make a bound read [ "" -ge 60 ],
-          #       which errors, short-circuits the &&, and silently turns the wait UNBOUNDED.
-          #     - A DOUBLE-dollar braced name is envsubst's escape and survives as the braced
-          #       form — still wrong for us, because on any path where envsubst does NOT run
-          #       (a plain `kustomize build` or `kubectl apply -k` of this directory) the shell
-          #       reads that leading double-dollar as its own PID, so a bound would compare a
-          #       string like 1234{i} against 60. The bare form is the only spelling that is
-          #       correct on BOTH paths.
-          #     - A braced name containing punctuation is a HARD ERROR ("unable to parse variable
-          #       name") that fails this whole Kustomization — and this Kustomization gates every
-          #       merge in the repo. So never write one anywhere in this file, INCLUDING in a
-          #       comment. That is why this note names the three forms in words rather than
-          #       showing them: an earlier draft of it spelled them out literally and broke
-          #       `flux envsubst` on the file. (Same trap the indexed-expansion warning in
-          #       clusters/base/apps-vms.yaml describes.)
           command:
             - /bin/sh
             - -c
             - |
-              # Stage 1: the sidecar's TLS client certs must appear in the shared emptyDir.
               i=0
               until [ -f /certs/client/ca.pem ]; do
                 i=$((i + 1))
                 [ "$i" -ge 60 ] && { echo "dind certs never appeared" >&2; exit 1; }
                 echo "waiting for dind TLS certs ($i)"; sleep 1
               done
-              # Stage 2: dockerd must actually accept a connection on :2376. busybox nc in this
-              # Alpine-based image (gitea/act_runner:0.2.11) supports -z (scan) and -w (timeout).
-              j=0
-              until nc -z -w1 127.0.0.1 2376; do
-                j=$((j + 1))
-                [ "$j" -ge 60 ] && { echo "dind daemon never accepted a connection on 2376" >&2; exit 1; }
-                echo "waiting for dind daemon on :2376 ($j)"; sleep 1
-              done
               exec /sbin/tini -- /opt/act/run.sh
           env:
             - name: GITEA_INSTANCE_URL
@@ -199,31 +155,6 @@ spec:
             limits:
               memory: 512Mi
         # ---- Docker-in-Docker: the job executor daemon (privileged) ----
-        #
-        # FOLLOW-UP (deliberately NOT bundled into the readiness fix above — own change):
-        #   Convert this container into a NATIVE SIDECAR and delete both shell wait-loops.
-        #   k3s is now v1.36, where sidecar containers are GA, so kubelet can enforce the
-        #   ordering that the loops above hand-roll — which also removes the envsubst hazard
-        #   from this file entirely (no shell, no $i/$j, nothing to escape or mis-escape).
-        #   Shape to build:
-        #     1. Move this whole container into `initContainers:` as the FIRST entry.
-        #     2. Add `restartPolicy: Always` to it. That single field is what makes an
-        #        initContainer a sidecar: it starts before the regular containers, keeps
-        #        running alongside them, and is terminated after them.
-        #     3. Give it a `startupProbe` that succeeds only when dockerd accepts a TLS
-        #        connection — e.g. `exec` a `docker -H tcp://127.0.0.1:2376 --tlsverify
-        #        --tlscacert /certs/client/ca.pem --tlscert /certs/client/cert.pem
-        #        --tlskey /certs/client/key.pem info`. Do NOT use a bare `tcpSocket` probe:
-        #        it would re-open the same "port accepting but TLS/certs not ready" gap that
-        #        Stage 1 above exists to close. `periodSeconds: 1` + `failureThreshold: 60`
-        #        preserves today's 60s cap.
-        #     4. kubelet then holds the `runner` container until that startupProbe passes, so
-        #        `runner` can drop its `command:` override completely and run the image's
-        #        default entrypoint.
-        #   Why it is a separate change: this Deployment is the ENFORCED MERGE GATE for this
-        #   entire repository. If the conversion is wrong, nothing can merge — so it must land
-        #   alone, where the failure is unambiguous and not entangled with anything else, and
-        #   only after a real workflow run has been observed green on the converted pod.
         - name: dind
           image: docker:27-dind
           securityContext:

apps/wave-1/kustomization.yaml

@@ -10,9 +10,7 @@ resources:
 - gitea-actions.yaml
 - nextcloud-bootstrap-secret.sops.yaml
 - nextcloud.yaml
-- radarr-oauth2-proxy-credentials.sops.yaml
 - radarr.yaml
 - semaphore-bootstrap-secret.sops.yaml
 - semaphore.yaml
-- sonarr-oauth2-proxy-credentials.sops.yaml
 - sonarr.yaml

apps/wave-1/nextcloud.yaml

@@ -205,10 +205,6 @@ spec:
   egress:
   - toEntities:
     - world
-  - toEndpoints:
-    - matchLabels:
-        io.kubernetes.pod.namespace: keycloak
-        app: keycloak
 ---
 # HTTPRoute nextcloud-80 (R-TLS/R-DNS)
 apiVersion: gateway.networking.k8s.io/v1

apps/wave-1/radarr-oauth2-proxy-credentials.sops.yaml

@@ -1,30 +0,0 @@
-apiVersion: v1
-kind: Secret
-metadata:
-    name: radarr-oauth2-proxy
-    namespace: radarr
-type: Opaque
-stringData:
-    client-id: ENC[AES256_GCM,data:vFLxqzxt,iv:6k0ZjKP8rvg56ZFPWEGdyj+FLgK+LBOE0pLBWfC0bjA=,tag:LiZb5BasJAEWM1c3I8vFTA==,type:str]
-    client-secret: ENC[AES256_GCM,data:JzfHF8JF4YUxBxiC738N4UsMlcCOvxn4/AE2B4meZNGAZb2uhG4n51uEY8OQsbsM0Ni4qM2Wi17KuLDwAicfUA==,iv:A3AnUhYxw/IWfvfcZf6MgC5u37dMoNToIWxRG9AxEJQ=,tag:1OY2pWGTbCxPYYdaiBa/oA==,type:str]
-    cookie-secret: ENC[AES256_GCM,data:LPt5nghLfoNvr7yBes8rLeEK3M6UVIO1h9POHMOAj8iZX3l2RJJa4w8GcSg=,iv:Q5+7zH8Tb1SWe92B0pxekfMgOXlBG8nixWn/ho4oJw4=,tag:D+41dBFPVTGj86xv5/GEng==,type:str]
-sops:
-    kms: []
-    gcp_kms: []
-    azure_kv: []
-    hc_vault: []
-    age:
-        - recipient: age175l8mdpdfu7hq0aml3exs3gykarwjmuvr23sq56s3wjejnyk44wsp4mfzg
-          enc: |
-            -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBHVG5MWVRyOVl1RWFpOExC
-            SjZKcEd1ZlFnZVZ5NmtyUlQzRGI1b1o3dG04CkwrTUNjd3crM1VzaUdPczliSDY1
-            a3dONnBFWk8yOW5kL2xES1RZbUN6L3cKLS0tIG9hWG9Cbm01RlMrcC9zMWkxeFgr
-            ZnRGRUVTTS81SFRyenNsZUZEcG1TUm8K5XUGP/1NVsoaFcBZUER3/zMPTtCuVisT
-            LrbSmTkIyVfzjN/vRpRH92b1VzENJvhGhRghhQaKY0rbG34i6yh3Vw==
-            -----END AGE ENCRYPTED FILE-----
-    lastmodified: "2026-08-31T13:42:25Z"
-    mac: ENC[AES256_GCM,data:TxdPz2zE6+ypgQG2b0Rp3VFiPL7LLls4NNkC7YQ/bVCLvyvaXdGsEWkiHJE+Po7hcESmZRt/NMS25mEXPMfvlSCltwFUTcrxoZDS0NI27kIzbtceTbxmznt/dMS/iZm21lCgA6Aag9ByLjWusPLkRKWhep+G+0G54qtfQzv3s80=,iv:44f0KikrpKpnSaLLyzTvnCfrwEXylRgVIlKMtSpMyNc=,tag:jDx43n0usDtbwzo8KzDBqw==,type:str]
-    pgp: []
-    encrypted_regex: ^(data|stringData)$
-    version: 3.9.4

apps/wave-1/radarr.yaml

@@ -84,70 +84,6 @@ spec:
         volumeMounts:
         - name: config
           mountPath: /data/config
-      - name: oauth2-proxy
-        image: quay.io/oauth2-proxy/oauth2-proxy:v7.8.1
-        imagePullPolicy: IfNotPresent
-        args:
-        - --http-address=0.0.0.0:4180
-        - --reverse-proxy=true
-        - --provider=oidc
-        - --oidc-issuer-url=https://keycloak.home.keithjackson.dev/realms/homelab
-        - --upstream=http://127.0.0.1:7878
-        - --email-domain=*
-        - --scope=openid email profile
-        - --oidc-groups-claim=groups
-        - --allowed-group=media-users
-        - --skip-provider-button=true
-        - --redirect-url=https://radarr.home.keithjackson.dev/oauth2/callback
-        - --cookie-domain=.home.keithjackson.dev
-        - --whitelist-domain=.home.keithjackson.dev
-        - --cookie-secure=true
-        - --pass-access-token=true
-        - --set-xauthrequest=true
-        env:
-        - name: OAUTH2_PROXY_CLIENT_ID
-          valueFrom:
-            secretKeyRef:
-              name: radarr-oauth2-proxy
-              key: client-id
-        - name: OAUTH2_PROXY_CLIENT_SECRET
-          valueFrom:
-            secretKeyRef:
-              name: radarr-oauth2-proxy
-              key: client-secret
-        - name: OAUTH2_PROXY_COOKIE_SECRET
-          valueFrom:
-            secretKeyRef:
-              name: radarr-oauth2-proxy
-              key: cookie-secret
-        ports:
-        - name: http
-          containerPort: 4180
-          protocol: TCP
-        resources:
-          requests:
-            cpu: 25m
-            memory: 64Mi
-          limits:
-            memory: 128Mi
-        securityContext:
-          allowPrivilegeEscalation: false
-          readOnlyRootFilesystem: true
-          capabilities:
-            drop:
-            - ALL
-        livenessProbe:
-          httpGet:
-            path: /ping
-            port: http
-          initialDelaySeconds: 5
-          periodSeconds: 15
-        readinessProbe:
-          httpGet:
-            path: /ping
-            port: http
-          initialDelaySeconds: 5
-          periodSeconds: 15
       volumes:
       - name: config
         persistentVolumeClaim:
@@ -168,9 +104,6 @@ spec:
   - name: port-7878
     port: 7878
     targetPort: 7878
-  - name: authproxy
-    port: 4180
-    targetPort: 4180
 ---
 # CiliumNetworkPolicy egress allow (R-NET-02): radarr-allow-qbittorrent-8080
 apiVersion: cilium.io/v2
@@ -255,7 +188,7 @@ spec:
     - ingress
     toPorts:
     - ports:
-      - port: '4180'
+      - port: '7878'
         protocol: TCP
 ---
 # CiliumNetworkPolicy world egress allow (egress class governs the path; R-NET-05): radarr-allow-internet-egress
@@ -273,10 +206,6 @@ spec:
   egress:
   - toEntities:
     - world
-  - toEndpoints:
-    - matchLabels:
-        io.kubernetes.pod.namespace: keycloak
-        app: keycloak
 ---
 # HTTPRoute radarr-7878 (R-TLS/R-DNS)
 apiVersion: gateway.networking.k8s.io/v1
@@ -303,7 +232,7 @@ spec:
         value: /
     backendRefs:
     - name: radarr
-      port: 4180
+      port: 7878
 ---
 # ServiceMonitor (R-OBS-01)
 apiVersion: monitoring.coreos.com/v1

apps/wave-1/semaphore.yaml

@@ -209,10 +209,6 @@ spec:
   egress:
   - toEntities:
     - world
-  - toEndpoints:
-    - matchLabels:
-        io.kubernetes.pod.namespace: keycloak
-        app: keycloak
 ---
 # HTTPRoute semaphore-3000 (R-TLS/R-DNS)
 apiVersion: gateway.networking.k8s.io/v1

apps/wave-1/sonarr-oauth2-proxy-credentials.sops.yaml

@@ -1,30 +0,0 @@
-apiVersion: v1
-kind: Secret
-metadata:
-    name: sonarr-oauth2-proxy
-    namespace: sonarr
-type: Opaque
-stringData:
-    client-id: ENC[AES256_GCM,data:JMYoJymH,iv:CM9HF71QZCeETzrOtkeTUs+BLTp/gGSExulXUP8yS/M=,tag:l73RaFQolO7+wZBo8nYCvw==,type:str]
-    client-secret: ENC[AES256_GCM,data:Fq271KdbtKWKWOe1Sun6+SLv/kSBsvAeFUhZIq6J2DNPGM4E8dgc2YANITcpezcJR07PAbSlUcS0L/qF6K+YhQ==,iv:/mVD1uOkweyO26RMV3qRQmLdyxXJR4I7p+OjYiiuSMU=,tag:9ZOKye5Sp2NHU2RXH1Y+wQ==,type:str]
-    cookie-secret: ENC[AES256_GCM,data:jMZZEi8fdh20BbPzCoVfwdS1g0/ma5n/QISU5A/gQCu/2Rkylt7mp798CUg=,iv:6IPqDNQyJHze2DUOQ93IwruWSge5nXjkVcQIOMfJSwY=,tag:cOKRcDKywV3PBOBrhoddRQ==,type:str]
-sops:
-    kms: []
-    gcp_kms: []
-    azure_kv: []
-    hc_vault: []
-    age:
-        - recipient: age175l8mdpdfu7hq0aml3exs3gykarwjmuvr23sq56s3wjejnyk44wsp4mfzg
-          enc: |
-            -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBoNU5NbnJnV0dKRnpIN2dw
-            QmhBRWI4aEpWajRHVkxZcGxCRk1hMDBjSlRVCm91b0U2N08vNi8rSEFaVlRJazFL
-            bllPcm9EMUgrQ3ZlUDlrdEZpZE8zNmMKLS0tIDU0ajdycXZrazlQb3RVRGQyY25h
-            R3RDajltcmFvQWdLYWQ2Mjh0VkplaWsKJprnLNJYzMd0yHbxjsD8P66Wu+sefmaV
-            nduEgVQY0t4EsYN6iKVDLMUAp+o3gXU8QuMuBnFZznrmBghfG+ZruQ==
-            -----END AGE ENCRYPTED FILE-----
-    lastmodified: "2026-08-31T13:42:49Z"
-    mac: ENC[AES256_GCM,data:ckVR9OyFvle0fLhBjHtJ6hH2St2CRJeu6abTCnqtxAfMDVXdbIkgHEesmN2kAgA7s3LkKJC2m7EFIpt3pO94VQNnZvC1KKD7qUSIfJN1DLVv78m8JbEQfVTJk/WgWy1ObxLriw5hDuMiEuOvKND+s6eqS5j6er4Q72lBsBpoEeY=,iv:uguThOQbR9Ks1qs1U0HO7EhUj1BT60fvM+3xVEC9hUU=,tag:m4P5nhlspCZyJRuOqffMog==,type:str]
-    pgp: []
-    encrypted_regex: ^(data|stringData)$
-    version: 3.9.4

apps/wave-1/sonarr.yaml

@@ -84,70 +84,6 @@ spec:
         volumeMounts:
         - name: config
           mountPath: /data/config
-      - name: oauth2-proxy
-        image: quay.io/oauth2-proxy/oauth2-proxy:v7.8.1
-        imagePullPolicy: IfNotPresent
-        args:
-        - --http-address=0.0.0.0:4180
-        - --reverse-proxy=true
-        - --provider=oidc
-        - --oidc-issuer-url=https://keycloak.home.keithjackson.dev/realms/homelab
-        - --upstream=http://127.0.0.1:8989
-        - --email-domain=*
-        - --scope=openid email profile
-        - --oidc-groups-claim=groups
-        - --allowed-group=media-users
-        - --skip-provider-button=true
-        - --redirect-url=https://sonarr.home.keithjackson.dev/oauth2/callback
-        - --cookie-domain=.home.keithjackson.dev
-        - --whitelist-domain=.home.keithjackson.dev
-        - --cookie-secure=true
-        - --pass-access-token=true
-        - --set-xauthrequest=true
-        env:
-        - name: OAUTH2_PROXY_CLIENT_ID
-          valueFrom:
-            secretKeyRef:
-              name: sonarr-oauth2-proxy
-              key: client-id
-        - name: OAUTH2_PROXY_CLIENT_SECRET
-          valueFrom:
-            secretKeyRef:
-              name: sonarr-oauth2-proxy
-              key: client-secret
-        - name: OAUTH2_PROXY_COOKIE_SECRET
-          valueFrom:
-            secretKeyRef:
-              name: sonarr-oauth2-proxy
-              key: cookie-secret
-        ports:
-        - name: http
-          containerPort: 4180
-          protocol: TCP
-        resources:
-          requests:
-            cpu: 25m
-            memory: 64Mi
-          limits:
-            memory: 128Mi
-        securityContext:
-          allowPrivilegeEscalation: false
-          readOnlyRootFilesystem: true
-          capabilities:
-            drop:
-            - ALL
-        livenessProbe:
-          httpGet:
-            path: /ping
-            port: http
-          initialDelaySeconds: 5
-          periodSeconds: 15
-        readinessProbe:
-          httpGet:
-            path: /ping
-            port: http
-          initialDelaySeconds: 5
-          periodSeconds: 15
       volumes:
       - name: config
         persistentVolumeClaim:
@@ -168,9 +104,6 @@ spec:
   - name: port-8989
     port: 8989
     targetPort: 8989
-  - name: authproxy
-    port: 4180
-    targetPort: 4180
 ---
 # CiliumNetworkPolicy egress allow (R-NET-02): sonarr-allow-qbittorrent-8080
 apiVersion: cilium.io/v2
@@ -255,7 +188,7 @@ spec:
     - ingress
     toPorts:
     - ports:
-      - port: '4180'
+      - port: '8989'
         protocol: TCP
 ---
 # CiliumNetworkPolicy world egress allow (egress class governs the path; R-NET-05): sonarr-allow-internet-egress
@@ -273,10 +206,6 @@ spec:
   egress:
   - toEntities:
     - world
-  - toEndpoints:
-    - matchLabels:
-        io.kubernetes.pod.namespace: keycloak
-        app: keycloak
 ---
 # HTTPRoute sonarr-8989 (R-TLS/R-DNS)
 apiVersion: gateway.networking.k8s.io/v1
@@ -303,7 +232,7 @@ spec:
         value: /
     backendRefs:
     - name: sonarr
-      port: 4180
+      port: 8989
 ---
 # ServiceMonitor (R-OBS-01)
 apiVersion: monitoring.coreos.com/v1

bootstrap/flux/README.md

@@ -13,16 +13,14 @@ Spec: [ADR-0004 (Flux)](../../../home-network-k3s/decisions/adr-0004-gitops-flux
 |---|---|
 | `bootstrap.sh` | **The single bootstrap entrypoint** the Infra worker's final Ansible play calls. `flux install` → verify `sops-age` Secret → `kubectl apply -k clusters/dev`. |
 | `patches/platform-sops-decryption.yaml` | Strategic-merge patch enabling `decryption.provider: sops` + `secretRef: sops-age` on the `platform` Kustomization (realises the base TODO). |
-| `patches/apps-sops-decryption.yaml` | Same, for the app-wave Kustomizations. There is **no single `apps` Kustomization** any more — the apps tier is now `apps-wave-0/1/2` + `apps-vms`, and SOPS decryption is layered on **per wave** (each wave Kustomization is patched by name in `clusters/dev`). |
+| `patches/apps-sops-decryption.yaml` | Same, for the `apps` Kustomization. |
 | `patches/flux-system-source-url.yaml` | Overrides the `flux-system` GitRepository `url` placeholder with the real bootstrap source (template — fill at bootstrap). |
 
 ## Why `flux install` + apply, not `flux bootstrap git` (Phase 1)
 
 The reconciliation graph is **already authored** in `clusters/base` (read-only to this worker):
-`flux-system.yaml` is the `flux-system` **GitRepository**; `platform.yaml`, `platform-config.yaml`
+`flux-system.yaml` is the `flux-system` **GitRepository**; `platform.yaml`/`apps.yaml` are the Flux
-and the app-wave Kustomizations (`apps-wave-0/1/2.yaml` + `apps-vms.yaml` — the single `apps.yaml`
+**Kustomizations**; `clusters/dev` is the overlay that patches `cluster-vars` (R-ENV-02).
-was retired) are the Flux **Kustomizations**; `clusters/dev` is the overlay that patches
-`cluster-vars` (R-ENV-02) and layers SOPS decryption onto each Kustomization by name.
 
 `flux bootstrap git` **generates and owns its own** `gotk-sync` GitRepository + Kustomization named
 `flux-system` and commits them into the repo — which would **duplicate/conflict** with the authored
@@ -43,9 +41,8 @@ tofu apply → cloud-init → Ansible (k3s + Cilium)          [Infra worker, T-1
 3. bootstrap/flux/bootstrap.sh           # flux install → verify sops-age → apply -k clusters/dev
         │
         ▼
-Flux reconciles: GitRepository(flux-system) → Kustomization(platform) → Kustomization(platform-config)
+Flux reconciles: GitRepository(flux-system) → Kustomization(platform) → Kustomization(apps),
-→ Kustomization(apps-wave-0 → apps-wave-1 → apps-wave-2) + Kustomization(apps-vms),
+with SOPS decryption enabled (patches/) and cluster-vars overlay (R-ENV-02).
-with per-Kustomization SOPS decryption enabled (patches/) and cluster-vars overlay (R-ENV-02).
 ```
 
 ## Interface note — for the Infra worker (T-1.3 Ansible)
@@ -75,8 +72,7 @@ patches:
   - path: ../../bootstrap/flux/patches/apps-sops-decryption.yaml      # SOPS on apps
 ```
 
-Alternatively the base TODOs in `clusters/base/platform.yaml` and each app-wave Kustomization
+Alternatively the base TODOs in `clusters/base/{platform,apps}.yaml` and the `url` in
-(`apps-wave-0/1/2.yaml`, `apps-vms.yaml`) and the `url` in
 `clusters/base/flux-system.yaml` can be filled in directly (same effect) — the clusters owner's call.
 Either way the `sops-age` `secretRef` name and `flux-system` source name are kept in lockstep with
 the authored base.

bootstrap/flux/bootstrap.sh

@@ -25,9 +25,7 @@
 #                         We do NOT use `flux bootstrap git` in Phase 1 because that command commits
 #                         and OWNS its own gotk-sync GitRepository+Kustomization named `flux-system`.
 #                         Our entrypoints are ALREADY authored in clusters/base (flux-system.yaml =
-#                         the GitRepository; platform.yaml + platform-config.yaml + the app-wave
+#                         the GitRepository; platform.yaml/apps.yaml = the Kustomizations). Letting
-#                         Kustomizations apps-wave-0/1/2.yaml + apps-vms.yaml = the Kustomizations;
-#                         the single apps.yaml was retired). Letting
 #                         bootstrap regenerate them would DUPLICATE/conflict. `flux install` +
 #                         applying the authored clusters/dev overlay is the reconciling path.
 #   2. sops-age Secret  — the operator installs the age PRIVATE key out-of-band BEFORE reconcile so
@@ -35,9 +33,8 @@
 #                         only VERIFIES it exists; it never handles key material.
 #   3. `kubectl apply -k clusters/dev` — materialises the authored reconciliation graph:
 #                         GitRepository `flux-system` (source) -> Kustomization `platform` ->
-#                         `platform-config` -> the app-wave Kustomizations `apps-wave-0/1/2`
+#                         Kustomization `apps` (dependsOn platform), with cluster-vars overlay
-#                         (chained dependsOn) + `apps-vms`, with cluster-vars overlay
+#                         (R-ENV-02) and SOPS decryption enabled via the patches in ./patches/.
-#                         (R-ENV-02) and per-Kustomization SOPS decryption enabled via ./patches/.
 #
 # PREREQUISITES the operator/PM resolves before this runs (see ../README.md "Handoff"):
 #   - clusters/base/flux-system.yaml url placeholder is overridden — wire ./patches/flux-system-

catalog/README.md

@@ -10,7 +10,7 @@ policy, egress, TLS, DNS, monitoring, and update policy from it (see
 [`docs/13-service-catalog.md`](../../home-network-k3s/docs/13-service-catalog.md) (14-service table +
 SSO/egress/dependency intent).
 
-## Current examples
+## Current examples (Phase 0)
 | File | docs/13 row | exposure | egress | auth | Notes |
 |---|---|---|---|---|---|
 | `pihole.yaml` | 6 | internal | internet | forward-auth | admin UI behind oauth2-proxy; config on Longhorn (DR-critical); also platform DNS |

catalog/render.py

@@ -52,20 +52,6 @@ KP_RELEASE = "kube-prometheus-stack"  # ServiceMonitor/PrometheusRule discovery
 EGRESS_CLASSES = ("internet", "vpn")
 AUTH_MODES = ("oidc", "forward-auth", "native", "none")
 
-# --- forward-auth (Option B) per-app oauth2-proxy sidecar ------------------------------
-# A `forward-auth` app has a bare web UI with no usable native OIDC (PiHole, *arr, Jackett,
-# qBittorrent). Cilium 1.16.x cannot inject ext_authz into the shared Gateway's Envoy config
-# (docs/plans/cilium-native-externalauth-evaluation.md; platform/oauth2-proxy/README.md), so
-# the platform's forward-auth CEC is an UNWIRED template. Instead each forward-auth app is
-# gated by a PER-POD oauth2-proxy REVERSE-PROXY sidecar (Option B) — EXACTLY the pattern
-# platform/crafty proves working: the shared Gateway routes <app>.home -> the sidecar (:4180),
-# the sidecar authenticates the browser against Keycloak (homelab realm, the app's auth.groups)
-# and only THEN proxies to the co-located app UI at http://127.0.0.1:<uiport>. The gate is IN
-# the data path (fail-closed: no valid session in an authorized group => no upstream; R-IDP-02).
-OAUTH2_PROXY_IMAGE = "quay.io/oauth2-proxy/oauth2-proxy:v7.8.1"
-OAUTH2_PROXY_PORT = 4180
-KEYCLOAK_ISSUER = f"https://keycloak.{ZONE}/realms/homelab"
-
 
 # =============================================================================
 # Defaults (renderer-contract.md "Default posture") — materialize BEFORE mapping.
@@ -266,13 +252,6 @@ def r_deployment(s: dict) -> dict:
 
     pod_spec = {"containers": [container]}
 
-    # forward-auth (Option B): append the per-pod oauth2-proxy reverse-proxy sidecar AFTER the app
-    # container (app stays containers[0]). The shared Gateway routes to the sidecar (:4180); it gates
-    # the co-located UI against Keycloak and only then proxies to http://127.0.0.1:<uiport>. `oidc`
-    # apps (native client) and `native`/`none` render byte-identical (no sidecar).
-    if _needs_oauth2_sidecar(s):
-        pod_spec["containers"].append(_oauth2_sidecar_container(s))
-
     # podSecurityContext (schema-optional): the k8s PodSecurityContext is emitted VERBATIM onto the
     # pod spec. Its primary use is `fsGroup`, which makes a mounted Longhorn PVC group-writable by a
     # non-root image (e.g. semaphore runs as UID 1001 and cannot otherwise write its config PVC).
@@ -315,14 +294,6 @@ def r_service(s: dict):
     ports = _service_ports(s)
     if not ports:
         return None
-    # forward-auth (Option B): the shared Gateway must reach the oauth2-proxy sidecar, so front it
-    # with a `authproxy` :4180 Service port (the HTTPRoute backendRef targets this). The app UI/API
-    # port(s) stay in the Service for EAST-WEST callers (e.g. arr -> qbittorrent:8080 / jackett:9117);
-    # north-south to those app ports is denied at the Gateway (see r_gateway_ingress_policy).
-    if _needs_oauth2_sidecar(s):
-        ports = ports + [
-            {"name": "authproxy", "port": OAUTH2_PROXY_PORT, "targetPort": OAUTH2_PROXY_PORT}
-        ]
     return {
         "apiVersion": "v1",
         "kind": "Service",
@@ -493,7 +464,7 @@ def r_gateway_ingress_policy(s: dict):
     north-south). This CiliumNetworkPolicy is the ADDITIVE, least-privilege allow that opens
     ONLY the north-south path (docs/09 "Ingress opens north-south only — it does not relax
     east-west default-deny"): ingress from Cilium's reserved `ingress` identity (the Gateway/
-    Ingress Envoy proxy, `reserved:ingress`, numeric identity 8 in Cilium 1.20.1) to ONLY the
+    Ingress Envoy proxy, `reserved:ingress`, numeric identity 8 in Cilium 1.16.5) to ONLY the
     exposed service port(s). Everything else stays denied; undeclared ports stay denied.
 
     Verified empirically against the cluster: with default-deny in place, Hubble shows the
@@ -503,15 +474,7 @@ def r_gateway_ingress_policy(s: dict):
     """
     if not s["ingressRoutes"]:
         return None
-    # forward-auth (Option B): the ONLY north-south path is the oauth2-proxy sidecar. Open the
+    ports = _service_ports(s)
-    # Gateway to :4180 ONLY — never the app UI port — so the un-gated UI is unreachable from the
-    # Gateway (fail-closed; R-IDP-02). Every other app opens its exposed service port(s) as before.
-    if _needs_oauth2_sidecar(s):
-        allow_ports = [{"port": str(OAUTH2_PROXY_PORT), "protocol": "TCP"}]
-    else:
-        allow_ports = [
-            {"port": str(p["port"]), "protocol": "TCP"} for p in _service_ports(s)
-        ]
     return {
         "apiVersion": "cilium.io/v2",
         "kind": "CiliumNetworkPolicy",
@@ -527,7 +490,14 @@ def r_gateway_ingress_policy(s: dict):
             "ingress": [
                 {
                     "fromEntities": ["ingress"],
-                    "toPorts": [{"ports": allow_ports}],
+                    "toPorts": [
+                        {
+                            "ports": [
+                                {"port": str(p["port"]), "protocol": "TCP"}
+                                for p in ports
+                            ]
+                        }
+                    ],
                 }
             ],
         },
@@ -557,28 +527,6 @@ def r_internet_egress(s: dict) -> dict:
     switch (a cluster-wide singleton keyed on the egress-class pod label) decide the PATH; this
     policy decides that north-south egress is ALLOWED at all.
     """
-    egress = [{"toEntities": ["world"]}]
-    # SSO apps (oidc/forward-auth) reach Keycloak at keycloak.home, which coredns split-horizon
-    # resolves to the shared Gateway VIP (an L2-announced LoadBalancer IP Cilium classifies as
-    # `world`), so the pod hairpins THROUGH the Gateway. Under Cilium >=1.16.8/1.17.2 (the
-    # CVE-2025-30162 fix, present in 1.20), the Gateway's L7 proxy re-evaluates THIS pod's egress
-    # policy against Keycloak's REAL backend cluster identity (enforce_policy_on_l7lb) — `world`
-    # matches the VIP at L3/L4 but NOT the resolved keycloak identity at L7, so OIDC discovery/
-    # token 403s ("Access denied") without an explicit allow to the keycloak backend. Scope it to
-    # the apps that actually talk to Keycloak (auth mode oidc or forward-auth).
-    if s.get("auth", {}).get("mode") in ("oidc", "forward-auth"):
-        egress.append(
-            {
-                "toEndpoints": [
-                    {
-                        "matchLabels": {
-                            "io.kubernetes.pod.namespace": "keycloak",
-                            "app": "keycloak",
-                        }
-                    }
-                ]
-            }
-        )
     return {
         "apiVersion": "cilium.io/v2",
         "kind": "CiliumNetworkPolicy",
@@ -589,7 +537,7 @@ def r_internet_egress(s: dict) -> dict:
         },
         "spec": {
             "endpointSelector": {"matchLabels": {"app.kubernetes.io/name": s["name"]}},
-            "egress": egress,
+            "egress": [{"toEntities": ["world"]}],
         },
     }
 
@@ -602,10 +550,6 @@ def r_httproutes(s: dict) -> list:
     """
     out = []
     for route in s["ingressRoutes"]:
-        # forward-auth (Option B): the Gateway backendRef targets the oauth2-proxy sidecar (:4180),
-        # NOT the app UI port — so the gate is in the data path and the bare UI is never served
-        # un-authenticated. Other apps route straight to their exposed app port.
-        backend_port = OAUTH2_PROXY_PORT if _needs_oauth2_sidecar(s) else route["port"]
         out.append(
             {
                 "apiVersion": "gateway.networking.k8s.io/v1",
@@ -626,7 +570,7 @@ def r_httproutes(s: dict) -> list:
                                 {"path": {"type": "PathPrefix", "value": "/"}}
                             ],
                             "backendRefs": [
-                                {"name": s["name"], "port": backend_port}
+                                {"name": s["name"], "port": route["port"]}
                             ],
                         }
                     ],
@@ -636,94 +580,6 @@ def r_httproutes(s: dict) -> list:
     return out
 
 
-def _needs_oauth2_sidecar(s: dict) -> bool:
-    """True iff this service gets a per-pod oauth2-proxy reverse-proxy sidecar (Option B).
-
-    A `forward-auth` app with an exposed UI has no usable native OIDC, so its Gateway route is
-    gated IN the data path by a co-located oauth2-proxy sidecar (crafty's proven pattern). `oidc`
-    apps authenticate themselves (native Keycloak client) and `native`/`none` are unchanged."""
-    return s["auth"]["mode"] == "forward-auth" and bool(s["ingressRoutes"])
-
-
-def _oauth2_sidecar_container(s: dict) -> dict:
-    """The oauth2-proxy REVERSE-PROXY sidecar (Option B) that gates a forward-auth app's UI.
-
-    Mirrors platform/crafty/deployment.yaml's `oauth2-proxy` container EXACTLY, adapted per app:
-    the upstream is the co-located app UI over plain HTTP (http://127.0.0.1:<uiport>) — the *arr /
-    PiHole / Jackett / qBittorrent UIs are HTTP, so (unlike crafty's self-signed HTTPS :8443) there
-    is NO --ssl-upstream-insecure-skip-verify. --allowed-group is emitted once per declared
-    auth.group (R-IDP-06: a valid session is not enough — the user must be in an authorized group,
-    else /oauth2 returns 403). client-id/secret + cookie-secret come by name from the SOPS-encrypted
-    `<app>-oauth2-proxy` Secret (R-IDP-04). Single provider => --skip-provider-button 302s straight
-    to Keycloak. The sidecar->app hop is pod-local (127.0.0.1) and not subject to network policy."""
-    route = s["ingressRoutes"][0]
-    host = route["host"]
-    upstream_port = route["port"]
-    args = [
-        f"--http-address=0.0.0.0:{OAUTH2_PROXY_PORT}",
-        "--reverse-proxy=true",
-        "--provider=oidc",
-        f"--oidc-issuer-url={KEYCLOAK_ISSUER}",
-        f"--upstream=http://127.0.0.1:{upstream_port}",
-        "--email-domain=*",
-        "--scope=openid email profile",
-        "--oidc-groups-claim=groups",
-    ]
-    # R-IDP-06: authorize by group. One flag per declared group (fail-closed to 403 otherwise).
-    for group in s["auth"].get("groups", []):
-        args.append(f"--allowed-group={group}")
-    args += [
-        "--skip-provider-button=true",
-        f"--redirect-url=https://{host}/oauth2/callback",
-        f"--cookie-domain=.{ZONE}",
-        f"--whitelist-domain=.{ZONE}",
-        "--cookie-secure=true",
-        "--pass-access-token=true",
-        "--set-xauthrequest=true",
-    ]
-    secret = f"{s['name']}-oauth2-proxy"
-    return {
-        "name": "oauth2-proxy",
-        "image": OAUTH2_PROXY_IMAGE,
-        "imagePullPolicy": "IfNotPresent",
-        "args": args,
-        "env": [
-            {
-                "name": "OAUTH2_PROXY_CLIENT_ID",
-                "valueFrom": {"secretKeyRef": {"name": secret, "key": "client-id"}},
-            },
-            {
-                "name": "OAUTH2_PROXY_CLIENT_SECRET",
-                "valueFrom": {"secretKeyRef": {"name": secret, "key": "client-secret"}},
-            },
-            {
-                "name": "OAUTH2_PROXY_COOKIE_SECRET",
-                "valueFrom": {"secretKeyRef": {"name": secret, "key": "cookie-secret"}},
-            },
-        ],
-        "ports": [{"name": "http", "containerPort": OAUTH2_PROXY_PORT, "protocol": "TCP"}],
-        "resources": {
-            "requests": {"cpu": "25m", "memory": "64Mi"},
-            "limits": {"memory": "128Mi"},
-        },
-        "securityContext": {
-            "allowPrivilegeEscalation": False,
-            "readOnlyRootFilesystem": True,
-            "capabilities": {"drop": ["ALL"]},
-        },
-        "livenessProbe": {
-            "httpGet": {"path": "/ping", "port": "http"},
-            "initialDelaySeconds": 5,
-            "periodSeconds": 15,
-        },
-        "readinessProbe": {
-            "httpGet": {"path": "/ping", "port": "http"},
-            "initialDelaySeconds": 5,
-            "periodSeconds": 15,
-        },
-    }
-
-
 def _auth_annotations(s: dict) -> dict:
     """Auth wiring hints (renderer-contract §auth): the platform (oauth2-proxy/Keycloak)
     consumes these to protect ingressRoutes. oidc/forward-auth carry the authorized groups
@@ -811,7 +667,7 @@ def r_image_automation(s: dict) -> list:
     upd = s["update"]
     interval = "1h0m0s"
     imagerepo = {
-        "apiVersion": "image.toolkit.fluxcd.io/v1",
+        "apiVersion": "image.toolkit.fluxcd.io/v1beta2",
         "kind": "ImageRepository",
         "metadata": {"name": s["name"], "namespace": s["name"]},
         "spec": {"image": image, "interval": interval},
@@ -826,7 +682,7 @@ def r_image_automation(s: dict) -> list:
     else:
         policy_spec["policy"] = {"semver": {"range": ">=0.0.0"}}
     imagepolicy = {
-        "apiVersion": "image.toolkit.fluxcd.io/v1",
+        "apiVersion": "image.toolkit.fluxcd.io/v1beta2",
         "kind": "ImagePolicy",
         "metadata": {
             "name": s["name"],
@@ -1180,7 +1036,6 @@ def plan_apps_waves(sds: list, present_files: set) -> dict:
     convention:
       - `<svc>.yaml`                        -> wave(svc)
       - `<svc>-bootstrap-secret.sops.yaml`  -> wave(svc)   (consumed by svc's env, same apply)
-      - `<svc>-oauth2-proxy-credentials.sops.yaml` -> wave(svc)  (forward-auth sidecar OIDC creds)
       - `<svc>-db.yaml`                     -> wave(svc-db) (a strictly-lower wave; CNPG DB)
       - `<svc>-db-app.sops.yaml`            -> wave(svc-db) (CNPG initdb owner secret)
       - `<svc>-db-backup-credentials.sops.yaml` -> wave(svc-db) (CNPG Barman S3 backup secret)
@@ -1201,10 +1056,6 @@ def plan_apps_waves(sds: list, present_files: set) -> dict:
         w = waves[name]
         add(w, f"{name}.yaml")
         add(w, f"{name}-bootstrap-secret.sops.yaml")
-        # forward-auth (Option B): the per-app oauth2-proxy sidecar's OIDC client + cookie secret.
-        # Same wave as the app (the sidecar reads it at pod start); placed only when the file exists,
-        # so non-forward-auth apps are unaffected. Mirrors the bootstrap-secret naming convention.
-        add(w, f"{name}-oauth2-proxy-credentials.sops.yaml")
         db = f"{name}{CO_DB_SUFFIX}"
         if any(dep["to"] == db for dep in by_name[name]["dependencies"]):
             wdb = waves[db]
@@ -1452,15 +1303,12 @@ def _selftest() -> int:
 
     # 4b) exposure (ingressRoutes) -> an ADDITIVE gateway-ingress CiliumNetworkPolicy that
     #     allows ONLY the reserved `ingress` (Gateway proxy) identity to the exposed port,
-    #     while default-deny still denies all other ingress (R-NET-01/02). Uses an explicit
+    #     while default-deny still denies all other ingress (R-NET-01/02).
-    #     non-forward-auth mode so this exercises the direct app-port gateway path (a
-    #     forward-auth app instead routes the Gateway to the :4180 sidecar — see 6c).
     exp_docs = render(
         {
             "name": "z",
             "source": {"image": "a:1"},
             "ingressRoutes": [{"host": "z.home.keithjackson.dev", "port": 80}],
-            "auth": {"mode": "none"},
         }
     )
     gw = [
@@ -1574,91 +1422,6 @@ def _selftest() -> int:
         "an SD without podSecurityContext must not gain a pod securityContext",
     )
 
-    # 6c) forward-auth (Option B): an exposed forward-auth app gets an oauth2-proxy sidecar that
-    #     fronts the Gateway path (fail-closed), while oidc/native/none apps are unchanged.
-    fa_docs = render(
-        {
-            "name": "fa",
-            "source": {"image": "a:1"},
-            "ingressRoutes": [{"host": "fa.home.keithjackson.dev", "port": 9117}],
-            "auth": {"mode": "forward-auth", "groups": ["media-users"]},
-        }
-    )
-    fa_dep = next(d[1] for d in fa_docs if d[1]["kind"] == "Deployment")
-    fa_containers = fa_dep["spec"]["template"]["spec"]["containers"]
-    check(
-        len(fa_containers) == 2
-        and fa_containers[0]["name"] == "fa"
-        and fa_containers[1]["name"] == "oauth2-proxy",
-        "forward-auth app must render the app container FIRST + an oauth2-proxy sidecar SECOND",
-    )
-    side = fa_containers[1]
-    check(
-        "--upstream=http://127.0.0.1:9117" in side["args"],
-        "sidecar upstream must be the co-located app UI over http on 127.0.0.1:<uiport>",
-    )
-    check(
-        "--allowed-group=media-users" in side["args"],
-        "sidecar must enforce the declared auth.group (R-IDP-06)",
-    )
-    check(
-        "--redirect-url=https://fa.home.keithjackson.dev/oauth2/callback" in side["args"],
-        "sidecar redirect-url must be the app host's /oauth2/callback",
-    )
-    check(
-        not any(a.startswith("--ssl-upstream-insecure-skip-verify") for a in side["args"]),
-        "http upstream must NOT skip TLS verify (that flag is only for crafty's self-signed https)",
-    )
-    check(
-        side["env"][0]["valueFrom"]["secretKeyRef"]["name"] == "fa-oauth2-proxy",
-        "sidecar creds must come by name from the per-app <app>-oauth2-proxy Secret (R-IDP-04)",
-    )
-    fa_svc = next(d[1] for d in fa_docs if d[1]["kind"] == "Service")
-    fa_svc_ports = {p["port"] for p in fa_svc["spec"]["ports"]}
-    check(
-        fa_svc_ports == {9117, 4180},
-        "forward-auth Service must expose the app port (east-west) AND :4180 (gateway->sidecar)",
-    )
-    fa_route = next(d[1] for d in fa_docs if d[1]["kind"] == "HTTPRoute")
-    check(
-        fa_route["spec"]["rules"][0]["backendRefs"][0]["port"] == 4180,
-        "forward-auth HTTPRoute backendRef must target the sidecar :4180, not the app UI port",
-    )
-    fa_gw = next(
-        d[1] for d in fa_docs
-        if d[1]["kind"] == "CiliumNetworkPolicy"
-        and d[1]["metadata"]["name"] == "fa-allow-gateway-ingress"
-    )
-    fa_gw_ports = {
-        p["port"] for r in fa_gw["spec"]["ingress"] for tp in r["toPorts"] for p in tp["ports"]
-    }
-    check(
-        fa_gw_ports == {"4180"},
-        "fail-closed: the Gateway may reach ONLY the sidecar :4180, never the un-gated app port",
-    )
-    # an oidc app (native client) must NOT gain a sidecar or a :4180 path.
-    oidc_docs = render(
-        {
-            "name": "oi",
-            "source": {"image": "a:1"},
-            "ingressRoutes": [{"host": "oi.home.keithjackson.dev", "port": 3000}],
-            "auth": {"mode": "oidc", "groups": ["platform-admins"]},
-        }
-    )
-    oi_containers = next(
-        d[1] for d in oidc_docs if d[1]["kind"] == "Deployment"
-    )["spec"]["template"]["spec"]["containers"]
-    check(
-        len(oi_containers) == 1
-        and not any("oauth2-proxy" in c["name"] for c in oi_containers),
-        "an oidc app must NOT gain an oauth2-proxy sidecar (native client authenticates itself)",
-    )
-    oi_route = next(d[1] for d in oidc_docs if d[1]["kind"] == "HTTPRoute")
-    check(
-        oi_route["spec"]["rules"][0]["backendRefs"][0]["port"] == 3000,
-        "an oidc app's HTTPRoute must still target its own app port (unchanged)",
-    )
-
     # 7) wave partitioning: longest-path over the dependency DAG, CNPG-DB strictly below.
     wsds = [
         {"name": "a", "source": {"image": "x:1"}},                                    # leaf

ci/Makefile

@@ -1,9 +1,8 @@
 # ci/Makefile — portable, single-command local equivalent of the Gitea Actions
 # merge gate (R-TEST-02: the gates are runnable TODAY, without Gitea).
 #
-# The LIVE Gitea Actions workflow (.gitea/workflows/gate.yml) invokes `make -C ci gate`
+# The Gitea Actions workflow (.gitea/workflows/merge-gate.yaml) calls these exact
-# directly, so `make -C ci gate` locally IS the CI merge gate — byte-for-byte. Run from
+# scripts, so `make -C ci gate` locally == the CI merge gate. Run from anywhere:
-# anywhere:
 #   make -C ci gate
 #
 # Every stage is a thin wrapper over a ci/*.sh script; see ci/README.md.
@@ -12,7 +11,7 @@ CI_DIR := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
 
 .DEFAULT_GOAL := help
 
-.PHONY: help spec-immutability lint schema l0 render-golden l1 conformance gate all touched versions tools test
+.PHONY: help spec-immutability lint schema l0 render-golden l1 conformance gate all touched versions tools
 
 help: ## Show this help
 	@awk 'BEGIN{FS=":.*##"} /^[a-zA-Z0-9_-]+:.*##/{printf "  \033[36m%-14s\033[0m %s\n",$$1,$$2}' $(MAKEFILE_LIST)
@@ -43,13 +42,6 @@ all: gate ## Alias for `gate`
 touched: ## Print the touched-requirement subset for the current branch vs main
 	@"$(CI_DIR)touched-requirements.sh"
 
-test: ## Run ci/ script unit tests (no cluster; e.g. touched-requirements content scan)
-	@rc=0; for t in "$(CI_DIR)"tests/*.test.sh; do \
-	  echo "== $$t"; bash "$$t" || rc=1; \
-	done; \
-	if [ $$rc -ne 0 ]; then echo "ci/tests: FAILED" >&2; else echo "ci/tests: all suites passed"; fi; \
-	exit $$rc
-
 versions: ## Print pinned tool versions (R-TEST-02)
 	@cat "$(CI_DIR)versions.env"
 

ci/README.md

@@ -20,54 +20,17 @@ Individual stages:
 
 ```bash
 make -C ci lint        # yaml/lint                          (no cluster; live now)
-make -C ci l0          # schema + Kyverno + L0 static conformance (no cluster; live now)
+make -C ci l0          # ServiceDefinition schema + Kyverno  (no cluster; live now)
 make -C ci l1          # L1 rendering checks                 (no cluster; live now)
 make -C ci conformance # L2/L3 in-cluster Chainsaw Job       (SKIPS unless CLUSTER_AVAILABLE=true)
-make -C ci test        # unit tests for these scripts themselves (ci/tests/*.test.sh)
 ```
 
-The `l0` stage has three sub-stages: ServiceDefinition schema validation, the Kyverno policy gate
-(`kyverno test`), and **`conformance/run.sh --layer l0`** — the whole-repo L0 invariants a
-per-resource policy cannot express (secret hygiene across every tracked file, least-privilege RBAC,
-spec-immutability enforcement). See `conformance/tests/README.md`.
-
 ## Exit-code contract
 
 Every stage script uses: **`0` = pass · `1` = fail (blocks merge) · `3` = SKIPPED** (cluster-gated
 with no cluster, or an unwired seam / missing tool). A `3` is **never** counted as green — that is how
 the gate stays honest in Phase 0 (no fake green).
 
-### The touched-requirement subset can also be *undeterminable* (exit 3)
-
-`touched-requirements.sh` is the input to the R-TEST-03 decision. It resolves a diff base in this
-order, **verifying each candidate exists** before using it:
-
-```
-BASE_REF (explicit)  >  GITHUB_BASE_REF (Actions PR runs)  >  origin/main  >  main
-```
-
-If no candidate resolves, if there is no merge base, if the diff command fails, or if **HEAD is not
-ahead of the base** (you are sitting on `main`, or the branch is already merged), it prints a loud
-banner, writes **nothing** to stdout, and exits **3** — plus a machine-readable reason code
-(`on-base` / `no-base` / `diff-failed`) to `$TOUCHED_REASON_FILE` when that variable names a path.
-
-**Why this is not cosmetic.** Every one of those conditions used to degrade *silently* to an empty
-touched set and exit 0. An empty set means `conformance/run.sh` is invoked with no `--requirements`,
-which means its coverage `--require` gate never runs — so on 2026-08-31 the local gate printed
-`MERGE GATE: PASS` having checked nothing, while CI (correct base, 17 touched requirements) BLOCKED
-on real coverage debt. There is no silent narrowing left: the `HEAD~1` last resort and the two-dot
-diff fallback are both gone. Locked down by `ci/tests/touched-requirements-base.test.sh`.
-
-`gate.sh` turns exit 3 into an **UNDECIDED** merge decision, never a pass:
-
-| reason | headline | exit |
-|---|---|---|
-| `on-base` | `MERGE GATE: NOT APPLICABLE — no branch under gate` | 0 (a push-to-`main` CI run legitimately lands here; it just must not read as "my branch is green") |
-| `no-base`, `diff-failed` | `MERGE GATE: BLOCKED` | 1 (the environment cannot support a merge decision; refusing to decide must not look like deciding yes) |
-
-A stage that needs the subset (`l0.sh`, `l1.sh`) still runs its FULL layer as a superset sanity net
-in that case — nothing goes untested — and says plainly that the *scoped* check did not happen.
-
 ## Reproducibility (R-TEST-02)
 
 - **Version-pinned:** all tool versions live in [`versions.env`](versions.env) — the single source of
@@ -80,19 +43,16 @@ in that case — nothing goes untested — and says plainly that the *scoped* ch
 
 ```
 ci/
-  Makefile                 targets: lint · l0 · l1 · conformance · gate · touched · test · tools · versions
+  Makefile                 targets: lint · l0 · l1 · conformance · gate · touched · tools · versions
   versions.env             PINNED tool versions (R-TEST-02) — single source of truth
   lib.sh                   shared helpers: logging, tool checks, cluster detection, artifact paths
   yamllint.yaml            lint config for the yaml/lint stage
   lint.sh                  Stage: yaml/lint
-  l0.sh                    Stage: schema validation + Kyverno L0 gate + L0 static conformance
+  l0.sh                    Stage: ServiceDefinition schema validation + Kyverno L0 policy gate
   l1.sh                    Stage: L1 rendering checks (Chainsaw L1 via the conformance runner)
   conformance.sh           Stage: L2/L3 in-cluster Chainsaw Job (trigger + collect)
-  touched-requirements.sh  compute the R-* subset a change touches (drives the R-TEST-03 filter);
+  touched-requirements.sh  compute the R-* subset a change touches (drives the R-TEST-03 filter)
-                           exits 3 LOUDLY rather than guessing when the diff base is unreliable
   gate.sh                  run all stages + the merge decision (the single command)
-  tests/*.test.sh          unit tests for these scripts (`make -C ci test`), run against
-                           throwaway git repos via the REPO_ROOT override — no network, no cluster
   README.md                this file
 ```
 
@@ -107,7 +67,7 @@ below are this workstream's **assumptions** — flagged for the PM to verify at
 |---|---|---|---|---|
 | ServiceDefinition schema validator | WS-GITOPS · **T-0.3** (`schemas/`) | `SCHEMA_VALIDATE_CMD` | `schemas/validate.sh` | `validate.sh catalog/*.yaml` → exit 0/1; `--version` supported |
 | Kyverno L0 policies + tests | WS-TEST · **T-0.5** (`platform/kyverno/`) | `KYVERNO_POLICY_DIR` | `platform/kyverno` | contains `kyverno-test.yaml` files discoverable by `kyverno test <dir>` |
-| Conformance runner (Chainsaw) | WS-TEST · **T-0.4** (`conformance/`) | `CONFORMANCE_CMD` | `conformance/run.sh` | `run.sh --layer <l0\|l1\|l2,l3> [--requirements R-A,R-B]`; artifacts land in `conformance/_artifacts/` |
+| Conformance runner (Chainsaw) | WS-TEST · **T-0.4** (`conformance/`) | `CONFORMANCE_CMD` | `conformance/run.sh` | `run.sh --layer <l1\|l2,l3> [--requirements R-A,R-B] [--trigger in-cluster-job] --junit <file> --coverage <dir>` |
 | Catalog (ServiceDefinitions) | WS-APPS | `CATALOG_DIR` | `catalog` | `*.yaml` ServiceDefinition files |
 
 Artifact locations (git-ignored): JUnit → `results/junit/*.xml`; coverage → `reports/coverage/**`.

ci/gate.sh

@@ -48,32 +48,6 @@ echo >&2
 log "==== gate decision ===="
 BLOCK=0
 
-# --- R-TEST-03 input: the touched-requirement subset --------------------------
-# Computed ONCE here. ci/touched-requirements.sh exits 3 when it cannot establish a
-# reliable diff base; that used to degrade to an empty set, which silently disabled the
-# coverage `--require` gate and let the gate print "MERGE GATE: PASS" having checked
-# nothing (the 2026-08-31 local false green). An undeterminable subset is now surfaced
-# as an UNDECIDED merge decision, never as a pass.
-_reason_file="$(mktemp)"
-# NOTE: command substitution, NOT `mapfile < <(...)` — process substitution does not
-# propagate the child's exit status, and the whole point here is to SEE the exit 3.
-TOUCHED_OUT="$(TOUCHED_REASON_FILE="$_reason_file" "${CI_DIR}/touched-requirements.sh")"
-TOUCHED_RC=$?
-TOUCHED_REASON="$(cat "$_reason_file" 2>/dev/null || true)"
-rm -f "$_reason_file"
-TOUCHED=()
-[ -n "$TOUCHED_OUT" ] && mapfile -t TOUCHED <<< "$TOUCHED_OUT"
-UNDECIDED=0
-if [ "$TOUCHED_RC" -eq 3 ]; then
-  UNDECIDED=1
-  echo >&2
-  err "R-TEST-03 DECISION INPUT MISSING: the touched-requirement subset could not be"
-  err "  determined (${TOUCHED_REASON:-unknown}). The scoped coverage gate did NOT run."
-  summary "- **R-TEST-03 subset UNDETERMINED** (\`${TOUCHED_REASON:-unknown}\`) — the scoped coverage gate did not run."
-elif [ "${#TOUCHED[@]}" -gt 0 ]; then
-  log "touched requirements: ${TOUCHED[*]}"
-fi
-
 # spec-immutability (R-TEST-06): a FAIL always blocks (the spec was modified/drifted).
 # A skip means the spec repo wasn't available to verify — surfaced, not green.
 case "${OUTCOME[spec-immutability]}" in
@@ -103,11 +77,9 @@ case "${OUTCOME[conformance]}" in
   skip)
     if cluster_available; then
       err "conformance: skipped despite CLUSTER_AVAILABLE=true -> blocks merge"; BLOCK=1
-    elif [ "$UNDECIDED" -eq 1 ]; then
-      warn "conformance L2/L3 UNVERIFIED, and the touched subset is unknown — nothing certified."
     else
-      if [ "${#TOUCHED[@]}" -gt 0 ]; then
+      mapfile -t reqs < <("${CI_DIR}/touched-requirements.sh" || true)
-        reqs=("${TOUCHED[@]}")
+      if [ "${#reqs[@]}" -gt 0 ]; then
         warn "conformance L2/L3 UNVERIFIED for touched requirements: ${reqs[*]}"
         warn "  (no cluster — Phase 0). These are NOT certified green."
         summary "- **UNVERIFIED (no cluster):** ${reqs[*]}"
@@ -121,35 +93,15 @@ case "${OUTCOME[conformance]}" in
     fi ;;
 esac
 
-# An UNDECIDED subset is never a pass. Two outcomes, by reason:
-#   on-base  -> this checkout has no branch commits to gate (you are on main, or the
-#               branch is already merged; a push-to-main CI run lands here). Legitimate,
-#               so the job is not failed — but the headline says NOT APPLICABLE, never
-#               PASS, so nobody reads it as "my branch is green".
-#   anything -> the environment cannot support a merge decision (no base ref, broken
-#      else     diff). Refusing to decide must not look like deciding yes: BLOCK.
-if [ "$UNDECIDED" -eq 1 ] && [ "$TOUCHED_REASON" != "on-base" ]; then
-  err "cannot compute the R-TEST-03 subset (${TOUCHED_REASON:-unknown}) -> blocks merge"
-  BLOCK=1
-fi
-
 echo >&2
-if [ "$BLOCK" -ne 0 ]; then
+if [ "$BLOCK" -eq 0 ]; then
-  err "MERGE GATE: BLOCKED"
-  summary ""
-  summary "**Result: BLOCKED** — see failures above."
-  exit 1
-elif [ "$UNDECIDED" -eq 1 ]; then
-  warn "MERGE GATE: NOT APPLICABLE — no branch under gate (${TOUCHED_REASON})."
-  warn "  The stages above ran and did not fail, but NO touched-requirement subset was"
-  warn "  checked. This is NOT a green branch: check out the branch you mean to gate"
-  warn "  (or set BASE_REF) and run it again."
-  summary ""
-  summary "**Result: NOT APPLICABLE** — no branch under gate (\`${TOUCHED_REASON}\`); the R-TEST-03 subset was not checked."
-  exit 0
-else
   ok "MERGE GATE: PASS (touched-requirement subset green for the layers that can run)"
   summary ""
   summary "**Result: PASS** — L0/L1 green. L2/L3 status per cluster availability above."
   exit 0
+else
+  err "MERGE GATE: BLOCKED"
+  summary ""
+  summary "**Result: BLOCKED** — see failures above."
+  exit 1
 fi

ci/l0.sh

@@ -7,9 +7,6 @@
 #   Stage 3  L0     : `kyverno test` over platform/kyverno/ — default-deny baseline
 #                     present, no plaintext secrets, required labels/egress-class,
 #                     ServiceDefinition schema conformance (owner: T-0.5, ADR-0011).
-#   Stage 3b L0     : conformance/run.sh --layer l0 — the WHOLE-REPO L0 invariants that
-#                     a per-resource policy cannot express (secret hygiene across every
-#                     tracked file, least-privilege RBAC, spec-immutability enforcement).
 #
 # Integration seams are env-overridable (see ci/lib.sh + README). When an owning
 # subtree is still a placeholder, this reports SKIPPED (exit 3) for that sub-stage
@@ -76,30 +73,6 @@ else
   rm -rf "$l0_render"
 fi
 
-# ---- Stage 3b: L0 STATIC conformance checks ---------------------------------
-# docs/14's L0 row is "manifests/ServiceDefinitions obey invariants", which is wider
-# than what Kyverno alone can express over single resources: whole-repo invariants
-# (no plaintext secrets anywhere in git, least-privilege RBAC across every tracked
-# binding, the spec-immutability gate's own behaviour) live in the conformance
-# harness's tests/l0-static subtree and run with no cluster. This stage executes them
-# through the same single-command runner CI and a laptop both use (R-TEST-02).
-log "Stage: L0 static conformance checks (${CONFORMANCE_CMD} --layer l0)"
-if ! entrypoint_ready "$CONFORMANCE_CMD"; then
-  skip "conformance runner not present at '${CONFORMANCE_CMD}' — UNWIRED, not green"
-  summary "- l0-static: **SKIPPED** (conformance runner missing)"
-  if [ "$rc" -eq 0 ]; then rc=3; fi
-else
-  load_touched_requirements
-  if "$CONFORMANCE_CMD" --layer l0 ${REQ_ARGS[@]+"${REQ_ARGS[@]}"}; then
-    ok "L0 static conformance checks passed"
-    summary "- l0-static: **passed**"
-  else
-    err "L0 static conformance checks failed"
-    summary "- l0-static: **FAILED**"
-    rc=1
-  fi
-fi
-
 case "$rc" in
   0) ok "L0 stage complete" ;;
   3) skip "L0 stage incomplete (unwired seam / missing tool) — not a pass" ;;

ci/l1.sh

@@ -20,16 +20,19 @@ if ! entrypoint_ready "$CONFORMANCE_CMD"; then
   exit 3
 fi
 
-# Touched-requirement subset (R-TEST-03). An empty OR undeterminable subset => run all
+# Touched-requirement subset (R-TEST-03). Empty subset => run all L1 as a sanity net.
-# L1 as a superset sanity net, and SAY SO — see load_touched_requirements in ci/lib.sh.
+mapfile -t reqs < <("${CI_DIR}/touched-requirements.sh" || true)
-load_touched_requirements
+req_args=()
-req_args=(${REQ_ARGS[@]+"${REQ_ARGS[@]}"})
+if [ "${#reqs[@]}" -gt 0 ]; then
+  log "touched requirements: ${reqs[*]}"
+  req_args=(--requirements "$(IFS=,; echo "${reqs[*]}")")
+fi
 
 # RECONCILED (PM 2026-08-16): conformance/run.sh's real interface is `--layer <l1|l2|l3>`
 # [--requirements R-A,R-B]; it ALWAYS writes artifacts to a FIXED location
 # (conformance/_artifacts/) and does NOT accept --junit/--coverage. Invoke by layer +
 # touched subset, then COLLECT its artifacts into the CI artifact dirs.
-"$CONFORMANCE_CMD" --layer l1 ${req_args[@]+"${req_args[@]}"}
+"$CONFORMANCE_CMD" --layer l1 "${req_args[@]}"
 
 CONF_ART="$(dirname "$CONFORMANCE_CMD")/_artifacts"
 if [ -f "${CONF_ART}/report.junit.xml" ]; then

ci/lib.sh

@@ -10,12 +10,9 @@
 set -euo pipefail
 
 # --- Paths -------------------------------------------------------------------
-# CI_DIR = this ci/ directory (always the real script location). REPO_ROOT = its
+# CI_DIR = this ci/ directory; REPO_ROOT = its parent (impl repo root).
-# parent (impl repo root) by default, but OVERRIDABLE via the environment so the
-# scripts can be pointed at a throwaway git repo under test (see ci/tests/) without
-# copying them out of the tree. CI and normal runs leave it unset -> unchanged.
 CI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-: "${REPO_ROOT:=$(cd "${CI_DIR}/.." && pwd)}"
+REPO_ROOT="$(cd "${CI_DIR}/.." && pwd)"
 # shellcheck disable=SC1091
 source "${CI_DIR}/versions.env"
 
@@ -85,38 +82,3 @@ cluster_available() {
 }
 
 mkartifacts() { mkdir -p "$JUNIT_DIR" "$COVERAGE_DIR"; }
-
-# --- Touched-requirement subset (R-TEST-03 input) ----------------------------
-# Sets: TOUCHED_RC (0 ok / 3 not determinable), TOUCHED_REQS[] and REQ_ARGS[]
-# (ready to splice into a conformance/run.sh invocation).
-#
-# ci/touched-requirements.sh exits 3 — LOUDLY — when it cannot establish a reliable
-# diff base, instead of the old silent degradation to an empty set. An empty set
-# means run.sh gets no `--requirements`, which means its coverage `--require` gate
-# never runs; treating "could not determine" as "nothing touched" is precisely the
-# false green this helper exists to prevent. A stage still runs its FULL layer in
-# that case (a superset, so nothing goes untested), but says plainly that the
-# SCOPED check did not happen; ci/gate.sh turns that into an UNDECIDED decision.
-load_touched_requirements() {
-  local out rc=0
-  out="$("${CI_DIR}/touched-requirements.sh")" || rc=$?
-  TOUCHED_RC=$rc
-  TOUCHED_REQS=()
-  # `if`, not `[ ... ] && ...`: under `set -e` a false AND-list is the caller's last
-  # command status and would abort the stage script on an EMPTY touched set.
-  if [ -n "$out" ]; then mapfile -t TOUCHED_REQS <<< "$out"; fi
-  REQ_ARGS=()
-  if [ "$rc" -eq 3 ]; then
-    warn "touched-requirement subset UNDETERMINED (reason logged above)."
-    warn "  Running the FULL layer as a superset sanity net; the SCOPED R-TEST-03"
-    warn "  coverage check is NOT performed here — ci/gate.sh reports that as UNDECIDED."
-  elif [ "$rc" -ne 0 ]; then
-    warn "touched-requirements.sh exited ${rc}; proceeding with the full layer."
-  elif [ "${#TOUCHED_REQS[@]}" -gt 0 ]; then
-    log "touched requirements: ${TOUCHED_REQS[*]}"
-    REQ_ARGS=(--requirements "$(IFS=,; echo "${TOUCHED_REQS[*]}")")
-  else
-    log "empty touched-requirement set (base WAS established) — running the full layer."
-  fi
-  return 0
-}

ci/render-golden.sh

@@ -39,7 +39,6 @@ PYTHON="${PYTHON:-python3}"
 # DESIGN-waves.md §"Catalog services not yet onboarded"). They render as a
 # by-product of `--all catalog` but must NOT have a golden; the gate ignores
 # their rendered output. Onboarding one = add its golden and drop it here.
-#
 : "${RENDER_GOLDEN_NOT_ONBOARDED:=keycloak vm-demo}"
 
 log "Stage: render==golden reproducibility (canonical render vs committed apps/)"

ci/tests/touched-requirements-base.test.sh

@@ -1,169 +0,0 @@
-#!/usr/bin/env bash
-# ci/tests/touched-requirements-base.test.sh — regression tests for BASE-REF RESOLUTION
-# in ci/touched-requirements.sh.
-#
-# THE BUG THESE LOCK DOWN (2026-08-31, local/CI divergence):
-#   `make -C ci gate` on a feature branch printed
-#       "no changed files vs base; empty touched-requirement set"
-#   and then "MERGE GATE: PASS" — while CI, with a correct base, computed 17 touched
-#   requirements and BLOCKED on real coverage debt. Root cause: EVERY way of failing to
-#   establish a diff base degraded silently to an EMPTY touched set and exit 0. An empty
-#   set means conformance/run.sh is invoked without `--requirements`, which means its
-#   coverage `--require` gate never runs — so the gate reported PASS having checked
-#   nothing. A false green is worse than a red.
-#
-#   The fix: an undeterminable base is a LOUD exit 3 (ci/README.md's SKIPPED code) with
-#   EMPTY stdout and a machine-readable reason code, never exit 0. These tests assert
-#   that for each way the base can be undeterminable, and assert the good paths still work.
-#
-# The tests point the real script at THROWAWAY git repos via the REPO_ROOT override
-# (ci/lib.sh honours a pre-set REPO_ROOT); the script runs from its true location so it
-# still sources the real lib.sh. No network, no cluster, and the working repo is never
-# touched.
-#
-# Run:  ci/tests/touched-requirements-base.test.sh     (or `make -C ci test`)
-set -uo pipefail    # NOT -e: these tests deliberately run a script that exits non-zero.
-
-CI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
-SCRIPT="${CI_DIR}/touched-requirements.sh"
-
-PASS=0 FAIL=0
-ok()  { printf '  ok   %s\n' "$*"; PASS=$((PASS+1)); }
-bad() { printf '  FAIL %s\n' "$*" >&2; FAIL=$((FAIL+1)); }
-
-TMP="$(mktemp -d)"
-trap 'rm -rf "$TMP"' EXIT
-
-# newrepo <dir> — a throwaway repo with one baseline commit on `main`.
-newrepo() {
-  local d="$1"
-  mkdir -p "$d/notes"
-  git -C "$d" init -q -b main
-  git -C "$d" config user.email [email protected]
-  git -C "$d" config user.name t
-  git -C "$d" config commit.gpgsign false   # throwaway: never sign (ignore host gpg config)
-  echo baseline > "$d/notes/base.txt"
-  git -C "$d" add -A && git -C "$d" commit -qm baseline
-}
-
-# commit_touching <dir> <text> — a commit whose content declares an R-ID.
-commit_touching() {
-  local d="$1" text="$2"
-  printf 'name: %s\n' "$text" > "$d/notes/decl.yaml"
-  git -C "$d" add -A && git -C "$d" commit -qm change
-}
-
-# Runs the script in <dir>; sets RC, OUT (stdout), ERRTXT (stderr), REASON.
-run_in() {
-  local d="$1"; shift
-  local reason_file="${TMP}/reason.$$"
-  : > "$reason_file"
-  OUT="$(REPO_ROOT="$d" TOUCHED_REASON_FILE="$reason_file" "$@" "$SCRIPT" 2>"${TMP}/err.$$")"
-  RC=$?
-  ERRTXT="$(cat "${TMP}/err.$$")"
-  REASON="$(cat "$reason_file")"
-}
-
-# assert_refusal <label> <expected-reason>
-assert_refusal() {
-  local label="$1" want="$2"
-  [ "$RC" = 3 ]        && ok "${label}: exits 3 (SKIPPED), not 0" || bad "${label}: rc=${RC}, expected 3"
-  [ -z "$OUT" ]        && ok "${label}: stdout is EMPTY (no silent requirement list)" \
-                       || bad "${label}: stdout was '${OUT}'"
-  [ "$REASON" = "$want" ] && ok "${label}: reason code '${want}'" \
-                          || bad "${label}: reason '${REASON}', expected '${want}'"
-  grep -q "NOT DETERMINABLE" <<<"$ERRTXT" && ok "${label}: announces the refusal LOUDLY on stderr" \
-                                          || bad "${label}: stderr lacked the loud banner"
-}
-
-echo "== case 1: a branch ahead of local main (no remote) resolves and reports"
-R1="${TMP}/r1"; newrepo "$R1"
-git -C "$R1" checkout -q -b feature
-commit_touching "$R1" implements-R-ALPHA-01
-run_in "$R1"
-[ "$RC" = 0 ] && ok "ahead-of-main: exits 0" || bad "ahead-of-main: rc=${RC}"
-grep -qx "R-ALPHA-01" <<<"$OUT" && ok "ahead-of-main: emits the touched R-ID" \
-                                || bad "ahead-of-main: output was '${OUT}'"
-grep -q "diff base: main" <<<"$ERRTXT" && ok "ahead-of-main: names the base it used" \
-                                       || bad "ahead-of-main: base not reported: ${ERRTXT}"
-
-echo "== case 2: THE FALSE GREEN — HEAD is the base branch itself"
-# This is the state that produced 'no changed files vs base' + 'MERGE GATE: PASS'
-# locally while CI blocked. It must now refuse, not report an empty set.
-R2="${TMP}/r2"; newrepo "$R2"
-git -C "$R2" checkout -q -b feature
-commit_touching "$R2" implements-R-ALPHA-01
-git -C "$R2" checkout -q main            # <- sitting on the base branch
-run_in "$R2"
-assert_refusal "on-base" "on-base"
-
-echo "== case 3: a branch already merged into the base (HEAD is an ancestor)"
-R3="${TMP}/r3"; newrepo "$R3"
-git -C "$R3" checkout -q -b feature
-commit_touching "$R3" implements-R-ALPHA-01
-git -C "$R3" checkout -q main
-git -C "$R3" merge -q --no-edit feature
-git -C "$R3" checkout -q feature         # feature is now behind main
-run_in "$R3"
-assert_refusal "already-merged" "on-base"
-
-echo "== case 4: no base ref resolves at all (no main, no origin/main)"
-R4="${TMP}/r4"; newrepo "$R4"
-git -C "$R4" checkout -q -b work
-commit_touching "$R4" implements-R-ALPHA-01
-git -C "$R4" branch -q -D main
-run_in "$R4"
-assert_refusal "no-base-ref" "no-base"
-
-echo "== case 5: a single-commit repo with no main (the removed HEAD~1 last resort)"
-# HEAD~1 does not exist here. The old code fell back to it, the diff failed, and the
-# empty result was reported as 'no changed files' + exit 0 — a silent false green.
-R5="${TMP}/r5"; mkdir -p "$R5/notes"
-git -C "$R5" init -q -b work
-git -C "$R5" config user.email [email protected]; git -C "$R5" config user.name t
-git -C "$R5" config commit.gpgsign false
-echo only > "$R5/notes/only.txt"; git -C "$R5" add -A; git -C "$R5" commit -qm only
-run_in "$R5"
-assert_refusal "single-commit-no-main" "no-base"
-
-echo "== case 6: BASE_REF names a ref that does not exist"
-R6="${TMP}/r6"; newrepo "$R6"
-git -C "$R6" checkout -q -b feature
-commit_touching "$R6" implements-R-ALPHA-01
-run_in "$R6" env BASE_REF=refs/heads/nope
-assert_refusal "bogus-BASE_REF" "no-base"
-
-echo "== case 7: GITHUB_BASE_REF set but absent from the checkout (shallow clone)"
-R7="${TMP}/r7"; newrepo "$R7"
-git -C "$R7" checkout -q -b feature
-commit_touching "$R7" implements-R-ALPHA-01
-run_in "$R7" env GITHUB_BASE_REF=release-9
-assert_refusal "missing-GITHUB_BASE_REF" "no-base"
-
-echo "== case 8: origin/main is preferred over a stale local main, and staleness warns"
-R8="${TMP}/r8"; newrepo "$R8"
-UP="${TMP}/r8-origin.git"; git init -q --bare -b main "$UP"
-git -C "$R8" remote add origin "$UP"
-git -C "$R8" push -q origin main
-# advance origin/main beyond the local main ref, then branch off it
-git -C "$R8" checkout -q -b ahead
-echo more > "$R8/notes/more.txt"; git -C "$R8" add -A; git -C "$R8" commit -qm more
-git -C "$R8" push -q origin ahead:main
-git -C "$R8" fetch -q origin
-git -C "$R8" checkout -q -b feature origin/main
-commit_touching "$R8" implements-R-ALPHA-01
-run_in "$R8"
-[ "$RC" = 0 ] && ok "origin-preferred: exits 0" || bad "origin-preferred: rc=${RC}"
-grep -q "diff base: origin/main" <<<"$ERRTXT" && ok "origin-preferred: uses origin/main, not local main" \
-                                              || bad "origin-preferred: base line was: ${ERRTXT}"
-grep -q "may be stale" <<<"$ERRTXT" && ok "origin-preferred: warns that local main differs" \
-                                    || bad "origin-preferred: no staleness warning"
-grep -qx "R-ALPHA-01" <<<"$OUT" && ok "origin-preferred: emits the touched R-ID" \
-                                || bad "origin-preferred: output was '${OUT}'"
-
-echo
-if [ "$FAIL" -eq 0 ]; then
-  echo "touched-requirements base-resolution tests: PASS (${PASS} assertions)"; exit 0
-fi
-echo "touched-requirements base-resolution tests: FAIL (${FAIL} failed, ${PASS} passed)" >&2
-exit 1

ci/tests/touched-requirements.test.sh

@@ -1,98 +0,0 @@
-#!/usr/bin/env bash
-# ci/tests/touched-requirements.test.sh — regression tests for the content scan in
-# ci/touched-requirements.sh, focused on rule 3: R-IDs cited only in a whole-line
-# YAML/shell `#` comment are PROSE (a reference), not a declaration, and must NOT be
-# attributed — while R-IDs on real code/spec lines still ARE.
-#
-# The tests point the real script at a THROWAWAY git repo via the REPO_ROOT override
-# (ci/lib.sh honours a pre-set REPO_ROOT); the script itself is run from its true
-# location so it still sources the real lib.sh. No network, no cluster.
-#
-# Run:  ci/tests/touched-requirements.test.sh     (or `make -C ci test`)
-set -euo pipefail
-
-CI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
-SCRIPT="${CI_DIR}/touched-requirements.sh"
-
-PASS=0 FAIL=0
-ok()   { printf '  ok   %s\n' "$*"; PASS=$((PASS+1)); }
-bad()  { printf '  FAIL %s\n' "$*"; FAIL=$((FAIL+1)); }
-
-# assert_absent <needle> <haystack> <msg>
-assert_absent() {
-  if grep -qxF "$1" <<<"$2"; then bad "$3 (found unexpected '$1' in: $(tr '\n' ' ' <<<"$2"))";
-  else ok "$3"; fi
-}
-# assert_present <needle> <haystack> <msg>
-assert_present() {
-  if grep -qxF "$1" <<<"$2"; then ok "$3";
-  else bad "$3 (missing '$1' from: $(tr '\n' ' ' <<<"$2"))"; fi
-}
-
-TMP="$(mktemp -d)"
-trap 'rm -rf "$TMP"' EXIT
-
-git -C "$TMP" init -q -b main
-git -C "$TMP" config user.email [email protected]
-git -C "$TMP" config user.name t
-git -C "$TMP" config commit.gpgsign false   # throwaway repo: never sign (ignore host gpg config)
-mkdir -p "$TMP/notes"                 # a path OUTSIDE path_requirements() so we isolate the content scan
-echo "baseline" > "$TMP/notes/base.txt"
-git -C "$TMP" add -A && git -C "$TMP" commit -qm baseline
-
-git -C "$TMP" checkout -q -b feature
-
-# A YAML file: an R-ID in a whole-line comment (must be IGNORED) and an R-ID on a real
-# value line (must be DETECTED). Also an indented comment, to prove leading space is fine.
-cat > "$TMP/notes/example.yaml" <<'YAML'
-# This header explains R-DOC-01 and must NOT be attributed.
-kind: Thing
-metadata:
-  name: implements-R-REAL-01
-spec:
-  note: value  # trailing inline comment mentioning R-INLINE-01 (conservatively still seen)
-    # deeply indented comment about R-DOC-02 — also NOT attributed
-YAML
-
-# A shell file: a `#` comment R-ID (IGNORED) plus a real command line R-ID (DETECTED).
-cat > "$TMP/notes/example.sh" <<'SH'
-#!/usr/bin/env bash
-# doc: this script relates to R-SHDOC-01 (comment only — ignored)
-echo "runs check for R-SHREAL-01"
-SH
-
-git -C "$TMP" add -A && git -C "$TMP" commit -qm feature
-
-OUT="$(REPO_ROOT="$TMP" BASE_REF=main "$SCRIPT" 2>/dev/null)"
-printf 'touched set:\n%s\n---\n' "$OUT"
-
-# Comment-only citations are dropped (rule 3):
-assert_absent  "R-DOC-01"    "$OUT" "whole-line YAML comment R-ID is NOT attributed"
-assert_absent  "R-DOC-02"    "$OUT" "indented YAML comment R-ID is NOT attributed"
-assert_absent  "R-SHDOC-01"  "$OUT" "shell '#' comment R-ID is NOT attributed"
-# Real code/spec lines still count:
-assert_present "R-REAL-01"   "$OUT" "R-ID on a real YAML value line IS attributed"
-assert_present "R-SHREAL-01" "$OUT" "R-ID on a real shell command line IS attributed"
-# Conservative: a trailing inline comment on a REAL line is still seen (only whole-line
-# comments are dropped) — documents the intentional boundary of the fix.
-assert_present "R-INLINE-01" "$OUT" "inline comment on a real line is (conservatively) still seen"
-
-# Removed comment lines are also excluded, while removed real lines still count. The file
-# must EXIST at the base, so we diff the removal against the commit that added it.
-git -C "$TMP" checkout -q -b feature2 main
-cat > "$TMP/notes/todelete.yaml" <<'YAML'
-# will delete a comment citing R-DELDOC-01 and a real line citing R-DELREAL-01
-kind: Thing
-name: implements-R-DELREAL-01
-YAML
-git -C "$TMP" add -A && git -C "$TMP" commit -qm add
-ADD_SHA="$(git -C "$TMP" rev-parse HEAD)"
-git -C "$TMP" rm -q "$TMP/notes/todelete.yaml" && git -C "$TMP" commit -qm remove
-OUT2="$(REPO_ROOT="$TMP" BASE_REF="$ADD_SHA" "$SCRIPT" 2>/dev/null)"
-printf 'touched set (removal):\n%s\n---\n' "$OUT2"
-assert_absent  "R-DELDOC-01"  "$OUT2" "REMOVED whole-line comment R-ID is NOT attributed"
-assert_present "R-DELREAL-01"  "$OUT2" "REMOVED real-line R-ID IS attributed"
-
-echo
-if [ "$FAIL" -eq 0 ]; then echo "touched-requirements tests: PASS ($PASS assertions)"; exit 0;
-else echo "touched-requirements tests: FAIL ($FAIL failed, $PASS passed)"; exit 1; fi

ci/touched-requirements.sh

@@ -28,99 +28,28 @@ CI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 # shellcheck disable=SC1091
 source "${CI_DIR}/lib.sh"
 
-# ---------------------------------------------------------------------------------
+# Base ref to diff against. In Gitea/GitHub Actions PR runs this is the target
-# BASE-REF RESOLUTION — and why it now REFUSES rather than guesses.
+# branch (main). Locally, default to main; override with BASE_REF.
-#
-# BUG THIS FIXES (local/CI divergence, 2026-08-31): every failure to establish a
-# reliable diff base used to degrade SILENTLY to an EMPTY touched set, which main()
-# reported as "no changed files vs base" and exit 0. An empty set means the coverage
-# `--require` gate is never applied, so the merge gate printed "MERGE GATE: PASS"
-# having checked NOTHING — while CI, with a correct base, computed 17 touched
-# requirements and blocked. A false green is the worst outcome this project can
-# produce, so every one of those degradations is now a LOUD exit 3:
-#
-#   * HEAD~1 last resort  REMOVED. It silently narrowed the diff to the last commit
-#                         (and on a one-commit repo failed into an empty set).
-#   * two-dot fallback    REMOVED. It masked a failing three-dot diff as "no changes".
-#   * unresolvable base   exit 3 (was: HEAD~1 or empty).
-#   * diff command fails  exit 3 (was: empty set).
-#   * HEAD not ahead of   exit 3 (was: exit 0 with an empty set — this is the case
-#     the base                that produced the false green: running the gate while
-#                             sitting on `main`, or on a branch already merged).
-#
-# Exit 3 is the ci/README.md "SKIPPED / not-run" code: NOT a pass, NOT a fail. Callers
-# (ci/gate.sh) must surface it as an UNDECIDED merge decision. An honest, noisy skip
-# beats a quiet false pass.
-#
-# Order of preference: BASE_REF (explicit) > GITHUB_BASE_REF (Actions PR runs, target
-# branch) > origin/main > main. Each candidate is VERIFIED to resolve to a commit
-# before it is used, so a shallow checkout that lacks origin/main is reported, not
-# papered over.
 : "${BASE_REF:=}"
-BASE=""            # resolved base ref
-BASE_SOURCE=""     # where it came from (for the log line)
-BASE_PROBLEM=""    # why resolution failed, when it did
-
-rev() { git -C "$REPO_ROOT" rev-parse --verify -q "$1^{commit}" 2>/dev/null; }
-
-# Loud, structured refusal. Prints to stderr only; stdout stays empty so a caller that
-# captures stdout gets NO requirement list rather than a silently-empty one.
-#
-# $1 is a machine-readable REASON CODE, also written to $TOUCHED_REASON_FILE when that
-# env var names a path. ci/gate.sh uses it to tell the two cases apart:
-#   on-base      HEAD is the base branch (or already merged) — nothing to gate. Legitimate
-#                (a push-to-main CI run lands here), so the gate reports NOT APPLICABLE
-#                rather than PASS, and does not fail the job.
-#   no-base      no reliable base ref exists in this checkout — the ENVIRONMENT is broken,
-#   diff-failed  or git could not produce the diff. A merge decision is impossible, so the
-#                gate BLOCKS rather than guessing.
-undecidable() {
-  local code="$1"; shift
-  if [ -n "${TOUCHED_REASON_FILE:-}" ]; then printf '%s\n' "$code" > "$TOUCHED_REASON_FILE"; fi
-  err "TOUCHED-REQUIREMENT SET NOT DETERMINABLE (${code})"
-  err "  reason: $*"
-  err "  consequence: the R-TEST-03 coverage gate CANNOT be applied to this checkout."
-  err "  This is exit 3 (SKIPPED / not-run), never a pass — see ci/README.md."
-  err "  Fix one of:"
-  err "    * check out the feature branch you mean to gate (not the base branch), or"
-  err "    * git fetch origin main   (so origin/main exists and is current), or"
-  err "    * BASE_REF=<ref> make -C ci gate   to name the base explicitly."
-  exit 3
-}
-
 resolve_base() {
-  local cand
+  if [ -n "$BASE_REF" ]; then echo "$BASE_REF"; return; fi
-  if [ -n "$BASE_REF" ]; then
+  if [ -n "${GITHUB_BASE_REF:-}" ]; then echo "origin/${GITHUB_BASE_REF}"; return; fi
-    if rev "$BASE_REF" >/dev/null; then BASE="$BASE_REF"; BASE_SOURCE="BASE_REF"; return 0; fi
+  if git -C "$REPO_ROOT" rev-parse --verify -q origin/main >/dev/null; then echo "origin/main"; return; fi
-    BASE_PROBLEM="BASE_REF='${BASE_REF}' does not resolve to a commit in ${REPO_ROOT}"
+  if git -C "$REPO_ROOT" rev-parse --verify -q main >/dev/null; then echo "main"; return; fi
-    return 1
+  echo "HEAD~1"  # last resort
-  fi
-  if [ -n "${GITHUB_BASE_REF:-}" ]; then
-    for cand in "origin/${GITHUB_BASE_REF}" "${GITHUB_BASE_REF}"; do
-      if rev "$cand" >/dev/null; then BASE="$cand"; BASE_SOURCE="GITHUB_BASE_REF"; return 0; fi
-    done
-    BASE_PROBLEM="GITHUB_BASE_REF='${GITHUB_BASE_REF}' is set, but neither 'origin/${GITHUB_BASE_REF}' nor '${GITHUB_BASE_REF}' exists here (shallow checkout? the workflow needs fetch-depth: 0)"
-    return 1
-  fi
-  for cand in origin/main main; do
-    if rev "$cand" >/dev/null; then BASE="$cand"; BASE_SOURCE="default"; return 0; fi
-  done
-  BASE_PROBLEM="no base ref resolves (tried origin/main, main) in ${REPO_ROOT}"
-  return 1
 }
 
-# Both diff helpers assume BASE is already resolved and validated by main(). They return
-# git's exit status so the CALLER can distinguish "no changes" from "the diff failed" —
-# conflating those two is exactly what produced the false green.
 changed_files() {
+  local base; base="$(resolve_base)"
   # Three-dot: changes on this branch since it diverged from base.
-  git -C "$REPO_ROOT" diff --name-only "${BASE}...HEAD"
+  if git -C "$REPO_ROOT" diff --name-only "${base}...HEAD" 2>/dev/null; then return; fi
+  git -C "$REPO_ROOT" diff --name-only "${base}" 2>/dev/null || true
 }
 
 # The ADDED/REMOVED lines of the change, for the R-ID content scan — excluding
 # documentation.
 #
-# THREE scoping rules, all learned from the gate blocking merges it should not have:
+# TWO scoping rules, both learned from the gate blocking merges it should not have:
 #
 #  1. Scan the DIFF, not whole file contents. Grepping whole files meant a change
 #     inherited every requirement those files had EVER mentioned, on lines the author
@@ -137,34 +66,14 @@ changed_files() {
 #     that matter (conformance/, schemas/, catalog/, clusters/, ci/) are covered by
 #     the path->requirement fallback regardless of file type.
 #
-#  3. Skip COMMENT-ONLY lines in non-Markdown files — rule 2's logic, applied to the
-#     prose that lives INSIDE code files. A requirement ID cited in a YAML/shell `#`
-#     comment is documentation, not a declaration, exactly like an ID in a `*.md` file.
-#     Rule 2 drops whole Markdown files, but a `.gitea/workflows/*.yml` or `ci/*.sh`
-#     HEADER that explains a requirement would otherwise re-attribute it. Concretely:
-#     gate.yml's header cites R-TEST-06 (enforced by the spec-immutability STAGE, not by
-#     any Chainsaw test, so a known-uncovered ID); before this rule, editing that comment
-#     re-attributed R-TEST-06 and the coverage `--require` check false-blocked the merge —
-#     the same self-referential trap rule 2 fixed for `*.md`. Applied in main()'s content
-#     scan (see there): after the diff marker is stripped, a line whose first non-blank
-#     character is `#` is dropped BEFORE the R-ID grep. Deliberately conservative — only
-#     WHOLE-LINE comments are dropped; an R-ID on a real code/spec line is still detected
-#     (authoritative declarations — test tags, policies, ServiceDefinitions — are never a
-#     bare comment), so this only narrows PROSE, never a declaration.
-#
 # Removed lines count: deleting a test's requirement tag, or a policy's self-declared
 # ID, touches that requirement just as much as adding one.
-# `:(exclude)*.md` drops Markdown from the content scan (rule 2 above).
+# `:(exclude)*.md` drops Markdown from the content scan (rule 2 above). Three-dot to
-# `:(exclude)ci/tests/` drops this dir's OWN unit-test fixtures: those files embed example
+# match changed_files(): changes on this branch since it diverged from base.
-#   R-IDs as DATA (asserting the scan's behaviour), never as declarations — the same
-#   "reference, not declaration" logic as rule 2. Their gate coverage is unaffected: any
-#   ci/ change (ci/tests/ included) still attributes R-GITOPS-01/R-TEST-02/R-TEST-03 via the
-#   path->requirement map below. NOTE: this excludes ci/tests/ only — conformance/tests/
-#   carry AUTHORITATIVE `conformance.home.k3s/R-*` labels and are deliberately still scanned.
-# Three-dot to match changed_files(): changes on this branch since it diverged from base.
 changed_diff() {
-  local -a excl=(':(exclude)*.md' ':(exclude)ci/tests/')
+  local base; base="$(resolve_base)"
-  git -C "$REPO_ROOT" diff -U0 "${BASE}...HEAD" -- . "${excl[@]}"
+  if git -C "$REPO_ROOT" diff -U0 "${base}...HEAD" -- . ':(exclude)*.md' 2>/dev/null; then return; fi
+  git -C "$REPO_ROOT" diff -U0 "${base}" -- . ':(exclude)*.md' 2>/dev/null || true
 }
 
 # Coarse path -> requirement map. Deliberately conservative; extend as subtrees
@@ -189,34 +98,9 @@ path_requirements() {
 }
 
 main() {
-  local files reqs="" rc=0 base_sha head_sha local_main merge_base
+  local files reqs=""
-
+  files="$(changed_files)"
-  resolve_base || undecidable no-base "$BASE_PROBLEM"
+  [ -n "$files" ] || { log "no changed files vs base; empty touched-requirement set"; exit 0; }
-  base_sha="$(rev "$BASE")" || undecidable no-base "resolved base '${BASE}' vanished"
-  head_sha="$(rev HEAD)"    || undecidable no-base "HEAD does not resolve to a commit (empty repo?)"
-  log "diff base: ${BASE} (${BASE_SOURCE}) @ ${base_sha:0:12} ... HEAD @ ${head_sha:0:12}"
-
-  # Advisory: a never-fetched origin/main silently WIDENS the set (over-broad, never a
-  # false green) — surfaced so an unexpectedly large touched set has an explanation.
-  if [ "$BASE" = "origin/main" ] && local_main="$(rev main)" && [ -n "$local_main" ] \
-     && [ "$local_main" != "$base_sha" ]; then
-    warn "local 'main' (${local_main:0:12}) != origin/main (${base_sha:0:12}) — if origin/main"
-    warn "  was not fetched recently the touched set may be stale: git fetch origin main"
-  fi
-
-  merge_base="$(git -C "$REPO_ROOT" merge-base "$BASE" HEAD 2>/dev/null)" \
-    || undecidable no-base "no merge base between '${BASE}' and HEAD (unrelated histories, or a shallow clone)"
-  if [ "$merge_base" = "$head_sha" ]; then
-    undecidable on-base "HEAD is not ahead of '${BASE}' (merge-base == HEAD): this checkout contains no branch commits to gate. The empty touched set would be empty BY DEFAULT, not by verification."
-  fi
-
-  files="$(changed_files)" || rc=$?
-  [ "$rc" -eq 0 ] || undecidable diff-failed "git diff --name-only ${BASE}...HEAD failed (rc=${rc})"
-  if [ -z "$files" ]; then
-    warn "HEAD is ahead of ${BASE} but the three-dot diff is empty (self-cancelling commits?)"
-    warn "  -> genuinely empty touched-requirement set; base WAS established."
-    exit 0
-  fi
 
   while IFS= read -r f; do
     [ -n "$f" ] || continue
@@ -225,18 +109,9 @@ main() {
 
   # Self-declared R-IDs on the ADDED/REMOVED lines of the change (see changed_diff).
   # `^[+-]` keeps hunk headers (`@@`) and the `+++`/`---` file headers out of scope.
-  # `sed 's/^[+-]//'` then strips the single-char diff marker (safe: -U0 emits one), so
+  reqs+=" $(changed_diff \
-  #   what remains is the SOURCE line, and `grep -vE '^[[:space:]]*#'` drops whole-line
-  #   YAML/shell comments (rule 3 in changed_diff's header) BEFORE the R-ID grep — a
-  #   requirement cited only in a `#` comment is prose, not a declaration.
-  local diff_txt
-  diff_txt="$(changed_diff)" || rc=$?
-  [ "$rc" -eq 0 ] || undecidable diff-failed "git diff -U0 ${BASE}...HEAD failed (rc=${rc})"
-  reqs+=" $(printf '%s\n' "$diff_txt" \
     | grep -E '^[+-]' \
     | grep -vE '^(\+\+\+|---)' \
-    | sed -E 's/^[+-]//' \
-    | grep -vE '^[[:space:]]*#' \
     | grep -hoE 'R-[A-Z]+-[0-9]+' \
     | tr '\n' ' ' || true)"
 

ci/versions.env

@@ -9,7 +9,7 @@
 # whatever platform/kyverno and conformance/ pin — see integration notes in README.
 
 # L0 policy gate (Kyverno CLI) + ServiceDefinition validation (kyverno-json)
-KYVERNO_VERSION="1.19.0"
+KYVERNO_VERSION="1.13.4"
 KYVERNO_JSON_VERSION="0.0.3"
 
 # L1/L2/L3 declarative runner (Chainsaw) — invoked via conformance/ entrypoint,
@@ -18,7 +18,7 @@ CHAINSAW_VERSION="0.2.12"
 
 # Lint / manifest tooling
 YAMLLINT_VERSION="1.35.1"
-KUBECTL_VERSION="1.36.4"
+KUBECTL_VERSION="1.31.4"
 YQ_VERSION="4.44.6"
 
 # ServiceDefinition JSON-Schema validation (schemas/validate.sh). Pip-installed;

clusters/README.md

@@ -29,9 +29,8 @@ clusters/
 
 ## How the overlay model satisfies R-ENV-02
 
-The base authors the reconciliation graph **once**. `platform.yaml` and the app-wave Kustomizations
+The base authors the reconciliation graph **once**. `platform.yaml` and `apps.yaml` (Flux
-(`apps-wave-0/1/2.yaml` — the single `apps.yaml` was retired; `apps-vms.yaml` deliberately omits it,
+`Kustomization`s) both declare `postBuild.substituteFrom` the `cluster-vars` ConfigMap. Platform and
-see below) declare `postBuild.substituteFrom` the `cluster-vars` ConfigMap. Platform and
 app manifests use `${TOKEN}` placeholders for every value that legitimately differs between
 environments (issuer, replica counts, storage sizes, retention, node placement).
 
@@ -40,31 +39,6 @@ they add no resources of their own and never copy/fork a base manifest. Promotio
 ([`docs/02`](../../home-network-k3s/docs/02-environments-and-provisioning.md)) is a change to the
 prod overlay's variables, nothing else.
 
-### LAN addressing is a knob, not a literal (fixed 2026-08-31)
-
-Until 2026-08-31 the base manifests hardcoded the **dev** LAN: the k3s API host Cilium dials, the
-Cilium LB IP pool range, the pinned crafty/unifi L4 VIPs and the off-cluster S3 backup endpoint. That
-broke R-ENV-02 outright — `clusters/prod` would have rendered dev addresses (192.168.1.x) onto the
-prod LAN (192.168.2.x). They are now the `cluster-vars` "LAN addressing" block
-(`K3S_API_HOST`, `CILIUM_LB_POOL_START`/`_STOP`, `UNIFI_L4_VIP`, `CRAFTY_L4_VIP`,
-`S3_BACKUP_ENDPOINT`, alongside the pre-existing `GATEWAY_VIP`), set explicitly by BOTH overlays.
-
-Two rules when touching that block:
-
-1. **Substitution coverage.** A `${TOKEN}` only resolves in a path whose Flux Kustomization declares
-   `postBuild.substituteFrom: cluster-vars`; anywhere else the literal string `${TOKEN}` ships to the
-   cluster. `apps-vms` was the one path without it and now has it — which means every shell `${...}`
-   in a VM manifest's cloud-init must be escaped `$${...}` (see `clusters/base/apps-vms.yaml` for the
-   full envsubst escape contract).
-2. **`GATEWAY_VIP` must be inside `CILIUM_LB_POOL_START..STOP`,** and in practice equal to START: the
-   `shared` Gateway pins no address, so Cilium LB-IPAM gives it the pool's first free IP while
-   split-horizon CoreDNS answers with `${GATEWAY_VIP}`. A VIP outside the pool blackholes in-cluster
-   hairpin traffic while the Gateway still looks healthy.
-
-Values the operator has not decided are `TBD-PROD-*` placeholders, not guesses: prod inherits the
-DEV base defaults if an overlay op is simply omitted, so an invalid value that fails loudly is
-strictly safer than a plausible one that quietly comes up on the wrong network.
-
 ## How Flux is pointed at an environment
 
 `flux bootstrap` (run by WS-INFRA in Phase 1) targets one environment's path per cluster:

clusters/base/apps-vms.yaml

@@ -23,21 +23,7 @@ spec:
     kind: GitRepository
     name: flux-system
   path: ./apps/vms
-  # postBuild.substituteFrom ENABLED 2026-08-31 (R-ENV-02). It was previously omitted because Flux's
+  # NO postBuild.substituteFrom: Flux's envsubst would run over the ENTIRE manifest, including the
-  # envsubst runs over the ENTIRE manifest — including VM cloud-init shell scripts — and would mangle
+  # VM cloud-init shell scripts, and mangle every `${...}` (e.g. ${UOS_URL}, ${PIPESTATUS[0]}) —
-  # every `${...}`; the omission was safe only while no VM manifest referenced a cluster-var. It now
+  # corrupting the in-guest install. VM manifests reference no cluster-vars, so substitution is
-  # does: apps/vms/unifi/50-service-l4.yaml pins the UniFi L4 VIP from ${UNIFI_L4_VIP} instead of the
+  # unneeded here. A future VM that needs a cluster-var must escape its shell `${...}` as `$${...}`.
-  # hardcoded dev-LAN literal that would have shipped 192.168.1.54 onto the PROD LAN.
-  #
-  # The escape contract the old comment prescribed is now in force and MUST be honoured by every VM
-  # manifest under this path (verified with `flux envsubst`):
-  #   `${VAR}`  -> substituted (empty when unset)  => escape shell braces as `$${VAR}`, which
-  #                envsubst emits back as a literal `${VAR}`.
-  #   `$VAR`    -> left alone by envsubst          => bare shell vars need NO escaping.
-  #   `$(cmd)`  -> left alone by envsubst          => command substitution needs NO escaping.
-  #   `${ARR[0]}` (e.g. ${PIPESTATUS[0]}) -> HARD ERROR "missing closing brace", which fails this
-  #                whole Kustomization. NEVER use an indexed expansion here, escaped or not.
-  postBuild:
-    substituteFrom:
-      - kind: ConfigMap
-        name: cluster-vars

clusters/base/cluster-vars.yaml

@@ -21,56 +21,13 @@ data:
   DNS_ZONE: "keithjackson.dev"              # the actual Cloudflare ZONE (BASE_DOMAIN is a subdomain of it);
                                             # ExternalDNS domain-filter must be the zone, not the subdomain
   ACME_CLUSTER_ISSUER: "letsencrypt-staging" # dev: staging | prod: letsencrypt-prod (ADR-0006)
-  GATEWAY_VIP: "192.168.1.53"                # in-cluster split-horizon target (R-DNS-01/03): the L2-announced
+  GATEWAY_VIP: "10.73.0.240"                 # in-cluster split-horizon target (R-DNS-01/03): the L2-announced
-                                            # ingress VIP of the `shared` Gateway. MUST equal the FIRST address of
+                                            # ingress VIP of the `shared` Gateway (first IP of the Cilium
-                                            # ${CILIUM_LB_POOL_START}..${CILIUM_LB_POOL_STOP} below — the Gateway
+                                            # LB IP pool 10.73.0.240-.250, platform-config/cilium/l2-ip-pool.yaml).
-                                            # takes the pool's first free IP, and coredns-custom answers
+                                            # coredns-custom answers *.home.keithjackson.dev with THIS so pod->
-                                            # *.home.keithjackson.dev with THIS so pod -> public-hostname traffic
+                                            # public-hostname traffic hairpins through the Gateway (correct SNI ->
-                                            # hairpins through the Gateway (correct SNI -> wildcard cert).
+                                            # wildcard cert). Base default = dev VIP; a prod overlay must set the
-                                            # Base default = the DEV LAN VIP (was 10.73.0.240, an address on the
+                                            # prod LAN VIP alongside the prod L2 pool range when hardware is built.
-                                            # long-RETIRED libvirt network — corrected 2026-08-31 so an overlay that
-                                            # forgets this knob at least lands on a REAL network). A prod overlay
-                                            # MUST still set the prod LAN VIP alongside the prod L2 pool range.
-
-  # --- LAN addressing (R-ENV-02) — the ONE block every new environment must re-supply -------------
-  # These were HARDCODED into base manifests until 2026-08-31, which broke the golden constraint:
-  # `clusters/prod` would have rendered the DEV LAN (192.168.1.0/24) onto the prod LAN (192.168.2.0/24)
-  # — a silent, cross-environment IP collision. They are knobs now. Base defaults ARE the dev values
-  # (same convention as GATEWAY_VIP above: the base default is the known-good working value), so the
-  # dev overlay restates them verbatim and a prod overlay MUST override EVERY knob in this block
-  # before `flux bootstrap --path=clusters/prod`.
-  #
-  # Substitution coverage (verify before adding a knob here): every path that consumes these declares
-  # postBuild.substituteFrom cluster-vars — platform (K3S_API_HOST), platform-config
-  # (CILIUM_LB_POOL_*, S3_BACKUP_ENDPOINT), crafty (CRAFTY_L4_VIP), apps-wave-0 (S3_BACKUP_ENDPOINT),
-  # velero (S3_BACKUP_ENDPOINT), apps-vms (UNIFI_L4_VIP). A ${TOKEN} in an UNSUBSTITUTED path would
-  # ship the literal string into the cluster.
-
-  # k3s API endpoint Cilium's kube-proxy replacement dials (platform/cilium/helmrelease.yaml).
-  # [parity] MUST equal the Ansible bootstrap's k3s_api_host (infra/ansible/roles/cilium/defaults/
-  # main.yml -> cilium-values.yaml.j2, already inventory-parameterized). Wrong value = Cilium cannot
-  # reach the API server on adopt and the CNI breaks. Port stays 6443 in both envs, so it is NOT a knob.
-  K3S_API_HOST: "192.168.1.50"
-
-  # CiliumLoadBalancerIPPool `default-pool` (platform-config/cilium/l2-ip-pool.yaml): the VIP range
-  # L2-announced on the flat LAN. Must sit OUTSIDE the node IPs and any DHCP scope, and must match the
-  # tofu reservation (cilium_l2_vip_range). INVARIANT: ${GATEWAY_VIP} must be INSIDE this range and in
-  # practice EQUAL to START — the `shared` Gateway pins no address, so LB-IPAM hands it the pool's
-  # first free IP. (⚠ the operator-supplied PROD values currently break this; see l2-ip-pool.yaml and
-  # the flagged block in clusters/prod/kustomization.yaml.)
-  CILIUM_LB_POOL_START: "192.168.1.53"
-  CILIUM_LB_POOL_STOP: "192.168.1.60"
-
-  # Pinned VIPs carved out of that pool by name (lbipam.cilium.io/ips). These are L4 workloads the
-  # HTTPS Gateway cannot express (raw TCP/UDP), so they hold their own stable address:
-  UNIFI_L4_VIP: "192.168.1.54"               # UniFi controller VM: inform/STUN/adoption (apps/vms/unifi)
-  CRAFTY_L4_VIP: "192.168.1.55"              # Crafty: Minecraft Java (TCP) + Bedrock (UDP) game ports
-
-  # Off-cluster S3 backup target (R-STATE-03, R-IDP-05) — the on-Proxmox MinIO, plain HTTP, path-style.
-  # Consumed by BOTH backup paths: every CNPG Cluster's barmanObjectStore.endpointURL (keycloak-db,
-  # nextcloud-db, semaphore-db) and Velero's backupStorageLocation s3Url. Full URL incl. scheme+port.
-  # Prod needs its OWN target: a backup that lands on the dev LAN's MinIO is not off-cluster DR.
-  S3_BACKUP_ENDPOINT: "http://192.168.1.9:9000"
 
   # --- storage (ADR-0003) ---
   LONGHORN_DEFAULT_REPLICA_COUNT: "3"       # default StorageClass replica count (3 on 3 nodes)

clusters/dev/kustomization.yaml

@@ -148,44 +148,12 @@ patches:
       - op: replace
         path: /data/ACME_CLUSTER_ISSUER
         value: "letsencrypt-prod"
-      # --- dev LAN addressing (192.168.1.0/24 on the Proxmox LAN) -----------------------------
-      # EVERY knob in cluster-vars' "LAN addressing" block is restated here EXPLICITLY, even where it
-      # equals the base default. These values are what the LIVE dev cluster is running right now, so
-      # the substituted output is byte-identical to the pre-parameterization hardcoded manifests
-      # (verified 2026-08-31). Never let this block drift from the live cluster.
-      #
       # Split-horizon DNS target: the Cilium-L2 VIP of the `shared` Gateway on the Proxmox LAN
-      # (= CILIUM_LB_POOL_START, the pool's first address). coredns-custom answers
+      # (base default 10.73.0.240 was the old libvirt VIP). coredns-custom answers
       # *.home.keithjackson.dev with this so in-cluster OIDC clients reach Keycloak via the Gateway.
       - op: replace
         path: /data/GATEWAY_VIP
         value: "192.168.1.53"
-      # k3s API endpoint Cilium's kube-proxy replacement dials — the primary control-plane node
-      # (k3s-1). [parity] equals the Ansible inventory's k3s_api_host for dev.
-      - op: replace
-        path: /data/K3S_API_HOST
-        value: "192.168.1.50"
-      # Cilium LB IP pool = the tofu reservation cilium_l2_vip_range .53-.60, outside the node IPs
-      # (.50/.51/.52) and outside the DHCP scope. .53 Gateway, .54 unifi, .55 crafty; .56-.60 free.
-      - op: replace
-        path: /data/CILIUM_LB_POOL_START
-        value: "192.168.1.53"
-      - op: replace
-        path: /data/CILIUM_LB_POOL_STOP
-        value: "192.168.1.60"
-      # Pinned L4 VIPs out of that pool (lbipam.cilium.io/ips). Changing either MOVES a live device-
-      # facing endpoint: unifi is the inform/STUN target every adopted UniFi device holds.
-      - op: replace
-        path: /data/UNIFI_L4_VIP
-        value: "192.168.1.54"
-      - op: replace
-        path: /data/CRAFTY_L4_VIP
-        value: "192.168.1.55"
-      # Off-cluster backup target: the on-Proxmox MinIO (plain HTTP, path-style) that every CNPG
-      # barmanObjectStore and Velero's backupStorageLocation write to.
-      - op: replace
-        path: /data/S3_BACKUP_ENDPOINT
-        value: "http://192.168.1.9:9000"
       # dev mirrors prod's 3-node topology, so the replicated tier keeps 3 replicas.
       - op: replace
         path: /data/LONGHORN_DEFAULT_REPLICA_COUNT

clusters/prod/kustomization.yaml

@@ -49,68 +49,8 @@ patches:
       - op: replace
         path: /data/VELERO_RETENTION
         value: "720h"
-      # ==========================================================================================
-      # PROD LAN ADDRESSING (192.168.2.0/24). Nodes: k3s-prod-1/2/3 = .10 / .11 / .12.
-      # ==========================================================================================
-      # These knobs were HARDCODED to the dev LAN inside base manifests until 2026-08-31; prod would
-      # have come up ANNOUNCING DEV ADDRESSES (192.168.1.x) on the prod LAN. They are knobs now, and
-      # this overlay supplies every one of them. Anything the operator has NOT yet decided stays an
-      # INVALID `TBD-PROD-*` placeholder on purpose: base defaults are the DEV values, so an OMITTED
-      # op means prod silently inherits a dev address — far worse than a failed apply. `TBD-PROD-*`
-      # is not a valid IP/URL, so Cilium CRD validation / LB-IPAM / CNPG / Velero reject it LOUDLY.
-      #
-      # ------------------------------------------------------------------------------------------
-      # ADDRESS PLAN (operator, 2026-08-31) — see infra/PROD-PROVISIONING.md §4 for the full table.
-      # ------------------------------------------------------------------------------------------
-      #   nodes                 192.168.2.10 / .11 / .12   (Ansible inventory)
-      #   control-plane VIP     192.168.2.9                (Ansible group_vars/k3s_prod — NOT here)
-      #   ingress Gateway VIP   192.168.2.20               (GATEWAY_VIP, below)
-      #   Cilium LB pool        192.168.2.20 - .30         (CILIUM_LB_POOL_*, below)
-      #
-      # CONTAINMENT INVARIANT SATISFIED: GATEWAY_VIP (.20) IS CILIUM_LB_POOL_START, mirroring dev
-      # (.53 of .53-.60). This matters because the `shared` Gateway (platform/gateway/gateway.yaml)
-      # pins NOTHING — no `lbipam.cilium.io/ips`, no spec.addresses (unlike platform/crafty and
-      # apps/vms/unifi, which DO pin) — so Cilium LB-IPAM assigns it the FIRST FREE address of a
-      # matching pool block. Had GATEWAY_VIP stayed outside every pool (the original .21-.30
-      # proposal), the Gateway would have come up on .21 while split-horizon coredns-custom kept
-      # answering *.home.keithjackson.dev with .20 — an address nothing announces — blackholing
-      # in-cluster hairpin traffic (OIDC clients -> Keycloak/Grafana) while looking healthy.
-      # KEEP GATEWAY_VIP == CILIUM_LB_POOL_START when changing either.
-      # The control-plane VIP (.9) is deliberately OUTSIDE this pool: kube-vip floats it, and
-      # LB-IPAM must never be able to hand it to a Service.
-      - op: replace
-        path: /data/GATEWAY_VIP
-        value: "192.168.2.20"
-      # Cilium LB IP pool, operator-supplied: 192.168.2.20-.30 (START == GATEWAY_VIP, see above).
-      # Mirror this range into the prod tofu reservation (cilium_l2_vip_range) if prod ever gets one.
-      - op: replace
-        path: /data/CILIUM_LB_POOL_START
-        value: "192.168.2.20"
-      - op: replace
-        path: /data/CILIUM_LB_POOL_STOP
-        value: "192.168.2.30"
-      # --- still UNDECIDED -> loud placeholders (must be real before `flux bootstrap`) ------------
-      # k3s API endpoint Cilium's kube-proxy replacement dials. This is the prod CONTROL-PLANE VIP
-      # (kube-vip), NOT .20 (the ingress VIP) and NOT the single node address .10. It is owned
-      # Ansible-side (prod inventory k3s_api_host) and MUST equal that value — [parity] contract in
-      # platform/cilium/helmrelease.yaml. Not yet decided.
-      - op: replace
-        path: /data/K3S_API_HOST
-        value: "192.168.2.10"
-      # Pinned L4 VIPs, both carved out of the pool by lbipam.cilium.io/ips. Not yet assigned for
-      # prod. Leaving the dev literals in place would hand a prod LoadBalancer a dev-LAN address, so
-      # they stay placeholders; whatever is chosen MUST fall inside [POOL_START, POOL_STOP] and not
-      # collide with the Gateway VIP or the node IPs (.10/.11/.12).
-      - op: replace
-        path: /data/UNIFI_L4_VIP
-        value: "192.168.2.21"
-      - op: replace
-        path: /data/CRAFTY_L4_VIP
-        value: "192.168.2.22"
-      # Off-cluster DR target. Dev's versitygw/MinIO (192.168.1.9:9000) sits on the DEV side of the
-      # network — a prod backup written there is not off-cluster DR and would mix two environments'
-      # PITR streams in one bucket. Operator supplies the prod endpoint (full URL incl. scheme+port).
-      - op: replace
-        path: /data/S3_BACKUP_ENDPOINT
-        value: "TBD-PROD-S3-BACKUP-ENDPOINT"
       # TODO(WS-INFRA, Phase 7): pin prod node placement once the bare-metal role labels exist.
+      # TODO(WS-NET, Phase 7): set GATEWAY_VIP to the prod LAN ingress VIP (add a `replace` op here)
+      #   at the same time the prod Cilium LB IP pool range is supplied — the split-horizon
+      #   coredns-custom (platform/coredns) answers *.home.keithjackson.dev with ${GATEWAY_VIP}, so
+      #   the prod value must match the prod pool's Gateway address. Base default is the dev VIP.

conformance/.gitignore

@@ -5,6 +5,3 @@ _artifacts/
 
 # Downloaded pinned tool binaries (run.sh may fetch chainsaw/kubectl here).
 .bin/
-
-# Python bytecode from lib/ (status.py imports coverage.py); never committed.
-__pycache__/

conformance/README.md

@@ -18,7 +18,6 @@ behavioral / L3 fault-injection** (this harness).
 ./run.sh --requirements R-NET-01,R-NET-06   # merge-gate subset (tagged)
 ./run.sh --layer l2                   # one layer: l1 | l2 | l3 | meta | all
 ./run.sh --coverage-only              # regenerate the coverage report (no cluster)
-./run.sh --status                     # derived requirement-STATUS report (no cluster)
 ./run.sh --lint                       # structural-validate test defs (no cluster)
 ./run.sh --strict-coverage            # hygiene gate: fail on tag/annotation drift
 ./run.sh --require-all                # phase-exit gate: every requirement must be covered
@@ -38,33 +37,6 @@ behavioral / L3 fault-injection** (this harness).
   used to be here; it now has a real L3 test — `tests/l3-fault-injection/node-loss` — and
   is covered, not exempted.)
 
-### Coverage vs status — two different questions
-
-`coverage.md` answers **"does a requirement-tagged test EXIST for R-x?"** (R-TEST-04).
-`status.md` (`./run.sh --status`, generator `lib/status.py`) answers the operator's
-question — **"what is the real state of each requirement given what is actually in the
-repo?"** — deriving three orthogonal axes per requirement:
-
-1. **implementation** — is there a tracked artifact naming the requirement?
-2. **reconciliation** — is that artifact reachable by walking `resources:` from
-   `clusters/<env>/kustomization.yaml` and expanding every Flux `Kustomization`'s
-   `spec.path`? This is what separates **authored** from **actually reconciled**: a
-   commented-out `resources:` entry (e.g. `platform/image-automation`) is correctly
-   reported as in-git-but-not-on-any-cluster.
-3. **verification** — is the tagged test EXECUTABLE, or a TODO scaffold? Coverage counts
-   both as "covered"; status does not. TODO markers in a body always outrank an
-   `executable: "true"` annotation, so ambiguity never upgrades a test.
-
-**It deliberately refuses to derive proof-of-pass.** Nothing tracked in git records a test
-result — `_artifacts/` and `results/` are git-ignored and DECISIONS-LOG.md prose is not a
-signal — so every requirement's proof state is reported as `not-derivable` rather than
-defaulting to green. A status report that overstates completion is worse than none
-(`agents/definition-of-done.md`). Closing that gap needs a **committed, machine-readable
-run ledger**; until then the strongest claim the report makes is *wired and testable*.
-`--strict-coverage` also applies status's hygiene gate (registry↔spec drift,
-annotation/body conflicts, a safety-critical requirement whose only L3 test is a scaffold).
-Exit codes follow `ci/lib.sh`: `0` pass / `1` fail / `3` skipped — a skip is never green.
-
 `run.sh` is the **only** entrypoint. Local runs and the in-cluster CI Chainsaw Job call
 the *same* script with the same config, so results are identical.
 
@@ -74,8 +46,6 @@ the *same* script with the same config, so results are identical.
 | `_artifacts/report.junit.xml` | JUnit — Chainsaw results, or a **skipped-scaffold** shim when no cluster | CI gate |
 | `_artifacts/coverage/coverage.md` | Human requirement-coverage report | PR reviewers / logs |
 | `_artifacts/coverage/coverage.json` | Machine coverage: `{requirement: [tests]}`, gaps, safety gaps | CI gate |
-| `_artifacts/status/status.md` | Human requirement-STATUS report derived from repo state | operator / PM |
-| `_artifacts/status/status.json` | Machine status: implementation / reconciliation / verification per requirement | tooling |
 
 ## Reproducibility contract (R-TEST-02 — what the harness guarantees)
 - **Single command.** `./run.sh` (or a tagged/layer subset) runs everything and emits the
@@ -104,7 +74,6 @@ requirements/registry.yaml   mirror of EVERY spec R-* ID (coverage enumerates fr
 lib/
   common.sh                  tagging convention + test discovery/selection helpers
   coverage.py                requirement-coverage generator (stdlib only)
-  status.py                  requirement-STATUS generator (stdlib only; --selftest)
   emit_skipped_junit.py      Phase-0 skipped-JUnit shim (no cluster)
 tests/
   l1-rendering/  l2-behavioral/  l3-fault-injection/  meta/   (see tests/README.md)

conformance/incluster/rbac.yaml

@@ -64,22 +64,6 @@ rules:
   - apiGroups: [""]
     resources: [pods/log]
     verbs: [get]                            # read fault/probe Job logs on failure
-  # ---- Storage-tier precondition gate (READ-ONLY). Every storage test self-skips on the
-  # absence of the `longhorn` StorageClass rather than failing: data-survival (R-STATE-01),
-  # node-loss (R-STATE-02), dr-restore (R-STATE-03), vm-data-survival / vm-lifecycle /
-  # vm-default-deny (R-PLAT-03). Without this read the gate cannot tell "absent" from
-  # "forbidden" and would skip for the wrong reason.
-  - apiGroups: [storage.k8s.io]
-    resources: [storageclasses]
-    verbs: [get, list]
-  # ---- Controller-readiness preconditions (READ-ONLY). dr-restore (R-STATE-03) skips cleanly
-  # unless the Velero Deployment and its node-agent DaemonSet (kopia File System Backup, the
-  # volume-DATA path) are actually ready — a backup path that is not up is an absent
-  # precondition, not a failure. No write verbs: tests never scale or patch a platform
-  # controller.
-  - apiGroups: [apps]
-    resources: [deployments, daemonsets]
-    verbs: [get, list, watch]
   # ---- Probe / fault Jobs (R-TLS-01/03 tls-probe; R-NET-06 tunnel stop/start)
   # `create` cannot be constrained by resourceNames, and the fault Job names are
   # fixed per-test (tunnel-stop/tunnel-start/tunnel-start-finally); scoped by the
@@ -119,29 +103,10 @@ rules:
   #   resources: [imagepolicies, imagerepositories]
   #   verbs: [get, patch]
   #
-  # ---- R-STATE-03 (DR restore, tests/l3-fault-injection/dr-restore): the drill creates a
+  # ---- R-STATE-03 (DR restore): trigger a Velero restore
-  # Velero Backup of its OWN scratch namespace, destroys that namespace + its PV, creates a
+  # - apiGroups: [velero.io]
-  # Restore into a fresh namespace, and (in `finally`) issues a DeleteBackupRequest so the
+  #   resources: [restores, backups]
-  # objects it wrote are removed from the off-cluster bucket again. `create` cannot be
+  #   verbs: [create, get, list, watch]
-  # constrained by resourceNames; the blast radius is bounded by the fact that every object
-  # the test creates or deletes is named `<its-own-chainsaw-namespace>-dr-*`.
-  - apiGroups: [velero.io]
-    resources: [backups, restores, deletebackuprequests]
-    verbs: [create, delete, get, list, watch]
-  # Read-only evidence: the BSL is the off-cluster target whose Available phase + endpoint the
-  # test asserts; schedules prove backups are SCHEDULED (not one-off); podvolumebackups /
-  # podvolumerestores prove the kopia snapshot of the volume DATA reached/returned from the
-  # remote repository, and are dumped on failure for diagnosis.
-  - apiGroups: [velero.io]
-    resources: [backupstoragelocations, schedules, podvolumebackups, podvolumerestores]
-    verbs: [get, list, watch]
-  # R-STATE-03 needs `delete` on the PV, not merely read: the proof that the restore came from
-  # OFF-cluster is that no in-cluster copy of the data survived. The test asserts its own PV is
-  # gone, and force-deletes it if a Retain policy or a stuck finalizer left it behind. It only
-  # ever names the PV bound to the PVC it created itself.
-  - apiGroups: [""]
-    resources: [persistentvolumes]
-    verbs: [get, list, delete]
   #
   # ---- R-GITOPS-03 (drift): mutate a reconciled resource + observe Flux revert
   # - apiGroups: [kustomize.toolkit.fluxcd.io]

conformance/lib/common.sh

@@ -47,7 +47,7 @@ conf_dirs_for_requirements() {
   done < <(conf_all_test_files) | sort -u
 }
 
-# Directories for a given layer (subtree): l0 | l1 | l2 | l3 | meta | all.
+# Directories for a given layer (subtree): l1 | l2 | l3 | meta | all.
 # Accepts a comma/space separated LIST of layers (e.g. "l2,l3") and returns the UNION of
 # their test dirs, deduplicated. A single layer keeps its prior behavior (backward compatible);
 # this is what lets `run.sh --layer l2,l3` (the ci/conformance.sh L2/L3 stage) select both
@@ -58,7 +58,6 @@ conf_dirs_for_layer() {
   for layer in ${layers//,/ }; do
     case "$layer" in
       all)  roots=("${CONF_ROOT}/tests"); break ;;
-      l0)   roots+=("${CONF_ROOT}/tests/l0-static") ;;
       l1)   roots+=("${CONF_ROOT}/tests/l1-rendering") ;;
       l2)   roots+=("${CONF_ROOT}/tests/l2-behavioral") ;;
       l3)   roots+=("${CONF_ROOT}/tests/l3-fault-injection") ;;

conformance/lib/coverage.py

@@ -115,9 +115,7 @@ def parse_test(path: Path) -> dict:
     # Layer inferred from path as a fallback / cross-check.
     rel = str(path)
     if not layer:
-        if "l0-static" in rel:
+        if "l1-rendering" in rel:
-            layer = "L0"
-        elif "l1-rendering" in rel:
             layer = "L1"
         elif "l2-behavioral" in rel:
             layer = "L2"

conformance/lib/emit_skipped_junit.py

@@ -33,8 +33,7 @@ def collect(path: Path) -> tuple[str, list[str], str]:
     reqs = sorted({m.group(1).upper() for m in
                    (LABEL_LINE_RE.match(l) for l in text.splitlines()) if m})
     rel = str(path)
-    layer = ("L0" if "l0-static" in rel
+    layer = ("L1" if "l1-rendering" in rel else "L2" if "l2-behavioral" in rel
-             else "L1" if "l1-rendering" in rel else "L2" if "l2-behavioral" in rel
              else "L3" if "l3-fault-injection" in rel else "META")
     return name, reqs, layer
 

conformance/lib/run_l1_offline.py

@@ -1,24 +1,17 @@
 #!/usr/bin/env python3
-"""Run the CLUSTER-INDEPENDENT conformance checks OFFLINE and emit JUnit.
+"""Run the L1 *rendering* conformance checks OFFLINE (no cluster) and emit JUnit.
 
-docs/14 puts two layers in CI with no cluster:
+docs/14: "L1 rendering assertions run in CI without a cluster." L1 verifies the
-  * L0 static/policy  -- "manifests/ServiceDefinitions obey invariants" (tests/l0-static/,
+ServiceDefinition -> rendered-resources mapping against golden expectations; unlike L2/L3
-                         each dir carrying a `check.sh`).
+it needs no live cluster. Each l1-rendering test dir carries a `render-check.sh` (renderer
-  * L1 rendering      -- ServiceDefinition -> rendered resources vs golden expectations
++ golden diff + structural asserts). run.sh invokes this for `--layer l1` so those tests
-                         (tests/l1-rendering/, each dir carrying a `render-check.sh`).
+report a REAL pass/fail instead of being skipped with the cluster-gated layers.
-Both are executed here so they report a REAL pass/fail instead of being skipped with the
-cluster-gated layers. run.sh invokes this for `--layer l0`, `--layer l1`, and as the
-offline leg of a full run.
 
-Pure standard library (matches the harness's no-pip-dependency posture); the checks
+Pure standard library (matches the harness's no-pip-dependency posture); the render checks
-themselves shell out (render checks to catalog/render.py, whose PyYAML dependency lives
+themselves shell out to catalog/render.py (the renderer's PyYAML dependency lives there).
-there; L0 checks to git + PyYAML).
 
 Usage: run_l1_offline.py --junit <out.xml> [--label-prefix P] <test-dir> [<test-dir> ...]
-Exit 0 iff every selected offline check passes.
+Exit 0 iff every selected render check passes.
-
-The per-dir check script is whichever of `render-check.sh` / `check.sh` the directory
-carries; a dir with neither is reported SKIPPED (never a pass).
 """
 from __future__ import annotations
 
@@ -32,19 +25,6 @@ from xml.dom import minidom
 
 LABEL_LINE_RE_TMPL = r"^\s*{prefix}/(R-[A-Z0-9]+-[0-9]+)\s*:\s*[\"']?true"
 
-# The offline check script a test dir may carry, in preference order.
-CHECK_SCRIPTS = ("render-check.sh", "check.sh")
-
-
-def layer_of(d: Path) -> str:
-    """Layer for the JUnit properties, from the subtree the test lives in."""
-    rel = str(d)
-    if "l0-static" in rel:
-        return "L0"
-    if "l1-rendering" in rel:
-        return "L1"
-    return "OFFLINE"
-
 
 def collect_meta(test_yaml: Path, prefix: str):
     text = test_yaml.read_text() if test_yaml.exists() else ""
@@ -64,11 +44,10 @@ def main() -> int:
 
     cases = []  # (name, reqs, dir, rc, output)
     for d in args.dirs:
-        check = next((d / s for s in CHECK_SCRIPTS if (d / s).exists()), None)
+        check = d / "render-check.sh"
         name, reqs = collect_meta(d / "chainsaw-test.yaml", args.label_prefix)
-        if check is None:
+        if not check.exists():
-            cases.append((name, reqs, d, 3,
+            cases.append((name, reqs, d, 3, f"no render-check.sh in {d}"))
-                          f"no offline check script ({' or '.join(CHECK_SCRIPTS)}) in {d}"))
             continue
         proc = subprocess.run(
             ["bash", str(check)], capture_output=True, text=True
@@ -82,7 +61,7 @@ def main() -> int:
     suite = ET.Element(
         "testsuite",
         {
-            "name": "home-k3s-conformance L0/L1 (offline)",
+            "name": "home-k3s-conformance L1 (offline render)",
             "tests": str(len(cases)),
             "failures": str(failures),
             "errors": "0",
@@ -93,21 +72,19 @@ def main() -> int:
         tc = ET.SubElement(
             suite,
             "testcase",
-            {"classname": f"{layer_of(d)}:{','.join(reqs) if reqs else 'UNTAGGED'}",
+            {"classname": f"L1:{','.join(reqs) if reqs else 'UNTAGGED'}", "name": name},
-             "name": name},
         )
         props = ET.SubElement(tc, "properties")
         ET.SubElement(props, "property", {"name": "requirements", "value": ",".join(reqs)})
-        ET.SubElement(props, "property", {"name": "layer", "value": layer_of(d)})
+        ET.SubElement(props, "property", {"name": "layer", "value": "L1"})
         ET.SubElement(props, "property", {"name": "path", "value": str(d)})
         if rc == 3:
-            ET.SubElement(tc, "skipped", {"message": out or "no offline check script"})
+            ET.SubElement(tc, "skipped", {"message": out or "no render-check.sh"})
         elif rc != 0:
-            fail = ET.SubElement(tc, "failure", {"message": f"offline check exit {rc}"})
+            fail = ET.SubElement(tc, "failure", {"message": f"render-check exit {rc}"})
             fail.text = out
         status = "PASS" if rc == 0 else ("SKIP" if rc == 3 else "FAIL")
-        print(f"[offline] {layer_of(d):3s} {status:4s} {name} "
+        print(f"[l1-offline] {status:4s} {name} ({','.join(reqs) or 'untagged'})",
-              f"({','.join(reqs) or 'untagged'})",
               file=sys.stderr)
         if rc not in (0, 3):
             print(out, file=sys.stderr)
@@ -126,7 +103,7 @@ def main() -> int:
     args.junit.parent.mkdir(parents=True, exist_ok=True)
     xml = minidom.parseString(ET.tostring(root, "utf-8")).toprettyxml(indent="  ")
     args.junit.write_text(xml)
-    print(f"[offline] wrote JUnit ({len(cases)} case(s), {failures} failure(s)) -> "
+    print(f"[l1-offline] wrote JUnit ({len(cases)} case(s), {failures} failure(s)) -> "
           f"{args.junit}", file=sys.stderr)
     return 1 if failures else 0
 

conformance/lib/status.py

@@ -1,941 +0,0 @@
-#!/usr/bin/env python3
-"""Requirement STATUS report — derived from repository state, not from a task board.
-
-WHY THIS EXISTS
----------------
-`lib/coverage.py` answers exactly one question: *does a requirement-tagged test exist
-for R-x?* (R-TEST-04). That is necessary but not sufficient: a requirement can have a
-tagged test that is a pure TODO scaffold, and can have implementing manifests that are
-authored in git but never reconciled by Flux. Both read as "covered"/"delivered" while
-nothing is actually true of the cluster.
-
-This generator derives, from the REPOSITORY ALONE, three orthogonal axes per requirement:
-
-  1. IMPLEMENTATION  — is there implementing config in the repo, and is it reachable
-                       from a Flux Kustomization entrypoint (i.e. actually reconciled)?
-  2. VERIFICATION    — is there a requirement-tagged test, at which layer, and is it an
-                       EXECUTABLE test or a TODO scaffold?
-  3. PROOF           — has that test actually passed?  *** DELIBERATELY NOT DERIVED. ***
-
-WHAT THIS TOOL REFUSES TO DERIVE (and why)
-------------------------------------------
-PROOF-OF-PASS IS NOT DERIVABLE FROM REPO STATE. Nothing tracked in git records a test
-result: `conformance/_artifacts/` and `results/` are git-ignored, and the only prose
-record of a green run lives in DECISIONS-LOG.md — hand-maintained, unstructured, and
-exactly the kind of source the operator has ruled out. The project's own culture is that
-"merged" is never "done" (agents/definition-of-done.md, R-TEST-03), so this report will
-NOT infer, guess, or default a requirement to green. Every requirement's proof state is
-reported as `not-derivable`, and the headline says so. If an untracked JUnit artifact
-happens to be lying in the workspace it is surfaced ONLY as a quarantined advisory with
-its mtime — never counted, never mixed into a status.
-
-The honest fix is a committed, machine-readable run ledger; until that exists, the most
-this report can say is "wired and testable", never "proven".
-
-READS (all read-only)
-  requirements/registry.yaml     the authoritative R-* mirror  (via lib/coverage.py)
-  tests/**/chainsaw-test.yaml    requirement tags + layer + executable annotation
-  clusters/<env>/               Flux entrypoint -> the reconciliation graph
-  `git ls-files`                 TRACKED files only; untracked scratch is not repo state
-  ../home-network-k3s/docs/00-vision-and-scope.md   OPTIONAL registry<->spec parity check
-
-WRITES (under --out, default _artifacts/status/, git-ignored like coverage/)
-  status.json   machine-readable, one record per requirement + per-spec rollup
-  status.md     human-readable report
-
-EXIT CODES (ci/lib.sh convention: 0 pass / 1 fail / 3 skipped-not-green)
-  0  report generated.
-  1  a derivation failed, or --strict found a defect this tool owns (registry/spec
-     drift, a requirement-tag hygiene problem, a safety-critical requirement whose only
-     L3 test is a scaffold).
-  3  a requested check could not run (e.g. --spec-root given but absent). A SKIP IS
-     NEVER COUNTED AS GREEN.
-
-Pure standard library (no PyYAML), matching the harness's no-pip-dependency posture
-(lib/coverage.py); the file shapes parsed here are line-constrained and documented at
-each parser. `catalog/render.py` owns the one PyYAML dependency; the harness does not.
-"""
-from __future__ import annotations
-
-import argparse
-import json
-import os
-import re
-import subprocess
-import sys
-import time
-from pathlib import Path
-
-_LIB = Path(__file__).resolve().parent
-if str(_LIB) not in sys.path:
-    sys.path.insert(0, str(_LIB))
-# Importing a sibling module would otherwise litter conformance/lib/__pycache__/ into the
-# working tree; the lint stage scans untracked files, so keep the tree clean.
-sys.dont_write_bytecode = True
-import coverage as cov  # noqa: E402  — reuse the registry/test parsers, don't fork them
-
-CONF_ROOT = _LIB.parent
-REPO_ROOT = CONF_ROOT.parent
-DEFAULT_SPEC_ROOT = REPO_ROOT.parent / "home-network-k3s"
-
-RID_RE = cov.RID_RE
-LABEL_PREFIX = cov.LABEL_PREFIX
-
-# Flux Kustomization identity. A `kind: Kustomization` doc is only a *Flux* entrypoint if
-# it also carries the kustomize.toolkit apiVersion — plain kustomize overlays share the kind.
-FLUX_API = "kustomize.toolkit.fluxcd.io"
-
-# Files excluded from the IMPLEMENTATION scan, with the same reasoning ci/touched-
-# requirements.sh applies to its content scan:
-#   *.md              — an R-ID in prose is a REFERENCE, not an implementing artifact.
-#                       (DECISIONS-LOG.md alone names nearly every requirement.)
-#   conformance/tests — those are the TESTS; they are the VERIFICATION axis, counted there.
-#   ci/tests          — gate unit-test fixtures embed example R-IDs as DATA.
-#   registry.yaml     — the requirement mirror itself, not an implementation of anything.
-IMPL_EXCLUDE_PREFIXES = ("conformance/tests/", "ci/tests/")
-IMPL_EXCLUDE_PATHS = ("conformance/requirements/registry.yaml",)
-
-
-# ============================================================================ git
-def tracked_files(root: Path) -> list[str]:
-    """Repo state == TRACKED files. Untracked scratch must never count as delivery."""
-    out = subprocess.run(["git", "-C", str(root), "ls-files"],
-                         capture_output=True, text=True, check=True).stdout
-    return [l for l in out.splitlines() if l]
-
-
-# ================================================================ kustomize walk
-# Parser contract: a kustomization.yaml in this repo uses ONLY a block `resources:` list
-# of `  - <path>` entries (verified: `resources` is the sole reference key across all 30
-# kustomization.yaml files). Entries are plain scalars, optionally followed by a `#`
-# comment. A COMMENTED-OUT entry is not a list item and is therefore correctly EXCLUDED —
-# that is the whole point: `platform/kustomization.yaml` comments out velero and
-# image-automation, so those directories are authored but not reconciled through it.
-_RES_KEY_RE = re.compile(r"^resources:\s*$")
-_TOP_KEY_RE = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*:")
-# NOTE: list items may be indented (`  - x`) or flush-left (`- x`, which is what
-#       `catalog/render.py --emit-waves` generates for apps/wave-*/kustomization.yaml).
-_RES_ITEM_RE = re.compile(r"^\s*-\s+(?P<v>[^#\s]+)\s*(?:#.*)?$")
-
-
-def kustomize_resources(kfile: Path) -> list[str]:
-    """Return the literal `resources:` entries of a kustomization.yaml."""
-    items: list[str] = []
-    in_res = False
-    for raw in kfile.read_text().splitlines():
-        if _RES_KEY_RE.match(raw):
-            in_res = True
-            continue
-        if in_res:
-            m = _RES_ITEM_RE.match(raw)
-            if m:
-                items.append(m.group("v"))
-                continue
-            if raw.strip() == "" or raw.lstrip().startswith("#"):
-                continue
-            if _TOP_KEY_RE.match(raw):
-                in_res = False
-    return items
-
-
-# Flux Kustomization `spec.path` extraction. Constraint: `path:` appears at spec-level
-# indent inside a doc that carries the Flux apiVersion. Docs are split on a lone `---`.
-# Inline `patch: |-` blocks (clusters/dev) carry Flux Kustomizations WITHOUT a path, so
-# they contribute nothing — correct, they are patches, not entrypoints.
-_DOC_SPLIT_RE = re.compile(r"^---\s*$", re.M)
-_PATH_RE = re.compile(r"^\s{2,6}path:\s*(\S+)\s*$", re.M)
-_NAME_RE = re.compile(r"^\s{2,6}name:\s*(\S+)\s*$", re.M)
-
-
-def flux_paths(yfile: Path) -> list[tuple[str, str]]:
-    """[(flux-kustomization-name, spec.path)] declared by this file."""
-    found: list[tuple[str, str]] = []
-    try:
-        text = yfile.read_text()
-    except (OSError, UnicodeDecodeError):
-        return found
-    if FLUX_API not in text:
-        return found
-    for doc in _DOC_SPLIT_RE.split(text):
-        if FLUX_API not in doc or not re.search(r"^kind:\s*Kustomization\s*$", doc, re.M):
-            continue
-        pm = _PATH_RE.search(doc)
-        if not pm:
-            continue
-        nm = _NAME_RE.search(doc)
-        found.append((nm.group(1) if nm else "?", pm.group(1)))
-    return found
-
-
-def reconciled_files(repo: Path, entrypoint: str) -> tuple[set[str], list[dict]]:
-    """Walk the Flux reconciliation graph from `clusters/<env>` and return
-
-      (set of repo-relative files Flux actually applies, list of Flux Kustomization nodes)
-
-    Algorithm: kustomize-walk a directory's `resources:`; a directory entry recurses, a
-    file entry is applied. Every applied file is then scanned for Flux Kustomization
-    objects; each one's `spec.path` is a NEW kustomize-walk root (that is how Flux
-    expands the graph). Cycles are guarded (clusters/dev/sync.yaml points at its own dir).
-
-    This is what makes the difference between AUTHORED and RECONCILED derivable: a
-    manifest that no walk reaches is in git but is not applied to any cluster.
-    """
-    applied: set[str] = set()
-    nodes: list[dict] = []
-    seen_dirs: set[str] = set()
-    pending: list[tuple[str, str]] = [("<entrypoint>", entrypoint)]
-
-    def walk_dir(rel: str) -> None:
-        if rel in seen_dirs:
-            return
-        seen_dirs.add(rel)
-        kfile = repo / rel / "kustomization.yaml"
-        if not kfile.is_file():
-            return
-        applied.add(os.path.relpath(kfile, repo))
-        for item in kustomize_resources(kfile):
-            target = (repo / rel / item).resolve()
-            try:
-                trel = os.path.relpath(target, repo)
-            except ValueError:
-                continue
-            if trel.startswith(".."):
-                continue          # outside the repo: not repo state
-            if target.is_dir():
-                walk_dir(trel)
-            elif target.is_file():
-                applied.add(trel)
-                for name, p in flux_paths(target):
-                    pending.append((name, p))
-
-    while pending:
-        owner, p = pending.pop()
-        rel = os.path.normpath(p.lstrip("./")) if p not in (".", "./") else "."
-        node = {"kustomization": owner, "path": p, "resolved": rel,
-                "exists": (repo / rel).is_dir()}
-        nodes.append(node)
-        if node["exists"]:
-            walk_dir(rel)
-
-    return applied, nodes
-
-
-# ====================================================== implementation evidence
-_COMMENT_LINE_RE = re.compile(r"^\s*#")
-
-
-def scan_implementation(repo: Path, files: list[str]) -> dict[str, list[dict]]:
-    """Map R-ID -> the tracked, non-doc files that name it.
-
-    Two strengths are recorded per file, mirroring ci/touched-requirements.sh rule 3:
-      * a hit on a REAL line  -> the artifact carries the ID (label/annotation/field);
-      * a hit only in a `#` comment -> declared INTENT (a header saying what it serves).
-    Comment-only evidence is kept (it is the dominant convention in this repo's
-    manifests) but reported as such, so a reader can tell an implementation from a claim.
-    """
-    out: dict[str, list[dict]] = {}
-    for rel in files:
-        if rel.endswith(".md") or rel in IMPL_EXCLUDE_PATHS:
-            continue
-        if rel.startswith(IMPL_EXCLUDE_PREFIXES):
-            continue
-        path = repo / rel
-        try:
-            text = path.read_text()
-        except (OSError, UnicodeDecodeError):
-            continue
-        if "R-" not in text:
-            continue
-        # A file is RECONCILABLE only if it is a Kubernetes manifest: a *.yaml with a
-        # top-level `kind:`. Shell scripts / values files / READMEs under a GitOps root
-        # are never applied by Flux, so counting them as "authored but not wired" would
-        # manufacture a false gap.
-        manifest = (rel.endswith((".yaml", ".yml"))
-                    and re.search(r"^kind:\s*\S", text, re.M) is not None)
-        hits: dict[str, bool] = {}       # rid -> seen on a non-comment line
-        for line in text.splitlines():
-            ids = RID_RE.findall(line)
-            if not ids:
-                continue
-            real = not _COMMENT_LINE_RE.match(line)
-            for rid in ids:
-                hits[rid] = hits.get(rid, False) or real
-        for rid, real in hits.items():
-            out.setdefault(rid, []).append({"path": rel, "comment_only": not real,
-                                            "manifest": manifest})
-    for rid in out:
-        out[rid].sort(key=lambda h: h["path"])
-    return out
-
-
-# ================================================================ verification
-_EXECUTABLE_RE = re.compile(
-    r"^\s*" + re.escape(LABEL_PREFIX) + r"/executable:\s*[\"']?true", re.M | re.I)
-_BLOCKED_RE = re.compile(
-    r"^\s*" + re.escape(LABEL_PREFIX) + r"/blocked-by:\s*(.+)$", re.M)
-
-
-def classify_test(t: dict) -> dict:
-    """Decide EXECUTABLE vs SCAFFOLD for one discovered test, conservatively.
-
-    A tagged test is only worth counting if it can actually assert something. Rules, in
-    order, chosen so ambiguity NEVER upgrades a test:
-
-      scaffold             the body contains TODO markers (a placeholder), *even if* it
-                           is annotated executable — an annotation must never outrank the
-                           body. That mismatch is also reported as a hygiene defect.
-      executable           annotated `conformance.home.k3s/executable: "true"` and no TODOs.
-      executable-untagged  no TODOs and a real runnable body, but the annotation is
-                           missing (true of the L1 render tests, which execute a
-                           render-check.sh + golden diff, of the L0 static checks, and
-                           of the META self-check).
-                           Counted as executable, and flagged as an annotation gap.
-      scaffold             anything else.
-    """
-    p = Path(t["path"])
-    text = p.read_text() if p.is_file() else ""
-    # The offline assertion script a test dir may carry: `render-check.sh` (L1 rendering)
-    # or `check.sh` (L0 static). BOTH are read, so a placeholder body in either one still
-    # downgrades the test to a scaffold — an annotation must never outrank the body,
-    # whichever file the body lives in.
-    extra = ""
-    for script in ("render-check.sh", "check.sh"):
-        sp = p.parent / script
-        if sp.is_file():
-            extra += sp.read_text()
-    declared = bool(_EXECUTABLE_RE.search(text))
-    has_todo = "TODO" in text or "TODO" in extra
-    bm = _BLOCKED_RE.search(text)
-    has_body = bool(extra) or bool(re.search(r"^\s+- (script|assert|apply):", text, re.M))
-
-    if has_todo:
-        state = "scaffold"
-    elif declared:
-        state = "executable"
-    elif has_body:
-        state = "executable-untagged"
-    else:
-        state = "scaffold"
-
-    try:
-        rel = os.path.relpath(p, REPO_ROOT)
-    except ValueError:
-        rel = t["path"]
-    return {
-        "name": t["name"], "path": rel, "layer": t["layer"],
-        "state": state,
-        "declared_executable": declared,
-        "annotation_conflict": bool(declared and has_todo),
-        "blocked_by": bm.group(1).strip().strip('"') if bm else "",
-    }
-
-
-# ===================================================================== spec parity
-def spec_requirement_ids(spec_root: Path) -> set[str] | None:
-    """R-* IDs defined by the READ-ONLY spec (docs/00-vision-and-scope.md).
-
-    Read-only by construction (R-TEST-06): this tool never writes under the spec tree.
-    Returns None when the spec tree is not available (a sibling checkout is optional).
-    """
-    doc = spec_root / "docs" / "00-vision-and-scope.md"
-    if not doc.is_file():
-        return None
-    return set(RID_RE.findall(doc.read_text()))
-
-
-# ========================================================================== build
-IMPL_ORDER = {"none": 0, "off-cluster": 1, "unwired": 2, "reconciled-partial": 3,
-              "reconciled": 4}
-VERIFY_ORDER = {"none": 0, "scaffold-only": 1, "executable": 2}
-
-LABELS = {
-    ("reconciled-partial", "executable"): "PARTLY-WIRED+TESTABLE",
-    ("reconciled-partial", "scaffold-only"): "PARTLY-WIRED, TEST-IS-SCAFFOLD",
-    ("reconciled-partial", "none"): "PARTLY-WIRED, UNTESTED",
-    ("reconciled", "executable"): "WIRED+TESTABLE",
-    ("reconciled", "scaffold-only"): "WIRED, TEST-IS-SCAFFOLD",
-    ("reconciled", "none"): "WIRED, UNTESTED",
-    ("unwired", "executable"): "AUTHORED-NOT-WIRED",
-    ("unwired", "scaffold-only"): "AUTHORED-NOT-WIRED",
-    ("unwired", "none"): "AUTHORED-NOT-WIRED",
-    ("off-cluster", "executable"): "TOOLING+TESTABLE",
-    ("off-cluster", "scaffold-only"): "TOOLING, TEST-IS-SCAFFOLD",
-    ("off-cluster", "none"): "TOOLING, UNTESTED",
-    ("none", "executable"): "TEST-ONLY",
-    ("none", "scaffold-only"): "SPEC-ONLY (scaffold test)",
-    ("none", "none"): "SPEC-ONLY",
-}
-
-
-def build(repo: Path, registry: list[dict], tests: list[dict], impl: dict[str, list[dict]],
-          applied: set[str], gitops_roots: set[str], spec_ids: set[str] | None,
-          env: str) -> dict:
-    classified = [classify_test(t) for t in tests]
-    by_req_tests: dict[str, list[dict]] = {r["id"]: [] for r in registry}
-    for t, c in zip(tests, classified):
-        for rid in t["labels"]:
-            if rid in by_req_tests:
-                by_req_tests[rid].append(c)
-
-    records: dict[str, dict] = {}
-    for r in registry:
-        rid = r["id"]
-        files = impl.get(rid, [])
-        ev = [{**f,
-               "reconciled": f["path"] in applied,
-               "gitops": (f["path"].split("/", 1)[0] in gitops_roots
-                          and f.get("manifest", False))}
-              for f in files]
-        gitops_ev = [e for e in ev if e["gitops"]]
-        rec_ev = [e for e in gitops_ev if e["reconciled"]]
-        unrec_ev = [e for e in gitops_ev if not e["reconciled"]]
-        if not ev:
-            impl_state = "none"
-        elif rec_ev and unrec_ev:
-            # Some citing manifests ARE applied and some are NOT. Never round this up to
-            # "reconciled": e.g. R-UPD-01's only applied citation is a PrometheusRule that
-            # merely mentions the ID, while platform/image-automation/ — the component
-            # that actually implements it — is commented out of platform/kustomization.yaml.
-            impl_state = "reconciled-partial"
-        elif rec_ev:
-            impl_state = "reconciled"
-        elif gitops_ev:
-            impl_state = "unwired"
-        else:
-            impl_state = "off-cluster"
-
-        ts = by_req_tests[rid]
-        exec_ts = [t for t in ts if t["state"].startswith("executable")]
-        if not ts:
-            verify_state = "none"
-        elif exec_ts:
-            verify_state = "executable"
-        else:
-            verify_state = "scaffold-only"
-
-        safety_l3_ok = None
-        if r["safety_critical"]:
-            safety_l3_ok = any(t["layer"] == "L3" and t["state"].startswith("executable")
-                               for t in ts)
-
-        records[rid] = {
-            "doc": r["doc"],
-            "summary": r["summary"],
-            "safety_critical": r["safety_critical"],
-            "implementation": {
-                "state": impl_state,
-                "files_total": len(ev),
-                "files_gitops": len(gitops_ev),
-                "files_reconciled": len(rec_ev),
-                "comment_only_evidence_only": bool(ev) and all(e["comment_only"] for e in ev),
-                "files": ev,
-            },
-            "verification": {
-                "state": verify_state,
-                "executable_tests": len(exec_ts),
-                "scaffold_tests": len(ts) - len(exec_ts),
-                "layers": sorted({t["layer"] for t in exec_ts}),
-                "safety_critical_l3_executable": safety_l3_ok,
-                "blocked_by": sorted({t["blocked_by"] for t in ts if t["blocked_by"]}),
-                "tests": ts,
-            },
-            # NEVER derived. See the module docstring.
-            "proof": {"state": "not-derivable",
-                      "reason": "no committed, machine-readable record of a test run "
-                                "exists in the repo (_artifacts/ and results/ are "
-                                "git-ignored); DECISIONS-LOG.md prose is not a signal"},
-            "status": LABELS[(impl_state, verify_state)],
-        }
-
-    # --- hygiene defects this tool owns -------------------------------------------
-    hygiene = {
-        "annotation_conflicts": [t["path"] for t in classified if t["annotation_conflict"]],
-        "executable_untagged": [t["path"] for t in classified
-                                if t["state"] == "executable-untagged"],
-        "registry_not_in_spec": [],
-        "spec_not_in_registry": [],
-        "spec_parity_checked": spec_ids is not None,
-    }
-    if spec_ids is not None:
-        reg_ids = {r["id"] for r in registry}
-        hygiene["registry_not_in_spec"] = sorted(reg_ids - spec_ids)
-        hygiene["spec_not_in_registry"] = sorted(spec_ids - reg_ids)
-
-    safety_scaffold = sorted(rid for rid, v in records.items()
-                             if v["safety_critical"]
-                             and v["verification"]["safety_critical_l3_executable"] is False)
-
-    def count(pred) -> int:
-        return sum(1 for v in records.values() if pred(v))
-
-    summary = {
-        "environment": env,
-        "generated_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
-        "total_requirements": len(registry),
-        "impl_reconciled": count(lambda v: v["implementation"]["state"] == "reconciled"),
-        "impl_reconciled_partial": count(
-            lambda v: v["implementation"]["state"] == "reconciled-partial"),
-        "impl_unwired": count(lambda v: v["implementation"]["state"] == "unwired"),
-        "impl_off_cluster": count(lambda v: v["implementation"]["state"] == "off-cluster"),
-        "impl_none": count(lambda v: v["implementation"]["state"] == "none"),
-        "verify_executable": count(lambda v: v["verification"]["state"] == "executable"),
-        "verify_scaffold_only": count(lambda v: v["verification"]["state"] == "scaffold-only"),
-        "verify_none": count(lambda v: v["verification"]["state"] == "none"),
-        "wired_and_testable": count(
-            lambda v: v["implementation"]["state"] == "reconciled"
-            and v["verification"]["state"] == "executable"),
-        "highest_derivable_state_reached": count(
-            lambda v: v["implementation"]["state"] in ("reconciled", "off-cluster")
-            and v["verification"]["state"] == "executable"),
-        "wired_or_partial_and_testable": count(
-            lambda v: v["implementation"]["state"] in ("reconciled", "reconciled-partial")
-            and v["verification"]["state"] == "executable"),
-        "safety_critical_total": count(lambda v: v["safety_critical"]),
-        "safety_critical_without_executable_l3": len(safety_scaffold),
-        "proven": 0,
-        "proven_derivable": False,
-        "coverage_style_covered": count(lambda v: v["verification"]["state"] != "none"),
-    }
-    summary["uncited_by_any_reconciled_manifest"] = count(
-        lambda v: v["implementation"]["state"] in ("none", "off-cluster"))
-    return {"summary": summary, "by_requirement": records,
-            "safety_critical_without_executable_l3": safety_scaffold,
-            "hygiene": hygiene}
-
-
-def spec_rollup(rep: dict) -> dict:
-    """Per-spec-document rollup — 'the status of each spec', the operator's actual ask."""
-    docs: dict[str, dict] = {}
-    for rid, v in rep["by_requirement"].items():
-        d = docs.setdefault(v["doc"] or "(unmapped)", {
-            "requirements": [], "reconciled": 0, "partly_reconciled": 0,
-            "executable": 0, "wired_and_testable": 0})
-        d["requirements"].append(rid)
-        if v["implementation"]["state"] == "reconciled":
-            d["reconciled"] += 1
-        elif v["implementation"]["state"] == "reconciled-partial":
-            d["partly_reconciled"] += 1
-        if v["verification"]["state"] == "executable":
-            d["executable"] += 1
-        if (v["implementation"]["state"] in ("reconciled", "off-cluster")
-                and v["verification"]["state"] == "executable"):
-            d["wired_and_testable"] += 1
-    for d in docs.values():
-        d["requirements"].sort()
-        d["total"] = len(d["requirements"])
-    return dict(sorted(docs.items()))
-
-
-# ========================================================================= render
-def render_md(rep: dict, nodes: list[dict], advisory: dict | None) -> str:
-    s = rep["summary"]
-    o: list[str] = []
-    o.append("# Requirement status — derived from repository state\n")
-    o.append(f"Environment overlay: `clusters/{s['environment']}` · "
-             f"generated {s['generated_utc']} · **no cluster was contacted**.\n")
-    o.append("\n## Headline\n")
-    o.append(f"| Axis | Count (of {s['total_requirements']}) |")
-    o.append("|---|---|")
-    o.append(f"| Implementation fully reconciled by Flux | **{s['impl_reconciled']}** |")
-    o.append(f"| Implementation only PARTLY reconciled (some citing manifests not "
-             f"applied) | **{s['impl_reconciled_partial']}** |")
-    o.append(f"| Implementation authored but NOT wired into the Flux graph | "
-             f"**{s['impl_unwired']}** |")
-    o.append(f"| Implementation is off-cluster tooling (CI / harness / IaC) | "
-             f"{s['impl_off_cluster']} |")
-    o.append(f"| No implementing artifact in the repo | **{s['impl_none']}** |")
-    o.append(f"| Has an EXECUTABLE requirement-tagged test | **{s['verify_executable']}** |")
-    o.append(f"| Tagged test exists but is a TODO SCAFFOLD | **{s['verify_scaffold_only']}** |")
-    o.append(f"| No requirement-tagged test at all | **{s['verify_none']}** |")
-    o.append(f"| Reconciled **and** executably tested (highest derivable state) | "
-             f"**{s['highest_derivable_state_reached']}** |")
-    o.append(f"| Safety-critical requirements whose L3 test is a scaffold | "
-             f"**{s['safety_critical_without_executable_l3']}** of "
-             f"{s['safety_critical_total']} |")
-    o.append(f"| **PROVEN (a test actually passed)** | **{s['proven']} — NOT DERIVABLE** |")
-    o.append("")
-    o.append("> **Proof is deliberately not derived.** Nothing tracked in git records a "
-             "test result: `conformance/_artifacts/` and `results/` are git-ignored, and "
-             "the only record of a green run is DECISIONS-LOG.md prose — hand-maintained, "
-             "which is the very thing this report replaces. Per "
-             "`agents/definition-of-done.md` a requirement is done only when its tagged "
-             "test **passes**, so the strongest claim this report can make is "
-             "*wired and testable*. Closing the gap needs a committed, machine-readable "
-             "run ledger; until then, treat every row below as unproven.\n")
-    o.append(f"> For contrast, `coverage.md` counts **{s['coverage_style_covered']}** "
-             f"requirements as covered — that count includes the "
-             f"{s['verify_scaffold_only']} whose only verifying test is a TODO scaffold.\n")
-
-    o.append("\n## Status legend\n")
-    o.append("| Label | Meaning |")
-    o.append("|---|---|")
-    o.append("| `WIRED+TESTABLE` | implementing config is reconciled by Flux **and** an "
-             "executable tagged test exists (still unproven) |")
-    o.append("| `WIRED, TEST-IS-SCAFFOLD` | reconciled, but its only tagged test is a "
-             "TODO placeholder — coverage counts this green; it is not |")
-    o.append("| `WIRED, UNTESTED` | reconciled with no tagged test at all |")
-    o.append("| `PARTLY-WIRED…` | some citing manifests are applied, others are not — "
-             "check whether the unapplied one is the real implementation |")
-    o.append("| `AUTHORED-NOT-WIRED` | manifests exist under a GitOps root but no Flux "
-             "Kustomization reaches them — in git, not on any cluster |")
-    o.append("| `TOOLING…` | implemented by off-cluster tooling (ci/, conformance/, "
-             "infra/, bootstrap/, schemas/) — nothing for Flux to reconcile |")
-    o.append("| `TEST-ONLY` / `SPEC-ONLY` | a test but no implementation / neither |")
-
-    o.append("\n## Per-spec rollup\n")
-    o.append("| Spec document | Reqs | Reconciled | Partly | Executable test | "
-             "Wired+Testable |")
-    o.append("|---|---|---|---|---|---|")
-    for doc, d in spec_rollup(rep).items():
-        o.append(f"| `{doc}` | {d['total']} | {d['reconciled']} | "
-                 f"{d['partly_reconciled']} | {d['executable']} | "
-                 f"**{d['wired_and_testable']}/{d['total']}** |")
-
-    o.append("\n## Status by requirement\n")
-    o.append("| Requirement | Status | Implementation | Verification | Proof |")
-    o.append("|---|---|---|---|---|")
-    for rid, v in rep["by_requirement"].items():
-        i, ver = v["implementation"], v["verification"]
-        if i["state"] == "none":
-            icell = "—"
-        else:
-            icell = (f"{i['state']} ({i['files_reconciled']}/{i['files_gitops']} gitops "
-                     f"files applied, {i['files_total']} refs)")
-            if i["comment_only_evidence_only"]:
-                icell += " *(comment-only evidence)*"
-        if ver["state"] == "none":
-            vcell = "—"
-        else:
-            layers = ",".join(ver["layers"]) or "-"
-            vcell = (f"{ver['state']} [{layers}] "
-                     f"exec={ver['executable_tests']} scaffold={ver['scaffold_tests']}")
-            if ver["blocked_by"]:
-                vcell += f" · blocked-by: {'; '.join(ver['blocked_by'])}"
-        sc = " ⚠safety" if v["safety_critical"] else ""
-        o.append(f"| {rid}{sc} | **{v['status']}** | {icell} | {vcell} | not-derivable |")
-
-    if rep["safety_critical_without_executable_l3"]:
-        o.append("\n## Safety-critical requirements whose L3 fault test is a SCAFFOLD "
-                 "(R-TEST-05)\n")
-        o.append("`coverage.md` reports zero safety-critical gaps because a tagged L3 "
-                 "test file exists. These L3 tests have TODO bodies and assert nothing:\n")
-        for rid in rep["safety_critical_without_executable_l3"]:
-            v = rep["by_requirement"][rid]
-            bb = "; ".join(v["verification"]["blocked_by"]) or "—"
-            o.append(f"- **{rid}** — {v['summary']} · blocked-by: {bb}")
-
-    partial = [r for r, v in rep["by_requirement"].items()
-               if v["implementation"]["state"] == "reconciled-partial"]
-    if partial:
-        o.append("\n## Partly wired — citing manifests that Flux does NOT apply\n")
-        o.append("At least one manifest naming these requirements is reconciled, but "
-                 "others are not. Check whether the unapplied file is the one that "
-                 "actually implements the requirement.\n")
-        for rid in partial:
-            v = rep["by_requirement"][rid]
-            miss = [f"`{f['path']}`" for f in v["implementation"]["files"]
-                    if f["gitops"] and not f["reconciled"]]
-            o.append(f"- **{rid}** — not applied: {', '.join(miss)}")
-
-    unwired = [r for r, v in rep["by_requirement"].items()
-               if v["implementation"]["state"] == "unwired"]
-    if unwired:
-        o.append("\n## Authored but NOT reconciled (in git, not on any cluster)\n")
-        for rid in unwired:
-            v = rep["by_requirement"][rid]
-            fs = ", ".join(f"`{f['path']}`" for f in v["implementation"]["files"]
-                           if f["gitops"])[:400]
-            o.append(f"- **{rid}** — {v['summary']}\n  - {fs}")
-
-    uncited = [r for r, v in rep["by_requirement"].items()
-               if v["implementation"]["state"] in ("none", "off-cluster")]
-    if uncited:
-        o.append("\n## No reconciled manifest cites these requirements (traceability, "
-                 "R-TEST-04)\n")
-        o.append("Implementation evidence here is **a tracked, non-Markdown file that "
-                 "names the R-ID**. For these requirements no reconciled Kubernetes "
-                 "manifest names the ID, which means EITHER the requirement is genuinely "
-                 "unimplemented on the cluster, OR the implementing manifest exists but "
-                 "is missing its requirement annotation. This report cannot tell the two "
-                 "apart and deliberately does not guess — annotate the manifest and the "
-                 "row self-corrects.\n")
-        for rid in uncited:
-            v = rep["by_requirement"][rid]
-            o.append(f"- **{rid}** ({v['implementation']['state']}) — {v['summary']}")
-
-    o.append("\n## Flux reconciliation graph walked\n")
-    o.append("| Flux Kustomization | spec.path | present |")
-    o.append("|---|---|---|")
-    for n in sorted(nodes, key=lambda n: (n["resolved"], n["kustomization"])):
-        o.append(f"| `{n['kustomization']}` | `{n['path']}` | "
-                 f"{'yes' if n['exists'] else '**MISSING**'} |")
-
-    h = rep["hygiene"]
-    o.append("\n## Harness hygiene\n")
-    if h["spec_parity_checked"]:
-        drift = h["registry_not_in_spec"] + h["spec_not_in_registry"]
-        o.append(f"- registry ↔ spec parity: {'**DRIFT**: ' + str(drift) if drift else 'in sync'}")
-    else:
-        o.append("- registry ↔ spec parity: **NOT CHECKED** (spec tree not available; "
-                 "this is a skip, not a pass)")
-    o.append(f"- tests annotated executable but containing TODO bodies: "
-             f"{h['annotation_conflicts'] or 'none'}")
-    o.append(f"- executable tests missing the `{LABEL_PREFIX}/executable` annotation: "
-             f"{h['executable_untagged'] or 'none'}")
-
-    o.append("\n## Method — what is derived, and what is refused\n")
-    o.append("| Signal | How it is derived | Trust |")
-    o.append("|---|---|---|")
-    o.append("| requirement set | `conformance/requirements/registry.yaml`, "
-             "cross-checked against the read-only spec `docs/00-vision-and-scope.md` | "
-             "exact |")
-    o.append("| implementing artifact | a TRACKED, non-Markdown file that names the R-ID "
-             "(comments counted, and marked as such) | heuristic — under-reports "
-             "un-annotated manifests, never over-reports |")
-    o.append("| reconciled | the file is reachable by walking `resources:` from "
-             "`clusters/<env>/kustomization.yaml`, expanding every Flux Kustomization's "
-             "`spec.path` | exact — a commented-out `resources:` entry is correctly NOT "
-             "reconciled |")
-    o.append("| verifying test | `conformance.home.k3s/R-*` label on a Chainsaw test "
-             "(the authoritative tag, same source as `coverage.md`) | exact |")
-    o.append("| executable vs scaffold | TODO markers in the test body outrank the "
-             "`executable` annotation; a real body without the annotation still counts "
-             "and is flagged | conservative — ambiguity never upgrades |")
-    o.append("| **test actually passed** | **NOT DERIVED** — nothing in git records a "
-             "run result | refused |")
-    o.append("| cluster liveness (pods/CRs healthy) | **NOT DERIVED** — no cluster is "
-             "contacted; `reconciled` means Flux *would* apply it, not that it is "
-             "healthy | refused |")
-    o.append("| phase/board status | **NOT DERIVED** — `agents/task-board.md` is stale "
-             "and is not a signal | refused |")
-
-    o.append("\n## Advisory — untracked local run artifact (NOT counted)\n")
-    if advisory:
-        o.append(f"- `{advisory['path']}` (untracked, git-ignored), last written "
-                 f"{advisory['mtime']}: {advisory['tests']} testcase(s), "
-                 f"{advisory['failures']} failure(s), {advisory['skipped']} skipped.")
-        o.append(f"- requirement IDs it names: {', '.join(advisory['requirements']) or '—'}")
-        o.append("- This is one developer's local workspace, not repo state. It is shown "
-                 "for orientation only and contributes to NO status above.")
-    else:
-        o.append("- none present.")
-    return "\n".join(o) + "\n"
-
-
-def read_advisory(junit: Path) -> dict | None:
-    """Quarantined read of an untracked local JUnit. Never feeds a status."""
-    if not junit.is_file():
-        return None
-    import xml.etree.ElementTree as ET
-    try:
-        root = ET.parse(junit).getroot()
-    except ET.ParseError:
-        return None
-    cases = root.iter("testcase")
-    n = f = sk = 0
-    reqs: set[str] = set()
-    for c in cases:
-        n += 1
-        if c.find("failure") is not None or c.find("error") is not None:
-            f += 1
-        if c.find("skipped") is not None:
-            sk += 1
-        reqs |= set(RID_RE.findall(c.get("classname") or ""))
-        for p in c.iter("property"):
-            if p.get("name") == "requirements":
-                reqs |= set(RID_RE.findall(p.get("value") or ""))
-    return {"path": os.path.relpath(junit, REPO_ROOT), "tests": n, "failures": f,
-            "skipped": sk, "requirements": sorted(reqs),
-            "mtime": time.strftime("%Y-%m-%dT%H:%M:%SZ",
-                                   time.gmtime(junit.stat().st_mtime))}
-
-
-# ======================================================================= selftest
-def _selftest() -> int:
-    """Deterministic, no-cluster self-test of every derivation rule (house pattern:
-    `catalog/render.py --selftest`). Uses a throwaway tree, never the real repo."""
-    import tempfile
-    ok = True
-
-    def check(label: str, cond: bool) -> None:
-        nonlocal ok
-        if not cond:
-            sys.stderr.write(f"status.py selftest: FAIL — {label}\n")
-            ok = False
-
-    with tempfile.TemporaryDirectory() as td:
-        root = Path(td)
-        # A kustomization whose second entry is COMMENTED OUT must not be reconciled.
-        (root / "clusters" / "dev").mkdir(parents=True)
-        (root / "platform" / "on").mkdir(parents=True)
-        (root / "platform" / "off").mkdir(parents=True)
-        (root / "clusters" / "dev" / "kustomization.yaml").write_text(
-            "resources:\n  - platform-ks.yaml   # comment\n")
-        (root / "clusters" / "dev" / "platform-ks.yaml").write_text(
-            "apiVersion: kustomize.toolkit.fluxcd.io/v1\n"
-            "kind: Kustomization\nmetadata:\n  name: platform\nspec:\n"
-            "  path: ./platform\n")
-        (root / "platform" / "kustomization.yaml").write_text(
-            "resources:\n  - on\n  # - off   # deliberately disabled\n")
-        # flush-left items (the `render.py --emit-waves` style) must parse too.
-        (root / "platform" / "on" / "kustomization.yaml").write_text(
-            "resources:\n- r.yaml\npatches: []\n")
-        (root / "platform" / "on" / "r.yaml").write_text("# serves R-NET-01\nkind: X\n")
-        (root / "platform" / "off" / "kustomization.yaml").write_text(
-            "resources:\n  - r.yaml\n")
-        (root / "platform" / "off" / "r.yaml").write_text("# serves R-UPD-01\nkind: X\n")
-
-        applied, nodes = reconciled_files(root, "clusters/dev")
-        check("commented-out resource is NOT reconciled",
-              "platform/on/r.yaml" in applied and "platform/off/r.yaml" not in applied)
-        check("flux spec.path expands the graph",
-              any(n["path"] == "./platform" for n in nodes))
-
-        files = ["platform/on/r.yaml", "platform/off/r.yaml", "docs/x.md",
-                 "conformance/tests/t/chainsaw-test.yaml"]
-        (root / "docs").mkdir(); (root / "docs" / "x.md").write_text("R-NET-01 R-TLS-01\n")
-        (root / "conformance" / "tests" / "t").mkdir(parents=True)
-        (root / "conformance" / "tests" / "t" / "chainsaw-test.yaml").write_text("R-DNS-01\n")
-        impl = scan_implementation(root, files)
-        check("markdown is excluded from implementation evidence", "R-TLS-01" not in impl)
-        check("conformance/tests is excluded from implementation evidence",
-              "R-DNS-01" not in impl)
-        check("comment-only evidence is recorded as such",
-              impl["R-NET-01"][0]["comment_only"] is True)
-
-        (root / "platform" / "on" / "real.yaml").write_text(
-            "kind: ClusterPolicy\n  conformance.home.k3s/R-SEC-01: \"true\"\n")
-        impl2 = scan_implementation(root, ["platform/on/real.yaml"])
-        check("non-comment evidence is recorded as such",
-              impl2["R-SEC-01"][0]["comment_only"] is False)
-        check("a yaml with a top-level kind: is a reconcilable manifest",
-              impl2["R-SEC-01"][0]["manifest"] is True)
-        (root / "platform" / "on" / "s.sh").write_text("# R-SEC-02\n")
-        check("a shell script is never a reconcilable manifest",
-              scan_implementation(root, ["platform/on/s.sh"])["R-SEC-02"][0]["manifest"]
-              is False)
-
-        # classify_test: TODO body outranks an `executable: true` annotation.
-        td2 = root / "conformance" / "tests" / "s"
-        td2.mkdir(parents=True)
-        f = td2 / "chainsaw-test.yaml"
-        f.write_text("metadata:\n  annotations:\n"
-                     "    conformance.home.k3s/executable: \"true\"\n"
-                     "spec:\n  steps:\n    - script:\n"
-                     "        content: 'echo \"TODO: write me\"'\n")
-        c = classify_test({"name": "s", "path": str(f), "layer": "L2"})
-        check("TODO body downgrades a test annotated executable",
-              c["state"] == "scaffold" and c["annotation_conflict"])
-        f.write_text("metadata:\n  annotations:\n"
-                     "    conformance.home.k3s/executable: \"true\"\n"
-                     "spec:\n  steps:\n    - script:\n        content: 'true'\n")
-        check("annotated + no TODO == executable",
-              classify_test({"name": "s", "path": str(f), "layer": "L2"})["state"]
-              == "executable")
-        f.write_text("spec:\n  steps:\n    - script:\n        content: 'true'\n")
-        check("real body without the annotation == executable-untagged",
-              classify_test({"name": "s", "path": str(f), "layer": "L2"})["state"]
-              == "executable-untagged")
-        f.write_text("spec: {}\n")
-        check("empty body == scaffold",
-              classify_test({"name": "s", "path": str(f), "layer": "L2"})["state"]
-              == "scaffold")
-
-    if not ok:
-        return 1
-    sys.stderr.write("status.py selftest: PASS\n")
-    return 0
-
-
-# =========================================================================== main
-def main() -> int:
-    ap = argparse.ArgumentParser(description=__doc__,
-                                 formatter_class=argparse.RawDescriptionHelpFormatter)
-    ap.add_argument("--repo", type=Path, default=REPO_ROOT)
-    ap.add_argument("--registry", type=Path,
-                    default=CONF_ROOT / "requirements" / "registry.yaml")
-    ap.add_argument("--tests", type=Path, default=CONF_ROOT / "tests")
-    ap.add_argument("--out", type=Path, default=CONF_ROOT / "_artifacts" / "status")
-    ap.add_argument("--env", default="dev",
-                    help="cluster overlay to walk (clusters/<env>); default dev")
-    ap.add_argument("--spec-root", type=Path, default=None,
-                    help=f"read-only spec checkout for the registry parity check "
-                         f"(default {DEFAULT_SPEC_ROOT} if present)")
-    ap.add_argument("--strict", action="store_true",
-                    help="exit 1 on defects this tool owns: registry/spec drift, "
-                         "executable-annotation conflicts, or a safety-critical "
-                         "requirement whose only L3 test is a scaffold")
-    ap.add_argument("--selftest", action="store_true", help="run the built-in self-test")
-    args = ap.parse_args()
-
-    if args.selftest:
-        return _selftest()
-
-    repo = args.repo.resolve()
-    entry = f"clusters/{args.env}"
-    if not (repo / entry / "kustomization.yaml").is_file():
-        sys.stderr.write(f"[status] no Flux entrypoint at {entry}/kustomization.yaml — "
-                         f"cannot derive reconciliation. SKIPPED (not green).\n")
-        return 3
-
-    explicit_spec = args.spec_root is not None
-    spec_root = (args.spec_root or DEFAULT_SPEC_ROOT).resolve()
-    spec_ids = spec_requirement_ids(spec_root)
-    if spec_ids is None and explicit_spec:
-        sys.stderr.write(f"[status] --spec-root {spec_root} has no "
-                         f"docs/00-vision-and-scope.md. SKIPPED (not green).\n")
-        return 3
-
-    registry = cov.parse_registry(args.registry)
-    if not registry:
-        sys.stderr.write("[status] registry parsed to zero requirements — refusing to "
-                         "emit a report that would read as 100% of nothing.\n")
-        return 1
-    tests = cov.discover_tests(args.tests)
-    applied, nodes = reconciled_files(repo, entry)
-    gitops_roots = {n["resolved"].split("/", 1)[0] for n in nodes if n["exists"]}
-    impl = scan_implementation(repo, tracked_files(repo))
-    rep = build(repo, registry, tests, impl, applied, gitops_roots, spec_ids, args.env)
-    rep["flux_graph"] = nodes
-    rep["gitops_roots"] = sorted(gitops_roots)
-    rep["reconciled_file_count"] = len(applied)
-
-    advisory = read_advisory(CONF_ROOT / "_artifacts" / "report.junit.xml")
-    rep["advisory_untracked_junit"] = advisory
-
-    args.out.mkdir(parents=True, exist_ok=True)
-    (args.out / "status.json").write_text(json.dumps(rep, indent=2) + "\n")
-    (args.out / "status.md").write_text(render_md(rep, nodes, advisory))
-
-    s = rep["summary"]
-    print(f"[status] {s['total_requirements']} requirements | reconciled "
-          f"{s['impl_reconciled']} (+{s['impl_reconciled_partial']} partial) | "
-          f"executable test {s['verify_executable']} | "
-          f"wired+testable {s['highest_derivable_state_reached']} "
-          f"(+{s['wired_or_partial_and_testable'] - s['wired_and_testable']} partial) | "
-          f"scaffold-only {s['verify_scaffold_only']} | "
-          f"safety-critical w/o executable L3 "
-          f"{s['safety_critical_without_executable_l3']} | PROVEN: not derivable")
-    print(f"[status] wrote {args.out/'status.md'} and {args.out/'status.json'}")
-
-    rc = 0
-    if args.strict:
-        h = rep["hygiene"]
-        if h["registry_not_in_spec"] or h["spec_not_in_registry"]:
-            sys.stderr.write(f"[status] STRICT: registry/spec drift "
-                             f"{h['registry_not_in_spec']} {h['spec_not_in_registry']}\n")
-            rc = 1
-        if h["annotation_conflicts"]:
-            sys.stderr.write(f"[status] STRICT: tests annotated executable with TODO "
-                             f"bodies: {h['annotation_conflicts']}\n")
-            rc = 1
-        if rep["safety_critical_without_executable_l3"]:
-            sys.stderr.write(f"[status] STRICT: safety-critical requirement(s) whose L3 "
-                             f"test is a scaffold: "
-                             f"{rep['safety_critical_without_executable_l3']}\n")
-            rc = 1
-    return rc
-
-
-if __name__ == "__main__":
-    raise SystemExit(main())

conformance/run.sh

@@ -8,14 +8,9 @@
 #
 #   ./run.sh                          # full suite; skips L1/L2/L3 if no cluster
 #   ./run.sh --requirements R-NET-01,R-NET-06   # merge-gate subset (tagged)
-#   ./run.sh --layer l2               # one layer (l0|l1|l2|l3|meta|all; l0/l1 need no cluster)
+#   ./run.sh --layer l2               # one layer
 #   ./run.sh --requirements R-NET-01,R-NET-02 --layer l2   # intersect: tagged AND in layer
 #   ./run.sh --coverage-only          # regenerate coverage report only (no cluster)
-#   ./run.sh --status                 # DERIVED STATUS report only (no cluster): per
-#                                     #   requirement, is it implemented, is it actually
-#                                     #   reconciled by Flux, is its test executable or a
-#                                     #   scaffold. Proof-of-pass is deliberately NOT
-#                                     #   derived — see lib/status.py.
 #   ./run.sh --lint                   # structural-validate test defs (no cluster)
 #   ./run.sh --strict-coverage        # hygiene gate: fail on tag/annotation drift
 #   ./run.sh --requirements R-NET-01  # ALSO asserts those IDs are covered (merge gate)
@@ -25,8 +20,6 @@
 #   _artifacts/report.junit.xml        JUnit (Chainsaw output, or skipped-scaffold shim)
 #   _artifacts/coverage/coverage.md    human coverage report
 #   _artifacts/coverage/coverage.json  machine coverage report
-#   _artifacts/status/status.md        human requirement-status report (--status)
-#   _artifacts/status/status.json      machine requirement-status report (--status)
 #
 # Phase 0: no cluster exists. The behavior tests are valid scaffolds that cannot
 # execute yet, so run.sh emits a *skipped* JUnit and a coverage report, and exits
@@ -43,13 +36,12 @@ source "${CONF_ROOT}/versions.env"
 
 ARTIFACTS="${CONF_ROOT}/_artifacts"
 COVERAGE_OUT="${ARTIFACTS}/coverage"
-STATUS_OUT="${ARTIFACTS}/status"
 JUNIT_OUT="${ARTIFACTS}/report.junit.xml"
 PYTHON="${PYTHON:-python3}"
 
 LAYER="all"
 REQS=""
-MODE="run"          # run | coverage-only | status | lint
+MODE="run"          # run | coverage-only | lint
 STRICT_COVERAGE=0
 REQUIRE_ALL=0
 FORCE_NO_CLUSTER=0
@@ -92,7 +84,6 @@ while [ $# -gt 0 ]; do
     --layer)          LAYER="$2"; shift 2 ;;
     --requirements)   REQS="$(conf_normalize_reqs "$2")"; shift 2 ;;
     --coverage-only)  MODE="coverage-only"; shift ;;
-    --status)         MODE="status"; shift ;;
     --lint)           MODE="lint"; shift ;;
     --strict-coverage) STRICT_COVERAGE=1; shift ;;
     --require-all)    REQUIRE_ALL=1; shift ;;
@@ -102,7 +93,7 @@ while [ $# -gt 0 ]; do
   esac
 done
 
-mkdir -p "${ARTIFACTS}" "${COVERAGE_OUT}" "${STATUS_OUT}"
+mkdir -p "${ARTIFACTS}" "${COVERAGE_OUT}"
 
 log() { printf '\033[1m[run.sh]\033[0m %s\n' "$*"; }
 
@@ -120,23 +111,6 @@ gen_coverage() {
     "${flags[@]}"
 }
 
-# --- Derived STATUS report (no cluster) ------------------------------------------------
-# Coverage answers "does a tagged test EXIST?". Status answers the operator's question —
-# "what is the real state of each requirement given what is actually in the repo?" — by
-# deriving, per requirement: implementing artifact? reachable from a Flux Kustomization
-# (reconciled, not merely authored)? tagged test that is EXECUTABLE rather than a TODO
-# scaffold? It REFUSES to derive whether a test passed: nothing tracked in git records a
-# run result, and a status report that defaults to green is worse than none.
-gen_status() {
-  "${PYTHON}" "${CONF_ROOT}/lib/status.py" \
-    --repo "$(cd "${CONF_ROOT}/.." && pwd)" \
-    --registry "${CONF_ROOT}/requirements/registry.yaml" \
-    --tests "${CONF_ROOT}/tests" \
-    --out "${STATUS_OUT}" \
-    --env "${CONF_ENV:-dev}" \
-    "$@"
-}
-
 # --- Structural lint (no cluster): parse each test def and check the tag convention ----
 lint_tests() {
   log "Linting test definitions (structural, no cluster)"
@@ -280,40 +254,26 @@ run_chainsaw() {
   return 0
 }
 
-# --- L0/L1: the cluster-INDEPENDENT layers (docs/14) ------------------------------------
+# --- L1 rendering: cluster-INDEPENDENT render checks (docs/14) --------------------------
-# docs/14 puts TWO layers in CI without a cluster:
+# L1 verifies ServiceDefinition -> rendered-resources against golden expectations; it needs
-#   L0 static/policy  tests/l0-static/**   each dir carries a `check.sh`        (repo invariants)
+# no live cluster. Each l1-rendering test dir carries a `render-check.sh` (renderer + golden
-#   L1 rendering      tests/l1-rendering/** each dir carries a `render-check.sh` (render == golden)
+# diff + structural asserts). We run those directly and emit a REAL pass/fail JUnit, instead
-# Both are executed directly here and emit a REAL pass/fail JUnit, instead of being skipped
+# of skipping L1 with the cluster-gated layers (T-0.3). Honors the same --layer/--requirements
-# with the cluster-gated layers. Honors the same --layer/--requirements selection as the rest
+# selection as the rest of the suite (select_dirs), filtered to the l1-rendering subtree.
-# of the suite (select_dirs), filtered to those two subtrees.
+l1_selected_dirs() {
-OFFLINE_SUBTREES='/l0-static/|/l1-rendering/'
+  select_dirs | grep '/l1-rendering/' || true
-
-offline_selected_dirs() {
-  select_dirs | grep -E "${OFFLINE_SUBTREES}" || true
 }
 
-# True when EVERY requested layer is cluster-independent, so the offline result is the
+run_l1_offline() {
-# whole result and run.sh must exit on it (the ci/l0.sh + ci/l1.sh merge-gate paths).
+  local -a l1dirs
-layers_are_offline_only() {
+  mapfile -t l1dirs < <(l1_selected_dirs)
-  local l
+  if [ "${#l1dirs[@]}" -eq 0 ]; then
-  [ "${LAYER}" = all ] && return 1
+    log "No L1 render tests match the selection (layer=${LAYER} reqs='${REQS}')."
-  for l in ${LAYER//,/ }; do
-    case "$l" in l0|l1) ;; *) return 1 ;; esac
-  done
-  return 0
-}
-
-run_offline_checks() {
-  local -a dirs
-  mapfile -t dirs < <(offline_selected_dirs)
-  if [ "${#dirs[@]}" -eq 0 ]; then
-    log "No L0/L1 offline checks match the selection (layer=${LAYER} reqs='${REQS}')."
     return 0
   fi
-  log "Running ${#dirs[@]} offline L0/L1 check(s) (no cluster; docs/14)."
+  log "Running ${#l1dirs[@]} L1 render check(s) OFFLINE (no cluster; docs/14)."
   "${PYTHON}" "${CONF_ROOT}/lib/run_l1_offline.py" \
-    --junit "${JUNIT_OUT}" --label-prefix "${CONF_LABEL_PREFIX}" "${dirs[@]}"
+    --junit "${JUNIT_OUT}" --label-prefix "${CONF_LABEL_PREFIX}" "${l1dirs[@]}"
 }
 
 ###############################################################################
@@ -327,17 +287,6 @@ case "${MODE}" in
     log "coverage-only: done. See ${COVERAGE_OUT}/coverage.md"
     exit 0
     ;;
-  status)
-    # rc: 0 report written, 1 a derivation/strict failure, 3 a check could not run
-    # (SKIPPED, never counted as green — ci/lib.sh convention). Propagated as-is.
-    rc=0
-    # --strict-coverage is the harness's hygiene switch; apply it to status too.
-    status_flags=()
-    [ "${STRICT_COVERAGE}" = 1 ] && status_flags+=(--strict)
-    gen_status ${status_flags[@]+"${status_flags[@]}"} || rc=$?
-    log "status: done. See ${STATUS_OUT}/status.md"
-    exit "${rc}"
-    ;;
   lint)
     lint_tests
     log "lint: done."
@@ -345,30 +294,29 @@ case "${MODE}" in
     ;;
 esac
 
-# L0 static + L1 rendering are cluster-independent (docs/14): run the selected offline
+# L1 rendering is cluster-independent (docs/14): run the selected L1 render checks offline
-# checks and treat their result as authoritative REGARDLESS of cluster availability. An
+# and treat their result as authoritative REGARDLESS of cluster availability. A render-check
-# offline check failure is a real failure (nothing cluster-gated about it), so it always blocks.
+# failure is a real failure (nothing cluster-gated about it), so it always blocks.
-OFFLINE_RC=0
+L1_RC=0
-run_offline_checks || OFFLINE_RC=$?
+run_l1_offline || L1_RC=$?
-if layers_are_offline_only; then
+if [ "${LAYER}" = l1 ]; then
-  # Offline-only run (the ci/l0.sh + ci/l1.sh merge-gate paths): the offline result IS the result.
+  # L1-only run (the ci/l1.sh merge-gate path): the offline render result IS the result.
-  if [ "${OFFLINE_RC}" -eq 0 ]; then
+  if [ "${L1_RC}" -eq 0 ]; then
-    log "Done (offline ${LAYER}). JUnit: ${JUNIT_OUT} | Coverage: ${COVERAGE_OUT}/coverage.md"
+    log "Done (L1 render). JUnit: ${JUNIT_OUT} | Coverage: ${COVERAGE_OUT}/coverage.md"
   else
-    log "Offline ${LAYER} checks FAILED (rc=${OFFLINE_RC}). JUnit: ${JUNIT_OUT}"
+    log "L1 render checks FAILED (rc=${L1_RC}). JUnit: ${JUNIT_OUT}"
   fi
-  exit "${OFFLINE_RC}"
+  exit "${L1_RC}"
 fi
-if [ "${OFFLINE_RC}" -ne 0 ]; then
+if [ "${L1_RC}" -ne 0 ]; then
-  log "Offline L0/L1 checks FAILED (rc=${OFFLINE_RC}) — blocking (not cluster-gated)."
+  log "L1 render checks FAILED (rc=${L1_RC}) — blocking (render failure is not cluster-gated)."
-  exit "${OFFLINE_RC}"
+  exit "${L1_RC}"
 fi
 
 # run mode: need a cluster for L2/L3. Without one (Phase 0), emit skipped JUnit for the
 # cluster-gated layers. (L1 already ran offline above and passed.)
 if [ "${FORCE_NO_CLUSTER}" = 1 ] || ! conf_cluster_reachable; then
   log "No reachable cluster — emitting SKIPPED JUnit scaffold for L2/L3 (Phase 0 behavior)."
-  log "NOTE: the offline L0/L1 results above already ran for real; this shim covers L2/L3 only."
   log "Also running structural lint so scaffolds stay valid."
   lint_tests || { log "lint failed"; exit 1; }
   "${PYTHON}" "${CONF_ROOT}/lib/emit_skipped_junit.py" \

conformance/tests/README.md

@@ -4,7 +4,6 @@ Every test is a **Chainsaw `Test`** in its own directory:
 
 ```
 tests/
-  l0-static/           L0 — whole-repo invariants over the tracked tree (no cluster)
   l1-rendering/        L1 — ServiceDefinition -> rendered resources match golden
   l2-behavioral/       L2 — real runtime behavior (probes against a cluster)
   l3-fault-injection/  L3 — break something; assert fail-closed / no-data-loss
@@ -15,35 +14,6 @@ tests/
 The layer a test belongs to is its subtree **and** its `.../layer` annotation; the
 coverage generator cross-checks the two.
 
-### L0 static (`l0-static/`) — what belongs here, and what does not
-
-docs/14's L0 row is *"Pre-apply, in CI · manifests/ServiceDefinitions obey invariants"*.
-Kyverno (`platform/kyverno/`) covers the invariants expressible over a **single
-resource**. This subtree covers the L0 invariants that are **properties of the repository
-as a whole** and that no per-resource policy can state — e.g. *no plaintext secret exists
-anywhere in the tracked tree*, *no tracked RoleBinding grants a workload identity a
-privileged role*, *the spec-immutability gate actually blocks a modified spec*.
-
-Each `l0-static/<name>/` dir carries a **`check.sh`**: a self-contained offline assertion
-script (git + Python stdlib/PyYAML; no cluster, no network) that exits 0/1. It is executed:
-
-- by `conformance/run.sh --layer l0` (and as the offline leg of a full run), via
-  `lib/run_l1_offline.py`, producing a REAL pass/fail JUnit entry — never a skip; and
-- by the merge gate's **`ci/l0.sh`** stage, so these assertions run on every PR; and
-- by the wrapping Chainsaw `Test`, which runs the SAME `check.sh`, so an in-cluster run
-  executes byte-identical assertions.
-
-Two authoring rules specific to this layer:
-
-1. **State the gap.** A statically-provable half of a requirement is still only half.
-   Every `check.sh` header ends with an explicit *"what this does NOT prove"* list naming
-   the cluster-layer remainder. Covering a requirement at L0 does not finish it.
-2. **Exemptions must be self-policing.** Where a check must tolerate something (a
-   deliberately-plaintext policy fixture, an `.example` template), the exemption is
-   conditioned on a property that would break if the exemption were abused — e.g. the
-   plaintext Secret fixture is exempt *only while* a `kyverno-test.yaml` asserts the
-   policy rejects it. Never a bare path allowlist.
-
 ## Requirement-tagging convention (authoritative)
 
 A test declares the requirement(s) it verifies **three** ways; the label is the source
@@ -58,7 +28,7 @@ metadata:
   annotations:
     # Human-readable canonical list — MUST equal the label set.
     conformance.home.k3s/requirements: "R-NET-01"
-    # Declared layer (L0|L1|L2|L3|META) — must match the subtree.
+    # Declared layer (L1|L2|L3|META) — must match the subtree.
     conformance.home.k3s/layer: "L2"
     # true => R-TEST-05 requires this to be (or have) an L3 fault-injection test.
     conformance.home.k3s/safety-critical: "false"
@@ -95,7 +65,4 @@ and coverage extraction exact and robust.
   the PM — do not relax the assertion or the requirement.
 
 Scaffolds in this tree carry `TODO(T-x.y)` markers where a fixture/assertion lands as a
-later roadmap phase delivers the capability under test. `lib/status.py` treats such a
+later roadmap phase delivers the capability under test.
-marker — in the `chainsaw-test.yaml`, in `render-check.sh`, or in `check.sh` — as
-outranking any `conformance.home.k3s/executable: "true"` annotation, so a placeholder body
-can never be reported as an executable test.

conformance/tests/l0-static/least-privilege-rbac/chainsaw-test.yaml

@@ -1,44 +0,0 @@
-# =============================================================================
-# L0 STATIC — least privilege for workload identities and network reach (R-SEC-02)
-# Catalog: docs/14 L0 row, "Manifests/ServiceDefinitions obey invariants" — runs in
-#          CI WITHOUT a cluster.
-#
-# GIVEN the repository as git tracks it,
-# WHEN  every tracked RBAC object and every least-network-reach policy is inspected,
-# THEN  no in-repo role grants a wildcard or a privilege-escalation verb, no workload
-#       ServiceAccount is bound to a privileged role, the conformance runner's own
-#       privileged test identity stays scoped to its authored ClusterRole, and each
-#       least-network-reach policy both Enforces and is proven by an L0 negative row
-#       (or a generated-resource row) to actually close reach.
-#
-# STATUS: EXECUTABLE — and DELIBERATELY PARTIAL. `check.sh` is a pure offline assertion
-# script (git ls-files + PyYAML); conformance/run.sh --layer l0 and ci/l0.sh run it
-# directly, and this Chainsaw Test wraps the SAME script. It proves the STATIC half of
-# both clauses. The RUNTIME half — that the API server actually denies an unrequested
-# verb, and that pod-to-pod traffic is actually dropped — needs a cluster and is NOT
-# asserted under this requirement; see the gap list in check.sh's header. R-SEC-02 is
-# therefore covered, not finished.
-# =============================================================================
-apiVersion: chainsaw.kyverno.io/v1alpha1
-kind: Test
-metadata:
-  name: least-privilege-rbac
-  labels:
-    conformance.home.k3s/R-SEC-02: "true"
-  annotations:
-    conformance.home.k3s/requirements: "R-SEC-02"
-    conformance.home.k3s/layer: "L0"
-    conformance.home.k3s/safety-critical: "false"
-    conformance.home.k3s/executable: "true"
-    conformance.home.k3s/intent: >-
-      Workload identities carry no wildcard, no escalation verb and no privileged
-      binding; the least-network-reach policies are enforced and proven to reject.
-spec:
-  steps:
-    - name: scan-rbac-and-least-reach-policies
-      description: >-
-        Assert the authored least-privilege posture of every tracked RBAC object and
-        the enforced, test-proven status of the least-network-reach policies.
-      try:
-        - script:
-            content: bash ./check.sh

conformance/tests/l0-static/least-privilege-rbac/check.sh

@@ -1,269 +0,0 @@
-#!/usr/bin/env bash
-# =============================================================================
-# L0 static check (offline, no cluster) — least-privilege-rbac
-# Serves: R-SEC-02 ("The platform MUST follow least-privilege for workload
-#         identities and network reach." — spec docs/00-vision-and-scope.md;
-#         detail in docs/07 "least privilege ... least network reach").
-#
-# The requirement has TWO halves and this check covers the STATIC half of each,
-# at the layer docs/14 puts them ("L0 Static/policy | Pre-apply, in CI").
-#
-# GIVEN the repository as git tracks it,
-# WHEN  every tracked RBAC object and every least-network-reach policy is inspected,
-# THEN — workload identity —
-#       (1) no in-repo Role/ClusterRole grants a wildcard apiGroup, resource or verb,
-#           and none grants escalate/bind/impersonate (the privilege-escalation verbs);
-#       (2) no (Cluster)RoleBinding binds a ServiceAccount — a WORKLOAD identity — to
-#           `cluster-admin` or to any in-repo role holding wildcard rules. Human/Group
-#           subjects are out of scope: R-SEC-02 constrains workload identities, and the
-#           operator's own admin path is an OIDC group binding, deliberately;
-#       (3) every ServiceAccount subject is namespaced (no cluster-wide ambiguity);
-#       (4) the conformance runner's own SA — this workstream's PRIVILEGED TEST
-#           IDENTITY, whose least-privilege authoring docs/14 explicitly delegates to
-#           WS-TEST — is bound only to its own scoped ClusterRole, carries no wildcard
-#           and no privilege-escalation verb, and grants nothing over RBAC objects
-#           themselves (no rbac.authorization.k8s.io rules);
-#     — network reach —
-#       (5) the policies that implement least network reach exist, and each is Enforce
-#           (or carries the Flux token that substitutes to Enforce): the default-deny
-#           baseline generator, the baseline SHAPE validator, the host-namespace/
-#           privilege restriction, and the egress-class label requirement;
-#       (6) each of those policies is actually EXERCISED by the L0 Kyverno suite with a
-#           NEGATIVE row (result: fail) — a policy nobody proves rejects anything is a
-#           declaration, not a gate.
-#
-# WHAT THIS DOES *NOT* PROVE (honest gap, cluster layer — R-SEC-02 is NOT fully covered):
-#   - RUNTIME least privilege of workload identities: that the API server actually
-#     denies the conformance SA (or any workload SA) the verbs it was not granted.
-#     That needs a live cluster (L2) and no test asserts it today.
-#   - RUNTIME least network reach: that pod-to-pod traffic is actually denied. The
-#     behaviour is exercised by tests/l2-behavioral/{default-deny,deny-undeclared},
-#     but those are tagged R-NET-01/R-NET-02 and are cluster-gated (skipped in Phase 0);
-#     nothing proves it under an R-SEC-02 tag.
-#   - Helm-chart-generated RBAC (e.g. what a HelmRelease's chart creates in-cluster) is
-#     not in the repo, so it is outside this static scan entirely.
-# =============================================================================
-set -euo pipefail
-DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-ROOT="$(cd "${DIR}/../../../.." && pwd)"        # repo root
-PYTHON="${PYTHON:-python3}"
-
-cd "${ROOT}"
-"${PYTHON}" - "${ROOT}" <<'PY'
-import os, re, subprocess, sys
-from pathlib import Path
-
-try:
-    import yaml
-except ImportError:                                  # pragma: no cover
-    print("check FAIL: PyYAML is required (same dependency as catalog/render.py)",
-          file=sys.stderr)
-    raise SystemExit(1)
-
-ROOT = Path(sys.argv[1])
-FAILURES: list[str] = []
-CHECKS = 0
-
-
-def check(cond, msg):
-    global CHECKS
-    CHECKS += 1
-    if not cond:
-        FAILURES.append(msg)
-
-
-def tracked() -> list[str]:
-    out = subprocess.run(["git", "-C", str(ROOT), "ls-files", "-z"],
-                         capture_output=True, text=True, check=True).stdout
-    return [p for p in out.split("\0") if p]
-
-
-def docs_of(rel: str):
-    try:
-        return [d for d in yaml.safe_load_all((ROOT / rel).read_text(errors="replace"))
-                if isinstance(d, dict)]
-    except Exception:
-        return []
-
-
-FILES = tracked()
-YAML_FILES = [f for f in FILES if re.search(r"\.(ya?ml)(\.example)?$", f)]
-check(len(YAML_FILES) > 50, f"only {len(YAML_FILES)} YAML files seen — not a real checkout")
-
-ESCALATION_VERBS = {"escalate", "bind", "impersonate"}
-PRIVILEGED_ROLES = {"cluster-admin", "admin", "edit"}
-
-roles: dict[tuple[str, str], dict] = {}   # (kind, name) -> {"file":…, "wildcard":bool, …}
-bindings: list[dict] = []
-
-for rel in YAML_FILES:
-    for d in docs_of(rel):
-        kind = d.get("kind")
-        name = (d.get("metadata") or {}).get("name")
-        if kind in ("Role", "ClusterRole"):
-            wildcard, escalating, rbac_over_rbac = False, set(), False
-            for r in d.get("rules") or []:
-                if not isinstance(r, dict):
-                    continue
-                for field in ("apiGroups", "resources", "verbs"):
-                    if "*" in (r.get(field) or []):
-                        wildcard = True
-                escalating |= set(v for v in (r.get("verbs") or [])) & ESCALATION_VERBS
-                if "rbac.authorization.k8s.io" in (r.get("apiGroups") or []):
-                    rbac_over_rbac = True
-            roles[(kind, name)] = {"file": rel, "wildcard": wildcard,
-                                   "escalating": escalating, "rbac": rbac_over_rbac,
-                                   "doc": d}
-            # (1) no wildcards, no privilege-escalation verbs.
-            check(not wildcard,
-                  f"{rel}: {kind}/{name} grants a wildcard apiGroup/resource/verb — that is "
-                  "not least-privilege (R-SEC-02)")
-            check(not escalating,
-                  f"{rel}: {kind}/{name} grants privilege-escalation verb(s) "
-                  f"{sorted(escalating)} (R-SEC-02)")
-        elif kind in ("RoleBinding", "ClusterRoleBinding"):
-            bindings.append({"file": rel, "name": name, "doc": d})
-
-check(len(roles) > 0, "no Role/ClusterRole found in the repo — the RBAC scan is not looking at it")
-check(len(bindings) > 0, "no RoleBinding/ClusterRoleBinding found — the RBAC scan is not looking at it")
-
-for b in bindings:
-    d, rel, name = b["doc"], b["file"], b["name"]
-    ref = d.get("roleRef") or {}
-    ref_kind, ref_name = ref.get("kind"), ref.get("name")
-    sa_subjects = [s for s in (d.get("subjects") or [])
-                   if isinstance(s, dict) and s.get("kind") == "ServiceAccount"]
-    local = roles.get((ref_kind, ref_name))
-    ref_is_privileged = (ref_name in PRIVILEGED_ROLES) or bool(local and local["wildcard"])
-    # (2) a WORKLOAD identity must never hold a privileged role.
-    check(not (sa_subjects and ref_is_privileged),
-          f"{rel}: {d.get('kind')}/{name} binds ServiceAccount(s) "
-          f"{[s.get('name') for s in sa_subjects]} to the privileged role {ref_name!r} — "
-          "workload identities must be least-privilege (R-SEC-02)")
-    # (3) SA subjects must name their namespace (a ClusterRoleBinding without one is
-    #     ambiguous and, in a RoleBinding, silently defaults to the binding's namespace).
-    for s in sa_subjects:
-        check(bool(s.get("namespace")) or d.get("kind") == "RoleBinding",
-              f"{rel}: {d.get('kind')}/{name} has a ServiceAccount subject "
-              f"{s.get('name')!r} with no namespace (R-SEC-02)")
-
-# --------------------------------- (4) our own privileged test identity, in detail
-RBAC_FILE = "conformance/incluster/rbac.yaml"
-check(RBAC_FILE in FILES, f"{RBAC_FILE} is missing — the conformance runner has no authored RBAC")
-runner_role = roles.get(("ClusterRole", "conformance-runner"))
-check(runner_role is not None,
-      f"{RBAC_FILE}: no ClusterRole/conformance-runner — the test identity's grants are undefined")
-if runner_role:
-    check(runner_role["file"] == RBAC_FILE,
-          f"ClusterRole/conformance-runner is defined in {runner_role['file']}, not {RBAC_FILE}")
-    check(not runner_role["wildcard"], "ClusterRole/conformance-runner uses a wildcard (R-SEC-02)")
-    check(not runner_role["escalating"],
-          "ClusterRole/conformance-runner grants a privilege-escalation verb (R-SEC-02)")
-    check(not runner_role["rbac"],
-          "ClusterRole/conformance-runner grants rights over RBAC objects themselves — a test "
-          "identity that can rewrite RBAC is not least-privilege (R-SEC-02)")
-
-runner_bindings = [b for b in bindings
-                   if any(s.get("kind") == "ServiceAccount" and s.get("name") == "conformance-runner"
-                          for s in (b["doc"].get("subjects") or []))]
-check(len(runner_bindings) >= 1, "the conformance-runner ServiceAccount has no binding at all")
-for b in runner_bindings:
-    ref = b["doc"].get("roleRef") or {}
-    check(ref.get("name") == "conformance-runner",
-          f"{b['file']}: conformance-runner is bound to {ref.get('name')!r} instead of its own "
-          "scoped ClusterRole (R-SEC-02)")
-    for s in b["doc"].get("subjects") or []:
-        if s.get("kind") == "ServiceAccount":
-            check(s.get("namespace") == "conformance",
-                  f"{b['file']}: conformance-runner subject is in namespace "
-                  f"{s.get('namespace')!r}, expected 'conformance' (R-SEC-02)")
-
-# --------------------- (5)/(6) least NETWORK REACH: the policies exist, Enforce, and
-#                               are proven to reject something at L0.
-# Validating policies: must Enforce AND be proven to REJECT something at L0.
-REQUIRED_POLICIES = {
-    "validate-default-deny-shape":   "a baseline that leaves a direction open is rejected",
-    "restrict-host-namespaces":      "host namespaces / privileged containers are rejected",
-    "require-egress-class-label":    "workloads must declare their egress class",
-}
-# Generating policies have no validationFailureAction and no negative row: the proof
-# that they close network reach is that the L0 suite asserts the resource they GENERATE.
-REQUIRED_GENERATE_POLICIES = {
-    "require-default-deny-baseline": "every namespace gets a default-deny baseline",
-}
-policies: dict[str, dict] = {}
-for rel in YAML_FILES:
-    for d in docs_of(rel):
-        if d.get("kind") in ("ClusterPolicy", "Policy"):
-            policies[(d.get("metadata") or {}).get("name")] = {"file": rel, "doc": d}
-
-for pname, why in REQUIRED_POLICIES.items():
-    p = policies.get(pname)
-    check(p is not None, f"policy {pname!r} is missing — nothing enforces: {why} (R-SEC-02)")
-    if not p:
-        continue
-    action = str((p["doc"].get("spec") or {}).get("validationFailureAction", ""))
-    check(action == "Enforce" or action.startswith("${"),
-          f"{p['file']}: policy {pname!r} validationFailureAction is {action!r} — a least-reach "
-          "policy in Audit mode blocks nothing (R-SEC-02)")
-
-negatives: dict[str, int] = {}
-suites = [f for f in FILES if os.path.basename(f) == "kyverno-test.yaml"]
-check(len(suites) > 0, "no kyverno-test.yaml — the L0 policy gate is not exercised at all")
-for suite in suites:
-    for d in docs_of(suite):
-        for r in d.get("results") or []:
-            if isinstance(r, dict) and r.get("result") == "fail":
-                negatives[r.get("policy")] = negatives.get(r.get("policy"), 0) + 1
-
-for pname, why in REQUIRED_POLICIES.items():
-    check(negatives.get(pname, 0) > 0,
-          f"no L0 test row asserts that policy {pname!r} REJECTS a violating resource "
-          f"(result: fail) — unproven gate for: {why} (R-SEC-02)")
-
-# The generating half: the policy must exist, and the L0 suite must assert the resource
-# it generates (a `generatedResource` row that passes) — otherwise nothing proves a new
-# namespace actually arrives with its reach closed.
-for pname, why in REQUIRED_GENERATE_POLICIES.items():
-    p = policies.get(pname)
-    check(p is not None, f"policy {pname!r} is missing — nothing enforces: {why} (R-SEC-02)")
-    generated_rows = []
-    for suite in suites:
-        for d in docs_of(suite):
-            for r in d.get("results") or []:
-                if isinstance(r, dict) and r.get("policy") == pname \
-                        and r.get("generatedResource") and r.get("result") == "pass":
-                    generated_rows.append((suite, r.get("generatedResource")))
-    check(len(generated_rows) > 0,
-          f"no L0 test row asserts the resource generated by {pname!r} — unproven gate for: "
-          f"{why} (R-SEC-02)")
-    for suite, gen in generated_rows:
-        gp = os.path.normpath(os.path.join(os.path.dirname(suite), str(gen)))
-        check(gp in FILES, f"{suite}: generatedResource {gen!r} does not resolve to a tracked file")
-        golden = "\n".join(l for l in (ROOT / gp).read_text(errors="replace").splitlines()
-                           if not l.lstrip().startswith("#")) if gp in FILES else ""
-        check("NetworkPolicy" in golden and re.search(r"policyTypes:", golden),
-              f"{gp}: the asserted default-deny baseline is not a shaped NetworkPolicy (R-SEC-02)")
-
-# The host-namespace policy must prove BOTH of its rules reject, not just one.
-host_rules = set()
-for suite in suites:
-    for d in docs_of(suite):
-        for r in d.get("results") or []:
-            if isinstance(r, dict) and r.get("policy") == "restrict-host-namespaces" \
-                    and r.get("result") == "fail":
-                host_rules.add(r.get("rule"))
-check({"disallow-host-namespaces", "disallow-privileged-containers"} <= host_rules,
-      "the L0 suite does not prove BOTH restrict-host-namespaces rules reject a violating pod "
-      f"(negative rows seen for: {sorted(host_rules)}) — R-SEC-02")
-
-# --------------------------------------------------------------------------- verdict
-if FAILURES:
-    print(f"CHECK FAIL (least-privilege-rbac): {len(FAILURES)} of {CHECKS} assertions failed",
-          file=sys.stderr)
-    for f in FAILURES:
-        print(f"  - {f}", file=sys.stderr)
-    raise SystemExit(1)
-print(f"check OK: least-privilege-rbac — {CHECKS} assertions over {len(roles)} role(s), "
-      f"{len(bindings)} binding(s), {len(REQUIRED_POLICIES)} least-reach policies")
-PY

conformance/tests/l0-static/no-plaintext-secrets/chainsaw-test.yaml

@@ -1,43 +0,0 @@
-# =============================================================================
-# L0 STATIC — no plaintext secrets in git (R-SEC-01)
-# Catalog: docs/14 L0 row, "Manifests/ServiceDefinitions obey invariants ...
-#          no plaintext secrets" — runs in CI WITHOUT a cluster.
-#
-# GIVEN the repository as git tracks it,
-# WHEN  every tracked file is inspected,
-# THEN  every committed Secret is SOPS ciphertext, every ansible group_vars vault is
-#       ansible-vault encrypted, no private key material is tracked, decryption is
-#       delegated to an in-cluster SOPS key, and the L0 policy that rejects plaintext
-#       Secrets exists, Enforces, and is proven by a negative test row to reject one.
-#
-# STATUS: EXECUTABLE. `check.sh` is a pure offline assertion script (git ls-files +
-# PyYAML, no cluster, no network); conformance/run.sh --layer l0 and ci/l0.sh run it
-# directly, and this Chainsaw Test wraps the SAME script so an in-cluster run executes
-# the identical assertions. Encryption at rest is necessary but NOT sufficient: the
-# runtime half of "decrypted only in-cluster", and a scan of git HISTORY, are named as
-# explicit gaps in check.sh's header and are NOT claimed here.
-# =============================================================================
-apiVersion: chainsaw.kyverno.io/v1alpha1
-kind: Test
-metadata:
-  name: no-plaintext-secrets
-  labels:
-    conformance.home.k3s/R-SEC-01: "true"
-  annotations:
-    conformance.home.k3s/requirements: "R-SEC-01"
-    conformance.home.k3s/layer: "L0"
-    conformance.home.k3s/safety-critical: "false"
-    conformance.home.k3s/executable: "true"
-    conformance.home.k3s/intent: >-
-      No secret is stored in plaintext in git: committed Secrets are SOPS ciphertext,
-      ansible vaults are encrypted, no private key material is tracked, and the
-      no-plaintext-secrets policy gate is enforced and proven to reject.
-spec:
-  steps:
-    - name: scan-tracked-tree-for-plaintext-secrets
-      description: >-
-        Assert over the whole tracked tree that no secret material is stored in
-        plaintext and that decryption is delegated to the in-cluster SOPS key.
-      try:
-        - script:
-            content: bash ./check.sh

conformance/tests/l0-static/no-plaintext-secrets/check.sh

@@ -1,287 +0,0 @@
-#!/usr/bin/env bash
-# =============================================================================
-# L0 static check (offline, no cluster) — no-plaintext-secrets
-# Serves: R-SEC-01 ("Secrets MUST NOT be stored in plaintext in git; they MUST be
-#         encrypted at rest in the repo and decrypted only in-cluster."
-#         — spec docs/00-vision-and-scope.md; detail in docs/07.)
-#
-# docs/14 puts exactly this invariant at L0 ("Static/policy | Pre-apply, in CI |
-# ... no plaintext secrets"), so it is asserted here over the WHOLE tracked tree,
-# with no cluster. Run by `conformance/run.sh --layer l0` and by ci/l0.sh.
-#
-# GIVEN the repository as git tracks it,
-# WHEN  every tracked file is inspected,
-# THEN  (1) SOPS is configured at the repo root with a PUBLIC age recipient;
-#       (2) every committed Kubernetes Secret is SOPS-encrypted — a top-level
-#           `sops:` block with a mac AND every data/stringData value is ENC[...];
-#       (3) the only two exempt classes are narrow and self-policing:
-#             a. `*.example` templates, which must NOT be referenced by any
-#                kustomization and whose secret-named values must be placeholders;
-#             b. the Kyverno L0 negative fixture, which is exempt ONLY while it is
-#                actually wired into a kyverno-test.yaml that asserts
-#                `disallow-plaintext-secrets` REJECTS it (result: fail);
-#       (4) every ansible-vault group_vars file is `$ANSIBLE_VAULT;`-encrypted, and
-#           no secret-valued ansible key carries a literal outside a vault;
-#       (5) no private key material of any kind is tracked (PEM private keys, an
-#           age SECRET key, kubeconfig client-key-data);
-#       (6) decryption is delegated to the cluster: a Flux Kustomization declares
-#           `decryption.provider: sops` with a secretRef, and the private half of
-#           that key pair is nowhere in the repo (5).
-#
-# WHAT THIS DOES *NOT* PROVE (honest gap, cluster layer):
-#   - that the decrypted Secret only ever materialises in-cluster (the runtime half
-#     of "decrypted only in-cluster") — needs a live cluster (L2), untested today.
-#   - that no plaintext secret exists in git HISTORY. This walks the WORKING TREE at
-#     HEAD only; docs/07's history-scan acceptance gate needs a secret scanner over
-#     all revisions and is NOT implemented here.
-#   - value-level entropy: a placeholder-looking string that is in fact a real
-#     credential is not detectable statically.
-# =============================================================================
-set -euo pipefail
-DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-ROOT="$(cd "${DIR}/../../../.." && pwd)"        # repo root
-PYTHON="${PYTHON:-python3}"
-
-cd "${ROOT}"
-"${PYTHON}" - "${ROOT}" <<'PY'
-import os, re, subprocess, sys
-from pathlib import Path
-
-try:
-    import yaml
-except ImportError:                                  # pragma: no cover
-    print("check FAIL: PyYAML is required (same dependency as catalog/render.py)",
-          file=sys.stderr)
-    raise SystemExit(1)
-
-ROOT = Path(sys.argv[1])
-FAILURES: list[str] = []
-CHECKS = 0
-
-
-def check(cond, msg):
-    global CHECKS
-    CHECKS += 1
-    if not cond:
-        FAILURES.append(msg)
-
-
-def tracked() -> list[str]:
-    out = subprocess.run(["git", "-C", str(ROOT), "ls-files", "-z"],
-                         capture_output=True, text=True, check=True).stdout
-    return [p for p in out.split("\0") if p]
-
-
-FILES = tracked()
-check(len(FILES) > 50, f"git ls-files returned only {len(FILES)} paths — not a real checkout")
-
-
-def docs_of(rel: str):
-    """Parsed YAML documents of a tracked file, or [] when it is not parseable YAML."""
-    p = ROOT / rel
-    try:
-        return [d for d in yaml.safe_load_all(p.read_text(errors="replace")) if isinstance(d, dict)]
-    except Exception:
-        return []
-
-
-YAMLISH = re.compile(r"\.(ya?ml)(\.example)?$")
-YAML_FILES = [f for f in FILES if YAMLISH.search(f)]
-
-# ---------------------------------------------------------------- (1) SOPS config
-sops_cfg = ROOT / ".sops.yaml"
-check(sops_cfg.is_file(), ".sops.yaml is missing from the repository root (R-SEC-01)")
-if sops_cfg.is_file():
-    cfg = yaml.safe_load(sops_cfg.read_text()) or {}
-    rules = cfg.get("creation_rules") or []
-    check(len(rules) > 0, ".sops.yaml declares no creation_rules — new secrets would be committed unencrypted")
-    for r in rules:
-        pr = r.get("path_regex", "<none>")
-        recips = str(r.get("age", ""))
-        check(bool(recips), f".sops.yaml creation_rule {pr!r} has no age recipient")
-        check(all(x.startswith("age1") for x in re.split(r"[,\s]+", recips) if x),
-              f".sops.yaml creation_rule {pr!r} recipient is not a PUBLIC age1... key")
-        check("AGE-SECRET-KEY" not in recips.upper(),
-              f".sops.yaml creation_rule {pr!r} contains an age PRIVATE key")
-
-# --------------------------------------------- Kyverno negative-fixture exemption
-# A plaintext Secret is tolerated ONLY where it is the negative case that proves the
-# L0 policy rejects plaintext. We do not trust the path: the file must be listed in a
-# kyverno-test.yaml whose results assert disallow-plaintext-secrets returns `fail`.
-POLICY_NAME = "disallow-plaintext-secrets"
-negative_fixtures: set[str] = set()
-kyverno_suites = [f for f in FILES if os.path.basename(f) == "kyverno-test.yaml"]
-check(len(kyverno_suites) > 0, "no kyverno-test.yaml found — the L0 policy gate is not exercised at all")
-for suite in kyverno_suites:
-    docs = docs_of(suite)
-    if not docs:
-        continue
-    d = docs[0]
-    results = d.get("results") or []
-    rejects_plaintext = any(r.get("policy") == POLICY_NAME and r.get("result") == "fail"
-                            for r in results if isinstance(r, dict))
-    if not rejects_plaintext:
-        continue
-    base = os.path.dirname(suite)
-    for res in d.get("resources") or []:
-        negative_fixtures.add(os.path.normpath(os.path.join(base, str(res))))
-
-check(len(negative_fixtures) > 0,
-      f"no kyverno-test.yaml asserts that {POLICY_NAME} REJECTS a plaintext Secret "
-      "(result: fail) — the plaintext-secret gate is unproven, so nothing may be exempt")
-
-# --------------------------------------------------- (2)/(3) committed Secret scan
-SECRET_KEY_RE = re.compile(r"(pass|secret|token|key|credential|cookie|cert)", re.I)
-PLACEHOLDER_RE = re.compile(r"^\s*(|<[^>]*>|replace[-_ ]?me.*|change[-_ ]?me.*|placeholder.*|x{3,})\s*$", re.I)
-
-secrets_seen = 0
-for rel in YAML_FILES:
-    for d in docs_of(rel):
-        if d.get("kind") != "Secret":
-            continue
-        secrets_seen += 1
-        values = []
-        for field in ("data", "stringData"):
-            for k, v in (d.get(field) or {}).items():
-                values.append((k, "" if v is None else str(v)))
-
-        is_example = rel.endswith(".example")
-        is_negative_fixture = os.path.normpath(rel) in negative_fixtures
-
-        if is_example:
-            # Template: never applied, and its secret-named values must be placeholders.
-            refs = [k for k in YAML_FILES
-                    if os.path.basename(k) == "kustomization.yaml"
-                    and os.path.basename(rel) in (ROOT / k).read_text(errors="replace")]
-            check(not refs,
-                  f"{rel}: an unencrypted .example template is referenced by a kustomization "
-                  f"({refs}) — it would be applied as a plaintext Secret")
-            for k, v in values:
-                if SECRET_KEY_RE.search(k):
-                    check(bool(PLACEHOLDER_RE.match(v)),
-                          f"{rel}: template key {k!r} holds {v[:24]!r}, not a placeholder — "
-                          "a real credential may have been committed (R-SEC-01)")
-            continue
-
-        if is_negative_fixture:
-            # Exempt, but only as the proven negative case; assert it really is plaintext
-            # (an encrypted 'bad' fixture would silently stop testing anything).
-            continue
-
-        # Everything else: must be SOPS-encrypted at rest.
-        sops_block = d.get("sops")
-        check(isinstance(sops_block, dict) and bool(sops_block.get("mac")),
-              f"{rel}: committed Secret {d.get('metadata', {}).get('name')!r} has no top-level "
-              "`sops:` block with a mac — it is stored in plaintext (R-SEC-01)")
-        for k, v in values:
-            check(v.startswith("ENC["),
-                  f"{rel}: Secret value {k!r} is not SOPS ciphertext (expected ENC[...]) — "
-                  "plaintext secret material in git (R-SEC-01)")
-
-check(secrets_seen > 0, "no Kubernetes Secret manifests found at all — the scan is not looking at the repo")
-
-# ------------------------------------------------------------ (4) ansible vaults
-VAULT_HEADER = "$ANSIBLE_VAULT;"
-vault_files = [f for f in FILES
-               if os.path.basename(f) in ("vault.yml", "vault.yaml")]
-check(len(vault_files) > 0, "no ansible vault.yml files found — the vault scan is not looking at the repo")
-for rel in vault_files:
-    head = (ROOT / rel).read_text(errors="replace")[:len(VAULT_HEADER)]
-    check(head == VAULT_HEADER,
-          f"{rel}: an ansible group_vars vault file is NOT ansible-vault encrypted "
-          "(must start with the $ANSIBLE_VAULT header) — R-SEC-01")
-
-SECRET_ANSIBLE_KEYS = ("ansible_become_password", "ansible_become_pass", "ansible_password",
-                       "ansible_ssh_pass", "ansible_sudo_pass", "ansible_ssh_private_key_file")
-ASSIGN_RE = re.compile(
-    r"^\s*(" + "|".join(SECRET_ANSIBLE_KEYS) + r")\s*:\s*(.*?)\s*$")
-for rel in FILES:
-    p = ROOT / rel
-    if not p.is_file() or p.is_symlink():
-        continue
-    try:
-        text = p.read_text(errors="replace")
-    except Exception:
-        continue
-    if text.startswith(VAULT_HEADER):
-        continue                                   # encrypted at rest: opaque, by design
-    for lineno, line in enumerate(text.splitlines(), 1):
-        if line.lstrip().startswith("#"):
-            continue
-        m = ASSIGN_RE.match(line)
-        if not m:
-            continue
-        key, val = m.group(1), m.group(2).strip().strip("'\"")
-        ok = (val.startswith("{{") or val.startswith("$")
-              or bool(PLACEHOLDER_RE.match(val)) or val == "")
-        check(ok, f"{rel}:{lineno}: {key} is assigned the literal {val[:24]!r} outside an "
-                  "ansible-vault file — a plaintext secret in git (R-SEC-01)")
-
-# ------------------------------------------------------ (5) no private key material
-# NOTE: each marker is assembled from fragments so that THIS file never contains the
-# literal string it hunts for — otherwise the scanner would flag itself the moment it
-# is tracked, and the only way to silence it would be to exempt the scanner (which is
-# exactly the kind of self-exemption this suite exists to prevent).
-_DASH5 = "-" * 5
-PRIVKEY_PATTERNS = [
-    (re.compile(_DASH5 + r"BEGIN (?:[A-Z ]+ )?PRIVATE" + r" KEY" + _DASH5),
-     "a PEM private key block"),
-    (re.compile("AGE" + "-SECRET" + "-KEY-1"), "an age SECRET (decryption) key"),
-    (re.compile(r"^\s*client" + r"-key-data\s*:\s*\S"), "kubeconfig client-key-data"),
-]
-for rel in FILES:
-    p = ROOT / rel
-    if not p.is_file() or p.is_symlink():
-        continue
-    try:
-        text = p.read_text(errors="replace")
-    except Exception:
-        continue
-    if text.startswith(VAULT_HEADER):
-        continue
-    for rx, what in PRIVKEY_PATTERNS:
-        for lineno, line in enumerate(text.splitlines(), 1):
-            if line.lstrip().startswith("#"):
-                continue
-            if rx.search(line):
-                check(False, f"{rel}:{lineno}: tracked file contains {what} (R-SEC-01)")
-
-# ----------------------------------------- (6) decryption is delegated to the cluster
-sops_decryption = []
-for rel in YAML_FILES:
-    for d in docs_of(rel):
-        if d.get("kind") != "Kustomization":
-            continue
-        dec = d.get("spec", {}).get("decryption") if isinstance(d.get("spec"), dict) else None
-        if isinstance(dec, dict) and dec.get("provider") == "sops" and dec.get("secretRef"):
-            sops_decryption.append(rel)
-        # Flux overlay patches carry the same block inside a patch body.
-for rel in YAML_FILES:
-    text = (ROOT / rel).read_text(errors="replace")
-    if re.search(r"^\s*decryption:\s*$", text, re.M) and re.search(r"^\s*provider:\s*sops\s*$", text, re.M) \
-            and re.search(r"^\s*secretRef:\s*$", text, re.M):
-        sops_decryption.append(rel)
-check(len(sops_decryption) > 0,
-      "no Flux Kustomization declares `decryption: {provider: sops, secretRef: ...}` — nothing "
-      "proves decryption happens in-cluster rather than from committed key material (R-SEC-01)")
-
-# ------------------------------------------- (7) the L0 policy gate is real and strict
-policy_files = [f for f in FILES if f.endswith(f"{POLICY_NAME}.yaml")]
-check(len(policy_files) == 1, f"expected exactly one {POLICY_NAME} ClusterPolicy, found {policy_files}")
-for rel in policy_files:
-    d = (docs_of(rel) or [{}])[0]
-    action = str(d.get("spec", {}).get("validationFailureAction", ""))
-    check(action == "Enforce" or action.startswith("${"),
-          f"{rel}: validationFailureAction is {action!r} — the no-plaintext-secrets gate must "
-          "Enforce (or carry the Flux token that substitutes to Enforce)")
-
-# --------------------------------------------------------------------------- verdict
-if FAILURES:
-    print(f"CHECK FAIL (no-plaintext-secrets): {len(FAILURES)} of {CHECKS} assertions failed",
-          file=sys.stderr)
-    for f in FAILURES:
-        print(f"  - {f}", file=sys.stderr)
-    raise SystemExit(1)
-print(f"check OK: no-plaintext-secrets — {CHECKS} assertions over {len(FILES)} tracked files, "
-      f"{secrets_seen} committed Secret document(s), {len(vault_files)} ansible vault file(s)")
-PY

conformance/tests/l0-static/spec-immutability-enforced/chainsaw-test.yaml

@@ -1,46 +0,0 @@
-# =============================================================================
-# L0 STATIC — the specification is immutable to agents (R-TEST-06)
-# Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster.
-#
-# A governance rule is made testable by testing its ENFORCEMENT MECHANISM, not by
-# observing that no agent misbehaved today.
-#
-# GIVEN the enforcement mechanism (ci/spec-immutability.sh + ci/SPEC_PIN),
-# WHEN  it is driven against hermetic THROWAWAY spec repositories in each state,
-# THEN  a clean spec at the pinned SHA passes; an uncommitted edit under docs/ or
-#       decisions/, an untracked spec addition, and an unadopted committed drift each
-#       exit 1 and block the merge; an absent spec repo exits 3 (honest skip, never
-#       green); an unpinned SPEC_PIN fails closed; and the merge gate runs the stage
-#       and blocks on its failure.
-#
-# THE REAL SPEC REPO IS NEVER TOUCHED: every case runs against a git repo created under
-# mktemp with SPEC_REPO pointed at it and a copy of ci/ carrying a fixture SPEC_PIN.
-#
-# STATUS: EXECUTABLE. `check.sh` is a pure offline assertion script; conformance/run.sh
-# --layer l0 and ci/l0.sh run it directly, and this Chainsaw Test wraps the SAME script.
-# NOT proven (named in check.sh's header): that the Gitea server REQUIRES this status
-# check before merge — branch protection is operator configuration outside the repo.
-# =============================================================================
-apiVersion: chainsaw.kyverno.io/v1alpha1
-kind: Test
-metadata:
-  name: spec-immutability-enforced
-  labels:
-    conformance.home.k3s/R-TEST-06: "true"
-  annotations:
-    conformance.home.k3s/requirements: "R-TEST-06"
-    conformance.home.k3s/layer: "L0"
-    conformance.home.k3s/safety-critical: "false"
-    conformance.home.k3s/executable: "true"
-    conformance.home.k3s/intent: >-
-      A spec modification is detected and blocks the merge; an unverifiable spec skips
-      honestly rather than passing; and the gate wires that stage as blocking.
-spec:
-  steps:
-    - name: drive-the-spec-immutability-gate
-      description: >-
-        Run the real ci/spec-immutability.sh against throwaway spec repos in every
-        state and assert its exit code, then assert the gate wires it as blocking.
-      try:
-        - script:
-            content: bash ./check.sh

conformance/tests/l0-static/spec-immutability-enforced/check.sh

@@ -1,165 +0,0 @@
-#!/usr/bin/env bash
-# =============================================================================
-# L0 static check (offline, no cluster) — spec-immutability-enforced
-# Serves: R-TEST-06 ("AI agents MUST NOT modify the specifications (this docs/ tree,
-#         the requirements, or accepted ADRs). Specs are changed only by the human
-#         operator/architect." — spec docs/00-vision-and-scope.md.)
-#
-# HOW A GOVERNANCE RULE IS MADE TESTABLE
-# -------------------------------------
-# "An agent must not do X" cannot be proven by observing that no agent did X today.
-# What CAN be proven — and is what actually protects the spec — is that the MECHANISM
-# which detects and blocks a spec modification works, and is wired into the merge gate
-# as a blocking stage. So this test drives the real `ci/spec-immutability.sh` against
-# HERMETIC THROWAWAY spec repositories and asserts its exit-code behaviour.
-#
-# THE REAL SPEC REPO IS NEVER TOUCHED. Every case runs against a fixture git repo under
-# mktemp, with SPEC_REPO pointed at it and a COPY of ci/ (so the fixture's SPEC_PIN can
-# be set without editing the tracked one). Nothing here writes outside its temp dir —
-# which is itself the R-TEST-06 discipline this test exists to defend.
-#
-# GIVEN the enforcement mechanism (ci/spec-immutability.sh + ci/SPEC_PIN),
-# WHEN  it is run against a spec repo in each state,
-# THEN  (1) clean and at the pinned SHA           -> exit 0 (pass)
-#       (2) an UNCOMMITTED edit under docs/       -> exit 1 (blocks the merge)
-#       (3) an uncommitted edit under decisions/  -> exit 1 (accepted ADRs are spec too)
-#       (4) a NEW COMMIT (HEAD drifted from pin)  -> exit 1 (an "adopted" change must be
-#                                                    an explicit reviewed SPEC_PIN bump)
-#       (5) an untracked file added to the spec   -> exit 1 (porcelain catches additions)
-#       (6) the spec repo absent                  -> exit 3 (honest SKIP, never green)
-#       (7) an empty/comment-only SPEC_PIN        -> exit 1 (an unpinned gate is no gate)
-# AND   (8) the tracked ci/SPEC_PIN really is a 40-hex commit SHA, and
-#       (9) the gate wires spec-immutability as a stage whose FAIL sets the merge block —
-#           i.e. the mechanism above is load-bearing, not dead code.
-#
-# WHAT THIS DOES *NOT* PROVE (honest gap):
-#   - That the Gitea server actually REQUIRES this status check before merge. Branch
-#     protection is operator configuration outside the repo; nothing in-repo can assert it.
-#   - Intent: an agent could still open a spec PR. The rule is enforced by the gate
-#     blocking it, which is exactly what (1)-(9) assert; it is not enforced by prediction.
-# =============================================================================
-set -euo pipefail
-DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-ROOT="$(cd "${DIR}/../../../.." && pwd)"        # repo root
-CI_SRC="${ROOT}/ci"
-
-PASS=0
-FAILED=0
-note() { printf '  ok   %s\n' "$*"; PASS=$((PASS + 1)); }
-bad()  { printf '  FAIL %s\n' "$*" >&2; FAILED=$((FAILED + 1)); }
-
-WORK="$(mktemp -d)"
-trap 'rm -rf "${WORK}"' EXIT
-
-# --- a hermetic stand-in for the spec repo -----------------------------------------
-# Mirrors the real spec's shape: a docs/ tree and a decisions/ tree, one commit.
-make_spec_repo() {
-  local d="$1"
-  mkdir -p "${d}/docs" "${d}/decisions"
-  # Fixture prose deliberately carries NO R-* token: ci/touched-requirements.sh scans the
-  # non-comment lines of changed files, so an example ID here would be attributed to the
-  # change as a touched requirement that does not exist in the registry.
-  printf 'normative spec text (fixture)\n' > "${d}/docs/00-vision-and-scope.md"
-  printf 'accepted decision\n' > "${d}/decisions/adr-0001-example.md"
-  git -C "$d" init -q -b main
-  git -C "$d" config user.email [email protected]
-  git -C "$d" config user.name spec
-  git -C "$d" config commit.gpgsign false     # throwaway: never sign, ignore host gpg config
-  git -C "$d" add -A
-  git -C "$d" commit -qm "spec baseline"
-}
-
-# A COPY of ci/ whose SPEC_PIN we may set freely. The script resolves CI_DIR from its own
-# location and sources lib.sh + versions.env from there, so a copy is self-contained.
-make_ci_copy() {
-  local dst="$1" pin="$2"
-  mkdir -p "$dst"
-  cp "${CI_SRC}/spec-immutability.sh" "${CI_SRC}/lib.sh" "${CI_SRC}/versions.env" "$dst/"
-  printf '%s\n' "$pin" > "${dst}/SPEC_PIN"
-  chmod +x "${dst}/spec-immutability.sh"
-}
-
-# run_case <expected-rc> <description> <spec-repo|-> <pin>
-run_case() {
-  local want="$1" desc="$2" spec="$3" pin="$4" rc=0
-  local cidir; cidir="$(mktemp -d -p "$WORK" ci.XXXXXX)"
-  make_ci_copy "$cidir" "$pin"
-  SPEC_REPO="$spec" REPO_ROOT="$WORK" "${cidir}/spec-immutability.sh" >/dev/null 2>&1 || rc=$?
-  if [ "$rc" = "$want" ]; then note "${desc} -> exit ${rc}"; else
-    bad "${desc}: expected exit ${want}, got ${rc}"; fi
-}
-
-for f in spec-immutability.sh lib.sh versions.env SPEC_PIN; do
-  if [ -f "${CI_SRC}/${f}" ]; then note "ci/${f} present"; else bad "ci/${f} is missing"; fi
-done
-
-SPEC="${WORK}/spec"
-make_spec_repo "$SPEC"
-PINNED="$(git -C "$SPEC" rev-parse HEAD)"
-
-# (1) clean + at the pin -> pass.
-run_case 0 "clean spec at the pinned SHA passes" "$SPEC" "$PINNED"
-
-# (2) an uncommitted edit under docs/ -> block.
-printf 'normative spec text REWRITTEN BY AN AGENT\n' > "${SPEC}/docs/00-vision-and-scope.md"
-run_case 1 "uncommitted edit under docs/ BLOCKS" "$SPEC" "$PINNED"
-git -C "$SPEC" checkout -q -- docs/00-vision-and-scope.md
-
-# (3) an uncommitted edit to an accepted ADR -> block (decisions/ is spec too).
-printf 'reversed decision\n' > "${SPEC}/decisions/adr-0001-example.md"
-run_case 1 "uncommitted edit under decisions/ BLOCKS" "$SPEC" "$PINNED"
-git -C "$SPEC" checkout -q -- decisions/adr-0001-example.md
-
-# (5) an untracked addition to the spec tree -> block.
-printf 'a spec doc an agent invented\n' > "${SPEC}/docs/99-invented.md"
-run_case 1 "untracked file added to the spec BLOCKS" "$SPEC" "$PINNED"
-rm -f "${SPEC}/docs/99-invented.md"
-
-# clean again -> still passes (proves cases 2/3/5 failed for the edit, not permanently).
-run_case 0 "spec restored to clean passes again" "$SPEC" "$PINNED"
-
-# (4) a committed spec change that the pin has not adopted -> block.
-printf 'an extra normative line added by an agent\n' >> "${SPEC}/docs/00-vision-and-scope.md"
-git -C "$SPEC" add -A
-git -C "$SPEC" commit -qm "agent edits the spec"
-DRIFTED="$(git -C "$SPEC" rev-parse HEAD)"
-run_case 1 "committed spec drift from the pinned SHA BLOCKS" "$SPEC" "$PINNED"
-# ...and adopting it EXPLICITLY (a reviewed SPEC_PIN bump) is the only way to green.
-run_case 0 "an explicit SPEC_PIN bump adopts the advance" "$SPEC" "$DRIFTED"
-
-# (6) spec repo unavailable -> honest SKIP (exit 3), never a pass.
-run_case 3 "absent spec repo SKIPS (exit 3), never green" "${WORK}/no-such-spec" "$PINNED"
-
-# (7) an unpinned gate is not a gate.
-run_case 1 "empty SPEC_PIN FAILS closed" "$SPEC" "# only a comment, no SHA"
-
-# (8) the tracked pin is a real commit SHA.
-REAL_PIN="$(grep -vE '^\s*#' "${CI_SRC}/SPEC_PIN" | tr -d '[:space:]')"
-if printf '%s' "$REAL_PIN" | grep -Eq '^[0-9a-f]{40}$'; then
-  note "ci/SPEC_PIN is a 40-hex commit SHA"
-else
-  bad "ci/SPEC_PIN is not a 40-hex commit SHA: '${REAL_PIN}'"
-fi
-
-# (9) the mechanism is load-bearing: the merge gate runs it as a stage AND a FAIL there
-#     sets the block. Asserted against the gate source because running the whole gate
-#     from inside a conformance check would recurse into this very check.
-GATE="${CI_SRC}/gate.sh"
-if grep -Eq '^[[:space:]]*run_stage[[:space:]]+spec-immutability' "$GATE"; then
-  note "ci/gate.sh runs spec-immutability as a gate stage"
-else
-  bad "ci/gate.sh does not run spec-immutability as a stage — the gate is not wired"
-fi
-if awk '/OUTCOME\[spec-immutability\]/,/^esac/' "$GATE" | grep -Eq '^[[:space:]]*fail\).*BLOCK=1'; then
-  note "ci/gate.sh blocks the merge when spec-immutability FAILS"
-else
-  bad "ci/gate.sh does not set the merge block on a spec-immutability failure"
-fi
-
-echo
-if [ "$FAILED" -eq 0 ]; then
-  echo "check OK: spec-immutability-enforced — ${PASS} assertions (R-TEST-06)"
-  exit 0
-fi
-echo "CHECK FAIL (spec-immutability-enforced): ${FAILED} failed, ${PASS} passed" >&2
-exit 1

conformance/tests/l1-rendering/container-workload-render/golden/expected.yaml

@@ -61,70 +61,6 @@ spec:
         image: nginx:1.27
         ports:
         - containerPort: 8080
-      - name: oauth2-proxy
-        image: quay.io/oauth2-proxy/oauth2-proxy:v7.8.1
-        imagePullPolicy: IfNotPresent
-        args:
-        - --http-address=0.0.0.0:4180
-        - --reverse-proxy=true
-        - --provider=oidc
-        - --oidc-issuer-url=https://keycloak.home.keithjackson.dev/realms/homelab
-        - --upstream=http://127.0.0.1:8080
-        - --email-domain=*
-        - --scope=openid email profile
-        - --oidc-groups-claim=groups
-        - --allowed-group=platform-admins
-        - --skip-provider-button=true
-        - --redirect-url=https://paas-demo.home.keithjackson.dev/oauth2/callback
-        - --cookie-domain=.home.keithjackson.dev
-        - --whitelist-domain=.home.keithjackson.dev
-        - --cookie-secure=true
-        - --pass-access-token=true
-        - --set-xauthrequest=true
-        env:
-        - name: OAUTH2_PROXY_CLIENT_ID
-          valueFrom:
-            secretKeyRef:
-              name: paas-demo-oauth2-proxy
-              key: client-id
-        - name: OAUTH2_PROXY_CLIENT_SECRET
-          valueFrom:
-            secretKeyRef:
-              name: paas-demo-oauth2-proxy
-              key: client-secret
-        - name: OAUTH2_PROXY_COOKIE_SECRET
-          valueFrom:
-            secretKeyRef:
-              name: paas-demo-oauth2-proxy
-              key: cookie-secret
-        ports:
-        - name: http
-          containerPort: 4180
-          protocol: TCP
-        resources:
-          requests:
-            cpu: 25m
-            memory: 64Mi
-          limits:
-            memory: 128Mi
-        securityContext:
-          allowPrivilegeEscalation: false
-          readOnlyRootFilesystem: true
-          capabilities:
-            drop:
-            - ALL
-        livenessProbe:
-          httpGet:
-            path: /ping
-            port: http
-          initialDelaySeconds: 5
-          periodSeconds: 15
-        readinessProbe:
-          httpGet:
-            path: /ping
-            port: http
-          initialDelaySeconds: 5
-          periodSeconds: 15
 ---
 # Service
 apiVersion: v1
@@ -141,9 +77,6 @@ spec:
   - name: port-8080
     port: 8080
     targetPort: 8080
-  - name: authproxy
-    port: 4180
-    targetPort: 4180
 ---
 # CiliumNetworkPolicy gateway-ingress allow (R-NET-01/02): paas-demo-allow-gateway-ingress
 apiVersion: cilium.io/v2
@@ -162,7 +95,7 @@ spec:
     - ingress
     toPorts:
     - ports:
-      - port: '4180'
+      - port: '8080'
         protocol: TCP
 ---
 # CiliumNetworkPolicy world egress allow (egress class governs the path; R-NET-05): paas-demo-allow-internet-egress
@@ -180,10 +113,6 @@ spec:
   egress:
   - toEntities:
     - world
-  - toEndpoints:
-    - matchLabels:
-        io.kubernetes.pod.namespace: keycloak
-        app: keycloak
 ---
 # HTTPRoute paas-demo-8080 (R-TLS/R-DNS)
 apiVersion: gateway.networking.k8s.io/v1
@@ -210,7 +139,7 @@ spec:
         value: /
     backendRefs:
     - name: paas-demo
-      port: 4180
+      port: 8080
 ---
 # ServiceMonitor (R-OBS-01)
 apiVersion: monitoring.coreos.com/v1
@@ -233,7 +162,7 @@ spec:
     interval: 30s
 ---
 # Flux ImageRepository (R-UPD)
-apiVersion: image.toolkit.fluxcd.io/v1
+apiVersion: image.toolkit.fluxcd.io/v1beta2
 kind: ImageRepository
 metadata:
   name: paas-demo
@@ -243,7 +172,7 @@ spec:
   interval: 1h0m0s
 ---
 # Flux ImagePolicy (R-UPD)
-apiVersion: image.toolkit.fluxcd.io/v1
+apiVersion: image.toolkit.fluxcd.io/v1beta2
 kind: ImagePolicy
 metadata:
   name: paas-demo

conformance/tests/l1-rendering/default-deny-baseline-render/golden/expected.yaml

@@ -93,7 +93,7 @@ spec:
     interval: 30s
 ---
 # Flux ImageRepository (R-UPD)
-apiVersion: image.toolkit.fluxcd.io/v1
+apiVersion: image.toolkit.fluxcd.io/v1beta2
 kind: ImageRepository
 metadata:
   name: baseline-demo
@@ -103,7 +103,7 @@ spec:
   interval: 1h0m0s
 ---
 # Flux ImagePolicy (R-UPD)
-apiVersion: image.toolkit.fluxcd.io/v1
+apiVersion: image.toolkit.fluxcd.io/v1beta2
 kind: ImagePolicy
 metadata:
   name: baseline-demo

conformance/tests/l1-rendering/egress-class-render/golden/egress-internet.expected.yaml

@@ -84,70 +84,6 @@ spec:
         volumeMounts:
         - name: config
           mountPath: /data/config
-      - name: oauth2-proxy
-        image: quay.io/oauth2-proxy/oauth2-proxy:v7.8.1
-        imagePullPolicy: IfNotPresent
-        args:
-        - --http-address=0.0.0.0:4180
-        - --reverse-proxy=true
-        - --provider=oidc
-        - --oidc-issuer-url=https://keycloak.home.keithjackson.dev/realms/homelab
-        - --upstream=http://127.0.0.1:8989
-        - --email-domain=*
-        - --scope=openid email profile
-        - --oidc-groups-claim=groups
-        - --allowed-group=media-admins
-        - --skip-provider-button=true
-        - --redirect-url=https://sonarr.home.keithjackson.dev/oauth2/callback
-        - --cookie-domain=.home.keithjackson.dev
-        - --whitelist-domain=.home.keithjackson.dev
-        - --cookie-secure=true
-        - --pass-access-token=true
-        - --set-xauthrequest=true
-        env:
-        - name: OAUTH2_PROXY_CLIENT_ID
-          valueFrom:
-            secretKeyRef:
-              name: sonarr-oauth2-proxy
-              key: client-id
-        - name: OAUTH2_PROXY_CLIENT_SECRET
-          valueFrom:
-            secretKeyRef:
-              name: sonarr-oauth2-proxy
-              key: client-secret
-        - name: OAUTH2_PROXY_COOKIE_SECRET
-          valueFrom:
-            secretKeyRef:
-              name: sonarr-oauth2-proxy
-              key: cookie-secret
-        ports:
-        - name: http
-          containerPort: 4180
-          protocol: TCP
-        resources:
-          requests:
-            cpu: 25m
-            memory: 64Mi
-          limits:
-            memory: 128Mi
-        securityContext:
-          allowPrivilegeEscalation: false
-          readOnlyRootFilesystem: true
-          capabilities:
-            drop:
-            - ALL
-        livenessProbe:
-          httpGet:
-            path: /ping
-            port: http
-          initialDelaySeconds: 5
-          periodSeconds: 15
-        readinessProbe:
-          httpGet:
-            path: /ping
-            port: http
-          initialDelaySeconds: 5
-          periodSeconds: 15
       volumes:
       - name: config
         persistentVolumeClaim:
@@ -168,9 +104,6 @@ spec:
   - name: port-8989
     port: 8989
     targetPort: 8989
-  - name: authproxy
-    port: 4180
-    targetPort: 4180
 ---
 # CiliumNetworkPolicy gateway-ingress allow (R-NET-01/02): sonarr-allow-gateway-ingress
 apiVersion: cilium.io/v2
@@ -189,7 +122,7 @@ spec:
     - ingress
     toPorts:
     - ports:
-      - port: '4180'
+      - port: '8989'
         protocol: TCP
 ---
 # CiliumNetworkPolicy world egress allow (egress class governs the path; R-NET-05): sonarr-allow-internet-egress
@@ -207,10 +140,6 @@ spec:
   egress:
   - toEntities:
     - world
-  - toEndpoints:
-    - matchLabels:
-        io.kubernetes.pod.namespace: keycloak
-        app: keycloak
 ---
 # HTTPRoute sonarr-8989 (R-TLS/R-DNS)
 apiVersion: gateway.networking.k8s.io/v1
@@ -237,7 +166,7 @@ spec:
         value: /
     backendRefs:
     - name: sonarr
-      port: 4180
+      port: 8989
 ---
 # ServiceMonitor (R-OBS-01)
 apiVersion: monitoring.coreos.com/v1
@@ -283,7 +212,7 @@ spec:
         description: 'sonarr has been unreachable for 5m (SLI: up).'
 ---
 # Flux ImageRepository (R-UPD)
-apiVersion: image.toolkit.fluxcd.io/v1
+apiVersion: image.toolkit.fluxcd.io/v1beta2
 kind: ImageRepository
 metadata:
   name: sonarr
@@ -293,7 +222,7 @@ spec:
   interval: 1h0m0s
 ---
 # Flux ImagePolicy (R-UPD)
-apiVersion: image.toolkit.fluxcd.io/v1
+apiVersion: image.toolkit.fluxcd.io/v1beta2
 kind: ImagePolicy
 metadata:
   name: sonarr

conformance/tests/l1-rendering/egress-class-render/golden/egress-vpn.expected.yaml

@@ -84,70 +84,6 @@ spec:
         volumeMounts:
         - name: config
           mountPath: /data/config
-      - name: oauth2-proxy
-        image: quay.io/oauth2-proxy/oauth2-proxy:v7.8.1
-        imagePullPolicy: IfNotPresent
-        args:
-        - --http-address=0.0.0.0:4180
-        - --reverse-proxy=true
-        - --provider=oidc
-        - --oidc-issuer-url=https://keycloak.home.keithjackson.dev/realms/homelab
-        - --upstream=http://127.0.0.1:8989
-        - --email-domain=*
-        - --scope=openid email profile
-        - --oidc-groups-claim=groups
-        - --allowed-group=media-admins
-        - --skip-provider-button=true
-        - --redirect-url=https://sonarr.home.keithjackson.dev/oauth2/callback
-        - --cookie-domain=.home.keithjackson.dev
-        - --whitelist-domain=.home.keithjackson.dev
-        - --cookie-secure=true
-        - --pass-access-token=true
-        - --set-xauthrequest=true
-        env:
-        - name: OAUTH2_PROXY_CLIENT_ID
-          valueFrom:
-            secretKeyRef:
-              name: sonarr-oauth2-proxy
-              key: client-id
-        - name: OAUTH2_PROXY_CLIENT_SECRET
-          valueFrom:
-            secretKeyRef:
-              name: sonarr-oauth2-proxy
-              key: client-secret
-        - name: OAUTH2_PROXY_COOKIE_SECRET
-          valueFrom:
-            secretKeyRef:
-              name: sonarr-oauth2-proxy
-              key: cookie-secret
-        ports:
-        - name: http
-          containerPort: 4180
-          protocol: TCP
-        resources:
-          requests:
-            cpu: 25m
-            memory: 64Mi
-          limits:
-            memory: 128Mi
-        securityContext:
-          allowPrivilegeEscalation: false
-          readOnlyRootFilesystem: true
-          capabilities:
-            drop:
-            - ALL
-        livenessProbe:
-          httpGet:
-            path: /ping
-            port: http
-          initialDelaySeconds: 5
-          periodSeconds: 15
-        readinessProbe:
-          httpGet:
-            path: /ping
-            port: http
-          initialDelaySeconds: 5
-          periodSeconds: 15
       volumes:
       - name: config
         persistentVolumeClaim:
@@ -168,9 +104,6 @@ spec:
   - name: port-8989
     port: 8989
     targetPort: 8989
-  - name: authproxy
-    port: 4180
-    targetPort: 4180
 ---
 # CiliumNetworkPolicy gateway-ingress allow (R-NET-01/02): sonarr-allow-gateway-ingress
 apiVersion: cilium.io/v2
@@ -189,7 +122,7 @@ spec:
     - ingress
     toPorts:
     - ports:
-      - port: '4180'
+      - port: '8989'
         protocol: TCP
 ---
 # CiliumNetworkPolicy world egress allow (egress class governs the path; R-NET-05): sonarr-allow-internet-egress
@@ -207,10 +140,6 @@ spec:
   egress:
   - toEntities:
     - world
-  - toEndpoints:
-    - matchLabels:
-        io.kubernetes.pod.namespace: keycloak
-        app: keycloak
 ---
 # HTTPRoute sonarr-8989 (R-TLS/R-DNS)
 apiVersion: gateway.networking.k8s.io/v1
@@ -237,7 +166,7 @@ spec:
         value: /
     backendRefs:
     - name: sonarr
-      port: 4180
+      port: 8989
 ---
 # ServiceMonitor (R-OBS-01)
 apiVersion: monitoring.coreos.com/v1
@@ -283,7 +212,7 @@ spec:
         description: 'sonarr has been unreachable for 5m (SLI: up).'
 ---
 # Flux ImageRepository (R-UPD)
-apiVersion: image.toolkit.fluxcd.io/v1
+apiVersion: image.toolkit.fluxcd.io/v1beta2
 kind: ImageRepository
 metadata:
   name: sonarr
@@ -293,7 +222,7 @@ spec:
   interval: 1h0m0s
 ---
 # Flux ImagePolicy (R-UPD)
-apiVersion: image.toolkit.fluxcd.io/v1
+apiVersion: image.toolkit.fluxcd.io/v1beta2
 kind: ImagePolicy
 metadata:
   name: sonarr

conformance/tests/l2-behavioral/vm-default-deny/chainsaw-test.yaml

@@ -13,16 +13,11 @@
 #       podSelector:{} NetworkPolicy covers it). R-NET-01 for IaaS.
 #
 # METHOD (virtctl-free): the VM's ROOT is a CDI DataVolume on Longhorn (a mounted
-# PVC — NOT an ephemeral containerDisk), the spec-required, rebuild-surviving disk model
+# PVC — NOT an ephemeral containerDisk, whose boot path is broken on this k3s cluster:
-# for VMs (docs/06, R-STATE-01), matching fixtures/vm.yaml. (Feature-gate note,
+# KubeVirt v1.9 serves the container-disk binary via a k8s `image` volume gated by the
-# revalidated 2026-08: an earlier revision said the containerDisk boot path was broken on
+# ALPHA `ImageVolume` feature gate, disabled on k3s v1.31, so the apiserver prunes it
-# this cluster — KubeVirt v1.9 serves the container-disk binary via a k8s `image` volume
+# and the guest dies at OCI init. DataVolume roots bypass that path and are the
-# gated by the ALPHA `ImageVolume` feature gate, which k3s v1.31 had DISABLED, so the
+# spec-required disk model — docs/06). cloud-init runs the two probes and writes a
-# apiserver pruned it and the guest died at OCI init. That premise is now STALE: the
-# cluster is k3s v1.36.4, where ImageVolume is ENABLED by default
-# (kubernetes_feature_enabled{name="ImageVolume"}=1; a server-side dry-run RETAINS an
-# `image` volume rather than pruning it), so containerDisk roots would boot — DataVolume
-# is kept because it is the rebuild-surviving model.) cloud-init runs the two probes and writes a
 # PASS/FAIL verdict to a raw Longhorn block disk, then powers off (RerunOnFailure =>
 # stays down). A busybox reader pod reads the verdict back. The container default-deny
 # test proves the datapath fail-closed (default-deny-failclosed, L3); this asserts

conformance/tests/l2-behavioral/vm-lifecycle/chainsaw-test.yaml

@@ -21,25 +21,19 @@
 # (hermetic).
 #
 # DISK MODEL — CDI DataVolume on Longhorn (NOT an ephemeral containerDisk):
-#   The SPEC requires VMs to use Longhorn-backed disks that survive rebuild (docs/06,
+#   KubeVirt v1.9 supplies the container-disk binary to the `volumerootdisk` init
-#   R-STATE-01), so this test — like fixtures/vm.yaml and the L3 vm-data-survival partner
+#   container via a Kubernetes-native `image` volume (VolumeSource.Image) mounted at
-#   — boots from a CDI DataVolume on a Longhorn PVC (CDI imports the Fedora image into it
+#   /container-disk-binary. That k8s feature is gated by the ALPHA `ImageVolume`
-#   out-of-band via its importer pod, so the VM never touches the container-disk binary /
+#   feature gate, which is DISABLED on this k3s v1.31 cluster — so the apiserver
-#   image-volume path). Cost: a one-time CDI import per run (covered by the raised exec
+#   silently PRUNES the image volume + its mount, the binary is never present, and a
-#   timeout below).
+#   containerDisk VM dies at OCI init with:
-#   FEATURE-GATE NOTE (revalidated 2026-08 — the original "containerDisk is broken here"
+#     exec: "/container-disk-binary/usr/bin/container-disk": no such file or directory
-#   rationale is now STALE): KubeVirt v1.9 supplies the container-disk binary to the
+#   A DataVolume-backed root is a mounted Longhorn PVC (CDI imports the same Fedora
-#   `volumerootdisk` init container via a Kubernetes-native `image` volume
+#   image into it out-of-band via its importer pod) and therefore does NOT touch the
-#   (VolumeSource.Image) mounted at /container-disk-binary, gated by the `ImageVolume`
+#   container-disk binary / image-volume path at all — it boots on this cluster. It is
-#   feature gate. On the ORIGINAL k3s v1.31 cluster that ALPHA gate was DISABLED, so the
+#   also the disk model the SPEC requires for VMs (docs/06, R-STATE-01: Longhorn-backed
-#   apiserver silently PRUNED the image volume + its mount and a containerDisk VM died at
+#   disks that survive rebuild), so this mirrors the representative fixtures/vm.yaml.
-#   OCI init ("/container-disk-binary/usr/bin/container-disk: no such file or directory").
+#   Cost: a one-time CDI import per run (covered by the raised exec timeout below).
-#   The cluster is now k3s v1.36.4, where ImageVolume is ENABLED by default
-#   (kubernetes_feature_enabled{name="ImageVolume"}=1; a server-side dry-run of a Pod with
-#   an `image` volume RETAINS the volume + mount instead of pruning them). containerDisk
-#   roots would therefore boot on this cluster now — but DataVolume is retained regardless,
-#   because only it (not an ephemeral containerDisk) is the rebuild-surviving, spec-required
-#   disk model.
 # =============================================================================
 apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test

conformance/tests/l3-fault-injection/dr-restore/chainsaw-test.yaml

@@ -1,75 +1,17 @@
 # =============================================================================
-# L3 FAULT-INJECTION — DR restore from an OFF-CLUSTER backup  (R-STATE-03)
+# L3 FAULT-INJECTION SCAFFOLD — DR restore  (R-STATE-03)  *SAFETY-CRITICAL*
-#                      *SAFETY-CRITICAL*                    [RUNNABLE / self-skips]
+# Catalog: docs/14 "DR restore (R-STATE-03, L3)"
-# Catalog: docs/14-testing-and-conformance.md "DR restore (R-STATE-03, L3)"
-# DoD:     agents/definition-of-done.md — "DR-critical data restores from an
-#          **off-cluster** backup (drill for at least one service/phase)."
-# Spec:    docs/04-storage-and-state.md; ADR-0003 (Longhorn + Velero).
-# Impl:    platform/velero/** (Velero + kopia File System Backup -> the versitygw S3
-#          endpoint on the Proxmox HOST), reconciled by clusters/base/backup.yaml.
 #
-# GIVEN a DR-critical stateful workload with known data on the replicated `longhorn`
+# GIVEN an OFF-CLUSTER backup of a namespace + volumes,
-#       tier, and a Velero BackupStorageLocation whose object-store endpoint is
+# WHEN  the namespace + volumes are restored into a CLEAN state,
-#       PROVABLY OUTSIDE this cluster,
+# THEN  the service returns with its data.
-# WHEN  a backup of that namespace + its volume data is written to that off-cluster
-#       store, and then EVERY in-cluster copy of the data is DESTROYED (namespace,
-#       PVC and the underlying PersistentVolume/Longhorn replicas all deleted),
-# THEN  a Velero restore into a CLEAN, previously-nonexistent namespace brings the
-#       workload back and the known data reads back byte-identical.
 #
-# R-TEST-05 (fault injection, not happy-path): the restore is only meaningful because
+# R-TEST-05: proves recoverability by restoring into a genuinely clean namespace
-# the source is annihilated first. After the destroy step there is no PVC, no PV and
+# (not by reading the still-live original). Backup TARGET is OQ-BKP (OPEN) — the
-# no Longhorn volume left in the cluster holding this data — so the bytes that come
+# restore mechanism is Velero (T-4.3/T-4.4); this scaffold is target-agnostic.
-# back can ONLY have come from the off-cluster object store. A restore that silently
-# rebound a surviving in-cluster PV is explicitly excluded: the test captures the
-# original PV name, asserts it is GONE, and asserts the restored PVC is bound to a
-# DIFFERENT, newly provisioned PV.
 #
-# HOW "OFF-CLUSTER" IS PROVEN (not assumed) — step `assert-backup-target-is-off-cluster`:
+# STATUS: scaffold — needs T-4.3 (Velero + off-cluster schedule) + T-4.4 (drill).
-#   1. The BackupStorageLocation reports phase `Available` — Velero itself has
+# BLOCKED-BY spec open question OQ-BKP (off-cluster backup target) — see report.
-#      successfully reached and validated that object store.
-#   2. Its endpoint host is resolved to an IPv4 address and asserted to be NONE of:
-#      loopback; any Node InternalIP/ExternalIP; any Service ClusterIP; any Pod IP;
-#      inside any node's `spec.podCIDR(s)`; inside any `ServiceCIDR` (best-effort).
-#      It is also asserted not to be a cluster-internal DNS name (*.svc,
-#      *.cluster.local, localhost). => the bytes leave the cluster's address space.
-#   3. The backup's PodVolumeBackup carries a non-empty `status.snapshotID` — the
-#      kopia snapshot really landed in the remote repository, i.e. VOLUME DATA (not
-#      merely Kubernetes resource JSON) was uploaded off-cluster.
-#   4. Plus the destroy-then-restore sequence above, which removes every in-cluster
-#      alternative source for the data.
-#
-# KNOWN, DELIBERATE GAPS (asserted nowhere — do not read this test as claiming them):
-#   * OFF-HOST is NOT proven, only OFF-CLUSTER. On the dev estate the k3s nodes are
-#     VMs on the same Proxmox host that runs the versitygw S3 target (192.168.1.9),
-#     so this drill does not survive loss of that hypervisor. Off-site/off-host
-#     replication is an operator concern outside R-STATE-03's wording.
-#   * The test does NOT read the backup objects out of the bucket with an S3 client.
-#     Doing so would require handing the conformance runner the backup credential,
-#     widening a privileged test identity for weaker evidence than (1)-(4) above.
-#   * The test restores its OWN scratch workload, never a real service's data. A
-#     per-service drill (Nextcloud/Gitea/CNPG) stays an operator exercise.
-#   * Bare-metal DR (rebuild the cluster itself, then restore) is NOT exercised; this
-#     asserts namespace+volume restore into a live cluster, which is what the catalog
-#     entry and the DoD line ask for.
-#   * Backup ENCRYPTION at rest and retention/GC behaviour are not asserted.
-#
-# RUNNABLE / SELF-SKIPPING (a skip is honest; a fake pass is a defect). Skips CLEANLY
-# (green) when a precondition is absent: no `longhorn` StorageClass; no Velero API
-# (CRDs absent); no BackupStorageLocation configured at all; Velero server or its
-# node-agent DaemonSet not ready. It runs the full drill the moment those exist — no
-# edit needed. RBAC denial is NOT a skip: it fails loudly as a harness defect.
-#
-# DESTRUCTIVE BY DESIGN, SCOPED BY CONSTRUCTION: every object it destroys is one it
-# created itself, in two namespaces it names after its own Chainsaw namespace
-# (`<ns>-dr-src`, `<ns>-dr-dst`) and labels `conformance.home.k3s/created-by=<ns>`.
-# It never touches a real service's namespace, PVC or PV, and it never deletes a
-# Backup it did not create.
-#
-# TEARDOWN ALWAYS RUNS (`finally`, even on failure/timeout/panic): both scratch
-# namespaces, any leftover PV, the Restore CR, and the Backup — the latter via a
-# Velero DeleteBackupRequest so the objects are removed from the bucket too, with a
-# short `ttl` on the Backup as a belt-and-braces backstop if that request is lost.
 # =============================================================================
 apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
@@ -81,603 +23,31 @@ metadata:
     conformance.home.k3s/requirements: "R-STATE-03"
     conformance.home.k3s/layer: "L3"
     conformance.home.k3s/safety-critical: "true"
-    # executable => runs against a cluster; self-skips (green) until Velero + a
-    # BackupStorageLocation + the longhorn tier exist.
-    conformance.home.k3s/executable: "true"
     conformance.home.k3s/intent: >-
-      Seed known data on longhorn, back the namespace + volume data up to the Velero
+      Restore a namespace + volumes from an off-cluster backup into a clean state;
-      object store whose endpoint is proven to be outside this cluster, destroy every
+      the service returns with its data.
-      in-cluster copy (namespace, PVC and PV), restore into a clean namespace, and read
+    conformance.home.k3s/blocked-by: "OQ-BKP (off-cluster backup target, spec-open)"
-      the known data back from a newly provisioned volume. Skips cleanly until Velero,
-      a BackupStorageLocation and a longhorn StorageClass exist.
-    conformance.home.k3s/never-executed: >-
-      AUTHORED, NOT YET EXECUTED. No workstation in this repo has cluster access, and
-      the drill is destructive, so it must first be run by the operator against the dev
-      cluster in a maintenance window before R-STATE-03 may be called proven. Repo-derived
-      status deliberately reports proof as not-derivable (conformance/lib/status.py).
 spec:
-  # A full backup -> destroy -> restore cycle over kopia File System Backup, plus
-  # Longhorn provisioning on HDD, is far slower than any other test here. Bound it
-  # generously but explicitly (no sleep-and-hope; every wait below is a bounded poll).
-  # Worst-case bound of the drill step, summing every poll below (seed ~5m + stale-target
-  # clean ~5m + backup 15m + namespace destroy 12m + PV destroy 5m + restore 15m + restored
-  # workload Ready 10m ~= 67m). Declared at 75m so the step's own bounded polls, not this
-  # timeout, are what fail first and report WHICH phase stalled. Nothing here sleeps blindly.
-  timeouts:
-    exec: 75m
-    delete: 10m
-    cleanup: 15m
   steps:
-    # -----------------------------------------------------------------------------
+    - name: given-off-cluster-backup-exists
-    # Preconditions + the OFF-CLUSTER property of the backup target. This step makes
+      try:
-    # no changes to the cluster; it either skips cleanly or proves the target is
+        - script:
-    # genuinely outside the cluster's address space.
+            content: 'echo "TODO(T-4.3): ensure a Velero backup of ns+volumes exists off-cluster"'
-    # -----------------------------------------------------------------------------
+    - name: when-restore-into-clean-namespace
-    - name: assert-backup-target-is-off-cluster
       try:
         - script:
             content: |
-              set -eu
+              set -euo pipefail
-
+              echo "TODO(T-4.4): create a clean target ns; trigger Velero restore of ns+PVs"
-              # ---- helpers -------------------------------------------------------
+    - name: then-service-returns-with-data
-              # Classify a kubectl failure: an RBAC denial is a HARNESS DEFECT and must
-              # fail loudly; an absent API is a genuine precondition gap and may skip.
-              die_or_skip() {
-                case "$2" in
-                  *orbidden*|*Unauthorized*|*cannot\ list*|*cannot\ get*)
-                    echo "FAIL: conformance runner lacks RBAC for $1 -- fix conformance/incluster/rbac.yaml" >&2
-                    echo "$2" >&2
-                    exit 1 ;;
-                  *)
-                    echo "SKIP: $1 not available on this cluster -- $2"
-                    echo DR_RESTORE_SKIPPED_OK
-                    exit 0 ;;
-                esac
-              }
-
-              ip2int() {
-                _q="$(echo "$1" | tr '.' ' ')"
-                set -- $_q
-                [ $# -eq 4 ] || { echo -1; return 0; }
-                echo $(( ($1 * 16777216) + ($2 * 65536) + ($3 * 256) + $4 ))
-              }
-
-              # in_cidr <ip> <a.b.c.d/len> -> 0 when the ip is inside the prefix
-              in_cidr() {
-                case "$2" in *"/"*) ;; *) return 1 ;; esac
-                _net="${2%/*}"; _len="${2#*/}"
-                case "$_len" in ''|*[!0-9]*) return 1 ;; esac
-                [ "$_len" -ge 0 ] && [ "$_len" -le 32 ] || return 1
-                _i="$(ip2int "$1")"; _n="$(ip2int "$_net")"
-                [ "$_i" -ge 0 ] && [ "$_n" -ge 0 ] || return 1
-                [ "$_len" -eq 0 ] && return 0
-                _d=1; _s=0
-                while [ "$_s" -lt $((32 - _len)) ]; do _d=$((_d * 2)); _s=$((_s + 1)); done
-                [ $(( _i / _d )) -eq $(( _n / _d )) ]
-              }
-
-              # ---- Gate 1: the replicated storage tier the DR-critical data lives on --
-              if ! kubectl get storageclass longhorn >/dev/null 2>&1; then
-                echo "SKIP: 'longhorn' StorageClass absent -- no DR-critical data tier on this cluster yet"
-                echo DR_RESTORE_SKIPPED_OK
-                exit 0
-              fi
-
-              # ---- Gate 2: the Velero API + a configured BackupStorageLocation --------
-              BSL_LINES="$(kubectl get backupstoragelocations.velero.io -A \
-                -o jsonpath='{range .items[*]}{.metadata.namespace}{" "}{.metadata.name}{" "}{.status.phase}{" "}{.spec.provider}{" "}{.spec.default}{" "}{.spec.config.s3Url}{" "}{.spec.objectStorage.bucket}{"\n"}{end}' \
-                2>&1)" || die_or_skip "velero.io/backupstoragelocations" "$BSL_LINES"
-              if [ -z "$(echo "$BSL_LINES" | tr -d '[:space:]')" ]; then
-                echo "SKIP: Velero API present but NO BackupStorageLocation is configured -- no off-cluster backup target yet"
-                echo DR_RESTORE_SKIPPED_OK
-                exit 0
-              fi
-              echo "backup storage locations:"
-              echo "$BSL_LINES"
-
-              VNS="$(echo "$BSL_LINES" | head -1 | cut -d' ' -f1)"
-
-              # ---- Gate 3: Velero server + node-agent (kopia FSB) actually ready -------
-              # File System Backup is the volume-DATA path (platform/velero/helmrelease.yaml:
-              # deployNodeAgent=true, uploaderType=kopia, snapshotsEnabled=false). Without a
-              # ready node-agent no volume data can be captured, so the drill is not runnable.
-              VREADY=0; NAREADY=0
-              for i in $(seq 1 12); do
-                VREADY="$(kubectl -n "$VNS" get deploy velero -o jsonpath='{.status.readyReplicas}' 2>/dev/null || true)"
-                NAREADY="$(kubectl -n "$VNS" get daemonset node-agent -o jsonpath='{.status.numberReady}' 2>/dev/null || true)"
-                echo "poll $i: velero readyReplicas=${VREADY:-0} node-agent numberReady=${NAREADY:-0}"
-                if [ "${VREADY:-0}" -ge 1 ] && [ "${NAREADY:-0}" -ge 1 ]; then break; fi
-                sleep 10
-              done
-              if [ "${VREADY:-0}" -lt 1 ] || [ "${NAREADY:-0}" -lt 1 ]; then
-                echo "SKIP: Velero server and/or its node-agent DaemonSet are not ready in ns '$VNS' -- backup/restore path not available"
-                echo DR_RESTORE_SKIPPED_OK
-                exit 0
-              fi
-
-              # ---- Pick the BSL this drill will use: prefer the default, and it MUST be
-              #      Available. A CONFIGURED-BUT-UNAVAILABLE target is NOT a precondition
-              #      gap -- it is a live R-STATE-03 defect, so it FAILS (after a bounded
-              #      wait for first-validation on a cold start), it does not skip. --------
-              BSL_NS=""; BSL_NAME=""; BSL_URL=""; BSL_BUCKET=""
-              for i in $(seq 1 12); do
-                BSL_LINES="$(kubectl get backupstoragelocations.velero.io -A \
-                  -o jsonpath='{range .items[*]}{.metadata.namespace}{" "}{.metadata.name}{" "}{.status.phase}{" "}{.spec.provider}{" "}{.spec.default}{" "}{.spec.config.s3Url}{" "}{.spec.objectStorage.bucket}{"\n"}{end}' 2>/dev/null || true)"
-                CAND="$(echo "$BSL_LINES" | grep ' Available ' | grep ' true ' | head -1 || true)"
-                [ -n "$CAND" ] || CAND="$(echo "$BSL_LINES" | grep ' Available ' | head -1 || true)"
-                if [ -n "$CAND" ]; then
-                  BSL_NS="$(echo "$CAND" | cut -d' ' -f1)"
-                  BSL_NAME="$(echo "$CAND" | cut -d' ' -f2)"
-                  BSL_URL="$(echo "$CAND" | cut -d' ' -f6)"
-                  BSL_BUCKET="$(echo "$CAND" | cut -d' ' -f7)"
-                  break
-                fi
-                echo "poll $i: no Available BackupStorageLocation yet"
-                sleep 10
-              done
-              if [ -z "$BSL_NAME" ]; then
-                echo "FAIL: a BackupStorageLocation is configured but NONE reached phase 'Available'." >&2
-                echo "      The off-cluster backup target is unreachable -- R-STATE-03 is BROKEN, not merely untested." >&2
-                echo "$BSL_LINES" >&2
-                exit 1
-              fi
-              echo "using BSL ${BSL_NS}/${BSL_NAME}  bucket=${BSL_BUCKET}  endpoint=${BSL_URL}"
-
-              # ---- OFF-CLUSTER PROOF (1): an endpoint we can evaluate ------------------
-              if [ -z "$BSL_URL" ]; then
-                echo "FAIL: BackupStorageLocation ${BSL_NS}/${BSL_NAME} exposes no spec.config.s3Url;" >&2
-                echo "      the OFF-CLUSTER property of the backup target cannot be established, so this" >&2
-                echo "      test refuses to claim it. (Extend this check for a non-S3 provider.)" >&2
-                exit 1
-              fi
-              HOST="${BSL_URL#*://}"      # strip scheme
-              HOST="${HOST%%/*}"          # strip path
-              HOST="${HOST%%:*}"          # strip port
-              echo "backup endpoint host: $HOST"
-
-              case "$HOST" in
-                localhost|*.svc|*.svc.cluster.local|*.cluster.local)
-                  echo "FAIL: backup endpoint '$HOST' is a CLUSTER-INTERNAL name -- the backup target is IN the cluster." >&2
-                  echo "      An in-cluster backup store cannot satisfy R-STATE-03 (off-cluster backup)." >&2
-                  exit 1 ;;
-              esac
-
-              # Resolve to an IPv4 address. If it cannot be resolved we CANNOT establish the
-              # off-cluster property -- that is a failure, never a silent pass.
-              case "$HOST" in
-                *[!0-9.]*)
-                  # Endpoint given as a name. Try the resolvers that may exist in the
-                  # runner image, in order; if none is available or none resolves, the
-                  # test FAILS below rather than guessing.
-                  EPIP="$(getent ahostsv4 "$HOST" 2>/dev/null | head -1 | cut -d' ' -f1 || true)"
-                  [ -n "$EPIP" ] || EPIP="$(getent hosts "$HOST" 2>/dev/null | head -1 | cut -d' ' -f1 || true)"
-                  [ -n "$EPIP" ] || EPIP="$(nslookup "$HOST" 2>/dev/null | grep -A2 '^Name:' | grep '^Address' | head -1 | sed 's/.*: *//' || true)" ;;
-                *) EPIP="$HOST" ;;
-              esac
-              if [ -z "$EPIP" ]; then
-                echo "FAIL: could not resolve backup endpoint host '$HOST' to an IPv4 address;" >&2
-                echo "      the OFF-CLUSTER property cannot be asserted, so this test does not pass." >&2
-                exit 1
-              fi
-              echo "backup endpoint resolves to: $EPIP"
-
-              # ---- OFF-CLUSTER PROOF (2): the endpoint is outside every cluster address --
-              case "$EPIP" in 127.*|0.0.0.0) echo "FAIL: backup endpoint $EPIP is loopback -- not off-cluster" >&2; exit 1 ;; esac
-
-              NODE_IPS="$(kubectl get nodes -o jsonpath='{range .items[*]}{range .status.addresses[*]}{.address}{"\n"}{end}{end}' 2>/dev/null || true)"
-              for a in $NODE_IPS; do
-                if [ "$a" = "$EPIP" ]; then
-                  echo "FAIL: backup endpoint $EPIP is a NODE address of this cluster -- the backup is not off-cluster." >&2
-                  exit 1
-                fi
-              done
-              echo "endpoint is not any node address (checked $(echo "$NODE_IPS" | grep -c . || true) node addresses)"
-
-              SVC_IPS="$(kubectl get svc -A -o jsonpath='{range .items[*]}{.spec.clusterIP}{"\n"}{end}' 2>/dev/null || true)"
-              for a in $SVC_IPS; do
-                if [ "$a" = "$EPIP" ]; then
-                  echo "FAIL: backup endpoint $EPIP is a Service ClusterIP -- the backup target lives INSIDE the cluster." >&2
-                  exit 1
-                fi
-              done
-              echo "endpoint is not any Service ClusterIP (checked $(echo "$SVC_IPS" | grep -c . || true))"
-
-              POD_IPS="$(kubectl get pods -A -o jsonpath='{range .items[*]}{.status.podIP}{"\n"}{end}' 2>/dev/null || true)"
-              for a in $POD_IPS; do
-                if [ "$a" = "$EPIP" ]; then
-                  echo "FAIL: backup endpoint $EPIP is a POD IP -- the backup target is an in-cluster pod." >&2
-                  exit 1
-                fi
-              done
-              echo "endpoint is not any Pod IP (checked $(echo "$POD_IPS" | grep -c . || true))"
-
-              POD_CIDRS="$(kubectl get nodes -o jsonpath='{range .items[*]}{.spec.podCIDR}{"\n"}{range .spec.podCIDRs[*]}{@}{"\n"}{end}{end}' 2>/dev/null || true)"
-              for c in $POD_CIDRS; do
-                case "$c" in *:*) continue ;; esac      # IPv6 prefixes are out of scope here
-                if in_cidr "$EPIP" "$c"; then
-                  echo "FAIL: backup endpoint $EPIP falls inside node podCIDR $c -- it is in the cluster pod network." >&2
-                  exit 1
-                fi
-              done
-              echo "endpoint is outside every node podCIDR (checked: $(echo "$POD_CIDRS" | tr '\n' ' '))"
-
-              # Best-effort (needs networking.k8s.io/ServiceCIDR, which the least-privilege
-              # runner may not read): the checks above are the authoritative ones.
-              SVC_CIDRS="$(kubectl get servicecidrs.networking.k8s.io -o jsonpath='{range .items[*]}{range .spec.cidrs[*]}{@}{"\n"}{end}{end}' 2>/dev/null || true)"
-              if [ -n "$SVC_CIDRS" ]; then
-                for c in $SVC_CIDRS; do
-                  case "$c" in *:*) continue ;; esac
-                  if in_cidr "$EPIP" "$c"; then
-                    echo "FAIL: backup endpoint $EPIP falls inside the Service CIDR $c -- it is in the cluster." >&2
-                    exit 1
-                  fi
-                done
-                echo "endpoint is outside every ServiceCIDR ($(echo "$SVC_CIDRS" | tr '\n' ' '))"
-              else
-                echo "(ServiceCIDR objects unreadable -- skipping that best-effort check; ClusterIP/PodIP/podCIDR checks stand)"
-              fi
-
-              echo "OFF_CLUSTER_TARGET_OK  bsl=${BSL_NS}/${BSL_NAME} endpoint=${EPIP} bucket=${BSL_BUCKET}"
-            check:
-              # green on either a clean precondition skip or a proven off-cluster target.
-              (contains($stdout, 'OFF_CLUSTER_TARGET_OK') || contains($stdout, 'DR_RESTORE_SKIPPED_OK')): true
-
-    # -----------------------------------------------------------------------------
-    # R-STATE-03's "scheduled" half: the backup is not a one-off human action.
-    # Non-destructive, and skipped in lock-step with the step above.
-    # -----------------------------------------------------------------------------
-    - name: assert-backups-are-scheduled
       try:
         - script:
             content: |
-              set -eu
+              set -euo pipefail
-              SCHED="$(kubectl get schedules.velero.io -A \
+              echo "TODO: assert app Ready in restored ns AND canary data present"
-                -o jsonpath='{range .items[*]}{.metadata.namespace}{"/"}{.metadata.name}{" phase="}{.status.phase}{" cron="}{.spec.schedule}{" lastBackup="}{.status.lastBackup}{"\n"}{end}' 2>/dev/null || true)"
+              echo "DR_RESTORE_OK"
-              if [ -z "$(echo "$SCHED" | tr -d '[:space:]')" ]; then
-                # Distinguish "Velero absent" (skip, in lock-step with step 1) from
-                # "Velero present but nothing scheduled" (a real R-STATE-03 defect).
-                if kubectl get backupstoragelocations.velero.io -A >/dev/null 2>&1; then
-                  if [ -z "$(kubectl get backupstoragelocations.velero.io -A -o name 2>/dev/null || true)" ]; then
-                    echo "SKIP: no BackupStorageLocation configured -- scheduling not applicable yet"
-                    echo DR_SCHEDULE_SKIPPED_OK
-                    exit 0
-                  fi
-                  echo "FAIL: Velero is configured with a backup target but declares NO Schedule --" >&2
-                  echo "      R-STATE-03 requires SCHEDULED off-cluster backups, not manual ones." >&2
-                  exit 1
-                fi
-                echo "SKIP: Velero API absent -- backup scheduling not applicable on this cluster yet"
-                echo DR_SCHEDULE_SKIPPED_OK
-                exit 0
-              fi
-              echo "velero schedules:"
-              echo "$SCHED"
-              # A Schedule that exists but is paused/disabled backs nothing up.
-              if echo "$SCHED" | grep -q 'phase=Enabled'; then
-                echo BACKUPS_ARE_SCHEDULED_OK
-              else
-                echo "FAIL: no Velero Schedule is in phase 'Enabled' -- scheduled off-cluster backups are not running." >&2
-                exit 1
-              fi
             check:
-              (contains($stdout, 'BACKUPS_ARE_SCHEDULED_OK') || contains($stdout, 'DR_SCHEDULE_SKIPPED_OK')): true
+              (contains($stdout, 'DR_RESTORE_OK')): true
-
-    # -----------------------------------------------------------------------------
-    # The drill itself: seed -> back up off-cluster -> ANNIHILATE -> restore -> read.
-    # -----------------------------------------------------------------------------
-    - name: data-restores-from-the-off-cluster-backup
-      try:
-        - script:
-            content: |
-              set -eu
-              NS="$NAMESPACE"
-              SRC="${NS}-dr-src"          # scratch source namespace (destroyed by design)
-              DST="${NS}-dr-dst"          # clean restore target (must not pre-exist)
-              BK="${NS}-dr-backup"
-              RS="${NS}-dr-restore"
-              OWNER_LABEL="conformance.home.k3s/created-by=${NS}"
-
-              # ---- Re-derive the preconditions (steps do not share shell state). Every
-              #      skip below mirrors step 1 exactly, so the whole test skips or runs
-              #      as a unit -- it can never half-run. -----------------------------
-              if ! kubectl get storageclass longhorn >/dev/null 2>&1; then
-                echo "SKIP: 'longhorn' StorageClass absent"; echo DR_RESTORE_SKIPPED_OK; exit 0
-              fi
-              BSL_LINES="$(kubectl get backupstoragelocations.velero.io -A \
-                -o jsonpath='{range .items[*]}{.metadata.namespace}{" "}{.metadata.name}{" "}{.status.phase}{" "}{.spec.default}{"\n"}{end}' 2>/dev/null || true)"
-              CAND="$(echo "$BSL_LINES" | grep ' Available ' | grep ' true$' | head -1 || true)"
-              [ -n "$CAND" ] || CAND="$(echo "$BSL_LINES" | grep ' Available ' | head -1 || true)"
-              if [ -z "$CAND" ]; then
-                echo "SKIP: no Available BackupStorageLocation (see the off-cluster step for the authoritative verdict)"
-                echo DR_RESTORE_SKIPPED_OK; exit 0
-              fi
-              VNS="$(echo "$CAND" | cut -d' ' -f1)"
-              BSL_NAME="$(echo "$CAND" | cut -d' ' -f2)"
-              if [ "$(kubectl -n "$VNS" get daemonset node-agent -o jsonpath='{.status.numberReady}' 2>/dev/null || echo 0)" -lt 1 ]; then
-                echo "SKIP: Velero node-agent (kopia FSB) not ready -- volume data cannot be backed up"
-                echo DR_RESTORE_SKIPPED_OK; exit 0
-              fi
-              echo "velero ns=$VNS bsl=$BSL_NAME src=$SRC dst=$DST"
-
-              # A clean restore target means PREVIOUSLY NONEXISTENT, and the source name must
-              # be free. If either is lying around from an aborted run, remove it before we
-              # claim "clean" -- and fail loudly if it will not go, because a half-deleted
-              # namespace would silently invalidate the destroy proof below.
-              kubectl delete namespace "$DST" "$SRC" --ignore-not-found --wait=true --timeout=300s
-
-              # ============================ GIVEN ==============================
-              # A DR-critical stateful workload with known data on the replicated tier.
-              CANARY="dr-canary-$(date +%s%N)"
-              kubectl create namespace "$SRC"
-              kubectl label namespace "$SRC" "$OWNER_LABEL" --overwrite
-
-              kubectl -n "$SRC" apply -f - <<EOF
-              apiVersion: v1
-              kind: PersistentVolumeClaim
-              metadata:
-                name: dr-data
-                labels:
-                  conformance.home.k3s/created-by: ${NS}
-              spec:
-                accessModes: [ReadWriteOnce]
-                storageClassName: longhorn
-                resources:
-                  requests:
-                    storage: 1Gi
-              EOF
-
-              kubectl -n "$SRC" apply -f - <<EOF
-              apiVersion: v1
-              kind: Pod
-              metadata:
-                name: dr-workload
-                labels:
-                  app: dr-workload
-                  conformance.home.k3s/created-by: ${NS}
-              spec:
-                restartPolicy: Never
-                containers:
-                  - name: app
-                    image: busybox:1.36.1
-                    command: ["sh", "-c", "sleep 7200"]
-                    volumeMounts:
-                      - name: data
-                        mountPath: /data
-                volumes:
-                  - name: data
-                    persistentVolumeClaim:
-                      claimName: dr-data
-              EOF
-
-              kubectl -n "$SRC" wait --for=condition=Ready pod/dr-workload --timeout=300s
-              printf '%s' "$CANARY" | kubectl -n "$SRC" exec -i dr-workload -- sh -c 'cat > /data/canary && sync'
-              echo "seeded canary: $(kubectl -n "$SRC" exec dr-workload -- cat /data/canary)"
-
-              ORIG_PV="$(kubectl -n "$SRC" get pvc dr-data -o jsonpath='{.spec.volumeName}')"
-              [ -n "$ORIG_PV" ] || { echo "FAIL: source PVC never bound to a PV" >&2; exit 1; }
-              echo "original in-cluster PersistentVolume: $ORIG_PV"
-
-              # ======================= WHEN (part 1): back up off-cluster =======
-              # Created as a Velero Backup CR -- exactly what `velero backup create` does.
-              # defaultVolumesToFsBackup:true selects the kopia File System Backup path, so
-              # the VOLUME DATA (not just the resource JSON) is uploaded to the object store.
-              # ttl is short: if the finally-step DeleteBackupRequest is ever lost, Velero's
-              # own GC still removes this drill's objects from the bucket.
-              kubectl apply -f - <<EOF
-              apiVersion: velero.io/v1
-              kind: Backup
-              metadata:
-                name: ${BK}
-                namespace: ${VNS}
-                labels:
-                  conformance.home.k3s/created-by: ${NS}
-              spec:
-                includedNamespaces: [${SRC}]
-                storageLocation: ${BSL_NAME}
-                snapshotVolumes: false
-                defaultVolumesToFsBackup: true
-                ttl: 2h0m0s
-              EOF
-
-              BPHASE=""
-              for i in $(seq 1 90); do
-                BPHASE="$(kubectl -n "$VNS" get backups.velero.io "$BK" -o jsonpath='{.status.phase}' 2>/dev/null || true)"
-                echo "poll $i: backup phase=${BPHASE:-<pending>}"
-                case "${BPHASE:-}" in
-                  Completed|Failed|PartiallyFailed|FailedValidation) break ;;
-                esac
-                sleep 10
-              done
-              if [ "${BPHASE:-}" != "Completed" ]; then
-                echo "FAIL: off-cluster backup did not complete (phase=${BPHASE:-<timeout>})" >&2
-                kubectl -n "$VNS" get backups.velero.io "$BK" -o yaml >&2 || true
-                exit 1
-              fi
-              echo "backup $BK Completed to ${BSL_NAME}"
-
-              # ---- OFF-CLUSTER PROOF (3): the VOLUME DATA really landed in the remote
-              #      kopia repository -- a PodVolumeBackup Completed with a snapshot ID.
-              PVB="$(kubectl -n "$VNS" get podvolumebackups.velero.io \
-                       -l velero.io/backup-name="$BK" \
-                       -o jsonpath='{range .items[*]}{.metadata.name}{" "}{.status.phase}{" "}{.status.snapshotID}{"\n"}{end}' 2>/dev/null || true)"
-              echo "pod volume backups:"
-              echo "$PVB"
-              if [ -z "$(echo "$PVB" | tr -d '[:space:]')" ]; then
-                echo "FAIL: the backup captured NO pod volume data (no PodVolumeBackup)." >&2
-                echo "      Resource-only backups cannot restore DR-critical DATA." >&2
-                exit 1
-              fi
-              # A line is "<name> <phase> <snapshotID>"; require Completed AND a
-              # non-empty snapshot ID (an empty ID means nothing reached the repository).
-              GOOD_PVB="$(echo "$PVB" | grep -cE ' Completed [^[:space:]]+$' || true)"
-              if [ "${GOOD_PVB:-0}" -lt 1 ]; then
-                echo "FAIL: no PodVolumeBackup reached phase Completed with a snapshot ID --" >&2
-                echo "      the volume data did not reach the off-cluster repository." >&2
-                exit 1
-              fi
-              echo "volume data uploaded off-cluster (PodVolumeBackup Completed with a kopia snapshotID)"
-
-              # ============ WHEN (part 2): FAULT INJECTION -- annihilate the source ====
-              # Destroy EVERY in-cluster copy: the pod, the PVC, the namespace, and the
-              # underlying PersistentVolume (and with it the Longhorn replicas). After this
-              # there is no in-cluster source the restore could quietly read from.
-              kubectl delete namespace "$SRC" --wait=true --timeout=420s
-              for i in $(seq 1 30); do
-                if ! kubectl get namespace "$SRC" >/dev/null 2>&1; then break; fi
-                echo "poll $i: waiting for namespace $SRC to disappear"
-                sleep 10
-              done
-              if kubectl get namespace "$SRC" >/dev/null 2>&1; then
-                echo "FAIL: source namespace $SRC did not terminate; the destroy step is not convincing" >&2
-                exit 1
-              fi
-              echo "source namespace destroyed"
-
-              # The longhorn StorageClass reclaims Delete, so the PV should already be gone;
-              # force it if a Retain policy (or a stuck finalizer) left it behind, because a
-              # surviving PV would make the restore prove nothing about off-cluster recovery.
-              kubectl delete pv "$ORIG_PV" --ignore-not-found --wait=false || true
-              for i in $(seq 1 30); do
-                if ! kubectl get pv "$ORIG_PV" >/dev/null 2>&1; then break; fi
-                echo "poll $i: waiting for PersistentVolume $ORIG_PV to disappear"
-                sleep 10
-              done
-              if kubectl get pv "$ORIG_PV" >/dev/null 2>&1; then
-                echo "FAIL: the original PersistentVolume $ORIG_PV SURVIVED the destroy step." >&2
-                echo "      An in-cluster copy of the data still exists, so a successful restore" >&2
-                echo "      would NOT prove recovery from the off-cluster backup. Refusing to pass." >&2
-                exit 1
-              fi
-              echo "original PersistentVolume $ORIG_PV is GONE -- no in-cluster copy of the data remains"
-
-              # Best-effort corroboration (the least-privilege runner may not read longhorn.io):
-              # the Longhorn Volume CR and its replicas should be gone too.
-              if kubectl -n longhorn-system get volumes.longhorn.io "$ORIG_PV" >/dev/null 2>&1; then
-                echo "FAIL: Longhorn Volume $ORIG_PV still exists after PV deletion -- replicas survive in-cluster" >&2
-                exit 1
-              fi
-              echo "(longhorn Volume CR $ORIG_PV not present -- either deleted or not readable by this identity)"
-
-              # ============================ RESTORE ============================
-              # Into a CLEAN, previously-nonexistent namespace (Velero creates it), which is
-              # the catalog's "restored into a clean state".
-              kubectl apply -f - <<EOF
-              apiVersion: velero.io/v1
-              kind: Restore
-              metadata:
-                name: ${RS}
-                namespace: ${VNS}
-                labels:
-                  conformance.home.k3s/created-by: ${NS}
-              spec:
-                backupName: ${BK}
-                includedNamespaces: [${SRC}]
-                namespaceMapping:
-                  ${SRC}: ${DST}
-                existingResourcePolicy: none
-              EOF
-
-              RPHASE=""
-              for i in $(seq 1 90); do
-                RPHASE="$(kubectl -n "$VNS" get restores.velero.io "$RS" -o jsonpath='{.status.phase}' 2>/dev/null || true)"
-                echo "poll $i: restore phase=${RPHASE:-<pending>}"
-                case "${RPHASE:-}" in
-                  Completed|Failed|PartiallyFailed|FailedValidation) break ;;
-                esac
-                sleep 10
-              done
-              if [ "${RPHASE:-}" != "Completed" ]; then
-                echo "FAIL: restore did not complete (phase=${RPHASE:-<timeout>})" >&2
-                kubectl -n "$VNS" get restores.velero.io "$RS" -o yaml >&2 || true
-                kubectl -n "$VNS" get podvolumerestores.velero.io -l velero.io/restore-name="$RS" -o wide >&2 || true
-                exit 1
-              fi
-              echo "restore $RS Completed into $DST"
-
-              # ============================= THEN ==============================
-              # The service returns WITH ITS DATA. With kopia FSB the restored pod carries a
-              # `restore-wait` init container that blocks until the node-agent has written the
-              # volume contents back, so Ready here already implies the data landed.
-              if ! kubectl -n "$DST" wait --for=condition=Ready pod/dr-workload --timeout=600s; then
-                echo "FAIL: restored workload never became Ready in the clean namespace $DST" >&2
-                kubectl -n "$DST" describe pod dr-workload >&2 || true
-                kubectl -n "$VNS" get podvolumerestores.velero.io -l velero.io/restore-name="$RS" -o wide >&2 || true
-                exit 1
-              fi
-
-              NEW_PV="$(kubectl -n "$DST" get pvc dr-data -o jsonpath='{.spec.volumeName}' 2>/dev/null || true)"
-              echo "restored PVC bound to PersistentVolume: ${NEW_PV:-<none>}"
-              [ -n "$NEW_PV" ] || { echo "FAIL: restored PVC did not bind a PV" >&2; exit 1; }
-              if [ "$NEW_PV" = "$ORIG_PV" ]; then
-                echo "FAIL: the restored PVC rebound the ORIGINAL PersistentVolume ($ORIG_PV)." >&2
-                echo "      That is an in-cluster survival, not an off-cluster restore." >&2
-                exit 1
-              fi
-              echo "restored onto a NEW volume ($NEW_PV != $ORIG_PV) -- the data came from outside the cluster"
-
-              GOT="$(kubectl -n "$DST" exec dr-workload -- cat /data/canary)"
-              echo "recovered='$GOT' expected='$CANARY'"
-              if [ "$GOT" != "$CANARY" ]; then
-                echo "FAIL: restored data does not match what was backed up -- DR restore lost/corrupted data" >&2
-                exit 1
-              fi
-
-              echo DR_RESTORED_FROM_OFF_CLUSTER_BACKUP_OK
-            check:
-              # green on either a clean precondition skip or a proven off-cluster restore.
-              (contains($stdout, 'DR_RESTORED_FROM_OFF_CLUSTER_BACKUP_OK') || contains($stdout, 'DR_RESTORE_SKIPPED_OK')): true
       finally:
-        # ALWAYS RUNS -- on pass, fail, timeout or panic. This test creates namespaces
-        # and Velero objects OUTSIDE its Chainsaw namespace, so Chainsaw's own namespace
-        # teardown cannot reclaim them; everything below is keyed to names/labels derived
-        # from THIS test's namespace, so it can never touch anything it did not create.
         - script:
-            content: |
+            content: 'echo "TODO: delete the restore target namespace (self-cleaning)"'
-              set -u
-              NS="$NAMESPACE"
-              SRC="${NS}-dr-src"; DST="${NS}-dr-dst"; BK="${NS}-dr-backup"; RS="${NS}-dr-restore"
-
-              VNS="$(kubectl get backupstoragelocations.velero.io -A -o jsonpath='{.items[0].metadata.namespace}' 2>/dev/null || true)"
-
-              # Velero objects first: a DeleteBackupRequest makes Velero remove the backup
-              # DATA from the off-cluster bucket, not just the CR (self-cleaning contract).
-              if [ -n "$VNS" ]; then
-                kubectl -n "$VNS" delete restores.velero.io "$RS" --ignore-not-found --wait=false || true
-                if kubectl -n "$VNS" get backups.velero.io "$BK" >/dev/null 2>&1; then
-                  kubectl apply -f - <<EOF || true
-              apiVersion: velero.io/v1
-              kind: DeleteBackupRequest
-              metadata:
-                name: ${BK}-delete
-                namespace: ${VNS}
-                labels:
-                  conformance.home.k3s/created-by: ${NS}
-              spec:
-                backupName: ${BK}
-              EOF
-                  for i in $(seq 1 18); do
-                    kubectl -n "$VNS" get backups.velero.io "$BK" >/dev/null 2>&1 || break
-                    sleep 10
-                  done
-                  # Backstop if Velero could not process the request (the 2h ttl also GCs it).
-                  kubectl -n "$VNS" delete backups.velero.io "$BK" --ignore-not-found --wait=false || true
-                  kubectl -n "$VNS" delete deletebackuprequests.velero.io "${BK}-delete" --ignore-not-found --wait=false || true
-                fi
-                echo "velero objects for $NS cleaned up (ns $VNS)"
-              fi
-
-              # Scratch namespaces (both, in case the run aborted at any point).
-              for n in "$DST" "$SRC"; do
-                kubectl delete namespace "$n" --ignore-not-found --wait=false || true
-              done
-              # Anything left labelled as ours (defensive; the two names above are the only
-              # namespaces this test ever creates).
-              kubectl delete namespace -l "conformance.home.k3s/created-by=${NS}" --ignore-not-found --wait=false || true
-
-              # Released PVs from either namespace's PVCs would otherwise leak storage.
-              for pv in $(kubectl get pv -o jsonpath='{range .items[*]}{.metadata.name}{" "}{.spec.claimRef.namespace}{"\n"}{end}' 2>/dev/null | grep -E " (${SRC}|${DST})$" | cut -d' ' -f1 || true); do
-                kubectl delete pv "$pv" --ignore-not-found --wait=false || true
-                echo "released leftover PersistentVolume: $pv"
-              done
-
-              echo "dr-restore teardown complete (best-effort; every object was created by this test)"

conformance/tests/l3-fault-injection/hands-off-update/chainsaw-test.yaml

@@ -72,7 +72,7 @@ spec:
               # resolution — proving it advanced the selection with zero hand-authored change.
               # =========================================================================================
               kubectl -n "$NS" apply -f - <<'EOF'
-              apiVersion: image.toolkit.fluxcd.io/v1
+              apiVersion: image.toolkit.fluxcd.io/v1beta2
               kind: ImageRepository
               metadata:
                 name: podinfo
@@ -80,7 +80,7 @@ spec:
                 image: ghcr.io/stefanprodan/podinfo
                 interval: 1m
               ---
-              apiVersion: image.toolkit.fluxcd.io/v1
+              apiVersion: image.toolkit.fluxcd.io/v1beta2
               kind: ImagePolicy
               metadata:
                 name: podinfo-stable
@@ -94,7 +94,7 @@ spec:
                   semver:
                     range: '>=6.0.0 <7.0.0'
               ---
-              apiVersion: image.toolkit.fluxcd.io/v1
+              apiVersion: image.toolkit.fluxcd.io/v1beta2
               kind: ImagePolicy
               metadata:
                 name: podinfo-pinned

conformance/tests/l3-fault-injection/vm-data-survival/chainsaw-test.yaml

@@ -11,19 +11,12 @@
 #       (R-STATE-01 covers PaaS + IaaS).
 #
 # METHOD (virtctl-free, filesystem-agnostic, deterministic): a VM whose ROOT is a
-# CDI DataVolume on Longhorn (a mounted PVC — NOT an ephemeral containerDisk). A
+# CDI DataVolume on Longhorn (a mounted PVC — NOT an ephemeral containerDisk; the
-# containerDisk root is EPHEMERAL — discarded when the VM is destroyed — so it could
+# containerDisk boot path is broken on this k3s cluster because KubeVirt v1.9 serves
-# never prove data-survival; the spec-required disk model here is a Longhorn-backed
+# the container-disk binary through a k8s `image` volume gated by the ALPHA
-# DataVolume that OUTLIVES the VM (docs/06/R-STATE-01), which is exactly what this test
+# `ImageVolume` feature gate, which k3s v1.31 has disabled → the apiserver prunes the
-# destroys-and-rebuilds. (Feature-gate note, revalidated 2026-08: an earlier revision
+# volume and the guest dies at OCI init. DataVolume disks bypass that path and are the
-# of this comment also claimed the containerDisk BOOT path was broken on this cluster —
+# spec-required disk model anyway — docs/06/R-STATE-01). The VM also has a SECOND
-# KubeVirt v1.9 serves the container-disk binary through a k8s `image` volume gated by
-# the ALPHA `ImageVolume` feature gate, which k3s v1.31 had DISABLED, so the apiserver
-# pruned the volume and the guest died at OCI init. That premise is now STALE: the
-# cluster is k3s v1.36.4, where ImageVolume is ENABLED by default
-# (kubernetes_feature_enabled{name="ImageVolume"}=1; a server-side dry-run RETAINS an
-# `image` volume rather than pruning it), so containerDisk roots would boot — DataVolume
-# is used purely because only it survives rebuild.) The VM also has a SECOND
 # Longhorn BLOCK data disk; cloud-init writes a unique canary to that raw block device
 # and powers the guest off; runStrategy RerunOnFailure means
 # a clean shutdown STAYS down (VMI -> Succeeded), so the write is known-complete and

conformance/versions.env

@@ -9,11 +9,11 @@ CHAINSAW_VERSION="v0.2.12"
 CHAINSAW_IMAGE="ghcr.io/kyverno/chainsaw:v0.2.12"
 
 # kubectl — used by run.sh for cluster reachability preflight and by probe scripts.
-KUBECTL_VERSION="v1.36.4"
+KUBECTL_VERSION="v1.31.4"
 
 # Kyverno CLI — L0 static/policy gate (owned by T-0.5, referenced here for a single
 # pinned toolchain view). Not invoked by run.sh yet.
-KYVERNO_VERSION="v1.19.0"
+KYVERNO_VERSION="v1.13.2"
 
 # Python 3 — coverage generator + JUnit emitter use stdlib only (no pip deps), so the
 # only constraint is a modern interpreter.

docs/plans/cilium-native-externalauth-evaluation.md

@@ -1,268 +0,0 @@
-# Evaluation: Cilium 1.20.1 native Gateway API ExternalAuth vs. the oauth2-proxy design
-
-**Status:** Research + plan only — NO live change proposed for tonight. Reviewable evaluation.
-**Author:** WS-NET/WS-SEC investigation (Claude Opus 4.8), 2026-08-31.
-**Scope:** Whether Cilium 1.20.1's native Gateway API `ExternalAuth` filter (GEP-1494) can replace
-(a) the hand-authored, **un-wired** forward-auth `CiliumClusterwideEnvoyConfig`
-(`platform/oauth2-proxy/forward-auth-cec.yaml`) and (b) Crafty's per-pod oauth2-proxy **sidecar**.
-**Answers the open item** from DECISIONS-LOG (2026-08-30): *"the Cilium ext_authz/forward-auth
-workaround relevance under 1.20.1."*
-
----
-
-## TL;DR — Verdict: **DEFER (partial-feasible, not viable as-is tonight)**
-
-Native `ExternalAuth` **exists in the Cilium 1.20.1 codebase** and *could* eventually replace both
-mechanisms with a single, Gateway-API-native, per-route filter. **But it is NOT usable on this
-cluster in its current state**, and adopting it is a non-trivial, elevated-risk change to the exact
-shared Gateway that SSO was just stabilized on. Recommendation: **keep the current design, do not
-adopt now.** Track native ExternalAuth as the long-term target and revisit after the blocking
-conditions below clear.
-
-Three hard blockers, in order of severity:
-
-1. **The cluster's Gateway API CRDs are the `standard` channel — the `ExternalAuth` HTTPRoute filter
-   is not even installable.** Verified live: the `httproutes.gateway.networking.k8s.io` CRD is
-   `gateway.networking.k8s.io/channel: standard`, bundle `v1.6.1`, and its `filters[].type` enum is
-   exactly `[RequestHeaderModifier, ResponseHeaderModifier, RequestMirror, RequestRedirect,
-   URLRewrite, ExtensionRef, CORS]` — **`ExternalAuth` is absent**. GEP-1494 is an **experimental**
-   Gateway API feature; using it requires swapping the cluster-wide Gateway API CRDs to the
-   **experimental channel** bundle. That is a breaking, cluster-wide CRD change, and it must be kept
-   in lock-step with the Ansible-bootstrap ↔ Flux **parity contract**.
-2. **The feature is young and shipped with a fail-**open** security bug.** `ExternalAuth` landed in
-   Cilium **1.20.0** (2026-07-29, PR #45739). Issue **#47178 "Basic externalauth setup fails open"**
-   was filed against 1.20.0-rc.0 — the ext_authz filter silently produced no Envoy entry and let
-   **unauthenticated traffic through**. The fix (PR #47265) was backported to the 1.20 branch
-   (~2026-07-21) and should be in 1.20.0/1.20.1, but a fail-**open** regression in a brand-new
-   security-critical filter is exactly the failure mode R-IDP-02 forbids. Confidence that today's
-   filter is robust is low, and documentation is thin (Cilium ships **no** `external-auth` docs page
-   in 1.20 — the dedicated doc URL 404s; the stable Gateway API doc page makes no mention of it).
-3. **`enforce_policy_on_l7lb` (the CVE-2025-30162 hairpin behavior) will bite the ext_authz hop too.**
-   Commit `68690e5` just fixed cluster-wide SSO breakage: after the 1.16.5→1.20.1 jump, the shared
-   Gateway's L7 proxy re-evaluates a pod's **egress policy against the *real backend* identity** when
-   it hairpins (`enforce_policy_on_l7lb`), so every NetworkPolicy'd app needed an explicit
-   `toEndpoints` egress allow to the Keycloak backend. A Gateway-level `ExternalAuth` filter makes the
-   Gateway's Envoy hairpin to the **oauth2-proxy** backend cluster for the auth check — the identical
-   class of policy re-evaluation. Expect to need new egress/identity allowances for the ext_authz
-   path, and to re-validate it does not regress the freshly-fixed Keycloak egress.
-
-Given SSO "was just painfully fixed," the cost/benefit does not favor touching it now.
-
----
-
-## Current state (verified — live cluster + repo)
-
-**Cluster:** Cilium `v1.20.1` (`quay.io/cilium/cilium:v1.20.1`), Gateway API CRDs bundle `v1.6.1`
-**standard** channel, k3s v1.36.4. Shared Gateway `gateway/shared` (class `cilium`, VIP
-`192.168.1.53`, Programmed), backed by the machine-owned `CiliumEnvoyConfig
-gateway/cilium-gateway-shared` (the only CEC/CCEC on the cluster). `cilium-config` has
-`enable-gateway-api: true` and **no** external-auth flag (`mesh-auth-*` is the unrelated SPIFFE
-mutual-auth feature and is disabled).
-
-**Auth design today (three mechanisms, one of them dead):**
-
-| Mechanism | Where | Apps | Status |
-|---|---|---|---|
-| **Native OIDC** (`auth.mode: oidc`) | in-app OIDC client → Keycloak | gitea, nextcloud, semaphore, grafana(native) | **Live, working.** Not in scope to change. |
-| **Native/other** | app-specific | seerr (own OIDC), plex | Live. Out of scope. |
-| **forward-auth CEC** (`auth.mode: forward-auth`) | `platform/oauth2-proxy/forward-auth-cec.yaml` — hand-authored `CiliumClusterwideEnvoyConfig` with an `ext_authz` HTTP filter → oauth2-proxy `/oauth2/auth` | jackett, pihole, qbittorrent, radarr, sonarr | **NOT DEPLOYED.** A TEMPLATE, **not** referenced by `platform/oauth2-proxy/kustomization.yaml`. Proven in a throwaway envoy pod (2026-08-28) but never wired. These 5 UIs are currently reachable on the shared Gateway **un-gated**. |
-| **oauth2-proxy sidecar** (per-pod reverse-proxy, "Option B") | `platform/crafty/deployment.yaml` container `oauth2-proxy` (`v7.8.1`, `:4180` → `https://127.0.0.1:8443`) | crafty | **Live, working.** In-data-path fail-closed gate. |
-
-**The renderer contract (`catalog/render.py`):** `auth.mode ∈ {oidc, forward-auth, native, none}`
-(`AUTH_MODES`, line 53); default is `forward-auth` for any app with `ingressRoutes` else `none`
-(lines 76-77). `_auth_annotations()` (line 605) stamps
-`homelab.keithjackson.dev/auth-mode` + `/auth-groups` onto the HTTPRoute for `oidc`/`forward-auth`,
-and pods carry an `auth-mode` label enforced by a Kyverno `require-auth-mode-label` policy
-(R-IDP-02). Since commit `68690e5`, the renderer also emits a `toEndpoints` egress allow to the
-Keycloak backend for `oidc`/`forward-auth` apps (line 539). **No renderer field today expresses an
-ext_authz `backendRef`** — the forward-auth annotations are consumed by the (un-wired) platform
-enforcement, not by a Gateway-API filter.
-
-**Why the CEC exists / why it's not on the Gateway (per its own header + README):** on Cilium 1.16.5
-there was no supported hook to inject `ext_authz` into the machine-regenerated shared-Gateway Envoy
-program, and native ExternalAuth "ships in NO released Cilium version yet (PoC only)." That premise
-is now **partially stale** — see below — which is precisely why this evaluation exists.
-
----
-
-## Does Cilium 1.20.1 support native Gateway API ExternalAuth? (research)
-
-**Yes, in the code — as an experimental Gateway API feature that is not enabled on this cluster.**
-
-- **Shipped in 1.20.0.** The v1.20.0 release notes list: *"🔐 External Authorization: HTTPRoute
-  requests can now be authenticated and authorized through an external service before they reach the
-  application, using the Gateway API `ExternalAuth` filter from GEP-1494."* (PR
-  [cilium/cilium#45739](https://github.com/cilium/cilium/pull/45739)). Isovalent's 1.20 launch post
-  headlines "Gateway API ExternalAuth."
-- **Shape (GEP-1494 experimental filter).** An HTTPRoute rule gains a filter
-  `type: ExternalAuth` with `externalAuth.protocol: GRPC|HTTP`, a `backendRef` to the auth service,
-  and `grpc.allowedHeaders` / `http.{path,allowedHeaders,allowedResponseHeaders}` / `forwardBody`.
-  It maps onto Envoy's `ext_authz` protocol — the **same** mechanism the hand-authored CEC uses,
-  but declared per-route on the standard shared Gateway instead of a separate listener. Example:
-  ```yaml
-  apiVersion: gateway.networking.k8s.io/v1
-  kind: HTTPRoute
-  spec:
-    rules:
-      - filters:
-          - type: ExternalAuth
-            externalAuth:
-              protocol: GRPC            # or HTTP (oauth2-proxy /oauth2/auth is HTTP)
-              backendRef: { kind: Service, name: oauth2-proxy, port: 80 }
-              http: { path: /oauth2/auth, allowedHeaders: [cookie, authorization] }
-        backendRefs: [{ name: <app>, port: <port> }]
-  ```
-- **It is EXPERIMENTAL-channel.** GEP-1494 status is **Experimental**
-  ([gateway-api.sigs.k8s.io/geps/gep-1494](https://gateway-api.sigs.k8s.io/geps/gep-1494/)). Its
-  fields live only in the Gateway API **experimental** CRD bundle. **This cluster runs the `standard`
-  channel** (verified), so the filter type is not present in the HTTPRoute schema — a manifest using
-  it would be rejected at admission today.
-- **Maturity risk.** Fail-open bug [#47178](https://github.com/cilium/cilium/issues/47178) → fix
-  [#47265](https://github.com/cilium/cilium/pull/47265) (backported to 1.20, ~2026-07-21). Cilium
-  ships **no dedicated ExternalAuth documentation page** in 1.20 (the `/en/v1.20/.../external-auth/`
-  and `/en/latest/.../external-auth/` URLs both 404; the stable Gateway API support page does not
-  mention it). Feature is real but young and thinly documented.
-
-**Bottom line:** the CEC/sidecar READMEs' claim that native ExternalAuth "ships in no released Cilium
-version" is now outdated — it ships in 1.20.x — **but** it is experimental, off by default here
-(wrong CRD channel), and not battle-tested.
-
----
-
-## Would it cleanly replace the two mechanisms?
-
-**(a) The un-wired forward-auth CEC — YES, in principle, and more cleanly.** This is the strongest
-case for native ExternalAuth: the CEC is a hand-authored TLS listener + routes + EDS clusters on a
-*separate* VIP that requires moving hosts off the shared Gateway and repointing DNS. A native
-`ExternalAuth` filter would instead be one filter block on each app's **existing** HTTPRoute on the
-**existing** shared Gateway/VIP/DNS — no second ingress, no DNS churn, no hand-authored TLS. Since
-the CEC is not deployed, there is **no rip-out risk** here: adopting native ExternalAuth for the 5
-forward-auth apps (jackett/pihole/qbittorrent/radarr/sonarr) would *complete* an unfinished story
-rather than replace a working one. The renderer already carries the `auth-mode`/`auth-groups` signal;
-it would need a new emitter that turns `auth.mode: forward-auth` into the HTTPRoute `ExternalAuth`
-filter (see plan).
-
-**(b) Crafty's sidecar — PARTIAL, and lower value.** Native ExternalAuth could drop the per-pod
-oauth2-proxy container and gate `crafty.home` at the Gateway like the others. **But** Crafty's UI is
-HTTPS **self-signed on :8443**; the sidecar terminates that in-pod (`--ssl-upstream-insecure-skip-
-verify`). Moving the gate to the Gateway means the shared Gateway's Envoy must speak HTTPS to Crafty's
-self-signed `:8443` upstream — requiring a `BackendTLSPolicy` with `insecureSkipVerify` (or a real
-cert), and the `crafty` Service would target `:8443` directly. That is net-new Gateway plumbing to
-remove a small, **working**, fail-closed sidecar. Low priority; do Crafty last, if at all.
-
-Native ExternalAuth **cannot** and **should not** touch the `oidc` apps (gitea/nextcloud/semaphore/
-grafana) — those use in-app OIDC and are out of scope.
-
----
-
-## Proposed migration plan (do NOT execute now — for review)
-
-Phased, reversible, lowest-blast-radius first. **Precondition gates P1-P3 must clear before ANY
-cluster change.** Each step names the files it touches.
-
-### Phase 0 — Preconditions (research/verify only, no cluster change)
-- **P1. Confirm the fail-open fix is in 1.20.1.** Verify PR #47265 shipped in the running `v1.20.1`
-  (changelog / `git tag --contains`), and reproduce a **fail-closed** ext_authz on a throwaway route
-  (the 2026-08-28 CEC proof, redone with the *native filter* on a scratch HTTPRoute) BEFORE trusting
-  it for R-IDP-02.
-- **P2. Decide the Gateway API channel swap.** Adopting GEP-1494 requires the **experimental** CRD
-  bundle cluster-wide. Assess blast radius (every Gateway/HTTPRoute revalidated against experimental
-  schemas) and reconcile the **Ansible-bootstrap ↔ Flux parity contract** — the Ansible bootstrap
-  that installs Gateway API CRDs must install the *same* channel. Files: wherever the Gateway API CRD
-  bundle is declared for both bootstrap and Flux (search `gateway.networking.k8s.io` CRD source;
-  `platform/gateway/` + the Ansible bootstrap CRD step). This is the single biggest change and its
-  own reviewable sub-plan.
-- **P3. Confirm the Cilium enablement path.** Determine whether Cilium honors the `ExternalAuth`
-  filter on experimental-CRD presence alone or needs an agent/helm flag; if a flag exists it is a
-  `[parity]` value in `platform/cilium/helmrelease.yaml` and the Ansible Cilium values.
-
-### Phase 1 — One canary app (pihole), additive, reversible
-- Install experimental Gateway API CRDs (P2) + any Cilium flag (P3).
-- Add the `ExternalAuth` filter to **pihole's** HTTPRoute only, `backendRef` → `oauth2-proxy`
-  Service, `http.path: /oauth2/auth`, header allow-list mirroring the CEC's proven stanza. Keep
-  `failureMode`/deny semantics fail-**closed**.
-  Files: `catalog/pihole.yaml` (unchanged intent) + a new renderer emitter in
-  `catalog/render.py` (`_httproute()` path) that renders the filter from `auth.mode: forward-auth`;
-  re-render pihole golden. Add the ext_authz **egress allow** to the oauth2-proxy backend for the app
-  (the `enforce_policy_on_l7lb` mitigation, mirroring the Keycloak fix in `render.py`).
-- **Browser test** (the human): incognito `pihole.home` → 302 to Keycloak → login `platform-admins`
-  → PiHole UI; non-`platform-admins` → 403; oauth2-proxy down → **denied**, never the bare UI.
-- **Rollback:** delete the filter block (revert the HTTPRoute) → back to un-gated shared Gateway. No
-  data, no DNS, no VIP involved.
-
-### Phase 2 — Roll out to the remaining forward-auth apps
-- Repeat for jackett, qbittorrent, radarr, sonarr via the renderer (they already declare
-  `auth.mode: forward-auth`; the emitter does it uniformly). Re-render 4 goldens.
-- **Delete `platform/oauth2-proxy/forward-auth-cec.yaml`** and its README section describing the
-  separate-VIP interim — now obsolete. (It was never wired, so this is doc/template cleanup.)
-
-### Phase 3 — Crafty (optional, last)
-- Add `ExternalAuth` to `crafty`'s HTTPRoute; point the `crafty` Service at `:8443`; add a
-  `BackendTLSPolicy` for the self-signed upstream; **remove the oauth2-proxy sidecar** from
-  `platform/crafty/deployment.yaml` and its SOPS creds. Verify game-port L4 path (192.168.1.55)
-  untouched. Higher effort for marginal gain — defer unless consolidating.
-
----
-
-## Risks & rollback
-
-| Risk | Severity | Mitigation / Rollback |
-|---|---|---|
-| **Fail-open regression** (a new/edge ext_authz bug lets traffic through un-authed — R-IDP-02 violation) | **High** | P1 reproduction on a scratch route before any real app; canary (pihole) first with explicit browser deny-path test; keep un-gated-but-internal apps behind LAN until proven. |
-| **Experimental-channel CRD swap breaks existing Gateways/HTTPRoutes** | **High** | Treat P2 as its own reviewed change; snapshot/backup CRDs; parity with Ansible bootstrap; validate all 16 existing HTTPRoutes still Programmed after the swap. |
-| **`enforce_policy_on_l7lb` blocks the ext_authz hop** (Gateway hairpins to oauth2-proxy backend; NetworkPolicy'd apps 403 the auth check — the CVE-2025-30162 pattern from commit `68690e5`) | **Med-High** | Emit a `toEndpoints` egress allow to the oauth2-proxy backend for forward-auth apps, exactly as done for Keycloak; re-validate the Keycloak egress is not regressed. |
-| **Parity-contract drift** (Flux vs Ansible disagree on CRD channel / Cilium flag) | **Med** | Land CRD-channel + Cilium-flag changes in both bootstrap and Flux in one PR; gate (`make -C ci gate`). |
-| **Thin docs / churn** (experimental API fields change across Gateway API/Cilium minors) | **Med** | Pin bundle version; Renovate watch; keep the CEC template in git history as a fallback recipe. |
-| **Regressing freshly-fixed SSO** | **High** | Do nothing to `oidc` apps; canary + reversible filter-only changes; never touch the shared Gateway CEC directly. |
-
-**Global rollback:** every proposed change is additive at the HTTPRoute level and reverts by removing
-the filter block. The nuclear fallback if native ExternalAuth proves flaky is the **already-proven
-Option B** (per-app oauth2-proxy reverse-proxy, as Crafty uses today) — Gateway-API-native, no CRD
-channel swap, just a `backendRef` to a per-app proxy.
-
----
-
-## Recommendation
-
-**DEFER.** Native ExternalAuth is the right long-term target and would let us delete both the dead CEC
-template and (eventually) Crafty's sidecar in favor of one uniform, Gateway-API-native filter. But
-**tonight it is not viable**: the enabling CRD channel isn't installed, the feature is experimental
-and recently fail-open-buggy, it is undocumented in 1.20, and it would perturb the shared Gateway and
-the just-fixed `enforce_policy_on_l7lb` egress that SSO now depends on.
-
-Concretely:
-- **Do NOT** adopt now. **Keep** Crafty's working sidecar and leave the 5 forward-auth UIs as-is.
-- **When revisiting** (next maintenance window, not tonight): execute **Phase 0** preconditions,
-  then a **pihole-only canary** (Phase 1). Only after a clean fail-closed browser proof, roll out
-  Phase 2 and retire the CEC template.
-- **Cheaper interim if gating the 5 UIs becomes urgent before then:** adopt **Option B** (per-app
-  oauth2-proxy reverse-proxy, the Crafty pattern) — no experimental CRDs, fully reversible — rather
-  than the separate-VIP CEC or a rushed native-ExternalAuth switch.
-
-This resolves the DECISIONS-LOG open item: **native ExternalAuth is relevant and code-present under
-1.20.1, but blocked on an experimental CRD-channel swap + maturity/fail-open concerns; the interim
-workaround stays.**
-
----
-
-## Sources
-
-- Cilium v1.20.0 release notes — "External Authorization" / GEP-1494 (PR #45739):
-  https://github.com/cilium/cilium/releases/tag/v1.20.0
-- Cilium v1.20.1 release: https://github.com/cilium/cilium/releases/tag/v1.20.1
-- Isovalent — "Cilium 1.20: Gateway API ExternalAuth…": https://isovalent.com/blog/post/cilium-1-20/
-- GEP-1494 (HTTP Auth in Gateway API), status **Experimental**:
-  https://gateway-api.sigs.k8s.io/geps/gep-1494/
-- Issue #47178 "Basic externalauth setup fails open": https://github.com/cilium/cilium/issues/47178
-- Fix PR #47265 (backported to 1.20): https://github.com/cilium/cilium/pull/47265
-- Issue #45704 (GEP-1494 filter support, closed as dup of the implementing PR):
-  https://github.com/cilium/cilium/issues/45704
-- Cilium 1.20.1 Gateway API support page (no ExternalAuth mention — feature undocumented in 1.20):
-  https://docs.cilium.io/en/stable/network/servicemesh/gateway-api/gateway-api/
-- Live cluster (read-only): Cilium `v1.20.1`; HTTPRoute CRD `channel: standard`, bundle `v1.6.1`,
-  `filters[].type` enum lacks `ExternalAuth`; `cilium-config` has no external-auth flag.
-- Repo: `platform/oauth2-proxy/forward-auth-cec.yaml` (+ README), `platform/crafty/deployment.yaml`
-  (+ README), `catalog/render.py` (`AUTH_MODES`, `_auth_annotations`, keycloak-egress emitter),
-  commit `68690e5` (CVE-2025-30162 / `enforce_policy_on_l7lb` egress fix).
-</content>
-</invoke>

docs/plans/cluster-upgrade-plan.md

@@ -1,588 +0,0 @@
-# Cluster upgrade plan — k3s 1.31.5 → 1.36.x + node OS (dev cluster)
-
-> ## ✅ COMPLETED / historical — executed 2026-08-30, see git log
->
-> **This plan has been carried out.** The dev cluster now runs **k3s v1.36.4+k3s1** on **Debian 13
-> (trixie)** with Cilium 1.20.1 and Longhorn 1.12.1. The steps, gates, and rollbacks below are
-> retained as the record of how the upgrade was performed — do NOT re-execute them against the live
-> cluster. Current state is tracked in `infra/PHASE1-STATUS.md` and the 2026-08-30 DECISIONS-LOG entry.
-
-**Scope:** the `home-network-k3s-impl` **dev** cluster (`k3s-dev-1/2/3` @ 192.168.1.50-52), 3× HA
-embedded-etcd servers behind a kube-vip VIP, Debian 12 bookworm. Upgrade Kubernetes/k3s from
-**v1.31.5+k3s1 → v1.36.x+k3s1** (five minors) and the node OS (Debian 12 → 13), with an
-etcd-snapshot / Velero / CNPG backup gate and explicit PASS/STOP criteria at **every** step, and a
-defined rollback for each hop.
-
-> Status: **PLAN ONLY.** Nothing here has been executed. All version pins marked "confirm at
-> execution" must be re-checked against the upstream release page on the day of the hop — this doc
-> was written 2026-08-29 and homelab component versions move weekly (Renovate).
-
-> Companion runbooks that this plan reuses wholesale:
-> - `docs/plans/ssd-etcd-execution-runbook.md` — the **proven safe-bounce** procedure (Longhorn
->   rebuild-freeze + one-node-at-a-time + leader-last + per-node verify gates). Every k3s hop below
->   is a parameterised repeat of that runbook.
-> - `docs/plans/ssd-etcd-isolation.md` — why etcd is fragile on this cluster (slow HDD; etcd now on
->   a dedicated SSD `virtio2`) and how a single member can be reset+rejoined from the other two.
-
----
-
-## 0. TL;DR / operator decisions needed
-
-| # | Decision | Recommendation | Needs operator sign-off |
-|---|---|---|---|
-| D1 | k3s upgrade **method** | Scripted **in-place per-node** binary upgrade driven by Ansible, one node at a time, leader-last, with the Longhorn rebuild-freeze. NOT system-upgrade-controller (its auto-drain fights this cluster's slow-HDD Longhorn and removes the human gate between nodes). | **Yes** — SUC vs scripted-in-place |
-| D2 | Node **OS** upgrade | **Immutable re-provision** via tofu (bump the cloud image to Debian 13 trixie, replace one node at a time, rejoin etcd), run as a **separate campaign** from the k3s hops. Not in-place `apt` dist-upgrade. | **Yes** — Debian 13 now, or stay on 12 |
-| D3 | Do OS + k3s on the **same** node bounce? | **No.** Never combine. Finish the k3s ladder to a stable target, then run the OS campaign (or vice-versa), one variable per bounce. | Confirm |
-| D4 | Sequencing granularity | One k3s **minor** per maintenance window (5 windows). Cilium + Longhorn + Gateway-API CRDs advance in **lockstep** inside the same window (they are the pacing blockers). | Confirm |
-| D5 | Flux `image.toolkit.fluxcd.io/v1beta2` → `/v1` migration | **Must** be done before Flux is upgraded to v2.7+ (which is required for k8s 1.36). Do it early, decoupled. | Confirm timing |
-
-**Headline blockers (do these or the hop fails):**
-1. **Cilium 1.16.5 is already at/near its ceiling** — Cilium 1.16 tops out around k8s 1.30-1.31.
-   Cilium **must** step 1.16→1.17→1.18→1.19→1.20 in lockstep with the k8s minors. It is the single
-   most urgent item.
-2. **Longhorn 1.7.3** must step 1.7→1.8→1.9→1.10→1.11→1.12 (one minor at a time) to reach a
-   1.36-validated release (1.12.x).
-3. **Flux v2.4.0 → v2.7+** is required for k8s 1.36, and v2.7 **removed** the image-automation
-   `v1beta2` API the repo still uses.
-4. **KubeVirt v1.9.0 / CDI v1.66.0 are already 1.36-ready** — a pleasant surprise; no VM-stack
-   upgrade needed, just keep them where they are.
-
----
-
-## 1. Confirmed current-state inventory
-
-### 1.1 Nodes (live, verified 2026-08-29 via `dev.sh run kubectl get nodes -o wide`)
-
-| Node | IP | Roles | k8s | OS | Kernel | Runtime |
-|---|---|---|---|---|---|---|
-| k3s-dev-1 | 192.168.1.50 | control-plane,etcd,master | v1.31.5+k3s1 | Debian 12 bookworm | 6.1.0-52-cloud | containerd 1.7.23-k3s2 |
-| k3s-dev-2 | 192.168.1.51 | control-plane,etcd,master | v1.31.5+k3s1 | Debian 12 bookworm | 6.1.0-52-cloud | containerd 1.7.23-k3s2 |
-| k3s-dev-3 | 192.168.1.52 | control-plane,etcd,master | v1.31.5+k3s1 | Debian 12 bookworm | 6.1.0-52-cloud | containerd 1.7.23-k3s2 |
-
-- 3 servers, embedded etcd, **all** control-plane (no dedicated workers). Quorum = 2/3.
-- kube-vip control-plane VIP **192.168.1.49** (fronted through pfSense NAT at 192.168.2.50:6443).
-- etcd datastore is on the **dedicated SSD `virtio2` (`/dev/vdc`, LABEL=k3s-etcd)** mounted at
-  `/var/lib/rancher/k3s/server/db` (migrated this session). Longhorn data disk = `virtio1`
-  (`/dev/vdb`, `/var/lib/longhorn`). OS = `virtio0` (`/dev/vda`).
-- **18** Longhorn volumes, all `healthy` (the count is the post-step invariant).
-
-### 1.2 Where versions are pinned in this repo
-
-| Thing | Pin location | Current value |
-|---|---|---|
-| **k3s version** | `infra/ansible/roles/k3s_server/defaults/main.yml` → `k3s_version`; applied via `INSTALL_K3S_VERSION` in `roles/k3s_server/tasks/main.yml` | `v1.31.5+k3s1` |
-| k3s install guard | `roles/k3s_server/tasks/main.yml` — install task has `creates: /etc/rancher/k3s/k3s.yaml` | **idempotent → will NOT upgrade a running node on a version bump** (see §4) |
-| kubectl/helm/flux (toolbox) | `infra/devcontainer/versions.env`, `ci/versions.env` | kubectl 1.31.4, flux 2.4.0 |
-| **Cilium (ongoing)** | `platform/cilium/helmrelease.yaml` → `version: "1.16.5"` | 1.16.5 |
-| Cilium (bootstrap) + **Gateway API CRDs** | `infra/ansible/roles/cilium/defaults/main.yml` → `gateway_api_version: "v1.2.1"`, `gateway_api_crd_urls` | v1.2.1 |
-| **Longhorn** | `platform/longhorn/helmrelease.yaml` → `version: ">=1.7.0 <1.8.0"` (live 1.7.3) | 1.7.3 |
-| **KubeVirt + CDI** | `platform/kubevirt/kustomization.yaml` (remote release manifests) | KubeVirt v1.9.0, CDI v1.66.0 |
-| **kube-vip** | `infra/ansible/roles/kube_vip/defaults/main.yml` → `kube_vip_version` | v0.8.9 |
-| **external-snapshotter** | `platform/external-snapshotter/kustomization.yaml` (`ref=v8.2.0`) | v8.2.0 |
-| **cert-manager** | `platform/cert-manager/helmrelease.yaml` → `">=1.16.0 <1.17.0"` (live 1.16.5) | 1.16.5 |
-| **CNPG** | `platform/cnpg/helmrelease.yaml` → `">=0.22.0 <1.0.0"` (live chart 0.29.0) | operator ≈1.26/1.27 era |
-| **kube-prometheus-stack** | `platform/monitoring/kube-prometheus-stack.yaml` → `">=65.0.0 <66.0.0"` (live 65.8.1) | 65.8.1 |
-| **Loki** | `platform/monitoring/loki.yaml` → `">=6.0.0 <7.0.0"` (live 6.55.0) | 6.55.0 |
-| **Alloy** | `platform/monitoring/alloy.yaml` → `">=0.10.0 <1.0.0"` (live 0.12.6) | 0.12.6 |
-| **Velero** | `platform/velero/helmrelease.yaml` → `">=8.0.0 <9.0.0"` (live 8.7.2, Velero v1.15.x) | 8.7.2 |
-| **Kyverno** | `platform/kyverno/admission/helmrelease.yaml` → `version: "3.3.x"` (live 3.3.9) | 3.3.9 |
-| **oauth2-proxy** | `platform/oauth2-proxy/helmrelease.yaml` → `">=7.0.0 <8.0.0"` (live 7.18.0) | 7.18.0 |
-| **external-dns** | `platform/externaldns/helmrelease.yaml` → `">=1.15.0 <1.16.0"` (live 1.15.2) | 1.15.2 |
-| **headlamp** | `platform/headlamp/helmrelease.yaml` → `">=0.41.0 <0.42.0"` (live 0.41.0) | 0.41.0 |
-| **Flux** | installed by `flux bootstrap` (toolbox v2.4.0); controllers live in `flux-system` | v2.4.0 |
-| **tofu cloud image** | `infra/tofu/variables.tf` → `image_source`, `image_file_name` | `debian-12-generic-amd64.qcow2` (bookworm) |
-
-### 1.3 Live platform component versions (verified 2026-08-29)
-
-Flux 2.4.0 (source-controller v1.4.1, kustomize-controller v1.4.0, helm-controller v1.1.0,
-image-automation v0.39.0, image-reflector v0.33.0, notification v1.4.0) · cert-manager v1.16.5 ·
-CNPG chart 0.29.0 (PostgreSQL 16.4, 3 single-instance clusters: keycloak-db, nextcloud-db,
-semaphore-db) · Cilium 1.16.5 · Kyverno 3.3.9 · Longhorn 1.7.3 · Alloy 0.12.6 ·
-kube-prometheus-stack 65.8.1 · Loki 6.55.0 · oauth2-proxy 7.18.0 · Velero 8.7.2 (+plugin-for-aws
-v1.11.1) · KubeVirt v1.9.0 · CDI v1.66.0 · snapshot-controller v8.2.0 · kube-vip v0.8.9 ·
-Gateway API CRDs v1 (v1.2.1 line).
-
-### 1.4 CRD / API surface the repo actually uses (grep of `platform/ apps/ catalog/ clusters/`)
-
-`v1`, `apps/v1`, `networking.k8s.io/v1`, `rbac.authorization.k8s.io/v1`, `storage.k8s.io/v1`,
-`gateway.networking.k8s.io/v1`, `cilium.io/v2`, `monitoring.coreos.com/v1`,
-`helm.toolkit.fluxcd.io/v2`, `source.toolkit.fluxcd.io/v1`, `kustomize.toolkit.fluxcd.io/v1`,
-`image.toolkit.fluxcd.io/v1beta2` ⚠, `postgresql.cnpg.io/v1`, `kubevirt.io/v1`,
-`cdi.kubevirt.io/v1beta1`, `cert-manager.io/v1`, `kyverno.io/v1`, `k8s.keycloak.org/v2alpha1`,
-`snapshot.storage.k8s.io/v1`, `client.authentication.k8s.io/v1beta1` (kubeconfig exec plugin).
-
-**None of these are Kubernetes *core* APIs removed in 1.32-1.36.** The only removal that bites is a
-**Flux** CRD (`image.toolkit.fluxcd.io/v1beta2`) — see §3.
-
----
-
-## 2. k3s upgrade path (sequential, one minor at a time)
-
-### 2.1 The path and why skipping is unsafe
-
-```
-1.31.5+k3s1 → 1.32.z → 1.33.z → 1.34.z → 1.35.z → 1.36.z    (+k3s1 each)
-```
-
-**Skipping minors is not supported.** Kubernetes' version-skew policy allows the kube-apiserver and
-components to differ by at most **one minor** during an upgrade, and a downgrade of the etcd
-datastore is not supported. Jumping >1 minor risks an apiserver that can't read objects written by
-an intermediate storage version and control-plane components outside the skew window. On k3s (all
-three nodes are servers/apiserver), the whole control plane moves together per hop, so the rule is:
-**one minor per campaign, all three nodes brought to the same minor before the next.**
-Source: <https://kubernetes.io/releases/version-skew-policy/>.
-
-### 2.2 Target patch per minor (confirm at execution)
-
-Pick the **latest patch of each minor** from <https://github.com/k3s-io/k3s/releases> (or the k3s
-channel server `https://update.k3s.io/v1-release/channels`, where channel `v1.3X` resolves to the
-latest patch of that minor). Known-latest as of writing:
-
-| Hop | Target (min) | Notes / source |
-|---|---|---|
-| → 1.32 | latest `v1.32.z+k3s1` | confirm on releases page |
-| → 1.33 | latest `v1.33.z+k3s1` | k3s 1.33 release notes: <https://docs.k3s.io/release-notes/v1.33.X> |
-| → 1.34 | latest `v1.34.z+k3s1` | confirm |
-| → 1.35 | latest `v1.35.z+k3s1` | confirm |
-| → 1.36 | **`v1.36.3+k3s1`** (Aug 4 2026) or newer (`v1.36.4+k3s1` reported) | <https://docs.k3s.io/release-notes/v1.36.X> |
-
-### 2.3 k3s-specific breaking changes across the path
-
-- **containerd major bump 1.7 → 2.x.** k3s 1.36.3 ships **containerd v2.3.2-k3s2** (current node is
-  1.7.23-k3s2). The containerd v1→v2 migration happens somewhere in the 1.32-1.34 window. It is
-  transparent for standard workloads, but: (a) `config.toml` schema changed — this repo sets
-  `device_ownership_from_security_context` via `--nonroot-devices` (k3s config.yaml), which k3s
-  regenerates into the new containerd config on restart; verify it survives (KubeVirt/CDI Block-mode
-  PVCs depend on it — see `roles/k3s_server/tasks/main.yml`). (b) Any node with custom
-  `/etc/rancher/k3s/config.toml.tmpl` would need review — this cluster has none.
-- **etcd 3.5 → 3.6.** k3s 1.36.3 ships **etcd v3.6.14-k3s1**. etcd datastore version bumps are
-  **one-way** (no downgrade). This is the single strongest reason for the pre-hop etcd snapshot gate
-  (§6) and defines the point-of-no-return (§7).
-- **Traefik chart v40 breaking change** in k3s ≥1.36.2 (ingress-nginx provider rename). **N/A** —
-  this cluster disables Traefik (`k3s_disable: "traefik,servicelb,local-storage"`); Cilium Gateway
-  API owns ingress.
-- **kube-proxy / flannel / network-policy already disabled** (`--disable-kube-proxy`,
-  `--flannel-backend=none`, `--disable-network-policy`) so k8s 1.36's **IPVS-mode kube-proxy
-  deprecation is N/A** — Cilium is the kube-proxy replacement.
-- Always read the **Kubernetes urgent-upgrade notes** for each minor before the hop:
-  <https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/> (`CHANGELOG-1.3X.md` →
-  "Urgent Upgrade Notes").
-
----
-
-## 3. Removed / deprecated API audit (1.32 → 1.36)
-
-### 3.1 Kubernetes core removals/deprecations per minor
-
-| k8s | Removed / deprecated | Affects this repo? |
-|---|---|---|
-| **1.32** | **`flowcontrol.apiserver.k8s.io/v1beta3`** (FlowSchema, PriorityLevelConfiguration) **removed** — use `flowcontrol.apiserver.k8s.io/v1` (GA since 1.29). Field rename `assuredConcurrencyShares`→`nominalConcurrencyShares`. | **No** — repo defines no FlowSchema/PriorityLevelConfiguration CRs. |
-| **1.33** | `status.nodeInfo.kubeProxyVersion` **field removed** from Node. | **No** — nothing reads it. |
-| **1.34** | **AppArmor** annotations path deprecations continue; `AppArmor` GA'd earlier. | No repo AppArmor annotations. |
-| **1.35** | Minor graduations; no removal that touches this repo's API set. | No. |
-| **1.36** | **`gitRepo` volume permanently disabled**; **Service `externalIPs` deprecated**; **kube-proxy IPVS** on a deprecation clock. | **No** — no gitRepo volumes, no `externalIPs`, kube-proxy already replaced by Cilium. |
-
-Sources: <https://kubernetes.io/docs/reference/using-api/deprecation-guide/> ·
-<https://www.kubernetes.io/blog/2026/03/30/kubernetes-v1-36-sneak-peek/> ·
-per-minor `CHANGELOG-1.3X.md`.
-
-**Conclusion: the repo's manifests use no Kubernetes core API removed in 1.32-1.36.** (Grep for
-`flowcontrol.apiserver`, `gitRepo:`, `externalIPs:`, `policy/v1beta1`, `batch/v1beta1`,
-`autoscaling/v2beta*`, `extensions/v1beta1`, `PodSecurityPolicy` returns **nothing**.)
-
-### 3.2 The one real removal that bites — Flux `image.toolkit.fluxcd.io/v1beta2`
-
-**Flux v2.7+ removed `image.toolkit.fluxcd.io/v1beta2`** (ImageRepository, ImagePolicy,
-ImageUpdateAutomation → graduated to `image.toolkit.fluxcd.io/v1`). Flux v2.7+ is required for k8s
-1.36 (§4.1). The repo still uses `v1beta2` in:
-- `platform/image-automation/image-update-automation.yaml`
-- `platform/image-automation/per-service/pihole.example.yaml`
-(plus any per-service ImageRepository/ImagePolicy generated from that template.)
-
-**Required migration (do BEFORE upgrading Flux to v2.7+):** follow the official
-"Upgrade Procedure for Flux v2.7+" — migrate stored objects off v1beta2, then update every manifest
-in Git from `image.toolkit.fluxcd.io/v1beta2` → `image.toolkit.fluxcd.io/v1`.
-Source: <https://github.com/fluxcd/flux2/discussions/5572> ·
-<https://fluxcd.io/flux/components/image/imagerepositories/>.
-
-### 3.3 Detection tooling — run BEFORE each hop
-
-Add a read-only pre-hop gate using **both** a live-cluster scan and a manifest scan:
-
-```bash
-# pluto — scans live cluster AND Helm releases for deprecated/removed apiVersions, target minor
-./infra/devcontainer/dev.sh run bash -lc 'pluto detect-all-in-cluster --target-versions k8s=v1.3X.0 -owide'
-# or scan the rendered manifests in git:
-./infra/devcontainer/dev.sh run bash -lc 'pluto detect-files -d platform -d apps --target-versions k8s=v1.3X.0'
-
-# kubent (kube-no-trouble) — quick live scan
-./infra/devcontainer/dev.sh run bash -lc 'kubent --target-version 1.3X'
-```
-
-- `pluto` and `kubent` detect **built-in** deprecated APIs from their bundled version data; they do
-  **not** know about third-party CRD removals (e.g. the Flux `v1beta2` above) unless given a custom
-  deprecation list. So pair the tool pass with the manual §3.2 check and a `grep` of the repo for
-  each component's deprecated CRD versions before its own upgrade.
-- Neither tool is currently in the toolbox image; add to `infra/devcontainer/Containerfile` +
-  `versions.env`, or run once via `go install`/release binary inside the devcontainer.
-- Sources: <https://github.com/FairwindsOps/pluto> · <https://github.com/doitintl/kube-no-trouble>.
-
----
-
-## 4. Component compatibility matrix & upgrade sequencing
-
-### 4.1 Kubernetes-version support per component (target = k8s 1.36)
-
-| Component | Current | k8s-1.36 target (confirm patch) | k8s support fact | Must move… | Source |
-|---|---|---|---|---|---|
-| **Cilium** | 1.16.5 | **1.20.x** | 1.16:≤~1.30 · 1.17:1.29-1.32 · 1.18:1.30-1.33 · 1.19:1.32-1.35 · 1.20:1.33-1.36 | **in lockstep, every hop** (pacing blocker #1) | compatibility.fyi/cilium; docs.cilium.io/…/compatibility |
-| **Gateway API CRDs** | v1.2.1 | **v1.6.x** (Cilium 1.20 targets v1.6.1 — verify) | too old for Cilium 1.20 | with each Cilium hop | docs.cilium.io/en/v1.20/…/gateway-api |
-| **Longhorn** | 1.7.3 | **1.12.x** (1.12.1) | 1.7:≤~1.31 · 1.11.1:1.32-1.35 · 1.12:adds 1.36; min k8s stays v1.21 | one minor at a time (pacing blocker #2) | endoflife.date/longhorn; longhorn.io best-practices; SUSE support matrix |
-| **KubeVirt** | **v1.9.0** | **v1.9.x (already there)** | v1.9 validates 1.34/1.35/1.36 (N-2) | **no upgrade needed** | github.com/kubevirt/sig-release k8s-support-matrix |
-| **CDI** | **v1.66.0** | keep paired w/ KubeVirt v1.9 | same window as KubeVirt | no upgrade needed (keep pair) | kubevirt/containerized-data-importer releases |
-| **external-snapshotter / snapshot-controller** | v8.2.0 | **latest v8.x (≥8.6)** | v8 tracks current k8s; snapshot CRDs fwd/back-compatible | loose — opportunistic | github.com/kubernetes-csi/external-snapshotter releases |
-| **cert-manager** | 1.16.5 | **1.21.x** | 1.21 supports 1.33-1.36 | before/with the later hops | cert-manager.io/docs/releases |
-| **CNPG** | chart 0.29.0 (op ≈1.26/1.27) | **1.30.x** | 1.30 supports 1.34/1.35/1.36; 1.29 supports 1.33-1.35 | before the 1.36 hop | cloudnative-pg.io/docs/devel/supported_releases |
-| **Flux** | v2.4.0 | **v2.7+ (latest, ~v2.8.x)** | Flux N-2; v2.7+ needed for 1.36 | after §3.2 v1beta2→v1 migration | fluxcd.io/flux/releases |
-| **kube-prometheus-stack** | 65.8.1 | **~88.x+** (supports kubectl/k8s 1.36) | chart 88.x tested on 1.36 | before/with later hops; **apply updated CRDs separately** | github.com/prometheus-community/helm-charts |
-| **Loki (chart)** | 6.55.0 | **latest 6.x/newer** | uses stable core APIs — loose | opportunistic | grafana/helm-charts |
-| **Alloy (chart)** | 0.12.6 | **latest** | loose (DaemonSet, stable APIs) | opportunistic | grafana/alloy |
-| **Velero** | chart 8.7.2 (v1.15) | **latest (v1.17+/chart ~10.x)** | v1.16 tested→1.33, v1.17 tested→1.34; "1.18-latest" expected | before the 1.36 hop | velero.io support-matrix; compatibility.fyi/velero |
-| **Kyverno** | 3.3.9 | **latest 3.x (N-2 for 1.36)** | Kyverno follows k8s N-2; 3.3.x too old for 1.36 | before/with later hops | kyverno.io/docs/installation/releases |
-| **oauth2-proxy** | 7.18.0 | **latest 7.x** | loose | opportunistic | oauth2-proxy chart |
-| **external-dns** | 1.15.2 | **latest 1.x** | loose | opportunistic | kubernetes-sigs/external-dns |
-| **kube-vip** | v0.8.9 | **latest (k8s.io 0.36.x-based)** | leader-election uses coordination.k8s.io/v1 (stable); latest bumps client-go to 0.36 | before/with the 1.36 hop | github.com/kube-vip/kube-vip releases |
-| **headlamp** | 0.41.0 | latest | loose UI | opportunistic | headlamp chart |
-
-### 4.2 The pacing blockers, and the per-hop lockstep
-
-**Cilium and Longhorn are the two items that force the pace** — each supports only ~3-4 k8s minors
-and each upgrades **one minor at a time (no skipping, pre-flight required)**. KubeVirt/CDI are
-already at target, so the VM stack is *not* a blocker this cycle. The mapping:
-
-| Window | k8s target | Cilium (HelmRelease) | Gateway API CRDs | Longhorn (HelmRelease) | Other components landed in this window |
-|---|---|---|---|---|---|
-| **W1** | 1.32 | **1.17** (covers 1.29-1.32) | bump toward Cilium 1.17's required version | **1.8** | — |
-| **W2** | 1.33 | **1.18** (1.30-1.33) | bump | **1.9** | cert-manager→1.18/1.19; kube-prometheus-stack CRDs refresh |
-| **W3** | 1.34 | **1.19** (1.32-1.35) | bump | **1.10** | CNPG→1.28/1.29; Velero→latest |
-| **W4** | 1.35 | 1.19 still covers 1.35 (optional →1.20 early) | as needed | **1.11** | Kyverno→latest; **Flux v1beta2→v1 migration + Flux→v2.7+** |
-| **W5** | 1.36 | **1.20** (1.33-1.36) | **v1.6.x** | **1.12** | cert-manager→1.21; CNPG→1.30; kube-vip→latest; snapshot-controller→latest v8.x |
-
-Ordering rule **inside** each window (critical): **CNI/storage compatibility first, then the k3s
-control-plane hop, then verify.** Concretely per window:
-1. Land the **component** upgrades that must *precede* the new k8s minor (Cilium minor, Gateway API
-   CRDs, Longhorn minor, and any component whose *current* version won't run on the new k8s). Let
-   Flux reconcile; verify healthy on the **still-current** k8s.
-2. Run the **k3s minor hop** (§5) node-by-node.
-3. Verify the components again on the **new** k8s; then land the "opportunistic/loose" bumps.
-
-> Rationale for "Cilium before the apiserver hop": if the apiserver moves to a minor the running
-> Cilium doesn't support, CNI/agent may fail to start on that node and the node never goes Ready.
-> Cilium's own guidance is to run its **pre-flight check** DaemonSet before each Cilium minor bump
-> and to upgrade Cilium **before or alongside** the k8s minor within its supported window.
-
----
-
-## 5. Upgrade method — evaluation & recommendation (D1)
-
-### 5.1 Options
-
-**(a) Rancher system-upgrade-controller (SUC) with Plans.** Canonical k3s upgrade tool. A `Plan`
-targets nodes by label, concurrency 1, HA-aware, drains → replaces the k3s binary → uncordons.
-- **Pros:** official, declarative, handles server-vs-agent ordering; could live in GitOps.
-- **Cons for THIS cluster:** SUC's default **drain evicts pods**, which on this slow-HDD Longhorn
-  triggers replica **rebuild** storms that starve etcd WAL fsync and flap the VIP — the exact
-  failure `ssd-etcd-isolation.md` documents. You'd have to disable drain (`drain: {enabled:false}`)
-  and still manually apply the Longhorn freeze, at which point SUC's main value (drain automation) is
-  gone. Worse, SUC **automates away the human gate between nodes** — the user's hard requirement is
-  an explicit backup + PASS/STOP verification *between* every node, which SUC does not pause for.
-
-**(b) Scripted in-place per-node binary upgrade (Ansible-driven).** Re-run the k3s install script
-with the new `INSTALL_K3S_VERSION` on one node at a time; k3s replaces its binary and restarts,
-etcd re-joins at the new version. This is exactly the `ssd-etcd-execution-runbook.md` bounce shape.
-- **Pros:** reuses the **proven** freeze + leader-last + per-node-verify pattern; no drain (no
-  rebuild storm — replicas *reattach*); the human gate sits naturally between nodes; minimal moving
-  parts. **Note the `creates:` guard:** the existing install task self-skips on a running node, so
-  the upgrade path must invoke the install script *explicitly* (a dedicated play/tag, or an ad-hoc
-  `curl … | INSTALL_K3S_VERSION=<tag> sh -s - server`), not rely on bumping the var alone.
-- **Cons:** does not touch the OS (handled separately, §6/D2); slightly more manual than SUC.
-
-**(c) Immutable re-provision via tofu.** Bump `image_source` + `k3s_version`, `tofu replace` one VM
-at a time, node rejoins etcd fresh and Longhorn rebuilds that node's replicas.
-- **Pros:** **also solves the OS upgrade** in the same motion; clean, reproducible cattle nodes.
-- **Cons:** destroys the node's etcd member + **all** its Longhorn replicas → full **rebuild** (not
-  reattach) on the slow HDD per node (throttled but heavy, ~hours for 18 volumes' replicas across
-  three rebuilds); the etcd SSD `virtio2` is wiped on replace (fine for a 3-node HA etcd — the new
-  member syncs from the other two). Heavier and slower than (b) for a *pure k8s* hop.
-
-### 5.2 Recommendation
-
-- **k3s minor hops (W1-W5): choose (b), scripted in-place per-node.** It matches the cluster's
-  proven safe-bounce, avoids rebuild storms, and preserves the mandatory per-node human gate. Pin
-  the new tag in `roles/k3s_server/defaults/main.yml` (`k3s_version`) **and** drive the actual
-  upgrade with an explicit install-script invocation (bypassing the `creates:` guard), `serial: 1`,
-  leader-last.
-- **Node OS (Debian 12→13): choose (c), immutable re-provision**, as a **separate** campaign (§8).
-  A clean trixie image beats an in-place `apt` dist-upgrade for reproducibility, and the nodes are
-  already cattle (cloud-init + `node_baseline` rebuild them). Accept the per-node Longhorn rebuild
-  there (that campaign *wants* rebuild, throttled at limit=1, since replicas are genuinely gone).
-- SUC remains a reasonable future option **once the cluster is on fast disks** (the prod/mini-PC
-  Phase-7 cluster) where drain-driven rebuild is cheap; note it for later.
-
----
-
-## 6. Data-validation gates at EVERY step (hard requirement)
-
-Every **node bounce** (there are 3 per k3s window; 3 per OS window) is wrapped by a pre-step backup
-gate and a post-step verification gate. Every **window** additionally takes a full cluster backup
-before the first node.
-
-### 6.1 PRE-STEP backup gate (before the first node of a window, and re-confirm before each node)
-
-```bash
-# 1) etcd on-demand snapshot (k3s native) — the primary rollback artifact. Run on the leader.
-./infra/devcontainer/dev.sh run bash -lc 'cd infra/ansible && ansible <leader> -b -m shell \
-  -a "k3s etcd-snapshot save --name pre-1.3X-$(date +%Y%m%d%H%M)"'
-# verify it exists + is copied off-node (snapshots land in /var/lib/rancher/k3s/server/db/snapshots;
-# also archived to S3 if configured). Confirm size > 0 and note the name.
-
-# 2) Velero backup (cluster resources + PVC data via kopia FSB → MinIO 192.168.1.9:9000)
-./infra/devcontainer/dev.sh run bash -lc 'velero backup create pre-1.3X-$(date +%Y%m%d%H%M) --wait'
-./infra/devcontainer/dev.sh run bash -lc 'velero backup describe <name> --details'   # Phase=Completed, 0 errors
-
-# 3) CNPG logical backup + verify, for each of keycloak-db / nextcloud-db / semaphore-db
-#    (they already run continuous WAL + daily base backups to MinIO via barmanObjectStore).
-#    Trigger an on-demand base backup and confirm it completes:
-./infra/devcontainer/dev.sh run kubectl -n <ns> create -f - <<'EOF'
-apiVersion: postgresql.cnpg.io/v1
-kind: Backup
-metadata: { name: pre-upg-<ns>, namespace: <ns> }
-spec: { method: barmanObjectStore, cluster: { name: <cluster> } }
-EOF
-./infra/devcontainer/dev.sh run kubectl -n <ns> get backup pre-upg-<ns> -o jsonpath='{.status.phase}'  # completed
-# capture an integrity baseline to compare after:
-./infra/devcontainer/dev.sh run kubectl -n <ns> exec <cluster>-1 -- psql -tAc \
-  "select (select count(*) from pg_database), (select sum(('x'||substr(md5(t.oid::text),1,8))::bit(32)::bigint) from pg_class t)"
-```
-
-**PASS to proceed:** etcd snapshot present (>0 bytes, name recorded) **AND** Velero backup
-`Completed` with 0 errors **AND** all three CNPG on-demand backups `completed` with baseline
-captured.
-**STOP (do not start the hop):** any backup fails, errors, or cannot be verified restorable.
-
-### 6.2 The per-node bounce itself (reuse `ssd-etcd-execution-runbook.md`)
-
-Per node `NODE=k3s-dev-N`, `VMID`, **leader last** (re-verify leader before each node):
-
-```bash
-# leader check
-./infra/devcontainer/dev.sh run bash -lc 'cd infra/ansible && ansible k3s_servers -m shell \
-  -a "curl -s http://127.0.0.1:2381/metrics | grep etcd_server_is_leader"'
-
-# 1) FREEZE Longhorn rebuilds (reattach, not rebuild) — the proven safe-bounce
-./infra/devcontainer/dev.sh run kubectl -n longhorn-system patch settings.longhorn.io \
-  concurrent-replica-rebuild-per-node-limit --type=merge -p '{"value":"0"}'
-
-# 2) Upgrade k3s ON THIS NODE ONLY (explicit install, bypassing the creates: guard):
-./infra/devcontainer/dev.sh run bash -lc 'cd infra/ansible && ansible $NODE -b -m shell \
-  -a "curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.3X.z+k3s1 sh -s - server"'
-#    (k3s replaces its binary and restarts; embedded etcd re-joins at the new version)
-
-# 3) VERIFY this node fully before touching the next (see §6.3), THEN
-# 4) THAW rebuilds only after 18/18 healthy:
-./infra/devcontainer/dev.sh run kubectl -n longhorn-system patch settings.longhorn.io \
-  concurrent-replica-rebuild-per-node-limit --type=merge -p '{"value":"1"}'
-```
-
-### 6.3 POST-STEP verification gate (after each node; full set after the last node of a window)
-
-```bash
-./infra/devcontainer/dev.sh run kubectl get nodes -o wide            # NODE Ready, VERSION = new minor
-./infra/devcontainer/dev.sh run kubectl get --raw=/readyz/etcd       # ok
-./infra/devcontainer/dev.sh run bash -lc 'cd infra/ansible && ansible k3s_servers -m shell \
-  -a "curl -s http://127.0.0.1:2381/metrics | grep etcd_server_has_leader"'   # all = 1
-./infra/devcontainer/dev.sh run kubectl -n longhorn-system get volumes.longhorn.io \
-  -o custom-columns=R:.status.robustness --no-headers | grep -c healthy       # 18
-./infra/devcontainer/dev.sh run kubectl -n kube-system rollout status ds/cilium --timeout=300s
-./infra/devcontainer/dev.sh run bash -lc 'flux get kustomizations -A && flux get helmreleases -A'  # all Ready
-# etcd snapshot restorability spot-check (list; a scratch restore is the W-level drill, §7)
-./infra/devcontainer/dev.sh run bash -lc 'cd infra/ansible && ansible <leader> -b -m shell \
-  -a "k3s etcd-snapshot ls"'
-# CNPG integrity vs the §6.1 baseline
-./infra/devcontainer/dev.sh run kubectl -n <ns> exec <cluster>-1 -- pg_isready         # accepting connections
-# re-run the row-count/checksum query and diff against baseline (must match)
-```
-
-**App smoke tests (the QA-sweep URL set)** — after each *window* completes, expect HTTP 200/302
-(or the app's healthy code) from:
-`auth` · `crafty` · `gitea` · `grafana` · `headlamp` · `jackett` · `keycloak`
-(`/realms/homelab/.well-known/openid-configuration`) · `nextcloud` · `pihole` · `plex` ·
-`qbittorrent` · `radarr` · `seerr` · `semaphore` · `sonarr` · `unifi` — all under
-`*.home.keithjackson.dev`. Plus one KubeVirt VM check (a VMI Running / console reachable).
-
-**PASS to advance to the next node/window:** all nodes Ready at the expected version · etcd
-`/readyz/etcd`=ok and 3/3 have a leader · **18/18** Longhorn healthy (reattached, no rebuild storm)
-· Cilium DaemonSet rolled out · all Flux Kustomizations + HelmReleases Ready · CNPG `pg_isready` +
-row-count/checksum unchanged vs baseline · QA-sweep all green.
-**STOP (halt, leave freeze=0, do not advance):** etcd `/readyz` not ok · a member without a leader ·
-VIP `192.168.2.50:6443` flapping · a node not rejoining/not Ready · any Longhorn volume Faulting ·
-k3s crash-looping · a CNPG DB failing `pg_isready` or checksum drift · a Flux HR stuck not-Ready ·
-any QA-sweep URL down. Recovery per §7.
-
----
-
-## 7. Rollback per step & point-of-no-return
-
-### 7.1 Per-node (before etcd majority is on the new minor)
-
-The safest rollback for a single bad node is to **re-pin the old k3s version on that node** and
-re-run the install script for the old tag, then verify it rejoins at the old minor; or reset the
-member and rejoin from the other two (2/3 quorum holds), per `ssd-etcd-isolation.md` §4.5. Because
-you go **leader-last and one node at a time**, at any moment ≤1 node is on the new minor until the
-2nd node upgrades.
-
-### 7.2 Point-of-no-return per hop
-
-**When the *second* server reaches the new minor, etcd majority (2/3) is on the new datastore
-version** (etcd 3.5→3.6 across the path is one-way). Past that point, "roll back the node" is no
-longer enough — a true rollback is an **etcd snapshot restore** of the whole cluster to the pre-hop
-snapshot:
-
-```bash
-# cluster-wide restore from the pre-hop snapshot (DISRUPTIVE — full control-plane stop):
-# on ONE server, with k3s stopped on all three:
-./infra/devcontainer/dev.sh run bash -lc 'cd infra/ansible && ansible k3s_servers -b -m shell -a "systemctl stop k3s"'
-./infra/devcontainer/dev.sh run bash -lc 'cd infra/ansible && ansible <node1> -b -m shell \
-  -a "k3s server --cluster-reset --cluster-reset-restore-path=/var/lib/rancher/k3s/server/db/snapshots/<pre-1.3X-snap>"'
-# then wipe db on the other two and rejoin them (standard k3s cluster-reset-restore flow),
-# and PIN k3s_version back to the old tag before they re-join.
-```
-
-After an etcd restore, **Longhorn volume data is recovered by Velero (kopia FSB) / CNPG PITR** for
-anything written between snapshot and restore — this is why the §6.1 triad (etcd + Velero + CNPG)
-is mandatory, not just etcd.
-
-### 7.3 Window-level DR drill (do once, in W1, to prove the path)
-
-Before the first real hop, prove restorability end-to-end: `velero restore create --from-backup
-<pre-snap> --namespace-mappings <ns>:<ns>-scratch` into a scratch namespace and confirm data lands
-(this is the existing Phase-4 R-STATE-03 gate). If the drill fails, **do not begin the upgrade.**
-
-### 7.4 OS-campaign rollback
-
-Re-provision is inherently reversible per node: if a re-provisioned trixie node misbehaves,
-`tofu` re-apply with `image_source` reverted to bookworm re-creates it on Debian 12; the node
-rejoins etcd and rebuilds replicas as with any replacement. Point-of-no-return is the same etcd
-majority rule if OS re-provision is (against advice) combined with a k3s bump — which is why D3 says
-keep them separate.
-
----
-
-## 8. Node OS upgrade — Debian 12 → 13 (trixie)
-
-- **Debian 13 "trixie" released 2025-08-09, ships Linux 6.12 LTS** (up from bookworm's 6.1).
-  Support to 2028-08 (LTS 2030-08). Sources: <https://www.debian.org/News/2025/20250809> ·
-  <https://www.debian.org/releases/trixie/>.
-- **Kernel implications (positive):** 6.12 is newer than Cilium's/Longhorn's minimums and improves
-  eBPF/BPF feature coverage for Cilium and has current `iscsi_tcp`/`nfs`/`dm_crypt`/`overlay`/
-  `br_netfilter` modules Longhorn needs (all already pinned in
-  `roles/node_baseline/defaults/main.yml` → `node_baseline_kernel_modules`). No known regression for
-  this stack; `open-iscsi`/`nfs-common` packages exist on trixie.
-- **Method (D2): immutable re-provision.** Bump `infra/tofu/variables.tf`:
-  - `image_source` → `https://cloud.debian.org/images/cloud/trixie/latest/debian-13-generic-amd64.qcow2`
-  - `image_file_name` → `debian-13-generic-amd64.qcow2`
-  Then `tofu apply -replace=proxmox_virtual_environment_vm.node["k3s-dev-N"]` **one node at a time**,
-  let `node_baseline` + `k3s_server` (secondary/join) rebuild it, and let it rejoin etcd. Because a
-  replaced node loses its Longhorn replicas, run this campaign with rebuild **throttled at 1** (not
-  frozen at 0 — you *want* the rebuild here), wait for 18/18 healthy before the next node, and keep
-  the same per-node backup + verify gates (§6).
-- **Coordinate with k3s (D3):** run the OS campaign **after** the cluster is stable on a chosen k3s
-  minor (either finish the whole 1.36 ladder first, or interleave one full campaign between k3s
-  windows) — **never** re-provision to trixie *and* bump k3s in the same node bounce; that entangles
-  two failure domains and defeats per-step rollback.
-- **In-place `apt` dist-upgrade** is the fallback if re-provision is undesirable, but it is discarded
-  here: it mutates cattle nodes outside the tofu/cloud-init source of truth, risks half-migrated
-  states across three nodes, and violates the repo's "changes go through IaC" principle.
-
----
-
-## 9. Sequencing & HA safety
-
-- **Quorum:** 3 servers, embedded etcd, quorum 2/3. Exactly **one** node down at a time keeps
-  quorum. Never bounce two simultaneously.
-- **Leader last:** re-verify the etcd leader before each node (`etcd_server_is_leader` on :2381) and
-  upgrade followers first, leader last, so leadership only moves once and late.
-- **VIP stability:** kube-vip floats 192.168.1.49 to the current leader; with one node down the VIP
-  fails over to a surviving leader and the API stays reachable through the pfSense NAT
-  (192.168.2.50:6443). Watch for VIP flapping as a STOP signal.
-- **Rebuild-freeze (reattach, not rebuild):** for k3s in-place hops set
-  `concurrent-replica-rebuild-per-node-limit=0` around each bounce so a briefly-down node's replicas
-  **reattach** instead of triggering a rebuild storm that starves etcd WAL fsync (the whole reason
-  etcd is on a dedicated SSD). Thaw to `1` only after 18/18 healthy. (For the OS re-provision
-  campaign use `1`, not `0`, since replicas are genuinely gone and must rebuild.)
-- **CNPG note:** all three DBs are **single-instance** (`instances: 1`) — each incurs a brief
-  outage when its pod reschedules during a node bounce. Order/announce accordingly; the app
-  smoke-tests (§6.3) will blip while a DB pod moves. (Consider `instances: 3` for the prod cluster.)
-- **KubeVirt/CDI:** already 1.36-ready; VMs without `evictionStrategy: LiveMigrate` simply
-  Evict+restart on a node bounce (we do **not** drain in method (b), so VMs are only affected on
-  the node actually bouncing).
-
-**Estimated maintenance windows:** per k3s minor ≈ **60-120 min** (3 nodes × ~15-30 min bounce +
-verify + freeze/thaw + component reconcile), so **5 windows ≈ 5-10 h** total spread across days,
-plus the W-level backups. OS re-provision campaign ≈ **2-4 h per node** (dominated by Longhorn
-rebuild of that node's share of 18 volumes on HDD) → ~1-1.5 days, one node per session.
-
----
-
-## 10. Dev-first, then prod (Phase-7 carry-over)
-
-This dev run is the rehearsal for the eventual **prod / mini-PC** cluster. What carries over:
-- The **method** (scripted in-place per-node for k3s; immutable re-provision for OS), the
-  **leader-last + freeze + per-node-gate** discipline, and the **backup triad** gate are
-  cluster-agnostic and become the prod runbook verbatim.
-- The **component lockstep table** (§4.2) is the reusable dependency map; re-confirm exact patch
-  numbers at prod time (they will have moved).
-- **Differences to expect on prod:** fast disks (NVMe/SSD) make Longhorn rebuild cheap → **SUC
-  becomes viable** there (drain no longer dangerous), and multi-instance CNPG (`instances: 3`)
-  removes the single-DB outage blip. Prod may also run dedicated workers, changing SUC Plan
-  server-vs-agent ordering.
-- The **Flux v1beta2→v1 migration** (§3.2) is a one-time repo change shared by both clusters — do it
-  once, both benefit.
-
----
-
-## Appendix A — pre-flight checklist per window (copy/paste)
-
-```
-[ ] Read k3s release notes for the target minor + k8s CHANGELOG "Urgent Upgrade Notes"
-[ ] pluto/kubent scan (live + manifests) against target minor → clean (or Flux v1beta2 handled)
-[ ] Component pre-reqs landed & Ready on CURRENT k8s (Cilium minor, Gateway CRDs, Longhorn minor, …)
-[ ] Cilium pre-flight check DaemonSet passed for the Cilium minor bump
-[ ] PRE-STEP backup gate PASS: etcd snapshot + Velero Completed + 3× CNPG completed + baselines captured
-[ ] (W1 only) Velero restore DR drill into scratch ns PASS
-[ ] Leader identified; node order = followers → leader
-[ ] For each node: freeze(0) → upgrade → §6.3 verify PASS → thaw(1) → 18/18 healthy → next
-[ ] After last node: full §6.3 + QA-sweep all green + KubeVirt VM check
-[ ] Land opportunistic/loose component bumps; Flux all Ready
-[ ] Pin the new k3s_version in roles/k3s_server/defaults/main.yml; PR → main (never commit to main)
-[ ] Record outcome in DECISIONS-LOG.md
-```
-
-## Appendix B — sources
-
-- k3s releases & notes: <https://github.com/k3s-io/k3s/releases> · <https://docs.k3s.io/release-notes/v1.36.X> · <https://docs.k3s.io/release-notes/v1.33.X> · channels <https://update.k3s.io/v1-release/channels>
-- k8s version-skew: <https://kubernetes.io/releases/version-skew-policy/> · deprecation guide <https://kubernetes.io/docs/reference/using-api/deprecation-guide/> · v1.36 sneak peek <https://www.kubernetes.io/blog/2026/03/30/kubernetes-v1-36-sneak-peek/>
-- Cilium compat: <https://docs.cilium.io/en/stable/network/kubernetes/compatibility/> · <https://compatibility.fyi/projects/cilium/> · Gateway API <https://docs.cilium.io/en/v1.20/network/servicemesh/gateway-api/gateway-api/>
-- Longhorn: <https://endoflife.date/longhorn> · <https://longhorn.io/docs/> · <https://www.suse.com/suse-longhorn/support-matrix/> · <https://github.com/longhorn/longhorn/wiki/Version-Update-Policy>
-- KubeVirt/CDI: <https://github.com/kubevirt/sig-release/blob/main/releases/k8s-support-matrix.md> · <https://github.com/kubevirt/kubevirt/blob/main/docs/kubernetes-compatibility.md>
-- external-snapshotter: <https://github.com/kubernetes-csi/external-snapshotter/releases>
-- cert-manager: <https://cert-manager.io/docs/releases/>
-- CNPG: <https://cloudnative-pg.io/docs/devel/supported_releases/>
-- Flux: <https://fluxcd.io/flux/releases/> · v2.7 upgrade <https://github.com/fluxcd/flux2/discussions/5572> · image API <https://fluxcd.io/flux/components/image/imagerepositories/>
-- kube-prometheus-stack: <https://github.com/prometheus-community/helm-charts>
-- Velero: <https://velero.io/docs/main/> · <https://compatibility.fyi/projects/velero/>
-- Kyverno: <https://kyverno.io/docs/installation/releases/>
-- kube-vip: <https://github.com/kube-vip/kube-vip/releases> · <https://kube-vip.io/docs/upgrade/>
-- Debian 13: <https://www.debian.org/News/2025/20250809> · <https://www.debian.org/releases/trixie/>
-- Tools: pluto <https://github.com/FairwindsOps/pluto> · kubent <https://github.com/doitintl/kube-no-trouble>
-```

docs/plans/l2-l3-conformance-ci-wiring.md

@@ -1,185 +0,0 @@
-# L2/L3 conformance CI wiring — implementation plan
-
-**Status:** PLAN — deferred, awaiting operator decision on the trigger-credential model. No
-cluster changes made.
-**Author:** WS-GITOSS agent, 2026-08-31 (Hand-off 2).
-**Scope:** wire the L2 (behavioural) + L3 (fault-injection) conformance layers into the LIVE
-Gitea Actions merge gate, running them as an **in-cluster Chainsaw Job** so CI holds **no broad
-kubeconfig** (docs/14 "Where each layer runs", ADR-0011, R-SEC-02).
-
-> This companions the landed Hand-off 1 (commit `27bc024`: `make -C ci gate` parity + the
-> comment-only R-ID scan fix). The `conformance` job in `.gitea/workflows/gate.yml` is still an
-> honest `if: false` placeholder — this plan is what turns it on **safely**.
-
----
-
-## 1. Why this is a written plan, not a landed change
-
-The intended seam is real and mostly scaffolded, but finishing it autonomously would mean
-handing a shared, **live** cluster a new privileged identity reachable from CI, plus building
-several missing functional pieces. The hand-off's rule is explicit: *if wiring real cluster
-access from CI is too risky/ambiguous to finish autonomously, STOP and write a concrete plan —
-do NOT hand CI a broad kubeconfig.* The blocker is a genuine **privilege-escalation** design
-question (§4), not just plumbing, and the cluster is being changed concurrently by sibling
-agents. So the credential model must be decided by the operator before anything is applied.
-
----
-
-## 2. Current state (what exists today)
-
-| Piece | File | State |
-|---|---|---|
-| Live gate workflow | `.gitea/workflows/gate.yml` | LIVE. `gate` job runs `make -C ci gate`. `conformance` job is `if: false` placeholder. |
-| Dormant full blueprint | `.gitea/workflows/merge-gate.yaml` | Shows the intended shape: `conformance` job `if: ${{ vars.CLUSTER_AVAILABLE == 'true' }}`, `secrets.CONFORMANCE_TRIGGER_KUBECONFIG`, `make -C ci conformance`, then a `merge-gate` decision job (`scripts/decide.sh`). Manual-only (`workflow_dispatch`). |
-| CI conformance stage | `ci/conformance.sh` | SKIPS (exit 3) unless `CLUSTER_AVAILABLE=true`. When "on", it currently runs `conformance/run.sh --layer l2,l3` **locally in the CI job** and collects `_artifacts/`. Its own header carries `TODO(Phase 1): drive the in-cluster Chainsaw Job under the narrow CONFORMANCE_TRIGGER_KUBECONFIG` — i.e. the trigger path is **not implemented**. |
-| Conformance runner | `conformance/run.sh` + `conformance/lib/`, `conformance/tests/` | Works locally/in-cluster. `--layer l2,l3 [--requirements ...]`, writes `_artifacts/report.junit.xml` + `coverage/`. Needs bash + python3 + chainsaw (+ kubectl for `conf_cluster_reachable`). |
-| In-cluster Job skeleton | `conformance/incluster/chainsaw-job.yaml` | Skeleton: `generateName: conformance-run-`, ns `conformance`, `serviceAccountName: conformance-runner`, image `ghcr.io/kyverno/chainsaw:v0.2.12`, `command: ["./run.sh"]`, args default `--layer all --strict-coverage`. **`workspace` is an empty `emptyDir` with no init-container to populate it**, and `results` is an `emptyDir`. |
-| Scoped runner RBAC | `conformance/incluster/rbac.yaml` | ns `conformance` + SA `conformance-runner` + a **privileged** ClusterRole (namespaces create/delete/patch, pods+pods/exec, jobs create/delete, **nodes get/list/patch** for R-STATE-02 cordon, pods/log). Least-privilege, per-test-justified. **Not reconciled by Flux** (no kustomization references `conformance/incluster/`). |
-| Runner deployment | `apps/wave-1/gitea-actions.yaml` | act_runner + DinD (Docker executor). **`automountServiceAccountToken: false`** — the runner and its job containers hold NO SA token. Broad egress allowed (`- {}`), so it can *reach* the kube-API VIP `192.168.1.49:6443`, but has no credential. |
-| Secrets encryption | `.sops.yaml` (repo root) | age recipient `age175l8m…mfzg`; encrypts `data`/`stringData` of `*.sops.yaml` under `platform|apps|catalog/**` and `secrets/**`. Private key is the in-cluster `sops-age` Secret Flux uses. |
-
-**Cluster reality:** the "Phase 0 / no cluster" language in these files is stale — the dev cluster
-is LIVE (Phase 6). So `CLUSTER_AVAILABLE` is the only remaining gate, exactly as the blueprint says.
-
----
-
-## 3. Gaps to close (functional)
-
-1. **Runner image.** `ghcr.io/kyverno/chainsaw` is a minimal image; `run.sh` needs **bash +
-   python3 + chainsaw + kubectl**, and the repo-clone init needs **git**. The Job cannot run
-   `./run.sh` on the stock chainsaw image. → Build a pinned `conformance-runner` image (chainsaw
-   `v0.2.12` + kubectl `v1.36.4` + git + python3 + bash), push to the Gitea container registry,
-   pin its digest in `conformance/versions.env` (`CONFORMANCE_RUNNER_IMAGE`). Keep
-   `CHAINSAW_VERSION` as the single source of truth the image is built from.
-2. **Repo at the commit-under-test.** The Job must run the suite at the PR head SHA. → Add an
-   **init-container** (`git clone --depth 1` of the impl repo, then `git checkout <SHA>`) writing
-   into the shared `workspace` volume, using a **read-only** Gitea deploy token. SHA + the touched
-   `--requirements` are injected by the CI trigger step as Job env/args.
-3. **Results collection without giving CI exec/cp.** The Job writes `_artifacts/` to the `results`
-   volume; CI holds only trigger/read rights (§4). → Job's last step prints a delimited
-   base64 tar of `_artifacts/` to stdout (`=== ARTIFACTS-TGZ-BASE64 ===`…); CI decodes it from
-   `kubectl logs`. No `pods/exec`, no `kubectl cp`, no results PVC exposure required. (A results
-   PVC + a read-only sidecar is the durable alternative if log size becomes a problem.)
-4. **`ci/conformance.sh` rewrite.** Replace the local `run.sh` call with: render the Job manifest
-   (inject image digest, SHA, `--requirements <touched>`), `kubectl --kubeconfig=$CONFORMANCE_TRIGGER_KUBECONFIG
-   -n conformance create -f -`, `kubectl wait --for=condition=complete/failed`, stream logs,
-   decode artifacts into `results/junit` + `reports/coverage`, map Job success→0 / failure→1 /
-   no-cred→3 (skip). Keep the touched-subset selection it already computes.
-5. **Flux wiring.** Reconcile `conformance/incluster/` (ns + `conformance-runner` SA/RBAC + the
-   new trigger SA/Role + the admission policy of §4) via a **non-gating** Kustomization
-   (`wait:false`, `dependsOn: platform-config`), mirroring `apps-vms` — a conformance identity
-   must never gate the app tier.
-6. **Gitea provisioning (operator, out-of-GitOps).** Set repo/org `vars.CLUSTER_AVAILABLE=true`
-   and `secrets.CONFORMANCE_TRIGGER_KUBECONFIG` via the Gitea API
-   (`PUT /repos/{owner}/{repo}/actions/{variables,secrets}/{name}`). The kubeconfig embeds the
-   trigger SA's token only. SOPS-seal the SA token in-repo (`secrets/conformance-trigger.sops.yaml`)
-   for reproducibility/rotation; the Gitea secret is derived from it.
-
----
-
-## 4. The security crux — trigger credential must NOT grant `conformance-runner`'s power
-
-**Problem.** Kubernetes places no `use` check on a Pod/Job's `serviceAccountName` at creation.
-So **any principal that can `create jobs` in namespace `conformance` can create a Job with
-`serviceAccountName: conformance-runner` and an arbitrary `command`** — i.e. run arbitrary code
-with the privileged runner SA (nodes/patch, pods/exec, jobs, namespaces/patch). A CI-held token
-with plain `create jobs` therefore *is* the privileged identity. That breaks R-SEC-02 ("CI holds
-no broad cluster access") even though the token's RBAC "looks" narrow.
-
-**Recommended model — narrow trigger SA + an admission constraint (uses the existing Kyverno L0
-stack):**
-
-- SA `conformance-trigger` (ns `conformance`), Role: `create` on `jobs`; `get,list,watch` on
-  `jobs`; `get` on `pods` and `pods/log`. Nothing else. Bound only in the `conformance` ns.
-- A **Kyverno ClusterPolicy (validate, enforce, fail-closed)** that, *for Jobs created by
-  `system:serviceaccount:conformance:conformance-trigger`*, ADMITS only Jobs that match the
-  committed shape: image == the pinned `CONFORMANCE_RUNNER_IMAGE` digest,
-  `serviceAccountName == conformance-runner`, `command == ["./run.sh"]`, `args` matching
-  `^--(layer|requirements|strict-coverage)`, `securityContext` non-root/no-privilege/no host
-  namespaces, and no extra volumes beyond `workspace`/`results`. Any other Job from that SA is
-  DENIED. This is the real security boundary: it bounds CI to "run exactly the committed
-  conformance suite, nothing else." (Equivalent: a k8s ValidatingAdmissionPolicy bound to that SA.)
-- The `chainsaw-conf-*` per-test namespaces the runner creates are already flagged in
-  `rbac.yaml` as needing exclusion from any cluster-wide disallow-privileged/host-namespaces
-  policy for R-NET-06 — carry that exception through, documented and audited.
-
-**Alternative model — GitOps-native, zero CI cluster credential (more work, strongest):** CI does
-not reach the cluster at all. A `ConformanceRun` marker (a CR or a commit to a watched path) is
-reconciled by an in-cluster controller/Flux that creates the Job from the in-repo template; the
-Job publishes results back as a Gitea commit-status / a fetchable artifact that CI polls. Removes
-the trigger credential entirely, at the cost of a controller and a trigger/ordering mechanism that
-must complete before the merge decision. Recommended only if even the constrained trigger SA of
-the primary model is deemed unacceptable.
-
-**Do NOT:** give CI a `cluster-admin`/broad kubeconfig; give the trigger SA unconstrained
-`create jobs`; mount an SA token into the act_runner pod; or reuse `conformance-runner` as the
-trigger identity.
-
----
-
-## 5. Concrete step sequence
-
-Land these as separate reviewable PRs, gate-green at each step. Steps 1–5 are in-repo/GitOps and
-safe to land incrementally *without* turning L2/L3 on; only step 7 (the operator setting the Gitea
-var) actually enables it.
-
-1. **Runner image.** `conformance/incluster/Containerfile` (chainsaw+kubectl+git+python3+bash);
-   build+push to the Gitea registry; pin `CONFORMANCE_RUNNER_IMAGE=<registry>/conformance-runner@sha256:…`
-   in `conformance/versions.env`. Add a note tying it to `CHAINSAW_VERSION`.
-2. **Job manifest.** Rewrite `chainsaw-job.yaml`: swap to `CONFORMANCE_RUNNER_IMAGE`; add the
-   git-clone init-container (SHA + repo URL + read token env); parametrise `args` (CI injects
-   `--layer l2,l3 --requirements <touched>`); keep `restartPolicy: Never`, `backoffLimit: 0`,
-   non-root/seccomp/readonly-rootfs; add sane CPU/mem requests+limits (heavy L3 etcd load —
-   see `run.sh` HEAVY set + `config/chainsaw-config.yaml`).
-3. **Trigger SA/RBAC + admission policy.** Add `conformance-trigger` SA + Role/RoleBinding to
-   `rbac.yaml`; add the Kyverno ClusterPolicy (new file under `platform/kyverno/` or
-   `conformance/incluster/`) constraining that SA's Jobs (§4). Add an L0 Kyverno test fixture for
-   the policy (admit the good Job, deny a tampered one) so `make -C ci l0` covers it.
-4. **Flux Kustomization.** New `clusters/<cluster>/conformance.yaml` (or an `apps-conformance`
-   entry) — `wait:false`, `dependsOn: platform-config`, `prune:true`, pointing at
-   `conformance/incluster/`. No `postBuild.substituteFrom` unless needed. Verify it reconciles
-   Ready with the SA/RBAC/policy present and NO Job running (Jobs are created only on trigger).
-5. **`ci/conformance.sh` rewrite.** Implement the trigger→wait→collect→map flow of §3.4 behind
-   `CONFORMANCE_TRIGGER_KUBECONFIG`; keep exit 3 when the cred is absent (honest skip). Update
-   `ci/README.md`. `make -C ci gate` still SKIPS L2/L3 locally (no cred) — unchanged for humans.
-6. **Workflow.** Replace the `if: false` placeholder in `gate.yml`'s `conformance` job with the
-   blueprint shape: `if: ${{ vars.CLUSTER_AVAILABLE == 'true' }}`, `needs: [gate]`, install the
-   `conformance` toolchain, `env: CONFORMANCE_TRIGGER_KUBECONFIG: ${{ secrets.CONFORMANCE_TRIGGER_KUBECONFIG }}`,
-   `run: make -C ci conformance`. Decide: either keep the single-`gate` model (aggregate `gate.sh`
-   already SKIPS conformance and reports UNVERIFIED) and add conformance as a **second required
-   check**, or adopt the blueprint's separate `merge-gate` decision job (`scripts/decide.sh`).
-   Keep the job honest: skipped when the var is off, never faked green (HARD RULE 4). Add the
-   conformance check to branch protection only after step 7 proves it green.
-7. **Enable (operator).** Provision the SOPS-sealed trigger SA token; set the Gitea
-   `secrets.CONFORMANCE_TRIGGER_KUBECONFIG` and `vars.CLUSTER_AVAILABLE=true`. First run on a
-   throwaway PR; confirm the Job is created, runs the touched subset, results are collected, and
-   the gate blocks on a real L2/L3 failure and passes on green. Then make it required.
-
----
-
-## 6. Validation & rollback
-
-- **Dry-run the admission policy** before granting the trigger SA: `kubectl auth can-i` matrix for
-  `conformance-trigger`; attempt a *tampered* Job (different command) as that SA and confirm
-  Kyverno DENIES it; attempt the committed Job and confirm ADMIT.
-- **Fail-closed check:** temporarily break an L3 test, confirm the Job fails and the gate BLOCKS
-  (mirrors Hand-off 1's proof that `gate.sh` exit 1 fails the job).
-- **Rollback:** unset `vars.CLUSTER_AVAILABLE` (instant SKIP, gate reverts to L0/L1-only) and/or
-  suspend the Flux Kustomization; delete the trigger SA token from Gitea secrets. No app-tier
-  impact (conformance is a non-gating tier).
-
----
-
-## 7. Open decisions for the operator
-
-1. **Trigger model:** constrained trigger SA + Kyverno admission policy (§4 primary) vs.
-   GitOps-native zero-credential (§4 alternative). This plan recommends the primary.
-2. **Runner image host:** Gitea container registry (in-network, matches the Flux source loop) vs.
-   ghcr.io. Recommend Gitea registry, digest-pinned.
-3. **Gate topology:** conformance as a second required check on the single-job `gate.yml` vs.
-   restoring the blueprint's multi-job `merge-gate.yaml` with the `decide.sh` decision job.
-4. **Results transport:** base64-tar-over-logs (simple, no extra rights) vs. a results PVC +
-   read-only sidecar (durable). Recommend logs first; revisit if artifacts grow.
-5. **Which requirements go live first:** the R-NET/R-TLS/R-DNS/R-STATE tests already green against
-   the live cluster (per DECISIONS-LOG) are the natural first `--requirements` subset to certify
-   through the wired path before widening.

docs/plans/longhorn-ssd-tiering.md

@@ -1,468 +0,0 @@
-# Longhorn SSD tiering for the k3s dev cluster
-
-**Status:** PROPOSAL — awaiting user approval. No storage, tofu, Longhorn, or live
-infrastructure has been changed. This is a read-only investigation + reviewable plan.
-**Author:** investigation agent (read-only), 2026-08-31
-**Problem:** All 18 Longhorn data volumes live on the slow/failing Seagate-class HDD
-pool (`Storage`). Every node reboot marks that node's replicas stale and triggers a
-full 18-volume rebuild that spikes Proxmox IO-delay to 30 %+ and runs for a long time,
-making every upgrade / resize / reboot expensive and risky. etcd was already isolated
-onto a dedicated per-node SSD disk (`virtio2`, `local-zfs`) for exactly this reason
-(`docs/plans/ssd-etcd-isolation.md`). The recorded blocker was that the SSD "does not
-fit" the 3 × 128 GiB Longhorn data disks, so Longhorn stayed on HDD "interim until the
-all-SSD mini-PCs land."
-
-**Verdict (short):** The "doesn't fit" blocker was computed against the **provisioned**
-128 GiB disks. The **actual working set is ~26 GB per node (~79 GB across all three
-nodes)** — bulk media is on NFS, *not* Longhorn, so every Longhorn volume is a small
-config/DB volume. **A hot-tier move onto SSD is viable and recommended**; a full move
-is also feasible on space but re-couples rebuild I/O with etcd/host on the same SSD
-pool. Recommendation: **Phase 1 hot-tier move (Prometheus + the CNPG DBs + Loki +
-gitea) onto a tagged `longhorn-ssd` disk**, one node at a time.
-
----
-
-## 1. Measured facts (all collected read-only, 2026-08-31)
-
-### 1.1 The Longhorn working set — provisioned vs *actual*
-
-`kubectl -n longhorn-system get volumes.longhorn.io` — capacity (`spec.size`) vs
-**actual consumed** (`status.actualSize`, the real on-disk footprint of one replica,
-snapshots included). All 18 volumes are `numberOfReplicas: 3`, so each node holds one
-replica of each and a node's on-disk footprint ≈ the actualSize column below.
-
-| Volume / PVC | Namespace | App class | Provisioned | **Actual (per replica)** | Tier |
-|---|---|---|---:|---:|---|
-| prometheus-…-db-0 | monitoring | Prometheus TSDB | 8 GiB | **11.59 GB** | HOT |
-| unifi-rootdisk | unifi | KubeVirt VM disk (`longhorn-vm`) | 40 GiB | 6.20 GB | cold |
-| nextcloud-data | nextcloud | file data | 20 GiB | 1.44 GB | cold |
-| nextcloud-db-1 | nextcloud-db | CNPG Postgres | 5 GiB | 1.41 GB | HOT |
-| storage-loki-0 | monitoring | Loki chunks/index | 5 GiB | 1.24 GB | HOT |
-| keycloak-db-1 | keycloak | CNPG Postgres | 5 GiB | 0.93 GB | HOT |
-| gitea-data | gitea | git repos + SQLite/meta | 10 GiB | 0.90 GB | HOT |
-| semaphore-db-1 | semaphore-db | CNPG Postgres | 5 GiB | 0.79 GB | HOT |
-| crafty-data | crafty | Minecraft world | 20 GiB | 0.54 GB | cold |
-| plex-config | plex | Plex metadata/db | 8 GiB | 0.23 GB | cold |
-| seerr-config | seerr | config | 5 GiB | 0.17 GB | cold |
-| sonarr-config | sonarr | config db | 5 GiB | 0.15 GB | cold |
-| qbittorrent-config | qbittorrent | config | 5 GiB | 0.15 GB | cold |
-| radarr-config | radarr | config db | 5 GiB | 0.15 GB | cold |
-| pihole-config | pihole | config | 2 GiB | 0.10 GB | cold |
-| jackett-config | jackett | config | 2 GiB | 0.10 GB | cold |
-| seerr/act-runner-data | gitea-actions | CI cache | 1 GiB | 0.05 GB | cold |
-| semaphore-config | semaphore | config | 1 GiB | 0.05 GB | cold |
-| **TOTALS (per node / one replica)** | | | **152 GiB** | **~26.2 GB** | |
-
-- **Per node working set: ~26.2 GB actual** vs **152 GiB provisioned** (× 3 replicas =
-  456 GiB nominal, but Longhorn/ZFS are thin — nominal is not what lands on disk).
-- **Whole-cluster working set: ~79 GB** across the three nodes.
-- **Bulk media is not on Longhorn.** The StorageClass survey (§1.3) shows Plex media
-  libraries and the *arr download shares use the `bulk` NFS class
-  (`nfs.csi.k8s.io`), not Longhorn. So the "bulk stays on HDD while small/hot moves to
-  SSD" framing mostly dissolves: **there is no large Longhorn volume.** The biggest is
-  `unifi-rootdisk` at 6.2 GB actual, and it is a cold KubeVirt root disk.
-
-### 1.2 Longhorn per-node disk (the current single HDD data disk)
-
-`kubectl -n longhorn-system get nodes.longhorn.io` (all three nodes identical):
-
-| Field | Value |
-|---|---|
-| Disk | `default-disk`, path `/var/lib/longhorn`, tags `[]`, `allowScheduling: true` |
-| `storageMaximum` | 124.9 GiB (the ext4 fs on the 128 GiB `virtio1` HDD disk) |
-| `storageAvailable` | 100.4 GiB free |
-| `storageScheduled` | **152.0 GiB** — over-scheduled vs the 124.9 GiB max |
-
-The 152 GiB *scheduled* > 124.9 GiB *max* is Longhorn honouring
-`storage-over-provisioning-percentage: 200`. Actual usage is only ~24.5 GiB
-(124.9 − 100.4), matching §1.1.
-
-### 1.3 StorageClasses in play
-
-`kubectl get storageclass`:
-
-| Class | Provisioner | Notes |
-|---|---|---|
-| `longhorn` | `driver.longhorn.io` | default class, 3 replicas, ext4 — **17 of 18 PVCs** |
-| `longhorn-vm` | `driver.longhorn.io` | 3 replicas, `migratable: true` — `unifi-rootdisk` only |
-| `longhorn-scratch` | `driver.longhorn.io` | 1 replica, best-effort locality (ephemeral) |
-| `longhorn-static` | `driver.longhorn.io` | static-provisioning class |
-| `bulk` | `nfs.csi.k8s.io` | NFS — bulk media (unresolved `REPLACE-ME` server/share) |
-
-### 1.4 Longhorn settings relevant to a migration / rebuild storm
-
-`kubectl -n longhorn-system get settings.longhorn.io …`:
-
-| Setting | Value | Relevance |
-|---|---|---|
-| `concurrent-replica-rebuild-per-node-limit` | **1** | already throttled — one rebuild at a time per node (set during etcd work) |
-| `storage-over-provisioning-percentage` | 200 | dangerous on a *small* SSD disk — must lower for the SSD disk sizing |
-| `storage-minimal-available-percentage` | 25 | Longhorn stops scheduling when a disk is <25 % free — a safety floor |
-| `replica-auto-balance` | best-effort | will help spread replicas after a new disk appears |
-| `default-replica-count` | 3 | — |
-
-Longhorn version: **v1.12.1**. **No backup target is configured**
-(`backuptargets.longhorn.io/default`: URL empty, `available: false`) — so any
-migration path that relies on Longhorn *backup → restore-to-new-SC* is **not
-available** until an S3/NFS backup target is set up. This constrains the migration
-mechanism (§4) to in-cluster options: CSI clone, per-volume `diskSelector` rebuild, or
-app-level redeploy.
-
-### 1.5 The SSD pool (`local-zfs` = `rpool/data`)
-
-Measured read-only on 2026-08-29 during the etcd-isolation work
-(`docs/plans/ssd-etcd-isolation.md` §1.1) and unchanged in substance since — the only
-addition was the three 16 GiB etcd disks on 2026-08-30, which are thin and hold ~83 MB
-of etcd data each (~250 MB total actual):
-
-| Property | Value |
-|---|---|
-| Pool | `rpool`, mirror of 2 × Micron 5100/52x0 960 GB **enterprise SATA SSD w/ PLP** |
-| Health | 97 % lifetime remaining, 0 reallocated sectors — excellent |
-| `zpool` size / free | 888 G / **~350 G free** (60 % capacity) |
-| ZFS `AVAIL` to a new dataset/zvol | **~323 GiB** |
-| Proxmox host root on it | `rpool/ROOT/pve-1` = 430 G |
-| Storage entry | `local-zfs`, content `images,rootdir`, `sparse 1` (thin) |
-| Already carries | 3 × 16 GiB etcd zvols (thin, ~250 MB actual) |
-
-> **Re-verify at execution time.** Before applying, re-run `zpool list rpool` and
-> `pvesm status` on the Proxmox host to confirm current free space (this figure is
-> dated 2026-08-29). The vault-gated ansible path (`proxmox_hosts` group) or a Proxmox
-> UI check both work.
-
-### 1.6 Is the HDD *failing* — data-loss urgency?
-
-The problem statement calls the Seagate HDD "slow/failing." **SMART was not
-re-collected in this investigation** (the Proxmox host query is vault-gated in the
-dev container). The etcd-isolation doc characterised `Storage` as 2 × WDC WUH721818
-18 TB (healthy) and `Storage2` as 2 × Seagate ST8000VN004 — the Longhorn `virtio1`
-disks are on `Storage` (the WD mirror), a ZFS **mirror**, so a single-disk SMART fault
-is not immediate data loss. **Action item for the executor:** collect
-`smartctl -a` for both `Storage`-pool members and `zpool status -x` before scheduling;
-if a member is degraded, that raises urgency and argues for moving the *hot* data off
-sooner. Treat "failing" as unconfirmed until SMART is read — but the mirror + 3-way
-Longhorn replication means the plan is not gated on it.
-
----
-
-## 2. Capacity verdict — does it fit?
-
-| Scenario | Per-node SSD footprint (actual) | ×3 across nodes | Provisioned to size the disk | Fits in ~323 GiB SSD `AVAIL`? |
-|---|---:|---:|---:|---|
-| **Hot tier** (Prometheus + 3 CNPG DBs + Loki + gitea) | **~16.9 GB** | **~50.6 GB** | 38 GiB/node → suggest a **48 GiB** SSD disk/node | **Yes, comfortably** (~145 GiB of thin zvols max, ~51 GB actual) |
-| **Full move** (all 18 Longhorn volumes) | ~26.2 GB | ~79 GB | 152 GiB/node nominal, but thin | **Yes on space** (~79 GB actual) — the "doesn't fit" blocker was a *provisioned*-number artefact |
-| All-SSD mini-PC path (no move now) | — | — | — | N/A — hardware-blocked, no committed date |
-
-**The recorded blocker is overturned by the actual numbers.** 3 × 128 GiB = 384 GiB
-*provisioned* does not fit next to the 430 GiB host root on an 888 GiB pool — that is
-true and is why the earlier doc said "NO for Longhorn." But the **real data is ~79 GB**,
-and both ZFS (`sparse 1`) and Longhorn are thin, so what actually lands on the SSD is
-the working set, not the nominal disk size. Both a hot-tier move and a full move fit on
-free space today.
-
-The reason to still prefer the **hot tier** over a full move is **not space** — it is
-**I/O coupling** (see §3.4 and §6): the only SSD pool is `rpool`, which now also carries
-the Proxmox host root *and* the freshly-isolated etcd disks. A full move puts every
-Longhorn rebuild back onto the same pool as etcd, partially undoing the isolation just
-completed. A hot-tier move keeps the SSD footprint and the rebuild-I/O burst small while
-capturing the single largest and churniest contributor (Prometheus = 11.6 GB = 44 % of
-the whole working set) and all the latency-sensitive Postgres DBs.
-
----
-
-## 3. Options evaluated
-
-### 3.1 Option A — Hot-tier move onto a tagged SSD Longhorn disk (RECOMMENDED)
-
-Add a second, SSD-backed Longhorn data disk per node (a new `virtio3` zvol on
-`local-zfs`), tag it `ssd`, tag the existing HDD disk `hdd`, publish a `longhorn-ssd`
-StorageClass with `diskSelector: ["ssd"]`, and migrate the hot volumes onto it. Cold
-config volumes stay on the HDD `default-disk`.
-
-- **Hot set:** `prometheus-…-db-0` (monitoring), `storage-loki-0` (monitoring),
-  `keycloak-db-1`, `nextcloud-db-1`, `semaphore-db-1` (the three CNPG Postgres DBs),
-  `gitea-data`. ~16.9 GB/node actual, 38 GiB provisioned.
-- **Why these:** Prometheus is the biggest and churniest (TSDB compaction + snapshot
-  bloat — 11.6 GB actual on an 8 GiB nominal volume). Postgres DBs are fsync-latency
-  sensitive and small. Loki and gitea are moderate-churn. Moving these removes the bulk
-  of rebuild bytes *and* the latency-sensitive I/O from the HDD.
-- **Footprint:** small and bounded → the I/O-coupling risk on `rpool` is minimal.
-- **Downside:** the ~12 cold config volumes (~9 GB/node) still rebuild on the HDD on a
-  reboot — but they are tiny and low-churn, so the storm shrinks from 18 volumes /
-  ~26 GB to ~12 volumes / ~9 GB, and the SSD ones rebuild fast.
-
-### 3.2 Option B — Full move (all 18 Longhorn volumes to SSD)
-
-Size the SSD disk per node for the full working set (e.g. 96 GiB thin, lower
-over-provisioning) and repoint the default StorageClass / migrate every volume.
-
-- **Pro:** eliminates HDD Longhorn rebuild entirely — reboots no longer touch the HDD
-  for storage; SSD rebuild is fast and low-IO-delay.
-- **Con:** re-couples *all* Longhorn rebuild I/O with etcd + host root on `rpool`,
-  partially undoing `docs/plans/ssd-etcd-isolation.md`. SSD IOPS make this far less
-  harmful than the HDD case, but it is a real regression of the isolation just done.
-  Also consumes more of the small SSD pool's headroom.
-- **Verdict:** feasible, but hold it as a *Phase 2* only if the hot-tier move proves
-  insufficient. Not the opening move.
-
-### 3.3 Option C — Do nothing; wait for the all-SSD mini-PCs
-
-Zero work, zero risk, but leaves the cluster fragile to every node bounce
-indefinitely — the mini-PC migration has no committed date. What that path *requires*:
-new hardware (all-SSD mini-PCs), a fresh Proxmox/k3s build, and a full Longhorn data
-migration anyway. Option A is a cheap, reversible bridge that the mini-PC path later
-supersedes — it does not waste that work (the `longhorn-ssd` class and tagging carry
-over).
-
-### 3.4 The I/O-coupling caveat (applies to A and B)
-
-`rpool` is the **only** SSD pool. It already carries the host root and the isolated
-etcd disks. Putting Longhorn rebuild I/O back on it re-introduces *some* contention
-with etcd's WAL fsync — the exact thing the etcd disk was meant to escape. Two facts
-keep Option A safe: (1) SSD random IOPS are ~2–3 orders of magnitude above the HDD, so
-concurrent fsync + rebuild is tolerable where it was fatal on spinning rust; (2) the
-hot-tier footprint is small so rebuild bursts are short. **Mitigation:** keep
-`concurrent-replica-rebuild-per-node-limit: 1`, and watch
-`etcd_disk_wal_fsync_duration_seconds` (Prometheus) during the first SSD rebuild — if
-p99 fsync climbs materially, stop and reassess before proceeding to the next node.
-
----
-
-## 4. Recommended implementation — Option A, phased, one node at a time
-
-> Every step below is a *proposal*. Nothing here has been executed. Apply per node,
-> lowest-quorum-risk node first, verifying Longhorn/etcd health between nodes.
-
-### Phase 0 — Pre-flight (read-only, do before touching anything)
-
-1. Re-verify SSD free space: on the Proxmox host `zpool list rpool` and `pvesm status`
-   (confirm ≳ 100 GiB `AVAIL` remains after the etcd zvols).
-2. Collect HDD SMART: `smartctl -a` on both `Storage`-pool members + `zpool status -x`.
-   If a member is degraded, note the added urgency but the mirror + 3× replication mean
-   the plan is unchanged.
-3. Confirm Longhorn is fully healthy (all volumes `robustness: healthy`) and no rebuild
-   is in flight. Do not start during a rebuild.
-4. Snapshot the CNPG DBs (CNPG can take an on-demand base backup) and note that gitea /
-   Prometheus / Loki data is reproducible or non-critical — belt-and-braces before any
-   volume move, since there is no Longhorn backup target.
-
-### Phase 1 — Add an SSD Longhorn disk per node (tofu, mirrors the etcd-disk pattern)
-
-The VMs are bpg/OpenTofu-managed (`infra/tofu/main.tf`,
-`resource "proxmox_virtual_environment_vm" "node"`, `for_each = local.nodes`). Disks
-are `disk {}` blocks; `virtio0` (OS) and `virtio1` (Longhorn HDD) are on `var.datastore`
-(`Storage`), `virtio2` (etcd) is on `var.etcd_ssd_datastore` (`local-zfs`). Add a
-**fourth disk on the SSD datastore**, kept last so bpg does not reorder existing disks.
-
-1. New variables in `infra/tofu/variables.tf`:
-   ```hcl
-   variable "longhorn_ssd_datastore" {
-     description = "SSD-backed datastore for the tagged Longhorn SSD disk (local-zfs = rpool/data, Micron 5100 SATA SSD mirror w/ PLP)."
-     type        = string
-     default     = "local-zfs"
-   }
-   variable "longhorn_ssd_disk_gib" {
-     description = "Per-node SSD Longhorn disk, GiB. Hot tier is ~17 GiB actual / 38 GiB provisioned; 48 GiB gives headroom without crowding rpool."
-     type        = number
-     default     = 48
-   }
-   ```
-2. New `disk {}` block in the VM resource in `main.tf`, appended **after** the
-   `virtio2` etcd block (ascending interface → `virtio3` → `/dev/vdd`):
-   ```hcl
-   # Tagged SSD Longhorn disk (virtio3 -> /dev/vdd) for the hot tier (Prometheus, CNPG
-   # DBs, Loki, gitea). See docs/plans/longhorn-ssd-tiering.md. Kept last so bpg won't
-   # reorder virtio0/1/2. discard=on so ext4/Longhorn TRIM reclaims thin zvol space.
-   disk {
-     datastore_id = var.longhorn_ssd_datastore
-     interface    = "virtio3"
-     size         = var.longhorn_ssd_disk_gib
-     discard      = "on"
-     cache        = "none"
-     iothread     = false
-   }
-   ```
-3. `tofu plan` **must** show an in-place *disk add* (update) for each node — **not** a
-   replacement. If it shows replace, stop (never destroy a running etcd node).
-4. Apply **one node at a time** with `-target` on the name-keyed resource address,
-   e.g. `-target='proxmox_virtual_environment_vm.node["k3s-dev-3"]'`. A disk add is
-   applied online by bpg, but treat the node as maintenance-affected regardless.
-
-### Phase 2 — Format + mount the SSD disk (ansible `node_baseline`, mirrors etcd)
-
-`roles/node_baseline` already formats/mounts the HDD Longhorn disk (`/dev/vdb` →
-`/var/lib/longhorn`) and the etcd disk (`/dev/vdc`). Add the same idempotent,
-data-safe pattern for `/dev/vdd`:
-
-1. New defaults in `roles/node_baseline/defaults/main.yml`:
-   ```yaml
-   node_baseline_longhorn_ssd_device: /dev/vdd
-   node_baseline_longhorn_ssd_mount: /var/lib/longhorn-ssd
-   node_baseline_longhorn_ssd_fs_label: longhorn-ssd
-   ```
-2. New tasks in `roles/node_baseline/tasks/main.yml` (copy the HDD Longhorn
-   filesystem+mount pair; `community.general.filesystem` never reformats an existing
-   fs, so re-runs are safe):
-   ```yaml
-   - name: Create ext4 filesystem on the SSD Longhorn disk (data-safe; skips if present)
-     community.general.filesystem:
-       fstype: ext4
-       dev: "{{ node_baseline_longhorn_ssd_device }}"
-       opts: "-L {{ node_baseline_longhorn_ssd_fs_label }}"
-   - name: Mount the SSD Longhorn disk
-     ansible.posix.mount:
-       path: "{{ node_baseline_longhorn_ssd_mount }}"
-       src: "LABEL={{ node_baseline_longhorn_ssd_fs_label }}"
-       fstype: ext4
-       opts: "defaults,nofail,x-systemd.device-timeout=5s"
-       state: mounted
-   ```
-   Use `LABEL=` (not `/dev/vdd`) so the fstab entry survives device renames, and
-   `nofail` so a missing SSD Longhorn disk does not block boot (unlike etcd, an absent
-   Longhorn extra-disk should not wedge the node).
-
-### Phase 3 — Register + tag the disks in Longhorn
-
-Per node (via the Longhorn `nodes.longhorn.io` CR or UI):
-
-1. **Tag the existing HDD disk** `default-disk` with `hdd` and set the new
-   `longhorn-ssd` StorageClass's counterpart default class to select it (so cold
-   volumes stay put and nothing hot accidentally lands on HDD once the SSD class
-   exists).
-2. **Add the SSD disk** at `/var/lib/longhorn-ssd` as a new Longhorn disk on each node,
-   `allowScheduling: true`, tag `ssd`.
-   ```yaml
-   # patch each nodes.longhorn.io/<node> .spec.disks
-   disk-ssd:
-     path: /var/lib/longhorn-ssd
-     allowScheduling: true
-     tags: ["ssd"]
-     storageReserved: 0
-   ```
-3. Confirm each node now reports two disks with correct `storageMaximum` (~48 GiB for
-   the SSD one) and `Schedulable` condition true.
-
-### Phase 4 — Publish the `longhorn-ssd` StorageClass
-
-New GitOps manifest (wherever the Longhorn StorageClasses are declared in
-`platform/` — mirror the existing `longhorn` class and add the selector):
-
-```yaml
-apiVersion: storage.k8s.io/v1
-kind: StorageClass
-metadata:
-  name: longhorn-ssd
-provisioner: driver.longhorn.io
-allowVolumeExpansion: true
-parameters:
-  numberOfReplicas: "3"
-  staleReplicaTimeout: "30"
-  fsType: ext4
-  dataLocality: disabled
-  diskSelector: "ssd"          # only schedules replicas on ssd-tagged disks
-  disableRevisionCounter: "true"
-  unmapMarkSnapChainRemoved: "ignored"
-```
-
-Optionally add a matching `diskSelector: "hdd"` to the existing default `longhorn`
-class so newly-created cold volumes stay on HDD.
-
-### Phase 5 — Migrate the hot volumes (no Longhorn backup target → in-cluster methods)
-
-Because no backup target exists (§1.4), use one of these, **one volume at a time**,
-respecting `concurrent-replica-rebuild-per-node-limit: 1`:
-
-- **CNPG DBs (keycloak / nextcloud / semaphore) — cleanest via the operator.** Change
-  the CNPG `Cluster` `.spec.storage.storageClass` to `longhorn-ssd` and let CNPG do a
-  rolling replica re-creation (it provisions a fresh instance on the new class, streams
-  from the primary, then promotes) — no manual data copy, no downtime beyond a
-  switchover. Do one CNPG cluster at a time; verify replication caught up before the
-  next.
-- **Prometheus / Loki / gitea — per-volume `diskSelector` rebuild (in place, no PVC
-  swap).** Patch the existing Longhorn volume to select the SSD disk and let Longhorn
-  rebuild the replicas onto it, then drop the HDD replicas:
-  ```
-  kubectl -n longhorn-system patch volumes.longhorn.io <vol> --type merge \
-    -p '{"spec":{"diskSelector":["ssd"]}}'
-  ```
-  Then trigger a controlled rebuild by deleting **one** HDD replica at a time (Longhorn
-  reschedules onto an `ssd`-tagged disk; `replica-auto-balance: best-effort` helps).
-  Watch the volume return to `robustness: healthy` before touching the next replica or
-  the next volume. This avoids recreating PVCs and keeps the app running.
-  - Alternative if the in-place rebuild misbehaves: scale the app to 0, CSI-**clone**
-    the PVC onto `longhorn-ssd` (`dataSource` = existing PVC), repoint the workload to
-    the clone, scale back up. More disruptive; keep as fallback.
-
-### Phase 6 — Validate
-
-1. All hot volumes `robustness: healthy`, replicas only on `ssd`-tagged disks
-   (`kubectl -n longhorn-system get replicas.longhorn.io -o wide`).
-2. SSD Longhorn disk usage ≈ 17 GiB/node; HDD disk down to the cold ~9 GiB/node.
-3. Deliberately reboot **one** node and confirm the rebuild storm is materially smaller
-   (fewer volumes, SSD-fast) and Proxmox IO-delay stays low. Confirm etcd fsync p99 did
-   not regress during the SSD rebuild (§3.4).
-
----
-
-## 5. Rebuild-storm mitigation (built into the plan)
-
-- **One node at a time** for every phase (tofu `-target`, ansible `--limit`, one CNPG
-  cluster / one replica at a time) — etcd quorum is 3-node, never take two down.
-- **`concurrent-replica-rebuild-per-node-limit: 1`** already set — a rebooted node
-  rebuilds serially, capping IO-delay.
-- **Migrate during a quiet window**, never while another rebuild is in flight.
-- **Lower `storage-over-provisioning-percentage`** *for the small SSD disk sizing* —
-  200 % on a 48 GiB disk would let Longhorn schedule ~96 GiB of nominal claims onto
-  ~17 GiB of real data; keep the hot set well under `storageMaximum`, and rely on
-  `storage-minimal-available-percentage: 25` as the floor. (The setting is global —
-  consider dropping it to ~120–150 % cluster-wide, or simply keep the SSD disk sized so
-  200 % of the hot claims still fits.)
-- **Verify etcd fsync latency** during the first SSD rebuild before proceeding (§3.4).
-
----
-
-## 6. Risks and rollback
-
-| Risk | Likelihood | Mitigation / rollback |
-|---|---|---|
-| SSD rebuild I/O re-starves etcd on shared `rpool` | Low (SSD IOPS) | rebuild-limit 1; watch `etcd_disk_wal_fsync_duration`; stop after node 1 if p99 regresses |
-| SSD pool fills (host root 430 G + etcd + Longhorn on 888 G) | Low (hot set ~51 GB) | keep SSD disk small (48 GiB), `storage-minimal-available-percentage: 25`; re-check `zpool list` between nodes |
-| tofu shows *replace* instead of disk-add | Medium if module drift | inspect `tofu plan` per node; abort on replace — never destroy a running etcd node |
-| Volume-migration data loss (no backup target) | Low | CNPG native rolling; per-volume rebuild keeps HDD replica until SSD healthy; CNPG base backup + reproducible data taken in Phase 0 |
-| Data disk device rename (`/dev/vdd` shifts) | Low | mount by `LABEL=longhorn-ssd`, not device path |
-| HDD actually failing mid-migration | Unknown (SMART unread) | 3× replication + mirror; Phase 0 SMART check; hot data lands on SSD first, reducing exposure |
-
-**Rollback:** every phase is reversible. To back out: migrate the hot volumes back to
-the HDD `default-disk` (reverse the `diskSelector` patch / point CNPG back at
-`longhorn`), remove the `disk-ssd` from each Longhorn node, delete the `longhorn-ssd`
-StorageClass, then remove the `virtio3` disk block from tofu and re-apply one node at a
-time. The SSD zvols are thin and independent of the HDD data, so removal is clean.
-
----
-
-## 7. Recommendation
-
-**Adopt Option A (hot-tier move), phased, one node at a time.** The recorded "SSD
-doesn't fit Longhorn" blocker was based on the 384 GiB *provisioned* figure; the
-**actual working set is ~79 GB cluster-wide (~26 GB/node)** because all bulk media
-lives on NFS and every Longhorn volume is a small config/DB volume. The hot tier
-(Prometheus + the three CNPG Postgres DBs + Loki + gitea) is **~51 GB actual /
-114 GiB provisioned** across three nodes and fits the ~323 GiB SSD `AVAIL` with wide
-headroom, while capturing the single largest and churniest rebuild contributor
-(Prometheus, 44 % of the working set) and the latency-sensitive databases.
-
-A **full move is feasible on space** and would fully eliminate HDD Longhorn rebuilds,
-but it re-couples all rebuild I/O with etcd + host root on the only SSD pool, partially
-undoing the just-completed etcd isolation — hold it as an optional Phase 2 gated on the
-hot-tier move proving insufficient. **Option C (wait for mini-PCs) is not a solution**
-tonight — no committed date, and the tagging + `longhorn-ssd` class built here carry
-over to that migration anyway.
-
-**Do this first, before any apply:** re-verify live SSD free space (`zpool list
-rpool`, `pvesm status`) and read HDD SMART (`smartctl -a`, `zpool status -x`). If the
-Seagate/`Storage` pool shows a degraded member, the urgency rises and the hot-tier move
-should be scheduled sooner — but the mirror + 3× Longhorn replication mean the plan is
-not gated on the disk's health.
-
-> This document is a plan only. No storage, tofu, Longhorn, StorageClass, or live
-> infrastructure has been modified.

docs/plans/ssd-etcd-execution-runbook.md

@@ -1,104 +0,0 @@
-# Phase 2 execution runbook — SSD-backed etcd migration (interactive/manual)
-
-> ## ✅ COMPLETED / historical — executed 2026-08-30, see git log
->
-> **This runbook has been executed.** Every k3s node's etcd now lives on its dedicated SSD `virtio2`
-> disk. The command sequence below is retained as the record of how the migration was performed — do
-> NOT re-run it against the live cluster. Current state is tracked in `infra/PHASE1-STATUS.md` and the
-> 2026-08-30 DECISIONS-LOG entry.
-
-Companion to `ssd-etcd-isolation.md`. This is the **exact command sequence** to migrate
-each k3s node's etcd onto its dedicated SSD `virtio2` disk. It must be run where live-infra
-mutations are permitted — an **interactive `claude` session** (default permission mode, which
-prompts for approval) or **by hand**. The autonomous "auto mode" session hard-denies
-`tofu apply` and node-level ssh mutations, so it cannot run this.
-
-## Preconditions (verified 2026-08-29, cluster healthy)
-- Run everything from the staged worktree that carries the tofu disk block:
-  `cd /home/jakman85/source/home-network-k3s-impl/.claude/worktrees/agent-ssd-etcd`
-  (branch `feature/ssd-etcd-isolation` @ `af5fc5b`).
-- All tooling via the devcontainer: `./infra/devcontainer/dev.sh run <cmd>`.
-- `./infra/devcontainer/dev.sh run ./infra/tofu/tf.sh init` once (done).
-- Ansible must run from `infra/ansible/` (for the vault-pass path): wrap as
-  `./infra/devcontainer/dev.sh run bash -lc 'cd infra/ansible && ansible ...'`.
-- **Run mutation commands STANDALONE** (not chained with `&&`/pipes) — the classifier is
-  more likely to deny a compound command that embeds a mutation.
-
-## ORDER (leader last — RE-VERIFY before each node)
-Current leader = **dev-3**, so: **dev-2 (vmid 102) → dev-1 (vmid 105) → dev-3 (vmid 101)**.
-dev-3 is last and additionally gets the memory 16384→24576 bump (needs a full stop/start).
-Re-check the leader before each node and keep the leader last:
-```
-./infra/devcontainer/dev.sh run bash -lc 'cd infra/ansible && ansible k3s_servers -m shell -a "curl -s http://127.0.0.1:2381/metrics | grep etcd_server_is_leader"'
-```
-
-## Pre-flight before EACH node (read-only — all must pass)
-```
-./infra/devcontainer/dev.sh run kubectl get nodes
-./infra/devcontainer/dev.sh run kubectl get --raw=/readyz/etcd            # expect: ok
-./infra/devcontainer/dev.sh run kubectl -n longhorn-system get volumes.longhorn.io -o custom-columns=R:.status.robustness --no-headers | grep -c healthy   # expect: 18
-```
-
-## Per-node procedure (substitute NODE=k3s-dev-N, VMID)
-
-### 1. Freeze rebuilds (STANDALONE)
-```
-./infra/devcontainer/dev.sh run kubectl -n longhorn-system patch settings.longhorn.io concurrent-replica-rebuild-per-node-limit --type=merge -p '{"value":"0"}'
-```
-
-### 2. Add the SSD disk via tofu (STANDALONE). dev-2/dev-1 = disk only; dev-3 also memory.
-```
-./infra/devcontainer/dev.sh run ./infra/tofu/tf.sh plan  '-target=proxmox_virtual_environment_vm.node["k3s-dev-N"]'   # confirm: disk ADD, in-place, NOT replace
-./infra/devcontainer/dev.sh run ./infra/tofu/tf.sh apply -auto-approve '-target=proxmox_virtual_environment_vm.node["k3s-dev-N"]'
-```
-Confirm the guest sees the new disk (hot-plug):
-```
-./infra/devcontainer/dev.sh run bash -lc 'cd infra/ansible && ansible k3s-dev-N -m shell -a "lsblk /dev/vdc"'
-```
-If `/dev/vdc` is NOT present (no hot-plug), power-cycle the VM (freeze is holding):
-```
-./infra/devcontainer/dev.sh run bash -lc 'cd infra/ansible && ansible proxmox -m shell -a "qm stop VMID"'   # wait until stopped
-./infra/devcontainer/dev.sh run bash -lc 'cd infra/ansible && ansible proxmox -m shell -a "qm start VMID"'
-```
-(For dev-3 the memory bump requires this stop/start regardless.)
-
-### 3. Migrate etcd data (node commands, become=root). Do NOT skip the copy-before-mount order.
-```
-./infra/devcontainer/dev.sh run bash -lc 'cd infra/ansible && ansible k3s-dev-N -b -m shell -a "systemctl stop k3s"'
-# mkfs ONLY if /dev/vdc has no filesystem yet (fresh disk):
-./infra/devcontainer/dev.sh run bash -lc 'cd infra/ansible && ansible k3s-dev-N -b -m shell -a "blkid /dev/vdc || mkfs.ext4 -L k3s-etcd /dev/vdc"'
-./infra/devcontainer/dev.sh run bash -lc 'cd infra/ansible && ansible k3s-dev-N -b -m shell -a "mv /var/lib/rancher/k3s/server/db /var/lib/rancher/k3s/server/db.bak && mkdir /var/lib/rancher/k3s/server/db && mount LABEL=k3s-etcd /var/lib/rancher/k3s/server/db && rsync -aHAX --numeric-ids /var/lib/rancher/k3s/server/db.bak/ /var/lib/rancher/k3s/server/db/"'
-# Persist the mount (matches the node_baseline role: LABEL + defaults,noatime):
-./infra/devcontainer/dev.sh run bash -lc 'cd infra/ansible && ansible k3s-dev-N -b -m ansible.posix.mount -a "path=/var/lib/rancher/k3s/server/db src=LABEL=k3s-etcd fstype=ext4 opts=defaults,noatime dump=0 passno=2 state=mounted"'
-./infra/devcontainer/dev.sh run bash -lc 'cd infra/ansible && ansible k3s-dev-N -b -m shell -a "systemctl start k3s"'
-```
-
-### 4. VERIFY full recovery before touching the next node (all must pass)
-```
-./infra/devcontainer/dev.sh run kubectl get nodes                          # NODE Ready
-./infra/devcontainer/dev.sh run kubectl get --raw=/readyz/etcd             # ok
-./infra/devcontainer/dev.sh run bash -lc 'cd infra/ansible && ansible k3s-dev-N -m shell -a "findmnt /var/lib/rancher/k3s/server/db"'   # source = /dev/vdc
-./infra/devcontainer/dev.sh run bash -lc 'cd infra/ansible && ansible k3s_servers -m shell -a "curl -s http://127.0.0.1:2381/metrics | grep etcd_server_has_leader"'  # all =1
-./infra/devcontainer/dev.sh run kubectl -n longhorn-system get volumes.longhorn.io -o custom-columns=R:.status.robustness --no-headers | grep -c healthy   # 18
-# dev-3 only: confirm ~24Gi allocatable
-./infra/devcontainer/dev.sh run kubectl get node k3s-dev-3 -o jsonpath='{.status.allocatable.memory}'
-```
-
-### 5. Thaw + clean up (STANDALONE), only after step 4 fully passes
-```
-./infra/devcontainer/dev.sh run kubectl -n longhorn-system patch settings.longhorn.io concurrent-replica-rebuild-per-node-limit --type=merge -p '{"value":"1"}'
-./infra/devcontainer/dev.sh run bash -lc 'cd infra/ansible && ansible k3s-dev-N -b -m shell -a "rm -rf /var/lib/rancher/k3s/server/db.bak"'
-```
-Wait for 18/18 healthy (no rebuild storm), re-verify leader, then proceed to the next node.
-
-## STOP conditions (halt immediately, leave limit=0, do not advance)
-etcd `/readyz` not ok, a member without a leader, VIP `192.168.2.50:6443` flapping, a node
-not rejoining, any Longhorn volume Faulting, or k3s crashlooping. Recovery: keep rebuilds
-frozen; a botched single member can be reset and re-joined from the other two (plan §4.5).
-
-## Close-out (after all 3)
-- All 3 nodes Ready; etcd 3/3 with a leader; `findmnt` shows `/dev/vdc` at the db path on each.
-- 18/18 Longhorn volumes healthy; rebuild-limit back to **1**.
-- `./infra/devcontainer/dev.sh run ./infra/tofu/tf.sh plan` → **No changes** (all disks present, dev-3 memory reconciled).
-- Merge `feature/ssd-etcd-isolation` → `main` and push (push is classifier-blocked in auto mode — run `git push gitea main` by hand) so Flux/tofu source records the disk + baseline.
-- Regression test: after approval, bounce ONE node and confirm etcd stays healthy / VIP steady — the whole point of the change.

docs/plans/ssd-etcd-isolation.md

@@ -1,277 +0,0 @@
-# SSD-backed etcd isolation for the k3s dev cluster
-
-> ## ✅ COMPLETED / historical — executed 2026-08-30, see git log
->
-> **This proposal was approved and carried out.** Each node now has a dedicated `virtio2` SSD etcd
-> disk (`/dev/vdc`, `local-zfs`) hosting the etcd data dir. The **Status: PROPOSAL** line below and
-> the sizing/RAM figures are the original investigation record — retained for the reasoning, not as a
-> live to-do. Current state is tracked in `infra/PHASE1-STATUS.md` and the 2026-08-30 DECISIONS-LOG entry.
-
-**Status:** PROPOSAL — awaiting user approval. No changes have been made.
-**Author:** investigation agent (read-only), 2026-08-29
-**Problem:** Longhorn replica-rebuild I/O storms on the HDD-backed VM disks starve
-etcd's WAL fsync, crash-looping k3s and flapping the kube-vip VIP
-(`192.168.2.50:6443`). Even `concurrent-replica-rebuild-per-node-limit=1` tips etcd.
-**Question:** Can the Proxmox host's ~1 TB system SSD (`rpool`) host dedicated
-SSD-backed storage for etcd (and/or Longhorn) to end these incidents?
-
-**Verdict: YES for etcd (strongly recommended). NO for Longhorn.**
-
----
-
-## 1. Measured facts (all collected read-only on 2026-08-29)
-
-### 1.1 `rpool` — the system/boot SSD pool
-
-| Property | Value |
-|---|---|
-| Pool | `rpool`, **mirror** (`mirror-0`) of 2 devices |
-| Devices | `ata-MTFDDAK960TDS_...` × 2 — **Micron 5100/52x0 960 GB, 2.5" SATA** |
-| Class | **Enterprise SATA SSD with power-loss protection (PLP)** — safe for sync/fsync |
-| Health | `Percent_Lifetime_Remain` = **97%** (3% used), 13 904 power-on hours, 0 reallocated sectors, ~96 TB written. Excellent. |
-| Pool size / alloc / **free** | 888 G / 538 G / **350 G free** (`zpool list`, 60% cap) |
-| ZFS `AVAIL` | **~323 GiB** available to any new dataset/zvol |
-| System root | `rpool/ROOT/pve-1` = **430 G used** (Proxmox host `/`) |
-| Proxmox storage on it | `local-zfs` → pool `rpool/data`, `content images,rootdir`, **`sparse 1`** (thin), ~338 GiB available per `pvesm status` |
-| Pool settings | `compression on`, **`sync standard`** (inherited), `dedup off`, recordsize 128K |
-
-`local-zfs` already advertises `images,rootdir` content, so **Proxmox can place VM
-disks on the SSD today with no storage-config change.**
-
-### 1.2 Current VM disk backing — confirms the HDD assumption
-
-VMs (bpg/OpenTofu-provisioned): **dev-1 = vmid 105, dev-2 = 102, dev-3 = 101**.
-Each has two disks, **both on the `Storage` datastore**:
-
-- `Storage` = ZFS pool on **2× WDC WUH721818ALE6L4 18 TB HDD** (`sdc`/`sdd`, `ROTA=1`), mirror, 16.4 T.
-- Per node: `virtio0 = vm-<id>-disk-0` (64 G, OS/root) and `virtio1 = vm-<id>-disk-1` (128 G, Longhorn data). `volblocksize 16K`.
-
-So **etcd, the OS, and Longhorn all currently sit on spinning rust.** That is the
-root cause: etcd WAL fsync competes with Longhorn rebuild I/O on the same HDD spindles.
-
-(`Storage2` = 2× Seagate ST8000VN004 8 TB HDD is a separate HDD pool, not involved.)
-
-### 1.3 Size of the hot data
-
-- **etcd DB: ~82.6 MB.** Measured via apiserver metric
-  `apiserver_storage_size_bytes{storage_cluster_id="etcd-0"} = 8.264e7`.
-  A 4-day-old cluster running ~18 services. Even allowing generous WAL + growth
-  headroom, **etcd fits in single-digit GB with room to spare — trivially ×3.**
-- **Longhorn: ~22 GB actual data per node right now**, but the data disk is
-  **128 GiB** and Longhorn reports `storageScheduled ≈ 152 GiB` (thin-overcommitted)
-  vs `storageMaximum ≈ 125 GiB` per node. 18 PVCs, `numberOfReplicas=3`.
-  Provisioned total ≈ 152 GiB. Longhorn is the **churn source we want to isolate
-  etcd from**, and it can grow toward 128 GiB/node.
-
-### 1.4 Sizing conclusion
-
-| Candidate | Per-node footprint | ×3 on rpool | Fits in 323 GiB avail? |
-|---|---|---|---|
-| **etcd** | ~83 MB now; size a 16 GiB disk | ~48 GiB reserved (thin) | **Yes, easily** |
-| **Longhorn** | ~22 GB now, up to 128 GiB provisioned | up to **~384 GiB** | **No — would fill/threaten the boot disk** |
-
----
-
-## 2. Viability verdict
-
-**etcd on SSD: viable and high-leverage.** etcd is ~83 MB; three 16 GiB SSD-backed
-disks cost ~48 GiB of thin allocation (actual ~250 MB) against 323 GiB free on an
-enterprise, PLP-equipped, 97%-healthy SSD mirror. This directly removes the WAL
-fsync path from the HDD contention domain — the smallest-footprint fix for the exact
-failure mode. **This is the recommendation.**
-
-**Longhorn on SSD: not viable here.** Its provisioned footprint (~384 GiB across 3
-replicas at full size) does not fit alongside the 430 GiB Proxmox root on a 888 GiB
-pool, and co-locating heavy Longhorn rebuild I/O on the boot SSD reintroduces
-contention with the host itself. Keep Longhorn on HDD (`Storage`). Rejected.
-
----
-
-## 3. Options evaluated
-
-- **Option A — dedicated small SSD disk per node, etcd-only (RECOMMENDED).**
-  Add a ~16 GiB SSD-backed virtual disk (zvol on `rpool/data` via `local-zfs`) to
-  each of the 3 VMs, mount it at the k3s etcd directory
-  (`/var/lib/rancher/k3s/server/db`), migrate etcd data onto it one node at a time.
-  Isolates WAL fsync from HDD Longhorn churn. Minimal space, minimal blast radius.
-- **Option B — Longhorn replicas/disk on SSD.** Rejected (§2): does not fit and
-  re-couples host I/O.
-- **Option C — do nothing; wait for the SSD mini-PCs.** Zero risk, zero work, but
-  leaves the cluster fragile to every node bounce until the mini-PC migration lands
-  (deferred, no committed date). Option A is a cheap, reversible bridge that Option C
-  eventually supersedes (§7).
-
-**Recommendation: Option A now, as an interim fix, carried by tofu.**
-
----
-
-## 4. Recommended implementation — Option A, step-by-step
-
-### 4.1 Model it in tofu (preferred over manual)
-
-The VMs are bpg/OpenTofu-managed (`infra/tofu/`). Disks are declared as `disk`
-blocks in `main.tf`; both current disks (`virtio0`, `virtio1`) point at
-`var.datastore` (`Storage`). Add a **third disk on the SSD datastore**:
-
-1. **New variables** in `infra/tofu/variables.tf`:
-   ```hcl
-   variable "etcd_ssd_datastore" {
-     description = "SSD-backed datastore for the dedicated etcd disk (VERIFIED: local-zfs = rpool/data, ZFS mirror on Micron 5100 SATA SSDs w/ PLP)."
-     type        = string
-     default     = "local-zfs"
-   }
-   variable "etcd_disk_gib" {
-     description = "Dedicated SSD etcd disk per node, GiB (etcd DB is ~83 MB; 16 GiB is generous headroom)."
-     type        = number
-     default     = 16
-   }
-   ```
-2. **New disk block** in the VM resource in `main.tf`, appended AFTER the existing
-   `virtio1` block (keep interface order ascending so bpg does not reorder):
-   ```hcl
-   # Dedicated SSD etcd disk (virtio2 -> /dev/vdc). Isolates etcd WAL fsync from
-   # the HDD Storage pool where the OS + Longhorn disks live.
-   disk {
-     datastore_id = var.etcd_ssd_datastore
-     interface    = "virtio2"
-     size         = var.etcd_disk_gib
-     discard      = "on"
-     cache        = "none"   # honor guest fsync through to ZFS ZIL (PLP SSD = safe+fast)
-     iothread     = false
-   }
-   ```
-3. **Plan and inspect before applying.** `tofu plan` MUST show an **in-place update
-   (disk add)** for the targeted VM — NOT a replacement. If it shows replace, stop and
-   reassess (do not destroy a running etcd node).
-4. **Apply one node at a time** with `-target` (see §4.3 for quorum ordering), e.g.
-   `-target='proxmox_virtual_environment_vm.node["k3s-dev-3"]'` (adjust to the actual
-   resource address in this repo). A disk add is applied online by bpg, but treat the
-   node as going down for the etcd migration regardless.
-
-> **Coupling caveat — dev-3 memory drift.** dev-1/dev-2 are at `memory_mib = 24576`
-> but **dev-3 is live at 16384** (drift from tofu default 24576). A tofu apply that
-> touches dev-3 will also try to reset its memory to 24576. Per the Proxmox rule,
-> memory changes need a **full stop/start**, not a reboot. Decide deliberately:
-> either (a) let the same maintenance window fix the memory drift on dev-3 (stop →
-> set → start), or (b) `-target` narrowly and accept that dev-3's plan will include
-> the memory change. Do dev-3 in its own window with eyes on it.
-
-**Manual alternative (if not using tofu):** on the Proxmox host,
-`qm set <vmid> -virtio2 local-zfs:16,discard=on,cache=none` then stop/start the VM.
-This drifts from tofu state and is discouraged — if done, import/reconcile into tofu
-afterward. Prefer the tofu path.
-
-### 4.2 Optional space guarantee
-
-`local-zfs` is `sparse` (thin), so the etcd zvols consume only real usage. To
-guarantee etcd can never be squeezed if the system disk fills, set a modest
-`refreservation` on each etcd zvol (e.g. `zfs set refreservation=16G
-rpool/data/vm-<id>-disk-2`). 3×16 GiB = 48 GiB reserved out of 323 GiB avail — safe.
-This is a host-side ZFS tweak, not expressible in bpg; apply manually if desired.
-
-### 4.3 Migrate etcd onto the new disk — quorum-safe, ONE node at a time
-
-k3s has no flag to relocate only the etcd directory (`--data-dir` moves everything),
-so the clean approach is to **mount the SSD disk at
-`/var/lib/rancher/k3s/server/db`** (this directory holds `etcd/` — member data, WAL,
-snapshots — and nothing else that matters).
-
-**Pre-flight:** confirm all 3 etcd members healthy and the cluster is quiet (no active
-Longhorn rebuild). `kubectl get nodes` all Ready; check etcd member health.
-
-**Per node, repeat for dev-3, then dev-2, then dev-1 (leader last):**
-
-1. Verify 3/3 etcd members healthy BEFORE touching this node (losing one of three
-   keeps quorum at 2/3; never take a second node until the previous is fully back).
-2. Stop k3s on the node: `systemctl stop k3s`. (Node leaves quorum; 2/3 remain.)
-3. Identify the new disk (`/dev/vdc` — the third virtio disk) and format it:
-   `mkfs.ext4 -L k3s-etcd /dev/vdc` (ext4 or xfs; ext4 is fine and simple).
-4. Preserve and move existing data:
-   ```
-   mv /var/lib/rancher/k3s/server/db /var/lib/rancher/k3s/server/db.bak
-   mkdir /var/lib/rancher/k3s/server/db
-   mount /dev/vdc /var/lib/rancher/k3s/server/db
-   rsync -aHAX --numeric-ids /var/lib/rancher/k3s/server/db.bak/ /var/lib/rancher/k3s/server/db/
-   ```
-5. Persist the mount in `/etc/fstab` **by UUID** (`blkid /dev/vdc`), e.g.
-   `UUID=<uuid> /var/lib/rancher/k3s/server/db ext4 defaults,noatime 0 2`, then
-   `systemctl daemon-reload`.
-6. Start k3s: `systemctl start k3s`. Wait until the node is `Ready` AND its etcd
-   member is healthy and has re-joined (member list shows 3 started, no learner
-   stuck). Confirm the VIP `192.168.2.50:6443` is stable.
-7. Only after full recovery, remove the backup: `rm -rf
-   /var/lib/rancher/k3s/server/db.bak`. Then proceed to the next node.
-
-**Reproducibility for rebuilds:** the SSD disk + mount should also be baked into the
-node **cloud-init first-boot baseline** (the same template that already provisions the
-Longhorn `virtio1` disk and swap-off), so a `tofu`-rebuilt node comes up with
-`/dev/vdc` formatted and mounted at the etcd path automatically. Update that template
-in the same change set as the tofu disk block.
-
-### 4.4 Validation
-
-- `kubectl get nodes` — all 3 Ready; no flapping over a 30–60 min soak.
-- etcd: 3 members, all started/healthy, one leader; DB size sane (~tens–hundreds MB).
-- On each node, `df -h /var/lib/rancher/k3s/server/db` shows the new `/dev/vdc` mount.
-- **Regression test the original failure:** with the cluster healthy, trigger a
-  controlled Longhorn replica rebuild (e.g. bounce ONE node after approval) and
-  confirm etcd stays healthy and the VIP does not flap — the whole point of the change.
-- Host-side: `zpool status rpool` clean; `zfs list rpool/data` shows the 3 new
-  `vm-*-disk-2` zvols with small real usage.
-
-### 4.5 Rollback
-
-Per node, fully reversible while `db.bak` still exists (before step 7):
-`systemctl stop k3s` → `umount .../server/db` → `rmdir` the empty mountpoint →
-`mv db.bak db` → remove the fstab line → `systemctl start k3s`. To fully revert the
-disk, detach `virtio2` (`qm set <vmid> -delete virtio2` after stop) or `tofu` revert
-the disk block, then stop/start. Because etcd is a replicated 3-member cluster, a
-botched single node can also simply be reset and re-joined from the other two.
-
----
-
-## 5. Risks
-
-1. **System-disk contention / exhaustion (top risk).** `rpool` is also the Proxmox
-   boot/root disk (430 GiB used of 888 GiB). Mitigations: keep the etcd disks tiny
-   (16 GiB, thin), optional `refreservation` (§4.2), and **never** put Longhorn here
-   (§2). etcd's steady-state real usage is ~hundreds of MB; the marginal I/O it adds
-   to the SSD is negligible next to what it removes from the HDD.
-2. **Consumer-SSD fsync/PLP caveat — N/A here, but assert it.** etcd correctness under
-   power loss depends on honest fsync. These are **Micron 5100/52x0 enterprise SATA
-   SSDs with power-loss protection**, so `sync=standard` + `cache=none` is both fast
-   and durable. (If the disks were consumer SSDs without PLP this concern would be
-   real — it is not, for this hardware.)
-3. **Quorum during migration.** Mitigated by strict one-node-at-a-time with full
-   recovery gates (§4.3) and leader-last ordering. Never take a second node down
-   before the previous rejoins.
-4. **tofu coupling with dev-3 memory drift + apply semantics (§4.1).** An apply
-   touching dev-3 also resets memory 16384→24576 (needs stop/start). Verify `tofu
-   plan` shows disk *add*, not VM *replace*, before applying. Use `-target`.
-5. **Backup/restore assumptions.** k3s etcd snapshots (`/var/lib/rancher/k3s/server/db/snapshots`)
-   now live on the SSD too — fine, but confirm any snapshot/backup job still points at
-   the (unchanged) path.
-
----
-
-## 6. Recommendation summary
-
-Implement **Option A**: a 16 GiB SSD-backed `virtio2` disk per k3s VM (zvol on
-`rpool/data` via `local-zfs`), etcd-only, migrated one node at a time with the
-quorum-safe procedure in §4.3, modeled in tofu (+ cloud-init baseline) for
-reproducibility. Do **not** move Longhorn to SSD. This is a small, reversible,
-high-leverage fix targeting the exact WAL-starvation failure.
-
----
-
-## 7. Relationship to the deferred SSD-mini-PC migration
-
-This is explicitly an **interim bridge** on the current Proxmox dev host
-(`192.168.2.102`), whose bulk storage is HDD. Once the k3s cluster migrates to the
-planned **all-SSD mini-PCs**, every I/O path (OS, etcd, Longhorn) is already on SSD
-and the HDD-contention failure mode disappears — the dedicated etcd disk becomes
-unnecessary. At that point the isolation can be either dropped or carried forward
-harmlessly (a dedicated etcd disk is still good hygiene). Because Option A is captured
-in tofu + cloud-init, it travels with the cluster definition and can be removed in one
-reviewed change when the mini-PCs land. Until then it stops the recurring incidents
-at minimal cost.

infra/PHASE1-STATUS.md

@@ -1,19 +1,9 @@
 # Dev cluster — current state & reconnect guide
 
-**Last verified: 2026-08-31.** Phase 1 is **complete and long since exited**; the cluster has been
+**Last verified: 2026-08-26.** Phase 1 is **complete and long since exited**; the cluster has been
 running the full platform (phases 2–6) for some time. This doc is how a session — human or agent —
 reconnects to it and re-establishes the toolchain.
 
-> **New PM / agent taking over?** Read this doc (reconnect + toolchain), then the **latest
-> `DECISIONS-LOG.md` entry (2026-08-31)** — it is the current-state + gotchas + prioritized **Next
-> Steps** handoff — and the plans under `docs/plans/`. Start at Next Steps.
-
-> **Phase 7 (prod) is OPEN as of 2026-08-31 — prep only.** The first prod node
-> (`k3s-prod-1`, `192.168.2.10`) is on the bench. This document covers the **dev** cluster;
-> for prod see [`PROD-PROVISIONING.md`](PROD-PROVISIONING.md) (T-7.1 checklist) and the
-> Phase-7 entry in `DECISIONS-LOG.md`. Nothing is installed on prod hardware until the
-> R-ENV-02 tokenization blockers close and the gate is green.
-
 > **History note.** The cluster was originally 3 libvirt/KVM VMs on the workstation
 > (`10.73.0.0/24` NAT). That was **retired** after the workstation ran out of RAM/disk; the cluster
 > now lives on **Proxmox VE**. Anything mentioning `virsh`, `qemu:///system`, `tofu`+`dmacvicar/libvirt`,
@@ -25,12 +15,12 @@ reconnects to it and re-establishes the toolchain.
 | | |
 |---|---|
 | Substrate | Proxmox VE host, node `proxmox`, disks on the `Storage` ZFS datastore, bridge `vmbr0` |
-| Nodes | `k3s-dev-1/2/3` at `192.168.1.50 / .51 / .52` — 8 vCPU, 24 GiB, Debian 13 (trixie) |
+| Nodes | `k3s-dev-1/2/3` at `192.168.1.50 / .51 / .52` — 6 vCPU, 16 GiB, Debian 12 (bookworm) |
-| k3s | **v1.36.4+k3s1**, 3 servers, embedded etcd (HA); no flannel / kube-proxy / Traefik |
+| k3s | **v1.31.5+k3s1**, 3 servers, embedded etcd (HA); no flannel / kube-proxy / Traefik |
-| CNI | Cilium 1.20.1 (kube-proxy replacement, L2 announcements, egress gateway) |
+| CNI | Cilium 1.16.5 (kube-proxy replacement, L2 announcements, egress gateway) |
 | kube-API | kube-vip VIP **`192.168.1.49:6443`**, tls-san'd on the serving cert |
 | Ingress VIP | **`192.168.1.53`** (shared Gateway). Pool `192.168.1.53–.60` for LB VIPs; UniFi holds `.54` |
-| Storage | Longhorn 1.12.1 — classes `longhorn` (default), `longhorn-static`, `longhorn-vm` (RWX/Block, migratable), `longhorn-scratch`. `bulk` (NFS CSI) is a **non-provisioning stub** pending OQ-2 |
+| Storage | Longhorn 1.7.3 — classes `longhorn` (default), `longhorn-static`, `longhorn-vm` (RWX/Block, migratable), `longhorn-scratch`. `bulk` (NFS CSI) is a **non-provisioning stub** pending OQ-2 |
 | Nested virt | `cpu_type = host` → real KVM in-guest; KubeVirt + CDI run on it |
 
 Everything above is provisioned by IaC: [`tofu/`](tofu/) creates the VMs,

infra/PROD-PROVISIONING.md

@@ -1,336 +0,0 @@
-# Prod bare-metal provisioning checklist (T-7.1)
-
-**Task:** T-7.1 · **Workstream:** WS-INFRA · **Serves:** R-ENV-01, R-ENV-02, R-ENV-03, R-PLAT-01/03
-**Status:** Phase 7 **opened by the operator 2026-08-31** — *prep only*. Nothing is installed on prod
-hardware until the blockers in §7 are closed and `make -C ci gate` is green.
-
-> **Entry-gate note (recorded, not waived).** `roadmap/phases.md` gates Phase 7 on *"Phases 1–6
-> proven in dev/test; operator declares prod hardware ready."* The operator has declared hardware
-> ready. The first condition is **not** met: conformance is 37/51, and the uncovered set includes
-> **R-ENV-01/02/03** — the very requirements this document serves. We proceed with prep work
-> knowingly; see `DECISIONS-LOG.md` (2026-08-31 Phase-7 entry).
-
-Prod is **3× mini-PC bare metal** ([spec docs/02](../../home-network-k3s/docs/02-environments-and-provisioning.md)).
-The dev path (OpenTofu → Proxmox VM → cloud-init) **does not apply**: there is no hypervisor API to
-call. Stage 1 "Create" is a human imaging a box. Stages 2–4 (baseline, cluster-join, manage) reuse
-the *same* Ansible roles and the *same* Flux base as dev — that reuse is the R-ENV-02 obligation.
-
----
-
-## 0. Node inventory
-
-Fill a row per node as it lands. Node 1 is as-found on 2026-08-31.
-
-| | node 1 | node 2 | node 3 |
-|---|---|---|---|
-| Hostname | `k3s-prod-1` | `k3s-prod-2` | `k3s-prod-3` |
-| LAN IP | `192.168.2.10` | `192.168.2.11` | `192.168.2.12` |
-| Model | HP EliteDesk 800 G5 Desktop Mini | | |
-| CPU | i7-9700T, 8c/8t, VT-x | | |
-| RAM | 31 GiB | | |
-| Disks | 1× 477 GB SATA SSD (`C400-MTFDDAC512M`) | | |
-| OS | Debian 13 (trixie), kernel 6.12 | | |
-| Firmware | HP R21 Ver. 02.26.00 | | |
-| `/dev/kvm` | present | | |
-
-Dev nodes for comparison: 8 vCPU / 24 GiB / Debian 13, **two** dedicated data disks (Longhorn + etcd).
-
----
-
-## 1. Firmware / BIOS (per node, before OS install)
-
-- [ ] **Virtualization enabled** — VT-x *and* VT-d. Required for KubeVirt (R-PLAT-03); verify after
-      boot with `/dev/kvm` present and `grep -cE 'vmx|svm' /proc/cpuinfo` non-zero.
-- [ ] **AC power recovery = Power On.** A homelab node must return after a power cut without a
-      human pressing the button. (`docs/02` calls out wake/boot behaviour explicitly.)
-- [ ] **Wake-on-LAN enabled** on the onboard NIC (remote recovery when the box is off).
-- [ ] **Boot order** — internal disk first; USB above it only while imaging. Disable network boot
-      unless PXE is adopted.
-- [ ] **Secure Boot** — record the setting. k3s itself is fine either way; note it so all three
-      nodes match.
-- [ ] **Firmware version recorded** in §0 and consistent across all three nodes.
-
-## 2. Disk layout — **OPEN DECISION, blocks the build**
-
-`docs/02` requires the OS-vs-Longhorn device split to be **documented before prod build**. Dev went
-further and added a *third* device for etcd after HDD contention starved etcd WAL fsync
-(`docs/plans/ssd-etcd-isolation.md`).
-
-Node 1 as-found is a **single** 477 GB SATA SSD carrying everything:
-
-```
-sda  477G  C400-MTFDDAC512M (SSD, ~2011-era Micron/Crucial)
-├─sda1 976M  /boot/efi
-├─sda2  30G  /
-├─sda3  12G  /var
-├─sda4 977M  swap
-└─sda5 433G  /srv          <- the only large free space
-```
-
-Consequences to settle before imaging nodes 2 and 3:
-
-- [x] **Drive health MEASURED 2026-09-01** (`smartctl -a /dev/sda`, via the baseline package
-      floor — see §5). Raw attributes, so the numbers are auditable:
-
-      | | |
-      |---|---|
-      | Model | Crucial/Micron RealSSD m4/C400/P400 (`C400-MTFDDAC512MAM`, fw `070H`) |
-      | SMART self-assessment | **PASSED** — no attribute in `WHEN_FAILED` |
-      | Power-on hours | 6,814 (~284 days) · 147 power cycles |
-      | **Rated life used** | **31%** (`202`), i.e. 69% remaining (`173` VALUE 069) |
-      | **Grown failing blocks** | **15** (`170`), reallocation events **15** (`196`) |
-      | **Erase failures** | **187** (`172`) |
-      | **Unexpected power losses** | **32** (`174`) |
-      | Pending / offline-uncorrectable / reported-uncorrectable | **0 / 0 / 0** (`197`/`198`/`187`) |
-      | UDMA CRC errors | 0 (`199`) — cabling is clean |
-
-      **Read attribute `5` carefully.** Its raw value is `61440`, which is NOT a bad-sector
-      count: `61440 = 15 x 4096`, and both `170` and `196` independently read **15**. Its
-      normalised VALUE is 100 against a THRESH of 10. So the drive has **15 grown bad blocks**,
-      not sixty-one thousand bad sectors. The first `node_health` run FAILED on that raw value;
-      the check now judges on normalised VALUE/THRESH, `WHEN_FAILED`, and the vendor-independent
-      uncorrectable counters instead.
-
-      **Verdict: usable, but not for production state.** Nothing says imminent failure — no
-      pending or uncorrectable sectors, SMART self-assessment passes. But this is a ~2011
-      consumer MLC drive that has already consumed **31% of its rated life**, grown **15** bad
-      blocks, logged **187** erase failures, and taken **32** unclean power losses **with no
-      power-loss protection**, in a single failure domain. etcd is the worst possible tenant for
-      it: etcd fsyncs constantly, and an unclean power loss on a drive without PLP is exactly how
-      a member corrupts — this drive has already logged 32 of them.
-      (Non-ECC RAM is a separate, **accepted** risk — operator decision 2026-09-01, see
-      `DECISIONS-LOG.md`. It is not part of this verdict.)
-- [ ] **Decide the layout.**- [ ] **Decide the layout.** `roles/node_baseline` expects *block devices*
-      (`node_baseline_longhorn_device`, `node_baseline_etcd_device`) and makes a filesystem on them.
-      Its defaults `/dev/vdb` and `/dev/vdc` are **virtio names that do not exist on bare metal** —
-      prod must override them in `group_vars`. On a single-disk node there is no second device at
-      all, so one of:
-      - **Add a second drive** (M.2 NVMe if the chassis has a free slot; verify physically — only a
-        SATA controller is populated today). Cleanest: matches dev's isolation, gives etcd its own
-        spindle.
-      - **Use `/dev/sda5`** (the 433 GB `/srv` partition) as the Longhorn device, etcd stays on `/`.
-        Buildable today, zero spend, but re-creates the shared-spindle contention dev was burned by.
-      - **Repartition** into dedicated OS / etcd / Longhorn partitions. Better isolation, still one
-        physical failure domain.
-- [ ] **All three nodes get the identical layout.** Longhorn replica placement assumes symmetry.
-- [ ] Record the chosen device paths here and set them in
-      `infra/ansible/inventory/group_vars/k3s_prod/vars.yml` (they are deliberately
-      `/dev/BLOCKED-pending-disk-decision-…` placeholders today — do **not** guess). A full
-      `node-baseline` run against a node still carrying those placeholders now stops on an explicit
-      assert naming this section, instead of failing obscurely at `mkfs` time.
-- [x] **The SMART-vs-smartmontools circular dependency is broken** (2026-08-31). `roles/node_baseline`
-      is tagged `packages` / `system` / `disks`, so the package floor can land **without** the disk
-      tasks this decision blocks — see §5's drive-health box for the exact command.
-
-## 3. OS install (per node)
-
-- [ ] **Debian 13 (trixie)**, matching dev exactly — same kernel series, same package baseline.
-- [ ] Hostname `k3s-prod-N`.
-- [ ] Partition per the §2 decision.
-- [ ] **No cloud-init needed.** Dev's first-boot work was deliberately moved into
-      `roles/node_baseline` when provisioning became API-only (`infra/cloud-init/` is a README now),
-      so the same role covers bare metal. Nothing to port.
-
-## 4. Network (per node)
-
-- [ ] **Static IP** on the prod LAN, recorded in §0. Node 1 is `192.168.2.10/24`, gateway
-      `192.168.2.1`.
-- [x] **Prod address plan — DECIDED (operator, 2026-08-31).**
-
-      | Role | Address | Owned by |
-      |---|---|---|
-      | `k3s-prod-1` / `-2` / `-3` | `192.168.2.10` / `.11` / `.12` | Ansible inventory |
-      | **kube-vip control-plane VIP** | `192.168.2.9` | Ansible (`group_vars/k3s_prod`) |
-      | **Ingress Gateway VIP** (`GATEWAY_VIP`) | `192.168.2.20` | Flux (`clusters/prod`) |
-      | **Cilium L2 announcement pool** | `192.168.2.20` – `.30` | Flux (`platform-config/cilium`) |
-
-      Two deliberate properties, both mirroring dev:
-      - The **control-plane VIP sits outside the LB pool** (`.9`, below the nodes), so kube-vip and
-        Cilium LB-IPAM can never contend for the same address. Dev achieves this the other way
-        round (`.49` control-plane, pool `.53-.60`).
-      - **`GATEWAY_VIP` is the FIRST address of the pool.** Cilium LB-IPAM can only allocate from a
-        defined pool block, so a Gateway VIP outside every pool would not be allocatable — the pool
-        was widened from `.21-.30` to `.20-.30` for exactly this reason. Dev has the same shape
-        (`GATEWAY_VIP` `.53` = first of `.53-.60`).
-- [ ] **Reserve all of `192.168.2.9` and `.20-.30` outside any DHCP range** on the prod LAN before
-      the build, and confirm nothing else on the network already answers on them.
-- [ ] **Resolver: SECOND UPSTREAM IS AN OPEN DECISION — blocks `node-baseline.yml --tags dns`.**
-      Node 1 has a single IPv4 nameserver, `192.168.2.1`, which is also the LAN gateway. A single
-      upstream is a known SPOF that has already bitten this project — it broke containerd image
-      pulls and CoreDNS forwarding on dev.
-
-      The MECHANISM is now in the repo and needs no further work: the `dns`-tagged block in
-      `roles/node_baseline` detects whether a node uses systemd-resolved (dev: writes
-      `/etc/systemd/resolved.conf.d/99-k3s-dns.conf`, restarts resolved) or a directly-managed
-      `/etc/resolv.conf` (prod's shape: writes the file, refusing if resolvconf/NetworkManager
-      owns it), and configures the ordered upstream list either way. The earlier
-      `fix/node-dns-resilient-upstream` branch was **dropped, not merged** — it wrote a netplan
-      drop-in and ran `netplan apply`, and netplan is not installed on these Debian 13 trixie
-      nodes, so it would have broken `node-baseline.yml` on every host.
-
-      What is still MISSING is the VALUE. `group_vars/k3s_prod/vars.yml` sets
-      `node_baseline_dns_servers` to `192.168.2.1` followed by the deliberately invalid
-      `BLOCKED-pending-second-upstream-decision-see-PROD-PROVISIONING-md-4`, and the role's assert
-      stops the play on it before writing anything. Choosing the second upstream is a **network
-      decision, not a default the role may take**: a public resolver (`8.8.8.8`, `1.1.1.1`) would
-      resolve reliably but would also bypass any local zone and any filtering the gateway applies.
-      Decide between — and record the choice HERE, in the same change that edits the group_vars:
-      - a second on-LAN resolver on `192.168.2.x`;
-      - the dev-side PiHole `192.168.1.18`, **if** prod can actually route to it (unverified —
-        prod sits on the workstation side of the pfSense boundary);
-      - an explicit, written-down acceptance of a public resolver as fallback-only.
-
-      Also decide whether node 1's current IPv6 nameservers (`2001:4860:4860::8888` / `::8844`)
-      are kept: the direct-file write path replaces `/etc/resolv.conf` with exactly the listed
-      servers, so keeping them means listing them explicitly.
-      `node_baseline_dns_search` is `[]` (no search domain, node 1's behaviour today) until prod's
-      local domain is decided.
-- [ ] Confirm the prod LAN's relationship to the dev LAN (`192.168.1.x`) and to the pfSense NAT —
-      prod on `192.168.2.x` sits on the *workstation* side of the boundary dev lives behind.
-
-## 5. Access (per node) — **blocks all Ansible**
-
-- [ ] **`k3s` user with the `home_network` SSH key.** Working on node 1.
-- [ ] **Password-protected sudo, driven by Ansible `become` + a vaulted password.**
-      **Prod does NOT use NOPASSWD sudo** (operator decision, 2026-08-31). Dev nodes do; prod
-      deliberately keeps the sudo password in place and supplies it to Ansible as an
-      **ansible-vault-encrypted** var, exactly like the existing `hypervisor` and `proxmox_hosts`
-      groups:
-      ```
-      inventory/group_vars/k3s_prod/
-        README.md     how to (re)create the vault — committed
-        vault.yml     ansible_become_password: "..."  — committed, ENCRYPTED
-      ```
-      The passphrase that decrypts it lives **outside the repo** at
-      `~/.config/home-k3s/ansible-vault-pass` (`ansible.cfg :: vault_password_file`), so plays still
-      run unattended without a plaintext secret ever touching git.
-      ```bash
-      ansible-vault create inventory/group_vars/k3s_prod/vault.yml   # ansible_become_password: ...
-      ```
-      Prod nodes must therefore set `ansible_become: true` in their group_vars and must **not**
-      inherit any passwordless-sudo assumption from `group_vars/all.yml`.
-- [ ] **Drive health baseline** (§2). **Do not hand-install the tooling** — package installation is
-      the `node_baseline` role's job, never a manual `apt-get` on a node (operator ruling,
-      2026-08-31; same click-ops-never principle as the rest of the repo). `smartmontools` is part
-      of `node_baseline_packages`, so it arrives with the baseline play — but a *full* baseline
-      would hit the disk tasks §2 has not decided yet. Run the **packages-only** baseline instead
-      (the `disks`-tagged tasks, including the BLOCKED-device assert, are not selected at all, so
-      the placeholder device paths are never even templated):
-      ```bash
-      cd infra/ansible
-      ansible-playbook playbooks/node-baseline.yml \
-        -e k3s_hosts=k3s_prod --tags packages --limit k3s-prod-1
-      ```
-      (Needs `group_vars/k3s_prod/vault.yml` to exist first — the become password above.) Reading
-      the drive is then:
-      ```bash
-      sudo smartctl -a /dev/sda   # Power_On_Hours, Percent_Lifetime_Remaining, reallocated sectors
-      ```
-      Record the result in §2 and close the layout decision in the same change that replaces the
-      two `node_baseline_*_device` placeholders.
-
-## 6. Ansible baseline & cluster join
-
-Reuses the dev roles **unchanged** — that reuse **is** R-ENV-02. Only inventory and group_vars
-differ, plus a target-group override on the command line.
-
-### How prod is targeted (done, 2026-08-31)
-
-Every node-host play used to hardcode the literal dev groups `k3s_servers` / `k3s_primary` /
-`k3s_secondary`, so nothing here could select prod at all. Two knobs now close that, deliberately
-kept separate because Ansible resolves them at different times:
-
-| Layer | Knob | Set via |
-|---|---|---|
-| Which hosts a play **runs on** | `k3s_hosts`, `k3s_primary_hosts`, `k3s_secondary_hosts` (each defaults to the dev group) | `-e` **extra vars only** — a play's `hosts:` is templated before any host exists, so `group_vars` is not readable there |
-| Which group a **role** means by "my primary" / "my servers" | `k3s_primary_group`, `k3s_servers_group`, `cluster_kubeconfig_name` | `group_vars` — dev in `all.yml`, prod in `k3s_prod/vars.yml`; **no per-run flag** |
-
-So `roles/k3s_server` finds the prod `--cluster-init` founder, builds the prod join target and
-tls-san list, and `roles/cilium` fetches the prod kubeconfig, with **no role fork**. Running with
-no extra vars is byte-for-byte the dev behaviour it always had (verified by a before/after
-`--list-hosts` diff on every playbook). Full detail: `infra/ansible/README.md` → **Targeting prod**.
-
-- [x] Add a `k3s_prod` inventory group (`k3s_prod_primary` / `k3s_prod_secondary`). Prod is reached
-      **directly** on the LAN — no NAT port-forward hop, unlike dev's `192.168.2.102:225x` mapping.
-      (Nodes 2/3 stay commented out until they are physically on the bench.)
-- [x] `inventory/group_vars/k3s_prod/vars.yml`: `kube_vip_address` (`192.168.2.9`),
-      `cluster_controller_api_server`, `k3s_api_host`, the group-name overrides above — **and** the
-      §2 device overrides for `node_baseline_longhorn_device` / `node_baseline_etcd_device`, which
-      are still BLOCKED placeholders.
-- [ ] `inventory/group_vars/k3s_prod/vault.yml` — the vaulted `ansible_become_password` (§5). Until
-      it exists **no** prod play can escalate, and `--check` against prod cannot work either.
-- [ ] Run the **packages-only** baseline first, to unblock the §2 drive-health read:
-      ```bash
-      ansible-playbook playbooks/node-baseline.yml -e k3s_hosts=k3s_prod --tags packages
-      ```
-- [ ] Run the **full** `playbooks/node-baseline.yml -e k3s_hosts=k3s_prod` → packages, kernel
-      modules, sysctls, iscsid, data-disk filesystems + mounts. **Blocked until §2 is decided**: it
-      asserts on the placeholder device paths and stops. (`qemu-guest-agent` is a no-op on bare
-      metal — the role already only *starts* it when the virtio channel exists, so it degrades
-      cleanly.)
-- [ ] Run `playbooks/k3s.yml -e k3s_primary_hosts=k3s_prod_primary -e k3s_secondary_hosts=k3s_prod_secondary`
-      → HA embedded etcd, flannel/kube-proxy/Traefik/servicelb disabled. **Only once all three
-      nodes are present** — see §8.
-- [ ] Run `playbooks/kube-vip.yml -e k3s_hosts=k3s_prod` → control-plane VIP (`192.168.2.9`) on the
-      prod LAN.
-- [ ] Run `playbooks/cilium.yml -e k3s_hosts=k3s_prod` → CNI at bootstrap; Flux's HelmRelease adopts
-      it afterwards. The bootstrap values and the HelmRelease **must stay in parity**
-      (`kubeProxyReplacement`, `k8sServiceHost`/`Port`) or the two layers fight over CNI config
-      (`docs/02`). Note the prod kubeconfig is fetched to `.cache/prod.kubeconfig`, never over the
-      dev one the dev container exports as `$KUBECONFIG`.
-- [ ] Run `playbooks/verify.yml -e k3s_hosts=k3s_prod`.
-- [ ] `playbooks/flux-bootstrap.yml -e k3s_hosts=k3s_prod` **retargets but is not yet correct**:
-      `roles/flux_bootstrap` shells out to `bootstrap/flux/bootstrap.sh`, which bootstraps
-      `--path=clusters/dev`. Prod needs `--path=clusters/prod` — a `bootstrap/` change, not a
-      target-group change. Tracked with the other repo blockers in §7.
-- [ ] Decide whether prod runs a VPN egress gateway node (dev uses `k3s-dev-3`, ADR-0015).
-      `playbooks/vpn.egress.gateway.yml` takes `-e vpn_egress_gateway_hosts=<group>`, but there is
-      deliberately no prod egress group in the inventory yet, and it would need its **own** vaulted
-      PIA credentials — the existing vault is scoped to the dev group.
-
-## 7. Repo blockers — **prod cannot be reconciled until these land**
-
-`clusters/prod` is design-only today and **would build wrong**. R-ENV-02 promises dev and prod differ
-*only* by `cluster-vars` values; in practice dev LAN addresses are hardcoded into base manifests:
-
-| Hardcoded value | Where | Prod needs |
-|---|---|---|
-| `k8sServiceHost: 192.168.1.50` | `platform/cilium/helmrelease.yaml:50`, mirrored in `values.yaml:16` | prod primary node IP |
-| LB pool `192.168.1.53–.60` | `platform-config/cilium/l2-ip-pool.yaml:15` | a prod-LAN pool |
-| `192.168.1.55` (crafty) | `platform/crafty/service-l4.yaml` | prod-LAN address |
-| `192.168.1.54` (UniFi VM) | `apps/vms/unifi/50-service-l4.yaml` | prod-LAN address |
-| `192.168.1.9:9000` (S3 backup) | `platform/keycloak/postgres-cluster.yaml`, `apps/wave-0/{nextcloud,semaphore}-db.yaml` | reachable prod backup target |
-
-- [ ] **Tokenize each into a `cluster-vars` knob** with a base default, then set the value in *both*
-      overlays. Adding a knob before referencing `${KNOB}` is the sanctioned mechanism (see the
-      header comment in `clusters/base/cluster-vars.yaml`). `platform`, `platform-config`, the app
-      waves, `apps-vms` and `crafty` all already carry `postBuild.substituteFrom: cluster-vars`, so
-      the seam exists.
-- [ ] **`clusters/prod` must set `GATEWAY_VIP`.** It currently doesn't, so prod would inherit the
-      base default `10.73.0.240` — an address from the **retired libvirt network**. The base file's
-      own comment flags this as owed "when hardware is built."
-- [ ] **Dev overlay sets every new knob to today's exact value** so rendered dev output is
-      byte-identical and the live cluster sees zero drift. Verify with `make -C ci render-golden`.
-- [ ] Confirm prod's backup target: dev's S3 (`versitygw` on the Proxmox host, `192.168.1.9`) is on
-      the dev side of the network. Off-cluster backup is DR-critical (R-STATE-03).
-
-## 8. Build sequence (operator decision, 2026-08-31: **wait for all three nodes**)
-
-Node 1 is baselined now; the cluster is **not** initialized until nodes 2 and 3 are on the bench.
-This keeps prod spec-conformant (3-node HA embedded etcd) from its first boot, avoids a
-single-server-etcd window, and avoids a later member-add dance.
-
-1. Per-node §1–§5 on each of the three boxes.
-2. §7 repo work lands on `main` with the gate green.
-3. `node-baseline` → `k3s` (all three) → `kube-vip` → `cilium`.
-4. `flux bootstrap ... --path=clusters/prod`.
-5. Conformance against prod (§9).
-
-## 9. Exit criteria (Phase 7)
-
-- [ ] Prod runs the same config as dev via the prod overlay, **no base edits** (R-ENV-02).
-- [ ] Node destroy/rebuild from IaC + Ansible without data loss (R-ENV-01, R-STATE-01).
-- [ ] Full **automated** prod DR drill (R-STATE-03) — `definition-of-done.md` requires this for prod
-      specifically.
-- [ ] Monitoring and alerts live on prod.
-- [ ] The conformance suite passes green **against the prod cluster**, not just dev (R-TEST-03).

infra/README.md

@@ -20,7 +20,7 @@ Proxmox LAN and are reached across a pfSense NAT:
 
 | | |
 |---|---|
-| Nodes | `k3s-dev-1/2/3` at `192.168.1.50/.51/.52` — 6 vCPU, 24 GiB, Debian 13, k3s v1.36.4 |
+| Nodes | `k3s-dev-1/2/3` at `192.168.1.50/.51/.52` — 6 vCPU, 16 GiB, Debian 12, k3s v1.31.5 |
 | kube-API | kube-vip VIP `192.168.1.49:6443`, reached from the workstation via NAT `192.168.2.50:6443` |
 | Node SSH | pfSense NAT `192.168.2.102:2250/2251/2252` → node `:22` (Ansible only) |
 | Ingress VIP | `192.168.1.53` (Cilium L2, shared Gateway); `.53–.60` reserved for LB VIPs |

infra/ansible/.gitignore

@@ -1,13 +1,3 @@
 # Runtime cache written by the cilium/verify/flux plays (fetched kubeconfig,
 # rendered Cilium values). Never commit — the kubeconfig is a credential.
 .cache/
-
-# Generated node-health reports (roles/node_health, playbooks/node-health.yml).
-# Regenerated on every run and they carry drive serial numbers — never commit. Same
-# convention as conformance/_artifacts/: generated output is an artifact, not source.
-_artifacts/
-
-# Galaxy collections installed by `ansible-galaxy collection install -r
-# collections/requirements.yml -p collections/`. Pinned by requirements.yml, fetched at
-# setup time — vendored bytes are not source. ansible.cfg sets collections_path.
-collections/ansible_collections/

infra/ansible/README.md

@@ -4,9 +4,8 @@ The "Cluster-join" node lifecycle stage (docs/02 stage 3) and the Flux hand-off
 (T-1.4 call-in). Installs a 3-server HA k3s cluster (embedded etcd), Cilium as the
 CNI, verifies readiness, then invokes the `bootstrap/` entrypoints.
 
-> **LIVE.** These plays have been executed against the running dev cluster (k3s
+> **AUTHOR-ONLY — NOTHING HERE HAS BEEN EXECUTED.** No `ansible-playbook` run, no
-> v1.36.4+k3s1, 3 nodes). This is no longer author-only scaffolding — treat it as the
+> `tofu apply`, no cluster contacted. The PM static-validates (see below).
-> operational bring-up + re-provision runbook. See **Re-provisioning a node** below.
 
 Serves **R-ENV-01/02**, **R-PLAT-01**, ADR-0001 (k3s), ADR-0002 (Cilium),
 ADR-0004 (Flux hand-off), ADR-0007 (SOPS).
@@ -23,40 +22,14 @@ ADR-0004 (Flux hand-off), ADR-0007 (SOPS).
 ## Run order
 ```
 cd infra/ansible
-ansible-playbook playbooks/site.yml        # core bring-up, in order (baseline→k3s→cilium→verify→flux)
+ansible-playbook playbooks/site.yml        # everything, in order
 # — or step by step —
-ansible-playbook playbooks/node-baseline.yml    # 0. packages, kernel modules, Longhorn disk, swap-off
 ansible-playbook playbooks/k3s.yml         # 1. HA k3s (primary --cluster-init, then joins)
 ansible-playbook playbooks/cilium.yml      # 2. Cilium CNI (cluster is NotReady until this)
 ansible-playbook playbooks/verify.yml      # 3. gate #1: 3 nodes Ready, Hubble up, no kube-proxy
 ansible-playbook playbooks/flux-bootstrap.yml   # 4. hand off to bootstrap/ (SOPS then Flux)
 ```
 
-> **`site.yml` is NOT everything.** It covers the core bring-up only. Two node-host
-> layers live OUTSIDE `site.yml` and must be run separately:
-> ```
-> ansible-playbook playbooks/kube-vip.yml            # control-plane VIP (192.168.1.49); defaults to k3s_servers
-> ansible-playbook playbooks/vpn.egress.gateway.yml  # node-host OpenVPN split-tunnel + kill-switch; defaults to k3s-dev-3
-> ```
-> (Both take the same target-group override as the rest — see **Targeting prod** below.)
-
-## Re-provisioning a node (tofu -replace / fresh OS)
-A node rebuild wipes **everything on the host filesystem** — including host-level config
-that Flux/Helm do NOT manage. After `tofu apply`/`-replace` re-creates a node, re-run the
-node-host layers or the node comes back missing pieces:
-
-| Re-run | Why it doesn't survive a rebuild |
-|---|---|
-| `node-baseline.yml` | packages (incl. `cryptsetup`), kernel modules, Longhorn disk, swap-off |
-| `k3s.yml` | the k3s server binary + config + join (secondaries re-join the VIP) |
-| `kube-vip.yml` | the kube-vip auto-deploy manifest on the server's host path |
-| `vpn.egress.gateway.yml` | **the OpenVPN (PIA) client, `tun0`, systemd units + nftables kill-switch on k3s-dev-3 — miss this and `egress-class=vpn` pods (sonarr/radarr/qbittorrent) are kill-switch-dropped with no internet** |
-
-Everything Flux owns (platform + apps) self-heals once the node re-joins; only the
-node-host layers above need a manual re-run. Verify the VPN after: on k3s-dev-3,
-`systemctl is-active vpn-egress-tunnel.service` and `curl --interface tun0 https://ipinfo.io/ip`
-should return a PIA exit IP (not the home ISP).
-
 ## What each layer does
 | Playbook / role | Action |
 |---|---|
@@ -89,91 +62,6 @@ expose different entrypoints, override the two vars in
 `roles/flux_bootstrap/defaults/main.yml`; the task graph is unchanged. Flagged
 for PM reconciliation.
 
-## Targeting prod (R-ENV-02: same playbooks, same roles, scoped overrides)
-
-`inventory/hosts.ini` carries a `[k3s_prod_primary]` / `[k3s_prod_secondary]` /
-`[k3s_prod:children]` section for the three planned bare-metal nodes (`k3s-prod-1` at
-`192.168.2.10` exists and is reachable today; `k3s-prod-2`/`-3` at `.11`/`.12` are addressed but
-not yet racked, so their inventory lines stay commented out), with its own `group_vars/k3s_prod/`
-(vaulted `ansible_become_password` + non-secret overrides).
-
-**Nothing here is forked for prod.** Every node-host play takes its target group from a variable
-that defaults to the dev group, so the dev commands above are unchanged, and prod is the same
-command plus an override. Two separate mechanisms, for two separate reasons:
-
-| Layer | Knob | Set via | Why |
-|---|---|---|---|
-| Which hosts a play **runs on** | `k3s_hosts`, `k3s_primary_hosts`, `k3s_secondary_hosts`, `vpn_egress_gateway_hosts` | `-e` **extra vars only** | Ansible templates a play's `hosts:` *before* it has any host, so it cannot see `group_vars` at all |
-| Which group a **role** means by "my primary" / "my servers" | `k3s_primary_group`, `k3s_servers_group` (+ `cluster_kubeconfig_name`) | `group_vars` — dev in `all.yml`, prod in `k3s_prod/vars.yml` | Host context exists by then, so each environment answers for itself with **no per-run flag** — same mechanism `k3s_api_host` already used |
-
-`playbooks/node-health.yml` established the first pattern (`-e node_health_hosts=k3s_prod`); it
-keeps its own var name because it is a standalone diagnostic you point anywhere, not part of the
-bring-up chain.
-
-```bash
-# dev — completely unchanged, no extra vars
-ansible-playbook playbooks/site.yml
-
-# prod — one flag for the whole-cluster plays
-ansible-playbook playbooks/node-baseline.yml   -e k3s_hosts=k3s_prod
-ansible-playbook playbooks/cilium.yml          -e k3s_hosts=k3s_prod
-ansible-playbook playbooks/kube-vip.yml        -e k3s_hosts=k3s_prod
-ansible-playbook playbooks/verify.yml          -e k3s_hosts=k3s_prod
-ansible-playbook playbooks/flux-bootstrap.yml  -e k3s_hosts=k3s_prod   # see caveat below
-
-# prod — k3s.yml needs both halves of the primary/secondary split
-ansible-playbook playbooks/k3s.yml \
-  -e k3s_primary_hosts=k3s_prod_primary -e k3s_secondary_hosts=k3s_prod_secondary
-
-# prod — the whole chain (extra vars are global, so they reach imported playbooks)
-ansible-playbook playbooks/site.yml \
-  -e k3s_hosts=k3s_prod \
-  -e k3s_primary_hosts=k3s_prod_primary \
-  -e k3s_secondary_hosts=k3s_prod_secondary
-```
-
-What the roles then resolve **by themselves** for a prod host, from `group_vars/k3s_prod/vars.yml`:
-`k3s_primary_group=k3s_prod_primary` (the `--cluster-init` founder, the node-token `delegate_to`
-target, and the node Cilium fetches the kubeconfig from), `k3s_servers_group=k3s_prod` (the tls-san
-node-IP list), `k3s_api_host=192.168.2.10`, `kube_vip_address=192.168.2.9`, and
-`cluster_kubeconfig_name=prod.kubeconfig` — so a prod run can never overwrite the
-`.cache/dev.kubeconfig` that `infra/devcontainer/dev.sh` exports as `$KUBECONFIG`.
-
-### Packages-only baseline (`--tags packages`)
-`roles/node_baseline` is tagged `packages` / `system` / `disks`. An untagged run is unchanged (all
-three, in order). `--tags packages` runs **only** the apt task:
-
-```bash
-ansible-playbook playbooks/node-baseline.yml -e k3s_hosts=k3s_prod --tags packages --limit k3s-prod-1
-```
-
-This exists to break a circular dependency: prod's disk layout (`infra/PROD-PROVISIONING.md` §2) is
-blocked on a SMART read, SMART needs `smartmontools`, `smartmontools` ships in
-`node_baseline_packages` — but the role's disk tasks need block devices that decision hasn't chosen
-yet. With `--tags packages` no `disks`-tagged task is selected, so the deliberate
-`/dev/BLOCKED-pending-disk-decision-…` placeholders in `group_vars/k3s_prod/vars.yml` are never
-even templated, let alone handed to `mkfs`. A **full** run against a node whose devices are still
-placeholders stops on an explicit assert naming §2 instead of failing obscurely at mkfs time. Never
-hand-install packages on a node — that is this role's job (operator ruling, 2026-08-31).
-
-### Still not runnable end to end
-- The **disk-layout** (§2) and the `node_baseline_*_device` overrides remain BLOCKED placeholders.
-- `group_vars/k3s_prod/vault.yml` (the become password) does not exist yet, so **no** prod play can
-  escalate — and `--check` against prod fails for the same reason.
-- `flux-bootstrap.yml` retargets, but `roles/flux_bootstrap` shells out to
-  `bootstrap/flux/bootstrap.sh`, which bootstraps `--path=clusters/dev`. Prod needs
-  `--path=clusters/prod`: a `bootstrap/` change, not a target-group change (§7).
-- `vpn.egress.gateway.yml` is parameterised (`-e vpn_egress_gateway_hosts=…`) but prod has **no**
-  egress group yet — whether prod runs one at all is an open §6 decision, and it would need its own
-  vaulted PIA credentials.
-- The operator has ruled the prod cluster stays **uninitialized** until all three nodes are present
-  (§8). `192.168.2.20` is prod's *ingress* Gateway VIP — a Flux/`cluster-vars` concern, not an
-  Ansible one; the kube-vip **control-plane** VIP is `192.168.2.9` (`kube_vip_address`).
-
-Everything above this section describes the **dev** runbook; the values in it (`192.168.1.x`,
-`.cache/dev.kubeconfig`, `clusters/dev`) are dev's, supplied by `group_vars`, not baked into the
-roles.
-
 ## Static validation (PM, in dev container — NOT run here)
 ```
 yamllint -c ../../ci/yamllint.yaml infra/ansible

infra/ansible/ansible.cfg

@@ -1,10 +1,6 @@
 [defaults]
 inventory = inventory/hosts.ini
 roles_path = roles
-# Galaxy collections live in the repo (gitignored), installed from
-# collections/requirements.yml. The devcontainer image does NOT ship them, so without
-# this every play dies at parse time on community.general.modprobe.
-collections_path = collections
 host_key_checking = False
 retry_files_enabled = False
 # SSH key the k3s nodes authorize for the `k3s` user (see inventory ansible_user).

infra/ansible/inventory/group_vars/all.yml

@@ -4,35 +4,12 @@
 # cloud-init created this login user (tofu var login_user); it has NOPASSWD sudo.
 ansible_user: k3s
 
-# --- environment group NAMES (R-ENV-02: one set of roles, scoped overrides) ---------
-# The GROUP NAMES a role uses to answer "who is MY environment's primary / MY
-# environment's servers?". Roles must never hardcode `groups['k3s_primary']`, or they
-# only ever work against dev. These two are the dev values; group_vars/k3s_prod/vars.yml
-# overrides them with k3s_prod_primary / k3s_prod, so a host in the prod group resolves
-# its own primary automatically — exactly the way k3s_api_host below is overridden there.
-#
-# These are GROUP NAMES (keys into `groups`), NOT host patterns. The play-level target
-# pattern is a separate, extra-vars-only knob (`k3s_hosts` / `k3s_primary_hosts` /
-# `k3s_secondary_hosts`, see playbooks/*.yml) because Ansible templates a play's `hosts:`
-# before it has any host context and so cannot see group_vars at all.
-k3s_primary_group: k3s_primary
-k3s_servers_group: k3s_servers
-
-# Filename of the kubeconfig the controller-side roles (cilium, cluster_verify,
-# flux_bootstrap) fetch from the primary into infra/ansible/.cache/. Scoped per
-# environment so pointing those plays at prod can NEVER overwrite the dev kubeconfig
-# that infra/devcontainer/dev.sh exports as $KUBECONFIG — an operator's `kubectl get
-# nodes` silently answering from prod would be a genuinely dangerous mix-up.
-# group_vars/k3s_prod/vars.yml overrides this to prod.kubeconfig. (.cache/ is gitignored;
-# a kubeconfig is a credential.)
-cluster_kubeconfig_name: dev.kubeconfig
-
 # The kube-apiserver endpoint the whole stack agrees on: the primary's REAL node IP.
 # NOTE: ansible_host is now the pfSense NAT SSH target (192.168.2.102:225x), NOT the node
 # IP — so we read the real 192.168.1.x from k3s_node_ip instead. Used for k3s --tls-san /
 # secondary --server AND for Cilium k8sServiceHost so the Ansible bootstrap and the Flux
 # HelmRelease stay in PARITY (docs/02 contract).
-k3s_api_host: "{{ hostvars[groups[k3s_primary_group][0]]['k3s_node_ip'] }}"
+k3s_api_host: "{{ hostvars[groups['k3s_primary'][0]]['k3s_node_ip'] }}"
 k3s_api_port: 6443
 
 # Control-plane HA VIP floated by kube-vip (roles/kube_vip). A stable failover

infra/ansible/inventory/group_vars/k3s_prod/README.md

@@ -1,68 +0,0 @@
-# group_vars for the `k3s_prod` group (prod k3s nodes — Phase 7 PREP ONLY)
-
-Every node-host play that touches a k3s node (`playbooks/node-baseline.yml`, `k3s.yml`,
-`cilium.yml`, `kube-vip.yml`, ...) already runs with `become: true` at the play level. On **dev**
-that succeeds without a password (cloud-init's `login_user` gets NOPASSWD sudo). **Prod
-deliberately does NOT get that** — operator decision, 2026-08-31 (`DECISIONS-LOG.md`, "PHASE 7
-OPENED", finding 3; `infra/PROD-PROVISIONING.md` §5). Supply the sudo password as an
-**ansible-vault-encrypted** var here — it is encrypted, so it is safe to commit; the passphrase
-that decrypts it stays out of the repo, exactly like the existing `hypervisor` and `proxmox_hosts`
-groups.
-
-## Two files (one committed-encrypted, one never committed)
-
-1. **The vault passphrase** — a file **outside the repo**, referenced by `ansible.cfg`
-   (`vault_password_file = ~/.config/home-k3s/ansible-vault-pass`). Skip this if you already made
-   one for the `hypervisor` or `proxmox_hosts` vaults — it's the SAME passphrase file, shared by
-   every vaulted group in this repo:
-   ```bash
-   mkdir -p ~/.config/home-k3s
-   install -m600 /dev/stdin ~/.config/home-k3s/ansible-vault-pass <<<'YOUR_VAULT_PASSPHRASE'
-   ```
-
-2. **The encrypted become password** — `vault.yml` in **this directory** (committed, encrypted).
-   `vault.yml.example` alongside it shows the exact shape (see it for the reasoning); copy its
-   content, fill in the real password, then encrypt it in place — do not commit the `.example`
-   content unencrypted as `vault.yml`:
-   ```bash
-   cd infra/ansible
-   ansible-vault create inventory/group_vars/k3s_prod/vault.yml
-   # file content (single line):
-   #   ansible_become_password: "THE k3s USER'S SUDO PASSWORD ON k3s-prod-1"
-   ```
-   (`ansible-vault create` uses the passphrase file above automatically — no prompt.)
-
-Once `vault.yml` exists, prod plays run unattended — same as dev, just with a real sudo boundary
-behind them instead of NOPASSWD.
-
-## Notes
-- `vault.yml` is skipped by the yamllint gate (`ci/yamllint.yaml` ignores `**/vault.yml`) because a
-  vault blob is not parseable YAML. `vault.yml.example` is plain YAML and IS linted — it's a
-  copy-paste scaffold, not a secret (same convention as
-  `platform/backup/minio-credentials.sops.yaml.example`).
-- The k3s **dev** nodes keep passwordless sudo unchanged — this vault is scoped to the `k3s_prod`
-  group only.
-- `ansible_become: true` for this group is set explicitly in `vars.yml` (this directory) — belt
-  and braces alongside the playbooks' own `become: true`, so prod's posture is self-documenting
-  and doesn't rely on inheriting anything from `group_vars/all.yml` by accident.
-- Non-secret prod vars live in `vars.yml`, not here — this README is scoped to the vault hand-off.
-  `k3s_api_host` is set to a real value. `kube_vip_address`, `cluster_controller_api_server`,
-  `node_baseline_longhorn_device` and `node_baseline_etcd_device` are all still-BLOCKED
-  placeholders pending operator decisions (control-plane VIP and disk layout, respectively) — see
-  `vars.yml`'s comments for what IS already decided (node addresses, the separate *ingress* VIP,
-  which is a Flux/`cluster-vars` concern and does not live here) versus what is not.
-- This is the Ansible-bootstrap-layer secret (host credential), distinct from the platform's
-  in-cluster **SOPS + age** secrets (ADR-0007) — different layer, different tool, same convention
-  as `hypervisor`/`proxmox_hosts`.
-- **Targeting prod is solved** (2026-08-31): plays take `-e k3s_hosts=k3s_prod` (and, for `k3s.yml`,
-  `-e k3s_primary_hosts=k3s_prod_primary -e k3s_secondary_hosts=k3s_prod_secondary`), while the
-  roles resolve "my environment's primary/servers" from `k3s_primary_group` / `k3s_servers_group`
-  in `vars.yml` here — no role fork, no per-run flag for that half. See
-  `infra/ansible/README.md` → "Targeting prod".
-- **Do not run any prod playbook yet.** Phase 7 is prep-only: the vault above just makes the plays
-  *able* to authenticate once the §7 repo blockers close and the disk-layout (§2) decision in
-  `infra/PROD-PROVISIONING.md` is made. The ONE exception the operator has sanctioned is the
-  packages-only baseline that unblocks the §2 drive-health read —
-  `ansible-playbook playbooks/node-baseline.yml -e k3s_hosts=k3s_prod --tags packages` — which
-  cannot touch the BLOCKED device placeholders because every disk task is `disks`-tagged and so is
-  not selected. It still needs `vault.yml` (above) to exist first.

infra/ansible/inventory/group_vars/k3s_prod/vars.yml

@@ -1,172 +0,0 @@
----
-# group_vars/k3s_prod/vars.yml — NON-secret prod-scoped overrides for the [k3s_prod]
-# umbrella group (inventory/hosts.ini: [k3s_prod_primary]/[k3s_prod_secondary] under
-# [k3s_prod:children]). Ansible auto-loads group_vars/<groupname>/*.yml for every host
-# that is a (transitive) member of that group. See infra/PROD-PROVISIONING.md (T-7.1)
-# §5/§6 for the checklist this file implements, and DECISIONS-LOG.md "PHASE 7 OPENED"
-# (2026-08-31) for the operator decisions behind it.
-#
-# Pairs with vault.yml in this same directory (ansible-vault-encrypted, operator-
-# created — see README.md here) for the one secret prod needs: ansible_become_password.
-#
-# PHASE 7 IS PREP ONLY. Five vars below (kube_vip_address, cluster_controller_api_server,
-# node_baseline_longhorn_device, node_baseline_etcd_device, and the SECOND entry of
-# node_baseline_dns_servers) are deliberately BROKEN
-# PLACEHOLDERS, not real values — see their comments. Do not "fix" them with a guess;
-# follow the referenced section of PROD-PROVISIONING.md, record the decision there,
-# and update this file in the same change.
-
-# --- become posture (§5) ----------------------------------------------------------
-# Prod deliberately does NOT get dev's passwordless (NOPASSWD) sudo — operator
-# decision, 2026-08-31 (DECISIONS-LOG.md "PHASE 7 OPENED", finding 3). group_vars/
-# all.yml carries no explicit `ansible_become` of its own (dev's passwordless sudo
-# means become succeeds without a password prompt, not that become is switched off),
-# and every node-host playbook already sets `become: true` at the play level
-# (node-baseline.yml, k3s.yml, cilium.yml, kube-vip.yml) — for dev that silently
-# succeeds because sudo needs no password there. Setting ansible_become explicitly
-# here is belt-and-braces: this group's posture is self-documenting and does not rely
-# on inheriting dev's assumption by accident.
-ansible_become: true
-# The password itself is ansible_become_password — ansible-vault-encrypted, in
-# vault.yml (this directory), never here in plaintext. See README.md to create it.
-
-# --- environment group NAMES (R-ENV-02) -------------------------------------------
-# group_vars/all.yml defines these as the DEV group names (k3s_primary / k3s_servers);
-# here they name PROD's groups. Roles index `groups[k3s_primary_group]` /
-# `groups[k3s_servers_group]` instead of a literal, so roles/k3s_server and roles/cilium
-# resolve "my environment's primary" and "my environment's servers" correctly for prod
-# hosts with NO role fork and no per-run flag — the reuse R-ENV-02 promises.
-#
-# NOTE these are GROUP NAMES, not the play-level target pattern. Selecting which hosts a
-# play RUNS against is a separate extra-vars knob (`-e k3s_hosts=k3s_prod`, and for
-# k3s.yml `-e k3s_primary_hosts=k3s_prod_primary -e k3s_secondary_hosts=k3s_prod_secondary`),
-# because Ansible templates `hosts:` before any host — and therefore any group_vars —
-# exists. See infra/ansible/README.md "Targeting prod".
-k3s_primary_group: k3s_prod_primary
-k3s_servers_group: k3s_prod
-
-# Keep the fetched prod kubeconfig in its OWN cache file. group_vars/all.yml defaults this
-# to dev.kubeconfig, which is the file infra/devcontainer/dev.sh exports as $KUBECONFIG —
-# a prod play must never clobber it (a `kubectl` that silently answers from prod is worse
-# than no kubeconfig at all). .cache/ is gitignored; a kubeconfig is a credential.
-cluster_kubeconfig_name: prod.kubeconfig
-
-# --- k3s API host — safe to set now, the primary's real IP is already known -------
-# Dev's k3s_api_host (group_vars/all.yml) reads a separate k3s_node_ip hostvar because
-# dev's ansible_host is the pfSense NAT SSH target, not the node's real address. Prod
-# has NO NAT hop — ansible_host in hosts.ini IS already the node's real, routable LAN
-# address (see the KEY DIFFERENCE note there) — so point straight at it instead of
-# needing a k3s_node_ip split.
-k3s_api_host: "{{ hostvars[groups[k3s_primary_group][0]]['ansible_host'] }}"
-# cluster_controller_tls_server_name (group_vars/all.yml) derives from kube_vip_address
-# by template reference, so it needs no separate override here — it picks up the real
-# value set below automatically for prod hosts too.
-
-# --- control-plane VIP + controller access — DECIDED (§4) -------------------------
-# Prod addressing, all decided by the operator 2026-08-31:
-#   nodes k3s-prod-1/2/3   192.168.2.10 / .11 / .12   (hosts.ini)
-#   control-plane VIP      192.168.2.9                (THIS value — kube-vip, Ansible-side)
-#   ingress Gateway VIP    192.168.2.20               (Flux/cluster-vars — NOT here)
-#   ingress L2 pool        192.168.2.20 - .30         (Flux/cluster-vars — NOT here)
-#
-# The control-plane VIP sits BELOW the node addresses and deliberately OUTSIDE the
-# Cilium LB pool (.20-.30), so kube-vip and Cilium LB-IPAM can never contend for the
-# same address. That separation is the point: dev learned the same shape the other way
-# round (control-plane .49, ingress pool .53-.60).
-#
-# This is the address kube-vip floats for the API and that k3s tls-san's onto its
-# serving cert; secondaries use it as their --server join target. Dev's equivalent is
-# kube_vip_address: 192.168.1.49 in group_vars/all.yml.
-#
-# HISTORY: an earlier pass here wrongly set this to 192.168.2.20 — that is the INGRESS
-# VIP, not the control-plane VIP. Corrected. Keep the two distinct.
-kube_vip_address: "192.168.2.9"
-# Derived from kube_vip_address above (not repeated), so fixing the one line above is
-# the only change needed once a real control-plane VIP is chosen. Prod has no NAT hop
-# (unlike dev's cluster_controller_api_server, the dev-container's NAT path to its VIP
-# — see group_vars/all.yml), so once real this collapses to a direct "https://<VIP>:6443".
-cluster_controller_api_server: "https://{{ kube_vip_address }}:6443"
-
-# --- node_baseline data-disk devices — BLOCKED, open decision (§2) ----------------
-# roles/node_baseline defaults (node_baseline_longhorn_device: /dev/vdb,
-# node_baseline_etcd_device: /dev/vdc) assume tofu-provisioned virtio disks that DO
-# NOT EXIST on bare metal. Node 1 is a SINGLE SATA disk (/dev/sda) with no free block
-# device at all today; PROD-PROVISIONING.md §2 lists three candidate fixes (add a
-# second drive / repartition / reuse the /dev/sda5 "/srv" partition) that all wait on
-# a SMART health read of a ~15-year-old consumer SSD (operator decision, 2026-08-31).
-#
-# The overrides below are deliberately present-but-broken: leaving them UNSET would
-# let the role silently fall back to /dev/vdb//dev/vdc, which ALSO don't exist on this
-# hardware — a confusing failure that looks like a wiring bug. This value fails just
-# as hard, but says exactly why, and blocks a `find`/glob from accidentally matching
-# a real device. DO NOT replace with a guess — replace only once §2 is decided, and
-# record the chosen device paths in PROD-PROVISIONING.md §2 in the SAME change that
-# updates these two lines. All three nodes must end up with an identical layout
-# (Longhorn replica placement assumes symmetry) before any of this is used for real.
-node_baseline_longhorn_device: "/dev/BLOCKED-pending-disk-decision-see-PROD-PROVISIONING-md-2"
-node_baseline_etcd_device: "/dev/BLOCKED-pending-disk-decision-see-PROD-PROVISIONING-md-2"
-
-# --- node_baseline DNS upstreams — SECOND UPSTREAM BLOCKED, open decision (§4) -----
-# roles/node_baseline defaults are DEV's list (192.168.1.18 PiHole, then 192.168.1.1
-# pfSense) — addresses on a network prod cannot even reach. Inherited verbatim they would
-# point every prod node at a PiHole that does not answer it, so they are overridden here,
-# in the same seam as kube_vip_address / k3s_api_host / cluster_kubeconfig_name above.
-# (NOT clusters/base/cluster-vars.yaml: that ConfigMap is consumed by Flux
-# postBuild.substituteFrom for platform/ and apps/ manifests, and no Ansible code reads it.)
-#
-# WHAT IS DECIDED: 192.168.2.1 — node 1's current sole nameserver, and the LAN gateway.
-# It stays FIRST, so whatever local zone it serves keeps working (the dev outage taught
-# that ordering is policy, not preference — see the role's defaults).
-#
-# WHAT IS NOT: the SECOND upstream. 192.168.2.1 is presently gateway AND only IPv4
-# resolver, and there is no second address on this LAN that is known to answer. Picking
-# one is a NETWORK decision, not a default this role may take: a public resolver
-# (8.8.8.8, 1.1.1.1) would resolve reliably but would ALSO bypass any local zone and any
-# network-level filtering the gateway applies — precisely the bypass the dev fix refused
-# to make. Candidates the operator must choose between and record in
-# infra/PROD-PROVISIONING.md §4: a second on-LAN resolver, the dev-side PiHole IF prod can
-# route to 192.168.1.18, or an explicit, written-down decision to accept a public resolver
-# as fallback-only.
-#
-# Deliberately present-but-broken, exactly like node_baseline_longhorn_device above:
-# leaving the list at one entry, or unset, would either configure the very SPOF this exists
-# to remove or silently inherit dev's PiHole. This value can never parse as an address, and
-# the `dns`-tagged assert in roles/node_baseline stops the play on the string BLOCKED
-# before a single file is written. DO NOT replace it with a guess — replace it only once
-# §4 is decided, and record the chosen resolver there in the SAME change.
-#
-# Note the two Google IPv6 servers currently in node 1's /etc/resolv.conf
-# (2001:4860:4860::8888/::8844) are NOT carried over: the direct-file write path replaces
-# the file with exactly this list, so if they are to be kept they must be listed here
-# explicitly — as part of that same recorded decision.
-node_baseline_dns_servers:
-  # DECIDED (operator, 2026-09-01): prod's resolvers are PUBLIC and deliberately differ
-  # from dev's. Dev is PiHole-first (192.168.1.18) because dev needs the local jackson.lan
-  # zone AND split-horizon *.home.keithjackson.dev. Prod has no local resolver on its LAN,
-  # so it takes two INDEPENDENT public upstreams — which resolves the single-upstream SPOF
-  # (a single upstream broke containerd image pulls and CoreDNS forwarding on dev).
-  - "8.8.8.8"
-  - "1.1.1.1"
-  # SCOPE — this file sets NODE resolution only: containerd image pulls, and the upstream
-  # CoreDNS forwards `.` to. Three different resolvers answer *.home.keithjackson.dev, and
-  # only ONE of them is configured here:
-  #
-  #   pods (in-cluster) -> CoreDNS -> platform/coredns/coredns-custom.yaml, whose `template`
-  #       plugin answers the WHOLE home.keithjackson.dev zone with ${GATEWAY_VIP}. Entirely
-  #       self-contained in the cluster; PiHole is NOT in this path. Prod is already correct
-  #       (GATEWAY_VIP = 192.168.2.20 in clusters/prod).
-  #   LAN clients       -> PiHole. Not a cluster concern.
-  #   nodes             -> THIS FILE. Public resolvers, so the zone resolves PUBLICLY.
-  #
-  # THE ONE THING THAT BREAKS: the k3s API SERVER is a node process and carries
-  # `oidc-issuer-url=https://keycloak.home.keithjackson.dev/realms/homelab`
-  # (roles/k3s_server/tasks/main.yml). It reaches the issuer through the NODE resolver, so on
-  # prod it would get the PUBLIC record instead of the prod Gateway VIP — breaking kubectl
-  # OIDC SSO. Dev avoids this because its node resolver is PiHole, which answers the zone
-  # with the dev Gateway VIP.
-  #
-  # NOT blocking today (prod has no cluster and no Keycloak yet). Options, in
-  # PROD-PROVISIONING.md §4: a narrow Ansible-managed /etc/hosts entry pinning
-  # keycloak.home.keithjackson.dev -> 192.168.2.20 on prod nodes; or a local resolver on the
-  # prod LAN serving the zone; or run prod on the cert kubeconfig and defer OIDC.
-node_baseline_dns_search: []

infra/ansible/inventory/group_vars/k3s_prod/vault.yml

@@ -1,7 +0,0 @@
-$ANSIBLE_VAULT;1.1;AES256
-33643331386138666338353162353562613737343265376639646262663931663239333735366163
-3961623239313038313966326339336464383763616266310a313766653231636636393030306635
-37653764656235363738363535386539613938393839623462316134383631623464343930383133
-6433386532653564630a383036633266353965383565666639363733393531643162366265366537
-30313265343035653961666331663534313564333939383038306338666535316639333136363036
-6532313164613836663862303561646362303130316136306361

infra/ansible/inventory/group_vars/k3s_prod/vault.yml.example

@@ -1,25 +0,0 @@
-# =============================================================================
-# TEMPLATE — NOT ENCRYPTED, NOT LOADED (a plain `.example` file; Ansible only reads
-# `vault.yml` in this directory, never `vault.yml.example`). Reference scaffold +
-# OPERATOR HAND-OFF, same convention as platform/backup/minio-credentials.sops.yaml.example.
-#
-# WHAT THIS IS: the sudo (become) password for the `k3s` user on prod k3s nodes.
-# Prod deliberately does NOT use passwordless (NOPASSWD) sudo the way dev does —
-# operator decision, 2026-08-31 (DECISIONS-LOG.md "PHASE 7 OPENED", finding 3;
-# infra/PROD-PROVISIONING.md §5). Every play that becomes root on a k3s_prod host
-# (node-baseline.yml, k3s.yml, cilium.yml, kube-vip.yml, ...) needs this to escalate.
-#
-# HAND-OFF STEPS (operator):
-#  (a) Create the vault passphrase file ONCE, outside the repo, if you don't already
-#      have one from the `hypervisor`/`proxmox_hosts` vaults (it's the same file,
-#      shared by every vaulted group in this repo):
-#        mkdir -p ~/.config/home-k3s
-#        install -m600 /dev/stdin ~/.config/home-k3s/ansible-vault-pass <<<'YOUR_VAULT_PASSPHRASE'
-#  (b) Encrypt the REAL content into inventory/group_vars/k3s_prod/vault.yml — never
-#      commit this .example file's content unencrypted as vault.yml itself:
-#        cd infra/ansible
-#        ansible-vault create inventory/group_vars/k3s_prod/vault.yml
-#        # paste the single line below with the REAL password, save, exit
-#  (c) Verify: ansible-vault view inventory/group_vars/k3s_prod/vault.yml
-# =============================================================================
-ansible_become_password: "REPLACE-ME"

infra/ansible/inventory/hosts.ini

@@ -45,57 +45,3 @@ k3s-dev-3
 # `hypervisor` group above (that is `localhost` for host-prereqs.yml, a different machine).
 [proxmox_hosts]
 proxmox ansible_host=192.168.2.102 ansible_port=22 ansible_user=root ansible_ssh_private_key_file=~/.ssh/home_network
-
-# =============================================================================
-# PROD (k3s-prod-*) — Phase 7 PREP ONLY. See infra/PROD-PROVISIONING.md (T-7.1,
-# especially §5/§6) and DECISIONS-LOG.md "PHASE 7 OPENED" (2026-08-31). Node 1
-# (`k3s-prod-1`, HP EliteDesk 800 G5 Mini) is on the bench and SSH-reachable; nodes 2
-# and 3 have not arrived. This section is inventory/group_vars scaffolding only —
-# nothing has been run against node 1, and the operator has decided the CLUSTER stays
-# uninitialized (no k3s install, no --cluster-init) until all three nodes are present
-# (decision #2 in the DECISIONS-LOG entry above) — no single-server-etcd window, no
-# later member-add dance.
-#
-# KEY DIFFERENCE FROM DEV: prod sits DIRECTLY on the 192.168.2.x LAN — no pfSense NAT
-# hop in front of it. Dev's `ansible_host` above is a NAT target (192.168.2.102:225x)
-# that port-forwards to the node's real 192.168.1.x address, which is why dev also
-# carries a separate `k3s_node_ip` per host. Prod needs no such split: `ansible_host`
-# below IS already the node's real, routable address, so `k3s_node_ip` is set to the
-# SAME value (kept explicit, not omitted, only so the k3s_server role's hostvar lookup
-# — see the GAP note below — has something correct to find once that gap is closed).
-# `ansible_port` is the node's own real sshd (22). See group_vars/k3s_prod/vars.yml for
-# the k3s_api_host override that reflects this directly-reachable topology.
-#
-# GROUP-NAME GAP: CLOSED 2026-08-31. The roles and playbooks used to hardcode the
-# LITERAL dev groups (`hosts: k3s_servers` / `k3s_primary` / `k3s_secondary`, and
-# `groups['k3s_primary']` inside roles/k3s_server + roles/cilium), so nothing could
-# select the k3s_prod_* groups below. Both halves are now variables, deliberately kept
-# separate because Ansible resolves them at different times:
-#   * WHICH HOSTS A PLAY RUNS ON — `k3s_hosts` (whole-cluster plays), `k3s_primary_hosts`
-#     / `k3s_secondary_hosts` (k3s.yml's split), each defaulting to the dev group.
-#     EXTRA VARS ONLY: a play's `hosts:` is templated before any host exists, so
-#     group_vars cannot be read there. Prod: `-e k3s_hosts=k3s_prod` etc.
-#   * WHICH GROUP A ROLE MEANS BY "MY PRIMARY"/"MY SERVERS" — `k3s_primary_group` /
-#     `k3s_servers_group`, set in group_vars/all.yml (dev) and group_vars/k3s_prod/vars.yml
-#     (prod), so roles resolve their own environment with NO per-run flag — the same
-#     mechanism k3s_api_host already used.
-# Running with no extra vars is byte-for-byte the dev behaviour it always had. See
-# infra/ansible/README.md "Targeting prod" and PROD-PROVISIONING.md §6.
-# =============================================================================
-
-[k3s_prod_primary]
-k3s-prod-1 ansible_host=192.168.2.10 ansible_port=22 k3s_node_ip=192.168.2.10
-
-[k3s_prod_secondary]
-# Addresses for nodes 2/3 ARE decided (operator, 2026-08-31 — infra/PROD-PROVISIONING.md
-# §0), but the nodes themselves are NOT yet racked/imaged/reachable. Left commented so
-# no play can accidentally target a host that isn't there; uncomment each line only once
-# that node is actually on the bench and answering SSH.
-# k3s-prod-2 ansible_host=192.168.2.11 ansible_port=22 k3s_node_ip=192.168.2.11
-# k3s-prod-3 ansible_host=192.168.2.12 ansible_port=22 k3s_node_ip=192.168.2.12
-
-# Umbrella group group_vars/k3s_prod/ attaches to (mirrors dev's [k3s_servers:children]
-# pattern; named to match the group_vars directory so Ansible auto-loads it).
-[k3s_prod:children]
-k3s_prod_primary
-k3s_prod_secondary

infra/ansible/playbooks/cilium.yml

@@ -1,13 +1,8 @@
 ---
 # Install Cilium as the CNI (ADR-0002). The cluster is NotReady until this runs.
 # Executed from the controller against a fetched kubeconfig (role handles delegation).
-#
-# TARGET (R-ENV-02): `k3s_hosts` defaults to dev's `k3s_servers`; prod is
-#   ansible-playbook playbooks/cilium.yml -e k3s_hosts=k3s_prod
-# The role's "fetch the kubeconfig from the primary" delegation follows the
-# k3s_primary_group group_var, so it picks the prod primary automatically.
 - name: Install Cilium CNI
-  hosts: "{{ k3s_hosts | default('k3s_servers') }}"
+  hosts: k3s_servers
   become: true
   gather_facts: false
   roles:

infra/ansible/playbooks/flux-bootstrap.yml

@@ -1,15 +1,8 @@
 ---
 # FINAL step: hand off to Flux + SOPS (T-1.4). Calls into bootstrap/ (authored by
 # the Security + GitOps Engineers) — this play does not author that subtree.
-#
-# TARGET (R-ENV-02): `k3s_hosts` defaults to dev's `k3s_servers`; prod is
-#   ansible-playbook playbooks/flux-bootstrap.yml -e k3s_hosts=k3s_prod
-# NOTE the Flux PATH is a separate concern: roles/flux_bootstrap shells out to
-# bootstrap/flux/bootstrap.sh, which bootstraps --path=clusters/dev. Prod needs
-# --path=clusters/prod, which is a bootstrap/ change, NOT a target-group change
-# (infra/PROD-PROVISIONING.md §7 tracks it).
 - name: Flux + SOPS bootstrap hand-off
-  hosts: "{{ k3s_hosts | default('k3s_servers') }}"
+  hosts: k3s_servers
   become: false
   gather_facts: false
   roles:

infra/ansible/playbooks/k3s.yml

@@ -2,20 +2,9 @@
 # Install the HA k3s control plane (embedded etcd, 3 servers) — ADR-0001.
 # Primary founds the cluster (--cluster-init); secondaries join it in a second
 # play so the primary's node-token fact is available to them.
-#
-# TARGET (R-ENV-02): this is the one playbook that needs TWO patterns, because the
-# primary/secondary split is the whole point of the file. Both default to dev:
-#   ansible-playbook playbooks/k3s.yml                       # dev, unchanged
-#   ansible-playbook playbooks/k3s.yml \
-#     -e k3s_primary_hosts=k3s_prod_primary \
-#     -e k3s_secondary_hosts=k3s_prod_secondary              # prod
-# The role's OWN "who is the primary" lookups (join token delegate_to, tls-san list)
-# do NOT use these — they follow the k3s_primary_group / k3s_servers_group group_vars,
-# which prod sets once in group_vars/k3s_prod/vars.yml. These two vars only choose which
-# hosts the plays RUN on, which Ansible resolves before any group_vars exist.
 
 - name: k3s primary (cluster-init)
-  hosts: "{{ k3s_primary_hosts | default('k3s_primary') }}"
+  hosts: k3s_primary
   become: true
   gather_facts: true
   roles:
@@ -27,7 +16,7 @@
 # an updated tls-san set) triggers the Restart k3s handler: restarting both
 # secondaries at once would drop etcd below quorum. One-at-a-time keeps 2/3 up.
 - name: k3s secondaries (join)
-  hosts: "{{ k3s_secondary_hosts | default('k3s_secondary') }}"
+  hosts: k3s_secondary
   become: true
   serial: 1
   gather_facts: true

infra/ansible/playbooks/kube-vip.yml

@@ -1,14 +1,9 @@
 ---
-# Deploy kube-vip as the control-plane HA VIP (192.168.1.49 on dev) for the kube-API.
+# Deploy kube-vip as the control-plane HA VIP (192.168.1.49) for the kube-API.
 # Control-plane ONLY (svc_enable=false) — Cilium owns Service LoadBalancing.
 # Writes the manifest into each server's k3s auto-deploy dir; k3s applies it.
-#
-# TARGET (R-ENV-02): `k3s_hosts` defaults to dev's `k3s_servers`; prod is
-#   ansible-playbook playbooks/kube-vip.yml -e k3s_hosts=k3s_prod
-# (the VIP itself comes from kube_vip_address, already scoped per environment in
-# group_vars — dev 192.168.1.49, prod 192.168.2.9).
 - name: Deploy kube-vip control-plane VIP
-  hosts: "{{ k3s_hosts | default('k3s_servers') }}"
+  hosts: k3s_servers
   become: true
   gather_facts: true
   roles:

infra/ansible/playbooks/node-baseline.yml

@@ -2,21 +2,8 @@
 # Node baseline — first-boot floor for k3s/Cilium/Longhorn, moved from cloud-init into
 # Ansible so Proxmox provisioning stays API-only (no SSH/SFTP to the hypervisor). Runs
 # FIRST in site.yml, before k3s. Idempotent; safe to re-run.
-#
-# TARGET (R-ENV-02): `k3s_hosts` defaults to dev's `k3s_servers`; prod is
-#   ansible-playbook playbooks/node-baseline.yml -e k3s_hosts=k3s_prod
-# See README.md "Targeting prod". The pattern is the one node-health.yml established.
-#
-# TAGS — this role is the only way `smartmontools` (and the rest of the package floor)
-# reaches a node, but its data-disk tasks need block devices that prod has NOT decided
-# yet (infra/PROD-PROVISIONING.md §2). Run JUST the package floor with:
-#   ansible-playbook playbooks/node-baseline.yml -e k3s_hosts=k3s_prod --tags packages
-# `--tags packages` runs the apt task ONLY; every disk task carries the `disks` tag and
-# is therefore skipped, so the BLOCKED placeholder devices can never be touched. A full
-# (untagged) run against a node whose device vars are still placeholders fails fast on an
-# explicit assert instead of handing a bogus path to mkfs. See roles/node_baseline.
 - name: Baseline the k3s nodes (packages, kernel modules, Longhorn disk, swap-off)
-  hosts: "{{ k3s_hosts | default('k3s_servers') }}"
+  hosts: k3s_servers
   become: true
   gather_facts: true
   roles:

infra/ansible/playbooks/node-health.yml

@@ -1,32 +0,0 @@
----
-# Node hardware health — READ-ONLY inspection of a node's disks, memory, CPU, thermals,
-# network and pressure-stall state. Installs nothing, changes nothing, safe to run at any
-# time against dev or prod (see roles/node_health for the full contract).
-#
-#   ./dev.sh run bash -c 'cd /work/infra/ansible && \
-#       ansible-playbook playbooks/node-health.yml -e node_health_hosts=k3s_prod'
-#
-# TARGET GROUP IS A VARIABLE. This play established that pattern; the rest of the node
-# playbooks (node-baseline.yml, k3s.yml, cilium.yml, kube-vip.yml, verify.yml,
-# flux-bootstrap.yml) now follow it and take `k3s_hosts` / `k3s_primary_hosts` /
-# `k3s_secondary_hosts` instead of the literal dev groups they used to hardcode — see
-# infra/ansible/README.md "Targeting prod". This one keeps its OWN var name because it is
-# a standalone diagnostic you point anywhere, not a step in the bring-up chain: it must
-# stay independently targetable (`-e node_health_hosts=<group>`) even while a bring-up run
-# is scoped elsewhere. Run it against prod with `-e node_health_hosts=k3s_prod`, or a
-# single box with `--limit`.
-#
-# BECOME: left at the play level exactly as the inventory sets it. The role itself pins
-# `ansible_become: false` on every unprivileged task and probes escalation once before
-# attempting the only root-only check (SMART), so this play behaves correctly on dev
-# (NOPASSWD sudo) and on prod (vaulted become password, or none at all) without any
-# per-environment flags. See roles/node_health/tasks/main.yml for why that has to be a
-# variable override rather than `become: false`.
-- name: Inspect node hardware health (read-only)
-  hosts: "{{ node_health_hosts | default('k3s_servers') }}"
-  # The role's own collector supersedes ansible.builtin.setup for this purpose (setup
-  # exposes no thermal zones, no EDAC, no PSI, no per-disk error counters), so skip the
-  # fact gathering round trip entirely.
-  gather_facts: false
-  roles:
-    - node_health

infra/ansible/playbooks/site.yml

@@ -2,18 +2,6 @@
 # Full dev/test cluster bring-up, in order (docs/02 sequencing; run AFTER
 # `tofu apply` has created and cloud-init has baselined the VMs):
 #   k3s HA install -> Cilium CNI -> verify readiness -> Flux + SOPS hand-off.
-#
-# TARGET (R-ENV-02): every imported play takes its host pattern from a var that
-# defaults to the dev group, so `ansible-playbook playbooks/site.yml` with no extra
-# vars is byte-for-byte the dev run it always was. Extra vars are global and reach
-# imported playbooks, so the whole chain retargets with three flags:
-#   ansible-playbook playbooks/site.yml \
-#     -e k3s_hosts=k3s_prod \
-#     -e k3s_primary_hosts=k3s_prod_primary \
-#     -e k3s_secondary_hosts=k3s_prod_secondary
-# DO NOT run that yet — prod is blocked on the disk-layout decision
-# (infra/PROD-PROVISIONING.md §2) and the §7 repo blockers, and the operator has ruled
-# the prod cluster stays uninitialized until all three nodes are present (§8).
 - name: Import - node baseline (packages, kernel modules, Longhorn disk, swap-off)
   ansible.builtin.import_playbook: node-baseline.yml
 

infra/ansible/playbooks/verify.yml

@@ -1,10 +1,7 @@
 ---
 # Verify Phase-1 exit gate #1: 3 nodes Ready, Cilium/Hubble healthy, no kube-proxy.
-#
-# TARGET (R-ENV-02): `k3s_hosts` defaults to dev's `k3s_servers`; prod is
-#   ansible-playbook playbooks/verify.yml -e k3s_hosts=k3s_prod
 - name: Verify cluster readiness
-  hosts: "{{ k3s_hosts | default('k3s_servers') }}"
+  hosts: k3s_servers
   become: false
   gather_facts: false
   roles:

infra/ansible/playbooks/vpn.egress.gateway.yml

@@ -3,23 +3,15 @@
 # (ADR-0015). Host infra — the split-tunnel OpenVPN service + policy routing + scoped
 # nftables kill-switch. The CiliumEgressGatewayPolicy that steers vpn-class pods to this
 # node's tun0 is authored separately as GitOps (platform/), against the same contract:
-#   * inventory group `vpn_egress_gateway`  (this play's default target)
+#   * inventory group `vpn_egress_gateway`  (this play's target)
 #   * k8s node label   homelab.keithjackson.dev/egress-gateway=true  (set by the role)
 #   * host tunnel iface tun0                 (split-tunnel; never the node default route)
 #
 # Secrets come from Ansible vault (inventory/group_vars/vpn_egress_gateway/vault.yml):
 #   vault_vpn_egress_gateway_pia_username, vault_vpn_egress_gateway_pia_password, vault_vpn_egress_gateway_pia_ovpn
 # Run:  ansible-playbook playbooks/vpn.egress.gateway.yml
-#
-# TARGET (R-ENV-02): the group is a var, defaulting to dev's `vpn_egress_gateway`. Prod
-# has NOT decided whether it runs an egress gateway at all, nor on which node
-# (infra/PROD-PROVISIONING.md §6, last box) — so there is deliberately no prod egress
-# group in the inventory yet. When there is, point this at it with
-#   -e vpn_egress_gateway_hosts=<that group>
-# and it needs its OWN vaulted PIA credentials group_vars; this play's vault is scoped to
-# the dev group.
 - name: Configure the node-level VPN egress gateway
-  hosts: "{{ vpn_egress_gateway_hosts | default('vpn_egress_gateway') }}"
+  hosts: vpn_egress_gateway
   become: true
   gather_facts: true          # package module + service facts need os facts
   roles:

infra/ansible/roles/cilium/defaults/main.yml

@@ -4,25 +4,22 @@
 # HelmRelease adopts config afterward. The parity-critical values
 # (kubeProxyReplacement, k8sServiceHost/Port) are templated from k3s_api_host.
 
-# Exact pin — MUST equal the platform/cilium HelmRelease chart.spec.version (1.20.1, an
+# Pin inside the platform/cilium HelmRelease window (>=1.16.0 <1.17.0).
-# exact pin, not a range). Renovate bumps both together.
+cilium_version: "1.16.5"
-cilium_version: "1.20.1"
 cilium_chart_repo_name: "cilium"
 cilium_chart_repo_url: "https://helm.cilium.io/"
 cilium_namespace: "kube-system"
 
 # Gateway API CRDs (standard channel) must exist BEFORE Cilium with gatewayAPI.enabled.
-# Cilium 1.20.1 targets Gateway API v1.6.1.
+# Cilium 1.16 targets Gateway API v1.2.x.
-gateway_api_version: "v1.6.1"
+gateway_api_version: "v1.2.1"
 gateway_api_crd_urls:
   - "https://github.com/kubernetes-sigs/gateway-api/releases/download/{{ gateway_api_version }}/standard-install.yaml"
   # TLSRoute (experimental) is required by Cilium's Gateway API support.
   - "https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/{{ gateway_api_version }}/config/crd/experimental/gateway.networking.k8s.io_tlsroutes.yaml"
 
-# Local kubeconfig fetched from the primary. *.kubeconfig is gitignored. The FILENAME
+# Local kubeconfig fetched from the primary. *.kubeconfig is gitignored.
-# comes from cluster_kubeconfig_name (group_vars) so a prod run cannot overwrite the dev
+cilium_kubeconfig: "{{ playbook_dir }}/../.cache/dev.kubeconfig"
-# kubeconfig the dev container exports as $KUBECONFIG. R-ENV-02.
-cilium_kubeconfig: "{{ playbook_dir }}/../.cache/{{ cluster_kubeconfig_name | default('dev.kubeconfig') }}"
 cilium_values_path: "{{ playbook_dir }}/../.cache/cilium-values.yaml"
 
 # dev: single operator replica to save RAM (PHASE1-SCOPE RAM ceiling). NOT a

infra/ansible/roles/cilium/tasks/main.yml

@@ -4,14 +4,11 @@
 # The cluster is NotReady until this completes — expected (PHASE1-SCOPE risks).
 # run_once: this whole role acts on the cluster once, not per host.
 
-# k3s_primary_group (group_vars) names THIS environment's primary group — `k3s_primary`
-# on dev, `k3s_prod_primary` on prod — so the role fetches the right cluster's kubeconfig
-# without being forked per environment (R-ENV-02).
 - name: Fetch the k3s kubeconfig from the primary
   ansible.builtin.slurp:
     src: /etc/rancher/k3s/k3s.yaml
   register: k3s_kubeconfig_raw
-  delegate_to: "{{ groups[k3s_primary_group][0] }}"
+  delegate_to: "{{ groups['k3s_primary'][0] }}"
   run_once: true
 
 - name: Ensure the local cache dir exists

infra/ansible/roles/cluster_verify/defaults/main.yml

@@ -1,6 +1,4 @@
 ---
 # cluster_verify role defaults.
-# Filename comes from cluster_kubeconfig_name (group_vars) so dev and prod never share
+cluster_verify_kubeconfig: "{{ playbook_dir }}/../.cache/dev.kubeconfig"
-# one cache file — dev.kubeconfig for dev, prod.kubeconfig for prod. R-ENV-02.
-cluster_verify_kubeconfig: "{{ playbook_dir }}/../.cache/{{ cluster_kubeconfig_name | default('dev.kubeconfig') }}"
 cluster_verify_node_ready_timeout: "300s"

infra/ansible/roles/flux_bootstrap/defaults/main.yml

@@ -7,8 +7,7 @@
 # args below are ASSUMPTIONS the PM must reconcile with those deliverables — see
 # roles/flux_bootstrap/tasks/main.yml and infra/ansible/README.md "Interface".
 
-# Filename from cluster_kubeconfig_name (group_vars): dev.kubeconfig / prod.kubeconfig.
+flux_bootstrap_kubeconfig: "{{ playbook_dir }}/../.cache/dev.kubeconfig"
-flux_bootstrap_kubeconfig: "{{ playbook_dir }}/../.cache/{{ cluster_kubeconfig_name | default('dev.kubeconfig') }}"
 
 # Repo root, relative to playbooks/ (infra/ansible/playbooks -> ../../..).
 flux_bootstrap_repo_root: "{{ playbook_dir }}/../../.."