docs(prod): open Phase 7 (prep only) + T-7.1 bare-metal provisioning checklist

The operator added the first production node (k3s-prod-1, 192.168.2.10 — HP EliteDesk
800 G5 Mini, i7-9700T/31GiB, Debian 13, VT-x + /dev/kvm) and opened roadmap Phase 7 for
PREP WORK ONLY: no k3s install until the R-ENV-02 blockers close and the gate is green.

Records the Phase-7 entry-gate position honestly: the operator has declared hardware
ready, but "Phases 1-6 proven in dev/test" is NOT met (37/51 conformance; R-ENV-01/02/03
themselves uncovered). Proceeding is an explicit, accepted risk — not an oversight.

Adds infra/PROD-PROVISIONING.md (T-7.1, design-only): firmware/BIOS, disk layout, OS,
network, access, Ansible reuse, repo blockers, build sequence, Phase-7 exit criteria.

Key findings recorded in DECISIONS-LOG:
- R-ENV-02 is violated in the base — dev LAN addresses hardcoded into base manifests
  (cilium k8sServiceHost, the Cilium LB pool, crafty/UniFi LB IPs, 3 S3 backup
  endpoints). Prod is on 192.168.2.x, so prod would build WRONG today.
- clusters/prod never sets GATEWAY_VIP -> would inherit the base default 10.73.0.240,
  an address on the RETIRED libvirt network.
- No NOPASSWD sudo on the prod node -> all Ansible fails at its first become.
- node_baseline device defaults are virtio (/dev/vdb,vdc) and absent on bare metal.
- ESCALATION: agents/task-board.md is structurally unmaintainable — AGENTS.md designates
  it the one agent-editable surface, but ci/spec-immutability.sh fails the gate on ANY
  spec-repo dirt with no path exception. Explains the all-todo board after six phases.
  NOT worked around: exempting it would weaken an R-TEST-06 gate. Operator decision.

Operator decisions: Phase 7 open (prep only); wait for all 3 nodes before cluster-init;
check drive health before choosing the single-SSD disk layout.

Serves: R-ENV-01, R-ENV-02, R-ENV-03, R-PLAT-01/03. Task: T-7.1.
Gate: MERGE GATE: PASS.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

DECISIONS-LOG.md

@@ -1250,3 +1250,102 @@ fresh agent should read this entry + `infra/PHASE1-STATUS.md` + `docs/plans/` an
 4. **Remaining Phase-6 onboarding** — see DESIGN-waves.md / `catalog/` (services rendered-but-not-onboarded).
 5. **Vendor platform/kubevirt** remote bases proactively (same flake class as #8).
 6. **Human action**: finish Headlamp login = complete TOTP enrollment in the browser.
+
+---
+
+## 2026-08-31 — **PHASE 7 OPENED (prep only)**: first prod node on the bench
+
+The operator added the first **production** node and opened roadmap Phase 7. This entry records the
+opening, the state of the hardware, the operator's three decisions, and the blockers that stand
+between "a node exists" and "prod reconciles from `clusters/prod`".
+
+### Governance — Phase 7 entry gate, recorded not waived
+
+`roadmap/phases.md` gates Phase 7 on **two** conditions: *"Phases 1–6 proven in dev/test; operator
+declares prod hardware ready."* The operator has declared hardware ready. **The first condition is
+not met** — conformance is 37/51, and the uncovered set includes **R-ENV-01/02/03**, the exact
+requirements prod provisioning serves. We proceed with *prep only* on that basis, with no prod
+install until the §7 repo blockers close and the gate is green. This is an accepted, explicit risk,
+not an oversight.
+
+### Node 1 as-found (`192.168.2.10`, verified by SSH 2026-08-31)
+
+Not blank — imaged and reachable. **HP EliteDesk 800 G5 Desktop Mini** (matches the spec's `3× mini-PC
+bare metal`), i7-9700T 8c/8t, **31 GiB RAM**, Debian 13 trixie / kernel 6.12, hostname already
+`k3s-prod-1`, `k3s` user reachable with the `home_network` key, VT-x + `/dev/kvm` present
+(KubeVirt-capable), k3s not installed. Firmware HP R21 Ver. 02.26.00.
+
+**One 477 GB SATA SSD** (`C400-MTFDDAC512M` — a ~2011-era Micron/Crucial consumer drive) carrying
+everything: `/` 30G, `/var` 12G, swap, and `/srv` 433G. No second device. Dev, by contrast, runs two
+dedicated data disks per node (Longhorn + a dedicated SSD for etcd, added after HDD contention
+starved etcd WAL fsync).
+
+### Operator decisions (2026-08-31)
+
+1. **Phase 7 opens, prep work only** — repo work and node baselining now; no k3s install until the
+   parameterization blockers land and the gate is green.
+2. **Wait for all three nodes before `cluster-init`** — prod comes up as spec-conformant 3-node HA
+   embedded etcd on day one; no single-server-etcd window, no later member-add.
+3. **Check drive health before choosing the disk layout** — a 15-year-old consumer SSD is not an
+   obvious home for prod state.
+
+### Findings — prod would build WRONG today
+
+1. **R-ENV-02 is violated in the base (the headline blocker).** The overlay model promises dev and
+   prod differ *only* by `cluster-vars` values. In fact dev LAN addresses are hardcoded into base
+   manifests: `k8sServiceHost: 192.168.1.50` (`platform/cilium/helmrelease.yaml:50` + the
+   `values.yaml` mirror), the Cilium LB pool `192.168.1.53-.60`
+   (`platform-config/cilium/l2-ip-pool.yaml`), crafty `.55`, the UniFi VM `.54`, and three S3 backup
+   endpoints at `192.168.1.9:9000`. **Prod is on `192.168.2.x` — none of these follow.** Fix is to
+   tokenize each into a `cluster-vars` knob and set it in both overlays; the `postBuild.substituteFrom`
+   seam already exists on every relevant Kustomization. Note `platform/cilium/values.yaml` is only a
+   *reference mirror* — the authoritative pair is the HelmRelease (Flux) and the Ansible j2 template
+   (already parameterized), so the Ansible side needs no change.
+2. **`clusters/prod` never sets `GATEWAY_VIP`** → it would inherit the base default `10.73.0.240`,
+   an address on the **retired libvirt network**. `cluster-vars`' own comment flags this as owed
+   "when hardware is built".
+3. **No NOPASSWD sudo for `k3s` on node 1** — `sudo -n` demands a password, so every Ansible
+   playbook fails at its first `become`. Dev nodes have it; prod must match. Operator action.
+4. **`node_baseline` device defaults are virtio** (`/dev/vdb`, `/dev/vdc`) and do not exist on bare
+   metal; the role makes a filesystem on a *block device*, so a single-disk node needs either a
+   second drive, `/dev/sda5`, or a repartition. Prod `group_vars` must override.
+5. **Single upstream resolver on node 1** (`192.168.2.1`) — the same SPOF class that broke
+   containerd pulls and CoreDNS forwarding on dev. The fix is **stranded unmerged** on
+   `fix/node-dns-resilient-upstream`.
+
+### ESCALATION — the task board is structurally unmaintainable
+
+`AGENTS.md` §11 and `coordination-protocol.md` §1 designate `agents/task-board.md` as *the one
+surface agents may edit* — claim items, update status, raise `OQ-*`. But `ci/spec-immutability.sh`
+fails the merge gate on **any** dirt in the spec repo, with **no path exception** for the board, and
+committing the board there moves spec `HEAD` off `ci/SPEC_PIN` — which also fails. So editing the
+board as instructed **blocks every merge**.
+
+This fully explains why the board still reads all-`todo` after six delivered phases, and why
+`OQ-BKP` is still marked open although Velero + CNPG Barman → S3 landed on `main`. It is not
+neglect — the gate made the board unusable, and coordination state migrated here instead.
+
+I did **not** work around it: adding a path exception would weaken an R-TEST-06 gate, which the
+protocol forbids ("never weaken a test to get green"). Options for the operator/architect:
+
+- **(a) Preferred** — the human commits board updates in the spec repo, and the impl repo adopts each
+  via a reviewed `ci/SPEC_PIN` bump. That is exactly the flow `SPEC_PIN`'s own comment describes, and
+  it keeps both the board truthful and the gate honest. Cost: a pin bump per board update.
+- **(b)** Amend `spec-immutability.sh` to exempt `agents/task-board.md` only. Lower friction,
+  but it narrows an immutability gate — a **human** decision, not an agent's.
+- **(c)** Formally declare `DECISIONS-LOG.md` the board of record and retire the spec board. Matches
+  practice, but contradicts the spec as written, so it needs a spec amendment.
+
+Until this is resolved, Phase-7 coordination state lives in this log.
+
+### Next steps
+
+1. **Operator:** configure NOPASSWD sudo on `k3s-prod-1`, then capture `smartctl -a /dev/sda`
+   (commands in `infra/PROD-PROVISIONING.md` §5). The disk decision waits on that output.
+2. **Land the R-ENV-02 tokenization** — the critical path for prod reconciling at all. Dev overlay
+   must set every new knob to today's exact value so live dev sees zero drift (`make -C ci render-golden`).
+3. **Complete `clusters/prod`** — `GATEWAY_VIP`, the L2 pool, and the prod backup target.
+4. **Land the two stranded fix branches** (`fix/node-dns-resilient-upstream`,
+   `fix/act-runner-dind-readiness`) — the DNS one now matters for prod too.
+5. Resolve the task-board escalation above.
+6. Nodes 2 and 3 through `infra/PROD-PROVISIONING.md` §1–§5 as they arrive.