feat(ansible): parameterise target groups so prod reuses the roles unchanged (R-ENV-02)

roles/k3s_server, roles/cilium and every node-host playbook hardcoded the dev group
names k3s_primary/k3s_secondary/k3s_servers, so the prod inventory groups could not be
selected at all. That broke the R-ENV-02 promise that prod reuses the SAME roles, and
blocked PROD-PROVISIONING.md §6 outright.

TWO KNOBS, deliberately separate — Ansible templates a play's `hosts:` BEFORE any host
context exists, so group_vars is unreadable there, while roles run WITH host context:
  * which hosts a play RUNS ON — k3s_hosts, k3s_primary_hosts/k3s_secondary_hosts
    (k3s.yml's split), vpn_egress_gateway_hosts; each `| default(<dev group>)`, set by -e
    only. Follows node-health.yml's established `<name>_hosts | default(dev)` pattern.
  * which group a ROLE means by "my primary"/"my servers" — k3s_primary_group,
    k3s_servers_group in group_vars/all.yml (dev), overridden in group_vars/k3s_prod.
    Follows the k3s_api_host convention already in this repo. No per-run flag.

DEV IS UNCHANGED — the key evidence. --list-hosts/--list-tasks/--syntax-check captured
for all 9 playbooks at HEAD and after: sorted host lists IDENTICAL for all 9, same
primary/secondary split, same 6-play site.yml. (Raw --list-hosts PRINT ORDER varies run
to run; proven to be Ansible's own set ordering by running UNMODIFIED HEAD cilium.yml
four times -> 3,1,2 / 1,3,2 / 1,2,3 / 1,2,3. Hence sorted comparison.) --list-tasks
differs in exactly two files, only by added TAGS annotations and one new assert; no task
reordered, renamed or removed. Var resolution verified per environment with -c local:
dev -> k3s_primary/192.168.1.50/sans .50,.49,127.0.0.1,.51,.52; prod ->
k3s_prod_primary/192.168.2.10/vip .9/sans .10,.9,127.0.0.1,.11,.12.

BREAKS THE SMARTMONTOOLS CIRCULAR DEPENDENCY. node_baseline tasks are now tagged
packages/system/disks, so the package install can run alone:
  ansible-playbook playbooks/node-baseline.yml -e k3s_hosts=k3s_prod --tags packages
Skipping is the guard — no disks-tagged task is selected, so the
BLOCKED-pending-disk-decision paths are never templated, let alone passed to mkfs.
Belt-and-braces, a new disks-tagged assert fires first in a FULL run and names §2
explicitly instead of failing obscurely at mkfs. `tags: never` was deliberately NOT used
(it would change the default run).

FOOTGUN FOUND AND FIXED: roles/cilium, cluster_verify and flux_bootstrap all hardcoded
.cache/dev.kubeconfig — the exact file infra/devcontainer/dev.sh exports as $KUBECONFIG.
Once prod became targetable, a prod cilium/verify run would have SILENTLY OVERWRITTEN it,
leaving the operator's kubectl quietly answering from PROD. Now cluster_kubeconfig_name
(dev.kubeconfig -> prod.kubeconfig). Dev path byte-identical.

KNOWN, DOCUMENTED, NOT FIXED HERE: flux-bootstrap.yml retargets but roles/flux_bootstrap
shells out to bootstrap/flux/bootstrap.sh with a hardcoded --path=clusters/dev — a
bootstrap/ subtree change, not a target-group change (§6, cross-referenced to §7).
vpn.egress.gateway.yml is parameterised but prod has no egress group yet and would need
its own vaulted PIA credentials. --check against prod is still impossible until the prod
vault exists — even the packages-only run needs it.

Nothing was run against the dev cluster: --list-hosts/--list-tasks/--syntax-check, plus
debug/assert with -c local, only.

Serves: R-ENV-01, R-ENV-02, R-ENV-03. Task: T-7.1.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

infra/PROD-PROVISIONING.md

@@ -84,7 +84,14 @@ Consequences to settle before imaging nodes 2 and 3:
       - **Repartition** into dedicated OS / etcd / Longhorn partitions. Better isolation, still one
         physical failure domain.
 - [ ] **All three nodes get the identical layout.** Longhorn replica placement assumes symmetry.
-- [ ] Record the chosen device paths here and set them in `inventory/group_vars/k3s_prod.yml`.
+- [ ] Record the chosen device paths here and set them in
+      `infra/ansible/inventory/group_vars/k3s_prod/vars.yml` (they are deliberately
+      `/dev/BLOCKED-pending-disk-decision-…` placeholders today — do **not** guess). A full
+      `node-baseline` run against a node still carrying those placeholders now stops on an explicit
+      assert naming this section, instead of failing obscurely at `mkfs` time.
+- [x] **The SMART-vs-smartmontools circular dependency is broken** (2026-08-31). `roles/node_baseline`
+      is tagged `packages` / `system` / `disks`, so the package floor can land **without** the disk
+      tasks this decision blocks — see §5's drive-health box for the exact command.
 
 ## 3. OS install (per node)
 
@@ -149,29 +156,81 @@ Consequences to settle before imaging nodes 2 and 3:
 - [ ] **Drive health baseline** (§2). **Do not hand-install the tooling** — package installation is
       the `node_baseline` role's job, never a manual `apt-get` on a node (operator ruling,
       2026-08-31; same click-ops-never principle as the rest of the repo). `smartmontools` is part
-      of `node_baseline_packages`, so it arrives with the baseline play. Reading the drive is then:
+      of `node_baseline_packages`, so it arrives with the baseline play — but a *full* baseline
+      would hit the disk tasks §2 has not decided yet. Run the **packages-only** baseline instead
+      (the `disks`-tagged tasks, including the BLOCKED-device assert, are not selected at all, so
+      the placeholder device paths are never even templated):
+      ```bash
+      cd infra/ansible
+      ansible-playbook playbooks/node-baseline.yml \
+        -e k3s_hosts=k3s_prod --tags packages --limit k3s-prod-1
+      ```
+      (Needs `group_vars/k3s_prod/vault.yml` to exist first — the become password above.) Reading
+      the drive is then:
       ```bash
       sudo smartctl -a /dev/sda   # Power_On_Hours, Percent_Lifetime_Remaining, reallocated sectors
       ```
+      Record the result in §2 and close the layout decision in the same change that replaces the
+      two `node_baseline_*_device` placeholders.
 
 ## 6. Ansible baseline & cluster join
 
-Reuses the dev roles unchanged — that reuse **is** R-ENV-02. Only inventory and group_vars differ.
+Reuses the dev roles **unchanged** — that reuse **is** R-ENV-02. Only inventory and group_vars
+differ, plus a target-group override on the command line.
 
-- [ ] Add a `k3s_prod` inventory group (`k3s_prod_primary` / `k3s_prod_secondary`). Prod is reached
+### How prod is targeted (done, 2026-08-31)
+
+Every node-host play used to hardcode the literal dev groups `k3s_servers` / `k3s_primary` /
+`k3s_secondary`, so nothing here could select prod at all. Two knobs now close that, deliberately
+kept separate because Ansible resolves them at different times:
+
+| Layer | Knob | Set via |
+|---|---|---|
+| Which hosts a play **runs on** | `k3s_hosts`, `k3s_primary_hosts`, `k3s_secondary_hosts` (each defaults to the dev group) | `-e` **extra vars only** — a play's `hosts:` is templated before any host exists, so `group_vars` is not readable there |
+| Which group a **role** means by "my primary" / "my servers" | `k3s_primary_group`, `k3s_servers_group`, `cluster_kubeconfig_name` | `group_vars` — dev in `all.yml`, prod in `k3s_prod/vars.yml`; **no per-run flag** |
+
+So `roles/k3s_server` finds the prod `--cluster-init` founder, builds the prod join target and
+tls-san list, and `roles/cilium` fetches the prod kubeconfig, with **no role fork**. Running with
+no extra vars is byte-for-byte the dev behaviour it always had (verified by a before/after
+`--list-hosts` diff on every playbook). Full detail: `infra/ansible/README.md` → **Targeting prod**.
+
+- [x] Add a `k3s_prod` inventory group (`k3s_prod_primary` / `k3s_prod_secondary`). Prod is reached
       **directly** on the LAN — no NAT port-forward hop, unlike dev's `192.168.2.102:225x` mapping.
-- [ ] `group_vars/k3s_prod.yml`: `kube_vip_address`, `cluster_controller_api_server`, and the §2
-      device overrides for `node_baseline_longhorn_device` / `node_baseline_etcd_device`.
-- [ ] Run `playbooks/node-baseline.yml` → packages, kernel modules, sysctls, iscsid, data-disk
-      filesystems + mounts. (`qemu-guest-agent` is a no-op on bare metal — the role already only
-      *starts* it when the virtio channel exists, so it degrades cleanly.)
-- [ ] Run `playbooks/k3s.yml` → HA embedded etcd, flannel/kube-proxy/Traefik/servicelb disabled.
-      **Only once all three nodes are present** — see §8.
-- [ ] Run `playbooks/kube-vip.yml` → control-plane VIP on the prod LAN.
-- [ ] Run `playbooks/cilium.yml` → CNI at bootstrap; Flux's HelmRelease adopts it afterwards. The
-      bootstrap values and the HelmRelease **must stay in parity** (`kubeProxyReplacement`,
-      `k8sServiceHost`/`Port`) or the two layers fight over CNI config (`docs/02`).
+      (Nodes 2/3 stay commented out until they are physically on the bench.)
+- [x] `inventory/group_vars/k3s_prod/vars.yml`: `kube_vip_address` (`192.168.2.9`),
+      `cluster_controller_api_server`, `k3s_api_host`, the group-name overrides above — **and** the
+      §2 device overrides for `node_baseline_longhorn_device` / `node_baseline_etcd_device`, which
+      are still BLOCKED placeholders.
+- [ ] `inventory/group_vars/k3s_prod/vault.yml` — the vaulted `ansible_become_password` (§5). Until
+      it exists **no** prod play can escalate, and `--check` against prod cannot work either.
+- [ ] Run the **packages-only** baseline first, to unblock the §2 drive-health read:
+      ```bash
+      ansible-playbook playbooks/node-baseline.yml -e k3s_hosts=k3s_prod --tags packages
+      ```
+- [ ] Run the **full** `playbooks/node-baseline.yml -e k3s_hosts=k3s_prod` → packages, kernel
+      modules, sysctls, iscsid, data-disk filesystems + mounts. **Blocked until §2 is decided**: it
+      asserts on the placeholder device paths and stops. (`qemu-guest-agent` is a no-op on bare
+      metal — the role already only *starts* it when the virtio channel exists, so it degrades
+      cleanly.)
+- [ ] Run `playbooks/k3s.yml -e k3s_primary_hosts=k3s_prod_primary -e k3s_secondary_hosts=k3s_prod_secondary`
+      → HA embedded etcd, flannel/kube-proxy/Traefik/servicelb disabled. **Only once all three
+      nodes are present** — see §8.
+- [ ] Run `playbooks/kube-vip.yml -e k3s_hosts=k3s_prod` → control-plane VIP (`192.168.2.9`) on the
+      prod LAN.
+- [ ] Run `playbooks/cilium.yml -e k3s_hosts=k3s_prod` → CNI at bootstrap; Flux's HelmRelease adopts
+      it afterwards. The bootstrap values and the HelmRelease **must stay in parity**
+      (`kubeProxyReplacement`, `k8sServiceHost`/`Port`) or the two layers fight over CNI config
+      (`docs/02`). Note the prod kubeconfig is fetched to `.cache/prod.kubeconfig`, never over the
+      dev one the dev container exports as `$KUBECONFIG`.
+- [ ] Run `playbooks/verify.yml -e k3s_hosts=k3s_prod`.
+- [ ] `playbooks/flux-bootstrap.yml -e k3s_hosts=k3s_prod` **retargets but is not yet correct**:
+      `roles/flux_bootstrap` shells out to `bootstrap/flux/bootstrap.sh`, which bootstraps
+      `--path=clusters/dev`. Prod needs `--path=clusters/prod` — a `bootstrap/` change, not a
+      target-group change. Tracked with the other repo blockers in §7.
 - [ ] Decide whether prod runs a VPN egress gateway node (dev uses `k3s-dev-3`, ADR-0015).
+      `playbooks/vpn.egress.gateway.yml` takes `-e vpn_egress_gateway_hosts=<group>`, but there is
+      deliberately no prod egress group in the inventory yet, and it would need its **own** vaulted
+      PIA credentials — the existing vault is scoped to the dev group.
 
 ## 7. Repo blockers — **prod cannot be reconciled until these land**