feat(ansible): parameterise target groups so prod reuses the roles unchanged (R-ENV-02)

roles/k3s_server, roles/cilium and every node-host playbook hardcoded the dev group
names k3s_primary/k3s_secondary/k3s_servers, so the prod inventory groups could not be
selected at all. That broke the R-ENV-02 promise that prod reuses the SAME roles, and
blocked PROD-PROVISIONING.md §6 outright.

TWO KNOBS, deliberately separate — Ansible templates a play's `hosts:` BEFORE any host
context exists, so group_vars is unreadable there, while roles run WITH host context:
  * which hosts a play RUNS ON — k3s_hosts, k3s_primary_hosts/k3s_secondary_hosts
    (k3s.yml's split), vpn_egress_gateway_hosts; each `| default(<dev group>)`, set by -e
    only. Follows node-health.yml's established `<name>_hosts | default(dev)` pattern.
  * which group a ROLE means by "my primary"/"my servers" — k3s_primary_group,
    k3s_servers_group in group_vars/all.yml (dev), overridden in group_vars/k3s_prod.
    Follows the k3s_api_host convention already in this repo. No per-run flag.

DEV IS UNCHANGED — the key evidence. --list-hosts/--list-tasks/--syntax-check captured
for all 9 playbooks at HEAD and after: sorted host lists IDENTICAL for all 9, same
primary/secondary split, same 6-play site.yml. (Raw --list-hosts PRINT ORDER varies run
to run; proven to be Ansible's own set ordering by running UNMODIFIED HEAD cilium.yml
four times -> 3,1,2 / 1,3,2 / 1,2,3 / 1,2,3. Hence sorted comparison.) --list-tasks
differs in exactly two files, only by added TAGS annotations and one new assert; no task
reordered, renamed or removed. Var resolution verified per environment with -c local:
dev -> k3s_primary/192.168.1.50/sans .50,.49,127.0.0.1,.51,.52; prod ->
k3s_prod_primary/192.168.2.10/vip .9/sans .10,.9,127.0.0.1,.11,.12.

BREAKS THE SMARTMONTOOLS CIRCULAR DEPENDENCY. node_baseline tasks are now tagged
packages/system/disks, so the package install can run alone:
  ansible-playbook playbooks/node-baseline.yml -e k3s_hosts=k3s_prod --tags packages
Skipping is the guard — no disks-tagged task is selected, so the
BLOCKED-pending-disk-decision paths are never templated, let alone passed to mkfs.
Belt-and-braces, a new disks-tagged assert fires first in a FULL run and names §2
explicitly instead of failing obscurely at mkfs. `tags: never` was deliberately NOT used
(it would change the default run).

FOOTGUN FOUND AND FIXED: roles/cilium, cluster_verify and flux_bootstrap all hardcoded
.cache/dev.kubeconfig — the exact file infra/devcontainer/dev.sh exports as $KUBECONFIG.
Once prod became targetable, a prod cilium/verify run would have SILENTLY OVERWRITTEN it,
leaving the operator's kubectl quietly answering from PROD. Now cluster_kubeconfig_name
(dev.kubeconfig -> prod.kubeconfig). Dev path byte-identical.

KNOWN, DOCUMENTED, NOT FIXED HERE: flux-bootstrap.yml retargets but roles/flux_bootstrap
shells out to bootstrap/flux/bootstrap.sh with a hardcoded --path=clusters/dev — a
bootstrap/ subtree change, not a target-group change (§6, cross-referenced to §7).
vpn.egress.gateway.yml is parameterised but prod has no egress group yet and would need
its own vaulted PIA credentials. --check against prod is still impossible until the prod
vault exists — even the packages-only run needs it.

Nothing was run against the dev cluster: --list-hosts/--list-tasks/--syntax-check, plus
debug/assert with -c local, only.

Serves: R-ENV-01, R-ENV-02, R-ENV-03. Task: T-7.1.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

infra/ansible/playbooks/cilium.yml

@@ -1,8 +1,13 @@
 ---
 # Install Cilium as the CNI (ADR-0002). The cluster is NotReady until this runs.
 # Executed from the controller against a fetched kubeconfig (role handles delegation).
+#
+# TARGET (R-ENV-02): `k3s_hosts` defaults to dev's `k3s_servers`; prod is
+#   ansible-playbook playbooks/cilium.yml -e k3s_hosts=k3s_prod
+# The role's "fetch the kubeconfig from the primary" delegation follows the
+# k3s_primary_group group_var, so it picks the prod primary automatically.
 - name: Install Cilium CNI
-  hosts: k3s_servers
+  hosts: "{{ k3s_hosts | default('k3s_servers') }}"
   become: true
   gather_facts: false
   roles:

infra/ansible/playbooks/flux-bootstrap.yml

@@ -1,8 +1,15 @@
 ---
 # FINAL step: hand off to Flux + SOPS (T-1.4). Calls into bootstrap/ (authored by
 # the Security + GitOps Engineers) — this play does not author that subtree.
+#
+# TARGET (R-ENV-02): `k3s_hosts` defaults to dev's `k3s_servers`; prod is
+#   ansible-playbook playbooks/flux-bootstrap.yml -e k3s_hosts=k3s_prod
+# NOTE the Flux PATH is a separate concern: roles/flux_bootstrap shells out to
+# bootstrap/flux/bootstrap.sh, which bootstraps --path=clusters/dev. Prod needs
+# --path=clusters/prod, which is a bootstrap/ change, NOT a target-group change
+# (infra/PROD-PROVISIONING.md §7 tracks it).
 - name: Flux + SOPS bootstrap hand-off
-  hosts: k3s_servers
+  hosts: "{{ k3s_hosts | default('k3s_servers') }}"
   become: false
   gather_facts: false
   roles:

infra/ansible/playbooks/k3s.yml

@@ -2,9 +2,20 @@
 # Install the HA k3s control plane (embedded etcd, 3 servers) — ADR-0001.
 # Primary founds the cluster (--cluster-init); secondaries join it in a second
 # play so the primary's node-token fact is available to them.
+#
+# TARGET (R-ENV-02): this is the one playbook that needs TWO patterns, because the
+# primary/secondary split is the whole point of the file. Both default to dev:
+#   ansible-playbook playbooks/k3s.yml                       # dev, unchanged
+#   ansible-playbook playbooks/k3s.yml \
+#     -e k3s_primary_hosts=k3s_prod_primary \
+#     -e k3s_secondary_hosts=k3s_prod_secondary              # prod
+# The role's OWN "who is the primary" lookups (join token delegate_to, tls-san list)
+# do NOT use these — they follow the k3s_primary_group / k3s_servers_group group_vars,
+# which prod sets once in group_vars/k3s_prod/vars.yml. These two vars only choose which
+# hosts the plays RUN on, which Ansible resolves before any group_vars exist.
 
 - name: k3s primary (cluster-init)
-  hosts: k3s_primary
+  hosts: "{{ k3s_primary_hosts | default('k3s_primary') }}"
   become: true
   gather_facts: true
   roles:
@@ -16,7 +27,7 @@
 # an updated tls-san set) triggers the Restart k3s handler: restarting both
 # secondaries at once would drop etcd below quorum. One-at-a-time keeps 2/3 up.
 - name: k3s secondaries (join)
-  hosts: k3s_secondary
+  hosts: "{{ k3s_secondary_hosts | default('k3s_secondary') }}"
   become: true
   serial: 1
   gather_facts: true

infra/ansible/playbooks/kube-vip.yml

@@ -1,9 +1,14 @@
 ---
-# Deploy kube-vip as the control-plane HA VIP (192.168.1.49) for the kube-API.
+# Deploy kube-vip as the control-plane HA VIP (192.168.1.49 on dev) for the kube-API.
 # Control-plane ONLY (svc_enable=false) — Cilium owns Service LoadBalancing.
 # Writes the manifest into each server's k3s auto-deploy dir; k3s applies it.
+#
+# TARGET (R-ENV-02): `k3s_hosts` defaults to dev's `k3s_servers`; prod is
+#   ansible-playbook playbooks/kube-vip.yml -e k3s_hosts=k3s_prod
+# (the VIP itself comes from kube_vip_address, already scoped per environment in
+# group_vars — dev 192.168.1.49, prod 192.168.2.9).
 - name: Deploy kube-vip control-plane VIP
-  hosts: k3s_servers
+  hosts: "{{ k3s_hosts | default('k3s_servers') }}"
   become: true
   gather_facts: true
   roles:

infra/ansible/playbooks/node-baseline.yml

@@ -2,8 +2,21 @@
 # Node baseline — first-boot floor for k3s/Cilium/Longhorn, moved from cloud-init into
 # Ansible so Proxmox provisioning stays API-only (no SSH/SFTP to the hypervisor). Runs
 # FIRST in site.yml, before k3s. Idempotent; safe to re-run.
+#
+# TARGET (R-ENV-02): `k3s_hosts` defaults to dev's `k3s_servers`; prod is
+#   ansible-playbook playbooks/node-baseline.yml -e k3s_hosts=k3s_prod
+# See README.md "Targeting prod". The pattern is the one node-health.yml established.
+#
+# TAGS — this role is the only way `smartmontools` (and the rest of the package floor)
+# reaches a node, but its data-disk tasks need block devices that prod has NOT decided
+# yet (infra/PROD-PROVISIONING.md §2). Run JUST the package floor with:
+#   ansible-playbook playbooks/node-baseline.yml -e k3s_hosts=k3s_prod --tags packages
+# `--tags packages` runs the apt task ONLY; every disk task carries the `disks` tag and
+# is therefore skipped, so the BLOCKED placeholder devices can never be touched. A full
+# (untagged) run against a node whose device vars are still placeholders fails fast on an
+# explicit assert instead of handing a bogus path to mkfs. See roles/node_baseline.
 - name: Baseline the k3s nodes (packages, kernel modules, Longhorn disk, swap-off)
-  hosts: k3s_servers
+  hosts: "{{ k3s_hosts | default('k3s_servers') }}"
   become: true
   gather_facts: true
   roles:

infra/ansible/playbooks/node-health.yml

@@ -6,13 +6,15 @@
 #   ./dev.sh run bash -c 'cd /work/infra/ansible && \
 #       ansible-playbook playbooks/node-health.yml -e node_health_hosts=k3s_prod'
 #
-# TARGET GROUP IS A VARIABLE, unlike the other playbooks here. Every existing node
-# playbook (node-baseline.yml, k3s.yml, cilium.yml, kube-vip.yml, verify.yml) hardcodes
-# `hosts: k3s_servers` / `k3s_primary` / `k3s_secondary`, which is exactly why they cannot
-# select the prod groups today (the gap flagged in inventory/hosts.ini's prod section, and
-# the R-ENV-02 reuse promise it threatens). A health check is the one play you want to be
-# able to point anywhere, so it takes the pattern as a var and defaults to dev. Run it
-# against prod with `-e node_health_hosts=k3s_prod`, or a single box with `--limit`.
+# TARGET GROUP IS A VARIABLE. This play established that pattern; the rest of the node
+# playbooks (node-baseline.yml, k3s.yml, cilium.yml, kube-vip.yml, verify.yml,
+# flux-bootstrap.yml) now follow it and take `k3s_hosts` / `k3s_primary_hosts` /
+# `k3s_secondary_hosts` instead of the literal dev groups they used to hardcode — see
+# infra/ansible/README.md "Targeting prod". This one keeps its OWN var name because it is
+# a standalone diagnostic you point anywhere, not a step in the bring-up chain: it must
+# stay independently targetable (`-e node_health_hosts=<group>`) even while a bring-up run
+# is scoped elsewhere. Run it against prod with `-e node_health_hosts=k3s_prod`, or a
+# single box with `--limit`.
 #
 # BECOME: left at the play level exactly as the inventory sets it. The role itself pins
 # `ansible_become: false` on every unprivileged task and probes escalation once before

infra/ansible/playbooks/site.yml

@@ -2,6 +2,18 @@
 # Full dev/test cluster bring-up, in order (docs/02 sequencing; run AFTER
 # `tofu apply` has created and cloud-init has baselined the VMs):
 #   k3s HA install -> Cilium CNI -> verify readiness -> Flux + SOPS hand-off.
+#
+# TARGET (R-ENV-02): every imported play takes its host pattern from a var that
+# defaults to the dev group, so `ansible-playbook playbooks/site.yml` with no extra
+# vars is byte-for-byte the dev run it always was. Extra vars are global and reach
+# imported playbooks, so the whole chain retargets with three flags:
+#   ansible-playbook playbooks/site.yml \
+#     -e k3s_hosts=k3s_prod \
+#     -e k3s_primary_hosts=k3s_prod_primary \
+#     -e k3s_secondary_hosts=k3s_prod_secondary
+# DO NOT run that yet — prod is blocked on the disk-layout decision
+# (infra/PROD-PROVISIONING.md §2) and the §7 repo blockers, and the operator has ruled
+# the prod cluster stays uninitialized until all three nodes are present (§8).
 - name: Import - node baseline (packages, kernel modules, Longhorn disk, swap-off)
   ansible.builtin.import_playbook: node-baseline.yml
 

infra/ansible/playbooks/verify.yml

@@ -1,7 +1,10 @@
 ---
 # Verify Phase-1 exit gate #1: 3 nodes Ready, Cilium/Hubble healthy, no kube-proxy.
+#
+# TARGET (R-ENV-02): `k3s_hosts` defaults to dev's `k3s_servers`; prod is
+#   ansible-playbook playbooks/verify.yml -e k3s_hosts=k3s_prod
 - name: Verify cluster readiness
-  hosts: k3s_servers
+  hosts: "{{ k3s_hosts | default('k3s_servers') }}"
   become: false
   gather_facts: false
   roles:

infra/ansible/playbooks/vpn.egress.gateway.yml

@@ -3,15 +3,23 @@
 # (ADR-0015). Host infra — the split-tunnel OpenVPN service + policy routing + scoped
 # nftables kill-switch. The CiliumEgressGatewayPolicy that steers vpn-class pods to this
 # node's tun0 is authored separately as GitOps (platform/), against the same contract:
-#   * inventory group `vpn_egress_gateway`  (this play's target)
+#   * inventory group `vpn_egress_gateway`  (this play's default target)
 #   * k8s node label   homelab.keithjackson.dev/egress-gateway=true  (set by the role)
 #   * host tunnel iface tun0                 (split-tunnel; never the node default route)
 #
 # Secrets come from Ansible vault (inventory/group_vars/vpn_egress_gateway/vault.yml):
 #   vault_vpn_egress_gateway_pia_username, vault_vpn_egress_gateway_pia_password, vault_vpn_egress_gateway_pia_ovpn
 # Run:  ansible-playbook playbooks/vpn.egress.gateway.yml
+#
+# TARGET (R-ENV-02): the group is a var, defaulting to dev's `vpn_egress_gateway`. Prod
+# has NOT decided whether it runs an egress gateway at all, nor on which node
+# (infra/PROD-PROVISIONING.md §6, last box) — so there is deliberately no prod egress
+# group in the inventory yet. When there is, point this at it with
+#   -e vpn_egress_gateway_hosts=<that group>
+# and it needs its OWN vaulted PIA credentials group_vars; this play's vault is scoped to
+# the dev group.
 - name: Configure the node-level VPN egress gateway
-  hosts: vpn_egress_gateway
+  hosts: "{{ vpn_egress_gateway_hosts | default('vpn_egress_gateway') }}"
   become: true
   gather_facts: true          # package module + service facts need os facts
   roles: