feat(ansible): parameterise target groups so prod reuses the roles unchanged (R-ENV-02)
roles/k3s_server, roles/cilium and every node-host playbook hardcoded the dev group
names k3s_primary/k3s_secondary/k3s_servers, so the prod inventory groups could not be
selected at all. That broke the R-ENV-02 promise that prod reuses the SAME roles, and
blocked PROD-PROVISIONING.md §6 outright.
TWO KNOBS, deliberately separate — Ansible templates a play's `hosts:` BEFORE any host
context exists, so group_vars is unreadable there, while roles run WITH host context:
* which hosts a play RUNS ON — k3s_hosts, k3s_primary_hosts/k3s_secondary_hosts
(k3s.yml's split), vpn_egress_gateway_hosts; each `| default(<dev group>)`, set by -e
only. Follows node-health.yml's established `<name>_hosts | default(dev)` pattern.
* which group a ROLE means by "my primary"/"my servers" — k3s_primary_group,
k3s_servers_group in group_vars/all.yml (dev), overridden in group_vars/k3s_prod.
Follows the k3s_api_host convention already in this repo. No per-run flag.
DEV IS UNCHANGED — the key evidence. --list-hosts/--list-tasks/--syntax-check captured
for all 9 playbooks at HEAD and after: sorted host lists IDENTICAL for all 9, same
primary/secondary split, same 6-play site.yml. (Raw --list-hosts PRINT ORDER varies run
to run; proven to be Ansible's own set ordering by running UNMODIFIED HEAD cilium.yml
four times -> 3,1,2 / 1,3,2 / 1,2,3 / 1,2,3. Hence sorted comparison.) --list-tasks
differs in exactly two files, only by added TAGS annotations and one new assert; no task
reordered, renamed or removed. Var resolution verified per environment with -c local:
dev -> k3s_primary/192.168.1.50/sans .50,.49,127.0.0.1,.51,.52; prod ->
k3s_prod_primary/192.168.2.10/vip .9/sans .10,.9,127.0.0.1,.11,.12.
BREAKS THE SMARTMONTOOLS CIRCULAR DEPENDENCY. node_baseline tasks are now tagged
packages/system/disks, so the package install can run alone:
ansible-playbook playbooks/node-baseline.yml -e k3s_hosts=k3s_prod --tags packages
Skipping is the guard — no disks-tagged task is selected, so the
BLOCKED-pending-disk-decision paths are never templated, let alone passed to mkfs.
Belt-and-braces, a new disks-tagged assert fires first in a FULL run and names §2
explicitly instead of failing obscurely at mkfs. `tags: never` was deliberately NOT used
(it would change the default run).
FOOTGUN FOUND AND FIXED: roles/cilium, cluster_verify and flux_bootstrap all hardcoded
.cache/dev.kubeconfig — the exact file infra/devcontainer/dev.sh exports as $KUBECONFIG.
Once prod became targetable, a prod cilium/verify run would have SILENTLY OVERWRITTEN it,
leaving the operator's kubectl quietly answering from PROD. Now cluster_kubeconfig_name
(dev.kubeconfig -> prod.kubeconfig). Dev path byte-identical.
KNOWN, DOCUMENTED, NOT FIXED HERE: flux-bootstrap.yml retargets but roles/flux_bootstrap
shells out to bootstrap/flux/bootstrap.sh with a hardcoded --path=clusters/dev — a
bootstrap/ subtree change, not a target-group change (§6, cross-referenced to §7).
vpn.egress.gateway.yml is parameterised but prod has no egress group yet and would need
its own vaulted PIA credentials. --check against prod is still impossible until the prod
vault exists — even the packages-only run needs it.
Nothing was run against the dev cluster: --list-hosts/--list-tasks/--syntax-check, plus
debug/assert with -c local, only.
Serves: R-ENV-01, R-ENV-02, R-ENV-03. Task: T-7.1.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
1 parent
1157c1b071
commit
e623d27e2d
22 files changed
+384
-76
No files matched your search
@@ -4,12 +4,35 @@
|
||||
# cloud-init created this login user (tofu var login_user); it has NOPASSWD sudo.
|
||||
ansible_user: k3s
|
||||
|
||||
# --- environment group NAMES (R-ENV-02: one set of roles, scoped overrides) ---------
|
||||
# The GROUP NAMES a role uses to answer "who is MY environment's primary / MY
|
||||
# environment's servers?". Roles must never hardcode `groups['k3s_primary']`, or they
|
||||
# only ever work against dev. These two are the dev values; group_vars/k3s_prod/vars.yml
|
||||
# overrides them with k3s_prod_primary / k3s_prod, so a host in the prod group resolves
|
||||
# its own primary automatically — exactly the way k3s_api_host below is overridden there.
|
||||
#
|
||||
# These are GROUP NAMES (keys into `groups`), NOT host patterns. The play-level target
|
||||
# pattern is a separate, extra-vars-only knob (`k3s_hosts` / `k3s_primary_hosts` /
|
||||
# `k3s_secondary_hosts`, see playbooks/*.yml) because Ansible templates a play's `hosts:`
|
||||
# before it has any host context and so cannot see group_vars at all.
|
||||
k3s_primary_group: k3s_primary
|
||||
k3s_servers_group: k3s_servers
|
||||
|
||||
# Filename of the kubeconfig the controller-side roles (cilium, cluster_verify,
|
||||
# flux_bootstrap) fetch from the primary into infra/ansible/.cache/. Scoped per
|
||||
# environment so pointing those plays at prod can NEVER overwrite the dev kubeconfig
|
||||
# that infra/devcontainer/dev.sh exports as $KUBECONFIG — an operator's `kubectl get
|
||||
# nodes` silently answering from prod would be a genuinely dangerous mix-up.
|
||||
# group_vars/k3s_prod/vars.yml overrides this to prod.kubeconfig. (.cache/ is gitignored;
|
||||
# a kubeconfig is a credential.)
|
||||
cluster_kubeconfig_name: dev.kubeconfig
|
||||
|
||||
# The kube-apiserver endpoint the whole stack agrees on: the primary's REAL node IP.
|
||||
# NOTE: ansible_host is now the pfSense NAT SSH target (192.168.2.102:225x), NOT the node
|
||||
# IP — so we read the real 192.168.1.x from k3s_node_ip instead. Used for k3s --tls-san /
|
||||
# secondary --server AND for Cilium k8sServiceHost so the Ansible bootstrap and the Flux
|
||||
# HelmRelease stay in PARITY (docs/02 contract).
|
||||
k3s_api_host: "{{ hostvars[groups['k3s_primary'][0]]['k3s_node_ip'] }}"
|
||||
k3s_api_host: "{{ hostvars[groups[k3s_primary_group][0]]['k3s_node_ip'] }}"
|
||||
k3s_api_port: 6443
|
||||
|
||||
# Control-plane HA VIP floated by kube-vip (roles/kube_vip). A stable failover
|
||||
|
||||
@@ -54,9 +54,15 @@ behind them instead of NOPASSWD.
|
||||
- This is the Ansible-bootstrap-layer secret (host credential), distinct from the platform's
|
||||
in-cluster **SOPS + age** secrets (ADR-0007) — different layer, different tool, same convention
|
||||
as `hypervisor`/`proxmox_hosts`.
|
||||
- **Targeting prod is solved** (2026-08-31): plays take `-e k3s_hosts=k3s_prod` (and, for `k3s.yml`,
|
||||
`-e k3s_primary_hosts=k3s_prod_primary -e k3s_secondary_hosts=k3s_prod_secondary`), while the
|
||||
roles resolve "my environment's primary/servers" from `k3s_primary_group` / `k3s_servers_group`
|
||||
in `vars.yml` here — no role fork, no per-run flag for that half. See
|
||||
`infra/ansible/README.md` → "Targeting prod".
|
||||
- **Do not run any prod playbook yet.** Phase 7 is prep-only: the vault above just makes the plays
|
||||
*able* to authenticate once the §7 repo blockers close and the disk-layout (§2) and
|
||||
control-plane-VIP (§4) decisions in `infra/PROD-PROVISIONING.md` are made — and, separately, once
|
||||
the group-name gap flagged in `inventory/hosts.ini`'s prod section is resolved (the existing
|
||||
playbooks target `k3s_servers` / `k3s_primary` / `k3s_secondary` literally, not `k3s_prod_*`).
|
||||
See that file for the full picture.
|
||||
*able* to authenticate once the §7 repo blockers close and the disk-layout (§2) decision in
|
||||
`infra/PROD-PROVISIONING.md` is made. The ONE exception the operator has sanctioned is the
|
||||
packages-only baseline that unblocks the §2 drive-health read —
|
||||
`ansible-playbook playbooks/node-baseline.yml -e k3s_hosts=k3s_prod --tags packages` — which
|
||||
cannot touch the BLOCKED device placeholders because every disk task is `disks`-tagged and so is
|
||||
not selected. It still needs `vault.yml` (above) to exist first.
|
||||
@@ -29,13 +29,34 @@ ansible_become: true
|
||||
# The password itself is ansible_become_password — ansible-vault-encrypted, in
|
||||
# vault.yml (this directory), never here in plaintext. See README.md to create it.
|
||||
|
||||
# --- environment group NAMES (R-ENV-02) -------------------------------------------
|
||||
# group_vars/all.yml defines these as the DEV group names (k3s_primary / k3s_servers);
|
||||
# here they name PROD's groups. Roles index `groups[k3s_primary_group]` /
|
||||
# `groups[k3s_servers_group]` instead of a literal, so roles/k3s_server and roles/cilium
|
||||
# resolve "my environment's primary" and "my environment's servers" correctly for prod
|
||||
# hosts with NO role fork and no per-run flag — the reuse R-ENV-02 promises.
|
||||
#
|
||||
# NOTE these are GROUP NAMES, not the play-level target pattern. Selecting which hosts a
|
||||
# play RUNS against is a separate extra-vars knob (`-e k3s_hosts=k3s_prod`, and for
|
||||
# k3s.yml `-e k3s_primary_hosts=k3s_prod_primary -e k3s_secondary_hosts=k3s_prod_secondary`),
|
||||
# because Ansible templates `hosts:` before any host — and therefore any group_vars —
|
||||
# exists. See infra/ansible/README.md "Targeting prod".
|
||||
k3s_primary_group: k3s_prod_primary
|
||||
k3s_servers_group: k3s_prod
|
||||
|
||||
# Keep the fetched prod kubeconfig in its OWN cache file. group_vars/all.yml defaults this
|
||||
# to dev.kubeconfig, which is the file infra/devcontainer/dev.sh exports as $KUBECONFIG —
|
||||
# a prod play must never clobber it (a `kubectl` that silently answers from prod is worse
|
||||
# than no kubeconfig at all). .cache/ is gitignored; a kubeconfig is a credential.
|
||||
cluster_kubeconfig_name: prod.kubeconfig
|
||||
|
||||
# --- k3s API host — safe to set now, the primary's real IP is already known -------
|
||||
# Dev's k3s_api_host (group_vars/all.yml) reads a separate k3s_node_ip hostvar because
|
||||
# dev's ansible_host is the pfSense NAT SSH target, not the node's real address. Prod
|
||||
# has NO NAT hop — ansible_host in hosts.ini IS already the node's real, routable LAN
|
||||
# address (see the KEY DIFFERENCE note there) — so point straight at it instead of
|
||||
# needing a k3s_node_ip split.
|
||||
k3s_api_host: "{{ hostvars[groups['k3s_prod_primary'][0]]['ansible_host'] }}"
|
||||
k3s_api_host: "{{ hostvars[groups[k3s_primary_group][0]]['ansible_host'] }}"
|
||||
# cluster_controller_tls_server_name (group_vars/all.yml) derives from kube_vip_address
|
||||
# by template reference, so it needs no separate override here — it picks up the real
|
||||
# value set below automatically for prod hosts too.
|
||||
|
||||
@@ -66,19 +66,21 @@ proxmox ansible_host=192.168.2.102 ansible_port=22 ansible_user=root ansible_ssh
|
||||
# `ansible_port` is the node's own real sshd (22). See group_vars/k3s_prod/vars.yml for
|
||||
# the k3s_api_host override that reflects this directly-reachable topology.
|
||||
#
|
||||
# GAP flagged for the PM, NOT fixed in this pass (inventory/group_vars only): the
|
||||
# k3s_server role hardcodes the LITERAL group names `k3s_primary` / `k3s_secondary` /
|
||||
# `k3s_servers` in roles/k3s_server/defaults/main.yml (tls-san SAN list) and
|
||||
# roles/k3s_server/tasks/main.yml (node-token delegate_to), and every node-host
|
||||
# playbook (node-baseline.yml, k3s.yml, cilium.yml, kube-vip.yml, verify.yml,
|
||||
# flux-bootstrap.yml) declares `hosts: k3s_servers` / `k3s_primary` / `k3s_secondary`
|
||||
# literally. Naming prod's groups k3s_prod_primary/k3s_prod_secondary — as this
|
||||
# checklist and PROD-PROVISIONING.md §6 both call for — means those existing
|
||||
# playbooks will NOT select prod hosts as they stand today. That's harmless right now
|
||||
# (nothing can accidentally run against prod), but it means actually reusing the dev
|
||||
# playbooks against prod later (the R-ENV-02 promise) needs either parametrizing the
|
||||
# group name they target or some other mechanism — not yet designed, tracked here so
|
||||
# it isn't lost before §6 is attempted for real.
|
||||
# GROUP-NAME GAP: CLOSED 2026-08-31. The roles and playbooks used to hardcode the
|
||||
# LITERAL dev groups (`hosts: k3s_servers` / `k3s_primary` / `k3s_secondary`, and
|
||||
# `groups['k3s_primary']` inside roles/k3s_server + roles/cilium), so nothing could
|
||||
# select the k3s_prod_* groups below. Both halves are now variables, deliberately kept
|
||||
# separate because Ansible resolves them at different times:
|
||||
# * WHICH HOSTS A PLAY RUNS ON — `k3s_hosts` (whole-cluster plays), `k3s_primary_hosts`
|
||||
# / `k3s_secondary_hosts` (k3s.yml's split), each defaulting to the dev group.
|
||||
# EXTRA VARS ONLY: a play's `hosts:` is templated before any host exists, so
|
||||
# group_vars cannot be read there. Prod: `-e k3s_hosts=k3s_prod` etc.
|
||||
# * WHICH GROUP A ROLE MEANS BY "MY PRIMARY"/"MY SERVERS" — `k3s_primary_group` /
|
||||
# `k3s_servers_group`, set in group_vars/all.yml (dev) and group_vars/k3s_prod/vars.yml
|
||||
# (prod), so roles resolve their own environment with NO per-run flag — the same
|
||||
# mechanism k3s_api_host already used.
|
||||
# Running with no extra vars is byte-for-byte the dev behaviour it always had. See
|
||||
# infra/ansible/README.md "Targeting prod" and PROD-PROVISIONING.md §6.
|
||||
# =============================================================================
|
||||
|
||||
[k3s_prod_primary]
|
||||
|
||||
Reference in new issue
Block a user