jakman85andClaude Opus 5
e623d27e2d
roles/k3s_server, roles/cilium and every node-host playbook hardcoded the dev group
names k3s_primary/k3s_secondary/k3s_servers, so the prod inventory groups could not be
selected at all. That broke the R-ENV-02 promise that prod reuses the SAME roles, and
blocked PROD-PROVISIONING.md §6 outright.
TWO KNOBS, deliberately separate — Ansible templates a play's `hosts:` BEFORE any host
context exists, so group_vars is unreadable there, while roles run WITH host context:
* which hosts a play RUNS ON — k3s_hosts, k3s_primary_hosts/k3s_secondary_hosts
(k3s.yml's split), vpn_egress_gateway_hosts; each `| default(<dev group>)`, set by -e
only. Follows node-health.yml's established `<name>_hosts | default(dev)` pattern.
* which group a ROLE means by "my primary"/"my servers" — k3s_primary_group,
k3s_servers_group in group_vars/all.yml (dev), overridden in group_vars/k3s_prod.
Follows the k3s_api_host convention already in this repo. No per-run flag.
DEV IS UNCHANGED — the key evidence. --list-hosts/--list-tasks/--syntax-check captured
for all 9 playbooks at HEAD and after: sorted host lists IDENTICAL for all 9, same
primary/secondary split, same 6-play site.yml. (Raw --list-hosts PRINT ORDER varies run
to run; proven to be Ansible's own set ordering by running UNMODIFIED HEAD cilium.yml
four times -> 3,1,2 / 1,3,2 / 1,2,3 / 1,2,3. Hence sorted comparison.) --list-tasks
differs in exactly two files, only by added TAGS annotations and one new assert; no task
reordered, renamed or removed. Var resolution verified per environment with -c local:
dev -> k3s_primary/192.168.1.50/sans .50,.49,127.0.0.1,.51,.52; prod ->
k3s_prod_primary/192.168.2.10/vip .9/sans .10,.9,127.0.0.1,.11,.12.
BREAKS THE SMARTMONTOOLS CIRCULAR DEPENDENCY. node_baseline tasks are now tagged
packages/system/disks, so the package install can run alone:
ansible-playbook playbooks/node-baseline.yml -e k3s_hosts=k3s_prod --tags packages
Skipping is the guard — no disks-tagged task is selected, so the
BLOCKED-pending-disk-decision paths are never templated, let alone passed to mkfs.
Belt-and-braces, a new disks-tagged assert fires first in a FULL run and names §2
explicitly instead of failing obscurely at mkfs. `tags: never` was deliberately NOT used
(it would change the default run).
FOOTGUN FOUND AND FIXED: roles/cilium, cluster_verify and flux_bootstrap all hardcoded
.cache/dev.kubeconfig — the exact file infra/devcontainer/dev.sh exports as $KUBECONFIG.
Once prod became targetable, a prod cilium/verify run would have SILENTLY OVERWRITTEN it,
leaving the operator's kubectl quietly answering from PROD. Now cluster_kubeconfig_name
(dev.kubeconfig -> prod.kubeconfig). Dev path byte-identical.
KNOWN, DOCUMENTED, NOT FIXED HERE: flux-bootstrap.yml retargets but roles/flux_bootstrap
shells out to bootstrap/flux/bootstrap.sh with a hardcoded --path=clusters/dev — a
bootstrap/ subtree change, not a target-group change (§6, cross-referenced to §7).
vpn.egress.gateway.yml is parameterised but prod has no egress group yet and would need
its own vaulted PIA credentials. --check against prod is still impossible until the prod
vault exists — even the packages-only run needs it.
Nothing was run against the dev cluster: --list-hosts/--list-tasks/--syntax-check, plus
debug/assert with -c local, only.
Serves: R-ENV-01, R-ENV-02, R-ENV-03. Task: T-7.1.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
102 lines
6.1 KiB
INI
102 lines
6.1 KiB
INI
# Static inventory — k3s node IPs ALIGNED with infra/tofu defaults (Proxmox migration IPs),
|
|
# but the SSH target is the pfSense NAT (see NAT HAND-OFF below) — this file is
|
|
# HAND-MAINTAINED and must NOT be regenerated from `tofu output -raw ansible_inventory`.
|
|
#
|
|
# k3s_primary = the --cluster-init server (embedded-etcd founder + Cilium k8sServiceHost).
|
|
# k3s_secondary = the two servers that join it. All three are servers (HA embedded etcd).
|
|
#
|
|
# NAT HAND-OFF (LIVE): the nodes live on the Proxmox LAN at 192.168.1.50/.51/.52, and
|
|
# Ansible reaches them over the pfSense NAT at 192.168.2.102, port-forwarded as
|
|
# 2250 -> 192.168.1.50:22, 2251 -> 192.168.1.51:22, 2252 -> 192.168.1.52:22.
|
|
# Therefore `ansible_host`/`ansible_port` below are the NAT SSH target (192.168.2.102:225x),
|
|
# and each node's REAL 192.168.1.x lives in the `k3s_node_ip` var — consumed for the k3s
|
|
# --node-ip, the API/tls-san endpoint (k3s_api_host), and general in-cluster parity.
|
|
# This file is HAND-MAINTAINED for the NAT: do NOT blindly regenerate it from
|
|
# tofu output -raw ansible_inventory
|
|
# because tofu has no knowledge of the pfSense NAT and would overwrite the mapping below.
|
|
|
|
[k3s_primary]
|
|
k3s-dev-1 ansible_host=192.168.2.102 ansible_port=2250 k3s_node_ip=192.168.1.50
|
|
|
|
[k3s_secondary]
|
|
k3s-dev-2 ansible_host=192.168.2.102 ansible_port=2251 k3s_node_ip=192.168.1.51
|
|
k3s-dev-3 ansible_host=192.168.2.102 ansible_port=2252 k3s_node_ip=192.168.1.52
|
|
|
|
[k3s_servers:children]
|
|
k3s_primary
|
|
k3s_secondary
|
|
|
|
# The hypervisor/workstation host itself (target of host-prereqs.yml). Its become
|
|
# (sudo) password comes from the vaulted group_vars/hypervisor/vault.yml.
|
|
[hypervisor]
|
|
localhost ansible_connection=local
|
|
|
|
# VPN egress-gateway node(s) — run the node-level split-tunnel OpenVPN + kill-switch
|
|
# (ADR-0015, playbooks/vpn.egress.gateway.yml). Initially just k3s-dev-3 (single gateway;
|
|
# add a second host here for Cilium Egress Gateway HA). Members are already defined above
|
|
# with their ansible_host under k3s_secondary; this group only tags the egress role target.
|
|
[vpn_egress_gateway]
|
|
k3s-dev-3
|
|
|
|
# The Proxmox HYPERVISOR host itself (NOT a k3s node) — target of playbooks/backup-s3.yml,
|
|
# which stands up the off-cluster versitygw S3 backup target for Velero + CNPG. Reached DIRECTLY
|
|
# over SSH as root (its own 192.168.2.102:22, the pfSense NAT to the host's 192.168.1.9), so
|
|
# it overrides the cluster-wide `ansible_user: k3s` from group_vars/all.yml. Distinct from the
|
|
# `hypervisor` group above (that is `localhost` for host-prereqs.yml, a different machine).
|
|
[proxmox_hosts]
|
|
proxmox ansible_host=192.168.2.102 ansible_port=22 ansible_user=root ansible_ssh_private_key_file=~/.ssh/home_network
|
|
|
|
# =============================================================================
|
|
# PROD (k3s-prod-*) — Phase 7 PREP ONLY. See infra/PROD-PROVISIONING.md (T-7.1,
|
|
# especially §5/§6) and DECISIONS-LOG.md "PHASE 7 OPENED" (2026-08-31). Node 1
|
|
# (`k3s-prod-1`, HP EliteDesk 800 G5 Mini) is on the bench and SSH-reachable; nodes 2
|
|
# and 3 have not arrived. This section is inventory/group_vars scaffolding only —
|
|
# nothing has been run against node 1, and the operator has decided the CLUSTER stays
|
|
# uninitialized (no k3s install, no --cluster-init) until all three nodes are present
|
|
# (decision #2 in the DECISIONS-LOG entry above) — no single-server-etcd window, no
|
|
# later member-add dance.
|
|
#
|
|
# KEY DIFFERENCE FROM DEV: prod sits DIRECTLY on the 192.168.2.x LAN — no pfSense NAT
|
|
# hop in front of it. Dev's `ansible_host` above is a NAT target (192.168.2.102:225x)
|
|
# that port-forwards to the node's real 192.168.1.x address, which is why dev also
|
|
# carries a separate `k3s_node_ip` per host. Prod needs no such split: `ansible_host`
|
|
# below IS already the node's real, routable address, so `k3s_node_ip` is set to the
|
|
# SAME value (kept explicit, not omitted, only so the k3s_server role's hostvar lookup
|
|
# — see the GAP note below — has something correct to find once that gap is closed).
|
|
# `ansible_port` is the node's own real sshd (22). See group_vars/k3s_prod/vars.yml for
|
|
# the k3s_api_host override that reflects this directly-reachable topology.
|
|
#
|
|
# GROUP-NAME GAP: CLOSED 2026-08-31. The roles and playbooks used to hardcode the
|
|
# LITERAL dev groups (`hosts: k3s_servers` / `k3s_primary` / `k3s_secondary`, and
|
|
# `groups['k3s_primary']` inside roles/k3s_server + roles/cilium), so nothing could
|
|
# select the k3s_prod_* groups below. Both halves are now variables, deliberately kept
|
|
# separate because Ansible resolves them at different times:
|
|
# * WHICH HOSTS A PLAY RUNS ON — `k3s_hosts` (whole-cluster plays), `k3s_primary_hosts`
|
|
# / `k3s_secondary_hosts` (k3s.yml's split), each defaulting to the dev group.
|
|
# EXTRA VARS ONLY: a play's `hosts:` is templated before any host exists, so
|
|
# group_vars cannot be read there. Prod: `-e k3s_hosts=k3s_prod` etc.
|
|
# * WHICH GROUP A ROLE MEANS BY "MY PRIMARY"/"MY SERVERS" — `k3s_primary_group` /
|
|
# `k3s_servers_group`, set in group_vars/all.yml (dev) and group_vars/k3s_prod/vars.yml
|
|
# (prod), so roles resolve their own environment with NO per-run flag — the same
|
|
# mechanism k3s_api_host already used.
|
|
# Running with no extra vars is byte-for-byte the dev behaviour it always had. See
|
|
# infra/ansible/README.md "Targeting prod" and PROD-PROVISIONING.md §6.
|
|
# =============================================================================
|
|
|
|
[k3s_prod_primary]
|
|
k3s-prod-1 ansible_host=192.168.2.10 ansible_port=22 k3s_node_ip=192.168.2.10
|
|
|
|
[k3s_prod_secondary]
|
|
# Addresses for nodes 2/3 ARE decided (operator, 2026-08-31 — infra/PROD-PROVISIONING.md
|
|
# §0), but the nodes themselves are NOT yet racked/imaged/reachable. Left commented so
|
|
# no play can accidentally target a host that isn't there; uncomment each line only once
|
|
# that node is actually on the bench and answering SSH.
|
|
# k3s-prod-2 ansible_host=192.168.2.11 ansible_port=22 k3s_node_ip=192.168.2.11
|
|
# k3s-prod-3 ansible_host=192.168.2.12 ansible_port=22 k3s_node_ip=192.168.2.12
|
|
|
|
# Umbrella group group_vars/k3s_prod/ attaches to (mirrors dev's [k3s_servers:children]
|
|
# pattern; named to match the group_vars directory so Ansible auto-loads it).
|
|
[k3s_prod:children]
|
|
k3s_prod_primary
|
|
k3s_prod_secondary
|