Prod inventory scaffolding for the three bare-metal nodes, and the become posture the
operator ruled for production.
BECOME: prod does NOT get dev's NOPASSWD sudo. Prod keeps a real sudo boundary and
supplies the password to Ansible as an ansible-vault-encrypted `ansible_become_password`
in inventory/group_vars/k3s_prod/vault.yml, mirroring the existing hypervisor and
proxmox_hosts groups. The decrypting passphrase stays outside the repo
(~/.config/home-k3s/ansible-vault-pass), so plays still run unattended with no plaintext
secret in git. Committed here: README.md + vault.yml.example only — the operator creates
the encrypted vault (`ansible-vault create`), as with every other vaulted group.
PACKAGES BELONG IN THE ROLE (operator ruling): smartmontools is now declared in
node_baseline_packages, not hand-installed on a node. Drive health tooling is a
permanent part of a bare-metal node's baseline — dev nodes are VMs and never needed it.
Same click-ops-never principle as the rest of the repo.
ADDRESSING: nodes .10/.11/.12, reached DIRECTLY on the LAN (no pfSense NAT hop, unlike
dev's 192.168.2.102:225x mapping), so ansible_host is the real node IP. kube-vip
control-plane VIP is 192.168.2.9 — deliberately BELOW the nodes and OUTSIDE the Cilium
LB pool (.20-.30) so kube-vip and LB-IPAM can never contend for an address. The ingress
Gateway VIP (.20) and the LB pool are Flux/cluster-vars concerns and deliberately do
not appear in Ansible.
BLOCKED-BY-DESIGN: node_baseline_longhorn_device / node_baseline_etcd_device are present
but set to an obviously invalid /dev/BLOCKED-... path pending the operator's disk-layout
decision (node 1 is a single SATA SSD; the role's /dev/vdb,/dev/vdc defaults are virtio
names that do not exist on bare metal). Leaving them unset would silently fall back to
those equally-nonexistent defaults — a confusing failure that looks like a wiring bug.
This fails just as hard and says exactly why.
KNOWN FOLLOW-UP: roles/k3s_server and every node-host playbook hardcode the literal
group names k3s_primary/k3s_secondary/k3s_servers, so the k3s_prod_* groups are not
selected by them as-is. Harmless now (nothing can accidentally run against prod) but the
"same roles, unchanged" reuse R-ENV-02 promises needs the target group parameterized
before §6 of PROD-PROVISIONING.md can execute. Documented in hosts.ini and both READMEs.
Serves: R-ENV-01, R-ENV-03, R-SEC-01. Task: T-7.1. Gate: MERGE GATE: PASS.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Interim bridge to stop Longhorn replica-rebuild I/O storms on the shared HDD
Storage pool from starving etcd WAL fsync (crash-looping k3s / flapping the
kube-vip VIP). Adds a small dedicated SSD disk per node for etcd only; Longhorn
stays on HDD. See docs/plans/ssd-etcd-isolation.md.
tofu (infra/tofu):
- New vars etcd_ssd_datastore (default "local-zfs" = rpool/data, Micron 5100
SATA SSD mirror w/ PLP) and etcd_disk_gib (default 16).
- New virtio2 disk block on the SSD datastore per VM: discard=on, cache=none
(honor guest fsync straight to the ZFS ZIL — safe on PLP SSD), iothread=false.
Kept last (ascending interface) so bpg does not reorder virtio0/virtio1.
- Revert the debian_cloud_image resource to the long type name
proxmox_virtual_environment_download_file to match the R2 state: bpg >= 0.111
dropped the cross-type moved{} linkage to the short proxmox_download_file, so
the stale moved block failed EVERY plan with "Resource type mismatch" (broken
on main too). Aligning code to state yields a clean plan with no image
re-download. Short-name migration is deferred to a deliberate `tofu state mv`.
ansible (node_baseline):
- Format /dev/vdc (label k3s-etcd) and mount it at
/var/lib/rancher/k3s/server/db (defaults,noatime; no nofail so a missing etcd
disk fails loudly rather than silently landing etcd on the root HDD) before
k3s starts, so a rebuilt node comes up with etcd isolated on SSD. Data-safe:
filesystem module never reformats an existing fs.
Phase 1 only: non-disruptive. No tofu apply, no node/VM/etcd/Longhorn changes.
tofu plan verified: 0 to add, 3 to change, 0 to destroy — every VM an in-place
disk ADD (virtio2), dev-3 also an in-place memory 16384->24576; no VM replace.
Merge gate green (render==golden included).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
jakman85andClaude Opus 5