feat(tofu,ansible): SSD-backed etcd disk per k3s node (Phase 1: code + plan)

Interim bridge to stop Longhorn replica-rebuild I/O storms on the shared HDD
Storage pool from starving etcd WAL fsync (crash-looping k3s / flapping the
kube-vip VIP). Adds a small dedicated SSD disk per node for etcd only; Longhorn
stays on HDD. See docs/plans/ssd-etcd-isolation.md.

tofu (infra/tofu):
- New vars etcd_ssd_datastore (default "local-zfs" = rpool/data, Micron 5100
  SATA SSD mirror w/ PLP) and etcd_disk_gib (default 16).
- New virtio2 disk block on the SSD datastore per VM: discard=on, cache=none
  (honor guest fsync straight to the ZFS ZIL — safe on PLP SSD), iothread=false.
  Kept last (ascending interface) so bpg does not reorder virtio0/virtio1.
- Revert the debian_cloud_image resource to the long type name
  proxmox_virtual_environment_download_file to match the R2 state: bpg >= 0.111
  dropped the cross-type moved{} linkage to the short proxmox_download_file, so
  the stale moved block failed EVERY plan with "Resource type mismatch" (broken
  on main too). Aligning code to state yields a clean plan with no image
  re-download. Short-name migration is deferred to a deliberate `tofu state mv`.

ansible (node_baseline):
- Format /dev/vdc (label k3s-etcd) and mount it at
  /var/lib/rancher/k3s/server/db (defaults,noatime; no nofail so a missing etcd
  disk fails loudly rather than silently landing etcd on the root HDD) before
  k3s starts, so a rebuilt node comes up with etcd isolated on SSD. Data-safe:
  filesystem module never reformats an existing fs.

Phase 1 only: non-disruptive. No tofu apply, no node/VM/etcd/Longhorn changes.
tofu plan verified: 0 to add, 3 to change, 0 to destroy — every VM an in-place
disk ADD (virtio2), dev-3 also an in-place memory 16384->24576; no VM replace.
Merge gate green (render==golden included).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
jakman85andClaude Opus 4.8 committed 2026-08-29 17:21:06 -05:00
1 parent 7910128675
commit af5fc5b25f
5 files changed
+348 -17

No files matched your search

@@ -72,6 +72,31 @@
opts: "defaults,nofail,x-systemd.device-timeout=5s"
state: mounted
# Dedicated SSD etcd disk (docs/plans/ssd-etcd-isolation.md). Formats /dev/vdc and
# mounts it at the k3s etcd data dir BEFORE k3s starts, so a rebuilt node comes up with
# etcd already isolated on SSD. Data-safe: community.general.filesystem never reformats
# a disk that already carries a filesystem, so re-running against an already-migrated
# node leaves the etcd data intact. On EXISTING nodes the one-time data migration
# (stop k3s -> rsync db -> mount) is done manually per the plan's §4.3 before this
# role would mount over a populated directory.
- name: Create ext4 filesystem on the dedicated etcd SSD disk (data-safe; skips if present)
community.general.filesystem:
fstype: ext4
dev: "{{ node_baseline_etcd_device }}"
opts: "-L {{ node_baseline_etcd_fs_label }}"
- name: Mount the dedicated etcd SSD disk at the k3s etcd data dir
ansible.posix.mount:
path: "{{ node_baseline_etcd_mount }}"
src: "LABEL={{ node_baseline_etcd_fs_label }}"
fstype: ext4
# noatime: fewer metadata writes under etcd's fsync-heavy load. No `nofail`: a
# missing etcd disk MUST fail loudly rather than let k3s start etcd on the root HDD.
opts: "defaults,noatime"
dump: "0"
passno: "2"
state: mounted
- name: Disable swap now
ansible.builtin.command: swapoff -a
changed_when: false