--- # ===================================================================================== # node_baseline — DATA DISKS. Every task here carries the `disks` tag (applied once by # the `include_tasks` in main.yml), so `--tags packages` still skips the whole file. # Split into its own file rather than living in main.yml because it is the one part of # the role that DESTRUCTIVELY claims storage, and it deserves to be read on its own. # # TWO ENVIRONMENTS, TWO SHAPES — and the shape is DECLARED, never inferred: # # dev (group_vars/all.yml, role defaults) # node_baseline_longhorn_source: device -> mkfs /dev/vdb, mount LABEL=longhorn # at /var/lib/longhorn (== dataPath) # node_baseline_etcd_source: device -> mkfs /dev/vdc, mount LABEL=k3s-etcd # k3s_data_dir: (k3s default, /var/lib/rancher/k3s) # THREE devices: OS, Longhorn, etcd. Two tofu-provisioned virtio disks exist for # exactly this purpose. NOTHING about dev changes in the 2026-09-04 revision. # # prod (group_vars/k3s_prod/vars.yml — operator ruling, PROD-PROVISIONING.md §2) # node_baseline_longhorn_source: claim-mountpoint (claim mountpoint /srv) # node_baseline_data_mount: /data (LABEL=k3s-data) # node_baseline_etcd_source: shared-data-partition # k3s_data_dir: /data/k3s # ONE physical disk per node, already fully partitioned by the Debian installer. The # ~424 GB `/srv` partition is claimed and mounted at a NEUTRAL mountpoint, then split # between TWO tenants: /data/longhorn (Longhorn replicas) + /data/k3s (the whole k3s # data dir: etcd AND the containerd image store). # # WHY THE PARTITION IS NO LONGER LONGHORN-ONLY (revised 2026-09-04, PROD-PROVISIONING.md # §2). Prod's `/var` is a separate partition of just 11.8 GiB. k3s's default data dir puts # BOTH etcd's WAL and the containerd image store on it. Measured on the fully-deployed dev # cluster, containerd alone is 21-25 GB per node and /var totals 58-70 GB — so prod would # have filled /var during its FIRST image pulls, and a full /var means etcd cannot fsync its # WAL. That is cluster death, not degradation. Moving k3s's data dir onto the 424 GB # partition is the fix; the partition therefore serves two tenants, and neither its # mountpoint nor its filesystem label may name only one of them. # # WHY /data AND LABEL k3s-data. # * `/data` — short, neutral, tenant-agnostic, and NOT under /var (the whole point). # Deliberately NOT `/srv`: /srv is the CLAIM mountpoint this role discovers the device # at, and the idempotency contract is that /srv is gone from /proc/mounts and # /etc/fstab once the claim has happened. Re-using it as the destination would make # "claimed" and "not yet claimed" indistinguishable. # * label `k3s-data`, deliberately NOT the bare `data`: guard #4 in the assert wall below # accepts a partition whose label is empty OR ALREADY OURS, and its own comment names # `data` as an example of somebody else's named volume. Labelling ours `data` would make # a stranger's `data` volume look already-claimed and weaken that guard. `k3s-data` is # cluster-scoped rather than tenant-scoped and matches the existing `k3s-etcd` label # convention in this repo. # # WHY A "SOURCE" ENUM INSTEAD OF A BLANK VARIABLE. "etcd has no dedicated disk" and # "somebody forgot to set node_baseline_etcd_device" look identical if the signal is an # empty string, and the second one must never silently succeed. `root-filesystem` and # `shared-data-partition` are explicit, greppable statements of intent that the assert # below accepts; an unset or placeholder device is still a hard failure. The two differ in # a way that matters: `root-filesystem` says etcd is on the OS filesystem, and # `shared-data-partition` says it is on the claimed data partition. Prod was the FORMER # until 2026-09-04 and is now the LATTER — and the role ASSERTS which one is true rather # than printing a claim it never checked (see the etcd section). # # WHY DISCOVERY INSTEAD OF A PER-HOST DEVICE PATH. The prod partition is /dev/sda5 on # nodes 1 and 3 (SATA) but /dev/nvme0n1p5 on node 2 (NVMe), so no single group value is # correct — and per-host `host_vars` would encode a *guess about hardware* into inventory # that goes stale the moment a node is re-imaged, a disk is replaced, or a fourth node # arrives with a different controller. Worse, a fleet-wide `/dev/sda5` would eventually # point at the wrong partition on some future node and mkfs it. So the role asks the NODE # which block device is mounted at the claim mountpoint, and then proves that answer is # safe (see the assert wall below) before touching it. The inventory declares INTENT # ("the partition currently mounted at /srv"); the node supplies the FACT. # ===================================================================================== # Fail LOUD and EARLY rather than at mkfs time when an environment has not yet declared # its data-disk layout. Two failure shapes are caught: # * an undeclared/typo'd source enum, and # * source: device with a device path that is a PLACEHOLDER rather than a real device. # group_vars/k3s_prod/vars.yml deliberately carries `/dev/NOT-A-DEVICE-…` for both device # vars now that prod uses neither: if anyone ever flips a source back to `device` without # also supplying a real path, they get this assert instead of dev's /dev/vdb (which does # not exist on bare metal — a confusing failure that looks like a wiring bug). `BLOCKED` # stays in the token list because that was the pre-decision placeholder and may still be # carried by other environments/branches. - name: Assert this environment's data-disk layout has actually been declared ansible.builtin.assert: that: - node_baseline_longhorn_source in ['device', 'claim-mountpoint'] - node_baseline_etcd_source in ['device', 'root-filesystem', 'shared-data-partition'] # `shared-data-partition` is only coherent when there IS a claimed data partition to # share, and only when the k3s data dir actually sits INSIDE it. Catch the # contradiction here, in inventory terms, rather than three sections later when etcd # has silently landed back on the OS filesystem. - >- node_baseline_etcd_source != 'shared-data-partition' or node_baseline_longhorn_source == 'claim-mountpoint' - >- node_baseline_etcd_source != 'shared-data-partition' or node_baseline_k3s_data_dir.startswith(node_baseline_data_mount ~ '/') # The transition knobs are a matched PAIR: a retired label with no retired mountpoint # (or vice versa) cannot describe a state to migrate from. - >- (node_baseline_data_retired_fs_label | length > 0) == (node_baseline_data_retired_mount | length > 0) - >- node_baseline_longhorn_source != 'device' or not (node_baseline_longhorn_device is search('BLOCKED|NOT-A-DEVICE')) - >- node_baseline_etcd_source != 'device' or not (node_baseline_etcd_device is search('BLOCKED|NOT-A-DEVICE')) - >- node_baseline_longhorn_source != 'claim-mountpoint' or (node_baseline_longhorn_claim_mountpoint | length) > 0 fail_msg: >- This host's data-disk layout is not declared. longhorn_source={{ node_baseline_longhorn_source }} (device={{ node_baseline_longhorn_device }}, claim_mountpoint={{ node_baseline_longhorn_claim_mountpoint | default('') }}), etcd_source={{ node_baseline_etcd_source }} (device={{ node_baseline_etcd_device }}). Valid Longhorn sources: `device` (a dedicated block device, dev's shape) or `claim-mountpoint` (re-purpose the partition already mounted at node_baseline_longhorn_claim_mountpoint, prod's shape). Valid etcd sources: `device`, `root-filesystem` (etcd on the OS filesystem, no claimed partition under it) or `shared-data-partition` (etcd inside node_baseline_k3s_data_dir on the CLAIMED data partition — prod's shape since 2026-09-04, which additionally requires longhorn_source `claim-mountpoint` and a k3s data dir under node_baseline_data_mount={{ node_baseline_data_mount }}; k3s_data_dir={{ node_baseline_k3s_data_dir }}). Every one of these is an EXPLICIT operator decision recorded in infra/PROD-PROVISIONING.md §2, not an omission. Declare it in the environment's group_vars and record the reasoning in §2 in the SAME change. To install ONLY the package floor meanwhile, re-run with `--tags packages`, which skips every disk task including this assert. quiet: true # ===================================================================================== # THE DATA PARTITION (prod: Longhorn store + k3s data dir; dev: Longhorn store only) # ===================================================================================== # THE IDEMPOTENCY PIVOT for the claim path, and the reason a second run is a no-op # rather than a reformat. Once the claim has happened the partition carries the ext4 # label node_baseline_data_fs_label and is mounted at node_baseline_data_mount — the claim # mountpoint (/srv) is gone from both /proc/mounts and /etc/fstab, so "discover what is # mounted at /srv" would find nothing on run 2. Asking `blkid -L