home-network-k3s-impl/infra/ansible/roles/node_baseline/tasks/disks.yml

---
# =====================================================================================
# node_baseline — DATA DISKS.  Every task here carries the `disks` tag (applied once by
# the `include_tasks` in main.yml), so `--tags packages` still skips the whole file.
# Split into its own file rather than living in main.yml because it is the one part of
# the role that DESTRUCTIVELY claims storage, and it deserves to be read on its own.
#
# TWO ENVIRONMENTS, TWO SHAPES — and the shape is DECLARED, never inferred:
#
#   dev  (group_vars/all.yml, role defaults)
#     node_baseline_longhorn_source: device          -> mkfs /dev/vdb, mount LABEL=longhorn
#                                                       at /var/lib/longhorn (== dataPath)
#     node_baseline_etcd_source:     device          -> mkfs /dev/vdc, mount LABEL=k3s-etcd
#     k3s_data_dir: (k3s default, /var/lib/rancher/k3s)
#     THREE devices: OS, Longhorn, etcd. Two tofu-provisioned virtio disks exist for
#     exactly this purpose. NOTHING about dev changes in the 2026-09-04 revision.
#
#   prod (group_vars/k3s_prod/vars.yml — operator ruling, PROD-PROVISIONING.md §2)
#     node_baseline_longhorn_source: claim-mountpoint  (claim mountpoint /srv)
#     node_baseline_data_mount:      /data            (LABEL=k3s-data)
#     node_baseline_etcd_source:     shared-data-partition
#     k3s_data_dir:                  /data/k3s
#     ONE physical disk per node, already fully partitioned by the Debian installer. The
#     ~424 GB `/srv` partition is claimed and mounted at a NEUTRAL mountpoint, then split
#     between TWO tenants:  /data/longhorn (Longhorn replicas)  +  /data/k3s (the whole k3s
#     data dir: etcd AND the containerd image store).
#
# WHY THE PARTITION IS NO LONGER LONGHORN-ONLY (revised 2026-09-04, PROD-PROVISIONING.md
# §2). Prod's `/var` is a separate partition of just 11.8 GiB. k3s's default data dir puts
# BOTH etcd's WAL and the containerd image store on it. Measured on the fully-deployed dev
# cluster, containerd alone is 21-25 GB per node and /var totals 58-70 GB — so prod would
# have filled /var during its FIRST image pulls, and a full /var means etcd cannot fsync its
# WAL. That is cluster death, not degradation. Moving k3s's data dir onto the 424 GB
# partition is the fix; the partition therefore serves two tenants, and neither its
# mountpoint nor its filesystem label may name only one of them.
#
# WHY /data AND LABEL k3s-data.
#   * `/data` — short, neutral, tenant-agnostic, and NOT under /var (the whole point).
#     Deliberately NOT `/srv`: /srv is the CLAIM mountpoint this role discovers the device
#     at, and the idempotency contract is that /srv is gone from /proc/mounts and
#     /etc/fstab once the claim has happened. Re-using it as the destination would make
#     "claimed" and "not yet claimed" indistinguishable.
#   * label `k3s-data`, deliberately NOT the bare `data`: guard #4 in the assert wall below
#     accepts a partition whose label is empty OR ALREADY OURS, and its own comment names
#     `data` as an example of somebody else's named volume. Labelling ours `data` would make
#     a stranger's `data` volume look already-claimed and weaken that guard. `k3s-data` is
#     cluster-scoped rather than tenant-scoped and matches the existing `k3s-etcd` label
#     convention in this repo.
#
# WHY A "SOURCE" ENUM INSTEAD OF A BLANK VARIABLE. "etcd has no dedicated disk" and
# "somebody forgot to set node_baseline_etcd_device" look identical if the signal is an
# empty string, and the second one must never silently succeed. `root-filesystem` and
# `shared-data-partition` are explicit, greppable statements of intent that the assert
# below accepts; an unset or placeholder device is still a hard failure. The two differ in
# a way that matters: `root-filesystem` says etcd is on the OS filesystem, and
# `shared-data-partition` says it is on the claimed data partition. Prod was the FORMER
# until 2026-09-04 and is now the LATTER — and the role ASSERTS which one is true rather
# than printing a claim it never checked (see the etcd section).
#
# WHY DISCOVERY INSTEAD OF A PER-HOST DEVICE PATH. The prod partition is /dev/sda5 on
# nodes 1 and 3 (SATA) but /dev/nvme0n1p5 on node 2 (NVMe), so no single group value is
# correct — and per-host `host_vars` would encode a *guess about hardware* into inventory
# that goes stale the moment a node is re-imaged, a disk is replaced, or a fourth node
# arrives with a different controller. Worse, a fleet-wide `/dev/sda5` would eventually
# point at the wrong partition on some future node and mkfs it. So the role asks the NODE
# which block device is mounted at the claim mountpoint, and then proves that answer is
# safe (see the assert wall below) before touching it. The inventory declares INTENT
# ("the partition currently mounted at /srv"); the node supplies the FACT.
# =====================================================================================

# Fail LOUD and EARLY rather than at mkfs time when an environment has not yet declared
# its data-disk layout. Two failure shapes are caught:
#   * an undeclared/typo'd source enum, and
#   * source: device with a device path that is a PLACEHOLDER rather than a real device.
# group_vars/k3s_prod/vars.yml deliberately carries `/dev/NOT-A-DEVICE-…` for both device
# vars now that prod uses neither: if anyone ever flips a source back to `device` without
# also supplying a real path, they get this assert instead of dev's /dev/vdb (which does
# not exist on bare metal — a confusing failure that looks like a wiring bug). `BLOCKED`
# stays in the token list because that was the pre-decision placeholder and may still be
# carried by other environments/branches.
- name: Assert this environment's data-disk layout has actually been declared
  ansible.builtin.assert:
    that:
      - node_baseline_longhorn_source in ['device', 'claim-mountpoint']
      - node_baseline_etcd_source in ['device', 'root-filesystem', 'shared-data-partition']
      # `shared-data-partition` is only coherent when there IS a claimed data partition to
      # share, and only when the k3s data dir actually sits INSIDE it. Catch the
      # contradiction here, in inventory terms, rather than three sections later when etcd
      # has silently landed back on the OS filesystem.
      - >-
        node_baseline_etcd_source != 'shared-data-partition'
        or node_baseline_longhorn_source == 'claim-mountpoint'
      - >-
        node_baseline_etcd_source != 'shared-data-partition'
        or node_baseline_k3s_data_dir.startswith(node_baseline_data_mount ~ '/')
      # The transition knobs are a matched PAIR: a retired label with no retired mountpoint
      # (or vice versa) cannot describe a state to migrate from.
      - >-
        (node_baseline_data_retired_fs_label | length > 0)
        == (node_baseline_data_retired_mount | length > 0)
      - >-
        node_baseline_longhorn_source != 'device'
        or not (node_baseline_longhorn_device is search('BLOCKED|NOT-A-DEVICE'))
      - >-
        node_baseline_etcd_source != 'device'
        or not (node_baseline_etcd_device is search('BLOCKED|NOT-A-DEVICE'))
      - >-
        node_baseline_longhorn_source != 'claim-mountpoint'
        or (node_baseline_longhorn_claim_mountpoint | length) > 0
    fail_msg: >-
      This host's data-disk layout is not declared.
      longhorn_source={{ node_baseline_longhorn_source }}
      (device={{ node_baseline_longhorn_device }},
      claim_mountpoint={{ node_baseline_longhorn_claim_mountpoint | default('') }}),
      etcd_source={{ node_baseline_etcd_source }}
      (device={{ node_baseline_etcd_device }}).
      Valid Longhorn sources: `device` (a dedicated block device, dev's shape) or
      `claim-mountpoint` (re-purpose the partition already mounted at
      node_baseline_longhorn_claim_mountpoint, prod's shape). Valid etcd sources:
      `device`, `root-filesystem` (etcd on the OS filesystem, no claimed partition under
      it) or `shared-data-partition` (etcd inside node_baseline_k3s_data_dir on the CLAIMED
      data partition — prod's shape since 2026-09-04, which additionally requires
      longhorn_source `claim-mountpoint` and a k3s data dir under
      node_baseline_data_mount={{ node_baseline_data_mount }};
      k3s_data_dir={{ node_baseline_k3s_data_dir }}). Every one of these is an EXPLICIT
      operator decision recorded in infra/PROD-PROVISIONING.md §2, not an omission.
      Declare it in the environment's group_vars and record the reasoning in
      §2 in the SAME change. To install ONLY the package floor meanwhile, re-run with
      `--tags packages`, which skips every disk task including this assert.
    quiet: true

# =====================================================================================
# THE DATA PARTITION  (prod: Longhorn store + k3s data dir; dev: Longhorn store only)
# =====================================================================================

# THE IDEMPOTENCY PIVOT for the claim path, and the reason a second run is a no-op
# rather than a reformat. Once the claim has happened the partition carries the ext4
# label node_baseline_data_fs_label and is mounted at node_baseline_data_mount — the claim
# mountpoint (/srv) is gone from both /proc/mounts and /etc/fstab, so "discover what is
# mounted at /srv" would find nothing on run 2. Asking `blkid -L <label>` instead answers
# "has this node already been claimed?" from the durable, on-disk evidence: rc 0 + a device
# path means converged, rc != 0 means the claim has not been made. Everything destructive
# below is gated on rc != 0.
- name: "Data partition: look for a filesystem already carrying the data-partition label"
  ansible.builtin.command: "blkid -L {{ node_baseline_data_fs_label }}"
  register: node_baseline_lh_labelled
  changed_when: false
  failed_when: false
  check_mode: false

# --- The TRANSITION (2026-09-04 layout revision; one-time, then this whole block is dead) --
# WHY THIS EXISTS. The idempotency pivot above is "does the CURRENT label exist?". When the
# label ITSELF changes, an already-claimed node answers "no" and the greenfield claim path
# takes over — which then fails closed at "the claim mountpoint must actually be a
# mountpoint", because /srv was consumed by the previous claim. Fail-closed is correct
# behaviour but it is not convergence, and the prod nodes are in exactly that state today:
# claimed under LABEL=longhorn at /var/lib/longhorn by the previous version of this role.
# So the role must handle the TRANSITION explicitly instead of pretending prod is greenfield.
#
# WHAT IT DOES: unmount the retired mountpoint -> drop its fstab entry -> re-stamp the
# partition with the new label. It never touches the DATA and never reformats: e2label is a
# metadata write, exactly as in the claim path below. The common mount task then mounts
# LABEL=<new> at the new mountpoint, and the node is converged. On every subsequent run
# `blkid -L <new label>` succeeds and none of this is reachable.
#
# THREE INDEPENDENT REASONS THE LIVE DEV CLUSTER CANNOT ENTER THIS PATH — it has 18 Longhorn
# volumes whose replicas live on the partition labelled `longhorn` at /var/lib/longhorn, and
# unmounting it would orphan every one of them:
#   1. node_baseline_data_retired_fs_label defaults to "" and is set ONLY in
#      group_vars/k3s_prod/vars.yml. No retired label declared => no transition, ever.
#   2. Gated on node_baseline_longhorn_source == 'claim-mountpoint'. Dev is `device`, so
#      the same structural gate that keeps dev out of the claim path keeps it out of this one.
#   3. The emptiness assert below. Even if 1 and 2 were both defeated, a partition with
#      replica directories on it fails closed and nothing is unmounted.
# Dev's label and mountpoint are also simply NOT CHANGING, so there is nothing to migrate.
- name: "Data partition (transition): look for a filesystem still carrying the RETIRED label"
  ansible.builtin.command: "blkid -L {{ node_baseline_data_retired_fs_label }}"
  register: node_baseline_lh_retired_labelled
  changed_when: false
  failed_when: false
  check_mode: false
  when:
    - node_baseline_longhorn_source == 'claim-mountpoint'
    - node_baseline_data_retired_fs_label | length > 0
    - node_baseline_lh_labelled.rc != 0

- name: "Data partition: decide between TRANSITION, CLAIM, and already-converged"
  ansible.builtin.set_fact:
    # A node claimed under the RETIRED label. Mutually exclusive with the claim below.
    node_baseline_lh_transition_needed: >-
      {{ node_baseline_longhorn_source == 'claim-mountpoint'
         and node_baseline_data_retired_fs_label | length > 0
         and node_baseline_lh_labelled.rc != 0
         and (node_baseline_lh_retired_labelled.rc | default(1)) == 0 }}
    # True only for a claim-mountpoint environment that has NOT been claimed under EITHER
    # label. A `device` environment (dev) can never enter the claim path at all — that is
    # the structural guarantee that no generalisation of prod's behaviour can reach the
    # LIVE dev nodes and their /dev/vdb Longhorn replicas.
    node_baseline_lh_claim_needed: >-
      {{ node_baseline_longhorn_source == 'claim-mountpoint'
         and node_baseline_lh_labelled.rc != 0
         and (node_baseline_lh_retired_labelled.rc | default(1)) != 0 }}

- name: "Data partition (transition): resolve what is mounted at the retired mountpoint"
  ansible.builtin.command: >-
    findmnt --noheadings --first-only --output SOURCE
    --mountpoint {{ node_baseline_data_retired_mount }}
  register: node_baseline_lh_retired_findmnt
  changed_when: false
  failed_when: false
  check_mode: false
  when: node_baseline_lh_transition_needed | bool

# Read the retired mountpoint's top level, same contract as the claim path: `lost+found` is
# not data, hidden files cannot hide. THIS is the assert that makes the transition safe —
# an unmount of a populated Longhorn store would orphan every replica on the node.
- name: "Data partition (transition): read the contents of the retired mountpoint"
  ansible.builtin.find:
    paths: "{{ node_baseline_data_retired_mount }}"
    file_type: any
    hidden: true
    recurse: false
    excludes: ['lost+found']
  register: node_baseline_lh_retired_contents
  when: node_baseline_lh_transition_needed | bool

- name: "Data partition (transition): prove this node is safe to re-label"
  ansible.builtin.assert:
    that:
      # 1. The retired mountpoint is STILL A MOUNT, and it is backed by the very device the
      #    retired label names. The only state this role knows how to transition is the one
      #    the previous version of it produced; anything else is an operator decision.
      - (node_baseline_lh_retired_findmnt.rc | default(1)) == 0
      - >-
        (node_baseline_lh_retired_findmnt.stdout | default('') | trim)
        == (node_baseline_lh_retired_labelled.stdout | default('') | trim)
      # 2. It is EMPTY. Nothing is being orphaned. A populated store means either this is
      #    not the node we think it is, or a tenant has already written data — in both cases
      #    unmounting and re-labelling under it is destructive and the role must stop.
      - (node_baseline_lh_retired_contents.files | length) == 0
    fail_msg: >-
      REFUSING to transition this node's data partition from
      LABEL={{ node_baseline_data_retired_fs_label }} at {{ node_baseline_data_retired_mount }}
      to LABEL={{ node_baseline_data_fs_label }} at {{ node_baseline_data_mount }}.
      device by retired label='{{ node_baseline_lh_retired_labelled.stdout | default('') | trim }}',
      device mounted at the retired mountpoint='{{ node_baseline_lh_retired_findmnt.stdout | default('') | trim }}'
      (findmnt rc={{ node_baseline_lh_retired_findmnt.rc | default('n/a') }}),
      non-lost+found entries={{ node_baseline_lh_retired_contents.files | default([]) | length }}
      {{ node_baseline_lh_retired_contents.files | default([]) | map(attribute='path') | list }}.
      If the partition holds DATA, this transition would orphan it — Longhorn replicas are
      not recoverable from an unmounted, re-labelled filesystem. Migrate the data
      deliberately, or drop node_baseline_data_retired_fs_label and leave the node alone.
      Do NOT relax this assert.
    quiet: true
  when: node_baseline_lh_transition_needed | bool

- name: "Data partition (transition): report exactly what is about to change"
  ansible.builtin.debug:
    msg: >-
      Transitioning {{ node_baseline_lh_retired_labelled.stdout | trim }} (verified empty)
      from LABEL={{ node_baseline_data_retired_fs_label }} at {{ node_baseline_data_retired_mount }}
      to LABEL={{ node_baseline_data_fs_label }} at {{ node_baseline_data_mount }}:
      unmount -> drop its fstab entry -> relabel -> re-mount at the new path.
      No filesystem is created and no data is moved.
  when: node_baseline_lh_transition_needed | bool

# e2label refuses to run on a mounted filesystem, and the fstab entry must go in the same
# breath or the next boot re-mounts the partition at the RETIRED path underneath the new one.
- name: "Data partition (transition): unmount the retired mountpoint"
  ansible.posix.mount:
    path: "{{ node_baseline_data_retired_mount }}"
    state: unmounted
  when: node_baseline_lh_transition_needed | bool

- name: "Data partition (transition): remove the retired mountpoint's /etc/fstab entry"
  ansible.posix.mount:
    path: "{{ node_baseline_data_retired_mount }}"
    state: absent_from_fstab
    backup: true
  when: node_baseline_lh_transition_needed | bool

- name: "Data partition (transition): re-stamp the partition with the new filesystem label"
  ansible.builtin.command: >-
    e2label {{ node_baseline_lh_retired_labelled.stdout | trim }}
    {{ node_baseline_data_fs_label }}
  changed_when: true
  when: node_baseline_lh_transition_needed | bool

# --- The claim (prod, first run only) -------------------------------------------------
# Ask the node, do not assume. `--mountpoint` (not `--target`) matches ONLY an exact
# mountpoint: if /srv is not itself a mount, findmnt exits 1 rather than silently
# returning the device behind `/` — which is precisely the mistake that would hand the
# root partition to e2label.
- name: "Longhorn (claim): resolve the block device mounted at the claim mountpoint"
  ansible.builtin.command: >-
    findmnt --noheadings --first-only --output SOURCE
    --mountpoint {{ node_baseline_longhorn_claim_mountpoint }}
  register: node_baseline_lh_claim_findmnt
  changed_when: false
  failed_when: false
  check_mode: false
  when: node_baseline_lh_claim_needed | bool

- name: "Longhorn (claim): the claim mountpoint must actually be a mountpoint"
  ansible.builtin.assert:
    that:
      - node_baseline_lh_claim_findmnt.rc == 0
      - (node_baseline_lh_claim_findmnt.stdout | trim) is match('^/dev/')
    fail_msg: >-
      node_baseline_longhorn_source is `claim-mountpoint` and no filesystem is claimed
      yet, but nothing is mounted at {{ node_baseline_longhorn_claim_mountpoint }} on
      this node (findmnt rc={{ node_baseline_lh_claim_findmnt.rc | default('n/a') }},
      stdout='{{ node_baseline_lh_claim_findmnt.stdout | default('') | trim }}').
      The role will NOT go looking for "a big partition" to claim instead. Either the
      node is partitioned differently from the fleet — in which case fix the node or give
      this host its own declaration — or the claim already happened and the label was
      removed by hand.
    quiet: true
  when: node_baseline_lh_claim_needed | bool

- name: "Longhorn (claim): inspect the candidate partition"
  ansible.builtin.command: >-
    lsblk --json --nodeps --bytes
    --output PATH,TYPE,FSTYPE,LABEL,SIZE
    {{ node_baseline_lh_claim_findmnt.stdout | trim }}
  register: node_baseline_lh_claim_lsblk
  changed_when: false
  check_mode: false
  when: node_baseline_lh_claim_needed | bool

# The devices this role must never, under any circumstances, hand to e2label. Built from
# gathered facts (`gather_facts: true` in the play) plus the live swap list, so it is the
# NODE's account of what is load-bearing, not a hardcoded list of partition names.
- name: "Longhorn (claim): list the devices currently backing active swap"
  ansible.builtin.command: swapon --noheadings --show=NAME
  register: node_baseline_lh_swapdevs
  changed_when: false
  failed_when: false
  check_mode: false
  when: node_baseline_lh_claim_needed | bool

# Read the mountpoint's top level. `excludes: lost+found` because every freshly-mkfs'd
# ext4 has one and it is not data. hidden: true so a dotfile cannot hide from the check.
- name: "Longhorn (claim): read the contents of the mountpoint to be claimed"
  ansible.builtin.find:
    paths: "{{ node_baseline_longhorn_claim_mountpoint }}"
    file_type: any
    hidden: true
    recurse: false
    excludes: ['lost+found']
  register: node_baseline_lh_claim_contents
  when: node_baseline_lh_claim_needed | bool

# THE GUARD WALL. Five independent conditions, every one of which must hold before the
# role modifies a filesystem it did not create. Any single failure stops the play with
# the evidence in the message — nothing is unmounted, relabelled or mounted first.
- name: "Longhorn (claim): prove the candidate partition is safe to claim"
  ansible.builtin.assert:
    that:
      # 1. It is a real block device — a partition or a whole disk. Catches a claim
      #    mountpoint that is actually tmpfs, an NFS/CIFS export, or a bind mount, all of
      #    which would have produced a non-/dev SOURCE or an lsblk row of another type.
      - node_baseline_lh_candidate.type in ['part', 'disk']
      # 2. It is not backing anything the OS needs. Derived from ansible_mounts + swapon,
      #    so a differently-partitioned node is caught by its own facts.
      - node_baseline_lh_candidate.path not in node_baseline_lh_protected_devices
      # 3. It already holds ext4. The claim path RELABELS rather than reformats (see
      #    below), which is only meaningful for the filesystem type we are about to
      #    declare in fstab. Anything else is an operator decision, not a role's.
      - node_baseline_lh_candidate.fstype == 'ext4'
      # 4. It carries no filesystem label, or already carries ours. A partition labelled
      #    `boot`, `data`, `backup`… is somebody's named volume; stamping `longhorn` over
      #    it would make it unfindable by its own fstab entry.
      - (node_baseline_lh_candidate.label | default('', true)) in ['', node_baseline_data_fs_label]
      # 5. It is big enough to be the partition we meant. A data partition of a few GB is
      #    not "small", it is the wrong partition — and on prod it must hold BOTH tenants.
      - (node_baseline_lh_candidate.size | int) >= (node_baseline_longhorn_min_size_gb | int) * 1024 * 1024 * 1024
      # 6. It is EMPTY. The operator's premise is that prod's /srv holds nothing but the
      #    ext4 5% root reserve; this is where that premise is verified per node instead
      #    of trusted. If any node has real data on it, the play stops here — the role
      #    never wipes data it cannot account for.
      - (node_baseline_lh_claim_contents.files | length) == 0
    fail_msg: >-
      REFUSING to claim {{ node_baseline_lh_candidate.path }}
      (mounted at {{ node_baseline_longhorn_claim_mountpoint }}) for the Longhorn store.
      type={{ node_baseline_lh_candidate.type }},
      fstype={{ node_baseline_lh_candidate.fstype }},
      label='{{ node_baseline_lh_candidate.label | default('', true) }}',
      size={{ ((node_baseline_lh_candidate.size | int) / 1024 / 1024 / 1024) | round(1) }} GiB
      (minimum {{ node_baseline_longhorn_min_size_gb }} GiB),
      non-lost+found entries={{ node_baseline_lh_claim_contents.files | length }}
      {{ node_baseline_lh_claim_contents.files | map(attribute='path') | list }},
      devices this node needs and that must never be claimed:
      {{ node_baseline_lh_protected_devices }}.
      This is the destructive step of the role and it fails CLOSED. If the partition
      holds data, back it up or pick a different one; do NOT relax this assert.
    quiet: true
  vars:
    node_baseline_lh_candidate: "{{ (node_baseline_lh_claim_lsblk.stdout | from_json).blockdevices[0] }}"
    # The node's OWN account of what it cannot afford to lose: the devices behind the
    # mounts in node_baseline_protected_mounts (from gathered facts, so a node
    # partitioned differently is judged on its own layout) plus every active swap device.
    # Built here rather than hardcoded so no partition-name list can go stale.
    node_baseline_lh_protected_devices: >-
      {{ (ansible_facts.mounts | default([])
          | selectattr('mount', 'in', node_baseline_protected_mounts)
          | map(attribute='device') | list)
         + (node_baseline_lh_swapdevs.stdout_lines | default([]) | map('trim') | select | list) }}
  when: node_baseline_lh_claim_needed | bool

- name: "Longhorn (claim): report exactly what is about to be claimed"
  ansible.builtin.debug:
    msg: >-
      Claiming {{ node_baseline_lh_candidate.path }}
      ({{ ((node_baseline_lh_candidate.size | int) / 1024 / 1024 / 1024) | round(1) }} GiB ext4,
      currently mounted at {{ node_baseline_longhorn_claim_mountpoint }}, verified empty)
      as the data partition: unmount -> drop its fstab entry ->
      relabel LABEL={{ node_baseline_data_fs_label }} ->
      mount at {{ node_baseline_data_mount }}.
  vars:
    node_baseline_lh_candidate: "{{ (node_baseline_lh_claim_lsblk.stdout | from_json).blockdevices[0] }}"
  when: node_baseline_lh_claim_needed | bool

# --- Why the role must unmount and relabel, not just point fstab at the device ---------
# The mount task below mounts by `LABEL=`, deliberately: a label survives /dev/sda5 ->
# /dev/nvme0n1p5 renaming across the fleet and across a controller change, which a device
# path does not. The installer-created /srv filesystem has NO label (fstab pins it by
# UUID), so it must be given one. e2label refuses to run on a mounted filesystem, hence
# the unmount first — and the fstab entry must go in the same breath, or the next boot
# re-mounts the same partition at /srv underneath Longhorn.
#
# RELABEL, NOT REFORMAT. The partition is already ext4 and already verified empty, so
# mkfs would buy nothing and would destroy anything the emptiness check got wrong.
# e2label is a metadata write: if some assert above was too permissive, the worst case is
# a renamed filesystem, not a destroyed one. (This is also why `community.general.filesystem`
# cannot do this job: it declines to touch a device that already has a filesystem, so it
# would silently no-op and then the LABEL= mount would fail with no useful message.)
- name: "Longhorn (claim): unmount the claim mountpoint"
  ansible.posix.mount:
    path: "{{ node_baseline_longhorn_claim_mountpoint }}"
    state: unmounted
  when: node_baseline_lh_claim_needed | bool

- name: "Longhorn (claim): remove the claim mountpoint's /etc/fstab entry"
  ansible.posix.mount:
    path: "{{ node_baseline_longhorn_claim_mountpoint }}"
    state: absent_from_fstab
    backup: true
  when: node_baseline_lh_claim_needed | bool

- name: "Longhorn (claim): stamp the data-partition filesystem label onto the partition"
  ansible.builtin.command: >-
    e2label {{ (node_baseline_lh_claim_lsblk.stdout | from_json).blockdevices[0].path }}
    {{ node_baseline_data_fs_label }}
  changed_when: true
  when: node_baseline_lh_claim_needed | bool

# --- The dedicated-device path (dev) --------------------------------------------------
# Unchanged behaviour for `source: device`: data-safe, because community.general.filesystem
# never reformats a device that already carries a filesystem.
- name: Create ext4 filesystem on the Longhorn data disk (data-safe; skips if present)
  community.general.filesystem:
    fstype: ext4
    dev: "{{ node_baseline_longhorn_device }}"
    opts: "-L {{ node_baseline_data_fs_label }}"
  when: node_baseline_longhorn_source == 'device'

# --- Common to ALL paths (device, claim, transition) ----------------------------------
# Mounted by LABEL for every shape, so dev and prod share ONE fstab convention and one
# mount task. Idempotent: on a converged node this reports ok, which is what makes a
# re-run a no-op rather than a reformat. It is also what completes the transition above:
# the retired mountpoint is gone from fstab and the partition now answers to the new label,
# so this task mounts it at the new path.
- name: Mount the data partition
  ansible.posix.mount:
    path: "{{ node_baseline_data_mount }}"
    src: "LABEL={{ node_baseline_data_fs_label }}"
    fstype: ext4
    opts: "defaults,nofail,x-systemd.device-timeout=5s"
    state: mounted

# --- The per-tenant subdirectories ---------------------------------------------------
# WHY THIS IS CONDITIONAL RATHER THAN UNCONDITIONAL. On prod the partition serves two
# tenants and each needs its own subdirectory carved out of it. On DEV neither path is a
# subdirectory of the mount: Longhorn's dataPath IS the mount root, and k3s's data dir is
# /var/lib/rancher/k3s on the OS filesystem, owned and created by a LIVE k3s. Creating
# either there would at best be a no-op and at worst re-mode the running dev cluster's data
# dir, so the role only creates paths that are STRICT DESCENDANTS of the partition it
# mounted — i.e. only the directories it is actually responsible for. That test is derived
# from the paths themselves, so no environment has to remember to switch it off.
#
# MODES. Longhorn's dataPath keeps 0755, matching what mkfs gives dev's mount root and what
# the Longhorn manager expects to find. k3s's data dir is 0700 from the start: it holds the
# etcd database, the node token and the containerd image store, and k3s would tighten it to
# 0700 itself on first start — doing it here means it is never briefly world-readable.
- name: Create the per-tenant data directories on the data partition
  ansible.builtin.file:
    path: "{{ item.path }}"
    state: directory
    owner: root
    group: root
    mode: "{{ item.mode }}"
  loop:
    - { path: "{{ node_baseline_longhorn_data_path }}", mode: "0755", tenant: "Longhorn replicas" }
    - { path: "{{ node_baseline_k3s_data_dir }}", mode: "0700", tenant: "k3s data dir (etcd + containerd)" }
  loop_control:
    label: "{{ item.path }} ({{ item.tenant }})"
  when: item.path.startswith(node_baseline_data_mount ~ '/')

# =====================================================================================
# ETCD DATA DIR
# =====================================================================================

# --- The dedicated-device path (dev) --------------------------------------------------
# Dedicated SSD etcd disk (docs/plans/ssd-etcd-isolation.md). Formats /dev/vdc and
# mounts it at the k3s etcd data dir BEFORE k3s starts, so a rebuilt node comes up with
# etcd already isolated on SSD. Data-safe: community.general.filesystem never reformats
# a disk that already carries a filesystem, so re-running against an already-migrated
# node leaves the etcd data intact. On EXISTING nodes the one-time data migration
# (stop k3s -> rsync db -> mount) is done manually per the plan's §4.3 before this
# role would mount over a populated directory.
- name: Create ext4 filesystem on the dedicated etcd SSD disk (data-safe; skips if present)
  community.general.filesystem:
    fstype: ext4
    dev: "{{ node_baseline_etcd_device }}"
    opts: "-L {{ node_baseline_etcd_fs_label }}"
  when: node_baseline_etcd_source == 'device'

- name: Mount the dedicated etcd SSD disk at the k3s etcd data dir
  ansible.posix.mount:
    path: "{{ node_baseline_etcd_mount }}"
    src: "LABEL={{ node_baseline_etcd_fs_label }}"
    fstype: ext4
    # noatime: fewer metadata writes under etcd's fsync-heavy load. No `nofail`: a
    # missing etcd disk MUST fail loudly rather than let k3s start etcd on the root HDD.
    opts: "defaults,noatime"
    dump: "0"
    passno: "2"
    state: mounted
  when: node_baseline_etcd_source == 'device'

# --- The no-dedicated-device paths (prod) ---------------------------------------------
# Prod has ONE physical disk per node, already fully partitioned, with no free slot verified
# populated — so etcd gets no device of its own. That is an operator decision
# (PROD-PROVISIONING.md §2, 2026-09-03), and the shared-device I/O contention it accepts is
# UNCHANGED by the 2026-09-04 layout revision: dev's etcd/Longhorn contention incident was
# on a spinning HDD, and every prod node is SSD or NVMe, where concurrent rebuild I/O costs
# latency rather than collapsing it. Accepted to be MEASURED, not assumed.
#
# WHAT DID CHANGE ON 2026-09-04 is WHICH FILESYSTEM etcd shares. It used to be the OS
# filesystem (`root-filesystem`), which on these nodes means /var — a separate partition of
# 11.8 GiB carrying the containerd image store too. Measured on dev, containerd alone is
# 21-25 GB per node, so prod would have filled /var during its first image pulls and etcd
# could not have fsynced its WAL. So k3s's whole data dir moved onto the claimed 424 GB
# partition and etcd moved with it: `shared-data-partition`. Both enum values are still
# supported, and the role now PROVES which one is true instead of printing a claim.
#
# There is nothing to CREATE here for either shape — the data-partition section above
# already created node_baseline_k3s_data_dir, and k3s creates server/db inside it — so the
# role's job is to verify the assumption rather than act on it.
- name: "etcd (no dedicated device): check for anything already mounted at the etcd data dir"
  ansible.builtin.command: >-
    findmnt --noheadings --first-only --output SOURCE
    --mountpoint {{ node_baseline_etcd_mount }}
  register: node_baseline_etcd_existing_mount
  changed_when: false
  failed_when: false
  check_mode: false
  when: node_baseline_etcd_source in ['root-filesystem', 'shared-data-partition']

- name: "etcd (no dedicated device): assert no dedicated device is already mounted there"
  ansible.builtin.assert:
    that:
      - node_baseline_etcd_existing_mount.rc != 0
    fail_msg: >-
      node_baseline_etcd_source is `{{ node_baseline_etcd_source }}` (etcd deliberately has
      NO dedicated device — PROD-PROVISIONING.md §2) but {{ node_baseline_etcd_mount }} is
      ALREADY a separate mount, backed by
      {{ node_baseline_etcd_existing_mount.stdout | default('') | trim }}. That is a
      contradiction the role will not resolve on its own: either this node was previously
      built with a dedicated etcd device (in which case unmount it deliberately, after
      accounting for the etcd data on it) or the declaration is wrong. Refusing to leave
      the node in a state where the declared layout and the real one disagree.
    quiet: true
  when: node_baseline_etcd_source in ['root-filesystem', 'shared-data-partition']

# Evidence, not decoration: §2 records which filesystem etcd shares, and this establishes
# WHICH one per node so the claim is auditable from a run log. `findmnt --target` needs an
# existing path and k3s has not created server/db yet, so walk up to the deepest ancestor
# that does exist — the same filesystem the data dir will be created on. TARGET is captured
# separately from the human-readable line because the assert below judges on it.
- name: "etcd (no dedicated device): resolve the filesystem etcd will actually live on"
  ansible.builtin.shell: |
    set -eu
    p={{ node_baseline_etcd_mount | quote }}
    while [ ! -d "$p" ]; do p=$(dirname "$p"); done
    findmnt --noheadings --first-only --output TARGET --target "$p"
    findmnt --noheadings --first-only --output SOURCE,TARGET,SIZE,AVAIL --target "$p"
  register: node_baseline_etcd_backing_fs
  changed_when: false
  check_mode: false
  when: node_baseline_etcd_source in ['root-filesystem', 'shared-data-partition']

# THE ASSERT THAT MAKES THE WHOLE LAYOUT REVISION SELF-PROVING, and the reason
# `shared-data-partition` is a distinct enum value rather than a comment. Declaring
# k3s_data_dir: /data/k3s does NOT by itself put etcd on the data partition: if the mount
# above had silently failed, /data would be a plain directory on `/` and etcd plus 21-25 GB
# of container images would land back on the OS filesystem — the exact failure this change
# exists to prevent, and completely invisible until the disk filled. So the role checks that
# the filesystem backing the etcd data dir IS the data partition's mount, per node, on every
# run. `root-filesystem` is deliberately NOT asserted this way: it makes no claim about
# which filesystem it lands on beyond "not a dedicated one", and the debug line below is the
# whole record for that shape.
- name: "etcd (shared-data-partition): assert etcd really lands on the data partition"
  ansible.builtin.assert:
    that:
      - >-
        (node_baseline_etcd_backing_fs.stdout_lines[0] | default('') | trim)
        == node_baseline_data_mount
    fail_msg: >-
      node_baseline_etcd_source is `shared-data-partition`, which asserts that
      {{ node_baseline_etcd_mount }} sits on the claimed data partition mounted at
      {{ node_baseline_data_mount }} — but the filesystem actually backing it is mounted at
      '{{ node_baseline_etcd_backing_fs.stdout_lines[0] | default('') | trim }}'
      (full: {{ node_baseline_etcd_backing_fs.stdout_lines[1] | default('') | trim }}).
      The data partition is therefore NOT mounted where this layout requires, and k3s would
      put etcd's WAL AND the containerd image store (21-25 GB measured on dev) on that
      filesystem instead. On prod that is /var at 11.8 GiB: it would fill during the first
      image pulls and etcd would stop being able to fsync its WAL. DO NOT install k3s on
      this node until the mount is correct.
    quiet: true
  when: node_baseline_etcd_source == 'shared-data-partition'

- name: "etcd (no dedicated device): show which filesystem etcd will live on"
  ansible.builtin.debug:
    msg: >-
      etcd data dir {{ node_baseline_etcd_mount }} has NO dedicated device
      (source={{ node_baseline_etcd_source }}); it lands on
      {{ node_baseline_etcd_backing_fs.stdout_lines[1] | default('') | trim }}
      (SOURCE TARGET SIZE AVAIL). Shared-device I/O contention is an accepted, to-be-
      measured risk — §2.
  when: node_baseline_etcd_source in ['root-filesystem', 'shared-data-partition']