Phase 1 (author-only): node provisioning IaC — tofu + cloud-init + Ansible (T-1.1/1.2/1.3)

OpenTofu libvirt module (k3s-dev NAT 10.73.0.0/24, 3x 6GiB/4vCPU VMs, 20G OS + 40G thin data, host-passthrough), cloud-init baseline (k3s/Cilium/Longhorn prereqs, static net), Ansible k3s-HA + Cilium (kube-proxy replacement, Gateway API, Hubble; flannel/kube-proxy/Traefik disabled). PM fix: escape ${...} in a cloud-init template comment (tofu templatefile parse). tofu fmt/init/validate = Success; ansible --syntax-check + yamllint pass. NOTHING executed (no apply). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-08-16 01:21:00 -05:00
1 parent 224cd8ac35
commit f2a7e42068
@@ -0,0 +1,3 @@
+# Runtime cache written by the cilium/verify/flux plays (fetched kubeconfig,
+# rendered Cilium values). Never commit — the kubeconfig is a credential.
+.cache/
@@ -1,3 +1,70 @@
-# infra/ansible
+# infra/ansible — k3s HA + Cilium bootstrap (Phase 1 T-1.3)

-_Phase 1 (T-1.x) — to be authored on execution go. See infra/PHASE1-SCOPE.md._
+The "Cluster-join" node lifecycle stage (docs/02 stage 3) and the Flux hand-off
+(T-1.4 call-in). Installs a 3-server HA k3s cluster (embedded etcd), Cilium as the
+CNI, verifies readiness, then invokes the `bootstrap/` entrypoints.
+
+> **AUTHOR-ONLY — NOTHING HERE HAS BEEN EXECUTED.** No `ansible-playbook` run, no
+> `tofu apply`, no cluster contacted. The PM static-validates (see below).
+
+Serves **R-ENV-01/02**, **R-PLAT-01**, ADR-0001 (k3s), ADR-0002 (Cilium),
+ADR-0004 (Flux hand-off), ADR-0007 (SOPS).
+
+## Prerequisites
+1. `tofu apply` in `../tofu/` has created the VMs; cloud-init (`../cloud-init/`)
+   has baselined them (modules, sysctls, Longhorn disk, SSH user).
+2. Inventory reflects the node IPs. `inventory/hosts.ini` ships aligned with the
+   LOCKED defaults; regenerate from tofu if overridden:
+   `cd ../tofu && tofu output -raw ansible_inventory > ../ansible/inventory/hosts.ini`
+3. Run from the dev container (has `kubectl` + `helm`, used by the Cilium/verify/
+   Flux plays via `delegate_to: localhost`).
+
+## Run order
+```
+cd infra/ansible
+ansible-playbook playbooks/site.yml        # everything, in order
+# — or step by step —
+ansible-playbook playbooks/k3s.yml         # 1. HA k3s (primary --cluster-init, then joins)
+ansible-playbook playbooks/cilium.yml      # 2. Cilium CNI (cluster is NotReady until this)
+ansible-playbook playbooks/verify.yml      # 3. gate #1: 3 nodes Ready, Hubble up, no kube-proxy
+ansible-playbook playbooks/flux-bootstrap.yml   # 4. hand off to bootstrap/ (SOPS then Flux)
+```
+
+## What each layer does
+| Playbook / role | Action |
+|---|---|
+| `k3s.yml` / `k3s_server` | k3s server install with `--flannel-backend=none --disable-kube-proxy --disable-network-policy --disable=traefik,servicelb`; primary `--cluster-init`, secondaries join via the primary's node-token (fetched, never pre-stored) |
+| `cilium.yml` / `cilium` | Gateway API CRDs, then Helm `cilium` with `kubeProxyReplacement`, `k8sServiceHost/Port`, `gatewayAPI`, `egressGateway`, `hubble`, `l2announcements` |
+| `verify.yml` / `cluster_verify` | waits nodes Ready + Cilium/Hubble rollout; asserts kube-proxy absent |
+| `flux-bootstrap.yml` / `flux_bootstrap` | calls `bootstrap/` (SOPS age key, then `flux bootstrap --path=clusters/dev`) |
+
+## Cilium ownership / parity contract (docs/02)
+Ansible installs Cilium; the Flux `platform/cilium` HelmRelease adopts config
+afterward. The parity-critical values **must match** between this role's
+`templates/cilium-values.yaml.j2` and that HelmRelease:
+- `kubeProxyReplacement: true`
+- `k8sServiceHost: {{ k3s_api_host }}` (= primary node IP, `10.73.0.11` by default)
+- `k8sServicePort: 6443`
+
+The HelmRelease currently has `k8sServiceHost: "REPLACE-ME"` — WS-NET must set it
+to the same value this role uses (surfaced by `tofu output primary_node_ip`).
+
+## Interface assumed of `bootstrap/` (authored by other workers, NOT here)
+`roles/flux_bootstrap` invokes two entrypoints via overridable command vars:
+- `flux_bootstrap_sops_cmd` → `bootstrap/sops/install-age-key.sh` (Security
+  Engineer, T-1.4): installs the age **private** key as the `sops-age` Secret in
+  `flux-system`. Key never committed (`.gitignore` guards `age.key`/`*.key`).
+- `flux_bootstrap_flux_cmd` → `bootstrap/flux/bootstrap.sh` (GitOps Engineer,
+  T-1.4): runs `flux bootstrap ... --path=clusters/dev`.
+
+**These script names/args are assumptions** — if the bootstrap deliverables
+expose different entrypoints, override the two vars in
+`roles/flux_bootstrap/defaults/main.yml`; the task graph is unchanged. Flagged
+for PM reconciliation.
+
+## Static validation (PM, in dev container — NOT run here)
+```
+yamllint -c ../../ci/yamllint.yaml infra/ansible
+ansible-playbook -i inventory/hosts.ini --syntax-check playbooks/site.yml
+ansible-lint playbooks/ roles/          # if the profile is enforced for infra/
+```
@@ -0,0 +1,11 @@
+[defaults]
+inventory = inventory/hosts.ini
+roles_path = roles
+host_key_checking = False
+retry_files_enabled = False
+stdout_callback = yaml
+interpreter_python = auto_silent
+
+[ssh_connection]
+pipelining = True
+ssh_args = -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=accept-new
@@ -0,0 +1,11 @@
+---
+# Shared connection + cluster facts for every k3s node.
+
+# cloud-init created this login user (tofu var login_user); it has NOPASSWD sudo.
+ansible_user: k3s
+
+# The kube-apiserver endpoint the whole stack agrees on: the primary node IP.
+# Used for k3s --tls-san / secondary --server AND for Cilium k8sServiceHost so the
+# Ansible bootstrap and the Flux HelmRelease stay in PARITY (docs/02 contract).
+k3s_api_host: "{{ hostvars[groups['k3s_primary'][0]]['ansible_host'] }}"
+k3s_api_port: 6443
@@ -0,0 +1,17 @@
+# Static inventory — ALIGNED with infra/tofu defaults (LOCKED PHASE1-SCOPE IPs).
+# Regenerate from tofu when IPs/names are overridden:
+#   cd ../tofu && tofu output -raw ansible_inventory > ../ansible/inventory/hosts.ini
+#
+# k3s_primary = the --cluster-init server (embedded-etcd founder + Cilium k8sServiceHost).
+# k3s_secondary = the two servers that join it. All three are servers (HA embedded etcd).
+
+[k3s_primary]
+k3s-dev-1 ansible_host=10.73.0.11
+
+[k3s_secondary]
+k3s-dev-2 ansible_host=10.73.0.12
+k3s-dev-3 ansible_host=10.73.0.13
+
+[k3s_servers:children]
+k3s_primary
+k3s_secondary
@@ -0,0 +1,9 @@
+---
+# Install Cilium as the CNI (ADR-0002). The cluster is NotReady until this runs.
+# Executed from the controller against a fetched kubeconfig (role handles delegation).
+- name: Install Cilium CNI
+  hosts: k3s_servers
+  become: true
+  gather_facts: false
+  roles:
+    - role: cilium
@@ -0,0 +1,9 @@
+---
+# FINAL step: hand off to Flux + SOPS (T-1.4). Calls into bootstrap/ (authored by
+# the Security + GitOps Engineers) — this play does not author that subtree.
+- name: Flux + SOPS bootstrap hand-off
+  hosts: k3s_servers
+  become: false
+  gather_facts: false
+  roles:
+    - role: flux_bootstrap
@@ -0,0 +1,22 @@
+---
+# Install the HA k3s control plane (embedded etcd, 3 servers) — ADR-0001.
+# Primary founds the cluster (--cluster-init); secondaries join it in a second
+# play so the primary's node-token fact is available to them.
+
+- name: k3s primary (cluster-init)
+  hosts: k3s_primary
+  become: true
+  gather_facts: true
+  roles:
+    - role: k3s_server
+      vars:
+        k3s_server_mode: primary
+
+- name: k3s secondaries (join)
+  hosts: k3s_secondary
+  become: true
+  gather_facts: true
+  roles:
+    - role: k3s_server
+      vars:
+        k3s_server_mode: secondary
@@ -0,0 +1,15 @@
+---
+# Full dev/test cluster bring-up, in order (docs/02 sequencing; run AFTER
+# `tofu apply` has created and cloud-init has baselined the VMs):
+#   k3s HA install -> Cilium CNI -> verify readiness -> Flux + SOPS hand-off.
+- name: Import - install k3s HA control plane
+  ansible.builtin.import_playbook: k3s.yml
+
+- name: Import - install Cilium CNI
+  ansible.builtin.import_playbook: cilium.yml
+
+- name: Import - verify cluster readiness
+  ansible.builtin.import_playbook: verify.yml
+
+- name: Import - Flux + SOPS bootstrap hand-off
+  ansible.builtin.import_playbook: flux-bootstrap.yml
@@ -0,0 +1,8 @@
+---
+# Verify Phase-1 exit gate #1: 3 nodes Ready, Cilium/Hubble healthy, no kube-proxy.
+- name: Verify cluster readiness
+  hosts: k3s_servers
+  become: false
+  gather_facts: false
+  roles:
+    - role: cluster_verify
@@ -0,0 +1,28 @@
+---
+# cilium role defaults (ADR-0002, docs/02 Cilium-ownership contract).
+# This role performs the INITIAL Cilium install; the Flux platform/cilium
+# HelmRelease adopts config afterward. The parity-critical values
+# (kubeProxyReplacement, k8sServiceHost/Port) are templated from k3s_api_host.
+
+# Pin inside the platform/cilium HelmRelease window (>=1.16.0 <1.17.0).
+cilium_version: "1.16.5"
+cilium_chart_repo_name: "cilium"
+cilium_chart_repo_url: "https://helm.cilium.io/"
+cilium_namespace: "kube-system"
+
+# Gateway API CRDs (standard channel) must exist BEFORE Cilium with gatewayAPI.enabled.
+# Cilium 1.16 targets Gateway API v1.2.x.
+gateway_api_version: "v1.2.1"
+gateway_api_crd_urls:
+  - "https://github.com/kubernetes-sigs/gateway-api/releases/download/{{ gateway_api_version }}/standard-install.yaml"
+  # TLSRoute (experimental) is required by Cilium's Gateway API support.
+  - "https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/{{ gateway_api_version }}/config/crd/experimental/gateway.networking.k8s.io_tlsroutes.yaml"
+
+# Local kubeconfig fetched from the primary. *.kubeconfig is gitignored.
+cilium_kubeconfig: "{{ playbook_dir }}/../.cache/dev.kubeconfig"
+cilium_values_path: "{{ playbook_dir }}/../.cache/cilium-values.yaml"
+
+# dev: single operator replica to save RAM (PHASE1-SCOPE RAM ceiling). NOT a
+# parity field — Flux may differ here; only kubeProxyReplacement/k8sServiceHost
+# /k8sServicePort must match the HelmRelease.
+cilium_operator_replicas: 1
@@ -0,0 +1,88 @@
+---
+# Install Cilium as the CNI (ADR-0002). Runs from the controller (dev container,
+# which has kubectl + helm) against a kubeconfig fetched from the primary node.
+# The cluster is NotReady until this completes — expected (PHASE1-SCOPE risks).
+# run_once: this whole role acts on the cluster once, not per host.
+
+- name: Fetch the k3s kubeconfig from the primary
+  ansible.builtin.slurp:
+    src: /etc/rancher/k3s/k3s.yaml
+  register: k3s_kubeconfig_raw
+  delegate_to: "{{ groups['k3s_primary'][0] }}"
+  run_once: true
+
+- name: Ensure the local cache dir exists
+  ansible.builtin.file:
+    path: "{{ cilium_kubeconfig | dirname }}"
+    state: directory
+    mode: "0700"
+  delegate_to: localhost
+  become: false
+  run_once: true
+
+- name: Write kubeconfig locally (server rewritten to the primary IP)
+  ansible.builtin.copy:
+    content: >-
+      {{ k3s_kubeconfig_raw.content | b64decode
+         | regex_replace('https://127.0.0.1:6443',
+                         'https://' ~ k3s_api_host ~ ':' ~ k3s_api_port) }}
+    dest: "{{ cilium_kubeconfig }}"
+    mode: "0600"
+  delegate_to: localhost
+  become: false
+  run_once: true
+
+- name: Install Gateway API CRDs (must precede Cilium gatewayAPI.enabled)
+  ansible.builtin.command:
+    cmd: "kubectl apply -f {{ item }}"
+  environment:
+    KUBECONFIG: "{{ cilium_kubeconfig }}"
+  loop: "{{ gateway_api_crd_urls }}"
+  delegate_to: localhost
+  become: false
+  run_once: true
+  changed_when: true
+
+- name: Add the Cilium Helm repo
+  ansible.builtin.command:
+    cmd: "helm repo add {{ cilium_chart_repo_name }} {{ cilium_chart_repo_url }}"
+  delegate_to: localhost
+  become: false
+  run_once: true
+  register: cilium_repo_add
+  changed_when: "'has been added' in cilium_repo_add.stdout"
+  failed_when:
+    - cilium_repo_add.rc != 0
+    - "'already exists' not in cilium_repo_add.stderr"
+
+- name: Update Helm repos
+  ansible.builtin.command:
+    cmd: "helm repo update {{ cilium_chart_repo_name }}"
+  delegate_to: localhost
+  become: false
+  run_once: true
+  changed_when: false
+
+- name: Render the Cilium values file (parity-critical settings from k3s_api_host)
+  ansible.builtin.template:
+    src: cilium-values.yaml.j2
+    dest: "{{ cilium_values_path }}"
+    mode: "0644"
+  delegate_to: localhost
+  become: false
+  run_once: true
+
+- name: Install/upgrade Cilium via Helm
+  ansible.builtin.command:
+    cmd: >-
+      helm upgrade --install cilium {{ cilium_chart_repo_name }}/cilium
+      --version {{ cilium_version }}
+      --namespace {{ cilium_namespace }}
+      --values {{ cilium_values_path }}
+      --wait --timeout 10m
+  environment:
+    KUBECONFIG: "{{ cilium_kubeconfig }}"
+  delegate_to: localhost
+  become: false
+  run_once: true
+  changed_when: true
@@ -0,0 +1,40 @@
+# Cilium Helm values — rendered by the Ansible bootstrap (T-1.3).
+# PARITY CONTRACT (docs/02): the fields marked [parity] MUST equal the
+# platform/cilium HelmRelease values, or Flux and the bootstrap fight over the CNI.
+---
+# [parity] k3s kube-proxy is disabled → Cilium replaces it (ADR-0001/0002).
+kubeProxyReplacement: true
+# [parity] = the k3s API endpoint (primary node IP). The Flux HelmRelease's
+# k8sServiceHost "REPLACE-ME" must be set to this same value.
+k8sServiceHost: "{{ k3s_api_host }}"
+k8sServicePort: {{ k3s_api_port }}
+
+# Ingress via Gateway API — no Traefik (docs/09, ADR-0002).
+gatewayAPI:
+  enabled: true
+
+# Egress-class steering (ADR-0002; VPN model ADR-0013, Phase 2/3).
+egressGateway:
+  enabled: true
+
+# Flow observability / isolation audit (ADR-0002).
+hubble:
+  enabled: true
+  relay:
+    enabled: true
+  ui:
+    enabled: true
+
+# Ingress VIP announcement on the NAT LAN. IP pool / CiliumL2AnnouncementPolicy
+# (from the reserved 10.73.0.240-.250 range) is authored by WS-NET in Phase 2.
+l2announcements:
+  enabled: true
+
+# k3s ships its own cgroup mount; point Cilium at it instead of auto-mounting.
+cgroup:
+  autoMount:
+    enabled: false
+  hostRoot: /run/cilium/cgroupv2
+
+operator:
+  replicas: {{ cilium_operator_replicas }}
@@ -0,0 +1,4 @@
+---
+# cluster_verify role defaults.
+cluster_verify_kubeconfig: "{{ playbook_dir }}/../.cache/dev.kubeconfig"
+cluster_verify_node_ready_timeout: "300s"
@@ -0,0 +1,65 @@
+---
+# Verify the cluster reached the Phase-1 exit gate #1: 3 nodes Ready, Cilium
+# healthy (Hubble up), no flannel/kube-proxy. Runs from the controller once.
+
+- name: Wait for all nodes to be Ready (Cilium must be up first)
+  ansible.builtin.command:
+    cmd: >-
+      kubectl wait --for=condition=Ready nodes --all
+      --timeout={{ cluster_verify_node_ready_timeout }}
+  environment:
+    KUBECONFIG: "{{ cluster_verify_kubeconfig }}"
+  delegate_to: localhost
+  become: false
+  run_once: true
+  changed_when: false
+
+- name: Wait for the Cilium DaemonSet to roll out
+  ansible.builtin.command:
+    cmd: >-
+      kubectl -n kube-system rollout status ds/cilium --timeout=300s
+  environment:
+    KUBECONFIG: "{{ cluster_verify_kubeconfig }}"
+  delegate_to: localhost
+  become: false
+  run_once: true
+  changed_when: false
+
+- name: Wait for hubble-relay to be available
+  ansible.builtin.command:
+    cmd: >-
+      kubectl -n kube-system rollout status deploy/hubble-relay --timeout=300s
+  environment:
+    KUBECONFIG: "{{ cluster_verify_kubeconfig }}"
+  delegate_to: localhost
+  become: false
+  run_once: true
+  changed_when: false
+
+- name: Confirm kube-proxy is absent (Cilium replaces it)
+  ansible.builtin.command:
+    cmd: "kubectl -n kube-system get ds kube-proxy"
+  environment:
+    KUBECONFIG: "{{ cluster_verify_kubeconfig }}"
+  delegate_to: localhost
+  become: false
+  run_once: true
+  register: kube_proxy_check
+  changed_when: false
+  failed_when: kube_proxy_check.rc == 0
+
+- name: Summarise cluster state
+  ansible.builtin.command:
+    cmd: "kubectl get nodes -o wide"
+  environment:
+    KUBECONFIG: "{{ cluster_verify_kubeconfig }}"
+  delegate_to: localhost
+  become: false
+  run_once: true
+  register: cluster_nodes
+  changed_when: false
+
+- name: Show nodes
+  ansible.builtin.debug:
+    var: cluster_nodes.stdout_lines
+  run_once: true
@@ -0,0 +1,22 @@
+---
+# flux_bootstrap role defaults — the hand-off to the OTHER worker's bootstrap/.
+#
+# THIS ROLE AUTHORS THE CALL, IT DOES NOT OWN bootstrap/. The Security Engineer
+# (T-1.4 SOPS/age) and GitOps Engineer (T-1.4 Flux) author bootstrap/sops/ and
+# bootstrap/flux/. This role INVOKES their entrypoints. The exact script names /
+# args below are ASSUMPTIONS the PM must reconcile with those deliverables — see
+# roles/flux_bootstrap/tasks/main.yml and infra/ansible/README.md "Interface".
+
+flux_bootstrap_kubeconfig: "{{ playbook_dir }}/../.cache/dev.kubeconfig"
+
+# Repo root, relative to playbooks/ (infra/ansible/playbooks -> ../../..).
+flux_bootstrap_repo_root: "{{ playbook_dir }}/../../.."
+
+# Assumed interface of bootstrap/sops (Security Engineer, T-1.4):
+#   installs the age private key as the `sops-age` Secret in flux-system so Flux
+#   can decrypt. Private key stays OUT of git (.gitignore guards age.key/*.key).
+flux_bootstrap_sops_cmd: "bash {{ flux_bootstrap_repo_root }}/bootstrap/sops/install-age-key.sh"
+
+# Assumed interface of bootstrap/flux (GitOps Engineer, T-1.4):
+#   runs `flux bootstrap ... --path=clusters/dev` against this cluster.
+flux_bootstrap_flux_cmd: "bash {{ flux_bootstrap_repo_root }}/bootstrap/flux/bootstrap.sh"
@@ -0,0 +1,52 @@
+---
+# FINAL play of the bring-up: hand off to Flux + SOPS (Phase 1 T-1.4, docs/02
+# "hands off to Flux"; ADR-0004, ADR-0007). This role does NOT author bootstrap/
+# — it calls the entrypoints the Security + GitOps Engineers author there.
+#
+# Order matters: install the age decryption key FIRST (so Flux can decrypt on its
+# very first reconcile), THEN run `flux bootstrap` at clusters/dev.
+#
+# The two commands are variables (see defaults/main.yml). If the bootstrap workers
+# expose different entrypoints, override flux_bootstrap_sops_cmd / _flux_cmd — the
+# task graph does not change.
+
+- name: Assert the bootstrap/ subtree exists (authored by the other worker)
+  ansible.builtin.stat:
+    path: "{{ flux_bootstrap_repo_root }}/bootstrap"
+  register: bootstrap_dir
+  delegate_to: localhost
+  become: false
+  run_once: true
+  failed_when: not bootstrap_dir.stat.exists
+
+- name: Install the SOPS age key as a cluster Secret (bootstrap/sops)
+  ansible.builtin.command:
+    cmd: "{{ flux_bootstrap_sops_cmd }}"
+  environment:
+    KUBECONFIG: "{{ flux_bootstrap_kubeconfig }}"
+  delegate_to: localhost
+  become: false
+  run_once: true
+  changed_when: true
+
+- name: Bootstrap Flux against clusters/dev (bootstrap/flux)
+  ansible.builtin.command:
+    cmd: "{{ flux_bootstrap_flux_cmd }}"
+  environment:
+    KUBECONFIG: "{{ flux_bootstrap_kubeconfig }}"
+  delegate_to: localhost
+  become: false
+  run_once: true
+  changed_when: true
+
+- name: Wait for Flux to report the platform Kustomization Ready
+  ansible.builtin.command:
+    cmd: >-
+      kubectl -n flux-system wait kustomization/platform
+      --for=condition=Ready --timeout=600s
+  environment:
+    KUBECONFIG: "{{ flux_bootstrap_kubeconfig }}"
+  delegate_to: localhost
+  become: false
+  run_once: true
+  changed_when: false
@@ -0,0 +1,17 @@
+---
+# k3s_server role defaults (ADR-0001).
+
+# Best-effort version pin (kept near the ci/versions kubectl 1.31.x line). Bump
+# deliberately — a version change is a code change. Empty string => latest stable.
+k3s_version: "v1.31.5+k3s1"
+
+k3s_install_url: "https://get.k3s.io"
+k3s_install_script: "/usr/local/bin/k3s-install.sh"
+
+# Components disabled at install so Cilium + Gateway API own networking (ADR-0001/0002).
+# flannel + kube-proxy + network-policy are disabled via dedicated flags below;
+# traefik + servicelb via --disable.
+k3s_disable: "traefik,servicelb"
+
+# Set per-play: 'primary' (runs --cluster-init) or 'secondary' (joins the primary).
+k3s_server_mode: "secondary"
@@ -0,0 +1,66 @@
+---
+# Install k3s as an HA server (embedded etcd) with Cilium-required components
+# disabled (ADR-0001, docs/02 stage 3 "Cluster-join"). The primary runs
+# --cluster-init; secondaries join it with the shared node-token.
+
+- name: Download the k3s install script
+  ansible.builtin.get_url:
+    url: "{{ k3s_install_url }}"
+    dest: "{{ k3s_install_script }}"
+    mode: "0755"
+
+- name: Compose the common k3s server flags
+  ansible.builtin.set_fact:
+    k3s_common_args: >-
+      server
+      --flannel-backend=none
+      --disable-kube-proxy
+      --disable-network-policy
+      --disable={{ k3s_disable }}
+      --write-kubeconfig-mode=644
+      --tls-san={{ k3s_api_host }}
+      --node-ip={{ ansible_host }}
+
+# --- primary: found the cluster --------------------------------------------
+- name: Install k3s primary (cluster-init, embedded etcd)
+  ansible.builtin.command:
+    cmd: "{{ k3s_install_script }} {{ k3s_common_args }} --cluster-init"
+    creates: /etc/rancher/k3s/k3s.yaml
+  environment:
+    INSTALL_K3S_VERSION: "{{ k3s_version }}"
+  when: k3s_server_mode == 'primary'
+
+- name: Wait for the k3s node-token to exist (primary)
+  ansible.builtin.wait_for:
+    path: /var/lib/rancher/k3s/server/token
+    timeout: 180
+  when: k3s_server_mode == 'primary'
+
+- name: Read the k3s node-token (primary)
+  ansible.builtin.slurp:
+    src: /var/lib/rancher/k3s/server/token
+  register: k3s_token_raw
+  when: k3s_server_mode == 'primary'
+
+- name: Publish the node-token as a fact for secondaries (primary)
+  ansible.builtin.set_fact:
+    k3s_node_token: "{{ k3s_token_raw.content | b64decode | trim }}"
+  when: k3s_server_mode == 'primary'
+
+# --- secondaries: join the existing cluster --------------------------------
+- name: Install k3s secondary (join as server)
+  ansible.builtin.command:
+    cmd: >-
+      {{ k3s_install_script }} {{ k3s_common_args }}
+      --server https://{{ k3s_api_host }}:{{ k3s_api_port }}
+    creates: /etc/rancher/k3s/k3s.yaml
+  environment:
+    INSTALL_K3S_VERSION: "{{ k3s_version }}"
+    K3S_TOKEN: "{{ hostvars[groups['k3s_primary'][0]]['k3s_node_token'] }}"
+  when: k3s_server_mode == 'secondary'
+
+- name: Wait for the k3s API to be serving locally
+  ansible.builtin.wait_for:
+    host: "127.0.0.1"
+    port: "{{ k3s_api_port }}"
+    timeout: 180