60 lines
3.3 KiB
Markdown
60 lines
3.3 KiB
Markdown
# infra/cloud-init — first-boot baseline templates
|
|
|
|
> **RETIRED (API-only migration):** the `user-data.yaml.tftpl` snippet was removed
|
|
> when Proxmox provisioning moved to **API-only native cloud-init**. Custom snippets
|
|
> require SSH/SFTP to the Proxmox host, which is not reachable here, so bpg's native
|
|
> `initialization` block now sets only the login user + SSH keys and static IP/DNS
|
|
> (via the built-in cloud-init API options). The rich first-boot baseline (packages,
|
|
> kernel modules, Longhorn disk, swap-off) now lives in the Ansible **`node_baseline`
|
|
> role** (`infra/ansible`), run before k3s. The notes below are historical context.
|
|
|
|
The "Baseline" node lifecycle stage (docs/02 stage 2). `user-data.yaml.tftpl` was
|
|
rendered by OpenTofu (`templatefile()`) and shipped to Proxmox as a **cloud-init
|
|
user-data snippet** (`proxmox_virtual_environment_file`, one per node), referenced
|
|
by each VM's `initialization.user_data_file_id`. **Authoring only — never applied.**
|
|
|
|
> Migrated from libvirt: bpg/proxmox generates the cloud-init **network-config**
|
|
> (static IP/gateway/DNS) and **meta-data** natively from the VM's `initialization`
|
|
> block, so the old `network-config.yaml.tftpl` / `meta-data.yaml.tftpl` templates
|
|
> were removed. Only the user-data (this baseline) is still a hand-authored template.
|
|
> Requires the `snippets` content type enabled on the image datastore (see ../tofu/README).
|
|
|
|
Serves **R-ENV-01** (nodes are "created, baselined, cluster-joined, managed as
|
|
code") and lays the kernel/OS floor ADR-0001 (k3s) + ADR-0002 (Cilium) +
|
|
ADR-0003 (Longhorn) + ADR-0005 (KubeVirt) require.
|
|
|
|
## Files
|
|
- `user-data.yaml.tftpl` — `#cloud-config`: login user + injected SSH **public**
|
|
key, hostname, package baseline, kernel modules, sysctls, Longhorn data-disk
|
|
mount, swap-off.
|
|
|
|
`${...}` tokens are **OpenTofu** template vars filled by `../tofu/main.tf`
|
|
(`hostname`, `fqdn`, `username`, `ssh_authorized_key`) — they are not cloud-init/Jinja.
|
|
IP/gateway/DNS are no longer templated here; bpg/proxmox injects them from the VM's
|
|
`initialization` block.
|
|
|
|
## Baseline (cloud-init's job) vs. Ansible's job
|
|
|
|
| cloud-init does (here) | Ansible does (T-1.3) |
|
|
|---|---|
|
|
| user + SSH public key, hostname | install k3s (HA, embedded etcd) |
|
|
| static networking | join nodes, disable flannel/kube-proxy/traefik/servicelb |
|
|
| `overlay`, `br_netfilter`, `iscsi_tcp`, `dm_crypt`, `nfs` modules | install Cilium (kube-proxy replacement, Gateway API, Hubble) |
|
|
| bridge-nf + `ip_forward` sysctls, inotify limits | verify readiness |
|
|
| open-iscsi + nfs-common (Longhorn client prereqs) | Flux + SOPS bootstrap handoff |
|
|
| mount `/dev/vdb` at `/var/lib/longhorn` | — |
|
|
| `swapoff` + strip swap from fstab | — |
|
|
|
|
### Why these specifics
|
|
- **overlay / br_netfilter + bridge-nf-call sysctls** — pod networking / iptables
|
|
bridging Cilium and kube-networking need.
|
|
- **ip_forward / forwarding** — routing between pods, nodes, and egress.
|
|
- **iscsi_tcp + open-iscsi (iscsid) and nfs / nfs-common** — Longhorn attaches
|
|
volumes over iSCSI and uses the NFS client for backup/RWX (ADR-0003).
|
|
- **dm_crypt** — headroom for Longhorn volume encryption.
|
|
- **swap off** — kubelet/k3s expectation.
|
|
- **qemu-guest-agent** — lets libvirt/tofu read guest state.
|
|
|
|
Nested-virt for KubeVirt (Phase 5) comes from the tofu `host-passthrough` CPU
|
|
mode, not from here.
|