docs(ansible): add re-provision runbook + node-host playbook list
A node rebuild (tofu -replace / fresh OS) wipes host-level config Flux/Helm don't manage. site.yml omits kube-vip.yml and vpn.egress.gateway.yml, so a re-provision missed the node-host OpenVPN on k3s-dev-3 and dropped all egress-class=vpn pod traffic. Document the node-host layers that must be re-run after any re-provision, and correct the stale author-only banner (these plays are live). Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
1 parent
22db5fdb63
commit
1f921fbfed
1 file changed
+29
-3
+29
-3
@@ -4,8 +4,9 @@ The "Cluster-join" node lifecycle stage (docs/02 stage 3) and the Flux hand-off
|
||||
(T-1.4 call-in). Installs a 3-server HA k3s cluster (embedded etcd), Cilium as the
|
||||
CNI, verifies readiness, then invokes the `bootstrap/` entrypoints.
|
||||
|
||||
> **AUTHOR-ONLY — NOTHING HERE HAS BEEN EXECUTED.** No `ansible-playbook` run, no
|
||||
> `tofu apply`, no cluster contacted. The PM static-validates (see below).
|
||||
> **LIVE.** These plays have been executed against the running dev cluster (k3s
|
||||
> v1.36.4+k3s1, 3 nodes). This is no longer author-only scaffolding — treat it as the
|
||||
> operational bring-up + re-provision runbook. See **Re-provisioning a node** below.
|
||||
|
||||
Serves **R-ENV-01/02**, **R-PLAT-01**, ADR-0001 (k3s), ADR-0002 (Cilium),
|
||||
ADR-0004 (Flux hand-off), ADR-0007 (SOPS).
|
||||
@@ -22,14 +23,39 @@ ADR-0004 (Flux hand-off), ADR-0007 (SOPS).
|
||||
## Run order
|
||||
```
|
||||
cd infra/ansible
|
||||
ansible-playbook playbooks/site.yml # everything, in order
|
||||
ansible-playbook playbooks/site.yml # core bring-up, in order (baseline→k3s→cilium→verify→flux)
|
||||
# — or step by step —
|
||||
ansible-playbook playbooks/node-baseline.yml # 0. packages, kernel modules, Longhorn disk, swap-off
|
||||
ansible-playbook playbooks/k3s.yml # 1. HA k3s (primary --cluster-init, then joins)
|
||||
ansible-playbook playbooks/cilium.yml # 2. Cilium CNI (cluster is NotReady until this)
|
||||
ansible-playbook playbooks/verify.yml # 3. gate #1: 3 nodes Ready, Hubble up, no kube-proxy
|
||||
ansible-playbook playbooks/flux-bootstrap.yml # 4. hand off to bootstrap/ (SOPS then Flux)
|
||||
```
|
||||
|
||||
> **`site.yml` is NOT everything.** It covers the core bring-up only. Two node-host
|
||||
> layers live OUTSIDE `site.yml` and must be run separately:
|
||||
> ```
|
||||
> ansible-playbook playbooks/kube-vip.yml # control-plane VIP (192.168.1.49); targets k3s_servers
|
||||
> ansible-playbook playbooks/vpn.egress.gateway.yml # node-host OpenVPN split-tunnel + kill-switch; targets k3s-dev-3
|
||||
> ```
|
||||
|
||||
## Re-provisioning a node (tofu -replace / fresh OS)
|
||||
A node rebuild wipes **everything on the host filesystem** — including host-level config
|
||||
that Flux/Helm do NOT manage. After `tofu apply`/`-replace` re-creates a node, re-run the
|
||||
node-host layers or the node comes back missing pieces:
|
||||
|
||||
| Re-run | Why it doesn't survive a rebuild |
|
||||
|---|---|
|
||||
| `node-baseline.yml` | packages (incl. `cryptsetup`), kernel modules, Longhorn disk, swap-off |
|
||||
| `k3s.yml` | the k3s server binary + config + join (secondaries re-join the VIP) |
|
||||
| `kube-vip.yml` | the kube-vip auto-deploy manifest on the server's host path |
|
||||
| `vpn.egress.gateway.yml` | **the OpenVPN (PIA) client, `tun0`, systemd units + nftables kill-switch on k3s-dev-3 — miss this and `egress-class=vpn` pods (sonarr/radarr/qbittorrent) are kill-switch-dropped with no internet** |
|
||||
|
||||
Everything Flux owns (platform + apps) self-heals once the node re-joins; only the
|
||||
node-host layers above need a manual re-run. Verify the VPN after: on k3s-dev-3,
|
||||
`systemctl is-active vpn-egress-tunnel.service` and `curl --interface tun0 https://ipinfo.io/ip`
|
||||
should return a PIA exit IP (not the home ISP).
|
||||
|
||||
## What each layer does
|
||||
| Playbook / role | Action |
|
||||
|---|---|
|
||||
|
||||
Reference in new issue
Block a user