diff --git a/infra/PROD-PROVISIONING.md b/infra/PROD-PROVISIONING.md index bca74c7..0371ff5 100644 --- a/infra/PROD-PROVISIONING.md +++ b/infra/PROD-PROVISIONING.md @@ -24,7 +24,7 @@ Fill a row per node as it lands. Node 1 is as-found on 2026-08-31. | | node 1 | node 2 | node 3 | |---|---|---|---| | Hostname | `k3s-prod-1` | `k3s-prod-2` | `k3s-prod-3` | -| LAN IP | `192.168.2.10` | — | — | +| LAN IP | `192.168.2.10` | `192.168.2.11` | `192.168.2.12` | | Model | HP EliteDesk 800 G5 Desktop Mini | | | | CPU | i7-9700T, 8c/8t, VT-x | | | | RAM | 31 GiB | | | @@ -99,8 +99,25 @@ Consequences to settle before imaging nodes 2 and 3: - [ ] **Static IP** on the prod LAN, recorded in §0. Node 1 is `192.168.2.10/24`, gateway `192.168.2.1`. -- [ ] **Reserve the VIP addresses** before the build — prod needs its own kube-vip control-plane VIP - *and* its own Cilium L2 announcement pool for ingress. Neither exists yet (§7). +- [x] **Prod address plan — DECIDED (operator, 2026-08-31).** + + | Role | Address | Owned by | + |---|---|---| + | `k3s-prod-1` / `-2` / `-3` | `192.168.2.10` / `.11` / `.12` | Ansible inventory | + | **kube-vip control-plane VIP** | `192.168.2.9` | Ansible (`group_vars/k3s_prod`) | + | **Ingress Gateway VIP** (`GATEWAY_VIP`) | `192.168.2.20` | Flux (`clusters/prod`) | + | **Cilium L2 announcement pool** | `192.168.2.20` – `.30` | Flux (`platform-config/cilium`) | + + Two deliberate properties, both mirroring dev: + - The **control-plane VIP sits outside the LB pool** (`.9`, below the nodes), so kube-vip and + Cilium LB-IPAM can never contend for the same address. Dev achieves this the other way + round (`.49` control-plane, pool `.53-.60`). + - **`GATEWAY_VIP` is the FIRST address of the pool.** Cilium LB-IPAM can only allocate from a + defined pool block, so a Gateway VIP outside every pool would not be allocatable — the pool + was widened from `.21-.30` to `.20-.30` for exactly this reason. Dev has the same shape + (`GATEWAY_VIP` `.53` = first of `.53-.60`). +- [ ] **Reserve all of `192.168.2.9` and `.20-.30` outside any DHCP range** on the prod LAN before + the build, and confirm nothing else on the network already answers on them. - [ ] **Resolver: configure at least two upstreams.** Node 1 currently has a single IPv4 nameserver (`192.168.2.1`). A single upstream is a known SPOF that has already bitten this project — it broke containerd image pulls and CoreDNS forwarding on dev. A fix exists but is **stranded @@ -129,10 +146,12 @@ Consequences to settle before imaging nodes 2 and 3: ``` Prod nodes must therefore set `ansible_become: true` in their group_vars and must **not** inherit any passwordless-sudo assumption from `group_vars/all.yml`. -- [ ] **Drive health baseline** (§2), needs root: +- [ ] **Drive health baseline** (§2). **Do not hand-install the tooling** — package installation is + the `node_baseline` role's job, never a manual `apt-get` on a node (operator ruling, + 2026-08-31; same click-ops-never principle as the rest of the repo). `smartmontools` is part + of `node_baseline_packages`, so it arrives with the baseline play. Reading the drive is then: ```bash - sudo apt-get update && sudo apt-get install -y smartmontools - sudo smartctl -a /dev/sda # Power_On_Hours, Percent_Lifetime_Remaining, reallocated sectors + sudo smartctl -a /dev/sda # Power_On_Hours, Percent_Lifetime_Remaining, reallocated sectors ``` ## 6. Ansible baseline & cluster join