jakman85andClaude Opus 4.8
017ed0fb9c
Cluster is UP (3/3 Ready, Cilium OK); Phase 1 paused before Flux bootstrap (Option A: suspend platform/apps until Phase 2). Add infra/PHASE1-STATUS.md (what's done, state file locations, reconnect steps, remaining T-1.4/1.5, Phase-2 platform prereq), infra/cluster.sh (status/start/stop/kubeconfig), and document that tofu+ansible run on the host (libvirt provider limitation). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
58 lines
3.3 KiB
Markdown
58 lines
3.3 KiB
Markdown
# Phase 1 — Execution Status & Resume Guide
|
|
|
|
**Checkpoint: 2026-08-16.** The dev/test cluster is **built and running**. Phase 1 is paused before
|
|
the Flux/SOPS bootstrap (a deliberate decision — see "Remaining"). This doc is how a future session
|
|
picks up.
|
|
|
|
## DONE — a live 3-node HA k3s + Cilium cluster (T-1.1/1.2/1.3)
|
|
- `tofu apply` created the `k3s-dev` NAT network (10.73.0.0/24) + 3 VMs (`k3s-dev-1/2/3`,
|
|
6 GiB/4 vCPU, 20 G OS + 40 G data, host-passthrough) — all **running**.
|
|
- cloud-init baselined the nodes; k3s HA (embedded etcd, 3 servers) + Cilium installed by Ansible.
|
|
- **Live health:** `kubectl get nodes` → **3/3 Ready**; `cilium status` → **OK**; CoreDNS up.
|
|
|
|
### Fixes applied during execution (committed to main)
|
|
- `ansible.cfg`: `stdout_callback yaml` removed in ansible-core 2.20 → builtin default.
|
|
- `cilium-values`: `egressGateway` requires `bpf.masquerade: true` (+ `enableIPv4Masquerade`) or the
|
|
agent fatals — added.
|
|
|
|
## Toolchain reality (important for resume)
|
|
The pinned **dev container** (`infra/devcontainer/`) holds flux/sops/age/kubectl/helm. BUT:
|
|
- **`tofu` runs on the HOST** — the `dmacvicar/libvirt` provider's read-write connect fails inside a
|
|
container (`Cannot find start time for pid`). The pinned binary was extracted to
|
|
**`~/.local/bin/tofu`** (OpenTofu 1.9.0). Run tofu from the host with `LIBVIRT_DEFAULT_URI=qemu:///system`.
|
|
- **`ansible` runs on the HOST** (pre-installed ansible-core 2.20; needs native network to the nodes).
|
|
|
|
## State locations (persisted on disk; gitignored)
|
|
| What | Path |
|
|
|---|---|
|
|
| tofu state | `infra/tofu/terraform.tfstate` |
|
|
| tofu vars (ssh pubkey) | `infra/tofu/terraform.tfvars` |
|
|
| cluster kubeconfig | `infra/ansible/.cache/dev.kubeconfig` (API at 10.73.0.11:6443) |
|
|
| SSH key | `~/.ssh/home_network` (user `k3s`, NOPASSWD sudo) |
|
|
| VMs / network | libvirt `qemu:///system` (`virsh list`, net `k3s-dev`) |
|
|
|
|
## Reconnect / manage
|
|
```bash
|
|
infra/cluster.sh status # VMs + kubectl get nodes
|
|
infra/cluster.sh start|stop # boot/graceful-shutdown the 3 VMs (state persists on disk)
|
|
eval "$(infra/cluster.sh kubeconfig)" # export KUBECONFIG
|
|
kubectl get nodes
|
|
ssh -i ~/.ssh/home_network [email protected]
|
|
```
|
|
The VMs are **running** at checkpoint (~18 GiB RAM). `infra/cluster.sh stop` frees RAM; k3s/etcd/
|
|
Cilium recover on `start` (data on the disks).
|
|
|
|
## REMAINING Phase 1
|
|
- **T-1.4 Flux + SOPS bootstrap — Option A (decided):** start the host Gitea (`bootstrap/gitea/`),
|
|
push this repo, `flux install`, apply the `flux-system` GitRepository + age Secret — but **keep the
|
|
`platform` and `apps` Kustomizations SUSPENDED**. This proves Flux reconciles from the host Gitea
|
|
(Phase-1 exit) WITHOUT unleashing the incomplete Phase-0 platform stubs on the working cluster.
|
|
- **T-1.5** destroy/rebuild proof (`tofu destroy` a node → `apply` → re-join), then flip
|
|
`CLUSTER_AVAILABLE=true` so conformance L2/L3 runs.
|
|
|
|
## Phase 2 prerequisite (before un-suspending `platform/`)
|
|
The Phase-0 `platform/` HelmReleases are stubs. **Do NOT let Flux reconcile them onto this cluster
|
|
until:** `platform/cilium` `k8sServiceHost` is set to **10.73.0.11** (parity with the Ansible install,
|
|
docs/02) and the REPLACE-ME values / secrets (Cloudflare token T-2.4p, etc.) are filled — otherwise
|
|
Flux would `helm upgrade` Cilium with a broken value and could break the running CNI.
|