0 Fork 0

Files

T

jakman85andClaude Opus 4.8 017ed0fb9c Phase 1 checkpoint: status/resume doc + cluster helper + toolchain note

Cluster is UP (3/3 Ready, Cilium OK); Phase 1 paused before Flux bootstrap
(Option A: suspend platform/apps until Phase 2). Add infra/PHASE1-STATUS.md
(what's done, state file locations, reconnect steps, remaining T-1.4/1.5,
Phase-2 platform prereq), infra/cluster.sh (status/start/stop/kubeconfig), and
document that tofu+ansible run on the host (libvirt provider limitation).

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

2026-08-16 02:18:39 -05:00

3.3 KiB

Raw Blame History

Phase 1 — Execution Status & Resume Guide

Checkpoint: 2026-08-16. The dev/test cluster is built and running. Phase 1 is paused before the Flux/SOPS bootstrap (a deliberate decision — see "Remaining"). This doc is how a future session picks up.

DONE — a live 3-node HA k3s + Cilium cluster (T-1.1/1.2/1.3)

tofu apply created the k3s-dev NAT network (10.73.0.0/24) + 3 VMs (k3s-dev-1/2/3, 6 GiB/4 vCPU, 20 G OS + 40 G data, host-passthrough) — all running.
cloud-init baselined the nodes; k3s HA (embedded etcd, 3 servers) + Cilium installed by Ansible.
Live health: kubectl get nodes → 3/3 Ready; cilium status → OK; CoreDNS up.

Fixes applied during execution (committed to main)

ansible.cfg: stdout_callback yaml removed in ansible-core 2.20 → builtin default.
cilium-values: egressGateway requires bpf.masquerade: true (+ enableIPv4Masquerade) or the agent fatals — added.

Toolchain reality (important for resume)

The pinned dev container (infra/devcontainer/) holds flux/sops/age/kubectl/helm. BUT:

tofu runs on the HOST — the dmacvicar/libvirt provider's read-write connect fails inside a container (Cannot find start time for pid). The pinned binary was extracted to ~/.local/bin/tofu (OpenTofu 1.9.0). Run tofu from the host with LIBVIRT_DEFAULT_URI=qemu:///system.
ansible runs on the HOST (pre-installed ansible-core 2.20; needs native network to the nodes).

State locations (persisted on disk; gitignored)

What	Path
tofu state	`infra/tofu/terraform.tfstate`
tofu vars (ssh pubkey)	`infra/tofu/terraform.tfvars`
cluster kubeconfig	`infra/ansible/.cache/dev.kubeconfig` (API at 10.73.0.11:6443)
SSH key	`~/.ssh/home_network` (user `k3s`, NOPASSWD sudo)
VMs / network	libvirt `qemu:///system` (`virsh list`, net `k3s-dev`)

Reconnect / manage

infra/cluster.sh status        # VMs + kubectl get nodes
infra/cluster.sh start|stop    # boot/graceful-shutdown the 3 VMs (state persists on disk)
eval "$(infra/cluster.sh kubeconfig)"   # export KUBECONFIG
kubectl get nodes
ssh -i ~/.ssh/home_network [email protected]

The VMs are running at checkpoint (~18 GiB RAM). infra/cluster.sh stop frees RAM; k3s/etcd/ Cilium recover on start (data on the disks).

REMAINING Phase 1

T-1.4 Flux + SOPS bootstrap — Option A (decided): start the host Gitea (bootstrap/gitea/), push this repo, flux install, apply the flux-system GitRepository + age Secret — but keep the platform and apps Kustomizations SUSPENDED. This proves Flux reconciles from the host Gitea (Phase-1 exit) WITHOUT unleashing the incomplete Phase-0 platform stubs on the working cluster.
T-1.5 destroy/rebuild proof (tofu destroy a node → apply → re-join), then flip CLUSTER_AVAILABLE=true so conformance L2/L3 runs.

Phase 2 prerequisite (before un-suspending `platform/`)

The Phase-0 platform/ HelmReleases are stubs. Do NOT let Flux reconcile them onto this cluster until: platform/cilium k8sServiceHost is set to 10.73.0.11 (parity with the Ansible install, docs/02) and the REPLACE-ME values / secrets (Cloudflare token T-2.4p, etc.) are filled — otherwise Flux would helm upgrade Cilium with a broken value and could break the running CNI.

3.3 KiB Raw Blame History