diff --git a/infra/PHASE1-SCOPE.md b/infra/PHASE1-SCOPE.md index 1e36c29..b95fa3f 100644 --- a/infra/PHASE1-SCOPE.md +++ b/infra/PHASE1-SCOPE.md @@ -24,11 +24,16 @@ and the **SOPS age key** bootstrapped. Exit gates: | Present | ansible 2.20, virt-install, qemu-img, kubectl, helm, genisoimage | ok | | **Missing (prereqs)** | **OpenTofu(+libvirt provider), age, sops, flux CLI**, cloud-localds | must install before execution | -## Prerequisites to install (before execution) -- **OpenTofu** + the **`dmacvicar/libvirt`** provider — node-VM IaC. -- **age** + **sops** — the SOPS secret bootstrap (ADR-0007). -- **flux** CLI — Flux bootstrap. -- (`cloud-localds` optional — the tofu libvirt provider builds cloud-init disks natively.) +## Toolchain — pinned dev container (decided) +The prereqs are **not all Python** (OpenTofu/age/sops/flux are Go binaries), so per the operator's +rule they are delivered as a **pinned dev container**, not host installs (and not a venv). See +[`devcontainer/`](devcontainer/): OpenTofu + `dmacvicar/libvirt` deps, age, sops, flux, ansible, +kubectl, helm, libvirt-client — baked into a Fedora-44 image (matches host libvirt 12). Run with the +host libvirt socket + repo mounted: +``` +cd infra/devcontainer && ./dev.sh build && ./dev.sh +``` +This isolates the toolchain, keeps the host clean, and gives every worker the same baseline. ## Resource plan (constrained by ~22 GiB RAM / 137 GiB disk) Per node: **4 vCPU · 6 GiB RAM · 20 GiB OS disk + 40 GiB thin data disk** (Longhorn). @@ -79,11 +84,13 @@ verify reconcile → destroy/rebuild proof → `CLUSTER_AVAILABLE=true` → conf - **Nested virt (Phase 5, not now)** — verify `/sys/module/kvm_amd/parameters/nested=1` before KubeVirt. -## Operator decisions / hand-offs needed -1. **Approve VM sizing** (6 GiB / 4 vCPU × 3) given the RAM ceiling — or free more RAM for larger. -2. **Approve installing the prereqs** (OpenTofu + libvirt provider, age, sops, flux). -3. **Network**: dedicated NAT `k3s-dev` (recommended) vs reuse `default`. -4. Cloudflare token / PIA creds are **Phase 2/3** — not needed for Phase 1. +## Operator decisions / hand-offs — RESOLVED (2026-08-16) +1. **VM sizing:** ✅ **6 GiB / 4 vCPU × 3** (18 GiB; ~4 GiB host headroom). Full stack in later phases + will need trimming or more RAM. +2. **Toolchain:** ✅ **pinned dev container** (`infra/devcontainer/`) — prereqs baked in, not host- + installed. +3. **Network:** ✅ **dedicated NAT `k3s-dev` (10.73.0.0/24)**; nodes .11/.12/.13; VIPs .240–.250. +4. Cloudflare token / PIA creds remain **Phase 2/3** — not needed for Phase 1. ## Worker dispatch (on execution go) - **Infra/Cluster Engineer** — T-1.1/1.2/1.3 (tofu + cloud-init + Ansible k3s/Cilium). diff --git a/infra/devcontainer/Containerfile b/infra/devcontainer/Containerfile new file mode 100644 index 0000000..fc08f84 --- /dev/null +++ b/infra/devcontainer/Containerfile @@ -0,0 +1,45 @@ +# Reproducible dev toolbox for the home-network-k3s platform (Phase 1+). +# Bakes the NON-Python prereqs (OpenTofu, age, sops, flux) plus ansible + kube tooling +# and the libvirt client libraries into a pinned baseline image. Run it with the host +# libvirt socket mounted (see dev.sh) so OpenTofu (dmacvicar/libvirt provider) and +# Ansible can drive the local KVM cluster. Fedora base matches the host libvirt major +# version (host is fc44 / libvirt 12) for client<->daemon compatibility. +ARG FEDORA_VERSION=44 +FROM registry.fedoraproject.org/fedora:${FEDORA_VERSION} + +ARG OPENTOFU_VERSION +ARG AGE_VERSION +ARG SOPS_VERSION +ARG FLUX_VERSION +ARG KUBECTL_VERSION +ARG HELM_VERSION +ARG TARGETARCH=amd64 + +# Distro packages: libvirt client libs (required by the tofu libvirt provider + virsh), +# ansible, ssh, disk-image + cloud-init seed tooling, and common utilities. +RUN dnf -y install --setopt=install_weak_deps=False \ + libvirt-client qemu-img cloud-utils genisoimage openssh-clients \ + ansible-core python3-libvirt git make jq unzip tar gzip xz which ca-certificates curl \ + && dnf clean all + +# Pinned Go binaries from official releases. +RUN set -eux; \ + curl -fsSL "https://github.com/opentofu/opentofu/releases/download/v${OPENTOFU_VERSION}/tofu_${OPENTOFU_VERSION}_linux_${TARGETARCH}.zip" -o /tmp/tofu.zip; \ + unzip -d /usr/local/bin /tmp/tofu.zip tofu; chmod +x /usr/local/bin/tofu; \ + curl -fsSL "https://github.com/FiloSottile/age/releases/download/v${AGE_VERSION}/age-v${AGE_VERSION}-linux-${TARGETARCH}.tar.gz" | tar -xz -C /tmp; \ + install -m0755 /tmp/age/age /tmp/age/age-keygen /usr/local/bin/; \ + curl -fsSL "https://github.com/getsops/sops/releases/download/v${SOPS_VERSION}/sops-v${SOPS_VERSION}.linux.${TARGETARCH}" -o /usr/local/bin/sops; chmod +x /usr/local/bin/sops; \ + curl -fsSL "https://github.com/fluxcd/flux2/releases/download/v${FLUX_VERSION}/flux_${FLUX_VERSION}_linux_${TARGETARCH}.tar.gz" | tar -xz -C /usr/local/bin flux; \ + curl -fsSL "https://dl.k8s.io/release/v${KUBECTL_VERSION}/bin/linux/${TARGETARCH}/kubectl" -o /usr/local/bin/kubectl; chmod +x /usr/local/bin/kubectl; \ + curl -fsSL "https://get.helm.sh/helm-v${HELM_VERSION}-linux-${TARGETARCH}.tar.gz" | tar -xz -C /tmp; \ + install -m0755 "/tmp/linux-${TARGETARCH}/helm" /usr/local/bin/helm; \ + rm -rf /tmp/* + +# Record the baked toolchain for provenance. +RUN { echo "opentofu=${OPENTOFU_VERSION}"; echo "age=${AGE_VERSION}"; echo "sops=${SOPS_VERSION}"; \ + echo "flux=${FLUX_VERSION}"; echo "kubectl=${KUBECTL_VERSION}"; echo "helm=${HELM_VERSION}"; \ + } > /etc/dev-toolchain-versions + +ENV LIBVIRT_DEFAULT_URI=qemu:///system +WORKDIR /work +CMD ["/bin/bash"] diff --git a/infra/devcontainer/README.md b/infra/devcontainer/README.md new file mode 100644 index 0000000..5b4c027 --- /dev/null +++ b/infra/devcontainer/README.md @@ -0,0 +1,39 @@ +# Dev toolbox (containerized baseline) + +The Phase 1 prerequisites are **not all Python** (OpenTofu, `age`, `sops`, `flux` are Go binaries; +only Ansible is Python). Per the operator's rule, the toolchain is therefore a **pinned dev +container** baked with everything needed to drive the local KVM cluster — no host pollution, one +reproducible baseline. + +## What's inside (pinned in `versions.env`) +OpenTofu, `age` + `age-keygen`, `sops`, `flux`, `kubectl`, `helm`, `ansible-core`, `libvirt-client` +(so the tofu `dmacvicar/libvirt` provider can talk to the host libvirt), `qemu-img`, cloud-init seed +tooling (`cloud-utils`/`genisoimage`), `openssh-clients`, and common utilities. + +## Use +```bash +cd infra/devcontainer +./dev.sh build # build the image (needs network: base image + pinned binaries) +./dev.sh # build if needed + open a shell in /work (the repo) +./dev.sh run tofu -version +``` +`ENGINE=docker ./dev.sh` to use docker instead of rootless podman. + +## How it reaches the host KVM +`dev.sh` runs the container with: +- `--network host` — so Ansible can SSH to the cluster VMs on the `k3s-dev` NAT subnet + (10.73.0.0/24) and `kubectl`/`flux` can reach the API server. +- `-v /run/libvirt:/run/libvirt` + `LIBVIRT_DEFAULT_URI=qemu:///system` — so OpenTofu/virsh drive + the host libvirt daemon (the socket is world-writable; no group change needed). +- `--security-opt label=disable` — lets the container use the host libvirt socket without + relabeling it (SELinux hosts). +- `-v $REPO:/work` and `-v ~/.ssh:/root/.ssh:ro` — the repo and your SSH keys. + +## Notes +- Versions are **best-effort pins**; if a download 404s at build, bump the value in `versions.env` + (one line) to the current stable and rebuild. The image records the resolved set at + `/etc/dev-toolchain-versions`. +- Secrets never enter the image. The SOPS **age private key** is generated at bootstrap and kept + out of git (`.gitignore` guards `age.key`). +- This container is the toolchain, not the cluster. It provisions the 3 KVM nodes (T-1.1–T-1.4); + the cluster itself runs on libvirt on the host. diff --git a/infra/devcontainer/dev.sh b/infra/devcontainer/dev.sh new file mode 100755 index 0000000..edd424a --- /dev/null +++ b/infra/devcontainer/dev.sh @@ -0,0 +1,48 @@ +#!/usr/bin/env bash +# infra/devcontainer/dev.sh — build & enter the pinned dev toolbox. +# +# Mounts the host libvirt socket and the repo so OpenTofu (dmacvicar/libvirt) and Ansible +# drive the local KVM cluster from inside the container. Rootless podman by default +# (ENGINE=docker to switch). SELinux label is disabled for the run so the container can +# use the host libvirt socket without relabeling it. +# +# ./dev.sh build # build the image only +# ./dev.sh # build (if needed) + open a shell +# ./dev.sh run tofu -version # run one command in the toolbox +set -euo pipefail +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +REPO_ROOT="$(cd "${HERE}/../.." && pwd)" +IMAGE="${IMAGE:-home-k3s-dev:local}" +ENGINE="${ENGINE:-podman}" +# shellcheck disable=SC1091 +source "${HERE}/versions.env" + +build() { + "$ENGINE" build \ + --build-arg FEDORA_VERSION="$FEDORA_VERSION" \ + --build-arg OPENTOFU_VERSION="$OPENTOFU_VERSION" \ + --build-arg AGE_VERSION="$AGE_VERSION" \ + --build-arg SOPS_VERSION="$SOPS_VERSION" \ + --build-arg FLUX_VERSION="$FLUX_VERSION" \ + --build-arg KUBECTL_VERSION="$KUBECTL_VERSION" \ + --build-arg HELM_VERSION="$HELM_VERSION" \ + -t "$IMAGE" -f "${HERE}/Containerfile" "${HERE}" +} + +run() { + "$ENGINE" run --rm -it \ + --network host \ + --security-opt label=disable \ + -v /run/libvirt:/run/libvirt \ + -v "${REPO_ROOT}:/work" -w /work \ + -v "${HOME}/.ssh:/root/.ssh:ro" \ + -e LIBVIRT_DEFAULT_URI=qemu:///system \ + "$IMAGE" "$@" +} + +case "${1:-shell}" in + build) build ;; + run) shift; run "$@" ;; + shell) build; shift 2>/dev/null || true; run ;; + *) build; run "$@" ;; +esac diff --git a/infra/devcontainer/versions.env b/infra/devcontainer/versions.env new file mode 100644 index 0000000..a63e40e --- /dev/null +++ b/infra/devcontainer/versions.env @@ -0,0 +1,13 @@ +# Pinned toolchain for the home-network-k3s dev container (the reproducible baseline). +# The non-Python prereqs (OpenTofu, age, sops, flux) are Go binaries, so — per the +# operator's rule — they are baked into a container image instead of a venv. +# +# NOTE: these are best-effort pins; bump to current stable if a download 404s at build. +FEDORA_VERSION=44 +OPENTOFU_VERSION=1.9.0 +AGE_VERSION=1.2.1 +SOPS_VERSION=3.9.4 +FLUX_VERSION=2.4.0 +KUBECTL_VERSION=1.32.0 +HELM_VERSION=3.16.0 +# terraform-libvirt provider (dmacvicar/libvirt) is pinned in the tofu module (T-1.1), not here.