Phase 1 toolchain: pinned dev container (prereqs not all Python)

Per operator rule: OpenTofu/age/sops/flux are Go binaries, so bake the toolchain into a pinned Fedora-44 dev container (matches host libvirt 12) instead of a venv. infra/devcontainer/: Containerfile + versions.env (tofu 1.9.0, age 1.2.1, sops 3.9.4, flux 2.4.0, kubectl 1.32.0, helm 3.16.0) + dev.sh (runs with host libvirt socket + repo + ssh mounted) + README. Built & validated: all tools resolve; virsh reaches host libvirt from inside the container. Also record resolved Phase 1 decisions in PHASE1-SCOPE.md (6GiB/4vCPU x3; k3s-dev NAT 10.73.0.0/24; container toolchain). Gate green. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-08-16 00:59:47 -05:00
1 parent 814029f2da
commit 5d2b248b68
@@ -24,11 +24,16 @@ and the **SOPS age key** bootstrapped. Exit gates:
 | Present | ansible 2.20, virt-install, qemu-img, kubectl, helm, genisoimage | ok |
 | **Missing (prereqs)** | **OpenTofu(+libvirt provider), age, sops, flux CLI**, cloud-localds | must install before execution |

-## Prerequisites to install (before execution)
- **OpenTofu** + the **`dmacvicar/libvirt`** provider — node-VM IaC.
- **age** + **sops** — the SOPS secret bootstrap (ADR-0007).
- **flux** CLI — Flux bootstrap.
- (`cloud-localds` optional — the tofu libvirt provider builds cloud-init disks natively.)
+## Toolchain — pinned dev container (decided)
+The prereqs are **not all Python** (OpenTofu/age/sops/flux are Go binaries), so per the operator's
+rule they are delivered as a **pinned dev container**, not host installs (and not a venv). See
+[`devcontainer/`](devcontainer/): OpenTofu + `dmacvicar/libvirt` deps, age, sops, flux, ansible,
+kubectl, helm, libvirt-client — baked into a Fedora-44 image (matches host libvirt 12). Run with the
+host libvirt socket + repo mounted:
+```
+cd infra/devcontainer && ./dev.sh build && ./dev.sh
+```
+This isolates the toolchain, keeps the host clean, and gives every worker the same baseline.

 ## Resource plan (constrained by ~22 GiB RAM / 137 GiB disk)
 Per node: **4 vCPU · 6 GiB RAM · 20 GiB OS disk + 40 GiB thin data disk** (Longhorn).
@@ -79,11 +84,13 @@ verify reconcile → destroy/rebuild proof → `CLUSTER_AVAILABLE=true` → conf
 - **Nested virt (Phase 5, not now)** — verify `/sys/module/kvm_amd/parameters/nested=1` before
  KubeVirt.

-## Operator decisions / hand-offs needed
-1. **Approve VM sizing** (6 GiB / 4 vCPU × 3) given the RAM ceiling — or free more RAM for larger.
-2. **Approve installing the prereqs** (OpenTofu + libvirt provider, age, sops, flux).
-3. **Network**: dedicated NAT `k3s-dev` (recommended) vs reuse `default`.
-4. Cloudflare token / PIA creds are **Phase 2/3** — not needed for Phase 1.
+## Operator decisions / hand-offs — RESOLVED (2026-08-16)
+1. **VM sizing:** ✅ **6 GiB / 4 vCPU × 3** (18 GiB; ~4 GiB host headroom). Full stack in later phases
+   will need trimming or more RAM.
+2. **Toolchain:** ✅ **pinned dev container** (`infra/devcontainer/`) — prereqs baked in, not host-
+   installed.
+3. **Network:** ✅ **dedicated NAT `k3s-dev` (10.73.0.0/24)**; nodes .11/.12/.13; VIPs .240–.250.
+4. Cloudflare token / PIA creds remain **Phase 2/3** — not needed for Phase 1.

 ## Worker dispatch (on execution go)
 - **Infra/Cluster Engineer** — T-1.1/1.2/1.3 (tofu + cloud-init + Ansible k3s/Cilium).
@@ -0,0 +1,45 @@
+# Reproducible dev toolbox for the home-network-k3s platform (Phase 1+).
+# Bakes the NON-Python prereqs (OpenTofu, age, sops, flux) plus ansible + kube tooling
+# and the libvirt client libraries into a pinned baseline image. Run it with the host
+# libvirt socket mounted (see dev.sh) so OpenTofu (dmacvicar/libvirt provider) and
+# Ansible can drive the local KVM cluster. Fedora base matches the host libvirt major
+# version (host is fc44 / libvirt 12) for client<->daemon compatibility.
+ARG FEDORA_VERSION=44
+FROM registry.fedoraproject.org/fedora:${FEDORA_VERSION}
+
+ARG OPENTOFU_VERSION
+ARG AGE_VERSION
+ARG SOPS_VERSION
+ARG FLUX_VERSION
+ARG KUBECTL_VERSION
+ARG HELM_VERSION
+ARG TARGETARCH=amd64
+
+# Distro packages: libvirt client libs (required by the tofu libvirt provider + virsh),
+# ansible, ssh, disk-image + cloud-init seed tooling, and common utilities.
+RUN dnf -y install --setopt=install_weak_deps=False \
+      libvirt-client qemu-img cloud-utils genisoimage openssh-clients \
+      ansible-core python3-libvirt git make jq unzip tar gzip xz which ca-certificates curl \
+ && dnf clean all
+
+# Pinned Go binaries from official releases.
+RUN set -eux; \
+    curl -fsSL "https://github.com/opentofu/opentofu/releases/download/v${OPENTOFU_VERSION}/tofu_${OPENTOFU_VERSION}_linux_${TARGETARCH}.zip" -o /tmp/tofu.zip; \
+      unzip -d /usr/local/bin /tmp/tofu.zip tofu; chmod +x /usr/local/bin/tofu; \
+    curl -fsSL "https://github.com/FiloSottile/age/releases/download/v${AGE_VERSION}/age-v${AGE_VERSION}-linux-${TARGETARCH}.tar.gz" | tar -xz -C /tmp; \
+      install -m0755 /tmp/age/age /tmp/age/age-keygen /usr/local/bin/; \
+    curl -fsSL "https://github.com/getsops/sops/releases/download/v${SOPS_VERSION}/sops-v${SOPS_VERSION}.linux.${TARGETARCH}" -o /usr/local/bin/sops; chmod +x /usr/local/bin/sops; \
+    curl -fsSL "https://github.com/fluxcd/flux2/releases/download/v${FLUX_VERSION}/flux_${FLUX_VERSION}_linux_${TARGETARCH}.tar.gz" | tar -xz -C /usr/local/bin flux; \
+    curl -fsSL "https://dl.k8s.io/release/v${KUBECTL_VERSION}/bin/linux/${TARGETARCH}/kubectl" -o /usr/local/bin/kubectl; chmod +x /usr/local/bin/kubectl; \
+    curl -fsSL "https://get.helm.sh/helm-v${HELM_VERSION}-linux-${TARGETARCH}.tar.gz" | tar -xz -C /tmp; \
+      install -m0755 "/tmp/linux-${TARGETARCH}/helm" /usr/local/bin/helm; \
+    rm -rf /tmp/*
+
+# Record the baked toolchain for provenance.
+RUN { echo "opentofu=${OPENTOFU_VERSION}"; echo "age=${AGE_VERSION}"; echo "sops=${SOPS_VERSION}"; \
+      echo "flux=${FLUX_VERSION}"; echo "kubectl=${KUBECTL_VERSION}"; echo "helm=${HELM_VERSION}"; \
+    } > /etc/dev-toolchain-versions
+
+ENV LIBVIRT_DEFAULT_URI=qemu:///system
+WORKDIR /work
+CMD ["/bin/bash"]
@@ -0,0 +1,39 @@
+# Dev toolbox (containerized baseline)
+
+The Phase 1 prerequisites are **not all Python** (OpenTofu, `age`, `sops`, `flux` are Go binaries;
+only Ansible is Python). Per the operator's rule, the toolchain is therefore a **pinned dev
+container** baked with everything needed to drive the local KVM cluster — no host pollution, one
+reproducible baseline.
+
+## What's inside (pinned in `versions.env`)
+OpenTofu, `age` + `age-keygen`, `sops`, `flux`, `kubectl`, `helm`, `ansible-core`, `libvirt-client`
+(so the tofu `dmacvicar/libvirt` provider can talk to the host libvirt), `qemu-img`, cloud-init seed
+tooling (`cloud-utils`/`genisoimage`), `openssh-clients`, and common utilities.
+
+## Use
+```bash
+cd infra/devcontainer
+./dev.sh build        # build the image (needs network: base image + pinned binaries)
+./dev.sh              # build if needed + open a shell in /work (the repo)
+./dev.sh run tofu -version
+```
+`ENGINE=docker ./dev.sh` to use docker instead of rootless podman.
+
+## How it reaches the host KVM
+`dev.sh` runs the container with:
+- `--network host` — so Ansible can SSH to the cluster VMs on the `k3s-dev` NAT subnet
+  (10.73.0.0/24) and `kubectl`/`flux` can reach the API server.
+- `-v /run/libvirt:/run/libvirt` + `LIBVIRT_DEFAULT_URI=qemu:///system` — so OpenTofu/virsh drive
+  the host libvirt daemon (the socket is world-writable; no group change needed).
+- `--security-opt label=disable` — lets the container use the host libvirt socket without
+  relabeling it (SELinux hosts).
+- `-v $REPO:/work` and `-v ~/.ssh:/root/.ssh:ro` — the repo and your SSH keys.
+
+## Notes
+- Versions are **best-effort pins**; if a download 404s at build, bump the value in `versions.env`
+  (one line) to the current stable and rebuild. The image records the resolved set at
+  `/etc/dev-toolchain-versions`.
+- Secrets never enter the image. The SOPS **age private key** is generated at bootstrap and kept
+  out of git (`.gitignore` guards `age.key`).
+- This container is the toolchain, not the cluster. It provisions the 3 KVM nodes (T-1.1–T-1.4);
+  the cluster itself runs on libvirt on the host.
@@ -0,0 +1,48 @@
+#!/usr/bin/env bash
+# infra/devcontainer/dev.sh — build & enter the pinned dev toolbox.
+#
+# Mounts the host libvirt socket and the repo so OpenTofu (dmacvicar/libvirt) and Ansible
+# drive the local KVM cluster from inside the container. Rootless podman by default
+# (ENGINE=docker to switch). SELinux label is disabled for the run so the container can
+# use the host libvirt socket without relabeling it.
+#
+#   ./dev.sh build           # build the image only
+#   ./dev.sh                 # build (if needed) + open a shell
+#   ./dev.sh run tofu -version   # run one command in the toolbox
+set -euo pipefail
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO_ROOT="$(cd "${HERE}/../.." && pwd)"
+IMAGE="${IMAGE:-home-k3s-dev:local}"
+ENGINE="${ENGINE:-podman}"
+# shellcheck disable=SC1091
+source "${HERE}/versions.env"
+
+build() {
+  "$ENGINE" build \
+    --build-arg FEDORA_VERSION="$FEDORA_VERSION" \
+    --build-arg OPENTOFU_VERSION="$OPENTOFU_VERSION" \
+    --build-arg AGE_VERSION="$AGE_VERSION" \
+    --build-arg SOPS_VERSION="$SOPS_VERSION" \
+    --build-arg FLUX_VERSION="$FLUX_VERSION" \
+    --build-arg KUBECTL_VERSION="$KUBECTL_VERSION" \
+    --build-arg HELM_VERSION="$HELM_VERSION" \
+    -t "$IMAGE" -f "${HERE}/Containerfile" "${HERE}"
+}
+
+run() {
+  "$ENGINE" run --rm -it \
+    --network host \
+    --security-opt label=disable \
+    -v /run/libvirt:/run/libvirt \
+    -v "${REPO_ROOT}:/work" -w /work \
+    -v "${HOME}/.ssh:/root/.ssh:ro" \
+    -e LIBVIRT_DEFAULT_URI=qemu:///system \
+    "$IMAGE" "$@"
+}
+
+case "${1:-shell}" in
+  build) build ;;
+  run)   shift; run "$@" ;;
+  shell) build; shift 2>/dev/null || true; run ;;
+  *)     build; run "$@" ;;
+esac
@@ -0,0 +1,13 @@
+# Pinned toolchain for the home-network-k3s dev container (the reproducible baseline).
+# The non-Python prereqs (OpenTofu, age, sops, flux) are Go binaries, so — per the
+# operator's rule — they are baked into a container image instead of a venv.
+#
+# NOTE: these are best-effort pins; bump to current stable if a download 404s at build.
+FEDORA_VERSION=44
+OPENTOFU_VERSION=1.9.0
+AGE_VERSION=1.2.1
+SOPS_VERSION=3.9.4
+FLUX_VERSION=2.4.0
+KUBECTL_VERSION=1.32.0
+HELM_VERSION=3.16.0
+# terraform-libvirt provider (dmacvicar/libvirt) is pinned in the tofu module (T-1.1), not here.