jakman85andClaude Opus 4.8
bb48f675ee
Bake the conformance harness toolchain into infra/devcontainer so `make -C ci gate` runs identically here and in CI: - Containerfile: install kyverno CLI (1.13.4), kyverno-json, chainsaw (0.2.12), yamllint (1.35.1), check-jsonschema (0.29.4), yq (4.44.6) via new pinned RUN block; add diffutils + python3-pip; drop stale LIBVIRT_DEFAULT_URI; refresh header for the Proxmox(bpg)+conformance toolbox; extend /etc/dev-toolchain-versions provenance. - versions.env: add KYVERNO/KYVERNO_JSON/CHAINSAW/YAMLLINT/YQ/CHECK_JSONSCHEMA; align KUBECTL 1.32.0 -> 1.31.4. dev.sh build(): pass the new --build-args. - ci/versions.env: pin CHECK_JSONSCHEMA_VERSION=0.29.4 (single source of truth). Fix harness seams so the gate can run: - dev.sh run(): mount the sibling spec repo (home-network-k3s) read-only at /home-network-k3s for ci/spec-immutability.sh (R-TEST-06); conditional so the container still runs without it. - conformance/lib/common.sh conf_dirs_for_layer(): accept a comma/space list of layers and union the dirs, so ci/conformance.sh `run.sh --layer l2,l3` selects both layers in one pass (backward compatible for single layers). NOTE: ci/versions.env pins KYVERNO_JSON_VERSION=0.0.5, which has no upstream release (max tag is v0.0.3; platform/kyverno/scripts/run-l0.sh already defaults to v0.0.3). Baked v0.0.3 and flagged for PM to correct ci/versions.env. kyverno-json is not used by any static gate stage. Co-Authored-By: Claude Opus 4.8 <[email protected]>
Dev toolbox (containerized baseline)
The Phase 1 prerequisites are not all Python (OpenTofu, age, sops, flux are Go binaries;
only Ansible is Python). Per the operator's rule, the toolchain is therefore a pinned dev
container baked with everything needed to drive the local KVM cluster — no host pollution, one
reproducible baseline.
What's inside (pinned in versions.env)
OpenTofu, age + age-keygen, sops, flux, kubectl, helm, ansible-core, libvirt-client
(so the tofu dmacvicar/libvirt provider can talk to the host libvirt), qemu-img, cloud-init seed
tooling (cloud-utils/genisoimage), openssh-clients, and common utilities.
Use
cd infra/devcontainer
./dev.sh build # build the image (needs network: base image + pinned binaries)
./dev.sh # build if needed + open a shell in /work (the repo)
./dev.sh run tofu -version
ENGINE=docker ./dev.sh to use docker instead of rootless podman.
How it reaches the host KVM
dev.sh runs the container with:
--network host— so Ansible can SSH to the cluster VMs on thek3s-devNAT subnet (10.73.0.0/24) andkubectl/fluxcan reach the API server.-v /run/libvirt:/run/libvirt+LIBVIRT_DEFAULT_URI=qemu:///system— so OpenTofu/virsh drive the host libvirt daemon (the socket is world-writable; no group change needed).--security-opt label=disable— lets the container use the host libvirt socket without relabeling it (SELinux hosts).-v $REPO:/workand-v ~/.ssh:/root/.ssh:ro— the repo and your SSH keys.
Known limitation — run tofu and ansible on the HOST, not in the container
Discovered during Phase 1 bring-up (see ../PHASE1-STATUS.md):
- OpenTofu +
dmacvicar/libvirtfails inside the container — its read-writeqemu:///systemconnect hits a polkit/PID-identity path (Cannot find start time for pid) that a mounted socket can't satisfy across the PID namespace (--pid=hostdidn't help). Run the pinned tofu on the host (extract once:podman run --rm -v ~/.local/bin:/out home-k3s-dev:local cp /usr/local/bin/tofu /out/; thenLIBVIRT_DEFAULT_URI=qemu:///system ~/.local/bin/tofu ...). - Ansible talks to the cluster VMs over SSH; simplest to run the pre-installed host
ansible-corewith native networking to thek3s-devNAT subnet. - The container remains the baseline for flux/sops/age/kubectl/helm (used from inside with
--network hostonce the cluster exists).
Notes
- Versions are best-effort pins; if a download 404s at build, bump the value in
versions.env(one line) to the current stable and rebuild. The image records the resolved set at/etc/dev-toolchain-versions. - Secrets never enter the image. The SOPS age private key is generated at bootstrap and kept
out of git (
.gitignoreguardsage.key). - This container is the toolchain, not the cluster. It provisions the 3 KVM nodes (T-1.1–T-1.4); the cluster itself runs on libvirt on the host.