jakman85andClaude Opus 4.8
3a59ef1918
VMs did not boot on the dev cluster. Three distinct live-verified causes, all fixed GitOps-side: 1. containerDisk root failed at OCI init: exec "/container-disk-binary/usr/bin/container-disk": no such file. KubeVirt v1.9 supplies that binary to the volumerootdisk init container via a k8s-native `image` volume (VolumeSource.Image), gated by the ALPHA `ImageVolume` feature gate. k3s v1.31 has it DISABLED, so the apiserver silently prunes the image volume + its mount -> binary missing -> crash. (Proved: a server-side dry-run of a pod with an `image` volume drops it.) Fix: convert the VM roots in all three tests to CDI DataVolumes imported onto Longhorn PVCs (mounted disks bypass the container-disk/image-volume path). This is also the spec-required disk model (docs/06, R-STATE-01). vm-lifecycle now PASSES green live (VMI Running on start + restart). 2. Guest dropped to emergency mode: the containerdisks/fedora:41 image is UEFI-native (ESP + /boot mounted via systemd GPT auto-discovery); under KubeVirt's SeaBIOS default the ESP+/boot by-uuid mounts time out -> local-fs.target fails -> cloud-init runcmd/poweroff never runs. Fix: boot the VMs with EFI (OVMF, secureBoot:false) in all three tests and fixtures/vm.yaml. Confirmed live: emergency mode gone with EFI. 3. VMs ran under TCG software emulation (guests ~10x slow, saturating nodes until one went NotReady and VMs were evicted). Cause: useEmulation:true in the KubeVirt CR makes virt-controller NOT request devices.kubevirt.io/ kvm, so /dev/kvm is never mounted and qemu falls back to TCG even though all nodes advertise kvm=1k / hypervisor=true. (Proved: `ls /dev/kvm` in a virt-launcher compute container = "No such file"; qemu accel = tcg.) Fix: useEmulation:false so the kvm device is requested and VMs run on real (nested) KVM. Requires Flux reconcile to take effect live (PM action). The virt-handler clientcertificates tls.crt error is UNRELATED: node-labeller (294 labels, hypervisor=true), device plugins, and VMI scheduling all work. Also raised test exec timeouts to 30m (CDI import + boot) and, in vm-default-deny, import the root BEFORE applying the default-deny NetworkPolicy so the CDI importer's egress to quay.io is not severed by the policy. Co-Authored-By: Claude Opus 4.8 <[email protected]>
platform/kubevirt
KubeVirt + CDI — run application VMs (IaaS) as Kubernetes resources on the same control plane, network, storage, and GitOps flow as containers.
- Governing ADR: ADR-0005 — VM workloads: KubeVirt + CDI
- Spec: docs/06 — VM Workloads (KubeVirt)
- Serves: R-PLAT-03/04, R-STATE-01
- Owner: WS-VM
What this stub is
A Kustomization that will pull the pinned KubeVirt operator + CDI operator release manifests
(commented TODO — added in Phase 5), plus the KubeVirt and CDI custom resources that tune
them. VM disks land on Longhorn PVCs via CDI; VMs inherit Cilium default-deny + egress classes.
Layer distinction (docs/01): these are Layer-2 application VMs (a feature the platform offers), NOT the Layer-1 cluster-node VMs (which WS-INFRA provisions with OpenTofu/libvirt). Do not conflate them.
Cross-workstream prerequisite
Nodes must expose virtualization extensions (nested virt in dev KVMs; native on prod) — a node baseline prerequisite owned by WS-INFRA (docs/02). Coordinate before Phase 5.
TODO (later phases — Phase 5, gated on Phases 2–3)
- Add pinned operator release manifests as Kustomize bases; let Renovate maintain versions.
- Run one representative application VM via a ServiceDefinition
kind: vm; prove create/start/stop/ destroy viakubectl/virtctl, Longhorn-backed disk survives rebuild, default-deny + egress apply, guest auto-updates (Phase-5 exit gates).