jakman85andClaude Opus 4.8
d60e5531b1
The three KubeVirt VM conformance tests (vm-lifecycle, vm-default-deny,
vm-data-survival) carried comments asserting the ALPHA `ImageVolume` feature
gate is DISABLED "on this k3s v1.31 cluster" and that the apiserver therefore
prunes the container-disk `image` volume, breaking the containerDisk boot path.
The cluster was upgraded v1.31 -> v1.36.4. Revalidated the gate live:
- kubernetes_feature_enabled{name="ImageVolume"} = 1 (enabled by default;
stage="" — no longer ALPHA)
- a server-side `kubectl apply --dry-run=server` of a Pod with an `image`
volume RETAINS the volume + mount instead of pruning them.
So the old premise is stale: containerDisk roots would boot on 1.36.4. The
tests never asserted the gate state and never used containerDisk — they boot
from CDI DataVolume roots, which remains the correct choice because it is the
spec-required, rebuild-surviving disk model (docs/06, R-STATE-01); an ephemeral
containerDisk could never satisfy data-survival. Test logic is therefore
unchanged; only the explanatory comments are corrected (v1.31 -> v1.36.4, and
the "containerDisk is broken here" rationale marked stale with the live
evidence).
Merge gate: PASS (lint, l0, render-golden, l1 green).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
214 lines
10 KiB
YAML
214 lines
10 KiB
YAML
# =============================================================================
|
|
# L2 BEHAVIORAL — VM lifecycle (R-PLAT-03, R-PLAT-04) [RUNNABLE / self-skips]
|
|
# Catalog: docs/14 "VM lifecycle (R-PLAT-03, L2/L3)"
|
|
# Spec: docs/06-vm-workloads-kubevirt.md, ADR-0005.
|
|
#
|
|
# GIVEN a `kind: vm` workload (KubeVirt VirtualMachine),
|
|
# WHEN it is created / started / stopped / restarted / destroyed via the SAME
|
|
# Kubernetes API + kubectl surface as a container,
|
|
# THEN each transition is honoured — the VMI reaches phase=Running on start,
|
|
# disappears on stop, comes back on restart, and is gone after delete.
|
|
# (R-PLAT-03 "VMs through the K8s API"; R-PLAT-04 "create/destroy/rebuild
|
|
# on demand".) The Longhorn-disk-survives-rebuild partner proof is the L3
|
|
# vm-data-survival test; the default-deny-applies-to-the-VM proof is the L2
|
|
# vm-default-deny test — both share this VM model.
|
|
#
|
|
# RUNNABLE / SELF-SKIPPING: KubeVirt (Phase 5) may not be installed when the
|
|
# broader suite runs. This test SKIPS CLEANLY (green) until BOTH the
|
|
# virtualmachineinstances.kubevirt.io CRD and a `longhorn` StorageClass exist, then
|
|
# runs the full lifecycle the moment they do — mirroring the data-survival self-skip.
|
|
# Everything lives in the chainsaw per-test namespace and is torn down with it
|
|
# (hermetic).
|
|
#
|
|
# DISK MODEL — CDI DataVolume on Longhorn (NOT an ephemeral containerDisk):
|
|
# The SPEC requires VMs to use Longhorn-backed disks that survive rebuild (docs/06,
|
|
# R-STATE-01), so this test — like fixtures/vm.yaml and the L3 vm-data-survival partner
|
|
# — boots from a CDI DataVolume on a Longhorn PVC (CDI imports the Fedora image into it
|
|
# out-of-band via its importer pod, so the VM never touches the container-disk binary /
|
|
# image-volume path). Cost: a one-time CDI import per run (covered by the raised exec
|
|
# timeout below).
|
|
# FEATURE-GATE NOTE (revalidated 2026-08 — the original "containerDisk is broken here"
|
|
# rationale is now STALE): KubeVirt v1.9 supplies the container-disk binary to the
|
|
# `volumerootdisk` init container via a Kubernetes-native `image` volume
|
|
# (VolumeSource.Image) mounted at /container-disk-binary, gated by the `ImageVolume`
|
|
# feature gate. On the ORIGINAL k3s v1.31 cluster that ALPHA gate was DISABLED, so the
|
|
# apiserver silently PRUNED the image volume + its mount and a containerDisk VM died at
|
|
# OCI init ("/container-disk-binary/usr/bin/container-disk: no such file or directory").
|
|
# The cluster is now k3s v1.36.4, where ImageVolume is ENABLED by default
|
|
# (kubernetes_feature_enabled{name="ImageVolume"}=1; a server-side dry-run of a Pod with
|
|
# an `image` volume RETAINS the volume + mount instead of pruning them). containerDisk
|
|
# roots would therefore boot on this cluster now — but DataVolume is retained regardless,
|
|
# because only it (not an ephemeral containerDisk) is the rebuild-surviving, spec-required
|
|
# disk model.
|
|
# =============================================================================
|
|
apiVersion: chainsaw.kyverno.io/v1alpha1
|
|
kind: Test
|
|
metadata:
|
|
name: vm-lifecycle
|
|
labels:
|
|
conformance.home.k3s/R-PLAT-03: "true"
|
|
conformance.home.k3s/R-PLAT-04: "true"
|
|
annotations:
|
|
conformance.home.k3s/requirements: "R-PLAT-03, R-PLAT-04"
|
|
conformance.home.k3s/layer: "L2"
|
|
conformance.home.k3s/safety-critical: "false"
|
|
conformance.home.k3s/executable: "true"
|
|
conformance.home.k3s/intent: >-
|
|
A VM is created/started/stopped/restarted/destroyed via the K8s API; the VMI
|
|
honours each transition (Running on start, gone on delete). Skips cleanly
|
|
until the KubeVirt CRDs exist.
|
|
spec:
|
|
# CDI imports the Fedora image onto a Longhorn PVC, then the VM boots the guest —
|
|
# import + boot is well beyond the default 180s exec window.
|
|
timeouts:
|
|
exec: 30m
|
|
steps:
|
|
- name: vm-lifecycle-through-k8s-api
|
|
try:
|
|
- script:
|
|
content: |
|
|
set -euo pipefail
|
|
NS="$NAMESPACE"
|
|
|
|
# -- Gate: skip cleanly (green) until BOTH KubeVirt and longhorn are present. --
|
|
if ! kubectl get crd virtualmachineinstances.kubevirt.io >/dev/null 2>&1; then
|
|
echo "SKIP: KubeVirt CRDs absent — vm-lifecycle not applicable on this cluster yet"
|
|
echo VM_LIFECYCLE_SKIPPED_OK
|
|
exit 0
|
|
fi
|
|
if ! kubectl get storageclass longhorn >/dev/null 2>&1; then
|
|
echo "SKIP: 'longhorn' StorageClass absent — vm-lifecycle not applicable yet"
|
|
echo VM_LIFECYCLE_SKIPPED_OK
|
|
exit 0
|
|
fi
|
|
|
|
# -- GIVEN (disk): CDI imports the Fedora image onto a Longhorn PVC (standalone
|
|
# DataVolume — outlives VM start/stop/restart so the disk is reused, not re-imported). --
|
|
kubectl -n "$NS" apply -f - <<'EOF'
|
|
apiVersion: cdi.kubevirt.io/v1beta1
|
|
kind: DataVolume
|
|
metadata:
|
|
name: lc-rootdisk
|
|
spec:
|
|
source:
|
|
registry:
|
|
url: "docker://quay.io/containerdisks/fedora:41"
|
|
storage:
|
|
accessModes:
|
|
- ReadWriteOnce
|
|
storageClassName: longhorn
|
|
resources:
|
|
requests:
|
|
storage: 6Gi
|
|
EOF
|
|
# Wait for the CDI import to complete before booting (bounded poll; import pulls quay.io).
|
|
dv_phase=""
|
|
for i in $(seq 1 120); do
|
|
dv_phase="$(kubectl -n "$NS" get dv lc-rootdisk -o jsonpath='{.status.phase}' 2>/dev/null || true)"
|
|
[ "$dv_phase" = "Succeeded" ] && break
|
|
sleep 10
|
|
done
|
|
[ "$dv_phase" = "Succeeded" ] || { echo "FAIL: CDI import did not Succeed (last='$dv_phase')" >&2; exit 1; }
|
|
echo "CDI import complete: DataVolume lc-rootdisk Succeeded (Longhorn PVC)"
|
|
|
|
# -- GIVEN (VM): a stopped VirtualMachine bound to that Longhorn DataVolume disk. --
|
|
kubectl -n "$NS" apply -f - <<'EOF'
|
|
apiVersion: kubevirt.io/v1
|
|
kind: VirtualMachine
|
|
metadata:
|
|
name: lc
|
|
spec:
|
|
running: false
|
|
template:
|
|
metadata:
|
|
labels:
|
|
kubevirt.io/vm: lc
|
|
homelab.keithjackson.dev/egress-class: internet
|
|
spec:
|
|
domain:
|
|
# EFI (OVMF) boot: the containerdisks/fedora:41 image is UEFI-native; under the
|
|
# KubeVirt SeaBIOS default it boots to emergency mode (ESP + /boot mounts fail).
|
|
# Not strictly required for this test (it only asserts VMI phase transitions, not
|
|
# in-guest state) but kept consistent with the probe tests + fixtures/vm.yaml.
|
|
firmware:
|
|
bootloader:
|
|
efi:
|
|
secureBoot: false
|
|
cpu: { cores: 1 }
|
|
memory: { guest: 1024Mi }
|
|
devices:
|
|
disks:
|
|
- name: rootdisk
|
|
disk: { bus: virtio }
|
|
interfaces:
|
|
- name: default
|
|
masquerade: {}
|
|
resources:
|
|
requests:
|
|
memory: 1024Mi
|
|
networks:
|
|
- name: default
|
|
pod: {}
|
|
volumes:
|
|
- name: rootdisk
|
|
dataVolume:
|
|
name: lc-rootdisk
|
|
EOF
|
|
# No VMI should exist while running=false.
|
|
if kubectl -n "$NS" get vmi lc >/dev/null 2>&1; then
|
|
echo "FAIL: VMI exists though the VM is defined stopped" >&2; exit 1
|
|
fi
|
|
echo "created VM 'lc' (stopped)"
|
|
|
|
wait_phase() { # $1=expected phase, bounded poll
|
|
local want="$1" i phase
|
|
for i in $(seq 1 80); do
|
|
phase="$(kubectl -n "$NS" get vmi lc -o jsonpath='{.status.phase}' 2>/dev/null || true)"
|
|
[ "$phase" = "$want" ] && return 0
|
|
sleep 10
|
|
done
|
|
echo "FAIL: VMI did not reach phase=$want (last='$phase')" >&2; return 1
|
|
}
|
|
wait_gone() {
|
|
local i
|
|
for i in $(seq 1 30); do
|
|
kubectl -n "$NS" get vmi lc >/dev/null 2>&1 || return 0
|
|
sleep 5
|
|
done
|
|
echo "FAIL: VMI still present when it should be gone" >&2; return 1
|
|
}
|
|
|
|
# -- WHEN start (running=true) THEN VMI reaches Running. --
|
|
kubectl -n "$NS" patch vm lc --type merge -p '{"spec":{"running":true}}'
|
|
wait_phase Running
|
|
echo "STARTED: VMI phase=Running"
|
|
|
|
# -- WHEN stop (running=false) THEN the VMI is torn down. --
|
|
kubectl -n "$NS" patch vm lc --type merge -p '{"spec":{"running":false}}'
|
|
wait_gone
|
|
echo "STOPPED: VMI removed"
|
|
|
|
# -- WHEN restart (running=true again) THEN it comes back Running (rebuild-on-demand). --
|
|
kubectl -n "$NS" patch vm lc --type merge -p '{"spec":{"running":true}}'
|
|
wait_phase Running
|
|
echo "RESTARTED: VMI phase=Running"
|
|
|
|
# -- WHEN destroy (delete the VM) THEN both VM and VMI are gone. --
|
|
kubectl -n "$NS" delete vm lc --wait=true --timeout=300s
|
|
wait_gone
|
|
if kubectl -n "$NS" get vm lc >/dev/null 2>&1; then
|
|
echo "FAIL: VM object still present after delete" >&2; exit 1
|
|
fi
|
|
echo "DESTROYED: VM + VMI gone"
|
|
|
|
echo VM_LIFECYCLE_OK
|
|
check:
|
|
# green on a clean skip (no KubeVirt) OR a proven full lifecycle.
|
|
(contains($stdout, 'VM_LIFECYCLE_OK') || contains($stdout, 'VM_LIFECYCLE_SKIPPED_OK')): true
|
|
finally:
|
|
- script:
|
|
content: |
|
|
kubectl -n "$NAMESPACE" delete vm lc --ignore-not-found --wait=false || true
|
|
kubectl -n "$NAMESPACE" delete dv lc-rootdisk --ignore-not-found --wait=false || true
|
|
echo "cleanup best-effort; chainsaw namespace teardown finalizes"
|