---
# KubeVirt CR — tunes the KubeVirt install (ADR-0005). CONSUMES the kubevirt.io CRD registered by the
# operator (platform/kubevirt/kustomization.yaml), so this is applied by the `platform-config` Flux
# Kustomization (dependsOn: platform) — see platform-config/kustomization.yaml + F-MON1.
#
# VIRT MODE — use real KVM (nested) on this cluster:
#   developerConfiguration.useEmulation was previously `true` on the belief that it "does NOT force
#   emulation, KubeVirt still prefers KVM when /dev/kvm is present." That belief is EMPIRICALLY WRONG
#   on KubeVirt v1.9 (WS-VM verified live): when useEmulation=true, virt-controller does NOT add the
#   `devices.kubevirt.io/kvm` resource to the virt-launcher pod, so /dev/kvm is never mounted into the
#   compute container and qemu ALWAYS falls back to software emulation (TCG) — even on nodes that
#   advertise `devices.kubevirt.io/kvm=1k` and are fully CPU-labelled (`hypervisor=true`). Proof: on
#   the dev cluster with useEmulation=true, `ls /dev/kvm` inside a virt-launcher compute container
#   returned "No such file or directory" and the qemu accelerator was `tcg`. TCG guests are ~10x slow,
#   never finish the in-guest conformance probes in time, and saturate node CPUs (a node went NotReady
#   and its VMs were evicted -> VMI Failed).
#     * false (set here) -> virt-controller REQUIRES + requests the kvm device, /dev/kvm is mounted,
#                           and VMs run on real (nested) KVM. All three dev nodes have /dev/kvm
#                           (advertised =1k, vmx/svm present), and prod mini-PCs have native KVM, so
#                           this is safe cluster-wide. A node genuinely lacking /dev/kvm now fails LOUD
#                           (correct — we want to know, not silently run 10x-slow emulated VMs).
#     * true             -> the broken-on-1.9 behaviour above (silent TCG everywhere). Do NOT use while
#                           the nodes have KVM.
#   RUNBOOK STEP 1 remains: confirm `/dev/kvm` on a node (`virt-host-validate` / `ls -l /dev/kvm`).
apiVersion: kubevirt.io/v1
kind: KubeVirt
metadata:
  name: kubevirt
  namespace: kubevirt
  annotations:
    homelab.keithjackson.dev/adr: ADR-0005
spec:
  # LIVE MIGRATION (docs/06): GA in KubeVirt since v0.43 — on v1.9 (this install) it is enabled by
  # default and needs NO feature gate. developerConfiguration.featureGates is intentionally left
  # EMPTY: the `LiveMigration` gate no longer exists as an alpha/beta toggle, and adding it would be
  # a no-op (worse, an unknown gate name can be rejected). No spec.configuration.migrations tuning is
  # added either — the defaults (parallelMigrationsPerCluster=5, parallelOutboundMigrationsPerNode=2,
  # 64Mi/s bandwidth cap) are correct for a 3-node dev cluster; revisit only under WS-VM load tests.
  #
  # A VM live-migrates on node drain ONLY if it opts in per-VM with `evictionStrategy: LiveMigrate`
  # AND its disks are on a RWX-migratable StorageClass — `longhorn-vm` (platform/longhorn/
  # storageclasses.yaml: migratable=true, RWX/Block). We do NOT set a cluster-wide
  # spec.configuration.evictionStrategy: a blanket LiveMigrate would WEDGE drains for VMs whose disks
  # are on the RWO `longhorn` class (unmigratable). VMs without opt-in still Evict+restart on drain.
  configuration:
    developerConfiguration:
      # Use real (nested) KVM. `false` makes virt-controller REQUEST the kvm device so /dev/kvm is
      # mounted into virt-launcher (KVM acceleration). `true` is broken on v1.9 -> silent TCG. (See header.)
      useEmulation: false
    # Persistent EFI/TPM VM state (if a VM enables it) lands on the replicated Longhorn tier (ADR-0003),
    # inheriting the same durability as every other stateful volume.
    vmStateStorageClass: longhorn