jakman85andClaude Opus 4.8
df5979fceb
The UniFi VM boot-disk DataVolume (RWX + Block on longhorn-vm) failed to import: CDI's non-root importer (UID 107, cap-drop ALL) hit `blockdev: cannot open /dev/cdi-block-volume: Permission denied`. Kubernetes only applies fsGroup ownership to *filesystem* volumes — it never chowns a raw block device — and Longhorn's CSIDriver (fsGroupPolicy: ReadWriteOnceWithFSType) skips fsGroup on RWX/Block volumes regardless, so the device stays root:root. The CDI-documented fix is the container-runtime setting `device_ownership_from_security_context`, exposed on k3s (>= v1.31.3+k3s1, this cluster is v1.31.5) as `nonroot-devices: true`. It makes containerd set the in-container device node's owner from the pod's securityContext. Added to the Ansible-managed /etc/rancher/k3s/config.yaml alongside tls-san; applied via the existing rolling `Restart k3s` handler (serial:1, kube-vip keeps the API up, etcd holds 2/3). Node-scoped, security-neutral (pods stay non-root), and preserves the longhorn-vm RWX/Block/migratable contract unchanged. Co-Authored-By: Claude Opus 4.8 <[email protected]>
50 lines
2.5 KiB
YAML
50 lines
2.5 KiB
YAML
---
|
|
# STAGING / DRAFT — see namespace.yaml header.
|
|
#
|
|
# CDI DataVolume — the UniFi VM boot disk. This is the FIRST real CDI *http* import on
|
|
# this cluster (the vm-demo/conformance path imports a containerdisk from a *registry*;
|
|
# this pulls the official Ubuntu 24.04 LTS server cloud image (qcow2) over https and CDI
|
|
# converts it onto the PVC).
|
|
#
|
|
# STANDALONE DataVolume (NOT a dataVolumeTemplate) so the disk OUTLIVES `kubectl delete vm`
|
|
# and survives destroy/rebuild (R-STATE-01) — same durability contract as vm-demo.
|
|
#
|
|
# STORAGE = longhorn-vm (the RWX/migratable StorageClass the storage workstream is adding):
|
|
# - accessModes: ReadWriteMany + volumeMode: Block => REQUIRED for KubeVirt LIVE MIGRATION
|
|
# (evictionStrategy: LiveMigrate in the VM). A ReadWriteOnce disk cannot live-migrate.
|
|
# - 60Gi: the Ubuntu cloud image virtual size is ~3.5Gi; cloud-init growpart/growroot
|
|
# expands the root fs to fill the disk on first boot. 60Gi leaves ample room for the
|
|
# UniFi datastore + OS + podman images + updates.
|
|
#
|
|
# GOTCHA / FLAG: a Block RWX DataVolume import needs CDI scratch space and the importer pod
|
|
# needs egress to cloud-images.ubuntu.com (see the CDI-import allow in 40-networkpolicy.yaml —
|
|
# the importer runs in THIS default-deny namespace, unlike the conformance test whose ephemeral
|
|
# ns has no baseline). No checksum is verified on an http source; pin a dated build for full
|
|
# reproducibility if drift matters (see README "Open questions").
|
|
#
|
|
# NODE PREREQ (Block-mode CDI): the non-root importer opens the raw block device, which k8s
|
|
# never chowns for it. The k3s nodes set `nonroot-devices: true`
|
|
# (infra/ansible/roles/k3s_server) so containerd hands the device to the pod's UID/GID —
|
|
# without it the import dies with `blockdev: cannot open /dev/cdi-block-volume: Permission denied`.
|
|
apiVersion: cdi.kubevirt.io/v1beta1
|
|
kind: DataVolume
|
|
metadata:
|
|
name: unifi-rootdisk
|
|
namespace: unifi
|
|
labels:
|
|
app.kubernetes.io/name: unifi
|
|
spec:
|
|
source:
|
|
http:
|
|
# Ubuntu 24.04 LTS (noble) server cloud image, amd64, qcow2. "releases/.../release/"
|
|
# tracks the latest 24.04 point release. Pin to a dated build dir for reproducibility.
|
|
url: "https://cloud-images.ubuntu.com/releases/noble/release/ubuntu-24.04-server-cloudimg-amd64.img"
|
|
storage:
|
|
accessModes:
|
|
- ReadWriteMany # RWX => migratable (paired with evictionStrategy: LiveMigrate)
|
|
volumeMode: Block # block mode is the migratable/perf default for VM disks
|
|
storageClassName: longhorn-vm
|
|
resources:
|
|
requests:
|
|
storage: 60Gi
|