fix(cdi): single-replica scratch class + 40Gi UniFi disk (fix insufficient-storage import)
The UniFi Block import got past the device-permission fix but then failed ReplicaSchedulingFailure / "insufficient storage": CDI sizes scratch ~= the disk virtual size and, on the default 3-replica `longhorn` class, the ~60Gi scratch x3 (189G) on top of the disk's x3 (180G) overflowed the Longhorn tier (~125G/node). - platform/longhorn/storageclasses.yaml: NEW `longhorn-scratch` StorageClass (numberOfReplicas: 1) — CDI scratch is ephemeral/throwaway, replicating it 3x is waste; if a node dies mid-import CDI just retries. - platform/kubevirt/cdi-cr.yaml: spec.config.scratchSpaceStorageClass -> longhorn-scratch. - apps/vms/unifi/20-datavolume.yaml: boot disk 60Gi -> 40Gi (UOS min is 25Gi; 40Gi is comfortable for a home site and lighter on the replicated tier). Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
1 parent
d8101bca38
commit
7a3da96d9f
3 files changed
+28
-6
No files matched your search
@@ -12,8 +12,8 @@
|
|||||||
# STORAGE = longhorn-vm (the RWX/migratable StorageClass the storage workstream is adding):
|
# STORAGE = longhorn-vm (the RWX/migratable StorageClass the storage workstream is adding):
|
||||||
# - accessModes: ReadWriteMany + volumeMode: Block => REQUIRED for KubeVirt LIVE MIGRATION
|
# - accessModes: ReadWriteMany + volumeMode: Block => REQUIRED for KubeVirt LIVE MIGRATION
|
||||||
# (evictionStrategy: LiveMigrate in the VM). A ReadWriteOnce disk cannot live-migrate.
|
# (evictionStrategy: LiveMigrate in the VM). A ReadWriteOnce disk cannot live-migrate.
|
||||||
# - 60Gi: the Ubuntu cloud image virtual size is ~3.5Gi; cloud-init growpart/growroot
|
# - 40Gi: the Ubuntu cloud image virtual size is ~3.5Gi; cloud-init growpart/growroot
|
||||||
# expands the root fs to fill the disk on first boot. 60Gi leaves ample room for the
|
# expands the root fs to fill the disk on first boot. 40Gi (UOS min is 25Gi) leaves room for the
|
||||||
# UniFi datastore + OS + podman images + updates.
|
# UniFi datastore + OS + podman images + updates.
|
||||||
#
|
#
|
||||||
# GOTCHA / FLAG: a Block RWX DataVolume import needs CDI scratch space and the importer pod
|
# GOTCHA / FLAG: a Block RWX DataVolume import needs CDI scratch space and the importer pod
|
||||||
@@ -46,4 +46,4 @@ spec:
|
|||||||
storageClassName: longhorn-vm
|
storageClassName: longhorn-vm
|
||||||
resources:
|
resources:
|
||||||
requests:
|
requests:
|
||||||
storage: 60Gi
|
storage: 40Gi
|
||||||
@@ -18,9 +18,10 @@ spec:
|
|||||||
workload: {}
|
workload: {}
|
||||||
config:
|
config:
|
||||||
# CDI's transient scratch space (used while converting/decompressing an imported image) is carved
|
# CDI's transient scratch space (used while converting/decompressing an imported image) is carved
|
||||||
# from the replicated Longhorn tier — the same StorageClass the final VM DataVolumes bind to, so
|
# from a dedicated SINGLE-replica Longhorn class — scratch is ephemeral/throwaway, so replicating
|
||||||
# there is no second storage tier to reason about. (docs/04, ADR-0003.)
|
# it 3x is waste and, for a large VM disk, the 3x scratch on top of the 3x final disk exhausts the
|
||||||
scratchSpaceStorageClass: longhorn
|
# Longhorn tier (ReplicaSchedulingFailure). See platform/longhorn/storageclasses.yaml. (docs/04, ADR-0003.)
|
||||||
|
scratchSpaceStorageClass: longhorn-scratch
|
||||||
# Reserve headroom on the filesystem-mode PVC for qcow2 metadata so an import never wedges a volume
|
# Reserve headroom on the filesystem-mode PVC for qcow2 metadata so an import never wedges a volume
|
||||||
# at 100%. 0.055 is CDI's documented default; pinned here so it is explicit and reviewable.
|
# at 100%. 0.055 is CDI's documented default; pinned here so it is explicit and reviewable.
|
||||||
filesystemOverhead:
|
filesystemOverhead:
|
||||||
|
|||||||
@@ -75,6 +75,27 @@ parameters:
|
|||||||
# No fsType: migratable RWX volumes are Block-mode (raw device); a filesystem is never laid down
|
# No fsType: migratable RWX volumes are Block-mode (raw device); a filesystem is never laid down
|
||||||
# by Longhorn here — the guest owns the block device.
|
# by Longhorn here — the guest owns the block device.
|
||||||
---
|
---
|
||||||
|
# CDI scratch tier — SINGLE-replica, for CDI's EPHEMERAL import scratch space ONLY (ADR-0003).
|
||||||
|
#
|
||||||
|
# WHY a dedicated class: CDI carves transient scratch space (~= the imported disk's virtual size)
|
||||||
|
# while converting/decompressing an image, then throws it away when the import finishes. Replicating
|
||||||
|
# that 3x is pure waste — and for a large VM disk the 3x scratch (ON TOP of the 3x final disk) can
|
||||||
|
# exhaust the Longhorn tier and fail the import with ReplicaSchedulingFailure / "insufficient
|
||||||
|
# storage" (observed with the UniFi 60Gi import). One replica is correct for throwaway data: if the
|
||||||
|
# node dies mid-import CDI just retries. Referenced by the CDI CR (spec.config.scratchSpaceStorageClass).
|
||||||
|
apiVersion: storage.k8s.io/v1
|
||||||
|
kind: StorageClass
|
||||||
|
metadata:
|
||||||
|
name: longhorn-scratch
|
||||||
|
provisioner: driver.longhorn.io
|
||||||
|
allowVolumeExpansion: true
|
||||||
|
reclaimPolicy: Delete # scratch is ephemeral — always reclaim on PVC removal
|
||||||
|
volumeBindingMode: Immediate
|
||||||
|
parameters:
|
||||||
|
numberOfReplicas: "1" # ephemeral throwaway data — no HA replication
|
||||||
|
staleReplicaTimeout: "30"
|
||||||
|
dataLocality: "best-effort" # keep the lone replica local to the importer pod when possible
|
||||||
|
---
|
||||||
# Bulk media tier — large, NON-replicated data (Plex library, download staging, arr handoff).
|
# Bulk media tier — large, NON-replicated data (Plex library, download staging, arr handoff).
|
||||||
# docs/04 / R-STATE-04: 3x replicating multi-TB media across 3 mini-PCs is impractical, so bulk
|
# docs/04 / R-STATE-04: 3x replicating multi-TB media across 3 mini-PCs is impractical, so bulk
|
||||||
# media lives OUTSIDE the cluster and is consumed as an external NFS/SMB mount via this thin
|
# media lives OUTSIDE the cluster and is consumed as an external NFS/SMB mount via this thin
|
||||||
|
|||||||
Reference in new issue
Block a user