jakman85andClaude Opus 4.8
317b3f02f1
First KubeVirt VM app + first real CDI http import. UniFi OS Server (UOS) is Ubiquiti's current self-host product (the .deb is deprecated); it bundles UniFi OS + Network app + its own datastore, so no MongoDB companion. - apps/vms/unifi: CDI DataVolume (Ubuntu 24.04 cloud image -> longhorn-vm RWX/Block, live-migratable) + VirtualMachine (evictionStrategy: LiveMigrate) with cloud-init that installs podman/slirp4netns/jq and runs the UOS installer NON-INTERACTIVELY. The installer URL is resolved at boot from Ubiquiti's public firmware API (fw-update.ui.com/api/firmware-latest) — always-latest, no pinned URL to maintain (currently UOS v5.1.37). - L4 exposure: two LoadBalancer Services (TCP+UDP) sharing a pinned, L2-announced Cilium VIP 192.168.1.54 (device inform/adoption + STUN can't ride the L7 Gateway); external-dns publishes unifi.home.keithjackson.dev. UI on :11443. - default-deny + additive CNPs (controller ingress, egress, and CDI-importer egress for the image pull). - clusters/base/apps-vms.yaml: NON-GATING Flux Kustomization (wait:false, dependsOn platform-config) — VMs are a separate lifecycle from the app waves, so a slow VM import/install can't stall the app tier. Co-Authored-By: Claude Opus 4.8 <[email protected]>
138 lines
7.2 KiB
YAML
138 lines
7.2 KiB
YAML
---
|
|
# KubeVirt VirtualMachine — the self-hosted UniFi OS Server (UOS) controller.
|
|
#
|
|
# WHY A VM (PM directive): the classic `unifi` .deb is deprecated. UOS is Ubiquiti's current
|
|
# self-host product — a single installer that runs on Ubuntu Server and bundles UniFi OS + the
|
|
# Network application + its OWN embedded datastore inside a Podman container stack. NO separate
|
|
# MongoDB companion is needed.
|
|
#
|
|
# DELIVERY: this VM is NOT on the app "wave" track — it is reconciled by the non-gating `apps-vms`
|
|
# Flux Kustomization (clusters/base/apps-vms.yaml, wait:false) so the slow first-boot image import
|
|
# + UOS install can't stall the app waves. UniFi has no in-cluster dependents.
|
|
#
|
|
# LIVE MIGRATION: evictionStrategy: LiveMigrate + the RWX (longhorn-vm) root disk let the VM
|
|
# drain to another node without downtime (node maintenance / rebalancing).
|
|
#
|
|
# LABELS on the pod template (propagated to the virt-launcher pod):
|
|
# - kubevirt.io/vm: unifi -> Service/CNP selector target
|
|
# - app.kubernetes.io/name: unifi -> convention
|
|
# - egress-class: internet -> Cilium egress steering (default WAN path)
|
|
# - auth-mode: native -> UniFi has its OWN login UI (no Keycloak/oauth2-proxy in
|
|
# front of it). REQUIRED by Kyverno require-auth-mode-label
|
|
# for any pod in an app-namespace.
|
|
apiVersion: kubevirt.io/v1
|
|
kind: VirtualMachine
|
|
metadata:
|
|
name: unifi
|
|
namespace: unifi
|
|
labels:
|
|
app.kubernetes.io/name: unifi
|
|
spec:
|
|
running: true
|
|
template:
|
|
metadata:
|
|
labels:
|
|
kubevirt.io/vm: unifi
|
|
app.kubernetes.io/name: unifi
|
|
homelab.keithjackson.dev/egress-class: internet
|
|
homelab.keithjackson.dev/auth-mode: native
|
|
spec:
|
|
# LiveMigrate on eviction — needs the RWX migratable disk (longhorn-vm). Without a shared
|
|
# disk KubeVirt cannot migrate and eviction would hard-stop the guest.
|
|
evictionStrategy: LiveMigrate
|
|
domain:
|
|
# FLAG (verify-at-bringup): Ubuntu 24.04 amd64 cloud images are hybrid BIOS+UEFI bootable,
|
|
# so KubeVirt's default SeaBIOS SHOULD boot cleanly (unlike the Fedora containerdisk, which
|
|
# is UEFI-only and needed an explicit efi block). If the guest drops to emergency/no-boot,
|
|
# add: firmware: { bootloader: { efi: { secureBoot: false } } }
|
|
cpu:
|
|
cores: 2
|
|
memory:
|
|
guest: 4Gi # UOS min is 2Gi; 4Gi is comfortable for a home site. Bump to 8Gi
|
|
# for large device counts / heavy DPI. Keep guest == requests below.
|
|
devices:
|
|
disks:
|
|
- name: rootdisk
|
|
disk:
|
|
bus: virtio
|
|
- name: cloudinit
|
|
disk:
|
|
bus: virtio
|
|
interfaces:
|
|
- name: default
|
|
masquerade: {}
|
|
# Declaring ports on a masquerade iface (a) restricts pod->VM forwarding to exactly
|
|
# these and (b) surfaces them as containerPorts on the virt-launcher pod so the
|
|
# LoadBalancer Services + CiliumNetworkPolicy ingress can target/match them.
|
|
ports:
|
|
- { name: inform, port: 8080, protocol: TCP } # device inform / adoption
|
|
- { name: ui-legacy, port: 8443, protocol: TCP } # legacy Network UI (still served)
|
|
- { name: ui, port: 11443, protocol: TCP } # UOS web UI (modern default)
|
|
- { name: portal-http, port: 8880, protocol: TCP } # guest portal HTTP
|
|
- { name: portal-https, port: 8843, protocol: TCP } # guest portal HTTPS
|
|
- { name: throughput, port: 6789, protocol: TCP } # mobile speedtest / throughput
|
|
- { name: stun, port: 3478, protocol: UDP } # STUN (device<->controller)
|
|
- { name: discovery, port: 10001, protocol: UDP } # L2 device discovery
|
|
resources:
|
|
requests:
|
|
memory: 4Gi
|
|
networks:
|
|
- name: default
|
|
pod: {}
|
|
volumes:
|
|
- name: rootdisk
|
|
dataVolume:
|
|
name: unifi-rootdisk
|
|
- name: cloudinit
|
|
cloudInitNoCloud:
|
|
# UNATTENDED UOS INSTALL: install-uos resolves the LATEST UniFi OS Server linux-x64
|
|
# installer from Ubiquiti's PUBLIC firmware API (fw-update.ui.com/api/firmware-latest, no
|
|
# auth) — so there is no per-release URL to bump (override with UOS_URL=<href> to pin a
|
|
# build). cloud-init installs the prerequisites (podman + slirp4netns + uidmap + jq),
|
|
# downloads the installer (~840MB), and runs it NON-INTERACTIVELY (`yes |` — the installer
|
|
# reads its confirmations from stdin). The UI comes up on https://<vip>:11443 once installed.
|
|
userData: |-
|
|
#cloud-config
|
|
hostname: unifi
|
|
fqdn: unifi.home.keithjackson.dev
|
|
package_update: true
|
|
package_upgrade: false # skip slow first-boot upgrade; base image is recent
|
|
packages:
|
|
- podman
|
|
- uidmap
|
|
- slirp4netns
|
|
- ca-certificates
|
|
- curl
|
|
- jq
|
|
- cloud-guest-utils
|
|
write_files:
|
|
# Install (or reinstall) UniFi OS Server. URL is public/no-auth, pinned per release;
|
|
# override at runtime with: sudo UOS_URL='https://fw-download.ubnt.com/...' install-uos
|
|
- path: /usr/local/sbin/install-uos
|
|
permissions: '0755'
|
|
content: |
|
|
#!/usr/bin/env bash
|
|
set -euo pipefail
|
|
# Resolve the latest UOS linux-x64 installer from Ubiquiti's public firmware API
|
|
# (no auth). Override by exporting UOS_URL=<href> to pin a specific build.
|
|
if [ -z "${UOS_URL:-}" ]; then
|
|
UOS_URL="$(curl -fsSL https://fw-update.ui.com/api/firmware-latest | jq -r '
|
|
._embedded.firmware
|
|
| map(select(.product == "unifi-os-server" and .platform == "linux-x64"))
|
|
| sort_by(.version_major, .version_minor, .version_patch)
|
|
| last._links.data.href')"
|
|
fi
|
|
[ -n "${UOS_URL}" ] && [ "${UOS_URL}" != "null" ] || { echo "could not resolve UOS installer URL"; exit 1; }
|
|
tmp="$(mktemp /tmp/uos-installer.XXXXXX)"
|
|
echo "Downloading UOS installer: ${UOS_URL}"
|
|
curl -fL --retry 5 --retry-delay 10 -o "$tmp" "$UOS_URL"
|
|
chmod +x "$tmp"
|
|
echo "Running UOS installer non-interactively"
|
|
yes | "$tmp"
|
|
echo "UOS install finished; 'uosserver status' should report running (UI :11443)"
|
|
runcmd:
|
|
- [ systemctl, enable, --now, podman.socket ]
|
|
# Full unattended install; logs to /var/log/uos-install.log. Non-fatal so a failed
|
|
# install leaves the VM up for `sudo install-uos` via console (KubeVirt virtctl).
|
|
- [ 'bash', '-lc', '/usr/local/sbin/install-uos > /var/log/uos-install.log 2>&1 || echo "UOS install failed; see /var/log/uos-install.log then rerun: sudo install-uos"' ]
|