jakman85andClaude Opus 4.8
d5f8212415
The first UOS bring-up left no way into the guest (cloud-init set no credentials), so the in-guest UOS install couldn't be observed. Add: - ssh_authorized_keys (public key, safe to commit) on the default ubuntu user - qemu-guest-agent (KubeVirt AgentConnected + virtctl guest access) - tee the install-uos output to the serial console + /var/log/uos-install.log, and record the installer exit code, so the install is observable. Co-Authored-By: Claude Opus 4.8 <[email protected]>
146 lines
8.6 KiB
YAML
146 lines
8.6 KiB
YAML
---
|
|
# KubeVirt VirtualMachine — the self-hosted UniFi OS Server (UOS) controller.
|
|
#
|
|
# WHY A VM (PM directive): the classic `unifi` .deb is deprecated. UOS is Ubiquiti's current
|
|
# self-host product — a single installer that runs on Ubuntu Server and bundles UniFi OS + the
|
|
# Network application + its OWN embedded datastore inside a Podman container stack. NO separate
|
|
# MongoDB companion is needed.
|
|
#
|
|
# DELIVERY: this VM is NOT on the app "wave" track — it is reconciled by the non-gating `apps-vms`
|
|
# Flux Kustomization (clusters/base/apps-vms.yaml, wait:false) so the slow first-boot image import
|
|
# + UOS install can't stall the app waves. UniFi has no in-cluster dependents.
|
|
#
|
|
# LIVE MIGRATION: evictionStrategy: LiveMigrate + the RWX (longhorn-vm) root disk let the VM
|
|
# drain to another node without downtime (node maintenance / rebalancing).
|
|
#
|
|
# LABELS on the pod template (propagated to the virt-launcher pod):
|
|
# - kubevirt.io/vm: unifi -> Service/CNP selector target
|
|
# - app.kubernetes.io/name: unifi -> convention
|
|
# - egress-class: internet -> Cilium egress steering (default WAN path)
|
|
# - auth-mode: native -> UniFi has its OWN login UI (no Keycloak/oauth2-proxy in
|
|
# front of it). REQUIRED by Kyverno require-auth-mode-label
|
|
# for any pod in an app-namespace.
|
|
apiVersion: kubevirt.io/v1
|
|
kind: VirtualMachine
|
|
metadata:
|
|
name: unifi
|
|
namespace: unifi
|
|
labels:
|
|
app.kubernetes.io/name: unifi
|
|
spec:
|
|
running: true
|
|
template:
|
|
metadata:
|
|
labels:
|
|
kubevirt.io/vm: unifi
|
|
app.kubernetes.io/name: unifi
|
|
homelab.keithjackson.dev/egress-class: internet
|
|
homelab.keithjackson.dev/auth-mode: native
|
|
spec:
|
|
# LiveMigrate on eviction — needs the RWX migratable disk (longhorn-vm). Without a shared
|
|
# disk KubeVirt cannot migrate and eviction would hard-stop the guest.
|
|
evictionStrategy: LiveMigrate
|
|
domain:
|
|
# FLAG (verify-at-bringup): Ubuntu 24.04 amd64 cloud images are hybrid BIOS+UEFI bootable,
|
|
# so KubeVirt's default SeaBIOS SHOULD boot cleanly (unlike the Fedora containerdisk, which
|
|
# is UEFI-only and needed an explicit efi block). If the guest drops to emergency/no-boot,
|
|
# add: firmware: { bootloader: { efi: { secureBoot: false } } }
|
|
cpu:
|
|
cores: 2
|
|
memory:
|
|
guest: 4Gi # UOS min is 2Gi; 4Gi is comfortable for a home site. Bump to 8Gi
|
|
# for large device counts / heavy DPI. Keep guest == requests below.
|
|
devices:
|
|
disks:
|
|
- name: rootdisk
|
|
disk:
|
|
bus: virtio
|
|
- name: cloudinit
|
|
disk:
|
|
bus: virtio
|
|
interfaces:
|
|
- name: default
|
|
masquerade: {}
|
|
# Declaring ports on a masquerade iface (a) restricts pod->VM forwarding to exactly
|
|
# these and (b) surfaces them as containerPorts on the virt-launcher pod so the
|
|
# LoadBalancer Services + CiliumNetworkPolicy ingress can target/match them.
|
|
ports:
|
|
- { name: inform, port: 8080, protocol: TCP } # device inform / adoption
|
|
- { name: ui-legacy, port: 8443, protocol: TCP } # legacy Network UI (still served)
|
|
- { name: ui, port: 11443, protocol: TCP } # UOS web UI (modern default)
|
|
- { name: portal-http, port: 8880, protocol: TCP } # guest portal HTTP
|
|
- { name: portal-https, port: 8843, protocol: TCP } # guest portal HTTPS
|
|
- { name: throughput, port: 6789, protocol: TCP } # mobile speedtest / throughput
|
|
- { name: stun, port: 3478, protocol: UDP } # STUN (device<->controller)
|
|
- { name: discovery, port: 10001, protocol: UDP } # L2 device discovery
|
|
resources:
|
|
requests:
|
|
memory: 4Gi
|
|
networks:
|
|
- name: default
|
|
pod: {}
|
|
volumes:
|
|
- name: rootdisk
|
|
dataVolume:
|
|
name: unifi-rootdisk
|
|
- name: cloudinit
|
|
cloudInitNoCloud:
|
|
# UNATTENDED UOS INSTALL: install-uos resolves the LATEST UniFi OS Server linux-x64
|
|
# installer from Ubiquiti's PUBLIC firmware API (fw-update.ui.com/api/firmware-latest, no
|
|
# auth) — so there is no per-release URL to bump (override with UOS_URL=<href> to pin a
|
|
# build). cloud-init installs the prerequisites (podman + slirp4netns + uidmap + jq),
|
|
# downloads the installer (~840MB), and runs it NON-INTERACTIVELY (`yes |` — the installer
|
|
# reads its confirmations from stdin). The UI comes up on https://<vip>:11443 once installed.
|
|
userData: |-
|
|
#cloud-config
|
|
hostname: unifi
|
|
fqdn: unifi.home.keithjackson.dev
|
|
# Break-glass OS access on the default `ubuntu` user (public key — safe to commit) for
|
|
# ops/debugging. qemu-guest-agent (below) additionally enables `virtctl` guest access.
|
|
ssh_pwauth: false
|
|
ssh_authorized_keys:
|
|
- ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCq3JM7CD3P+OExaD4sQARLDQUf5QqDRemAO0qWA3fG7O/6ncgX0d/xN95lJ5Ei8LewlzXF5cCVe43oJ5CJRjh41igWZ38XXgOQcbAbY0lZTJlJg/SAj9Xni++JY3Jws5Nslm8qCrqt1bDsP3O3Lnp4hmBVZ+1eJy3v9HlJ2/1mVOukIob/zVXQNdOpVx1+oubNqo+UYt3l1IxqF8x+2JlHt+POaTBMqbC3JDCaStkkHN20xItpydHZ2uSamXaTx2MH+rQjrABjuD6Xxbhxhx0hG2XrVDWyAuq91aZUpuDkybcuHnpQUSHeZJCMqDCd3Q8yd/dyifDZE8QcnY5hiVHJF8VzJT/r5+em+I4LESlE5WNfq80TWG42d9gw7Yi/8xT38iDm+0nArMd/laAPpH0JGvdIyMdWcVZgfWTgcT6EZ9cIK2J4N+ijrh/V+xwzApmYyMdVsu2UjMeKrJG4VZk43XKnpfJCLJ2LmwtPMKqrI+wyge9Bav2+t+taznKP141gL6YJW3gwArzj+oktK+/fbahtBAUnjh5emLb2rv0gbAHC9cNbovEIdA3A6FDME96zAlozU4AtoCTA7SA1AqIHuAOHkqGYFBV77eriO85K0J1qo0FwuByFINI6G/JtEnNa+CjJLvh0clJonw+6+iR2vB1hwfDmLLwrE7QRT7Ho9w== [email protected]
|
|
package_update: true
|
|
package_upgrade: false # skip slow first-boot upgrade; base image is recent
|
|
packages:
|
|
- qemu-guest-agent # KubeVirt guest integration + virtctl guest access
|
|
- podman
|
|
- uidmap
|
|
- slirp4netns
|
|
- ca-certificates
|
|
- curl
|
|
- jq
|
|
- cloud-guest-utils
|
|
write_files:
|
|
# Install (or reinstall) UniFi OS Server. URL is public/no-auth, pinned per release;
|
|
# override at runtime with: sudo UOS_URL='https://fw-download.ubnt.com/...' install-uos
|
|
- path: /usr/local/sbin/install-uos
|
|
permissions: '0755'
|
|
content: |
|
|
#!/usr/bin/env bash
|
|
set -euo pipefail
|
|
# Resolve the latest UOS linux-x64 installer from Ubiquiti's public firmware API
|
|
# (no auth). Override by exporting UOS_URL=<href> to pin a specific build.
|
|
if [ -z "${UOS_URL:-}" ]; then
|
|
UOS_URL="$(curl -fsSL https://fw-update.ui.com/api/firmware-latest | jq -r '
|
|
._embedded.firmware
|
|
| map(select(.product == "unifi-os-server" and .platform == "linux-x64"))
|
|
| sort_by(.version_major, .version_minor, .version_patch)
|
|
| last._links.data.href')"
|
|
fi
|
|
[ -n "${UOS_URL}" ] && [ "${UOS_URL}" != "null" ] || { echo "could not resolve UOS installer URL"; exit 1; }
|
|
tmp="$(mktemp /tmp/uos-installer.XXXXXX)"
|
|
echo "Downloading UOS installer: ${UOS_URL}"
|
|
curl -fL --retry 5 --retry-delay 10 -o "$tmp" "$UOS_URL"
|
|
chmod +x "$tmp"
|
|
echo "Running UOS installer non-interactively"
|
|
yes | "$tmp"
|
|
echo "UOS install finished; 'uosserver status' should report running (UI :11443)"
|
|
runcmd:
|
|
- [ systemctl, enable, --now, qemu-guest-agent ]
|
|
- [ systemctl, enable, --now, podman.socket ]
|
|
# Full unattended install; tee to the serial console AND /var/log/uos-install.log for
|
|
# visibility. Non-fatal so a failed install leaves the VM up for `sudo install-uos`.
|
|
- [ 'bash', '-lc', '/usr/local/sbin/install-uos 2>&1 | tee /var/log/uos-install.log | tee /dev/console; echo "install-uos exit=${PIPESTATUS[0]}" | tee -a /var/log/uos-install.log > /dev/console' ]
|
|
final_message: "cloud-init finished; UOS install log at /var/log/uos-install.log (UI :11443)"
|