home-network-k3s-impl/apps/vms/unifi/README.md

# UniFi Network controller (UniFi OS Server) — VM onboarding **[STAGING / DRAFT]**

> **NOT WIRED IN.** This directory is deliberately absent from `apps/kustomization.yaml`.
> It lands **after** the storage workstream ships the **`longhorn-vm`** RWX StorageClass; the PM
> then places `unifi` in the correct apps delivery *wave*. Nothing here should be reconciled yet.

## What this is

A KubeVirt `VirtualMachine` running **UniFi OS Server (UOS)** — Ubiquiti's current self-host
product. The legacy `unifi` `.deb`/`unifi-network-server` is deprecated. UOS is a single installer
that runs on Ubuntu Server and bundles **UniFi OS + the Network application + its own embedded
datastore** inside a Podman container stack — so **no separate MongoDB companion is needed**
(this supersedes the earlier "unifi needs a Mongo container" note).

## Files

| File | Purpose |
|---|---|
| `namespace.yaml` | `unifi` namespace: app-namespace label (Kyverno + default-deny apply), `gateway-access=shared`, backup annotation |
| `10-default-deny.yaml` | R-NET-01 baseline: deny all, allow only DNS |
| `20-datavolume.yaml` | CDI **http** import of the Ubuntu 24.04 cloud image onto a `longhorn-vm` **RWX/Block** disk (60Gi). First real CDI http import on the cluster |
| `30-virtualmachine.yaml` | The VM: 2 vCPU / 4Gi, `running: true`, `evictionStrategy: LiveMigrate`, cloud-init that preps Podman + installs UOS |
| `40-networkpolicy.yaml` | Additive CNP allows: L4 ingress ports, controller world+DNS egress, and CDI-importer egress |
| `50-service-l4.yaml` | Two `type=LoadBalancer` Services (TCP+UDP) sharing **one pinned Cilium VIP** `192.168.1.54`; external-dns A record |
| `90-httproute-optional.yaml` | **Optional, disabled** L7 UI route via the shared Gateway (needs BackendTLSPolicy — see caveat) |
| `kustomization.yaml` | For `kubectl kustomize apps/unifi` validation only; excludes the optional HTTPRoute |

## UOS install method (researched)

- **Host:** Ubuntu 24.04 LTS x86-64, systemd, **Podman 4.9.3+**, `slirp4netns` 1.2+, `uidmap`. Min
  2 GB RAM / 10 GB disk (we give 4 GB / 60 GB). Podman must be at `/usr/bin/podman`.
- **Install:** download the signed installer from `fw-download.ubnt.com` (URL obtained from
  `ui.com/download/software/unifi-os-server`), `chmod +x`, run **as root**. UI then on
  **`https://<host>:11443`**.
- **Confidence:** *High* on requirements, ports, and the download→chmod→run shape (multiple
  independent 2025–2026 sources agree). *Lower* on **fully unattended** install — see risks.

### Ports (device adoption needs the L4 ones — the L7 Gateway cannot express them)

| Port | Proto | Use |
|---|---|---|
| 11443 | TCP | UOS web UI (modern default) |
| 8443 | TCP | Legacy Network UI (still served) |
| 8080 | TCP | **Device inform / adoption** |
| 8880 / 8843 | TCP | Guest portal HTTP / HTTPS |
| 6789 | TCP | Mobile throughput/speedtest |
| 3478 | UDP | **STUN** (device ⇄ controller) |
| 10001 | UDP | **L2 device discovery** |

## L4 exposure design (the hard part)

The shared Gateway is L7 HTTPS only (`:443` host-routed). It **cannot** carry raw TCP `8080`/`6789`
or any UDP (`3478`/`10001`) — exactly the ports adoption rides. So:

- **One VIP, two Services.** `unifi-l4-tcp` + `unifi-l4-udp` share a single LoadBalancer IP via
  Cilium `lbipam.cilium.io/sharing-key: unifi`, pinned to **`192.168.1.54`** (`lbipam.cilium.io/ips`).
  The VIP is pulled from the existing `default-pool` (192.168.1.53–.60) and **ARP-announced on the
  flat LAN by the existing `default-l2` CiliumL2AnnouncementPolicy** — no new Cilium config.
- `selector: kubevirt.io/vm=unifi` → the virt-launcher pod; the VM's `masquerade` iface forwards the
  declared ports into the guest. `externalTrafficPolicy: Local` preserves LAN client source IP.
- **Device-adoption story:** point devices' **inform host** at the VIP (`http://192.168.1.54:8080/inform`)
  — via the controller's *Override inform host* setting, `set-inform` on the device, or DHCP option 43.
  Because the VIP is L2-announced on the devices' own subnet (192.168.1.0/24), unicast inform + STUN
  reach it. **Limitation:** broadcast-based L2 auto-discovery (10001/udp) does **not** cross the pod
  masquerade boundary, so zero-touch "device just appears" adoption is unreliable — expect to use
  explicit inform. Devices on other VLANs must route to the VIP and be inform-pointed anyway.
- **Reusable pattern (== the deferred `minecraft` :25565 problem):** L4-only workload → Cilium
  LoadBalancer on a pinned, L2-announced VIP with a per-app sharing key, A record via external-dns.
  `minecraft` is the single-TCP-port special case of this multi-port TCP+UDP design.

**UI access:** `https://unifi.home.keithjackson.dev:11443` (A record → VIP). The port-less
`https://unifi.home.keithjackson.dev` via the Gateway is optional and needs a BackendTLSPolicy
(UOS is HTTPS-only on 11443) — see `90-httproute-optional.yaml`.

## Open questions / risks (need a PM decision or a live spike)

1. **Unattended install (HIGH).** The installer is a **signed, account-tied** URL that **rotates/
   expires** (cannot be committed) and is documented as **interactive** (no official silent flag).
   cloud-init preps the host fully and provides an `install-uos` helper + an optional baked-URL
   auto-run that pipes `yes`. **Decision needed:** accept "operator runs `install-uos` at bringup"
   vs. invest in a verified silent path. First bringup should be treated as a spike.
2. **CDI http import (MEDIUM).** First http/qcow2 CDI import on this cluster; no checksum on an http
   source. Consider pinning a dated build dir for reproducibility. Import needs the importer-egress
   CNP (included).
3. **Boot firmware (LOW).** Ubuntu cloud images are hybrid BIOS+UEFI; default SeaBIOS should boot.
   If not, add the `efi` firmware block (noted inline in `30-virtualmachine.yaml`).
4. **VIP `192.168.1.54` (LOW).** Confirm free, in-pool, and reserved in tofu (Gateway already holds
   one pool IP, typically .53). Could not verify live — no kubeconfig cache in this worktree.
5. **Longhorn RWX prerequisite (BLOCKING).** `longhorn-vm` must exist with RWX/Block + migratable
   before apply, else the DataVolume/VM stay Pending.
6. **Licensing (LOW).** UOS self-host is free; a Ubiquiti account is needed only to fetch the
   installer and for optional remote access.

## Post-merge bring-up + verification

1. **Prereq:** `longhorn-vm` StorageClass live (RWX, Block, migratable). Confirm `192.168.1.54` free.
2. PM adds `apps/unifi` to the right wave in `apps/kustomization.yaml`; reconcile.
3. Watch the CDI import: `kubectl -n unifi get dv unifi-rootdisk -w` → `Succeeded`.
4. VM starts: `kubectl -n unifi get vmi unifi` → `Running`. LB VIP: `kubectl -n unifi get svc` →
   both show `192.168.1.54`.
5. **Install UOS** (if not auto-run): get the signed URL from ui.com, then in the guest
   `sudo UOS_URL='https://fw-download.ubnt.com/...' install-uos`. Access the guest via
   `virtctl console unifi` / SSH once cloud-init adds a key.
6. Verify UI: browse `https://unifi.home.keithjackson.dev:11443`, complete first-run wizard.
7. Verify DNS: `dig unifi.home.keithjackson.dev` → `192.168.1.54`.
8. Adopt a device: set its inform to `http://192.168.1.54:8080/inform`; confirm it appears + STUN
   connects (green). Test guest portal / throughput if used.
9. Verify default-deny holds: only the declared L4 ports reachable; VM egress limited to world+DNS.
10. (Optional) Enable the L7 HTTPRoute once BackendTLSPolicy is confirmed on this Cilium version.