From 3173254cc4e2c04f22b32a4804e38e7e272482b9 Mon Sep 17 00:00:00 2001 From: Keith Jackson Date: Mon, 14 Sep 2026 02:24:01 -0500 Subject: [PATCH] =?UTF-8?q?docs:=20UniFi=20OS=20was=20never=20the=20blocke?= =?UTF-8?q?r=20=E2=80=94=20it=20is=20installed=20and=20the=20guest=20lost?= =?UTF-8?q?=20its=20IP?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Third confident claim of mine corrected in two days, and this one I took from a stale README rather than the machine. uosserver.service starts on the current boot and the root filesystem holds 6.3 of 40 GB, while enp1s0 is up with no address and does not answer ARP — so the controller runs and nothing can reach it. Records the near-miss: proceeding on my premise would have reinstalled over a working controller, which the new install-uos guard now prevents. Also records that there is no way into the guest at all, which is why the deliverable became console access, and the cloud-init instance-id discovery that explains why an earlier unattended install could never have taken effect. Co-Authored-By: Claude Opus 5 --- DECISIONS-LOG.md | 60 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) diff --git a/DECISIONS-LOG.md b/DECISIONS-LOG.md index 951a0d1..210b760 100644 --- a/DECISIONS-LOG.md +++ b/DECISIONS-LOG.md @@ -4123,3 +4123,63 @@ so the new state would have been unprotected until 03:00. I took a one-off backu Server", 20 slots, **no whitelist**), so the server is reachable by anyone who can reach the VIP. A whitelist or `loadBalancerSourceRanges` would restrict it — but restricting could cut off off-LAN players, which is why it is not mine to decide. + +--- + +## 2026-09-14 — UniFi OS was NEVER the blocker: it is installed; the guest lost its IP + +Third correction in two days to a claim I made confidently. I told the operator UniFi OS Server had +never been installed and that installing it was their next step. **Both halves were wrong**, and I had +taken them from `apps/vms/unifi/README.md`, which was itself stale — I repeated documentation instead +of checking the machine. + +**Verified independently on live prod:** + +| claim | evidence | +|---|---| +| UOS **is** installed and running | guest console log: `Started uosserver.service — UniFi OS Server Service` and `uosserver-updater.service` | +| it is a real install, not a stub | guest filesystem `/` holds **6.3 GB of 40 GB**; bare Ubuntu plus our packages is ~2 GB | +| **the guest has NO address** | `.status.interfaces` → `enp1s0`, `linkState: up`, **no `ipAddress`**; the VMI IP column is empty | +| confirmed at L2 | `ip neigh` inside virt-launcher: `10.0.2.2 dev k6t-eth0 **FAILED**` — the guest does not answer ARP | +| VIP / policy exonerated | `/dev/tcp` from virt-launcher to the pod IP *and* to `10.0.2.2`: 11443, 8443, 8080 and **22** all RST | + +So the controller runs and nothing can reach it, because its NIC lost its DHCP address. The cause is +not determinable from outside the guest. + +**This nearly caused real damage.** Had the work proceeded on my stated premise it would have +*reinstalled UniFi OS over a working controller*. The shipped `install-uos` now bails the moment +systemd knows a `uosserver` unit — a guard that exists only because the agent checked the premise +instead of executing it. + +**Second finding: there is no way into the guest at all.** No SSH key was ever installed, the stock +`ubuntu` account is locked, and SSH cannot work by construction — KubeVirt masquerade forwards only +declared ports and 22 is not among them, which the RST above confirms. So the deliverable became +*console access*: a SOPS secret carrying `installer-url` **and `console-password`**, mounted as a +read-only NoCloud disk and re-read on every boot. + +**The discovery that makes any of this work.** NoCloud's `instance-id` is +`spec.template.spec.domain.firmware.uuid`, stamped on the VM object — cloud-init compares it against +the DMI product UUID, so **editing `userData` alone changes nothing on a second boot**: the cached +datasource is restored and `write_files`/`runcmd` are skipped entirely. That is why a previously +shipped "unattended install" could never have taken effect, and why an earlier attempt was "tried and +reverted". The UUID is now pinned in git and bumped once. (Also: the inline `userData` cap is exactly +**2048 bytes**, established by server-side dry-run — 2048 accepted, 2049 rejected — and the payload +ships at 1454, so it stayed inline.) + +Deliberately NOT done: putting the whole cloud-config behind `secretRef`. That would hide the shell in +ciphertext, beyond the reach of the L0 dollar-collapse guard, while Flux's envsubst still runs over the +decrypted resource — exactly the nextcloud-db shape from 2026-09-13. Script plaintext, operator values +in SOPS. + +**Unattended install remains unverified BY US.** Ubiquiti documents no silent flag, answer file or +environment variables; `yes y | install` is convergent community practice and is shipped as +such, labelled third-party. No Ubiquiti credential was touched and no installer was downloaded. + +**The operator's real next step is diagnosis, not installation:** set `console-password`, restart the +VMI, then `virtctl -n unifi console unifi` and find out why `enp1s0` has no address — `ip -4 a`, +`networkctl status enp1s0`, `journalctl -u systemd-networkd`, `ls /etc/netplan`. Capture state before +restarting, since a restart may silently restore DHCP and destroy the evidence. + +The README's dev-era addressing was corrected throughout (VIP `.1.54`→`192.168.2.21`, pool and Gateway, +device subnet, node name) along with several stale factual claims, since it doubles as the rebuild +runbook.