jakman85andClaude Opus 4.8
317b3f02f1
First KubeVirt VM app + first real CDI http import. UniFi OS Server (UOS) is Ubiquiti's current self-host product (the .deb is deprecated); it bundles UniFi OS + Network app + its own datastore, so no MongoDB companion. - apps/vms/unifi: CDI DataVolume (Ubuntu 24.04 cloud image -> longhorn-vm RWX/Block, live-migratable) + VirtualMachine (evictionStrategy: LiveMigrate) with cloud-init that installs podman/slirp4netns/jq and runs the UOS installer NON-INTERACTIVELY. The installer URL is resolved at boot from Ubiquiti's public firmware API (fw-update.ui.com/api/firmware-latest) — always-latest, no pinned URL to maintain (currently UOS v5.1.37). - L4 exposure: two LoadBalancer Services (TCP+UDP) sharing a pinned, L2-announced Cilium VIP 192.168.1.54 (device inform/adoption + STUN can't ride the L7 Gateway); external-dns publishes unifi.home.keithjackson.dev. UI on :11443. - default-deny + additive CNPs (controller ingress, egress, and CDI-importer egress for the image pull). - clusters/base/apps-vms.yaml: NON-GATING Flux Kustomization (wait:false, dependsOn platform-config) — VMs are a separate lifecycle from the app waves, so a slow VM import/install can't stall the app tier. Co-Authored-By: Claude Opus 4.8 <[email protected]>
77 lines
3.8 KiB
YAML
77 lines
3.8 KiB
YAML
---
|
|
# STAGING / DRAFT — see namespace.yaml header.
|
|
#
|
|
# ============================================================================================
|
|
# L4 EXPOSURE (the hard part) — why the shared HTTPS Gateway is NOT enough for UniFi.
|
|
# ============================================================================================
|
|
# The Gateway API `shared` Gateway is an L7 HTTPS terminator (host-routed :443 -> backend). It
|
|
# CANNOT express: raw TCP on non-HTTP ports (8080 inform, 6789), UDP at all (3478 STUN, 10001
|
|
# discovery), or the device-facing protocols that never speak HTTP. UniFi device ADOPTION rides
|
|
# exactly those L4 ports. So the controller is exposed as a type=LoadBalancer Service whose VIP
|
|
# is pulled from the CiliumLoadBalancerIPPool `default-pool` (192.168.1.53-.60) and ARP-announced
|
|
# on the flat LAN by the existing CiliumL2AnnouncementPolicy `default-l2` — the SAME L2/LB-IPAM
|
|
# machinery the Gateway ingress VIP already uses (platform-config/cilium/*). No new Cilium config
|
|
# is required; these Services just consume the pool.
|
|
#
|
|
# ONE VIP, TWO SERVICES: a LoadBalancer Service is single-protocol-friendly, so TCP and UDP are
|
|
# split into two Services that SHARE ONE IP via Cilium's `lbipam.cilium.io/sharing-key`. The IP is
|
|
# PINNED (lbipam.cilium.io/ips) so devices have a STABLE inform/STUN target that never moves.
|
|
# (Alternative: one mixed-protocol Service — needs the MixedProtocolLBService feature, GA on
|
|
# k3s v1.31 — but the shared-key split is the lower-risk, Cilium-native pattern.)
|
|
#
|
|
# selector kubevirt.io/vm=unifi -> the virt-launcher pod; masquerade forwards the declared ports
|
|
# into the guest. externalTrafficPolicy: Local preserves the LAN client source IP and keeps the
|
|
# VIP pinned to the node running the VM (also what L2 announcement does).
|
|
#
|
|
# FLAG: confirm 192.168.1.54 is free, inside default-pool, and reserved in tofu (the Gateway
|
|
# already holds one pool IP, typically .53). Adjust if it collides.
|
|
#
|
|
# REUSABLE PATTERN (the deferred `minecraft` :25565/L4 problem is the same shape): expose an
|
|
# L4-only workload as a Cilium LoadBalancer on a pinned, L2-announced VIP with a per-app sharing
|
|
# key; publish its A record from the Service via external-dns. minecraft = one TCP port; unifi is
|
|
# the multi-port TCP+UDP generalization.
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: unifi-l4-tcp
|
|
namespace: unifi
|
|
labels:
|
|
app.kubernetes.io/name: unifi
|
|
annotations:
|
|
lbipam.cilium.io/sharing-key: "unifi"
|
|
lbipam.cilium.io/ips: "192.168.1.54"
|
|
# external-dns (source: service) publishes A unifi.home.keithjackson.dev -> 192.168.1.54.
|
|
# This is the browser + device target; UI is https://unifi.home.keithjackson.dev:11443.
|
|
external-dns.alpha.kubernetes.io/hostname: "unifi.home.keithjackson.dev"
|
|
spec:
|
|
type: LoadBalancer
|
|
externalTrafficPolicy: Local
|
|
selector:
|
|
kubevirt.io/vm: unifi
|
|
ports:
|
|
- { name: inform, port: 8080, targetPort: 8080, protocol: TCP }
|
|
- { name: ui, port: 11443, targetPort: 11443, protocol: TCP }
|
|
- { name: ui-legacy, port: 8443, targetPort: 8443, protocol: TCP }
|
|
- { name: portal-http, port: 8880, targetPort: 8880, protocol: TCP }
|
|
- { name: portal-https, port: 8843, targetPort: 8843, protocol: TCP }
|
|
- { name: throughput, port: 6789, targetPort: 6789, protocol: TCP }
|
|
---
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: unifi-l4-udp
|
|
namespace: unifi
|
|
labels:
|
|
app.kubernetes.io/name: unifi
|
|
annotations:
|
|
lbipam.cilium.io/sharing-key: "unifi" # same key + same IP => shares the VIP with -tcp
|
|
lbipam.cilium.io/ips: "192.168.1.54"
|
|
spec:
|
|
type: LoadBalancer
|
|
externalTrafficPolicy: Local
|
|
selector:
|
|
kubevirt.io/vm: unifi
|
|
ports:
|
|
- { name: stun, port: 3478, targetPort: 3478, protocol: UDP }
|
|
- { name: discovery, port: 10001, targetPort: 10001, protocol: UDP }
|