Lets users kubectl into the cluster via Keycloak SSO, mapping realm groups to
cluster RBAC. The cert-based admin kubeconfig is untouched and remains a fully
independent break-glass path (works even if Keycloak/OIDC is down).
- keycloak-realm-import.yaml: new `kubernetes` PUBLIC client + PKCE (S256) — no
client secret in any kubeconfig; kubelogin runs auth-code+PKCE to localhost.
groups mapper + email/profile/roles scopes.
- k8s-oidc-rbac.yaml (NEW, platform-config): ClusterRoleBindings
oidc:platform-admins -> cluster-admin, oidc:viewers -> view.
- k3s_server role: kube-apiserver-arg OIDC flags (issuer=homelab realm,
client-id=kubernetes, username-claim=email + oidc: prefix, groups-claim=groups
+ oidc: prefix). No oidc-ca-file (nodes reach the issuer via the gateway VIP
and trust the public LE cert). Applied via the rolling Restart k3s handler.
Keycloak/RBAC land via Flux now; the API-server flags take effect on the next
`ansible-playbook playbooks/k3s.yml` rolling restart (deferred until cluster load
from the VM-import incident settles).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The UniFi VM boot-disk DataVolume (RWX + Block on longhorn-vm) failed to import:
CDI's non-root importer (UID 107, cap-drop ALL) hit
`blockdev: cannot open /dev/cdi-block-volume: Permission denied`.
Kubernetes only applies fsGroup ownership to *filesystem* volumes — it never
chowns a raw block device — and Longhorn's CSIDriver (fsGroupPolicy:
ReadWriteOnceWithFSType) skips fsGroup on RWX/Block volumes regardless, so the
device stays root:root. The CDI-documented fix is the container-runtime setting
`device_ownership_from_security_context`, exposed on k3s (>= v1.31.3+k3s1, this
cluster is v1.31.5) as `nonroot-devices: true`. It makes containerd set the
in-container device node's owner from the pod's securityContext.
Added to the Ansible-managed /etc/rancher/k3s/config.yaml alongside tls-san;
applied via the existing rolling `Restart k3s` handler (serial:1, kube-vip keeps
the API up, etcd holds 2/3). Node-scoped, security-neutral (pods stay non-root),
and preserves the longhorn-vm RWX/Block/migratable contract unchanged.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Add an HA control-plane API VIP for the k3s cluster with kube-vip, and add
the VIP + all node IPs to the kube-apiserver serving-cert SANs so TLS via the
VIP validates. Prepares the stable failover endpoint the operator fronts with
a pfSense port-forward to 192.168.1.49:6443.
roles/kube_vip + playbooks/kube-vip.yml:
- Templates a single manifest (RBAC + DaemonSet kube-vip-ds) into each server's
k3s auto-deploy dir (/var/lib/rancher/k3s/server/manifests) so it is
reproducible and independent of Flux; k3s auto-applies it.
- Control-plane ONLY (cp_enable=true, svc_enable=false): Cilium keeps ownership
of Service LoadBalancing (L2 pool .53-.60) — no ARP conflict.
- Runs on control-plane nodes (nodeSelector node-role.kubernetes.io/control-plane),
hostNetwork, NET_ADMIN/NET_RAW, image pinned ghcr.io/kube-vip/kube-vip:v0.8.9.
- vip_interface auto-detected per node (the iface holding 192.168.1.5x = eth0).
- Leader election (plndr-cp-lock) with 15/10/2s lease timings.
- This cluster runs without kube-proxy (Cilium replacement), so a hostNetwork
pod cannot reach the kubernetes ClusterIP; kube-vip is pointed at the
node-local apiserver (KUBERNETES_SERVICE_HOST=127.0.0.1:6443) so leader
election works.
roles/k3s_server tls-san:
- New k3s_tls_sans (VIP + all three node IPs + 127.0.0.1, deduped); install
args now emit one --tls-san per entry.
- Persist the SAN list to /etc/rancher/k3s/config.yaml (read by a running k3s)
and Restart k3s handler so an existing cluster regenerates its serving cert.
- k3s.yml secondaries run serial:1 so a SAN change rolls one node at a time,
keeping etcd quorum (kube-vip keeps the API reachable during each restart).
Verified live: DS 3/3 ready, VIP answers on 192.168.1.49:6443, all 3 certs and
an s_client against the VIP carry 192.168.1.49 + .50/.51/.52 + 127.0.0.1, all
nodes Ready throughout the rolling restart.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
- inventory routes SSH via 192.168.2.102:2250/2251/2252; new k3s_node_ip carries
each node's real 192.168.1.x for --node-ip / k3s_api_host / Cilium parity
- dev.sh bind-mounts ~/.config/home-k3s/ansible-vault-pass (ro) so vaulted
playbooks can run in the pinned container (ansible.cfg vault_password_file)
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Implements T-1.5 findings F1/F2 (author-first, no cluster change).
F1 (tofu): refactor libvirt node resources count -> for_each keyed by
node name so single-node -replace/destroy cannot cascade to siblings.
Add ordered-list/primary helper locals; update outputs.tf off the map;
add moved.tf (12 moved blocks). tofu plan proves a pure state-address
migration: 0 to add, 0 to change, 0 to destroy (all 12 'has moved to').
State-persist via a 0-change apply is operator-permission-gated.
F2 (ansible): k3s_server secondary fetches the join token via
delegate_to the primary (published fact as fast path), so a single-node
rejoin with --limit k3s-dev-N is self-contained and no longer depends on
a fact an earlier play had to set. Syntax-check + CI gate green.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
jakman85andClaude Opus 4.8