home-network-k3s-impl/platform/crafty

platform/crafty

Crafty Controller 4 — a web panel that creates and manages multiple Minecraft servers (Java + Bedrock) as subprocesses inside its own container. Its web UI is Keycloak-gated via a per-pod oauth2-proxy reverse-proxy sidecar (the decided Option B, see ../oauth2-proxy/README.md). Game ports are exposed un-gated on a Cilium L2 LoadBalancer (game protocols authenticate players themselves).

• Owner: WS-APP (auth wiring shared with WS-SEC)
• Flux tier: own non-gating clusters/base/crafty.yaml (dependsOn platform-config, wait:false)
• Namespace: crafty (PLATFORM namespace — gateway-access=shared, NOT app-namespace=true)

Deploy summary

Thing	Value
Image	registry.gitlab.com/crafty-controller/crafty-4:4.10.8 (current stable; bare-semver tags, no v)
Web UI	HTTPS self-signed on :8443 (reached only via the sidecar / same-pod)
Sidecar	quay.io/oauth2-proxy/oauth2-proxy:v7.8.1, reverse-proxy on :4180
Data	ONE Longhorn RWO PVC crafty-data (20Gi), mounted 5× by subPath
PVC subPaths	servers→/crafty/servers, backups→/crafty/backups, logs→/crafty/logs, import→/crafty/import, config→/crafty/app/config
Identity	runAsUser:1000 runAsGroup:0 fsGroup:0 runAsNonRoot:true (see below)
Web host	crafty.home.keithjackson.dev → shared Gateway → crafty Service :80 → sidecar :4180 → Crafty :8443
Game VIP	192.168.1.55 (Cilium L2 LoadBalancer; .53=Gateway, .54=unifi, .55=crafty)
Game ports	starter set: TCP 25565-25567 (Java), UDP 19132-19134 (Bedrock)

fsGroup / user decision (the seerr lesson, applied)

Crafty's Dockerfile does useradd -g root -M crafty (UID 1000, primary group root/GID 0), chown -R crafty:root /crafty, and makes /crafty group-writable — the OpenShift/K8s "arbitrary-UID, root-group-perms" pattern. A fresh Longhorn PVC is root:root and not group-writable, so the non-root crafty process could not create /crafty/servers/* etc. and would CrashLoop (exactly seerr's failure at UID 1000 on a root:root PVC). Fix: fsGroup: 0 makes the kubelet group-own the mounted volume to GID 0, mark it group-writable+SGID, and add GID 0 as a supplemental group; runAsGroup: 0 + runAsUser: 1000 pin the image's own crafty:root identity. Longhorn's CSIDriver fsGroupPolicy: ReadWriteOnceWithFSType applies fsGroup to this RWO filesystem volume.

HUMAN runtime check: confirm the pod writes its SQLite DB and can start a server. If the image's crafty UID is ever not 1000, adjust runAsUser (the launcher's non-root path runs directly as the current UID, so any non-root UID with GID 0 works, but 1000 matches getpwuid cleanly).

Auth = Option B (per-app oauth2-proxy reverse-proxy)

Cilium 1.16.5 cannot inject ext_authz into the shared Gateway's machine-generated Envoy config, and native Gateway-API ExternalAuth ships in no released Cilium yet (see ../oauth2-proxy/README.md). So Crafty's UI is gated by a dedicated oauth2-proxy sidecar in the Crafty pod (--reverse-proxy, --upstream=https://127.0.0.1:8443, --ssl-upstream-insecure-skip-verify=true, --allowed-groups=platform-admins, --skip-provider-button, cookie/redirect for crafty.home.keithjackson.dev). The crafty Service targets the sidecar, never Crafty's :8443, so the gate is in the data path (fail-closed: no valid platform-admins session ⇒ no upstream).

Note: Crafty ALSO has its own local admin login (it has no OIDC). After oauth2-proxy admits the user, they still see Crafty's own login page — a second, defense-in-depth factor. The operator logs in with Crafty's local admin credentials (set on first boot). This is expected, not a bug.

Keycloak client

Confidential auth-code client crafty in the homelab realm — redirect https://crafty.home.keithjackson.dev/oauth2/callback, webOrigins the same host, a groups membership mapper (bare names). Provisioned live via kcadm at authoring time (2026-08-28) and captured as the durable GitOps source of truth in ../keycloak/keycloak-realm-import.yaml (secret injected via the CRAFTY_CLIENT_SECRET placeholder).

Secret locations (SOPS/age)

Secret	File	Key
oauth2-proxy client id/secret + cookie secret	crafty-oauth2-proxy-credentials.sops.yaml (this dir)	client-id, client-secret, cookie-secret
Keycloak client secret (mirror)	../keycloak/keycloak-realm-clients.sops.yaml	crafty-client-secret

client-secret here must equal crafty-client-secret there (cross-namespace mirror — a Secret can't span namespaces). Both are age-encrypted to the repo sops-age recipient and Flux-decryptable (verified: both decrypt and the two values match).

Game-port L4 exposure

The Gateway is L7-HTTPS-only; raw TCP/UDP game ports can't ride it. service-l4.yaml exposes them as two type=LoadBalancer Services (TCP + UDP) sharing the pinned VIP 192.168.1.55 (lbipam.cilium.io/sharing-key: crafty), ARP-announced by the existing default-l2 policy. This path is NOT auth-gated — Minecraft Java (Mojang/Microsoft) and Bedrock (Xbox Live) authenticate players themselves. external-dns publishes crafty-mc.home.keithjackson.dev → 192.168.1.55.

Adding more game ports (when the operator creates servers)

The operator has not created any servers yet; the starter set covers 3 Java + 3 Bedrock slots. To add capacity:

1. In the Crafty UI, create the server and bind it to a NEW port (Java TCP or Bedrock UDP).
2. Add that port to the matching Service in service-l4.yaml (one { name, port, targetPort, protocol } line — K8s Services enumerate ports, no ranges).
3. Add it to the game-port ingress rule in networkpolicy.yaml (crafty-allow-ingress).
4. (Optional) add a containerPort in deployment.yaml for documentation.
5. Commit → Flux reconciles.

Policies

namespace.yaml sets only gateway-access=shared (platform namespace, like headlamp/homepage), so Kyverno does not auto-generate the app baseline. networkpolicy.yaml ships it explicitly (defense in depth): default-deny (DNS-only egress) + Cilium allows — Gateway→sidecar :4180, world/LAN→game ports, host→probe ports, and egress world (server-jar downloads + the OIDC path to Keycloak via the Gateway VIP). The sidecar→Crafty hop is pod-local (127.0.0.1) and not subject to network policy.

Verification

• make -C ci gate — green (hand-authored platform manifests; render-golden unaffected).
• kustomize build of platform/crafty, clusters/base, clusters/dev — clean.

Human tests (post-deploy)

1. Browser OIDC round-trip: incognito → https://crafty.home.keithjackson.dev → expect a 302 to https://keycloak.home.keithjackson.dev/realms/homelab/... (NOT the Crafty UI). Log in as a platform-admins user (MFA/OTP enforced) → land on Crafty's own login → sign in with Crafty's local admin. A non-platform-admins user → 403 (authenticated but not authorized).
2. Pod health / fsGroup: kubectl -n crafty get pod Ready 2/2; Crafty writes its DB under /crafty/app/config and /crafty/servers without EACCES.
3. Server-create + game reach: in the Crafty UI create a Java server bound to 25565; start it; connect a Minecraft client to crafty-mc.home.keithjackson.dev:25565 (or 192.168.1.55:25565) from the LAN and confirm the world loads. (Bedrock: a server on 19132/udp reached at the VIP.)