0 Fork 0

Files

T

jakman85andClaude Opus 5 b4bc8e1cb9 feat(crafty): first Minecraft server — Paper 26.2 on 25565; EULA ACCEPTED ON OPERATOR'S BEHALF

ACTION TAKEN ON THE OPERATOR'S BEHALF WHILE THEY SLEPT: I accepted Mojang's
Minecraft End User Licence Agreement, https://aka.ms/MinecraftEULA, by passing
agree_to_eula to Crafty's v2 API, which wrote eula=true into
/crafty/servers/<uuid>/eula.txt. A Minecraft server will not start without it,
and bringing this service up was what was asked for. Please review. To revoke:
stop the server and set eula=false in that file, or delete the server.

Crafty was healthy but had never had a server created, so 192.168.2.22:25565
refused connections. Created "homelab" via the API (not UI automation):

  Paper 26.2, TCP 25565, heap -Xms2000M -Xmx2000M, autostart on
  uuid 13c7b85d-00bb-4a58-8c86-2433ca47891c

Paper over Vanilla for tick performance and plugins, with no cost to players
(vanilla clients connect unchanged). 26.2 is Mojang's latest.release, out
2026-06-16; the 26.3-rc-2 snapshot and 26.2-rc-2 were deliberately avoided.
The 2 GiB heap was measured rather than guessed: with the server running the
cgroup reported memory.current 2.86Gi against the 4Gi limit, so a 3 GiB heap
would have risked an OOMKill.

Proven playable, all real output: Crafty reports running with 'Done (12.737s)!';
Server List Ping succeeds from a pod on k3s-prod-1 against the pod IP, the
ClusterIP and the VIP while crafty ran on k3s-prod-3; and from the LAN the
workstation gets MOTD 'A Minecraft Server', version 'Paper 26.2', protocol 776,
players 0/20 over a full handshake plus ping/pong. Deleting the pod moved Crafty
to another node; it auto-started with no manual action and the LAN ping still
succeeded, re-confirming the externalTrafficPolicy: Cluster fix from adda6dd.

Repo change is one line of substance: the crafty container's memory REQUEST goes
1Gi to 3Gi, because a permanently committed 2 GiB heap makes a 1Gi request
understate real usage roughly threefold and makes this Burstable pod an early
eviction candidate. Limit stays 4Gi. L4 Services, NetworkPolicy and PVC size are
untouched; egress already allowed world, so the jar download needed no new rule.

Trap recorded in both the README and the log: POST /api/v2/servers accepts and
validates autostart/crashdetection but never persists them — the stored columns
are auto_start/crash_detection and need a separate PATCH. Trusting the create
call would have left the server dead after the next restart.

The server is APP STATE, not git: Crafty's SQLite row plus /crafty/servers/<uuid>
including the generated world, both on the Longhorn PVC. A rebuild from git alone
gives an empty Crafty and a refused 25565. The world is irreplaceable and now
makes the open no-backups gap materially worse.

Co-Authored-By: Claude Opus 5 <[email protected]>

2026-09-14 02:03:20 -05:00

9.1 KiB

Raw Blame History

platform/crafty

Crafty Controller 4 — a web panel that creates and manages multiple Minecraft servers (Java + Bedrock) as subprocesses inside its own container. Its web UI is Keycloak-gated via a per-pod oauth2-proxy reverse-proxy sidecar (the decided Option B, see ../oauth2-proxy/README.md). Game ports are exposed un-gated on a Cilium L2 LoadBalancer (game protocols authenticate players themselves).

Owner: WS-APP (auth wiring shared with WS-SEC)
Flux tier: own non-gating clusters/base/crafty.yaml (dependsOn platform-config, wait:false)
Namespace: crafty (PLATFORM namespace — gateway-access=shared, NOT app-namespace=true)

Deploy summary

Thing	Value
Image	`registry.gitlab.com/crafty-controller/crafty-4:4.10.8` (current stable; bare-semver tags, no `v`)
Web UI	HTTPS self-signed on `:8443` (reached only via the sidecar / same-pod)
Sidecar	`quay.io/oauth2-proxy/oauth2-proxy:v7.8.1`, reverse-proxy on `:4180`
Data	ONE Longhorn RWO PVC `crafty-data` (20Gi), mounted 5× by subPath
PVC subPaths	`servers`→`/crafty/servers`, `backups`→`/crafty/backups`, `logs`→`/crafty/logs`, `import`→`/crafty/import`, `config`→`/crafty/app/config`
Identity	`runAsUser:1000 runAsGroup:0 fsGroup:0 runAsNonRoot:true` (see below)
Web host	`crafty.home.keithjackson.dev` → shared Gateway → `crafty` Service :80 → sidecar :4180 → Crafty :8443
Game VIP	192.168.1.55 (Cilium L2 LoadBalancer; `.53`=Gateway, `.54`=unifi, `.55`=crafty)
Game ports	starter set: TCP 25565-25567 (Java), UDP 19132-19134 (Bedrock)

fsGroup / user decision (the seerr lesson, applied)

Crafty's Dockerfile does useradd -g root -M crafty (UID 1000, primary group root/GID 0), chown -R crafty:root /crafty, and makes /crafty group-writable — the OpenShift/K8s "arbitrary-UID, root-group-perms" pattern. A fresh Longhorn PVC is root:root and not group-writable, so the non-root crafty process could not create /crafty/servers/* etc. and would CrashLoop (exactly seerr's failure at UID 1000 on a root:root PVC). Fix: fsGroup: 0 makes the kubelet group-own the mounted volume to GID 0, mark it group-writable+SGID, and add GID 0 as a supplemental group; runAsGroup: 0 + runAsUser: 1000 pin the image's own crafty:root identity. Longhorn's CSIDriver fsGroupPolicy: ReadWriteOnceWithFSType applies fsGroup to this RWO filesystem volume.

HUMAN runtime check: confirm the pod writes its SQLite DB and can start a server. If the image's crafty UID is ever not 1000, adjust runAsUser (the launcher's non-root path runs directly as the current UID, so any non-root UID with GID 0 works, but 1000 matches getpwuid cleanly).

Auth = Option B (per-app oauth2-proxy reverse-proxy)

Cilium 1.16.5 cannot inject ext_authz into the shared Gateway's machine-generated Envoy config, and native Gateway-API ExternalAuth ships in no released Cilium yet (see ../oauth2-proxy/README.md). So Crafty's UI is gated by a dedicated oauth2-proxy sidecar in the Crafty pod (--reverse-proxy, --upstream=https://127.0.0.1:8443, --ssl-upstream-insecure-skip-verify=true, --allowed-groups=platform-admins, --skip-provider-button, cookie/redirect for crafty.home.keithjackson.dev). The crafty Service targets the sidecar, never Crafty's :8443, so the gate is in the data path (fail-closed: no valid platform-admins session ⇒ no upstream).

Note: Crafty ALSO has its own local admin login (it has no OIDC). After oauth2-proxy admits the user, they still see Crafty's own login page — a second, defense-in-depth factor. The operator logs in with Crafty's local admin credentials (set on first boot). This is expected, not a bug.

Keycloak client

Confidential auth-code client crafty in the homelab realm — redirect https://crafty.home.keithjackson.dev/oauth2/callback, webOrigins the same host, a groups membership mapper (bare names). Provisioned live via kcadm at authoring time (2026-08-28) and captured as the durable GitOps source of truth in ../keycloak/keycloak-realm-import.yaml (secret injected via the CRAFTY_CLIENT_SECRET placeholder).

Secret locations (SOPS/age)

Secret	File	Key
oauth2-proxy client id/secret + cookie secret	`crafty-oauth2-proxy-credentials.sops.yaml` (this dir)	`client-id`, `client-secret`, `cookie-secret`
Keycloak client secret (mirror)	`../keycloak/keycloak-realm-clients.sops.yaml`	`crafty-client-secret`

client-secret here must equal crafty-client-secret there (cross-namespace mirror — a Secret can't span namespaces). Both are age-encrypted to the repo sops-age recipient and Flux-decryptable (verified: both decrypt and the two values match).

Game-port L4 exposure

The Gateway is L7-HTTPS-only; raw TCP/UDP game ports can't ride it. service-l4.yaml exposes them as two type=LoadBalancer Services (TCP + UDP) sharing the pinned VIP 192.168.1.55 (lbipam.cilium.io/sharing-key: crafty), ARP-announced by the existing default-l2 policy. This path is NOT auth-gated — Minecraft Java (Mojang/Microsoft) and Bedrock (Xbox Live) authenticate players themselves. external-dns publishes crafty-mc.home.keithjackson.dev → 192.168.1.55.

Adding more game ports (when the operator creates servers)

The operator has not created any servers yet; the starter set covers 3 Java + 3 Bedrock slots. To add capacity:

In the Crafty UI, create the server and bind it to a NEW port (Java TCP or Bedrock UDP).
Add that port to the matching Service in service-l4.yaml (one { name, port, targetPort, protocol } line — K8s Services enumerate ports, no ranges).
Add it to the game-port ingress rule in networkpolicy.yaml (crafty-allow-ingress).
(Optional) add a containerPort in deployment.yaml for documentation.
Commit → Flux reconciles.

Policies

namespace.yaml sets only gateway-access=shared (platform namespace, like headlamp/homepage), so Kyverno does not auto-generate the app baseline. networkpolicy.yaml ships it explicitly (defense in depth): default-deny (DNS-only egress) + Cilium allows — Gateway→sidecar :4180, world/LAN→game ports, host→probe ports, and egress world (server-jar downloads + the OIDC path to Keycloak via the Gateway VIP). The sidecar→Crafty hop is pod-local (127.0.0.1) and not subject to network policy.

Verification

make -C ci gate — green (hand-authored platform manifests; render-golden unaffected).
kustomize build of platform/crafty, clusters/base, clusters/dev — clean.

Human tests (post-deploy)

Browser OIDC round-trip: incognito → https://crafty.home.keithjackson.dev → expect a 302 to https://keycloak.home.keithjackson.dev/realms/homelab/... (NOT the Crafty UI). Log in as a platform-admins user (MFA/OTP enforced) → land on Crafty's own login → sign in with Crafty's local admin. A non-platform-admins user → 403 (authenticated but not authorized).
Pod health / fsGroup: kubectl -n crafty get pod Ready 2/2; Crafty writes its DB under /crafty/app/config and /crafty/servers without EACCES.
Server-create + game reach: in the Crafty UI create a Java server bound to 25565; start it; connect a Minecraft client to crafty-mc.home.keithjackson.dev:25565 (or 192.168.1.55:25565) from the LAN and confirm the world loads. (Bedrock: a server on 19132/udp reached at the VIP.)

Runtime state: the servers themselves are NOT in git

As of 2026-09-14 one Java server exists — "homelab", Paper 26.2, TCP 25565, 2 GiB heap, autostart on, UUID 13c7b85d-00bb-4a58-8c86-2433ca47891c. It was created through Crafty's v2 API, and Mojang's EULA (https://aka.ms/MinecraftEULA) was accepted on the operator's behalf to let it start; see the DECISIONS-LOG entry of that date.

Everything in this directory describes only the capacity to run Minecraft servers. The servers are application state on the crafty-data Longhorn PVC, in two places, and Flux cannot recreate either:

/crafty/app/config/db/ (config subPath) — Crafty's SQLite row: name, port, jar, launch command, heap sizes, auto_start / crash_detection.
/crafty/servers/<uuid>/ (servers subPath) — paper.jar, eula.txt, server.properties, and the generated world. The jar is re-downloadable; the world is not.

So a rebuild from git alone yields a healthy but empty Crafty with nothing listening on 25565. Recreating a server by hand means: create it in the UI or API, accept the EULA again, and PATCH auto_start to true — see the next note. The world starts over. Treat this PVC as backup-relevant.

Trap: the create API silently ignores `autostart`

POST /api/v2/servers accepts and validates autostart and crashdetection, but never stores them — the persisted columns are auto_start and crash_detection. After a create that asked for true, both read back false. Set them afterwards with PATCH /api/v2/servers/<uuid> and read them back to confirm, or the server will not come up after a pod restart.

9.1 KiB Raw Blame History Unescape Escape