jakman85andClaude Opus 5
b4bc8e1cb9
ACTION TAKEN ON THE OPERATOR'S BEHALF WHILE THEY SLEPT: I accepted Mojang's
Minecraft End User Licence Agreement, https://aka.ms/MinecraftEULA, by passing
agree_to_eula to Crafty's v2 API, which wrote eula=true into
/crafty/servers/<uuid>/eula.txt. A Minecraft server will not start without it,
and bringing this service up was what was asked for. Please review. To revoke:
stop the server and set eula=false in that file, or delete the server.
Crafty was healthy but had never had a server created, so 192.168.2.22:25565
refused connections. Created "homelab" via the API (not UI automation):
Paper 26.2, TCP 25565, heap -Xms2000M -Xmx2000M, autostart on
uuid 13c7b85d-00bb-4a58-8c86-2433ca47891c
Paper over Vanilla for tick performance and plugins, with no cost to players
(vanilla clients connect unchanged). 26.2 is Mojang's latest.release, out
2026-06-16; the 26.3-rc-2 snapshot and 26.2-rc-2 were deliberately avoided.
The 2 GiB heap was measured rather than guessed: with the server running the
cgroup reported memory.current 2.86Gi against the 4Gi limit, so a 3 GiB heap
would have risked an OOMKill.
Proven playable, all real output: Crafty reports running with 'Done (12.737s)!';
Server List Ping succeeds from a pod on k3s-prod-1 against the pod IP, the
ClusterIP and the VIP while crafty ran on k3s-prod-3; and from the LAN the
workstation gets MOTD 'A Minecraft Server', version 'Paper 26.2', protocol 776,
players 0/20 over a full handshake plus ping/pong. Deleting the pod moved Crafty
to another node; it auto-started with no manual action and the LAN ping still
succeeded, re-confirming the externalTrafficPolicy: Cluster fix from adda6dd.
Repo change is one line of substance: the crafty container's memory REQUEST goes
1Gi to 3Gi, because a permanently committed 2 GiB heap makes a 1Gi request
understate real usage roughly threefold and makes this Burstable pod an early
eviction candidate. Limit stays 4Gi. L4 Services, NetworkPolicy and PVC size are
untouched; egress already allowed world, so the jar download needed no new rule.
Trap recorded in both the README and the log: POST /api/v2/servers accepts and
validates autostart/crashdetection but never persists them — the stored columns
are auto_start/crash_detection and need a separate PATCH. Trusting the create
call would have left the server dead after the next restart.
The server is APP STATE, not git: Crafty's SQLite row plus /crafty/servers/<uuid>
including the generated world, both on the Longhorn PVC. A rebuild from git alone
gives an empty Crafty and a refused 25565. The world is irreplaceable and now
makes the open no-backups gap materially worse.
Co-Authored-By: Claude Opus 5 <[email protected]>
platform/crafty
Crafty Controller 4 — a web panel that creates and manages multiple Minecraft servers (Java +
Bedrock) as subprocesses inside its own container. Its web UI is Keycloak-gated via a per-pod
oauth2-proxy reverse-proxy sidecar (the decided Option B, see
../oauth2-proxy/README.md). Game ports are exposed un-gated on a
Cilium L2 LoadBalancer (game protocols authenticate players themselves).
- Owner: WS-APP (auth wiring shared with WS-SEC)
- Flux tier: own non-gating
clusters/base/crafty.yaml(dependsOn platform-config,wait:false) - Namespace:
crafty(PLATFORM namespace —gateway-access=shared, NOTapp-namespace=true)
Deploy summary
| Thing | Value |
|---|---|
| Image | registry.gitlab.com/crafty-controller/crafty-4:4.10.8 (current stable; bare-semver tags, no v) |
| Web UI | HTTPS self-signed on :8443 (reached only via the sidecar / same-pod) |
| Sidecar | quay.io/oauth2-proxy/oauth2-proxy:v7.8.1, reverse-proxy on :4180 |
| Data | ONE Longhorn RWO PVC crafty-data (20Gi), mounted 5× by subPath |
| PVC subPaths | servers→/crafty/servers, backups→/crafty/backups, logs→/crafty/logs, import→/crafty/import, config→/crafty/app/config |
| Identity | runAsUser:1000 runAsGroup:0 fsGroup:0 runAsNonRoot:true (see below) |
| Web host | crafty.home.keithjackson.dev → shared Gateway → crafty Service :80 → sidecar :4180 → Crafty :8443 |
| Game VIP | 192.168.1.55 (Cilium L2 LoadBalancer; .53=Gateway, .54=unifi, .55=crafty) |
| Game ports | starter set: TCP 25565-25567 (Java), UDP 19132-19134 (Bedrock) |
fsGroup / user decision (the seerr lesson, applied)
Crafty's Dockerfile does useradd -g root -M crafty (UID 1000, primary group root/GID 0),
chown -R crafty:root /crafty, and makes /crafty group-writable — the OpenShift/K8s
"arbitrary-UID, root-group-perms" pattern. A fresh Longhorn PVC is root:root and not group-writable,
so the non-root crafty process could not create /crafty/servers/* etc. and would CrashLoop (exactly
seerr's failure at UID 1000 on a root:root PVC). Fix: fsGroup: 0 makes the kubelet group-own
the mounted volume to GID 0, mark it group-writable+SGID, and add GID 0 as a supplemental group;
runAsGroup: 0 + runAsUser: 1000 pin the image's own crafty:root identity. Longhorn's
CSIDriver fsGroupPolicy: ReadWriteOnceWithFSType applies fsGroup to this RWO filesystem volume.
HUMAN runtime check: confirm the pod writes its SQLite DB and can start a server. If the image's
craftyUID is ever not 1000, adjustrunAsUser(the launcher's non-root path runs directly as the current UID, so any non-root UID with GID 0 works, but 1000 matchesgetpwuidcleanly).
Auth = Option B (per-app oauth2-proxy reverse-proxy)
Cilium 1.16.5 cannot inject ext_authz into the shared Gateway's machine-generated Envoy config, and
native Gateway-API ExternalAuth ships in no released Cilium yet (see ../oauth2-proxy/README.md). So
Crafty's UI is gated by a dedicated oauth2-proxy sidecar in the Crafty pod (--reverse-proxy,
--upstream=https://127.0.0.1:8443, --ssl-upstream-insecure-skip-verify=true,
--allowed-groups=platform-admins, --skip-provider-button, cookie/redirect for
crafty.home.keithjackson.dev). The crafty Service targets the sidecar, never Crafty's :8443, so
the gate is in the data path (fail-closed: no valid platform-admins session ⇒ no upstream).
Note: Crafty ALSO has its own local admin login (it has no OIDC). After oauth2-proxy admits the user, they still see Crafty's own login page — a second, defense-in-depth factor. The operator logs in with Crafty's local admin credentials (set on first boot). This is expected, not a bug.
Keycloak client
Confidential auth-code client crafty in the homelab realm — redirect
https://crafty.home.keithjackson.dev/oauth2/callback, webOrigins the same host, a groups
membership mapper (bare names). Provisioned live via kcadm at authoring time (2026-08-28) and
captured as the durable GitOps source of truth in
../keycloak/keycloak-realm-import.yaml (secret injected via
the CRAFTY_CLIENT_SECRET placeholder).
Secret locations (SOPS/age)
| Secret | File | Key |
|---|---|---|
| oauth2-proxy client id/secret + cookie secret | crafty-oauth2-proxy-credentials.sops.yaml (this dir) |
client-id, client-secret, cookie-secret |
| Keycloak client secret (mirror) | ../keycloak/keycloak-realm-clients.sops.yaml |
crafty-client-secret |
client-secret here must equal crafty-client-secret there (cross-namespace mirror — a Secret
can't span namespaces). Both are age-encrypted to the repo sops-age recipient and Flux-decryptable
(verified: both decrypt and the two values match).
Game-port L4 exposure
The Gateway is L7-HTTPS-only; raw TCP/UDP game ports can't ride it. service-l4.yaml exposes them as
two type=LoadBalancer Services (TCP + UDP) sharing the pinned VIP 192.168.1.55
(lbipam.cilium.io/sharing-key: crafty), ARP-announced by the existing default-l2 policy. This path
is NOT auth-gated — Minecraft Java (Mojang/Microsoft) and Bedrock (Xbox Live) authenticate players
themselves. external-dns publishes crafty-mc.home.keithjackson.dev → 192.168.1.55.
Adding more game ports (when the operator creates servers)
The operator has not created any servers yet; the starter set covers 3 Java + 3 Bedrock slots. To add capacity:
- In the Crafty UI, create the server and bind it to a NEW port (Java TCP or Bedrock UDP).
- Add that port to the matching Service in
service-l4.yaml(one{ name, port, targetPort, protocol }line — K8s Services enumerate ports, no ranges). - Add it to the game-port ingress rule in
networkpolicy.yaml(crafty-allow-ingress). - (Optional) add a
containerPortindeployment.yamlfor documentation. - Commit → Flux reconciles.
Policies
namespace.yaml sets only gateway-access=shared (platform namespace, like headlamp/homepage), so
Kyverno does not auto-generate the app baseline. networkpolicy.yaml ships it explicitly (defense in
depth): default-deny (DNS-only egress) + Cilium allows — Gateway→sidecar :4180, world/LAN→game
ports, host→probe ports, and egress world (server-jar downloads + the OIDC path to Keycloak via the
Gateway VIP). The sidecar→Crafty hop is pod-local (127.0.0.1) and not subject to network policy.
Verification
make -C ci gate— green (hand-authored platform manifests; render-golden unaffected).kustomize buildofplatform/crafty,clusters/base,clusters/dev— clean.
Human tests (post-deploy)
- Browser OIDC round-trip: incognito →
https://crafty.home.keithjackson.dev→ expect a 302 tohttps://keycloak.home.keithjackson.dev/realms/homelab/...(NOT the Crafty UI). Log in as a platform-admins user (MFA/OTP enforced) → land on Crafty's own login → sign in with Crafty's local admin. A non-platform-admins user → 403 (authenticated but not authorized). - Pod health / fsGroup:
kubectl -n crafty get podReady 2/2; Crafty writes its DB under/crafty/app/configand/crafty/serverswithout EACCES. - Server-create + game reach: in the Crafty UI create a Java server bound to
25565; start it; connect a Minecraft client tocrafty-mc.home.keithjackson.dev:25565(or192.168.1.55:25565) from the LAN and confirm the world loads. (Bedrock: a server on19132/udpreached at the VIP.)
Runtime state: the servers themselves are NOT in git
As of 2026-09-14 one Java server exists — "homelab", Paper 26.2, TCP 25565, 2 GiB heap, autostart
on, UUID 13c7b85d-00bb-4a58-8c86-2433ca47891c. It was created through Crafty's v2 API, and Mojang's
EULA (https://aka.ms/MinecraftEULA) was accepted on the operator's behalf to let it start; see the
DECISIONS-LOG entry of that date.
Everything in this directory describes only the capacity to run Minecraft servers. The servers are
application state on the crafty-data Longhorn PVC, in two places, and Flux cannot recreate either:
/crafty/app/config/db/(configsubPath) — Crafty's SQLite row: name, port, jar, launch command, heap sizes,auto_start/crash_detection./crafty/servers/<uuid>/(serverssubPath) —paper.jar,eula.txt,server.properties, and the generated world. The jar is re-downloadable; the world is not.
So a rebuild from git alone yields a healthy but empty Crafty with nothing listening on 25565.
Recreating a server by hand means: create it in the UI or API, accept the EULA again, and PATCH
auto_start to true — see the next note. The world starts over. Treat this PVC as backup-relevant.
Trap: the create API silently ignores autostart
POST /api/v2/servers accepts and validates autostart and crashdetection, but never stores them —
the persisted columns are auto_start and crash_detection. After a create that asked for true,
both read back false. Set them afterwards with PATCH /api/v2/servers/<uuid> and read them back to
confirm, or the server will not come up after a pod restart.