jakman85andClaude Opus 5
b4bc8e1cb9
ACTION TAKEN ON THE OPERATOR'S BEHALF WHILE THEY SLEPT: I accepted Mojang's
Minecraft End User Licence Agreement, https://aka.ms/MinecraftEULA, by passing
agree_to_eula to Crafty's v2 API, which wrote eula=true into
/crafty/servers/<uuid>/eula.txt. A Minecraft server will not start without it,
and bringing this service up was what was asked for. Please review. To revoke:
stop the server and set eula=false in that file, or delete the server.
Crafty was healthy but had never had a server created, so 192.168.2.22:25565
refused connections. Created "homelab" via the API (not UI automation):
Paper 26.2, TCP 25565, heap -Xms2000M -Xmx2000M, autostart on
uuid 13c7b85d-00bb-4a58-8c86-2433ca47891c
Paper over Vanilla for tick performance and plugins, with no cost to players
(vanilla clients connect unchanged). 26.2 is Mojang's latest.release, out
2026-06-16; the 26.3-rc-2 snapshot and 26.2-rc-2 were deliberately avoided.
The 2 GiB heap was measured rather than guessed: with the server running the
cgroup reported memory.current 2.86Gi against the 4Gi limit, so a 3 GiB heap
would have risked an OOMKill.
Proven playable, all real output: Crafty reports running with 'Done (12.737s)!';
Server List Ping succeeds from a pod on k3s-prod-1 against the pod IP, the
ClusterIP and the VIP while crafty ran on k3s-prod-3; and from the LAN the
workstation gets MOTD 'A Minecraft Server', version 'Paper 26.2', protocol 776,
players 0/20 over a full handshake plus ping/pong. Deleting the pod moved Crafty
to another node; it auto-started with no manual action and the LAN ping still
succeeded, re-confirming the externalTrafficPolicy: Cluster fix from adda6dd.
Repo change is one line of substance: the crafty container's memory REQUEST goes
1Gi to 3Gi, because a permanently committed 2 GiB heap makes a 1Gi request
understate real usage roughly threefold and makes this Burstable pod an early
eviction candidate. Limit stays 4Gi. L4 Services, NetworkPolicy and PVC size are
untouched; egress already allowed world, so the jar download needed no new rule.
Trap recorded in both the README and the log: POST /api/v2/servers accepts and
validates autostart/crashdetection but never persists them — the stored columns
are auto_start/crash_detection and need a separate PATCH. Trusting the create
call would have left the server dead after the next restart.
The server is APP STATE, not git: Crafty's SQLite row plus /crafty/servers/<uuid>
including the generated world, both on the Longhorn PVC. A rebuild from git alone
gives an empty Crafty and a refused 25565. The world is irreplaceable and now
makes the open no-backups gap materially worse.
Co-Authored-By: Claude Opus 5 <[email protected]>
146 lines
9.1 KiB
Markdown
146 lines
9.1 KiB
Markdown
# platform/crafty
|
||
|
||
**Crafty Controller 4** — a web panel that creates and manages multiple Minecraft servers (Java +
|
||
Bedrock) as **subprocesses inside its own container**. Its web UI is **Keycloak-gated** via a per-pod
|
||
**oauth2-proxy reverse-proxy sidecar** (the decided **Option B**, see
|
||
[`../oauth2-proxy/README.md`](../oauth2-proxy/README.md)). Game ports are exposed **un-gated** on a
|
||
Cilium L2 LoadBalancer (game protocols authenticate players themselves).
|
||
|
||
- **Owner:** WS-APP (auth wiring shared with WS-SEC)
|
||
- **Flux tier:** own non-gating `clusters/base/crafty.yaml` (`dependsOn platform-config`, `wait:false`)
|
||
- **Namespace:** `crafty` (PLATFORM namespace — `gateway-access=shared`, NOT `app-namespace=true`)
|
||
|
||
## Deploy summary
|
||
|
||
| Thing | Value |
|
||
|---|---|
|
||
| Image | `registry.gitlab.com/crafty-controller/crafty-4:4.10.8` (current stable; bare-semver tags, no `v`) |
|
||
| Web UI | HTTPS **self-signed** on `:8443` (reached only via the sidecar / same-pod) |
|
||
| Sidecar | `quay.io/oauth2-proxy/oauth2-proxy:v7.8.1`, reverse-proxy on `:4180` |
|
||
| Data | ONE Longhorn RWO PVC `crafty-data` (20Gi), mounted 5× by subPath |
|
||
| PVC subPaths | `servers`→`/crafty/servers`, `backups`→`/crafty/backups`, `logs`→`/crafty/logs`, `import`→`/crafty/import`, `config`→`/crafty/app/config` |
|
||
| Identity | `runAsUser:1000 runAsGroup:0 fsGroup:0 runAsNonRoot:true` (see below) |
|
||
| Web host | `crafty.home.keithjackson.dev` → shared Gateway → `crafty` Service :80 → sidecar :4180 → Crafty :8443 |
|
||
| Game VIP | **192.168.1.55** (Cilium L2 LoadBalancer; `.53`=Gateway, `.54`=unifi, `.55`=crafty) |
|
||
| Game ports | starter set: **TCP 25565-25567** (Java), **UDP 19132-19134** (Bedrock) |
|
||
|
||
### fsGroup / user decision (the seerr lesson, applied)
|
||
|
||
Crafty's Dockerfile does `useradd -g root -M crafty` (UID **1000**, primary group **root/GID 0**),
|
||
`chown -R crafty:root /crafty`, and makes `/crafty` group-writable — the OpenShift/K8s
|
||
"arbitrary-UID, root-group-perms" pattern. A fresh Longhorn PVC is `root:root` and not group-writable,
|
||
so the non-root crafty process could not create `/crafty/servers/*` etc. and would CrashLoop (exactly
|
||
seerr's failure at UID 1000 on a `root:root` PVC). Fix: **`fsGroup: 0`** makes the kubelet group-own
|
||
the mounted volume to GID 0, mark it group-writable+SGID, and add GID 0 as a supplemental group;
|
||
**`runAsGroup: 0`** + **`runAsUser: 1000`** pin the image's own `crafty:root` identity. Longhorn's
|
||
`CSIDriver fsGroupPolicy: ReadWriteOnceWithFSType` applies fsGroup to this RWO filesystem volume.
|
||
|
||
> **HUMAN runtime check:** confirm the pod writes its SQLite DB and can start a server. If the image's
|
||
> `crafty` UID is ever not 1000, adjust `runAsUser` (the launcher's non-root path runs directly as the
|
||
> current UID, so any non-root UID with GID 0 works, but 1000 matches `getpwuid` cleanly).
|
||
|
||
## Auth = Option B (per-app oauth2-proxy reverse-proxy)
|
||
|
||
Cilium 1.16.5 cannot inject `ext_authz` into the shared Gateway's machine-generated Envoy config, and
|
||
native Gateway-API ExternalAuth ships in no released Cilium yet (see `../oauth2-proxy/README.md`). So
|
||
Crafty's UI is gated by a **dedicated oauth2-proxy sidecar in the Crafty pod** (`--reverse-proxy`,
|
||
`--upstream=https://127.0.0.1:8443`, `--ssl-upstream-insecure-skip-verify=true`,
|
||
`--allowed-groups=platform-admins`, `--skip-provider-button`, cookie/redirect for
|
||
`crafty.home.keithjackson.dev`). The `crafty` Service targets the sidecar, never Crafty's `:8443`, so
|
||
the gate is **in the data path** (fail-closed: no valid `platform-admins` session ⇒ no upstream).
|
||
|
||
> **Note:** Crafty ALSO has its own local admin login (it has no OIDC). After oauth2-proxy admits the
|
||
> user, they still see Crafty's own login page — a second, defense-in-depth factor. The operator logs
|
||
> in with Crafty's local admin credentials (set on first boot). This is expected, not a bug.
|
||
|
||
### Keycloak client
|
||
|
||
Confidential auth-code client **`crafty`** in the `homelab` realm — redirect
|
||
`https://crafty.home.keithjackson.dev/oauth2/callback`, `webOrigins` the same host, a `groups`
|
||
membership mapper (bare names). Provisioned **live via kcadm** at authoring time (2026-08-28) and
|
||
captured as the durable GitOps source of truth in
|
||
[`../keycloak/keycloak-realm-import.yaml`](../keycloak/keycloak-realm-import.yaml) (secret injected via
|
||
the `CRAFTY_CLIENT_SECRET` placeholder).
|
||
|
||
### Secret locations (SOPS/age)
|
||
|
||
| Secret | File | Key |
|
||
|---|---|---|
|
||
| oauth2-proxy client id/secret + cookie secret | `crafty-oauth2-proxy-credentials.sops.yaml` (this dir) | `client-id`, `client-secret`, `cookie-secret` |
|
||
| Keycloak client secret (mirror) | `../keycloak/keycloak-realm-clients.sops.yaml` | `crafty-client-secret` |
|
||
|
||
`client-secret` here **must equal** `crafty-client-secret` there (cross-namespace mirror — a Secret
|
||
can't span namespaces). Both are age-encrypted to the repo `sops-age` recipient and Flux-decryptable
|
||
(verified: both decrypt and the two values match).
|
||
|
||
## Game-port L4 exposure
|
||
|
||
The Gateway is L7-HTTPS-only; raw TCP/UDP game ports can't ride it. `service-l4.yaml` exposes them as
|
||
two `type=LoadBalancer` Services (TCP + UDP) sharing the pinned VIP **192.168.1.55**
|
||
(`lbipam.cilium.io/sharing-key: crafty`), ARP-announced by the existing `default-l2` policy. This path
|
||
is **NOT auth-gated** — Minecraft Java (Mojang/Microsoft) and Bedrock (Xbox Live) authenticate players
|
||
themselves. `external-dns` publishes `crafty-mc.home.keithjackson.dev → 192.168.1.55`.
|
||
|
||
### Adding more game ports (when the operator creates servers)
|
||
|
||
The operator has **not created any servers yet**; the starter set covers 3 Java + 3 Bedrock slots. To
|
||
add capacity:
|
||
|
||
1. In the Crafty UI, create the server and bind it to a NEW port (Java TCP or Bedrock UDP).
|
||
2. Add that port to the matching Service in `service-l4.yaml` (one
|
||
`{ name, port, targetPort, protocol }` line — K8s Services enumerate ports, no ranges).
|
||
3. Add it to the game-port ingress rule in `networkpolicy.yaml` (`crafty-allow-ingress`).
|
||
4. (Optional) add a `containerPort` in `deployment.yaml` for documentation.
|
||
5. Commit → Flux reconciles.
|
||
|
||
## Policies
|
||
|
||
`namespace.yaml` sets only `gateway-access=shared` (platform namespace, like headlamp/homepage), so
|
||
Kyverno does not auto-generate the app baseline. `networkpolicy.yaml` ships it explicitly (defense in
|
||
depth): default-deny (DNS-only egress) + Cilium allows — Gateway→sidecar `:4180`, world/LAN→game
|
||
ports, host→probe ports, and egress `world` (server-jar downloads + the OIDC path to Keycloak via the
|
||
Gateway VIP). The sidecar→Crafty hop is pod-local (`127.0.0.1`) and not subject to network policy.
|
||
|
||
## Verification
|
||
|
||
- `make -C ci gate` — green (hand-authored platform manifests; render-golden unaffected).
|
||
- `kustomize build` of `platform/crafty`, `clusters/base`, `clusters/dev` — clean.
|
||
|
||
### Human tests (post-deploy)
|
||
|
||
1. **Browser OIDC round-trip:** incognito → `https://crafty.home.keithjackson.dev` → expect a 302 to
|
||
`https://keycloak.home.keithjackson.dev/realms/homelab/...` (NOT the Crafty UI). Log in as a
|
||
**platform-admins** user (MFA/OTP enforced) → land on Crafty's own login → sign in with Crafty's
|
||
local admin. A **non-platform-admins** user → **403** (authenticated but not authorized).
|
||
2. **Pod health / fsGroup:** `kubectl -n crafty get pod` Ready 2/2; Crafty writes its DB under
|
||
`/crafty/app/config` and `/crafty/servers` without EACCES.
|
||
3. **Server-create + game reach:** in the Crafty UI create a Java server bound to `25565`; start it;
|
||
connect a Minecraft client to `crafty-mc.home.keithjackson.dev:25565` (or `192.168.1.55:25565`)
|
||
from the LAN and confirm the world loads. (Bedrock: a server on `19132/udp` reached at the VIP.)
|
||
|
||
## Runtime state: the servers themselves are NOT in git
|
||
|
||
As of 2026-09-14 one Java server exists — **"homelab"**, Paper 26.2, TCP 25565, 2 GiB heap, autostart
|
||
on, UUID `13c7b85d-00bb-4a58-8c86-2433ca47891c`. It was created through Crafty's v2 API, and **Mojang's
|
||
EULA (<https://aka.ms/MinecraftEULA>) was accepted on the operator's behalf** to let it start; see the
|
||
DECISIONS-LOG entry of that date.
|
||
|
||
Everything in this directory describes only the *capacity* to run Minecraft servers. The servers are
|
||
application state on the `crafty-data` Longhorn PVC, in two places, and **Flux cannot recreate either**:
|
||
|
||
- `/crafty/app/config/db/` (`config` subPath) — Crafty's SQLite row: name, port, jar, launch command,
|
||
heap sizes, `auto_start` / `crash_detection`.
|
||
- `/crafty/servers/<uuid>/` (`servers` subPath) — `paper.jar`, `eula.txt`, `server.properties`, and
|
||
**the generated world**. The jar is re-downloadable; the world is not.
|
||
|
||
So a rebuild from git alone yields a healthy but **empty** Crafty with nothing listening on 25565.
|
||
Recreating a server by hand means: create it in the UI or API, accept the EULA again, and PATCH
|
||
`auto_start` to true — see the next note. The world starts over. **Treat this PVC as backup-relevant.**
|
||
|
||
### Trap: the create API silently ignores `autostart`
|
||
|
||
`POST /api/v2/servers` accepts and validates `autostart` and `crashdetection`, but never stores them —
|
||
the persisted columns are `auto_start` and `crash_detection`. After a create that asked for `true`,
|
||
both read back `false`. Set them afterwards with `PATCH /api/v2/servers/<uuid>` and **read them back to
|
||
confirm**, or the server will not come up after a pod restart.
|