Home-Network/home-network-k3s-impl
2
0 Fork 0
Code Issues 2 Pull Requests 5 Actions Packages Projects Releases Wiki Activity

merge: a Minecraft server exists and 192.168.2.22:25565 is playable
gate / gate (push) Has been cancelled
Details
gate / conformance (push) Has been cancelled
Details

Browse Source 
Crafty was healthy but had never had a server created — /crafty/servers was
empty, which is why the port refused. Paper 26.2 (Mojang's latest.release,
jar SHA256-verified by Crafty) with a 2 GiB heap sized from measurement: with
the server running the container cgroup reported 2.86Gi against a 4Gi limit,
so a 3 GiB heap would have risked OOMKill.

EULA ACCEPTED ON THE OPERATOR'S BEHALF — https://aka.ms/MinecraftEULA — via
Crafty's agree_to_eula, writing eula=true. A server refuses to start without
it and the operator asked for the service brought up while asleep. No account,
key or payment involved. To revoke: stop the server and set eula=false, or
delete the server.

Proven playable by a real Server List Ping handshake from the LAN, not merely
an open port: MOTD, Paper 26.2, protocol 776, 0/20 players, 12.3 ms RTT.

Caught a trap worth recording: Crafty's create API ACCEPTS and validates
'autostart' but stores 'auto_start', so it silently read back false. Only
reading the record back caught it; trusting the create call would have
reported autostart working while the server stayed dead after any restart.
Fixed by PATCH and then proven — deleting the pod moved Crafty prod-3 ->
prod-1 and it auto-started with the same world, which also independently
re-confirms the externalTrafficPolicy: Cluster fix, since under Local the VIP
would have blackholed exactly there.

Raises crafty's memory request 1Gi -> 3Gi (limit untouched): a permanently
committed 2 GiB heap made the old request understate real usage threefold,
misleading the scheduler and making this Burstable pod an early eviction
candidate.
This commit is contained in:
jakman85 committed 2026-09-14 02:06:54 -05:00
2 parents 83f13f4b0a b4bc8e1cb9
commit 97257515e8
3 files changed
+150 -1
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

DECISIONS-LOG.md
+112
View File
@@ -3968,3 +3968,115 @@ Two further things the agent got right by checking rather than assuming:
**The operator's actual next step here is not networking:** install UniFi OS in the guest, and create a
Minecraft server in Crafty.
---
## 2026-09-14 — Crafty's first Minecraft server exists: Paper 26.2 on 25565, and the EULA was accepted on the operator's behalf
The previous entry closed with *"create a Minecraft server in Crafty"*. Done. `/crafty/servers` is no
longer empty, and `192.168.2.22:25565` now answers a real Minecraft Server List Ping from the LAN.
**I ACCEPTED MOJANG'S EULA ON THE OPERATOR'S BEHALF.** This is a legal act and it was taken while they
were asleep, so it is recorded here, in the commit message, and at the top of the handoff rather than
buried. The agreement is the Minecraft End User Licence Agreement at <https://aka.ms/MinecraftEULA>
(Mojang's canonical redirect; historically `https://account.mojang.com/documents/minecraft_eula`). It
was accepted by passing `agree_to_eula: true` to Crafty's server-creation API, which writes
`eula=true` into `/crafty/servers/<uuid>/eula.txt` (the mechanism is `main_controller.py:457-463`). A
Minecraft server refuses to start without it, so the service the operator asked for could not have
been brought up otherwise. **To revoke: stop the server and set `eula=false` in that file, or delete
the server in Crafty.** No account, licence key or payment was involved — the EULA is a free
click-through covering running the server software.
**What was created, and why each choice.**
- **Paper, not Vanilla.** Paper is a drop-in Vanilla-compatible server with substantially better tick
performance and a plugin API; unmodified Minecraft clients connect to it exactly as they would to
Vanilla, so nothing is given up on the player side. Vanilla would only be the right pick if the
operator wanted strict parity with Mojang's own jar or a mod loader (Fabric/Forge/NeoForge are also
in Crafty's bucket if that changes). Purpur and Folia were available and not chosen: Purpur adds
gameplay-changing configuration nobody asked for, and Folia's threaded-regions model targets very
large player counts and is the wrong shape for a home server.
- **Version 26.2.** Mojang's `version_manifest_v2.json` reports `latest.release = 26.2`, released
2026-06-16 — about three months mature, so not a fresh drop. The current snapshot is `26.3-rc-2`,
and Crafty's bucket also lists `26.2-rc-2`; both were deliberately avoided, as the brief asked for a
current stable release and not a snapshot. Crafty verifies the jar against a published SHA256.
- **A 2 GiB heap (`-Xms2000M -Xmx2000M`), not larger.** This was measured, not guessed. The container's
memory limit is 4Gi. With the server idle-but-running, the cgroup reported `memory.current` of
2.86Gi against a 4Gi `memory.max` — that is the 2 GiB committed heap, plus roughly half a gigabyte of
JVM non-heap (metaspace, code cache, GC structures, thread stacks, memory-mapped region files), plus
the Crafty panel itself. A 3 GiB heap would have landed at or above the 4Gi cap and invited an
OOMKill. Minimum and maximum heap are set equal, which is the standard Paper recommendation to avoid
heap-resize pauses. Node `k3s-prod-3` has about 31 GiB allocatable and was only 38 percent used, so
the node was never the constraint — the container limit was.
**The one repo change.** `platform/crafty/deployment.yaml`: the crafty container's memory *request*
went 1Gi to 3Gi. The limit is untouched at 4Gi. Before a server existed, a 1Gi request was honest;
now that a 2 GiB heap is permanently committed, a 1Gi request understates real usage roughly threefold,
which both misleads the scheduler and makes this Burstable pod an early eviction candidate under node
memory pressure. Nothing else was altered — not the L4 Services, not the NetworkPolicy, not the PVC.
The existing egress policy already allowed `world`, so the jar download needed no new rule.
**A trap worth recording: the create API silently ignores `autostart`.** The server-creation schema
accepts `autostart` and `crashdetection` and validates them happily, but the values never reach the
database — the stored columns are `auto_start` and `crash_detection`, and they came back `false` after
a creation that had asked for `true`. They must be set with a separate `PATCH /api/v2/servers/<uuid>`.
Had I trusted the create call, I would have reported autostart working when it was not, and the server
would have stayed dead after the next pod restart. Both are now `true`, verified by reading them back.
**THIS SERVER IS APP STATE. IT IS NOT IN GIT, AND A REBUILD FROM GIT ALONE WILL NOT RECREATE IT.**
Same shape as the arr download-client rows. What git holds is only the *capacity* to run Minecraft
servers: the Deployment, the PVC, the L4 Services and the policies. The server itself lives in two
places, both on the `crafty-data` Longhorn PVC and neither reconstructible by Flux:
1. **Crafty's SQLite database** (`/crafty/app/config/db/`, the `config` subPath) — the row defining
server `13c7b85d-00bb-4a58-8c86-2433ca47891c` ("homelab"): its name, port, jar, launch command,
heap sizes, autostart and crash-detection flags.
2. **The server directory** (`/crafty/servers/13c7b85d-00bb-4a58-8c86-2433ca47891c/`, the `servers`
subPath) — `paper.jar`, `eula.txt`, `server.properties`, and **the generated world**, which is the
genuinely irreplaceable part. A jar can be re-downloaded; a world cannot be regenerated.
If that PVC is lost and restored from nothing, `flux reconcile` brings back a healthy, *empty* Crafty
with no servers and a refused connection on 25565 — precisely the state this entry is closing out.
Recreating it by hand means: create the server in Crafty (Paper, 26.2, port 25565, 2 GiB heap), accept
the EULA again, PATCH `auto_start` on — and accept that the world starts over. **The world is
therefore backup-relevant in a way the manifests are not**, which makes the still-open no-backups gap
(Proxmox migration entry) materially worse than it was yesterday: there is now player-authored state on
this cluster. Crafty has its own scheduled-backup feature writing to the `backups` subPath, but that is
same-PVC and so protects against an in-app mistake, not against loss of the volume. Configuring it, and
getting the volume itself into a real backup, are both still open and both need the operator.
**Proofs, run in the order the brief asked for.** Every one of these is real output, not inference.
(a) Crafty reports started running=True ver='Paper 26.2' players=0/20
server log: 'Done (12.737s)! For help, type "help"'
(b) in-cluster, cross-node from a pod on k3s-prod-1 while crafty ran on k3s-prod-3,
python3 sockets (no curl in these images):
10.0.1.95:25565 pod IP SLP OK
10.43.5.109:25565 ClusterIP SLP OK
192.168.2.22:25565 the VIP SLP OK
(c) from the LAN, VIP workstation host, bash /dev/tcp to 192.168.2.22:25565 -> connect OK
(d) Server List Ping full handshake -> status -> ping/pong, from the workstation:
MOTD 'A Minecraft Server' Version 'Paper 26.2'
Protocol 776 Players 0/20
round trip 12.3 ms enforcesSecureChat True
Proof (d) is the one that matters: it is the same exchange a Minecraft client performs to draw the
server in its list, so it demonstrates the game protocol end to end rather than merely an open port.
**Restart persistence — tested, and it moved nodes.** Deleting the pod rescheduled Crafty from
`k3s-prod-3` onto `k3s-prod-1`. It came back `2/2 Running`, and with **no manual start issued** the
server auto-started about 30 seconds later: same UUID, same world directory, `eula=true` intact,
`Paper 26.2` answering. The LAN Server List Ping was then re-run against the VIP and still succeeded
with the pod on a different node — which independently re-confirms the `externalTrafficPolicy: Cluster`
fix from `adda6dd`, since under the old `Local` policy the VIP would have blackholed exactly here.
**Left alone and verified unbroken:** the oauth2-proxy sidecar still returns 302 to
`keycloak.home.keithjackson.dev/realms/homelab/protocol/openid-connect/auth` for an unauthenticated
request, so the Keycloak gate on the web UI is intact. Note that `crafty.home.keithjackson.dev` does
not resolve from the workstation shell — that is the pre-existing split-horizon DNS gap described in
the entry above, not a regression from this work.
**Still open for the operator:** review the EULA acceptance; decide whether the defaults are wanted
(MOTD is still "A Minecraft Server", 20 player slots, no whitelist, online-mode on, and **the server is
open to anyone who can reach the VIP** — a whitelist or `loadBalancerSourceRanges` is a judgement call I
did not make alone); and get the world into a backup.
platform/crafty/README.md
+26
View File
@@ -117,3 +117,29 @@ Gateway VIP). The sidecar→Crafty hop is pod-local (`127.0.0.1`) and not subjec
3. **Server-create + game reach:** in the Crafty UI create a Java server bound to `25565`; start it;
connect a Minecraft client to `crafty-mc.home.keithjackson.dev:25565` (or `192.168.1.55:25565`)
from the LAN and confirm the world loads. (Bedrock: a server on `19132/udp` reached at the VIP.)
## Runtime state: the servers themselves are NOT in git
As of 2026-09-14 one Java server exists — **"homelab"**, Paper 26.2, TCP 25565, 2 GiB heap, autostart
on, UUID `13c7b85d-00bb-4a58-8c86-2433ca47891c`. It was created through Crafty's v2 API, and **Mojang's
EULA (<https://aka.ms/MinecraftEULA>) was accepted on the operator's behalf** to let it start; see the
DECISIONS-LOG entry of that date.
Everything in this directory describes only the *capacity* to run Minecraft servers. The servers are
application state on the `crafty-data` Longhorn PVC, in two places, and **Flux cannot recreate either**:
- `/crafty/app/config/db/` (`config` subPath) — Crafty's SQLite row: name, port, jar, launch command,
heap sizes, `auto_start` / `crash_detection`.
- `/crafty/servers/<uuid>/` (`servers` subPath) — `paper.jar`, `eula.txt`, `server.properties`, and
**the generated world**. The jar is re-downloadable; the world is not.
So a rebuild from git alone yields a healthy but **empty** Crafty with nothing listening on 25565.
Recreating a server by hand means: create it in the UI or API, accept the EULA again, and PATCH
`auto_start` to true — see the next note. The world starts over. **Treat this PVC as backup-relevant.**
### Trap: the create API silently ignores `autostart`
`POST /api/v2/servers` accepts and validates `autostart` and `crashdetection`, but never stores them —
the persisted columns are `auto_start` and `crash_detection`. After a create that asked for `true`,
both read back `false`. Set them afterwards with `PATCH /api/v2/servers/<uuid>` and **read them back to
confirm**, or the server will not come up after a pod restart.
platform/crafty/deployment.yaml
+12 -1
View File
@@ -79,8 +79,19 @@ spec:
resources:
requests:
cpu: 250m
memory: 1Gi
# Raised 1Gi -> 3Gi when the first Minecraft server ("homelab", Paper 26.2) was created:
# that JVM runs with -Xms2000M -Xmx2000M, so the container's steady-state floor is now the
# panel (~100Mi) PLUS a permanently-committed 2 GiB heap plus JVM overhead. Measured
# cgroup memory.current with the server idle-but-running: 2.86Gi. A 1Gi request understated
# that by ~3x, which both misleads the scheduler and makes this Burstable pod an early
# eviction candidate under node memory pressure (kubelet ranks by usage-over-request).
memory: 3Gi
# Minecraft JVMs are memory-hungry; cap generously for a starter set of small servers.
# Headroom check for the 4Gi cap: 2 GiB heap + ~0.5 GiB JVM non-heap (metaspace, code cache,
# GC structures, thread stacks, mmap'd region files) + ~0.1 GiB panel = ~2.9Gi observed,
# leaving ~1.2Gi. A 3 GiB heap would have landed near 4Gi and risked an OOMKill, which is
# why the server was sized at a 2 GiB heap rather than larger. Raise BOTH this limit and the
# server's own memory in Crafty together if a bigger heap is ever wanted.
limits:
memory: 4Gi
securityContext: