diff --git a/DECISIONS-LOG.md b/DECISIONS-LOG.md index f48751f..314fb8a 100644 --- a/DECISIONS-LOG.md +++ b/DECISIONS-LOG.md @@ -5285,3 +5285,181 @@ section (now a two-row table). 5. **After step 1, watch for the SNAT/RakNet question (h6 above)** — if a *second* simultaneous player evicts the first, that is the SNAT theory becoming real and it needs its own measured investigation (masquerade port reuse, UDP conntrack), not a guess. + +## 2026-09-29 (later) — Bedrock IS reachable on 19132/udp. The `TRANSPORT TYPE ERROR` banner is advisory, and the last entry's "REASONED, NOT OBSERVED" call has now been OBSERVED — correct + +**Do not read this as overturning the previous entry's recommendation. It confirms it.** What it +overturns is the conclusion the operator and I both drew from the server's *reaction* to it. + +### The sequence + +The entry above ended with operator action 1: stop `K3s-Bedrock`, set `transport=raknet`, start it — +explicitly flagged **"REASONED, NOT OBSERVED … Nobody has seen a packet answered on 19132/udp on this +cluster."** The operator did it. `/crafty/logs/audit.log`, measured: + + 21:32:32 issued command stop_server for server K3s-Bedrock + 21:33:31 Edited file /crafty/servers/c5837301-…/server.properties + 21:33:58 issued command start_server for server K3s-Bedrock + +BDS then printed, at ERROR level, `================ TRANSPORT TYPE ERROR ===================` / +`Your current connection type is not set to NetherNet. In this release, NetherNet is the only supported +transport type.` / `Players will not be able to connect to your game without NetherNet.` That was +reported to me as *"the server rejected it"*, and I was briefed to treat the shipped property comment +(`Allowed values: "raknet" or "nethernet"`) as stale relative to the binary and to write up an +architecture finding: that BDS is NetherNet-only, can never bind `19132/udp`, and that the whole +`crafty-l4-udp` exposure is therefore inert. + +**That brief was wrong, and so was the inference behind it. The server did not reject anything.** + +### What is actually true (all OBSERVED, 2026-09-29 21:38–21:41) + +- `./bedrock_server` pid **47006** is running, started 21:34 and still up — it printed that banner and + carried on. It did not exit and Crafty did not restart it. +- It **bound the RakNet sockets**: `/proc/net/udp` `00000000:4ABC` (19132/IPv4) and `/proc/net/udp6` + `…:4ABD` (19133/IPv6), both owned by that pid (`/proc/47006/fd/25`, `fd/26` → those inodes). The + **TCP** bind on `4ABC` that the last entry measured under nethernet is **gone**. +- Crafty's warning changed kind at that exact moment: last `Got timeout while issuing bedrock ping` at + **`21:34:04,306`**, first `Bedrock ping returned a string with too few fields` at **`21:34:09,854`** + (155 timeout lines, then 86 answered and counting). Per `raknet_ping.py`, `Got timeout` is emitted + *only* when `sock.recv` raises `TimeoutError`; `too few fields` is reachable *only after* a `0x1C` + Unconnected Pong arrives **and its magic validates**. The listener answers. +- **The end-to-end test the last entry admitted it had never run, run:** a RakNet Unconnected Ping from + the toolbox on the LAN (host network, off-cluster) to the prod VIP **`192.168.2.22:19132`** returns a + 139-byte `0x1c` pong in ~12 ms, magic valid: + + MCPE;Dedicated Server;2193;1.26.52;0;10;11676629557768398479;Bedrock level;Survival;1;19132;19133;0;0;0; + + BDS **1.26.52**, protocol 2193, 0/10 players. `:19133` returns the **same** pong from the same + `server_guid` (that is `enable-lan-visibility=true`, now observed rather than read off a comment); + `:19134` times out, correctly. **A Bedrock player on this LAN can connect right now.** +- The vendor's own doc on the PVC, `bedrock_server_how_to.html` (shipped in the same archive as the + binary, same mtime 20:52), documents `transport` as *"raknet, nethernet"* and: *"**raknet** — The + traditional Bedrock UDP transport. The server listens directly on `server-port` (IPv4) and + `server-portv6` (IPv6), and clients connect to that address and port."* +- RakNet is still fully compiled in: `strings bedrock_server` yields `RakNetConnector`, + `createUniqueRakPeer`, `RakPeerInterface`, and the vendored build path + `/mnt/vss/_work/1/s/handheld/src-deps/raknet/raknet/RakNetSocket2.cpp`. + +So `NetherNet is the only supported transport type` is a **support** statement, not a **capability** +one. Microsoft declines to support RakNet and ships it working, at ERROR log level, in 1.26.52. + +### Why nethernet could never have worked here anyway — a finding the last entry missed + +The same vendor doc explains the last entry's oddest measurement (19132 bound **TCP**): nethernet +*"listens on a single dual-stack **TCP** socket on `server-port` for an HTTP-based signaling handshake, +then negotiates a UDP connection with each client."* And this repo exposes **no TCP 19132**: +`crafty-l4-tcp` carries 25565-25567 only, and `crafty-allow-ingress` allows `19132 UDP` and not +`19132 TCP`. The signaling port was unreachable at the Service layer *and* the policy layer. The +server's own `packet-statistics.txt` for that 2293-second nethernet run agrees: `Total Received: 0, 0B`. +The 26 packets / 27660 bytes the last entry found on the `19132/UDP` policy map were players' RakNet +pings arriving at a port where nethernet had opened no UDP socket. Consistent, and now closed. + +### Where the reasoning failed — the part worth keeping + +The previous entry's reasoning was **sound and its recommendation was right**; it was correctly +labelled as unobserved, and the observation vindicated it. The failure was downstream, in reading the +server's response: + +1. **An ERROR-level log line was read as a refusal.** It was a deprecation notice. Nothing checked + whether the process survived it or what it bound — and both were one command away. +2. **"The vendor comment is stale relative to the binary" was asserted from the binary's *complaint*, + never from the binary's *behaviour*.** The binary had two other testimonies on disk — the shipped + how-to, and its own symbol table — and both said `raknet` works. Neither was consulted. +3. The tiebreak between two contradicting vendor statements is never a third piece of prose. It is + `/proc/net/udp` and a pong on the wire. That is what settled it, in about four minutes. + +This is CLAUDE.md §2b applied one level further in: don't infer platform state from the repo, **and +don't infer application state from the application's own error message.** Query the socket. + +### Decision and recommendation + +**Keep `transport=raknet`. No manifest change.** `service-l4.yaml`, `networkpolicy.yaml` and the +`containerPort`s were already shaped for exactly RakNet and are now proven end-to-end rather than +assumed. Ranked alternatives, all rejected for today and written up in the README: + +1. **Keep `transport=raknet`** — zero manifest change, works now, vanilla parity, real Bedrock world. +2. **GeyserMC + Floodgate** on the existing `homelab` Paper server — also zero manifest change (Geyser + itself binds `19132/udp`, so the existing Service is already correct), held in reserve for the day + RakNet is genuinely removed. It **cannot run alongside** the live BDS: both want 19132. Cost is that + Bedrock players join the **Java** world. Geyser's current Bedrock-version support is + **external and unmeasured**. +3. **Pin/downgrade BDS** — unnecessary (the current build speaks RakNet) and fragile. +4. **PocketMine-MP / Nukkit** — no; loses vanilla parity and world format. +5. **Accept NetherNet** — worst option: it needs *more* manifest work than the one that already works + (TCP 19132 in `crafty-l4-tcp` **and** in `crafty-allow-ingress`, plus `server-udp-ports` pinned and + exposed), and WebRTC media behind a shared VIP and SNAT is unproven here. + +The 19132-19134/UDP exposure **stays** and is **not** inert. That part of the brief is refuted. + +### What changed in this repo + +`platform/crafty/README.md` only — again **no manifest touched**, and this time that is a measured +conclusion rather than a cautious one. + +- Replaced *"the transport trap"* with a two-trap section: nethernet hides the UDP socket, and then the + `TRANSPORT TYPE ERROR` banner frightens you into reverting the fix. Records the full evidence chain, + the vendor how-to quotes, the binary symbols, and the off-cluster pong. +- Documented that Crafty's perpetual `too few fields - missing values [None]` warning is the **success** + signature, not a failure: BDS 1.26.52's pong has 16 `;`-fields, Crafty 4.10.8's + `pingpong_string_fields` knows 14 and wants an empty terminator, gets `'0'`. Cosmetic. The failure + signature is the *other* message, `Got timeout`. +- Added the ranked options table with each option's manifest consequence, and an *"If RakNet is ever + really removed"* section naming Geyser+Floodgate as the migration with the exact PVC/panel steps. +- Upgraded the `enable-lan-visibility` note from "read off a comment" to OBSERVED, and **corrected** the + claim that `19133` is unreachable — it is reachable, on IPv4, and slot 1 genuinely is occupied. +- Corrected the **Game VIP**: the README stated `192.168.1.55` as fact. That is the base/dev value; + `clusters/prod/kustomization.yaml` patches `CRAFTY_L4_VIP` to `192.168.2.22`, which is the live + `EXTERNAL-IP` on both `crafty-l4-tcp` and `crafty-l4-udp`. Rewrote as the variable plus the measured + prod value. +- Noted that `transport=raknet` is PVC state Flux cannot restore, and that a recreated or reinstalled + Bedrock server silently regresses to the vendor default `nethernet`. +- Re-confirmed `K3s-Bedrock` `auto_start=0` / `crash_detection=0` straight out of `crafty.sqlite`. + +### What I did NOT verify — read this before acting + +- **No human Bedrock client has connected.** I proved the RakNet *ping/pong* handshake end-to-end from + off-cluster; I did not complete a login, which additionally requires Xbox Live auth + (`online-mode=true`) from the client. **UNVERIFIED that a player reaches the world**, and it is the + one test only a human with a Minecraft client can run. +- **The multi-player SNAT question from the last entry is still open and still untested.** Whether a + *second* simultaneous Bedrock player evicts the first (masquerade port reuse / UDP conntrack behind a + shared VIP) is unmeasured. One ping proves one flow. +- **Whether a console can reach it is still no** — unchanged and still not tested; consoles have no + server-address field. +- **Geyser is entirely unmeasured here.** Nothing was installed, downloaded, or run. Its Bedrock-version + support, its jar names and its config keys as written in the README are **web/documentation-sourced, + not observed on this cluster**. Verify before acting on that section. +- **Whether Crafty can pin a BDS version is UNVERIFIED** — I did not read its Bedrock installer. +- **A web search on this error was attempted and was low-quality** — results largely echoed the error + string back. Nothing in the README or this entry rests on it. The load-bearing evidence is all local: + `/proc/net/*`, `/proc/47006/fd/*`, `audit.log`, `commander.log`, `raknet_ping.py`, the shipped + `bedrock_server_how_to.html`, `strings bedrock_server`, and the pong. +- **Whether a future BDS removes RakNet for real is unknowable from here.** Treat the banner as a + deprecation notice with an unknown expiry. +- **Nothing was applied or written.** No `kubectl apply/edit/patch/scale/delete`, no `flux reconcile`, + no write anywhere inside the container — `server.properties` was read, never touched. The running pod + is still `crafty-64f7b6d78-pkpgp` (2/2, 15d, 0 restarts). All probes were reads plus outbound UDP + pings from the toolbox. +- **`fix/crafty-oauth2-proxy-signin` is still unmerged**, so the pod still runs the pre-that-PR spec. + +### Operator actions required + +1. **Nothing to do about the transport.** `transport=raknet` is correct, live, and answering. **Do not + revert it because of the `TRANSPORT TYPE ERROR` banner, and do not chase Crafty's `too few fields` + warning** — both appear on a fully working server. +2. **Get a human to actually join**, from a Windows/mobile client: **Add Server** → + `crafty-mc.home.keithjackson.dev`, port **19132**. This is the only remaining unproven step. It will + never appear in the LAN/Friends tab; a console cannot reach it at all. +3. **Fix `auto_start` — still outstanding, and now more urgent.** `K3s-Bedrock` is still + `auto_start=0`, `crash_detection=0`. `PATCH /api/v2/servers/c5837301-12b1-404e-b50e-70eafb37226b` + setting both true **and read them back** (the create API silently drops them). Until then the + Bedrock server does not survive a pod restart — and merging `fix/crafty-oauth2-proxy-signin` + restarts the pod (`strategy: Recreate`, RWO volume), stopping **both** Minecraft servers **and** + leaving Bedrock down until someone starts it by hand. +4. **Set `enable-lan-visibility=false` on `K3s-Bedrock`** if a second Bedrock server is ever wanted. + It currently occupies **both** 19132 and 19133 (measured), so slot 1 is not actually free. LAN + discovery cannot work behind the VIP anyway, so nothing is lost. +5. **Re-check after any Bedrock server recreate or BDS update:** confirm `transport=raknet` survived, + then re-ping. The vendor default is `nethernet`, and the regression is silent. +6. **If a second simultaneous player evicts the first**, that is the SNAT/UDP-conntrack theory becoming + real; it needs its own measured investigation, not a guess. diff --git a/platform/crafty/README.md b/platform/crafty/README.md index 7fc18e5..be1443e 100644 --- a/platform/crafty/README.md +++ b/platform/crafty/README.md @@ -21,7 +21,7 @@ Cilium L2 LoadBalancer (game protocols authenticate players themselves). | PVC subPaths | `servers`→`/crafty/servers`, `backups`→`/crafty/backups`, `logs`→`/crafty/logs`, `import`→`/crafty/import`, `config`→`/crafty/app/config` | | Identity | `runAsUser:1000 runAsGroup:0 fsGroup:0 runAsNonRoot:true` (see below) | | Web host | `crafty.home.keithjackson.dev` → shared Gateway → `crafty` Service :80 → sidecar :4180 → Crafty :8443 | -| Game VIP | **192.168.1.55** (Cilium L2 LoadBalancer; `.53`=Gateway, `.54`=unifi, `.55`=crafty) | +| Game VIP | `${CRAFTY_L4_VIP}` (Cilium L2 LoadBalancer) — **prod `192.168.2.22`** (live `EXTERNAL-IP`, measured 2026-09-29); base/dev `192.168.1.55` | | Game ports | starter set: **TCP 25565-25567** (Java), **UDP 19132-19134** (Bedrock) | ### fsGroup / user decision (the seerr lesson, applied) @@ -76,10 +76,11 @@ can't span namespaces). Both are age-encrypted to the repo `sops-age` recipient ## Game-port L4 exposure The Gateway is L7-HTTPS-only; raw TCP/UDP game ports can't ride it. `service-l4.yaml` exposes them as -two `type=LoadBalancer` Services (TCP + UDP) sharing the pinned VIP **192.168.1.55** +two `type=LoadBalancer` Services (TCP + UDP) sharing the pinned VIP `${CRAFTY_L4_VIP}` (**prod +`192.168.2.22`**) (`lbipam.cilium.io/sharing-key: crafty`), ARP-announced by the existing `default-l2` policy. This path is **NOT auth-gated** — Minecraft Java (Mojang/Microsoft) and Bedrock (Xbox Live) authenticate players -themselves. `external-dns` publishes `crafty-mc.home.keithjackson.dev → 192.168.1.55`. +themselves. `external-dns` publishes `crafty-mc.home.keithjackson.dev → ${CRAFTY_L4_VIP}`. ### Adding more game ports (when the operator creates servers) @@ -93,42 +94,122 @@ Bedrock `K3s-Bedrock` on 19132. The starter set covers 3 Java + 3 Bedrock slots. 4. (Optional) add a `containerPort` in `deployment.yaml` for documentation. 5. Commit → Flux reconciles. -### Bedrock on UDP 19132 — the transport trap (MEASURED on prod 2026-09-29) +### Bedrock on UDP 19132 — the transport trap, and the ERROR banner that is NOT a refusal (MEASURED on prod 2026-09-29) -**A Bedrock server that Crafty created, started, and reports as running can still have nothing bound -to `19132/udp`, with every layer in this repo provably correct.** This is the first thing to check -when "Bedrock players cannot connect" — before the Service, the policy, the lease or the VIP. +Two traps in sequence. The first hides the UDP socket; the second frightens you into undoing the fix. +As of 2026-09-29 21:34 this server is **reachable on `19132/udp` and answering**, measured from +off-cluster. Read both traps before changing anything. -Measured on prod 2026-09-29 against server `K3s-Bedrock` (`c5837301-12b1-404e-b50e-70eafb37226b`, -DB `type=minecraft-bedrock`, `server_port=19132`, process `./bedrock_server` alive): +#### Trap 1 — `transport=nethernet` (the shipped default) opens no RakNet UDP listener -- `/proc/net/udp` and `/proc/net/udp6` inside the crafty container held **no socket on 19132** - (`0x4ABC`) at all — only ephemeral UDP sockets. Port 19132 was bound as **TCP** - (`/proc/net/tcp6`, `…:4ABC`, state `0A`/LISTEN), not UDP. -- Cause, from the server's own `server.properties`: **`transport=nethernet`** — the value Microsoft's - BDS archive *ships by default*. Its own inline docs: `# Allowed values: "raknet" or "nethernet"`, - and against `server-portv6`: `# This property is ignored when transport=nethernet and a dual-stack - socket will be opened on server-port instead`. NetherNet is the WebRTC/Xbox-signalled transport; it - **never opens a RakNet UDP listener**, so nothing binds `19132/udp` however `server-port` is set. -- Crafty does not manage this property — `transport` does not appear anywhere under - `/crafty/app/classes/` — and Crafty's *only* Bedrock reachability probe is `raknet_ping.py`, so the - panel spams `WARNING app.classes.remote_stats.raknet_ping - Got timeout while issuing bedrock ping - to 127.0.0.1:19132` every 5–30 s against a server it started successfully. **That warning is the - signature of this trap**, and it comes from `127.0.0.1` — i.e. it indicts the server, not the LAN. -- The platform path was verified sound at the same time: `crafty-l4-udp` live as - `externalTrafficPolicy: Cluster` with `status.loadBalancer.ingress[0].ip` = the VIP, a non-empty - EndpointSlice, `cilium-dbg service list` showing `:19132/UDP → 10.0.0.130:19132/UDP (active)` - on *every* node, and the crafty endpoint's datapath policy map carrying - `Allow Ingress reserved:world 19132/UDP` **with 26 packets / 27660 bytes already counted through - it**. Real player packets reached the pod and were allowed; the application had no socket to take - them. +Measured 20:54–21:32 against `K3s-Bedrock` (`c5837301-12b1-404e-b50e-70eafb37226b`, DB +`type=minecraft-bedrock`, `server_port=19132`, process `./bedrock_server` alive): -**Operator fix — application state on the PVC, not git and not Flux:** stop the server in Crafty, set -`transport=raknet` in `/crafty/servers//server.properties`, start it, and **confirm a UDP -listener on 19132 exists** (`grep 4ABC /proc/net/udp /proc/net/udp6`, or the `raknet_ping` warnings -stopping) before suspecting the network. *(That `raknet` restores the UDP bind is read from the -shipped property documentation, not yet observed on this cluster — see the DECISIONS-LOG entry of -2026-09-29.)* +- `/proc/net/udp` and `/proc/net/udp6` held **no socket on 19132** (`0x4ABC`) at all. Port 19132 was + bound as **TCP** (`/proc/net/tcp6`, `…:4ABC`, state `0A`/LISTEN), not UDP. +- That is nethernet working as designed, and the **shipped `bedrock_server_how_to.html`** — the + vendor's own doc, sitting on the PVC beside the binary — says exactly why: nethernet *"listens on a + single dual-stack **TCP** socket on `server-port` for an HTTP-based signaling handshake, then + negotiates a UDP connection with each client that carries the gameplay traffic."* +- **So nethernet could never have worked here anyway, on two independent counts:** `service-l4.yaml` + exposes 19132 **only in the UDP Service** (`crafty-l4-tcp` carries 25565-25567 and nothing else), and + `crafty-allow-ingress` allows `19132 UDP` and **not** `19132 TCP`. The signaling port was unreachable + at both the Service and the policy layer. The server's own `packet-statistics.txt` for that run + agrees: `Total Received: 0, 0B` over 2293 s. +- Crafty does not manage this property (`transport` appears nowhere under `/crafty/app/classes/`) and + its *only* Bedrock probe is `raknet_ping.py`, so the panel logged `WARNING + app.classes.remote_stats.raknet_ping - Got timeout while issuing bedrock ping to 127.0.0.1:19132` + every ~5 s — **155 such lines** — against a server it had started successfully. + +#### Trap 2 — under `transport=raknet` this build prints a scary ERROR banner and then works anyway + +The operator set `transport=raknet` at 21:33:31 (`/crafty/logs/audit.log`: `Edited file +/crafty/servers/c5837301-…/server.properties`) and started the server at 21:33:58. BDS **1.26.52** +printed, at ERROR level: + +``` +[ERROR] ================ TRANSPORT TYPE ERROR =================== +[ERROR] Your current connection type is not set to NetherNet. In this release, NetherNet is the only + supported transport type. +[ERROR] Players will not be able to connect to your game without NetherNet. +[ERROR] To switch, set 'transport=nethernet' in server.properties. +``` + +**That banner is advisory, not fatal. Do not revert `transport` on seeing it.** Every claim it makes +about connectivity was false on this cluster within five seconds of it being printed: + +- The process **kept running** (`./bedrock_server`, pid 47006) and **bound the RakNet sockets**: + `/proc/net/udp` `00000000:4ABC` (19132, IPv4) and `/proc/net/udp6` `…:4ABD` (19133, IPv6), both owned + by `bedrock_server` — `/proc/47006/fd/25` and `fd/26` → those socket inodes. The earlier **TCP** bind + on `4ABC` is **gone**. +- Crafty's warning **changed kind** at that instant: last `Got timeout` at `21:34:04,306`, first + `Bedrock ping returned a string with too few fields` at `21:34:09,854`. That distinction is + load-bearing — read `raknet_ping.py`: `Got timeout` is emitted *only* when `sock.recv` raises + `TimeoutError` (nothing came back at all), whereas `too few fields` is reachable *only after* a + `0x1C` Unconnected Pong has arrived **and its magic has validated**. The server is answering pings. +- **End-to-end, from off-cluster:** a RakNet Unconnected Ping sent from the LAN to the prod VIP + `192.168.2.22:19132` returns a 139-byte `0x1c` Unconnected Pong in ~12 ms, magic valid: + + MCPE;Dedicated Server;2193;1.26.52;0;10;11676629557768398479;Bedrock level;Survival;1;19132;19133;0;0;0; + + Edition `MCPE`, protocol `2193`, version **`1.26.52`**, `0/10` players. **Players can reach this + server now**, through the VIP, the Service, and the CiliumNetworkPolicy, with no manifest change. +- The vendor doc on disk contradicts the banner and matches the observed behaviour. `transport` is + documented as *"Allowed values: raknet, nethernet"*, and: *"**raknet** — The traditional Bedrock UDP + transport. The server listens directly on `server-port` (IPv4) and `server-portv6` (IPv6), and + clients connect to that address and port."* RakNet is also still fully compiled into the binary — + `strings bedrock_server` yields `RakNetConnector`, `createUniqueRakPeer`, `RakPeerInterface` and the + vendored build path `…/src-deps/raknet/raknet/RakNetSocket2.cpp`. + +So *"NetherNet is the only supported transport type"* is a **support statement, not a capability +statement**: Microsoft declines to support RakNet and ships it working. Treat the banner as a +deprecation notice with an **unknown expiry** — a future BDS may genuinely remove RakNet, and that is +the day this exposure needs GeyserMC (see *If RakNet is ever really removed* below). + +> **The shipped-comment trap, stated plainly.** `server.properties` lists `raknet` as allowed and the +> binary logs an ERROR saying it is not supported. Both are the vendor, and they disagree. The tiebreak +> is neither prose: it is `/proc/net/udp` and a pong on the wire. Whenever this file says a Bedrock +> server is or is not reachable, it is because someone pinged it — not because a comment said so. + +#### The steady-state `too few fields` warning is benign and is now the expected signature + +Crafty will log `Bedrock ping returned a string with too few fields - missing values [None]` every +~5 s forever. **That means the ping SUCCEEDED.** BDS 1.26.52's pong carries 16 `;`-separated fields; +Crafty 4.10.8's `pingpong_string_fields` table knows 14 and expects the last to be an empty +terminator. It instead finds `'0'`, so `last_enumeration` stops at index 12 and the parser reports the +one `None` terminator entry as missing. Every field that matters (edition, MOTD, protocol, version +name, player count, max players, uuid, game mode, both ports) parsed correctly. **Do not chase this +warning.** The failure signature is the *other* message, `Got timeout`. + +#### Decision: `transport=raknet` stays; the 19132-19134/UDP exposure is correct and live + +No manifest change is needed or wanted. `service-l4.yaml`, `networkpolicy.yaml` and the +`containerPort`s were already shaped for exactly RakNet, and that shape is now proven end-to-end +rather than assumed. The three options that would have required manifest work are all **not +recommended** today: + +| Option | Manifest consequence | Verdict | +|---|---|---| +| **Keep `transport=raknet`** (recommended) | **none** — the existing Service, policy and ports are already correct and now proven | **Take this.** Zero change, works today, vanilla parity, Bedrock clients join a real Bedrock world. | +| **GeyserMC + Floodgate** on the `homelab` Paper server | none either (Geyser also binds `19132/udp`), but it would **collide** with the live BDS on 19132 — one of them must move | Hold in reserve. Only worth it if RakNet is really removed, or if you *want* one shared world. Cost: Bedrock players join the **Java** world, not a Bedrock world; Java-only mechanics and some content differ; it is another moving part to patch. | +| **Pin/downgrade BDS** to a pre-nethernet release | none | Unnecessary — the *current* build already speaks RakNet. Also fragile: Microsoft removes old BDS downloads, and Crafty's Bedrock installer tracks latest. UNVERIFIED whether Crafty can pin a version at all. | +| **A third-party server** (PocketMine-MP, Nukkit) | none | No. Loses vanilla parity and the world format; a large change to solve a problem that is not occurring. | +| **Accept NetherNet** | 19132-19134/UDP becomes **inert**; nethernet would instead need **TCP** 19132 added to `crafty-l4-tcp` *and* to `crafty-allow-ingress`, plus `server-udp-ports` pinned and exposed for the negotiated media path | No. Strictly more manifest work than the option that already works, and the WebRTC media path behind a shared VIP and SNAT is unproven here. | + +#### If RakNet is ever really removed + +Then BDS opens no UDP listener and this exposure has nothing to bind to — an architecture problem, not +a config one. The migration is **GeyserMC + Floodgate as a plugin on the existing `homelab` Paper +server**, because Geyser speaks RakNet on `19132/udp` itself: the VIP, the Service, the policy and the +`containerPort`s all stay **byte-for-byte as they are**, and Bedrock clients keep connecting to +`crafty-mc.home.keithjackson.dev:19132`. What changes is the world: Bedrock players land in the Java +world via Floodgate (no Xbox-authenticated Bedrock account needed on the Java side), the Bedrock server +is retired, and `K3s-Bedrock` stops the day Geyser starts — **they cannot share 19132**. Installing it +is a PVC write plus a Paper restart, i.e. an **operator action**: drop `Geyser-Spigot.jar` and +`floodgate-spigot.jar` into `/crafty/servers/13c7b85d-…/plugins/` and restart `homelab`, then set +Geyser's `bedrock.port: 19132` and `clone-remote-port: false` in its generated `config.yml`. +Geyser's own Bedrock-version support is **external and unmeasured here** — check that it supports the +Bedrock client version in use before committing. ### Bedrock realities that are NOT platform defects but present identically @@ -141,14 +222,20 @@ shipped property documentation, not yet observed on this cluster — see the DEC - **Consoles cannot reach this at all.** Switch/Xbox/PlayStation Bedrock clients have no arbitrary-server-address field; only Windows/mobile/Fire do. Reaching it from a console needs a DNS redirect of a featured-server hostname, which is out of scope for `service-l4.yaml`. -- **`enable-lan-visibility=true` (the shipped default) collides with Bedrock slots 1 and 2.** Under - `transport=raknet` it makes a server bind the **default** ports 19132/19133 *even when `server-port` - is set to something else* — the shipped comment says exactly that. So a second Bedrock server put on - 19133 or 19134 will fight the first for 19132. Set `enable-lan-visibility=false` on every Bedrock - server after the first; LAN discovery cannot work here anyway (above). -- **`server-portv6` is not exposed and is not meant to be.** `service-l4.yaml` is IPv4 single-stack, - so the default `server-portv6=19133` is unreachable from the LAN. Harmless — but do not read - `19133` in `server.properties` as "slot 1 is in use". +- **`enable-lan-visibility=true` (the shipped default) collides with Bedrock slots 1 and 2 — now + OBSERVED, not merely documented.** Under `transport=raknet` it makes a server bind the **default** + ports 19132 *and* 19133 even when `server-port`/`server-portv6` say otherwise; the shipped + `bedrock_server_how_to.html` says so, and on 2026-09-29 a RakNet ping to + `192.168.2.22:19133` returned **the same pong, from the same `server_guid`, as `:19132`** — one + server occupying two of the three Bedrock slots. `:19134` correctly timed out. So a second Bedrock + server put on 19133 will fight the first. Set `enable-lan-visibility=false` on every Bedrock server + after the first; LAN discovery cannot work here anyway (above). +- **`server-portv6` is the IPv6 listener and is not exposed; `19133` is nonetheless in use.** + `service-l4.yaml` is IPv4 single-stack, so the *IPv6* socket `…:4ABD` that BDS opens for + `server-portv6=19133` is unreachable from the LAN. But because `enable-lan-visibility=true` (above) + the server *also* answers on **IPv4 19133**, which the UDP Service does expose — measured. So + `19133` in `server.properties` genuinely does mean "slot 1 is taken", contrary to what this file + previously said. ## Policies @@ -173,11 +260,16 @@ Gateway VIP). The sidecar→Crafty hop is pod-local (`127.0.0.1`) and not subjec `/crafty/app/config` and `/crafty/servers` without EACCES. 3. **Server-create + game reach (Java):** in the Crafty UI create a Java server bound to `25565`; start it; connect a Minecraft client to `crafty-mc.home.keithjackson.dev:25565` (or - `192.168.1.55:25565`) from the LAN and confirm the world loads. -4. **Server-create + game reach (Bedrock):** creating and starting the server is **not sufficient** — - check `transport=raknet` and a live UDP listener on 19132 first (see *the transport trap* above), - then **Add Server** by hand in a Windows/mobile client at `crafty-mc.home.keithjackson.dev` port - `19132`. It will never show up in the LAN/Friends tab, and a console cannot reach it. + `192.168.2.22:25565`) from the LAN and confirm the world loads. +4. **Server-create + game reach (Bedrock):** creating and starting the server is **not sufficient**. + Check, in this order (see *the transport trap* above): `transport=raknet` in `server.properties`; + a live **UDP** listener (`grep 4ABC /proc/net/udp /proc/net/udp6` → a wildcard bind); and a real + pong from off-cluster. **Ignore the `TRANSPORT TYPE ERROR` banner in the console and ignore + Crafty's `too few fields` warning — both appear on a fully working server.** The authoritative + test is a RakNet Unconnected Ping to `192.168.2.22:19132` returning a `0x1c` packet whose MOTD + string starts `MCPE;`. Then **Add Server** by hand in a Windows/mobile client at + `crafty-mc.home.keithjackson.dev` port `19132`. It will never show up in the LAN/Friends tab, and a + console cannot reach it. ## Runtime state: the servers themselves are NOT in git @@ -188,10 +280,11 @@ As of 2026-09-29 **two** servers exist (both measured live, `select * from serve | `homelab` | `minecraft-java` (Paper 26.2, 2 GiB heap) | TCP 25565 | `13c7b85d-00bb-4a58-8c86-2433ca47891c` | **1** | | `K3s-Bedrock` | `minecraft-bedrock` | UDP 19132 | `c5837301-12b1-404e-b50e-70eafb37226b` | **0** ⚠ | -`K3s-Bedrock` was created 2026-09-29 20:51 and has `auto_start=0` **and** `crash_detection=0` — the -create-API trap documented at the bottom of this file, live again. **It will not come back after a pod -restart**, and merging any change to `deployment.yaml` restarts the pod (`strategy: Recreate`). PATCH -both to true and read them back. +`K3s-Bedrock` was created 2026-09-29 20:51 and **still** has `auto_start=0` **and** +`crash_detection=0` — re-measured 2026-09-29 21:41 straight out of `crafty.sqlite` (read-only), so the +create-API trap documented at the bottom of this file is still live and still unfixed. **It will not +come back after a pod restart**, and merging any change to `deployment.yaml` restarts the pod +(`strategy: Recreate`). PATCH both to true and read them back. `homelab` was created through Crafty's v2 API, and **Mojang's EULA () was accepted on the operator's behalf** to let it start; see the @@ -204,6 +297,10 @@ application state on the `crafty-data` Longhorn PVC, in two places, and **Flux c heap sizes, `auto_start` / `crash_detection`. - `/crafty/servers//` (`servers` subPath) — `paper.jar`, `eula.txt`, `server.properties`, and **the generated world**. The jar is re-downloadable; the world is not. + **`transport=raknet` lives here too.** It is the one setting that makes Bedrock reachable at all (see + *the transport trap*), it is not in git, Flux cannot restore it, and a freshly created or reinstalled + Bedrock server gets the vendor default `transport=nethernet` back — i.e. it silently regresses to + "nothing bound on 19132/udp". Re-apply and re-ping after any Bedrock server recreate or BDS update. So a rebuild from git alone yields a healthy but **empty** Crafty with nothing listening on 25565. Recreating a server by hand means: create it in the UI or API, accept the EULA again, and PATCH