From c59a4b76bb0b7015104f22597e93d7460aedd72f Mon Sep 17 00:00:00 2001 From: Keith Jackson Date: Tue, 29 Sep 2026 22:08:54 -0500 Subject: [PATCH] docs(crafty): a player has joined -- the allowlist chain is closed, recommendation now PROVEN Operator added one gamertag and the Bedrock client connected and joined. The three-pass investigation closes with a working server, no manifest change, no transport change, no network change. Moved out of UNVERIFIED: * A successful join -- OPERATOR-REPORTED (I did not watch it; the distinction is kept deliberately). Corroborated directly on disk, which is stronger than the report alone: allowlist.json went [] -> one entry, 75 bytes, and that entry's xuid is POPULATED (mtime 22:03:17). audit.log shows the operator wrote only the name (Edited file .../allowlist.json, 22:02:13). Vendor doc: xuid "will be populated when someone with a matching name connects". So the server wrote the XUID on connect, ~64s after the edit -- server-authored evidence of an XBL-authenticated join. * The reasoned mapping of "You are not invited" to the empty allowlist is CONFIRMED: one gamertag added, one successful join. allow-list=true with allowlist.json == [] was the wall, and nothing else was. * transport=raknet survives a restart: the fix involved one (pid 47006 -> 47875); afterwards server.properties mtime is still 21:33:31 with transport=raknet, UDP 4ABC bound, still NO TCP 4ABC, and a fresh pong from 192.168.2.22:19132 at 22:06. * The operator's actual route was file-edit + restart (route 2), not the console `allowlist add` I listed first. Both proven; console still preferred (no restart). README: join-gate section marked RESOLVED; decision heading and option 1 promoted from provisional to PROVEN; Bedrock human-test 4 rewritten as a 5-step verified connect procedure, each step with its distinct failure signature, stating plainly that the TRANSPORT TYPE ERROR banner and the "too few fields" warning BOTH appear on a fully working server -- that pair misled two consecutive sessions. NOT weakened, and sharpened where possible: * auto_start=0 / crash_detection=0 STILL 0, re-measured at 22:06 after the join and after the operator's 22:02:28 "modified the server" action, which did not change them. The failure mode is now worse, not better: "worked, then vanished". Merging fix/crafty-oauth2-proxy-signin is such a restart. * Silent-reset now precisely characterised: a RESTART preserves both files (observed) but leaves the server down (auto_start=0); a server RECREATE or reinstall silently resets server.properties to transport=nethernet AND allowlist.json to []. * L2/L3 conformance still uncertified; multi-player SNAT still untested (exactly one player has joined -- that is precisely the evidence that does not settle it); BDS console output still never seen (log_path is ''). No manifest touched, in any of the three passes. Read-only throughout. Co-Authored-By: Claude Opus 5 (1M context) --- DECISIONS-LOG.md | 98 ++++++++++++++++++++++++++++++++++++ platform/crafty/README.md | 103 +++++++++++++++++++++++++++++--------- 2 files changed, 176 insertions(+), 25 deletions(-) diff --git a/DECISIONS-LOG.md b/DECISIONS-LOG.md index fe46e6a..f04a856 100644 --- a/DECISIONS-LOG.md +++ b/DECISIONS-LOG.md @@ -5620,3 +5620,101 @@ and human test 4 to put the allowlist check before blaming the network. **No man /api/v2/servers/c5837301-12b1-404e-b50e-70eafb37226b`, both true, **read them back**. 6. **Treat `allowlist.json` and `transport=raknet` as backup-relevant PVC state.** A server recreate resets the allowlist to `[]` and the transport to `nethernet` — both silently. + +## 2026-09-29 (closing) — a player has joined. The chain is closed: `allow-list=true` + an empty allowlist was the wall, and nothing else was + +The operator added one gamertag to the allowlist and the Bedrock client connected and joined. **The +three-entry investigation above is now closed by a working server**, with no manifest change, no +transport change, and no network change. + +### What is now OBSERVED (moved out of UNVERIFIED) + +- **A successful join.** *Operator-reported, 2026-09-29* — they added the player's gamertag and the + client connected and joined. **I did not watch the join**, and that distinction is kept deliberately; + this repo has been burned by collapsing it. +- **Corroborated directly on disk, though**, which is stronger than the report alone: + `allowlist.json` went `[]` → **one entry**, 75 bytes, and that entry's **`xuid` is populated** + (`keys=['ignoresPlayerLimit','name','xuid']`, mtime **22:03:17**). `audit.log` shows the operator wrote + only the name — `Edited file …/allowlist.json` at **22:02:13**. The vendor doc: `xuid` is *"Optional. + If it's not set then it will be populated when someone with a matching name **connects**."* So the XUID + was written **by the server, on connect**, ~64 s after the operator's edit. That is server-authored + evidence of an Xbox-Live-authenticated join. *(Observed: file state, both timestamps, the audit line. + Reasoned from the vendor doc: populated XUID ⇒ a connect happened.)* +- **The reasoned mapping is confirmed.** The client's *"You are not invited to play on this server"* was + mapped to the empty allowlist **by reasoning** from the vendor doc, because Crafty persists no BDS + stdout and there was no log to read it from. Adding **exactly one** gamertag and getting **exactly one** + successful join confirms that mapping. **`allow-list=true` + `allowlist.json == []` was the wall, and + nothing else was.** +- **`transport=raknet` survives a restart.** The fix involved one: `stop_server` 22:02:37 → + `start_server` 22:02:41, pid **47006 → 47875**. Afterwards `server.properties` mtime is **still + 21:33:31** with `transport=raknet`, UDP `4ABC` is bound by the new pid, there is still **no TCP 4ABC**, + and `192.168.2.22:19132` answered a fresh pong at 22:06 (`MCPE;…;1.26.52;0;10;…`, new `server_guid` + matching the new pid, `players=0` — the player had left). +- **The route the operator used was file-edit + restart**, not the console `allowlist add` I put first: + `audit.log` edit 22:02:13 → stop → start. A restart is a valid substitute for `allowlist reload`. + Both routes now documented as proven; the console route is still preferred because it needs no restart. + +### The recommendation is now PROVEN, not provisional + +**Keep `transport=raknet` and enumerate players in the allowlist. Zero manifest change.** This is no +longer the best-reasoned option — it is the configuration a player has demonstrably joined over +`192.168.2.22:19132`. `service-l4.yaml`, `networkpolicy.yaml` and the `containerPort`s needed nothing. +Ranking otherwise unchanged: Geyser #2 (reserve, and it cannot share 19132), pinning BDS #3, +PocketMine/Nukkit #4, accept-NetherNet last — NetherNet is still **not in use**, would still need TCP +19132 exposed in two places, and would still admit not one extra player. + +### Still open — NOT weakened by the success + +- **`K3s-Bedrock` still has `auto_start=0` and `crash_detection=0`.** Re-measured from `crafty.sqlite` at + 22:06, *after* the successful join and after the operator's 22:02:28 `modified the server` action — + which did **not** change them. **The operator now has a working server that will silently not come back + after any pod restart**, and merging `fix/crafty-oauth2-proxy-signin` **is** such a restart + (`strategy: Recreate`, RWO volume) and would stop **both** Minecraft servers. This matters *more* now, + not less: the failure mode has changed from "never worked" to "worked, then vanished". +- **The silent-reset exposure is unchanged, and now precisely characterised:** + a **restart** preserves `transport=raknet` and `allowlist.json` (observed above) but does not restart + the server itself (`auto_start=0`); a **server recreate or reinstall** resets `server.properties` to + `transport=nethernet` (nothing bound on 19132/udp) **and** `allowlist.json` to `[]` (everyone refused). + Neither reset announces itself, and the panel will show the server healthy and started. Both files are + PVC state Flux cannot restore. **Re-run the 5-step connect procedure after any recreate or BDS update.** +- **L2/L3 conformance is still uncertified.** `CLUSTER_AVAILABLE` is unset, so the gate's `conformance` + stage does not run; `R-DNS-01 R-NET-01 R-PLAT-02 R-STATE-01` remain UNVERIFIED in CI. A player joining + is not a conformance test. +- **Multi-player SNAT/UDP-conntrack is still untested.** Exactly **one** player has joined. Whether a + *second simultaneous* Bedrock player evicts the first behind the shared VIP and SNAT is unmeasured, and + a single successful session is precisely the evidence that does *not* settle it. +- **I still never saw BDS's own console output** — `log_path` is `''`, `Dedicated_Server.txt` is 0 bytes. + Anything BDS itself said about the join is visible only live in the panel. +- **Consoles still cannot reach this**, and LAN/Friends discovery still cannot work behind the VIP. +- **Geyser remains entirely unmeasured**; the pinnability of a BDS version remains UNVERIFIED. + +### What changed in this repo + +`platform/crafty/README.md` and `DECISIONS-LOG.md`. **No manifest touched, in any of the three passes.** + +- Marked the join-gate section **RESOLVED and confirmed**, with the XUID-population evidence and the + operator-reported join labelled as exactly that. +- Promoted the decision heading and option-1 row from provisional to **PROVEN**, and recorded that the + operator's actual route was file-edit + restart (route 2), with route 1 still preferred. +- Rewrote Bedrock human-test 4 as a **5-step verified connect procedure**, each step with its distinct + failure signature, and stated plainly that the `TRANSPORT TYPE ERROR` banner and the `too few fields` + warning both appear on a fully working server — that pair misled two consecutive sessions. +- Sharpened the PVC warning to distinguish **restart** (preserves both files; server stays down because + `auto_start=0`) from **recreate/reinstall** (silently resets transport *and* allowlist). + +### Operator actions — one is now urgent + +1. **`PATCH /api/v2/servers/c5837301-12b1-404e-b50e-70eafb37226b`** setting `auto_start` **and** + `crash_detection` true, **and read them back**. Both are still `0`. This is the only thing standing + between the working server and it disappearing at the next pod restart — including the restart that + merging `fix/crafty-oauth2-proxy-signin` will cause. +2. **Leave `transport=raknet` and the allowlist alone.** Keep ignoring the `TRANSPORT TYPE ERROR` banner + and the `too few fields` warning. +3. **Add a gamertag per additional player** — console `allowlist add ` (no restart needed). + Do **not** switch to `allow-list=false`: the game port is un-gated by design, so the allowlist is the + only join control. +4. **After any Bedrock recreate or BDS update**, re-run the 5-step connect procedure — transport and + allowlist both reset silently. +5. **Before announcing this to more than one player**, note that only one simultaneous session has been + proven; if a second player evicts the first, that is the SNAT/conntrack question and needs its own + measured investigation. diff --git a/platform/crafty/README.md b/platform/crafty/README.md index f3c368e..37b07f7 100644 --- a/platform/crafty/README.md +++ b/platform/crafty/README.md @@ -181,7 +181,7 @@ one `None` terminator entry as missing. Every field that matters (edition, MOTD, name, player count, max players, uuid, game mode, both ports) parsed correctly. **Do not chase this warning.** The failure signature is the *other* message, `Got timeout`. -### "You are not invited to play on this server" — the join gate is the allowlist, not the network (MEASURED 2026-09-29 21:56) +### "You are not invited to play on this server" — the join gate is the allowlist, not the network (MEASURED 2026-09-29 21:56; **RESOLVED and confirmed 22:03**) A Bedrock client reported **"Unable to connect / You are not invited to play on this server"**. That is an **authorization** refusal, and it is the expected, guaranteed behaviour of this server's current @@ -202,6 +202,34 @@ then the server will only allow selected users to connect. To allow a user to co their Xbox Live **Gamertag**."* An empty list selects nobody. This is not a network problem, not a transport problem, and not a platform problem — and no amount of Service or policy work will change it. +#### CONFIRMED: one gamertag added, and a player joined + +**This is no longer a hypothesis. The fix was applied and it worked.** *(The join itself is +**operator-reported** — 2026-09-29, the operator added the player's gamertag and the Bedrock client +connected and joined. I did not watch it.)* Two independent things corroborate it **directly on disk**: + +- `allowlist.json` went from `[]` to **one entry**, and that entry's **`xuid` is populated** + (`keys=['ignoresPlayerLimit','name','xuid']`, file 75 bytes, mtime **22:03:17**). Per the vendor doc, + `xuid` is *"Optional. If it's not set then it will be populated when someone with a matching name + **connects**."* The operator wrote only the name — `audit.log` records `Edited file + …/allowlist.json` at **22:02:13** — so **the XUID was written by the server, on connect, at 22:03:17**. + That is server-authored evidence of an Xbox-Live-authenticated join. *(Observed: the file state and + both timestamps. Reasoned from the vendor doc: that a populated XUID implies a connect.)* +- The RakNet path still answers after the restart this involved: a pong from `192.168.2.22:19132` at + 22:06, `MCPE;…;1.26.52;0;10;…` (`players=0` — the player had left by then), new `server_guid` matching + the new pid. + +**So the causal chain is closed: `allow-list=true` + `allowlist.json == []` was the wall, and nothing +else was.** The earlier mapping of the client's *"You are not invited"* string to the empty allowlist was +**reasoned** from the vendor doc (there is no BDS log on disk to read it from); adding exactly one +gamertag and getting exactly one successful join **confirms that mapping**. No transport change, no +manifest change, and no network change was involved in the fix. + +> **Note the route the operator actually used:** they **edited `allowlist.json` directly** and then +> **restarted** the server (`audit.log`: edit 22:02:13 → `stop_server` 22:02:37 → `start_server` +> 22:02:41; pid 47006 → **47875**). That is route 2 below, and a restart is a valid substitute for +> `allowlist reload`. Route 1 (`allowlist add` in the console) avoids the restart and is still preferred. + **This is also positive evidence that the RakNet path works end to end.** A refusal authored by *our* server required the client to reach our server and get an answer back. The only listener on 19132 is the **RakNet UDP** socket — there is **no TCP 19132 listener at all**, and NetherNet requires a TCP @@ -230,7 +258,8 @@ gamertag), optional `xuid` (*"populated when someone with a matching name connec group, role, or Keycloak integration, and **none of this touches the oauth2-proxy/Keycloak gate**, which protects only the web panel. -Three ways to do it, all **operator actions**: +Three ways to do it, all **operator actions**. **Routes 1 and 2 are both proven** — route 2 is the one +that produced the confirmed join above: 1. **Preferred — the panel console.** In Crafty, open `K3s-Bedrock` → Console and send: `allowlist add ` (quote it if it contains a space: `allowlist add "Example Name"`). @@ -253,27 +282,40 @@ Three ways to do it, all **operator actions**: exposure*), so the allowlist is the **only** thing deciding who may join. Keep it `true` and enumerate players. -> **`allowlist.json` is application state on the PVC.** It is not in git, Flux cannot restore it, and a -> server recreate resets it to `[]` — i.e. back to refusing everyone. Same class of problem as -> `transport=raknet`. Treat both as backup-relevant. +> **`allowlist.json` is application state on the PVC**, exactly like `transport=raknet`. Neither is in +> git and **Flux can restore neither**. Be precise about what breaks them, because the two cases differ: +> +> - **A pod or server *restart* preserves both** — observed: the 22:02:48 restart (pid 47006 → 47875) +> kept `transport=raknet` (`server.properties` mtime unchanged at 21:33:31) and kept the allowlist +> entry. What a **pod** restart *does* break is that `K3s-Bedrock` has `auto_start=0`, so the Bedrock +> server simply **does not come back** until someone starts it by hand. +> - **A server *recreate* or reinstall silently resets both** — a fresh `server.properties` ships +> `transport=nethernet` (nothing bound on 19132/udp) and a fresh `allowlist.json` is `[]` (everyone +> refused). Neither change announces itself, and the server will look healthy and started in the panel +> while being unreachable *and* unjoinable. +> +> **Treat both files as backup-relevant, and re-run the 5-step connect procedure above after any Bedrock +> recreate or BDS update.** #### Why this does not change the transport recommendation -`transport=raknet` is still correct and still the recommendation. The allowlist refusal is orthogonal — -it would refuse players identically under NetherNet. Switching to NetherNet would *add* work (TCP 19132 -in `crafty-l4-tcp` **and** in `crafty-allow-ingress`, plus `server-udp-ports` pinned and exposed) and -would not admit a single extra player. The ranking below stands, with the allowlist as the live blocker. +`transport=raknet` is correct, and with the allowlist entry in place the combination is **proven +end-to-end by a player who has joined** — not merely the best-reasoned option. The allowlist refusal was +orthogonal to transport all along: it would have refused players identically under NetherNet. Switching +to NetherNet would *add* work (TCP 19132 in `crafty-l4-tcp` **and** in `crafty-allow-ingress`, plus +`server-udp-ports` pinned and exposed) and would admit **not one extra player**. The ranking below stands, +and option 1 is now settled rather than provisional. -#### Decision: `transport=raknet` stays; the 19132-19134/UDP exposure is correct and live +#### Decision (PROVEN, not provisional): `transport=raknet` stays; the 19132-19134/UDP exposure is correct and live No manifest change is needed or wanted. `service-l4.yaml`, `networkpolicy.yaml` and the -`containerPort`s were already shaped for exactly RakNet, and that shape is now proven end-to-end -rather than assumed. The three options that would have required manifest work are all **not -recommended** today: +`containerPort`s were already shaped for exactly RakNet, and that shape is now **proven by a player who +has connected and joined over `192.168.2.22:19132`** — not assumed, and not merely reasoned. The options +that would have required manifest work are all **not recommended**: | Option | Manifest consequence | Verdict | |---|---|---| -| **Keep `transport=raknet`** (recommended) | **none** — the existing Service, policy and ports are already correct and now proven end-to-end | **Take this.** Zero change, reachable today, vanilla parity, Bedrock clients join a real Bedrock world. The remaining blocker is the **allowlist**, not the transport (above). | +| **Keep `transport=raknet`** + allowlist entries (**this is what works**) | **none** — the existing Service, policy and ports were already correct | **Done; proven.** A player has joined over the VIP. Zero manifest change, vanilla parity, real Bedrock world. | | **GeyserMC + Floodgate** on the `homelab` Paper server | none either (Geyser also binds `19132/udp`), but it would **collide** with the live BDS on 19132 — one of them must move | Hold in reserve. Only worth it if RakNet is really removed, or if you *want* one shared world. Cost: Bedrock players join the **Java** world, not a Bedrock world; Java-only mechanics and some content differ; it is another moving part to patch. | | **Pin/downgrade BDS** to a pre-nethernet release | none | Unnecessary — the *current* build already speaks RakNet. Also fragile: Microsoft removes old BDS downloads, and Crafty's Bedrock installer tracks latest. UNVERIFIED whether Crafty can pin a version at all. | | **A third-party server** (PocketMine-MP, Nukkit) | none | No. Loses vanilla parity and the world format; a large change to solve a problem that is not occurring. | @@ -344,17 +386,28 @@ Gateway VIP). The sidecar→Crafty hop is pod-local (`127.0.0.1`) and not subjec 3. **Server-create + game reach (Java):** in the Crafty UI create a Java server bound to `25565`; start it; connect a Minecraft client to `crafty-mc.home.keithjackson.dev:25565` (or `192.168.2.22:25565`) from the LAN and confirm the world loads. -4. **Server-create + game reach (Bedrock):** creating and starting the server is **not sufficient**. - Check, in this order (see *the transport trap* above): `transport=raknet` in `server.properties`; - a live **UDP** listener (`grep 4ABC /proc/net/udp /proc/net/udp6` → a wildcard bind); and a real - pong from off-cluster. **Ignore the `TRANSPORT TYPE ERROR` banner in the console and ignore - Crafty's `too few fields` warning — both appear on a fully working server.** The authoritative - test is a RakNet Unconnected Ping to `192.168.2.22:19132` returning a `0x1c` packet whose MOTD - string starts `MCPE;`. **Then check the allowlist** — `allow-list=true` with `allowlist.json == []` - refuses every player with *"You are not invited to play on this server"* even though the server is - fully reachable (see *the join gate* above); add the player's **Xbox Gamertag** first. Then - **Add Server** by hand in a Windows/mobile client at `crafty-mc.home.keithjackson.dev` port `19132`. - It will never show up in the LAN/Friends tab, and a console cannot reach it. +4. **Bedrock: the connect procedure (VERIFIED end-to-end 2026-09-29 — a player has joined).** + Creating and starting the server is **not sufficient**. Do these in order; each step has a distinct + failure signature: + + 1. **`transport=raknet`** in `server.properties`. The vendor default is `nethernet`, which opens + **no UDP listener at all**. + 2. **A live UDP listener:** `grep 4ABC /proc/net/udp /proc/net/udp6` → a wildcard bind. Under + `nethernet` you instead see `4ABC` on **TCP**, which nothing here exposes. + 3. **A pong from off-cluster:** a RakNet Unconnected Ping to `192.168.2.22:19132` returns a `0x1c` + packet whose MOTD string starts `MCPE;`. This proves the VIP, Service and policy path. + 4. **An allowlist entry for the player's Xbox Gamertag.** `allow-list=true` with + `allowlist.json == []` refuses **every** player with *"You are not invited to play on this + server"* while the server is otherwise perfectly reachable. In the panel console: + `allowlist add `. + 5. **Add Server** by hand in a Windows/mobile client: `crafty-mc.home.keithjackson.dev`, port + `19132`. It will never appear in the LAN/Friends tab, and a console cannot reach it at all. + + **Two log messages appear on a fully working server and must be ignored:** BDS's + `================ TRANSPORT TYPE ERROR ===================` banner (advisory — see *Trap 2*), and + Crafty's perpetual `Bedrock ping returned a string with too few fields`. Neither indicates a fault; + chasing them is what cost two sessions. The **real** failure signatures are `Got timeout while + issuing bedrock ping` (no listener) and *"You are not invited"* (allowlist). ## Runtime state: the servers themselves are NOT in git