home-network-k3s-impl/platform/crafty/service-l4.yaml

---
# ============================================================================================
# GAME-PORT L4 EXPOSURE — why the shared HTTPS Gateway is NOT enough (same shape as apps/vms/unifi).
# ============================================================================================
# The Minecraft servers Crafty creates listen on RAW TCP (Java) and UDP (Bedrock) game ports inside
# the crafty pod's network namespace. The Gateway API `shared` Gateway is an L7 HTTPS terminator and
# CANNOT express raw TCP on arbitrary ports or UDP at all. So the game ports are exposed as
# type=LoadBalancer Services whose VIP is pulled from the CiliumLoadBalancerIPPool `default-pool`
# (dev 192.168.1.53-.60; see CILIUM_LB_POOL_* in cluster-vars) and ARP-announced on the flat LAN by
# the existing CiliumL2AnnouncementPolicy
# `default-l2` (platform-config/cilium/*) — the SAME machinery the Gateway VIP and unifi already use.
# No new Cilium config is needed; these Services just consume the pool.
#
# THIS PATH IS NOT AUTH-GATED — and that is correct: Minecraft's Java (Mojang/Microsoft auth) and
# Bedrock (Xbox Live) protocols authenticate the PLAYER themselves. Only the Crafty WEB UI (admin
# control plane) is Keycloak-gated (via the oauth2-proxy sidecar); the game ports carry player traffic
# and must reach the server subprocesses directly.
#
# VIP: CRAFTY_L4_VIP (cluster-vars, R-ENV-02; dev 192.168.1.55) — the next free address in
# default-pool (dev: .53 = shared Gateway, .54 = unifi; verified free live 2026-08-28). PINNED via
# lbipam.cilium.io/ips so players have a stable target. The `crafty` Flux Kustomization substitutes it.
# ONE VIP, TWO SERVICES: a LoadBalancer Service is single-protocol-friendly, so TCP and UDP split into
# two Services sharing the IP via lbipam.cilium.io/sharing-key (same pattern as unifi).
#
# STARTER PORT SET (the operator has not created any servers yet): TCP 25565-25567 (three Java slots)
# + UDP 19132-19134 (three Bedrock slots). To add capacity: create the server in Crafty bound to a NEW
# port, then add that port to the matching Service below (and a containerPort in deployment.yaml for
# documentation) and to the game-port ingress CiliumNetworkPolicy (networkpolicy.yaml). K8s Services
# enumerate ports individually (no ranges), so add one `{ name, port, targetPort, protocol }` line each.
#
# external-dns (source: service) publishes the A record from the -tcp Service.
#
# -------------------------------------------------------------------------------------------------
# externalTrafficPolicy: Cluster — CORRECTED 2026-09-14 after an 8-DAY LAN OUTAGE.
# -------------------------------------------------------------------------------------------------
# This file used to say `Local`, justified as "preserves the LAN client source IP and pins the VIP to
# the node running the crafty pod (also what L2 announcement does)". THE SECOND HALF IS FALSE, and it
# is what took the Minecraft port off the LAN.
#
# L2 ANNOUNCEMENT DOES NOT FOLLOW THE BACKEND. Cilium's L2 announcer elects one lease holder per
# Service among the nodes matching the CiliumL2AnnouncementPolicy `default-l2` — which matches ALL
# linux nodes — and it does NOT filter that set by which nodes hold a backend. MEASURED on prod
# 2026-09-14: the crafty pod ran on k3s-prod-1, `cilium-l2announce-crafty-crafty-l4-udp` was held by
# k3s-prod-1 (so Bedrock/UDP happened to work) but `-tcp` was held by k3s-prod-2 — and with `Local` an
# announcing node holding no backend has nowhere to send the packet and answers ICMP port unreachable.
# 192.168.2.22:25565, the Java port, was dead from the LAN for up to 8 days. That the two protocols
# landed on different nodes, one working and one not, is the whole tell: the election is per-Service
# and random, not backend-aware. It also does NOT self-heal — the holder keeps RENEWING after the pod
# moves, so only deleting the Lease forces a (still random) re-election. See
# apps/wave-0/pihole-dns.yaml, which hit this first and is the reference implementation.
#
# `Cluster` makes EVERY node able to serve the VIP by forwarding to wherever the pod is, which is what
# makes this survive a reschedule and a rebuild.
#
# NO CHANGE IS NEEDED IN networkpolicy.yaml, AND THAT WAS VERIFIED RATHER THAN ASSUMED. Under
# `Cluster` the forwarded second hop is sourced from the ingress node and carries Cilium's reserved
# `remote-node` identity. pihole-dns.yaml has to name `remote-node` explicitly in its
# CiliumNetworkPolicy — but that is because its policy deliberately omits `cluster`.
# crafty-allow-ingress admits `cluster`, and Cilium's `cluster` entity SUBSUMES `remote-node` (along
# with host/init/unmanaged). Confirmed on prod 2026-09-14 by dumping the crafty endpoint's datapath
# policy map (`cilium-dbg bpf policy get`) while the CNP was in its unmodified state:
# `reserved:remote-node` was already an Allow on 25565-25567/TCP and 19132-19134/UDP. Adding
# `remote-node` to that rule would be a no-op, so it is deliberately NOT added here — do not "fix" it.
#
# THE CLIENT-IP COST, STATED PLAINLY AND ACCEPTED. Cilium runs in SNAT mode here (no
# `loadBalancer.mode: dsr` in platform/cilium/helmrelease.yaml), so unlike UniFi — a KubeVirt VM whose
# `masquerade` binding already destroyed the client address below the Service — crafty is a plain pod
# and DOES lose a real client IP it was previously receiving. Every player will appear to the server
# subprocesses as the ingress node's address. What that actually costs here:
#   * PLAYER IDENTITY IS UNAFFECTED. Minecraft Java authenticates against Mojang/Microsoft and Bedrock
#     against Xbox Live; a player is a UUID, not an address. Vanilla `/ban` and the whitelist are
#     UUID-keyed (banned-players.json, whitelist.json) and keep working exactly as before.
#   * `/ban-ip` (banned-ips.json) DEGRADES, and would degrade dangerously: with every player behind one
#     apparent address, an IP ban would hit EVERY player at once. Use `/ban` (by player), not
#     `/ban-ip`, on this cluster. No IP ban is configured today.
#   * Per-IP rate limiting / connection throttling degrades the same way — the whole LAN counts as one
#     client. Nothing in platform/crafty configures any, and Crafty's own web UI is a separate,
#     Keycloak-gated path that does not ride these Services.
# This is a real cost and it is worth paying: the alternative on offer is a Java port that is dead
# outright, and the only fix that keeps BOTH the client IP and a working VIP is `loadBalancer.mode:
# dsr` on the Cilium HelmRelease — a platform-wide change with an Ansible [parity] contract that rolls
# the CNI across all three nodes, which belongs in its own change, not in an outage fix.
#
# NO loadBalancerSourceRanges HERE, DELIBERATELY — this is where this file departs from
# pihole-dns.yaml. PiHole added `loadBalancerSourceRanges: [${LAN_CIDR}]` because it was giving up a
# restriction it had and genuinely needed to be LAN-only. These Services never had a source
# restriction at all (crafty-allow-ingress admits `world` on the game ports, and the block above is
# explicit that player auth — not network position — is the control). `Cluster` therefore removes no
# protection and there is nothing to compensate for. Pinning the game ports to ${LAN_CIDR} during an
# outage fix would be a NEW restriction that would cut off any player reaching these ports from off
# the LAN, trading this outage for a subtler one.
apiVersion: v1
kind: Service
metadata:
  name: crafty-l4-tcp
  namespace: crafty
  labels:
    app.kubernetes.io/name: crafty
  annotations:
    lbipam.cilium.io/sharing-key: "crafty"
    lbipam.cilium.io/ips: "${CRAFTY_L4_VIP}"
    # external-dns publishes A crafty-mc.home.keithjackson.dev -> the pinned VIP (Java server target).
    external-dns.alpha.kubernetes.io/hostname: "crafty-mc.${BASE_DOMAIN}"
spec:
  type: LoadBalancer
  externalTrafficPolicy: Cluster
  selector:
    app.kubernetes.io/name: crafty
  ports:
    - { name: mc-java-0, port: 25565, targetPort: 25565, protocol: TCP }
    - { name: mc-java-1, port: 25566, targetPort: 25566, protocol: TCP }
    - { name: mc-java-2, port: 25567, targetPort: 25567, protocol: TCP }
---
apiVersion: v1
kind: Service
metadata:
  name: crafty-l4-udp
  namespace: crafty
  labels:
    app.kubernetes.io/name: crafty
  annotations:
    lbipam.cilium.io/sharing-key: "crafty"    # same key + same IP => shares the VIP with -tcp
    lbipam.cilium.io/ips: "${CRAFTY_L4_VIP}"
spec:
  type: LoadBalancer
  externalTrafficPolicy: Cluster
  selector:
    app.kubernetes.io/name: crafty
  ports:
    - { name: mc-bedr-0, port: 19132, targetPort: 19132, protocol: UDP }
    - { name: mc-bedr-1, port: 19133, targetPort: 19133, protocol: UDP }
    - { name: mc-bedr-2, port: 19134, targetPort: 19134, protocol: UDP }