feat(vpn-ingress): WireGuard remote access on udp/51820 (hostNetwork, declarative peers)
Lets the operator's Android / Linux / Windows / macOS devices dial in from anywhere and behave like LAN machines: full reach to 192.168.2.0/24 plus internet egress out the house WAN. Opposite direction to platform/vpn-egress, which sends pods OUT through PIA; the two share nothing but the word "VPN". WireGuard because it is the only option with a first-party client on all four named platforms, and because its handshake MAC is keyed with the responder's own public key: an unauthenticated packet gets NO reply, so udp/51820 is indistinguishable from a closed port. That, not "it is encrypted", is what makes an unfiltered WAN port defensible, and why there is no source allowlist. hostNetwork: true is REQUIRED, not convenient. A pod-networked server must SNAT clients to its own pod IP, so every remote client would carry that pod's Cilium identity -- and a pod-sourced request to the shared Gateway VIP returns 403 from Envoy for ALL 16 hostnames (measured 2026-10-04), because Cilium's Gateway preserves the original source identity and enforces the destination's ingress policy. The obvious build would have shipped a healthy-looking tunnel on which every cluster web UI was 403 and *.home.keithjackson.dev did not even resolve. In the host netns traffic is SNATed to the node and arrives as host/remote-node, exactly like a LAN client. It also sidesteps the unsafe ip_forward sysctl, the L2-announcement lease lottery, and double encapsulation (hence MTU 1420). Permitted by restrict-host-namespaces' own carve-out for platform namespaces. The cost is stated in networkpolicy.yaml rather than papered over: no NetworkPolicy constrains a hostNetwork pod. A DaemonSet so the hand-made WAN forward cannot be invalidated by a reschedule; all three nodes answer the same keys, so failover is a one-line router change. Peer PUBLIC keys are plaintext in peers.yaml, which makes "who can get into my house" a reviewable PR diff. The only secret -- the server's private key -- is an out-of-band Secret: an agent must not mint a long-lived private key, and the two halves are joined only at pod start into a tmpfs that dies with the pod. Ships INERT-BUT-HEALTHY with zero peers. THREE off-repo manual steps remain (kernel module, server key, WAN forward) -- runbooks/vpn-remote-access.md. Touches clusters/** , so ci/approval-policy.sh will block pending the operator's /approve-override. Deliberately not posted by me (CLAUDE.md §1). Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
jakman85andClaude Opus 5 committed
1 parent
22c94effff
commit
1a67fc1b3c
16 files changed
+1391
-1
No files matched your search
@@ -74,12 +74,23 @@ node_baseline_backports_kernel_packages:
|
||||
# kernel, and how do I undo it?" from the machine itself.
|
||||
node_baseline_backports_kernel_stamp: /etc/node-baseline-backports-kernel
|
||||
|
||||
# Modules written to /etc/modules-load.d/k3s-cilium-longhorn.conf (persistent) AND modprobe'd now.
|
||||
node_baseline_kernel_modules:
|
||||
- overlay
|
||||
- br_netfilter
|
||||
- iscsi_tcp
|
||||
- dm_crypt
|
||||
- nfs
|
||||
# Remote-access VPN (platform/vpn-ingress). The WireGuard DaemonSet creates its wg0 link in the
|
||||
# HOST netns with CAP_NET_ADMIN only -- it is deliberately NOT granted CAP_SYS_MODULE, so it
|
||||
# cannot autoload this itself: `ip link add wg0 type wireguard` fails outright when the module is
|
||||
# absent, and the pod CrashLoops. Loading it here is therefore a PREREQUISITE of that component,
|
||||
# not a nicety. In-tree since kernel 5.6, so Debian 13's 7.1.8 ships it and there is nothing to
|
||||
# install. Naming it here also means the kernel does the crypto: without the module a WireGuard
|
||||
# container's only fallback is slow userspace wireguard-go, which we do not use (and which would
|
||||
# need /dev/net/tun -- a device a hostPath mount cannot actually grant, as the /dev/dri attempt
|
||||
# proved: visible but EPERM from the device cgroup).
|
||||
- wireguard
|
||||
|
||||
# --- DATA DISKS (tasks/disks.yml, tag `disks`) ----------------------------------------
|
||||
# The SHAPE of a node's storage is DECLARED per environment, never inferred from whether
|
||||
|
||||
Executable
+195
@@ -0,0 +1,195 @@
|
||||
#!/usr/bin/env bash
|
||||
# =================================================================================================
|
||||
# new-peer.sh — enrol one device on the remote-access VPN (platform/vpn-ingress).
|
||||
#
|
||||
# Generates a WireGuard keypair ON THIS MACHINE, renders the device's client profile, optionally
|
||||
# prints a QR code for a phone, and prints the one `[Peer]` stanza to paste into
|
||||
# platform/vpn-ingress/peers.yaml. The private half NEVER leaves this machine: it is not sent to the
|
||||
# cluster, not committed, and not written to any log.
|
||||
#
|
||||
# WHY A SCRIPT AND NOT A RUNBOOK STEP. Three of these fields are easy to get wrong in ways that are
|
||||
# either silently insecure or silently broken, and all three are mechanical:
|
||||
# * AllowedIPs on the SERVER side must be a /32. On the server, AllowedIPs is simultaneously the
|
||||
# cryptokey routing table AND the inbound source filter, so a /24 here lets one device spoof
|
||||
# every other device's tunnel address.
|
||||
# * PersistentKeepalive must be set on the CLIENT. Without it the NAT mapping at the client's end
|
||||
# expires after ~30s of idle and nothing can reach the device until it speaks first — which
|
||||
# presents as "the VPN works for a minute and then half-works".
|
||||
# * The endpoint must be the DDNS name, never the current WAN IP (see --endpoint below).
|
||||
#
|
||||
# USAGE
|
||||
# ./infra/vpn-ingress/new-peer.sh <device-name> <last-octet> [--split] [--out DIR]
|
||||
#
|
||||
# <device-name> short, lowercase, e.g. pixel-8, thinkpad, work-laptop. Used for the filename
|
||||
# and the comment that ties the key to a human-recognisable device.
|
||||
# <last-octet> 2-254, the device's address inside WG_CLIENT_CIDR. Must be unused.
|
||||
# --split split tunnel: route ONLY the LAN and the tunnel subnet. Default is a FULL
|
||||
# tunnel (0.0.0.0/0) so the device also egresses the house WAN.
|
||||
# --out DIR where to write the profile (default: a 0700 dir under $TMPDIR).
|
||||
#
|
||||
# ENVIRONMENT
|
||||
# WG_SERVER_KEY_FILE server PRIVATE key (default ~/.config/wireguard-home/server.key); its public
|
||||
# half is derived from it. Alternatively set WG_SERVER_PUBKEY directly.
|
||||
# WG_ENDPOINT_HOST default wan.home.keithjackson.dev
|
||||
# WG_CLIENT_CIDR default 10.90.0.0/24 (must match clusters/base/cluster-vars.yaml)
|
||||
# WG_LAN_CIDR default 192.168.2.0/24 (prod LAN_CIDR)
|
||||
# WG_DNS default 192.168.2.23 (prod PIHOLE_DNS_VIP)
|
||||
#
|
||||
# REQUIRES: wireguard-tools (`wg`). `qrencode` is optional and only used for the QR code.
|
||||
# =================================================================================================
|
||||
set -euo pipefail
|
||||
|
||||
die() { printf 'new-peer.sh: %s\n' "$*" >&2; exit 1; }
|
||||
|
||||
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
PEERS_FILE="${REPO_ROOT}/platform/vpn-ingress/peers.yaml"
|
||||
|
||||
# --- defaults, each mirroring a value that is DECLARED elsewhere in this repo -------------------
|
||||
# These are duplicated here rather than parsed out of the YAML on purpose: this script is meant to
|
||||
# run on an operator's laptop with no checkout-shaped assumptions beyond its own location, and a
|
||||
# silent mis-parse would be worse than a visible default. If you change the knob, change it here.
|
||||
: "${WG_CLIENT_CIDR:=10.90.0.0/24}" # clusters/base/cluster-vars.yaml
|
||||
: "${WG_LAN_CIDR:=192.168.2.0/24}" # clusters/prod LAN_CIDR
|
||||
: "${WG_DNS:=192.168.2.23}" # clusters/prod PIHOLE_DNS_VIP
|
||||
: "${WG_SERVER_KEY_FILE:=${HOME}/.config/wireguard-home/server.key}"
|
||||
# wan.<base-domain> is an UNPROXIED A record that platform/externaldns/ddns.yaml keeps pointed at
|
||||
# the current WAN IP (checked every 5 min). Use it, never a literal address: this house has a
|
||||
# DYNAMIC WAN IP, and a profile pinned to today's address stops working without warning and with no
|
||||
# symptom on the server side. It is a PUBLIC Cloudflare record, so it also resolves from a cellular
|
||||
# connection — unlike any `*.home.keithjackson.dev` name PiHole answers, which a remote client
|
||||
# cannot resolve until it is already connected.
|
||||
: "${WG_ENDPOINT_HOST:=wan.home.keithjackson.dev}"
|
||||
: "${WG_LISTEN_PORT:=51820}"
|
||||
: "${WG_MTU:=1420}"
|
||||
|
||||
SPLIT=0
|
||||
OUT_DIR=""
|
||||
NAME=""
|
||||
OCTET=""
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--split) SPLIT=1; shift ;;
|
||||
--out) OUT_DIR="${2:-}"; [ -n "$OUT_DIR" ] || die "--out needs a directory"; shift 2 ;;
|
||||
-h|--help) sed -n '2,40p' "${BASH_SOURCE[0]}"; exit 0 ;;
|
||||
-*) die "unknown option: $1" ;;
|
||||
*) if [ -z "$NAME" ]; then NAME="$1"; elif [ -z "$OCTET" ]; then OCTET="$1"; else die "unexpected argument: $1"; fi; shift ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[ -n "$NAME" ] || die "need a device name. See --help"
|
||||
[ -n "$OCTET" ] || die "need a last octet (2-254). See --help"
|
||||
printf '%s' "$NAME" | grep -Eq '^[a-z0-9][a-z0-9-]{0,30}$' \
|
||||
|| die "device name must be lowercase letters/digits/hyphens: got '$NAME'"
|
||||
printf '%s' "$OCTET" | grep -Eq '^[0-9]+$' || die "octet must be a number: got '$OCTET'"
|
||||
[ "$OCTET" -ge 2 ] && [ "$OCTET" -le 254 ] || die "octet must be 2-254 (.1 is the server): got '$OCTET'"
|
||||
|
||||
command -v wg >/dev/null || die "wireguard-tools not installed ('wg' not on PATH). Debian/Ubuntu: apt install wireguard-tools"
|
||||
|
||||
# --- the server's public key --------------------------------------------------------------------
|
||||
if [ -z "${WG_SERVER_PUBKEY:-}" ]; then
|
||||
[ -r "$WG_SERVER_KEY_FILE" ] || die "cannot read server key '$WG_SERVER_KEY_FILE'.
|
||||
Either point WG_SERVER_KEY_FILE at it, or set WG_SERVER_PUBKEY to the public half.
|
||||
If the server key does not exist yet, see runbooks/vpn-remote-access.md step 1."
|
||||
WG_SERVER_PUBKEY="$(tr -d ' \r\n' < "$WG_SERVER_KEY_FILE" | wg pubkey)"
|
||||
fi
|
||||
|
||||
CLIENT_BASE="${WG_CLIENT_CIDR%/*}"; CLIENT_BASE="${CLIENT_BASE%.*}"
|
||||
CLIENT_ADDR="${CLIENT_BASE}.${OCTET}"
|
||||
|
||||
# --- refuse a collision, because the failure is otherwise invisible -----------------------------
|
||||
# Two devices sharing an address does not error anywhere: the server routes that address to
|
||||
# whichever peer handshook most recently, so BOTH devices appear to work intermittently.
|
||||
if [ -r "$PEERS_FILE" ] && grep -q "AllowedIPs *= *${CLIENT_ADDR}/32" "$PEERS_FILE"; then
|
||||
die "${CLIENT_ADDR}/32 is already allocated in platform/vpn-ingress/peers.yaml. Pick another octet."
|
||||
fi
|
||||
if [ -r "$PEERS_FILE" ] && grep -qi "^ *# *${NAME} " "$PEERS_FILE"; then
|
||||
printf 'new-peer.sh: WARNING a peer comment already mentions "%s" in peers.yaml.\n' "$NAME" >&2
|
||||
fi
|
||||
|
||||
# --- output location: 0700, and never inside the repo -------------------------------------------
|
||||
if [ -z "$OUT_DIR" ]; then
|
||||
OUT_DIR="$(mktemp -d "${TMPDIR:-/tmp}/wg-peer-XXXXXX")"
|
||||
else
|
||||
mkdir -p "$OUT_DIR"
|
||||
fi
|
||||
chmod 700 "$OUT_DIR"
|
||||
case "$(cd "$OUT_DIR" && pwd)" in
|
||||
"$REPO_ROOT"|"$REPO_ROOT"/*) die "refusing to write a private key inside the git checkout ($OUT_DIR)" ;;
|
||||
esac
|
||||
PROFILE="${OUT_DIR}/${NAME}.conf"
|
||||
|
||||
umask 077
|
||||
PRIVKEY="$(wg genkey)"
|
||||
PUBKEY="$(printf '%s' "$PRIVKEY" | wg pubkey)"
|
||||
|
||||
if [ "$SPLIT" -eq 1 ]; then
|
||||
ALLOWED="${WG_LAN_CIDR}, ${WG_CLIENT_CIDR}"
|
||||
TUNNEL_KIND="split tunnel (LAN only; the device keeps using its own internet)"
|
||||
else
|
||||
ALLOWED="0.0.0.0/0"
|
||||
TUNNEL_KIND="FULL tunnel (all IPv4 traffic egresses the house WAN)"
|
||||
fi
|
||||
|
||||
cat > "$PROFILE" <<EOF
|
||||
# ${NAME} — remote-access VPN profile for home.keithjackson.dev
|
||||
# Generated $(date -u +%Y-%m-%dT%H:%M:%SZ) by infra/vpn-ingress/new-peer.sh
|
||||
# ${TUNNEL_KIND}
|
||||
#
|
||||
# THIS FILE CONTAINS A PRIVATE KEY. Move it to the device, then delete it from here.
|
||||
# It is the ONLY copy: the server never sees it and it cannot be recovered or re-derived.
|
||||
[Interface]
|
||||
PrivateKey = ${PRIVKEY}
|
||||
Address = ${CLIENT_ADDR}/32
|
||||
DNS = ${WG_DNS}
|
||||
MTU = ${WG_MTU}
|
||||
|
||||
[Peer]
|
||||
PublicKey = ${WG_SERVER_PUBKEY}
|
||||
Endpoint = ${WG_ENDPOINT_HOST}:${WG_LISTEN_PORT}
|
||||
AllowedIPs = ${ALLOWED}
|
||||
# Hold the NAT mapping open from the client side. Required for the server to be able to reach the
|
||||
# device at all once it goes idle; harmless otherwise (one small packet every 25s).
|
||||
PersistentKeepalive = 25
|
||||
EOF
|
||||
chmod 600 "$PROFILE"
|
||||
|
||||
cat <<EOF
|
||||
|
||||
Device : ${NAME}
|
||||
Address : ${CLIENT_ADDR}/32
|
||||
Routing : ${TUNNEL_KIND}
|
||||
DNS : ${WG_DNS} (PiHole — gives the device LAN answers for *.home.keithjackson.dev)
|
||||
Profile : ${PROFILE} (0600; contains the private key)
|
||||
|
||||
------------------------------------------------------------------------------------------
|
||||
STEP 1 — add this to platform/vpn-ingress/peers.yaml, under data.peers.conf, then PR it:
|
||||
|
||||
# ${NAME} — <owner>, enrolled $(date -u +%Y-%m-%d)
|
||||
[Peer]
|
||||
PublicKey = ${PUBKEY}
|
||||
AllowedIPs = ${CLIENT_ADDR}/32
|
||||
|
||||
Keep the comment line: \`wg show\` can only print the key, so it is the only record of which
|
||||
stanza belongs to which device when one has to be revoked.
|
||||
|
||||
STEP 2 — get the profile onto the device:
|
||||
Android / iOS : scan the QR code below in the WireGuard app ("+" -> Scan from QR code)
|
||||
Windows / macOS: WireGuard app -> "Import tunnel(s) from file" -> ${NAME}.conf
|
||||
Linux : install as /etc/wireguard/${NAME}.conf, then
|
||||
sudo systemctl enable --now wg-quick@${NAME}
|
||||
|
||||
STEP 3 — delete the profile from this machine once the device has it:
|
||||
shred -u ${PROFILE} 2>/dev/null || rm -f ${PROFILE}
|
||||
|
||||
The tunnel will not work until the peers.yaml PR is MERGED and Flux has rolled the DaemonSet
|
||||
(~1 min). Until then the device handshakes and gets silence, exactly like an unknown key.
|
||||
------------------------------------------------------------------------------------------
|
||||
EOF
|
||||
|
||||
if command -v qrencode >/dev/null; then
|
||||
printf ' QR code (this IS the private key — your terminal scrollback now holds a credential):\n\n'
|
||||
qrencode -t ansiutf8 < "$PROFILE"
|
||||
else
|
||||
printf ' qrencode not installed, so no QR code was printed (Debian/Ubuntu: apt install qrencode).\n'
|
||||
printf ' Transfer %s to the device by another route instead.\n' "$PROFILE"
|
||||
fi
|
||||
Reference in new issue
Block a user