jakman85andClaude Opus 5.5
8f60e6103a
new-peer.sh generated MTU 1420, which stalled traffic on mobile data despite a healthy handshake (one phone, one carrier, 2026-11-09). Default to 1280, overridable through WG_MTU. Correct stale claims in the runbook and the DaemonSet header against live BPF conntrack: LoadBalancer VIPs see 10.90.0.x as world, not a SNATed host identity. Add a troubleshooting section. The DaemonSet render is unchanged, so WireGuard pods do not roll. Co-Authored-By: Claude Opus 5.5 <[email protected]>
259 lines
14 KiB
Bash
Executable File
259 lines
14 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# =================================================================================================
|
|
# new-peer.sh — enrol one device on the remote-access VPN (platform/vpn-ingress).
|
|
#
|
|
# Generates a WireGuard keypair, renders the device's client profile, prints a QR code for a phone,
|
|
# and prints the one `[Peer]` stanza to paste into platform/vpn-ingress/peers.conf. The device's
|
|
# private half is written to ONE 0600 file and nowhere else: not sent to the cluster, not committed,
|
|
# never logged.
|
|
#
|
|
# RUN IT IN THE DEV CONTAINER — that is where the toolchain lives (CLAUDE.md §5):
|
|
#
|
|
# ./infra/devcontainer/dev.sh run bash -c './infra/vpn-ingress/new-peer.sh pixel-8 2'
|
|
#
|
|
# It needs `wg` (wireguard-tools), `qrencode` and `sops`, all three baked into the image. It will
|
|
# run on a host that happens to have them too, but the container is the supported way and the only
|
|
# one that is reproducible.
|
|
#
|
|
# THE SERVER'S PUBLIC KEY IS DERIVED FROM THE ENCRYPTED FILE, not from a copy you keep lying around:
|
|
# `sops -d` the private half out of platform/vpn-ingress/wg-server-key.sops.yaml and pipe it
|
|
# straight into `wg pubkey`. The plaintext never touches disk and never reaches a terminal. (This
|
|
# is why both tools being in ONE image matters — see the Containerfile comment.) Override with
|
|
# WG_SERVER_PUBKEY or WG_SERVER_KEY_FILE if you are working off-tree.
|
|
#
|
|
# WHY A SCRIPT AND NOT A RUNBOOK STEP. Three of these fields are easy to get wrong in ways that are
|
|
# either silently insecure or silently broken, and all three are mechanical:
|
|
# * AllowedIPs on the SERVER side must be a /32. On the server, AllowedIPs is simultaneously the
|
|
# cryptokey routing table AND the inbound source filter, so a /24 here lets one device spoof
|
|
# every other device's tunnel address.
|
|
# * PersistentKeepalive must be set on the CLIENT. Without it the NAT mapping at the client's end
|
|
# expires after ~30s of idle and nothing can reach the device until it speaks first — which
|
|
# presents as "the VPN works for a minute and then half-works".
|
|
# * The endpoint must be the DDNS name, never the current WAN IP (see --endpoint below).
|
|
#
|
|
# USAGE
|
|
# ./infra/vpn-ingress/new-peer.sh <device-name> <last-octet> [--split] [--out DIR]
|
|
#
|
|
# <device-name> short, lowercase, e.g. pixel-8, thinkpad, work-laptop. Used for the filename
|
|
# and the comment that ties the key to a human-recognisable device.
|
|
# <last-octet> 2-254, the device's address inside WG_CLIENT_CIDR. Must be unused.
|
|
# --split split tunnel: route ONLY the LAN and the tunnel subnet. Default is a FULL
|
|
# tunnel (0.0.0.0/0) so the device also egresses the house WAN.
|
|
# --out DIR where to write the profile. Default: `.cache/wg-peers/` in the repo, which is
|
|
# GITIGNORED (`.gitignore` `.cache/`) and, unlike /tmp inside the container,
|
|
# survives the container exiting — the repo is its only persistent read-write
|
|
# mount, so a profile written anywhere else is gone before you can move it to a
|
|
# desktop. The script refuses any destination git does not ignore.
|
|
#
|
|
# ENVIRONMENT
|
|
# WG_SERVER_PUBKEY the server's public key, if you want to skip the SOPS decrypt entirely.
|
|
# WG_SERVER_KEY_FILE a local copy of the server PRIVATE key to derive the public half from,
|
|
# instead of reading the SOPS file.
|
|
# WG_SERVER_SOPS_FILE default platform/vpn-ingress/wg-server-key.sops.yaml
|
|
# WG_ENDPOINT_HOST default wan.home.keithjackson.dev
|
|
# WG_CLIENT_CIDR default 10.90.0.0/24 (must match clusters/base/cluster-vars.yaml)
|
|
# WG_LAN_CIDR default 192.168.2.0/24 (prod LAN_CIDR)
|
|
# WG_MTU default 1280 (client tunnel MTU; see the comment on the default below)
|
|
# WG_DNS default 192.168.2.23 (prod PIHOLE_DNS_VIP)
|
|
#
|
|
# REQUIRES: `wg`, `qrencode`, `sops` — all three in the dev container.
|
|
# =================================================================================================
|
|
set -euo pipefail
|
|
|
|
die() { printf 'new-peer.sh: %s\n' "$*" >&2; exit 1; }
|
|
|
|
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
PEERS_FILE="${REPO_ROOT}/platform/vpn-ingress/peers.conf"
|
|
|
|
# --- defaults, each mirroring a value that is DECLARED elsewhere in this repo -------------------
|
|
# These are duplicated here rather than parsed out of the YAML on purpose: this script is meant to
|
|
# run on an operator's laptop with no checkout-shaped assumptions beyond its own location, and a
|
|
# silent mis-parse would be worse than a visible default. If you change the knob, change it here.
|
|
: "${WG_CLIENT_CIDR:=10.90.0.0/24}" # clusters/base/cluster-vars.yaml
|
|
: "${WG_LAN_CIDR:=192.168.2.0/24}" # clusters/prod LAN_CIDR
|
|
: "${WG_DNS:=192.168.2.23}" # clusters/prod PIHOLE_DNS_VIP
|
|
: "${WG_SERVER_SOPS_FILE:=${REPO_ROOT}/platform/vpn-ingress/wg-server-key.sops.yaml}"
|
|
# wan.<base-domain> is an UNPROXIED A record that platform/externaldns/ddns.yaml keeps pointed at
|
|
# the current WAN IP (checked every 5 min). Use it, never a literal address: this house has a
|
|
# DYNAMIC WAN IP, and a profile pinned to today's address stops working without warning and with no
|
|
# symptom on the server side. It is a PUBLIC Cloudflare record, so it also resolves from a cellular
|
|
# connection — unlike any `*.home.keithjackson.dev` name PiHole answers, which a remote client
|
|
# cannot resolve until it is already connected.
|
|
: "${WG_ENDPOINT_HOST:=wan.home.keithjackson.dev}"
|
|
: "${WG_LISTEN_PORT:=51820}"
|
|
# The CLIENT's tunnel MTU. Deliberately lower than the server's 1420 (daemonset.yaml), which only
|
|
# describes the 1500-byte path at the node's end. A phone's path is the carrier's, often behind
|
|
# carrier NAT, and is not known to carry 1500. Measured 2026-11-09, one Android phone on one
|
|
# carrier: at 1420 the handshake was healthy but names and pages failed; clearing the app's MTU
|
|
# field (its userspace backend then uses 1280) fixed it. 1280 is also the IPv6 minimum link MTU, so
|
|
# it stays valid if the tunnel ever gains IPv6. For a device only ever on a wired or known-1500
|
|
# path, WG_MTU=1420 recovers about 10% of per-packet payload.
|
|
: "${WG_MTU:=1280}"
|
|
|
|
SPLIT=0
|
|
OUT_DIR=""
|
|
NAME=""
|
|
OCTET=""
|
|
while [ $# -gt 0 ]; do
|
|
case "$1" in
|
|
--split) SPLIT=1; shift ;;
|
|
--out) OUT_DIR="${2:-}"; [ -n "$OUT_DIR" ] || die "--out needs a directory"; shift 2 ;;
|
|
-h|--help) sed -n '2,40p' "${BASH_SOURCE[0]}"; exit 0 ;;
|
|
-*) die "unknown option: $1" ;;
|
|
*) if [ -z "$NAME" ]; then NAME="$1"; elif [ -z "$OCTET" ]; then OCTET="$1"; else die "unexpected argument: $1"; fi; shift ;;
|
|
esac
|
|
done
|
|
|
|
[ -n "$NAME" ] || die "need a device name. See --help"
|
|
[ -n "$OCTET" ] || die "need a last octet (2-254). See --help"
|
|
printf '%s' "$NAME" | grep -Eq '^[a-z0-9][a-z0-9-]{0,30}$' \
|
|
|| die "device name must be lowercase letters/digits/hyphens: got '$NAME'"
|
|
printf '%s' "$OCTET" | grep -Eq '^[0-9]+$' || die "octet must be a number: got '$OCTET'"
|
|
[ "$OCTET" -ge 2 ] && [ "$OCTET" -le 254 ] || die "octet must be 2-254 (.1 is the server): got '$OCTET'"
|
|
|
|
command -v wg >/dev/null || die "'wg' not on PATH. Run this inside the dev container:
|
|
./infra/devcontainer/dev.sh run bash -c './infra/vpn-ingress/new-peer.sh $NAME $OCTET'"
|
|
|
|
# --- the server's public key --------------------------------------------------------------------
|
|
# Order: an explicit pubkey, then a local private-key file, then the SOPS file in the tree. The
|
|
# last is the normal path and needs no local copy of anything.
|
|
if [ -z "${WG_SERVER_PUBKEY:-}" ]; then
|
|
if [ -n "${WG_SERVER_KEY_FILE:-}" ]; then
|
|
[ -r "$WG_SERVER_KEY_FILE" ] || die "cannot read WG_SERVER_KEY_FILE '$WG_SERVER_KEY_FILE'"
|
|
WG_SERVER_PUBKEY="$(tr -d ' \r\n' < "$WG_SERVER_KEY_FILE" | wg pubkey)"
|
|
else
|
|
command -v sops >/dev/null || die "'sops' not on PATH. Run this inside the dev container (see --help)."
|
|
[ -r "$WG_SERVER_SOPS_FILE" ] || die "cannot read '$WG_SERVER_SOPS_FILE'"
|
|
# The decrypted private key goes straight into `wg pubkey` — never to a file, never to a tty.
|
|
# `wg pubkey` fails on the committed REPLACE-ME placeholder, which is the useful error here.
|
|
WG_SERVER_PUBKEY="$(sops -d --extract '["stringData"]["privatekey"]' "$WG_SERVER_SOPS_FILE" \
|
|
| tr -d ' \r\n' | wg pubkey 2>/dev/null)" || WG_SERVER_PUBKEY=""
|
|
[ -n "$WG_SERVER_PUBKEY" ] || die "could not derive the server public key from
|
|
$WG_SERVER_SOPS_FILE
|
|
Either it still holds the REPLACE-ME placeholder (see runbooks/vpn-remote-access.md step 2),
|
|
or this shell cannot decrypt it (the age keyring is mounted at ~/.config/sops in the container)."
|
|
fi
|
|
fi
|
|
|
|
CLIENT_BASE="${WG_CLIENT_CIDR%/*}"; CLIENT_BASE="${CLIENT_BASE%.*}"
|
|
CLIENT_ADDR="${CLIENT_BASE}.${OCTET}"
|
|
|
|
# --- refuse a collision, because the failure is otherwise invisible -----------------------------
|
|
# Two devices sharing an address does not error anywhere: the server routes that address to
|
|
# whichever peer handshook most recently, so BOTH devices appear to work intermittently.
|
|
# COMMENT LINES ARE NOT ALLOCATIONS. This used to grep the whole file and so matched the
|
|
# COMMENTED-OUT EXAMPLE STANZA in the register, which made `.2` — the first address anyone would
|
|
# ask for — permanently un-allocatable with a message claiming it was already taken. Strip
|
|
# comments first: a real stanza's `AllowedIPs` line is never commented, while both the template
|
|
# and every peer's `# <name> — <owner>` label are.
|
|
# FAIL if the register is missing rather than skipping the check. The path was silently wrong once
|
|
# (it still said peers.yaml after the file became peers.conf), and because the old guard was
|
|
# `[ -r "$PEERS_FILE" ] && ...` the only symptom would have been duplicate detection quietly
|
|
# never firing again — the exact silent-skip shape this repo treats as a defect.
|
|
[ -r "$PEERS_FILE" ] || die "cannot read the peer register '$PEERS_FILE'.
|
|
Duplicate-address detection cannot run, so this script refuses to issue a profile."
|
|
if grep -vE '^[[:space:]]*#' "$PEERS_FILE" | grep -q "AllowedIPs *= *${CLIENT_ADDR}/32"; then
|
|
die "${CLIENT_ADDR}/32 is already allocated in platform/vpn-ingress/peers.conf. Pick another octet."
|
|
fi
|
|
if grep -qi "^ *# *${NAME} " "$PEERS_FILE"; then
|
|
printf 'new-peer.sh: WARNING a peer comment already mentions "%s" in peers.conf.\n' "$NAME" >&2
|
|
fi
|
|
|
|
# --- output location: 0700, and never inside the repo -------------------------------------------
|
|
[ -n "$OUT_DIR" ] || OUT_DIR="${REPO_ROOT}/.cache/wg-peers"
|
|
mkdir -p "$OUT_DIR"
|
|
chmod 700 "$OUT_DIR"
|
|
OUT_ABS="$(cd "$OUT_DIR" && pwd)"
|
|
# A profile holds a private key, so the destination must not be committable. Inside the repo that
|
|
# means git must IGNORE it — test the real property rather than a proxy, because the repo is the
|
|
# container's only persistent read-write mount and so the only sane place to write. Outside the
|
|
# repo, anywhere goes.
|
|
case "$OUT_ABS" in
|
|
"$REPO_ROOT"|"$REPO_ROOT"/*)
|
|
if ! git -C "$REPO_ROOT" check-ignore -q "$OUT_ABS/probe.conf" 2>/dev/null; then
|
|
die "refusing to write a private key to '$OUT_ABS': it is inside the git checkout and NOT
|
|
gitignored, so the profile would be committable. Use the default (.cache/wg-peers) or add an
|
|
ignore rule first. Verify with: git check-ignore -v <path>"
|
|
fi
|
|
;;
|
|
esac
|
|
PROFILE="${OUT_DIR}/${NAME}.conf"
|
|
|
|
umask 077
|
|
PRIVKEY="$(wg genkey)"
|
|
PUBKEY="$(printf '%s' "$PRIVKEY" | wg pubkey)"
|
|
|
|
if [ "$SPLIT" -eq 1 ]; then
|
|
ALLOWED="${WG_LAN_CIDR}, ${WG_CLIENT_CIDR}"
|
|
TUNNEL_KIND="split tunnel (LAN only; the device keeps using its own internet)"
|
|
else
|
|
ALLOWED="0.0.0.0/0"
|
|
TUNNEL_KIND="FULL tunnel (all IPv4 traffic egresses the house WAN)"
|
|
fi
|
|
|
|
cat > "$PROFILE" <<EOF
|
|
# ${NAME} — remote-access VPN profile for home.keithjackson.dev
|
|
# Generated $(date -u +%Y-%m-%dT%H:%M:%SZ) by infra/vpn-ingress/new-peer.sh
|
|
# ${TUNNEL_KIND}
|
|
#
|
|
# THIS FILE CONTAINS A PRIVATE KEY. Move it to the device, then delete it from here.
|
|
# It is the ONLY copy: the server never sees it and it cannot be recovered or re-derived.
|
|
[Interface]
|
|
PrivateKey = ${PRIVKEY}
|
|
Address = ${CLIENT_ADDR}/32
|
|
DNS = ${WG_DNS}
|
|
MTU = ${WG_MTU}
|
|
|
|
[Peer]
|
|
PublicKey = ${WG_SERVER_PUBKEY}
|
|
Endpoint = ${WG_ENDPOINT_HOST}:${WG_LISTEN_PORT}
|
|
AllowedIPs = ${ALLOWED}
|
|
# Hold the NAT mapping open from the client side. Required for the server to be able to reach the
|
|
# device at all once it goes idle; harmless otherwise (one small packet every 25s).
|
|
PersistentKeepalive = 25
|
|
EOF
|
|
chmod 600 "$PROFILE"
|
|
|
|
cat <<EOF
|
|
|
|
Device : ${NAME}
|
|
Address : ${CLIENT_ADDR}/32
|
|
Routing : ${TUNNEL_KIND}
|
|
DNS : ${WG_DNS} (PiHole — gives the device LAN answers for *.home.keithjackson.dev)
|
|
Profile : ${PROFILE} (0600; contains the private key)
|
|
|
|
------------------------------------------------------------------------------------------
|
|
STEP 1 — append this to platform/vpn-ingress/peers.conf, then PR it:
|
|
|
|
# ${NAME} — <owner>, enrolled $(date -u +%Y-%m-%d)
|
|
[Peer]
|
|
PublicKey = ${PUBKEY}
|
|
AllowedIPs = ${CLIENT_ADDR}/32
|
|
|
|
Keep the comment line: \`wg show\` can only print the key, so it is the only record of which
|
|
stanza belongs to which device when one has to be revoked.
|
|
|
|
STEP 2 — get the profile onto the device:
|
|
Android / iOS : scan the QR code below in the WireGuard app ("+" -> Scan from QR code)
|
|
Windows / macOS: WireGuard app -> "Import tunnel(s) from file" -> ${NAME}.conf
|
|
Linux : install as /etc/wireguard/${NAME}.conf, then
|
|
sudo systemctl enable --now wg-quick@${NAME}
|
|
|
|
STEP 3 — delete the profile from this machine once the device has it:
|
|
shred -u ${PROFILE} 2>/dev/null || rm -f ${PROFILE}
|
|
|
|
The tunnel will not work until the peers.conf PR is MERGED and Flux has rolled the DaemonSet
|
|
(~1 min). Until then the device handshakes and gets silence, exactly like an unknown key.
|
|
------------------------------------------------------------------------------------------
|
|
EOF
|
|
|
|
if command -v qrencode >/dev/null; then
|
|
printf ' QR code (this IS the private key — your terminal scrollback now holds a credential):\n\n'
|
|
qrencode -t ansiutf8 < "$PROFILE"
|
|
else
|
|
# Should not happen in the dev container: qrencode is baked into the image. Kept as a real
|
|
# fallback for a host run, rather than failing after the profile has already been written.
|
|
printf ' qrencode not on PATH, so no QR code was printed. It is in the dev container —\n'
|
|
printf ' rerun there, or transfer %s to the device by another route.\n' "$PROFILE"
|
|
fi
|