fix(vpn-ingress): content-hash the peer register, or a merged peer never reaches the tunnel
The operator enrolled a phone, merged the peer, and `wg show` listed none. Measured on prod 2026-10-06 with the peer already on main: live wg-peers ConfigMap, active AllowedIPs .. 0 DaemonSet volume reference ................. configMap.name: wg-peers (FIXED) DaemonSet generation/observedGeneration .... 1 / 1 (never rolled) init container, all three pods ............. "0 peer(s)" Part of it was just the 10m reconcile interval. The defect is the fixed name: Flux updates a fixed-name ConfigMap IN PLACE, so the pod template is byte-identical, so no rollout is triggered -- and the init container assembles wg0.conf ONCE at pod start into a tmpfs, so the running tunnel keeps the old peer list indefinitely. Adding or revoking a peer had no effect until something unrelated restarted the pods. For a REVOCATION that is a security defect, not an inconvenience: delete a stanza, see a green PR, device keeps working. The runbook and README both claimed "PR -> merge -> Flux rolls the DaemonSet". Never true. And this repo already documents the fix, in a file I had read -- runbooks/kustomization.yaml: "The generated name carries kustomize's content hash, so editing any runbook rolls the server automatically and a stale projection is impossible." So: peers.yaml (inline ConfigMap) -> peers.conf (plain file) + a configMapGenerator. The operator's enrolled peer is carried across verbatim. Every line is WireGuard config and `#` is a legal comment there, so the documentation now lives inside the file that is actually mounted. `namespace: vpn-ingress` on the kustomization is REQUIRED, not tidiness: a generated ConfigMap inherits no namespace, everything else here declares vpn-ingress, and kustomize's nameReference will not rewrite across a namespace boundary -- without it the build mounts the unhashed name and the pods wedge on a missing volume. Caught only by building the output and reading the volumes. new-peer.sh: PEERS_FILE repointed, and a missing register is now FATAL rather than skipped -- the old `[ -r "$PEERS_FILE" ] && grep ...` guard meant that after the rename the only symptom would have been duplicate detection quietly never firing again. Stale peers.yaml references swept from six files including the script's own user-facing output. Verified: the DaemonSet now mounts wg-peers-m9kcckcb5t; the hash tracks content (edit -> dgg7ht9c74 -> restore -> m9kcckcb5t); the enrolled peer is byte-identical to main's; and `new-peer.sh dupe 2` is now correctly refused by the operator's real allocation. No sensitive paths; merges on a green gate. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
1 parent
e1bebb857e
commit
2acfed460a
9 files changed
+226
-65
No files matched your search
@@ -3,7 +3,7 @@
|
||||
# new-peer.sh — enrol one device on the remote-access VPN (platform/vpn-ingress).
|
||||
#
|
||||
# Generates a WireGuard keypair, renders the device's client profile, prints a QR code for a phone,
|
||||
# and prints the one `[Peer]` stanza to paste into platform/vpn-ingress/peers.yaml. The device's
|
||||
# and prints the one `[Peer]` stanza to paste into platform/vpn-ingress/peers.conf. The device's
|
||||
# private half is written to ONE 0600 file and nowhere else: not sent to the cluster, not committed,
|
||||
# never logged.
|
||||
#
|
||||
@@ -62,7 +62,7 @@ set -euo pipefail
|
||||
die() { printf 'new-peer.sh: %s\n' "$*" >&2; exit 1; }
|
||||
|
||||
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
PEERS_FILE="${REPO_ROOT}/platform/vpn-ingress/peers.yaml"
|
||||
PEERS_FILE="${REPO_ROOT}/platform/vpn-ingress/peers.conf"
|
||||
|
||||
# --- defaults, each mirroring a value that is DECLARED elsewhere in this repo -------------------
|
||||
# These are duplicated here rather than parsed out of the YAML on purpose: this script is meant to
|
||||
@@ -134,16 +134,21 @@ CLIENT_ADDR="${CLIENT_BASE}.${OCTET}"
|
||||
# Two devices sharing an address does not error anywhere: the server routes that address to
|
||||
# whichever peer handshook most recently, so BOTH devices appear to work intermittently.
|
||||
# COMMENT LINES ARE NOT ALLOCATIONS. This used to grep the whole file and so matched the
|
||||
# COMMENTED-OUT EXAMPLE STANZA in peers.yaml, which made `.2` — the first address anyone would
|
||||
# COMMENTED-OUT EXAMPLE STANZA in the register, which made `.2` — the first address anyone would
|
||||
# ask for — permanently un-allocatable with a message claiming it was already taken. Strip
|
||||
# comments first: a real stanza's `AllowedIPs` line is never commented, while both the template
|
||||
# and every peer's `# <name> — <owner>` label are.
|
||||
if [ -r "$PEERS_FILE" ] \
|
||||
&& grep -vE '^[[:space:]]*#' "$PEERS_FILE" | grep -q "AllowedIPs *= *${CLIENT_ADDR}/32"; then
|
||||
die "${CLIENT_ADDR}/32 is already allocated in platform/vpn-ingress/peers.yaml. Pick another octet."
|
||||
# FAIL if the register is missing rather than skipping the check. The path was silently wrong once
|
||||
# (it still said peers.yaml after the file became peers.conf), and because the old guard was
|
||||
# `[ -r "$PEERS_FILE" ] && ...` the only symptom would have been duplicate detection quietly
|
||||
# never firing again — the exact silent-skip shape this repo treats as a defect.
|
||||
[ -r "$PEERS_FILE" ] || die "cannot read the peer register '$PEERS_FILE'.
|
||||
Duplicate-address detection cannot run, so this script refuses to issue a profile."
|
||||
if grep -vE '^[[:space:]]*#' "$PEERS_FILE" | grep -q "AllowedIPs *= *${CLIENT_ADDR}/32"; then
|
||||
die "${CLIENT_ADDR}/32 is already allocated in platform/vpn-ingress/peers.conf. Pick another octet."
|
||||
fi
|
||||
if [ -r "$PEERS_FILE" ] && grep -qi "^ *# *${NAME} " "$PEERS_FILE"; then
|
||||
printf 'new-peer.sh: WARNING a peer comment already mentions "%s" in peers.yaml.\n' "$NAME" >&2
|
||||
if grep -qi "^ *# *${NAME} " "$PEERS_FILE"; then
|
||||
printf 'new-peer.sh: WARNING a peer comment already mentions "%s" in peers.conf.\n' "$NAME" >&2
|
||||
fi
|
||||
|
||||
# --- output location: 0700, and never inside the repo -------------------------------------------
|
||||
@@ -210,7 +215,7 @@ cat <<EOF
|
||||
Profile : ${PROFILE} (0600; contains the private key)
|
||||
|
||||
------------------------------------------------------------------------------------------
|
||||
STEP 1 — add this to platform/vpn-ingress/peers.yaml, under data.peers.conf, then PR it:
|
||||
STEP 1 — append this to platform/vpn-ingress/peers.conf, then PR it:
|
||||
|
||||
# ${NAME} — <owner>, enrolled $(date -u +%Y-%m-%d)
|
||||
[Peer]
|
||||
@@ -229,7 +234,7 @@ cat <<EOF
|
||||
STEP 3 — delete the profile from this machine once the device has it:
|
||||
shred -u ${PROFILE} 2>/dev/null || rm -f ${PROFILE}
|
||||
|
||||
The tunnel will not work until the peers.yaml PR is MERGED and Flux has rolled the DaemonSet
|
||||
The tunnel will not work until the peers.conf PR is MERGED and Flux has rolled the DaemonSet
|
||||
(~1 min). Until then the device handshakes and gets silence, exactly like an unknown key.
|
||||
------------------------------------------------------------------------------------------
|
||||
EOF
|
||||
|
||||
Reference in new issue
Block a user