jakman85andClaude Opus 5
2acfed460a
The operator enrolled a phone, merged the peer, and `wg show` listed none. Measured on prod 2026-10-06 with the peer already on main: live wg-peers ConfigMap, active AllowedIPs .. 0 DaemonSet volume reference ................. configMap.name: wg-peers (FIXED) DaemonSet generation/observedGeneration .... 1 / 1 (never rolled) init container, all three pods ............. "0 peer(s)" Part of it was just the 10m reconcile interval. The defect is the fixed name: Flux updates a fixed-name ConfigMap IN PLACE, so the pod template is byte-identical, so no rollout is triggered -- and the init container assembles wg0.conf ONCE at pod start into a tmpfs, so the running tunnel keeps the old peer list indefinitely. Adding or revoking a peer had no effect until something unrelated restarted the pods. For a REVOCATION that is a security defect, not an inconvenience: delete a stanza, see a green PR, device keeps working. The runbook and README both claimed "PR -> merge -> Flux rolls the DaemonSet". Never true. And this repo already documents the fix, in a file I had read -- runbooks/kustomization.yaml: "The generated name carries kustomize's content hash, so editing any runbook rolls the server automatically and a stale projection is impossible." So: peers.yaml (inline ConfigMap) -> peers.conf (plain file) + a configMapGenerator. The operator's enrolled peer is carried across verbatim. Every line is WireGuard config and `#` is a legal comment there, so the documentation now lives inside the file that is actually mounted. `namespace: vpn-ingress` on the kustomization is REQUIRED, not tidiness: a generated ConfigMap inherits no namespace, everything else here declares vpn-ingress, and kustomize's nameReference will not rewrite across a namespace boundary -- without it the build mounts the unhashed name and the pods wedge on a missing volume. Caught only by building the output and reading the volumes. new-peer.sh: PEERS_FILE repointed, and a missing register is now FATAL rather than skipped -- the old `[ -r "$PEERS_FILE" ] && grep ...` guard meant that after the rename the only symptom would have been duplicate detection quietly never firing again. Stale peers.yaml references swept from six files including the script's own user-facing output. Verified: the DaemonSet now mounts wg-peers-m9kcckcb5t; the hash tracks content (edit -> dgg7ht9c74 -> restore -> m9kcckcb5t); the enrolled peer is byte-identical to main's; and `new-peer.sh dupe 2` is now correctly refused by the operator's real allocation. No sensitive paths; merges on a green gate. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
93 lines
6.0 KiB
Plaintext
93 lines
6.0 KiB
Plaintext
# THIS FILE IS THE MOUNTED CONTENT. It is turned into the `wg-peers` ConfigMap by
|
|
# `configMapGenerator` in kustomization.yaml, whose generated name carries a CONTENT HASH — so
|
|
# editing this file changes the ConfigMap name, which changes the DaemonSet's pod template, which
|
|
# ROLLS THE PODS. That is the whole reason it is a generator and not an inline ConfigMap manifest:
|
|
# a fixed-name ConfigMap updates in place, the pod template never changes, no rollout happens, and
|
|
# the init container's wg0.conf (built once at pod start, in tmpfs) keeps the OLD peer list
|
|
# forever. Measured 2026-10-06: a merged peer sat in the ConfigMap while `wg show` listed none and
|
|
# the DaemonSet reported generation=1 — it had never rolled. Same mechanism and same fix as
|
|
# runbooks/kustomization.yaml.
|
|
#
|
|
# Every line here is WireGuard config. `#` is a legal comment in it, so the documentation below
|
|
# travels inside the file that is actually mounted, rather than in a wrapper that gets stripped.
|
|
#
|
|
# ============================================================================================
|
|
# THE PEER REGISTER — every device allowed to dial in, in PLAINTEXT, in git, on purpose.
|
|
# ============================================================================================
|
|
# WireGuard has no usernames, no passwords and no PKI. Authorisation is exactly one thing: a
|
|
# device's PUBLIC key appears in this file, or it cannot complete a handshake. An unlisted key
|
|
# gets no reply at all — not a rejection, silence (see README "Why this is safe to expose").
|
|
# Revoking a device is therefore deleting its stanza, and nothing else.
|
|
#
|
|
# WHY PLAINTEXT, AND WHY THIS IS NOT A SECRET LEAK. A WireGuard peer's PUBLIC key is public by
|
|
# construction — it is the half you are supposed to hand to the server, and holding it grants
|
|
# nothing. The only secret in the whole scheme is the SERVER's private key, which is NOT here:
|
|
# it lives in the `wg-server-key` Secret and is assembled into the running config at pod start
|
|
# (daemonset.yaml). Keeping the register in cleartext buys the property that matters most for an
|
|
# access-control list: **"who can get into my house" is reviewable in a pull-request diff**, and
|
|
# its history is `git log` on this file. A SOPS-encrypted register would hide exactly the change
|
|
# a reviewer most needs to see — a new line granting a new device entry to the whole LAN.
|
|
#
|
|
# This is the same division of labour the repo already uses for the opposite case: a Secret's KEY
|
|
# NAMES are cleartext while values are encrypted (.sops.yaml `encrypted_regex`), so diffs stay
|
|
# readable. Here the whole file is the key-name half.
|
|
#
|
|
# ADDING A DEVICE — never hand-roll this, and never generate a key in a chat transcript.
|
|
# ./infra/vpn-ingress/new-peer.sh <device-name> <last-octet>
|
|
# It generates the keypair ON THE OPERATOR'S MACHINE, writes/renders the client profile (and a
|
|
# QR code for phones), prints ONLY the public half for pasting below, and never transmits or
|
|
# logs the private half. Then: add the stanza, PR, merge; Flux rolls the DaemonSet.
|
|
#
|
|
# ADDRESS ALLOCATION. ${WG_CLIENT_CIDR} is the tunnel subnet; .1 is the server (every node).
|
|
# Give each device ONE address as a /32 in AllowedIPs. A /32 is not a formality — on the SERVER
|
|
# side AllowedIPs is the cryptokey ROUTING table AND the inbound source filter: a peer may only
|
|
# send from, and receive traffic for, the addresses listed in its own stanza. A sloppy /24 here
|
|
# would let any one device impersonate every other device's tunnel address.
|
|
#
|
|
# .1 the server (this DaemonSet, on every node)
|
|
# .2 - .254 devices, one each, allocated below. Keep this list dense and in order so the
|
|
# next free octet is obvious; never re-use an octet after revoking a device
|
|
# (an old client profile would silently start working again as someone else).
|
|
#
|
|
# PRESHARED KEYS are deliberately NOT used. They add a symmetric post-quantum hedge on top of
|
|
# the Noise handshake and would be a genuine secret — which would force this register into SOPS
|
|
# and cost the reviewability above. Revisit only if the threat model changes; if it does, the PSK
|
|
# belongs in the `wg-server-key` Secret, not here. See README "Hardening not taken".
|
|
#
|
|
# name: wg-peers
|
|
# namespace: vpn-ingress
|
|
# labels:
|
|
# app.kubernetes.io/name: vpn-ingress
|
|
# Appended VERBATIM to the generated [Interface] block, in this order, at pod start.
|
|
#
|
|
# *** THERE ARE NO PEERS YET. THAT IS THE CORRECT INITIAL STATE, NOT AN OVERSIGHT. ***
|
|
# An agent must not mint key material (CLAUDE.md §6), so no device could be enrolled as part of
|
|
# authoring this component. A zero-peer WireGuard interface is valid and starts cleanly: the
|
|
# DaemonSet will run and listen, and every handshake will be refused because no key is trusted.
|
|
# The VPN is therefore INERT-BUT-HEALTHY until the first stanza lands — fail-closed, and visible
|
|
# (`wg show` reports an interface with no peers).
|
|
#
|
|
# COPY THIS TEMPLATE, uncomment, one block per device:
|
|
#
|
|
# # <device-name> — <owner>, enrolled <YYYY-MM-DD>
|
|
# [Peer]
|
|
# PublicKey = <the public key new-peer.sh printed>
|
|
# AllowedIPs = 10.90.0.<octet>/32
|
|
#
|
|
# The placeholders above are deliberately NOT valid values. An earlier version of this template
|
|
# used a real-looking `10.90.0.2/32`, and new-peer.sh's duplicate-address check — which grepped
|
|
# the whole file — matched it, so `.2` could never be allocated and the script insisted it was
|
|
# already taken. The check now ignores comments; keeping the example unmatchable too means a
|
|
# future grep cannot be fooled either.
|
|
#
|
|
# Keep the `# <name> — <owner>, enrolled <date>` comment line. It is the only place a key is
|
|
# tied to a human-recognisable device; `wg show` can only ever print the key itself, so without
|
|
# it an incident responder cannot tell which stanza to delete.
|
|
#
|
|
# No peers enrolled yet. See the template in this file's comments and
|
|
# ./infra/vpn-ingress/new-peer.sh. One [Peer] block per device.
|
|
# keith-phone — <owner>, enrolled 2026-10-06
|
|
[Peer]
|
|
PublicKey = pBRSHrRyG2hnsuZ6FvHfEzF8hOVTF5stHUb/7fW2Km8=
|
|
AllowedIPs = 10.90.0.2/32
|