jakman85andClaude Opus 5
2acfed460a
The operator enrolled a phone, merged the peer, and `wg show` listed none. Measured on prod 2026-10-06 with the peer already on main: live wg-peers ConfigMap, active AllowedIPs .. 0 DaemonSet volume reference ................. configMap.name: wg-peers (FIXED) DaemonSet generation/observedGeneration .... 1 / 1 (never rolled) init container, all three pods ............. "0 peer(s)" Part of it was just the 10m reconcile interval. The defect is the fixed name: Flux updates a fixed-name ConfigMap IN PLACE, so the pod template is byte-identical, so no rollout is triggered -- and the init container assembles wg0.conf ONCE at pod start into a tmpfs, so the running tunnel keeps the old peer list indefinitely. Adding or revoking a peer had no effect until something unrelated restarted the pods. For a REVOCATION that is a security defect, not an inconvenience: delete a stanza, see a green PR, device keeps working. The runbook and README both claimed "PR -> merge -> Flux rolls the DaemonSet". Never true. And this repo already documents the fix, in a file I had read -- runbooks/kustomization.yaml: "The generated name carries kustomize's content hash, so editing any runbook rolls the server automatically and a stale projection is impossible." So: peers.yaml (inline ConfigMap) -> peers.conf (plain file) + a configMapGenerator. The operator's enrolled peer is carried across verbatim. Every line is WireGuard config and `#` is a legal comment there, so the documentation now lives inside the file that is actually mounted. `namespace: vpn-ingress` on the kustomization is REQUIRED, not tidiness: a generated ConfigMap inherits no namespace, everything else here declares vpn-ingress, and kustomize's nameReference will not rewrite across a namespace boundary -- without it the build mounts the unhashed name and the pods wedge on a missing volume. Caught only by building the output and reading the volumes. new-peer.sh: PEERS_FILE repointed, and a missing register is now FATAL rather than skipped -- the old `[ -r "$PEERS_FILE" ] && grep ...` guard meant that after the rename the only symptom would have been duplicate detection quietly never firing again. Stale peers.yaml references swept from six files including the script's own user-facing output. Verified: the DaemonSet now mounts wg-peers-m9kcckcb5t; the hash tracks content (edit -> dgg7ht9c74 -> restore -> m9kcckcb5t); the enrolled peer is byte-identical to main's; and `new-peer.sh dupe 2` is now correctly refused by the operator's real allocation. No sensitive paths; merges on a green gate. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
77 lines
4.7 KiB
Plaintext
77 lines
4.7 KiB
Plaintext
# =============================================================================
|
|
# TEMPLATE — NOT APPLIED, NOT ENCRYPTED (.yaml.example is ignored by kustomize and never
|
|
# auto-encrypted by SOPS). Reference scaffold + OPERATOR HAND-OFF.
|
|
#
|
|
# WHAT THIS IS: the WireGuard SERVER's long-term private key for the remote-access VPN
|
|
# (platform/vpn-ingress). It is the ONE secret in the whole component. The init container in
|
|
# daemonset.yaml reads it at `/run/wg-server-key/privatekey` and joins it to the PLAINTEXT peer
|
|
# register (peers.conf) in a tmpfs, so the assembled wg0.conf never touches a node's disk.
|
|
#
|
|
# WHY IT IS IN GIT, SOPS-ENCRYPTED, AND NOT HAND-CREATED. R-SEC-01 / spec docs/07: secrets live in
|
|
# source, encrypted, and reach the cluster only through Flux. An out-of-band `kubectl create secret`
|
|
# would put the single credential every enrolled device depends on OUTSIDE the GitOps tree, where a
|
|
# cluster rebuild silently loses it — and losing it means RE-ENROLLING EVERY DEVICE, because each
|
|
# client profile pins this key's public half.
|
|
#
|
|
# *** THE REAL FILE IS COMMITTED WITH `privatekey: REPLACE-ME`, ALREADY ENCRYPTED. ***
|
|
# That is deliberate, so the wiring is complete and green before the key exists: the resource is
|
|
# already listed in kustomization.yaml and both overlays already carry the `vpn-ingress` decryption
|
|
# patch. Your job is ONLY to replace the value (step 3 below). Until you do, the DaemonSet
|
|
# CrashLoops with an explicit FATAL naming this file — the init container validates the value with
|
|
# `wg pubkey` before writing wg0.conf, so a placeholder fails CLOSED and LOUDLY rather than
|
|
# producing a server that silently cannot complete a handshake.
|
|
#
|
|
# ---------------------------------------------------------------------------------------------
|
|
# EVERYTHING RUNS IN THE DEV CONTAINER. `wg`, `sops` and `qrencode` are all baked into the image
|
|
# (infra/devcontainer/Containerfile) because that image IS the toolchain — CLAUDE.md §5, and the
|
|
# devcontainer README's "Don't install the toolchain on the host". One shell, one toolchain, so
|
|
# `wg genkey | sops set` is just a pipeline.
|
|
# ---------------------------------------------------------------------------------------------
|
|
#
|
|
# HAND-OFF STEPS (operator — an agent must not mint a long-lived private key, CLAUDE.md §6).
|
|
# Full procedure with verification: runbooks/vpn-remote-access.md step 2.
|
|
#
|
|
# 1. Generate the key STRAIGHT INTO this file. The plaintext never lands on disk and never
|
|
# reaches a terminal; `sops set` edits the already-encrypted document in place:
|
|
# ./infra/devcontainer/dev.sh run bash -c '
|
|
# cd /work
|
|
# umask 077
|
|
# sops set platform/vpn-ingress/wg-server-key.sops.yaml \
|
|
# "[\"stringData\"][\"privatekey\"]" "\"$(wg genkey)\""
|
|
# '
|
|
# (Or edit it: `./infra/devcontainer/dev.sh run sops platform/vpn-ingress/wg-server-key.sops.yaml`.)
|
|
#
|
|
# 2. Record BOTH halves in your PASSWORD MANAGER. The encrypted file is in git and Velero captures
|
|
# this namespace to R2, so this is belt-and-braces — but it is the only copy that survives
|
|
# losing the cluster AND the backups, and the key cannot be regenerated without invalidating
|
|
# every client profile.
|
|
# # public half (safe to display; it is what every client profile carries)
|
|
# ./infra/devcontainer/dev.sh run bash -c 'cd /work && sops -d \
|
|
# --extract "[\"stringData\"][\"privatekey\"]" \
|
|
# platform/vpn-ingress/wg-server-key.sops.yaml | wg pubkey'
|
|
#
|
|
# 3. Confirm the value is a real key without printing it — `wg pubkey` succeeds only on a valid
|
|
# 32-byte curve25519 private key, which is exactly the check the pod's init container runs:
|
|
# ./infra/devcontainer/dev.sh run bash -c 'cd /work && sops -d \
|
|
# --extract "[\"stringData\"][\"privatekey\"]" \
|
|
# platform/vpn-ingress/wg-server-key.sops.yaml | wg pubkey >/dev/null \
|
|
# && echo "valid WireGuard key"'
|
|
#
|
|
# 4. PR it. Nothing else to wire: the resource line and both overlay decryption patches already
|
|
# exist. Flux rolls the DaemonSet and the pods leave CrashLoop.
|
|
#
|
|
# NOTE infra/vpn-ingress/new-peer.sh reads THIS FILE to derive each client profile's
|
|
# `[Peer] PublicKey`, so there is no separate copy of the key to keep in step with anything.
|
|
# =============================================================================
|
|
apiVersion: v1
|
|
kind: Secret
|
|
metadata:
|
|
name: wg-server-key
|
|
namespace: vpn-ingress
|
|
type: Opaque
|
|
stringData:
|
|
# The server's WireGuard private key: `wg genkey` output — 44 chars of base64 ending in '='.
|
|
# NOT a passphrase, and NOT the public key. The init container validates it with `wg pubkey`
|
|
# before writing wg0.conf, so a placeholder or a mangled value fails closed with a clear message.
|
|
privatekey: REPLACE-ME
|