fix(vpn-ingress): content-hash the peer register, or a merged peer never reaches the tunnel #66

2026-10-06T07:56:38Z

gitea-admin commented

2026-10-06 07:56:38 +00:00

You enrolled a phone, merged the peer, and wg show lists none. Measured on prod with the peer already on main:


peer stanza on `main`	present (`AllowedIPs = 10.90.0.2/32`)
GitRepository artifact	`main@sha1:e1bebb8` — Flux had fetched it
`vpn-ingress` Kustomization `lastAppliedRevision`	`5928d81` — behind, 10m interval
live `wg-peers` ConfigMap, active `AllowedIPs`	0
DaemonSet volume reference	`configMap.name: wg-peers` — a fixed name
DaemonSet `generation` / `observedGeneration`	1 / 1 — it had never rolled
init container, all three pods	`wg0.conf assembled: 0 peer(s)`

Part of it was just the reconcile interval. The defect is the fixed name. Flux updates a fixed-name ConfigMap in place, so the pod template stays byte-identical and no rollout is triggered — and the init container assembles wg0.conf once at pod start into a tmpfs, so the running tunnel keeps the old peer list indefinitely. Adding a peer did nothing until something unrelated restarted the pods.

For a revocation that's a security defect, not an inconvenience: you'd delete a stanza, see a green PR, and the device would keep working.

My runbook and README both asserted "PR → merge → Flux rolls the DaemonSet (~1 min)". That was never true.

The fix the repo already documented

runbooks/kustomization.yaml, a file I had read: "The generated name carries kustomize's content hash, so editing any runbook rolls the server automatically and a stale projection is impossible."

peers.yaml (inline ConfigMap) → peers.conf (plain file) + a configMapGenerator. Your enrolled peer is carried across verbatim — I diffed the active stanza against git show origin/main:…/peers.yaml to be sure. Every line is WireGuard config and # is a legal comment there, so the documentation now travels inside the mounted file.
namespace: vpn-ingress on the kustomization is required, not tidiness. A generated ConfigMap inherits no namespace, everything else here declares vpn-ingress, and kustomize's nameReference won't rewrite across a namespace boundary. Without it the build emitted ConfigMap wg-peers-m9kcckcb5t while the DaemonSet still mounted the unhashed wg-peers — the pods would wedge on a missing volume. I only caught that by building the output and reading the volumes block.
new-peer.sh: PEERS_FILE repointed, and a missing register is now fatal rather than skipped. The old [ -r "$PEERS_FILE" ] && grep … guard meant that after the rename the only symptom would have been duplicate detection quietly never firing again.
Stale peers.yaml references swept from six files — including the script's own user-facing output, which told you to paste into "peers.yaml, under data.peers.conf".
The runbook gained the check that actually distinguishes this failure: the DaemonSet's generation must increase after a merge.

Verified

Reference rewritten end to end: generated ConfigMap wg-peers-m9kcckcb5t in vpn-ingress, DaemonSet mounts wg-peers-m9kcckcb5t. Measured both ways — without namespace: it mounted the unhashed name.
The hash tracks content: m9kcckcb5t → add a peer → dgg7ht9c74 → restore → m9kcckcb5t. So an edit necessarily changes the pod template.
new-peer.sh dupe 2 is now refused because your real .2 occupies it — the first time the duplicate check has run against real data rather than its own example.
Build still emits all five objects; the Namespace object's own name is untouched.
Gate: rc 3, all runnable stages green, two environmental skips. Not green — reported as rc 3.

Not verified

That the roll actually happens — the point of the change is unproven until Flux applies it. Success: generation increases, init log says 1 peer(s), wg show lists the peer. And no handshake has ever been observed, so the tunnel still hasn't carried a packet I can see. Revocation — the case where this was a security problem — is also untested.

No sensitive paths. Merges on a green gate, no override.

🤖 Generated with Claude Code

You enrolled a phone, merged the peer, and **`wg show` lists none.** Measured on prod with the peer already on `main`: | | | |---|---| | peer stanza on `main` | present (`AllowedIPs = 10.90.0.2/32`) | | GitRepository artifact | `main@sha1:e1bebb8` — Flux had fetched it | | `vpn-ingress` Kustomization `lastAppliedRevision` | `5928d81` — behind, 10m interval | | live `wg-peers` ConfigMap, active `AllowedIPs` | **0** | | DaemonSet volume reference | `configMap.name: wg-peers` — a **fixed** name | | DaemonSet `generation` / `observedGeneration` | **1 / 1** — it had **never rolled** | | init container, all three pods | `wg0.conf assembled: 0 peer(s)` | Part of it was just the reconcile interval. **The defect is the fixed name.** Flux updates a fixed-name ConfigMap *in place*, so the pod template stays byte-identical and no rollout is triggered — and the init container assembles `wg0.conf` **once at pod start** into a tmpfs, so the running tunnel keeps the old peer list indefinitely. Adding a peer did nothing until something unrelated restarted the pods. **For a revocation that's a security defect, not an inconvenience:** you'd delete a stanza, see a green PR, and the device would keep working. My runbook and README both asserted "PR → merge → Flux rolls the DaemonSet (~1 min)". That was never true. ## The fix the repo already documented `runbooks/kustomization.yaml`, a file I had read: *"The generated name carries kustomize's content hash, so editing any runbook rolls the server automatically and a stale projection is impossible."* - `peers.yaml` (inline ConfigMap) → **`peers.conf`** (plain file) + a `configMapGenerator`. **Your enrolled peer is carried across verbatim** — I diffed the active stanza against `git show origin/main:…/peers.yaml` to be sure. Every line is WireGuard config and `#` is a legal comment there, so the documentation now travels *inside* the mounted file. - **`namespace: vpn-ingress` on the kustomization is required, not tidiness.** A generated ConfigMap inherits no namespace, everything else here declares `vpn-ingress`, and kustomize's `nameReference` won't rewrite across a namespace boundary. Without it the build emitted `ConfigMap wg-peers-m9kcckcb5t` while the DaemonSet still mounted the unhashed `wg-peers` — the pods would wedge on a missing volume. I only caught that by building the output and reading the volumes block. - **`new-peer.sh`**: `PEERS_FILE` repointed, and a missing register is now **fatal** rather than skipped. The old `[ -r "$PEERS_FILE" ] && grep …` guard meant that after the rename the only symptom would have been duplicate detection quietly never firing again. - Stale `peers.yaml` references swept from six files — including the script's own **user-facing output**, which told you to paste into "peers.yaml, under data.peers.conf". - The runbook gained the check that actually distinguishes this failure: the DaemonSet's `generation` must **increase** after a merge. ## Verified - **Reference rewritten end to end**: generated `ConfigMap wg-peers-m9kcckcb5t` in `vpn-ingress`, DaemonSet mounts `wg-peers-m9kcckcb5t`. Measured both ways — without `namespace:` it mounted the unhashed name. - **The hash tracks content**: `m9kcckcb5t` → add a peer → `dgg7ht9c74` → restore → `m9kcckcb5t`. So an edit necessarily changes the pod template. - **`new-peer.sh dupe 2` is now refused** because your real `.2` occupies it — the first time the duplicate check has run against real data rather than its own example. - Build still emits all five objects; the `Namespace` object's own name is untouched. - Gate: **rc 3**, all runnable stages green, two environmental skips. **Not green — reported as rc 3.** ## Not verified **That the roll actually happens** — the point of the change is unproven until Flux applies it. Success: `generation` increases, init log says `1 peer(s)`, `wg show` lists the peer. And **no handshake has ever been observed**, so the tunnel still hasn't carried a packet I can see. **Revocation** — the case where this was a security problem — is also untested. No sensitive paths. **Merges on a green gate, no override.** 🤖 Generated with [Claude Code](https://claude.com/claude-code)

gitea-admin added 1 commit 2026-10-06 07:56:38 +00:00

fix(vpn-ingress): content-hash the peer register, or a merged peer never reaches the tunnel

gate / gate (pull_request) Successful in 1m52s

Details

gate / conformance (pull_request) Has been skipped

Details

2acfed460a

The operator enrolled a phone, merged the peer, and `wg show` listed none.
Measured on prod 2026-10-06 with the peer already on main:

  live wg-peers ConfigMap, active AllowedIPs .. 0
  DaemonSet volume reference ................. configMap.name: wg-peers (FIXED)
  DaemonSet generation/observedGeneration .... 1 / 1   (never rolled)
  init container, all three pods ............. "0 peer(s)"

Part of it was just the 10m reconcile interval. The defect is the fixed name:
Flux updates a fixed-name ConfigMap IN PLACE, so the pod template is
byte-identical, so no rollout is triggered -- and the init container assembles
wg0.conf ONCE at pod start into a tmpfs, so the running tunnel keeps the old
peer list indefinitely. Adding or revoking a peer had no effect until something
unrelated restarted the pods. For a REVOCATION that is a security defect, not
an inconvenience: delete a stanza, see a green PR, device keeps working.

The runbook and README both claimed "PR -> merge -> Flux rolls the DaemonSet".
Never true. And this repo already documents the fix, in a file I had read --
runbooks/kustomization.yaml: "The generated name carries kustomize's content
hash, so editing any runbook rolls the server automatically and a stale
projection is impossible."

So: peers.yaml (inline ConfigMap) -> peers.conf (plain file) + a
configMapGenerator. The operator's enrolled peer is carried across verbatim.
Every line is WireGuard config and `#` is a legal comment there, so the
documentation now lives inside the file that is actually mounted.

`namespace: vpn-ingress` on the kustomization is REQUIRED, not tidiness: a
generated ConfigMap inherits no namespace, everything else here declares
vpn-ingress, and kustomize's nameReference will not rewrite across a namespace
boundary -- without it the build mounts the unhashed name and the pods wedge on
a missing volume. Caught only by building the output and reading the volumes.

new-peer.sh: PEERS_FILE repointed, and a missing register is now FATAL rather
than skipped -- the old `[ -r "$PEERS_FILE" ] && grep ...` guard meant that
after the rename the only symptom would have been duplicate detection quietly
never firing again. Stale peers.yaml references swept from six files including
the script's own user-facing output.

Verified: the DaemonSet now mounts wg-peers-m9kcckcb5t; the hash tracks content
(edit -> dgg7ht9c74 -> restore -> m9kcckcb5t); the enrolled peer is byte-identical
to main's; and `new-peer.sh dupe 2` is now correctly refused by the operator's
real allocation.

No sensitive paths; merges on a green gate.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

jakman85 merged commit ad37acdf53 into main

2026-10-06 07:58:49 +00:00

jakman85 referenced this issue from a commit

2026-10-06 07:58:50 +00:00

Merge pull request 'fix(vpn-ingress): content-hash the peer register, or a merged peer never reaches the tunnel' (#66) from fix/peers-configmap-content-hash into main

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Home-Network/home-network-k3s-impl#66