fix(vpn-ingress): keep the server key in source through SOPS, not as an out-of-band Secret
Operator correction: secrets are kept in source through sops. #55 shipped the WireGuard server private key as a Secret to `kubectl create` by hand. Two real costs, not just a convention break: * R-SEC-01 / spec docs/07 -- secrets live in source, SOPS-encrypted, and reach the cluster only through Flux. * It broke DR for the one credential that cannot be re-derived. A rebuild restores everything in git; it does not restore a hand-made Secret, and losing this key means re-enrolling EVERY device, because each client profile pins its public half. #55 papered over that with "keep a copy in your password manager". (Note secret-provenance did NOT forbid the original shape -- it scopes to app-namespace=true and vpn-ingress is deliberately a platform namespace. The rule broken was the written requirement, not an enforced control.) An agent still must not mint a long-lived private key (CLAUDE.md §6), so the file is committed ALREADY ENCRYPTED with `privatekey: REPLACE-ME`. That makes the wiring complete and green from the first commit -- resource line plus both overlay decryption patches -- leaving the operator exactly one act: replace the value. A commented-out resource to uncomment later is the "in git, no generator entry, reviewed, merged, still broken" shape this repo keeps hitting. The init guard is now a validation rather than an emptiness check: the value is piped through `wg pubkey`, which accepts only a real curve25519 private key, and the pod names which case it hit -- placeholder, empty, mangled, or (the silent one) still `ENC[` ciphertext because an overlay is missing its decryption patch, which otherwise applies cleanly with every Flux condition green. Second, independent finding that also fixes #55's step 2: `wg` is on the HOST only and `sops` is in the CONTAINER only, so `sops -d | wg pubkey` cannot run in one shell -- the command #55 documented could not have worked. Every step is now split by where it runs, and the written key is verified by comparing sha256 digests rather than printing anything. `qrencode` is on neither, so the QR step says so and gives the install command. Touches clusters/** and adds a **/*.sops.yaml, so approval-policy will block pending the operator's /approve-override. Deliberately not posted by me. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
jakman85andClaude Opus 5 committed
1 parent
95597bd688
commit
e35612dc7e
11 files changed
+471
-66
No files matched your search
@@ -8,8 +8,8 @@
|
||||
# NOTHING downstream reconciles — the incident behind the headlamp/oauth2-proxy/homepage moves
|
||||
# (DECISIONS-LOG: "Headlamp gating stall").
|
||||
# * SPECIFIC, AND THE REASON THIS IS NOT A JUDGEMENT CALL: this DaemonSet cannot start until the
|
||||
# operator creates the out-of-band `wg-server-key` Secret (runbooks/vpn-remote-access.md
|
||||
# step 1). That is the correct fail-closed state — an unenrolled VPN must not come up — but in
|
||||
# operator replaces the `REPLACE-ME` placeholder in platform/vpn-ingress/wg-server-key.sops.yaml
|
||||
# with a real key (runbooks/vpn-remote-access.md step 2). That is the correct fail-closed state — an unenrolled VPN must not come up — but in
|
||||
# the gating tier it would have been a self-inflicted platform-wide outage ON MERGE, with the
|
||||
# cause (a missing manual step) invisible from any Flux condition.
|
||||
#
|
||||
@@ -51,6 +51,9 @@ spec:
|
||||
substituteFrom:
|
||||
- kind: ConfigMap
|
||||
name: cluster-vars
|
||||
# NO `decryption:` stanza is needed in the overlays: this path carries no *.sops.yaml. The server
|
||||
# key is an out-of-band Secret (platform/vpn-ingress/kustomization.yaml says what to do if that
|
||||
# ever changes).
|
||||
# SOPS decryption is layered per-environment by the overlay (clusters/dev and clusters/prod each
|
||||
# patch THIS Kustomization by name), identical to headlamp/crafty/velero — this path carries
|
||||
# platform/vpn-ingress/wg-server-key.sops.yaml, the WireGuard server private key. The `sops-age`
|
||||
# Secret is created out of band and referenced by NAME only. Without the overlay patch the Secret
|
||||
# applies cleanly full of ENC[...] and only the tunnel is broken, so the patch is as load-bearing
|
||||
# as the encrypted file itself.
|
||||
@@ -122,6 +122,24 @@ patches:
|
||||
provider: sops
|
||||
secretRef:
|
||||
name: sops-age
|
||||
# vpn-ingress (the WireGuard remote-access VPN, clusters/base/vpn-ingress.yaml) carries
|
||||
# platform/vpn-ingress/wg-server-key.sops.yaml — the WireGuard SERVER private key, the one
|
||||
# secret in that component. Without this stanza kustomize hands Flux the raw ciphertext and the
|
||||
# Secret applies full of ENC[...]: the APPLY SUCCEEDS and every Flux condition stays green, and
|
||||
# the only symptom is a VPN that no client can handshake with. The DaemonSet's init container
|
||||
# validates the value with `wg pubkey` specifically so that state is a named CrashLoop instead.
|
||||
# The `sops-age` Secret is out-of-band (name only).
|
||||
- patch: |-
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: vpn-ingress
|
||||
namespace: flux-system
|
||||
spec:
|
||||
decryption:
|
||||
provider: sops
|
||||
secretRef:
|
||||
name: sops-age
|
||||
# pm-agent (the Program Manager agent service, clusters/base/pm-agent.yaml) carries
|
||||
# pm-agent-credentials.sops.yaml — the Gitea write token for the dedicated non-admin bot
|
||||
# account and the Anthropic subscription token from `claude setup-token`. Without this
|
||||
|
||||
@@ -113,6 +113,24 @@ patches:
|
||||
provider: sops
|
||||
secretRef:
|
||||
name: sops-age
|
||||
# vpn-ingress (the WireGuard remote-access VPN, clusters/base/vpn-ingress.yaml) carries
|
||||
# platform/vpn-ingress/wg-server-key.sops.yaml — the WireGuard SERVER private key, the one
|
||||
# secret in that component. Without this stanza kustomize hands Flux the raw ciphertext and the
|
||||
# Secret applies full of ENC[...]: the APPLY SUCCEEDS and every Flux condition stays green, and
|
||||
# the only symptom is a VPN that no client can handshake with. The DaemonSet's init container
|
||||
# validates the value with `wg pubkey` specifically so that state is a named CrashLoop instead.
|
||||
# The `sops-age` Secret is out-of-band (name only).
|
||||
- patch: |-
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: vpn-ingress
|
||||
namespace: flux-system
|
||||
spec:
|
||||
decryption:
|
||||
provider: sops
|
||||
secretRef:
|
||||
name: sops-age
|
||||
# pm-agent (the Program Manager agent service, clusters/base/pm-agent.yaml) carries
|
||||
# pm-agent-credentials.sops.yaml — the Gitea write token for the dedicated non-admin bot
|
||||
# account and the Anthropic subscription token from `claude setup-token`. Without this
|
||||
|
||||
@@ -0,0 +1,146 @@
|
||||
---
|
||||
date: 2026-10-05
|
||||
branch: fix/vpn-server-key-in-sops
|
||||
title: VPN server key moves into SOPS-encrypted git — the out-of-band Secret broke R-SEC-01 and the DR story
|
||||
---
|
||||
|
||||
## What changed
|
||||
|
||||
**Operator correction (2026-10-05): "secrets are kept in source through sops".** They are right, and
|
||||
#55 shipped the WireGuard server private key the wrong way: as a Secret the operator was told to
|
||||
`kubectl create` out of band, referenced by name only. This branch moves it into the tree as
|
||||
`platform/vpn-ingress/wg-server-key.sops.yaml`, where every other secret in this repo lives.
|
||||
|
||||
**Why the original choice was wrong, not merely unconventional.** Two concrete costs, both of which
|
||||
#55's own README half-admitted and then shipped anyway:
|
||||
|
||||
1. **R-SEC-01 / spec docs/07 says secrets live in source, SOPS-encrypted, reaching the cluster only
|
||||
through Flux.** An out-of-band Secret is outside the GitOps tree by construction.
|
||||
2. **It broke DR for the one credential that cannot be re-derived.** A cluster rebuild restores
|
||||
everything in git; it does not restore a hand-made Secret. Losing this key means **re-enrolling
|
||||
every device**, because each client profile pins its public half. #55 papered over this with
|
||||
"keep a copy in your password manager", which is a workaround for a self-inflicted gap.
|
||||
|
||||
Note `platform/kyverno/admission/secret-provenance.yaml` did **not** forbid the original shape —
|
||||
it scopes to `app-namespace=true` namespaces and `vpn-ingress` is deliberately a platform namespace,
|
||||
so admission would have accepted the hand-applied Secret. The rule it violated was the written
|
||||
requirement, not an enforced control. Recording that because "the gate let it through" was never
|
||||
evidence it was correct.
|
||||
|
||||
**How the unfillable-key problem is solved.** An agent still must not mint a long-lived private key
|
||||
(CLAUDE.md §6), so the file is committed **already encrypted, with `privatekey: REPLACE-ME`**. That
|
||||
makes the wiring complete and the gate green from the first commit — the resource line and both
|
||||
overlay decryption patches are present — and leaves the operator exactly one act: replace the value.
|
||||
The alternative, a commented-out resource line to uncomment later, is precisely the "file in git with
|
||||
no generator entry: reviewed, merged, apparently done, still broken" shape this repo keeps getting
|
||||
bitten by (`alert-runbooks-resolvable` exists because of it).
|
||||
|
||||
**The init-container guard is now a validation, not an emptiness check.** It was `[ -z "$key" ]`.
|
||||
Because the committed placeholder is non-empty, the likely failure is now a value that is present
|
||||
and *wrong*, so the guard pipes the value through `wg pubkey` — which accepts only a real 32-byte
|
||||
curve25519 private key — and names which of four cases it hit:
|
||||
|
||||
| case | why it matters |
|
||||
|---|---|
|
||||
| `REPLACE-ME` placeholder | the expected fresh-install state |
|
||||
| empty | blanked value |
|
||||
| **starts with `ENC[`** | **the Secret reached the cluster UNDECRYPTED.** A missing overlay decryption patch applies cleanly and leaves every Flux condition green — this is the silent failure the guard exists for |
|
||||
| rejected, none of the above | mangled value, or a *public* key pasted by mistake |
|
||||
|
||||
Files:
|
||||
|
||||
- `platform/vpn-ingress/wg-server-key.sops.yaml` (new) — SOPS-encrypted to the repo's age recipient,
|
||||
`stringData.privatekey` only, placeholder value.
|
||||
- `platform/vpn-ingress/wg-server-key.sops.yaml.example` (new) — plaintext template + operator
|
||||
hand-off, following the `crafty-oauth2-proxy-credentials.sops.yaml.example` convention.
|
||||
- `platform/vpn-ingress/kustomization.yaml` — the resource added; the "DELIBERATELY ABSENT: a
|
||||
Secret" block replaced with the overlay-patch warning.
|
||||
- `platform/vpn-ingress/daemonset.yaml` — the `wg pubkey` guard; volume comment rewritten.
|
||||
- `clusters/dev/kustomization.yaml`, `clusters/prod/kustomization.yaml` — **the `vpn-ingress`
|
||||
`decryption:` patch, in both.** This is the load-bearing half: #55's own kustomization comment
|
||||
predicted that forgetting it applies ciphertext cleanly.
|
||||
- `clusters/base/vpn-ingress.yaml` — its "NO decryption stanza is needed" comment was now false.
|
||||
- `platform/vpn-ingress/README.md`, `runbooks/vpn-remote-access.md` — step 2 rewritten.
|
||||
- `infra/vpn-ingress/new-peer.sh` — stale "step 1" pointer.
|
||||
|
||||
**A second finding, which changes the procedure independently of SOPS: `wg` and `sops` are not in
|
||||
the same place.** Measured: `wg` is on the **host** only (`/usr/bin/wg`, absent from the dev
|
||||
container); `sops` is in the **container** only (`/usr/local/bin/sops`, absent from the host). So
|
||||
`sops -d … | wg pubkey` cannot be run in one shell, and #55's step 2 — which assumed it could —
|
||||
would have failed. The runbook and the `.example` now split every command by where it runs, and
|
||||
verify the written key by comparing **sha256 digests** of the decrypted value and the operator's
|
||||
local copy, which never puts key material on a terminal. `qrencode` is on neither, so the QR
|
||||
instruction now says so and gives the install command.
|
||||
|
||||
**SENSITIVE PATHS — this PR touches `clusters/**` (two overlays) AND adds a `**/*.sops.yaml`, so
|
||||
`ci/approval-policy.sh` will BLOCK it.** It needs the operator's `/approve-override <reason>` as a
|
||||
PR **review** of type "Comment". I have deliberately not posted it (CLAUDE.md §1).
|
||||
|
||||
## What I verified
|
||||
|
||||
- **The encrypted file is real and correctly targeted.** `sops -e -i` in the pinned toolbox
|
||||
produced `stringData.privatekey: ENC[AES256_GCM,...]` with cleartext keys, `encrypted_regex
|
||||
^(data|stringData)$`, and exactly one age recipient:
|
||||
`age175l8mdpdfu7hq0aml3exs3gykarwjmuvr23sq56s3wjejnyk44wsp4mfzg` — the recipient in the root
|
||||
`.sops.yaml`, i.e. the key Flux holds as `sops-age`.
|
||||
- **Round-trip, without exposing anything**: `sops -d` then a Python read of key names only →
|
||||
`stringData keys: ['privatekey']`, and `value is still the placeholder: True`.
|
||||
- **Both overlay patches are present and name the right Kustomization** (`name: vpn-ingress`,
|
||||
`namespace: flux-system`, `decryption.secretRef.name: sops-age`), inserted after each overlay's
|
||||
`velero` stanza, mirroring headlamp/crafty/velero.
|
||||
- **`kubectl kustomize platform/vpn-ingress` builds** and now emits five objects — Namespace,
|
||||
ConfigMap, **Secret**, DaemonSet, NetworkPolicy — with `${WG_CLIENT_CIDR}` still an unexpanded
|
||||
token for Flux.
|
||||
- **`wg` / `sops` / `qrencode` locations**, measured on both host and container, as described above.
|
||||
- **Toolbox gate** from an isolated worktree: **rc 3**, with `spec-immutability`, `decisions-log`,
|
||||
`lint`, `l0`, `render-golden`, `l1` and `approval-policy` all **pass**; rc 3 is the two
|
||||
environmental skips (`gitea-settings` — no local token; conformance L2/L3 — no cluster).
|
||||
**Not green. Reported as rc 3, not as pass.**
|
||||
- **The L0 gate caught THREE real defects in this branch before it could merge — including one
|
||||
in the fix for another — and I had asserted one of them was fine without running it.** Both are recorded because the gate's value here is the
|
||||
whole point of the exercise:
|
||||
1. **`no-plaintext-secrets` FAILED**: "`wg-server-key.sops.yaml.example`: an unencrypted .example
|
||||
template is referenced by a kustomization — it would be applied as a plaintext Secret". My
|
||||
kustomization named the template in a *comment*, and the check matches comments too (correctly:
|
||||
naming one is how a plaintext Secret gets applied by accident). `platform/crafty/kustomization.yaml`
|
||||
never names its own example either — the convention was there to copy and I missed it. Fixed by
|
||||
not naming the file; the hand-off lives inside the example, where it belongs.
|
||||
**An earlier draft of this entry claimed this check passed. It did not. The claim was written
|
||||
from expectation rather than from a run.**
|
||||
2. **`flux-postbuild-substitution` FAILED** on `${key#ENC[}` in my new guard
|
||||
(`daemonset.yaml:155`), flagged `[SILENT-EMPTY]`: envsubst consumes every `${...}`, so my
|
||||
shell prefix-strip would have rendered to the empty string — the ciphertext branch of the guard
|
||||
would never have fired, which is precisely the silent failure the guard was added to catch.
|
||||
Rewritten with `case`, which uses no braces and so has nothing to escape.
|
||||
3. **And then it failed a THIRD time, on the fix's own comment** — `[FATAL-PARSE]
|
||||
daemonset.yaml:156 ${...}`, the prose explaining the rule, written with the literal form. That
|
||||
class is worse than it looks: a `FATAL-PARSE` hazard fails the **entire** Kustomization, every
|
||||
resource in the path, not just the line. Rewritten as "dollar-brace expansion". This is the
|
||||
second time in this component that writing the hazard literally inside a comment about the
|
||||
hazard has been the defect (the first was in #55), so the habit to build is: **in any
|
||||
substituted path, describe the construct in words, never type it.**
|
||||
- **After both fixes, `no-plaintext-secrets` and `flux-postbuild-substitution` pass** (446 and 198
|
||||
assertions respectively), and `l0` is green.
|
||||
|
||||
## What I did NOT verify
|
||||
|
||||
- **That Flux decrypts this file on prod.** The overlay patches have never reconciled. This is the
|
||||
whole risk of the change and it has exactly one observable outcome: after merge the pods should
|
||||
CrashLoop reporting the **`REPLACE-ME` placeholder** (proving decryption worked), *not* reporting
|
||||
**`ENC[`** (which would prove a patch is wrong). I built the guard to tell those apart because I
|
||||
cannot test them.
|
||||
- **That `wg pubkey` accepts the operator's real key through this path.** The guard is reasoned from
|
||||
`wg pubkey`'s behaviour and has never run — `wg` is absent from both this workstation's container
|
||||
and the gate, so the first execution will be in the pod.
|
||||
- **The `sops set` command in the hand-off.** I verified `sops -e -i` and `sops -d` in the toolbox;
|
||||
I did **not** run `sops set` with the quoting shown, and nested quoting inside
|
||||
`dev.sh run bash -c` is exactly where a documented command tends to break. If it misbehaves, the
|
||||
editor form (`sops <file>`) is the fallback and does not depend on quoting.
|
||||
- **The sha256 comparison in step 2 step 4** is untested end to end (it needs the real key).
|
||||
- **Whether anything else in #55 assumed a tool is available where it is not.** I checked `wg`,
|
||||
`sops` and `qrencode` because step 2 depended on them. I did not audit every command in the
|
||||
runbook the same way — and that class of defect has now produced two corrections in a row
|
||||
(`site.sh`, then `sops | wg`), so assume more remain until each step has actually been run.
|
||||
- **The live VPN still carries no traffic**, and the earlier gaps stand unchanged: the
|
||||
iptables-vs-nftables question, `allowPrivilegeEscalation: false` versus the image's s6 init, and
|
||||
any off-LAN reachability.
|
||||
@@ -89,7 +89,7 @@ command -v wg >/dev/null || die "wireguard-tools not installed ('wg' not on PATH
|
||||
if [ -z "${WG_SERVER_PUBKEY:-}" ]; then
|
||||
[ -r "$WG_SERVER_KEY_FILE" ] || die "cannot read server key '$WG_SERVER_KEY_FILE'.
|
||||
Either point WG_SERVER_KEY_FILE at it, or set WG_SERVER_PUBKEY to the public half.
|
||||
If the server key does not exist yet, see runbooks/vpn-remote-access.md step 1."
|
||||
If the server key does not exist yet, see runbooks/vpn-remote-access.md step 2."
|
||||
WG_SERVER_PUBKEY="$(tr -d ' \r\n' < "$WG_SERVER_KEY_FILE" | wg pubkey)"
|
||||
fi
|
||||
|
||||
|
||||
@@ -14,7 +14,7 @@ egress out the house WAN.
|
||||
| Where it runs | `hostNetwork` **DaemonSet** — one warm listener on every node |
|
||||
| Client subnet | `${WG_CLIENT_CIDR}` = `10.90.0.0/24`; `.1` is the server, `.2`–`.254` devices |
|
||||
| Authorisation | a device's **public key** is in [`peers.yaml`](./peers.yaml), or it cannot connect |
|
||||
| The one secret | the server's private key, Secret `wg-server-key`, **created out of band** |
|
||||
| The one secret | the server's private key — **SOPS-encrypted in this tree**, `wg-server-key.sops.yaml` |
|
||||
| Exposure | one hand-made WAN port-forward on the SOHO router, off-repo |
|
||||
| Enrolling a device | [`infra/vpn-ingress/new-peer.sh`](../../infra/vpn-ingress/new-peer.sh) |
|
||||
| Procedure / bring-up | [`runbooks/vpn-remote-access.md`](../../runbooks/vpn-remote-access.md) |
|
||||
@@ -130,11 +130,17 @@ translations win here.
|
||||
| half | where | why |
|
||||
|---|---|---|
|
||||
| peer **public** keys + addresses | [`peers.yaml`](./peers.yaml), **plaintext in git** | a public key grants nothing by itself, and this makes *"who can get into my house"* a reviewable PR diff with `git log` as its audit trail |
|
||||
| the server's **private** key | Secret `wg-server-key`, **out of band** | the only real secret; an agent must not mint a long-lived private key (CLAUDE.md §6) |
|
||||
| the server's **private** key | `wg-server-key.sops.yaml`, **SOPS-encrypted in git** | the only real secret (R-SEC-01, spec docs/07). Ships with a `REPLACE-ME` placeholder: an agent must not mint a long-lived private key (CLAUDE.md §6), so the operator `sops set`s it in |
|
||||
|
||||
The two are joined only at pod start, by the init container, into a **`medium: Memory` emptyDir** —
|
||||
so the assembled config holding the private key never touches any node's disk and dies with the pod.
|
||||
|
||||
Both halves are therefore in git, which is the point: a cluster rebuild restores the server key with
|
||||
everything else. An out-of-band `kubectl create secret` — the first shape this component shipped in,
|
||||
corrected after review — would have put the one credential every enrolled device depends on *outside*
|
||||
the GitOps tree, and losing it means re-enrolling every device, because each client profile pins this
|
||||
key's public half.
|
||||
|
||||
Encrypting the peer register would have hidden exactly the change a reviewer most needs to see: a
|
||||
new line granting a new device access to the whole LAN.
|
||||
|
||||
@@ -173,8 +179,14 @@ NetworkPolicy applies.
|
||||
(`infra/ansible/roles/node_baseline/defaults/main.yml`); the operator runs the role. The pod has
|
||||
`NET_ADMIN` but deliberately **not** `SYS_MODULE`, so it cannot load the module itself — it
|
||||
CrashLoops with a clear `wg-quick` error instead of quietly degrading to userspace WireGuard.
|
||||
2. **The `wg-server-key` Secret.** Runbook step 1. Until it exists the DaemonSet cannot start —
|
||||
the correct fail-closed state for an unenrolled VPN.
|
||||
2. **The server private key.** `wg-server-key.sops.yaml` is committed, encrypted, with a
|
||||
`REPLACE-ME` placeholder, and both overlays already carry its decryption patch — so the wiring
|
||||
is complete and the only outstanding act is replacing the value (runbook step 2). Until then the
|
||||
DaemonSet CrashLoops with an explicit FATAL naming the file: the init container validates the
|
||||
value with `wg pubkey`, which rejects the placeholder, an empty value, a public key pasted by
|
||||
mistake, **and** a Secret that arrived as undecrypted `ENC[...]` ciphertext. That last case is
|
||||
the one worth having a guard for — a missing overlay decryption patch applies cleanly and leaves
|
||||
every Flux condition green.
|
||||
3. **The WAN port-forward**, udp/51820 → one node address. Hand-made on the SOHO router, off-repo,
|
||||
and the half CI can never check.
|
||||
|
||||
|
||||
@@ -136,9 +136,38 @@ spec:
|
||||
# The key arrives as a Secret file; strip any trailing newline the operator's shell
|
||||
# added, or wg-quick rejects the config with "Key is not the correct length or format".
|
||||
key=$(tr -d ' \r\n' < /run/wg-server-key/privatekey)
|
||||
if [ -z "$key" ]; then
|
||||
echo "FATAL: /run/wg-server-key/privatekey is empty." >&2
|
||||
echo " See runbooks/vpn-remote-access.md step 1 (create the server key)." >&2
|
||||
# VALIDATE, don't just check for empty. The Secret ships committed with a REPLACE-ME
|
||||
# placeholder so the GitOps wiring is complete before the key exists, so the common
|
||||
# failure is a value that is present and WRONG — a placeholder, a public key pasted by
|
||||
# mistake, or a SOPS document that reached the cluster undecrypted (ENC[...], which
|
||||
# happens if an overlay is missing its `vpn-ingress` decryption patch). `wg pubkey`
|
||||
# accepts a value only if it is a real 32-byte curve25519 private key, so it rejects
|
||||
# all four. Without this the bad value would be written into wg0.conf and the pod would
|
||||
# fail later and less legibly, or come up with a server nobody can handshake with.
|
||||
if ! printf '%s' "$key" | wg pubkey >/dev/null 2>&1; then
|
||||
echo "FATAL: the server private key in Secret 'wg-server-key' is not a valid" >&2
|
||||
echo " WireGuard private key (wg pubkey rejected it)." >&2
|
||||
if [ -z "$key" ]; then
|
||||
echo " It is EMPTY." >&2
|
||||
elif [ "$key" = "REPLACE-ME" ]; then
|
||||
echo " It is still the committed REPLACE-ME PLACEHOLDER — the key has not" >&2
|
||||
echo " been generated yet. This is the expected state on a fresh install." >&2
|
||||
else
|
||||
# `case`, NOT a dollar-brace prefix-strip. This manifest is rendered through
|
||||
# envsubst, which consumes every dollar-brace expansion it sees, so a shell
|
||||
# parameter expansion here renders to the empty string and silently disables this
|
||||
# very branch. `case` uses no braces, so there is nothing to escape.
|
||||
# (ci: flux-postbuild-substitution caught both the original expansion AND the
|
||||
# first version of this comment, which spelled the hazard out literally.)
|
||||
case "$key" in
|
||||
ENC\[*)
|
||||
echo " It is still SOPS CIPHERTEXT (starts with ENC[). The Secret" >&2
|
||||
echo " reached the cluster undecrypted — check that this environment's" >&2
|
||||
echo " overlay carries the 'vpn-ingress' decryption patch." >&2
|
||||
;;
|
||||
esac
|
||||
fi
|
||||
echo " Fix: runbooks/vpn-remote-access.md step 2." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
@@ -272,14 +301,22 @@ spec:
|
||||
# during startup rather than at a point anyone would connect to the volume size. Still
|
||||
# counted against the container's 128Mi limit, and still far more than wg0.conf needs.
|
||||
sizeLimit: 8Mi
|
||||
# The ONLY secret in this component. Created OUT OF BAND by the operator and referenced by
|
||||
# name (R-IDP-04) — runbooks/vpn-remote-access.md step 1. It is NOT a *.sops.yaml in this
|
||||
# tree on purpose: an agent must not mint a long-lived private key (CLAUDE.md §6), and a key
|
||||
# that passed through a session transcript would have to be treated as compromised.
|
||||
# DR: Velero's `daily` schedule excludes only `monitoring` and `unifi`, so this namespace
|
||||
# and this Secret ARE captured off-cluster (platform/velero/helmrelease.yaml). Keep a copy
|
||||
# in the operator's password manager as well — restoring it is what makes every already
|
||||
# enrolled client work again without re-enrolling.
|
||||
# The ONLY secret in this component. SOPS-encrypted IN THIS TREE (R-SEC-01, spec docs/07) as
|
||||
# wg-server-key.sops.yaml, delivered by Flux, referenced here by name only (R-IDP-04).
|
||||
#
|
||||
# It ships committed with a REPLACE-ME placeholder: an agent must not mint a long-lived
|
||||
# private key (CLAUDE.md §6), so the operator generates it and `sops set`s it in
|
||||
# (runbooks/vpn-remote-access.md step 2). Shipping the encrypted placeholder rather than no
|
||||
# file at all is what lets the resource line and both overlay decryption patches be present
|
||||
# and GREEN from the first commit — the alternative, a commented-out resource to uncomment
|
||||
# later, is exactly the "file in git with no generator entry: reviewed, merged, apparently
|
||||
# done, still broken" shape this repo keeps getting bitten by.
|
||||
#
|
||||
# DR: in git means a cluster rebuild restores it with everything else — the property an
|
||||
# out-of-band `kubectl create secret` would NOT have had, and losing this key means
|
||||
# re-enrolling every device because each client profile pins its public half. Velero also
|
||||
# captures this namespace (its `daily` schedule excludes only `monitoring` and `unifi`).
|
||||
# Keep a password-manager copy too: that is the one that survives losing cluster AND backups.
|
||||
- name: wg-server-key
|
||||
secret:
|
||||
secretName: wg-server-key
|
||||
|
||||
@@ -4,9 +4,9 @@
|
||||
# same placement as crafty/headlamp/homepage/runbooks, for the same reason: a leaf whose pod can be
|
||||
# legitimately unhealthy must never sit in the gating `platform` tier, where one component's
|
||||
# unhealth stalls platform-config and every apps wave behind it (DECISIONS-LOG: "Headlamp gating
|
||||
# stall"). That rationale is unusually concrete here: until the operator creates the `wg-server-key`
|
||||
# Secret (runbooks/vpn-remote-access.md step 1) this DaemonSet CANNOT start, so putting it in the
|
||||
# gating tier would have taken the whole platform down on merge.
|
||||
# stall"). That rationale is unusually concrete here: until the operator replaces the `REPLACE-ME`
|
||||
# placeholder in wg-server-key.sops.yaml (runbooks/vpn-remote-access.md step 2) this DaemonSet
|
||||
# CANNOT start, so putting it in the gating tier would have taken the whole platform down on merge.
|
||||
#
|
||||
# NOT in platform/kustomization.yaml — see clusters/base/vpn-ingress.yaml.
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
@@ -19,12 +19,22 @@ resources:
|
||||
- daemonset.yaml
|
||||
# Namespace default-deny baseline. NOTE: does NOT constrain the hostNetwork DaemonSet — the file
|
||||
# says so at length rather than implying a protection it cannot provide.
|
||||
- networkpolicy.yaml
|
||||
- networkpolicy.yaml # The server's WireGuard private key — the ONE secret here. SOPS-encrypted (R-SEC-01, spec
|
||||
# docs/07), decrypted per-env by the `vpn-ingress` Kustomization, referenced by name only from
|
||||
# daemonset.yaml (R-IDP-04). Its plaintext `.example` sibling carries the operator hand-off --
|
||||
# deliberately NOT named here: conformance/tests/l0-static/no-plaintext-secrets fails any
|
||||
# kustomization that mentions an unencrypted template, because naming one is how a plaintext
|
||||
# Secret gets applied by accident. (It matches comments too. That is correct, not overzealous.)
|
||||
#
|
||||
# SHIPS WITH A `REPLACE-ME` PLACEHOLDER, already encrypted. An agent must not
|
||||
# mint a long-lived private key (CLAUDE.md §6), and the init container validates the value with
|
||||
# `wg pubkey`, so the unfilled state is a loud CrashLoop naming this file, never a server that
|
||||
# quietly cannot handshake.
|
||||
- wg-server-key.sops.yaml
|
||||
#
|
||||
# DELIBERATELY ABSENT: a Secret. The server's private key (`wg-server-key`) is created out of band
|
||||
# by the operator and referenced BY NAME from daemonset.yaml. There is no `*.sops.yaml` here, so
|
||||
# this component needs NO SOPS decryption stanza in clusters/dev or clusters/prod — unlike
|
||||
# crafty/headlamp/velero. If that key is ever migrated into SOPS, BOTH halves must land together:
|
||||
# the file here, and a `decryption:` patch naming the `vpn-ingress` Kustomization in each overlay.
|
||||
# Without the patch, kustomize hands Flux the raw ciphertext and the pod mounts a key full of
|
||||
# ENC[...] — which applies cleanly and fails only at handshake time.
|
||||
# THIS COMPONENT CARRIES AN ENCRYPTED SECRET, so BOTH overlays must patch the `vpn-ingress`
|
||||
# Kustomization with a `decryption:` stanza — clusters/dev and clusters/prod both do, added by the
|
||||
# same change as this file. That patch is as load-bearing as the file itself: without it kustomize
|
||||
# hands Flux the raw ciphertext, the Secret APPLIES CLEANLY full of ENC[...], and nothing in any
|
||||
# Flux condition looks wrong. The init container's `wg pubkey` guard exists partly to turn exactly
|
||||
# that into a named error rather than a silently non-working tunnel.
|
||||
@@ -0,0 +1,34 @@
|
||||
# WireGuard SERVER private key for platform/vpn-ingress — SOPS-ENCRYPTED (R-SEC-01, spec docs/07).
|
||||
# Keys stay cleartext for reviewable diffs; only `stringData` values are encrypted (.sops.yaml
|
||||
# `encrypted_regex`). Plaintext template + operator hand-off: wg-server-key.sops.yaml.example.
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: wg-server-key
|
||||
namespace: vpn-ingress
|
||||
type: Opaque
|
||||
stringData:
|
||||
#ENC[AES256_GCM,data:fUoPkfWvasisdTtwdhjeOVOlmdQkTl6Bp9YxYyji8g6k+ClATds3BnHJWOj2iplaGpT2XaNakq1+D4YyOi7QbFvNJui+e4qRpqcY8hWnazvZIHJYxjxfAfzW,iv:xtKd6oMZQumAQJAsNLHqJB+o0wdQJ+vL1kI1mc9jL7w=,tag:aImqLgbSmhsRI36/t5kpyg==,type:comment]
|
||||
#ENC[AES256_GCM,data:npDsCW6rQleolIpur5gFkX2jSRrIQzBlGaIPs2Schv0YU3q8AanG39z4QqT3ASDmskV5YuH29Bo0ujEAgBf52YIjQCp58PDDETofrVKOJzhHMa7bV+HCxUFE,iv:7E7thyFBleyqZpqJRGApKvIq3ubCFOkshF/mxim3YPc=,tag:joIrhMe2I6kUnx1ouOPdZw==,type:comment]
|
||||
#ENC[AES256_GCM,data:wHUFx09FgP/2+LTyw61EFbKA5UaqfjU7CeGk6Xc5av2y4r4AvQJRefDM5EGlahKecDFEFsf7fSQRFRHtUJqrQTtw+6tcj3ubSCPW4iJvhyJJv0xdb/0puRmcgj+2NfgF,iv:xgDOSDLqMQ1wcI9vrkKqfNMgIDFEusFnnO7Rzq/aOa4=,tag:m1m/fgjkBsG91c+3daqOpA==,type:comment]
|
||||
privatekey: ENC[AES256_GCM,data:irLENWa1Z3IJSQ==,iv:Qu2Yr6F5+M6P8cQRuhB5J7joxCcudUhwoEWDiL941Lo=,tag:ZNwcBZEkswtnkX0vGKfMAQ==,type:str]
|
||||
sops:
|
||||
kms: []
|
||||
gcp_kms: []
|
||||
azure_kv: []
|
||||
hc_vault: []
|
||||
age:
|
||||
- recipient: age175l8mdpdfu7hq0aml3exs3gykarwjmuvr23sq56s3wjejnyk44wsp4mfzg
|
||||
enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBMays2OXY0VkpRc2FQQmhU
|
||||
Q0owM3N0VkhYR0lYTUdNZWRqeTFIMjNHOVJ3CnJSMnc5NFEyU0IzWTFtUkw2YVFS
|
||||
MU02VFo1TE14YVpZRzNhVjZwRTZwd0UKLS0tIDFaMWF4NUZjZmhRdXRuZFJYNkg3
|
||||
SUUzN1FnMlI4bkkzbVNlMWhUVnJZODAKQ+Iy86CIv/Oj9E9j4E/0y5hDbcctRuqB
|
||||
OR825KRPRaTdF/G2hudE1R1JbM4wGbPLMzv078R4ZXHDPGvDttSEkA==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
lastmodified: "2026-10-06T04:20:58Z"
|
||||
mac: ENC[AES256_GCM,data:GpP/zTX4FtJyrlaCS/R0/sAtFbkM0W8A5sqNxb5ZziSgLUXI+oUKYB09igN6vx5cjRNH7rSmXduS/actaT5jhP8kuF1n/iQfG56PsCJHL2vneefzcEOKXeGquAwDOY+iGmNsZ5R8ySrM/Fq7HYLMJasWosyTh3YlBs8GRYiHmU4=,iv:438+pDzHLGyh+OMQCMzGBEBrZUPNp5BbjrrMdiEiqJ8=,tag:JqLu6zZWKtcoB8mp4HSZig==,type:str]
|
||||
pgp: []
|
||||
encrypted_regex: ^(data|stringData)$
|
||||
version: 3.9.4
|
||||
@@ -0,0 +1,82 @@
|
||||
# =============================================================================
|
||||
# TEMPLATE — NOT APPLIED, NOT ENCRYPTED (.yaml.example is ignored by kustomize and never
|
||||
# auto-encrypted by SOPS). Reference scaffold + OPERATOR HAND-OFF.
|
||||
#
|
||||
# WHAT THIS IS: the WireGuard SERVER's long-term private key for the remote-access VPN
|
||||
# (platform/vpn-ingress). It is the ONE secret in the whole component. The init container in
|
||||
# daemonset.yaml reads it at `/run/wg-server-key/privatekey` and joins it to the PLAINTEXT peer
|
||||
# register (peers.yaml) in a tmpfs, so the assembled wg0.conf never touches a node's disk.
|
||||
#
|
||||
# WHY IT IS IN GIT, SOPS-ENCRYPTED, AND NOT HAND-CREATED. R-SEC-01 / spec docs/07: secrets live in
|
||||
# source, encrypted, and reach the cluster only through Flux. An out-of-band `kubectl create secret`
|
||||
# would put the single credential every enrolled device depends on OUTSIDE the GitOps tree, where a
|
||||
# cluster rebuild silently loses it — and losing it means RE-ENROLLING EVERY DEVICE, because each
|
||||
# client profile pins this key's public half.
|
||||
#
|
||||
# *** THE REAL FILE IS COMMITTED WITH `privatekey: REPLACE-ME`, ALREADY ENCRYPTED. ***
|
||||
# That is deliberate, so the wiring is complete and green before the key exists: the resource is
|
||||
# already listed in kustomization.yaml and both overlays already carry the `vpn-ingress` decryption
|
||||
# patch. Your job is ONLY to replace the value (step 3 below). Until you do, the DaemonSet
|
||||
# CrashLoops with an explicit FATAL naming this file — the init container validates the value with
|
||||
# `wg pubkey` before writing wg0.conf, so a placeholder fails CLOSED and LOUDLY rather than
|
||||
# producing a server that silently cannot complete a handshake.
|
||||
#
|
||||
# ---------------------------------------------------------------------------------------------
|
||||
# MIND WHERE EACH COMMAND RUNS. The two tools do not live in the same place (measured 2026-10-05):
|
||||
# wg -> the HOST only (/usr/bin/wg; NOT in the dev container)
|
||||
# sops -> the dev CONTAINER only (/usr/local/bin/sops; NOT on the host)
|
||||
# So you cannot pipe `sops -d ... | wg pubkey` in one shell. The steps below are split accordingly,
|
||||
# and they never put the key on a terminal or in a pipeline that crosses the boundary.
|
||||
# ---------------------------------------------------------------------------------------------
|
||||
#
|
||||
# HAND-OFF STEPS (operator — an agent must not mint a long-lived private key, CLAUDE.md §6).
|
||||
# Full procedure with verification: runbooks/vpn-remote-access.md step 2.
|
||||
#
|
||||
# 1. HOST — generate it. This file is also what infra/vpn-ingress/new-peer.sh reads to derive the
|
||||
# public half for each client profile, so keep it where the script's default expects it:
|
||||
# umask 077
|
||||
# mkdir -p ~/.config/wireguard-home && chmod 700 ~/.config/wireguard-home
|
||||
# wg genkey > ~/.config/wireguard-home/server.key
|
||||
#
|
||||
# 2. Put that value in your PASSWORD MANAGER. It cannot be regenerated without invalidating every
|
||||
# client profile. (Velero backs this namespace up to R2 too, but the password-manager copy is
|
||||
# the one that survives losing the cluster AND the backups.)
|
||||
# Record the public half alongside it — safe to display, and useful when debugging a client:
|
||||
# wg pubkey < ~/.config/wireguard-home/server.key
|
||||
#
|
||||
# 3. Stage it where the CONTAINER can read it, then write it into the encrypted file in place:
|
||||
# mkdir -p .cache && cp ~/.config/wireguard-home/server.key .cache/server.key
|
||||
# git check-ignore -v .cache/server.key # MUST print `.gitignore:11:*.key` — see below
|
||||
# ./infra/devcontainer/dev.sh run bash -c \
|
||||
# 'cd /work && sops set platform/vpn-ingress/wg-server-key.sops.yaml \
|
||||
# "[\"stringData\"][\"privatekey\"]" "\"$(tr -d "\r\n" < /work/.cache/server.key)\""'
|
||||
# shred -u .cache/server.key
|
||||
# `sops set` edits the ENCRYPTED document directly — no plaintext copy of it is ever written.
|
||||
#
|
||||
# KEEP THE `.key` SUFFIX on that staged copy. `.gitignore` line 11 is `*.key`, and it is the
|
||||
# ONLY rule covering that path: `.cache/` itself is NOT ignored. A copy named `server.pem` or
|
||||
# `privatekey` in the same directory is COMMITTABLE, and the gate's `no-plaintext-secrets` stage
|
||||
# does not recognise a bare WireGuard key.
|
||||
#
|
||||
# 4. Verify you wrote the right bytes WITHOUT printing them — compare digests, not values:
|
||||
# ./infra/devcontainer/dev.sh run bash -c \
|
||||
# 'cd /work && sops -d --extract "[\"stringData\"][\"privatekey\"]" \
|
||||
# platform/vpn-ingress/wg-server-key.sops.yaml | tr -d "\r\n" | sha256sum'
|
||||
# tr -d '\r\n' < ~/.config/wireguard-home/server.key | sha256sum
|
||||
# The two digests must match. (This is the only sanctioned decrypt of this file, and its output
|
||||
# goes straight into sha256sum — never onto a terminal. CLAUDE.md §6.)
|
||||
#
|
||||
# 5. PR it. Nothing else to wire: the resource line and both overlay decryption patches already
|
||||
# exist. Flux rolls the DaemonSet and the pods leave CrashLoop.
|
||||
# =============================================================================
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: wg-server-key
|
||||
namespace: vpn-ingress
|
||||
type: Opaque
|
||||
stringData:
|
||||
# The server's WireGuard private key: `wg genkey` output — 44 chars of base64 ending in '='.
|
||||
# NOT a passphrase, and NOT the public key. The init container validates it with `wg pubkey`
|
||||
# before writing wg0.conf, so a placeholder or a mangled value fails closed with a clear message.
|
||||
privatekey: REPLACE-ME
|
||||
@@ -62,53 +62,90 @@ lsmod | grep '^wireguard'
|
||||
grep -w wireguard /etc/modules-load.d/k3s-cilium-longhorn.conf
|
||||
```
|
||||
|
||||
## Step 2 — create the server key (once)
|
||||
## Step 2 — fill in the server key (once)
|
||||
|
||||
This is the **only** secret in the component, and the one thing that must never be generated in an
|
||||
agent session or pasted into a transcript. Generate it on your own machine:
|
||||
The key is **SOPS-encrypted in this tree** (R-SEC-01, spec docs/07), not created by hand:
|
||||
`platform/vpn-ingress/wg-server-key.sops.yaml`. It is already committed — encrypted, with a
|
||||
`REPLACE-ME` placeholder — and it is already listed in the component's `resources:` with a
|
||||
`decryption:` patch in both overlays. **Nothing needs wiring. You are only replacing one value.**
|
||||
|
||||
Why in git rather than `kubectl create secret`: an out-of-band Secret puts the one credential every
|
||||
enrolled device depends on outside the GitOps tree, where a cluster rebuild silently loses it — and
|
||||
losing it means re-enrolling every device, because each client profile pins this key's public half.
|
||||
|
||||
> **MIND WHERE EACH COMMAND RUNS.** The two tools are not in the same place (measured 2026-10-05):
|
||||
> `wg` is on the **HOST** only (`/usr/bin/wg`, absent from the dev container); `sops` is in the
|
||||
> **CONTAINER** only (`/usr/local/bin/sops`, absent from the host). You cannot pipe
|
||||
> `sops -d … | wg pubkey` in one shell. The steps below are split accordingly and never put the key
|
||||
> on a terminal.
|
||||
|
||||
**1. HOST — generate it.** Keep it at this path: it is where `new-peer.sh` looks by default to
|
||||
derive the public half for each client profile.
|
||||
|
||||
```bash
|
||||
mkdir -p ~/.config/wireguard-home && chmod 700 ~/.config/wireguard-home
|
||||
umask 077
|
||||
wg genkey > ~/.config/wireguard-home/server.key # keep this; see "Back it up" below
|
||||
mkdir -p ~/.config/wireguard-home && chmod 700 ~/.config/wireguard-home
|
||||
wg genkey > ~/.config/wireguard-home/server.key
|
||||
```
|
||||
|
||||
Load it into the cluster. **Do not echo it, do not pass it on an interactive command line that
|
||||
lands in shell history, and never print the resulting Secret's `.data`** (CLAUDE.md §6):
|
||||
**2. Store it in your PASSWORD MANAGER**, together with its public half (safe to display):
|
||||
|
||||
```bash
|
||||
wg pubkey < ~/.config/wireguard-home/server.key
|
||||
```
|
||||
|
||||
This copy is the one that survives losing the cluster *and* the backups. Velero does capture this
|
||||
namespace to R2 (its `daily` schedule excludes only `monitoring` and `unifi`), and the encrypted
|
||||
file is in git — but the key itself cannot be regenerated without invalidating every client profile.
|
||||
|
||||
**3. Write it into the encrypted file in place.**
|
||||
|
||||
```bash
|
||||
mkdir -p .cache && cp ~/.config/wireguard-home/server.key .cache/server.key
|
||||
git check-ignore -v .cache/server.key # MUST print `.gitignore:11:*.key` — see the warning below
|
||||
|
||||
./infra/devcontainer/dev.sh run bash -c \
|
||||
'cd /work && sops set platform/vpn-ingress/wg-server-key.sops.yaml \
|
||||
"[\"stringData\"][\"privatekey\"]" "\"$(tr -d "\r\n" < /work/.cache/server.key)\""'
|
||||
|
||||
shred -u .cache/server.key
|
||||
```
|
||||
|
||||
`sops set` edits the **encrypted** document directly, so no plaintext copy of it is ever written.
|
||||
(`sops platform/vpn-ingress/wg-server-key.sops.yaml` opens an editor instead, if you prefer.)
|
||||
|
||||
> **KEEP THE `.key` SUFFIX** on the staged copy. `.gitignore` line 11 is `*.key`, and it is the
|
||||
> **only** rule covering that path — `.cache/` itself is NOT ignored
|
||||
> (`git check-ignore -v .cache/serverkey.txt` matches nothing). A copy named `server.pem` or
|
||||
> `privatekey` in that directory is **committable**, and the gate's `no-plaintext-secrets` stage
|
||||
> does not recognise a bare WireGuard key.
|
||||
|
||||
**4. Verify you wrote the right bytes, without printing them** — compare digests, not values:
|
||||
|
||||
```bash
|
||||
./infra/devcontainer/dev.sh run bash -c \
|
||||
'kubectl --context k3s-prod -n vpn-ingress create secret generic wg-server-key \
|
||||
--from-file=privatekey=/work/.cache/server.key'
|
||||
'cd /work && sops -d --extract "[\"stringData\"][\"privatekey\"]" \
|
||||
platform/vpn-ingress/wg-server-key.sops.yaml | tr -d "\r\n" | sha256sum'
|
||||
|
||||
tr -d '\r\n' < ~/.config/wireguard-home/server.key | sha256sum
|
||||
```
|
||||
|
||||
…where `/work/.cache/server.key` is a path you copied the key to (`.cache/` is not tracked; create
|
||||
it if absent). Delete that copy as soon as the Secret exists: `shred -u .cache/server.key`.
|
||||
The two digests must match. This is the only sanctioned decrypt of this file, and its output goes
|
||||
straight into `sha256sum` — never onto a terminal (CLAUDE.md §6).
|
||||
|
||||
> **KEEP THE `.key` SUFFIX.** That filename is what keeps the key out of git: `.gitignore` line 11
|
||||
> is `*.key`, and it is the **only** rule covering this path — `.cache/` itself is NOT ignored
|
||||
> (`git check-ignore -v .cache/serverkey.txt` matches nothing). So a copy named `server.pem`,
|
||||
> `wg-server` or `privatekey` sitting in that directory is **committable**, and the gate's
|
||||
> `no-plaintext-secrets` stage does not know a WireGuard private key when it sees one. Verify before
|
||||
> you trust it: `git check-ignore -v .cache/<your-filename>` must print a rule, and
|
||||
> `git status --short` must not list the file.
|
||||
**5. PR it, and merge.** Flux rolls the DaemonSet and the pods leave CrashLoop.
|
||||
|
||||
Confirm it exists **by key name only**:
|
||||
### What the pod tells you if this went wrong
|
||||
|
||||
```bash
|
||||
./infra/devcontainer/dev.sh run bash -c \
|
||||
'kubectl --context k3s-prod -n vpn-ingress get secret wg-server-key -o jsonpath="{.data}" | tr "," "\n" | cut -d: -f1'
|
||||
# expected: {"privatekey -- a NAME. If you see base64 payload, you printed the key. Rotate it.
|
||||
```
|
||||
The init container validates the value with `wg pubkey` before writing `wg0.conf`, and names the
|
||||
specific failure in its log — so you never get a server that silently cannot complete a handshake:
|
||||
|
||||
**Why out of band and not SOPS.** An agent must not mint a long-lived private key, and a key that
|
||||
passed through a session transcript would have to be treated as compromised. The trade-off is that
|
||||
this Secret is **not reconstructible from git** — so:
|
||||
|
||||
**Back it up.** Keep `server.key` in your password manager. Velero's `daily` schedule excludes only
|
||||
`monitoring` and `unifi` (`platform/velero/helmrelease.yaml`), so this namespace *is* captured
|
||||
off-cluster to R2 as well. Restoring this one key is what makes every already-enrolled device work
|
||||
again after a cluster rebuild — **losing it means re-enrolling every device.**
|
||||
| pod log says | meaning |
|
||||
|---|---|
|
||||
| `still the committed REPLACE-ME PLACEHOLDER` | step 3 not done (the expected fresh-install state) |
|
||||
| `It is EMPTY` | the value was blanked |
|
||||
| `still SOPS CIPHERTEXT (starts with ENC[)` | the Secret reached the cluster undecrypted — this environment's overlay is missing its `vpn-ingress` decryption patch. **This is the dangerous one:** the Secret applies cleanly and every Flux condition stays green |
|
||||
| `wg pubkey rejected it` with none of the above | a mangled value, or a *public* key pasted by mistake |
|
||||
|
||||
## Step 3 — forward udp/51820 on the router (once)
|
||||
|
||||
@@ -153,8 +190,16 @@ unauthenticated traffic either way. If the pod is Ready, the interface exists.
|
||||
./infra/vpn-ingress/new-peer.sh thinkpad 3 --split # split tunnel (LAN only)
|
||||
```
|
||||
|
||||
It generates the keypair **on your machine**, writes a `0600` client profile outside the checkout,
|
||||
prints a QR code if `qrencode` is installed, and prints the `[Peer]` stanza to add. Then:
|
||||
It generates the keypair **on your machine**, writes a `0600` client profile outside the checkout
|
||||
(it refuses to write inside the git checkout), prints a QR code, and prints the `[Peer]` stanza to
|
||||
add.
|
||||
|
||||
> **`qrencode` is installed neither on this workstation nor in the dev container** (measured
|
||||
> 2026-10-05), so the QR path needs it first: `sudo dnf install qrencode` (or `apt install
|
||||
> qrencode`). Without it the script says so plainly and leaves you to move the `.conf` to the
|
||||
> device by another route — fine for a desktop, awkward for a phone.
|
||||
|
||||
Then:
|
||||
|
||||
1. Paste the stanza into `platform/vpn-ingress/peers.yaml` under `data.peers.conf`, keeping the
|
||||
`# <name> — <owner>, enrolled <date>` comment. `wg show` can only ever print the key, so that
|
||||
|
||||
Reference in new issue
Block a user