jakman85andClaude Opus 5.5
250fb54154
The operator ruled the implementation repo's main (50c8719) the latest and accepted state. Sync the spec to it. Accepted changes are reflected: one bare-metal environment, TrueNAS bulk storage, R2 off-cluster backup, CloudNativePG, single-node VPN gateway with Cilium fail-closed, inbound WireGuard, per-app auth sidecars, Renovate, the CI gate as approver, and the pm-agent. New ADR-0017..0025; ADR-0016 corrected and accepted; R-* IDs unchanged, eight amended. Where the implementation has simply not met a requirement, the requirement stands and the shortfall is recorded in docs/16-known-gaps.md. Co-Authored-By: Claude Opus 5.5 <[email protected]>
68 lines
4.4 KiB
Markdown
68 lines
4.4 KiB
Markdown
# ADR-0021 — Inbound remote access: WireGuard on a hostNetwork DaemonSet
|
|
- Status: Accepted
|
|
- Date: 2026-11-08
|
|
- Accepted by: operator ruling of 2026-11-08 that the implementation is the accepted state; effective on merge.
|
|
- Serves: remote operator access (no dedicated requirement ID); bounds R-NET-01, R-SEC-02, R-SEC-04
|
|
- Supersedes: —
|
|
|
|
## Context
|
|
The operator's Android, Linux, Windows and other devices need to reach the home network from anywhere
|
|
as though on the LAN. Several services are LAN/VPN-only by design (ADR-0016), and the Kubernetes API is
|
|
LAN/VPN-only (R-SEC-04), so the "VPN" in that phrase needs a defined mechanism. This is unrelated to
|
|
`vpn`-class egress (ADR-0015, ADR-0020), which sends pods out through a provider tunnel.
|
|
|
|
Two facts about this cluster shape the design:
|
|
- Cilium decides what a packet may reach from the **security identity of its source**. Measured: a
|
|
request to the shared Gateway from inside a pod is refused for every Gateway hostname. A
|
|
pod-networked VPN server would give every client that pod's identity: a healthy tunnel on which
|
|
every web UI is refused.
|
|
- The SOHO edge has no automation and cannot hairpin NAT. An inbound forward is a manual step that names
|
|
one node address.
|
|
|
|
## Decision
|
|
**Remote access is WireGuard on udp/51820, run as a Flux-managed DaemonSet in the host network
|
|
namespace on every node.**
|
|
|
|
- **Host network** so that clients never carry a pod identity.
|
|
- **DaemonSet** so that every node holds an identical warm listener, and losing the forwarded node is a
|
|
one-line router change with no cluster work and no new client profiles.
|
|
- **Authorization is a reviewable git diff:** peer public keys are in plain text in git, so granting or
|
|
revoking a device is a reviewed change that rolls the listeners. The server's private key is
|
|
SOPS-encrypted (ADR-0007), and joined with the peer list only in memory at pod start.
|
|
- **Reach equals a LAN machine:** the whole LAN, the Gateway, Pi-hole, and the internet through the
|
|
house WAN. Clients are `internet` class, never steered to the provider tunnel.
|
|
- **Endpoint:** the DNS-only `wan.` record (ADR-0016). A client inside the house cannot use it, as there
|
|
is no hairpin, and uses a node's LAN address instead.
|
|
|
|
**This is a deliberate exception to default-deny (R-NET-01).** A host-network workload has no Cilium
|
|
endpoint, so no network policy applies to it. The exception is scoped to this platform component;
|
|
application namespaces remain barred from the host network by admission policy. The boundary is
|
|
WireGuard's peer authentication: a packet from an unknown key gets no reply, so the port is
|
|
indistinguishable from a closed one.
|
|
|
|
## Alternatives considered
|
|
- **OpenVPN** — rejected: a CA, per-device certificates and a revocation list are long-lived state to
|
|
protect and rotate.
|
|
- **IKEv2/IPsec** — rejected: painful to containerize and brittle behind double NAT.
|
|
- **Pod-networked WireGuard** — rejected: clients inherit the pod's identity and every Gateway hostname
|
|
refuses them, while the tunnel reports healthy.
|
|
- **A LoadBalancer VIP for the listener** — rejected: adds an L2-announcement lease, SNAT and a
|
|
node-to-node hop to the one path whose failure means the operator is locked out while away.
|
|
- **Single-replica Deployment** — rejected: a reschedule silently invalidates the router forward.
|
|
- **Carry it over the Cloudflare Tunnel** — not possible: the tunnel carries HTTP(S), and WireGuard is
|
|
UDP to a node.
|
|
- **Encrypt the peer list / add preshared keys** — rejected: hides the change a reviewer most needs to
|
|
see, and a preshared key would be a further secret for a threat not in the model.
|
|
|
|
## Consequences
|
|
- **No network policy constrains this workload.** Peer authentication and the single forwarded port are
|
|
the controls.
|
|
- **An enrolled device is as privileged as a machine in the house**, including reach to the nodes'
|
|
control-plane ports. A stolen device is that device until revoked, so revocation must be prompt.
|
|
Narrowing is possible per profile (split tunnel) or by host firewall rules in the listener.
|
|
- The WAN IP stays public for this port (ADR-0016); the forward is a manual edge step.
|
|
- The tunnel carries IPv4 only. Client profiles default to a conservative MTU, because a client's own
|
|
path can be narrower than the server's.
|
|
- No per-peer metrics exist; a WireGuard exporter would need a second privileged container beside the
|
|
internet-facing tunnel.
|