Merge pull request 'fix(vpn): enrol keith-phone-home and admit the WireGuard subnet to PiHole DNS' (#158) from fix/wg-home-peer-pihole-sources into main
Reviewed-on: #158
This commit was merged in pull request #158.
This commit is contained in:
jakman85 committed
commit
da0515c8c8
7 files changed
+363
-17
No files matched your search
@@ -42,8 +42,9 @@ no version string to match against a CVE, and no amplification to abuse.
|
||||
|
||||
That is the whole security argument, and it is why the exposed port carries no IP allowlist (which
|
||||
would also have made roaming clients unusable — a phone's address changes with every cell tower).
|
||||
Compare `apps/wave-0/pihole-dns.yaml`, which *does* pin `loadBalancerSourceRanges` to the LAN: a
|
||||
resolver answers anybody, so it must be scoped by address. WireGuard answers nobody.
|
||||
Compare `apps/wave-0/pihole-dns.yaml`, which *does* pin `loadBalancerSourceRanges` to the LAN and
|
||||
this tunnel's client subnet: a resolver answers anybody, so it must be scoped by address. WireGuard
|
||||
answers nobody.
|
||||
|
||||
What it does **not** protect against is a stolen device — anyone holding a device's private key
|
||||
*is* that device. That is what revocation is for: delete the stanza, merge, done (~1 min).
|
||||
@@ -55,9 +56,9 @@ What it does **not** protect against is a stolen device — anyone holding a dev
|
||||
┌──────────────┐ udp/51820 ┌───────────────┐ forward ┌────────────────────────────┐
|
||||
│ WireGuard app│─────────────▶│ WAN 51820 ──▶ │───────────▶│ wg0 10.90.0.1/24 │
|
||||
│ 10.90.0.2/32 │ over the │ node IP:51820 │ │ decrypt, then FORWARD │
|
||||
└──────────────┘ internet └───────────────┘ │ SNAT to the node's addr │
|
||||
└──────────────┘ internet └───────────────┘ │ SNAT (LAN), eBPF LB (VIPs)│
|
||||
└─────────────┬──────────────┘
|
||||
│ as `host`/`remote-node`
|
||||
│ translated per destination (below)
|
||||
┌─────────────────────────────────┼──────────────────────┐
|
||||
▼ ▼ ▼
|
||||
LAN 192.168.2.0/24 Gateway VIP .20 (16 web UIs) WAN / internet
|
||||
@@ -86,9 +87,21 @@ resolved, because `apps/wave-0/pihole-dns.yaml`'s ingress policy deliberately om
|
||||
entity. A VPN that looks up and is useless is the worst available outcome, and nothing in the
|
||||
manifests would have hinted at it.
|
||||
|
||||
In the **host** netns there is no pod identity to carry. Forwarded client traffic is SNATed to the
|
||||
node's own address, so it reaches the Gateway, PiHole and every LAN host as `host`/`remote-node` —
|
||||
byte-for-byte what a LAN client looks like. Three further problems vanish with it:
|
||||
In the **host** netns there is no pod identity to carry, which is what avoids the 403s. Clients do
|
||||
**not**, however, all arrive SNATed to the node. Measured 2026-11-08: Cilium counts `wg0` among its
|
||||
kube-proxy-replacement devices (`cilium-dbg status`: `eno1`, `wg0`, `tun0`), so what a packet looks
|
||||
like depends on where it is going:
|
||||
|
||||
| destination | handled by | source the destination sees |
|
||||
|---|---|---|
|
||||
| LAN hosts, the internet | kernel FORWARD, then the iptables MASQUERADE in [`daemonset.yaml`](./daemonset.yaml) | the node's address |
|
||||
| LoadBalancer VIPs (Gateway `.20`, PiHole `.23`, ...) | Cilium's eBPF load balancer, as the packet enters from `wg0`, **before** MASQUERADE | `10.90.0.x` at the VIP's source-range check, identity `world` |
|
||||
|
||||
The second row is why PiHole's `loadBalancerSourceRanges` has to list `${WG_CLIENT_CIDR}` as well as
|
||||
the LAN: left LAN-only, tunnel DNS is predicted to die with `DROP_NOT_IN_SRC_RANGE`. That prediction
|
||||
and the row's source column come from reading Cilium's datapath source; **no peer has ever completed
|
||||
a handshake, so neither has been observed live.** The Gateway VIP carries no source ranges and needs
|
||||
no change. Three further problems vanish with the host netns:
|
||||
|
||||
* `net.ipv4.ip_forward` is **already 1** on these nodes (`node_baseline` writes
|
||||
`/etc/sysctl.d/99-k3s-cilium.conf`). It is *not* in the kubelet's safe-sysctl allowlist, so a
|
||||
@@ -163,6 +176,30 @@ internet path. To enforce less *server-side*, add filtering to the `PostUp` chai
|
||||
before the accept rules) — the FORWARD chain there is the only enforcement point, since no
|
||||
NetworkPolicy applies.
|
||||
|
||||
## Phone profiles: one tunnel for home, one for away
|
||||
|
||||
A phone that needs the LAN-only apps runs full tunnel (`AllowedIPs = 0.0.0.0/0`, `DNS = 192.168.2.23`)
|
||||
everywhere, so it needs **two tunnels**, each its own peer with its own key and address:
|
||||
|
||||
| tunnel | peer | address | Endpoint | use when |
|
||||
|---|---|---|---|---|
|
||||
| `away` | `keith-phone` | `10.90.0.2` | `wan.home.keithjackson.dev:51820` | off the home network |
|
||||
| `home` | `keith-phone-home` | `10.90.0.3` | `192.168.2.10:51820` | on the home Wi-Fi |
|
||||
|
||||
**`wan.` cannot work from inside the house.** It is an unproxied A record holding the house's WAN
|
||||
address (`platform/externaldns/README.md`), and the router has no hairpin NAT, so a LAN client that
|
||||
sends to it never comes back in through the port-forward. The Cloudflare Tunnel is no way round: it
|
||||
carries HTTP(S) to the Gateway and WireGuard is UDP to a node, so `wan.` has to stay DNS-only and
|
||||
`away` has to keep using it.
|
||||
|
||||
**`home` depends on `k3s-prod-1`.** Its Endpoint is that node's address, with no failover: if
|
||||
prod-1 is down, `home` cannot handshake even though the other two listeners are healthy. The fix
|
||||
is client-side: edit the tunnel's Endpoint to `192.168.2.11` or `.12`, since every node answers the
|
||||
same keys. No server change and no new key is needed. The same dependency applies to `away` if the
|
||||
WAN forward points at prod-1.
|
||||
|
||||
Android allows one active VPN at a time, so switch tunnels by hand when leaving or arriving.
|
||||
|
||||
## Operating it
|
||||
|
||||
* **Enrol a device:** `./infra/vpn-ingress/new-peer.sh <name> <octet>` → paste the printed `[Peer]`
|
||||
@@ -215,10 +252,13 @@ right way; one was a defect in my own manifest.
|
||||
handshake — and would be a real secret, forcing `peers.conf` into SOPS and costing the
|
||||
reviewability above. Revisit only if the threat model changes; if it does, the PSK belongs in the
|
||||
`wg-server-key` Secret.
|
||||
* **No IPv6.** The server offers no IPv6 tunnel address, so a full-tunnel client with native IPv6
|
||||
still reaches IPv6 sites *outside* the tunnel. Functionally harmless, a privacy leak if the point
|
||||
was to hide the client's network. Fix is an `fd..` ULA on `wg0` plus `::/0` in the client's
|
||||
`AllowedIPs`; not done because nothing here needs it.
|
||||
* **No IPv6.** The server offers no IPv6 tunnel address and `new-peer.sh` emits only `0.0.0.0/0`.
|
||||
What happens to a client's IPv6 then depends on the client. The **Android app does not leak it**:
|
||||
its `GoBackend` skips `allowFamily()` when the config is one peer with a default route, so Android
|
||||
blocks IPv6 for the VPN's duration and apps fall back to IPv4. Other clients are not covered by
|
||||
that code; expect `wg-quick` on a dual-stack Linux host to leave IPv6 outside the tunnel. Fix is
|
||||
an `fd..` ULA on `wg0` plus `::/0` in the client's `AllowedIPs`; not done because nothing here
|
||||
needs it.
|
||||
* **No in-cluster-service reach beyond what the LAN has.** Clients reach cluster apps the way a LAN
|
||||
browser does — through the Gateway VIP. Pod IPs and `10.43.0.0/16` service IPs are not routed
|
||||
into the tunnel.
|
||||
@@ -90,3 +90,7 @@
|
||||
[Peer]
|
||||
PublicKey = pBRSHrRyG2hnsuZ6FvHfEzF8hOVTF5stHUb/7fW2Km8=
|
||||
AllowedIPs = 10.90.0.2/32
|
||||
# keith-phone-home — <owner>, enrolled 2026-11-08 (at-home profile, LAN Endpoint; no hairpin NAT)
|
||||
[Peer]
|
||||
PublicKey = AT8lBJHqrVkv8RL9n0qWDsh5nyl0e3qgrfTAaYKgBz0=
|
||||
AllowedIPs = 10.90.0.3/32
|
||||
Reference in new issue
Block a user