#63 - fix(vpn-ingress): stop trying to set net.ipv4.ip_forward — a hostNetwork pod cannot, and wg-quick dies on a failed PostUp - home-network-k3s-impl

Conversation 0 Commits 1 Files Changed 4

+200 -5

1 Commits

Author	SHA1	Message	Date
jakman85andClaude Opus 5	5c43fbac65	fix(vpn-ingress): stop trying to set net.ipv4.ip_forward — a hostNetwork pod cannot, and wg-quick dies on a failed PostUp gate / gate (pull_request) Successful in 1m51s Details gate / conformance (pull_request) Has been skipped Details The tunnel never came up on its first reconcile with a real key. One line of my own manifest was the whole cause. Measured on prod 2026-10-06: [#] ip link add dev wg0 type wireguard <- fine [#] wg addconf wg0 /dev/fd/63 <- fine [#] ip -4 address add 10.90.0.1/24 dev wg0 <- fine [#] ip link set mtu 1420 up dev wg0 <- fine [#] sysctl -q -w net.ipv4.ip_forward=1 sysctl: permission denied on key "net.ipv4.ip_forward" [#] ip link delete dev wg0 ** Tunnel failed, will stop all others! ** Not a missing capability: with hostNetwork the runtime mounts /proc/sys READ-ONLY (confirmed in the pod's /proc/mounts), because those sysctls are the HOST's globals. CAP_NET_ADMIN cannot lift a read-only mount, so the write can never succeed here -- and wg-quick treats a failed PostUp as fatal, so it deleted the interface it had just built. The line was never needed. node_baseline already sets ip_forward=1, and reading it from the pod returns 1. My header comment said exactly that ("so no pod sysctl is needed") and then emitted the sysctl anyway, justified as "re-asserted so this component does not silently depend on another role's sysctl file". That defensive re-assertion is the entire outage. In a hostNetwork pod: VERIFY host state, never write it. Replaced with a read in the init container -- `grep -qx 1 /proc/sys/net/ipv4/ip_forward` -- failing closed with a message naming the node_baseline role. Still fail-closed on purpose: a tunnel that comes up unable to forward satisfies every probe here and silently drops all client traffic. Also documents, in the manifest and the runbook, that the image's `chown: Operation not permitted` run and its "we will not provide support for it" warning are expected and harmless -- capabilities are dropped to NET_ADMIN so its s6 init has no CAP_CHOWN, /config is a writable tmpfs, everything runs as root, and wg-quick proceeds (it did; the failure was 30 lines later). Three other unknowns closed by the same reconcile, now recorded in the README: the kernel module reaches the pod (ip link add type wireguard works, no CAP_SYS_MODULE needed); allowPrivilegeEscalation:false does NOT break the s6 init; and the image's iptables is nf_tables (xtables-nft-multi) matching Debian 13, which closes the worst documented fear -- rules landing in a table the kernel never consults. No sensitive paths; merges on a green gate with no override. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>	2026-10-06 01:30:08 -05:00

Author

SHA1

Message

Date

jakman85andClaude Opus 5

5c43fbac65

fix(vpn-ingress): stop trying to set net.ipv4.ip_forward — a hostNetwork pod cannot, and wg-quick dies on a failed PostUp

gate / gate (pull_request) Successful in 1m51s

Details

gate / conformance (pull_request) Has been skipped

Details

The tunnel never came up on its first reconcile with a real key. One line of my
own manifest was the whole cause. Measured on prod 2026-10-06:

    [#] ip link add dev wg0 type wireguard        <- fine
    [#] wg addconf wg0 /dev/fd/63                 <- fine
    [#] ip -4 address add 10.90.0.1/24 dev wg0    <- fine
    [#] ip link set mtu 1420 up dev wg0           <- fine
    [#] sysctl -q -w net.ipv4.ip_forward=1
    sysctl: permission denied on key "net.ipv4.ip_forward"
    [#] ip link delete dev wg0
    **** Tunnel failed, will stop all others! ****

Not a missing capability: with hostNetwork the runtime mounts /proc/sys
READ-ONLY (confirmed in the pod's /proc/mounts), because those sysctls are the
HOST's globals. CAP_NET_ADMIN cannot lift a read-only mount, so the write can
never succeed here -- and wg-quick treats a failed PostUp as fatal, so it
deleted the interface it had just built.

The line was never needed. node_baseline already sets ip_forward=1, and reading
it from the pod returns 1. My header comment said exactly that ("so no pod
sysctl is needed") and then emitted the sysctl anyway, justified as "re-asserted
so this component does not silently depend on another role's sysctl file". That
defensive re-assertion is the entire outage. In a hostNetwork pod: VERIFY host
state, never write it.

Replaced with a read in the init container -- `grep -qx 1
/proc/sys/net/ipv4/ip_forward` -- failing closed with a message naming the
node_baseline role. Still fail-closed on purpose: a tunnel that comes up unable
to forward satisfies every probe here and silently drops all client traffic.

Also documents, in the manifest and the runbook, that the image's `chown:
Operation not permitted` run and its "we will not provide support for it"
warning are expected and harmless -- capabilities are dropped to NET_ADMIN so
its s6 init has no CAP_CHOWN, /config is a writable tmpfs, everything runs as
root, and wg-quick proceeds (it did; the failure was 30 lines later).

Three other unknowns closed by the same reconcile, now recorded in the README:
the kernel module reaches the pod (ip link add type wireguard works, no
CAP_SYS_MODULE needed); allowPrivilegeEscalation:false does NOT break the s6
init; and the image's iptables is nf_tables (xtables-nft-multi) matching Debian
13, which closes the worst documented fear -- rules landing in a table the
kernel never consults.

No sensitive paths; merges on a green gate with no override.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

2026-10-06 01:30:08 -05:00

fix(vpn-ingress): stop trying to set net.ipv4.ip_forward — a hostNetwork pod cannot, and wg-quick dies on a failed PostUp #63