jakman85andClaude Opus 5
989649e64a
The requirement is log visibility only -- which address connected to Plex.
Plex's own log cannot show it. Every LoadBalancer Service here is
externalTrafficPolicy: Cluster, so the node holding the VIP's L2 lease SNATs
the client before forwarding to the backend pod. Measured: Plex had logged
3,214 requests from 10.0.1.x (lease holder k3s-prod-3's own address, Cilium
identity 1 = reserved:host) and ZERO from any 192.168.x address, ever. No
X-Forwarded-For either -- it serves raw tcp/32400, not HTTP via the Gateway.
Nothing on the router does this; there is no hairpin NAT. It is Cilium.
The address is not lost, only unrecorded. Cilium observes the flow at ingress,
BEFORE its own translation, so Hubble already has it:
192.168.2.133 -> 10.0.0.135 real LAN client -> the Plex pod
10.0.1.78 -> 10.0.0.135 same traffic post-SNAT, what Plex sees
What was missing is RETENTION: 4095/4095 flows at ~303/s is about thirteen
seconds of history, metrics disabled, no exporter configured.
So enable the static exporter, port-scoped, to the agent's stdout -- Alloy
already tails every pod into Loki, so this needs no new mount, no rotation
handling and no Alloy change.
{namespace="kube-system", app="cilium"} |= "32400"
Scoped BY PORT, not namespace, on evidence: --to-namespace plex returned only
the post-SNAT flows; the record carrying the real client address did not match
it. A namespace filter would drop exactly what this exists to capture.
Rejected, with reasons: eTP:Local is incompatible with cilium 1.20's L2
announcer per upstream docs for this version ("may cause service IPs to be
announced on nodes without pods causing traffic drops") -- and the VIP is on
prod-3 while the pod is on prod-1, so it would black-hole 32400 immediately;
making it safe costs Plex its ability to reschedule. DSR: routing-mode is
vxlan, not native/geneve, and DSR returns bypass the ingress node, breaking
the router's stateful NAT on the remote path.
Also corrects the repo's own record: 2026-10-03 says the SNAT lands in
192.168.0.0/16 (the node's LAN address). It is 10.0.1.x, the cilium_host
inside the pod CIDR -- which is why LanNetworksBandwidth can never separate
local from remote while eTP is Cluster.
UNVERIFIED and stated in the handoff: nothing is reconciled; `filePath: stdout`
is taken from a verbatim upstream sentence but not confirmed against 1.20.1
source (a wrong notation is a silent no-op, with the file+Alloy fallback
documented); the port FlowFilter has no upstream example; and this change
ROLLS CILIUM -- prod-3 announces the Gateway, PiHole DNS and Plex VIPs, so a
lease move during the rollout could briefly interrupt LAN DNS.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
platform/cilium
Cilium — CNI, kube-proxy replacement, Gateway API ingress, Egress Gateway, and Hubble.
- Governing ADR: ADR-0002 — CNI, network policy & ingress: Cilium + Gateway API
- Spec: docs/03 — Networking, docs/09 — DNS & Ingress
- Serves: R-NET-01..07, R-DNS-01, R-SEC-02
- Owner: WS-NET (bootstrap shared with WS-INFRA)
What this stub is
A Flux HelmRelease + HelmRepository stub with placeholder values. No cluster exists (Phase 0);
this does not reconcile. Cilium replaces k3s' bundled flannel + kube-proxy + Traefik
(ADR-0001/0002).
Bootstrap vs. GitOps (important)
Per docs/02, Cilium's initial
install is done by the Ansible k3s bootstrap (Phase 1, WS-INFRA), because a cluster needs a CNI
before Flux can run. This HelmRelease manages Cilium configuration thereafter. The Helm values
here MUST stay consistent with the Ansible bootstrap values (kubeProxyReplacement, k8sServiceHost,
k8sServicePort) — otherwise Flux and the bootstrap fight. Coordinate this contract with WS-INFRA.
TODO (later phases — not Phase 0)
- Phase 1 (WS-INFRA/WS-NET): align Helm values with the Ansible bootstrap; pin chart version.
- Phase 2 (WS-NET): default-deny network baseline; Egress Gateway policies; L2 announcement IP
pool +
CiliumL2AnnouncementPolicyfor the ingress VIP; Hubble metrics → kube-prometheus-stack. - Phase 3 (WS-NET): egress-class routing integration with the VPN gateway (ADR-0013).