jakman85andClaude Opus 5
d0db63911a
A merged Cilium config change did nothing, and everything reported success. The Hubble flow exporter (#50, merged 2026-10-04) never took effect: HelmRelease cilium Ready=True, "Helm upgrade succeeded ... cilium.v6" cilium-config hubble-export-file-path: stdout + the allowlist, PRESENT cilium agent pods all three still PRE-MERGE, startTime 2026-10-04T07:11Z exported flow lines 0, 0, 0 Flux every Kustomization Ready=True Cause: the chart's rollOutCiliumPods defaults to false, so Helm rewrites the ConfigMap and leaves the DaemonSet's pod template untouched -- and cilium-agent reads that ConfigMap only at STARTUP. Merged, believed delivered, green everywhere, doing nothing. The same class of failure this repo already guards against elsewhere: a green signal over something that never ran. Confirmed by contrast rather than reasoned: the cgroup fix (#42) DID apply -- both the DaemonSet and the running pod mount /sys/fs/cgroup -- because it changed the POD TEMPLATE, which rolls the DaemonSet on its own. That also explains why the agents are dated 2026-10-04T07:11Z: that was #42's rollout, and nothing has rolled them since. The inertness is specific to ConfigMap-only changes. Set in all three value files to hold the [parity] contract. Cost, and it is the point: every future Cilium config change now rolls the agents instead of waiting for a manual restart. Strictly more honest -- it applies when merged or fails visibly -- but config edits become disruptive by default. maxUnavailable: 1 bounds it to one node at a time. The lease-holding node briefly loses the Gateway, PiHole-DNS and Plex VIPs as it cycles, so merge at a quiet moment. Merging this IS one rollout (the ConfigMap checksum annotation changes the pod template) -- which is what finally activates the Hubble exporter. UNVERIFIED: unreconciled; `filePath: stdout` still unproven and only answerable after the agents restart (check for flow lines immediately); and the rollout's actual impact on LAN DNS is unmeasured. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
87 lines
4.5 KiB
YAML
87 lines
4.5 KiB
YAML
# Cilium Helm values — REFERENCE COPY of the parity set (docs/02 Cilium-ownership contract).
|
|
#
|
|
# The AUTHORITATIVE values that Flux reconciles are inlined in helmrelease.yaml. This file is a
|
|
# readable, un-nested mirror of that value set so WS-NET can diff it against the live cluster and
|
|
# the Ansible bootstrap render at a glance. Keep the two in sync.
|
|
#
|
|
# SOURCE OF TRUTH (all [parity] fields MUST equal these):
|
|
# infra/ansible/roles/cilium/templates/cilium-values.yaml.j2
|
|
# infra/ansible/roles/cilium/defaults/main.yml (cilium_version: 1.20.1)
|
|
# infra/ansible/inventory/group_vars/all.yml (k3s_apiserver_endpoint = the kube-vip VIP,
|
|
# k3s_api_port) — NOT k3s_api_host, which is the
|
|
# founder's own address and bootstrap-only
|
|
#
|
|
# chart version (helmrelease.yaml chart.spec.version): 1.20.1 # [parity] = cilium_version
|
|
|
|
# [parity] k3s kube-proxy disabled → Cilium replaces it (ADR-0001/0002)
|
|
kubeProxyReplacement: true
|
|
# [parity] = the k3s API endpoint + port. THE CONTROL-PLANE VIP (kube-vip), not the primary node —
|
|
# changed 2026-09-06; the Ansible half is k3s_apiserver_endpoint, NOT k3s_api_host.
|
|
# Mirrors the HelmRelease's ${K3S_API_HOST} cluster-vars token (R-ENV-02). This file is NOT a
|
|
# kustomize resource, so nothing substitutes it — the token is shown with its dev value beside it.
|
|
k8sServiceHost: "${K3S_API_HOST}" # the CONTROL-PLANE VIP — dev: 192.168.1.49 | prod: 192.168.2.9
|
|
k8sServicePort: 6443
|
|
# [parity] ingress via Gateway API, no Traefik (docs/09)
|
|
gatewayAPI:
|
|
enabled: true
|
|
# [parity] egress-class steering (ADR-0002; VPN model ADR-0013)
|
|
egressGateway:
|
|
enabled: true
|
|
# [parity] Egress Gateway REQUIRES masquerade; ipv4 + eBPF is the Phase-1 crash fix — keep BOTH
|
|
enableIPv4Masquerade: true
|
|
bpf:
|
|
masquerade: true
|
|
# [parity] flow observability / isolation audit
|
|
# [parity] roll the agents when cilium-config changes. The chart default (false) rewrites the
|
|
# ConfigMap but leaves the pod template alone, and cilium-agent only reads it at STARTUP — so a
|
|
# merged config change is SILENTLY INERT. Measured 2026-10-04..05: the Hubble exporter keys were
|
|
# live in cilium-config and the HelmRelease said UpgradeSucceeded, while all three agents were
|
|
# still pre-merge pods exporting nothing for over a day. Cost: config edits now roll the agents
|
|
# (one node at a time via maxUnavailable: 1), which can briefly blip the VIPs on the lease-holding
|
|
# node. Full rationale in helmrelease.yaml.
|
|
rollOutCiliumPods: true
|
|
|
|
hubble:
|
|
enabled: true
|
|
relay:
|
|
enabled: true
|
|
ui:
|
|
enabled: true
|
|
# [parity] static flow export -> cilium-agent stdout -> Alloy -> Loki. Records the PRE-SNAT
|
|
# client address for Plex's raw tcp/32400, which Plex's own log cannot show (every LB Service here
|
|
# is externalTrafficPolicy: Cluster, so the VIP lease holder SNATs the client). Scoped BY PORT,
|
|
# not namespace: the flow carrying the real client IP does not match a namespace filter. Full
|
|
# rationale, measurements and the rejected alternatives are in helmrelease.yaml.
|
|
export:
|
|
static:
|
|
enabled: true
|
|
filePath: stdout
|
|
allowList:
|
|
- '{"destination_port":["32400"]}'
|
|
- '{"source_port":["32400"]}'
|
|
# [parity] ingress VIP announcement on the flat LAN (docs/09, OQ-1)
|
|
l2announcements:
|
|
enabled: true
|
|
# [parity] the host's cgroup2 root is /sys/fs/cgroup (systemd mounts it); do not auto-mount.
|
|
# NOT /run/cilium/cgroupv2: nothing mounts cgroup2 there, so socket-LB attached to the agent's own
|
|
# cgroup and host processes lost Service/LB translation (2026-10-03 — see helmrelease.yaml).
|
|
cgroup:
|
|
autoMount:
|
|
enabled: false
|
|
hostRoot: /sys/fs/cgroup
|
|
# [parity] roll the agent DaemonSet ONE node at a time. Overrides the cilium 1.20.1 chart
|
|
# default of 2, which on 3 nodes lets a single bad render break 2/3 agents at once — as the
|
|
# 2026-09-01 incident did (k8sServiceHost: "" → the `config` init container fell back to the
|
|
# ClusterIP 10.43.0.1:443, translatable only by the very agent being replaced). A CNI upgrade
|
|
# has no healthy-peer fallback, least of all on a cold prod bootstrap; maxUnavailable: 1 stops
|
|
# the rollout at the first unready pod. Cost: serialised, slower upgrades (fine at 3 nodes).
|
|
updateStrategy:
|
|
type: RollingUpdate
|
|
rollingUpdate:
|
|
maxUnavailable: 1
|
|
# NON-parity (Flux/prod may differ; dev sets 1 to save RAM) — omitted from the HelmRelease:
|
|
# operator.replicas
|
|
#
|
|
# TODO (Phase 2, WS-NET): ipam / ipv4NativeRoutingCIDR pinned to cluster CIDRs; east-west
|
|
# encryption (wireguard/ipsec) decision (WS-SEC/WS-NET).
|