monitoring: retarget VPNTunnelDown to node tun0 presence; drop obsolete EgressGatewayUnhealthy
gluetun retirement removes the gluetun_vpn_up gauge and the in-cluster gateway pod. Point
VPNTunnelDown at absent(node_network_receive_bytes_total{device=tun0}) — tun0 exists iff the
host OpenVPN tunnel is up (node_network_up unreliable: tun0 operstate=unknown). This also
fires if the gateway node/node-exporter is down (series goes stale). Remove the pod-readiness
EgressGatewayUnhealthy alert — there is no gateway pod under ADR-0015.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
1 parent
7b7982e28d
commit
d519ede4e3
1 file changed
+12
-14
@@ -213,11 +213,14 @@ spec:
|
||||
groups:
|
||||
- name: platform-health.networking
|
||||
rules:
|
||||
# VPN tunnel health. The concrete metric depends on the egress-gateway VPN sidecar chosen by
|
||||
# WS-NET (docs/03). Written against a gluetun-style up gauge; confirm/adjust at bring-up.
|
||||
# VPN tunnel health (ADR-0015: node-level host OpenVPN tunnel on the egress-gateway node, steered
|
||||
# by Cilium Egress Gateway — there is NO in-cluster gateway pod). OpenVPN creates/destroys the tun0
|
||||
# device with the connection, so the tun0 netdev series exists iff the tunnel is up. node_network_up
|
||||
# is unreliable here (tun0 operstate=unknown => 0), so key on the byte-counter's PRESENCE. absent()
|
||||
# also fires if the gateway node / its node-exporter is down (the tun0 series goes stale), covering
|
||||
# gateway-node-down as well. Metric confirmed live on the dev cluster 2026-08-24.
|
||||
- alert: VPNTunnelDown
|
||||
# TODO(WS-NET): confirm metric name of the egress VPN exporter (docs/03).
|
||||
expr: max(gluetun_vpn_up) == 0 or absent(gluetun_vpn_up)
|
||||
expr: absent(node_network_receive_bytes_total{device="tun0"})
|
||||
for: 5m
|
||||
labels:
|
||||
severity: critical
|
||||
@@ -225,16 +228,11 @@ spec:
|
||||
summary: "Egress VPN tunnel down"
|
||||
description: "The egress-gateway VPN tunnel is down (or its metric is absent) for 5m — outbound traffic could leak or stall (R-SEC-03 leak risk)."
|
||||
runbook_url: "https://home.keithjackson.dev/runbooks/vpn-down"
|
||||
- alert: EgressGatewayUnhealthy
|
||||
expr: |
|
||||
max by (pod) (kube_pod_status_ready{namespace="egress", condition="true"}) == 0
|
||||
for: 10m
|
||||
labels:
|
||||
severity: critical
|
||||
annotations:
|
||||
summary: "Egress gateway unhealthy"
|
||||
description: "No ready egress-gateway pod for 10m — VPN-scoped egress is unavailable."
|
||||
runbook_url: "https://home.keithjackson.dev/runbooks/egress-gateway"
|
||||
# (ADR-0015) The former `EgressGatewayUnhealthy` pod-readiness alert was REMOVED with the gluetun
|
||||
# retirement: there is no in-cluster egress-gateway pod anymore (the tunnel is a host systemd
|
||||
# service, steering is Cilium eBPF). Tunnel + gateway-node health is covered by `VPNTunnelDown`
|
||||
# above (tun0 presence, which also goes absent if the gateway node is down); Cilium datapath
|
||||
# health is covered by the Cilium platform alerts.
|
||||
- alert: HubbleDeniedFlowSpike
|
||||
# Cilium/Hubble denied (policy-drop) flows. Requires hubble metrics enabled (docs/03).
|
||||
expr: sum(rate(hubble_drop_total{reason="POLICY_DENIED"}[5m])) > 5
|
||||
|
||||
Reference in new issue
Block a user