home-network-k3s-impl/apps/wave-0/pihole-dns.yaml

---
# =================================================================================================
# PiHole DNS EXPOSURE — the LAN half of split-horizon (R-DNS-01 / R-DNS-03), and NETWORK POLICY.
# =================================================================================================
# HAND-WRITTEN, NOT RENDERED. This file is a non-catalog app-tier unit anchored to the `pihole`
# ServiceDefinition and placed into wave 0 by `catalog/render.py --emit-waves` via its EXTRA_UNITS
# table (the same mechanism gitea-actions uses). It is hand-written because the ServiceDefinition
# schema genuinely cannot express it: schemas/servicedefinition.schema.json is
# `additionalProperties: false` and has NO field for a type=LoadBalancer Service, for a pinned
# lbipam VIP, for a sharing key, for loadBalancerSourceRanges, or for UDP north-south exposure at
# all. `ingressRoutes` renders HTTPRoutes on the shared Gateway, which is an L7 HTTPS terminator and
# cannot carry DNS. The two existing L4 workloads in this repo are hand-written for exactly this
# reason and in exactly this shape — apps/vms/unifi/50-service-l4.yaml and
# platform/crafty/service-l4.yaml. The ANSWERS PiHole gives are declared in the catalog
# (catalog/pihole.yaml `env`), so the only thing living outside the catalog is the exposure the
# schema has no word for.
#
# WHY THIS EXISTS AT ALL: platform/coredns/coredns-custom.yaml is explicit that its split-horizon is
# for PODS and defers LAN clients to "PiHole split-horizon" — but PiHole served no DNS. Its Service
# was ClusterIP with ports 80 and 4180 only. The LAN half of the split-horizon that docs/09
# (R-DNS-03) and the README's Phase 2 row both claim simply did not exist, and LAN clients ran on
# hand-maintained per-client dnsmasq files instead. Upstream cannot substitute: the SOHO router that
# owns this LAN strips RFC1918 answers for public domains as rebind protection (NOERROR, ANSWER 0).
#
# VIP: ${PIHOLE_DNS_VIP} (cluster-vars, R-ENV-02 — dev 192.168.1.56, prod 192.168.2.23), PINNED via
# lbipam.cilium.io/ips out of the CiliumLoadBalancerIPPool `default-pool`
# (${CILIUM_LB_POOL_START}..${CILIUM_LB_POOL_STOP}) and ARP-announced on the flat LAN by the
# existing CiliumL2AnnouncementPolicy `default-l2` (platform-config/cilium/*). No new Cilium config
# is needed; these Services just consume the pool. PINNING IS NOT OPTIONAL HERE: this address is
# typed into every LAN client's resolver configuration (or handed out by the router's DHCP), so it
# can never be allowed to move.
#
# ONE VIP, TWO SERVICES: a LoadBalancer Service is single-protocol-friendly, so udp/53 and tcp/53
# split into two Services sharing one address via lbipam.cilium.io/sharing-key — the same
# arrangement unifi and crafty use. BOTH PROTOCOLS ARE REQUIRED, not belt-and-braces: UDP is the
# normal query path, and TCP is what a resolver falls back to for any answer that does not fit in a
# UDP datagram and what some clients use unconditionally. A UDP-only resolver fails intermittently
# and confusingly.
#
# NO external-dns ANNOTATION, DELIBERATELY. unifi and crafty both carry
# `external-dns.alpha.kubernetes.io/hostname` so external-dns publishes their address as a PUBLIC
# Cloudflare record. This Service must not: `pihole-dns.${BASE_DOMAIN}` is answered by PiHole itself
# (catalog/pihole.yaml's dnsmasq override), it is an RFC1918 address that is useless in public DNS,
# and publishing a resolver's address in a public zone invites exactly the traffic the source-range
# filter below exists to refuse. Internal name, internal answer, no public record — which is also
# why this change touches nothing under platform/externaldns/.
#
# -------------------------------------------------------------------------------------------------
# externalTrafficPolicy: Cluster — AND WHY IT IS NOT `Local`, WHICH IS WHAT unifi AND crafty USE.
# -------------------------------------------------------------------------------------------------
# `Local` is the obvious choice: it preserves the client source IP and, in theory, pins the VIP to
# the node running the backend. The second half is NOT TRUE on this cluster, and that was measured,
# not assumed (Cilium 1.20.1, prod, 2026-09-14):
#
#   * Cilium's L2 announcer elects one lease holder per Service among the nodes matching the
#     CiliumL2AnnouncementPolicy's nodeSelector — `default-l2` matches ALL linux nodes — and it does
#     NOT filter that candidate set by which nodes hold a backend. Created fresh, the two Services
#     landed on DIFFERENT nodes (`cilium-l2announce-pihole-pihole-dns-tcp` -> k3s-prod-3,
#     `-udp` -> k3s-prod-2) while the pihole pod ran on k3s-prod-1.
#   * With `Local`, a node that announces the VIP but holds no backend has nowhere to send the
#     packet, and the query fails with ICMP port unreachable (dig: "connection refused"). On three
#     nodes with one replica that is a 1-in-3 chance of working, drawn independently per protocol.
#   * It does not self-heal. The lease holder keeps RENEWING after the pod moves away: a pod
#     rescheduled from k3s-prod-1 to k3s-prod-2 left the lease on k3s-prod-1, still renewing three
#     minutes later, with LAN resolution dead the whole time. Only deleting the Lease forced a
#     re-election — and that re-election is random, so it may pick a backend-less node again.
#
# A resolver that works one time in three and stays broken after any reschedule is not a resolver.
# `Cluster` makes every node able to serve the VIP by forwarding to wherever the pod is, which is
# what makes this survive a reschedule and a rebuild. VERIFIED in exactly the broken arrangement
# above — leases on k3s-prod-2 and k3s-prod-3, pod on k3s-prod-1 — udp/53, tcp/53 and recursion all
# answering.
#
# THE COST, STATED PLAINLY: Cilium runs in SNAT mode here (no `loadBalancer.mode: dsr` in
# platform/cilium/helmrelease.yaml), so a query forwarded between nodes reaches PiHole with the
# INGRESS NODE's address as its source, not the client's. PiHole's per-client query log, per-client
# groups and per-client rate limiting therefore see one apparent client instead of the LAN. Two
# consequences are handled: the rate limit is disabled in catalog/pihole.yaml (otherwise the whole
# LAN counts as one client against PiHole's 1000-queries-per-60s default and PiHole starts
# REFUSING), and the LAN restriction is enforced in the datapath rather than by source identity —
# see loadBalancerSourceRanges below. The fix that would restore client IPs is `loadBalancer.mode:
# dsr` on the Cilium HelmRelease, which is a platform-wide change with an Ansible [parity] contract
# and would also fix the same latent defect in unifi and crafty; it belongs in its own change.
#
# -------------------------------------------------------------------------------------------------
# loadBalancerSourceRanges — THE LAN RESTRICTION, ENFORCED IN THE DATAPATH.
# -------------------------------------------------------------------------------------------------
# Cilium enforces this list in eBPF on the ingress node, on the VIP, BEFORE load-balancing and
# before SNAT. A packet to ${PIHOLE_DNS_VIP}:53 from outside ${LAN_CIDR} is dropped by the
# datapath — it never reaches a backend and never reaches a network policy. This is what makes
# "LAN clients only" a real, tested restriction rather than a comment: VERIFIED by narrowing the
# range to a /24 that excludes the test client, after which the same query that had been answering
# stopped being answered at all, and restoring it, after which it answered again.
#
# It is also the layer that survives the SNAT above. A CiliumNetworkPolicy cannot express this
# restriction once the source has been rewritten: measured on this cluster, the forwarded query
# arrives from the ingress node's internal router address and Cilium resolves that source to the
# reserved `world` identity, which a `fromCIDR` clause does not match even when the CIDR covers the
# address (tried: fromCIDR ${CLUSTER_POD_CIDR}, still dropped as `world`). So the peer restriction
# lives here, where it is actually enforceable, and the policy below restricts the PORTS.
apiVersion: v1
kind: Service
metadata:
  name: pihole-dns-udp
  namespace: pihole
  labels:
    app.kubernetes.io/name: pihole
  annotations:
    lbipam.cilium.io/sharing-key: "pihole-dns"
    lbipam.cilium.io/ips: "${PIHOLE_DNS_VIP}"
spec:
  type: LoadBalancer
  externalTrafficPolicy: Cluster
  loadBalancerSourceRanges:
    - ${LAN_CIDR}
  selector:
    app.kubernetes.io/name: pihole
  ports:
    - { name: dns-udp, port: 53, targetPort: 53, protocol: UDP }
---
apiVersion: v1
kind: Service
metadata:
  name: pihole-dns-tcp
  namespace: pihole
  labels:
    app.kubernetes.io/name: pihole
  annotations:
    lbipam.cilium.io/sharing-key: "pihole-dns"   # same key + same IP => shares the VIP with -udp
    lbipam.cilium.io/ips: "${PIHOLE_DNS_VIP}"
spec:
  type: LoadBalancer
  externalTrafficPolicy: Cluster
  loadBalancerSourceRanges:
    - ${LAN_CIDR}
  selector:
    app.kubernetes.io/name: pihole
  ports:
    - { name: dns-tcp, port: 53, targetPort: 53, protocol: TCP }
---
# =================================================================================================
# NETWORK POLICY — a NEW ingress path from off-cluster, restricted to the two DNS ports.
# =================================================================================================
# The pihole namespace is default-deny on ingress (the baseline NetworkPolicy in the rendered
# apps/wave-0/pihole.yaml: `ingress: []`). Cilium enforces the UNION of that NetworkPolicy and the
# CiliumNetworkPolicies in the namespace, so this file ADDS one path and nothing else. Everything
# undeclared stays denied, and the existing `pihole-allow-gateway-ingress` (Gateway -> :4180, the
# oauth2-proxy gate in front of the admin UI) is untouched. PiHole's own :80 is NOT in this rule:
# the admin UI stays reachable only through the Gateway and oauth2-proxy, and a LAN client that
# talks to ${PIHOLE_DNS_VIP} on any port other than 53 gets nothing — there is no Service port for
# it and no policy for it.
#
# PORTS ARE THE NARROWING HERE; THE LAN RESTRICTION IS THE SOURCE-RANGE FILTER ABOVE. The peer is
# `world` for a mechanical reason, not a lazy one: with externalTrafficPolicy Cluster in Cilium's
# SNAT mode a forwarded query arrives from the ingress node's internal router address, which Cilium
# resolves to the reserved `world` identity — a `fromCIDR` clause does not match it, which was
# measured (a fromCIDR of ${CLUSTER_POD_CIDR}, which contains that address, still produced
# "Policy denied ... identity world" drops in `cilium-dbg monitor`). So `world` is the only peer
# selector that can describe LAN traffic that has been through the LoadBalancer, and the set of
# addresses that can actually BE that traffic is fixed by loadBalancerSourceRanges before this
# policy is consulted. Together: ${LAN_CIDR} in the datapath, udp/53 + tcp/53 here.
#
# This is NOT wider than the precedent it follows. unifi-allow-l4-ingress and crafty-allow-ingress
# admit `fromEntities: [world, cluster, host]` on their L4 ports with NO source-range filter at all;
# this rule admits fewer entities (no `cluster` — in-cluster pods resolve through CoreDNS and none
# needs PiHole on :53) on fewer ports, behind a filter those two do not have.
#
# WHAT REMAINS REACHABLE WITHOUT THE VIP, ACKNOWLEDGED: Cilium applies source ranges to the
# LoadBalancer address, not to the NodePorts the same Services allocate, so a host that can route to
# a node IP can still reach udp/53 and tcp/53 through the NodePort. On this LAN that is the same
# population the source range admits, and nothing at the edge forwards 53 inward, so PiHole is not
# an internet-facing open resolver. Say so out loud rather than implying the filter is total.
#
# `host` and `remote-node` are named explicitly because node-sourced traffic carries a reserved
# Cilium identity that no CIDR or `world` selector matches: the kubelet's probes and the
# node-to-node forwarding that IS the Cluster-policy datapath both need them.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: pihole-allow-lan-dns-ingress
  namespace: pihole
  labels:
    app.kubernetes.io/name: pihole
spec:
  endpointSelector:
    matchLabels:
      app.kubernetes.io/name: pihole
  ingress:
    - fromEntities:
        - world         # LAN clients, post-LoadBalancer: see the block above for why not fromCIDR
        - host          # this node's kubelet probes
        - remote-node   # the node-to-node hop that forwards a query to wherever the pod runs
      toPorts:
        - ports:
            - { port: "53", protocol: UDP }
            - { port: "53", protocol: TCP }