jakman85andClaude Opus 4.8
e8c555cfeb
Greenfield spec set for a GitOps-driven PaaS+IaaS home platform: vision & requirements (R-* IDs), architecture, 14 domain specs, 11 ADRs, roadmap (phases + workstreams), and AI-agent coordination (roles, protocol, definition-of-done, task board). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2.3 KiB
2.3 KiB
ADR-0002 — CNI, network policy & ingress: Cilium + Gateway API
- Status: Accepted
- Date: 2026-08-15
- Serves: R-NET-01, R-NET-02, R-NET-03, R-NET-04, R-NET-05, R-NET-06, R-NET-07, R-DNS-01, R-SEC-02
- Supersedes: —
Context
The platform requires: default-deny east-west with explicit port-level allows (firewall-like), network isolation/segmentation, multi-WAN/egress routing including per-service VPN egress, and north-south ingress with TLS. These are the most safety-critical requirements. A single, coherent networking layer that does policy, egress steering, observability, and ingress reduces moving parts.
Decision
Use Cilium as the CNI (replacing k3s' flannel), with kube-proxy replacement, and adopt:
- CiliumNetworkPolicy / NetworkPolicy for default-deny + explicit L3/L4 (optionally L7) allows.
- Cilium Egress Gateway to steer selected pods' egress through chosen nodes/identities/SNAT IPs — the mechanism for egress-class routing and part of the VPN model (with ADR-0008).
- Hubble for flow observability — the audit/troubleshooting tool for isolation and egress.
- Gateway API implemented by Cilium for north-south ingress and TLS termination (no separate ingress controller).
Alternatives considered
- Calico — strong policy engine; lacks Cilium's integrated egress gateway + Hubble + Gateway API in one stack.
- Flannel + separate ingress-nginx/Traefik + separate egress solution — more components, no identity-aware policy or built-in flow visibility.
- Service mesh (Istio/Linkerd) for policy/mTLS — heavier; not needed at launch. mTLS can be added later via Cilium/mesh as a future ADR (08).
Consequences
- One networking layer covers policy, egress steering, observability, and ingress — fewer components, coherent model, VMs (KubeVirt) inherit the same policy fabric.
- Requires disabling k3s bundled flannel/kube-proxy/Traefik (see ADR-0001).
- Egress Gateway + VPN model must be validated end-to-end including kill-switch (ADR-0008, R-NET-06).
- Physical multi-WAN (dual ISP) remains partly an edge-router responsibility (OQ-1); the cluster decides egress class, the edge router decides physical WAN for the internet class.
- Gateway LB/VIP mechanism (Cilium L2 announcement vs BGP) to be pinned during networking bring-up.