jakman85andClaude Opus 4.8
e8c555cfeb
Greenfield spec set for a GitOps-driven PaaS+IaaS home platform: vision & requirements (R-* IDs), architecture, 14 domain specs, 11 ADRs, roadmap (phases + workstreams), and AI-agent coordination (roles, protocol, definition-of-done, task board). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
38 lines
2.3 KiB
Markdown
38 lines
2.3 KiB
Markdown
# ADR-0002 — CNI, network policy & ingress: Cilium + Gateway API
|
|
- Status: Accepted
|
|
- Date: 2026-08-15
|
|
- Serves: R-NET-01, R-NET-02, R-NET-03, R-NET-04, R-NET-05, R-NET-06, R-NET-07, R-DNS-01, R-SEC-02
|
|
- Supersedes: —
|
|
|
|
## Context
|
|
The platform requires: default-deny east-west with explicit port-level allows (firewall-like),
|
|
network isolation/segmentation, multi-WAN/egress routing including per-service VPN egress, and
|
|
north-south ingress with TLS. These are the most safety-critical requirements. A single, coherent
|
|
networking layer that does policy, egress steering, observability, and ingress reduces moving parts.
|
|
|
|
## Decision
|
|
Use **Cilium** as the CNI (replacing k3s' flannel), with **kube-proxy replacement**, and adopt:
|
|
- **CiliumNetworkPolicy / NetworkPolicy** for default-deny + explicit L3/L4 (optionally L7) allows.
|
|
- **Cilium Egress Gateway** to steer selected pods' egress through chosen nodes/identities/SNAT IPs
|
|
— the mechanism for egress-class routing and part of the VPN model (with ADR-0008).
|
|
- **Hubble** for flow observability — the audit/troubleshooting tool for isolation and egress.
|
|
- **Gateway API implemented by Cilium** for north-south ingress and TLS termination (no separate
|
|
ingress controller).
|
|
|
|
## Alternatives considered
|
|
- **Calico** — strong policy engine; lacks Cilium's integrated egress gateway + Hubble + Gateway API
|
|
in one stack.
|
|
- **Flannel + separate ingress-nginx/Traefik + separate egress solution** — more components, no
|
|
identity-aware policy or built-in flow visibility.
|
|
- **Service mesh (Istio/Linkerd) for policy/mTLS** — heavier; not needed at launch. mTLS can be
|
|
added later via Cilium/mesh as a future ADR ([08](../docs/08-tls-and-certificates.md)).
|
|
|
|
## Consequences
|
|
- One networking layer covers policy, egress steering, observability, and ingress — fewer components,
|
|
coherent model, VMs (KubeVirt) inherit the same policy fabric.
|
|
- Requires disabling k3s bundled flannel/kube-proxy/Traefik (see ADR-0001).
|
|
- Egress Gateway + VPN model must be validated end-to-end including kill-switch (ADR-0008, R-NET-06).
|
|
- Physical multi-WAN (dual ISP) remains partly an edge-router responsibility (OQ-1); the cluster
|
|
decides egress *class*, the edge router decides physical WAN for the internet class.
|
|
- Gateway LB/VIP mechanism (Cilium L2 announcement vs BGP) to be pinned during networking bring-up.
|