The operator ruled the implementation repo's main (50c8719) the latest and accepted state. Sync the spec to it. Accepted changes are reflected: one bare-metal environment, TrueNAS bulk storage, R2 off-cluster backup, CloudNativePG, single-node VPN gateway with Cilium fail-closed, inbound WireGuard, per-app auth sidecars, Renovate, the CI gate as approver, and the pm-agent. New ADR-0017..0025; ADR-0016 corrected and accepted; R-* IDs unchanged, eight amended. Where the implementation has simply not met a requirement, the requirement stands and the shortfall is recorded in docs/16-known-gaps.md. Co-Authored-By: Claude Opus 5.5 <[email protected]>
2.0 KiB
2.0 KiB
ADR-0001 — Kubernetes distribution: k3s
- Status: Accepted — k3s, embedded etcd and its disables stand (dev provisioning clause superseded by ADR-0017)
- Date: 2026-08-15
- Serves: R-PLAT-01, R-PLAT-02, R-ENV-01/02/03
- Supersedes: —
Context
The platform runs on 3 small nodes (dev: KVMs; prod: mini-PCs) and must offer a full Kubernetes API for both containers and VMs. It must be lightweight enough for constrained hardware yet production-capable, HA, and identical across dev/test and prod.
Decision
Use k3s as the Kubernetes distribution, deployed as a 3-node HA cluster with embedded etcd (all three nodes are servers). Bundled components that conflict with our chosen stack are disabled at install time: flannel (→ Cilium), kube-proxy (→ Cilium kube-proxy replacement), Traefik (→ Gateway API via Cilium), and servicelb where a different LB strategy is selected. Nodes are provisioned by OpenTofu (libvirt, dev) / bare-metal imaging (prod) + cloud-init, and joined/configured by Ansible.
Alternatives considered
- Full kubeadm/vanilla Kubernetes — heavier operationally for 3 small nodes; k3s gives the same API with far less overhead.
- k0s / MicroK8s / Talos — all viable. Talos is attractive (immutable, API-driven) and may be revisited via a superseding ADR for prod hardening; k3s chosen now for its ubiquity, small footprint, single-binary simplicity, strong community, and the operator's existing familiarity.
- Managed cloud Kubernetes — out of scope; this is on-prem home hardware.
Consequences
- Lightweight, HA, uniform dev↔prod topology (R-ENV-02).
- Must explicitly disable bundled CNI/ingress/proxy to make room for Cilium + Gateway API (ADR-0002) — captured in the Ansible bootstrap.
- Embedded etcd across 3 nodes tolerates single-node loss; do not reduce below 3 servers in prod.
- Revisit Talos for prod if immutability/hardening becomes a priority.