Initial baseline: home platform (k3s + Longhorn) specification repository

Greenfield spec set for a GitOps-driven PaaS+IaaS home platform: vision & requirements (R-* IDs), architecture, 14 domain specs, 11 ADRs, roadmap (phases + workstreams), and AI-agent coordination (roles, protocol, definition-of-done, task board). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-08-15 22:37:56 -05:00
commit e8c555cfeb
@@ -0,0 +1,108 @@
+# Roadmap — Delivery Phases
+
+Delivery is phased. Each phase has an **entry gate** (what must be true to start), **exit criteria**
+(demonstrable outcomes tied to requirement IDs), and the **workstreams** ([workstreams.md](workstreams.md))
+that execute it. **Everything is proven in dev/test; prod is design-only until Phase 7.**
+
+A phase's exit criteria are **passing automated conformance tests** (per [`../AGENTS.md` §7](../AGENTS.md)
+and [`../docs/14-testing-and-conformance.md`](../docs/14-testing-and-conformance.md)), not "manifests
+merged" and not manual demos. Every "**Prove:**" / "**demonstrate**" item below is an automated,
+reproducible, requirement-tagged behavior test committed with the work. A phase exits only when its
+full conformance subset is green. **The conformance harness (WS-TEST) is stood up in Phase 0, so
+every subsequent phase ships with tests, not after them.**
+
+---
+
+## Phase 0 — Foundations & Repo Bootstrap
+**Entry:** specs (this repo) approved.
+**Do:**
+- Create the implementation repo structure ([`../docs/05-gitops-and-delivery.md`](../docs/05-gitops-and-delivery.md)).
+- Stand up CI (lint, schema validation, dry-run/build) for manifests and ServiceDefinitions.
+- Author the ServiceDefinition schema + renderer contract ([`../docs/13-service-catalog.md`](../docs/13-service-catalog.md)).
+- **Stand up the conformance harness (WS-TEST):** single-command runner, reproducibility contract,
+  requirement-tagged JUnit + coverage report, L0 policy-as-code gate, and the CI merge gate
+  ([`../docs/14-testing-and-conformance.md`](../docs/14-testing-and-conformance.md), [ADR-0010](../decisions/adr-0010-conformance-testing.md)).
+**Exit:** implementation repo + CI exist; ServiceDefinition schema validated; **conformance harness
+runs green with the coverage report wired**; no cluster yet.
+**Serves:** R-GITOPS-02, R-ENV-02, R-TEST-01..04.
+
+## Phase 1 — dev/test Cluster Bring-up (IaC)
+**Entry:** Phase 0 done.
+**Do:** OpenTofu libvirt node VMs + cloud-init baseline + Ansible k3s HA install with **Cilium**;
+disable bundled flannel/kube-proxy/Traefik; bootstrap **Flux**; bootstrap **SOPS age key**.
+**Exit:** 3-node k3s HA cluster with Cilium up; Flux reconciling from git; a dev node VM can be
+destroyed and rebuilt from IaC. **Prove:** node-loss re-provision.
+**Serves:** R-PLAT-01, R-ENV-01, R-GITOPS-01, ADR-0001/0002/0004/0007.
+
+## Phase 2 — Platform Core: Network, Storage, TLS, DNS, Ingress
+**Entry:** Phase 1 done.
+**Do:** Longhorn (3-replica) + bulk-tier plan; default-deny network baseline + Hubble; Cilium
+Egress Gateway; cert-manager + Let's Encrypt (DNS-01, wildcard); ExternalDNS; Gateway API listeners;
+split-horizon DNS foundation.
+**Exit (demonstrate):**
+- Default-deny holds: a probe pod can reach only DNS (R-NET-01).
+- A PVC survives pod destroy/rebuild (R-STATE-01).
+- A test host serves a **trusted, non-self-signed** cert at `*.home.keithjackson.dev` (R-TLS-01/03).
+- Auto DNS record appears for a test route (R-DNS-02).
+**Serves:** R-NET-01/02/03, R-STATE-01/02, R-TLS-01/02/03, R-DNS-01/02/03, ADR-0002/0003/0006.
+
+## Phase 3 — Egress, VPN & Multi-WAN
+**Entry:** Phase 2 done; OQ-1/OQ-3 answered enough to build.
+**Do:** implement egress classes; VPN egress (Gluetun sidecar default) with **kill-switch**;
+integrate edge-router multi-WAN split for the `internet` class.
+**Exit (demonstrate):**
+- `vpn` test workload egresses via VPN IP; `internet` workload via home/WAN IP (R-NET-05).
+- **Kill-switch:** tunnel forced down ⇒ `vpn` workload leaks nothing; `internet` unaffected (R-NET-06).
+- Flipping `egress` class in git re-routes without workload rewrite (R-NET-07).
+**Serves:** R-NET-04/05/06/07, ADR-0008.
+
+## Phase 4 — Observability, Backup/DR & Automated Updates
+**Entry:** Phase 2 done (can overlap Phase 3).
+**Do:** kube-prometheus-stack + Loki + Alloy + Alertmanager(email); platform-health alerts; Velero +
+Longhorn off-cluster backup; Flux image-automation + Renovate; per-service update policy plumbing.
+**Exit (demonstrate):**
+- Injected fault emails an alert; dashboards render (R-OBS-01/02/03/04).
+- DR drill: restore a namespace + volumes from off-cluster backup (R-STATE-03).
+- A test service auto-advances a stable version on schedule with no human/AI edit; a bad update
+  auto-rolls-back and alerts (R-UPD-01/02/03).
+**Serves:** R-OBS-*, R-STATE-03, R-UPD-*, ADR-0009/0003.
+
+## Phase 5 — IaaS Proof (KubeVirt)
+**Entry:** Phases 2–3 done.
+**Do:** KubeVirt + CDI; run one representative application VM via ServiceDefinition `kind: vm`.
+**Exit (demonstrate):** create/start/stop/destroy a VM via `kubectl`/`virtctl`; Longhorn-backed disk
+survives rebuild; default-deny + `egress` apply to the VM; guest auto-updates.
+**Serves:** R-PLAT-03/04, R-STATE-01, R-NET-*, R-UPD-*, ADR-0005.
+
+## Phase 6 — Service Catalog Onboarding + C2 UI
+**Entry:** Phases 2–4 done.
+**Do:** onboard the 13 services via ServiceDefinitions in dependency order (see workstream WS-APPS);
+stand up the C2 UI (Homepage root + Grafana + Headlamp + Flux status).
+**Exit (demonstrate):** each service fully wired from its ServiceDefinition (compute/storage/policy/
+egress/TLS/DNS/monitoring/updates) with default-deny + correct egress; the arr↔qbittorrent↔jackett↔
+seerr↔plex dependency graph works and nothing undeclared does; `home.keithjackson.dev` serves the C2
+with live links + health.
+**Serves:** R-C2-01/02, all app-facing requirements, [`../docs/13-service-catalog.md`](../docs/13-service-catalog.md).
+
+## Phase 7 — Production Readiness (design → build)
+**Entry:** Phases 1–6 proven in dev/test; operator declares prod hardware ready.
+**Do:** finalize bare-metal provisioning checklist; apply the **prod overlay** (no base edits);
+full DR drill on prod; cutover plan.
+**Exit:** prod 3× mini-PC cluster runs the same config via the prod overlay; DR proven; monitoring +
+alerts live.
+**Serves:** R-ENV-02/03, R-PLAT-01.
+**Note:** Do not begin building on prod hardware before this phase is explicitly opened
+([`../AGENTS.md` §6](../AGENTS.md)).
+
+---
+
+## Dependency flow
+
+```
+P0 → P1 → P2 ─┬→ P3 ─┐
+              ├→ P4 ──┼→ P6 → P7
+              └───────┘   ↑
+                    P5 ───┘
+```
+
+P3, P4, P5 can proceed in parallel once P2 lands; P6 needs P2–P4; P7 needs everything proven.
@@ -0,0 +1,59 @@
+# Roadmap — Workstreams
+
+A **workstream** is a durable area of ownership. Each maps to an agent role
+([`../agents/roles.md`](../agents/roles.md)), owns a set of specs, exposes **interfaces** other
+workstreams depend on, and delivers against requirement IDs. Interfaces are contracts — change them
+only via [`../agents/coordination-protocol.md`](../agents/coordination-protocol.md).
+
+| ID | Workstream | Owns (specs) | Serves | Depends on |
+|---|---|---|---|---|
+| **WS-INFRA** | Infra & cluster bring-up | [02](../docs/02-environments-and-provisioning.md) | R-PLAT-01, R-ENV-* | — |
+| **WS-GITOPS** | GitOps, CI, catalog renderer | [05](../docs/05-gitops-and-delivery.md), [13](../docs/13-service-catalog.md) | R-GITOPS-*, R-UPD-02 | WS-INFRA |
+| **WS-NET** | Networking, isolation, egress, ingress | [03](../docs/03-networking-isolation-and-egress.md), [09](../docs/09-dns-and-ingress.md) | R-NET-*, R-DNS-* | WS-INFRA |
+| **WS-STORAGE** | Storage, state, backup/DR | [04](../docs/04-storage-and-state.md) | R-STATE-* | WS-INFRA |
+| **WS-SEC** | Secrets, TLS, RBAC, auditability | [07](../docs/07-security-and-secrets.md), [08](../docs/08-tls-and-certificates.md) | R-SEC-*, R-TLS-* | WS-INFRA, WS-NET |
+| **WS-OBS** | Observability & alerting | [10](../docs/10-observability-and-alerting.md) | R-OBS-* | WS-INFRA, WS-STORAGE |
+| **WS-UPDATE** | Automated updates | [11](../docs/11-automated-updates.md) | R-UPD-* | WS-GITOPS, WS-OBS |
+| **WS-VM** | KubeVirt IaaS | [06](../docs/06-vm-workloads-kubevirt.md) | R-PLAT-03 | WS-NET, WS-STORAGE |
+| **WS-APPS** | Service onboarding (13 services) | [13](../docs/13-service-catalog.md) | app requirements | WS-GITOPS, WS-NET, WS-STORAGE, WS-SEC, WS-OBS |
+| **WS-C2** | Command & Control UI | [12](../docs/12-command-center-ui.md) | R-C2-* | WS-OBS, WS-APPS, WS-SEC |
+| **WS-TEST** | Conformance testing (cross-cutting) | [14](../docs/14-testing-and-conformance.md) | R-TEST-* | WS-INFRA, WS-GITOPS |
+
+## Key interfaces (contracts between workstreams)
+
+- **ServiceDefinition schema** (WS-GITOPS owns) — the operator surface every app/VM is expressed in.
+  WS-APPS, WS-VM consume it; WS-NET/STORAGE/SEC/OBS render pieces of it. **The most important
+  contract in the system.**
+- **Network baseline & policy rendering** (WS-NET) — default-deny + how `dependencies` and `egress`
+  compile to CiliumNetworkPolicies/egress routing. Consumed by WS-APPS/WS-VM.
+- **StorageClasses** (WS-STORAGE) — `longhorn` (replicated) and `bulk` (media). Consumed by everyone
+  with state.
+- **Issuer + Gateway TLS** (WS-SEC + WS-NET) — wildcard cert + Gateway listeners that HTTPRoutes
+  attach to.
+- **Observability contract** (WS-OBS) — how a service declares metrics/SLIs/alerts and gets
+  dashboards. Consumed by WS-APPS/WS-VM.
+- **Update policy contract** (WS-UPDATE) — ImagePolicy/Renovate conventions a ServiceDefinition's
+  `update` block maps to.
+- **Conformance harness + gate** (WS-TEST) — the reproducible test framework (**Kyverno** for L0
+  policy, **Chainsaw** for L1/L2/L3 behavior + fault-injection, [ADR-0011](../decisions/adr-0011-test-tooling-kyverno-chainsaw.md)),
+  requirement-coverage report, and CI/phase gates every workstream's tests plug into. Each workstream
+  **authors its own behavior tests** against this harness ([14](../docs/14-testing-and-conformance.md)).
+
+## Parallelization guidance
+- **WS-TEST stands up the conformance harness early (Phase 0–1)** so every other workstream lands
+  with passing, reproducible tests from day one — testing is not a trailing phase.
+- After WS-INFRA delivers a cluster + Flux (Phase 1), WS-NET, WS-STORAGE, WS-SEC can proceed largely
+  in parallel (Phase 2).
+- WS-OBS, WS-UPDATE, WS-VM layer on next (Phases 3–5), overlapping.
+- WS-APPS is gated on the platform contracts being stable; onboard services in the dependency order
+  below.
+- WS-C2 is last-ish (Phase 6) since it aggregates everything.
+
+## WS-APPS onboarding order (respect dependencies from [13](../docs/13-service-catalog.md))
+1. **pihole** (DNS underpinning) → 2. **plex** (largest; validates bulk tier + public exposure) →
+3. **qbittorrent, jackett** (VPN + kill-switch proof) → 4. **sonarr, radarr** (VPN + arr↔downloader
+allows) → 5. **seerr** (cross-service allows to arr + plex) → 6. **nextcloud, gitea** (DR-critical
+data + backups) → 7. **minecraft, unifi, semaphore, kasm** (remaining).
+
+Each onboarding is a full acceptance-gate pass ([`../agents/definition-of-done.md`](../agents/definition-of-done.md)),
+not just a merged manifest.