jakman85andClaude Opus 4.8 7598cff945 ADR-0015: VPN egress steering — node-level tunnel + Cilium Egress Gateway
The in-cluster Gluetun-pod gateway (ADR-0013) cannot be steered: a pod's tun0 is
invisible to the node datapath, and hostNetwork Gluetun hijacks the node's own
default route (breaks k3s/etcd). The tunnel itself works (PIA egress validated),
but not the steering.

New decision (ADR-0015): move the tunnel to a HOST OpenVPN systemd service on
egress-gateway node(s) (Ansible-managed, split-tunnel so it doesn't touch the node
default route), steer egress-class=vpn pods with a CiliumEgressGatewayPolicy (native
eBPF, no client-pod changes), and enforce the kill-switch with a host nftables rule
(fail-closed independent of Cilium). PIA/OpenVPN-UDP provider decision (ADR-0013)
stands; the .ovpn transfers over; PF re-solved host-side (Phase 6).

Amended for consistency: ADR-0013 (client+steering superseded), decisions/README,
docs/00, docs/01, docs/03, docs/07, agents/task-board.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-08-24 00:46:18 -05:00

Home Platform — k3s + Longhorn Onboarding Platform

This repository is a specification, not an implementation. It defines what to build and how the work is organized so that teams of AI agents (and their human reviewers) can implement a self-service onboarding platform for home infrastructure at both the PaaS (containerized apps) and IaaS (virtual machines) level.

What we are building

A single, GitOps-driven platform, running on a k3s Kubernetes cluster backed by Longhorn distributed storage, that lets an operator onboard, run, update, and tear down services — both containers and virtual machines — through one consistent Kubernetes API, CLI, and YAML configuration surface.

The platform must provide:

  • PaaS: containerized workloads onboarded via GitOps.
  • IaaS: virtual machines managed through the same Kubernetes API/CLI/YAML (via KubeVirt), for workloads that cannot be containerized.
  • Network isolation: default-deny east-west traffic; explicit, port-level allow rules between services (firewall-like).
  • Multi-WAN egress: some services egress the normal internet; others are routed through a custom VPN connection. Egress path is a declarative property of a service.
  • Stateful durability: any service can be destroyed and rebuilt on a whim without losing data.
  • Hands-off updates: services update to latest stable versions on a schedule — no human, no AI, no hand-written GitOps change required.
  • Trusted TLS: every service gets an auto-issued, auto-renewed, non-self-signed certificate.
  • Observability: services are monitored; issues are reported by email and on dashboards.
  • A command & control center: the root of home.keithjackson.dev is a UI that surfaces and controls everything running in the cluster.
  • Provable conformance: every requirement is verified by an automated, reproducible, behavior- based test — and the specifications themselves are human-owned and immutable to the implementing agents.

Two environments

Environment Substrate Purpose Status
dev/test Local KVM virtual machines on a workstation, provisioned by IaC Prove the platform end-to-end before hardware exists Active development target
prod 3× mini-PC bare-metal k3s cluster, each with local storage The real deployment Not ready — design for it, do not build on it yet

The dev/test environment is the proving ground. Everything must be authored so that the same GitOps configuration promotes to production with only environment-scoped overrides.

How to use this repository (start here)

  1. Read AGENTS.md — the rules of engagement for every AI agent working here.
  2. Read docs/00-vision-and-scope.md — the north star and the non-negotiable requirements (with stable requirement IDs).
  3. Read docs/01-architecture-overview.md — the chosen stack and how the pieces fit.
  4. Consult decisions/ — the Architecture Decision Records (ADRs) that lock in every major technology choice and why. Do not re-litigate a decision; if you disagree, open a superseding ADR per the process in decisions/README.md.
  5. Pick up work from roadmap/ and agents/task-board.md — phased delivery plan, workstreams, dependencies, and the live task board.

Repository map

home-network-k3s/
├── README.md                     ← you are here
├── AGENTS.md                     ← rules of engagement for AI agents
├── CONTRIBUTING.md               ← git workflow (feature branches → main; Gitea/CI-CD to come)
├── GLOSSARY.md                   ← shared vocabulary (read to avoid ambiguity)
├── docs/                         ← the specification, by domain
│   ├── 00-vision-and-scope.md
│   ├── 01-architecture-overview.md
│   ├── 02-environments-and-provisioning.md
│   ├── 03-networking-isolation-and-egress.md
│   ├── 04-storage-and-state.md
│   ├── 05-gitops-and-delivery.md
│   ├── 06-vm-workloads-kubevirt.md
│   ├── 07-security-and-secrets.md
│   ├── 08-tls-and-certificates.md
│   ├── 09-dns-and-ingress.md
│   ├── 10-observability-and-alerting.md
│   ├── 11-automated-updates.md
│   ├── 12-command-center-ui.md
│   ├── 13-service-catalog.md
│   ├── 14-testing-and-conformance.md
│   └── 15-identity-and-access.md
├── decisions/                    ← ADRs (one decision per file) + index/template
├── roadmap/                      ← phases.md, workstreams.md
└── agents/                       ← roles, coordination protocol, DoD, task board

Requirement traceability

Every hard requirement has a stable ID (e.g. R-NET-01) defined in docs/00-vision-and-scope.md. Implementation work, ADRs, and task-board items must reference the requirement IDs they satisfy. This is how we prove the built system meets the spec.

Ground rules (the short version — full detail in AGENTS.md)

  • Specs are the source of truth — and are immutable to agents. Code and tests serve the spec. AI agents may not change the docs/, requirements, or accepted ADRs; a spec that is wrong or can't be met is an escalation to the human, never an edit (R-TEST-06).
  • Done means a green, reproducible, behavior-based test. Every requirement is verified by an automated, requirement-tagged conformance test (docs/14-testing-and-conformance.md, ADR-0010); a manual demo or "it merged" is not acceptance.
  • Everything is GitOps. No click-ops in the cluster except where an ADR explicitly allows it.
  • Nothing is self-signed, nothing is manual, nothing loses data. These three properties are acceptance gates, not aspirations — and each is proven by a test.
  • Design once, deploy to two environments. dev/test and prod differ only by scoped overrides.
S
Description
No description provided
Readme
420 KiB
0 Stars 2 Watchers 0 Forks
Languages
Markdown 100%