From 70a98675ac55abe85718e0cb2fd9ac5fab58bc84 Mon Sep 17 00:00:00 2001 From: Keith Jackson Date: Sun, 16 Aug 2026 00:27:24 -0500 Subject: [PATCH] Phase 0 clarifications: resolve 5 ambiguities surfaced by implementation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - docs/07: correct plaintext-secret enforcement — CI-time ciphertext check vs admission SOPS-provenance (Flux decrypts before the API server, so an admitted Secret carries no ciphertext; a 'reject plaintext' admission rule is impossible). - docs/13: add Contract clarifications (v1.1) — source exactly-one-of; longhorn⇒size (L0 policy); backup is a {policy} ref; update.channel free-string/stable default; auth.groups required for oidc|forward-auth; public⇒auth.mode≠none (schema); irreplaceable⇒off-cluster-backup (L0 policy). - docs/02: Cilium install ownership — Ansible bootstraps, Flux adopts config, value-parity contract. - docs/03: platform label-key convention (homelab.keithjackson.dev/{egress-class,auth-mode,...}). - docs/14: ServiceDefinition validation is two complementary L0 gates (JSON Schema shape + Kyverno invariants). Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/02-environments-and-provisioning.md | 9 +++++++++ docs/03-networking-isolation-and-egress.md | 7 +++++++ docs/07-security-and-secrets.md | 8 +++++++- docs/13-service-catalog.md | 20 ++++++++++++++++++++ docs/14-testing-and-conformance.md | 6 ++++++ 5 files changed, 49 insertions(+), 1 deletion(-) diff --git a/docs/02-environments-and-provisioning.md b/docs/02-environments-and-provisioning.md index eb5b575..cf1bb87 100644 --- a/docs/02-environments-and-provisioning.md +++ b/docs/02-environments-and-provisioning.md @@ -48,6 +48,15 @@ Never fork the base config per environment. - Nodes are both control-plane and workload (3-node converged cluster); with only 3 nodes we do not isolate control-plane nodes. +### Cilium install ownership (Ansible bootstrap ↔ Flux) +There is a deliberate hand-off between the two control layers: **Ansible installs Cilium at cluster +bootstrap** (the CNI must exist before Flux can run any workload), and thereafter the **Flux +HelmRelease adopts and manages Cilium's configuration** ([05](05-gitops-and-delivery.md)). This +resolves the apparent overlap between "Ansible installs the CNI" (here) and "Flux manages +`platform/`" (docs/05). The contract: the bootstrap values and the Flux HelmRelease values MUST stay +in **parity** for the settings that define cluster networking — notably `kubeProxyReplacement` and +`k8sServiceHost`/`k8sServicePort` — or the two layers will fight over the CNI config. + ### Bare-metal specifics (prod, design-only for now) - Each mini-PC contributes local NVMe/SSD to Longhorn; document the disk layout (OS vs. Longhorn data device) before prod build. diff --git a/docs/03-networking-isolation-and-egress.md b/docs/03-networking-isolation-and-egress.md index ba6f481..8925c52 100644 --- a/docs/03-networking-isolation-and-egress.md +++ b/docs/03-networking-isolation-and-egress.md @@ -35,6 +35,13 @@ Cilium replaces the default k3s CNI and provides: - **Platform namespace(s)** (Flux, cert-manager, monitoring, storage) are isolated from application namespaces; only the specific flows they need (e.g. metrics scraping) are opened. +**Platform label-key convention.** The spec fixes label *values* (e.g. `internet|vpn`, +`oidc|forward-auth|native|none`); the label *keys* the platform uses are: +`homelab.keithjackson.dev/egress-class`, `homelab.keithjackson.dev/auth-mode`, and the supporting +`homelab.keithjackson.dev/{app-namespace,baseline}`. These keys are the contract that network +policies, Kyverno rules, and renderers key off (values per this doc, [13](13-service-catalog.md), +and [15](15-identity-and-access.md)). + ### Default-deny baseline (R-NET-01) Every namespace gets a baseline policy set applied by GitOps at namespace creation: - **Deny all ingress** and **deny all egress** by default. diff --git a/docs/07-security-and-secrets.md b/docs/07-security-and-secrets.md index 3f20052..c6d6e95 100644 --- a/docs/07-security-and-secrets.md +++ b/docs/07-security-and-secrets.md @@ -23,10 +23,16 @@ Defense in depth with declarative, auditable controls: control, complementing network default-deny (the primary *isolation* control). - **Policy enforcement (admission)**: **Kyverno** runs in-cluster as an admission controller and rejects non-conforming resources at reconcile time — e.g. a namespace missing its default-deny - baseline, a plaintext Secret, a workload without a required egress-class label. The *same* policies + baseline, or a workload without a required egress-class/auth-mode label. The *same* policies gate CI ([14](14-testing-and-conformance.md), [ADR-0011](../decisions/adr-0011-test-tooling-kyverno-chainsaw.md)), so invariants are enforced both before merge and at admission (defense in depth). Enforcement complements — never replaces — the runtime behavior tests. + - **Plaintext-secret enforcement is split by layer (do not conflate):** the "no plaintext Secret" + invariant is a **CI-time** check (the committed Secret must carry SOPS ciphertext). It **cannot** + be an admission rule, because Flux **decrypts SOPS before** the object reaches the API server — + an admitted Secret never carries ciphertext, so an admission "reject plaintext" rule would reject + *every* real Secret. Admission instead enforces **SOPS provenance** (the Secret was reconciled + through the Flux/SOPS path), not ciphertext presence. - **Auditability**: isolation, egress path, cert validity, and backup integrity are all demonstrable (R-SEC-03) via Hubble, cert status, and DR drills. diff --git a/docs/13-service-catalog.md b/docs/13-service-catalog.md index e219bb2..fc5616c 100644 --- a/docs/13-service-catalog.md +++ b/docs/13-service-catalog.md @@ -112,6 +112,26 @@ radarr → qbittorrent, jackett **"…but not limited to":** the catalog is open-ended. New services follow the same ServiceDefinition and inherit all guarantees automatically — that is the point of the abstraction. +## Contract clarifications (v1.1 — resolved 2026-08-16) + +Points the schema left ambiguous, now pinned so the JSON Schema, Kyverno policies, and renderers agree: +- **`source` is exactly-one-of** — `image` for `kind: container`, `vmTemplate` for `kind: vm` + (schema-enforced conditional). +- **`storage[].class: longhorn ⇒ size` is required** — enforced by an **L0 Kyverno policy** (a + replicated volume with no size is a defect). `bulk` size may be omitted (external mount). +- **`backup` is a reference** — `backup: {policy: }` where `policy` is either the literal + `reacquirable` (bulk/re-acquirable data) or a named off-cluster backup schedule. Cadence/retention + live in the referenced schedule object, **not** inline in the ServiceDefinition. +- **`update.channel` is a free string defaulting to `stable`** — not a closed enum, so the sanctioned + "opt-in to a non-stable channel" (R-UPD-01) is expressible per service. +- **`auth.groups` is required for `auth.mode ∈ {oidc, forward-auth}`** — access must be scoped to at + least one Keycloak group (R-IDP-06); optional for `native`/`none`. +- **`exposure: public ⇒ auth.mode ≠ none`** — a public service may not be unauthenticated + (docs/15, R-IDP-02); **schema-enforced** invariant. +- **Irreplaceable ⇒ off-cluster backup** — DR-critical services MUST NOT be `reacquirable`. This + depends on per-service criticality the schema can't see, so it is enforced by an **L0 Kyverno + policy** keyed on the service's criticality signal, not by the JSON Schema. + ## Cross-cutting notes - **Plex media** is the R-STATE-04 case: keep the metadata DB on replicated Longhorn; keep the large library on the bulk tier; downloads stage on bulk and are handed off by the arr stack. diff --git a/docs/14-testing-and-conformance.md b/docs/14-testing-and-conformance.md index c1a4c57..2ef5b5c 100644 --- a/docs/14-testing-and-conformance.md +++ b/docs/14-testing-and-conformance.md @@ -122,6 +122,12 @@ from the **Kyverno ecosystem**: behavior tests**: a policy admitting a manifest is not proof the runtime behaves (R-TEST-01), so L2/L3 Chainsaw tests remain mandatory. Policies are authored once and shared between the CLI gate and the admission controller. + - **ServiceDefinition validation is two complementary gates, not one** (do not consolidate): the + **JSON Schema** (`schemas/servicedefinition.schema.json`) checks *shape* — types, enums, required + fields, exactly-one-of `source`, `public ⇒ auth.mode≠none` — as a fast, cluster-free PR check; + **Kyverno `kyverno-json`** enforces the *cross-cutting invariants* the schema can't express + (`longhorn ⇒ size`, irreplaceable⇒off-cluster-backup, label conventions). Both are L0 and run in + CI; they overlap by design (defense in depth), not by redundancy. - **Chainsaw** — **L1 rendering, L2 behavioral, and L3 fault-injection.** Declarative steps (apply/assert/error, `script`/`command`, `try`/`catch`/`finally`), bounded polling with explicit timeouts, per-test namespace isolation, and automatic cleanup satisfy the reproducibility contract.