Admission controller was crashlooping (9 restarts/4h, exit 255, NOT OOM: live usage ~65Mi/384Mi limit, 7m cpu). Root cause: chart-default 100m CPU request starved the process under node contention, so client-go calls hit context-deadline-exceeded, leader-election lease renewal timed out, leadership was lost and the webhook server closed. Raise the CPU request to 250m (no CPU limit, to avoid CFS throttling) with modest memory headroom (256Mi/512Mi), and raise dev admission replicas 1->2 so a single fail-closed pod restart no longer 502s cluster-wide admission-gated writes. Co-Authored-By: Claude Opus 4.8 <[email protected]>
platform/kyverno — L0 policy-as-code gate & admission enforcer (T-0.5)
Kyverno is the platform's L0 static/policy gate (in CI, no cluster) and its
in-cluster admission enforcer (ADR-0011, docs/14, docs/07). The policies here
encode spec invariants as code: they run in CI as a merge gate today (Phase 0, no
cluster) and the same policies are wired to run in-cluster as an admission controller
later (Phase 2, admission/).
Enforcement complements, never replaces, behavior tests. A policy admitting a manifest is not proof the runtime behaves (R-TEST-01). The L2/L3 Chainsaw behavior + fault-injection tests remain mandatory. See
admission/README.md.
Layout
platform/kyverno/
├── README.md # this file: policy→requirement map + CI entrypoint
├── labels.md # label/annotation convention (PROPOSED contract — escalated)
├── policies/ # ClusterPolicies (authored once; CI + admission)
│ ├── require-default-deny-baseline.yaml # R-NET-01 (generate baseline)
│ ├── validate-default-deny-shape.yaml # R-NET-01 (reject malformed baseline)
│ ├── disallow-plaintext-secrets.yaml # R-SEC-01 (CI-only: sops block required)
│ ├── restrict-host-namespaces.yaml # R-SEC-02 (+R-NET-01 bypass guard)
│ ├── require-egress-class-label.yaml # R-NET-05
│ └── require-auth-mode-label.yaml # R-IDP-02
├── servicedefinition/ # ServiceDefinition shape validation (kyverno-json)
│ ├── validate-servicedefinition.yaml # R-GITOPS-02
│ ├── payloads/ (valid-* / invalid-*) # pass & fail fixtures
│ └── README.md
├── tests/ # `kyverno test` suite — runnable L0 tests NOW, no cluster
│ ├── kyverno-test.yaml # policies × fixtures × expected results
│ └── resources/ (good-* / bad-* fixtures + generated golden)
├── admission/ # in-cluster enforcing bundle (scaffolding, T-2.7)
│ ├── namespace.yaml · helmrelease.yaml · kustomization.yaml
│ ├── secret-provenance.yaml # R-SEC-01 admission variant (provenance)
│ └── README.md
└── scripts/run-l0.sh # the single-command CI entrypoint for the gate
Policy → requirement map
| Policy / artifact | Rule(s) | Requirement | Spec source | Behaves |
|---|---|---|---|---|
require-default-deny-baseline |
generate-default-deny |
R-NET-01 | docs/03 default-deny baseline | Every app namespace auto-gets a deny-all (ingress+egress, DNS-only) NetworkPolicy |
validate-default-deny-shape |
baseline-must-deny-both-directions |
R-NET-01 | docs/03 | A baseline that leaves a direction open is rejected |
disallow-plaintext-secrets |
secrets-must-be-sops-encrypted |
R-SEC-01 | docs/07, docs/05, ADR-0007 | Committed Secrets must be SOPS-encrypted (sops block); plaintext rejected. CI-only |
restrict-host-namespaces |
disallow-host-namespaces, disallow-privileged-containers |
R-SEC-02 (+R-NET-01) | docs/07 least reach | App pods can't use hostNetwork/PID/IPC or privileged (would bypass default-deny) |
require-egress-class-label |
require-egress-class |
R-NET-05 | docs/03, docs/13 | App pods must declare egress-class ∈ {internet,vpn} |
require-auth-mode-label |
require-auth-mode |
R-IDP-02 | docs/13, docs/15 | App pods must declare auth-mode ∈ {oidc,forward-auth,native,none} |
validate-servicedefinition (kyverno-json) |
shape / dependencies / storage | R-GITOPS-02 | docs/13 (+03,15) | ServiceDefinition is schema/invariant valid before render; public⇒not none auth |
admission/secret-provenance |
secrets-must-be-flux-managed |
R-SEC-01 | docs/07 | Admission-only variant: app-ns Secrets must come from Flux/SOPS (see below) |
Coverage of the T-0.5 requirement set: R-TEST-01 (behavior-tagged, reproducible
tests — this whole suite), R-NET-01, R-SEC-01, R-SEC-02, R-IDP-02,
R-NET-05, R-GITOPS-02. Every policy carries its R-* IDs in the
homelab.keithjackson.dev/requirements annotation so the coverage report can pick
them up automatically.
CI entrypoint (the L0 gate)
CI (WS-GITOPS .gitea/workflows, then Gitea Actions) runs the single command:
platform/kyverno/scripts/run-l0.sh
which performs, with pinned tool versions (KYVERNO_VERSION, KYVERNO_JSON_VERSION):
# 1) ClusterPolicy tests — offline, no cluster
kyverno test platform/kyverno/tests
# 2) ServiceDefinition validation — offline, no cluster
kyverno-json scan --policy platform/kyverno/servicedefinition/validate-servicedefinition.yaml \
--payload <each catalog/ ServiceDefinition and the valid-* fixtures> # must pass
# plus each invalid-* fixture must be REJECTED (script asserts non-zero exit)
kyverno test emits pass/fail per policy×rule×resource and supports JUnit output for
the coverage/merge gate:
kyverno test platform/kyverno/tests --output junit > results/kyverno-l0.junit.xml
Exit non-zero fails the merge gate (docs/14 gating rules). Deterministic, hermetic,
self-contained — no cluster, no external state (reproducibility contract, R-TEST-02).
Notes / handoffs
- Label keys are a proposed contract (
labels.md) shared with WS-NET / WS-SEC / WS-GITOPS. Values are spec-fixed; keys need owner sign-off (coordination-protocol §2). - Secrets CI vs admission divergence is deliberate and documented in
admission/README.mdandpolicies/disallow-plaintext-secrets.yaml. - The ServiceDefinition JSON Schema (WS-GITOPS T-0.3) is the complementary structural
contract; this Kyverno policy encodes the invariants from
docs/13. - Tool availability: neither
kyvernonorkyverno-jsonis installed on the WS-TEST workstation, so this suite was authored but not executed here; the CI agent (T-0.2/T-0.4 harness) runsscripts/run-l0.shas the gate.