Files
home-network-k3s-impl/platform/kyverno
jakman85andClaude Opus 4.8 28a34c4477 fix(kyverno): raise admission-controller resources + dev replicas 2 to stop fail-closed crashloop/502s
Admission controller was crashlooping (9 restarts/4h, exit 255, NOT OOM:
live usage ~65Mi/384Mi limit, 7m cpu). Root cause: chart-default 100m CPU
request starved the process under node contention, so client-go calls hit
context-deadline-exceeded, leader-election lease renewal timed out, leadership
was lost and the webhook server closed. Raise the CPU request to 250m (no CPU
limit, to avoid CFS throttling) with modest memory headroom (256Mi/512Mi), and
raise dev admission replicas 1->2 so a single fail-closed pod restart no longer
502s cluster-wide admission-gated writes.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-08-25 16:43:43 -05:00
..

platform/kyverno — L0 policy-as-code gate & admission enforcer (T-0.5)

Kyverno is the platform's L0 static/policy gate (in CI, no cluster) and its in-cluster admission enforcer (ADR-0011, docs/14, docs/07). The policies here encode spec invariants as code: they run in CI as a merge gate today (Phase 0, no cluster) and the same policies are wired to run in-cluster as an admission controller later (Phase 2, admission/).

Enforcement complements, never replaces, behavior tests. A policy admitting a manifest is not proof the runtime behaves (R-TEST-01). The L2/L3 Chainsaw behavior + fault-injection tests remain mandatory. See admission/README.md.

Layout

platform/kyverno/
├── README.md                     # this file: policy→requirement map + CI entrypoint
├── labels.md                     # label/annotation convention (PROPOSED contract — escalated)
├── policies/                     # ClusterPolicies (authored once; CI + admission)
│   ├── require-default-deny-baseline.yaml   # R-NET-01 (generate baseline)
│   ├── validate-default-deny-shape.yaml     # R-NET-01 (reject malformed baseline)
│   ├── disallow-plaintext-secrets.yaml      # R-SEC-01 (CI-only: sops block required)
│   ├── restrict-host-namespaces.yaml        # R-SEC-02 (+R-NET-01 bypass guard)
│   ├── require-egress-class-label.yaml      # R-NET-05
│   └── require-auth-mode-label.yaml         # R-IDP-02
├── servicedefinition/            # ServiceDefinition shape validation (kyverno-json)
│   ├── validate-servicedefinition.yaml      # R-GITOPS-02
│   ├── payloads/ (valid-* / invalid-*)      # pass & fail fixtures
│   └── README.md
├── tests/                        # `kyverno test` suite — runnable L0 tests NOW, no cluster
│   ├── kyverno-test.yaml                     # policies × fixtures × expected results
│   └── resources/ (good-* / bad-* fixtures + generated golden)
├── admission/                    # in-cluster enforcing bundle (scaffolding, T-2.7)
│   ├── namespace.yaml · helmrelease.yaml · kustomization.yaml
│   ├── secret-provenance.yaml                # R-SEC-01 admission variant (provenance)
│   └── README.md
└── scripts/run-l0.sh             # the single-command CI entrypoint for the gate

Policy → requirement map

Policy / artifact Rule(s) Requirement Spec source Behaves
require-default-deny-baseline generate-default-deny R-NET-01 docs/03 default-deny baseline Every app namespace auto-gets a deny-all (ingress+egress, DNS-only) NetworkPolicy
validate-default-deny-shape baseline-must-deny-both-directions R-NET-01 docs/03 A baseline that leaves a direction open is rejected
disallow-plaintext-secrets secrets-must-be-sops-encrypted R-SEC-01 docs/07, docs/05, ADR-0007 Committed Secrets must be SOPS-encrypted (sops block); plaintext rejected. CI-only
restrict-host-namespaces disallow-host-namespaces, disallow-privileged-containers R-SEC-02 (+R-NET-01) docs/07 least reach App pods can't use hostNetwork/PID/IPC or privileged (would bypass default-deny)
require-egress-class-label require-egress-class R-NET-05 docs/03, docs/13 App pods must declare egress-class ∈ {internet,vpn}
require-auth-mode-label require-auth-mode R-IDP-02 docs/13, docs/15 App pods must declare auth-mode ∈ {oidc,forward-auth,native,none}
validate-servicedefinition (kyverno-json) shape / dependencies / storage R-GITOPS-02 docs/13 (+03,15) ServiceDefinition is schema/invariant valid before render; public⇒not none auth
admission/secret-provenance secrets-must-be-flux-managed R-SEC-01 docs/07 Admission-only variant: app-ns Secrets must come from Flux/SOPS (see below)

Coverage of the T-0.5 requirement set: R-TEST-01 (behavior-tagged, reproducible tests — this whole suite), R-NET-01, R-SEC-01, R-SEC-02, R-IDP-02, R-NET-05, R-GITOPS-02. Every policy carries its R-* IDs in the homelab.keithjackson.dev/requirements annotation so the coverage report can pick them up automatically.

CI entrypoint (the L0 gate)

CI (WS-GITOPS .gitea/workflows, then Gitea Actions) runs the single command:

platform/kyverno/scripts/run-l0.sh

which performs, with pinned tool versions (KYVERNO_VERSION, KYVERNO_JSON_VERSION):

# 1) ClusterPolicy tests — offline, no cluster
kyverno test platform/kyverno/tests

# 2) ServiceDefinition validation — offline, no cluster
kyverno-json scan --policy platform/kyverno/servicedefinition/validate-servicedefinition.yaml \
  --payload <each catalog/ ServiceDefinition and the valid-* fixtures>   # must pass
#   plus each invalid-* fixture must be REJECTED (script asserts non-zero exit)

kyverno test emits pass/fail per policy×rule×resource and supports JUnit output for the coverage/merge gate:

kyverno test platform/kyverno/tests --output junit > results/kyverno-l0.junit.xml

Exit non-zero fails the merge gate (docs/14 gating rules). Deterministic, hermetic, self-contained — no cluster, no external state (reproducibility contract, R-TEST-02).

Notes / handoffs

  • Label keys are a proposed contract (labels.md) shared with WS-NET / WS-SEC / WS-GITOPS. Values are spec-fixed; keys need owner sign-off (coordination-protocol §2).
  • Secrets CI vs admission divergence is deliberate and documented in admission/README.md and policies/disallow-plaintext-secrets.yaml.
  • The ServiceDefinition JSON Schema (WS-GITOPS T-0.3) is the complementary structural contract; this Kyverno policy encodes the invariants from docs/13.
  • Tool availability: neither kyverno nor kyverno-json is installed on the WS-TEST workstation, so this suite was authored but not executed here; the CI agent (T-0.2/T-0.4 harness) runs scripts/run-l0.sh as the gate.