Files
home-network-k3s-impl/platform/kyverno
jakman85andClaude Opus 5 8eb5baffa4
gate / gate (pull_request) Successful in 1m38s
gate / conformance (pull_request) Has been skipped
feat(sec): constrain the conformance CI trigger at admission (R-SEC-02)
Implements the credential model the operator chose for wiring L2/L3 into CI: a narrow
trigger SA bounded by a fail-closed Kyverno admission policy, rather than the
zero-credential GitOps-native alternative (plan section 4, primary model).

THE ESCALATION THIS CLOSES. Kubernetes places NO `use` check on a Job's
serviceAccountName. Any principal that can `create jobs` in ns conformance can create a Job
running as conformance-runner -- nodes/patch, pods/exec, namespaces/patch -- with an
arbitrary command. A CI token with plain `create jobs` IS that privileged identity no matter
how narrow its own Role looks. RBAC cannot fix this; only constraining the SHAPE of what the
trigger may create can.

Adds:
  * SA/Role/RoleBinding conformance-trigger -- create+get+list+watch jobs, get pods and
    pods/log, nothing else. Namespaced. No delete (a trigger that can delete can hide a
    failed run), no exec, no secrets. Results come back as a tarball on stdout read via
    pods/log, so no execution channel into the runner pod is ever granted.
  * ClusterPolicy conformance-trigger-job-shape -- Enforce, failurePolicy: Fail. It does NOT
    use ${KYVERNO_VALIDATION_ACTION}: a security boundary an overlay can downgrade to Audit
    is not a boundary. It constrains ONLY conformance-trigger, so a mistake here cannot wedge
    the rest of the cluster.
  * Flux Kustomization `conformance` -> ./conformance/incluster, wait:false and non-gating
    like apps-vms, so a test identity can never stall the platform or app tiers.
  * A kyverno test with seven cases, six of them real escalation paths.

VERIFIED by `kyverno apply` (pass: 10, fail: 6, error: 0), not assumed:
  evil-command    DENIED at /spec/template/spec/containers/0/command/   <- THE attack
  evil-image      DENIED at .../image/
  tag-not-digest  DENIED at .../image/          (mutable tag is republishable)
  host-namespaces DENIED at /spec/template/spec/hostPID/
  extra-volume    DENIED by the volumes foreach
  injected-args   DENIED by the args foreach
  good-run        ADMITTED

ALSO FIXES A FAIL-OPEN IN THE GATE ITSELF. `kyverno test` does NOT fail when a policy cannot
LOAD: every row becomes "Skip / Invalid Policy" and it still prints "N tests passed" and
exits 0. Measured while writing this policy -- a schema error (subjects in a separate
match.all entry from resources) produced exactly that, and ci/l0.sh would have certified a
security policy that does not load as green. This is not specific to this policy: a typo in
restrict-host-namespaces or secret-provenance would silently disable it with the gate still
green. ci/l0.sh now scans for "Invalid Policy" and fails. Proven: broken policy -> l0 rc=1,
restored -> rc=0.

IMAGE PINNING, HONEST LIMIT: the plan specifies pinning the exact runner image digest. That
image does not exist yet, so this enforces the enforceable half -- the image must be under
the conformance repository path AND digest-pinned, never a mutable tag. Residual surface:
anyone who can PUSH to that path could publish a different digest. Tighten to the literal
digest when the image is built; it is a one-line change to the pattern, reviewed in a PR,
which is where a change to a security boundary belongs.

NOT YET WIRED, deliberately: no runner image, no trigger kubeconfig, no CLUSTER_AVAILABLE.
The identities here are inert until those exist, and PR #12 makes CLUSTER_AVAILABLE=true
block rather than fake a green in the meantime.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-29 03:22:41 -05:00
..

platform/kyverno — L0 policy-as-code gate & admission enforcer (T-0.5)

Kyverno is the platform's L0 static/policy gate (in CI, no cluster) and its in-cluster admission enforcer (ADR-0011, docs/14, docs/07). The policies here encode spec invariants as code: they run in CI as a merge gate today (Phase 0, no cluster) and the same policies are wired to run in-cluster as an admission controller later (Phase 2, admission/).

Enforcement complements, never replaces, behavior tests. A policy admitting a manifest is not proof the runtime behaves (R-TEST-01). The L2/L3 Chainsaw behavior + fault-injection tests remain mandatory. See admission/README.md.

Layout

platform/kyverno/
├── README.md                     # this file: policy→requirement map + CI entrypoint
├── labels.md                     # label/annotation convention (PROPOSED contract — escalated)
├── policies/                     # ClusterPolicies (authored once; CI + admission)
│   ├── require-default-deny-baseline.yaml   # R-NET-01 (generate baseline)
│   ├── validate-default-deny-shape.yaml     # R-NET-01 (reject malformed baseline)
│   ├── disallow-plaintext-secrets.yaml      # R-SEC-01 (CI-only: sops block required)
│   ├── restrict-host-namespaces.yaml        # R-SEC-02 (+R-NET-01 bypass guard)
│   ├── require-egress-class-label.yaml      # R-NET-05
│   ├── require-auth-mode-label.yaml         # R-IDP-02
│   └── label-cdi-worker-pods.yaml           # R-NET-05+R-IDP-02 (mutate: CDI's own worker pods)
├── servicedefinition/            # ServiceDefinition shape validation (kyverno-json)
│   ├── validate-servicedefinition.yaml      # R-GITOPS-02
│   ├── payloads/ (valid-* / invalid-*)      # pass & fail fixtures
│   └── README.md
├── tests/                        # `kyverno test` suite — runnable L0 tests NOW, no cluster
│   ├── kyverno-test.yaml                     # policies × fixtures × expected results
│   ├── resources/ (good-* / bad-* fixtures + generated golden)
│   └── controller-worker-pods/               # the CDI-worker mutation, proven BOTH ways
│       ├── kyverno-test.yaml                 #   2 Tests: cdi-sa requester vs ordinary requester
│       ├── userinfo-cdi-sa.yaml / userinfo-ordinary.yaml
│       └── resources/ + patched/             #   real + FORGED importer pods; expected patch
├── admission/                    # in-cluster enforcing bundle (scaffolding, T-2.7)
│   ├── namespace.yaml · helmrelease.yaml · kustomization.yaml
│   ├── secret-provenance.yaml                # R-SEC-01 admission variant (provenance)
│   └── README.md
└── scripts/run-l0.sh             # the single-command CI entrypoint for the gate

Policy → requirement map

Policy / artifact Rule(s) Requirement Spec source Behaves
require-default-deny-baseline generate-default-deny R-NET-01 docs/03 default-deny baseline Every app namespace auto-gets a deny-all (ingress+egress, DNS-only) NetworkPolicy
validate-default-deny-shape baseline-must-deny-both-directions R-NET-01 docs/03 A baseline that leaves a direction open is rejected
disallow-plaintext-secrets secrets-must-be-sops-encrypted R-SEC-01 docs/07, docs/05, ADR-0007 Committed Secrets must be SOPS-encrypted (sops block); plaintext rejected. CI-only
restrict-host-namespaces disallow-host-namespaces, disallow-privileged-containers R-SEC-02 (+R-NET-01) docs/07 least reach App pods can't use hostNetwork/PID/IPC or privileged (would bypass default-deny)
require-egress-class-label require-egress-class R-NET-05 docs/03, docs/13 App pods must declare egress-class ∈ {internet,vpn}
require-auth-mode-label require-auth-mode R-IDP-02 docs/13, docs/15 App pods must declare auth-mode ∈ {oidc,forward-auth,native,none}
label-cdi-worker-pods (mutate) add-worker-pod-labels R-NET-05, R-IDP-02 docs/03, docs/13 CDI's controller-generated worker pods (importer-*/uploader-*/cloner-*, created by SA cdi/cdi-sa) get egress-class: internet + auth-mode: none added, so they satisfy the two policies above instead of being exempted from them. Matched on the generating ServiceAccount, never a pod-name prefix
validate-servicedefinition (kyverno-json) shape / dependencies / storage R-GITOPS-02 docs/13 (+03,15) ServiceDefinition is schema/invariant valid before render; public⇒not none auth
admission/secret-provenance secrets-must-be-flux-managed R-SEC-01 docs/07 Admission-only variant: app-ns Secrets must come from Flux/SOPS (see below)

Coverage of the T-0.5 requirement set: R-TEST-01 (behavior-tagged, reproducible tests — this whole suite), R-NET-01, R-SEC-01, R-SEC-02, R-IDP-02, R-NET-05, R-GITOPS-02. Every policy carries its R-* IDs in the homelab.keithjackson.dev/requirements annotation so the coverage report can pick them up automatically.

CI entrypoint (the L0 gate)

CI (WS-GITOPS .gitea/workflows, then Gitea Actions) runs the single command:

platform/kyverno/scripts/run-l0.sh

which performs, with pinned tool versions (KYVERNO_VERSION, KYVERNO_JSON_VERSION):

# 1) ClusterPolicy tests — offline, no cluster
kyverno test platform/kyverno/tests

# 2) ServiceDefinition validation — offline, no cluster
kyverno-json scan --policy platform/kyverno/servicedefinition/validate-servicedefinition.yaml \
  --payload <each catalog/ ServiceDefinition and the valid-* fixtures>   # must pass
#   plus each invalid-* fixture must be REJECTED (script asserts non-zero exit)

kyverno test emits pass/fail per policy×rule×resource and supports JUnit output for the coverage/merge gate:

kyverno test platform/kyverno/tests --output junit > results/kyverno-l0.junit.xml

Exit non-zero fails the merge gate (docs/14 gating rules). Deterministic, hermetic, self-contained — no cluster, no external state (reproducibility contract, R-TEST-02).

Notes / handoffs

  • Label keys are a proposed contract (labels.md) shared with WS-NET / WS-SEC / WS-GITOPS. Values are spec-fixed; keys need owner sign-off (coordination-protocol §2).
  • Controller-generated pods are a cold-start hazard. A policy that matches any Pod in an app namespace also matches pods that no manifest in git authors. Prod hit this on 2026-09-06: require-auth-mode-label + require-egress-class-label denied CDI's own importer-prime-* pod, so DataVolume unifi/unifi-rootdisk stuck in ImportScheduled and the UniFi VM had no disk. Fix: policies/label-cdi-worker-pods.yaml mutates those pods to carry truthful labels (matched on the generating ServiceAccount cdi/cdi-sa), rather than excluding them. virt-launcher-* pods need nothing — they inherit the VM pod-template labels. KubeVirt hotplug (hp-volume-*) and export (virt-export-*) pods do not inherit them and are unused here; extend the same policy if either is adopted.
  • PolicyException is NOT usable in this install: the Kyverno admission/background/reports controllers all run --enablePolicyException=false (chart default, verified on prod 2026-09-06). The CRD exists (from the pre-applied CRD bundle) so a PolicyException would APPLY cleanly and be silently ignored — a policy hole that looks like a fix and isn't.
  • Secrets CI vs admission divergence is deliberate and documented in admission/README.md and policies/disallow-plaintext-secrets.yaml.
  • The ServiceDefinition JSON Schema (WS-GITOPS T-0.3) is the complementary structural contract; this Kyverno policy encodes the invariants from docs/13.
  • Tool availability: neither kyverno nor kyverno-json is installed on the WS-TEST workstation, so this suite was authored but not executed here; the CI agent (T-0.2/T-0.4 harness) runs scripts/run-l0.sh as the gate.