fix(kyverno): unblock CDI imports by SATISFYING the label requirement, not exempting it
Prod's UniFi VM has no disk. `DataVolume unifi/unifi-rootdisk` stuck in
ImportScheduled for hours, because Kyverno denied CDI's own generated pod:
validate.kyverno.svc-fail denied Pod/unifi/importer-prime-<uid>
require-auth-mode-label (R-IDP-02)
require-egress-class-label (R-NET-05)
Both policies match any Pod in an `app-namespace: "true"` namespace with no
exclusion, and CDI generates `importer-*` itself — it cannot carry repo labels.
A COLD-START defect, not a prod quirk: dev's identical import Succeeded 9 days
ago, before the policies enforced, so dev has never exercised it and would hit it
on any rebuild. Fourth bug of this shape in two days.
MECHANISM: a MUTATE ClusterPolicy that ADDS
`egress-class: internet` + `auth-mode: none` (add-if-absent anchor) to Pods on
CREATE in app namespaces WHEN the requesting subject is ServiceAccount
`cdi/cdi-sa`. Both validate policies are untouched — no exclude, no exception,
`background: true` preserved on them.
Why mutate beats exempting: the requirement stays literally true ("every pod in an
app namespace declares both labels") and the values are HONEST — the importer does
pull over the default WAN path, and `internet` is the least-reach class (it does
not grant the VPN path); a transient worker pod with no UI genuinely is
`auth-mode: none`.
THE MATCH IS IDENTITY, NOT A NAME. `subjects: ServiceAccount cdi/cdi-sa` is the
AdmissionReview userInfo, taken from the real prod denial
(`system:serviceaccount:cdi:cdi-sa`, name importer-prime-…, op CREATE). A name
prefix is attacker-choosable; a requesting ServiceAccount is not — and the test
proves it, with a fixture `importer-prime-forged-by-an-app` that copies both the
name prefix AND CDI's labels and is still DENIED. Covers every CDI worker kind
(importer/uploader/cloner) without enumerating names that change per release.
Confirmed on prod that cdi-sa is the only cdi-namespace SA that can create pods.
REJECTED, and the first one is a trap worth recording:
* PolicyException — the CRD EXISTS (v2/v2beta1, from the pre-applied bundle) but
all controllers run `--enablePolicyException=false` (verified live on prod:
admission, background and reports controllers all carry the flag). An exception
would apply cleanly through Flux and be SILENTLY IGNORED. Enabling the gate is a
cluster-wide capability change for no gain over the mutate.
* `exclude` scoped to cdi-sa — sound identity, worse consequences: any userInfo
match forces `background: false`, which would drop background scanning and
policy-report coverage of R-NET-05/R-IDP-02 for ALL EXISTING PODS to fix one pod
class. It also leaves the importer with no labels at all — the unlabelled
fall-through that require-egress-class-label's own header calls a fail-open leak.
* name-prefix / CDI-label matching — attacker-choosable, and proven walkable.
TWO-WAY PROOF, non-vacuous by construction — same resource, different requester,
opposite outcomes (verified by me, not just reported):
userinfo cdi-sa -> mutate pass, require-egress-class pass, require-auth-mode pass
userinfo ordinary -> mutate SKIP, require-egress-class FAIL, require-auth-mode FAIL
Kyverno L0 went 17 -> 29 tests passed, 0 failed. The mutate policy is deliberately
kept OUT of the existing tests/kyverno-test.yaml: that suite declares no userinfo,
and without userinfo the offline engine does not discriminate on subjects — it
mutates everything, which would have silently turned its negative rows into passes.
platform-config/kustomization.yaml is the live apply path for these ClusterPolicies;
without it the policy never reaches a cluster.
RESIDUAL TRADE-OFF, reversible by deleting one file: anything created by cdi-sa in
an app namespace now has these labels supplied rather than being rejected for not
declaring them — including a pod that is not a CDI worker. That identity already
holds `pods create/delete` cluster-wide via ClusterRole `cdi`, so it is not a new
capability; what changes is that for that one identity the declaration is assumed
rather than demanded. restrict-host-namespaces, the namespace default-deny baseline
and the CiliumNetworkPolicies all still apply.
FIRST resource-mutating policy on this cluster: kyverno-resource-mutating-webhook-cfg
holds 0 webhooks today; with autoUpdateWebhooks=true Kyverno registers
mutate.kyverno.svc-fail for Pods within seconds of this applying. Prod's
forceFailurePolicyIgnore=false makes it failurePolicy: Fail — same posture as the
validating webhook already in that path.
CDI retries importer creation on a loop, so the stuck import should proceed once
this lands. The detached prime-be3f9d1c-… 40 GiB scratch PVC is separate operator
cleanup; untouched, as is the DataVolume.
VERIFIED: full CI gate PASS. Nothing applied to any cluster.
Refs R-IDP-02, R-NET-05, R-PLAT-01.
Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
jakman85andClaude Opus 5 committed
1 parent
834928c237
commit
b7d6ba3d5f
10 files changed
+377
-2
No files matched your search
@@ -22,14 +22,19 @@ platform/kyverno/
|
||||
│ ├── disallow-plaintext-secrets.yaml # R-SEC-01 (CI-only: sops block required)
|
||||
│ ├── restrict-host-namespaces.yaml # R-SEC-02 (+R-NET-01 bypass guard)
|
||||
│ ├── require-egress-class-label.yaml # R-NET-05
|
||||
│ └── require-auth-mode-label.yaml # R-IDP-02
|
||||
│ ├── require-auth-mode-label.yaml # R-IDP-02
|
||||
│ └── label-cdi-worker-pods.yaml # R-NET-05+R-IDP-02 (mutate: CDI's own worker pods)
|
||||
├── servicedefinition/ # ServiceDefinition shape validation (kyverno-json)
|
||||
│ ├── validate-servicedefinition.yaml # R-GITOPS-02
|
||||
│ ├── payloads/ (valid-* / invalid-*) # pass & fail fixtures
|
||||
│ └── README.md
|
||||
├── tests/ # `kyverno test` suite — runnable L0 tests NOW, no cluster
|
||||
│ ├── kyverno-test.yaml # policies × fixtures × expected results
|
||||
│ └── resources/ (good-* / bad-* fixtures + generated golden)
|
||||
│ ├── resources/ (good-* / bad-* fixtures + generated golden)
|
||||
│ └── controller-worker-pods/ # the CDI-worker mutation, proven BOTH ways
|
||||
│ ├── kyverno-test.yaml # 2 Tests: cdi-sa requester vs ordinary requester
|
||||
│ ├── userinfo-cdi-sa.yaml / userinfo-ordinary.yaml
|
||||
│ └── resources/ + patched/ # real + FORGED importer pods; expected patch
|
||||
├── admission/ # in-cluster enforcing bundle (scaffolding, T-2.7)
|
||||
│ ├── namespace.yaml · helmrelease.yaml · kustomization.yaml
|
||||
│ ├── secret-provenance.yaml # R-SEC-01 admission variant (provenance)
|
||||
@@ -47,6 +52,7 @@ platform/kyverno/
|
||||
| `restrict-host-namespaces` | `disallow-host-namespaces`, `disallow-privileged-containers` | **R-SEC-02** (+R-NET-01) | docs/07 least reach | App pods can't use hostNetwork/PID/IPC or privileged (would bypass default-deny) |
|
||||
| `require-egress-class-label` | `require-egress-class` | **R-NET-05** | docs/03, docs/13 | App pods must declare `egress-class` ∈ {internet,vpn} |
|
||||
| `require-auth-mode-label` | `require-auth-mode` | **R-IDP-02** | docs/13, docs/15 | App pods must declare `auth-mode` ∈ {oidc,forward-auth,native,none} |
|
||||
| `label-cdi-worker-pods` (mutate) | `add-worker-pod-labels` | **R-NET-05**, **R-IDP-02** | docs/03, docs/13 | CDI's **controller-generated** worker pods (`importer-*`/`uploader-*`/`cloner-*`, created by SA `cdi/cdi-sa`) get `egress-class: internet` + `auth-mode: none` **added**, so they satisfy the two policies above instead of being exempted from them. Matched on the **generating ServiceAccount**, never a pod-name prefix |
|
||||
| `validate-servicedefinition` (kyverno-json) | shape / dependencies / storage | **R-GITOPS-02** | docs/13 (+03,15) | ServiceDefinition is schema/invariant valid before render; public⇒not `none` auth |
|
||||
| `admission/secret-provenance` | `secrets-must-be-flux-managed` | **R-SEC-01** | docs/07 | **Admission-only** variant: app-ns Secrets must come from Flux/SOPS (see below) |
|
||||
|
||||
@@ -90,6 +96,19 @@ self-contained — no cluster, no external state (reproducibility contract, R-TE
|
||||
|
||||
- **Label keys are a proposed contract** (`labels.md`) shared with WS-NET / WS-SEC /
|
||||
WS-GITOPS. Values are spec-fixed; keys need owner sign-off (coordination-protocol §2).
|
||||
- **Controller-generated pods are a cold-start hazard.** A policy that matches *any* Pod in an
|
||||
app namespace also matches pods that no manifest in git authors. Prod hit this on 2026-09-06:
|
||||
`require-auth-mode-label` + `require-egress-class-label` denied CDI's own `importer-prime-*`
|
||||
pod, so `DataVolume unifi/unifi-rootdisk` stuck in `ImportScheduled` and the UniFi VM had no
|
||||
disk. Fix: `policies/label-cdi-worker-pods.yaml` **mutates** those pods to carry truthful
|
||||
labels (matched on the generating ServiceAccount `cdi/cdi-sa`), rather than excluding them.
|
||||
`virt-launcher-*` pods need nothing — they inherit the VM pod-template labels. KubeVirt
|
||||
hotplug (`hp-volume-*`) and export (`virt-export-*`) pods do **not** inherit them and are
|
||||
unused here; extend the same policy if either is adopted.
|
||||
- **`PolicyException` is NOT usable in this install**: the Kyverno admission/background/reports
|
||||
controllers all run `--enablePolicyException=false` (chart default, verified on prod
|
||||
2026-09-06). The CRD exists (from the pre-applied CRD bundle) so a `PolicyException` would
|
||||
APPLY cleanly and be silently ignored — a policy hole that looks like a fix and isn't.
|
||||
- **Secrets CI vs admission divergence** is deliberate and documented in
|
||||
`admission/README.md` and `policies/disallow-plaintext-secrets.yaml`.
|
||||
- The ServiceDefinition JSON Schema (WS-GITOPS T-0.3) is the complementary structural
|
||||
|
||||
@@ -13,6 +13,10 @@ resources:
|
||||
- ../policies/restrict-host-namespaces.yaml
|
||||
- ../policies/require-egress-class-label.yaml
|
||||
- ../policies/require-auth-mode-label.yaml
|
||||
# mutate: gives CDI's OWN generated worker pods (importer-*/uploader-*/cloner-*) the two
|
||||
# labels the policies above require -- they are controller-generated, so nothing in git can
|
||||
# label them. Ships with them, never separately.
|
||||
- ../policies/label-cdi-worker-pods.yaml
|
||||
# --- secrets: the admission variant REPLACES the CI-only sops-block policy ---
|
||||
# (../policies/disallow-plaintext-secrets.yaml is CI-only; see secret-provenance.yaml)
|
||||
- secret-provenance.yaml
|
||||
@@ -15,6 +15,17 @@ Shared contract with WS-NET (egress rendering), WS-SEC (auth), and WS-GITOPS (re
|
||||
| `homelab.keithjackson.dev/baseline` | NetworkPolicy | `default-deny` | (convention) identifies the baseline NP | `require-default-deny-baseline` (generate), `validate-default-deny-shape` |
|
||||
| `homelab.keithjackson.dev/secret-managed-by` | Secret | `controller` | (convention) exempts controller-generated Secrets | `disallow-plaintext-secrets` (exclude), admission `secret-provenance` |
|
||||
|
||||
## Controller-generated pods (who sets the labels when git cannot)
|
||||
|
||||
`egress-class` / `auth-mode` are normally projected onto pod templates by the renderer (below).
|
||||
Some pods are generated by a **controller at runtime** and therefore have no manifest in git to
|
||||
label — CDI's `importer-*` / `uploader-*` / `cloner-*` worker pods are the live example: both
|
||||
label policies denied CDI's importer on prod (2026-09-06) and the UniFi VM got no disk. For those,
|
||||
`policies/label-cdi-worker-pods.yaml` **adds** the labels at admission (`+(…)` add-if-absent),
|
||||
matched on the **generating ServiceAccount** (`cdi/cdi-sa`) — never on a pod-name prefix or on
|
||||
CDI's own labels, both of which any pod author in the namespace can copy. `virt-launcher-*` needs
|
||||
nothing: KubeVirt propagates the VM pod-template labels onto it (verified on dev).
|
||||
|
||||
## Rendering expectation (for WS-GITOPS)
|
||||
|
||||
The ServiceDefinition (`docs/13`) declares intent — `egress:` and `auth.mode:` — at the
|
||||
|
||||
@@ -0,0 +1,136 @@
|
||||
# Serves: R-NET-05 (egress is a declared property) + R-IDP-02 (auth mode is declared) —
|
||||
# this policy does NOT weaken either requirement. It makes the one class of pod that
|
||||
# CANNOT carry the labels from git SATISFY them, instead of being exempted from them.
|
||||
#
|
||||
# THE DEFECT IT FIXES (observed on LIVE prod, 2026-09-05/06):
|
||||
# `DataVolume unifi/unifi-rootdisk` sat in ImportScheduled for hours holding a detached
|
||||
# 40Gi `prime-…` scratch PVC. `kubectl -n cdi logs deploy/cdi-deployment` showed:
|
||||
# admission webhook "validate.kyverno.svc-fail" denied the request: resource
|
||||
# Pod/unifi/importer-prime-<uid> was blocked due to the following policies
|
||||
# require-auth-mode-label / require-egress-class-label
|
||||
# CDI generates its own short-lived worker pods (importer-*, uploader-*, cloner-*/
|
||||
# clone-source-*) from inside the cdi-deployment controller. Those pods are not in git,
|
||||
# so nothing can put the platform labels on them, and both label policies match ANY Pod
|
||||
# in a namespace labelled homelab.keithjackson.dev/app-namespace=true. Result: the VM
|
||||
# never gets a disk. This is a COLD-START defect, not a prod quirk — dev's identical
|
||||
# import Succeeded before the policies were enforcing, so dev has never exercised it and
|
||||
# would hit it on any rebuild.
|
||||
#
|
||||
# WHY MUTATE (and not an exception/exclusion):
|
||||
# The honest reading of R-NET-05/R-IDP-02 is "every pod's egress path and auth posture is
|
||||
# DECLARED". A CDI importer pod has truthful values for both:
|
||||
# egress-class: internet — it pulls the image from the public internet over https
|
||||
# (cloud-images.ubuntu.com / quay.io) and must take the
|
||||
# default WAN path, which is exactly what Cilium's egress
|
||||
# gateway does for `internet` (ADR-0013/ADR-0015). It is NOT
|
||||
# a `vpn` workload.
|
||||
# auth-mode: none — the pod serves no UI and terminates when the import ends;
|
||||
# there is nothing to put an IdP in front of. docs/13 permits
|
||||
# `none` for internal, non-public workloads (the
|
||||
# ServiceDefinition policy forbids only public + none).
|
||||
# So the requirement is SATISFIED rather than bypassed: the invariant "every pod in an app
|
||||
# namespace carries both labels" stays literally true cluster-wide, both label policies
|
||||
# stay untouched (no `exclude`, no PolicyException, background scanning still on), and the
|
||||
# labels are functional — the pod is now selectable by the Cilium egress fabric instead of
|
||||
# being an unlabelled pod that silently falls through to the default path.
|
||||
#
|
||||
# THE MATCH IS THE GENERATING ServiceAccount, NOT A NAME PREFIX:
|
||||
# `subjects: ServiceAccount cdi/cdi-sa` is the identity in the AdmissionReview's userInfo —
|
||||
# the controller that issued the CREATE. Verified on prod from the Kyverno admission log of
|
||||
# the blocked request:
|
||||
# user={"username":"system:serviceaccount:cdi:cdi-sa", ...} roles=["cdi:cdi-deployment"]
|
||||
# name=importer-prime-be3f9d1c-… namespace=unifi operation=CREATE
|
||||
# A pod NAME (`importer-*`) or CDI's own labels (`cdi.kubevirt.io`) are attacker-CHOOSABLE:
|
||||
# anyone who can create a pod in the namespace can name it `importer-prime-x` and copy the
|
||||
# labels, and would then inherit the exemption. The requesting ServiceAccount cannot be
|
||||
# chosen by the pod author — only by someone who already holds cdi-sa's token, which already
|
||||
# confers "create any pod in any namespace" cluster-wide. So this adds no reachable
|
||||
# privilege. It is also VERSION-ROBUST: it covers every CDI worker kind (importer, uploader,
|
||||
# cloner/clone-source) without enumerating names that change between CDI releases —
|
||||
# confirmed on prod that cdi-sa is the ONLY cdi-namespace ServiceAccount that can create
|
||||
# pods (`kubectl auth can-i create pods -n unifi --as …`: cdi-sa yes; cdi-apiserver no;
|
||||
# cdi-uploadproxy no).
|
||||
#
|
||||
# DELIBERATELY NOT COVERED — KubeVirt's own generated pods:
|
||||
# virt-launcher-* pods INHERIT the VM's pod-template labels, so a VM authored in this repo
|
||||
# already passes (verified on dev: virt-launcher-unifi-22dg7 carries egress-class=internet,
|
||||
# auth-mode=native from apps/vms/unifi/30-virtualmachine.yaml). Exempting
|
||||
# kubevirt-controller would REMOVE that useful enforcement on VM authors, so it is not done.
|
||||
# KubeVirt's hotplug (`hp-volume-*`) and export (`virt-export-*`) pods do NOT inherit those
|
||||
# labels, but nothing in this repo uses volume hotplug or VirtualMachineExport; if either is
|
||||
# adopted, extend this policy then — with the same ServiceAccount-scoped shape.
|
||||
#
|
||||
# OPERATOR NOTE — this is the cluster's FIRST resource-MUTATING policy. Verified on prod
|
||||
# 2026-09-06 that `kyverno-resource-mutating-webhook-cfg` currently holds 0 webhooks; with the
|
||||
# chart's `autoUpdateWebhooks=true` Kyverno registers `mutate.kyverno.svc-fail` for Pods within
|
||||
# seconds of this policy being applied. Prod runs `forceFailurePolicyIgnore=false`, so that
|
||||
# webhook is failurePolicy: Fail — the same posture as the validating webhook that is ALREADY in
|
||||
# the path for these pods, so no new fail-open surface and no new blocking surface. CDI retries
|
||||
# the importer pod on a loop, so the stuck import proceeds once the policy lands (the detached
|
||||
# `prime-…` scratch PVC left behind by the blocked attempts is a separate operator cleanup).
|
||||
#
|
||||
# RESIDUAL TRADE-OFF (state it plainly, it is reversible by deleting this file):
|
||||
# Anything created by `cdi:cdi-sa` in an app namespace now gets `egress-class: internet` +
|
||||
# `auth-mode: none` supplied for it instead of being rejected for not declaring them. Whoever
|
||||
# holds cdi-sa's token can therefore run a pod in an app namespace without declaring its egress
|
||||
# class. That identity can already create arbitrary pods in every namespace (ClusterRole `cdi`:
|
||||
# pods create/delete cluster-wide), so it is not a new capability — but the declaration is now
|
||||
# assumed rather than demanded for that one identity. Nothing else changes: the pod is still
|
||||
# subject to restrict-host-namespaces, the namespace default-deny baseline, and every other
|
||||
# policy, and `internet` is the LEAST-reach of the two egress classes (it does not grant the VPN
|
||||
# path).
|
||||
#
|
||||
# PROOF (offline, no cluster — platform/kyverno/tests/controller-worker-pods/):
|
||||
# (a) a CDI importer pod created by cdi:cdi-sa is mutated and then PASSES both policies;
|
||||
# (b) the SAME namespace's ordinary pod — including one that FORGES the importer name and
|
||||
# CDI's labels — is NOT mutated and is still DENIED by both policies.
|
||||
apiVersion: kyverno.io/v1
|
||||
kind: ClusterPolicy
|
||||
metadata:
|
||||
name: label-cdi-worker-pods
|
||||
annotations:
|
||||
policies.kyverno.io/title: Label CDI-generated worker pods
|
||||
policies.kyverno.io/category: Network Isolation
|
||||
policies.kyverno.io/severity: medium
|
||||
homelab.keithjackson.dev/requirements: R-NET-05,R-IDP-02
|
||||
policies.kyverno.io/description: >-
|
||||
Adds homelab.keithjackson.dev/egress-class=internet and auth-mode=none to the
|
||||
short-lived worker Pods that the CDI controller (ServiceAccount cdi/cdi-sa)
|
||||
generates in application namespaces, so they satisfy require-egress-class-label
|
||||
and require-auth-mode-label instead of being exempted from them.
|
||||
spec:
|
||||
# No validationFailureAction: this is a MUTATE policy, there is nothing to fail.
|
||||
# background MUST be false: `match.subjects` is userInfo, which exists only on an
|
||||
# AdmissionReview. Kyverno rejects a userInfo match with background scanning enabled
|
||||
# ("invalid variable used at path ... policy contains 'userInfo'"), and a background scan
|
||||
# of an already-created pod has no requester to match anyway.
|
||||
background: false
|
||||
rules:
|
||||
- name: add-worker-pod-labels
|
||||
match:
|
||||
# One filter entry: the resource conditions AND the subject must both match.
|
||||
all:
|
||||
- resources:
|
||||
kinds:
|
||||
- Pod
|
||||
operations:
|
||||
- CREATE
|
||||
namespaceSelector:
|
||||
matchLabels:
|
||||
homelab.keithjackson.dev/app-namespace: "true"
|
||||
subjects:
|
||||
# The CDI controller (cdi-deployment) ServiceAccount — the identity that
|
||||
# CREATEs importer-*/uploader-*/cloner-* pods. Not the pods' own SA.
|
||||
- kind: ServiceAccount
|
||||
namespace: cdi
|
||||
name: cdi-sa
|
||||
mutate:
|
||||
patchStrategicMerge:
|
||||
metadata:
|
||||
labels:
|
||||
# `+(key)` is Kyverno's ADD-IF-ABSENT anchor: an explicit value already on the
|
||||
# pod is never overwritten (so a future CDI worker that legitimately needs
|
||||
# egress-class: vpn can be given it, and virt-launcher-style inherited labels
|
||||
# would survive).
|
||||
+(homelab.keithjackson.dev/egress-class): internet
|
||||
+(homelab.keithjackson.dev/auth-mode): none
|
||||
@@ -0,0 +1,91 @@
|
||||
# L0 policy-as-code suite for the CDI worker-pod label mutation — runs in CI WITHOUT a cluster
|
||||
# (discovered recursively by the gate's `kyverno test platform/kyverno`; ci/l0.sh).
|
||||
#
|
||||
# WHY A SEPARATE SUITE from ../kyverno-test.yaml: a `kyverno test` manifest carries ONE
|
||||
# `userinfo`, and the whole point here is that the SAME pods get different outcomes for
|
||||
# different requesters. Two Test documents, two identities, one file:
|
||||
# 1. cdi-controller-worker-pods-admitted (requester = cdi/cdi-sa) -> mutated, ADMITTED
|
||||
# 2. cdi-worker-exemption-is-narrow (requester = ordinary user) -> not mutated, DENIED
|
||||
# Keeping the mutate policy OUT of ../kyverno-test.yaml is deliberate: that suite declares no
|
||||
# userinfo, and with no userinfo the offline engine does not discriminate on `subjects`, which
|
||||
# would silently turn its negative (result: fail) rows into passes.
|
||||
apiVersion: cli.kyverno.io/v1alpha1
|
||||
kind: Test
|
||||
metadata:
|
||||
name: cdi-controller-worker-pods-admitted
|
||||
# Namespace labels for offline namespaceSelector resolution (shared with the main suite).
|
||||
variables: ../values.yaml
|
||||
userinfo: userinfo-cdi-sa.yaml
|
||||
policies:
|
||||
- ../../policies/label-cdi-worker-pods.yaml
|
||||
- ../../policies/require-egress-class-label.yaml
|
||||
- ../../policies/require-auth-mode-label.yaml
|
||||
resources:
|
||||
- ../resources/namespaces.yaml
|
||||
- resources/cdi-worker-pods.yaml
|
||||
results:
|
||||
# ---- the mutation fires for the CDI controller and adds exactly the two labels ----
|
||||
- policy: label-cdi-worker-pods
|
||||
rule: add-worker-pod-labels
|
||||
kind: Pod
|
||||
resources:
|
||||
- importer-prime-be3f9d1c-c3ce-455a-a450-3d9e95c7baa2
|
||||
patchedResources: patched/importer-mutated.yaml
|
||||
result: pass
|
||||
# ---- R-NET-05 + R-IDP-02: the mutated CDI worker pod now SATISFIES both policies, so the
|
||||
# import CDI could not start is admitted. (The engine validates the mutated resource.)
|
||||
- policy: require-egress-class-label
|
||||
rule: require-egress-class
|
||||
kind: Pod
|
||||
resources:
|
||||
- importer-prime-be3f9d1c-c3ce-455a-a450-3d9e95c7baa2
|
||||
result: pass
|
||||
- policy: require-auth-mode-label
|
||||
rule: require-auth-mode
|
||||
kind: Pod
|
||||
resources:
|
||||
- importer-prime-be3f9d1c-c3ce-455a-a450-3d9e95c7baa2
|
||||
result: pass
|
||||
---
|
||||
apiVersion: cli.kyverno.io/v1alpha1
|
||||
kind: Test
|
||||
metadata:
|
||||
name: cdi-worker-exemption-is-narrow
|
||||
variables: ../values.yaml
|
||||
userinfo: userinfo-ordinary.yaml
|
||||
policies:
|
||||
- ../../policies/label-cdi-worker-pods.yaml
|
||||
- ../../policies/require-egress-class-label.yaml
|
||||
- ../../policies/require-auth-mode-label.yaml
|
||||
resources:
|
||||
- ../resources/namespaces.yaml
|
||||
- resources/cdi-worker-pods.yaml
|
||||
results:
|
||||
# ---- the mutation does NOT fire for an ordinary requester, not even for a pod that forges
|
||||
# the importer's name and CDI's labels ----
|
||||
- policy: label-cdi-worker-pods
|
||||
rule: add-worker-pod-labels
|
||||
kind: Pod
|
||||
resources:
|
||||
- importer-prime-be3f9d1c-c3ce-455a-a450-3d9e95c7baa2
|
||||
- importer-prime-forged-by-an-app
|
||||
- plain-app-pod-no-labels
|
||||
result: skip
|
||||
# ---- so every unlabelled pod in the SAME app namespace is still DENIED (R-NET-05) ----
|
||||
- policy: require-egress-class-label
|
||||
rule: require-egress-class
|
||||
kind: Pod
|
||||
resources:
|
||||
- importer-prime-be3f9d1c-c3ce-455a-a450-3d9e95c7baa2
|
||||
- importer-prime-forged-by-an-app
|
||||
- plain-app-pod-no-labels
|
||||
result: fail
|
||||
# ---- ... and by R-IDP-02 ----
|
||||
- policy: require-auth-mode-label
|
||||
rule: require-auth-mode
|
||||
kind: Pod
|
||||
resources:
|
||||
- importer-prime-be3f9d1c-c3ce-455a-a450-3d9e95c7baa2
|
||||
- importer-prime-forged-by-an-app
|
||||
- plain-app-pod-no-labels
|
||||
result: fail
|
||||
@@ -0,0 +1,16 @@
|
||||
# Expected result of the mutation: CDI's own labels untouched, the two platform labels ADDED.
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: importer-prime-be3f9d1c-c3ce-455a-a450-3d9e95c7baa2
|
||||
namespace: app-ns
|
||||
labels:
|
||||
app: containerized-data-importer
|
||||
cdi.kubevirt.io: importer
|
||||
homelab.keithjackson.dev/egress-class: internet
|
||||
homelab.keithjackson.dev/auth-mode: none
|
||||
spec:
|
||||
restartPolicy: OnFailure
|
||||
containers:
|
||||
- name: importer
|
||||
image: quay.io/kubevirt/cdi-importer:v1.62.0
|
||||
@@ -0,0 +1,53 @@
|
||||
# Fixtures for the CDI-worker-pod label mutation (policies/label-cdi-worker-pods.yaml).
|
||||
#
|
||||
# All three pods live in `app-ns` (../resources/namespaces.yaml — carries
|
||||
# homelab.keithjackson.dev/app-namespace: "true"), so all three are IN SCOPE of
|
||||
# require-egress-class-label + require-auth-mode-label. What separates them is WHO creates
|
||||
# them, which is the whole point of the mechanism: the two "forged" pods are byte-for-byte
|
||||
# as CDI-looking as the real one.
|
||||
#
|
||||
# ---- (a) the REAL CDI worker pod: shape copied from the pod CDI actually tried to create on
|
||||
# prod (name importer-prime-<pvc-uid>, CDI's own labels, no platform labels). It is
|
||||
# created by ServiceAccount cdi/cdi-sa — see userinfo-cdi-sa.yaml.
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: importer-prime-be3f9d1c-c3ce-455a-a450-3d9e95c7baa2
|
||||
namespace: app-ns
|
||||
labels:
|
||||
app: containerized-data-importer
|
||||
cdi.kubevirt.io: importer
|
||||
# NOTE: no homelab.keithjackson.dev/* labels — CDI generates this pod itself, nothing in
|
||||
# git can label it. This is exactly why the import was blocked.
|
||||
spec:
|
||||
restartPolicy: OnFailure
|
||||
containers:
|
||||
- name: importer
|
||||
image: quay.io/kubevirt/cdi-importer:v1.62.0
|
||||
---
|
||||
# ---- (b) FORGED: an ordinary pod that copies the importer's NAME PREFIX and CDI's labels.
|
||||
# Proves that a name-prefix or label-based exemption would have been walkable, and
|
||||
# that this ServiceAccount-scoped one is not.
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: importer-prime-forged-by-an-app
|
||||
namespace: app-ns
|
||||
labels:
|
||||
app: containerized-data-importer
|
||||
cdi.kubevirt.io: importer
|
||||
spec:
|
||||
containers:
|
||||
- name: app
|
||||
image: registry.example/app:1.0.0
|
||||
---
|
||||
# ---- (c) a plain unlabelled application pod in the SAME namespace. Must still be DENIED.
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: plain-app-pod-no-labels
|
||||
namespace: app-ns
|
||||
spec:
|
||||
containers:
|
||||
- name: app
|
||||
image: registry.example/app:1.0.0
|
||||
@@ -0,0 +1,21 @@
|
||||
# The requester identity Kyverno actually saw on prod when CDI created the importer pod
|
||||
# (from the kyverno-admission-controller log of the blocked request, 2026-09-06):
|
||||
# user={"username":"system:serviceaccount:cdi:cdi-sa","groups":["system:serviceaccounts",
|
||||
# "system:serviceaccounts:cdi","system:authenticated"], ...}
|
||||
# roles=["cdi:cdi-deployment"] clusterroles=["cdi", …]
|
||||
# `kyverno test` feeds this to the engine as the AdmissionReview userInfo so
|
||||
# `match.subjects` can be evaluated offline.
|
||||
apiVersion: cli.kyverno.io/v1alpha1
|
||||
kind: UserInfo
|
||||
metadata:
|
||||
name: cdi-controller
|
||||
clusterRoles:
|
||||
- cdi
|
||||
roles:
|
||||
- cdi:cdi-deployment
|
||||
userInfo:
|
||||
username: system:serviceaccount:cdi:cdi-sa
|
||||
groups:
|
||||
- system:serviceaccounts
|
||||
- system:serviceaccounts:cdi
|
||||
- system:authenticated
|
||||
@@ -0,0 +1,16 @@
|
||||
# An ORDINARY requester in the application namespace — the attacker/mistake case. Anything
|
||||
# that is not cdi/cdi-sa: an app's own ServiceAccount, a human's kubectl, Flux.
|
||||
# The mutation must NOT fire for this identity, so both label policies must still DENY —
|
||||
# even for a pod that forged the importer's name and CDI's labels.
|
||||
apiVersion: cli.kyverno.io/v1alpha1
|
||||
kind: UserInfo
|
||||
metadata:
|
||||
name: ordinary-namespace-user
|
||||
clusterRoles:
|
||||
- edit
|
||||
userInfo:
|
||||
username: system:serviceaccount:app-ns:default
|
||||
groups:
|
||||
- system:serviceaccounts
|
||||
- system:serviceaccounts:app-ns
|
||||
- system:authenticated
|
||||
Reference in new issue
Block a user