jakman85andClaude Opus 5
a7eadd4054
The registry had two blockers. #59 fixed the first (nodes resolved the hostname to the non-hairpinned WAN and failed TLS against the router's CN=%D cert). This is the second: the pull is unauthenticated, and this registry refuses unauthenticated clients outright — measured 2026-10-05, its anonymous token carries access: null and every path 401s. pm-agent-registry is a dockerconfigjson Secret (SOPS, placeholders) referenced by imagePullSecrets on the dispatcher Deployment and on the committed worker Job template. It holds the SAME Gitea token as pm-agent-credentials, because the two consumers need two formats: the application reads a raw token from env, the kubelet reads a dockerconfigjson before any container starts. Same one-value-in-two-files pattern as the Keycloak client secret, documented in both .example files. The operator's part is the value plus one new scope: read:package. ADMISSION GAINED A SEVENTH RULE. imagePullSecrets is a pod-spec field no container references, which is exactly why it is easy to leave unread — the same shape of omission as initContainers, ephemeralContainers, env and envFrom before it. The kubelet reads the named Secret, in this namespace, before anything runs. Rule 1's pattern now REQUIRES the committed entry; rule 7 forbids any other and any second one. Both halves are needed: a foreach over an absent list yields no result. The leverage is small and the policy says so: the image is digest-pinned, so a different pull credential cannot change WHAT runs. Pinned anyway — 'only the committed shape' is the premise, and the rule costs four lines. kyverno test: 33 passed, 0 failed (was 28). Three new fixtures, each a real path: the committed entry plus a second one, a different Secret, and none at all. The gate also caught me shipping the Secret in plaintext — I never ran sops -e -i on the placeholder. Recorded in the handoff entry, because the file looked finished. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
419 lines
21 KiB
YAML
419 lines
21 KiB
YAML
# Serves: R-SEC-02 (least privilege for workload identities)
|
|
#
|
|
# THIS POLICY IS THE SECURITY BOUNDARY. The RBAC in platform/pm-agent/rbac.yaml is not.
|
|
#
|
|
# Kubernetes places NO `use` check on a Pod's `serviceAccountName`. Any principal that can
|
|
# `create jobs` in namespace `pm-agent` can therefore create a Job that runs as any
|
|
# ServiceAccount in that namespace, with an ARBITRARY command — so the dispatcher's Role,
|
|
# however narrow it looks, IS whatever the most privileged identity in the namespace can do,
|
|
# unless something constrains the SHAPE of what it may create. This is the same finding
|
|
# conformance/incluster/rbac.yaml records for the conformance trigger, and the model here is
|
|
# deliberately the same one.
|
|
#
|
|
# Two things make it bite harder for this service than for that one:
|
|
#
|
|
# 1. THE CREATOR IS DRIVEN BY A LANGUAGE MODEL READING UNTRUSTED TEXT. The dispatcher asks
|
|
# the model what to do about an issue thread written by whoever can reach the
|
|
# repository. The model holds no tools and the dispatcher accepts only a validated
|
|
# vocabulary (an action from a fixed set, a branch matching CLAUDE.md §4) — but those
|
|
# are both one program away from the API server. This policy is the layer that does not
|
|
# depend on that program being correct.
|
|
# 2. THE WORKER RUNS MODEL-AUTHORED CODE WITH CREDENTIALS. Each worker holds a Gitea write
|
|
# token and an Anthropic subscription token. What stops a worker Job from being pointed
|
|
# at a different command, a different image, a mounted hostPath or a projected
|
|
# ServiceAccount token is this file.
|
|
#
|
|
# ENFORCE AND FAIL-CLOSED, NOT SUBSTITUTED. Note this does NOT use
|
|
# ${KYVERNO_VALIDATION_ACTION} like the app-scoped policies: a security boundary an
|
|
# environment overlay can downgrade to Audit is not a boundary. `failurePolicy: Fail` means a
|
|
# Kyverno outage blocks worker Jobs rather than silently admitting unconstrained ones.
|
|
# Blocking an agent's work is the correct failure direction; admitting arbitrary code that
|
|
# holds a git write token is not.
|
|
#
|
|
# SCOPE: it constrains ONLY Jobs created by the `pm-agent` ServiceAccount in the `pm-agent`
|
|
# namespace. Jobs created by humans, by Flux, or by any other SA are untouched — so a broken
|
|
# rule here cannot wedge the rest of the cluster. A human debugging this service can still
|
|
# create a one-off Job by hand, which is deliberate: the boundary exists to bound the
|
|
# AUTOMATED creator, not to lock the operator out of their own namespace.
|
|
#
|
|
# IMAGE PINNING — HONEST LIMIT, read before relying on this. The rules below require the
|
|
# image to live under the agent-runner repository path AND to be digest-pinned
|
|
# (`@sha256:`), never a mutable tag. Residual surface: anyone who can PUSH to that registry
|
|
# path could publish a different digest. TIGHTEN IT once the image exists by replacing the
|
|
# wildcard with the literal digest — a one-line change, reviewed in a pull request, which is
|
|
# exactly where a change to the security boundary belongs. (Same limit, same wording, as
|
|
# conformance-trigger-job-shape.yaml: that is not copy-paste, it is the same unresolved
|
|
# tradeoff in both places.)
|
|
#
|
|
# EVERY RULE BELOW EXISTS BECAUSE OF A FIELD THAT WAS ONCE UNREAD. The conformance policy
|
|
# shipped with three rules and then needed four more — `initContainers`, then
|
|
# `ephemeralContainers`, then `env`, then `envFrom` — because pinning `command` and `args`
|
|
# only moves the injection to whatever adjacent field nobody enumerated. This policy starts
|
|
# with all of them, and `trigger`-shaped rules are named `worker-…` for the same reason.
|
|
apiVersion: kyverno.io/v1
|
|
kind: ClusterPolicy
|
|
metadata:
|
|
name: pm-agent-job-shape
|
|
annotations:
|
|
policies.kyverno.io/title: Constrain what the PM agent dispatcher may create
|
|
policies.kyverno.io/category: Supply chain
|
|
policies.kyverno.io/severity: critical
|
|
homelab.keithjackson.dev/requirements: R-SEC-02
|
|
homelab.keithjackson.dev/scope: admission-only
|
|
spec:
|
|
validationFailureAction: Enforce
|
|
failurePolicy: Fail
|
|
background: false
|
|
rules:
|
|
# =====================================================================================
|
|
# Rule 1 — the committed worker shape, as a pattern.
|
|
#
|
|
# `resources` AND `subjects` in the SAME filter entry: that is how Kyverno ANDs them. As
|
|
# two separate `all` entries it is a schema error ("match resources not specified") and
|
|
# the WHOLE policy loads as INVALID — which the CLI reports as a SKIP, i.e. a boundary
|
|
# that silently is not there. That mistake has already been made once in this repository.
|
|
# =====================================================================================
|
|
- name: worker-must-be-the-committed-shape
|
|
match:
|
|
all:
|
|
- resources:
|
|
kinds:
|
|
- Job
|
|
namespaces:
|
|
- pm-agent
|
|
subjects:
|
|
- kind: ServiceAccount
|
|
name: pm-agent
|
|
namespace: pm-agent
|
|
validate:
|
|
message: >-
|
|
The PM dispatcher may create ONLY the committed worker shape: a single container,
|
|
the digest-pinned agent-runner image, serviceAccountName pm-agent-worker with NO
|
|
mounted token, command ["/usr/local/bin/pm-worker"], no host namespaces, non-root
|
|
with privilege escalation disabled and a read-only root filesystem. See
|
|
infra/agent/worker-job-template.yaml and
|
|
platform/kyverno/admission/pm-agent-job-shape.yaml.
|
|
pattern:
|
|
spec:
|
|
template:
|
|
spec:
|
|
# The powerless identity. A Job that ran as `pm-agent` instead could create
|
|
# more Jobs — i.e. fork itself without limit — which is why this is pinned
|
|
# rather than merely defaulted.
|
|
serviceAccountName: pm-agent-worker
|
|
# A worker needs no Kubernetes API access at all, so it gets no token. This
|
|
# is the field that makes the empty `pm-agent-worker` Role moot rather than
|
|
# merely narrow.
|
|
automountServiceAccountToken: false
|
|
# A retried agent would duplicate its push, its pull request and its issue
|
|
# comment, from a tree the first attempt had already changed.
|
|
restartPolicy: Never
|
|
=(hostNetwork): "false"
|
|
=(hostPID): "false"
|
|
=(hostIPC): "false"
|
|
# The kubelet reads this Secret BEFORE any container starts, so it is part
|
|
# of the shape even though no container references it. A pattern on an array
|
|
# asserts "at least one element matches", which is enough to REQUIRE the
|
|
# committed entry; rule 7 is what forbids a second one.
|
|
imagePullSecrets:
|
|
- name: pm-agent-registry
|
|
containers:
|
|
- image: "gitea.home.keithjackson.dev/home-network/agent-runner@sha256:*"
|
|
command:
|
|
- "/usr/local/bin/pm-worker"
|
|
securityContext:
|
|
runAsNonRoot: true
|
|
allowPrivilegeEscalation: false
|
|
readOnlyRootFilesystem: true
|
|
=(privileged): "false"
|
|
|
|
# =====================================================================================
|
|
# Rule 2 — exactly one container, and no args on it.
|
|
#
|
|
# Rule 1's `containers` pattern is an ARRAY pattern, and a Kyverno array pattern asserts
|
|
# "at least one element matches". So rule 1 alone would admit the compliant worker
|
|
# container PLUS a second container with an arbitrary image and command, sharing the
|
|
# pod's network namespace and its two credential env vars. The count has to be asserted
|
|
# separately.
|
|
#
|
|
# `args` is the other half: the command is pinned, so caller-supplied code moves to the
|
|
# arguments. pm-worker refuses arguments itself (the image build asserts it), and this
|
|
# refuses them one layer earlier.
|
|
# =====================================================================================
|
|
- name: worker-must-be-one-container-with-no-args
|
|
match:
|
|
all:
|
|
- resources:
|
|
kinds:
|
|
- Job
|
|
namespaces:
|
|
- pm-agent
|
|
subjects:
|
|
- kind: ServiceAccount
|
|
name: pm-agent
|
|
namespace: pm-agent
|
|
validate:
|
|
message: >-
|
|
A worker Job must declare EXACTLY ONE container and give it NO args. A second
|
|
container shares the pod's namespaces and its credentials; args on a pinned command
|
|
are caller-supplied code.
|
|
deny:
|
|
conditions:
|
|
any:
|
|
- key: "{{ length(request.object.spec.template.spec.containers) }}"
|
|
operator: NotEquals
|
|
value: 1
|
|
- key: "{{ length(not_null(request.object.spec.template.spec.containers[0].args, `[]`)) }}"
|
|
operator: GreaterThan
|
|
value: 0
|
|
|
|
# =====================================================================================
|
|
# Rule 3 — only the `workspace` and `tmp` volumes.
|
|
#
|
|
# foreach, NOT a `pattern` list, for the reason in rule 2: an array pattern is satisfied
|
|
# by one matching element, so it would happily admit `workspace` PLUS a hostPath, a
|
|
# secret projection, or another SA's token. This asserts EVERY volume is one of the two,
|
|
# and that each is an emptyDir — a volume named `workspace` whose source is a hostPath
|
|
# would otherwise pass a name-only check.
|
|
# =====================================================================================
|
|
- name: worker-volumes-must-be-workspace-or-tmp-emptydirs
|
|
match:
|
|
all:
|
|
- resources:
|
|
kinds:
|
|
- Job
|
|
namespaces:
|
|
- pm-agent
|
|
subjects:
|
|
- kind: ServiceAccount
|
|
name: pm-agent
|
|
namespace: pm-agent
|
|
validate:
|
|
message: >-
|
|
A worker Job may mount only the `workspace` and `tmp` volumes, and both must be
|
|
emptyDir. An extra volume — or a hostPath wearing an allowed name — is how a
|
|
constrained pod reaches the host, a Secret, or another ServiceAccount's token.
|
|
foreach:
|
|
- list: "request.object.spec.template.spec.volumes"
|
|
deny:
|
|
conditions:
|
|
any:
|
|
- key: "{{ element.name }}"
|
|
operator: AnyNotIn
|
|
value: ["workspace", "tmp"]
|
|
# The SOURCE must be emptyDir, asserted over the volume's KEY SET rather
|
|
# than over emptyDir's contents: `emptyDir: {}` is a perfectly valid
|
|
# volume with no sizeLimit, and a length check on it denies the good case
|
|
# (measured — this is what row 5 of the test caught). Checking the keys
|
|
# also catches every other source type without naming them, so a future
|
|
# `csi:` or `projected:` is denied by default rather than by enumeration.
|
|
- key: "{{ keys(element) }}"
|
|
operator: AnyNotIn
|
|
value: ["name", "emptyDir"]
|
|
|
|
# =====================================================================================
|
|
# Rule 4 — no initContainers and no ephemeralContainers.
|
|
#
|
|
# These are the second and third container lists on a PodSpec, and every rule above reads
|
|
# only `containers`. On the conformance policy this exact gap was MEASURED: a Job with a
|
|
# fully compliant containers[0] and a privileged initContainer running an arbitrary
|
|
# command was ADMITTED (pass 3, fail 0) before the equivalent rule existed.
|
|
#
|
|
# Unlike the conformance run, a worker needs NO init container at all — pm-worker does its
|
|
# own clone, inside the one pinned entrypoint — so this is a flat prohibition rather than
|
|
# a second pinned shape. That is strictly simpler to reason about, and it is the whole
|
|
# reason the clone lives in the entrypoint instead of in an init step.
|
|
#
|
|
# `ephemeralContainers` is believed to be rejected by upstream Kubernetes in a Job's pod
|
|
# template (the field is settable only via the pods/ephemeralcontainers subresource).
|
|
# THAT HAS NOT BEEN VERIFIED on this cluster, and "probably rejected upstream" is not a
|
|
# boundary. The condition costs nothing and is a no-op when the field is absent.
|
|
# =====================================================================================
|
|
- name: worker-may-not-declare-init-or-ephemeral-containers
|
|
match:
|
|
all:
|
|
- resources:
|
|
kinds:
|
|
- Job
|
|
namespaces:
|
|
- pm-agent
|
|
subjects:
|
|
- kind: ServiceAccount
|
|
name: pm-agent
|
|
namespace: pm-agent
|
|
validate:
|
|
message: >-
|
|
A worker Job may not declare initContainers or ephemeralContainers. pm-worker does
|
|
its own clone inside the pinned entrypoint, so neither has a legitimate purpose
|
|
here — and either is a second container list running with the worker's credentials
|
|
under a command this policy does not read.
|
|
deny:
|
|
conditions:
|
|
any:
|
|
- key: "{{ length(not_null(request.object.spec.template.spec.initContainers, `[]`)) }}"
|
|
operator: GreaterThan
|
|
value: 0
|
|
- key: "{{ length(not_null(request.object.spec.template.spec.ephemeralContainers, `[]`)) }}"
|
|
operator: GreaterThan
|
|
value: 0
|
|
|
|
# =====================================================================================
|
|
# Rule 5 — only the committed environment variable NAMES.
|
|
#
|
|
# The pattern from the conformance policy, applied before it has to be learned again:
|
|
# pin `command` and `args`, and injection moves to the environment. It is not
|
|
# hypothetical here either —
|
|
# * pm-worker keys its whole behaviour on this environment: PM_BRANCH decides which ref
|
|
# is created and pushed, PM_GITEA_URL decides which host is cloned from and pushed to
|
|
# (it validates that one against a baked-in allowlist, which is the other half of
|
|
# this defence), PM_TASK_MODE decides whether an existing branch is reused;
|
|
# * `claude` reads ANTHROPIC_API_KEY and ANTHROPIC_AUTH_TOKEN AHEAD of the subscription
|
|
# token, so one injected variable redirects every model call to a different
|
|
# credential and a different endpoint (pm-worker unsets them, which is defence in
|
|
# depth, not a reason to omit this);
|
|
# * and bash itself reads BASH_ENV, which it sources before a non-interactive script —
|
|
# so `command: ["/usr/local/bin/pm-worker"]` plus one env var would be arbitrary code
|
|
# execution holding a git write token.
|
|
#
|
|
# NOT CONSTRAINED, deliberately: the VALUES. They are legitimately per-assignment (a
|
|
# branch, a brief, an issue number), and pm-worker validates every one it acts on. Names
|
|
# are the vector that matters, because a name is what a program keys its behaviour on.
|
|
# The `valueFrom` SOURCES are likewise unconstrained here and constrained in the one place
|
|
# that can: the committed Job template holds them and pm-dispatch refuses to overwrite an
|
|
# entry that has a `valueFrom`.
|
|
#
|
|
# This list must stay equal to WORKER_ENV_ALLOWED in infra/agent/pm-dispatch.py. The
|
|
# duplication is deliberate — the policy must decide without trusting that program — and
|
|
# platform/kyverno/tests/pm-agent/ pins them together with the real template as a fixture.
|
|
# =====================================================================================
|
|
- name: worker-env-names-must-be-the-committed-set
|
|
match:
|
|
all:
|
|
- resources:
|
|
kinds:
|
|
- Job
|
|
namespaces:
|
|
- pm-agent
|
|
subjects:
|
|
- kind: ServiceAccount
|
|
name: pm-agent
|
|
namespace: pm-agent
|
|
validate:
|
|
message: >-
|
|
A worker Job may set only the committed environment variables (PM_TASK_MODE,
|
|
PM_ISSUE_NUMBER, PM_ISSUE_TITLE, PM_BRANCH, PM_BASE_BRANCH, PM_BRIEF, PM_GITEA_URL,
|
|
PM_GITEA_OWNER, PM_GITEA_REPO, PM_GITEA_USER, PM_MODEL, PM_MAX_TURNS,
|
|
PM_MAX_BUDGET_USD, PM_PUSH_ON_RED_GATE, GITEA_TOKEN, CLAUDE_CODE_OAUTH_TOKEN, HOME,
|
|
TMPDIR). An arbitrary env var changes what the pinned command does — BASH_ENV alone
|
|
is arbitrary code execution, and ANTHROPIC_API_KEY silently outranks the
|
|
subscription token. Widening this list is a reviewed one-line change, in two files.
|
|
foreach:
|
|
- list: "request.object.spec.template.spec.containers[0].env"
|
|
deny:
|
|
conditions:
|
|
all:
|
|
- key: "{{ element.name }}"
|
|
operator: AnyNotIn
|
|
value:
|
|
- PM_TASK_MODE
|
|
- PM_ISSUE_NUMBER
|
|
- PM_ISSUE_TITLE
|
|
- PM_BRANCH
|
|
- PM_BASE_BRANCH
|
|
- PM_BRIEF
|
|
- PM_GITEA_URL
|
|
- PM_GITEA_OWNER
|
|
- PM_GITEA_REPO
|
|
- PM_GITEA_USER
|
|
- PM_MODEL
|
|
- PM_MAX_TURNS
|
|
- PM_MAX_BUDGET_USD
|
|
- PM_PUSH_ON_RED_GATE
|
|
- GITEA_TOKEN
|
|
- CLAUDE_CODE_OAUTH_TOKEN
|
|
- HOME
|
|
- TMPDIR
|
|
|
|
# =====================================================================================
|
|
# Rule 6 — no envFrom.
|
|
#
|
|
# It injects an entire ConfigMap or Secret as environment variables whose KEYS are not
|
|
# visible in the Job at all, which would make rule 5 unenforceable by inspection. The
|
|
# dispatcher itself uses envFrom (it is a Deployment, not a Job, and not matched by this
|
|
# policy); a worker must not.
|
|
# =====================================================================================
|
|
# =====================================================================================
|
|
# Rule 7 — exactly one imagePullSecret, and it is the committed one.
|
|
#
|
|
# `imagePullSecrets` is a pod-spec field that no container references, which is precisely
|
|
# why it is easy to leave unread — the same shape of omission as `initContainers` and
|
|
# `env` before it. The kubelet reads the named Secret, in this namespace, before any
|
|
# container starts. A Job naming a different one would have the kubelet read a Secret the
|
|
# worker was never meant to touch.
|
|
#
|
|
# The leverage is genuinely small: the image is digest-pinned, so a different credential
|
|
# cannot change WHAT runs, and the only Secrets in this namespace are the agent's own.
|
|
# It is pinned anyway, because "only the committed shape" is the whole premise of this
|
|
# policy, and because the cost of the rule is four lines while the cost of rediscovering
|
|
# the omission is an incident.
|
|
#
|
|
# Rule 1's pattern REQUIRES the committed entry; this forbids any OTHER entry and any
|
|
# second entry. A foreach over an absent list yields no result, which is why both halves
|
|
# are needed.
|
|
# =====================================================================================
|
|
- name: worker-pull-secret-must-be-the-committed-one
|
|
match:
|
|
all:
|
|
- resources:
|
|
kinds:
|
|
- Job
|
|
namespaces:
|
|
- pm-agent
|
|
subjects:
|
|
- kind: ServiceAccount
|
|
name: pm-agent
|
|
namespace: pm-agent
|
|
validate:
|
|
message: >-
|
|
A worker Job may declare exactly ONE imagePullSecret, named `pm-agent-registry`.
|
|
The kubelet reads that Secret before any container starts, so naming another one
|
|
reaches a credential the worker was never meant to touch.
|
|
foreach:
|
|
- list: "request.object.spec.template.spec.imagePullSecrets"
|
|
deny:
|
|
conditions:
|
|
any:
|
|
- key: "{{ length(request.object.spec.template.spec.imagePullSecrets) }}"
|
|
operator: NotEquals
|
|
value: 1
|
|
- key: "{{ element.name }}"
|
|
operator: AnyNotIn
|
|
value: ["pm-agent-registry"]
|
|
|
|
- name: worker-may-not-use-envfrom
|
|
match:
|
|
all:
|
|
- resources:
|
|
kinds:
|
|
- Job
|
|
namespaces:
|
|
- pm-agent
|
|
subjects:
|
|
- kind: ServiceAccount
|
|
name: pm-agent
|
|
namespace: pm-agent
|
|
validate:
|
|
message: >-
|
|
A worker Job may not use envFrom. It injects a whole ConfigMap or Secret as
|
|
environment variables whose keys the Job does not name, which would make the
|
|
env-name allowlist unenforceable by inspection.
|
|
deny:
|
|
conditions:
|
|
any:
|
|
- key: "{{ length(not_null(request.object.spec.template.spec.containers[0].envFrom, `[]`)) }}"
|
|
operator: GreaterThan
|
|
value: 0
|
|
- key: "{{ length(not_null(request.object.spec.template.spec.initContainers[].envFrom[], `[]`)) }}"
|
|
operator: GreaterThan
|
|
value: 0
|