feat(pm-agent): the kubelet's pull credential, and the seventh field admission had not read
gate / gate (pull_request) Failing after 1m53s
gate / conformance (pull_request) Has been skipped

The registry had two blockers. #59 fixed the first (nodes resolved the hostname to the
non-hairpinned WAN and failed TLS against the router's CN=%D cert). This is the second:
the pull is unauthenticated, and this registry refuses unauthenticated clients outright —
measured 2026-10-05, its anonymous token carries access: null and every path 401s.

pm-agent-registry is a dockerconfigjson Secret (SOPS, placeholders) referenced by
imagePullSecrets on the dispatcher Deployment and on the committed worker Job template. It
holds the SAME Gitea token as pm-agent-credentials, because the two consumers need two
formats: the application reads a raw token from env, the kubelet reads a dockerconfigjson
before any container starts. Same one-value-in-two-files pattern as the Keycloak client
secret, documented in both .example files. The operator's part is the value plus one new
scope: read:package.

ADMISSION GAINED A SEVENTH RULE. imagePullSecrets is a pod-spec field no container
references, which is exactly why it is easy to leave unread — the same shape of omission
as initContainers, ephemeralContainers, env and envFrom before it. The kubelet reads the
named Secret, in this namespace, before anything runs. Rule 1's pattern now REQUIRES the
committed entry; rule 7 forbids any other and any second one. Both halves are needed: a
foreach over an absent list yields no result.

The leverage is small and the policy says so: the image is digest-pinned, so a different
pull credential cannot change WHAT runs. Pinned anyway — 'only the committed shape' is the
premise, and the rule costs four lines.

kyverno test: 33 passed, 0 failed (was 28). Three new fixtures, each a real path: the
committed entry plus a second one, a different Secret, and none at all.

The gate also caught me shipping the Secret in plaintext — I never ran sops -e -i on the
placeholder. Recorded in the handoff entry, because the file looked finished.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
jakman85andClaude Opus 5 committed 2026-10-06 01:03:54 -05:00
1 parent 523f482957
commit 7fa5e84d25
13 files changed
+413 -21

No files matched your search

@@ -0,0 +1,88 @@
---
date: 2026-10-06
branch: feat/pm-agent-registry-pull-secret
title: The kubelet's pull credential — the second half of making agent-runner pullable, and the seventh field the admission policy had not read
---
## What changed
The registry had two blockers. The first — the nodes resolving the registry hostname to the
non-hairpinned WAN and failing TLS against the router's `CN=%D` certificate — was diagnosed
and fixed by an `/etc/hosts` pin, merged in #59 and applied by the operator. This is the
second: **the pull is unauthenticated, and this registry refuses unauthenticated clients
outright.**
`pm-agent-registry` is a new `kubernetes.io/dockerconfigjson` Secret, SOPS-encrypted with
placeholder values, referenced by `imagePullSecrets` on both the dispatcher Deployment and
the committed worker Job template. It is a **second Secret holding the same Gitea token** as
`pm-agent-credentials`, because the two consumers need two formats: the application reads a
raw token from env, the kubelet reads a dockerconfigjson before any container starts. That
duplication is the established pattern here (the Keycloak client secret is mirrored the same
way, for the same reason) and it is documented in both `.example` files.
The operator's part is the value, and one scope: the Gitea token must now carry
**`read:package`** alongside `write:repository` and `write:issue`. Steps are in
`platform/pm-agent/pm-agent-registry.sops.yaml.example`, including the trap that
`.dockerconfigjson`'s registry key must be the bare host as the image reference spells it —
with a scheme or a port it matches nothing, the kubelet pulls anonymously, and the failure is
indistinguishable from the Secret being absent.
**The admission policy gained a seventh rule.** `imagePullSecrets` is a pod-spec field that
no container references, which is exactly why it is easy to leave unread — the same shape of
omission as `initContainers`, `ephemeralContainers`, `env` and `envFrom` before it. The
kubelet reads the named Secret, in this namespace, before anything runs. Rule 1's pattern now
REQUIRES the committed entry and rule 7 forbids any other or any second one; both halves are
needed because a `foreach` over an absent list yields no result.
The leverage is small and worth saying so: the image is digest-pinned, so a different pull
credential cannot change *what* runs, and the only Secrets in this namespace are the agent's
own. It is pinned anyway, because "only the committed shape" is this policy's whole premise
and the rule costs four lines.
## What I verified
- **`kyverno test platform/kyverno/tests/pm-agent` → 33 passed, 0 failed** under the pinned
CLI (was 28 before this change). Three new fixtures, each a real path rather than a style
violation: `extra-pull-secret` (the committed entry PLUS a second one — rule 1's array
pattern is satisfied by the first element, which is why the count is asserted separately),
`wrong-pull-secret` (a different Secret in this namespace), and `no-pull-secret` (absent
entirely, refused by rule 1's pattern because rule 7's foreach cannot see it).
- **`PM_SELFTEST=1 pm-dispatch` → exit 0**, now also asserting that the committed Job
template names exactly `pm-agent-registry` — so the template and the policy cannot drift
apart without the image build failing.
- **`./ci/gate.sh` in the pinned toolbox, from a §0 worktree → exit 3 (UNDETERMINED)**, with
`gitea-settings` the only stage that did not run (no `SETTINGS_API_TOKEN` on this
workstation). Every other stage green, `spec-immutability` included. rc 3 is not a pass.
- **The gate caught me shipping the Secret in plaintext.** I wrote the placeholder
`pm-agent-registry.sops.yaml` and never ran `sops -e -i` on it. `no-plaintext-secrets`
refused it with both halves named — "has no top-level `sops:` block with a mac" and
"Secret value '.dockerconfigjson' is not SOPS ciphertext" — which is exactly the check
earning its place: the file looked finished, and the value was a placeholder, so nothing
about reading the diff would have made it obvious. Encrypted, and the stage is green.
- **The registry really does refuse anonymous clients**, measured before building this
(2026-10-05): the anonymous registry token's `access` claim is `null`, every `/v2/…`
path answers 401 whether it exists or not, and `/api/v1/packages/Home-Network` replies
`{"message":"token is required"}`. That is why this is a credential and not a posture
tweak.
## What I did NOT verify
- **That the pull now works.** Nothing has been pushed to that registry and the committed
digest is still the all-zero placeholder, so neither half of the registry fix has been
exercised end to end. The first real pull is what proves both at once — and if it fails, the
error names which half: `x509 … not valid for any names` is the resolution pin, `401` /
`unauthorized` is this credential.
- **That the `/etc/hosts` pin from #59 took effect.** The operator applied it; I could not
confirm the effect, because the dispatcher pod's retained events are all repeating
`BackOff` with no fresh `Pulling`, so the `x509` message still visible is the one from
before the play ran. Not evidence either way.
- **Whether `read:package` is sufficient** for a pull from a Gitea package of this type. It is
the documented read scope for packages; nothing here has exercised it, because no package
exists yet.
- **The dockerconfigjson placeholder is not a valid credential and is not meant to be.** Its
`auth` field is a `REPLACE-ME` string rather than base64, so a pull with it will fail at
authentication rather than be mistaken for working.
- **Anything about runtime.** Unchanged from the previous entries: the image has never been
built or pushed, no dispatcher has authenticated to Gitea, no issue has been triaged, no
worker Job has ever been created, and the admission policy — live, `Enforce`, now seven
rules — has still never admitted or refused a real Job.