Files
home-network-k3s-impl/decisions/2026-10-06-pm-agent-registry-pull-secret.md
T
jakman85andClaude Opus 5 a7eadd4054 feat(pm-agent): the kubelet's pull credential, and the seventh field admission had not read
The registry had two blockers. #59 fixed the first (nodes resolved the hostname to the
non-hairpinned WAN and failed TLS against the router's CN=%D cert). This is the second:
the pull is unauthenticated, and this registry refuses unauthenticated clients outright —
measured 2026-10-05, its anonymous token carries access: null and every path 401s.

pm-agent-registry is a dockerconfigjson Secret (SOPS, placeholders) referenced by
imagePullSecrets on the dispatcher Deployment and on the committed worker Job template. It
holds the SAME Gitea token as pm-agent-credentials, because the two consumers need two
formats: the application reads a raw token from env, the kubelet reads a dockerconfigjson
before any container starts. Same one-value-in-two-files pattern as the Keycloak client
secret, documented in both .example files. The operator's part is the value plus one new
scope: read:package.

ADMISSION GAINED A SEVENTH RULE. imagePullSecrets is a pod-spec field no container
references, which is exactly why it is easy to leave unread — the same shape of omission
as initContainers, ephemeralContainers, env and envFrom before it. The kubelet reads the
named Secret, in this namespace, before anything runs. Rule 1's pattern now REQUIRES the
committed entry; rule 7 forbids any other and any second one. Both halves are needed: a
foreach over an absent list yields no result.

The leverage is small and the policy says so: the image is digest-pinned, so a different
pull credential cannot change WHAT runs. Pinned anyway — 'only the committed shape' is the
premise, and the rule costs four lines.

kyverno test: 33 passed, 0 failed (was 28). Three new fixtures, each a real path: the
committed entry plus a second one, a different Secret, and none at all.

The gate also caught me shipping the Secret in plaintext — I never ran sops -e -i on the
placeholder. Recorded in the handoff entry, because the file looked finished.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-10-30 01:09:39 -05:00

5.9 KiB

date, branch, title
date branch title
2026-10-06 feat/pm-agent-registry-pull-secret The kubelet's pull credential — the second half of making agent-runner pullable, and the seventh field the admission policy had not read

What changed

The registry had two blockers. The first — the nodes resolving the registry hostname to the non-hairpinned WAN and failing TLS against the router's CN=%D certificate — was diagnosed and fixed by an /etc/hosts pin, merged in #59 and applied by the operator. This is the second: the pull is unauthenticated, and this registry refuses unauthenticated clients outright.

pm-agent-registry is a new kubernetes.io/dockerconfigjson Secret, SOPS-encrypted with placeholder values, referenced by imagePullSecrets on both the dispatcher Deployment and the committed worker Job template. It is a second Secret holding the same Gitea token as pm-agent-credentials, because the two consumers need two formats: the application reads a raw token from env, the kubelet reads a dockerconfigjson before any container starts. That duplication is the established pattern here (the Keycloak client secret is mirrored the same way, for the same reason) and it is documented in both .example files.

The operator's part is the value, and one scope: the Gitea token must now carry read:package alongside write:repository and write:issue. Steps are in platform/pm-agent/pm-agent-registry.sops.yaml.example, including the trap that .dockerconfigjson's registry key must be the bare host as the image reference spells it — with a scheme or a port it matches nothing, the kubelet pulls anonymously, and the failure is indistinguishable from the Secret being absent.

The admission policy gained a seventh rule. imagePullSecrets is a pod-spec field that no container references, which is exactly why it is easy to leave unread — the same shape of omission as initContainers, ephemeralContainers, env and envFrom before it. The kubelet reads the named Secret, in this namespace, before anything runs. Rule 1's pattern now REQUIRES the committed entry and rule 7 forbids any other or any second one; both halves are needed because a foreach over an absent list yields no result.

The leverage is small and worth saying so: the image is digest-pinned, so a different pull credential cannot change what runs, and the only Secrets in this namespace are the agent's own. It is pinned anyway, because "only the committed shape" is this policy's whole premise and the rule costs four lines.

What I verified

  • kyverno test platform/kyverno/tests/pm-agent → 33 passed, 0 failed under the pinned CLI (was 28 before this change). Three new fixtures, each a real path rather than a style violation: extra-pull-secret (the committed entry PLUS a second one — rule 1's array pattern is satisfied by the first element, which is why the count is asserted separately), wrong-pull-secret (a different Secret in this namespace), and no-pull-secret (absent entirely, refused by rule 1's pattern because rule 7's foreach cannot see it).
  • PM_SELFTEST=1 pm-dispatch → exit 0, now also asserting that the committed Job template names exactly pm-agent-registry — so the template and the policy cannot drift apart without the image build failing.
  • ./ci/gate.sh in the pinned toolbox, from a §0 worktree → exit 3 (UNDETERMINED), with gitea-settings the only stage that did not run (no SETTINGS_API_TOKEN on this workstation). Every other stage green, spec-immutability included. rc 3 is not a pass.
  • The gate caught me shipping the Secret in plaintext. I wrote the placeholder pm-agent-registry.sops.yaml and never ran sops -e -i on it. no-plaintext-secrets refused it with both halves named — "has no top-level sops: block with a mac" and "Secret value '.dockerconfigjson' is not SOPS ciphertext" — which is exactly the check earning its place: the file looked finished, and the value was a placeholder, so nothing about reading the diff would have made it obvious. Encrypted, and the stage is green.
  • The registry really does refuse anonymous clients, measured before building this (2026-10-05): the anonymous registry token's access claim is null, every /v2/… path answers 401 whether it exists or not, and /api/v1/packages/Home-Network replies {"message":"token is required"}. That is why this is a credential and not a posture tweak.

What I did NOT verify

  • That the pull now works. Nothing has been pushed to that registry and the committed digest is still the all-zero placeholder, so neither half of the registry fix has been exercised end to end. The first real pull is what proves both at once — and if it fails, the error names which half: x509 … not valid for any names is the resolution pin, 401 / unauthorized is this credential.
  • That the /etc/hosts pin from #59 took effect. The operator applied it; I could not confirm the effect, because the dispatcher pod's retained events are all repeating BackOff with no fresh Pulling, so the x509 message still visible is the one from before the play ran. Not evidence either way.
  • Whether read:package is sufficient for a pull from a Gitea package of this type. It is the documented read scope for packages; nothing here has exercised it, because no package exists yet.
  • The dockerconfigjson placeholder is not a valid credential and is not meant to be. Its auth field is a REPLACE-ME string rather than base64, so a pull with it will fail at authentication rather than be mistaken for working.
  • Anything about runtime. Unchanged from the previous entries: the image has never been built or pushed, no dispatcher has authenticated to Gitea, no issue has been triaged, no worker Job has ever been created, and the admission policy — live, Enforce, now seven rules — has still never admitted or refused a real Job.