--- date: 2026-10-05 branch: docs/pm-agent-registry-pull-prereq title: pm-agent reconciled exactly as designed — and is the first workload ever to pull from the in-cluster registry, which does not work yet --- ## What changed Documentation only. No manifest, no policy, no code. PR #57 merged and `pm-agent` reconciled. I measured what landed, and the measurement turned up **two prerequisites for the next operator step that nothing in this repository declares** — and that the handoff entry for #57 could not have known, because it listed the image pull as unverified and that is exactly where it bit. So this entry records the post-merge measurement, and `platform/pm-agent/README.md`, `docs/plans/pm-agent-issue-to-pr.md` and `runbooks/pm-agent.md` now name both prerequisites where they say "build, push, pin the digest". Before this, that step read as one action; it is three, and the two new ones are not an agent's to perform. **The finding is not specific to pm-agent.** `conformance-runner` has the identical registry path (`gitea.home.keithjackson.dev/home-network/...`), the identical "operator pushes, a digest goes in a PR" model, and has never been pulled either — so the same two blockers sit in front of the L2/L3 conformance wiring. Measured: **no pod anywhere in the cluster references that registry** except the pm-agent dispatcher, which is therefore the first workload ever to try it. ## What I verified Read-only, through the pinned toolbox, against the live prod cluster. **The merge reconciled, completely and exactly as designed.** - `GitRepository/flux-system` artifact revision: `main@sha1:95597bd6883637…` — the merge commit. `platform-config`, `pm-agent` and `apps-wave-0` all report that same `lastAppliedRevision` with `Ready=True`. Compared field to field, per CLAUDE.md §2b, rather than assumed from the merge. - **The security boundary is live.** `ClusterPolicy/pm-agent-job-shape`: `validationFailureAction=Enforce`, `failurePolicy=Fail`, `Ready=True`, and all six rules present (`worker-must-be-the-committed-shape`, `worker-must-be-one-container-with-no-args`, `worker-volumes-must-be-workspace-or-tmp-emptydirs`, `worker-may-not-declare-init-or-ephemeral-containers`, `worker-env-names-must-be-the-committed-set`, `worker-may-not-use-envfrom`). This is the check that mattered most: a ClusterPolicy added to the wrong kustomization list ships green and never reaches the cluster, which is what happened to the conformance boundary on 2026-09-29. - Namespace contents: both ServiceAccounts, `pm-agent-config` (17 keys), the `pm-agent-credentials` Secret, the `default-deny` NetworkPolicy, and both CiliumNetworkPolicies with `VALID=True`. - **SOPS decryption works through the new overlay patch.** Checked by KEY NAMES and VALUE LENGTHS only, never values (CLAUDE.md §6): keys `gitea-token`, `claude-code-oauth-token`, lengths 76 and 52 bytes — which are the base64 lengths of the committed placeholder strings, so the Secret holds the decrypted placeholders and not ciphertext. A ciphertext apply succeeds silently and would have left `ENC[AES256_GCM,…]` values an order of magnitude longer. - **`pm-agent-worker` really is bound to nothing.** The namespace has exactly one RoleBinding (`pm-agent` → Role `pm-agent`, subject `ServiceAccount/pm-agent`), and no ClusterRoleBinding anywhere names either pm-agent ServiceAccount. - `gitea-allow-pm-agent-ingress` is applied in the `gitea` namespace, matching `io.kubernetes.pod.namespace: pm-agent`. - The dispatcher is `ImagePullBackOff` on the all-zero placeholder digest — the designed, visible failure. **The two prerequisites, measured.** 1. **The nodes cannot verify the registry's TLS certificate.** containerd's error on the pull is not about the digest: failed to resolve reference "gitea.home.keithjackson.dev/home-network/agent-runner@sha256:000…": tls: failed to verify certificate: x509: certificate is not valid for any names, but wanted to match gitea.home.keithjackson.dev "not valid for **any** names" means the certificate presented to the node carries no matching SAN at all. From the operator's workstation the same name is healthy: it resolves to `192.168.2.20` and serves a Let's Encrypt certificate `CN=*.home.keithjackson.dev` with SANs `*.home.keithjackson.dev, home.keithjackson.dev` — and `Certificate gateway/wildcard-home` is `Ready=True` for exactly those names. So the certificate that *should* be served is fine, and what the NODE reaches is not what the workstation reaches. **Correcting the digest will not clear this.** 2. **Anonymous pulls from this registry are refused.** Not inferred from a bare 401 — that would have been wrong, because this Gitea answers 401 for *everything* anonymously: `/v2/home-network/agent-runner/manifests/latest`, `/v2/home-network/conformance-runner/…`, a randomly-named nonexistent repository, and `nosuchowner/nosuchimage` all return 401, and `/api/v1/packages/Home-Network` returns `{"message":"token is required"}`. The registry *does* issue a token to an anonymous client, but its `access` claim is `null` — no pull scope. So the conclusion that holds is the simple one: **an unauthenticated client gets nothing from this registry**, so a pull with no credential cannot succeed. And `grep` over the tree finds **no `imagePullSecrets` anywhere** — not in `platform/pm-agent/deployment.yaml`, not in `infra/agent/worker-job-template.yaml`, not in `conformance/incluster/`. ## What I did NOT verify - **Why the node sees a nameless certificate.** I measured that it does, and that the LAN does not. I did not diagnose it: I did not read the nodes' resolver or `/etc/rancher/k3s/registries.yaml` (node files, not reachable read-only through the API), and I did not check whether the node resolves the name to the same `192.168.2.20` the workstation does or to something else. Note `cluster-vars` has `GATEWAY_VIP: 192.168.1.53` while the nodes are `192.168.2.10-12` and the name resolves to `192.168.2.20` — three different subnets in one sentence is a reason to measure rather than to theorise, and I have not. - **Whether the package would be pullable once authenticated.** Nothing has ever been pushed to that path, so "a credential is sufficient" is untested. The only thing established is that no credential is insufficient. - **Which fix the operator wants.** Two routes, and they are not equivalent: (a) an `imagePullSecret` — a `dockerconfigjson` holding a Gitea token, SOPS-encrypted like `pm-agent-credentials`, referenced from the Deployment *and* from the committed worker Job template; note the Job template's shape is pinned by admission, and `imagePullSecrets` is a field that policy does not currently constrain, so adding one is a template change that needs a deliberate look at whether the policy should pin it too. (b) allowing anonymous reads of that package in Gitea — no manifest change at all, but it makes the image world-readable to anything that can reach the host. I have implemented neither: both need an operator action I must not take, and (b) is a posture decision. The docs now present both. - **Everything the #57 entry listed as unverified about RUNTIME remains unverified.** The image has still never been built or pushed. No dispatcher has authenticated to Gitea, no issue has been triaged, no worker Job has been created — so the admission policy, though now provably applied and Enforce, has still never refused or admitted a real Job. The network policies have not carried traffic. - **I did not re-run the merge gate against `main` after the merge.** This branch's own gate run is the evidence for this change; the merged state's is CI run on PR #57.