Home-Network/home-network-k3s-impl
2
0 Fork 0
Code Issues 2 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
Files
main
home-network-k3s-impl/decisions/2026-10-30-pm-agent-live.md
T
jakman85andClaude Opus 5.5 7f0cbc7ba2
gate / gate (pull_request) Failing after 2m56s
Details
images / images (pull_request) Successful in 32s
Details
gate / conformance (pull_request) Has been skipped
Details
feat(pm-agent): pin the first published agent-runner digest, start dry, and fix the token scopes 
Rebased #61 onto the CI image pipeline (#81). This commit makes the remaining in-git
blockers to going live go away:

- deployment.yaml and PM_WORKER_IMAGE pin images run 340's digest (sha256:c4eba5b3...,
  tag sha-523cd9118f95 = main), read from the registry. Renovate markers unchanged.
- PM_DRY_RUN=true and PM_REQUIRE_LABEL=agent for the first run. Dry run is required, not
  just cautious: the run-340 image's baked worker template predates imagePullSecrets, so
  admission would deny its Jobs until CI re-publishes after this merge and Renovate bumps.
- Token scopes: Gitea 1.24.3 scopes GET /api/v1/user (the dispatcher's first call) under
  read:user. The documented list was missing it. Now read:user, write:repository,
  write:issue, read:package everywhere.
- Org-package read documented as a team (Packages=Read, no repos), not just a scope.
- sops is toolbox-only; the .examples say so and give the in-place edit.
- runbook: key-name check no longer emits Secret values before the pipe.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 01:16:21 -05:00

7.1 KiB
Raw Permalink Blame History

date, branch, title
date branch title
2026-10-30 feat/pm-agent-registry-pull-secret PM agent —

What changed

PR #61 (feat/pm-agent-registry-pull-secret, the kubelet pull credential and admission rule 7) was rebased onto origin/main after #81 (CI image pipeline), #82 and #83. Conflicts in docs/plans/pm-agent-issue-to-pr.md, platform/pm-agent/README.md and runbooks/pm-agent.md were resolved keeping both sides: #61's "pull credential now wired" and #81's "CI publishes, Renovate bumps". No manual build-and-push text came back.

On top of the rebase, as a second commit:

  • The first published digest is pinned: deployment.yaml and PM_WORKER_IMAGE in configmap.yaml now name gitea.home.keithjackson.dev/home-network/agent-runner@sha256:c4eba5b3c508bf456a03391ac76f8275e7ea28a829da1450d99f6f6f25c6eb42 (images run 340, tag sha-523cd9118f95, which is also main). The # renovate-digest: markers are unchanged and still sit on the line directly above each value, so later bumps stay Renovate's. Pinning by hand once removes the dependency on Renovate's first-ever run.
  • The first run is dry, and scoped to a label: PM_DRY_RUN: "true", PM_REQUIRE_LABEL: "agent" (the plan's follow-up 3); config-generation bumped to 2. Dry run is also a correctness requirement, not only caution: the pinned digest was built before this PR added imagePullSecrets to the worker Job template, which is baked into the image. Admission now requires that entry, so a worker Job built from the run-340 image would be denied. Merging this PR touches infra/agent/**, so images.yaml re-publishes agent-runner; Renovate's bump to that digest is what makes workers admissible. Only then flip PM_DRY_RUN to false.
  • The token scope list was wrong; it is fixed. Gitea 1.24.3's routers/api/v1/api.go closes the /user route group with tokenRequiresScopes(AccessTokenScopeCategoryUser), and the dispatcher's first call is GET /user. The documented scopes are now read:user, write:repository, write:issue, read:package everywhere (both .examples, README, runbook row).
  • Org-package permission is documented separately from the scope. Per services/context/package.go determineAccessMode (v1.24.3), a non-admin's access to an org-owned package is the highest Packages-unit mode across its org teams, else read if the org is visible to it. Home-Network is public, so it works today by visibility alone; the registry .example now has the operator create a pm-agent-pull team (no repositories, Packages = Read) so it does not depend on that.
  • sops is toolbox-only: both .examples now say so and give the in-place ./infra/devcontainer/dev.sh run sops <file> edit (the same procedure runbooks/vpn-remote-access.md documents), which writes no plaintext file.
  • The runbook's Secret-key check used -o jsonpath='{.data}', which prints values before the pipe trims them; it is now a go-template that emits key names only. Two new runbook rows: the read:user 403, and the admission denial from a pre-#61 digest.

What I verified

  • Registry digest, read-only from the registry API (bearer token from /v2/token, HEAD /v2/home-network/agent-runner/manifests/<tag> with OCI and Docker accept headers): main and sha-523cd9118f95 both answer 200, Docker-Content-Digest: sha256:c4eba5b3…6eb42, application/vnd.oci.image.manifest.v1+json. Tags list: [main, sha-523cd9118f95]. The admission pattern gitea.home.keithjackson.dev/home-network/agent-runner@sha256:* matches the pinned value.
  • #81's claim that the template's digest needs no tracking holds: pm-dispatch.build_job sets containers[0].image = cfg.worker_image (from PM_WORKER_IMAGE) on every Job. Renovate's renovate-digest custom manager only matches platform/pm-agent/*.yaml and versions.env, so the template is not touched by it.
  • Node path to the registry, read-only over SSH, all three prod nodes: /etc/hosts line 9 is 192.168.2.20 keycloak.home.keithjackson.dev gitea.home.keithjackson.dev, nsswitch is hosts: files dns, and curl https://gitea.home.keithjackson.dev/v2/ from the node answers 401 with ssl_verify_result=0 and remote_ip=192.168.2.20. That is the host resolver, which is what containerd (a node process) uses; a pod's view would not have shown it.
  • Network path from the namespace: dispatcher and worker both use PM_GITEA_URL=http://gitea.gitea.svc.cluster.local:3000 (ConfigMap, Job template, and pm-worker's hard allowlist, which refuses anything else); the git remote and the API base are both derived from it. So nothing goes through the Gateway and #83's toEndpoints-through-Gateway issue does not apply. Egress: toEndpoints gitea pods :3000 (both CNPs), kube-apiserver (dispatcher only), world:443 (the model). Ingress on the Gitea side: gitea-allow-pm-agent-ingress exists live and is VALID True.
  • Live state (read-only, --context k3s-prod): namespace pm-agent exists; Deployment 0/1, pod ImagePullBackOff for 24d on the all-zero digest; Secret pm-agent-credentials (2 keys) exists, pm-agent-registry does not yet (it ships in this PR); Flux Kustomization pm-agent Ready at main@dfc432c; ClusterPolicy pm-agent-job-shape Ready.
  • Gitea state (read-only API): 1.24.3; org Home-Network public; no user pm-agent exists; teams are Owners and registry-push; collaborators are jakman85 (owner) and renovate-bot (write); the repository has no labels and no open issues.
  • Placeholders, without decrypting: the registry ciphertext decodes to 155 bytes, exactly the length of the .example's placeholder JSON. The credentials values decode to 57 and 39 bytes — neither a 40-char Gitea token nor a ~100-char Claude OAuth token. Both files' lastmodified is the commit that created them as placeholders.
  • kyverno test, PM_SELFTEST=1 pm-dispatch and the gate: numbers in the PR description and the final handoff report for this run.

What I did NOT verify

  • Any authenticated pull, poll or worker. No pm-agent account or token exists, so the scope set has not been exercised end to end. The read:user requirement is read from Gitea's v1.24.3 source, not from a live 403.
  • That the credentials ciphertext is a placeholder. Lengths are strong evidence, not proof; I did not decrypt.
  • That Renovate will open the post-merge bump. Its token was being re-minted at the time of writing and it has never completed a run here. Renovate's kubernetes manager also scans infra/agent/*.yaml and platform/**; what it does with a digest-only reference with no tag (the template) is unmeasured.
  • The dispatcher's claude --print call to api.anthropic.com through world:443, and whether the run-340 image's dispatcher behaves as this tree's code does (it predates #61's selftest line, nothing else in pm-dispatch.py).
  • The admission policy against a real Job. Still never admitted or refused one live.
Reference in new issue View Git Blame Copy Permalink