home-network-k3s-impl/runbooks/plex-claim.md

plex-claim — claiming the Plex Media Server inside the token window

Alert: none. This is an operator procedure, not an incident page — it is indexed here because it is the page you need at the moment you need it, with a stopwatch running. Files: catalog/plex.yaml (env PLEX_CLAIM), apps/wave-0/plex-claim-secret.sops.yaml (the SOPS Secret plex-claim, key PLEX_CLAIM), apps/wave-0/plex.yaml (the rendered golden).

What this means here

Plex will not let you create libraries over the mounted bulk media share while the server is unclaimed — the 8.33 T NFS tree at /media is there, readable, and unusable. Claiming binds the server to a plex.tv account. It is a one-shot operation: the token is exchanged once for a durable PlexOnlineToken written into Preferences.xml on the plex-config Longhorn PVC, and from then on the PLEX_CLAIM env is ignored on every start (upstream gate: [ ! -z "${PLEX_CLAIM}" ] && [ -z "${token}" ] in root/etc/cont-init.d/40-plex-first-run).

A https://plex.tv/claim token is short-lived — minutes, not hours. Read the countdown the claim page shows you; do not budget against a number written here. That is shorter than PR → gate → merge → Flux's 10-minute reconcile interval → pod roll if you let those run on their own timers, which is why this page exists: the loop is scripted and the reconcile is forced rather than waited for. Measured end to end on 2026-09-29: ~2m45s.

An expired or wrong token does not break anything. Verified against the image's own first-run script (and the copy inside plexinc/pms-docker:latest): a failed exchange gets a 403 with an empty body, curl's status is never checked, the token sed yields empty, the if does not fire, there is no else and no exit 1, and the script (no set -e) exits 0. The pod comes up unclaimed and serving. So a burnt token costs you one more attempt, not an outage — and the repo deliberately ships a placeholder in that Secret for exactly this reason (the env secretKeyRef has no optional, so the Secret must always exist even when its value is useless).

Running the commands in this runbook

kubectl does not work on the operator workstation. The toolchain lives only in the pinned dev container, and the host's k3s-oidc context hangs on a browser prompt once its token expires — that hang is not the cluster being down. Every command below runs as:

infra/devcontainer/dev.sh run bash -c 'kubectl --context k3s-prod get nodes'

Always pass --context k3s-prod explicitly. dev.sh merges BOTH clusters into one kubeconfig and picks a current-context per invocation, and that default has already changed once (k3s-dev until dev was decommissioned on 2026-09-13, k3s-prod now). Do not run two dev.sh invocations at once.

Mutating kubectl and flux reconcile are blocked for agents here. This whole page is operator work — the reconcile and the pod roll below are deliberate operator actions, and the durable change is still a git change.

First commands

Before spending a token, find out whether the server is already claimed. This prints only the attribute NAME, never its value — do not widen it to print the token:

infra/devcontainer/dev.sh run bash -c \
  'kubectl --context k3s-prod -n plex exec deploy/plex -- sh -c \
     "grep -o PlexOnlineToken= \"/config/Library/Application Support/Plex Media Server/Preferences.xml\" | head -1"'

PlexOnlineToken= printed ⇒ already claimed; stop, there is nothing to do and no token to burn. Empty output ⇒ unclaimed, continue.

Is the workload healthy enough to claim at all, and is the wave reconciling?

infra/devcontainer/dev.sh run bash -c \
  'kubectl --context k3s-prod -n plex get deploy,pod,pvc'
infra/devcontainer/dev.sh run bash -c \
  'kubectl --context k3s-prod -n flux-system get kustomization apps-wave-0'

The procedure

STATUS: this server was CLAIMED on 2026-09-29 and PlexOnlineToken is present in /config on the plex-config PVC. The upstream exchange is gated on that pref being absent, so PLEX_CLAIM is now a no-op on every start whatever it holds. You only need this page again after a /config wipe or a rebuild onto an empty PVC.

On the clock. The token is short-lived — read the countdown that plex.tv/claim shows you rather than trusting a number written here. (Earlier revisions of this page asserted a "~4-minute TTL" in four places with no citation; upstream pms-docker documents no expiry at all. The page is the only authority, and it is on screen at T+0.)

A burnt token costs nothing. An expired token gets a 403 with an empty body, curl's status is never checked, the if does not fire, there is no else and no exit 1, and the script (no set -e) exits 0. The pod comes up unclaimed and serving. So retry freely.

One command

1. T+0 — get the token. Sign in to plex.tv as the account that should OWN this server, open https://plex.tv/claim, copy the claim-… string.

2. Put it in the Secret — in place, encrypted, never touching a plaintext file:

   infra/devcontainer/dev.sh run sops apps/wave-0/plex-claim-secret.sops.yaml
   

3. Ship it. This drives the whole delivery loop and prints a running stopwatch:

   ./infra/plex/claim-ship.sh
   

   branch → commit → push → PR → gate → merge → flux reconcile --with-source → rollout restart → verdict. It refuses to run if the file is unchanged or not encrypted.

Measured end to end, 2026-09-29: ~2m45s. Gate ~1m40s, merge + reconcile ~25s, pod roll and container start ~30s.

Why it is a script and not six pasted commands

main is a protected branch — the direct push this page used to instruct is now rejected — so the deploy is a PR merged after a green gate. Three of those steps are easy to get wrong under time pressure, and two are silent when skipped:

• --with-source is load-bearing. Without it the Kustomization applies the previously-fetched revision, not your commit.
• The pod roll is mandatory. Only the Secret's ciphertext changes, so the Deployment's pod template is byte-identical and nothing restarts on its own. PLEX_CLAIM is read once, at container start. Skip this and the new Secret sits in the cluster while the running pod keeps the old value forever.
• The branch is named feat/… on purpose. act_runner capacity is 1, and gate.yml's push trigger covers feature/** fix/** docs/** chore/** but not feat/**. A chore/ branch fires a second redundant run that serialises ahead of the PR run and roughly doubles the wall clock.

Reading the verdict

• Attempting to obtain server token from claim token and Token obtained successfully ⇒ claimed. Re-run the First-commands PlexOnlineToken= check to confirm it persisted, then create the libraries over /media/movies, /media/tv, /media/photos.

• Attempting… with no success line ⇒ the token was expired or invalid. Nothing is broken; go back to step 1 with a new one.

• Neither line ⇒ the exchange was skipped: /config already had a PlexOnlineToken (already claimed — the expected case now), or the env did not reach the container. Check by NAME only:

  infra/devcontainer/dev.sh run bash -c \
    'kubectl --context k3s-prod -n plex get pod -l app.kubernetes.io/name=plex \
       -o jsonpath="{.items[0].spec.containers[0].env[*].name}"'
  

Likely causes, most likely first

When the claim does not take, in order of how often it is the answer here:

1. The token expired — you exceeded the countdown, usually by reviewing the PR instead of merging it the moment the gate went green. Symptom: the "Attempting…" line with no success line. Remedy: a new token, and let claim-ship.sh drive the loop rather than pasting steps.
2. The pod was never restarted (step 5 skipped). Symptom: no claim log lines at all from a pod whose AGE predates your push. The Secret is correct in git and in the cluster and the running container still holds the placeholder.
3. Flux had not applied the commit yet when you rolled the pod — step 5 run before step 4 finished, so the new pod read the OLD Secret. Check kubectl --context k3s-prod -n flux-system get kustomization apps-wave-0 -o jsonpath='{.status.lastAppliedRevision}' against your commit, then roll again.
4. Already claimed — the exchange is gated on PlexOnlineToken being absent from /config, so a server claimed earlier ignores every later token silently. This is the benign case.
5. plex.tv unreachable. The upstream curl has no --max-time/--connect-timeout, so a stalled connection can leave the container sitting in cont-init with nothing listening on 32400 — the one genuine hang risk in this path. Symptom: pod Running but never Ready, log stops after "Attempting…". Check egress (plex is egress: internet) rather than re-rolling repeatedly.
6. Wrong plex.tv account. The claim succeeds and binds the server to whoever was signed in at step 1. Unclaiming is a plex.tv-side operation on that account — see "escalate or stop".

When to escalate or stop

• Stop after two failed token attempts and fix the timing, not the token. The third rushed attempt is how people start editing live Secrets by hand, which leaves the cluster holding a value that is in no git commit — precisely the drift qbittorrent-bootstrap had to be recovered from.
• Never kubectl edit/patch the plex-claim Secret in the cluster, and never kubectl create secret it. Everything durable goes through the SOPS file; a live-only Secret is reverted (or adopted with the wrong value) on the next reconcile.
• Never sops -d this file into a terminal, a log, or a shared screen, and never widen the PlexOnlineToken= check into one that prints the value. The token is a plex.tv account credential.
• If the server got claimed to the wrong account, stop and get the operator. Reversing it means removing the server from that account on plex.tv and clearing PlexOnlineToken from Preferences.xml on the PVC — a destructive edit of app state on the replicated tier, not a git change.
• Do not delete the plex-config PVC to "start clean". It holds the library database, metadata and watch state (policy: daily-offcluster), and it is expensive to rebuild. The claim is a two-line pref, not a reason to lose the DB.
• After a successful claim, leave the expired token in the SOPS file. It is inert (the exchange is skipped once claimed) and the Secret must keep existing, because the env secretKeyRef has no optional — deleting it would put the pod into CreateContainerConfigError.

---

Source of truth: runbooks/plex-claim.md. Index: /runbooks/ Upstream behaviour cited here is from plexinc/pms-docker branch master, root/etc/cont-init.d/40-plex-first-run (s6-overlay v2), cross-checked against the same file inside the running plexinc/pms-docker:latest image.