Home-Network/home-network-k3s-impl
2
0 Fork 0
Code Issues 2 Pull Requests 5 Actions Packages Projects Releases Wiki Activity
Files
005e80d38a7082b549a459d540d76e2cf936e4fa
home-network-k3s-impl/apps/wave-0/plex.yaml
T
jakman85andClaude Opus 5 874c6703f0
gate / gate (pull_request) Successful in 1m50s
Details
gate / conformance (pull_request) Has been skipped
Details
fix(plex): advertise a resolver-independent plex.direct name again (LAN-wide outage) 
I took Plex down for the whole LAN. ADVERTISE_IP was
http://plex-vip.${BASE_DOMAIN}:32400 -- a name that exists ONLY in PiHole's
split-horizon answers and in no public zone.

A Plex client resolves the advertised host ITSELF, with whatever resolver it
uses, and on a real LAN that is frequently not PiHole: browsers ship DoH on by
default, phones have "private DNS", guest/IoT devices take the router's DNS.
Those clients could not resolve the name, the router answered the failed lookup
with ITSELF, and they all landed on its web UI -- hence the %D certificate.

Measured 2026-10-05:
  192.168.2.1:443      -> C=ZZ, O=%D342e7e6b, CN=%D   (router, LAN side)
  136.51.115.252:443   -> the same %D cert            (router, WAN side)
  pihole-0 AND pihole-1 -> plex -> .20, plex-vip -> .24   (both CORRECT)
  gateway .20:443      -> CN=*.home.keithjackson.dev, valid
  http://192.168.2.24:32400/identity -> 200
  Plex's published connections: all port 32400, nothing on 443

The DNS was right; the DEPENDENCY on it was the bug. And since nothing Plex
advertises listens on 443, the clients had left the Plex path entirely.

Fix: <dashed-ip>.<serverhash>.plex.direct, which is RESOLVER-INDEPENDENT -- a
public zone returning the private address embedded in the label, so every
resolver (PiHole, the router, 1.1.1.1, a DoH browser) answers it with
192.168.2.24. No split-horizon entry, no Cloudflare record and no certificate
work needed: the wildcard cert Plex already serves on 32400 covers that name.
Re-verified today with full validation (no curl -k): 200, TLS verified.

The nicer hostname cannot be made correct for clients that bypass PiHole -- a
public record resolves to the WAN address and there is no NAT hairpin, so it is
right only for PiHole users and wrong for the rest. There is no configuration of
it that is right for both. PiHole's plex-vip answer is KEPT; it is just not what
Plex advertises.

Cost restated, not buried: the value embeds the VIP and the cert hash as a
literal and nothing checks they still agree. Accepted because an unresolvable
name is a worse failure -- silent, LAN-wide, and it points users at the router.

UNVERIFIED: unreconciled, and the client-side resolver path is INFERRED -- my
public-resolver probe was broken (NXDOMAIN even for a name that resolves) so I
discarded it. The chain is the only explanation consistent with the
measurements, but the NXDOMAIN-hijack step was not observed directly.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-10-05 20:58:55 -05:00

343 lines
13 KiB
YAML
Raw Blame History

---
# Namespace (trust zone; default-deny + policies apply)
apiVersion: v1
kind: Namespace
metadata:
name: plex
labels:
homelab.keithjackson.dev/app-namespace: 'true'
gateway-access: shared
annotations:
homelab.keithjackson.dev/backup-policy: daily-offcluster
---
# Default-deny baseline (R-NET-01): deny all, allow only DNS
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny
namespace: plex
labels:
homelab.keithjackson.dev/baseline: default-deny
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
ingress: []
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
ports:
- protocol: UDP
port: 53
- protocol: TCP
port: 53
---
# PersistentVolume plex-media - static external NFS (bulk tier)
apiVersion: v1
kind: PersistentVolume
metadata:
name: plex-media
labels:
app.kubernetes.io/name: plex
spec:
capacity:
storage: 9Ti
accessModes:
- ReadOnlyMany
persistentVolumeReclaimPolicy: Retain
storageClassName: ''
mountOptions:
- nfsvers=4.1
- hard
- noatime
- timeo=600
- retrans=2
nfs:
server: ${BULK_NFS_SERVER}
path: /mnt/Storage/plex
readOnly: true
claimRef:
namespace: plex
name: plex-media
---
# PVC plex-config (R-STATE)
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: plex-config
namespace: plex
labels:
app.kubernetes.io/name: plex
spec:
accessModes:
- ReadWriteOnce
storageClassName: longhorn
resources:
requests:
storage: 8Gi
---
# PVC plex-media (R-STATE)
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: plex-media
namespace: plex
labels:
app.kubernetes.io/name: plex
spec:
accessModes:
- ReadOnlyMany
storageClassName: ''
volumeName: plex-media
resources:
requests:
storage: 9Ti
---
# Deployment (R-PLAT-02)
apiVersion: apps/v1
kind: Deployment
metadata:
name: plex
namespace: plex
labels:
app.kubernetes.io/name: plex
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app.kubernetes.io/name: plex
template:
metadata:
labels:
app.kubernetes.io/name: plex
homelab.keithjackson.dev/egress-class: internet
homelab.keithjackson.dev/auth-mode: native
spec:
containers:
- name: plex
image: plexinc/pms-docker:latest
ports:
- containerPort: 32400
env:
- name: PLEX_UID
value: '1000'
- name: PLEX_GID
value: '1000'
- name: TZ
value: Etc/UTC
- name: ALLOWED_NETWORKS
value: 10.0.0.0/8,192.168.0.0/16
- name: PLEX_CLAIM
valueFrom:
secretKeyRef:
name: plex-claim
key: PLEX_CLAIM
- name: ADVERTISE_IP
value: https://192-168-2-24.130072f20c334d6c8f63d0dfef87309b.plex.direct:32400
- name: PLEX_TLS_CRT
valueFrom:
secretKeyRef:
name: plex-vip-tls
key: tls.crt
optional: true
- name: PLEX_TLS_KEY
valueFrom:
secretKeyRef:
name: plex-vip-tls
key: tls.key
optional: true
resources:
limits:
gpu.intel.com/i915: 1
volumeMounts:
- name: config
mountPath: /config
- name: media
mountPath: /media
readOnly: true
lifecycle:
postStart:
exec:
command:
- /bin/bash
- -c
- "# NOTE \u2014 LOAD-BEARING: SHELL VARIABLES HERE ARE WRITTEN BARE, NEVER BRACE-WRAPPED.\n# This whole script is rewritten by Flux postBuild envsubst BEFORE it ever runs, so a\n# brace-wrapped name is read as a cluster-vars knob: an undeclared one is a HARD\n# Kustomization failure under StrictPostBuildSubstitutions, and a declared one would be\n# replaced with its value. Shell vars are therefore `$PX`, `$TOKEN`, `$P12` \u2014 bare. Only\n# BASE_DOMAIN below is brace-wrapped, because it IS a knob and IS meant to be\n# substituted. Command substitution `$(cmd)` is left alone and needs no escaping. A shell\n# variable written with braces would be blanked or would fail the whole Kustomization \u2014\n# this repo has already lost a gitea OIDC bootstrap to exactly that mistake, and\n# catalog/render.py now refuses to emit it (which is how this very comment got caught).\n# DELIBERATELY NO `set -u`. The TLS env vars are declared `optional: true`, so when the\n# certificate has not issued yet they are genuinely UNSET, not empty \u2014 and under `set -u`\n# the first `[ -n \"$PLEX_TLS_CRT\" ]` would abort the script, which for a postStart hook\n# means kubelet KILLS THE CONTAINER. That is the precise failure the `optional` flag exists\n# to prevent, so turning it back on here would defeat the whole design. (The alternative,\n# a brace-wrapped `:-` default, would need envsubst escaping \u2014 the trap the note above\n# describes.) Every value below is explicitly guarded instead.\nPX=\"/config/Library/Application Support/Plex Media Server/Preferences.xml\"\nCERTDIR=/config/certs\nP12=$CERTDIR/plex-vip.p12\n# The PKCS#12 passphrase protects nothing and is deliberately not a secret: Plex needs it\n# in cleartext in customCertificateKey, and it and the bundle sit on the same PVC. It is a\n# container-format requirement, not an access control.\nP12PASS=plexvip\nLANNETS='10.0.0.0/8,192.168.0.0/16'\nCERTDOMAIN=plex-vip.${BASE_DOMAIN}\nAPI=http://127.0.0.1:32400\n\n# (1) Build the PKCS#12 bundle from the PEM the Secret supplies. Rebuilt on every start so\n# a cert-manager RENEWAL is picked up with no extra moving part.\nif [ -n \"$PLEX_TLS_CRT\" ] && [ -n \"$PLEX_TLS_KEY\" ]; then\n mkdir -p $CERTDIR && chmod 700 $CERTDIR\n printf '%s' \"$PLEX_TLS_CRT\" > $CERTDIR/tls.crt\n printf '%s' \"$PLEX_TLS_KEY\" > $CERTDIR/tls.key\n chmod 600 $CERTDIR/tls.key\n if openssl pkcs12 -export -out $P12.tmp -inkey $CERTDIR/tls.key \\\n -in $CERTDIR/tls.crt -passout pass:$P12PASS 2>/dev/null; then\n mv -f $P12.tmp $P12\n chown 1000:1000 $P12 2>/dev/null || true\n chmod 600 $P12\n echo \"[plex-settings] PKCS#12 bundle written\"\n else\n rm -f $P12.tmp\n echo \"[plex-settings] WARN: pkcs12 export failed; keeping any previous bundle\" >&2\n fi\n rm -f $CERTDIR/tls.key\nelse\n echo \"[plex-settings] WARN: TLS env empty; certificate not installed this start\" >&2\nfi\n\n# (2) Wait, bounded (~90s), for the local API. /identity needs no auth.\ni=0\nwhile [ \"$i\" -lt 45 ]; do\n if [ \"$(curl -s -o /dev/null -w '%{http_code}' --max-time 5 $API/identity 2>/dev/null)\" = \"200\" ]; then\n break\n fi\n i=$(expr $i + 1); sleep 2\ndone\n\n# (3) The prefs API needs the server token. It already lives in Preferences.xml on this\n# PVC; it is never echoed here, and Plex redacts tokens in its own request log. An\n# UNCLAIMED server has none \u2014 a normal state on a fresh /config \u2014 so skip, do not fail.\nTOKEN=$(grep -o 'PlexOnlineToken=\"[^\"]*\"' \"$PX\" 2>/dev/null | head -1 | cut -d'\"' -f2)\nif [ -z \"$TOKEN\" ]; then\n echo \"[plex-settings] not claimed yet \u2014 skipping prefs, will retry next start\" >&2\n exit 0\nfi\n\n# Percent-encode what our values actually contain. Plex takes prefs as QUERY parameters on\n# a PUT, so a raw '/' or ',' would be read as structure rather than data.\nenc() {\n printf '%s' \"$1\" | sed -e 's|%|%25|g' -e 's|/|%2F|g' -e 's|,|%2C|g' \\\n -e 's|:|%3A|g' -e 's| |%20|g'\n}\nsetpref() {\n code=$(curl -s -o /dev/null -w '%{http_code}' --max-time 10 -X PUT\
\ \\\n -H \"X-Plex-Token: $TOKEN\" \"$API/:/prefs?$1=$(enc \"$2\")\" 2>/dev/null)\n if [ \"$code\" = \"200\" ]; then\n echo \"[plex-settings] set $1\"\n else\n echo \"[plex-settings] WARN: $1 -> HTTP $code\" >&2\n fi\n}\n\n# (4) THE LAN DEFINITION \u2014 the reason this hook was written. Without it Plex has NO notion\n# of a local network and tags EVERY client remote. Measured 2026-10-04: 468 requests\n# logged \"Allowed Network (WAN)\" and ZERO \"(LAN)\", including Seerr from inside the\n# cluster, which is why the server showed as Remote to LAN clients. Note allowedNetworks\n# (the ALLOWED_NETWORKS env) only waives AUTH \u2014 it does NOT make a client local. Two\n# different settings; only one of them has an env hook.\nsetpref LanNetworksBandwidth \"$LANNETS\"\n\n# (5) THE CERTIFICATE, which is what makes the `https` in ADVERTISE_IP servable. Set only\n# if the bundle exists \u2014 pointing Plex at a missing file would break its TLS listener.\nif [ -s \"$P12\" ]; then\n setpref customCertificatePath \"$P12\"\n setpref customCertificateKey \"$P12PASS\"\n setpref customCertificateDomain \"$CERTDOMAIN\"\nelse\n echo \"[plex-settings] WARN: no bundle; leaving certificate prefs untouched\" >&2\nfi\nexit 0\n"
volumes:
- name: config
persistentVolumeClaim:
claimName: plex-config
- name: media
persistentVolumeClaim:
claimName: plex-media
---
# Service
apiVersion: v1
kind: Service
metadata:
name: plex
namespace: plex
labels:
app.kubernetes.io/name: plex
spec:
selector:
app.kubernetes.io/name: plex
ports:
- name: port-32400
port: 32400
targetPort: 32400
---
# CiliumNetworkPolicy reciprocal ingress allow (R-NET-02): plex-allow-radarr-32400
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: plex-allow-radarr-32400
namespace: plex
labels:
app.kubernetes.io/name: plex
spec:
endpointSelector:
matchLabels:
app.kubernetes.io/name: plex
ingress:
- fromEndpoints:
- matchLabels:
app.kubernetes.io/name: radarr
k8s:io.kubernetes.pod.namespace: radarr
toPorts:
- ports:
- port: '32400'
protocol: TCP
---
# CiliumNetworkPolicy reciprocal ingress allow (R-NET-02): plex-allow-seerr-32400
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: plex-allow-seerr-32400
namespace: plex
labels:
app.kubernetes.io/name: plex
spec:
endpointSelector:
matchLabels:
app.kubernetes.io/name: plex
ingress:
- fromEndpoints:
- matchLabels:
app.kubernetes.io/name: seerr
k8s:io.kubernetes.pod.namespace: seerr
toPorts:
- ports:
- port: '32400'
protocol: TCP
---
# CiliumNetworkPolicy reciprocal ingress allow (R-NET-02): plex-allow-sonarr-32400
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: plex-allow-sonarr-32400
namespace: plex
labels:
app.kubernetes.io/name: plex
spec:
endpointSelector:
matchLabels:
app.kubernetes.io/name: plex
ingress:
- fromEndpoints:
- matchLabels:
app.kubernetes.io/name: sonarr
k8s:io.kubernetes.pod.namespace: sonarr
toPorts:
- ports:
- port: '32400'
protocol: TCP
---
# CiliumNetworkPolicy gateway-ingress allow (R-NET-01/02): plex-allow-gateway-ingress
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: plex-allow-gateway-ingress
namespace: plex
labels:
app.kubernetes.io/name: plex
spec:
endpointSelector:
matchLabels:
app.kubernetes.io/name: plex
ingress:
- fromEntities:
- ingress
toPorts:
- ports:
- port: '32400'
protocol: TCP
---
# CiliumNetworkPolicy world egress allow (egress class governs the path; R-NET-05): plex-allow-internet-egress
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: plex-allow-internet-egress
namespace: plex
labels:
app.kubernetes.io/name: plex
spec:
endpointSelector:
matchLabels:
app.kubernetes.io/name: plex
egress:
- toEntities:
- world
---
# HTTPRoute plex-32400 (R-TLS/R-DNS)
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: plex-32400
namespace: plex
labels:
app.kubernetes.io/name: plex
spec:
parentRefs:
- name: shared
namespace: gateway
sectionName: https
hostnames:
- plex.${BASE_DOMAIN}
rules:
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: plex
port: 32400
---
# PrometheusRule SLI alerts (R-OBS-01)
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
name: plex-slis
namespace: plex
labels:
release: kube-prometheus-stack
app.kubernetes.io/name: plex
spec:
groups:
- name: plex.slis
rules:
- alert: plex-down
expr: kube_deployment_status_replicas_available{namespace="plex", deployment="plex"} == 0 or absent(kube_deployment_status_replicas_available{namespace="plex", deployment="plex"})
for: 5m
labels:
severity: critical
service: plex
annotations:
summary: plex is down
description: 'plex: its Deployment has no available/ready replica for 5m, or the signal is gone entirely (SLI: up, via kube_deployment_status_replicas_available).'
Reference in new issue View Git Blame Copy Permalink