fix(bulk-nfs): repoint at the TrueNAS (192.168.2.30); the PVs must be recreated, not reconciled
Both ZFS pools moved off the Proxmox host onto a standalone TrueNAS on 192.168.2.30 -- prod's OWN /24, so the pfSense crossing that made this knob load-bearing is gone, along with the DNAT at 192.168.2.102 it mounted through. Measured 2026-10-07: 192.168.2.102:2049 no longer answers; .30:2049 and :111 do. BULK_NFS_SERVER -> 192.168.2.30 in prod, base and dev. The dev value is marked inert (dev is decommissioned); it carries the real address only so nothing in the tree renders a host that no longer exists. THE OPERATIONAL POINT, and why this is not just a value change: a bound PersistentVolume's NFS source is immutable. Server dry-run, 2026-10-07: spec.persistentvolumesource: Forbidden: spec.persistentvolumesource is immutable after creation so Flux's apply FAILS rather than updating, and the pods keep their old, dead mount. The five PVs must be deleted and recreated -- safe, because Retain never touches data and render.py emits a pre-set claimRef so each PV rebinds its own PVC. The sequence is docs/BULK-NFS-STORAGE.md section 10 and is an OPERATOR action; this commit does not perform it. Pairs with the hazard already on the page: spec.nfs.server is unvalidated at create (any string binds) and immutable at update. A wrong value is accepted silently and then cannot be corrected in place. docs/BULK-NFS-STORAGE.md: sections 1 and 5 rewritten for TrueNAS -- the measured exports, the service settings that decide whether uid 1000 survives the wire, the three stale ZFS sharenfs properties that had to be cleared, and the ACL hazard (acltype=nfsv4 + aclinherit=passthrough means one click in the ACL editor propagates to the whole future library). Section 7 rewritten now that the placeholder is gone, section 9 retargeted from dev to prod, section 10 new. Sections 2/3/4/6 are unchanged by the move. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
1 parent
ad37acdf53
commit
0747849837
7 files changed
+369
-188
No files matched your search
@@ -199,7 +199,7 @@ data:
|
||||
"id": 15,
|
||||
"type": "timeseries",
|
||||
"title": "CAUSE 4 — NFS RPC retransmission ratio, per node",
|
||||
"description": "node_nfs_rpc_retransmissions_total / node_nfs_rpcs_total, from /proc/net/rpc/nfs. Alert line at 1% (NodeNFSRetransmissionRateHigh). MEASURED BASELINE IS EXACTLY ZERO on all three nodes over 48h, including throughout the 2026-09-14 spike — so any visible line here means the export at 192.168.2.102, or the path to it, is dropping requests. A flat zero does NOT clear NFS: it only clears the TRANSPORT. A merely SLOW export needs per-mount RPC round-trip time, which arrives with the mountstats collector (see kube-prometheus-stack.yaml) and is the known gap recorded in runbooks/node-io.md.",
|
||||
"description": "node_nfs_rpc_retransmissions_total / node_nfs_rpcs_total, from /proc/net/rpc/nfs. Alert line at 1% (NodeNFSRetransmissionRateHigh). MEASURED BASELINE IS EXACTLY ZERO on all three nodes over 48h, including throughout the 2026-09-14 spike — so any visible line here means the export at 192.168.2.30, or the path to it, is dropping requests. A flat zero does NOT clear NFS: it only clears the TRANSPORT. A merely SLOW export needs per-mount RPC round-trip time, which arrives with the mountstats collector (see kube-prometheus-stack.yaml) and is the known gap recorded in runbooks/node-io.md.",
|
||||
"gridPos": {"h": 8, "w": 12, "x": 12, "y": 39},
|
||||
"fieldConfig": {"defaults": {"unit": "percentunit", "min": 0, "custom": {"drawStyle": "line", "fillOpacity": 5, "thresholdsStyle": {"mode": "line"}}, "thresholds": {"mode": "absolute", "steps": [
|
||||
{"color": "green", "value": null}, {"color": "red", "value": 0.01}
|
||||
|
||||
Reference in new issue
Block a user