fix(bulk-nfs): repoint at the TrueNAS (192.168.2.30); the PVs must be recreated, not reconciled
Both ZFS pools moved off the Proxmox host onto a standalone TrueNAS on 192.168.2.30 -- prod's OWN /24, so the pfSense crossing that made this knob load-bearing is gone, along with the DNAT at 192.168.2.102 it mounted through. Measured 2026-10-07: 192.168.2.102:2049 no longer answers; .30:2049 and :111 do. BULK_NFS_SERVER -> 192.168.2.30 in prod, base and dev. The dev value is marked inert (dev is decommissioned); it carries the real address only so nothing in the tree renders a host that no longer exists. THE OPERATIONAL POINT, and why this is not just a value change: a bound PersistentVolume's NFS source is immutable. Server dry-run, 2026-10-07: spec.persistentvolumesource: Forbidden: spec.persistentvolumesource is immutable after creation so Flux's apply FAILS rather than updating, and the pods keep their old, dead mount. The five PVs must be deleted and recreated -- safe, because Retain never touches data and render.py emits a pre-set claimRef so each PV rebinds its own PVC. The sequence is docs/BULK-NFS-STORAGE.md section 10 and is an OPERATOR action; this commit does not perform it. Pairs with the hazard already on the page: spec.nfs.server is unvalidated at create (any string binds) and immutable at update. A wrong value is accepted silently and then cannot be corrected in place. docs/BULK-NFS-STORAGE.md: sections 1 and 5 rewritten for TrueNAS -- the measured exports, the service settings that decide whether uid 1000 survives the wire, the three stale ZFS sharenfs properties that had to be cleared, and the ACL hazard (acltype=nfsv4 + aclinherit=passthrough means one click in the ACL editor propagates to the whole future library). Section 7 rewritten now that the placeholder is gone, section 9 retargeted from dev to prod, section 10 new. Sections 2/3/4/6 are unchanged by the move. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
1 parent
ad37acdf53
commit
0747849837
7 files changed
+369
-188
No files matched your search
@@ -234,26 +234,28 @@ data:
|
||||
WG_CLIENT_CIDR: "10.90.0.0/24" # remote VPN clients; .1 = server. NOT a LAN range
|
||||
|
||||
# --- Bulk (external NFS) storage host (docs/04 bulk tier) ---------------------------------------
|
||||
# THE address of the Proxmox host whose ZFS datasets back every `class: bulk` volume: the media
|
||||
# library (Storage/plex, 8.33 T — plex, sonarr, radarr) and the legacy Nextcloud archive
|
||||
# THE address of the NAS whose ZFS datasets back every `class: bulk` volume: the media library
|
||||
# (Storage/plex, 8.33 T — plex, sonarr, radarr, qbittorrent) and the legacy Nextcloud archive
|
||||
# (Storage/nextcloud, 270 G). The renderer emits it into each static NFS PersistentVolume's
|
||||
# `spec.nfs.server` (catalog/render.py, BULK_NFS_SERVER); the export PATH is per-dataset and is
|
||||
# declared in the ServiceDefinition, because the path is identical in every environment and only
|
||||
# the ADDRESS is not.
|
||||
#
|
||||
# WHY THIS IS A KNOB AND NOT A LITERAL — the two clusters are on OPPOSITE SIDES of pfSense.
|
||||
# The storage host has exactly one interface, vmbr0 192.168.1.9/24, on pfSense's LAN.
|
||||
# dev nodes are 192.168.1.40/.41/.42 — the SAME L2 segment. They reach it directly, and one
|
||||
# NFS export to them already exists on that host (/mnt/Storage/k8s-volumes in
|
||||
# /etc/exports), so this path is proven, not theoretical. Base default = the dev value,
|
||||
# per the convention of the LAN block above.
|
||||
# prod nodes are 192.168.2.10/.11/.12 — pfSense's WAN. tcp/2049 does NOT traverse that today;
|
||||
# 22 and 8006 answer only because pfSense port-forwards them via 192.168.2.102. Opening
|
||||
# the storage path is a PENDING OPERATOR DECISION with two candidate outcomes that give
|
||||
# DIFFERENT values here: 192.168.2.102 (another port-forward) or 192.168.1.9 (a route).
|
||||
# So prod holds a loud TBD-PROD-* placeholder — see clusters/prod/kustomization.yaml for the
|
||||
# blast radius of shipping it unset, which is NOT the usual "fails to apply".
|
||||
BULK_NFS_SERVER: "192.168.1.9"
|
||||
# IT IS NO LONGER A CROSS-NETWORK KNOB (2026-10-07). This used to read "the two clusters are on
|
||||
# OPPOSITE SIDES of pfSense": the datasets lived on a Proxmox host whose only interface was
|
||||
# vmbr0 192.168.1.9/24 on pfSense's LAN, which prod (192.168.2.0/24, pfSense's WAN) could reach
|
||||
# only through a DNAT at 192.168.2.102. Both pools were physically moved onto a standalone
|
||||
# TrueNAS at 192.168.2.30 — prod's OWN /24 — so there is no crossing to arrange and no candidate
|
||||
# values to choose between. Measured 2026-10-07: 192.168.2.102:2049 no longer answers;
|
||||
# 192.168.2.30:2049 and :111 do. There is now exactly ONE storage host and one correct value;
|
||||
# this stays a knob only because the renderer reads it per-environment.
|
||||
#
|
||||
# *** CHANGING THIS VALUE IS NOT A PLAIN RECONCILE. *** A bound PersistentVolume's spec.nfs is
|
||||
# immutable — measured by server dry-run 2026-10-07:
|
||||
# spec.persistentvolumesource: Forbidden: spec.persistentvolumesource is immutable after creation
|
||||
# so Flux's apply FAILS rather than updating, and the pods keep their old mount. The PVs must be
|
||||
# deleted and recreated; the operator sequence is docs/BULK-NFS-STORAGE.md §10.
|
||||
BULK_NFS_SERVER: "192.168.2.30"
|
||||
|
||||
# Off-cluster S3 backup target (R-STATE-03, R-IDP-05) — the on-Proxmox MinIO, plain HTTP, path-style.
|
||||
# Consumed by BOTH backup paths: every CNPG Cluster's barmanObjectStore.endpointURL (keycloak-db,
|
||||
|
||||
Reference in new issue
Block a user