# ============================================================================= # L0 STATIC — the bulk tier's capacity strategy is DISTINCT from the replicated tier's # Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster. # # THE BUG CLASS THIS CLOSES. The `bulk` StorageClass existed from Phase 0 as a stub: # `provisioner: nfs.csi.k8s.io`, `server: REPLACE-ME`, `share: REPLACE-ME`, TODO(OQ-2). # Four ServiceDefinitions carried a comment promising to attach their media once OQ-2 was # resolved, and the ONLY thing keeping a multi-terabyte library off the 3-replica # `longhorn` class was those four comments. A bulk volume that lands on the replicated # tier does not fail loudly — Longhorn accepts the PVC and starts replicating 3x of data # three mini-PCs cannot hold. R-STATE-04 is precisely that the two tiers are distinct, so # the distinctness is asserted rather than reviewed. # # It also pins the four decisions that make the STATIC form of this tier safe, each with a # plausible "simplification" that breaks something silently: `Retain` (bulk data pre-exists # the cluster and has consumers outside it, so anything else makes `delete pvc` a data-loss # operation); NO dynamic provisioning (a dynamic bulk class hands out a FRESH EMPTY # directory that mounts and Binds perfectly while the real 8 TiB is never attached — the # most deceptive failure available here); `hard` not `soft` (a soft mount hands a server # hiccup to the app as a short read, which for a media library is silent corruption); and # NFSv4 (mount and lock in-protocol ⇒ ONE reachable port, tcp/2049, which is what the # operator must open in pfSense for prod to reach this server at all). # # AND the EXDEV trap, which is not about storage. Sonarr/Radarr import by hardlinking or # atomically renaming, both same-filesystem operations, and to the kernel "same filesystem" # means SAME MOUNT. Splitting one export into two subPath mounts looks TIGHTER and breaks # link() with EXDEV; the arrs then degrade to copy-then-delete, rewriting every import over # NFS and destroying the seeding copy. It surfaces weeks later as "imports are slow and my # ratio is gone". A reviewer cannot see that; this check can. # # GIVEN the repository as authored, # WHEN every `class: bulk` storage entry in catalog/ is compared against the objects the # COMMITTED apps/ goldens carry for it (what Flux reconciles — deliberately not the # renderer, which would only prove it agrees with itself), # THEN (1) no bulk volume is on a Longhorn-provisioned class, and its class is explicitly # EMPTY so nothing can dynamically provision an empty stand-in; # (2) its capacity is declared and is at least an order of magnitude above the same # service's replicated volume — the requirement stated as arithmetic; # (3) PV and PVC are pre-bound from BOTH ends (claimRef + volumeName) and agree; # (4) the PV reclaim policy is Retain; # (5) mountOptions carry `hard` and exactly one `nfsvers=4.x`, and never `soft`; # (6) no workload mounts one export more than once (the EXDEV trap); # (7) a readOnly entry renders nfs.readOnly AND a readOnly volumeMount AND # ReadOnlyMany; a writable one renders neither and is ReadWriteMany; every bulk # access mode is a `...Many`, since one export serves several namespaces and # several nodes; # (8) the PV's nfs.server is a BARE cluster-vars token (never a literal — dev and # prod reach this host by different addresses), that knob has a base default, and # EVERY overlay supplies it; # AND (9) the check is LOAD-BEARING (ci/l0.sh runs it unconditionally, ci/gate.sh blocks # on the l0 stage) and NON-VACUOUS (an empty bulk corpus FAILS rather than # passing trivially, and a floor on the assertion count keeps a pass from # degenerating into a no-op). # # STATUS: EXECUTABLE. `check.sh` is a pure offline assertion script (standard library + # PyYAML); conformance/run.sh --layer l0 runs it directly and this Chainsaw Test wraps the # SAME script. # # NOT proven — the runtime half, which needs a cluster: that the server is reachable, that # the export admits these clients, that the PVC binds, that a write succeeds, or that a # hardlink between the library and the download path actually works. The uid analysis is # reasoning (docs/BULK-NFS-STORAGE.md §6) and the in-cluster probes that would settle it # are written down in §9 of that document, unrun. Nor can this prove prod's address is # CORRECT — only that it is a supplied knob; while it is a placeholder, prod's pods will # not mount, and catching that statically needs the placeholder guard, which is absent from # main (§7). Read-only on the media tree is a CLIENT-side guarantee only: the export serves # one reader and two writers from the same node addresses, so the server cannot tell them # apart (§5). # ============================================================================= apiVersion: chainsaw.kyverno.io/v1alpha1 kind: Test metadata: name: bulk-tier-distinct-from-replicated labels: conformance.home.k3s/R-STATE-04: "true" annotations: conformance.home.k3s/requirements: "R-STATE-04" conformance.home.k3s/layer: "L0" # WHOLE-REPO SCOPE — relocated verbatim from ci/l0.sh's hand-written "Stage 3e # (UNCONDITIONAL)", which this annotation replaces: # # The `bulk` StorageClass sat as a REPLACE-ME stub for six roadmap phases, and the only # thing keeping a multi-terabyte media library off the 3-replica `longhorn` class was four # hand-written comments in ServiceDefinitions. A bulk volume that lands on the replicated # tier does not fail loudly — Longhorn accepts the PVC and starts replicating 3x of data # three mini-PCs cannot hold. # # WHY IT MUST NOT BE REQUIREMENT-SCOPED: ci/touched-requirements.sh's path map keys # NEITHER apps/* nor platform/longhorn/* to the bulk requirement. So the change most # likely to break this — moving a bulk volume onto the replicated tier, or narrowing one # export into two mounts — could attribute no requirement at all and be gated by nothing. # The invariant spans catalog/, apps/ and clusters/. conformance.home.k3s/scope: "whole-repo" conformance.home.k3s/safety-critical: "false" conformance.home.k3s/executable: "true" conformance.home.k3s/intent: >- Bulk media storage is provably NOT on the 3-replica Longhorn tier, is sized for a different order of magnitude, cannot be silently replaced by a dynamically provisioned empty volume, is never reclaimable by the cluster, mounts hard over NFSv4, and is never split into two mounts of one export — so hardlinked imports keep working. Its server is a per-environment knob, not a literal, because the two clusters reach the same storage host by different addresses. spec: steps: - name: assert-bulk-tier-is-distinct-and-statically-bound description: >- Read every `class: bulk` storage entry from catalog/, pair it with the PersistentVolume, PersistentVolumeClaim and volumeMount the committed apps/ goldens carry for it, and assert the tier separation, the static pre-binding, the reclaim and mount-option decisions, the read-only coupling, the one-export-one-mount invariant, and that the server address is supplied by every environment overlay. try: - script: content: bash ./check.sh