The second requirement ever proven on this project, and the one the operator had attested was working but the suite could not attest to. Test authored 2026-08-31 carrying a `conformance.home.k3s/never-executed` annotation; executed for the first time tonight. test dr-restore run 20260902T001846Z-fdc3265 req R-STATE-03 (safety-critical, DR) junit passed / 0 failed / 0 skipped sentinel DR_RESTORED_FROM_OFF_CLUSTER_BACKUP_OK proof proven shafdc3265(clean tree) kube v1.36.4+k3s1 WHAT IT ACTUALLY PROVED, in three steps: 1. The backup target is genuinely OFF-CLUSTER. BSL velero/default phase=Available at http://192.168.1.9:9000 — endpoint resolved and asserted to be none of: loopback, any node Internal/ExternalIP, any Service ClusterIP, any Pod IP, inside any node podCIDR, or inside any ServiceCIDR. 2. Backups are SCHEDULED, not merely possible — velero-daily and velero-daily-resources-only, both phase=Enabled, last run 2026-09-01T03:00:27Z. 3. The drill: seed a nanosecond-unique canary onto a longhorn PVC; Velero Backup to Completed with a kopia PodVolumeBackup carrying a real snapshotID (volume DATA reached the remote repository, not just resource JSON); ANNIHILATE the source — delete the namespace, poll until gone, force-delete the PV, FAIL if the original PV survived; Restore into a clean namespace; assert the restored PVC bound a DIFFERENT PV and the canary bytes are byte-identical. The bytes could only have come from outside the cluster. VALIDATES THE PARSER FIX FROM57d4df8. This log contains SEVEN occurrences of DR_RESTORE_SKIPPED_OK — every one of them inside a `=== COMMAND` script echo — and the pass token twice in real `=== STDOUT`. Under the pre-fix scanner this genuine pass would have been recorded as `skipped`, exactly as node-loss was. The section-parsing rule is doing real work, not theoretical work. CLEANUP VERIFIED against a pre-run baseline: Velero backups back to 7 (its own chainsaw-*-dr-backup deleted via DeleteBackupRequest), 0 chainsaw backups, 0 restores, 0 orphan Released/Available PVs, both scratch namespaces gone, 13/13 Kustomizations Ready, 0 pods not Running. It wrote into and cleaned out of the SAME bucket and kopia repository the live nightly schedule uses, and left it as it found it. SEPARATE FINDING, cleaned up: `chainsaw-stable-worm` was still Active from 2026-08-25 — empty residue from an earlier conformance session that never tore its namespace down. Deleted. Worth knowing that Chainsaw namespace leaks can outlive a run by a week unnoticed. NOT proven by this test, stated in its own header rather than implied: off-HOST DR (the dev nodes are VMs on the same Proxmox host as the S3 target, so this does not survive hypervisor loss); no direct S3 client read; real service data (Nextcloud/Gitea/CNPG drills remain operator exercises); bare-metal rebuild DR; encryption-at-rest; retention/GC. Serves: R-STATE-03, R-TEST-03, R-TEST-05. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
3 lines
1.1 KiB
JSON
3 lines
1.1 KiB
JSON
{"chainsaw_version": "v0.2.12", "env": "dev", "git_sha": "57d4df8c4f992732a49f3e2daddcace866778ab0", "junit_result": "passed", "kube_version": "v1.36.4+k3s1", "layer": "L3", "note": "run log OUTPUT carries every declared pass token (NODE_LOSS_VOLUME_AVAILABLE_OK) and no skip token [log scoped to this testcase; tokens read from real-output sections only]", "proof": "proven", "requirements": ["R-STATE-02"], "run_id": "20260901T235117Z-e9a4b66", "sentinel": "NODE_LOSS_VOLUME_AVAILABLE_OK", "test": "node-loss", "ts_utc": "2026-09-01T23:51:17Z"}
|
|
{"chainsaw_version": "v0.2.12", "env": "dev", "git_sha": "fdc3265585f4b775c0a6ae3fb33c837b0c643655", "junit_result": "passed", "kube_version": "v1.36.4+k3s1", "layer": "L3", "note": "run log OUTPUT carries every declared pass token (OFF_CLUSTER_TARGET_OK, BACKUPS_ARE_SCHEDULED_OK, DR_RESTORED_FROM_OFF_CLUSTER_BACKUP_OK) and no skip token [log scoped to this testcase; tokens read from real-output sections only]", "proof": "proven", "requirements": ["R-STATE-03"], "run_id": "20260902T001907Z-fdc3265", "sentinel": "DR_RESTORED_FROM_OFF_CLUSTER_BACKUP_OK", "test": "dr-restore", "ts_utc": "2026-09-02T00:19:07Z"}
|