Home-Network/home-network-k3s-impl
2
0 Fork 0
Code Issues 2 Pull Requests 5 Actions Packages Projects Releases Wiki Activity
Files
d8b25a67feeea46f3e1311d4e8fb15aedaaad8e8
home-network-k3s-impl/conformance/runs/20260913T014437Z-19fc8bc/run.log
T
jakman85andClaude Opus 5 9d0814ae61 test(conformance): R-NET-06 run against prod — L1 proven, L3 pass recorded -dirty 
Two recorded runs against PROD (CONF_ENV=prod, context k3s-prod):

  20260913T014321Z-19fc8bc  egress-class-render (L1)  proven
  20260913T014437Z-19fc8bc  vpn-killswitch      (L3)  proven, but git_sha -dirty

The L3 run PASSED for real on prod — every declared sentinel present
(NS_PRIVILEGED_OK, TUNNEL_HEALTHY_OK, NO_LEAK_OK, INTERNET_UNAFFECTED_OK,
TUNNEL_SELF_HEAL_OK, TUNNEL_RECOVERED_OK), tunnel stopped 01:44:48Z and
recovered unaided by 01:46:25Z.

Its row is nevertheless NOT usable as proof: git_sha carries `-dirty`, so
status.py's ancestry check cannot resolve it and the row degrades to
stale-proof. Cause is a harness flaw, not the cluster:
lib/record_run.py's git_sha() ignores UNTRACKED files (so a run's own
runs/<id>/ does not mark it dirty) but conformance/runs/ledger.jsonl is
TRACKED — so the FIRST --record run of a session appends a row, dirties the
tree, and every SUBSEQUENT --record run in that same session is stamped
-dirty. Recording one test poisons the proof of the next.

Committed as-is rather than rewritten: the ledger is append-only and this is
honest evidence of a real run. A clean re-run follows and supersedes it by
latest-wins (lib/status.py latest_by_test).

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-09-12 20:48:55 -05:00

17 KiB
Raw Blame History

[run.sh] Coverage report first (always)
[coverage] 49/51 requirements have a scaffolded test (96.1%); 2 gap(s), 0 safety-critical gap(s).
[coverage] wrote /work/.claude/conf-clone/conformance/_artifacts/coverage/coverage.md and /work/.claude/conf-clone/conformance/_artifacts/coverage/coverage.json
[run.sh] No L0/L1 offline checks match the selection (layer=l3 reqs='R-NET-06').
[run.sh] LIGHT pass: 1 test(s), parallel=2 (pinned v0.2.12)
Version: 0.2.12
Loading config (/work/.claude/conf-clone/conformance/config/chainsaw-config.yaml)...
- Using test file: chainsaw-test
- TestDirs [/work/.claude/conf-clone/conformance/tests/l3-fault-injection/vpn-killswitch]
- SkipDelete false
- FailFast false
- ReportFormat 'JUNIT-TEST'
- ReportName 'chainsaw-report-light'
- ReportPath '/work/.claude/conf-clone/conformance/_artifacts'
- Namespace ''
- FullName false
- IncludeTestRegex ''
- ExcludeTestRegex ''
- ApplyTimeout 1m30s
- AssertTimeout 3m0s
- CleanupTimeout 2m0s
- DeleteTimeout 2m0s
- ErrorTimeout 1m0s
- ExecTimeout 3m0s
- DeletionPropagationPolicy Background
- Parallel 2
- Template true
- NoCluster false
- PauseOnFailure false
Loading tests...
- vpn-killswitch (/work/.claude/conf-clone/conformance/tests/l3-fault-injection/vpn-killswitch)
Loading values...
Running tests...
=== RUN chainsaw
=== PAUSE chainsaw
=== CONT chainsaw
=== RUN chainsaw/vpn-killswitch
=== PAUSE chainsaw/vpn-killswitch
=== CONT chainsaw/vpn-killswitch
| 01:44:38 | vpn-killswitch | @chainsaw | CREATE | OK | v1/Namespace @ chainsaw-credible-bird
| 01:44:38 | vpn-killswitch | given-healthy-tunnel-and-two-workloads | TRY | BEGIN |
| 01:44:38 | vpn-killswitch | given-healthy-tunnel-and-two-workloads | CMD | RUN |
=== COMMAND
/usr/bin/sh -c set -eu
kubectl label namespace "$NAMESPACE" \
pod-security.kubernetes.io/enforce=privileged \
pod-security.kubernetes.io/warn=privileged \
pod-security.kubernetes.io/audit=privileged \
--overwrite
echo "NS_PRIVILEGED_OK"
| 01:44:39 | vpn-killswitch | given-healthy-tunnel-and-two-workloads | SCRIPT | LOG |
=== STDOUT
namespace/chainsaw-credible-bird labeled
NS_PRIVILEGED_OK
| 01:44:39 | vpn-killswitch | given-healthy-tunnel-and-two-workloads | SCRIPT | DONE |
| 01:44:39 | vpn-killswitch | given-healthy-tunnel-and-two-workloads | APPLY | RUN | v1/Pod @ chainsaw-credible-bird/egress-internet
| 01:44:39 | vpn-killswitch | given-healthy-tunnel-and-two-workloads | CREATE | OK | v1/Pod @ chainsaw-credible-bird/egress-internet
| 01:44:39 | vpn-killswitch | given-healthy-tunnel-and-two-workloads | APPLY | DONE | v1/Pod @ chainsaw-credible-bird/egress-internet
| 01:44:39 | vpn-killswitch | given-healthy-tunnel-and-two-workloads | APPLY | RUN | v1/Pod @ chainsaw-credible-bird/egress-vpn
| 01:44:39 | vpn-killswitch | given-healthy-tunnel-and-two-workloads | CREATE | OK | v1/Pod @ chainsaw-credible-bird/egress-vpn
| 01:44:39 | vpn-killswitch | given-healthy-tunnel-and-two-workloads | APPLY | DONE | v1/Pod @ chainsaw-credible-bird/egress-vpn
| 01:44:39 | vpn-killswitch | given-healthy-tunnel-and-two-workloads | ASSERT | RUN | v1/Pod @ chainsaw-credible-bird/egress-internet
| 01:44:41 | vpn-killswitch | given-healthy-tunnel-and-two-workloads | ASSERT | DONE | v1/Pod @ chainsaw-credible-bird/egress-internet
| 01:44:41 | vpn-killswitch | given-healthy-tunnel-and-two-workloads | ASSERT | RUN | v1/Pod @ chainsaw-credible-bird/egress-vpn
| 01:44:41 | vpn-killswitch | given-healthy-tunnel-and-two-workloads | ASSERT | DONE | v1/Pod @ chainsaw-credible-bird/egress-vpn
| 01:44:41 | vpn-killswitch | given-healthy-tunnel-and-two-workloads | CMD | RUN |
=== COMMAND
/usr/bin/sh -c set -eu
ns="$NAMESPACE"
get_ip() {
kubectl -n "$ns" exec "$1" -c curl -- \
curl -fsS --max-time 10 https://api.ipify.org 2>/dev/null || true
}
wan="" vpn=""
for i in $(seq 1 12); do
[ -n "$wan" ] || wan="$(get_ip egress-internet)"
[ -n "$vpn" ] || vpn="$(get_ip egress-vpn)"
if [ -n "$wan" ] && [ -n "$vpn" ] && [ "$wan" != "$vpn" ]; then break; fi
[ "$wan" = "$vpn" ] && vpn="" # not steered yet; re-poll
sleep 5
done
echo "wan=$wan vpn=$vpn"
[ -n "$wan" ] || { echo "FAIL: no WAN IP from internet-class pod" >&2; exit 1; }
[ -n "$vpn" ] || { echo "FAIL: vpn-class egress not working at baseline" >&2; exit 1; }
[ "$wan" != "$vpn" ] || { echo "FAIL: vpn-class == WAN at baseline (not tunneled)" >&2; exit 1; }
echo "TUNNEL_HEALTHY_OK"
| 01:44:43 | vpn-killswitch | given-healthy-tunnel-and-two-workloads | SCRIPT | LOG |
=== STDOUT
wan=136.51.115.252 vpn=158.173.241.11
TUNNEL_HEALTHY_OK
| 01:44:43 | vpn-killswitch | given-healthy-tunnel-and-two-workloads | SCRIPT | DONE |
| 01:44:43 | vpn-killswitch | given-healthy-tunnel-and-two-workloads | TRY | END |
| 01:44:43 | vpn-killswitch | when-force-tunnel-down-then-no-leak | TRY | BEGIN |
| 01:44:43 | vpn-killswitch | when-force-tunnel-down-then-no-leak | APPLY | RUN | batch/v1/Job @ chainsaw-credible-bird/tunnel-stop
| 01:44:43 | vpn-killswitch | when-force-tunnel-down-then-no-leak | CREATE | OK | batch/v1/Job @ chainsaw-credible-bird/tunnel-stop
| 01:44:43 | vpn-killswitch | when-force-tunnel-down-then-no-leak | APPLY | DONE | batch/v1/Job @ chainsaw-credible-bird/tunnel-stop
| 01:44:43 | vpn-killswitch | when-force-tunnel-down-then-no-leak | ASSERT | RUN | batch/v1/Job @ chainsaw-credible-bird/tunnel-stop
| 01:44:48 | vpn-killswitch | when-force-tunnel-down-then-no-leak | ASSERT | DONE | batch/v1/Job @ chainsaw-credible-bird/tunnel-stop
| 01:44:48 | vpn-killswitch | when-force-tunnel-down-then-no-leak | CMD | RUN |
=== COMMAND
/usr/bin/sh -c set -u
ns="$NAMESPACE"
wan="$(kubectl -n "$ns" exec egress-internet -c curl -- \
curl -fsS --max-time 10 https://api.ipify.org 2>/dev/null || true)"
for i in $(seq 1 8); do
out="$(kubectl -n "$ns" exec egress-vpn -c curl -- \
curl -fsS --max-time 8 https://api.ipify.org 2>/dev/null || true)"
if [ -n "$out" ]; then
if [ -n "$wan" ] && [ "$out" = "$wan" ]; then
echo "LEAK: vpn-class reached the WAN IP ($out) with the tunnel DOWN" >&2; exit 1
fi
echo "LEAK: vpn-class reached the internet ($out) with the tunnel DOWN" >&2; exit 1
fi
echo "attempt $i: vpn-class blackholed (no response), good"
sleep 3
done
echo "NO_LEAK_OK"
| 01:46:18 | vpn-killswitch | when-force-tunnel-down-then-no-leak | SCRIPT | LOG |
=== STDOUT
attempt 1: vpn-class blackholed (no response), good
attempt 2: vpn-class blackholed (no response), good
attempt 3: vpn-class blackholed (no response), good
attempt 4: vpn-class blackholed (no response), good
attempt 5: vpn-class blackholed (no response), good
attempt 6: vpn-class blackholed (no response), good
attempt 7: vpn-class blackholed (no response), good
attempt 8: vpn-class blackholed (no response), good
NO_LEAK_OK
| 01:46:18 | vpn-killswitch | when-force-tunnel-down-then-no-leak | SCRIPT | DONE |
| 01:46:18 | vpn-killswitch | when-force-tunnel-down-then-no-leak | CMD | RUN |
=== COMMAND
/usr/bin/sh -c set -eu
ns="$NAMESPACE"
out="$(kubectl -n "$ns" exec egress-internet -c curl -- \
curl -fsS --max-time 10 https://api.ipify.org 2>/dev/null || true)"
echo "internet_ip=$out"
[ -n "$out" ] || { echo "FAIL: internet-class egress broke while tunnel down (must be unaffected)" >&2; exit 1; }
echo "INTERNET_UNAFFECTED_OK"
| 01:46:18 | vpn-killswitch | when-force-tunnel-down-then-no-leak | SCRIPT | LOG |
=== STDOUT
internet_ip=136.51.115.252
INTERNET_UNAFFECTED_OK
| 01:46:18 | vpn-killswitch | when-force-tunnel-down-then-no-leak | SCRIPT | DONE |
| 01:46:18 | vpn-killswitch | when-force-tunnel-down-then-no-leak | TRY | END |
| 01:46:18 | vpn-killswitch | when-force-tunnel-down-then-no-leak | FINALLY | BEGIN |
| 01:46:18 | vpn-killswitch | when-force-tunnel-down-then-no-leak | CMD | RUN |
=== COMMAND
/usr/bin/sh -c set -eu
ns="$NAMESPACE"
# Distinct Job name (tunnel-restart) so it never collides with the
# tunnel-stop Job. Left-aligned manifest so the block scalar yields
# valid YAML at column 0 at runtime.
cat <<'JOB' | kubectl -n "$ns" apply -f -
apiVersion: batch/v1
kind: Job
metadata:
name: tunnel-restart
labels: { app: vpn-killswitch-fault }
spec:
backoffLimit: 3
activeDeadlineSeconds: 90
ttlSecondsAfterFinished: 600
template:
metadata:
labels: { app: vpn-killswitch-fault }
spec:
restartPolicy: Never
hostPID: true
nodeSelector:
homelab.keithjackson.dev/egress-gateway: "true"
tolerations:
- operator: Exists
containers:
- name: nsenter
image: busybox:1.37.0
securityContext:
privileged: true
command: ["nsenter","-t","1","-m","-u","-i","-n","-p","--","systemctl","start","vpn-egress-tunnel"]
JOB
# Block teardown until the restart actually completed on the host — a
# leaked-down tunnel is unacceptable. `wait` exits non-zero on failure
# or timeout, which (set -e) fails the step and surfaces the problem.
kubectl -n "$ns" wait --for=condition=complete --timeout=110s job/tunnel-restart
echo "TUNNEL_SELF_HEAL_OK"
| 01:46:21 | vpn-killswitch | when-force-tunnel-down-then-no-leak | SCRIPT | LOG |
=== STDOUT
job.batch/tunnel-restart created
job.batch/tunnel-restart condition met
TUNNEL_SELF_HEAL_OK
| 01:46:21 | vpn-killswitch | when-force-tunnel-down-then-no-leak | SCRIPT | DONE |
| 01:46:21 | vpn-killswitch | when-force-tunnel-down-then-no-leak | FINALLY | END |
| 01:46:21 | vpn-killswitch | when-tunnel-recovers-connectivity-returns | TRY | BEGIN |
| 01:46:21 | vpn-killswitch | when-tunnel-recovers-connectivity-returns | CMD | RUN |
=== COMMAND
/usr/bin/sh -c set -eu
ns="$NAMESPACE"
wan="$(kubectl -n "$ns" exec egress-internet -c curl -- \
curl -fsS --max-time 10 https://api.ipify.org 2>/dev/null || true)"
vpn=""
for i in $(seq 1 24); do
vpn="$(kubectl -n "$ns" exec egress-vpn -c curl -- \
curl -fsS --max-time 8 https://api.ipify.org 2>/dev/null || true)"
if [ -n "$vpn" ] && [ "$vpn" != "$wan" ]; then break; fi
vpn=""
sleep 5
done
echo "recovered_vpn=$vpn wan=$wan"
[ -n "$vpn" ] || { echo "FAIL: vpn-class egress did not recover after tunnel restart" >&2; exit 1; }
[ "$vpn" != "$wan" ] || { echo "FAIL: vpn-class egress recovered onto the WAN (not tunneled)" >&2; exit 1; }
echo "TUNNEL_RECOVERED_OK"
| 01:46:25 | vpn-killswitch | when-tunnel-recovers-connectivity-returns | SCRIPT | LOG |
=== STDOUT
recovered_vpn=158.173.241.11 wan=136.51.115.252
TUNNEL_RECOVERED_OK
| 01:46:25 | vpn-killswitch | when-tunnel-recovers-connectivity-returns | SCRIPT | DONE |
| 01:46:25 | vpn-killswitch | when-tunnel-recovers-connectivity-returns | TRY | END |
| 01:46:25 | vpn-killswitch | when-force-tunnel-down-then-no-leak | CLEANUP | BEGIN |
| 01:46:25 | vpn-killswitch | when-force-tunnel-down-then-no-leak | DELETE | OK | batch/v1/Job @ chainsaw-credible-bird/tunnel-stop
| 01:46:25 | vpn-killswitch | when-force-tunnel-down-then-no-leak | CLEANUP | END |
| 01:46:25 | vpn-killswitch | given-healthy-tunnel-and-two-workloads | CLEANUP | BEGIN |
| 01:46:25 | vpn-killswitch | given-healthy-tunnel-and-two-workloads | DELETE | OK | v1/Pod @ chainsaw-credible-bird/egress-vpn
| 01:46:56 | vpn-killswitch | given-healthy-tunnel-and-two-workloads | DELETE | OK | v1/Pod @ chainsaw-credible-bird/egress-internet
| 01:47:26 | vpn-killswitch | given-healthy-tunnel-and-two-workloads | CLEANUP | END |
| 01:47:26 | vpn-killswitch | @chainsaw | CLEANUP | BEGIN |
| 01:47:26 | vpn-killswitch | @chainsaw | DELETE | OK | v1/Namespace @ chainsaw-credible-bird
| 01:47:33 | vpn-killswitch | @chainsaw | CLEANUP | END |
--- PASS: chainsaw (0.00s)
--- PASS: chainsaw/vpn-killswitch (174.63s)
PASS
Tests Summary...
- Passed tests 1
- Failed tests 0
- Skipped tests 0
Done.
[run.sh] HEAVY pass: no heavy tests selected.
merge_junit: merged 1 report(s) -> /work/.claude/conf-clone/conformance/_artifacts/report.junit.xml (tests=1 failures=0 errors=0 skipped=0)
[run.sh] Done. JUnit: /work/.claude/conf-clone/conformance/_artifacts/report.junit.xml | Coverage: /work/.claude/conf-clone/conformance/_artifacts/coverage/coverage.md
Reference in new issue View Git Blame Copy Permalink