jakman85andClaude Opus 4.8
c6a480eed1
New runnable chainsaw test gateway-ingress-reachable (R-NET-01/02/03): stands up an
exposed-app fixture (namespace default-deny + the renderer's additive gateway-ingress
CiliumNetworkPolicy allowing fromEntities:[ingress] to the declared port 8080 only, a
backend listening on 8080 + an undeclared 9091, and declared/undeclared HTTPRoutes on
the shared Gateway) and proves BEHAVIOR through the REAL Gateway VIP:
* then-reachable-through-gateway -> the Gateway (ingress identity) reaches backend:8080
through default-deny (backend marker returned) [R-NET-02/03]
* then-undeclared-port-denied -> port 9091 (no allow) is NOT reachable through the
Gateway (marker never returned; envoy 503) [R-NET-02]
* then-east-west-still-denied -> a non-ingress in-namespace peer is DROPPED on 8080;
the allow is least-privilege, east-west default-deny intact (hermetic) [R-NET-01]
Verified live: reachable=REACHED (200, marker), undeclared=DENIED, east-west=DENIED.
Self-skipping: the gateway probes classify each attempt REACHED / DENIED / NO-GATEWAY
(curl 000). NO-GATEWAY self-skips (green + note) because a node-local Cilium L7LB/envoy
desync can make pods on an affected node unable to reach the Gateway VIP at all — a
SEPARATE platform fault, not default-deny — so this test never false-fails on it. The
east-west denial is hermetic and always asserts. VIP discovered per-step via a kubectl
output binding (env-agnostic; chainsaw bindings are step-scoped). L2 layer stays gated by
CLUSTER_AVAILABLE in run.sh. chainsaw v0.2.12 quirks honored (scalar/JMESPath asserts,
no readinessProbe under default-deny, step-level finally delete).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
303 lines
15 KiB
YAML
303 lines
15 KiB
YAML
# =============================================================================
|
|
# L2 BEHAVIORAL — Exposed app reachable through the Gateway under default-deny
|
|
# (R-NET-01, R-NET-02, R-NET-03) [RUNNABLE]
|
|
# Catalog: docs/14 (default-deny + explicit allow family)
|
|
# Spec: docs/03 "Default-deny baseline" / "Explicit allow rules";
|
|
# docs/09 "Interaction with default-deny" (ingress opens NORTH-SOUTH only).
|
|
#
|
|
# THE GAP THIS CLOSES: an onboarded app's rendered default-deny has `ingress: []`
|
|
# (deny ALL), so the Cilium Gateway (Envoy, reserved identity `ingress`) cannot
|
|
# reach the backend — an Accepted HTTPRoute is still unreachable. The renderer now
|
|
# also emits an ADDITIVE gateway-ingress CiliumNetworkPolicy allowing ONLY the
|
|
# `ingress` identity to ONLY the declared port. This test proves that end-to-end
|
|
# through the REAL shared Gateway:
|
|
#
|
|
# GIVEN an exposed app namespace with default-deny + the gateway-ingress allow
|
|
# (fromEntities: [ingress] -> declared port 8080 only), attached to the
|
|
# shared Gateway with a declared route (8080) and an undeclared route (9091),
|
|
# WHEN a client dials the Gateway VIP for the DECLARED host -> THEN it reaches the
|
|
# backend (proves the additive allow opens the north-south path, R-NET-02/03);
|
|
# WHEN a client dials the Gateway VIP for the UNDECLARED port -> THEN the backend is
|
|
# NOT reached (default-deny still drops it, R-NET-02);
|
|
# WHEN an in-namespace peer (NOT the `ingress` identity) dials the declared port
|
|
# east-west -> THEN it is DENIED (the allow is least-privilege / gateway-only;
|
|
# default-deny east-west is intact, R-NET-01).
|
|
#
|
|
# This asserts observable BEHAVIOR through the live Gateway (green Job = the intended
|
|
# verdict), not object existence (R-TEST-01).
|
|
#
|
|
# TWO FAILURE MODES ARE DISTINGUISHED (per the WS-NET pod->Gateway-VIP reachability
|
|
# finding, DECISIONS-LOG): the gateway probes classify each attempt as
|
|
# * REACHED (backend marker in the body) -> the intended verdict for the declared host;
|
|
# * DENIED (a real HTTP status, e.g. 503, but NO backend marker) -> default-deny drop
|
|
# (the intended verdict for the undeclared port);
|
|
# * NO-GATEWAY (curl code 000 / connect timeout) -> the probe's node cannot reach the
|
|
# Gateway VIP AT ALL. That is a SEPARATE platform fault (a node-local Cilium L7LB/
|
|
# envoy desync makes pods on an affected node unable to reach the Gateway VIP or
|
|
# ClusterIP), NOT default-deny. The probe SELF-SKIPS that case (exits 0 with a SKIP
|
|
# note) so this test never false-fails on the unrelated reachability bug. The
|
|
# hermetic east-west denial (below) needs no Gateway and always asserts.
|
|
#
|
|
# SELF-SKIPPING: the whole L2 layer is gated by CLUSTER_AVAILABLE in conformance/
|
|
# run.sh (skipped with no cluster). Within the test, `given` asserts the shared
|
|
# Gateway is Programmed (a missing Gateway fails fast/diagnostically), and the gateway
|
|
# probes self-skip the node-local reachability fault as above. The backend uses NO
|
|
# readinessProbe: the default-deny baseline blocks the kubelet node->pod probe, so a
|
|
# probe-bearing pod would never go Ready (DECISIONS-LOG K4); `phase: Running` is the
|
|
# correct readiness signal. Cleanup deletes both fixture namespaces in `finally`.
|
|
# =============================================================================
|
|
apiVersion: chainsaw.kyverno.io/v1alpha1
|
|
kind: Test
|
|
metadata:
|
|
name: gateway-ingress-reachable
|
|
labels:
|
|
conformance.home.k3s/R-NET-01: "true"
|
|
conformance.home.k3s/R-NET-02: "true"
|
|
conformance.home.k3s/R-NET-03: "true"
|
|
annotations:
|
|
conformance.home.k3s/requirements: "R-NET-01, R-NET-02, R-NET-03"
|
|
conformance.home.k3s/layer: "L2"
|
|
conformance.home.k3s/safety-critical: "false"
|
|
conformance.home.k3s/executable: "true"
|
|
conformance.home.k3s/intent: >-
|
|
An exposed app is reachable through the shared Gateway under default-deny via an
|
|
additive gateway-ingress allow (ingress identity -> declared port only); an
|
|
undeclared port is not reachable through the Gateway and east-west stays denied.
|
|
spec:
|
|
# A generous timeout: pulling images + programming two HTTPRoutes on the Gateway.
|
|
timeouts:
|
|
apply: 60s
|
|
assert: 180s
|
|
steps:
|
|
# -- GIVEN: the exposed-app fixture (default-deny + gateway-ingress allow + backend +
|
|
# Service + declared/undeclared HTTPRoutes) and a probe namespace. Assert the shared
|
|
# Gateway is Programmed (precondition) and the backend pod is Running (listeners up).
|
|
# Each gateway-probe step below re-captures the Gateway VIP into a binding (chainsaw
|
|
# output bindings are step-scoped) so probes dial the real ingress address that
|
|
# split-horizon DNS resolves service hostnames to — env-agnostic, no hardcoded IP.
|
|
- name: given-exposed-app-under-default-deny
|
|
try:
|
|
- apply:
|
|
file: fixtures/app-and-probe.yaml
|
|
- assert:
|
|
resource:
|
|
apiVersion: gateway.networking.k8s.io/v1
|
|
kind: Gateway
|
|
metadata:
|
|
name: shared
|
|
namespace: gateway
|
|
# JMESPath binding (parenthesized key) rather than a list literal: chainsaw does
|
|
# STRICT length-equal list matching, so asserting one element against the full
|
|
# conditions[] would fail (DECISIONS-LOG K4). This pinpoints Programmed=True.
|
|
status:
|
|
(conditions[?type == 'Programmed'] | [0].status): "True"
|
|
- assert:
|
|
resource:
|
|
apiVersion: v1
|
|
kind: Pod
|
|
metadata:
|
|
labels:
|
|
app.kubernetes.io/name: gw-backend
|
|
namespace: conformance-gw-ingress
|
|
status:
|
|
# scalar match — Running means both httpd listeners are up (no readinessProbe
|
|
# under default-deny; DECISIONS-LOG K4).
|
|
phase: Running
|
|
|
|
# -- THEN(reachable): a client in the unrestricted probe namespace dials the Gateway VIP
|
|
# for the DECLARED host. --connect-to sets SNI/Host to the hostname but dials the VIP
|
|
# (the real ingress address; -k because we test REACHABILITY, not TLS — trusted-tls
|
|
# covers the chain). The backend's marker in the body proves the Gateway (identity
|
|
# `ingress`) reached the backend THROUGH default-deny (green Job = REACHED). If the
|
|
# probe's node cannot reach the VIP at all (code 000), self-skip (green + SKIP note):
|
|
# that is the separate pod->Gateway-VIP platform fault, not default-deny.
|
|
- name: then-reachable-through-gateway
|
|
try:
|
|
# chainsaw output bindings are step-scoped, so re-capture the VIP in this step.
|
|
- command:
|
|
entrypoint: kubectl
|
|
args: [-n, gateway, get, gateway, shared, -o, "jsonpath={.status.addresses[0].value}"]
|
|
outputs:
|
|
- name: gwvip
|
|
value: ($stdout)
|
|
- apply:
|
|
resource:
|
|
apiVersion: batch/v1
|
|
kind: Job
|
|
metadata:
|
|
name: probe-reachable
|
|
namespace: conformance-gw-probe
|
|
spec:
|
|
backoffLimit: 6
|
|
activeDeadlineSeconds: 180
|
|
template:
|
|
spec:
|
|
restartPolicy: Never
|
|
containers:
|
|
- name: probe
|
|
image: curlimages/curl:8.11.1
|
|
env:
|
|
- name: GWVIP
|
|
value: ($gwvip)
|
|
command:
|
|
- sh
|
|
- -c
|
|
- |
|
|
set -u
|
|
host="gw-declared-conformance.home.keithjackson.dev"
|
|
last=000
|
|
i=0
|
|
while [ "$i" -lt 30 ]; do
|
|
code="$(curl -k -sS --max-time 8 -o /tmp/body \
|
|
-w '%{http_code}' \
|
|
--connect-to "${host}:443:${GWVIP}:443" \
|
|
"https://${host}/" 2>/dev/null || echo 000)"
|
|
last="$code"
|
|
if grep -q GW_BACKEND_REACHED /tmp/body 2>/dev/null; then
|
|
echo "REACHED: Gateway reached the backend through default-deny (code ${code})"
|
|
exit 0
|
|
fi
|
|
i=$((i+1)); sleep 5
|
|
done
|
|
if [ "$last" = "000" ]; then
|
|
echo "SKIP: this node cannot reach the Gateway VIP (code 000) —" 1>&2
|
|
echo " separate pod->Gateway-VIP platform fault, NOT default-deny" 1>&2
|
|
exit 0
|
|
fi
|
|
echo "FAIL: Gateway returned ${last} but the backend marker was absent — default-deny drop of the declared port?" 1>&2
|
|
exit 1
|
|
- assert:
|
|
resource:
|
|
apiVersion: batch/v1
|
|
kind: Job
|
|
metadata: { name: probe-reachable, namespace: conformance-gw-probe }
|
|
status: { succeeded: 1 }
|
|
|
|
# -- THEN(undeclared port denied through the Gateway): same Gateway VIP, same backend, the
|
|
# UNDECLARED host -> port 9091. Routing is identical to the declared route (proven to
|
|
# reach the backend), so the ONLY difference is the missing allow on 9091: the Gateway
|
|
# Envoy's upstream connect is DROPPED by default-deny. The backend marker MUST be
|
|
# absent. A leak (the 9091 marker appears) fails the Job. If the node cannot reach the
|
|
# VIP at all (code 000), self-skip (same platform fault as above).
|
|
- name: then-undeclared-port-denied-through-gateway
|
|
try:
|
|
# chainsaw output bindings are step-scoped, so re-capture the VIP in this step.
|
|
- command:
|
|
entrypoint: kubectl
|
|
args: [-n, gateway, get, gateway, shared, -o, "jsonpath={.status.addresses[0].value}"]
|
|
outputs:
|
|
- name: gwvip
|
|
value: ($stdout)
|
|
- apply:
|
|
resource:
|
|
apiVersion: batch/v1
|
|
kind: Job
|
|
metadata:
|
|
name: probe-undeclared
|
|
namespace: conformance-gw-probe
|
|
spec:
|
|
backoffLimit: 4
|
|
activeDeadlineSeconds: 150
|
|
template:
|
|
spec:
|
|
restartPolicy: Never
|
|
containers:
|
|
- name: probe
|
|
image: curlimages/curl:8.11.1
|
|
env:
|
|
- name: GWVIP
|
|
value: ($gwvip)
|
|
command:
|
|
- sh
|
|
- -c
|
|
- |
|
|
set -u
|
|
host="gw-undeclared-conformance.home.keithjackson.dev"
|
|
reached_gw=no
|
|
i=0
|
|
while [ "$i" -lt 8 ]; do
|
|
code="$(curl -k -sS --max-time 8 -o /tmp/body \
|
|
-w '%{http_code}' \
|
|
--connect-to "${host}:443:${GWVIP}:443" \
|
|
"https://${host}/" 2>/dev/null || echo 000)"
|
|
if grep -q GW_BACKEND_UNDECLARED /tmp/body 2>/dev/null; then
|
|
echo "LEAK: undeclared port 9091 reachable through the Gateway (code ${code})" 1>&2
|
|
exit 1
|
|
fi
|
|
[ "$code" != "000" ] && reached_gw=yes
|
|
i=$((i+1)); sleep 5
|
|
done
|
|
if [ "$reached_gw" = "yes" ]; then
|
|
echo "DENIED: undeclared port not reachable through the Gateway (backend marker never returned)"
|
|
exit 0
|
|
fi
|
|
echo "SKIP: this node cannot reach the Gateway VIP (code 000) —" 1>&2
|
|
echo " separate pod->Gateway-VIP platform fault, NOT default-deny" 1>&2
|
|
exit 0
|
|
- assert:
|
|
resource:
|
|
apiVersion: batch/v1
|
|
kind: Job
|
|
metadata: { name: probe-undeclared, namespace: conformance-gw-probe }
|
|
status: { succeeded: 1 }
|
|
|
|
# -- THEN(east-west still denied): HERMETIC (no Gateway needed). An in-namespace peer
|
|
# (NOT the `ingress` identity), covered by the namespace-wide default-deny, dials the
|
|
# DECLARED port east-west. The gateway-ingress allow is least-privilege (only
|
|
# fromEntities: [ingress]), so this connect MUST be dropped — default-deny east-west
|
|
# is intact (R-NET-01). The probe EXITS 0 only when the connection is BLOCKED (green
|
|
# Job = denied). This step always runs the real assertion (independent of the pod->VIP
|
|
# reachability fault above).
|
|
- name: then-east-west-still-denied
|
|
try:
|
|
- apply:
|
|
resource:
|
|
apiVersion: batch/v1
|
|
kind: Job
|
|
metadata:
|
|
name: probe-eastwest
|
|
namespace: conformance-gw-ingress
|
|
spec:
|
|
backoffLimit: 4
|
|
activeDeadlineSeconds: 120
|
|
template:
|
|
metadata:
|
|
labels: { app: eastwest-peer }
|
|
spec:
|
|
restartPolicy: Never
|
|
containers:
|
|
- name: probe
|
|
image: busybox:1.36.1
|
|
command:
|
|
- sh
|
|
- -c
|
|
- |
|
|
set -u
|
|
# DNS resolves (baseline allows it); the TCP connect must drop.
|
|
if nc -w 5 gw-backend 8080 </dev/null 2>/dev/null; then
|
|
echo "LEAK: east-west reached the declared port under default-deny" 1>&2
|
|
exit 1
|
|
fi
|
|
echo "DENIED: east-west blocked; gateway allow is least-privilege"
|
|
exit 0
|
|
- assert:
|
|
resource:
|
|
apiVersion: batch/v1
|
|
kind: Job
|
|
metadata: { name: probe-eastwest, namespace: conformance-gw-ingress }
|
|
status: { succeeded: 1 }
|
|
# -- Cleanup: delete both fixture namespaces (fixed names, outside chainsaw's managed
|
|
# namespace). finally accepts delete (DECISIONS-LOG: step-level finally = script/
|
|
# command/delete/wait only).
|
|
finally:
|
|
- delete:
|
|
ref:
|
|
apiVersion: v1
|
|
kind: Namespace
|
|
name: conformance-gw-ingress
|
|
- delete:
|
|
ref:
|
|
apiVersion: v1
|
|
kind: Namespace
|
|
name: conformance-gw-probe
|