Merge feature/vpn-egress-gateway-role into main (ADR-0015 host tunnel role, not yet deployed)

2026-08-24 07:31:27 -05:00
commit 33d048d379
@@ -0,0 +1,52 @@
 # group_vars for the `vpn_egress_gateway` node(s) — ADR-0015 host tunnel secrets
 The node-level VPN tunnel is **host infra**, so (per ADR-0015 / docs/07) its PIA credentials
 live **host-side in Ansible vault** here — NOT in a cluster SOPS Secret. Create an encrypted
 `vault.yml` in this directory (committed, encrypted; the passphrase file stays out of the repo,
 see `ansible.cfg` `vault_password_file`).
 ## Required vault keys (operator provides the VALUES — none are committed in plaintext)
 ```bash
 cd infra/ansible
 ansible-vault create inventory/group_vars/vpn_egress_gateway/vault.yml
 ```
 `vault.yml` must define exactly these three keys (the role maps them to its
 `vpn_egress_gateway_*` vars in `roles/vpn_egress_gateway/defaults/main.yml`):
 | Vault key | What it is |
 |---|---|
 | `vault_vpn_egress_gateway_pia_username` | PIA account username (for the `.ovpn`'s `auth-user-pass`) |
 | `vault_vpn_egress_gateway_pia_password` | PIA account password |
 | `vault_vpn_egress_gateway_pia_ovpn`     | The **full text** of the proven PIA `.ovpn` — the same `sweden-aes-128-cbc-udp-ip.ovpn` (PIA over OpenVPN-UDP, AES-128-CBC/SHA1/RSA-2048, IP-pinned Stockholm remote `158.173.241.x:1198/udp`) currently held in the k8s `pia-custom-ovpn` Secret. Use a YAML block scalar so the inline `<ca>`/`<crl-verify>` PEM survives verbatim. |
 Example skeleton (values are placeholders — DO NOT commit unencrypted):
 ```yaml
 vault_vpn_egress_gateway_pia_username: "pXXXXXXX"
 vault_vpn_egress_gateway_pia_password: "REDACTED"
 vault_vpn_egress_gateway_pia_ovpn: |
  client
  dev tun
  proto udp
  remote 158.173.241.36 1198
  cipher aes-128-cbc
  auth sha1
  auth-user-pass
  ...
  <ca>
  -----BEGIN CERTIFICATE-----
  ...
  -----END CERTIFICATE-----
  </ca>
 ```
 > To lift the exact `.ovpn` out of the existing cluster Secret:
 > `sops -d platform/vpn-egress/pia-custom-ovpn.sops.yaml` → the `sweden-aes-128-cbc-udp-ip.ovpn`
 > value is the text to paste under `vault_vpn_egress_gateway_pia_ovpn`.
 ## Notes
 - `vault.yml` is skipped by the yamllint gate (`ci/yamllint.yaml` ignores `**/vault.yml`).
 - This is the Ansible-bootstrap-layer secret (host credential), distinct from the platform's
  in-cluster SOPS + age secrets (ADR-0007) — different layer, different tool, by design (ADR-0015).
@@ -20,3 +20,10 @@ k3s_secondary
 # (sudo) password comes from the vaulted group_vars/hypervisor/vault.yml.
 [hypervisor]
 localhost ansible_connection=local
 # VPN egress-gateway node(s) — run the node-level split-tunnel OpenVPN + kill-switch
 # (ADR-0015, playbooks/vpn.egress.gateway.yml). Initially just k3s-dev-3 (single gateway;
 # add a second host here for Cilium Egress Gateway HA). Members are already defined above
 # with their ansible_host under k3s_secondary; this group only tags the egress role target.
 [vpn_egress_gateway]
 k3s-dev-3
@@ -0,0 +1,18 @@
 ---
 # Stand up the node-level VPN egress tunnel on the designated egress-gateway node(s)
 # (ADR-0015). Host infra — the split-tunnel OpenVPN service + policy routing + scoped
 # nftables kill-switch. The CiliumEgressGatewayPolicy that steers vpn-class pods to this
 # node's tun0 is authored separately as GitOps (platform/), against the same contract:
 #   * inventory group `vpn_egress_gateway`  (this play's target)
 #   * k8s node label   homelab.keithjackson.dev/egress-gateway=true  (set by the role)
 #   * host tunnel iface tun0                 (split-tunnel; never the node default route)
 #
 # Secrets come from Ansible vault (inventory/group_vars/vpn_egress_gateway/vault.yml):
 #   vault_vpn_egress_gateway_pia_username, vault_vpn_egress_gateway_pia_password, vault_vpn_egress_gateway_pia_ovpn
 # Run:  ansible-playbook playbooks/vpn.egress.gateway.yml
 - name: Configure the node-level VPN egress gateway
  hosts: vpn_egress_gateway
  become: true
  gather_facts: true          # package module + service facts need os facts
  roles:
    - role: vpn_egress_gateway
@@ -0,0 +1,101 @@
 # role: vpn_egress_gateway (ADR-0015)
 Stands up the **node-level VPN tunnel** on the egress-gateway node(s): a host **OpenVPN
 systemd service** (PIA over OpenVPN-UDP), **split-tunnel** via policy routing, and a
 **scoped nftables kill-switch**. This is the host half of ADR-0015; the
 `CiliumEgressGatewayPolicy` that steers `egress-class=vpn` pods to this node's `tun0` is
 separate GitOps authored against the shared **interface contract** below.
 > **CONFORMANCE IS GATED ON A LIVE TUNNEL.** This role cannot be proven green without the
 > operator's PIA host credentials (see "What requires a live tunnel"). Authoring + lint pass
 > in CI; the `egress-class` / `vpn-killswitch` conformance tests (R-NET-05/06/07) go green only
 > once the tunnel is live in dev/test.
 ## Interface contract (shared with the Cilium-policy agent)
 - **Inventory group** `vpn_egress_gateway` — the egress-gateway node(s) (initially `k3s-dev-3`).
 - **k8s node label** `homelab.keithjackson.dev/egress-gateway=true` — set by this role; the
  `CiliumEgressGatewayPolicy` selects the gateway node by it.
 - **Host tunnel iface** `tun0` — the Cilium policy SNATs vpn-class pods out of it.
 - **Optional fwmark** `0x1c8` — a contract *hook* (see kill-switch layering) the Cilium side
  MAY stamp on vpn-class packets for a tunnel-down-independent host backstop.
 ## How the four ADR-0015 invariants are met
 ### 1 & 2 — split-tunnel; tun0 is never a default route
 `templates/vpn-egress.conf.j2` includes the operator's proven `.ovpn` and layers on
 `route-noexec` + `pull-filter ignore "redirect-gateway"`, so **OpenVPN installs no routes and
 never touches the main routing table**. The node's own default route stays on the WAN whether
 the tunnel is up or down — k3s/etcd/kubelet are unaffected.
 Steering is done with **policy routing** in a dedicated table (`51820`, named `vpn_egress`):
 - The OpenVPN **up-script** adds `default dev tun0` to table `51820` and an ip-rule
  **`from <tun0-addr> lookup 51820`** (priority 5000). Only packets whose source is the tunnel
  address — i.e. the vpn-class pods Cilium has SNAT'd to `tun0` — consult that table. Nothing
  else does, so nothing else is re-routed.
 - The **down-script** removes the ip-rule and the tunnel default (leaving the blackhole).
 ### 3 & 4 — kill-switch scoped to vpn-class, opposite fail directions
 Two independent layers, both scoped so they can only ever affect vpn-class traffic:
 1. **Routing blackhole (fail-closed fallback).** Table `51820` always carries a permanent
   `blackhole default` (installed by `vpn-egress-killswitch.service`, metric 4000). When the
   tunnel default is withdrawn (flap / partial teardown) while the ip-rule still steers, packets
   hit the blackhole instead of falling through to the WAN.
 2. **nftables backstop (`templates/vpn-egress-killswitch.nft.j2`).** Own `inet
   vpn_egress_gateway_killswitch` table, base chain **policy `accept`** (so it can only *drop* the
   narrow matches, never break anything). In `postrouting`:
   - `oifname tun0 ip saddr @vpn_egress_gateway_src accept` — tunnel up: let steered traffic out.
   - `ip saddr @vpn_egress_gateway_src oifname != tun0 drop` — **fail closed:** a tunnel-sourced packet
     trying to leave any other interface is dropped (no WAN leak).
   - `meta mark 0x1c8 oifname != tun0 drop` — the optional fwmark backstop.
 `@vpn_egress_gateway_src` holds the **live tun0 address** (added by the up-script, flushed by the
 down-script). Node traffic and `internet`-class traffic are never in the set, never carry the
 mark, and are never targeted — so they always hit `policy accept` and **fail open toward the
 WAN** (invariant 4), tunnel up or down. The opposite directions are structural, not incidental.
 ## Why the node default route is provably unaffected (design-level verification)
 - OpenVPN runs with `route-noexec` and `pull-filter ignore "redirect-gateway"` → it adds **zero**
  routes and cannot install a `0.0.0.0/0` via `tun0`. (This is exactly the failure mode that made
  `hostNetwork` gluetun non-viable in ADR-0015; here OpenVPN is explicitly told not to route.)
 - All tunnel routing lives in **table 51820 only**; the up/down scripts touch that table and
  ip-rules, never `table main` / `table default`.
 - The ip-rule matches **`from <tun0-addr>`** — an address the node itself never sources traffic
  from — so node-originated and internet-class packets never enter the tunnel table.
 - The nft table is **policy accept** and matches only `@vpn_egress_gateway_src` / the fwmark.
 - Live check (post-deploy): `ip route show table main default` still points at the WAN NIC, and
  `curl ipinfo` from the node shows the WAN IP, with the tunnel both up and forced down.
 ## Kill-switch layering & ESCALATION (honest limit)
 - The nft `@vpn_egress_gateway_src` rule and the routing blackhole robustly cover the case where the
  tunnel is up but a vpn-class packet is (mis)routed toward the WAN, and flaps.
 - The **tunnel fully-down** case relies on **Cilium Egress Gateway itself failing closed** when
  its egress interface `tun0` is absent (Cilium 1.16 drops egress-gateway-marked traffic with no
  valid egress path rather than falling back to the node's default route) **plus** the routing
  blackhole. That is the ADR's expected behavior.
 - For a host backstop that is **fully independent of Cilium even in a fail-open regression** when
  the tunnel is down (no tun0 address to key on), the host needs a stable marker on vpn-class
  packets. The only contract-available marker is the **`0x1c8` fwmark hook** — which the
  *Cilium-policy* side must stamp. **ESCALATION for the PM:** either (a) confirm the Cilium
  Egress Gateway 1.16 fail-closed + routing-blackhole guarantee is accepted as sufficient for
  R-NET-06, or (b) add the shared fwmark to the interface contract so the CiliumEgressGatewayPolicy
  marks vpn-class traffic and this role's fwmark rule becomes the tunnel-down-independent backstop.
  This is flagged rather than worked around — no spec was weakened to reach done.
 ## What requires a LIVE tunnel to validate (cannot be tested without operator creds)
 1. The PIA `.ovpn` + username/password actually authenticate and `tun0` comes up as a host iface.
 2. `egress-class`: a vpn-class pod's public IP is the PIA Stockholm IP; internet-class shows the WAN IP.
 3. `vpn-killswitch`: forcing `tun0`/the tunnel service down blackholes vpn-class egress (no leak)
   while internet-class is unaffected — the R-NET-06 fault-injection gate.
 4. That Cilium Egress Gateway SNAT to `tun0` traverses host `postrouting` so the nft set-scoped
   rules actually see the packets (eBPF datapath interaction — see escalation).
 5. MTU/PMTU across `pod → tun0 → PIA` (in-pod gluetun needed 1300 for Cilium VXLAN; on the node
   the tunnel rides the physical WAN, so `vpn_egress_gateway_tun_mtu` defaults empty — verify live).
 6. Port forwarding (PIA NAT-PMP) for qBittorrent — ADR-0015 Phase-6 follow-up, not in this role.
 ## Files
 - `defaults/main.yml` — all `vpn_egress_gateway_*` vars; the three vault-backed credential vars.
 - `tasks/main.yml` — packages, rt_tables, config/scripts/units, enable+start, node label.
 - `handlers/main.yml` — daemon-reload, kill-switch reload, tunnel restart.
 - `templates/` — wrapper `.conf`, credentials, up/down scripts, nft ruleset, both systemd units.
 - Playbook: `playbooks/vpn.egress.gateway.yml`. Vault keys: `inventory/group_vars/vpn_egress_gateway/README.md`.
@@ -0,0 +1,66 @@
 ---
 # vpn_egress_gateway role defaults (ADR-0015).
 # Node-level split-tunnel host OpenVPN + policy routing + scoped nftables kill-switch.
 # All vars are role-name prefixed. NO secret VALUES live here — the three credential
 # vars below reference operator-provided Ansible-vault keys (see the role README and
 # inventory/group_vars/vpn_egress_gateway/README.md). They are intentionally NOT
 # defaulted so a missing vault fails loud rather than silently deploying a broken tunnel.
 # --- packages -------------------------------------------------------------------
 vpn_egress_gateway_packages:
  - openvpn
  - nftables
 # --- credentials / config (from Ansible vault — operator provides the values) ----
 # The PIA .ovpn PROVEN in Phase 3 (ADR-0013/0015): sweden-aes-128-cbc-udp-ip.ovpn —
 # PIA over OpenVPN-UDP, AES-128-CBC / SHA1 / RSA-2048, IP-pinned PIA Stockholm remote
 # (158.173.241.x:1198/udp), auth-user-pass. Same file that drove the working gluetun pod.
 vpn_egress_gateway_pia_username: "{{ vault_vpn_egress_gateway_pia_username }}"
 vpn_egress_gateway_pia_password: "{{ vault_vpn_egress_gateway_pia_password }}"
 # The FULL text of the PIA .ovpn (inline <ca>/<crl-verify>, cipher/auth, remote line).
 vpn_egress_gateway_pia_ovpn: "{{ vault_vpn_egress_gateway_pia_ovpn }}"
 # --- on-disk layout -------------------------------------------------------------
 vpn_egress_gateway_config_dir: /etc/openvpn/client
 vpn_egress_gateway_scripts_dir: /etc/openvpn/scripts
 vpn_egress_gateway_ovpn_path: "{{ vpn_egress_gateway_config_dir }}/pia.ovpn"
 vpn_egress_gateway_credentials_path: "{{ vpn_egress_gateway_config_dir }}/pia-credentials.txt"
 vpn_egress_gateway_wrapper_conf: "{{ vpn_egress_gateway_config_dir }}/vpn-egress.conf"
 # --- tunnel interface + split-tunnel policy routing (invariants 1 & 2) ----------
 # The host tunnel interface Cilium Egress Gateway steers vpn-class pods out of.
 vpn_egress_gateway_tun_iface: tun0
 # Dedicated policy-routing table for the tunnel (NEVER the main table — the node's
 # own default route stays on the WAN). Numeric id + a name added to rt_tables.
 vpn_egress_gateway_route_table_id: 51820
 vpn_egress_gateway_route_table_name: vpn_egress
 # ip-rule priority for the `from <tun0-ip> lookup vpn_egress` rule (well below main).
 vpn_egress_gateway_rule_priority: 5000
 # Route metrics inside the dedicated table: the live tunnel default is preferred;
 # a permanent blackhole is the fail-closed fallback during flaps / partial teardown.
 vpn_egress_gateway_tun_route_metric: 50
 vpn_egress_gateway_blackhole_metric: 4000
 # Optional tun MTU override. Empty => honor whatever the .ovpn/PIA negotiates. On the
 # NODE the tunnel rides the physical WAN (1500), so unlike the in-pod gluetun case
 # (Cilium VXLAN 1450 → needed 1300) no shrink is required by default.
 vpn_egress_gateway_tun_mtu: ""
 # --- nftables kill-switch (invariants 3 & 4 — scoped to vpn-class ONLY) ----------
 vpn_egress_gateway_nft_table: vpn_egress_gateway_killswitch
 # Named set holding the LIVE tun0 source address (maintained by the up/down scripts).
 # The kill-switch matches ONLY packets SNAT'd to this address by Cilium Egress Gateway
 # — i.e. exactly the egress-class=vpn traffic. Node / internet-class traffic is never
 # in this set and is never matched (fails OPEN toward the WAN).
 vpn_egress_gateway_nft_src_set: vpn_egress_gateway_src
 # Contract-hook fwmark: if the CiliumEgressGatewayPolicy side stamps vpn-class packets
 # with this mark, the kill-switch also drops marked packets not leaving via tun0 — an
 # extra, tunnel-down-independent backstop. See README "Kill-switch layering / escalation".
 vpn_egress_gateway_fwmark: "0x1c8"
 # --- kubernetes node label (steering contract) ----------------------------------
 # The label the parallel CiliumEgressGatewayPolicy selects the gateway node by.
 vpn_egress_gateway_node_label_key: homelab.keithjackson.dev/egress-gateway
 vpn_egress_gateway_node_label_value: "true"
 # k3s ships its kubeconfig here; the gateway node is a k3s server so `k3s kubectl` works.
 vpn_egress_gateway_kubeconfig: /etc/rancher/k3s/k3s.yaml
@@ -0,0 +1,14 @@
 ---
 - name: Reload systemd
  ansible.builtin.systemd_service:
    daemon_reload: true
 - name: Reload vpn-egress kill-switch
  ansible.builtin.systemd_service:
    name: vpn-egress-killswitch.service
    state: restarted
 - name: Restart vpn-egress tunnel
  ansible.builtin.systemd_service:
    name: vpn-egress-tunnel.service
    state: restarted
@@ -0,0 +1,139 @@
 ---
 # vpn_egress_gateway — node-level split-tunnel PIA OpenVPN + policy routing + scoped
 # nftables kill-switch on the egress-gateway node(s) (ADR-0015). Host infra, Ansible-
 # managed like k3s itself; the CiliumEgressGatewayPolicy that steers pods is separate GitOps.
 - name: Install OpenVPN and nftables
  ansible.builtin.package:
    name: "{{ vpn_egress_gateway_packages }}"
    state: present
 - name: Register the dedicated policy-routing table name
  ansible.builtin.lineinfile:
    path: /etc/iproute2/rt_tables
    line: "{{ vpn_egress_gateway_route_table_id }} {{ vpn_egress_gateway_route_table_name }}"
    regexp: "^{{ vpn_egress_gateway_route_table_id }}\\s"
    state: present
    create: true
    mode: "0644"
 - name: Ensure the OpenVPN config and scripts directories exist
  ansible.builtin.file:
    path: "{{ item }}"
    state: directory
    owner: root
    group: root
    mode: "0750"
  loop:
    - "{{ vpn_egress_gateway_config_dir }}"
    - "{{ vpn_egress_gateway_scripts_dir }}"
 - name: Deploy the operator's proven PIA .ovpn (from Ansible vault)
  ansible.builtin.copy:
    content: "{{ vpn_egress_gateway_pia_ovpn }}"
    dest: "{{ vpn_egress_gateway_ovpn_path }}"
    owner: root
    group: root
    mode: "0600"
  no_log: true
  notify: Restart vpn-egress tunnel
 - name: Deploy the PIA auth-user-pass credentials (from Ansible vault)
  ansible.builtin.template:
    src: pia-credentials.txt.j2
    dest: "{{ vpn_egress_gateway_credentials_path }}"
    owner: root
    group: root
    mode: "0600"
  no_log: true
  notify: Restart vpn-egress tunnel
 - name: Deploy the split-tunnel OpenVPN wrapper config
  ansible.builtin.template:
    src: vpn-egress.conf.j2
    dest: "{{ vpn_egress_gateway_wrapper_conf }}"
    owner: root
    group: root
    mode: "0600"
  notify: Restart vpn-egress tunnel
 - name: Deploy the split-tunnel up/down policy-routing scripts
  ansible.builtin.template:
    src: "{{ item }}.j2"
    dest: "{{ vpn_egress_gateway_scripts_dir }}/{{ item }}"
    owner: root
    group: root
    mode: "0750"
  loop:
    - vpn-egress-up.sh
    - vpn-egress-down.sh
  notify: Restart vpn-egress tunnel
 - name: Deploy the scoped nftables kill-switch ruleset
  ansible.builtin.template:
    src: vpn-egress-killswitch.nft.j2
    dest: "{{ vpn_egress_gateway_config_dir }}/vpn-egress-killswitch.nft"
    owner: root
    group: root
    mode: "0600"
  notify: Reload vpn-egress kill-switch
 - name: Deploy the kill-switch systemd unit
  ansible.builtin.template:
    src: vpn-egress-killswitch.service.j2
    dest: /etc/systemd/system/vpn-egress-killswitch.service
    owner: root
    group: root
    mode: "0644"
  notify:
    - Reload systemd
    - Reload vpn-egress kill-switch
 - name: Deploy the tunnel systemd unit
  ansible.builtin.template:
    src: vpn-egress-tunnel.service.j2
    dest: /etc/systemd/system/vpn-egress-tunnel.service
    owner: root
    group: root
    mode: "0644"
  notify:
    - Reload systemd
    - Restart vpn-egress tunnel
 - name: Flush handlers so units/routing exist before enable+start
  ansible.builtin.meta: flush_handlers
 - name: Enable and start the kill-switch (must be up before the tunnel)
  ansible.builtin.systemd_service:
    name: vpn-egress-killswitch.service
    enabled: true
    state: started
 - name: Enable and start the split-tunnel VPN service
  ansible.builtin.systemd_service:
    name: vpn-egress-tunnel.service
    enabled: true
    state: started
 # --- steering contract: label the k8s node the Cilium Egress Gateway policy selects ----
 - name: Read the current egress-gateway node label
  ansible.builtin.command:
    cmd: >-
      k3s kubectl get node {{ inventory_hostname }}
      -o jsonpath={.metadata.labels.{{ vpn_egress_gateway_node_label_key | replace('.', '\\.') | replace('/', '\\/') }}}
  environment:
    KUBECONFIG: "{{ vpn_egress_gateway_kubeconfig }}"
  register: vpn_egress_gateway_label_current
  changed_when: false
  failed_when: false
 - name: Label this node as the VPN egress gateway
  ansible.builtin.command:
    cmd: >-
      k3s kubectl label node {{ inventory_hostname }}
      {{ vpn_egress_gateway_node_label_key }}={{ vpn_egress_gateway_node_label_value }} --overwrite
  environment:
    KUBECONFIG: "{{ vpn_egress_gateway_kubeconfig }}"
  register: vpn_egress_gateway_label_apply
  changed_when: "'not labeled' not in vpn_egress_gateway_label_apply.stdout"
  when: vpn_egress_gateway_label_current.stdout | default('') != vpn_egress_gateway_node_label_value
@@ -0,0 +1,2 @@
 {{ vpn_egress_gateway_pia_username }}
 {{ vpn_egress_gateway_pia_password }}
@@ -0,0 +1,29 @@
 #!/usr/bin/env bash
 # {{ ansible_managed }}
 # OpenVPN --down hook (with `down-pre`, runs BEFORE tun0 is destroyed). Tears down the
 # split-tunnel steering so vpn-class traffic FAILS CLOSED, never leaks to the WAN
 # (ADR-0015 invariants 3 & 4).
 #
 # Order matters: (1) empty the kill-switch source set and drop the ip-rule so no packet
 # can be routed toward a vanishing tun0, then (2) remove the tunnel default from the
 # dedicated table. The permanent `blackhole default` in that table is LEFT in place as
 # the fail-closed fallback. The node's main-table WAN default is never touched here.
 set -uo pipefail
 TABLE_ID="{{ vpn_egress_gateway_route_table_id }}"
 RULE_PRIO="{{ vpn_egress_gateway_rule_priority }}"
 NFT_TABLE="{{ vpn_egress_gateway_nft_table }}"
 NFT_SET="{{ vpn_egress_gateway_nft_src_set }}"
 IFACE="${dev:-{{ vpn_egress_gateway_tun_iface }}}"
 SRC="${ifconfig_local:-}"
 # 1) Stop scoping/steering to the (now dead) tunnel source.
 nft flush set inet "${NFT_TABLE}" "${NFT_SET}" 2>/dev/null || true
 if [ -n "${SRC}" ]; then
  while ip rule del from "${SRC}" lookup "${TABLE_ID}" priority "${RULE_PRIO}" 2>/dev/null; do :; done
 fi
 # 2) Remove the tunnel default from the dedicated table; keep the blackhole fallback.
 ip route del default dev "${IFACE}" table "${TABLE_ID}" 2>/dev/null || true
 echo "vpn-egress-down: steering removed; vpn-class fails closed (blackhole retained)"
@@ -0,0 +1,40 @@
 #!/usr/sbin/nft -f
 # {{ ansible_managed }}
 # ADR-0015 kill-switch (invariants 3 & 4) — SCOPED TO vpn-class ONLY, fails CLOSED.
 #
 # Own dedicated table so re-loading it never disturbs Cilium's or the node's own rules.
 # The base chains are policy ACCEPT: this table can ONLY ever drop the narrow vpn-class
 # traffic matched below; node traffic and internet-class traffic are untouched and keep
 # egressing the WAN (fail OPEN — invariant 4). It matches on:
 #   * @{{ vpn_egress_gateway_nft_src_set }} — the LIVE tun0 address Cilium Egress Gateway SNATs
 #     vpn-class pods to (populated by the OpenVPN up-script; emptied by the down-script).
 #     This IS the egress-class=vpn traffic and nothing else.
 #   * meta mark {{ vpn_egress_gateway_fwmark }} — optional contract-hook: a tunnel-DOWN-independent
 #     backstop, active only if the CiliumEgressGatewayPolicy side stamps this fwmark.
 table inet {{ vpn_egress_gateway_nft_table }} {
    # Live tunnel source address(es). Empty whenever the tunnel is down => the two
    # saddr rules below match nothing => vpn-class has no path at all (fail closed).
    set {{ vpn_egress_gateway_nft_src_set }} {
        type ipv4_addr
    }
    chain postrouting {
        # oifname is resolved at postrouting; SNAT/masquerade has already applied here,
        # so the Cilium-SNAT'd vpn-class packet's saddr is the tunnel address.
        type filter hook postrouting priority 100; policy accept;
        # Tunnel UP: let the steered vpn-class traffic out tun0.
        oifname "{{ vpn_egress_gateway_tun_iface }}" ip saddr @{{ vpn_egress_gateway_nft_src_set }} accept
        # FAIL CLOSED: a vpn-class (tunnel-sourced) packet trying to leave ANY other
        # interface (i.e. the WAN) is dropped — never leaks. Scoped by the set, so it
        # can never match node or internet-class traffic.
        ip saddr @{{ vpn_egress_gateway_nft_src_set }} oifname != "{{ vpn_egress_gateway_tun_iface }}" drop
        # Tunnel-down-independent backstop (contract hook, see role README): if vpn-class
        # packets carry the agreed fwmark, drop any not leaving via tun0 even when the set
        # is empty (tunnel fully down). Harmless if the Cilium side does not set the mark.
        meta mark {{ vpn_egress_gateway_fwmark }} oifname != "{{ vpn_egress_gateway_tun_iface }}" drop
    }
 }
@@ -0,0 +1,25 @@
 # {{ ansible_managed }}
 # Installs the ADR-0015 kill-switch state that must exist INDEPENDENTLY of (and BEFORE)
 # the tunnel: the scoped nftables table/set and the permanent fail-closed blackhole route
 # in the dedicated policy-routing table. Ordered before the tunnel so the nft set exists
 # when the OpenVPN up-script registers the live tunnel address.
 [Unit]
 Description=VPN egress gateway kill-switch (nftables + fail-closed blackhole) — ADR-0015
 Documentation=file://{{ vpn_egress_gateway_config_dir }}/README
 After=network-pre.target nftables.service
 Wants=network-pre.target
 Before=vpn-egress-tunnel.service
 [Service]
 Type=oneshot
 RemainAfterExit=yes
 # Load the scoped kill-switch table (idempotent: delete-then-add).
 ExecStart=/usr/sbin/nft -f {{ vpn_egress_gateway_config_dir }}/vpn-egress-killswitch.nft
 # Ensure the dedicated table has a permanent blackhole default (fail closed) even before
 # the tunnel first connects. The tunnel up-script adds the preferred `default dev tun0`.
 ExecStart=/sbin/ip route replace blackhole default table {{ vpn_egress_gateway_route_table_id }} metric {{ vpn_egress_gateway_blackhole_metric }}
 # Tear down cleanly: remove the whole scoped table (never touches other nft rules).
 ExecStop=/usr/sbin/nft delete table inet {{ vpn_egress_gateway_nft_table }}
 [Install]
 WantedBy=multi-user.target
@@ -0,0 +1,25 @@
 # {{ ansible_managed }}
 # Node-level host OpenVPN tunnel (ADR-0015): PIA over OpenVPN-UDP, split-tunnel. Creates
 # {{ vpn_egress_gateway_tun_iface }} but — via the wrapper's route-noexec / pull-filter — NEVER sets
 # the node default route. The node (k3s/etcd/kubelet) keeps working whether this is up or down.
 [Unit]
 Description=VPN egress gateway tunnel (PIA OpenVPN-UDP, split-tunnel) — ADR-0015
 Documentation=file://{{ vpn_egress_gateway_config_dir }}/README
 After=network-online.target vpn-egress-killswitch.service
 Wants=network-online.target
 Requires=vpn-egress-killswitch.service
 [Service]
 Type=notify
 # openvpn signals readiness to systemd once the tunnel is up.
 ExecStart=/usr/sbin/openvpn --suppress-timestamps --nobind --config {{ vpn_egress_gateway_wrapper_conf }}
 # The up/down scripts need CAP_NET_ADMIN to manage tun0, ip rules/routes and the nft set.
 CapabilityBoundingSet=CAP_NET_ADMIN CAP_NET_RAW CAP_SETPCAP CAP_DAC_OVERRIDE
 AmbientCapabilities=CAP_NET_ADMIN CAP_NET_RAW
 Restart=on-failure
 RestartSec=5
 # Not the node default route, so a tunnel crash must NOT cascade to the node — just retry.
 KillMode=process
 [Install]
 WantedBy=multi-user.target
@@ -0,0 +1,43 @@
 #!/usr/bin/env bash
 # {{ ansible_managed }}
 # OpenVPN --up hook. Runs as root AFTER tun0 is created, with the tunnel's dynamic
 # address available in $ifconfig_local (and $dev = the interface, forced to tun0).
 #
 # Installs the SPLIT-TUNNEL policy routing (ADR-0015 invariant 2): only traffic whose
 # source is the tunnel address (i.e. the vpn-class pods Cilium Egress Gateway SNATs to
 # tun0) is routed via the dedicated table out tun0. The node's own default route in the
 # MAIN table is never touched.
 set -euo pipefail
 TABLE_ID="{{ vpn_egress_gateway_route_table_id }}"
 RULE_PRIO="{{ vpn_egress_gateway_rule_priority }}"
 TUN_METRIC="{{ vpn_egress_gateway_tun_route_metric }}"
 BH_METRIC="{{ vpn_egress_gateway_blackhole_metric }}"
 NFT_TABLE="{{ vpn_egress_gateway_nft_table }}"
 NFT_SET="{{ vpn_egress_gateway_nft_src_set }}"
 IFACE="${dev:-{{ vpn_egress_gateway_tun_iface }}}"
 SRC="${ifconfig_local:-}"
 if [ -z "${SRC}" ]; then
  echo "vpn-egress-up: ifconfig_local not set; refusing to install partial routing" >&2
  exit 1
 fi
 # Preferred default via the tunnel inside the DEDICATED table (not the main table).
 ip route replace default dev "${IFACE}" table "${TABLE_ID}" metric "${TUN_METRIC}"
 # Permanent fail-closed fallback: if the tunnel default is ever withdrawn while the
 # ip-rule still steers traffic here (flap / partial teardown), packets blackhole
 # instead of falling through to the main table's WAN default (invariant 4: fail closed).
 ip route replace blackhole default table "${TABLE_ID}" metric "${BH_METRIC}"
 # Steer ONLY tunnel-sourced (Cilium-SNAT'd vpn-class) packets into the dedicated table.
 # Idempotent: delete any stale copy of this exact rule first.
 while ip rule del from "${SRC}" lookup "${TABLE_ID}" priority "${RULE_PRIO}" 2>/dev/null; do :; done
 ip rule add from "${SRC}" lookup "${TABLE_ID}" priority "${RULE_PRIO}"
 # Register the live tunnel source with the kill-switch set so the nft backstop can scope
 # to exactly this traffic. Flush any stale members first (address changes on reconnect).
 nft flush set inet "${NFT_TABLE}" "${NFT_SET}" 2>/dev/null || true
 nft add element inet "${NFT_TABLE}" "${NFT_SET}" "{ ${SRC} }"
 echo "vpn-egress-up: tun=${IFACE} src=${SRC} table=${TABLE_ID} — split-tunnel routing active"
@@ -0,0 +1,45 @@
 # {{ ansible_managed }}
 # Split-tunnel wrapper for the node-level PIA egress tunnel (ADR-0015, invariants 1 & 2).
 # It INCLUDES the operator's proven PIA .ovpn and layers the split-tunnel directives on top,
 # so the same file that drove the working gluetun pod is reused verbatim (only the packaging
 # changes: pod -> host systemd service).
 # Pull in the operator's PIA .ovpn (inline <ca>/<crl-verify>, cipher aes-128-cbc, auth sha1,
 # remote 158.173.241.x 1198 udp, auth-user-pass). Directives AFTER this include augment/override it.
 config {{ vpn_egress_gateway_ovpn_path }}
 # Force a deterministic interface name so the Cilium Egress Gateway policy + kill-switch can
 # reference it by name. (The .ovpn's own `dev tun` is overridden here.)
 dev {{ vpn_egress_gateway_tun_iface }}
 dev-type tun
 # --- SPLIT-TUNNEL: create tun0 but NEVER become the node default route (invariant 2) ---------
 # route-noexec: OpenVPN installs NO routes itself (no `redirect-gateway`, no pushed routes).
 route-noexec
 # Belt-and-suspenders: even if the server pushes redirect-gateway, drop it before it is applied.
 pull-filter ignore "redirect-gateway"
 # Do not let the VPN rewrite the NODE's DNS — the node/kubelet keep the local resolver.
 pull-filter ignore "dhcp-option DNS"
 pull-filter ignore "dhcp-option DOMAIN"
 # Credentials for the .ovpn's `auth-user-pass` (rendered from Ansible vault, mode 0600).
 auth-user-pass {{ vpn_egress_gateway_credentials_path }}
 auth-nocache
 # The dedicated policy-routing table + ip-rule + nft set membership are installed by the
 # up/down scripts (they receive tun0's dynamic address + gateway from OpenVPN's env).
 script-security 2
 up {{ vpn_egress_gateway_scripts_dir }}/vpn-egress-up.sh
 down {{ vpn_egress_gateway_scripts_dir }}/vpn-egress-down.sh
 # Run the down script BEFORE tearing the tunnel down so it can clean up while tun0 still exists.
 down-pre
 persist-tun
 {% if vpn_egress_gateway_tun_mtu | length > 0 %}
 tun-mtu {{ vpn_egress_gateway_tun_mtu }}
 {% endif %}
 # Keep the tunnel resilient; systemd also restarts the unit on hard failure.
 keepalive 10 60
 resolv-retry infinite
 nobind
 verb 3
@@ -1,88 +0,0 @@
 ---
 # ============================================================================
 # Cilium Egress Gateway steering for `egress-class=vpn` pods — ADR-0015.
 # Serves: R-NET-05, R-NET-06, R-NET-07.
 #
 # ##########################################################################
 # ##  NOT WIRED INTO THE RUNTIME YET — DO NOT ADD TO kustomization.yaml   ##
 # ##  UNTIL THE HOST tun0 TUNNEL IS LIVE AND THIS DATAPATH IS VALIDATED.  ##
 # ##########################################################################
 #
 # This policy steers vpn-class pods out a gateway node's host `tun0` interface. It is INERT and
 # potentially disruptive until the Ansible-managed host OpenVPN split-tunnel service actually exists
 # on a node labeled `homelab.keithjackson.dev/egress-gateway=true`. If Flux shipped this before the
 # tunnel is up, vpn-class pods would be SNAT'd to a non-existent interface — an unvalidated datapath.
 # Per platform/vpn-egress/README.md: "the steering datapath is intentionally NOT committed [to the
 # runtime kustomization] until the mechanism is chosen + validated." ADR-0015 chose the mechanism
 # (node tunnel + Cilium Egress Gateway); this file IS that datapath, authored but held out of the
 # resource list for the PM to enable after a live-tunnel validation (see TODO at the bottom).
 #
 # ---------------------------------------------------------------------------
 # PREREQUISITES CONFIRMED (2026-08-24):
 #   * Cilium Egress Gateway is ENABLED per ADR-0002 — platform/cilium/helmrelease.yaml and
 #     platform/cilium/values.yaml both set (all [parity] with the Ansible bootstrap render):
 #         egressGateway.enabled: true
 #         bpf.masquerade:        true   (Egress Gateway REQUIRES eBPF masquerade)
 #         kubeProxyReplacement:  true
 #     No Cilium HelmRelease change is needed for this policy. (If any of the three were false this
 #     would be an ESCALATION, not a silent HelmRelease edit — they are all true, so we proceed.)
 #
 # INTERFACE CONTRACT (shared verbatim with the parallel Ansible host-tunnel role — ADR-0015):
 #   * Gateway node label ...... homelab.keithjackson.dev/egress-gateway=true   (applied by Ansible)
 #   * Host tunnel interface .... tun0                                          (split-tunnel OpenVPN)
 #   * Client selection ......... homelab.keithjackson.dev/egress-class=vpn     (Kyverno-enforced)
 # ---------------------------------------------------------------------------
 apiVersion: cilium.io/v2
 kind: CiliumEgressGatewayPolicy
 metadata:
  name: vpn-egress-steering
  annotations:
    homelab.keithjackson.dev/requirements: R-NET-05,R-NET-06,R-NET-07
    homelab.keithjackson.dev/adr: ADR-0015
 spec:
  # --- Client selection (INVARIANTS 1 & 2) --------------------------------
  # ONLY pods explicitly opted into the vpn class are matched. Default/internet-class pods and node
  # traffic are never selected, so they stay on the local/private-WAN path (invariant 1: default is
  # the local path). Because ONLY these flows are steered, tun0 never becomes a default route for the
  # node or for any other pod (invariant 2). The selector is intentionally cluster-wide (no namespace
  # scope) so the class label alone is the contract — every namespace's vpn-class pods route here.
  selectors:
    - podSelector:
        matchLabels:
          homelab.keithjackson.dev/egress-class: vpn
  # --- What gets tunneled (INVARIANT 1: only PUBLIC internet leaves via the VPN) ------------------
  # destinationCIDRs is the full internet; excludedCIDRs carves out everything that must stay LOCAL:
  # all RFC1918 private ranges (which also cover the Cilium cluster-pool pod CIDR 10.0.0.0/8, the k3s
  # service/pod ranges, and the LAN 192.168.x/10.73.x node network). So intra-cluster east-west,
  # pod->service, and LAN traffic are NOT SNAT'd out tun0 — only genuinely public destinations are.
  # Excluded flows fall through to Cilium's normal local datapath (invariant 1).
  destinationCIDRs:
    - "0.0.0.0/0"
  excludedCIDRs:
    - "10.0.0.0/8"       # RFC1918 + Cilium cluster-pool pod CIDR (CLUSTER_POD_CIDR) + k3s service net
    - "172.16.0.0/12"    # RFC1918
    - "192.168.0.0/16"   # RFC1918 (home LAN / node network family)
  # --- Where it egresses --------------------------------------------------
  # The gateway node (Ansible-labeled) SNATs matched traffic out its host split-tunnel `tun0`. When
  # tun0 is absent/down, Cilium has no egress interface here; the host nftables kill-switch (Ansible,
  # ADR-0015) is the authoritative fail-closed backstop that DROPS this vpn-class traffic — scoped to
  # vpn-class only, so default/internet-class is unaffected (invariants 3 & 4: vpn fails CLOSED,
  # internet fails OPEN).
  egressGateway:
    nodeSelector:
      matchLabels:
        homelab.keithjackson.dev/egress-gateway: "true"
    interface: tun0
 # ---------------------------------------------------------------------------
 # TODO(PM — enable after live validation, then retire Gluetun):
 #   1. Confirm the Ansible host-tunnel role has brought `tun0` up on a node labeled
 #      homelab.keithjackson.dev/egress-gateway=true, and the host nftables kill-switch is in place.
 #   2. Add `- egress-steering.yaml` to platform/vpn-egress/kustomization.yaml so Flux ships it.
 #   3. Run the `egress-class` conformance test: vpn-class pod shows the PIA IP, internet-class shows
 #      the WAN IP, and a git label flip re-routes (R-NET-05/07).
 #   4. Run the `vpn-killswitch` conformance test: force tun0 down => vpn-class blackholed (no leak),
 #      internet-class unaffected (R-NET-06).
 #   5. Only AFTER both pass green: retire the Gluetun Deployment (remove gluetun-gateway.yaml from
 #      kustomization.yaml) per ADR-0015 ("Gluetun is dropped as the gateway"). Do NOT retire it in
 #      the same change that enables this — keep the validated fallback until steering is proven.
 # ---------------------------------------------------------------------------
		`@@ -0,0 +1,2 @@`
							`{{ vpn_egress_gateway_pia_username }}`
							`{{ vpn_egress_gateway_pia_password }}`