home-network-k3s-impl/infra/ansible/roles/unattended_upgrades/defaults/main.yml

---
# roles/unattended_upgrades — automatic Debian SECURITY patching on the k3s nodes, with the kernel
# deliberately excluded and with reboots delegated to kured (platform/kured).
#
# OFF BY DEFAULT, ON FOR PROD. Same seam as node_baseline_backports_kernel / etcd_io_priority_enabled:
# a plain boolean, default false, turned on in inventory/group_vars/k3s_prod/vars.yml. Not a hostname
# test — that would encode the environment into the role (see node_baseline's defaults for the full
# argument, which this role follows rather than restates).
#
# ═════════════════════════════════════════════════════════════════════════════════════════════════
# STATE BEFORE THIS ROLE (measured on all three prod nodes, 2026-10-17, via `dpkg-query -W`)
# ═════════════════════════════════════════════════════════════════════════════════════════════════
#   unattended-upgrades      NOT INSTALLED
#   update-notifier-common   NOT INSTALLED
#   needrestart              NOT INSTALLED
#   apt-mark showhold        EMPTY (no package is held on any node)
#   /var/run/reboot-required ABSENT, and nothing on the node can create it
#   /etc/apt/preferences.d/  EMPTY (no apt pin declares the backports-kernel intent)
# So: zero automatic patching, and no mechanism by which a reboot could ever be signalled. That last
# point is why this role and platform/kured must land together — kured watches
# /var/run/reboot-required, and on these nodes today NOTHING WRITES IT. Deploying kured alone would
# produce a daemon that is permanently, silently idle while looking perfectly healthy.
#
# ═════════════════════════════════════════════════════════════════════════════════════════════════
# DECISION 1 — SECURITY ONLY, NOT FULL UPGRADES
# ═════════════════════════════════════════════════════════════════════════════════════════════════
# Measured `apt-cache policy` on k3s-prod-1, 2026-10-17, gives the four archives and their pins:
#   trixie            o=Debian,            a=stable,            l=Debian          pin 500
#   trixie-updates    o=Debian,            a=stable-updates,    l=Debian          pin 500
#   trixie-security   o=Debian,            a=stable-security,   l=Debian-Security pin 500
#   trixie-backports  o=Debian Backports,  a=stable-backports,  l=Debian Backports pin 100
#
# Only `Debian-Security` is allowed below. The reasoning, in order of weight:
#
#   * Security updates are the class where the cost of NOT applying them is unbounded and the
#     change itself is minimal by Debian policy (targeted fixes against the frozen stable version,
#     no feature churn). It is the one class worth applying with nobody watching.
#   * A full `apt upgrade` on these nodes would also pull trixie point-release changes, which is a
#     larger surface than anyone has reviewed for a 3-node etcd cluster, for no urgency.
#   * `trixie-updates` is DELIBERATELY EXCLUDED, not overlooked. It carries non-security fixes that
#     are too urgent to wait for a point release — useful, but it is the archive through which an
#     unreviewed change would most plausibly arrive, and it is not what "unattended" should mean on
#     a control-plane node. Add it consciously later if the operator wants it; the knob is below.
#   * `trixie-backports` is EXCLUDED, AND THAT IS THE KERNEL ARGUMENT — see DECISION 2.
#
# ═════════════════════════════════════════════════════════════════════════════════════════════════
# DECISION 2 — THE KERNEL IS EXCLUDED TWICE OVER. THIS IS THE HIGHEST-RISK PART OF THE CHANGE.
# ═════════════════════════════════════════════════════════════════════════════════════════════════
# The constraint: HP's R21 firmware on these EliteDesk 800 G5 minis publishes a corrupt EFI Memory
# Attributes Table, and Linux < 6.13 does not validate it before reserving memory from its contents
# — k3s-prod-2 twice booted with 2.6 GiB of 16 GB. Upstream 8fbe4c49c0cc (first in v6.13) is the
# ONLY fix; there is no kernel cmdline workaround. So these nodes run the backports metapackage
# `linux-image-amd64` (measured 2026-10-17: version 7.1.8-1~bpo13+1 installed and running on all
# three, well above the floor). Full write-up: infra/PROD-PROVISIONING.md §0b, and the rationale
# block in roles/node_baseline/defaults/main.yml:21-51.
#
# A NODE THAT COMES BACK FROM AN UNATTENDED UPGRADE ON A 6.12 KERNEL IS A NODE THAT MAY COME BACK
# WITH A FRACTION OF ITS RAM. Note that 6.12 kernels are not hypothetical here — measured
# 2026-10-17, `linux-image-6.12.107+deb13-amd64` and `linux-image-6.12.94+deb13-amd64` are BOTH
# still installed on all three nodes, below the floor, bootable, and nothing holds them.
#
# Two independent mechanisms keep unattended-upgrades away from the kernel, and the role asserts
# both rather than trusting either:
#
#   1. ORIGIN EXCLUSION (structural). The running kernel comes from `o=Debian Backports`, which is
#      not in Allowed-Origins. u-u only ever considers packages from allowed origins, so it cannot
#      see a backports kernel upgrade at all. This also means the backports kernel receives no
#      automatic patching — accepted, and in any case backports carries no security guarantees
#      (node_baseline's defaults state that cost plainly: "no security-tracker commitment, no DSAs").
#   2. PACKAGE BLACKLIST (belt and braces). `linux-*` is blacklisted below, so even if the origin
#      list is later widened — the most likely future edit to this file, and the one that would
#      silently re-arm the hazard — the kernel is still untouchable. Without this, widening origins
#      to include backports would hand an unattended process the EFI-memattr-critical package.
#
# WHAT THIS DOES *NOT* DO, AND MUST NOT BE READ AS DOING: it does not stop the kernel moving. The
# operator still advances it deliberately with
#   ansible-playbook playbooks/node-baseline.yml -e k3s_hosts=k3s_prod --tags kernel --limit <host>
# which is the only thing that maintains the backports source, the metapackage and the on-node
# provenance stamp. node_baseline's own comment ("`ButAutomaticUpgrades: yes` ... means `apt upgrade`
# keeps it current once installed") still holds for an operator-run apt; the measured pin priority of
# 100 is exactly the NotAutomatic + ButAutomaticUpgrades signature that produces that behaviour.
# Nothing here changes it. What changes is that a NEW unattended actor is forbidden from the kernel.
#
# NOT BLACKLISTED, BECAUSE THEY ARE NOT APT PACKAGES AT ALL — measured 2026-10-17: `dpkg-query` finds
# no `k3s`, no `containerd` and no `containerd.io`. k3s is a single static binary at
# /usr/local/bin/k3s installed by roles/k3s_server, and it bundles its own containerd and kubelet.
# So the entire Kubernetes control plane and container runtime are OUTSIDE apt's reach and cannot be
# upgraded by this role even in principle. That is worth knowing before worrying about it.
unattended_upgrades_enabled: false

# The packages this role installs. Each one is load-bearing; see the notes.
unattended_upgrades_packages:
  # The upgrader itself.
  - unattended-upgrades
  # NOTE: `update-notifier-common` was listed here and REMOVED (2026-10-18). It is an UBUNTU
  # package and does not exist in Debian at all -- `apt-cache policy update-notifier-common` on
  # trixie returns `Candidate: (none)`, so the play failed outright with "No package matching
  # 'update-notifier-common' is available" on all three nodes. The sentinel writer is now provided
  # by this role itself; see unattended_upgrades_sentinel_* below and tasks/main.yml.
  # Restarts services still mapped to a deleted library after a security upgrade. Without it a
  # security-only policy patches bytes on disk that no running process reads — see DECISION 3.
  - needrestart

# --- Allowed origins -----------------------------------------------------------------------------
# Verbatim apt origin expressions, substituted into 50unattended-upgrades. The ${distro_*} tokens
# here are UNATTENDED-UPGRADES' OWN placeholders, expanded by it at runtime — they are not Ansible
# or Flux substitutions, which is why they are single-quoted and why this role's files never travel
# through Flux envsubst.
#
# `l=Debian-Security` plus the `-security` codename is the form Debian's own shipped config uses, and
# both halves were confirmed against the measured archive metadata above (l=Debian-Security,
# n=trixie-security). Matching on the LABEL rather than the suite name is what makes this survive the
# next Debian release.
unattended_upgrades_allowed_origins:
  - 'origin=Debian,codename=${distro_codename},label=Debian-Security'
  - 'origin=Debian,codename=${distro_codename}-security,label=Debian-Security'
# Set this to add 'origin=Debian,codename=${distro_codename}-updates,label=Debian' — a deliberate
# widening, kept as a knob so that turning it on is a reviewable one-line change rather than an edit
# to a heredoc. See DECISION 1 for why it is off.
unattended_upgrades_include_updates: false

# --- Package blacklist ---------------------------------------------------------------------------
# Regular expressions matched against package names. See DECISION 2: `linux-` is the EFI-memattr
# guard and must not be removed without re-reading PROD-PROVISIONING.md §0b.
unattended_upgrades_blacklist:
  # Every kernel package: linux-image-*, linux-headers-*, linux-image-amd64 (the metapackage that
  # actually tracks backports), plus linux-kbuild / linux-libc-dev. Anchored at the start so it
  # cannot accidentally match an unrelated package containing the substring.
  - '^linux-'
  # Microcode changes take effect only on reboot and can alter boot behaviour on the very firmware
  # that is already misbehaving here. Operator-only, same argument as the kernel.
  - '^intel-microcode$'
  - '^amd64-microcode$'
  # The bootloader. A grub upgrade rewrites boot configuration on a machine with a documented
  # warm-reboot hang (k3s-prod-2, 2026-09-03). Never unattended.
  - '^grub-'
  - '^grub2-common$'

# --- Reboot policy -------------------------------------------------------------------------------
# FALSE, AND THIS IS THE DIVISION OF LABOUR WITH kured, NOT AN OVERSIGHT.
# unattended-upgrades can reboot a node itself (Automatic-Reboot "true"). It must not here: it has
# no notion of a cluster, so it would not cordon, would not drain, would not evict the 25 attached
# Longhorn volumes cleanly, would not check whether another node is already down, and — on three
# control-plane nodes where etcd quorum is 2 of 3 — three nodes each deciding independently to
# reboot at their own `Automatic-Reboot-Time` is precisely how you lose the cluster.
# u-u therefore only ever leaves the sentinel behind; kured owns every reboot, one node at a time,
# under a cluster-wide lock. See platform/kured/helmrelease.yaml.
unattended_upgrades_automatic_reboot: false

# --- needrestart (DECISION 3) --------------------------------------------------------------------
# See the role README for the full argument. Summary: 'a' (restart services automatically) is what
# makes a security-only policy actually effective — a patched libssl on disk does nothing for a
# process still mapping the old one — but k3s.service is EXCLUDED, because restarting the
# Kubernetes control plane unattended at 06:00 is a worse outcome than a delayed library fix.
# Measured 2026-10-17: the only matching unit on these nodes is `k3s.service` (no separate
# containerd or kubelet unit exists; k3s bundles both), so this blacklist is complete today.
unattended_upgrades_needrestart_mode: a
unattended_upgrades_needrestart_blacklist:
  - '^k3s'

# --- Schedule ------------------------------------------------------------------------------------
# apt's own systemd timers (apt-daily, apt-daily-upgrade) drive u-u; these set their windows.
# 04:00 with a 1h randomised delay, so the upgrade lands INSIDE kured's 03:00-06:00 reboot window
# (platform/kured/helmrelease.yaml) rather than hours away from it: if an upgrade ever does set the
# sentinel, the reboot can be taken the same night instead of waiting ~24h. The randomised delay
# keeps the three nodes from hitting the mirror simultaneously; it does NOT serialise anything, and
# it does not need to — u-u never reboots, so three concurrent upgrades are harmless.
unattended_upgrades_download_time: "03:30"
unattended_upgrades_upgrade_time: "04:00"
unattended_upgrades_randomised_delay: 60m


# ── THE REBOOT SENTINEL, PROVIDED BY THIS ROLE ──────────────────────────────────────────────────
# /var/run/reboot-required is the ONLY thing kured watches, and on Debian trixie **nothing creates
# it**. Measured on all three nodes 2026-10-18:
#   * `update-notifier-common` (which owns that hook on Ubuntu) has `Candidate: (none)` — the
#     package does not exist in Debian. Installing it is what broke the first run of this play.
#   * `/usr/share/update-notifier/notify-reboot-required` is absent.
#   * NO `linux-image-*` postinst references `notify-reboot-required` — calling it is an Ubuntu
#     patch, not Debian behaviour, so merely dropping a script at that path would never be invoked.
#   * No hook under /etc/apt/apt.conf.d or /etc/kernel/ mentions reboot-required.
# So the writer has to be ours. Without it kured is a permanent, INVISIBLE no-op: it stays 1/1
# Running and reports kured_reboot_required=0 forever, which is the "failure mode is a green tick"
# shape this repo keeps getting burned by.
#
# MECHANISM: a dpkg Post-Invoke hook runs a script that compares the RUNNING kernel against the
# newest INSTALLED one and touches the sentinel when they differ. Deliberately kernel-only, which
# matches the design: unattended-upgrades is security-only with `^linux-` blacklisted, so it will
# essentially never set this — the sentinel appears when the OPERATOR deliberately runs the
# node_baseline kernel play, and kured then automates only the mechanical reboot.
#
# It uses `linux-version` (from `linux-base`, a dependency of every linux-image package, so
# guaranteed present — measured `linux-base 4.12.1` installed) rather than parsing `dpkg -l`
# output, because kernel version ordering is not string ordering.
#
# /run is a tmpfs (measured), so the sentinel self-clears on boot: after a reboot the running
# kernel IS the newest and the script writes nothing. No cleanup task is needed, and none exists.
unattended_upgrades_sentinel_script: /usr/local/sbin/k3s-notify-reboot-required
unattended_upgrades_sentinel_hook: /etc/apt/apt.conf.d/99-k3s-reboot-required