Home-Network/home-network-k3s-impl
2
0 Fork 0
Code Issues 2 Pull Requests 5 Actions Packages Projects Releases Wiki Activity

docs(prod): second independent confirmation + no cmdline workaround exists

Browse Source 
A second research pass reached the same root cause by a different route and pinned the
arithmetic. Recording it because convergence from two independent lines is what turns this from
a good hypothesis into a settled diagnosis.

MECHANISM TO THE BYTE:
- the garbage 'version' node 2 read was 1463964749 = 0x5742544D = ASCII 'MTBW'. The kernel is
  reading unrelated data. HP publishes the MEMATTR pointer into memory /sys/firmware/memmap calls
  plain System RAM, so the pre-Linux boot chain overwrites it.
- tbl_size is a SIGNED int; num_entries * desc_size wraps mod 2^32 and goes NEGATIVE when bit 31
  lands. memblock_reserve sign-extends it and memblock_cap_size clamps to PHYS_ADDR_MAX - base,
  so EVERYTHING above 0xbeeb0018 is reserved.
- memblock_free_all then hands the allocator nothing above that line while present_pages (from
  memblock.memory) is untouched -> present 3340288 / managed 0.
- 'Failed to map EFI Memory Attributes table' is emitted LATER by efi_memattr_apply_permissions();
  it is positive PROOF the reservation already happened, a consequence rather than the cause.

CLEANEST A/B TELL, one dmesg line: NODE_DATA(0) on prod-2 ends at 0xbeeb0000 — pgdat forced
immediately BELOW the reservation — while on prod-1 it ends at 0x82f800000, the top of RAM.
Identical 0x2a980 allocation, opposite placement.

THERE IS NO KERNEL CMDLINE WORKAROUND. efi_memattr_init() is called from
efi_config_parse_tables(), and x86's efi_init() calls efi_config_init() BEFORE the
efi_runtime_disabled() bail-out — so noefi / efi=noruntime / efi=novamap do NOT prevent the
reservation. A fixed kernel is the only real fix.

EFI_MEMORY_SP / soft-reserved was the strongest competing theory and is EXCLUDED BY MEASUREMENT:
/sys/firmware/memmap/10/type reads 'System RAM'. Soft-reserved would print as 'Soft Reserved' and
is never memblock_add()ed, which would have left present at ~0 rather than 3340288.

Also records that the hardening landed exactly one release late for Debian 13 (bug present since
2016; 8fbe4c49c0cc first in v6.13; follow-up 5d0faa8e8369) and that a Debian BTS report against
src:linux is warranted — 6.12 is Debian 13's kernel for the release lifetime.

Serves: R-ENV-01, R-ENV-03. Task: T-7.1.
This commit is contained in:
jakman85 committed 2026-09-01 22:16:22 -05:00
1 parent 49d82491dd
commit 47f02c6e06
1 file changed
+46
Download Patch File Download Diff File Expand all files Collapse all files
infra/PROD-PROVISIONING.md
+46
View File
@@ -96,6 +96,41 @@ DMA (3840) + DMA32 (684481) pages = **2,753,284 kB exactly**, with Normal contri
Node 1 is one unlucky boot away from the identical fault. **Fix both.**
### Mechanism, traced to the byte (second, independent confirmation)
A second research pass reached the same root cause by a different route and pinned the arithmetic:
- The garbage `version` node 2 read was `1463964749` = `0x5742544D` = the ASCII bytes **`MTBW`**. The
kernel is reading unrelated data, not a table. HP publishes the MEMATTR pointer into memory that
`/sys/firmware/memmap` calls plain `System RAM`, so the pre-Linux boot chain overwrites it.
- `tbl_size` is a **signed `int`**. `num_entries * desc_size` is `u32 * u32`, wraps mod 2^32, and
when bit 31 lands the result is **negative**.
- `memblock_reserve(base, tbl_size)` converts that negative `int` to `phys_addr_t`, sign-extending to
~`0xFFFFFFFF_xxxxxxxx`; `memblock_cap_size()` clamps it to `PHYS_ADDR_MAX - base`. **Every physical
address above `0xbeeb0018` is reserved.**
- `memblock_free_all()` then hands the buddy allocator nothing above that line, while `present_pages`
(which comes from `memblock.memory`) is untouched — producing `present 3340288 / managed 0`.
- The `Failed to map EFI Memory Attributes table` message is emitted *later*, by
`efi_memattr_apply_permissions()`, whose `memremap()` of the absurd size fails. **That message is
therefore positive proof the reservation already happened** — it is a consequence, not the cause.
The cleanest A/B tell between the two nodes is one line of dmesg:
| | `NODE_DATA(0) allocated` | meaning |
|---|---|---|
| prod-2 (bad) | `[mem 0xbee85680-0xbeeaffff]` | ends at `0xbeeb0000` — pgdat forced immediately **below** the reservation |
| prod-1 (good) | `[mem 0x82f7d5680-0x82f7fffff]` | ends at `0x82f800000` — **top of RAM**, the normal placement |
Identical `0x2a980` allocation size, opposite placement. Node 2's DMA32 `managed` is also 684481 vs
node 1's 775777 on identical `present` — ~356 MiB of extra structures forced below 4 GiB.
### THERE IS NO KERNEL CMDLINE WORKAROUND
`efi_memattr_init()` is called from `efi_config_parse_tables()` under only
`!CONFIG_X86_32 && efi_enabled(EFI_MEMMAP)`, and x86's `efi_init()` calls `efi_config_init()`
**before** the `efi_runtime_disabled()` bail-out. So `noefi`, `efi=noruntime` and `efi=novamap`
**do not prevent the reservation.** Booting a fixed kernel is the only real fix.
### The fix
**Boot a kernel ≥ 6.13** — upstream commit `8fbe4c49c0cc` *"efi/memattr: Ignore table if the size is
@@ -104,6 +139,13 @@ describes this bug verbatim. Verified absent from v6.12 and from the current `li
branch. On Debian 13 that means `linux-image-amd64` from **trixie-backports**. A fixed kernel prints
`FW_BUG Corrupted EFI Memory Attributes Table detected!` and keeps all the memory.
The hardening landed exactly one release too late for Debian 13: the bug has existed since the
feature was introduced in 2016, `8fbe4c49c0cc` first shipped in **v6.13**, and a follow-up
`5d0faa8e8369` (2026-03-26) tightened it further to the current form, whose comment reads verbatim
*"This prevents a corrupted table from eating all system RAM."* Debian's 6.12.107-1 demonstrably
lacks it. **Worth a Debian BTS report against `src:linux`** — 6.12 is Debian 13's kernel for the
release lifetime, and the fix is small and already upstream.
Same signature reported elsewhere: [Ubuntu LP #2039382](https://bugs.launchpad.net/ubuntu/+source/linux-signed-hwe-6.2/+bug/2039382)
(Dell, 16 GiB → 2.5 GiB, memtest clean, "fixed" by unplugging a USB mouse — which merely moved the
table) and [LP #1944019](https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1944019) (**HP ZBook
@@ -128,6 +170,10 @@ Studio G5**, 16 GiB → 1.5 GiB).
- **Memory remap / above-4G decoding** — no such attribute exists among the 241 exposed by
`hp-bioscfg` on this platform.
- **Integrated graphics** — `Graphics Stolen Memory` is exactly 64 MiB, below 4 GiB.
- **EFI_MEMORY_SP / "soft reserved" memory** — the strongest competing theory, **excluded by
measurement**: `/sys/firmware/memmap/10/type` reads `System RAM` for `0x100000000-0x42f7fffff`.
Soft-reserved ranges become `E820_TYPE_SOFT_RESERVED`, print as `Soft Reserved`, and are never
`memblock_add()`ed — which would have left `present` at ~0, not 3340288.
- **SGX** — ruled out at runtime: `x86/cpu: SGX disabled or unsupported by BIOS`.
- **MTRR trim** — MTRRs near-identical to node 1, no trim warning.
- **A `last_pfn` cap** — a misreading; the second value is `max_low_pfn` and is normal.