Compare commits
4
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0d4bd7297d | ||
|
|
c340bbfc4b | ||
|
|
a9a529750d | ||
|
|
793c3940a8 |
No files matched your search
@@ -74,12 +74,26 @@ name: renovate
|
|||||||
|
|
||||||
on:
|
on:
|
||||||
schedule:
|
schedule:
|
||||||
# 03:15 UTC daily. renovate.json ALSO carries `"schedule": ["after 3am and before 6am"]` in
|
# 08:15 UTC daily. renovate.json ALSO carries `"schedule": ["after 3am and before 6am"]` with
|
||||||
# America/New_York, and that inner schedule is the authoritative one — Renovate re-checks it and
|
# `"timezone": "America/New_York"`, and THAT inner schedule is authoritative — Renovate re-checks
|
||||||
# will decline to open PRs outside it. The two are deliberately both present: this cron decides
|
# it and DECLINES TO OPEN PRs outside it. So this cron must fire INSIDE the local window, not
|
||||||
# when the process RUNS, renovate.json decides when it may ACT. Keeping the cron slightly ahead
|
# merely near it.
|
||||||
# of the repo window means a run is always in flight when the window opens.
|
#
|
||||||
- cron: "15 3 * * *"
|
# WAS `15 3 * * *`, WHICH NEVER WOULD HAVE WORKED (fixed 2026-10-18, before the first scheduled
|
||||||
|
# run). Gitea Actions cron is UTC, so 03:15 UTC is 23:15 the PREVIOUS EVENING in New York --
|
||||||
|
# 3h45m BEFORE the window opens, not "slightly ahead" of it as the original comment claimed.
|
||||||
|
# Every nightly run would have started, logged "not within schedule", opened zero PRs and
|
||||||
|
# exited SUCCESSFULLY. A green tick forever, and no updates ever: the exact "failure mode is a
|
||||||
|
# green tick" shape this repo keeps getting burned by, and indistinguishable from "nothing
|
||||||
|
# needed updating".
|
||||||
|
#
|
||||||
|
# THE WINDOW IN UTC MOVES WITH US DST, and a cron cannot follow it:
|
||||||
|
# EDT (UTC-4, Mar-Nov): local 03:00-06:00 == 07:00-10:00 UTC
|
||||||
|
# EST (UTC-5, Nov-Mar): local 03:00-06:00 == 08:00-11:00 UTC
|
||||||
|
# 08:15 UTC is the only quarter-hour comfortably inside BOTH (04:15 EDT / 03:15 EST), so the
|
||||||
|
# schedule keeps working across both switchovers without an edit. Do not "tidy" this back to an
|
||||||
|
# early-morning-UTC value without redoing this arithmetic.
|
||||||
|
- cron: "15 8 * * *"
|
||||||
workflow_dispatch:
|
workflow_dispatch:
|
||||||
inputs:
|
inputs:
|
||||||
dry_run:
|
dry_run:
|
||||||
|
|||||||
@@ -445,8 +445,21 @@ join = re.search(r"--server https://\{\{\s*k3s_api_host", K3S_TEXT)
|
|||||||
check(join is not None,
|
check(join is not None,
|
||||||
"roles/k3s_server does not join secondaries with `--server https://{{ k3s_api_host }}` "
|
"roles/k3s_server does not join secondaries with `--server https://{{ k3s_api_host }}` "
|
||||||
"— without a join target the servers are separate single-node clusters")
|
"— without a join target the servers are separate single-node clusters")
|
||||||
join_when = re.search(r"--server https://.*?\n(?:.*\n){0,10}?\s*when:\s*k3s_server_mode\s*==\s*'secondary'",
|
# LINE-BOUNDED ON PURPOSE -- DO NOT "SIMPLIFY" BACK TO `.*` + re.S.
|
||||||
K3S_TEXT, re.S)
|
# This was `--server https://.*?\n(?:.*\n){0,10}?\s*when:...` with re.S, which made every `.`
|
||||||
|
# match newlines too. That is nested unbounded repetition (a `{0,10}` of a quantifier that can each
|
||||||
|
# swallow the whole document), so a FAILED or late match backtracks exponentially. It cost
|
||||||
|
# **19m54s of CPU in CI** (measured 2026-10-18 via `docker top` on the live job container) and made
|
||||||
|
# the whole merge gate take ~28 minutes instead of ~2 on any change touching `infra/**` -- because
|
||||||
|
# `ci/touched-requirements.sh` maps `infra/*` to R-ENV-01, which pulls in this whole-repo check.
|
||||||
|
# It was latent for as long as the regex existed and only crossed the threshold when
|
||||||
|
# roles/k3s_server grew by ~100 lines (the kubelet graceful-shutdown config, PR #71): K3S_TEXT is
|
||||||
|
# every *.yml in that role concatenated, so the role growing is the input growing.
|
||||||
|
# `[^\n]*` cannot cross a line, so the `{0,10}` is a genuine ten-line window with nothing to
|
||||||
|
# backtrack into. Measured on the same input: 0.2 ms, same verdict (match found).
|
||||||
|
join_when = re.search(
|
||||||
|
r"--server https://[^\n]*\n(?:[^\n]*\n){0,10}?[ \t]*when:[ \t]*k3s_server_mode[ \t]*==[ \t]*'secondary'",
|
||||||
|
K3S_TEXT)
|
||||||
check(join_when is not None,
|
check(join_when is not None,
|
||||||
"the join install task is not gated on k3s_server_mode == 'secondary' — the founder "
|
"the join install task is not gated on k3s_server_mode == 'secondary' — the founder "
|
||||||
"would also try to join itself")
|
"would also try to join itself")
|
||||||
|
|||||||
@@ -0,0 +1,76 @@
|
|||||||
|
---
|
||||||
|
date: 2026-10-18
|
||||||
|
branch: fix/ha-check-catastrophic-regex
|
||||||
|
title: "One regex made the merge gate take 28 minutes instead of 2"
|
||||||
|
---
|
||||||
|
|
||||||
|
## What changed
|
||||||
|
|
||||||
|
The merge gate started taking **~28 minutes** instead of its usual **<2 minutes**, on every change
|
||||||
|
touching `infra/**`. One regex was responsible for essentially all of it.
|
||||||
|
|
||||||
|
`conformance/tests/l0-static/ha-control-plane-as-code/check.sh` contained:
|
||||||
|
|
||||||
|
```python
|
||||||
|
join_when = re.search(r"--server https://.*?\n(?:.*\n){0,10}?\s*when:\s*k3s_server_mode\s*==\s*'secondary'",
|
||||||
|
K3S_TEXT, re.S)
|
||||||
|
```
|
||||||
|
|
||||||
|
With `re.S`, every `.` also matches newlines, so `.*?` and the nested `(?:.*\n){0,10}?` can each
|
||||||
|
consume the entire document. That is **nested unbounded repetition** — ten levels of a quantifier
|
||||||
|
that can each swallow everything — and a failed or late match backtracks exponentially.
|
||||||
|
|
||||||
|
Replaced `.` with `[^\n]*` and dropped `re.S`, so the `{0,10}` is a genuine ten-line window with
|
||||||
|
nothing to backtrack into. Same intent, same verdict, no DOTALL.
|
||||||
|
|
||||||
|
**Why it surfaced now, and it was mine.** `K3S_TEXT` is every `*.yml` in `roles/k3s_server`
|
||||||
|
concatenated. The regex was latent for as long as it existed; **PR #71 added ~100 lines to that role
|
||||||
|
(the kubelet graceful-shutdown config), which grew the input past the point where exponential
|
||||||
|
backtracking stopped being survivable.** Nothing about the check changed — my change grew its input.
|
||||||
|
|
||||||
|
**Why only `infra/**` changes were slow.** `ci/touched-requirements.sh:268` maps `infra/*` to
|
||||||
|
`R-ENV-01`, and the resulting L0 run is `--include-scope whole-repo --requirements R-ENV-01`, which
|
||||||
|
pulls in this whole-repo check. Doc-only and `platform/**` changes never selected it, which is why
|
||||||
|
#69 (1m55s) and #72 (1m53s) stayed fast while #71 (27m44s), the #73 merge (27m46s) and #75 all ran
|
||||||
|
~28 minutes.
|
||||||
|
|
||||||
|
## What I verified
|
||||||
|
|
||||||
|
* **The hot spot, by measurement rather than reading.** `docker top` on the live CI job container for
|
||||||
|
run #299 showed `python3 -` with **19:54 of CPU time** under
|
||||||
|
`ha-control-plane-as-code/check.sh`, inside
|
||||||
|
`conformance/run.sh --layer l0 --include-scope whole-repo --requirements R-ENV-01`.
|
||||||
|
* **Reproduced locally**: the check did not finish within a 100-second budget on an unmodified
|
||||||
|
checkout.
|
||||||
|
* **Localised to the exact line** by extracting the inline Python and running it under
|
||||||
|
`faulthandler.dump_traceback_later(25)`, which named `re/__init__.py:177 in search` called from
|
||||||
|
the `join_when` line — not inferred from reading the patterns.
|
||||||
|
* **The pattern is unique in the repo**: `grep -rlnE "\{0,[0-9]+\}\?" conformance/tests/l0-static/*/check.sh`
|
||||||
|
returns only this file, so there is no sibling with the same hazard.
|
||||||
|
* **The fix is fast AND verdict-preserving**, tested against the real `K3S_TEXT` (22,179 chars, 403
|
||||||
|
lines): the new regex matches (`YES`) in **0.2 ms**. Sanity-checked that the thing it looks for
|
||||||
|
genuinely exists — one `--server https://` and three `k3s_server_mode == 'secondary'` occurrences.
|
||||||
|
* **The whole check now runs in 0.199s, rc 0, 115 assertions**, with the same success message as
|
||||||
|
before.
|
||||||
|
|
||||||
|
## What I did NOT verify
|
||||||
|
|
||||||
|
* **I did not re-time the full gate in CI after the fix.** The expectation is that `infra/**` changes
|
||||||
|
return to the ~2-minute range, but that is arithmetic from the 19m54s figure, not a measured
|
||||||
|
post-fix run. The PR's own CI run is the first real datapoint — read it.
|
||||||
|
* **I did not audit the other 23 L0 checks for different performance pathologies.** This one was
|
||||||
|
found because it was actively burning CPU in a live job; a check that is merely slow (not
|
||||||
|
exponential) would not have stood out, and `ci/gate.sh` has **no per-stage timing** at all
|
||||||
|
(`grep -nE "SECONDS|date \+|elapsed"` finds nothing), so there is no data on where the remaining
|
||||||
|
time goes.
|
||||||
|
* **Nothing prevents this class of bug returning.** No check asserts gate stage runtime, and no lint
|
||||||
|
forbids nested quantifiers under `re.S`. A timing budget per stage, or a `re` lint, would close it;
|
||||||
|
neither is written here.
|
||||||
|
* **I did not verify my claim about *why* only `infra/**` was affected beyond the path mapping and
|
||||||
|
the observed durations.** The correlation is strong (5 runs, two clear clusters) and the live
|
||||||
|
process list confirms the `--requirements R-ENV-01` scope, but I did not read
|
||||||
|
`conformance/run.sh`'s scope-selection logic end to end.
|
||||||
|
* The regex rewrite assumes the `when:` clause is indented with spaces or tabs only (`[ \t]*`). A
|
||||||
|
file using some other whitespace before `when:` would now fail to match where the old DOTALL
|
||||||
|
version would have succeeded. Unlikely in YAML, and the current input matches, but it is a
|
||||||
|
narrowing.
|
||||||
@@ -0,0 +1,78 @@
|
|||||||
|
---
|
||||||
|
date: 2026-10-18
|
||||||
|
branch: fix/renovate-cron-window
|
||||||
|
title: "The Renovate cron fired outside Renovate's own schedule, so it would never have opened a PR"
|
||||||
|
---
|
||||||
|
|
||||||
|
## What changed
|
||||||
|
|
||||||
|
`.gitea/workflows/renovate.yaml` shipped (PR #73) with `cron: "15 3 * * *"`. **That schedule could
|
||||||
|
never have produced a single PR.**
|
||||||
|
|
||||||
|
Gitea Actions cron is **UTC**. `renovate.json` carries `"schedule": ["after 3am and before 6am"]`
|
||||||
|
with `"timezone": "America/New_York"`, and that inner schedule is **authoritative** — Renovate
|
||||||
|
re-checks it and declines to open PRs outside it. So:
|
||||||
|
|
||||||
|
```
|
||||||
|
cron 03:15 UTC -> 23:15 EDT, the PREVIOUS EVENING in New York
|
||||||
|
window -> 03:00-06:00 local
|
||||||
|
inside? -> NO (3h45m before the window opens)
|
||||||
|
```
|
||||||
|
|
||||||
|
The original comment claimed the cron was "slightly ahead of the repo window ... so a run is always
|
||||||
|
in flight when the window opens". It is 3h45m *behind*, on the previous day. Every nightly run would
|
||||||
|
have started, logged "not within schedule", opened **zero** PRs and exited **successfully** — a
|
||||||
|
green tick forever, indistinguishable from "nothing needed updating". That is the same
|
||||||
|
"failure-mode-is-a-green-tick" shape this repo keeps hitting, and it would have hidden the ~30
|
||||||
|
pending updates PR #73 was written to surface.
|
||||||
|
|
||||||
|
Changed to `cron: "15 8 * * *"`, chosen to survive DST rather than to be tidy. The window's UTC
|
||||||
|
position moves:
|
||||||
|
|
||||||
|
| | local window | in UTC |
|
||||||
|
|---|---|---|
|
||||||
|
| EDT (UTC-4, Mar–Nov) | 03:00–06:00 | 07:00–10:00 |
|
||||||
|
| EST (UTC-5, Nov–Mar) | 03:00–06:00 | 08:00–11:00 |
|
||||||
|
|
||||||
|
`08:15 UTC` is inside **both** (04:15 EDT / 03:15 EST), so no edit is needed at either switchover.
|
||||||
|
A cron cannot follow DST, so this is the only way to get it right once.
|
||||||
|
|
||||||
|
## What I verified
|
||||||
|
|
||||||
|
* **The defect, by arithmetic rather than assertion:** `datetime(2026,10,18,03,15, UTC)` converted to
|
||||||
|
`America/New_York` via `zoneinfo` is `2026-10-17 23:15 EDT`; `3 <= hour < 6` is **False**.
|
||||||
|
* **The fix, at both DST extremes:** `08:15 UTC` → `04:15 EDT` in July and `03:15 EST` in December,
|
||||||
|
both inside `03:00–06:00`. Computed, not reasoned.
|
||||||
|
* **The inner schedule really is authoritative and really is set:** `renovate.json` on `main` carries
|
||||||
|
`timezone = America/New_York` and `schedule = ['after 3am and before 6am']`.
|
||||||
|
* **`RENOVATE_TOKEN` now exists** as a repository Actions secret (listed by name via the Gitea API
|
||||||
|
alongside `SETTINGS_API_TOKEN`), so the workflow's fail-hard preflight will pass. It did not exist
|
||||||
|
when PR #73 was authored.
|
||||||
|
* The edited workflow parses as YAML.
|
||||||
|
|
||||||
|
## What I did NOT verify
|
||||||
|
|
||||||
|
* **Renovate has still never executed.** This fixes *when* it runs, not *whether it works*. No run
|
||||||
|
has happened, so nothing here proves it extracts dependencies, authenticates to Gitea, or opens a
|
||||||
|
usable PR.
|
||||||
|
* **`enabledManagers` may silently disable the custom managers — UNRESOLVED and the likelier
|
||||||
|
remaining bug.** `renovate.json` lists `enabledManagers: [..., 'regex', ...]` and defines
|
||||||
|
`customManagers` (correct modern key; `regexManagers` is absent, which is right). But modern
|
||||||
|
Renovate names the custom manager **`custom.regex`** in `enabledManagers`, not bare `regex`. If
|
||||||
|
the bare name is invalid or ignored, every inline `# renovate:` marker in the tree does nothing —
|
||||||
|
including the `external-snapshotter` `newTag` pin PR #73 flagged as probably inert. **I did not
|
||||||
|
change it**, because guessing at a config key that could fail validation and take the whole run
|
||||||
|
down is worse than leaving a question that one dry run answers definitively. The `dry_run: true`
|
||||||
|
dispatch log settles it: look for a per-manager dependency-extraction count, and for any config
|
||||||
|
validation warning naming `regex`.
|
||||||
|
* **Whether a dry run inside vs outside the window differs usefully.** Renovate honours its schedule
|
||||||
|
even in dry-run, so a manual dispatch outside 03:00–06:00 local will also log "not within
|
||||||
|
schedule". That makes the first dispatch's output less informative than it looks; to see what
|
||||||
|
Renovate *would* do, either dispatch inside the local window or override the schedule for that
|
||||||
|
run. Not tested either way.
|
||||||
|
* **No alert fires if the schedule stops firing.** PR #73 recorded this gap and it is still open: a
|
||||||
|
Renovate workflow that silently stops running (disabled Actions, expired token, deleted secret)
|
||||||
|
produces no signal at all. Closing it needs a check on last-successful-run age, which nothing
|
||||||
|
does.
|
||||||
|
* No conformance test asserts the cron falls inside `renovate.json`'s window, so this exact bug can
|
||||||
|
return. A static check comparing the two is feasible and is not written.
|
||||||
Reference in new issue
Block a user