`platform/kyverno/admission/helmrelease.yaml` sets `crds: {install: false}`
because the CRD bundle is 5,636,155 bytes, far past kubectl's 256 KiB
last-applied-configuration limit — a client-side apply fails with
"metadata.annotations: Too long" and STALLS the HelmRelease.
The consequence nobody automated: with CRD install off, NOTHING installed
them. `platform` has `wait: true` and every other tier chains transitively
through it, so on a cold cluster Kyverno's API types never exist, `platform`
never goes Ready, and all 10 downstream tiers stall. Dev only ever had these
CRDs because they were applied BY HAND — exactly the undocumented manual step
this repo exists to eliminate.
New `bootstrap/flux/preapply-crds.sh`, wired into bootstrap.sh as step [2/4]
(after `flux install`, before the graph is applied) and standalone-safe so an
already-bootstrapped cluster can be unblocked without re-running bootstrap.
NO SECOND PIN: all four chart coordinates are read with yq out of the same
HelmRelease/HelmRepository pair Flux reconciles. The version is passed through
as the CONSTRAINT ("3.9.x"), not a resolved tag — helm and helm-controller both
use Masterminds/semver, so the render is the version Flux installs. Renovate
bumps one line and this step follows; there is nothing here to drift from.
Only `kind: CustomResourceDefinition` documents are applied. The by-hand
command recorded in the HelmRelease comment piped the WHOLE template into
kubectl, which would create Deployments/webhooks outside Helm's ownership and
then fight the HelmRelease over field managers.
THREE GUARDS, because a silent skip here reproduces a failure mode this repo
has been bitten by repeatedly:
1. any derived coordinate that is empty/null/multi-valued aborts;
2. rendered CRD count must be > 0 — this caught a real bug in the first
draft: on an empty render `yq .metadata.name` emits one BLANK line, which
counted as 1 CRD and sailed past the guard into an apply of an empty file
returning 0, i.e. the exact silent-success trap. Blank/null lines are now
dropped and a chart with no CRDs exits 1;
3. every applied CRD is waited on for condition=Established, so "kubectl
returned 0" is not the only evidence.
No opt-out flag, deliberately: a skipped CRD apply yields a cluster that looks
bootstrapped and reconciles nothing.
`roles/flux_bootstrap` is untouched — it already shells to bootstrap.sh, the
documented single entrypoint, so it inherits the step with no dev/prod fork
(R-ENV-02). A second role task would double-apply.
MEASURED, not assumed — reproduced under the PINNED toolchain (helm 3.16.0,
yq 4.44.6) in infra/devcontainer, matching an independent run on helm 3.20.0
byte-for-byte: 3.9.x resolves to 3.9.0, 22 CRDs, 5,636,155 bytes CRD-only
(5,870,019 full render). Largest single CRD document is 1,378,544 bytes —
under etcd's 1.5 MiB default, but not by much; worth watching on chart bumps.
NOT YET EXERCISED against a live API server: the server-side apply and the
Established wait were tested with a stub kubectl. velero and
kube-prometheus-stack also say "CRDs pre-applied" but both still get theirs
from Helm's native crds/ dir on FIRST install, so neither is a cold-start
blocker; CRD_HELMRELEASE makes extending this a one-liner if that changes.
Refs R-GITOPS-01, R-ENV-02.
Co-Authored-By: Claude Opus 5 <[email protected]>
