jakman85andClaude Opus 4.8
485b9e3888
Local-only tfstate meant losing the workstation would strand a rebuild (apply would try to recreate the 3 live nodes). Move state to Cloudflare R2, which sits OUTSIDE the Proxmox->VM->cluster->Gitea chain, so a full cluster rebuild can still read it; encrypted at rest via OpenTofu native AES-GCM. Adds a partial s3 backend (R2 skip-flags), gitignored backend.hcl/.r2.env for the account-specific + secret bits, their .example templates, and a migration+DR runbook. Locking (use_lockfile) deferred to an OpenTofu >=1.10 bump (1.9.0 lacks it). Migration (tofu init -migrate-state) runs once the operator provisions the R2 bucket. Co-Authored-By: Claude Opus 4.8 <[email protected]>
3.4 KiB
Tofu remote state — Cloudflare R2 (off-cluster, encrypted)
Why
State was local (terraform.tfstate on one workstation, git-ignored). Losing that box meant a
tofu apply from anywhere else would try to recreate the three live k3s nodes. State must live
off the cluster it manages — the chain is Proxmox host → k3s VMs → k3s cluster → Gitea/apps, so
an in-cluster backend (Gitea) is unreadable in the one scenario it exists for (cluster gone, rebuild).
Cloudflare R2 sits outside that chain entirely: a full cluster rebuild can still read state, it survives total-site loss (off-site), and it works from any machine. State bytes are encrypted at rest (OpenTofu native AES-GCM) so R2 only ever holds ciphertext.
One-time setup (operator)
1. Create the R2 bucket + token (Cloudflare dashboard)
- R2 → Create bucket → name
homelab-tofu-state(any name; matchbackend.hcl). - R2 → Manage API Tokens → Create API Token → permission Object Read & Write, scoped to that bucket. Copy the Access Key ID and Secret Access Key (shown once).
- Note your Account ID (32 hex chars, in the dashboard URL / the R2 S3 endpoint).
2. Fill the two gitignored files (secrets stay out of git and out of chat)
cd infra/tofu
cp backend.hcl.example backend.hcl # set bucket/key + endpoints.s3 = https://<ACCOUNT_ID>.r2.cloudflarestorage.com
cp .r2.env.example .r2.env # set AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY + a strong TF_ENCRYPTION passphrase
Store the encryption passphrase in your password manager. It is required to read state on any
machine and after any rebuild. Also record the R2 keys there. (Both files are in .gitignore.)
3. Migrate the existing local state -> R2 (one shot, moves state only — touches NO infra)
cd infra/tofu
cp terraform.tfstate terraform.tfstate.premigrate.bak # extra belt-and-suspenders backup
source .r2.env
tofu init -backend-config=backend.hcl -migrate-state # answer "yes" to copy local -> R2
tofu plan # MUST report "No changes." — proves parity
If plan shows no changes, the migration is correct. If it wants to create/destroy nodes, STOP —
the state did not carry over; re-check backend.hcl/creds and do not apply.
Day-2 use (any machine)
cd infra/tofu
source .r2.env # R2 creds + TF_ENCRYPTION passphrase
tofu init -backend-config=backend.hcl
tofu plan
That's the whole "work from a different computer" story: clone the repo, create the two local files
from the templates (creds + passphrase from your password manager), init, done.
DR: rebuild from nothing
Because state is in R2 (not the cluster), a destroyed cluster is recoverable: on a fresh box, install
OpenTofu, create backend.hcl + .r2.env, tofu init -backend-config=backend.hcl, then tofu apply
to rebuild the VMs against the preserved state. R2 itself should ALSO be covered by the off-cluster
backup target (OQ-BKP) so state survives even an R2/account loss.
Follow-up (tracked)
- Locking. OpenTofu 1.9.0 has no
use_lockfile; state locking is currently OFF (safe for a single operator). After bumping the toolchain to OpenTofu ≥ 1.10 (infra/devcontainer/versions.envversions.tfrequired_version), setuse_lockfile = trueinbackend.tffor native R2 conditional-write locking (no DynamoDB) — do this before regular two-machine use.