# Tofu remote state — Cloudflare R2 (off-cluster, encrypted)

## Why
State was **local** (`terraform.tfstate` on one workstation, git-ignored). Losing that box meant a
`tofu apply` from anywhere else would try to **recreate the three live k3s nodes**. State must live
**off the cluster it manages** — the chain is `Proxmox host → k3s VMs → k3s cluster → Gitea/apps`, so
an in-cluster backend (Gitea) is unreadable in the one scenario it exists for (cluster gone, rebuild).

**Cloudflare R2** sits outside that chain entirely: a full cluster rebuild can still read state, it
survives total-site loss (off-site), and it works from any machine. State bytes are **encrypted at
rest** (OpenTofu native AES-GCM) so R2 only ever holds ciphertext.

## One-time setup (operator)

### 1. Create the R2 bucket + token (Cloudflare dashboard)
1. **R2 → Create bucket** → name `homelab-tofu-state` (any name; match `backend.hcl`).
2. **R2 → Manage API Tokens → Create API Token** → permission **Object Read & Write**, scoped to
   that bucket. Copy the **Access Key ID** and **Secret Access Key** (shown once).
3. Note your **Account ID** (32 hex chars, in the dashboard URL / the R2 S3 endpoint).

### 2. Fill the two gitignored files (secrets stay out of git and out of chat)
```bash
cd infra/tofu
cp backend.hcl.example backend.hcl     # set bucket/key + endpoints.s3 = https://<ACCOUNT_ID>.r2.cloudflarestorage.com
cp .r2.env.example     .r2.env         # set AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY + a strong TF_ENCRYPTION passphrase
```
**Store the encryption passphrase in your password manager.** It is required to read state on any
machine and after any rebuild. Also record the R2 keys there. (Both files are in `.gitignore`.)

### 3. Migrate the existing local state -> R2 (one shot, moves state only — touches NO infra)
```bash
cd infra/tofu
cp terraform.tfstate terraform.tfstate.premigrate.bak   # extra belt-and-suspenders backup
source .r2.env
tofu init -backend-config=backend.hcl -migrate-state     # answer "yes" to copy local -> R2
tofu plan                                                # MUST report "No changes." — proves parity
```
If `plan` shows no changes, the migration is correct. If it wants to create/destroy nodes, STOP —
the state did not carry over; re-check backend.hcl/creds and do not apply.

> **Adopting encryption on pre-existing *unencrypted* state (one-time — already done on first
> migration).** OpenTofu refuses to read a plaintext state while encryption is configured
> (`encountered unencrypted payload without unencrypted method configured`). For that ONE migration
> only, `TF_ENCRYPTION` must include a fallback so tofu can read plaintext and rewrite it encrypted:
> add `method "unencrypted" "migrate" {}` and give the `state` block a
> `fallback { method = method.unencrypted.migrate }`. Run the migrate, then **remove both** and
> `tofu init -reconfigure` so encryption is enforced. A fresh machine adopting the ALREADY-encrypted
> R2 state does NOT need this — the encrypt-only `.r2.env.example` is correct as-is.

## Day-2 use (any machine)
```bash
cd infra/tofu
source .r2.env                 # R2 creds + TF_ENCRYPTION passphrase
tofu init -backend-config=backend.hcl
tofu plan
```
That's the whole "work from a different computer" story: clone the repo, create the two local files
from the templates (creds + passphrase from your password manager), `init`, done.

## DR: rebuild from nothing
Because state is in R2 (not the cluster), a destroyed cluster is recoverable: on a fresh box, install
OpenTofu, create `backend.hcl` + `.r2.env`, `tofu init -backend-config=backend.hcl`, then `tofu apply`
to rebuild the VMs against the preserved state. R2 itself should ALSO be covered by the off-cluster
backup target (OQ-BKP) so state survives even an R2/account loss.

## Follow-up (tracked)
- **Locking.** OpenTofu 1.9.0 has no `use_lockfile`; state locking is currently OFF (safe for a single
  operator). After bumping the toolchain to **OpenTofu ≥ 1.10** (`infra/devcontainer/versions.env`
  + `versions.tf` `required_version`), set `use_lockfile = true` in `backend.tf` for native R2
  conditional-write locking (no DynamoDB) — do this before regular two-machine use.