Replace MinIO backup target with versitygw (s3_backup_target role)
MinIO's open-source edition is de-prioritised (console removed, commercial
pivot), so swap the off-cluster S3 backup target on the Proxmox host to
versitygw v1.7.0 (github.com/versity/versitygw) — a maintained Go S3 gateway
serving a POSIX filesystem as S3.
Rename role minio_backup_target -> s3_backup_target and playbook backup-minio
-> backup-s3. The role now:
- decommissions any pre-existing MinIO (service/binary/env/mc-client/user/
object-layout), KEEPING the ZFS dataset Storage/k3s-backups and its mount
- installs the pinned, checksummed versitygw binary (fetched+extracted on the
controller, host has no internet), a dedicated versitygw system user, and a
Type=simple hardened systemd unit running the posix backend rooted at
/mnt/Storage/k3s-backups/data, bound to 192.168.1.9:9000 (same endpoint),
plain HTTP / path-style
- ensures the dataset has xattr=sa (versitygw stores object metadata in xattrs)
- pre-creates the velero + cnpg buckets (bucket == top-level subdir)
Credentials are REUSED: versitygw's ROOT key is set to the same scoped
access_key/secret the cluster already uses, so endpoint + key + secret + bucket
names are byte-identical to the old MinIO target. The GitOps SOPS secret
(platform/backup/minio-credentials.sops.yaml) and all cluster-side Velero/CNPG
config need ZERO change. The vaulted group_vars now expose
s3_backup_target_access_key/_secret_key (same values, MinIO root creds retired).
Verified live on the Proxmox host: versitygw active+enabled on 192.168.1.9:9000,
MinIO fully removed, dataset intact; SigV4 ListBuckets with the reused key
returns velero+cnpg; a throwaway in-cluster pod reaches http://192.168.1.9:9000
(versitygw S3 response). Re-run is idempotent (no host changes).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
1 parent
1bee2554f8
commit
6739339f53
17 files changed
+499
-491
No files matched your search
@@ -1,29 +1,27 @@
|
||||
# group_vars for the `proxmox_hosts` group (the Proxmox hypervisor host)
|
||||
|
||||
Target of `playbooks/backup-minio.yml` (role `minio_backup_target`). The role needs the MinIO
|
||||
credentials as **ansible-vault-encrypted** vars here — encrypted, so safe to commit; the passphrase
|
||||
that decrypts them stays out of the repo (`ansible.cfg` → `vault_password_file`).
|
||||
Target of `playbooks/backup-s3.yml` (role `s3_backup_target`, versitygw). The role needs the S3
|
||||
backup-target credentials as **ansible-vault-encrypted** vars here — encrypted, so safe to commit;
|
||||
the passphrase that decrypts them stays out of the repo (`ansible.cfg` → `vault_password_file`).
|
||||
|
||||
## `vault.yml` (committed, encrypted)
|
||||
|
||||
```bash
|
||||
cd infra/ansible
|
||||
ansible-vault create inventory/group_vars/proxmox_hosts/vault.yml
|
||||
ansible-vault edit inventory/group_vars/proxmox_hosts/vault.yml
|
||||
# contents:
|
||||
# minio_backup_target_root_user: "…" # MinIO root (host admin only)
|
||||
# minio_backup_target_root_password: "…"
|
||||
# minio_backup_target_access_key: "k3s-backup" # scoped key handed to Velero + CNPG
|
||||
# minio_backup_target_secret_key: "…"
|
||||
# s3_backup_target_access_key: "k3s-backup" # versitygw ROOT key = key handed to Velero + CNPG
|
||||
# s3_backup_target_secret_key: "…"
|
||||
```
|
||||
|
||||
The **scoped** access key/secret is mirrored (SOPS-encrypted) for the GitOps side at
|
||||
`platform/backup/minio-credentials.sops.yaml` — the two MUST stay in sync (same key/secret), since
|
||||
the cluster authenticates to this MinIO with them. Rotating means updating BOTH and re-running the
|
||||
playbook (recreates the MinIO user) + reconciling Flux.
|
||||
versitygw authenticates every S3 request against this single **ROOT** key. It is mirrored
|
||||
(SOPS-encrypted) for the GitOps side at `platform/backup/minio-credentials.sops.yaml` — the two MUST
|
||||
stay in sync (same key/secret), since the cluster authenticates to this endpoint with them. Rotating
|
||||
means updating BOTH and re-running the playbook (rewrites the env file + restarts versitygw) +
|
||||
reconciling Flux.
|
||||
|
||||
## Notes
|
||||
- This is the **Ansible-bootstrap-layer** secret (host credential), distinct from the platform's
|
||||
in-cluster **SOPS + age** secret (ADR-0007) — different layer, different tool, same values for the
|
||||
scoped key.
|
||||
in-cluster **SOPS + age** secret (ADR-0007) — different layer, different tool, same values.
|
||||
- `vault.yml` is a vault blob (not parseable YAML) — excluded from the yamllint gate like the other
|
||||
`**/vault.yml`.
|
||||
@@ -1,29 +1,25 @@
|
||||
$ANSIBLE_VAULT;1.1;AES256
|
||||
30633837396565613861623931383238383836356238326436633738613935326461346138373432
|
||||
6234643535303037313337363463633765326239376238380a333331333136396239313733366163
|
||||
33353036366461393530633139636430313736366338313331353163616135303930323038636134
|
||||
6366353362313537330a373863623861663537353539376664306338643539346139396130396632
|
||||
30336231363731616334383863326638636238323565316364303530396636373163393765376438
|
||||
63373263623063656662353434623339316238663535386465653164326236616265616638656639
|
||||
30613466656439396534316339646139343334656165323965323965313361633933633038376663
|
||||
31653539653763623132313533396333643964373032313333613432383433626338376431323731
|
||||
63316163363361303439313266356361613737333066643931656564633038306236363439666435
|
||||
39343536656336343834663464356566373564636531343534343365376332343832343235663339
|
||||
66383561306533323938613738376632626263306337346532643063373166333838333033346365
|
||||
32383136363636643163613334366631653238613166303962656463363663643830343531363533
|
||||
39333630323765643963396562363537396564666462306163336366643739313035636633646661
|
||||
61653637376535393034383631303563363539633265333161363334616239323866353162636632
|
||||
65663830323465393138306361333662383934346438663562316235353035633064643861353334
|
||||
63646435653639316439366139316539653833656561646439323766633139306565383565656261
|
||||
61323230306635343236646438336165643737626132613962633930653461393131303731623438
|
||||
63313265653166643334396461353231343235386434363438623034323762653433303932326630
|
||||
62633464316139353433366338313837646162323231363135353964623035366132643332366662
|
||||
38393465326363616139376465383738626636393639363338653437323832613439313138353033
|
||||
39393765613735666331636230363463306462373238623138316263326339366330333332643833
|
||||
33363764303939326662343665643963333463356638613231363834386132666339393132356538
|
||||
38303362643337366430303165633432666632343536383366653836663135383336316232393936
|
||||
66626632386537333739313432313564633866383132353862626433626366633562346464613066
|
||||
35376332303132326438313132303264313763376163396137356132306131373335346266646532
|
||||
31666230616461386339366461653661323138353662373866316639626134326339643231613337
|
||||
30343361643166353162363866653061313966663435373332353562656164663962326562333038
|
||||
35363834656361393930
|
||||
33623463393666373861613531626336396164356361643939326332626634663234653164313437
|
||||
3963326465326364643639633066636564656565333461360a383438323530663932326462376661
|
||||
64343766366132373436366336353338383365396563636138333333306330643337636532383662
|
||||
6561323836656637380a656436656538613263646461386635303762376163363161336436386263
|
||||
34653437386364363161653132343434333230393732373364383564336162643136636437376231
|
||||
39376463336666396462363431636161306632326134663038343364626232333838333039393338
|
||||
64313762343363666139393864366561316362633465323361633132373533356531353534396433
|
||||
62333131356630343764303438663766623737333137383264623739646539373535306136303530
|
||||
34663063383032643238373537663633343735613265653964356638306566636538363063656435
|
||||
65643666366665393238363037313337633366333338633163316533656134336463363630353961
|
||||
37656637343063373266643237663633613638303165656262313133313535313665336335333039
|
||||
30626537373530663430343637386263353765353538376263333130623536306635353966393561
|
||||
62313362373061303035376635646335303133613365326163643566333031356436326464616430
|
||||
34373433343733386666626234623064383063393330343034363036663632373865643636623961
|
||||
31666131636538303131376339623733353964653266356339313466616363646336663433666631
|
||||
61663564633931626639323235633332366338333837316535376330633534626537666166643262
|
||||
63646239656165303366626163613238393266336566316164356330393836323332336330633635
|
||||
64636636613164366338396666316137353837313639323630353234373936393335366364666361
|
||||
35356163363636623031333733653432636539396235633436313334333663343532303239373238
|
||||
37646230326164396339386339356663396661306165343630316261306331383632626662323234
|
||||
31396539313766356633333736383036623839626663383763626332626237373338653832646463
|
||||
34366434386139306336633964313939643565363838346163656236656561363865386136346530
|
||||
36316661323262323366646332316535306637316663656264303137653831656136633432643765
|
||||
61646433653530646339
|
||||
@@ -38,8 +38,8 @@ localhost ansible_connection=local
|
||||
[vpn_egress_gateway]
|
||||
k3s-dev-3
|
||||
|
||||
# The Proxmox HYPERVISOR host itself (NOT a k3s node) — target of playbooks/backup-minio.yml,
|
||||
# which stands up the off-cluster MinIO S3 backup target for Velero + CNPG. Reached DIRECTLY
|
||||
# The Proxmox HYPERVISOR host itself (NOT a k3s node) — target of playbooks/backup-s3.yml,
|
||||
# which stands up the off-cluster versitygw S3 backup target for Velero + CNPG. Reached DIRECTLY
|
||||
# over SSH as root (its own 192.168.2.102:22, the pfSense NAT to the host's 192.168.1.9), so
|
||||
# it overrides the cluster-wide `ansible_user: k3s` from group_vars/all.yml. Distinct from the
|
||||
# `hypervisor` group above (that is `localhost` for host-prereqs.yml, a different machine).
|
||||
|
||||
Reference in new issue
Block a user