Existing bucket stays Pending: CreateBucket returns 503 while the bucket is written to #286

Problem

ensure_bucket sends CreateBucket (PUT /<bucket>) on every provisioning run and treats BucketAlreadyOwnedByYou / BucketAlreadyExists as success, like minio/operator does. On RustFS, CreateBucket for an existing bucket first takes the bucket's exclusive lifecycle and namespace locks and only then finds that the bucket exists (rustfs/rustfs#8741). Every PutObject holds the lifecycle read lock, so while the bucket receives steady writes the request times out after 5 s with 503.

The operator treats 503 as transient. The bucket stays Pending ("failed to create RustFS bucket: upstream returned 503") and the Tenant stays Ready=False ProvisioningPending for as long as writes continue. The bucket exists the whole time and needs no change.

Seen on RustFS 1.0.2 (1.0.3 ships the same server code) with a bucket receiving steady mc mirror writes. The Tenant was not Ready for over 30 minutes, which also blocked a Flux Kustomization that waits for the Tenant. Scaling the writer to 0 made it Ready within seconds.

Proposal

Send HeadBucket before CreateBucket. HeadBucket takes no exclusive lock and stays fast under the same load. If it succeeds, report the bucket as already existing and go on to versioning, policy, encryption and lifecycle. For any other status (404, 403, 5xx), keep the current CreateBucket path and its error classification. A connection error fails the same way a failed CreateBucket does today.