You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
storage: parallel composite upload parts inherit retention/holds from the destination object, so part cleanup fails and the upload errors after a successful compose #14892
With TransferManagerConfig.setAllowParallelCompositeUpload(true), an upload larger than 4 × perWorkerBufferSize uses parallel composite upload. If the BlobInfo returned by ParallelUploadConfig.UploadBlobInfoFactory sets Retention (either mode), temporaryHold, or eventBasedHold:
ParallelCompositeUploadWritableByteChannel.definePart builds every part and intermediate compose object from ultimateObject.toBuilder(). It clears only crc32c, md5, and kmsKeyName, so each part gets the destination's retention and holds.
The final compose succeeds.
cleanupParts (default PartCleanupStrategy.always()) can't delete the parts, because GCS rejects deletes on retained or held objects. deleteAsync only tolerates 404, so the failure propagates through quietAllAsList, and close() / the UploadResult reports FAILED even though the destination object exists.
The part objects stay behind. With Retention.Mode.LOCKED they can't be removed until the retain-until time.
Expected
Parts and intermediate compose objects are transient and shouldn't carry the destination's retention or holds. Only the final composed object should get them, and the final compose already applies them from ultimateObject.
Why there's no workaround through the public API
TransferManagerConfig doesn't expose PartMetadataFieldDecorator or PartCleanupStrategy.
PartMetadataFieldDecorator can't be extended outside the package (its newInstance is package-private), and the only factories are noOp() and setCustomTimeInFuture().
The only option left is to disable parallel composite upload for these objects.
Suggested fix
In definePart, clear retention, temporaryHold, and eventBasedHold on part / intermediate objects. Optionally, also let callers set PartMetadataFieldDecorator and PartCleanupStrategy through TransferManagerConfig.
Notes
This report comes from reading the 2.75.0 source (ParallelCompositeUploadWritableByteChannel#definePart, #cleanupParts, #deleteAsync); it hasn't been reproduced against a live bucket yet. Repro outline: a bucket with object retention enabled, TransferManager.uploadFiles of a file over 64 MiB (default buffer) with parallel composite upload enabled and a BlobInfo carrying Retention (UNLOCKED, for easy cleanup) or temporaryHold=true.
Activity
blobstore: skip GCP parallel composite upload for object-locked direc…
blobstore: document GCP parallel composite upload limitation with obj…