Note
Multipart Write was previously referred to as Parallel Write; the functionality is the same.
With Multipart Write, you can upload parts of a large object from multiple clients at the same time. Multipart Write allow client applications to split a large file into multiple pieces, transfer the pieces concurrently to Swarm, and then request that Swarm combine the separately uploaded parts together as a single object, thereby minimizing the upload time.
Multipart write requires erasure coding (EC). The health processor (HP) has the ability to consolidate the segments of erasure-coded objects that have sub-optimal segment usage, such as can happen when you do SCSP or S3 multipart writes of objects using small parts. We recommend configuring clients to use 50MB-100MB parts. To enable consolidation, set the configuration setting, ec.segmentConsolidationFrequency, to 10 (recommended), which performs all consolidations over 20 HP cycles, if consolidation is needed.
Tip
Every multipart write must be erasure-coded for upload; however, if the uploaded object does not meet the current policy for EC encoding, the HP converts it to a replicated object. To maintain erasure coding for the lifetime of the object, be sure to add a lifepoint to that effect.
To upload a large object in parts using multipart write, you must perform three distinct actions in this order:
- Initiate a multipart write.
- Upload or copy the parts.
- Complete or cancel the procedure.