1 Using Object Part Numbers
2 Uploading a Part
3 Uploading a Part by Copying from an Existing Object
- 3.1 Copy-Source-Headers
4 Validating the Uploaded Parts

The individual parts of a larger object are ready to upload when the initiate request is complete. Swarm allows any number of parts in a multipart upload. (v9.1)

Create a POST request with the object name used in the initiate request, the upload ID returned from the initiate request, and a unique part number for each part to upload a part.

The part uploads for an upload ID must include the same domain query argument if the initiate request included a domain query argument for a specific domain. The part uploads must not include an encoding query argument as they inherit whatever was specified in the initiate request, unlike the initiate request. A failed part upload can be retried without affecting the outcome of the multipart upload.

A part is stored as an immutable object whose Content-UUID is returned in the request response when it is successfully uploaded. Client applications must keep track of the part number used for the upload and the Content-UUID Swarm assigned when stored to eventually complete the multipart upload, as described below.

Parts are Unnamed

Each part is an immutable object that returns a Content-UUID even if the initiated object is named, even though a POST on a named object does not ordinarily return that header. The parts are tenanted in the same domain as the destination object, but parts are unnamed, so they cannot reside in buckets.

Using Object Part Numbers

Swarm uses part numbers to identify the position of each part in an object. Include the upload ID and a unique part number for each part so Swarm can assemble the parts in the correct order when uploading parts. Non-sequential part numbers for each part can be selected (2, 4, 6, 8), but Swarm assembles the parts in sequential order.

Record each part number and corresponding Content-UUID. This information is required to complete the multipart write procedure.

Uploading a Part

The following must be included to upload each part:

The object name or UUID (or Content-UUID, for immutable objects) returned by the initiate request
The upload ID returned from the initiate request
A unique part number for each part uploaded

POST /ObjectNameorUUID?partNumber=PartNumber&uploadId=UploadId HTTP/1.1 
	Host: cluster.example.com 
	User-Agent: Swarm Client/0.1 
	Content-Length: 43402 
	Expect: 100-continue
	[ content ]

The content to be uploaded for the part is placed in the body of the request, like a normal POST operation.

Uploading a Part by Copying from an Existing Object

Create a POST request that uses the content from existing objects if the required parts currently exist in the storage cluster. Swarm creates an EC copy of the object for the multipart write when the part copy request is completed. The request fails if the source object does not exist or cannot be read from the specified range. This process leaves the source and destination versions unrelated to each other.

Content Gateway

The user making the "PUT with copy" request must have read access to the source object when going through Gateway.

The additional headers used in this request specify the source object and the range in bytes. The source object must be specified in the x-castor-copy-source header by UUID or object name in bucket/object-name format:

curl -i "$HOST/fd9cf39f056fb0dd858d8fb288c22885?PartNumber=3
		&UploadID=ddd080eb400bd5531f580191e3c5a916dd66c7c1e3244dc6cad46183097677e6dd66c7c1e3244dc6cad46183097677e60P" 
	-XPOST 
	-H "x-castor-copy-source: a08212d59b5bd306a52008dfef335be2" 
	-H "x-castor-copy-source-range: 5-8"

HTTP/1.1 201 Created
Location: http://192.168.1.171:80/09938e338c3590b93855d7cca2179aec
Location: http://192.168.1.109:80/09938e338c3590b93855d7cca2179aec
Volume: 3f5ef63dab992ebcf28e092bb56103c3
Volume: 12e08e29145f277501a6490b602ea287
Manifest: ec
Castor-System-UploadID: ddd080eb400bd5531f580191e3c5a916dd66c7c1e3244dc6cad46183097677e6dd66c7c1e3244dc6cad46183097677e60P
Content-UUID: 09938e338c3590b93855d7cca2179aec
Last-Modified: Tue, 27 Sep 2016 20:49:46 GMT
Entity-MD5: 9g0GoVLSYSXc/PMI4FWKbQ==
Stored-Digest: f60d06a152d26125dcfcf308e0558a6d
Castor-System-Encoding: zfec 1.4(1, 1, 524288, 200000000)
Castor-System-Version: 1475009386.549
Etag: "4c760a34ee534bcdba91680919378e2e"
Content-Range: bytes 5-8/10
Replica-Count: 2
Date: Tue, 27 Sep 2016 20:49:46 GMT
Server: CAStor Cluster/8.2.a
Content-Length: 46
Content-Type: text/html
Keep-Alive: timeout=14400

<html><body>New stream created</body></html>

The Content-MD5 is applied to the range read if a gencontentmd5 query argument (or the deprecated Expect: Content-MD5 header) is applied to a part copy with a range read.

These are the arguments and headers that are required for a part upload request that copies data from an existing object:

POST /ObjectNameorUUID?partNumber=PartNumber&uploadId=UploadId HTTP/1.1 
	x-castor-copy-source: uuid/name 
	x-castor-copy-source-domain: domain_name

Response Headers

The result code for the operation (which always responds with chunked encoding) is in the trailing header Castor-System-Result when the x-castor-copy-source header is used.

See "Response Headers for Multipart Writes" in Completing the Multipart Write.

Copy-Source-Headers

The following headers in the POST request for part uploads are optional, except for the first. They perform the same filtering as the regular headers of those names (range, if-match, etc.) performed against the source object being copied if included.

Type	Header	Notes

Type	Header	Notes
Source	x-castor-copy-source	Required. Must be a valid name or UUID.
Domain	x-castor-copy-source-domain	Required (unless untenanted)
Range	x-castor-copy-source-range: bytes=first-last	If the range values are out of bounds for the data, the request returns 416 (Range Not Satisfiable). Tip: To copy from a start range to the end of the object, omit the end range. (v11.1)
Conditional	x-castor-copy-source-if-match: "<ETag>" x-castor-copy-source-if-none-match: "<ETag>"	The ETag must be enclosed in quotes.
Conditional	x-castor-copy-source-if-unmodified-since: <timestamp> x-castor-copy-source-if-modified-since: <timestamp>	Uses the format of the standard HTTP last-modified header.

Validating the Uploaded Parts

Include a gencontentmd5 query argument or Content-MD5 header in the POST argument to return a Content-MD5 header to validate the content of the uploaded part. See Content-MD5 Checksums. (v9.2)

Verify each part has an MD5 stored with it if intending to validate the full transfer on the complete using the Composite-Content-MD5 header.

Swarm Documentation

Uploading the Parts