When the initiate request is complete, you are ready to upload the individual parts of a larger object. Swarm allows any number of parts in a multipart upload. (v9.1)
To upload a part, create a POST request with the object name used in the initiate request, the upload ID returned from the initiate request, and a unique part number for each part.
If the initiate request included a domain query argument for a specific domain, the part uploads for that upload ID must also include the same domain query argument. Unlike the initiate request, the part uploads must not include an encoding query argument as they will inherit whatever was specified in the initiate request. A failed part upload can be retried without affecting the outcome of the multipart upload.
When a part is successfully uploaded, it is stored as an immutable object whose Content-UUID is returned in the request response. Your client application must keep track of the part number used for the upload and the Content-UUID Swarm assigned it when it was stored in order to eventually complete the multipart upload, as described below.
Parts are unnamed
Even if the initiated object is named, each part is an immutable object that will return a Content-UUID, even though a POST on a named object does not ordinarily return that header. The parts are tenanted in the same domain as the destination object, but parts are unnamed, so they cannot reside in buckets.
Using object part numbers
Swarm uses part numbers to identify the position of each part in an object. When you upload the parts, include the upload ID and a unique part number for each part so that Swarm can assemble the parts in the correct order. You can select non-sequential part numbers for each part (for example: 2, 4, 6, 8), but Swarm will assemble the parts in sequential order.
Record each part number and its corresponding Content-UUID. This information is required to complete the multipart write procedure.
Uploading a part
To upload each part, you must include three things:
the object name or UUID (or Content-UUID, for immutable objects) that is returned by the initiate request
the upload ID returned from the initiate request
a unique part number for each part that is uploaded
POST /ObjectNameorUUID?partNumber=PartNumber&uploadId=UploadId HTTP/1.1 Host: cluster.example.com User-Agent: Swarm Client/0.1 Content-Length: 43402 Expect: 100-continue [ content ]
The content to be uploaded for the part should be in the body of the request, like a normal POST operation.
Uploading a part by copying from an existing object
If the required parts currently exist in your storage cluster, you can create a POST request that uses the content from existing objects. When the part copy request is completed, Swarm creates an EC copy of the object for the multipart write. If the source object does not exist or cannot be read from the specified range, the request will fail. This process leaves the source and destination versions unrelated to each other.
Content Gateway
When going through Gateway, the user who is making the "PUT with copy" request must have read access to the source object.
The additional headers used in this request specify the source object and its range, in bytes. The source object must be specified in the x-castor-copy-source header by UUID or object name in bucket/object-name format:
curl -i "$HOST/fd9cf39f056fb0dd858d8fb288c22885?PartNumber=3 &UploadID=ddd080eb400bd5531f580191e3c5a916dd66c7c1e3244dc6cad46183097677e6dd66c7c1e3244dc6cad46183097677e60P" -XPOST -H "x-castor-copy-source: a08212d59b5bd306a52008dfef335be2" -H "x-castor-copy-source-range: 5-8" HTTP/1.1 201 Created Location: http://192.168.1.171:80/09938e338c3590b93855d7cca2179aec Location: http://192.168.1.109:80/09938e338c3590b93855d7cca2179aec Volume: 3f5ef63dab992ebcf28e092bb56103c3 Volume: 12e08e29145f277501a6490b602ea287 Manifest: ec Castor-System-UploadID: ddd080eb400bd5531f580191e3c5a916dd66c7c1e3244dc6cad46183097677e6dd66c7c1e3244dc6cad46183097677e60P Content-UUID: 09938e338c3590b93855d7cca2179aec Last-Modified: Tue, 27 Sep 2016 20:49:46 GMT Entity-MD5: 9g0GoVLSYSXc/PMI4FWKbQ== Stored-Digest: f60d06a152d26125dcfcf308e0558a6d Castor-System-Encoding: zfec 1.4(1, 1, 524288, 200000000) Castor-System-Version: 1475009386.549 Etag: "4c760a34ee534bcdba91680919378e2e" Content-Range: bytes 5-8/10 Replica-Count: 2 Date: Tue, 27 Sep 2016 20:49:46 GMT Server: CAStor Cluster/8.2.a Content-Length: 46 Content-Type: text/html Keep-Alive: timeout=14400 <html><body>New stream created</body></html>
If you apply a gencontentmd5 query argument (or the deprecated Expect: Content-MD5 header) to a part copy with a range read, the Content-MD5 is only applied to the range read.
These are the arguments and headers that are required for a part upload request that copies data from an existing object:
POST /ObjectNameorUUID?partNumber=PartNumber&uploadId=UploadId HTTP/1.1 x-castor-copy-source: uuid/name x-castor-copy-source-domain: domain_name
Response Headers
Whenever the x-castor-copy-source header is used, the result code for the operation (which always responds with chunked encoding) is in the trailing header Castor-System-Result.
See "Response Headers for Multipart Writes" in Completing the Multipart Write.
-copy-source- headers
The following headers in the POST request for part uploads are optional, except for the first. If you include them, they perform the same filtering as the regular headers of those names (range, if-match, etc.), performed against the source object being copied.
Note
Error responses on conditional headers come back immediately, in place of a 202 (Accepted for processing) response. Condition failures (such as the ETags not matching) are reported in the initial HTTP response, not the castor-system-result header.
Type | Header | Notes |
|---|---|---|
Source | x-castor-copy-source | Required. Must be a valid name or UUID. |
Domain | x-castor-copy-source-domain | Required (unless untenanted) |
Range | x-castor-copy-source-range: bytes=first-last | If the range values are out of bounds for the data, the request returns 416 (Range Not Satisfiable). Tip: To copy from a start range to the end of the object, omit the end range. (v11.1) |
Conditional | x-castor-copy-source-if-match: "<ETag>" | The ETag must be enclosed in quotes. |
x-castor-copy-source-if-unmodified-since: <timestamp> | Uses the format of the standard HTTP last-modified header. |
Validating the uploaded parts
To validate the content of the uploaded part, include a gencontentmd5 query argument or Content-MD5 header in your POST argument to return a Content-MD5 header. See Content-MD5 Checksums. (v9.2)
If you intend to validate the full transfer on the complete using the Composite-Content-MD5 header, ensure that each part has an MD5 stored with it.