SCSP Headers
Header Limits: Headers (metadata) are constrained not only by Swarm but by all services, proxies, and clients (such as Elasticsearch, Twisted, Jetty, HAProxy) handling objects. A Swarm object may have any number of annotations associated with it, but each annotation object is subject to these limits. The persisted metadata on Swarm objects must fall within these limits by default:
500 - The total number of headers on an object
32 KB - The combined length of all headers (key/value pairs); exceeding this returns a 400 (Bad Request) error
16 KB - The maximum length for a given header (key/value pair)
The following tables list all headers supported or used by Swarm. The headers described apply to buckets, named objects, and aliased objects unless otherwise noted.
Methods are the SCSP methods to which the header applies.
Request (signified by X)
Response (signified by X)
V means it appears only with
verbose=true
T means it appears in a trailing response
Writable (signified by W)
R means it is read-only
Persisted indicates whether the header is persisted with the object. Note: some persisted headers are not writable (signified by P).
Best Practice
Always use case-insensitive header matching.
Swarm-Specific Headers
Header | Description | Method | Request | Response [Trailing] [Verbose] | Writable [Read-only] | Persisted |
---|---|---|---|---|---|---|
Castor-System-Accessed | Used on GET/HEAD requests to indicate the time the object was last accessed (written or GET). | GET, HEAD | X V | |||
Castor-System-CompositeMD5 | Multipart-written objects only. This composite value, stored with the completed object, is computed on the multipart complete as the sum of the Content-MD5 values of all parts. This header value is reused on subsequent COPY operations but removed on other updates (PUT, APPEND). | GET, HEAD, POST | T | |||
Castor-System-Decorates | Set to the ETag of a Swarm object to be decorated. When the decorated object is deleted or overwritten, the object with this header is reclaimed by the health processor. | All | X | X | W | P |
Castor-System-Headers-Filtered | If response headers are filtered by a whitelist or blacklist, this header is added. | GET, HEAD | X | R | ||
Castor-System-InProgress | Returned with value yes on a verbose request when the inprogress flag is set on a stream. Not returned at all when the flag is not set. | GET, HEAD | X V | |||
Castor-System-Result | Reports the response code, such as 201 (Created) or 404 (Not Found), as a trailing header for a multipart write completion. | POST | T | |||
Castor-System-Tiered | Audit headers for objects restored by an S3 Backup feed. One copy captures the date and source cluster of the backup, and another captures the date and S3 host/bucket of the restore. Dates are same format as Last-Modified. | GET, HEAD | X V | R | P | |
Castor‑{anything} | {anything} cannot start with "system". Uninterpreted; meant for client customization. | All | W | P | ||
Castor‑Authorization | Content‑level authorization. | All | X | W | P | |
Castor‑System-Bytes‑Used | With bucket list du, the number of bytes in the query without replicas. | GET (listing) | X | |||
Castor‑System-Bytes‑Used‑With‑Reps | With bucket list du / withreps requests, the number of bytes in the query with replicas. | GET (listing) | X | |||
Castor‑System-Object‑Count | In a bucket listing request, the number of objects returned by the query. | X | ||||
Castor‑System‑Alias | The alias UUID of an alias object, bucket, or domain. This UUID cannot be used in any SCSP method on a domain, bucket, or named object. Use it to execute SCSP methods on an aliased object or to delete an unnamed immutable object. | GET, HEAD, DELETE | X | R | P | |
Castor‑System‑Auth | Special headers for a GET/retrieve request. The username:password for an administrative account on the remote cluster, only if it differs from the source cluster. | SEND | X | |||
Castor‑System‑CID | For named and tenanted unnamed objects, the alias UUID of the owning context. | GET, HEAD, DELETE | X | R | P | |
Castor‑System‑Cluster | Name of the cluster where the object was created or last updated. For SEND, The value of the cluster.name setting of the destination cluster. | GET, HEAD, DELETE, SEND | X | R | P | |
Castor‑System‑Created | A timestamp that specifies when the object was created or last updated. Uses HTTP time, an ASCII format based on GMT. | GET, HEAD, DELETE | X | R | P | |
Castor‑System‑Domain | Named objects only. Shows the name of the domain where the object was created. The domain name in which this object is tenanted. Computed on GET and HEAD. Returns <domain>. | GET, HEAD | X V | R | P | |
Castor‑System‑EnforceTenancy | True or False depending on cluster.enforceTenancy. A response header on the status page. | GET | X | |||
Castor‑System‑Error‑Code | Sent with any error response. The request error code (if applicable). This code is usually a 4xx or 5xx HTTP response code. May appear as a trailing header on multipart write complete requests. | All | X | |||
Castor‑System‑Error‑Text | Sent with any error response. The request error description (if applicable). Provides a description of the error. May appear as a trailing header on multipart write complete requests. | All | X | |||
Castor‑System‑Error‑Token | Sent with any error response. A unique error token (if applicable). Provides a parsed token that uniquely identifies the error. May appear as a trailing header on multipart write complete requests. | All | X | |||
Castor‑System‑IsVersioned | Header indicating the object versioning state in effect at the time the object was written. May only appear on alias or named objects. True if the object was written in the versioning‑Enabled state. "False" if the object was written in the versioning‑Suspended state. There is no header otherwise. A flag for the object's Swarm versioning status, which does not appear on objects created in a versioning‑disabled context. True means it was created in the versioning‑Enabled state. False means it was created in the versioning‑Suspended state. | GET, HEAD | R | P | ||
Castor‑System‑LicenseCapacityTB | Return on a cluster status request. Indicates the licensed capacity, in TB. | X | ||||
Castor‑System‑LicenseSerialNumber | Returned on a cluster status request. Indicates the license serial number, or 'unregistered.' | X | ||||
Castor‑System‑Name | Named objects only. Symbolic name of the object, which is the user‑Specified (partial) name for this object. | GET, HEAD, DELETE | X | R | P | |
Castor‑System‑Next‑Version | Appears only on admin or administrative GET or HEAD responses. When object versioning is enabled or suspended, named and alias objects include these headers if there is a previous or next version in the version chain for the object. | GET, HEAD | X V | |||
Castor‑System‑Owner | The user id of the authenticated user who wrote the object. Name of the user who created or last modified the object. If the object was created anonymously (that is, without authenticating), this header is absent. | All | R | P | ||
Castor‑System‑PartNumber | Special headers for multipart write requests. Identifies the part number of a part in the temporary manifest for a multipart write. | POST, PUT | X | W | P | |
Castor‑System‑Path | For named objects only, shows the full path of the object in /domain/bucket/object format. Example: /cluster.example.com/mybucket/mypath/myobject.html. Computed on GET and HEAD. Returns <domain>/<bucket>/<name>. | GET, HEAD | X V | R | P | |
Castor‑System‑Persisted-Headers | Indicates which of the other headers on the response are persisted. | GET, HEAD | X V | |||
Castor‑System‑Previous‑Version | Appears only on admin or administrative GET or HEAD responses. When object versioning is enabled or suspended, named and alias objects include these headers if there is a previous or next version in the version chain for the object. | GET, HEAD | X V | |||
Castor‑System‑TotalGBAvailable | A response header on the status page. | GET | X | |||
Castor‑System‑TotalGBCapacity | A response header on the status page. | GET | X | |||
Castor‑System‑UploadID | Special header for multipart write requests. As a response header, it identifies the multipart write request for all subsequent requests relating to the upload. As a persisted header, it associates a stream with a multipart upload, either as the init stream or a part, based on its Castor-System-PartNumber value. | GET, HEAD | X | R | P | |
Castor‑System‑Version | A numerical version number associated with mutable objects. This is a UNIX‑Style floating point time since GMT epoch with millisecond accuracy. The timestamp (in seconds since epoch) when the object was written, which corresponds to Last‑Modified. Not writable: Although it is persisted, this header cannot be supplied on any non‑admin requests. | All | X | R | P | |
Composite-Content-MD5 | Provides an end‑to‑end integrity check of the content (excluding metadata) of a parallel write request at the time of completion. The supplied value is a base-64 encoding of the concatenation of the binary Content-MD5 hashes of the parts (in order) followed by a dash, followed by the number of parts as text value. On the complete request, Swarm computes the value for the overall request and reject the request with a 409 error if the values do not match. | POST | X | X | W | P |
Content-MD5 | Users can compute the md5 sum of the object prior to write. The header is preserved and checked on GET. A persisted Content-MD5 can also be generated by Swarm. | POST, PUT, GET, HEAD | X | X | W | P |
Content-type: | Required to create a context. Content-Type is a standard HTTP header. | POST, PUT | X | |||
Content‑UUID | One header per UUID specified. Indicates the alias or primary uuid from a diskless countrep HEAD request. Also on a write request indicating the new content uuid. Unnamed objects only. COPY and PUT are invalid methods for immutable objects. Not writable: Although it is persisted, this header cannot be supplied on any non‑admin requests. Is returned as a trailing header on multipart write completes. | All | X | W | P | |
Feed‑{id}‑Status | On a verbose GET or HEAD, returns the status of each defined feed, one per header. The {id} is the ID field from the feed definition. The value is 0 for success (no writes remaining), 1 for timeout, or else the number of attempts made to write the object. | GET, HEAD | X V | |||
Feed‑{id}‑StatusTime | On a verbose GET or HEAD, returns the status time of each defined feed, one per header. Time is HTTP time. | GET, HEAD | X V | |||
Lifepoint | Time‑based SCSP and HP directives. Returned for an object created with a lifepoint header. | All | X | X | W | P |
Manifest | Indicates the object is erasure coded. Indicates the manifest type of the object being created or queried. Internal. Used with replica transfer. Currently, "ec" is the only value used. Indicates the manifest type of the object being created or queried. Also returned on a verbose GET/HEAD for EC objects. | All | X V | |||
Node-Status | Return on a cluster status request. Indicates the current node status. | X | ||||
Overlay‑IPs | A list of space‑separated IP addresses associated with the overlay key. Only on a countreps HEAD request. Up to two IP headers may appear. | X | ||||
Overlay‑Key | An overlay index key used for this object. Only on a countreps HEAD request. Up to two keys may appear. | X | ||||
Policy-* | Include Policy-* headers in the header inclusion list or use the preserve query argument for COPY. | COPY | W | P | ||
Policy-ECEncoding | Optional; context objects only. Stores the encoding policy for erasure coding named objects in this context. Valid values: unspecified, disabled, k:p (a tuple such as 5:2 that specifies the data and parity encoding to use). For a domain, appending "anchored" cancels any policies on its buckets. | All | X | X | W | P |
Policy-ECEncoding-Unnamed | Optional; domain objects only. Stores the encoding policy for erasure coding unnamed objects tenanted in this domain. Valid values: unspecified, disabled, k:p (a tuple such as 5:2 that specifies the data and parity encoding to use). | All | X | X | W | P |
Policy-ECMinStreamSize | Optional; context objects only. Stores the minimum object size that triggers erasure coding of named objects in this context. In units of megabytes (MB) or gigabytes (GB); must be 1MB (default) or greater. For a domain, appending "anchored" cancels any policies on its buckets. | All | X | X | W | P |
Policy-ECMinStreamSize-Unnamed | Optional; domain objects only. Stores the minimum object size that triggers erasure coding of unnamed objects tenanted in this domain. In units of megabytes (MB) or gigabytes (GB); must be 1MB (default) or greater. | All | X | X | W | P |
Policy-Replicas | Optional; context objects only. Stores the min, max, and/or default number of replicas that the cluster maintains for the objects in this context. For a domain, appending "anchored" cancels any policies on its buckets. | All | X | X | W | P |
Policy‑{Feature}‑Evaluated | Evaluated policy literals, but only on contexts and the cluster status page. This header appears on GET or HEAD requests of context objects. To view it for all objects, add the "verbose" query argument. | GET, HEAD | X V | |||
Policy‑{Feature}‑Unnamed-Evaluated | Evaluated policy literals for domain objects only, related to unnamed objects therein. This header appears on GET or HEAD requests of domain objects. | GET, HEAD | X | |||
Policy‑{Feature}‑Unnamed-Evaluated-Constrained | Reports what existing constraints affect the policy evaluation for unnamed objects in a domain. If "no", nothing constrains the policy at this level. If "yes", something (anchor, override, conflict) is canceling the policy at this level. Values such as "min>2" summarize the constraints in force. This header appears on GET or HEAD requests of domain objects. | GET, HEAD | X | |||
Policy‑{Feature}‑Value-Constrained | Reports what existing constraints affect the policy evaluation. If "no", nothing constrains the policy at this level. If "yes", something (anchor, override, conflict) is canceling the policy at this level. Values such as "min>2" summarize the constraints in force. This header appears on GET or HEAD requests of context objects. To view it for all objects, add the "verbose" query argument. | GET, HEAD | X V | |||
Policy‑Versioning | Optional; context objects only. Stores the versioning policy for objects in this context. Valid states: disabled, enabled, suspended, required. | All | X | X | W | P |
Replica‑Count | Returns the number of replicas found with a countreps HEAD. Returns the number of replicas created on a ROW or RmOW write. One header per UUID with a diskless countreps HEAD. Specifies the number of known replicas of the object in the cluster. Is returned only if the countreps=yes query argument is included in the request. Not writable: Although it is persisted, this header cannot be supplied on any non‑admin requests. | HEAD | X | W | P | |
ScspHoldBucket | The SCSP hold bucket on a hold request. | HOLD | X | |||
ScspHoldDomain | The SCSP hold domain on a hold request | HOLD | X | |||
Volume | Indicates the volume UUID where one or more replicas is stored. Used with replica transfer. On a HEAD with the countreps query argument, the ids of the volume of all replicas are individually returned as separate headers. For a replicate‑on‑write request, there are two sets of Location and Volume headers. Not writable: Although it is persisted, this header cannot be supplied on any non‑admin requests. Indicates the volume UUID where one or more replicas is stored. The Volume headers generated usually correspond to the Location headers generated. | POST, PUT, COPY, APPEND | X | X | R | P |
X‑{anything}‑Meta[‑{anything}] | Swarm custom metadata header. Uninterpreted; meant for client customization. Custom metadata has the format X‑*‑Meta[‑*], such as X‑color‑Meta or X‑phonehome‑Meta‑Castor‑cluster‑id. The name is case‑insensitive (consistent with the HTTP/1.1 RFC). Important ‑ Custom metadata that does not match this form (such as X‑Meta‑color) is not persisted. | POST, PUT, COPY | X | X | W | P |
X‑Castor‑Copy‑Source | Required for parallel write part upload that copies content from an existing object. Specifies the source object by alias, UUID, or name (not Etag). Names are in the form "bucket/object." | POST | X | |||
X‑Castor‑Copy‑Source-Range | Optional for parallel write part upload as a copy, failing the request if it cannot be read from the specified range. Given as bytes in the form of first - last. | POST | X | |||
X‑Castor‑Copy‑Source‑Domain | Optional for parallel write part upload as a copy. Supplies the domain for a named or tenanted object, failing the request if it does not exist. | POST | X | |||
X‑Castor‑Copy‑Source‑If‑Match | Optional for parallel write part upload as a copy, returning 304 or 412 if the condition is not met. Value is an ETag (enclosed in quotes). | POST | X | |||
X‑Castor‑Copy‑Source‑If‑Modified‑Since | Optional for parallel write part upload as a copy, returning 304 or 412 if the condition is not met. Timestamp in the format of the Last-Modified header. | POST | X | |||
X‑Castor‑Copy‑Source‑If‑None‑Match | Optional for parallel write part upload as a copy, returning 304 or 412 if the condition is not met. Value is an ETag (enclosed in quotes). | POST | X | |||
X‑Castor‑Copy‑Source‑If‑Unmodified‑Since | Optional for parallel write part upload as a copy, returning 304 or 412 if the condition is not met. Timestamp in the format of the Last-Modified header. | POST | X | |||
X‑Timestamp | On a bucket list request, the value of the Castor‑System‑Created header. | GET | X |
Standard HTTP Headers
Header | RFC Section | Methods | Rq | Rs | W | P | Description |
---|---|---|---|---|---|---|---|
Accept | Not used by Swarm Storage. | ||||||
Accept-Charset | X | Specifies which character encodings (charsets) are acceptable for the response. Used in the console to resolve client requests. | |||||
Accept-Encoding | X | Implemented in accordance with RFC-2616. | |||||
Accept-Language | X | Specifies which natural languages are acceptable for the response. Used in the console to resolve client requests. | |||||
Accept-Ranges | Not used by Swarm Storage. | ||||||
Age | GET, HEAD | X | The age in seconds of an item read from cache. Also sent on the status page (Age: 0). Returned when an object is served from the Swarm content cache. If Age is absent, indicates the object was retrieved from disk. | ||||
Allow | All | X | X | W | P | For a POST request on an object, restricts subsequent access to the object. For a GET request, returns the list of allowed methods for that object. Delete protection — Allow headers have no effect on automatic deletes specified in Lifepoint headers. Use | |
Authorization | X | Answer an authorization challenge. Usually a repeated request after receiving a 401 (Unauthorized). | |||||
Cache-Control | GET, HEAD, POST | X | X | W | P | Caching-related header. Specifies directives to be obeyed by all caching mechanisms along the request/response chain. Values include max-age, no-cache, and no-cache-context. Returned as metadata. Exactly matches what was sent with the object on POST. Serves as both a request header and a persisted header, which Specifies whether readers of the object access the object from cache. | |
Connection | All | X | X | An action to perform on the connection. Includes "close", to request the client close the connection. The Swarm software implements HTTP/1.1 persistent connections. A client application is not required to close the socket/connection after each request. This header may also be absent. | |||
Content-Encoding | All | X | W | P | Activates decoding behavior. Uses registered decoders for decoding. | ||
Content-Language | All | W | P | Uninterpreted. Specifies the language(s) of the entity. | |||
Content-Length | All | X | X | W | P | The exact number of bytes (possibly zero) comprising the content contained in the message body. Exception: If adding Swarm does not update Content-Length on a COPY, and it fails any COPY requests with a Content-Length > 0. Do not send this header on a COPY unless it has the value 0. | |
Content-Location | W | P | Not used by Swarm Storage. | ||||
Content-MD5 | POST, PUT | X | W | P | An MD5 hash of the content of the object. Can be set on a write operation or request it be computed. This value may be synthesized for some GET requests if it is not part of the persisted headers. This header provides an end-to-end message integrity check of the content (excluding metadata) as it is sent to and returned from Swarm. A proxy or client can check this header to detect accidental modification of the entity-body in transit. A client can provide this header to indicate Swarm needs to compute and check it as it is storing or returning the object data. Swarm fails a COPY request with Content-MD5 if the object is stored erasure-coded and the Content-MD5 was not stored in the original POST or PUT. Do not add Content-MD5 with a COPY to an EC object; include it in COPY requests for both erasure-coded and non-erasure-coded objects if it already existed on the object. S3 compatibility — The Swarm setting scsp.autoContentMD5Computation improves S3 compatibility by automating Content-MD5 hashing. The gencontentmd5 query argument or the deprecated Expect: Content-MD5 header on writes does not need to be included (although a separate Content-MD5 header for content integrity checking may want to be supplied). This setting is ignored wherever it is invalid, such as on a multipart initiate/complete or an EC APPEND. (v9.1) | ||
Content-Range | X | Sent with a partial entity-body to specify where in the full entity-body the partial body should be applied. Appears on read range responses to indicate the actual ranges returned. | |||||
Content-Type | All | X | W | P | Media type as specified in the corresponding POST or PUT request. The value should be a valid media type registered with the IANA, but Swarm does not verify this or make assumptions about the content type or structure. This response can include other headers that contain meta-information supplied by the application that stored the content. In addition to a persisted content type, this value may appear on read range responses to indicate a multi-part response. castorcontext — | ||
Cookie | Not used by Swarm Storage. | ||||||
Date | All | X | X | The HTTP time of the message. The current date/time on the Swarm node at the time of the request. | |||
Destination | Not used by Swarm Storage. | ||||||
ETag | All | X | R | P | Caching-related header. The ETag (entity tag) of the specific variant of the object. The value is a double-quoted UUID. The ETag of an immutable unnamed object does not change during the entire lifecycle of the object, whereas alias object ETags change each time the object is mutated by a PUT. When Versioning is enabled, the ETag identifies the version of the object. Not writable: Although it is persisted, this header cannot be supplied on any non-admin requests. | ||
Expect | X | Indicates that particular server behaviors are required by the client. Values include | |||||
Expires | GET, HEAD | X | W | P | Caching-related header. Specifies the date/time after which the response is considered stale, for caching purposes. Uninterpreted. Returned as metadata. Matches what was sent with the object on POST. | ||
From | Not used by Swarm Storage. | ||||||
Host | The Host header field in a request provides the host and port information from the target URI, enabling the origin server to distinguish among resources while servicing requests for multiple host names on a single IP address. Swarm uses the Host header in many cases as a means of specifying the domain of the request. | ||||||
If-Match | All | X | Caching-related header. Used with a method to make it conditional. | ||||
If-Modified-Since | GET, HEAD | X | Caching-related header. Cache coherency headers. Per the RFC, Swarm makes no attempt to enforce "If-Modified-Since" on DELETE, PUT, or COPY requests. (v9.2) | ||||
If-None-Match | All | X | Caching-related header. Cache coherency headers. Note: If-None-Match:* can erroneously report an object exists during the time window after it is flagged for deletion by policy but before it is removed from disk. This window is determined by the HP cycle time. | ||||
If-Range | GET | X | Caching-related header. Cache coherency headers. | ||||
If-Unmodified-Since | GET, PUT, DELETE | X | Caching-related header. Cache coherency headers. | ||||
Last-Modified | All but POST | X | R | P | Caching-related header. Exactly the same as Castor-System-Created. Not writable: Although it is persisted, this header cannot be supplied on any non-admin requests. | ||
Link | Not used by Swarm Storage. | ||||||
Location | X | Values indicate how to access one or more replicas of the object directly. May be multi-valued indicating the locations of multiple new replicas. | |||||
Max-Forwards | Not used by Swarm Storage. | ||||||
Pragma | Not used by Swarm Storage. | ||||||
Proxy-Authenticate | Not used by Swarm Storage. | ||||||
Proxy-Authorization | Not used by Swarm Storage. | ||||||
Range | X | Indicates a range of data is requested. | |||||
Referer | Not used by Swarm Storage. | ||||||
Retry-After | Not used by Swarm Storage. | ||||||
Server | All | X | Swarm software version running on the responding node. The server name and version. CAStor Cluster/{version}. | ||||
Set-Cookie | Not used by Swarm Storage. | ||||||
TE | Not used by Swarm Storage. | ||||||
Trailer | X | X | Indicates a trailer is sent during chunked transfer encoding. | ||||
Transfer-Encoding: chunked | POST, PUT, APPEND | X | X | Standard header that indicates a large object to be sent to the cluster using chunked transfer encoding. Indicates that the data is being sent with an alternate transfer encoding. Values include "chunked" and "bundle". The latter is used internally for FVR of small objects (non-Standard). | |||
Upgrade | Not used by Swarm Storage. | ||||||
User-Agent | X | Standard HTTP header for a client to identify itself. | |||||
Vary | Not used by Swarm Storage. | ||||||
Via | Not used by Swarm Storage. | ||||||
Warning | Not used by Swarm Storage. | ||||||
WWW-Authenticate | X | Indicates an authentication challenge. Usually associated with a 401 (Unauthorized) response. |
© DataCore Software Corporation. · https://www.datacore.com · All rights reserved.