SCSP Headers

Header Limits: Headers (metadata) are constrained not only by Swarm but by all services, proxies, and clients (such as Elasticsearch, Twisted, Jetty, HAProxy) handling objects. A Swarm object may have any number of annotations associated with it, but each annotation object is subject to these limits. The persisted metadata on Swarm objects must fall within these limits by default:

  • 500 - The total number of headers on an object

  • 32 KB - The combined length of all headers (key/value pairs); exceeding this returns a 400 (Bad Request) error

  • 16 KB - The maximum length for a given header (key/value pair)

The following tables list all headers supported or used by Swarm. The headers described apply to buckets, named objects, and aliased objects unless otherwise noted.

  • Methods are the SCSP methods to which the header applies.

  • Request (signified by X)

  • Response (signified by X)

    • V means it appears only with verbose=true

    • T means it appears in a trailing response

  • Writable (signified by W)

    • R means it is read-only

  • Persisted indicates whether the header is persisted with the object. Note: some persisted headers are not writable (signified by P).

Best Practice

Always use case-insensitive header matching.

Swarm-Specific Headers

Header

Description

Method

Request

Response [Trailing] [Verbose]

Writable [Read-only]

Persisted

Header

Description

Method

Request

Response [Trailing] [Verbose]

Writable [Read-only]

Persisted

Castor-System-Accessed

Used on GET/HEAD requests to indicate the time the object was last accessed (written or GET).

GET, HEAD



X V





Castor-System-CompositeMD5

Multipart-written objects only. This composite value, stored with the completed object, is computed on the multipart complete as the sum of the Content-MD5 values of all parts. This header value is reused on subsequent COPY operations but removed on other updates (PUT, APPEND).

GET, HEAD, POST



T





Castor-System-Decorates

Set to the ETag of a Swarm object to be decorated. When the decorated object is deleted or overwritten, the object with this header is reclaimed by the health processor.

All

X

X

W

P

Castor-System-Headers-Filtered

If response headers are filtered by a whitelist or blacklist, this header is added.

GET, HEAD



X

R



Castor-System-InProgress

Returned with value yes on a verbose request when the inprogress flag is set on a stream. Not returned at all when the flag is not set.

GET, HEAD



X V





Castor-System-Result

Reports the response code, such as 201 (Created) or 404 (Not Found), as a trailing header for a multipart write completion.

POST



T





Castor-System-Tiered

Audit headers for objects restored by an S3 Backup feed. One copy captures the date and source cluster of the backup, and another captures the date and S3 host/bucket of the restore. Dates are same format as Last-Modified.

GET, HEAD



X V

R

P

Castor‑{anything}

{anything} cannot start with "system". Uninterpreted; meant for client customization.

All





W

P

Castor‑Authorization

Content‑level authorization.

All

X



W

P

Castor‑System-Bytes‑Used

With bucket list du, the number of bytes in the query without replicas.

GET (listing)



X





Castor‑System-Bytes‑Used‑With‑Reps

With bucket list du / withreps requests, the number of bytes in the query with replicas.

GET (listing)



X





Castor‑System-Object‑Count

In a bucket listing request, the number of objects returned by the query.





X





Castor‑System‑Alias

The alias UUID of an alias object, bucket, or domain. This UUID cannot be used in any SCSP method on a domain, bucket, or named object. Use it to execute SCSP methods on an aliased object or to delete an unnamed immutable object.

GET, HEAD, DELETE



X

R

P

Castor‑System‑Auth

Special headers for a GET/retrieve request. The username:password for an administrative account on the remote cluster, only if it differs from the source cluster.

SEND

X







Castor‑System‑CID

For named and tenanted unnamed objects, the alias UUID of the owning context.

GET, HEAD, DELETE



X

R

P

Castor‑System‑Cluster

Name of the cluster where the object was created or last updated. For SEND, The value of the cluster.name setting of the destination cluster.

GET, HEAD, DELETE, SEND



X

R

P

Castor‑System‑Created

A timestamp that specifies when the object was created or last updated. Uses HTTP time, an ASCII format based on GMT.

GET, HEAD, DELETE



X

R

P

Castor‑System‑Domain

Named objects only. Shows the name of the domain where the object was created. The domain name in which this object is tenanted. Computed on GET and HEAD. Returns <domain>.

GET, HEAD



X V

R

P

Castor‑System‑EnforceTenancy

True or False depending on cluster.enforceTenancy. A response header on the status page.

GET



X





Castor‑System‑Error‑Code

Sent with any error response. The request error code (if applicable). This code is usually a 4xx or 5xx HTTP response code. May appear as a trailing header on multipart write complete requests.

All



X





Castor‑System‑Error‑Text

Sent with any error response. The request error description (if applicable). Provides a description of the error. May appear as a trailing header on multipart write complete requests.

All



X





Castor‑System‑Error‑Token

Sent with any error response. A unique error token (if applicable). Provides a parsed token that uniquely identifies the error. May appear as a trailing header on multipart write complete requests.

All



X





Castor‑System‑IsVersioned

Header indicating the object versioning state in effect at the time the object was written. May only appear on alias or named objects. True if the object was written in the versioning‑Enabled state. "False" if the object was written in the versioning‑Suspended state. There is no header otherwise. A flag for the object's Swarm versioning status, which does not appear on objects created in a versioning‑disabled context. True means it was created in the versioning‑Enabled state. False means it was created in the versioning‑Suspended state.
9.6.a: Maps to "versioned" (boolean) search query argument.

GET, HEAD





R

P

Castor‑System‑LicenseCapacityTB

Return on a cluster status request. Indicates the licensed capacity, in TB.





X





Castor‑System‑LicenseSerialNumber

Returned on a cluster status request. Indicates the license serial number, or 'unregistered.'





X





Castor‑System‑Name

Named objects only. Symbolic name of the object, which is the user‑Specified (partial) name for this object.

GET, HEAD, DELETE



X

R

P

Castor‑System‑Next‑Version

Appears only on admin or administrative GET or HEAD responses. When object versioning is enabled or suspended, named and alias objects include these headers if there is a previous or next version in the version chain for the object.

GET, HEAD



X V





Castor‑System‑Owner

The user id of the authenticated user who wrote the object. Name of the user who created or last modified the object. If the object was created anonymously (that is, without authenticating), this header is absent.

All





R

P

Castor‑System‑PartNumber

Special headers for multipart write requests. Identifies the part number of a part in the temporary manifest for a multipart write.

POST, PUT

X



W

P

Castor‑System‑Path

For named objects only, shows the full path of the object in /domain/bucket/object format. Example: /cluster.example.com/mybucket/mypath/myobject.html. Computed on GET and HEAD. Returns <domain>/<bucket>/<name>.

GET, HEAD



X V

R

P

Castor‑System‑Persisted-Headers

Indicates which of the other headers on the response are persisted.

GET, HEAD



X V





Castor‑System‑Previous‑Version

Appears only on admin or administrative GET or HEAD responses. When object versioning is enabled or suspended, named and alias objects include these headers if there is a previous or next version in the version chain for the object.

GET, HEAD



X V





Castor‑System‑TotalGBAvailable

A response header on the status page.

GET



X





Castor‑System‑TotalGBCapacity

A response header on the status page.

GET



X





Castor‑System‑UploadID

Special header for multipart write requests. As a response header, it identifies the multipart write request for all subsequent requests relating to the upload. As a persisted header, it associates a stream with a multipart upload, either as the init stream or a part, based on its Castor-System-PartNumber value.

GET, HEAD

X



R

P

Castor‑System‑Version

A numerical version number associated with mutable objects. This is a UNIX‑Style floating point time since GMT epoch with millisecond accuracy. The timestamp (in seconds since epoch) when the object was written, which corresponds to Last‑Modified. Not writable: Although it is persisted, this header cannot be supplied on any non‑admin requests.

All



X

R

P

Composite-Content-MD5

Provides an end‑to‑end integrity check of the content (excluding metadata) of a parallel write request at the time of completion. The supplied value is a base-64 encoding of the concatenation of the binary Content-MD5 hashes of the parts (in order) followed by a dash, followed by the number of parts as text value. On the complete request, Swarm computes the value for the overall request and reject the request with a 409 error if the values do not match.

POST

X

X

W

P

Content-MD5

Users can compute the md5 sum of the object prior to write. The header is preserved and checked on GET. A persisted Content-MD5 can also be generated by Swarm.

POST, PUT, GET, HEAD

X

X

W

P

Content-type:
application/castorcontext

Required to create a context. Content-Type is a standard HTTP header.

POST, PUT

X







Content‑UUID

One header per UUID specified. Indicates the alias or primary uuid from a diskless countrep HEAD request. Also on a write request indicating the new content uuid. Unnamed objects only. COPY and PUT are invalid methods for immutable objects. Not writable: Although it is persisted, this header cannot be supplied on any non‑admin requests. Is returned as a trailing header on multipart write completes.

All



X

W

P

Feed‑{id}‑Status

On a verbose GET or HEAD, returns the status of each defined feed, one per header. The {id} is the ID field from the feed definition. The value is 0 for success (no writes remaining), 1 for timeout, or else the number of attempts made to write the object.

GET, HEAD



X V





Feed‑{id}‑StatusTime

On a verbose GET or HEAD, returns the status time of each defined feed, one per header. Time is HTTP time.

GET, HEAD



X V





Lifepoint

Time‑based SCSP and HP directives. Returned for an object created with a lifepoint header.

All

X

X

W

P

Manifest

Indicates the object is erasure coded. Indicates the manifest type of the object being created or queried. Internal. Used with replica transfer. Currently, "ec" is the only value used. Indicates the manifest type of the object being created or queried. Also returned on a verbose GET/HEAD for EC objects.

All



X V





Node-Status

Return on a cluster status request. Indicates the current node status.





X





Overlay‑IPs

A list of space‑separated IP addresses associated with the overlay key. Only on a countreps HEAD request. Up to two IP headers may appear.





X





Overlay‑Key

An overlay index key used for this object. Only on a countreps HEAD request. Up to two keys may appear.





X





Policy-*

Include Policy-* headers in the header inclusion list or use the preserve query argument for COPY.

COPY





W

P

Policy-ECEncoding

Optional; context objects only. Stores the encoding policy for erasure coding named objects in this context. Valid values: unspecified, disabled, k:p (a tuple such as 5:2 that specifies the data and parity encoding to use). For a domain, appending "anchored" cancels any policies on its buckets.

All

X

X

W

P

Policy-ECEncoding-Unnamed

Optional; domain objects only. Stores the encoding policy for erasure coding unnamed objects tenanted in this domain. Valid values: unspecified, disabled, k:p (a tuple such as 5:2 that specifies the data and parity encoding to use).

All

X

X

W

P

Policy-ECMinStreamSize

Optional; context objects only. Stores the minimum object size that triggers erasure coding of named objects in this context. In units of megabytes (MB) or gigabytes (GB); must be 1MB (default) or greater. For a domain, appending "anchored" cancels any policies on its buckets.

All

X

X

W

P

Policy-ECMinStreamSize-Unnamed

Optional; domain objects only. Stores the minimum object size that triggers erasure coding of unnamed objects tenanted in this domain. In units of megabytes (MB) or gigabytes (GB); must be 1MB (default) or greater.

All

X

X

W

P

Policy-Replicas

Optional; context objects only. Stores the min, max, and/or default number of replicas that the cluster maintains for the objects in this context. For a domain, appending "anchored" cancels any policies on its buckets.

All

X

X

W

P

Policy‑{Feature}‑Evaluated

Evaluated policy literals, but only on contexts and the cluster status page. This header appears on GET or HEAD requests of context objects. To view it for all objects, add the "verbose" query argument.

GET, HEAD



X V





Policy‑{Feature}‑Unnamed-Evaluated

Evaluated policy literals for domain objects only, related to unnamed objects therein. This header appears on GET or HEAD requests of domain objects.

GET, HEAD



X





Policy‑{Feature}‑Unnamed-Evaluated-Constrained

Reports what existing constraints affect the policy evaluation for unnamed objects in a domain. If "no", nothing constrains the policy at this level. If "yes", something (anchor, override, conflict) is canceling the policy at this level. Values such as "min>2" summarize the constraints in force. This header appears on GET or HEAD requests of domain objects.

GET, HEAD



X





Policy‑{Feature}‑Value-Constrained

Reports what existing constraints affect the policy evaluation. If "no", nothing constrains the policy at this level. If "yes", something (anchor, override, conflict) is canceling the policy at this level. Values such as "min>2" summarize the constraints in force. This header appears on GET or HEAD requests of context objects. To view it for all objects, add the "verbose" query argument.

GET, HEAD



X V





Policy‑Versioning

Optional; context objects only. Stores the versioning policy for objects in this context. Valid states: disabled, enabled, suspended, required.

All

X

X

W

P

Replica‑Count

Returns the number of replicas found with a countreps HEAD. Returns the number of replicas created on a ROW or RmOW write. One header per UUID with a diskless countreps HEAD. Specifies the number of known replicas of the object in the cluster. Is returned only if the countreps=yes query argument is included in the request. Not writable: Although it is persisted, this header cannot be supplied on any non‑admin requests.

HEAD



X

W

P

ScspHoldBucket

The SCSP hold bucket on a hold request.

HOLD



X





ScspHoldDomain

The SCSP hold domain on a hold request

HOLD



X





Volume

Indicates the volume UUID where one or more replicas is stored. Used with replica transfer. On a HEAD with the countreps query argument, the ids of the volume of all replicas are individually returned as separate headers. For a replicate‑on‑write request, there are two sets of Location and Volume headers. Not writable: Although it is persisted, this header cannot be supplied on any non‑admin requests. Indicates the volume UUID where one or more replicas is stored. The Volume headers generated usually correspond to the Location headers generated.

POST, PUT, COPY, APPEND

X

X

R

P

X‑{anything}‑Meta[‑{anything}]

Swarm custom metadata header. Uninterpreted; meant for client customization. Custom metadata has the format X‑*‑Meta[‑*], such as X‑color‑Meta or X‑phonehome‑Meta‑Castor‑cluster‑id. The name is case‑insensitive (consistent with the HTTP/1.1 RFC). Important ‑ Custom metadata that does not match this form (such as X‑Meta‑color) is not persisted.

POST, PUT, COPY

X

X

W

P

X‑Castor‑Copy‑Source

Required for parallel write part upload that copies content from an existing object. Specifies the source object by alias, UUID, or name (not Etag). Names are in the form "bucket/object."

POST

X







X‑Castor‑Copy‑Source-Range

Optional for parallel write part upload as a copy, failing the request if it cannot be read from the specified range. Given as bytes in the form of first - last.

POST

X







X‑Castor‑Copy‑Source‑Domain

Optional for parallel write part upload as a copy. Supplies the domain for a named or tenanted object, failing the request if it does not exist.

POST

X







X‑Castor‑Copy‑Source‑If‑Match

Optional for parallel write part upload as a copy, returning 304 or 412 if the condition is not met. Value is an ETag (enclosed in quotes).

POST

X







X‑Castor‑Copy‑Source‑If‑Modified‑Since

Optional for parallel write part upload as a copy, returning 304 or 412 if the condition is not met. Timestamp in the format of the Last-Modified header.

POST

X







X‑Castor‑Copy‑Source‑If‑None‑Match

Optional for parallel write part upload as a copy, returning 304 or 412 if the condition is not met. Value is an ETag (enclosed in quotes).

POST

X







X‑Castor‑Copy‑Source‑If‑Unmodified‑Since

Optional for parallel write part upload as a copy, returning 304 or 412 if the condition is not met. Timestamp in the format of the Last-Modified header.

POST

X







X‑Timestamp

On a bucket list request, the value of the Castor‑System‑Created header.

GET



X





Standard HTTP Headers

Header

RFC Section

Methods

Rq

Rs

W

P

Description

Header

RFC Section

Methods

Rq

Rs

W

P

Description

Accept

7231 5.3.2











Not used by Swarm Storage.

Accept-Charset

7231 5.3.3



X







Specifies which character encodings (charsets) are acceptable for the response. Used in the console to resolve client requests.

Accept-Encoding

7231 5.3.4



X







Implemented in accordance with RFC-2616.

Accept-Language

7231 5.3.5



X







Specifies which natural languages are acceptable for the response. Used in the console to resolve client requests.

Accept-Ranges

7233 2.3











Not used by Swarm Storage.

Age

7234 5.1

GET, HEAD



X





The age in seconds of an item read from cache. Also sent on the status page (Age: 0). Returned when an object is served from the Swarm content cache. If Age is absent, indicates the object was retrieved from disk.

Allow

7231 7.4.1

All

X

X

W

P

For a POST request on an object, restricts subsequent access to the object. For a GET request, returns the list of allowed methods for that object.

Delete protection — Allow headers have no effect on automatic deletes specified in Lifepoint headers. Use deletable=no lifepoints for best protection from deletes. Using lifepoints allows blocking recursive deletes when a bucket or domain is deleted, causing Swarm to log a CRITICAL error that non-deletable content is present.

Authorization

7235 4.2



X







Answer an authorization challenge. Usually a repeated request after receiving a 401 (Unauthorized).

Cache-Control

7234 5.2

GET, HEAD, POST

X

X

W

P

Caching-related header. Specifies directives to be obeyed by all caching mechanisms along the request/response chain. Values include max-age, no-cache, and no-cache-context. Returned as metadata. Exactly matches what was sent with the object on POST. Serves as both a request header and a persisted header, which Specifies whether readers of the object access the object from cache.

Connection

7230 6.1

All

X

X





An action to perform on the connection. Includes "close", to request the client close the connection. The Swarm software implements HTTP/1.1 persistent connections. A client application is not required to close the socket/connection after each request. This header may also be absent.

Content-Encoding

7231 3.1.2.2

All



X

W

P

Activates decoding behavior. Uses registered decoders for decoding.

Content-Language

7231 3.1.3.2

All





W

P

Uninterpreted. Specifies the language(s) of the entity.

Content-Length

7230 3.3.2

All

X

X

W

P

The exact number of bytes (possibly zero) comprising the content contained in the message body. Exception: If adding checkIntegrity to HEAD and GET requests for the same object, different Content-Length values are returned in the responses. This occurs because the HEAD response returns the Content-Length of the manifest rather than the object. This is a standard HTTP header used on all requests.

Swarm does not update Content-Length on a COPY, and it fails any COPY requests with a Content-Length > 0. Do not send this header on a COPY unless it has the value 0.

Content-Location

7231 3.1.4.2







W

P

Not used by Swarm Storage.

Content-MD5

2616 14.15

POST, PUT



X

W

P

An MD5 hash of the content of the object. Can be set on a write operation or request it be computed. This value may be synthesized for some GET requests if it is not part of the persisted headers.

This header provides an end-to-end message integrity check of the content (excluding metadata) as it is sent to and returned from Swarm. A proxy or client can check this header to detect accidental modification of the entity-body in transit. A client can provide this header to indicate Swarm needs to compute and check it as it is storing or returning the object data.

Swarm fails a COPY request with Content-MD5 if the object is stored erasure-coded and the Content-MD5 was not stored in the original POST or PUT. Do not add Content-MD5 with a COPY to an EC object; include it in COPY requests for both erasure-coded and non-erasure-coded objects if it already existed on the object.

S3 compatibility — The Swarm setting scsp.autoContentMD5Computation improves S3 compatibility by automating Content-MD5 hashing. The gencontentmd5 query argument or the deprecated Expect: Content-MD5 header on writes does not need to be included (although a separate Content-MD5 header for content integrity checking may want to be supplied). This setting is ignored wherever it is invalid, such as on a multipart initiate/complete or an EC APPEND. (v9.1)

Content-Range

7233 4.2





X





Sent with a partial entity-body to specify where in the full entity-body the partial body should be applied. Appears on read range responses to indicate the actual ranges returned.

Content-Type

7231 3.1.1.5

All



X

W

P

Media type as specified in the corresponding POST or PUT request. The value should be a valid media type registered with the IANA, but Swarm does not verify this or make assumptions about the content type or structure. This response can include other headers that contain meta-information supplied by the application that stored the content. In addition to a persisted content type, this value may appear on read range responses to indicate a multi-part response.

castorcontext Content-type: application/castorcontext specifies that the object is a context (domain or bucket). If the setting scsp.requireExplicitContextCreate is enabled (recommended), Swarm  does not create a context object unless it includes the required header, which protects against erroneous context creation. (v9.1)

Cookie

2109 4.3.4











Not used by Swarm Storage.

Date

7231 7.1.1.2

All

X

X





The HTTP time of the message. The current date/time on the Swarm node at the time of the request.

Destination

2518 9.3











Not used by Swarm Storage.

ETag

7231 2.3

All



X

R

P

Caching-related header. The ETag (entity tag) of the specific variant of the object. The value is a double-quoted UUID. The ETag of an immutable unnamed object does not change during the entire lifecycle of the object, whereas alias object ETags change each time the object is mutated by a PUT. When Versioning is enabled, the ETag identifies the version of the object. Not writable: Although it is persisted, this header cannot be supplied on any non-admin requests.

Expect

7231 5.1.1



X







Indicates that particular server behaviors are required by the client. Values include 100‑continue and Content‑MD5 (unsupported by Gateway and unneeded if scsp.autoContentMD5Computation is enabled; see Content-MD5 Checksums).

Expires

7234 5.3

GET, HEAD



X

W

P

Caching-related header. Specifies the date/time after which the response is considered stale, for caching purposes. Uninterpreted. Returned as metadata. Matches what was sent with the object on POST.

From

7231 5.5.2











Not used by Swarm Storage.

Host

7230 5.4











The Host header field in a request provides the host and port information from the target URI, enabling the origin server to distinguish among  resources while servicing requests for multiple host names on a single IP address. Swarm uses the Host header in many cases as a means of specifying the domain of the request.

If-Match

7232 3.1

All

X







Caching-related header. Used with a method to make it conditional.

If-Modified-Since

7232 3.3

GET, HEAD

X







Caching-related header. Cache coherency headers.

Per the RFC, Swarm makes no attempt to enforce "If-Modified-Since" on DELETE, PUT, or COPY requests. (v9.2)

If-None-Match

7232 3.2

All

X







Caching-related header. Cache coherency headers.

Note: If-None-Match:* can erroneously report an object exists during the time window after it is flagged for deletion by policy but before it is removed from disk. This window is determined by the HP cycle time.

If-Range

7233 3.2

GET

X







Caching-related header. Cache coherency headers.

If-Unmodified-Since

7232 3.4

GET, PUT, DELETE

X







Caching-related header. Cache coherency headers.

Last-Modified

7232 2.2

All but POST



X

R

P

Caching-related header. Exactly the same as Castor-System-Created. Not writable: Although it is persisted, this header cannot be supplied on any non-admin requests.

Link

5988 5











Not used by Swarm Storage.

Location

7231 7.1.2





X





Values indicate how to access one or more replicas of the object directly. May be multi-valued indicating the locations of multiple new replicas.

Max-Forwards

7231 5.1.2











Not used by Swarm Storage.

Pragma

7234 5.4











Not used by Swarm Storage.

Proxy-Authenticate

7235 4.3











Not used by Swarm Storage.

Proxy-Authorization

7235 4.4











Not used by Swarm Storage.

Range

7233 3.1



X







Indicates a range of data is requested.

Referer

7231 5.5.2











Not used by Swarm Storage.

Retry-After

7231 7.1.3











Not used by Swarm Storage.

Server

7231 7.4.2

All



X





Swarm software version running on the responding node. The server name and version. CAStor Cluster/{version}.

Set-Cookie

2109 4.2.2











Not used by Swarm Storage.

TE

7230 4.3











Not used by Swarm Storage.

Trailer

7230 4.4



X

X





Indicates a trailer is sent during chunked transfer encoding.

Transfer-Encoding: chunked

7230 3.3.1

POST, PUT, APPEND

X

X





Standard header that indicates a large object to be sent to the cluster using chunked transfer encoding. Indicates that the data is being sent with an alternate transfer encoding. Values include "chunked" and "bundle". The latter is used internally for FVR of small objects (non-Standard).

Upgrade

7230 6.7











Not used by Swarm Storage.

User-Agent

7231 5.5.3



X







Standard HTTP header for a client to identify itself.

Vary

7231 7.1.4











Not used by Swarm Storage.

Via

7230 5.7.1











Not used by Swarm Storage.

Warning

7234 5.5











Not used by Swarm Storage.

WWW-Authenticate

7235 4.1





X





Indicates an authentication challenge. Usually associated with a 401 (Unauthorized) response.



© DataCore Software Corporation. · https://www.datacore.com · All rights reserved.