The SCSP SEND method applies to both named and unnamed objects. The SEND request allows explicit transmission of a newly written object from a source cluster to a remote one, such as for keeping two clusters immediately synchronized. The feed SEND method works with any feed type as of Swarm 11.2, so it can force synchronous processing of a specific object on one or more of those feeds.
Best practice — Use this feature with a replication feed, which acts as a catch-up mechanism if the intracluster network is down or the SEND command fails.
Legacy SEND
The legacy behavior of the SEND method (which was limited to legacy replication feeds) is replaced by the following expanded SEND method. The legacy behavior is preserved for backwards compatibility: invoke it by omitting the required “feedid” or “feedtype” query arguments. (v11.2)
SEND Requests
With a SEND request, provide the path or UUID for the Swarm object to be sent to one or more feeds. Swarm checks the destination cluster to verify whether the object already exists there.
Which node to SEND to — Content Gateway determines the optimal target node for the request if using SEND through Content Gateway (under development for future release); select the node if going direct to Storage. All replicas must perform feed processing, but, on a new write, one replica begins replication and S3 backup processing. A SEND request has the best chance of arriving with the replication already in progress, which speeds completion if pointing SEND to the optimal node (which holds this first replica). Determine whether the request is an EC write and whether the request is a multipart completion to find the optimal SEND node. Identify an EC write by the “Manifest: ec” response header.
Use the first
Locationheader’shostfor the SEND for normal EC write responses.Use the last
Locationheader’shostfor non-EC write responses and for SEND after multipart completes.
Request headers — No special headers are expected or used on the request. Do not include the legacy SEND request headers:
Castor-System-Cluster
Castor-System-Type
Castor-System-Auth
Query Arguments for SEND
The SEND request needs query arguments for feedid, feedtype, or both, which is a union of all those provided arguments; SEND reverts to the legacy behavior if neither is provided.
Argument | Values, Examples | Notes |
|---|---|---|
admin | none | SEND is used by an |
feedid | all | integer
| Specifies one or more specific feeds as the replication destination. Reference existing feed IDs (  Swarm returns a 400 Bad Request error if an integer value is not an existing feed. The SEND operation succeeds if the object being sent does not match the feed definition (because of a domain restriction), but no data is transferred. |
feedtype | all | search | replication | s3backup
| Specifies one or more types of feeds as the replication destination, from among these values: Swarm returns an HTTP 400 Bad Request error if the feedtype is not a valid value. The SEND operation succeeds if the object being sent does not match the feed definition (because of a domain restriction), but no data is transferred. |
timeout | true | number of seconds | false
| Sets how long to wait for replication to complete; if disabled ( Using timeout=true waits for the Swarm setting scsp.defaultFeedSendTimeout time in seconds, which defaults to 30. Specifying a positive number for the timeout overrides the value in the Swarm setting. |
force | yes | no | true | false | If the value is true or yes, Swarm forces re-processing of the object though any associated feeds for objects previously processed. |
SEND Responses
The request returns information about that request in the body of the response. SEND operates like a HEAD request, with the headers of the response resembling that of a HEAD request.
Chunked encoding
The SEND response is chunked transfer encoded, so the client of the SEND request must be prepared for chunked transfer encoding.
The response body may contain additional leading newlines sent incrementally, which keeps the connection open in long requests. The body of the response can be ignored; the trailing headers are repeated at the end of the body to support clients that cannot handle trailing headers, such as curl.
SEND SCSP returns an HTTP 200 OK (request has been completed), even for timeouts.
Response Headers
Check the replication status for the object in these response headers:
Feed-<id>-Status — (The same value as a verbose HEAD/GET request.) The ID refers to the Swarm-assigned ID field in the feed definition.
0 is a success; no writes remain to be completed.
1 is a timeout.
Any other positive number is a failure.
Feed-<id>-StatusTime — (The same value as a verbose HEAD/GET request). The HTTP time of the last replication attempt or success. Feed-<id>-StatusTime are blank if Feed-<id>-Status is 1 because the object has not been processed by the feed.
The response is chunked encoded and may include trailing headers:
The above feed statuses are provided immediately in the HTTP 200 response headers for feeds with a successful status (0) prior to the request. There are no feed status headers if there is no matching feed.
Any other feed statuses are provided as trailing headers. A newline keep-alive chunk is sent periodically as per scsp.keepAliveInterval during processing. At the end of the quest, the trailing headers are sent both in the body of the request and as trailing headers. All results are sent at the end of the response, not when processing completes for an individual feed.
A failure or timeout response is provided with no automatic retries if a feed-to-be-processed is blocked or paused.
Example of SEND
The following request transfers the unnamed object to all clusters that are the targets of replication feeds:
curl -i -X SEND --location-trusted
--anyauth -u admin:ourpwdofchoicehere
"http://192.168.1.12:80/97f7149dec6cbc0aa1e9425688158969
?feedtype=replication&timeout=true&admin"
See also SCSP SEND - legacy.