The S3 protocol personality is a front-end storage protocol for client applications. It runs within the Gateway itself. All of the Gateway deployment and scaling features apply when you use the S3 front-end protocol.
An administrator can configure the Gateway to run several ways:
only the SCSP protocol
only the S3 protocol
both protocols at the same time
Additionally, Gateways can be scaled horizontally with any combination of front-end protocols as needed for a particular deployment. This allows for a heterogeneous mix of client types that utilize the same Swarm storage cluster and that share content with each other. Doing this allows an administrator to provide a unified object storage platform.
Sharing Storage across S3 and SCSP
Content Gateway provides the mechanism to unify the back-end Swarm object storage so that S3 applications and SCSP applications can share content. In a unified object storage deployment, a device like a load balancer is used to route incoming client traffic through the appropriate port number or pool of Gateway servers.
Routing Methods
These are some routing methods that can be used for a unified object storage front-end:
IP address | Listen to multiple virtual IP addresses and distribute traffic based upon the IP address used by the client. | |
---|---|---|
X-Forwarded-Host | Use Layer 7 inspection of the requests to distribute traffic based upon headers contained in the client requests. | The X-Forwarded-Host header can be used by a proxy or load balancer to let client applications use a different host name for each supported HTTP storage protocol while sharing the same storage domain for their content. For example, consider the storage domain In order to direct the storage requests to the shared storage domain, the load balancer needs to add the header: X-Forwarded-Host: castor.example.com |
DNS | Cause the DNS name resolution to be different for the clients using one storage protocol than for the clients using another protocol. | For example, the clients using S3 may resolve the storage domain castor.example.com to 10.100.100.81 while clients using SCSP resolve the same storage domain to 10.100.100.82. While this method does not require in-line modification of the HTTP requests, it does require that the administrator have control of the hosts where the client applications run so as to allow alteration of the DNS/host resolution. |
OPTIONS | Route request method OPTIONS with "Origin" to the S3 port rather than default to the SCSP port (which happens when an Authorization header does not exist or have "AWS"). | S3 must handle "bucket in Host" style requests because SCSP reports that the domain was not found. curl -i -H 'Origin: http://www.example.com' -H 'Access-Control-Request-Method: PUT' -X OPTIONS https://mycorsbucket.elsewhere.com/ |
Pattern-matching | Use Layer 7 inspection of the requests to switch incoming traffic based on the object storage protocol. See below: | This is done by looking for the distinctive S3 Authorization header pattern or one of the query string arguments for authenticated S3 requests. The pattern-matching rules for these authenticated S3 requests are as follows. |
| Use pattern-matching on the request header. | Authorization ~= ^AWS.* |
| Use pattern-matching on the query string arguments. | {RequestURL} ~= [?&](AWSAccessKeyId|X-Amz-Credential)= |
| Use pattern-matching to test for the absence of all AWS request patterns. | The request is either an anonymous S3 request or an SCSP request. Since anonymous S3 requests should not create, update, or delete content, they are most likely GET requests, and it is safe to allow SCSP to handle these. |