Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The S3 protocol personality is a front-end storage protocol for client applications. It runs within the Gateway itself. All of the Gateway deployment and scaling features apply when you use using the S3 front-end protocol.

...

An administrator can configure the Gateway to run several ways:

  • only the SCSP protocol

  • only the S3 protocol

  • both protocols at the same time

Additionally, Gateways can be scaled horizontally with any combination of front-end protocols as needed for a particular deployment. This allows for a heterogeneous mix of client types that utilize utilizing the same Swarm storage cluster and that share sharing content with each other. Doing this allows an administrator to provide a unified object storage platform.

...

Content Gateway provides the mechanism to unify the back-end Swarm object storage so that S3 applications and SCSP applications can share content. In a unified object storage deployment, a device like a load balancer is used to route incoming client traffic through the appropriate port number or pool of Gateway servers.

...

These are some routing methods that can be used usable for a unified object storage front-end:

IP address

Listen to multiple virtual IP addresses and distribute traffic based upon the IP address used by the client.


X-Forwarded-Host

Use Layer 7 inspection of the requests to distribute traffic based upon headers contained in the client requests.

The X-Forwarded-Host header can be used by a proxy or load balancer to let allow client applications to use a different host name for each supported HTTP storage protocol while sharing the same storage domain for their the content. For example, consider Consider the storage domain castor.example.com with two protocol-specific host names scsp.castor.example.com and s3.castor.example.com. Each host resolves to a different IP address so that a load balancer can direct the traffic to the appropriate Gateway server pool for that the chosen storage protocol.

In The load balancer needs to add the header in order to direct the storage requests to the shared storage domain, the load balancer needs to add the header

Code Block
languagexml
X-Forwarded-Host: castor.example.com


DNS

Cause the DNS name resolution to be different for the clients using one storage protocol than for the clients using another protocol.

For example, the clients Clients using S3 may resolve the storage domain castor.example.com to 10.100.100.81 while clients using SCSP resolve the same storage domain to 10.100.100.82. While this method does not require in-line modification of the HTTP requests, it does require that the administrator have This method requires the administrator has control of the hosts where the client applications run so as to allow alteration of the DNS/host resolution but does not require in-line modification of the HTTP requests.

OPTIONS

Route request method OPTIONS with "Origin" to the S3 port rather than default to the SCSP port (which happens when an Authorization header does not exist or have "AWS").

S3 must handle "bucket in Host" style requests because SCSP reports that the domain was not found.


Code Block
languagebash
curl -i 
	-H 'Origin: http://www.example.com' 
	-H 'Access-Control-Request-Method: PUT' 
	-X OPTIONS https://mycorsbucket.elsewhere.com/


Pattern-matching

Use Layer 7 inspection of the requests to switch incoming traffic based on the object storage protocol. See below:

This is done by looking for the distinctive S3 Authorization header pattern or one of the query string arguments for authenticated S3 requests. The pattern-matching rules for these authenticated S3 requests are as follows.

  • Headers

Use pattern-matching on the request header.


Code Block
Authorization ~= ^AWS.*


  • Arguments

Use pattern-matching on the query string arguments.


Code Block
{RequestURL} ~= [?&](AWSAccessKeyId|X-Amz-Credential)=


  • Absence

Use pattern-matching to test for the absence of all AWS request patterns.

The request is either an anonymous S3 request or an SCSP request. Since anonymous S3 requests should do not create, update, or delete content, they are most likely GET requests, and it is safe to allow SCSP to handle these.

...