Versions Compared

Old Version 5

changes.mady.by.user Kyle Miller

Saved on

compared with

New Version Current

changes.mady.by.user Bala Harish A(AC)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Table of Contents
minLevel1
maxLevel2
outlinefalse
typelist
printablefalse

The S3 protocol personality is a front-end storage protocol for client applications. It runs within the Gateway itself. All of the Gateway deployment and scaling features apply when you use using the S3 front-end protocol.

...

An administrator can configure the Gateway to run several ways:

  • only the SCSP protocol

  • only the S3 protocol

  • both protocols at the same time

Additionally, Gateways can be scaled horizontally with any combination of front-end protocols as needed for a particular deployment. This allows for a heterogeneous mix of client types that utilize utilizing the same Swarm storage cluster and that share sharing content with each other. Doing Performing this allows an administrator to provide a unified object storage platform.

Sharing Storage across S3 and SCSP

Content Gateway provides the mechanism to unify the back-end Swarm object storage so that S3 applications and SCSP applications can share content. In a unified object storage deployment, a device like a load balancer is used to route incoming client traffic through the appropriate port number or pool of Gateway servers.

...

Routing Methods

These are some routing methods that can be used usable for a unified object storage front-end:

IP

address

Address

Listen to multiple virtual IP addresses and distribute traffic based upon the IP address used by the client.


X-Forwarded-Host

Use Layer 7 inspection of the requests to distribute traffic based upon headers contained in the client requests.

The X-Forwarded-Host header can be used by a proxy or load balancer to

let

allow client applications to use a different host name for each supported HTTP storage protocol while sharing the same storage domain for

their

the content.

For example, consider

Consider the storage domain castor.example.com with two protocol-specific host names scsp.castor.example.com and s3.castor.example.com. Each host resolves to a different IP address so

that

a load balancer can direct the traffic to the appropriate Gateway server pool for

that

the chosen storage protocol.

In order

The load balancer needs to add the header to direct

the

storage requests to the shared storage domain

, the load balancer needs to add the header

: 

Code Block
languagexml
X-Forwarded-Host: castor.example.com

DNS

Cause the DNS name resolution to be different for the clients using one storage protocol than for the clients using another protocol.

For example, the clients

Clients using S3 may resolve the storage domain castor.example.com to 10.100.100.81 while clients using SCSP resolve the same storage domain to 10.100.100.82.

While this method does not require in-line modification of the HTTP requests, it does require that the administrator have

This method requires the administrator has control of the hosts where the client applications run so as to allow alteration of the DNS/host resolution but does not require in-line modification of the HTTP requests.

OPTIONS

Route request method OPTIONS with "Origin" to the S3 port rather than default to the SCSP port (which happens when an Authorization header does not exist or have "AWS").

S3 must handle "bucket in Host" style requests because SCSP reports

that

the domain was not found.

Code Block
languagebash
curl -i 
	-H 'Origin: http://www.example.com' 
	-H 'Access-Control-Request-Method: PUT' 
	-X OPTIONS https://mycorsbucket.elsewhere.com/

Pattern-

matching

Matching

Use Layer 7 inspection of the requests to switch incoming traffic based on the object storage protocol. See below:

This is

done

performed by looking for the distinctive S3 Authorization header pattern or one of the query string arguments for authenticated S3 requests. The pattern-matching rules for these authenticated S3 requests are as follows.

  • Headers

Use pattern-matching on the request header.

Code Block
Authorization ~= ^AWS.*

  • Arguments

Use pattern-matching on the query string arguments.

Code Block
{RequestURL} ~= [?&](AWSAccessKeyId|X-Amz-Credential)=

  • Absence

Use pattern-matching to test for the absence of all AWS request patterns.

The request is either an anonymous S3 request or an SCSP request. Since anonymous S3 requests

should

do not create, update, or delete content, they are most likely GET requests, and it is safe to allow SCSP to handle these. Using SCSP for video (e.g., mp4) objects also avoids an issue with If-Range handling on the Gateway S3 port.