How to set up a FileFly S3 endpoint for Gateway

FileFly 3.1 added the ability to archive data directly to an S3-compatible endpoint.

FileFly offers two S3 compatible endpoints: Amazon AWS S3 and Generic S3 Endpoint. Use the Generic option to use Content Gateway as the endpoint.

Preparation

Install the Amazon S3 Plugin and Amazon S3 Config utilities on the FileFly Migration Agents that write to the destination storage.

1. Locate the executables in the FileFly distribution:
   
   

   Open

   

2. Install the utilities.

   • Install the two executables and configure as follows if installing on a new server not previously used for migration operations.

   • It is best to wait for a maintenance window before proceeding if installing the plugins on a running migration agent, as the plugin installation stops the running agent until the target is configured.

Creating the S3 Endpoint

1. Launch the Amazon S3 Config tool from the Start menu on the Windows server.

2. Choose Generic S3 Endpoint when prompted to select between Amazon AWS S3 and Generic S3 Endpoint:
   
   

   Open

   

3. Enter the server FQDN and the S3 port on the FileFly Generic S3 Endpoint Config:
   
   

   1. Server FQDN is the domain set up in Content UI.

   2. S3 Port: The port is 9090 as the server has no SSL offload. Verify the port running S3 by opening /etc/caringo/cloudgateway/gateway.cfg on the Gateway server and checking the [S3] section.
      
      

4. Create a token and secret key pair in the Content UI to complete the next configuration section.

   1. Browse to Content UI on the Gateway server.

   2. Click on the Settings (cog) icon on the far right and select Tokens.
      
      

   3. Click on + Add, beside the Settings cog, and create the token.
      
      

      
      Security best practices: Pick a complicated secret key and change the token and keypair regularly.

   4. Record the token information in a secure location as this is the one chance to obtain it in plain text:
      
      

5. Add the details for the new token in the FileFly S3 Configuration utility.
   
   

   
   Authorization refers to the choice of signature validations available for the AWS protocol. Both AWS2 and AWS4 signature validations are supported.
   
   

   • AWS2 is slightly less secure but faster, because it is performing fewer header checks per chunk.

   • AWS4 is more secure but slower and more CPU intensive. AWSv4 also requires HTTPS.

6. Select Manage Buckets:
   
   

   1. Select an existing bucket target or create a new one:

   2. Set a partition.

   3. Select Get Migration URI:

Configuring the Destination

The next step is to configure the new Destination (target for migration) in FileFly.

1. Browse to the FileFly Admin Portal.

2. Select Destinations and then Create Destination:
   
   

3. Paste the destination URI placed in the clipboard on the Create Destination page:

4. Verify the Name chosen makes clear what the target is and add any useful details in the Comments field.

5. Update the URI pasted with the correct value for the placeholder, GATEWAY_HERE, which is the FQDN of the migration agent.

   In this example, the complete URI (FQDN) is as follows: 

   s3generic://GATEWAY_HERE/tony.demo.sales.local/filefly-bucket-example

6. Select Save, and then the destination is ready for use with a migration policy.

Configuring the Task

Create a task chain using the destination configured.

1. Set up these components (refer to FileFly Documentation for details) to move data to Gateway.

   • Source

   • Rule

   • Policy

   • Task

2. Run the Task to test when these are configured.