How to set up a FileFly S3 endpoint for Amazon

Data can be archived directly to Amazon's S3 cloud service as of version 3.0 of FileFly. This article walks through those setup steps.

Creating an AWS S3 Bucket for FileFly

Best practice is to create an S3 bucket devoted exclusively to FileFly for use with AWS storage.

1. Browse to the AWS console: https://console.aws.amazon.com

2. Log in with valid credentials.
   
   

   Open

   

3. Select the S3 service from Services.
   
   

   Open

   

4. From there, create a new S3 bucket.
   
   

5. Name and region — Name the bucket a DNS-compliant name unique across all AWS regions.
   Use lower-case letters, numbers, and hyphens, with no special characters.
   
   

   
   
   

6. Configure options — Accept the defaults here:
   
   

7. Set permissions — Accept the defaults here, too:
   
   

8. Review and create the bucket, which appears in the S3 Buckets listing:
   
   

Creating an AWS user for FileFly

To avoid using the root credentials for FileFly access (which is bad practice), create an IAM user with access to S3.

1. In the AWS console, select the Services menu.

2. From Security, Identity, & Compliance (bottom right), select IAM.
   
   

3. From the left menu, select Users: 
   
   

4. Select Add user: 
   
   

5. A wizard walks through creating a user for IAM. On the first page, name the user to indicate it is dedicated to FileFly access, and select Programmatic access:
   
   

6. Next, assign the permissions for the FileFly user. Start by creating a group to add the user to; select Create group:
   
   

7. Name the group and filter for applicable prebuilt policies by searching on "s3":
   
   

8. Select AmazonS3FullAccess, which allows the FileFly user to create buckets and read/write/delete content.

9. Select the policy and attach it to the group, then click Create group.

10. Add tags to apply to the user, to help identify the purpose of the account:
    
    

11. Next, Review the summary and complete. 
    
    

12. A prompt appears to download a CSV file with the access key ID/secret access key pair. 
    
    

    
    This is the sole opportunity, so download this file to a secure location. These are used in the next setup step.

Accessing the S3 Account

With the S3 bucket and dedicated IAM user created, configure FileFly to access the AWS S3 account.

1. Navigate to the FileFly Migration agent used to write data to AWS.

2. Install "Caringo FileFly Amazon S3 Config" and "Caringo FileFly Amazon S3 Plugin".
   
   

   
   Note: Migration jobs are paused until a plugin is configured when a new plugin for FileFly is installed.

3. After the FileFly S3 Config tool is installed on the migration agent, open it via the Start menu.
   Search for "s3" if not listed:
   
   

4. Select Amazon AWS S3 when prompted for the type, which is the option for direct connectivity to Amazon:
   
   

   
   The second, Generic S3 Endpoint, is for connecting to any other S3-compatible target (such as Content Gateway access to Swarm object storage, or another external provider).

5. Enter the Amazon account credentials, the Access Key ID and Secret Access Key from the CSV file saved above on the main configuration screen:
   
   

6. Next, select Manage Buckets...
   
   

7. Select the S3 bucket from the list:
   
   

   
   Note: Writes to a bucket created from selecting New... are not immediately possible: it can take 15-25 minutes for Amazon to propagate DNS across regions.

8. Highlight the bucket to be used and select the Get Migration URI button:
   
   

9. When prompted to Select a partition, accept the Default partition.
   Creating a partition here creates a prefix or pseudo folder, for separating content in a mixed-use bucket. Best practice is to dedicate this bucket to FileFly, which prevents accidental deletion of FileFly files by other sources.

10. Click OK, and copy the URI from the clipboard:
    
    

11. Save the config file so the target works if the server reboots.
    Click OK, Close, then Save at to the bottom of the screen.

12. "C:\Program Files\” is the default location to save the configuration.
    
    

13. Per the instructions on the dialog:

    1. Copy the config file to any additional FileFly Gateway machines, in this location:

       c:\Program files\Caringo Filefly\data\FileFly Agent\s3.cfg

    2. Restart the FileFly Agent service to apply the changes.

Configuring the Destination

The next step is to configure the new Destination (target for migration) in FileFly.

1. Browse to the FileFly Admin Portal.

2. Select Destinations and then Create Destination:
   
   

3. Paste the destination URI from the clipboard on the Create Destination page:
   
   

4. Verify the Name makes clear what the target is and add any useful details in the Comments field.

5. Update the URI pasted with the correct value for the placeholder, YOUR_GATEWAY_HERE, which is the FQDN of the migration agent.

   In this example, the FQDN is ffservertonyl.saleslab01.dom, so the complete URI is as follows: 

   s3://ffservertonyl.saleslab01.dom/example-bucket-for-filefly

   
   
   

6. Select Save, and then the destination is ready for use with a migration policy.

Configuring the Task

Create a task chain to make use of the new destination.

1. Set up these components (refer to FileFly Documentation for details) to move data to AWS.

   • Source

   • Rule

   • Policy

   • Task

2. When these are configured, run the Task to test it: