Walkthrough: How to install FileFly 3.x

This is a detailed walkthrough on how to install FileFly with all roles on one server.

Order of Installation

Install FileFly Tools
1. Apply license
Install FileFly Agent(s)
1. Plugin Tools
2. Plugins (if not done already)
Create domain and bucket in Swarm
Configure target destination

Downloading the Package

Browse to connect.caringo.com, and register for an account (if you have not done so already).
Locate FileFly on the left pane under Software Downloads:
On the right side of the FileFly product page, locate the Downloads option, with the Documentation links below. Agree to the EULA and click Download.
Save the package, which is a dated ZIP file, around 450 MB in size.
Transfer the ZIP to the server where you are installing on.
Unzip the package, which has numerous executables for the required and optional components as well as tools to configure them and PDF documentation for both editions of FileFly.

Installing FileFly Tools

The FileFly Tools server is primarily used for admin tasks, scheduling and license management. It provides a web interface for configuring sources and destinations on FileFly.

Important

Active Directory authentication needs to be manually set up later if it is not set up during installation. See How to integrate FileFly portal with Active Directory.

Locate the Caringo FileFly Tools installer among the executables in the unzipped bundle.
Launch the installation wizard EXE file for the correct FileFly version and follow the prompts.
Verify the server meets the minimum requirements and runs one of the supported operating systems required by FileFly Tools:
- Windows Server 2016
- Windows Server 2012 R2 (Apr 2014 update rollup)
- Windows Server 2012
- Windows Server 2008 R2 SP1
Accept the License Agreement:
For Installation Type, specify the Standard Install (default).
Specify the installation path:
Configure access for your FileFly Admin Portal users:
Choose the authentication method for your Admin Portal:
- Local admin account — Create a local administration account (adequate for many implementations)
  
  A local user is not tied to Active Directory, and the credentials apply to the Admin Portal only.
- Active Directory authentication — To integrate with your Active Directory, you need to complete steps to add the needed group and users:
With authentication configured, complete the installation.
After installation, open the FileFly Admin Portal.
Log in to the portal.
When prompted to install the license, click Select file to upload your license file, which you should have received via email from Caringo and unzipped locally.
FileFly 3.0 activates its license with a global licensing server, so internet access is required for the initial activation:
Below the warning, you are prompted to set your Backup & Scrub Grace Period. The grace period is the default length of time to keep files in a destination, which matches the amount of time the stubs are backed up. The intention is that, if a stub is restored from a backup, the end destination file remains there. You can change this setting after the environment is installed and configured, so set a reasonable number of days (1 or higher) and reevaluate later.
Click Next, and view the summary screen. Nothing useful appears here, but the FileFly Tools are now installed.

Installing FileFly Agents

The FileFly Agents are required for moving data to a destination. See Where to install Migration Agents versus Gateway Agents.

Locate the Caringo FileFly Agent installer among the executables in the unzipped bundle.
Launch the installation wizard EXE file for the correct FileFly version and follow the prompts.
If installing an agent onto the FileFly Tools server, you are warned about additional memory requirements:
Verify the server meets the minimum requirements and runs one of the supported operating systems required by FileFly Tools:
Accept the license agreement.
Set the installation location:
FileFly Agents come in two modes. Choose what type of agent to install:
- Agent for migration — Used to migrate files to other destinations in our case Swarm, Content Gateway, S3, Azure, or Drive. Most standalone Windows deployments install the agent for migration only.
- Gateway Agent — Used to provide access to a Swarm/Gateway destination for servers without direct access. It does not migrate files.
After the installation, you are prompted to activate the agent. The activation code for this agent (which is automatically copied to your clipboard) must be registered in your Admin Portal.
Open the FileFly Admin Portal and navigate to the Servers tab. If this is a re-install, first retire the server that appears there, and then add it as new:
On the Add Server page, select Standalone Server:
Enter the fully qualified domain name of the server being configured, add comments, and click Next.
Verify the Hostname and IP address, and click Next.
Click Next, and the Admin Portal contacts the server and gather the details required to register it with the FileFly Tools.
Paste the key on the clipboard into the Activation Code field, and click Activate:
With the server registered, close the Agent installer.

Installing Plugins

Plugins tell tell the server how and where to move data to.

The Swarm plugin is used when a migration agent has direct access to the storage cluster via HTTP.
The CloudScaler plugin is used to provide access to the storage cluster via Content Gateway.

Now the migration agent is installed, but it has no plugins. What the plugin installer does is to drop the appropriate DLL and config files into the FileFly Agent directory.

Note

If this step is delayed, the plugin shows errors until it is configured with a configuration file:

To check this is the issue, navigate to the default log directory for the FileFly Agent:

C:\Program Files (x86)\Caringo FileFly\logs\FileFly Agent\agent.log

Open the log in a text editor. The bottom of the log file has an entry such as follows:

2019-03-14 04:44:42.326 (GMT-07:00) checking for clients...
2019-03-14 04:44:45.013 (GMT-07:00) <H05> ATTENTION: Configuration file is missing or contains no settings:
  C:\Program Files (x86)\Caringo FileFly\data\FileFly Agent\cloudscaler.cfg
Please configure.

This is expected; the next step is to install the configuration tools.

Installation for the Swarm plugin and the CloudScaler plugin is the same:

Locate the appropriate Caringo FileFly Plugin installer among the executables in the unzipped bundle.
Launch the installation wizard EXE file for the correct FileFly version and follow the prompts.
Verify the correct plugin setup is running:
Accept the License Agreement.
Click Finish to close the installation wizard.

Installing the Configuration Tools

The configuration tools are needed to make the connection between FileFly agents and the Destination storage.

The configuration installation for Swarm and for CloudScaler are the same:

Locate the appropriate Caringo FileFly Config installer among the executables in the unzipped bundle.
Launch the installation wizard EXE file for the correct FileFly version and follow the prompts.
Before proceeding, verify the minimum requirements are met:
Accept the license agreement.
Set the Destination folder for the installation.
After the installation completes, select Close.

Creating a Destination in Swarm

To create a Swarm or Content Gateway (CloudScaler) destination for FileFly, complete three steps:

Create a domain
Create a bucket
Resolve the domain to one or more IP addresses, either Swarm nodes (for a Swarm target) or Gateways (for a Gateway target).

The easiest way to create Swarm domains and buckets is using Swarm's Content UI; se the command line, with PowerShell and cURL. The command to issue is a custom web request with special header information causing Swarm to create an empty context (domain or bucket) object.

Best practice

Domain and bucket names are case-sensitive. To avoid problems, define the domain and bucket names with lowercase letters and use alphanumeric characters. See Naming Rules in Swarm documentation.

Examples with Content UI

Create a domain and bucket for FileFly using the Swarm Content UI, if it is installed and enabled.

Navigate to the Content UI in a browser:
http://<ipaddressofgateway>:scspport/_admin/portal http://gatewayadmin.example.com:8090/_admin/portal
Log in as an admin user:
Select an existing tenant, or click + Add to create one.
Click + Add to create a new domain for FileFly.
Name the FileFly domain and click Add.

Edit the Settings (gear icon) to customize any content policies after saved.
Open the new domain, and click + Add to create a bucket:
Select Bucket, then name and Add the bucket, verifying S3 Compatible is enabled:

Edit the Settings (gear icon) to customize any content policies after saved.
Note the domain and bucket names for use in FileFly configuration.

Examples with PowerShell

These examples show domain and bucket creation in PowerShell:

Create a domain

PS C:\Users\Administrator> Invoke-WebRequest 
 -Uri http://172.30.2.246/?domain=newdomain.example.com 
 -ContentType "application/castorcontext" -Method Post

Create a bucket

Examples with cURL

These examples show domain and bucket creation in cURL:

Create a domain

Create a bucket

Configuring a Swarm Destination in FileFly

To configure a swarm destination for FileFly, use the Caringo FileFly Swarm Config tool.

From the Windows Start menu, open the Caringo FileFly Swarm Config tool:

FileFly Encryption	Enable encryption	If it is enabled, any files that are migrated to a destination are encrypted before sending to the destination and decrypted before being served back to the client.
	Key ID
Metadata Options	Include metadata HTTP headers	Recommended. Optional headers to attach to FileFly writes. Include metadata relating to the original location of the file like owner, original name, and directory found in.
	Include Content-Disposition	Recommended. Optional headers to attach to FileFly writes. Include the mime-type (if possible) for the file.

Enable the needed options that then select Create Index.
1. Swarm domain — Enter the domain created when prompted for a Swarm domain:
  
  Swarm attempts to resolve the domain as entered. If the domain does not resolve to one or more Swarm nodes, you are stopped at this point to correct it:
2. Gateway domain — A prompted displays for the FQDN of the migration agent or gateway used to connect to the Swarm target. For deploying everything on one system, enter the FQDN of the current server:
3. Port — When prompted for the port number, accept the default. The port should not change, because Swarm's port for inbound HTTP traffic is port 80. Check the cluster.cfg to verify the port has not been changed if unsure.
4. Bucket — When prompted to enter the bucket name to use with the destination, match the case of the bucket created:
5. Partition — Although unnamed objects are supported in a FileFly destination, use named objects, if possible. For the partition, leave the field blank, because it is not used:
6. Confirm — Review the summary, and select Yes:
7. A Success message displays, which shows the URI that is now on the clipboard for pasting into the Admin Portal:
8. When clicking OK, there is a post to the cluster on the URL specified, which creates the initial index for FileFly.
Before closing the config tool, save the configuration. On the main window of FileFly Swarm Config, select Save (bottom of dialog).
A File Explorer opens, defaulting to this path: C:\Program Files\Caringo FileFly\data\Filefly Agent\
If the FileFly agent is installed on a partition other than C:\, the path reflects that. Save the file.
Once the config file is saved in the Caringo FileFly\data\Filefly Agent\ directory, a prompt to restart the Caringo FileFly Agent service displays. Select OK.

Note: if multiple agents exist, this config file needs to be copied to each agent machine.
To get to Windows Services, click the Start button and start typing "services":
Click Restart to restart the service. Once the agent service restarts, the agent is available for use in the Admin Portal.
At this point, you are ready to add this URI in the FileFly Admin Portal as a new Destination.
Open the FileFly Admin Portal, which defaults to: http://<fileflyadminservername>:8080/ffap/AdminPortal
Select the Destinations tab, then select Create Destination:
These are the options for Create Destination, which are the same for creating a Swarm or a Gateway destination:
- Name — Best practice is to choose a name that is relevant to the destination or is identifiable.
- Tag — Tags are optional, but they can help you quickly ID destinations and are handy when filtering on the various areas of the UI.
- Comments — Add any notes that help to maintain or explain the destination. For example, you may document the protection level on the destination or the locality.
- URI — Paste in the URI that the configuration tool copied to the clipboard. The format is as follows:
  swarm://ffservertony.sales.local/newdomain.example.com/fileflybucket/
  - The type of connection: swarm://
  - FQDN of the machine that is accessing the resource: ffservertony.sales.local (in this example, my FileFly Admin server)
  - FQDN of the Swarm (SCSP) domain created : fileflyexample.sales.local
  - Bucket, if being used : /fileflybucket/
- WORM — Enable the Write Once Read Many checkbox if this destination never deletes any data from the target.
Click Save, and the destination are ready for use.

Configuring a CloudScaler Destination in FileFly

From the Windows Start menu, open the Caringo FileFly CloudScaler Config tool:

Connection Type	Use TLS	Whether to use HTTPS with certificate validation. The certificate on the destination must be trusted by the FileFly migration agent or gateway agent. The destination needs to be configured to provide an SSL endpoint.
	Use a HTTP Proxy	It is only required to fill this out if the destination endpoint is only available over WAN rather than a LAN connection and connection for it goes via a proxy.
CloudScaler Domain Credentials		Configure a new domain and auth credentials for the cluster being connecting to.
FileFly Encryption	Enable encryption	If it is enabled, any files that are migrated to a destination are encrypted before sending to the destination and decrypted before being served back to the client.
	Key ID
Metadata Options	Include metadata HTTP headers	Optional headers to attach to FileFly writes. Include metadata relating to the original location of the file like owner, original name, and directory found in.
	Include Content-Disposition	Optional headers to attach to FileFly writes. Include the mime-type (if possible) for the file.

To create the needed credentials, click the Add button on the middle panel.
1. Fill out the Edit Auth Entry dialog that opens. Here is an example of the entry for a domain of tony.demo.sales.local and a bucket named filefly:
2. The Username and Password need to be Swarm SCSP credentials.
After adding the credentials, return to the previous panel.
Select the domain and select Create Index...
1. Gateway domain — A prompt appears for the fully qualified domain name of the FileFly Gateway used to access this CloudScaler destination. Enter the hostname for the machine or the alias for the machine relating to the service if installing all items on a single server:
2. Port — A prompt displays for the port number used by Swarm (the SCSP interface) to access the CloudScaler domain. This is usually port 80, but it can be different depending on how the Content Gateways are configured (gateway.cfg). Often this is port 8090 or 443 if using an SSL setup. The port refers to SCSP interface not the S3 one.
3. Partition — A prompted displays to specify a partition for the index. Leave it blank to take the default (recommended), or fill in a value, which prefix each write within the FileFly bucket. This is like a pseudo folder in S3, and there is no benefit to writing to a "partition" as it all resides in the same namespace. Usually having one bucket per destination target.
4. Confirm — Review the summary, and select Yes:
The configuration tool runs these two tests:
- HTTP HEAD the domain specified via DNS with the credentials used.
- HTTP PUT and create an index for FileFly to use with this destination.
If both succeed, a Success message displays, which shows the URI that is now on the clipboard for pasting into the Admin Portal:
Before closing the config tool, save the configuration. On the main window of FileFly Swarm Config, select Save (bottom of dialog).
A File Explorer opens, defaulting to this path: C:\Program Files\Caringo FileFly\data\Filefly Agent\
If the FileFly agent is installed on a partition other than C:\, then path reflects that. Save the file.
Once the config file is saved in the Caringo FileFly\data\Filefly Agent\ directory, a prompt displays to restart the Caringo FileFly Agent service. Select OK.

Note: if multiple agents exist, this config file needs to be copied to each of the agent machines.
To get to Windows Services, click the Start button and start typing "services":
Click Restart to restart the service.
Once the agent service restarts, the agent is available for use in the Admin Portal.
At this point, you are ready to add this URI in the FileFly Admin Portal as a new Destination. Creating destinations for each agent type is largely the same for Swarm and Gateway (CloudScaler).
Open the FileFly Admin Portal, which defaults to: http://<fileflyadminservername>:8080/ffap/AdminPortal
1. Select the Destinations tab, then select Create Destination:
2. These are the options for Create Destination, which are the same for creating a Swarm or a Gateway destination:
  - Name — Best practice is to choose a name that is relevant to the destination or is identifiable.
  - Tag — Tags are optional, but they can help you quickly ID destinations and are handy when filtering on the various areas of the UI.
  - Comments — Add any notes that help to maintain or explain the destination. For example, you may document the protection level on the destination or the locality.
  - URI — Paste in the URI that the configuration tool copied to the clipboard. The format is as follows:
    cloudscaler://ffservertony.sales.local/fileflyexample.sales.local/fileflybucket/
    - The type of connection: cloudscaler://
    - FQDN of the machine that is accessing the resource: ffservertony.sales.local (in this example, my FileFly Admin server)
    - FQDN of the Swarm (SCSP) domain you created : fileflyexample.sales.local
    - Bucket, if being used : /fileflybucket/
  - WORM — Enable the Write Once Read Many checkbox if this destination never deletes any data from the target.
Click Save, and the destination is ready for use.