2 Deployment

This chapter will cover:

• Installing Caringo FileFly Tools
• Installing Caringo FileFly Agent on file servers
• Installing Caringo FileFly Gateways as required
• Getting started with FileFly^TM policies
• Production readiness

Refer to these instructions during initial deployment and when adding new components. For upgrade instructions, please refer to Chapter 8 instead.

For further details and usage instructions for each platform, refer to Chapter 4.

2.1 DNS Best Practice

In a production deployment, Fully Qualified Domain Names (FQDNs) should always be used in preference to bare IP addresses.

Storage locations in Caringo FileFly^TM are referred to by URI. Relationships between files must be maintained over a long period of time. It is therefore advisable to take steps to ensure that the FQDNs used in these URIs are valid long-term, even as individual server roles are changed or consolidated.

Create DNS aliases for each logical storage role for each server. For example, use different DNS aliases when storing your finance department’s data as opposed to your engineering department’s data – even if they initially reside on the same server.

2.2 Installing FileFly Tools

The Caringo FileFly Tools package consists of the FileFly Admin Portal and the FileFly DrTool application. The FileFly Admin Portal provides central management of policy execution while the FileFly DrTool is used in disaster recovery situations.

FileFly Tools must be installed before any other components.

2.2.1 System Requirements

• A dedicated server with a supported operating system:
• Windows Server 2016
• Windows Server 2012 R2 (Apr 2014 update rollup)
• Windows Server 2012
• Windows Server 2008 R2 SP1
• Minimum 4GB RAM
• Minimum 2GB disk space for log files
• Active clock synchronization (e.g. via NTP)

Internet Explorer 11 or higher (possibly on a separate workstation) will be required to access the FileFly Admin Portal web interface.

2.2.2 Setup

1. Run Caringo FileFly Tools.exe
2. Follow the instructions on screen

After completing the installation process, FileFly Tools must be configured via the Admin Portal web interface. The FileFly Admin Portal will be opened automatically and can be found later via the Start Menu.

The interface will lead you through the process for installing your license.

For production licensed installations, a ‘Backup & Scrub Grace Period’ setup page will be displayed. Please read the text carefully and set the minimum grace period as appropriate and after consulting with your backup plan – see also §7.2 . This value may be revised later via the ‘Settings’ page.

2.3 Installing FileFly Agents

Once FileFly Tools installation is complete, proceed to install Caringo FileFly Agents as described below. FileFly Agents perform file operations as directed by Admin Portal Policies. Also, in the case of user/application initiated demigration, agents retrieve the file data from secondary storage autonomously.

2.3.1 FileFly Agent Server Roles

Each FileFly Agent server may fulfill one of two roles, selected at installation time.

In the ‘FileFly Agent for migration’ role, an agent assists the operating system to migrate and demigrate files. It is essential for the agent to be installed on all machines from which files will be migrated.

By contrast, in the Gateway role, the agent provides access to CloudScaler and Swarm destinations.

2.3.2 High-Availability Gateway Configuration

A high-availability gateway configuration is recommended. Such FileFly Gateways must be activated as ‘High-Availability FileFly Gateways’.

High-Availability Gateway DNS Setup

At least two FileFly Gateways are required for High-Availability.

1. Add each FileFly Gateway server to DNS
2. Create a single alias that maps to each of the IP addresses
3. Use this alias in FileFly^TM destination URIs, never individual nodes Example:

• gw-1.example.com→192.168.0.1
• gw-2.example.com→192.168.0.2
• gw.example.com→192.168.0.1, 192.168.0.2

Note: The servers that form the High-Availability Gateway cluster must NOT be members of a Windows failover cluster.

For further DNS recommendations, refer to §2.1.

2.3.3 Installing FileFly Agent for Windows Servers

System Requirements

• Supported Windows Server operating system:
• Windows Server 2016
• Windows Server 2012 R2 (Apr 2014 update rollup)
• Windows Server 2012
• Windows Server 2008 R2 SP1
• Minimum 4GB RAM
• Minimum 2GB disk space for log files
• Active clock synchronization (e.g. via NTP)

Note: When installed in the Gateway role, a dedicated server is required, unless it is to be co-located on the FileFly Tools server. When co-locating, create separate DNS aliases to refer to the Gateway and the FileFly Admin Portal web interface.

Setup

1. Run the Caringo FileFly Agent.exe
2. Select install location
3. Select migration or Gateway role as appropriate, refer to §2.3.1
4. If installing a FileFly Gateway, select the desired plugins
5. Follow the instructions to activate the agent via FileFly Admin Portal

Activation

• If no clustering is required, activate as a ‘Standalone Server’
• If installing the FileFly Gateway for High-Availability, activate as a High-Availability FileFly Gateway
• If the server is part of a Windows failover cluster, and this clustered resource is to be used as a FileFly^TM Source, activate as a Windows failover cluster node

For further information see §5.3.1 .

Important: If any type of clustering is used, ensure that FileFly Agent for Windows is installed on ALL cluster nodes.

2.3.4 Installing Caringo FileFly FPolicy Server for NetApp Filers

A Caringo FileFly FPolicy Server provides migration support for one or more NetApp

Filers through the FPolicy protocol. This component is the equivalent of Caringo File-

Fly Agent for NetApp Filers. Typically FileFly FPolicy Servers are installed in a high-availability configuration.

System Requirements

• A dedicated server with a supported operating system:
• Windows Server 2016
• Windows Server 2012 R2 (Apr 2014 update rollup)
• Windows Server 2012
• Windows Server 2008 R2 SP1
• Minimum 4GB RAM
• Minimum 2GB disk space for log files
• Active clock synchronization (e.g. via NTP)

Setup

Installation of the FileFly FPolicy Server software requires careful preparation of the NetApp Filer and the FileFly FPolicy Server machines. Instructions are provided in §4.3 .

Note: Legacy 7-Mode Filers require a different procedure at FileFly FPolicy Server installation time – see §4.4 .

2.4 Installing Config Tools

In addition to the components described above, it may also be necessary to install one or more Config Tools. Full details are provided where required for each storage platform in Chapter 4.

2.5 Getting Started

2.5.1 Analyzing Volumes

Once the software has been installed, the first step in any new Caringo deployment is to analyze the characteristics of the primary storage volumes. The following steps describe how to generate file statistics reports for each volume.

In the FileFly Admin Portal web interface (see Chapter 5 for full documentation):

1. Create Sources for each volume to analyze
2. Create a ‘Gather Statistics’ Policy and select all defined Sources
3. Create a Task for the ‘Gather Statistics’ Policy 4. On the ‘Overview’ tab, click Quick Run
   1. Click on the Task’s name to run it immediately
   2. When the Task has finished, expand the details by clicking on the Task name under ‘Recent Task History’
   3. Click Go to Task to go to the ‘Task Details’ page
   4. Access the report by clicking on View Last Stats

Pay particular attention to the ‘Last Modified % by size’ graph. This graph will help identify how much data would be affected by a migration policy based on the age of files.

Examine ‘File types by size’ to see if the data profile matches the expected usage of the volume.

2.5.2 Preparing for Migration

Using the information from the reports, create tasks to migrate files:

1. Prepare a destination for migrated files – see Chapter 4
2. Create a Destination in FileFly Admin Portal
3. Create a Rule and a Migration Policy

• A typical rule might limit migrations to files modified more than six months ago – do not use an ‘all files’ rule
• To avoid unnecessary migration of active files, be conservative with your first Migration Policy

1. Create a Task for the new Policy

• For now, disable the schedule

1. Save the task, then click on its name to open the ‘Task Details’ page
2. Click Simulate Now to run a Task simulation
3. Examine the resultant reports (view the Task and click View Last Stats)

If the results of simulation differ from expectations, it may be necessary to modify the rules and re-run the simulation.

Note: The simulation reports created above show details of the subset of files matched by the rules in the policies only.

Note: Reports are generated for simulations only – a real Task run will log each file operation, but will not generate a statistics report.

2.5.3 Running and Scheduling Migration

Use Quick Run on the ‘Overview’ tab to run the migration Task immediately.

Migration is typically performed periodically: configure a schedule on the migration Task’s details page.

2.5.4 Next Steps

Chapter 3 describes all FileFly^TM Policy Operations in detail and will help you to get the most out of FileFly^TM.

The remainder of this chapter gives guidance on using FileFly^TM in a production environment.

2.6 Production Readiness Checklist

Backup

Refer to Chapter 6 for details of how to backup FileFly^TM configuration.

Next, be sure to test that your backup and restore software respects stubs appropriately.

Specifically:

1. Review the backup and restore procedures described in Chapter 7
2. Check backup software can backup stubs without triggering demigration
3. Check backup software restores stubs and that they can be demigrated

As an additional measure to increase the range of available restore options, schedule regular ‘Create DrTool File From Source’ Policies on your migration sources.

Antivirus

Generally, antivirus software will not cause demigrations during normal file access. However, some antivirus software will demigrate files when performing scheduled file system scans.

Prior to production deployment, always check that installed antivirus software does not cause unwanted demigrations. Some software must be configured to skip offline files in order to avoid these inappropriate demigrations. Consult the antivirus software documentation for further details.

If the antivirus software does not provide an option to skip offline files during a scan, Caringo FileFly Agent may be configured to deny demigration rights to the antivirus software. Refer to §D.5 for more information.

Other System-wide Applications

Check for other applications that open all the files on the whole volume. Audit scheduled processes on the file server – if such processes cause unwanted demigration, it may be possible to block them (see §D.5 ).

Monitoring and Notification

To facilitate proactive monitoring, it is recommended to configure one or both of the following mechanisms:

1. Configure email notifications to monitor system health and Task activity – see §5.10
2. Enable syslog on agents – see §D.3

2.7 Policy Tuning

Periodically re-assess file distribution and access behavior:

1. Run ‘Gather Statistics’ Policies

• Examine reports

1. Examine Server statistics – see §5.3

• For more detail, examine demigrates in file server agent.log files

Consider:

• Are there unexpected peaks in demigration activity?
• Are there any file types that should not be migrated?
• Should different rules be applied to different file types?
• Is the Migration Policy migrating data that is regularly accessed?
• Are the Rules aggressive enough or too aggressive?
• What is the data growth rate on primary and secondary storage?
• Are there subtrees on the source file system that should be addressed by separate policies or excluded from the source entirely?