version 4.0 | document revision 1
...
| Anchor | ||||
|---|---|---|---|---|
|
1.1 provides an overview of a FileFly deployment. All communication between FileFly components is secured with Transport Layer Security (TLS). The individual components are described below.
| Anchor | ||||
|---|---|---|---|---|
|
FileFly System Overview
...
DataCore FileFly Admin Portal is the web-based interface that provides central management of a FileFly deployment. It is installed as part of the FileFly Tools package.
When entering the FileFly Admin Portal, the 'Dashboard' will be displayed – we will come back to the dashboard in 1.5. For now, the remainder of this section will follow the Admin Portal's navigation menu.
...
- view additional server information
- configure storage plugins
- add / retire / restart cluster nodes
- upgrade a standalone server to high-availability
- view detailed charts of recent activity
- edit server-specific configuration (see E)
| Anchor | ||||
|---|---|---|---|---|
|
Sources describe volumes or folders to which Policies may be applied (e.g., locations on the network from which files may be Migrated).
A Source location is specified by a URI. Platform-specific information for all supported sources is detailed in 5. A filesystem browser is provided to assist in setting the URI location interactively.
...
Destinations are storage locations that Policies may write files to (e.g., locations on the network to which files are Migrated). Platform-specific information for all supported sources is detailed in 5.
Optionally, a Destination may be configured to use Write Once Read Many (WORM) semantics for migration operations. No attempt will be made thereafter to update the resultant secondary storage objects. This option is useful when the underlying storage device has WORM-like behavior, but is exposed using a generic protocol.
...
- wildcard patterns, e.g. *.doc (see A.1)
- regular expressions, e.g. /2004-06-[0-9][0-9]\.log/ (see A.2)
Note that:
- files match if any one of the patterns in the list match
- whitespace before and after each pattern is ignored
- patterns are case-insensitive
- filename patterns starting with '/' match the path from the point specified by the Source URI
- filename patterns NOT starting with '/' match files in any subtree
- literal commas within a pattern must be escaped with a backslash
...
A Policy specifies an operation to perform on a set of files. Depending on the type of operation, a Policy will specify Source(s) and/or Destination(s), and possibly Rules to limit the Policy to a subset of files.
Each operation has different parameters, refer to 4 for a full reference.
| Anchor | ||||
|---|---|---|---|---|
|
A Task schedules one or more Policies for execution. Tasks can be scheduled to run at specific times, or can be run on-demand via the Quick Run control on the 'Dashboard'.
While a Task is running, its status is displayed in the 'Running Tasks' panel of the 'Dashboard'. When Tasks finish they are moved to the 'Recent Tasks' panel.
Operation statistics are updated in real time as the task runs. Operations will automatically be executed in parallel, see E for more details.
If multiple Tasks are scheduled to start simultaneously, Policies on each Source are grouped such that only a single traversal of each file system is required.
...
The 'Recovery' page provides access to multiple versions of the recovery files produced by each Create Recovery File From Source/Destination Policy. Retention options may be adjusted in 'Settings'.
Refer to 6 for more information on performing recovery operations.
...
- email notification
- configuration backup (see 3.3)
- work hours
- Admin Portal logging
- user interface language selection
...
The 'Processed' line chart graphs both the rate of operations successfully performed and data processed over time. Data transfer and bytes Quick-Remigrated (i.e. without any transfer required) are shown separately.
The 'Operations' breakdown chart shows successful activity by operation type across the whole system over time. Additionally, per-server operations charts are available via the 'Servers' page – see 1.4.1.
The 'Operations' radar chart shows a visual representation of the relative operation profile across your deployment. Two figures are drawn, one for each of the two preceding 7-day periods. This allows behavioral change from week to week to be seen at a glance.
...
Refer to these instructions during initial deployment and when adding new components. For upgrade instructions, please refer to 3.7 instead.
For further information about each supported storage platform, refer to 5.
| Anchor | ||||
|---|---|---|---|---|
|
...
Installation of the FileFly FPolicy Server software requires careful preparation of the NetApp Filer and the FileFly FPolicy Server machines. Instructions are provided in 5.3.
| Anchor | ||||
|---|---|---|---|---|
|
...
Installation of the FileFly LinkConnect Server software requires careful configuration of both the NAS / file server and the FileFly LinkConnect Server machines. Instructions are provided in 5.4 for OneFS and 5.2 for Windows file servers. Other devices are not supported.
...
- Create a Destination to store your migrated data
- see 5 for platform-specific instructions
- Create a Migration Policy and add the Source(s), Rule and Destination
- Use the 'Simulate rule matching…' button to explore the effect of your rule
- Create a Task for the new Policy
- Run the task
...
| Anchor | ||||
|---|---|---|---|---|
|
4 describes all FileFly Policy Operations in detail and will help you to get the most out of FileFly.
The remainder of this chapter gives guidance on using FileFly in a production environment.
...
This section describes how to backup DataCore FileFly configuration (for primary and secondary storage backup considerations, see 3.4).
| Anchor | ||||
|---|---|---|---|---|
|
...
- Policy configuration
- Server registrations
- Per-Server settings, including plugin configuration, keys etc.
- Note: FileFly FPolicy Server configuration is not included – see 3.3.2.1
- Recovery files
- Settings from the Admin Portal 'Settings' page
- Settings specified when FileFly Tools was installed
...
Most enterprise Windows backup software will respect FileFly stubs and will back them up correctly without causing any unwanted demigrations. For some backup software, it may be necessary to refer to the software documentation for options regarding Offline files.
When testing backup software configuration, test that backup of stubs does not cause unwanted demigration.
Additional backup testing may be required if Stub Deletion Monitoring is required. Please refer to E for more details.
| Anchor | ||||
|---|---|---|---|---|
|
Please consult 5.3.5 for information regarding snapshot restore on NetApp Filers.
...
- Check your FileFly configuration is adequately backed up – see 3.3
- Review the storage backup and restore procedures described in 3.4
- Check backup software can backup stubs without triggering demigration
- Check backup software restores stubs and that they can be demigrated
- Schedule regular 'Create Recovery File From Source' Policies on your migration sources – see 4.10
| Anchor | ||||
|---|---|---|---|---|
|
Generally, antivirus software will not cause demigrations during normal file access. However, some antivirus software will demigrate files when performing scheduled file system scans.
Prior to production deployment, always check that installed antivirus software does not cause unwanted demigrations. Some software must be configured to skip offline files in order to avoid these inappropriate demigrations. Consult the antivirus software documentation for further details.
If the antivirus software does not provide an option to skip offline files during a scan, DataCore FileFly Agent may be configured to deny demigration rights to the antivirus software. Refer to E for more information.
It may be necessary for some antivirus products to exempt the DataCore FileFly Agent process from real-time protection (scan-on-access). If the exclusion configuration requires the path of the executable to be specified, be sure to update the exclusion whenever FileFly is upgraded (since the path will change on upgrade).
...
Check for other applications that open all the files on the whole volume. Audit scheduled processes on file servers – if such processes cause unwanted demigration, it may be possible to block them (see E).
| Anchor | ||||
|---|---|---|---|---|
|
...
- Configure email notifications to monitor system health and Task activity
- Enable syslog – see E
| Anchor | ||||
|---|---|---|---|---|
|
For further information on platform-specific interoperability considerations, please refer to the appropriates sections of 5.
| Anchor | ||||
|---|---|---|---|---|
|
...
- Run 'Gather Statistics' Policies
- Examine reports
- Examine Server statistics – see 1.4.1
- For more detail, examine demigrates in file server agent.log files
...
Requires: Source(s), Rule(s), Destination Included in Community Edition: yes
Migrate file data from selected Source(s) to a Destination. Stub files remain at the Source location as placeholders until files are demigrated. File content will be transparently demigrated (returned to primary storage) when accessed by a user or application. Stub files retain the original logical size and file metadata. Files containing no data will not be migrated.
Each Migrate operation will be logged as a Migrate, Remigrate, or Quick-Remigrate.
A Remigrate is the same as a Migrate except it explicitly recognizes that a previous version of the file had been migrated in the past and that stored data pertaining to that previous version is no longer required and so is eligible for removal via a Scrub policy.
A Quick-Remigrate occurs when a file has been demigrated and NOT modified. In this case it is not necessary to retransfer the data to secondary storage so the operation can be performed very quickly. Quick-remigration does not change the secondary storage location of the migrated data.
Optionally, quick-remigration of files demigrated within a specified number of days may be prevented. This option can be used to avoid quick-remigrations occurring in an overly aggressive fashion.
Additionally, this policy may be configured to pause during the globally configured work hours.
Note: For Sources using a FileFly LinkConnect Server, such as Dell EMC OneFS shares, see 4.9 instead.
| Anchor | ||||
|---|---|---|---|---|
|
...
Requires: Source(s) Included in Community Edition: yes
Scan all stubs present on a given Source, revalidating the relationship between the stubs and the corresponding files on secondary storage. This operation is required following a restore from backup and should be performed on the root of the restored source volume.
If only Write Once Read Many (WORM) destinations are in use, this policy is not required.
Important: This revalidation operation MUST be integrated into backup/restore procedures, see 3.4.1.
| Anchor | ||||
|---|---|---|---|---|
|
...
Requires: Source(s), Rule(s), Destination Included in Community Edition: yes
For platforms that do not support standard stub-based migration, Link-Migrate file data from selected Source(s) to a Destination.
Files at the source location will be replaced with FileFly-encoded links (MigLinks) which allow client applications to transparently read data without returning files to primary storage. If an application attempts to modify a link, the file will be automatically returned to primary storage and then modified in-place. Files containing no data will be skipped by this policy.
MigLinks present the original logical size and file metadata.
Since MigLinks remain links when read by client applications, there is no analogue of quick-remigration for link-migrate.
This policy may be configured to pause during the globally configured work hours.
Note: To perform link-migration to Swarm targets, the destination should use the s3generic scheme, see 5.8.
| Anchor | ||||
|---|---|---|---|---|
|
...
Windows NTFS volumes may be used as migration sources. On Windows Server 2016 and above, ReFS volumes are also supported as migration sources.
Windows stub files can be identified by the 'O' (Offline) attribute in Explorer. Depending on the version of Windows, files with this flag may be displayed with an overlay icon.
Note: If it is not possible to install the DataCore FileFly Agent directly on the file server, see 5.2 for an alternative solution using Link-Migration.
...
When creating a production deployment plan, please refer to 3.5.
| Anchor | ||||
|---|---|---|---|---|
|
...
See Installing FileFly Agent for Windows 2.2.2
| Anchor | ||||
|---|---|---|---|---|
|
This section describes Windows-specific considerations only and should be read in conjunction with 3.5.
| Anchor | ||||
|---|---|---|---|---|
|
FileFly supports Microsoft Storage Replica.
If Storage Replica is configured for asynchronous replication, a disaster failover effectively reverts the volume to a previous point in time. As such, this kind of failover is directly equivalent to a volume restore operation (albeit to a very recent state).
As with any restore, a Post-Restore Revalidate Policy (see 4.5) should be run across the restored volume within the scrub grace period window. This will ensure correct operation of any future scrub policies by accounting for discrepancies between the demigration state of the files on the (failed) replication source volume and the replication destination volume.
Important: integrate this process into your recovery procedures prior to production deployment of asynchronous storage replication.
...
Windows Shadow Copy – also known as Volume Snapshot Service (VSS) – allows previous versions of files to be restored, e.g. from Windows Explorer. This mechanism cannot be used to restore a stub. Restore stubs from backup instead – see 3.4.
| Anchor | ||||
|---|---|---|---|---|
|
...
On Windows, the FileFly Agent can monitor stub deletions to identify secondary storage files that are no longer referenced in order to maximize the usefulness of Scrub Policies. This feature extends not only to stubs that are directly deleted by the user, but also to other cases of stub file destruction such as overwriting a stub or renaming a different file over the top of a stub.
Stub Deletion Monitoring is disabled by default. To enable it, please refer to E.
| Anchor | ||||
|---|---|---|---|---|
|
...
This section details the configuration of a DataCore FileFly LinkConnect Server to enable Link-Migration of files from Windows Server SMB shares. This option should be used when it is not possible to install DataCore FileFly Agent directly on the Windows file server in question. For other cases – where FileFly Agent can be installed on the server – please refer to 5.1.
Refer to 4.2 and 4.9 for details of the Migrate and Link-Migrate operations respectively.
Link-Migration works by pairing a Windows SMB share with a corresponding LinkConnect Cache Share. Typically a top-level share on each Windows file server volume is mapped to a unique share (or subdivision) on a FileFly LinkConnect Server. Multiple file server shares may use Cache Shares / subdivisions on the same FileFly LinkConnect Server if desired.
Once this configuration is completed, Link-Migrate policies convert files on the source Windows Server SMB share to links pointing to the destination files via the LinkConnect Cache Share, according to configured rules.
Link-Migrated files can be identified by the 'O' (Offline) attribute in Explorer. Depending on the version of Windows, files with this flag may be displayed with an overlay icon.
...
- An NTFS Cache Volume of at least 1TB – see 2.2.4
- A FileFly license that includes an entitlement for FileFly LinkConnect Server.
- A supported secondary storage destination (excluding scsp and scspdirect)
When creating a production deployment plan, please refer to 3.5.
| Anchor | ||||
|---|---|---|---|---|
|
...
In order to access link-migrated files, the LinkConnect Client Driver must be installed on each client machine – see 2.3.
| Anchor | ||||
|---|---|---|---|---|
|
...
This list of sub-shares can be updated later as more subdirectories are shared. Where MigLink access occurs on unexpected shares, warnings will be written to the LinkConnect agent.log.
Save the configuration and restart the DataCore FileFly Agent service.
Important: Refer to 3.3.1 to ensure that the configuration on this FileFly LinkConnect Server is included in your backup. If the FileFly LinkConnect Server needs to be rebuilt, the secret key will be required to enable previously link-migrated files to be securely accessed.
...
DataCore FileFly FPolicy Servers require EXCLUSIVE use of SMB connections to their associated NetApp Vservers. This means Explorer windows must not be opened, drives must not be mapped, nor should any UNC paths to the filer be accessed from the FileFly FPolicy Server machine. Failure to observe this restriction will result in unpredictable FPolicy disconnections and interrupted service.
When creating a production deployment plan, please refer to 3.5.
| Anchor | ||||
|---|---|---|---|---|
|
...
After an entire volume containing stubs is restored from snapshot, a Post-Restore Revalidate Policy must be run, as per the restore procedure described in 3.4.
| Anchor | ||||
|---|---|---|---|---|
|
...
IMPORTANT: Until the demigration above is performed, the restored stub(s) may occupy space for the full size of the file.
As with any other FileFly restore procedure, be sure to run a Post-Restore Revalidate Policy across the volume before the next Scrub – see 3.4.
| Anchor | ||||
|---|---|---|---|---|
|
...
Except when following the procedure in 5.3.5 Robocopy must not be used with the /b (backup mode) switch when copying FileFly NetApp stubs.
When in backup mode, robocopy attempts to copy stub files as-is rather than demigrating them as they are read. This behavior is not supported.
Note: The /b switch requires Administrator privilege – it is not available to normal users.
...
OneFS does not provide an interface for performing FileFly stub-based migration. As an alternative, FileFly provides a link-based migration mechanism via a FileFly LinkConnect Server. See 4.9 for details of the Link-Migrate operation.
Link-Migration works by pairing a OneFS SMB share with a corresponding LinkConnect Cache Share. Typically a top-level share on each OneFS device is mapped to a unique share (or subdivision) on a FileFly LinkConnect Server. Multiple OneFS systems may use shares/subdivisions on the same FileFly LinkConnect Server if desired.
Once this configuration is completed, Link-Migrate policies convert files on the source OneFS share to links pointing to the destination files via the LinkConnect Cache Share, according to configured rules.
Link-Migrated files can be identified by the 'O' (Offline) attribute in Explorer. Depending on the version of Windows, files with this flag may be displayed with an overlay icon.
...
- An NTFS Cache Volume of at least 1TB – see 2.2.4
- A FileFly license that includes an entitlement for FileFly LinkConnect Server.
- A supported secondary storage destination (excluding scsp and scspdirect)
When creating a production deployment plan, please refer to 3.5.
| Anchor | ||||
|---|---|---|---|---|
|
...
In order to access link-migrated files, the LinkConnect Client Driver must be installed on each client machine – see 2.3.
| Anchor | ||||
|---|---|---|---|---|
|
...
This list of sub-shares can be updated later as more subdirectories are shared. Where MigLink access occurs on unexpected shares, warnings will be written to the LinkConnect agent.log.
Save the configuration and restart the DataCore FileFly Agent service.
Important: Refer to 3.3.1 to ensure that the configuration on this FileFly LinkConnect Server is included in your backup. If the FileFly LinkConnect Server needs to be rebuilt, the secret key will be required to enable previously link-migrated files to be securely accessed.
...
The TCP port used to access the Swarm Content Gateway via HTTP or HTTPS must be allowed by any firewalls between the DataCore FileFly Gateway and the Swarm endpoint. For further information regarding firewall configuration see 8.
| Anchor | ||||
|---|---|---|---|---|
|
...
Enable 'Include metadata headers' to store per-file HTTP metadata with the destination objects, such as original filename and location, content-type, owner and timestamps – see 5.5.6 for details.
Also enable 'Include Content-Disposition' to include original filename for use when downloading the target objects directly using a web browser.
...
The scspdirect scheme should only be used when accessing Swarm storage nodes directly. Swarm may be used as a migration destination only.
Swarm (SCSP) traffic is not encrypted in transit when using this scheme. Optionally, the plugin can employ client-side encryption to protect migrated data at rest.
Normally, Swarm will be accessed via a Swarm Content Gateway, in which case the scsp scheme must be used instead, see 5.5.
| Anchor | ||||
|---|---|---|---|---|
|
...
The Swarm storage node port must be allowed by any firewalls between the DataCore FileFly Gateway and the Swarm storage nodes. For further information regarding firewall configuration see B.
| Anchor | ||||
|---|---|---|---|---|
|
...
Enable 'Include metadata headers' to store per-file HTTP metadata with the destination objects, such as original filename and location, content-type, owner and timestamps – see 5.5.6 for details.
Also enable 'Include Content-Disposition' to include original filename for use when downloading the target objects directly using a web browser.
...
| Anchor | ||||
|---|---|---|---|---|
|
Refer to 5.5.5.
| Anchor | ||||
|---|---|---|---|---|
|
...
| Anchor | ||||
|---|---|---|---|---|
|
Please refer to 5.7.4 for S3 metadata field details.
...
The FileFly DrTool application allows for the recovery of files where normal backup and restore procedures have failed. Storage backup recommendations and considerations are covered in 3.4.
It is recommended to regularly run a 'Create Recovery File From Source' Policy to generate an up-to-date list of source–destination mappings.
FileFly DrTool is installed as part of DataCore FileFly Tools.
Note: Community Edition licenses do not include FileFly DrTool functionality.
...
The 'File Pattern' field will match either filenames only (and search within all directories), or filenames qualified with directory paths in the same manner as filename patterns in FileFly Admin Portal Rules – see A.
For the purposes of file pattern matching, the top-level directory is considered to be the top level of the entire URI path. This may be different to the top-level of the original Source URI.
...
All files may be recovered either as a batch process using the command line (see 6.7) or interactively as follows:
...
When recovering to a new location, always use an up-to-date Recovery file generated by a 'Create Recovery File From Source' Policy.
To rewrite source file URIs to the new location, use the -csu command line option to update the prefix of each URI. Once these URI substitutions have been applied (and checked in the GUI) files may be recovered as previously outlined. The -csu option is further detailed in 6.7.
Important: DO NOT create stubs in a new location and then continue to use the old location. To avoid incorrect reference counts, only one set of stubs should exist at any given time.
...
In FileFly DrTool, source files may be updated to reflect a destination URI change through use of the -cmu command line option – detailed in 6.7.
To apply the destination URI substitution to existing files on the source, select 'Update All Source Files…' from the Edit menu. When given the option, elect to update substituted entries only.
Note: This operation must always be performed using an up-to-date Recovery file generated by a 'Create Recovery File From Source' Policy.
...
Run a 'Create Recovery File From Destination' Policy, see 4.10.
| Anchor | ||||
|---|---|---|---|---|
|
This appendix details the specifics of the pattern-matching syntax for filename and owner patterns used in Rules (see 1.4.1).
| Anchor | ||||
|---|---|---|---|---|
|
...
As shown above, wildcard patterns ending with '/**' match all files in a particular tree.
Directory inclusion and exclusion can also be performed using Subdirectory Filtering (see 1.4.2.1).
| Anchor | ||||
|---|---|---|---|---|
|
...
| Anchor | ||||
|---|---|---|---|---|
|
As described in 5.1.6, on Windows file systems, FileFly can monitor stub deletion events in order to make corresponding secondary storage files eligible for removal using Scrub Policies.
This feature is not enabled by default. It must be enabled on a per-volume basis either by specifying volume GUIDs (preferred) or drive letters. Volume GUIDs may be determined by running the Windows mountvol command or Powershell Get-WmiObject -Class win32_volume. For Windows clustered volumes, the cluster volume must be specified using a volume GUID.
Note: This feature should not be configured to monitor events on backup destination volumes. In particular, some basic backup tools such as Windows Server Backup copy individual files to VHDX backup volumes in a manner which is not supported and so such volumes must not be configured for Stub Deletion Monitoring. Of course, deletions may still be monitored on source data volumes.
...
Storage plugins may be configured and enabled as necessary for each server. For plugin-specific details, refer to the appropriate section of 5.
| Anchor | ||||
|---|---|---|---|---|
|
...
Location: C:\Program Files\DataCore FileFly\logs\FileFly Agent
There are two types of FileFly Agent log file. The agent.log contains all FileFly Agent messages, including startup, shutdown, and error information, as well as details of each individual file operation (migrate, demigrate, etc.). Use this log to determine which operations have been performed on which files and to check any errors that may have occurred.
The messages.log contains a subset of the FileFly Agent messages, related to startup, shutdown, critical events and system-wide notifications.
Log messages in both logs are prefixed with a timestamp and thread tag. The thread tag (e.g. <A123>) can be used to distinguish messages from concurrent threads of activity.
Log files are regularly rotated to keep the size of individual log files manageable. Old rotations are compressed as gzip (.gz) files, and can be read using many common tools such as 7-zip, WinZip, or zless. To adjust logging parameters, including how much storage to allow for log files before removing old rotations, see E.
Log information for operations performed as the result of an Admin Portal Policy will also be available via the web interface.
...
- DataCore FileFly version
- Swarm version
- A description of the issue – be sure to include:
- how long the issue has been present
- how regularly the issue occurs
- any changes made to the environment or configuration
- any specific circumstances which trigger the issue
- does the issue occur for a particular file and/or server?
- Operating System(s)
- Saved system info (.NFO) from msinfo32.exe
- Plugins in use
- Source and Destination URIs
- Applicable Log Files
- see F.1 for log locations
- include Admin Portal logs
- include source agent logs
- include destination / gateway agent logs
- remember to include all nodes in each agent cluster
- zip the entire log folders wherever possible
- Generate a system configuration file (support.zip) from the Admin Portal 'Help' page
- Any other error messages – include screenshots if necessary
...