E Troubleshooting

Before contacting Caringo Support, please review log files for error messages.

E.1 Log Files

FileFly Agent Logs

Location: C:\Program Files\Caringo FileFly\logs\FileFly Agent

There are two types of FileFly Agent log file. The agent.log contains all FileFly Agent messages, including startup, shutdown, and error information, as well as details of each individual file operation (migrate, demigrate, etc.). Use this log to determine which operations have been performed on which files and to check any errors that may have occurred.

The messages.log contains a subset of the FileFly Agent messages, related to startup, shutdown, critical events and system-wide notifications. This log is often most useful to troubleshoot configuration issues.

Log messages in both logs are prefixed with a timestamp and thread tag. The thread tag (e.g. <A123>) can be used to distinguish messages from concurrent threads of activity.

Log files are regularly rotated to keep the size of individual log files manageable. Old rotations are compressed as gzip (.gz) files, and can be read using many common tools such as 7-zip, WinZip, or zless. To adjust logging parameters, including how much storage to allow for log files before removing old rotations, see §D.1 .

Log information for operations performed as the result of an Admin Portal Policy will also be available via the web interface.

Admin Portal Logs

Location: C:\Program Files\Caringo FileFly\logs\AdminPortal

Normally Admin Portal logs are accessed through the web interface. If the logs available in the interface have been rotated, consult this directory to find the older logs.

DrTool Logs

Location: C:\Program Files\Caringo FileFly\logs\DrTool

FileFly DrTool operations such as recreating stubs are logged in this location. FileFly DrTool will provide the exact name of the log file in the interface.

E.2 Interpreting Errors

Logged errors are typically recorded in an ‘error tree’ format which enables userdiagnosis of errors / issues in the environment or configuration, as well as providing sufficient detail for further investigation by support engineers if necessary.

Error trees are structured to show WHAT failed, and WHY, at various levels of detail.

This section provides a rough guide to extracting the salient features from an error tree.

Each numbered line consists of the following fields:

• WHAT failed – e.g. a migration operation failed
• WHY the failure occurred – the ‘[ERR ADD...]’ code
• optionally, extra DETAILS about the failure – e.g. the path to a file

As can be seen in the example below, most lines only have a WHAT component, as the reason is further explained by the following line.

A Simple Error

ERROR demigrate win://server.test/G/source/data.dat
[0]ERR_DMAGENT_DEMIGRATE_FAILED [] []
[1]ERR_DMMIGRATESUPPORTWIN_DEMIGRATE_FAILED [] []
[2]ERR_DMAGENT_DEMIGRATEIMP_FAILED [] []
[3]ERR_DMAGENT_COPYDATA_FAILED [] []
[4]ERR_DMSTREAMWIN_WRITE_FAILED [ERR_ADD_DISK_FULL] [112: There is not enough space on the disk (or a quota has been reached).] 

To expand the error above into English:

• demigration failed for the file: win://server.test/G/source/data.dat
• because copying the data failed
• because one of the writes failed with a disk full error
• the full text of the Windows error (112) is provided

So, G: drive on server.test is full (or a quota has been reached).

Errors with Multiple Branches

Some errors result in further action being taken which may itself fail. Errors with multiple branches are used to convey this to the administrator. Consider an error with the following structure:

[0] ERR...
[1] ERR...
[2] ERR...
[3] ERR...
[4] ERR...
[5] ERR...
[6] ERR...
  [3] ERR...
  [4] ERR...
  [5] ERR...

Whatever ultimately went wrong in line 6 caused the operation in question to fail. However, the function at line 2 chose to take further action following the error – possibly to recover from the original error or simply to clean up after it. This action also failed, the details of which are given by the additional errors in lines 3, 4, and 5 at the end.

Check the Last Line First

For many errors, the most salient details are to be found in the last line of the error tree (or the last line of the first branch of the error tree). Consider the following last line:

[11] ERR_DMSOCKETUTIL_GETROUNDROBINCONNECTEDSOCKET_FAILED [ERR_ADD_COULD_NOT_RESOLVE_HOSTNAME] [host was [svr1279.example.com]]

It is fairly clear that this error represents a failure to resolve the server hostname svr1279.example.com. As with any other software, the administrator’s next steps will include checking the spelling of the DNS name, the server’s DNS configuration and whether the hostname is indeed present in DNS.

E.3 Contacting Support

If an issue cannot be resolved after reviewing the logs, contact Caringo Support:

https://support.caringo.com/

Include the following information in any support request (where possible):

1. A description of the issue – be sure to include:
   1. how long the issue has been present
   2. how regularly the issue occurs
   3. any changes made to the environment or configuration
   4. any specific circumstances which trigger the issue
   5. does the issue occur for a particular file and/or server?
2. Caringo FileFly version
3. Swarm and CloudScaler versions
4. Operating System(s)
5. Plugins in use
6. Source and Destination URIs
7. Applicable Log Files
   1. see §E.1 for log locations
   2. include Admin Portal logs
   3. include source agent logs
   4. include destination/gateway agent logs
   5. remember to include all nodes in each agent cluster
   6. zip the entire log folders wherever possible
8. Generate a system configuration file (support.zip) by clicking the link on the Admin Portal ‘About’ page
9. Any other error messages – include screenshots if necessary

Important: Failure to include all relevant details will delay resolution of your issue.