The best practices for using DataCore Swarm object storage as the capacity tier in a Veeam Backup and Replication Scale-out Backup Repository include sizing and configuration guidelines as well as operational tips for optimizing the backup and recovery processes. These practices may be updated periodically as new lessons are learned.

We recommend reading Veeam’s best practice documentation before reading this KB article, Repository Design

Use Multiple SOBR

Currently, Veeam supports only one object storage bucket per SOBR configuration. We recommend using multiple SOBRs pointing to different buckets and storage domains per client and workload type (VB365 Backup, Archive backup jobs etc.).

This will allow you to take advantage of future support for separate elasticsearch indices per storage domain.

Each SOBR requires its performance tier (primary storage), whereby each SOBR could be a subdirectory of the same primary storage.

Another advantage of multiple SOBR is that you can set different retention policies per SOBR.

Backup Job Storage Settings

Veeam allows you to configure a block size per Backup Job. The block size affects deduplication and incremental backup size. By default, blocks are compressed and the compression ratio is about 50%.

The smaller the block size, the more calls are needed to the object storage to upload the data. DataCore recommends to use “Local Target (large blocks)”. This is equal to a 4MB (=2MB after compression) object size. Performance Tier storage requirements depend on this setting, too.

Consult with Veeam or your partner before setting this.

See Veeam V12 Guidelines for more details on how to set the storage block size.

SOBR Offloading

For the SOBR offloading the aggregated number of concurrent tasks of all performance tier extents is used. If an extent has no limit on concurrent tasks, SOBR offload uses an unlimited number of tasks (up to the # of backup chains).

Each SOBR offload task will use up to 64 connections to the capacity tier. This is defined by S3ConcurrentTaskLimit in the Windows registry settings. (HKLM\SOFTWARE\Veeam\Veeam Backup and Replication\S3ConcurrentTaskLimit , requires a reboot to apply).

We recommend 2,000 connections (approximately) for On-Premise object storage.

The default value of 64 (decimal) is applied to every worker process, meaning that the “SOBR Offloading” job overall may be using hundreds or thousands of concurrent connections, resulting in sub-optimal performance.

The performance will depend on the Veeam setup in the environment. If you have a single repository with 8 concurrent tasks and the above setting is left standard, you may have the ability to overpower the object store. For example: 8 x 64 connections, which may result in high CPU usage.

Since Veeam 11, the number of tasks can be throttled on the repository.

Open

To calculate the number of S3 operations that are processed in parallel (max), use this calculation: Available Repository Task Slots * S3ConcurrentTaskLimit = Max used S3 operations in parallel.

Use the Veeam Object Storage IO sizing calculator VSE FrontEnd for the best setting.

Task Throttling

Max Concurrent Tasks per Backup Proxy, by default, Veeam recommends this to be set to 2x the number of CPU cores in your VBR server. (Apply the same modifications on the VM backup proxy).

For more information, see Limitation of Concurrent Tasks - User Guide for VMware vSphere

Open

Handling Multiple Veeam Backup Agents

Veeam recommends the use of VBR “managed mode” which provides network traffic rules via their backup proxies. In this mode, all data protection and administration tasks are performed by a backup administrator in Veeam Backup and Replication.

Reference: Veeam Agent Management Infrastructure - Veeam Agent Management Guide

Optimization Inside DataCore Swarm

By default, the cloud gateway 7.10.4 uses 50 threads to execute S3 MultiDelete requests, you can improve performance by increasing recursiveDeleteMaxThreads to 250 in /etc/caringo/cloudgateway/gateway.cfg

recursiveDeleteMaxThreads must always be lower than threads and maxConnections

so if you set recursiveDeleteMaxThreads then we recommend also changing:

Example for a cluster of 4 Swarm Storage Nodes

threads = 400

maxConnections = 400

MaxConnectionsPerRoute = (maxConnections / Total Swarm Storage Nodes)

Final Remarks

For additional technical details, see Veeam V12 Guidelines.

For additional configuration steps, see the deployment guide for using Integrating Swarm Object Storage with Veeam Backup and Replication.