This guide outlines the best practices for using Datacore Swarm as a capacity tier for Veeam. Emphasis has been placed on Veeam Scale-out Backup Repository (SOBR) capacity planning, configuration, and workflow optimization.

Use Scale-Out Backup Repository not Direct to S3 mode

As an administrator you need to have the ability to put a backup repository into maintenance mode, ( for example in the event of a bucket re-shard event ) , this is only possible with a Scale-Out backup repository configuration.

SOBR supports setting S3 backup repository as a performance tier, in combination with a backup-copy job this would fall under 3-2-1 backup rule and not rely on a single backup tier.

Comply with the 3-2-1 backup rule do not rely on a single backup tier

This rule is a robust guideline for data protection, ensuring redundancy, resilience, and the ability to recover data even in the face of unexpected events or disasters.

What is the 3-2-1 backup rule?

Use Multiple SOBR

Currently, Veeam supports only one object storage bucket per SOBR configuration. We recommend using multiple SOBRs pointing to different buckets and storage domains per client and workload type (VB365 Backup, Archive backup jobs etc.).

This will allow you to take advantage of “Per domain indexing” , where we create separate Elasticsearch indices per storage domain.

For best performance, we recommend using a performance tier in a SOBR as primary target.

Another advantage of multiple SOBR is that you can set different retention policies per SOBR.

Backup Job Storage Settings

Veeam allows you to configure a block size per Backup Job. The block size affects deduplication and incremental backup size. By default, blocks are compressed and the compression ratio is about 50%.

The smaller the block size, the more calls are needed to the object storage to upload the data. DataCore recommends to use “Local Target (large blocks)”. This is equal to a 4MB (=2MB after compression) object size. Performance Tier storage requirements depend on this setting, too.

Consult with Veeam or your partner before setting this.

See Veeam V12 Guidelines for more details on how to set the storage block size.

SOBR Offloading

For the SOBR offloading the aggregated number of concurrent tasks of all performance tier extents is used. If an extent has no limit on concurrent tasks, SOBR offload uses an unlimited number of tasks (up to the # of backup chains).

Each SOBR offload task will use up to 64 connections to the capacity tier. This is defined by S3ConcurrentTaskLimit in the Windows registry settings. (HKLM\SOFTWARE\Veeam\Veeam Backup and Replication\S3ConcurrentTaskLimit , requires a reboot to apply).

The default value of 64 (decimal) is applied to every worker process, meaning that the “SOBR Offloading” job overall may be using hundreds or thousands of concurrent connections, resulting in sub-optimal performance.

The performance will depend on the Veeam setup in the environment. If you have a single repository with 8 concurrent tasks and the above setting is left standard, you may have the ability to overpower the object store. For example: 8 x 64 connections, which may result in high CPU usage.

Since Veeam 11, the number of tasks can be throttled on the repository.

Open

To calculate the number of S3 operations that are processed in parallel (max), use this calculation: Available Repository Task Slots * S3ConcurrentTaskLimit = Max used S3 operations in parallel.

Use the Veeam Object Storage IO sizing calculator Veeam Calculators for the best setting.

Task Throttling

Max Concurrent Tasks per Backup Proxy, by default, Veeam recommends this to be set to 2x the number of CPU cores in your VBR server. (Apply the same modifications on the VM backup proxy).

For more information, see Limitation of Concurrent Tasks - User Guide for VMware vSphere

Open

Handling Multiple Veeam Backup Agents

Veeam recommends the use of VBR “managed mode” which provides network traffic rules via their backup proxies. In this mode, all data protection and administration tasks are performed by a backup administrator in Veeam Backup and Replication.

Reference: Veeam Agent Management Infrastructure - Veeam Agent Management Guide

Optimization Inside DataCore Swarm

By default, the cloud gateway 7.10.4 uses 50 threads to execute S3 MultiDelete requests, you can improve performance by increasing recursiveDeleteMaxThreads to 200 in /etc/caringo/cloudgateway/gateway.cfg

recursiveDeleteMaxThreads must always be lower than threads and maxConnections

so if you set recursiveDeleteMaxThreads then we recommend also changing:

threads = 100 x number of CPU in Gateway

maxConnections = (threads/2) * swarm storage nodes

maxConnectionsPerRoute = threads/2

Example often used in the field: ( 10 storage nodes )

Veeam v12.3 auto-provisioning of buckets feature

Veeam v12.3 introduced the concept of auto-provisioning of buckets in a single backup repository based on a configurable total objects threshold.

Final Remarks

For additional technical details, see Veeam V12 Guidelines.

For additional configuration steps, see the deployment guide for using Integrating Swarm Object Storage with Veeam Backup and Replication.