Swarm is designed to provide both fast and secure data storage. Selecting the appropriate trade-offs between performance, capacity, and data protection that best meet your business needs is a critical part of a successful storage solution.

Role of the Health Processor

Swarm includes the Health Processor (HP) that provides end-to-end, disk-level, and life-cycle data protection. The Health Processor monitors both data integrity and cardinality (the number of object replicas) continuously, and it heals any degradation or non-conformity within the storage cluster that it finds.

The Health Processor regularly scans every object on disk to verify its integrity. If the object is corrupt (such as due to a bad sector), HP removes it. HP then detects and corrects the object's under-replication by triggering the creation of another replica of that object elsewhere in the cluster, thus restoring the correct number of replicas.

Using the Health Processor, Swarm can:

• Verify the correct object protection level exists in the cluster.

• Distribute replicas and erasure coding segments properly across subclusters.

• Enforce lifepoint policies by enforcing replica and/or erasure coding segment counts and delete policies (terminal lifepoints) at different policy time intervals.

• Validate the object on-disk integrity and create a new replica if the integrity is compromised.

• Economically store and load balance by periodically evaluating if an object is optimally stored in its current location.

• Defragment storage space on an as-needed basis.

See https://perifery.atlassian.net/wiki/spaces/public/pages/2443821972.

Content-MD5 Integrity Checking

In addition to the protection provided by the Health Processor, Swarm uses the Content-MD5 metadata header to provide end-to-end message integrity check of the object body (excluding metadata) as it is sent to and from Swarm.

See https://perifery.atlassian.net/wiki/spaces/public/pages/2443822141.