Swarm is designed to provide both fast and secure data storage. Selecting the appropriate trade-offs between performance, capacity, and data protection that best meet your business needs is a critical part of a successful storage solution.
Role of the Health Processor
Swarm includes the Health Processor (HP) that provides end-to-end, disk-level, and life-cycle data protection. The Health Processor monitors both data integrity and cardinality (the number of object replicas) continuously, and it heals any degradation or non-conformity within the storage cluster that it finds.
The Health Processor regularly scans every object on disk to verify its integrity. If the object is corrupt (such as due to a bad sector), HP removes it. HP then detects and corrects the object's under-replication by triggering creation of another replica of that object elsewhere in the cluster, thus restoring the correct number of replicas.
Using the Health Processor, Swarm can:
- Verify the correct object protection level exists in the cluster.
- Distribute replicas and erasure coding segments properly across subclusters.
- Enforce lifepoint policies by enforcing replica and/or erasure coding segment counts and delete policies (specifically, terminal lifepoints) at different policy time intervals.
- Validate the object on-disk integrity and create a new replica if the integrity is compromised.
- Economically store and load balance by periodically evaluating if an object is optimally stored in its current location.
- Defragment storage space on an as-needed basis.
See Lifepoint Metadata Headers.
Content-MD5 Integrity Checking
In addition to the protection provided by the Health Processor, Swarm uses the Content-MD5 metadata header to provide end-to-end message integrity check of the object body (excluding metadata) as it is sent to and from Swarm.