Setting Storage Policies

The following options are available to specify storage policies to apply to the objects it contains when editing the Properties of a domain or bucket:

A domain or bucket inherits the protection settings in force above it (cluster, domain) by default.

Required

Grant users these specific permissions: PutDomain and CopyBucket if users are allowed to change the content policies (replication, erasure coding, or versioning) on a domain or bucket through the Content UI.

  • To see the options for a storage policy, deselect the Inherit checkbox, which expands the policy section.

  • For guidance about the policy options, click the information icon, which toggles the help text.

Protection

Swarm allows flexibility in determining the type and level of content protection best fitting the storage needs. In Swarm storage, objects can be replicated and/or erasure-coded, with objects of both types co-existing in the same cluster.

Tip

Erasure coding helps cost-effectively scale clusters with many nodes and larger objects, while replication is better for smaller clusters and with smaller objects.

These settings allow a choice of replication and erasure-coding protection policies for the objects in this immediate context, subject to overrides by higher-level (cluster or domain) settings.

Replication

Replication protection requires the cluster to keep a specified number of copies (replicas) of each object.

image-20240209-115333.png
  • Default Replicas: Accept the inherited number or enter how many replicas are desired (subject to existing min and max values and query arguments).

  • Minimum and Maximum Replicas: A minimum number of replicas is two and the maximum is sixteen for an object.

  • Anchored: Select to override any lower-level policies.

For more about replication in Swarm, see https://perifery.atlassian.net/wiki/spaces/public/pages/2443812060.

For implementation, see https://perifery.atlassian.net/wiki/spaces/public/pages/2443812082.

Erasure Coding

Erasure coding (EC) protection divides very large objects into multiple data (k) and parity (p) segments for distribution across k+p nodes, which is more space-efficient than storing very large replicas.

  • Enabled: Select to allow erasure coding at this level and below (subject to higher-level policies). 

  • EC Size Threshold (MB): (not settable) Reports the object size that triggers erasure coding rather than replication. 

  • Default Encoding: Accept the inherited encoding, or enter data (k) and parity (p) values such as these examples:

    • 5 : 2 (1.4x footprint) - protection for 2 simultaneous disk failures.

    • 9 : 6 (1.7x footprint) - protection for 6 simultaneous disk failures and 1 subcluster failure in clusters of 3 or more subclusters.

  • Anchored: Select to override any lower-level policies.

For more about erasure coding in Swarm, see https://perifery.atlassian.net/wiki/spaces/public/pages/2443812123.

For implementation, see .

Versioning

Swarm supports object-level versioning, which is a powerful content protection option that tracks, secures, and provides access to historical versions of objects, even after they are deleted. With versioning, applications can read, list, revert, and purge prior versions as well as restore objects deleted by mistake.

Best Practice

  • Plan for higher disk utilization with versioning: each update to a versioned object adds a new object to the cluster (one object updated twice results in three objects stored).

  • Where possible, make use of lifepoints to control the lifetime – and thus the cost of storing – multiple versions of objects.

  • For optimal resource management, limit versioning to the specific domains and/or buckets for which it is needed.

For more about versioning in Swarm, see .

For implementation, see .

The versioning state of the immediate context applies to every object in that context, without exception. Each domain and bucket has one of these versioning states:

disabled

(default)
No versioning exists, so no versions are created.

This state is the normal behavior of Swarm.

The Health Processor cleans up all residual prior versions, regaining space if changed from enabled to disabled in the domain or bucket.

This feature (changing to disabled) is not available in Amazon S3.

suspended

No new versions are accumulated but old versions are retained.

This is a hybrid between enabled and disabled that preserves history.

The chain of versions resumes from where it stopped if versioning is re-enabled from this state.

enabled

New versions are accumulated as they are created, starting with any version that exists at the time versioning becomes enabled for the object.

This is the only parent state from which domains and buckets can enforce a versioning policy that differs from its parent.

required

Domains only. Requires each of its buckets to have versioning enabled.

Use this to prevent bucket owners from disabling versioning policy.

 

Policies for Unnamed Objects

All named objects are controlled by the policies on the buckets, but unnamed objects are handled separately for each domain. 

To view and set the storage policy for unnamed objects in the domain, view All Buckets and open the special Content IDs bucket, which is the system-controlled container for all unnamed objects in the domain. Open the settings (cog icon) and specify the policy to apply to the unnamed objects in the domain:

Because immutable objects cannot be updated, they cannot be versioned.

Override Alerts

Because the defined policies are subject to override by policies at higher levels (such as the cluster settings), alert icons, and messages inform if the specified policy is blocked from going in to effect.

© DataCore Software Corporation. · https://www.datacore.com · All rights reserved.