Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

To make the most cost-effective use of your storage footprint, you can reduce the redundancy of Swarm's content protection. These are types of content that may be candidates for reduced redundancy:

...

Balancing the downside to losing the file versus the likelihood that you need that particular one is needed, you protection can lower protection be lowered as described below.

How to enable reduced redundancy

  1. First, lower the cluster-wide minimum for replication, which sets the absolute lower bound for the cluster: Change policy.replicas: min  to 1, if it is a 2 or higher.

    Code Block
    policy.replicas min:1 max:<int> default:<int>

    Note: the minimum is not the default. This setting can be changed dynamically, using the UI or SNMP. See Implementing Replication Policy.

    Info
    titleCaution

    Parity of 1 for erasure coding lowers protection and so should not be used as your cluster's default (policy.ecEncoding=N:1); as N increases, N:1 approaches the protection of having only a single copy of the object in the cluster.

  2. Best practice: If you know that you regularly need to store a category of lesser content regularly needs to be store, provide it its own container (context) and set reduced redundancy on that container only.

    1. Create or designate a specific domain or bucket to be dedicated to this content.

    2. Update the domain or bucket properties to enforce single replicas, no erasure coding, and no versioning, using the Content UI or a manual command:

      Code Block
      curl -iL -XPOST --post301 --data-binary ""
          -H "Policy-Replicas: default:1"
          -H "Policy-ECEncoding: disabled"
      	-H "Policy-Versioning: disabled"
          "http://{cluster}/myBucket?domain=myDomain" 

      See Implementing EC Encoding Policy.

      Info
      titleWhy disable erasure coding?

      When storing a category of lesser content with reduced redundancy, erasure coding should not be used. If an erasure-coded object degrades (loses more than p segments in a k:p encoding, such as 2:1), then the remaining segments still take up cluster space, and the health processor continues to look for k segments to attempt to reconstruct missing ones. With whole replica encoding, losing lesser data results in the entire object being removed from the cluster, reclaiming that space and avoiding extra processing.

  3. Apart from your special domain or bucket, when you have an individual file (such as a backup) that is a candidate for reduced redundancy, write it with lifepoint headers that lower redundancy. See Lifepoint Metadata Headers.
    In this example, a chunked upload (which must be EC encoded) has two lifepoints: the first lifepoint specifies an EC encoding that expires in one day, and the second specifies that the cluster needs to keep only one replica after that.

    Code Block
    Transfer-Encoding: chunked
    Lifepoint: [Wed, 4 Apr 2019 15:59:02 GMT] reps=2:1
    Lifepoint: [] reps=1