Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 8 Next »

To make the most cost-effective use of your storage footprint, you can reduce the redundancy of Swarm's content protection. These are types of content that may be candidates for reduced redundancy:

  • Nightly backups, which you can restore from using any save set within a few days

  • Intermediate files in a multi-step operation

  • Files that can be recreated on demand, as needed

  • Content of little business impact if lost

Balancing the downside to losing the file versus the likelihood that you need that particular one, you can lower protection as described below.

How to enable reduced redundancy

  1. First, lower the cluster-wide minimum for replication, which sets the absolute lower bound for the cluster: Change policy.replicas: min  to 1, if it is a 2 or higher.

    policy.replicas min:1 max:<int> default:<int>

    Note: the minimum is not the default. This setting can be changed dynamically, using the UI or SNMP. See Implementing Replication Policy.

    Caution

    Parity of 1 for erasure coding lowers protection and so should not be used as your cluster's default (policy.ecEncoding=N:1); as N increases, N:1 approaches the protection of having only a single copy of the object in the cluster.

  2. Best practice: If you know that you regularly need to store a category of lesser content, provide it its own container (context) and set reduced redundancy on that container only.

    1. Create or designate a specific domain or bucket to be dedicated to this content.

    2. Update the domain or bucket properties to enforce single replicas, no erasure coding, and no versioning, using the Content UI or a manual command:

      curl -iL -XPOST --post301 --data-binary ""
          -H "Policy-Replicas: default:1"
          -H "Policy-ECEncoding: disabled"
      	-H "Policy-Versioning: disabled"
          "http://{cluster}/myBucket?domain=myDomain" 

      See Implementing EC Encoding Policy.

      Why disable erasure coding?

      When storing a category of lesser content with reduced redundancy, erasure coding should not be used. If an erasure-coded object degrades (loses more than p segments in a k:p encoding, such as 2:1), then the remaining segments still take up cluster space, and the health processor continues to look for k segments to attempt to reconstruct missing ones. With whole replica encoding, losing lesser data results in the entire object being removed from the cluster, reclaiming that space and avoiding extra processing.

  3. Apart from your special domain or bucket, when you have an individual file (such as a backup) that is a candidate for reduced redundancy, write it with lifepoint headers that lower redundancy. See Lifepoint Metadata Headers.
    In this example, a chunked upload (which must be EC encoded) has two lifepoints: the first lifepoint specifies an EC encoding that expires in one day, and the second specifies that the cluster needs to keep only one replica after that.

    Transfer-Encoding: chunked
    Lifepoint: [Wed, 4 Apr 2019 15:59:02 GMT] reps=2:1
    Lifepoint: [] reps=1
  • No labels