Versions Compared

Old Version 11

changes.mady.by.user Kyle Miller

Saved on

compared with

New Version 12

changes.mady.by.user Kyle Miller

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

The S3 Backup Restore Tool is the standalone utility for performing DR from your S3 backup bucket, either to the original cluster or to an empty cluster that is meant to replace the original. See S3 Backup Feeds.

Once your data is backed up in S3, the restore tool allows both examining a backup and control how, what, and where it is restored:

...

Section

Settings

[s3] 

  • host — The hostname of the S3 service.

  • port — The port to use for the S3 service. Use 443 or else 80, if SSL (sslOption) is disabled.

  • accessKeyID — The S3 access key ID.

  • secretAccessKey — The S3 secret access key.

  • bucketName — The name of the destination bucket in S3.

  • sslOption — The S3 connection constraint, with one of two values:

    • "trusted" (the default) specifies use of SSL and requires a trusted server certificate from the destination server.

    • “none” disables use of SSL. Use only for testing and troubleshooting, and change the port to 80.

[s3]

archival only

If you are using an S3 bucket with an archival storage class (Glacier, Glacier Deep Archive), set these additional parameters:

  • performArchiveRetrieval — Whether restoration from archival storage is needed. If false (default), performing a restore will not incur any expenses for the bucket owner.

  • retrievalTier — Which S3 Glacier retrieval tier to use for restoration: ‘Standard' (default), 'Expedited', or 'Bulk'. Each tier has its own cost and expected restoration time; see Amazon S3 Storage Classes.

  • accountID — Specifies the 9-digit AWS account ID of the bucket owner, granting the tool permission to incur archive restoration expenses at the tier requested. This setting appears in the x-amz-expected-bucket-owner header on the restore object request.

  • activeLifetimeDays — How many days an object restored from archive should remain active before expiring (returning to archival storage). The default is 7 (1 week).

[forwardProxy] 

This section is for use only with an optional forward proxy:

  • host — The forward proxy hostname or IP address.

  • port — The forward proxy host to use.

  • username — (optional) The user name.

  • password — (optional) The password.

[log] 

You may use the same log settings as your Swarm cluster; if you do so, identify the logs by looking for those with the component "RESTORE".

  • host — The log host. Leave blank to disable logging.

  • port — (optional) The log port. Defaults to 514.

  • file — (optional) The log filename. Accepts the value of “stdout” for logging to the console screen. Defaults to /dev/null.

  • level — The log level. Defaults to 30 (Warning). Levels are the same used by Swarm: 20 (Info), 15 (Audit), 10 (Debug).

[swarm] 

  • host — A list of host names or IP addresses of Swarm nodes or Gateway nodes.

  • port — (optional) The SCSP port. Defaults to 80.

  • user — The cluster administrator user name, usually "admin".

  • password — The cluster administrator password.

...

Info

Full cluster restore

Before undertaking a restore of a large cluster, contact DataCore Support. They will help you balance the speed of your restore with your bandwidth constraints by examining the space used by the S3 backup bucket, estimating the bandwidth needed, and recommending best use of the -p command line option (for multiple simultaneously running restore commands on different hosts). They will also advise you on whether you need a forward proxy, to reduce bandwidth usage.

If you are using an AWS Glacier storage class, you might may pull your AWS bucket out of cold storage before your full cluster restore by changing the storage class to Standard.

...

  • -R or --recursive — Recursively lists the given domain or bucket, or else the entire cluster. Without this option, the command lists only the top-level contents of the object.

  • -v or --versions — List previous versions of versioned objects. Versions are not listed by default.

  • -l or --long — Lists details for each item returned in the output:

    • Creation date

    • Content length of the body

    • ETag

    • Archive status:

      • AN — Archived; not available for restoration

      • AR — Archived with an archive restore in progress; not available for restoration

      • AA — Archived with a copy available for restoration

      • OK — Not archived and fully available

    • Objectspec

    • Alias UUID, if the object is a domain or bucket

  • <objectspec> — If none, the command runs across the entire contents of the S3 backup. If present, filters the command to a specific domain or bucket (context object) in Swarm. Use this format:

Cluster


Domain

mydomain/

Bucket

mydomain/mybucket/

Named object

mydomain/mybucket/myobject/name/with/slashes.jpg

Named version

mydomain/mybucket/myobject/name/with/slashes.jpg//645f3912802bb4c31311afc46de2cfc3

Unnamed object

mydomain/06ea262a860af23504261f50c09a6b29 (no domain if untenanted)

Unnamed version

mydomain/06ea262a860af23504261f50c09a6b29//137a88d550041ecda9b8ec4bc36ebea2

Info

...

Note

Use the double-slash format (//) before including a specific version ID for an object. Newlines separate objects.

When you run the command without any options, it returns the list of domains that are included in this S3 bucket for your Swarm cluster:

...

Cluster


Domain

mydomain/

Bucket

mydomain/mybucket/

Named object

mydomain/mybucket/myobject/name/with/slashes.jpg

Named version

mydomain/mybucket/myobject/name/with/slashes.jpg//645f3912802bb4c31311afc46de2cfc3

Unnamed object

mydomain/06ea262a860af23504261f50c09a6b29 (no domain if untenanted)

Unnamed version

mydomain/06ea262a860af23504261f50c09a6b29//137a88d550041ecda9b8ec4bc36ebea2

Info

Note

Use the double-slash format (//) before including a specific version ID for an object. Newlines separate objects.

...

Rate of Restore — Restoration might may take a long time run, especially if recursion (-R) is used on domains or buckets. To boost the rate of restore, you can install the S3 Backup Restore tool on multiple servers and then run the restore command with partitioning parameters (-p) across all the instances of the tool, which allows restoring faster in parallel, with minimal overlap.

...