Versions Compared

Old Version 20

changes.mady.by.user Kyle Miller

Saved on

compared with

New Version 21

changes.mady.by.user Anjali Tyagi

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Updated as per SWAR-9700

The S3 Backup Restore Tool is the standalone utility for performing DR from the S3 backup bucket, either to the original cluster or to an empty cluster that is meant to replace the original. See S3 Backup Feeds.

Once the data is backed up in S3, the restore tool allows both examining a backup and control how, what, and where it is restored:

...

The S3 Backup Restore tool has a separate install package included in the Swarm download bundleavailable by Support request. Install it on one or more (for parallel restores) systems where the restore processes run.

...

The swarmrestore package is delivered as a Python pip3 source distribution. Each machine needs to be prepared to be able to install this and future versions of swarmrestore.

  1. As root, run the following command:

    Code Block
    languagebash
    yum install python3

  2. Verify version 3.6 is installed:

    Code Block
    python3 --version

  3. Upgrade pip

Code Block
languagebash
pip3 install --upgrade pip

...

Rerun this installation when a new version of swarmrestore is obtained:

  1. Copy the latest version of the swarmrestore package to the server.

  2. Run the following as root:

    Code Block
    languagebash
    pip3 install caringo-swarmrestore-<version>.tar.gz

  3. swarmrestore is likely in /usr/local/bin and is already in the path.

  4. Repeat for any additional servers if planning to perform partitioning for parallel restores.

Restore Tool Settings

The tool uses a configuration file, .swarmrestore.cfg. Because the file contains sensitive passwords, the tool warns if the configuration file is not access-protected (chmod mode 600 or 400).

The configuration file follows the format of Swarm Storage settings files, using sections listing name = value pairs. These setting names map to the S3 Backup feed definition, where the values have the same meaning.

  1. Locate the sample configuration file where it is installed:

    Code Block
    /usr/local/sample-.swarmrestore.cfg

  2. Copy the file into the home directory and rename it, and open it for editing:

    Code Block
    languagebash
    cp /usr/local/sample-.swarmrestore.cfg ~/.swarmrestore.cfg
vi ~/.swarmrestore.cfg 	           # Edit config settings
sample-.swarmrestore.cfg
Code Block
# This is a sample configuration file for the swarmrestore utility.
# Save this file as ~/.swarmrestore.cfg and chmod 600 ~/.swarmrestore.cfg to keep passwords private.

# S3 host must be a fully qualified host name. The virtual host access style is supported if
# the host's first component is the bucket name.
# See https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region for Amazon S3 endpoints.
[s3]
host=s3.amazonaws.com
port=443
accessKeyID=<youraccesskeyid>
secretAccessKey=<yoursecretaccesskey>
bucketName=<yourbucketname>
region=us-east-1
# The option below uses HTTPS for access. For HTTP, set sslOption=none and adjust port.
sslOption=trusted

# The 4 options below are for swarmrestore initiating archival restore of content, such as GLACIER.
performArchiveRetrieval=false
retrievalTier=Standard
accountID=<ninedigitaccountid>
activeLifetimeDays=7

# Use these only if you need a forward proxy to reach the S3 service.
[forwardProxy]
host=
port=80
username=
password=

# The log file can be /dev/null, but logs are useful for diagnosing problems.
[log]
filename=swarmrestore.log
level=30

# The Swarm cluster must either be directly accessible or accessible via
# a proxy. The password below is the administrative password for the cluster.
[swarm]
host=<space separated list of swarm host IPs or gateway host>
password=ourpwdofchoicehere
cluster=<yourclustername>

Section

Settings

[s3] 

  • host — The hostname of the S3 service.

  • port — The port to use for the S3 service. Use 443 or else 80, if SSL (sslOption) is disabled.

  • accessKeyID — The S3 access key ID.

  • secretAccessKey — The S3 secret access key.

  • bucketName — The name of the destination bucket in S3.

  • sslOption — The S3 connection constraint, with one of two values:

    • "trusted" (the default) specifies use of SSL and requires a trusted server certificate from the destination server.

    • “none” disables use of SSL. Use for testing and troubleshooting, and change the port to 80.

[s3]

archival

Set these additional parameters if using an S3 bucket with an archival storage class (Glacier, Glacier Deep Archive):

  • performArchiveRetrieval — Whether restoration from archival storage is needed. Performing a restore does not incur any expenses for the bucket owner if false (default), .

  • retrievalTier — Which S3 Glacier retrieval tier to use for restoration: ‘Standard' (default), 'Expedited', or 'Bulk'. Each tier has its own cost and expected restoration time; see Amazon S3 Storage Classes.

  • accountID — Specifies the 9-digit AWS account ID of the bucket owner, granting the tool permission to incur archive restoration expenses at the tier requested. This setting appears in the x-amz-expected-bucket-owner header on the restore object request.

  • activeLifetimeDays — How many days an object restored from archive should remain active before expiring (returning to archival storage). The default is 7 (1 week).

[forwardProxy] 

This section is for use with an optional forward proxy:

  • host — The forward proxy hostname or IP address.

  • port — The forward proxy host to use.

  • username — (optional) The user name.

  • password — (optional) The password.

[log] 

The same log settings as the Swarm cluster may be used; identify the logs by looking for those with the component "RESTORE" if done so.

  • host — The log host. Leave blank to disable logging.

  • port — (optional) The log port. Defaults to 514.

  • file — (optional) The log filename. Accepts the value of “stdout” for logging to the console screen. Defaults to /dev/null.

  • level — The log level. Defaults to 30 (Warning). Levels are the same used by Swarm: 20 (Info), 15 (Audit), 10 (Debug).

[swarm] 

  • host — A list of host names or IP addresses of Swarm nodes or Gateway nodes.

  • port — (optional) The SCSP port. Defaults to 80.

  • user — The cluster administrator user name, usually "admin".

  • password — The cluster administrator password.

...