The backup software "restic" (https://restic.net) earns its tagline "Backups done right!" as a safe and easy cross-platform command-line utility for backing up file systems.
It is not a “single namespace” kind of client like s3cmd or rclone, where the named streams in Swarm match the names of the original files. Restic stores all directories and files in 5MB deduped and client-side encrypted blobs. While it uses the S3 protocol it does not use S3 multipart uploads. It does not support compression (https://github.com/restic/restic/issues/21) or adjustable block sizes (https://github.com/restic/restic/issues/1071).
As you’d expect restic performs incremental backups, letting you make and recover snapshots and tags.
By default the encryption keys are kept on the server alongside the data, so you don’t have to worry about managing those keys. But the keys are encrypted with a password you enter when you run "restic init". Remember this is warrant-proof storage — if you forget the password you won’t be able to recover any backups!
A killer feature on Linux and macOS is that restic lets you mount your backups to a local virtual (FUSE) directory. This lets you e.g. verify your backup by mounting it then doing a simple recursive diff.
Step-by-step guide
Restic configuration is via environment variables for the access key, secret key, and domain endpoint.
- Install "restic" on the machine with access to the directory you will be backing up. Restic should be available in your operating system's package manager or see https://restic.readthedocs.io/en/stable/020_installation.html.
$ brew install restic # or apt-get install -y restic or yum install -y restic - Create an S3 token via the Content UI or a curl to your domain endpoint:
$ curl -i -u caringoadmin -X POST -H 'X-User-Secret-Key-Meta: secret' \HTTP/1.1 201 Created
-H 'X-User-Token-Expires-Meta: +365' 'https://support.cloud.caringo.com/.TOKEN/'
Enter host password for user 'caringoadmin': ...
Date: Tue, 03 Jul 2018 18:10:18 GMT
Gateway-Request-Id: 2FE9C3C985129777
Server: CAStor Cluster/9.6.0
Via: 1.1 support.cloud.caringo.com (Cloud Gateway SCSP/5.3.0)
Gateway-Protocol: scsp
Set-Cookie: token=ce1a7e2f8ee53b035ab6d2f3979946a4; expires=Wed, 03-Jul-2019 18:10:18 GMT; path=/
Content-Type: text/plain;charset=utf-8
Content-Length: 87Token
ce1a7e2f8ee53b035ab6d2f3979946a4issued for caringoadmin in [root] with secret secret - Export the configuration variables:
$ export AWS_ACCESS_KEY_ID=ce1a7e2f8ee53b035ab6d2f3979946a4$ export AWS_SECRET_ACCESS_KEY=secret$ export RESTIC_REPOSITORY=s3:support.cloud.caringo.com/mybucket Initialize the restic repo the first time you create a backup. This will also create the bucket if it does not exist.
$ restic initenter password for new repository:enter password again:created restic repository 07617a1b05 at s3:support.cloud.caringo.com/mybucketPlease note that knowledge of your password is required to accessthe repository. Losing your password means that your data isirrecoverably lost.Now you're ready to backup a directory, using the same password you used with "restic init". This initial backup of 230GB took ~2 hours.
$ restic backup /var/jenkins_home/enter password for repository:repository 07617a1b opened successfully, password is correctFiles: 461496 new, 0 changed, 0 unmodifiedDirs: 1 new, 0 changed, 0 unmodifiedAdded: 228.495 GiBprocessed 461496 files, 277.912 GiB in 1:50:50snapshot f6f0e61f saved- On Linux or macOS with FUSE support you can mount the backup as a read-only virtual directory to directly access backed up files. Enter the same password you used with "restic init".
$ restic mount /mnt/restic-remote # run this in a separate window - Now you can access snapshots directly, eg..
$ diff -u -r /var/jenkins_home /mnt/restic-remote/snapshots/latest/var/jenkins_home
Caringo does not directly support or guarantee third-party utilities but let us know if you have questions or run into problems.
Related articles