...

1. Create a Snapshot Repository

   First, You need to create a snapshot repository where the snapshots will be stored. This can be a shared file system , Amazon S3, HDFS, etc. For this example, we'll use a shared file system.or an S3 bucket.
   Using a Shared File System:
   First, specify the shared repository location in the elasticsearch.yml file:

   Code Block
   path.repo: "/mount/backups/my_backup"

   Then, create the repository using the following command:

   Code Block
   curl -X PUT "http://<es_node_ip>:9200/_snapshot/my_backup" -H 'Content-Type: application/json' -d' { "type": "fs", "settings": { "location": "/mount/backups/my_backup" } }'

   When the target repository is another Swarm cluster, the command to create the snapshot repository would be as follows:

   Code Block
   curl -X PUT "http://<es_node_ip>:9200/_snapshot/my_s3_backup" -H 'Content-Type: application/json' -d' { "type": "s3", "settings": { "bucket": "my-elasticsearch-backup-bucket", "endpoint": "https://datacore-swarm.example.com", "access_key": "your_access_key", "secret_key": "your_secret_key", "protocol": "https" } }'

Replace ‘my-elasticsearch-backup-backup’, ‘https://datacore-swarm.example.com’, ‘your_access_key’, and ‘your_secret_key’ with your actual S3 bucket name, DataCore Swarm endpoint URL, and AWS credentials.
NOTE: Ensure that the location path is accessible and writable by all nodes in the cluster.
NOTE: The shared repository location will have to be specified in the elasticsearch.yml file as a “path.repo” parameter:path.repo: "/mount/backups/my_backup"

2. Verify the Repository

   After creating the repository, verify it to ensure it is set up correctly:

   Code Block
   curl -X GET "http://<es_node_ip>:9200/_snapshot/my_backup"

3. Create a Snapshot

   Once the repository is set up and verified, create a snapshot of your index. Replace index_mumbkctcomobs.datacore.com.com0 with your index name.

   Code Block
   curl -X PUT "http://<es_node_ip>:9200/_snapshot/my_backup/snapshot_$(date +\%Y\%m\%d\%H\%M)" -H 'Content-Type: application/json' -d' { "indices": "index_swarm.datacore.com.com0", "ignore_unavailable": true, "include_global_state": false }'

4. Automate Snapshot Creation

   To automate the creation of snapshots, you can use cron jobs on Linux or scheduled tasks on Windows.

   Example using a cron job (runs daily at 2 AM):

   Code Block
   0 2 * * * curl -X PUT "http://<es_node_ip>:9200/_snapshot/my_backup/snapshot_$(date +\%Y\%m\%d\%H\%M)" -H 'Content-Type: application/json' -d' { "indices": "index_swarm.datacore.com.com0", "ignore_unavailable": true, "include_global_state": false }'

5. Monitor Snapshots

   Regularly check the status of your snapshots to ensure they are completing successfully:

   Code Block
   curl -X GET "http://<es_node_ip>:9200/_snapshot/my_backup/_all/_status"

6. Restoring a Snapshot (if needed)
   If you need to restore a snapshot, you can do so with the following command:

   Code Block
   curl -X POST "http://<es_node_ip>:9200/_snapshot/my_backup/snapshot_<snapshot_date>/_restore" -H 'Content-Type: application/json' -d' { "indices": "index_swarm.datacore.com.com0", "ignore_unavailable": true, "include_global_state": false }'

NOTE:

...

2. For the S3 bucket:

   Code Block
   curl -X POST "http://<es_node_ip>:9200/_snapshot/my_s3_backup/snapshot_<snapshot_date>/_restore" -H 'Content-Type: application/json' -d' { "indices": "index_swarm.datacore.com.com0", "ignore_unavailable": true, "include_global_state": false

...

2. }'

   • '"include_global_state": false' means that only the data

...

2. • stored in the particular index is restored.

...

2. • If you wan to restore everything from the cluster

...

2. • , including templates, persistent cluster settings, and

...

2. • more, set '"include_global_state": true'.

Automating with Elasticsearch Curator

...