...
With Swarm 16.1.4 and Gateway 8.1.0 we introduced a new feature called “Index per Domain”, this means within our Elasticsearch database we will now store metadata from a each domain in separate indices. It is enabled by setting search.perDomainIndex=True before creating a Search Feed.
Data in Elasticsearch is organized into indices. Each index is made up of one or more shards. Each shard is an instance of a Lucene index, which you can think of as a self-contained search engine that indexes and handles queries for a subset of the data in an Elasticsearch cluster.
...
| Panel | ||
|---|---|---|
| ||
Option 3: You split the index , which amounts to increasing the shard number for that domain. Note: this option requires scheduled maintenance window as the index needs to be in read-only mode during this split process. Object backup repository can only be put into maintenance mode if they are a member of a SOBR ( scale-out backup repository ), it does not exist when using Direct to S3 backup mode. If you do not use SOBR, you will need to suspend all backup jobs and disable background retention before proceeding with a split procedure. Veeam v12.1 introduced a new feature to temporarily disable background retention, you will need to activate this during the split operation. ( See https://helpcenter.veeam.com/docs/backup/vsphere/backup_background_retention_disable.html?ver=120 ) For more details see: TODO final link [DRAFT] Steps to split or reduce the number of shards in perdomain index /wiki/spaces/~5f4d1b7773a268004df5fcac/pages/3696558130 |
| Note |
|---|
Keep in mind that splitting an existing index will increase the shards it uses and account towards the maximum 600 shards per node limit. |
...
In the example above we had 5 data nodes, with 600 shards per node this means Swarm can support up to 250 domains ( some indexes indices are needed for csmetrics, kibana, etc.. )
...