Elasticsearch requires configuration and settings file changes to be made consistently across your Elasticsearch cluster.
...
Bulk
Usage | This method is most efficient if you have a large number of nodes and/or have manual configurations to apply to the elasticsearch.yml (see next section). On the first Elasticsearch node, run the configuration script provided in /usr/share/caringo-elasticsearch-search/bin/. This script prompts you for the needed values as it goes: | Code Block |
|---|
| /usr/share/caringo-elasticsearch-search/bin/configure_elasticsearch_with_swarm_search.py |
- The script will generate generates custom configuration files for each of the nodes in your Elasticsearch cluster. (v10.x)
- The current node's file is
/etc/elasticsearch/elasticsearch.yml. - The other nodes' files (if any) will beare
/etc/elasticsearch/elasticsearch.yml.<node-name-or-ip>
Follow the Customization details (below) to update the YAML files further, such as to change Elasticsearch's path.data (data directory). | Info |
|---|
| - Update log files to match your data path or other customizations.
- Update the rollingfile appender to delete rotated logs archives, to prevent running out of space.
|
- For the next and all remaining nodes, complete these steps:
- On the next Elasticsearch node, copy over the appropriate file as
/tmp/elasticsearch.yml.esnode8. With the YAML file in place, run the configuration script with the -c argument, so that it uses the existing file. | Code Block |
|---|
| configure_elasticsearch_with_swarm_search.py -c \
/tmp/elasticsearch.yml.esnode8 |
- Go to the next node, if any.
- Resume your installation to turn on the service: Installing Elasticsearch or Migrating from Older Elasticsearch
|
|---|
Non-Bulk
Usage | On the first Elasticsearch node, run the configuration script provided in /usr/share/caringo-elasticsearch-search/bin/. This script prompts you for the needed values as it goes: | Code Block |
|---|
| configure_elasticsearch_with_swarm_search.py |
- The script will generate generates a custom
/etc/elasticsearch/elasticsearch.yml configuration file for the current node as well as ones for each of the nodes, which you can ignore. (v10.x) Following the Customization details (below) if you need to update the YAML file further, such as to change Elasticsearch's path.data (data directory). | Info |
|---|
| - Update log files to match your data path or other customizations.
- Update the rollingfile appender to delete rotated logs archives, to prevent running out of space.
|
- Run the script the same way on each remaining ES node, answering the prompts consistently and reapplying any manual configurations.
- Resume your installation to turn on the service: Installing Elasticsearch or Migrating from Older Elasticsearch
|
|---|
...
| Info |
|---|
|
- Errors in adding and completing these settings can prevent the Elasticsearch service from working properly.
- If you customize Elasticsearch's
path.data location from the default, you must adjust all references to it below to reflect the new location.
|
Elasticsearch Config File
Edit the Elasticsearch config file: /etc/elasticsearch/elasticsearch.yml
action.auto_create_index: "+csmeter*,+*_nfsconnector,.watches,
.triggered_watches,.watcher-history-*" | Needed to disable automatic index creation, csmeter indices, and Swarm NFS connectors. (v10.1) |
| cluster.name: <ES_cluster_name> | Give your Elasticsearch cluster a unique name, which is unrelated to your Swarm cluster name. Do not use periods in the name. | Info |
|---|
| To prevent merging, it must differ from the cluster.name of your legacy ES cluster, if you have one operating. |
|
| node.name: <ES_node_name> | Optional. Elasticsearch will supply supplies a node name if you do not set one. Do not use periods in the name. |
| network.host: _site_ | Assign a specific hostname or IP address, which requires clients to access the ES server using that address. If you use a hostname, update /etc/hosts. Defaults to the special value, _site_. |
| cluster.initial_master_nodes | (ES 7+) For first-time bootstrapping of a production ES cluster. Set to an array or comma-delimited list of the hostnames of the master-eligible ES nodes whose votes should be counted in the very first election. |
discovery.zen.
minimum_master_nodes: 3 | (ES 6 only) Set to (number of master-eligible nodes / 2, rounded down) + 1. Prevents split-brain scenarios by setting the minimum number of ES nodes that must be online before deciding on electing a new master. |
| discovery.seed_hosts | (ES 7+) Enables auto-clustering of ES nodes across hosts. Set to an array or comma-delimited list of the addresses of all the master-eligible nodes in the cluster. |
| discovery.zen.ping.unicast.hosts: ["es0", "es1"] | (ES 6 only) Set to the list of node names/IPs in your cluster, making sure to include all of your ES servers. By default, multicast is disabled. |
| gateway.expected_nodes: 4 | Add and set to the number of nodes in your ES cluster. Recovery of local shards will start starts as soon as this number of nodes have joined the cluster. It falls back to the recover_after_nodes value after 5 minutes. This example is for a 4-node cluster. |
| gateway.recover_after_nodes: 2 | Set to the minimum number of ES nodes that must be started before going into operation status, computed as such: - If total nodes is 1 or 2, set to 1.
- If total nodes is 3 or 4, set to 2.
- If total nodes is 5 to 7, set to your number – 2.
- If total nodes 8 or more, set to your number – 3.
|
| bootstrap.memory_lock: true | Set to lock the memory on startup to ensure that Elasticsearch never swaps (swapping makes it perform poorly). Ensure that enough system memory resources are available for all processes running on the server. To allow the elasticsearch user to disable swapping and to increase the number of open file descriptors, the RPM installer makes these edits to/etc/security/limits.d/10-caringo-elasticsearch.conf: | Code Block |
|---|
| # Custom for Caringo Swarm
elasticsearch soft nofile 65536
elasticsearch hard nofile 65536
elasticsearch soft nproc 4096
elasticsearch hard nproc 4096
# allow user 'elasticsearch' memlock
elasticsearch soft memlock unlimited
elasticsearch hard memlock unlimited |
|
| path.data: <path_to_data_directory> | By default, path.data goes to /var/lib/elasticsearch with the needed ownership. If you want to move the Elasticsearch data directory, choose a separate, dedicated partition of ample size, and be sure to make the elasticsearch user the owner of that directory: | Code Block |
|---|
| chown -R elasticsearch:elasticsearch <path_to_data_directory> |
|
| thread_pool.write.queue_size | The size of the queue used for bulk indexing. This variable was called threadpool.bulk.queue_size in earlier Elasticsearch versions. |
...
Create the override file:
| Code Block |
|---|
|
/etc/systemd/system/elasticsearch.service.d/override.conf |
Add this content:
| Code Block |
|---|
|
[Service]
LimitMEMLOCK=infinity |
Load the override file (otherwise, the setting will does not take effect until the next reboot):
| Code Block |
|---|
|
sudo systemctl daemon-reload |
...
By default, deprecation logging is enabled at the WARN level, the level at which all deprecation log messages will be are emitted. To avoid having large warning logs, change the log level to ERROR:
...
