Partition Size

The following is the set of requirements for partitioning specific to OS capability:

CentOS Setup

The below table is an example:

Download the Latest Swarm Bundle Package

For OVF-based installation, see Swarm VMware Bundle Packages to download the latest Swarm VMWare bundle.

For RPM-based installations, see Swarm Bundles to download the latest Swarm bundle.

Install Elasticsearch

Refer to the following steps:

1. Update CentOS.

   yum -y update

2. Install EPEL.

   yum -y install epel-release

3. Install NTP Server.

   yum -y install ntp systemctl enable ntpd hwclock --systohc systemctl restart ntpd

4. Unzip the Swarm bundle package to /root/datacore.

5. Obtain the latest Elasticsearch RPM and Swarm Search RPM from the downloaded Swarm bundle package.

   ll ~/datacore/Elasticsearch/  -rwxr-xr-x. 1 root root     38380 Jan  8 18:33 caringo-elasticsearch-search-<version>.noarch.rpm -rwxr-xr-x. 1 root root 290237676 Jan  8 18:33 elasticsearch-<version>-x86_64.rpm -rwxr-xr-x. 1 root root       345 Jan  8 18:33 README.txt -rwxr-xr-x. 1 root root      1927 Jan  8 18:33 RPM-GPG-KEY

6. Install Swarm RPM public key included with the distribution bundle.

   rpm --import RPM-GPG-KEY

7. Install and configure the Elasticsearch components on each Elasticsearch node.

   yum -y install elasticsearch-<version>-x86_64.rpm yum -y install caringo-elasticsearch-search-<version>.noarch.rpm

Configure a 3-Node Elasticsearch Cluster for Swarm

Verify the Data Folder for Swarm Elasticsearch

1. The default data folder for Swarm Elasticsearch on all nodes is /var/lib/elasticsearch. This folder must not be on the boot partition to verify Elasticsearch cannot fill the boot partition.

2. Verify ownership of that folder. Use ls -la /var/lib/elasticsearch. It should be elasticsearch:elasticsearch.

3. Configure the firewall to allow Elasticsearch vi /etc/firewalld/services/elasticsearch.xml.

   <?xml version="1.0" encoding="utf-8"?> <service>    <short>Elasticsearch</short>    <description>Elasticsearch server REST API, which is based on http traffic.</description>    <port protocol="tcp" port="9100"/>    <port protocol="tcp" port="9200"/>    <port protocol="tcp" port="9300"/>  </service>

4. Restart the firewall service:

   systemctl restart firewalld

5. Add Elasticsearch firewall rule permanently:

   firewall-cmd --permanent --add-service elasticsearch

6. Reload the firewall service:

   firewall-cmd --reload

Configure Elasticsearch Cluster

The following information is about the example deployment used to configure a 3-node Elasticsearch cluster:

1. Run the Swarm Elasticsearch configuration script /usr/share/caringo-elasticsearch-search/bin/configure_elasticsearch_with_swarm_search.py on the first Elasticsearch node (e.g., es75-01).

2. Enter the Elasticsearch cluster name (e.g., swarm-es75).

   Checking ES version... elasticsearch 7 detected =================  About to prompt for Elasticsearch config information, config=None  Prompting admin for configuration input  Enter Elasticsearch cluster name [A string]: swarm-es75

3. Enter the list of all Elasticsearch server names or private network IP addresses.

   Enter List of all the Elasticsearch server names in cluster [Comma-separated list of DNS-resolvable names or IP addresses]:192.168.9.21,192.168.9.22,192.168.9.23 

4. Enter the name of the current Elasticsearch node.

   Enter this Elasticsearch node's name [A string name from the list entered above]: 192.168.9.21 

   The configuration script generates a custom Elasticsearch configure file for each node in the Elasticsearch cluster:

   1. The other nodes’ files are /etc/elasticsearch/elasticsearch.yml.<node-ip>

   2. The current node’s file is /etc/elasticsearch/elasticsearch.yml

      ll /etc/elasticsearch/  -rw-rw----. 1 root elasticsearch   199 Jan 13 11:50 elasticsearch.keystore  -rwxr-xr--. 1 root elasticsearch  3549 Jan 13 11:57 elasticsearch.yml  -rwxr-xr--. 1 root root           3355 Jan 13 11:57 elasticsearch.yml.192.168.9.22  -rwxr-xr--. 1 root root           3355 Jan 13 11:57 elasticsearch.yml.192.168.9.23  -rw-rw----. 1 root elasticsearch  2847 Jan 15  2020 elasticsearch.yml-2022-01-13T11:57:17.bak  -rw-rw-r--. 1 root elasticsearch  2436 Jan 13 11:57 jvm.options  -rw-rw----. 1 root elasticsearch  2276 Jan 15  2020 jvm.options-2022-01-13T11:57:17.bak  -rw-rw----. 1 root elasticsearch 17545 Jan 15  2020 log4j2.properties  -rw-rw----. 1 root elasticsearch   473 Jan 15  2020 role_mapping.yml  -rw-rw----. 1 root elasticsearch   197 Jan 15  2020 roles.yml  -rw-rw----. 1 root elasticsearch     0 Jan 15  2020 users  -rw-rw----. 1 root elasticsearch     0 Jan 15  2020 users_roles

5. Verify the Elasticsearch configuration files of all Elasticsearch nodes.

   vi /etc/elasticsearch/elasticsearch.yml # ======================== Elasticsearch Configuration ========================= # Caringo (elasticsearch 7) render date: 2022-01-13T03:51:17Z # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: swarm-es75 # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # node.name: 192.168.9.21 # # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # path.data: /var/lib/elasticsearch # # Path to log files: # path.logs: /var/log/elasticsearch # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # bootstrap.memory_lock: true # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # #network.host: 0.0.0.0 network.host: 192.168.9.21 # # Set a custom port for HTTP: # #http.port: 9200 # # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when this node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # discovery.seed_hosts: ["192.168.9.21", "192.168.9.22", "192.168.9.23"] # # Bootstrap the cluster using an initial set of master-eligible nodes: # cluster.initial_master_nodes: ["192.168.9.21", "192.168.9.22", "192.168.9.23"] # # For more information, consult the discovery and cluster formation module documentation. # # By default nodes can be both master or data but if set will be preserved # node.master: true node.data: true # # This should be set to 1 in Production # node.max_local_storage_nodes: 1 # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # gateway.recover_after_nodes: 2 # gateway.expected_nodes: 3 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true # # Disable automatic index creation, except for csmeter indices, Swarm NFS connectors, and ES watcher feature. action.auto_create_index: "+csmeter*,+*_nfsconnector,.watches,.triggered_watches,.watcher-history-*"

6. Copy the generated Elasticsearch configure file to other nodes.

   scp /etc/elasticsearch/elasticsearch.yml.192.168.9.22 root@192.168.9.22:/etc/elasticsearch/. scp /etc/elasticsearch/elasticsearch.yml.192.168.9.23 root@192.168.9.23:/etc/elasticsearch/.

7. SSH to other nodes of the Elasticsearch cluster to run the configuration script.

   1. On ES75-02 node,

      ssh root@192.168.9.22 /usr/share/caringo-elasticsearch-search/bin/configure_elasticsearch_with_swarm_search.py -c \ /etc/elasticsearch/elasticsearch.yml.192.168.9.22

   2. On ES75-03 node,

      ssh root@192.168.9.23 /usr/share/caringo-elasticsearch-search/bin/configure_elasticsearch_with_swarm_search.py -c \ /etc/elasticsearch/elasticsearch.yml.192.168.9.23

See Configuring Elasticsearch to update the environment and log settings. Perform a rolling restart of Elasticsearch services to apply changes once all updates are made.
See Hardware Requirements for Elasticsearch to know the hardware requirements for Elasticsearch.