Setup Elasticsearch Cluster
Partition Size
The following is the set of requirements for partitioning specific to OS capability:
Device | Mount | Partition Type | File System Type | Size | Options |
---|---|---|---|---|---|
/dev/sda1 | /boot | GPT | ext4 | 10 GiB | Bootable |
/dev/sda2 | / | GPT | ext4 | 20 GiB | NA |
/dev/sda3 | /var/lib | GPT | ext4 | >=120GiB (expand to Fill Disk) | NA |
/dev/sda4 | /var/log | GPT | ext4 | 15 GiB | NA |
Important
It is recommended to run Elasticsearch on SSDs.
CentOS Setup
See here to set up time synchronization for RHEL/CentOS 7 or RHEL/Rocky Linux 8.
The below table is an example:
Minimal Install Mode | Hostname |
|
Private Network | IP Address |
|
Network Gateway | 172.16.33.1 | |
NTP |
|
Download the Latest Swarm Bundle Package
For OVF-based installation, see Swarm VMware Bundle Packages to download the latest Swarm VMWare bundle.
For RPM-based installations, see Swarm Bundles to download the latest Swarm bundle.
Install Elasticsearch
Refer to the following steps:
Update CentOS.
yum -y update
Install EPEL.
yum -y install epel-release
Install NTP Server.
yum -y install ntp systemctl enable ntpd hwclock --systohc systemctl restart ntpd
Unzip the Swarm bundle package to
/root/datacore
.Obtain the latest Elasticsearch RPM and Swarm Search RPM from the downloaded Swarm bundle package.
Install Swarm RPM public key included with the distribution bundle.
Install and configure the Elasticsearch components on each Elasticsearch node.
Configure a 3-Node Elasticsearch Cluster for Swarm
Verify the Data Folder for Swarm Elasticsearch
The default data folder for Swarm Elasticsearch on all nodes is
/var/lib/elasticsearch
. This folder must not be on the boot partition to verify Elasticsearch cannot fill the boot partition.Verify ownership of that folder. Use
ls -la /var/lib/elasticsearch
. It should beelasticsearch:elasticsearch
.Configure the firewall to allow Elasticsearch
vi /etc/firewalld/services/elasticsearch.xml
.Restart the firewall service:
Add Elasticsearch firewall rule permanently:
Reload the firewall service:
Configure Elasticsearch Cluster
The following information is about the example deployment used to configure a 3-node Elasticsearch cluster:
Role | Private Network | vCPU | RAM | Disk Size (GB) | Remark |
ES75-01 | 192.168.9.21 | 4 | 8 | 100 | Elasticsearch 7.5.2 Cluster (SSD) |
ES75-02 | 192.168.9.22 | 4 | 8 | 100 | Elasticsearch 7.5.2 Cluster (SSD) |
ES75-03 | 192.168.9.23 | 4 | 8 | 100 | Elasticsearch 7.5.2 Cluster (SSD) |
Run the Swarm Elasticsearch configuration script
/usr/share/caringo-elasticsearch-search/bin/configure_elasticsearch_with_swarm_search.py
on the first Elasticsearch node (e.g., es75-01).Enter the Elasticsearch cluster name (e.g., swarm-es75).
Enter the list of all Elasticsearch server names or private network IP addresses.
Enter the name of the current Elasticsearch node.
The configuration script generates a custom Elasticsearch configure file for each node in the Elasticsearch cluster:
The other nodes’ files are
/etc/elasticsearch/elasticsearch.yml.<node-ip>
The current node’s file is
/etc/elasticsearch/elasticsearch.yml
Verify the Elasticsearch configuration files of all Elasticsearch nodes.
Copy the generated Elasticsearch configure file to other nodes.
SSH to other nodes of the Elasticsearch cluster to run the configuration script.
On ES75-02 node,
On ES75-03 node,
See Configuring Elasticsearch to update the environment and log settings. Perform a rolling restart of Elasticsearch services to apply changes once all updates are made.
See Hardware Requirements for Elasticsearch to know the hardware requirements for Elasticsearch.
Only for the New Setup of Elasticsearch Cluster
Refer to the below steps if Elasticsearch nodes are unable to join the cluster and return the ‘master not discovered exception’ error:
Stop Elasticsearch service on all nodes.
Delete
files/folders inside /var/lib/elasticsearch/
on all nodes.Restart Elasticsearch nodes one by one.
© DataCore Software Corporation. · https://www.datacore.com · All rights reserved.