Versions Compared

Old Version 26

changes.mady.by.user Bala Harish A(AC)

Saved on

compared with

New Version 27

changes.mady.by.user Bala Harish A(AC)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

This is the process for in-place upgrades of Elasticsearch (ES), using an existing Search feed and index data.

...

  1. Before upgrading, query the Elasticsearch cluster for the list of nodes.

    Code Block
    languagebash
    curl -i http://ELASTICSEARCH:9200/_cat/nodes

    Note which node is starred; that is the Elasticsearch master node, which is recommended to upgrade last to avoid problems electing a new one.

  2. Backup the existing elasticsearch.yml, so a record of any customizations made exists. Run /root/dist/indexer-grab.sh and gather stats for support.

  3. Start by installing the latest Swarm Search, which is the caringo-elasticsearch-search RPM.

    Code Block
    languagebash
    yum install caringo-elasticsearch-search-VERSION.noarch.rpm
    Info
    titleTip

    The error: "ES_PATH_CONF must be set to the configuration path chown: cannot access '/etc/elasticsearch/elasticsearch.keystore': No such file or directory" displays if Elasticsearch 7 RPM is inadvertently installed. Install caringo-elasticsearch-search-7.0.0 RPM to proceed.

  4. Run the script that installs and configures the upgrade.
    If the script detects that Elasticsearch is installed and configured, it runs with --upgrade instead of configuring a new cluster.

    Code Block
    languagebash
    /usr/share/caringo-elasticsearch-search/bin/configure_elasticsearch_with_swarm_search.py
Warning

The upgrade requires Internet access to download the Elasticsearch rpm. Place elasticsearch-7.17.*.rpm in the current directory if Internet access is unavailable.

  1. Compare the backup file to the newly created elasticsearch.yml and add back any customizations needed, such as node.attr.rack and systemctl restart elasticsearch if the configuration was modified.

Note

thread_pool.write.queue_size: 1000, xpack.ml.enabled, xpack.security.enabled, prometheus.indices, and bootstrap.memory_lock are no longer required.

...

The script updates the configuration files and restarts the service if Elasticsearch 7 is already installed.

Info

Troubleshooting

Change permissions if the Elasticsearch service fails and journalctl -u elasticsearch shows access is denied (BootstrapException/AccessDeniedException): 

chown elasticsearch /etc/elasticsearch

Info

Important

Type Ctrl-C once , when the upgrade script is stuck in retrying status checks and proceed to the next node after the script finishes if the cluster loses master during the upgrade process and does not recover. Review /etc/elasticsearch/elasticsearch.yml and tail -F /var/log/elasticsearch/<cluster-name>.log for configuration errors.

The nodes elect the master and recover once all nodes have started on Elasticsearch 7.5.2. Health status goes yellow, then eventually green. Re-enable shard allocation, otherwise /_cat/health?v stops at 50% with health status yellow.

...

  1. It fixes /etc/sysconfig/elasticsearch to the correct ES6 version (the same as ES7).

  2. It increases the systemd timeout in /etc/systemd/system/elasticsearch.service.d/override.conf (see github.com/elastic/elasticsearch/issues/60140)

  3. A prompt to continue with the yum upgrade to 7.5.2 appears after refreshing the config files for Elasticsearch 6.

  4. It disables shard allocation and does a POST synced-flush for safer rolling upgrades. 

    Info
    titleImportant

    Disabling shard allocation or sync-flush can fail to contact the node, but do not proceed to upgrade the next node until the cluster health is green again.

  5. It uninstalls the Prometheus Exporter plugin if it exists.

  6. It shells out to yum to install the Elasticsearch 7 RPM in the current directory or from artifacts.elastic.co, if unavailable.

  7. It updates elasticsearch.yml for version 7 compatibility, including discovery.initial_master_nodes instead of discovery.zen.unicast.hosts, and jvm.options.

  8. It starts the upgraded Elasticsearch 7 and waits for it to be ready.

  9. The cluster re-enables shard allocation and prompts to repeat these two steps on the next node if the cluster health is green or yellow.