Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Table of Contents
minLevel1
maxLevel2
outlinefalse
typelist
printablefalse

SNMP Commands

Storage cluster nodes are controlled through the SNMP action commands. The following OIDs enable you to disable allow disabling nodes and volumes with nodes from a storage cluster:

  • castorShutdownAction. : Disable nodes and volumes within nodes for servicing.

  • castorRetireAction. : Disable nodes and volumes within nodes for retirement.

Shutdown Action for Nodes

Info

Required

If you permanently remove a storage node's IP address from a storage cluster, you must also remove the reference to the storage node from the hosts parameter in /etc/caringo/cloudgateway/gateway.cfg on every Content Gateway. Once the storage node's IP address has been removed, restart the Content Gateway service to enable the change:

systemctl restart cloudgateway

To gracefully shut down a Swarm node, the string shutdown is written to the castorShutdownAction OID. Similarly, writing Writing the string reboot to this OID causes a Swarm node to reboot.

When a node receives a shutdown or reboot action, it A node initiates a graceful stop by unmounting all of its volumes and removing itself from the cluster . For when it receives a shutdown , the or reboot action. The node is powered off if the hardware supports this action . For a reboot, the node will reboot to machinefor a shutdown. The node reboots, re-read reads the node or cluster configuration files, and startup starts up Swarm for a reboot.

A graceful shutdown is required to perform a quick reboot. Performing an ungraceful shutdown forces the node to perform consistency checks on all its volumes before it can rejoin rejoining the cluster.

Info

Tip

Before shutting down or rebooting a node, check the node status page or the SNMP castorErrTable OID for critical error messages. Any logged critical messages will be are cleared upon reboot.

Info

Note

...

Wait at least 10 seconds in between each node reboot if rebooting more than one node at a time, but not the whole cluster

...

. This pause

...

verifies that each node can communicate

...

the rebooting state to the rest of the cluster, so

...

other nodes do not initiate recovery for the rebooting node.

Retire Action for Nodes and Volumes

Info

Required

If you permanently remove a storage node's IP address from a storage cluster, you must also remove the reference to the storage node from the hosts parameter in /etc/caringo/cloudgateway/gateway.cfg on every Content Gateway. Once the storage node's IP address has been removed, restart the Content Gateway service to enable the change:

systemctl restart cloudgateway

The Retire action is used to permanently remove a node or a volume within a node from the cluster. This action is intended for retiring legacy hardware or pre-emptively preemptively pushing content away from a volume with a history of I/O errors. Retired volumes and nodes are visible in the Swarm Admin Console until the cluster is rebooted.

Seehttps://perifery.atlassian.net/wiki/spaces/public/pages/2443811993/Retiring+Hardware#Retire-Rate and https://perifery.atlassian.net/wiki/spaces/public/pages/2443811993/Retiring+Hardware#Retiring-Volumes.

...

Note

The Retire action may take an extended amount of time to complete and requires at least three health processor cycles.

Single

...

Volumes

All stored objects are moved to other nodes in the storage cluster . After you initiate when a volume retirement, the is retired. The volume becomes a read-only volume and no additional objects can be stored on it . After all of the after initiating a volume retirement. The volume is idled with no further read/write requests after all objects are moved to other locations in the cluster, the volume is idled with no further read/write requests.

Each volume is given a unique name within its the node – the device string from the vols line in the configuration file. To retire a volume, its the name is written as a string to the castorRetireAction OID. The volume retirement process is initiated immediately upon receipt and the action cannot be aborted after it starts.

To manually retire a volume,

  1. Open the Swarm UI (or legacy Admin Console).

  2. Click the targeted chassis/node (IP address).

  3. For the targeted disk/volume, select Retire.

Entire

...

Node

Retiring a node means all volumes on the node are retired at the same time. After all volumes in the node are retired and the node data is copied elsewhere in the cluster, the node is permanently out of service and will does not respond to further requests.

To retire a node and all of its volumes, the all string is written to the castorRetireAction OID. The node retirement process is initiated immediately upon receipt and the action cannot be aborted after it starts.

Infonote

Warning

Ensure that Verify the cluster has enough free space and nodes to store the objects from the retiring volume. For subclusters, this applies to the subcluster where the retiring volume resides. If The retiring node cannot complete the retirement process until adding additional nodes if the number of nodes in the cluster or subcluster do not have enough space to store at least two replicas of all objects, the retiring node cannot complete the retirement process until you add additional nodes. .

The Retire action does not require that the configured default replicas (policy.replicas default) is are maintained to complete retirement. If Messages are logged indicating sufficient replicas cannot be created if there are not enough nodes to maintain the minimum number of replicas, messages will be logged that sufficient replicas can not be created.