Table of Contents | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
|
Info |
---|
ImportantWhen using SCS 1.5.x or later with Swarm 15.3, the process for configuring static IP addresses is greatly simplified. Carefully select the correct set of instructions for your environment. |
...
SCS
...
1.6
Note |
---|
This section ONLY applies to Swarm 15.3 or later running on SCS 1.6 |
SCS will remember the IP addresses of storage nodes once they have been booted, as it did in SCS 1.5 (.1).
Note
If the SCS instance must be recreated in the absence of an up-to-date backup file, the cluster must be rebooted. The IP addresses of nodes may change.
Assigning Pre-Defined IP Addresses to Storage Nodes
In version 1.6, static IP address assignment is by default enabled. SCS supports the same automatic assignment of static IP addresses to nodes but adds the additional ability to manually assign static IP addresses.
New Installations
Follow the normal installation procedures for both SCS and the Swarm Storage bundle for SCS, including a run of scsctl diagnostics config scan_missing
at the end.
...
dhcpd.conf
only mentions a single pool of IP addressesscsctl storage config show -d "network.useStaticAddresses"
should show a value of true. This is required for both automatic and manual assignment of static IP addresses
Assigning Pre-Defined Static IP Addresses
Before assigning static IP addresses, you need to list the storage nodes and record their chassis IDs.
Info |
---|
ImportantIf the nodes have not been booted previously, boot the cluster in the dynamic IP assignment mode so the SCS has a record of the storage instances. Make a list of the instances once the entire cluster is up and running. |
...
scsctl storage config set -d -i {chassis ID} "network.ipV4Address={your.static.address.here}"
Info |
---|
RequiredIf the storage cluster was previously using static IP addresses defined in a previous version of SCS (1.5 or earlier), remove the customized node.cfg files for each storage node. Review any other customizations made that do not relate to manual IP address assignment.
|
...
scsctl storage instance list
SCS 1.5.1
Note |
---|
This section only applies to Swarm 15.3 or later running on SCS 1.5.1. For environments running earlier versions of either, see SCS 1.4 and Earlier. |
Static IP address assignment is by default enabled, beginning with SCS 1.5 and Swarm 15.3. However, the process to ensure correct behavior differs somewhat based on whether the environment is being upgraded or is a fresh installation.
Architecture
From 1.5 onward, SCS no longer uses a dual-pool system of address allocation. Older versions used a “transient” DHCP pool to assign addresses while nodes were first booting, with the expectation that the nodes would later acquire an address from the more permanent “storage” pool.
Now, SCS uses a single pool of addresses, bounded by the upper and lower reserved ranges. Once a node begins booting and reports its IP address to SCS, SCS will associate that IP address with the node’s chassis ID and embed the IP address within the node’s node.cfg configuration file. This ensures that the node ends up with the same IP address, regardless of how DHCP responds during the initial boot process on subsequent boots.
Note
If the chassis ID changes (due to a change in NICs on a booting node), then a new IP address may be assigned to the node.
Fresh Installations
The assignment of static IP addresses is by default enabled. To ensure this is the case, follow the normal installation procedures for both SCS and the Swarm Storage bundle for SCS, including a run of scsctl diagnostics config scan_missing
at the end and initializing DHCP (scsctl init dhcp ...
).
...
Once these have been confirmed, the configuration is complete for static IP address assignments.
Upgrading Environments
From Dynamic IP Assignments
If the current environment relies on dynamic IP addresses for storage nodes, then no extra steps are required.
Follow the normal upgrade procedures for both SCS and the Swarm Storage bundle for SCS. When upgrading SCS, be sure to run scsctl diagnostics upgrade_check -a
to ensure that all SCS-related software components are properly updated, verifying that all steps are completed successfully. Once all upgrade steps have been completed, including a run of scsctl diagnostics config scan_missing
at the end, re-initialize DHCP (scsctl init dhcp ...
) using the same upper and lower reserved ranges as in the current configuration.
Follow the verification steps mentioned in Fresh Installations. Once done, configuration is complete for static IP address assignments.
From Static IP Assignments
In order to set up static IP assignments in a pre-upgrade environment, several manual steps had to be taken:
...
Both of these should be undone when upgrading.
Remove Custom Templates
First, obtain a list of nodes that SCS is aware of:
...
Code Block |
---|
scsctl storage config file unset -d -i "{instance ID}" node.cfg |
DHCP Updates
...
Info |
---|
TipSee Network Planning documentation regarding SCS network planning |
...
. |
It is likely that in the initial setup of static IP address support, either: (1) an
An upper reserved range was defined
...
or
The lower reserved range was extended
...
Neither is needed, but it is not harmful to leave either in place other than a reduced number of IP addresses available for booting nodes. In most cases, this is not a significant impact.
If these addresses should be added back to the pool, record the desired lower and upper range sizes and save them for later in the upgrade process.
Upgrade
Remove any custom node.cfg templates, as described above.
Note the desired lower and upper reserved ranges for DHCP
Follow the normal upgrade procedures for both SCS and the Swarm Storage bundle for SCS.
When upgrading SCS, be sure to run
scsctl diagnostics upgrade_check -a
to ensure that all SCS-related software components are properly updated, verifying that all steps complete successfully.Run
scsctl diagnostics config scan_missing
at the end.
Using the saved values for lower and upper reserved ranges, initialize DHCP:
scsctl init dhcp --dhcp-reserve-lower {size_of_lower_range} --dhcp-reserve-upper {size_of_upper_range}
Follow the verification steps mentioned in Fresh Installations.
Once done, configuration is complete for static IP address assignments.
SCS 1.4 and Earlier
Info |
---|
This section ONLY only applies to environments that are running older versions of SCS (older than 1.5) or Swarm Storage (older than 15.3). If newer versions are being used, please see SCS 1.5 and Later. |
The Basics
To support static IP assignments, two main updates are needed: DHCP configuration and node.cfg for each booting node. The DHCP configuration updates are needed to ensure the static IP addresses used are not part of the DHCP pool. The changes to node.cfg are needed so the Swarm Storage nodes know what network information to use.
When making these changes, the DHCP updates should be made first, followed by the node.cfg changes. Once all the changes are complete, the Swarm Storage nodes can be (re-)booted to use the static IP assignments.
Updating DHCP Configuration
...
Info |
---|
TipSee Network Planning documentation regarding SCS network planning |
...
. |
SCS DHCP configuration has three major ranges that it manages:
...
The lower and upper reserved ranges are optional, though one of them MUST must be defined. In practice, this is usually the lower range, and it is used for things like the SCS interface itself, attached Gateway instances, etc. The upper range typically goes undefined and unused (a size of zero).
For static IP assignments, this upper reserved range is typically the part of the network carved out for statically-addressed nodes. Alternatively, the lower range may be extended to allow room for the static IP addresses. However this may cause IP address conflicts with existing address assignments unless all nodes are booted together.
Using the Upper Reserved Range
The upper reserved range is determined not by providing specific IP address boundaries, but by specifying the number of addresses within the subnet that should be reserved. One address per storage node is sufficient, though adding a buffer for future expansion may be desired.
...
This command updates the DHCP configuration and restarts the dhcpd
service.
Info |
---|
ImportantTo confirm the range is as expected, inspect the DHCP configuration at |
Extending the Lower Reserved Range
Generally a lower reserved range has already been defined for an SCS installation. The first step in extending it is to make note of the current size of the lower reserved range. Then determine the size of the extension. One address per storage node is sufficient, though adding a buffer for future expansion may be desired. The new size of the lower reserved range is the sum of the current size and the size of the extension.
...
Info |
---|
To confirm the range is as expected, inspect the DHCP configuration at |
Updating node.cfg Templates
SCS does not manage configuration files in the traditional sense. Rather, it uses templates that are rendered when needed, with values populated from settings. Typically the template for node.cfg is inherited from a generic template provided by the storage
service within SCS, but it may be overridden for the entire cluster or for individual nodes. This method relies on the latter.
Note |
---|
The base template is specific to the current active version of Swarm Storage. These steps need to be repeated in order to pick up essential changes within the node.cfg template if the version changes in the future. Failure to do so may prevent nodes from booting properly!. |
Download the Base Template
This is the file that needs to be customized for each node booting with a static IP address. To obtain the template:
...
Verify no extraneous line breaks are present in the template as this prevents nodes from booting properly. These can be introduced if a terminal session wraps the lines of text in the template. Example: Using :set list
in vi displays unprintable characters and line endings.
List the Nodes
Note
This process needs to be repeated for each new node if nodes are added in the future.
...
Make note of all of the instance IDs that are listed; they are needed for future commands.
Info |
---|
ImportantBoot a node using the standard DHCP mechanism if it does not appear in this list. Shut it down once it is online. Obtain a new instance list and the missing ID displays. |
Update Each Node
A custom node.cfg template needs to be created with the node’s networking information for each instance ID. That The updated template then needs to be set for that node.
Customize the Template
Create a copy of the base template specific to the node:
...
Code Block |
---|
[network] ipAddress = {static IP address} netmask = {Swarm internal network netmask} gateway = {internal IP address of SCS} |
Info |
---|
Under DHCP booting, the DHCP server provides the netmask and gateway along with the IP address. Since DHCP is not being used aside from PXE booting, these values must be explicitly defined. |
Upload the Template
Use the following command to tell SCS to use the customized template when booting:
Code Block |
---|
scsctl storage config file set -d -i {instance_id} -f "./node.cfg.{instance_id}" node.cfg |
Reboot the Cluster
Reboot the entire cluster once both DHCP and the node.cfg templates have been updated. The nodes use the static IP addresses assigned to them in the custom node.cfg templates once they are online.