The Swarm Cluster Services (SCS) server isolates Swarm within an internal network. The network is defined by both the IP address assigned to the interface on the SCS server connected to that network as well as the subnet defined for that network. The SCS server runs a DHCP service on the network for booting Swarm storage nodes up in the network.

The DHCP server allows the following IP addresses:

• Reserved Range: Two configurable reserved ranges in DHCP are at the upper and lower extremities of the subnet. These reserved ranges are special IP addresses for routers, network devices, etc., and are used for any servers that require direct participation in the Swarm cluster (Elasticsearch nodes, Content Gateway instances, etc.). None of these ranges has any default size., and at least one MUST be specified during DHCP initialization.
  One of the reserved ranges is set aside when the site needs a static IP allocation for Swarm nodes.

• Transient Pool: The dynamic addresses that are used during the early stages of booting Swarm nodes. This is configured as a percentage of the overall DHCP-managed range.

• Storage Pool: IP addresses that are used for Swarm storage nodes. If the number of storage nodes exceeds this pool, the transient pool is also used for the excess nodes. This is a remainder of the DHCP-managed range, which is not allocated to the transient pool.

The DCHP server utilizes the following breakdown of the subnet:

Subnet Layout:

  |                     |                    |                      |                     |
  | <- reserve lower -> | <- storage pool -> | <- transient pool -> | <- reserve upper -> |
  |                     |                    |                      |                     |
subnet                  | <--------- DHCP Managed Range ----------> |                 broadcast
address  

Range Sizing

For most installations, the DHCP-managed range (the “middle” of the subnet) spans the majority of the network’s address space, with small or nonexistent lower or upper ranges. However, at least one of the lower or upper ranges must have a non-zero number of IP addresses set aside for it.

SCS Internal Interface

The Swarm-facing network interface on the SCS server acts as a contact point for all SCS services for the cluster (logging, PXE booting, NTP, etc.) and as a default network gateway. Choose an appropriate static IP address cautiously for this interface. The IP address is often at the bottom of the lower range (but it is not required).

However, the IP address must be within either the upper or lower range but placing it outside of these ranges conflicts with the IP allocation.

Other Swarm Services

Gateway or Elasticsearch instances run for the Swarm cluster on the Swarm network. Their IP addresses are statically allocated in either the lower or upper reserved ranges.

Static IP Address Pool

If the Swarm deployment requires storage nodes to be given static IP addresses, then one of the reserved ranges is set aside for these IP addresses.

Other IP Addresses

Sometimes, the organizational policy represents that other services have access to the network while the Swarm network is private. The static IP addresses are provided to these services in either the lower or upper reserved ranges.

Swarm (Internal) Network MTU

If the MTU needs to be configured for the internal Swarm network, see https://perifery.atlassian.net/wiki/spaces/public/pages/1223491667/SCS+Administration#Swarm-(Internal)-Network-MTU

Examples

Following two scenarios of the Network Planning assume the default transient pool allocation is 50%. The pool size and the size of the upper and lower reserved ranges are configurable.

The below table represents the mapping of CSN concepts with SCS:

Small Network (/24)

Implications

• 222 Storage nodes (sum of transient and storage pool) on the network

• Up to 111 chassis can PXE boot simultaneously (determined by the transient pool size)

Large Network (/16)

Implications

• 65,000 Storage nodes (sum of transient pool and storage pool) on the network

• Up to 32,500 chassis can PXE boot simultaneously (determined by the transient pool size)

Podman Network Ranges

Podman sets up an internal network for containers to talk to each other. The range 10.88.0.0/16 is used internally for the podman network on SCS nodes and is reserved for container communication. If this IP range (or an overlapping IP address range) is used for normal network communications, conflicts occur and network traffic becomes unpredictable. Verify the IP range is not used outside of Swarm.