QUESTION: I am adding a new server in order to gain room for new writes as my cluster is almost filled with existing data, and perhaps even under-replicated. I don't want my new server and disks to be immediately overwhelmed by replicas for existing streams as I need room to write new streams.
ANSWER: bidding.relocationThreshold and health.relocationVolumeFillRate
The value of this first parameter is an integer between 0 and 100, representing the allowed difference between volume percent utilizations. Lower values ensure better load balancing and higher throughput. Higher values minimize data movement at the expense of lower maximum throughput."
Basically, the default is 5%, so if the existing volumes are 97% full and you add a new empty server with new disks, Swarm will fill the new ones to 92% full with replicas of existing objects.
We also suggest changing the hpRelocationVolumeFillRate to 1. This parameter is the percentage of available space on new volumes that may be filled for stream relocation during one cluster HP cycle.
To set these easily, you can use the snmp-castor-tool.sh from the support tools bundle like this:
swarmctl -C bidding.relocationThreshold -V 95 -d [Swarm Node IP] -p [ADMINUSER:ADMINPASS]swarmctl -C health.relocationVolumeFillRate -V 1 -d [Swarm Node IP] -p [ADMINUSER:ADMINPASS]
Using snmp-castor-tool.sh (the older method):
snmp-castor-tool.sh -C relocationThreshold -V 95
snmp-castor-tool.sh -C hpRelocationVolumeFillRate -V 1