Settings Reference

Following are the published settings allowing configuration of how storage cluster nodes operate. They are sorted alphabetically by section, and some do not appear in the provided node.cfg.sample configuration file. Use the Swarm UI to change these settings.

Dynamic Settings and SNMP: Many cluster settings (and a few node settings) are dynamic, accepting runtime changes without any hardware restarting. Dynamic settings are those with SNMP names defined below. Settings that cannot be changed dynamically are flagged with a restart icon in the Swarm UI, and settings that cannot be changed dynamically are flagged with a restart icon: The changes are applied and take effect after the next restart. See Persisted Settings (SNMP) to change settings by SNMP.

When a setting is not dynamic, it is because changing it on a running cluster can destabilize the cluster; examples are cluster.name and cluster.enforceTenancy (see How enforceTenancy Works).

Node (Chassis) Settings: Each physical or virtual machine is one Swarm node. The local config file is read and the node inherits the settings from that file on reboot. Most of the node (chassis) settings (those without an SNMP Name) require a reboot of the affected machine (chassis) to take effect; the Swarm UI flags such settings with a restart icon.

Tip

The special value -1 is often used to direct Swarm to take the value of another setting. Follow the guidance in the setting description to use it correctly in each context.

Cluster Settings

Name SNMP

Default

Description and
Examples

Name SNMP

Default

Description and
Examples

bidding.relocationThreshold
SNMP: relocationThreshold

5

Percentage, 0-100. How much difference between volume utilizations will cause a lower bid on another node to relocate or rebalance a replica to the other node. Lower values improve load balancing and throughput. Higher values minimize data movement at the expense of lower maximum throughput.

cip.group
SNMP: group

224.0.10.100

The multicast IP address for the cluster, as a Class D IP address in the 224.0.0.0 - 239.255.255.255 range. This address must be unique for each cluster. When configuring multiple, distinct clusters, take care that the multicast groups do not overlap, as any node with the same multicast group will become part of a single cluster. Set to blank to not use multicast. If this is done, the cluster.name must be unique within your network.
Examples:
224.5.5.7
239.255.255.253

cip.queryRetryMultiplier
SNMP: queryRetryMultiplier

1

What multiple of time to wait on each successive UDP multicast read retry.

cip.ttl

1

Controls configuration of multicast network traffic TTL (time to live). When set to 1, the multicast traffic should remain on the subnet.

cluster.enforceTenancy
SNMP: enforceTenancy

false

Setting to True (recommended) ensures that all content is written into a domain named in the request or else into the default domain. Setting to False (default) allows backward compatibility for applications in use before Swarm 5.0 that access data outside of domains and is required when using Gateway in legacy only mode to access this kind of content. Set to True for new deployments.

cluster.name
SNMP: cluster

 

The name of the cluster. Use an IANA-compatible domain name, such as cluster.example.com, and create one domain with the same name as the cluster, which sets up a default cluster domain that holds all unnamed objects. Do not use spaces in the name. To prevent confusion, configure all nodes in the cluster with the same cluster name.
Example:
swarm1.yourcompany.com

cluster.proxyIPAddress

 

[deprecated] The reverse proxy IP address for the cluster. Use cluster.proxyIPList instead.
Example:
129.3.7.14

cluster.proxyIPList
SNMP: clusterProxyIpList

 

For use with bidirectional GET replication only, to configure proxies on the source side for the target nodes to connect to. A comma-separated list of reverse proxy IP addresses or names, including ports in name:port format.
Example:
129.3.7.14:80, 129.3.7.15:80

cluster.proxyPort

80

[deprecated] The reverse proxy access port for the cluster. Use cluster.proxyIPList instead.

console.expiryErrInterval

10

Number of days before the cluster license expires to generate an error as a log message and a console indicator.

console.expiryWarnInterval

30

Number of days before the cluster license expires to generate a warning as a log message and a console indicator.

console.indexErrorLevel

90

Percentage, 0-100. How much index utilization will generate an error as a log message and a console indicator.

console.indexWarningLevel

80

Percentage, 0-100. How much index utilization will generate a warning as a log message and a console indicator.

console.messageExpirationSeconds
SNMP: messageExpirationSeconds

1209600

In seconds; defaults to 2 weeks. How long until an error expires out of the error table.

console.port

90

Which port Swarm uses to listen for requests. All nodes in the same cluster must be set to the same port. When deploying Swarm into untrusted network environments, firewall this port so that only administrators can access it.

console.reportStyleUrl

 

The URL for the path to the stylesheet and image files for configuring Swarm console.
Example:
http://10.10.15.32/css/swarm-reports.css

console.spaceErrorLevel

10

Percentage, 0-100. How much cluster capacity remaining will generate an error as a log message and a console indicator.

console.spaceWarnLevel

25

Percentage, 0-100. How much cluster capacity remaining will generate a warning as a log message and a console indicator.

console.styleUrl

 

The URL for the path to the stylesheet and image files for configuring the Swarm console.
Example:
http://10.10.15.32/css/swarm.css

disk.atimeEnabled
SNMP: accessedTimeEnabled

false

Whether to track the time of last access on GET requests, stored in the Castor-System-Accessed header and indexed as the search field 'accessed'. Increases load on the cluster and Elasticsearch.

disk.atimeGranularity
SNMP: accessedTimeGranularity

86400

In seconds; defaults to 1 day. The window during which accessed time will not be updated. Lowering the value affects GET performance.

disk.contextDeleteMarkerLifespan

31536000

In seconds; defaults to 1 year. How long a delete marker lives for a context (domain or bucket) object.

disk.deleteMarkerLifespan

1209600

In seconds; defaults to 2 weeks. How long the cluster remembers a deleted named object.

disk.obsoleteTimeout

7776000

In seconds; defaults to 3 months. The amount of time after which an unused volume is considered "stale" and will not recover, except with use of the 'k' modifier.

ec.conversionPercentage
SNMP: ecConversionPercentage

0

Percentage, 1-100; 0 stops all conversion. Adjusts the rate at which the Health Processor consolidates multi-set erasure-coded objects each HP cycle. Lower to reduce cluster load; increase to convert a large number of eligible objects faster, at the cost of load on the cluster. Requires policy.eCEncoding to be specified.

ec.convertToPolicy
SNMP: ecConvertToPolicy

false

When true, convert existing EC objects to whole replicas or to the EC encoding as specified by policy.

ec.convertVersionedObjects
SNMP: ecConvertVersionedObjects

false

When true, Swarm performs lifepoint conversions and consolidations of multi-set erasure-coded versioned objects.

ec.maxManifests
SNMP: ecMaxManifests

6

Range, 2-36. The maximum number of manifests written for an EC object. Usually p+1 are written for a k:p encoding. Do not set above 6 unless directed by Support.

ec.minParity
SNMP: ecMinParity

-1

Range -1 or 1-4; default of -1 is max(policyminreps - 1, 1), where policyminreps is the min value in policy.replicas. The minimum number of parity segments the cluster requires. This is the lower limit on p for EC content protection, regardless of the parity value expressed on individual objects through query arguments or lifepoints.

ec.protectionLevel
SNMP: ecProtectionLevel

node

Either 'node', 'subcluster', 'device', or 'volume'. At what level segments must be distributed for an EC write to succeed; note that multiple segments are allowed per level, if needed. 'node' (default) distributes segments across the cluster's physical/virtual machines. 'subcluster' requires node.subcluster to be defined across sets of nodes. You must have (k+p)/p nodes/subclusters for those levels; at minimum, you must have k+p volumes/devices.

ec.s3Mode
SNMP: ecS3Mode

true

Defaults to True. When set, assume EC writes are mostly through S3, enabling some space optimiations.

ec.segmentConsolidationFrequency
SNMP: ecSegmentConsolidationFrequency

10

Percentage, 1-100, 0 to disable. How quickly the health processor consolidates object segments after ingest. Increase this value (such as to 25, to consolidate over 4 HP cycles) to make new content readable sooner by clients. For multipart uploads via S3 clients, 10 is recommended; for SwarmFS, 100 is recommended, with extra space allowances for trapped space. Consolidation changes the ETag (which affects If-Match requests) and Castor-System-Version headers, but Content-MD5 and Castor-System-CompositeMD5 headers are unchanged. Therefore, have clients use hash and last-modified date, rather than ETag, to find if an object has changed.

ec.segmentSize
SNMP: ecSegmentSize

-1

In bytes; default of -1 implies 200 MB, with recommended minimum of 100 MB. The maximum size allowed for an EC segment before triggering another level of erasure coding. For mostly large (1+ GB) objects, increase to minimize the number of EC sets, which reduces index memory usage. Alternatively, increase the size as needed per write request using the 'segmentsize' query arg.

feeds.pauseDisconnectPerHourLimit
SNMP: feedsPauseDisconnectPerHourLimit

1000

Pause a feed if the number of disconnections per hour exceeds this value. 0 disables.

feeds.retry
SNMP: feedsRetryDelays

[30, 300, 1200]

In seconds. The progressive number of retry attempts by the plug-in, when blocked.
Example:
[60, 60, 60, 3600]

feeds.statsReportInterval

300

In seconds. How frequently to report statistics.

health.defragInterval
SNMP: healthDefragInterval

3600

In seconds; defaults to 1 hour. How long to wait between attempts to defrag a volume during an HP cycle.

health.ecrSegmentDelay
SNMP: healthECRSegmentDelay

0.0

In seconds; defaults to 0.0. Tunes ECRs by defining the length of the forced delay after each segment is relocated. Change from default only as directed.

health.examDelay
SNMP: healthExamDelay

0.19

In seconds; defaults to 0.19. Tunes the health processor by defining the length of the forced delay until the next HP exam, or removes the delay altogether (-1). Change from default only as directed.

health.fvrPushDelay
SNMP: healthFVRPushDelay

0.7

In seconds; defaults to 0.7. Tunes FVRs by defining the length of the forced delay after each replica/bundle is pushed to another node. Change from default only as directed.

health.neonatalROWProtection

true

If the exam queue for newly written objects is close to overflow, enables Swarm to override the data protection scheme of transitioning to ROW (scsp.replicateOnWrite). All subsequent replicas are processed out of this queue.

health.offloadPauseInterval
SNMP: healthOffloadPauseInterval

600

The delay between attempts to bulk offload to the cluster, in seconds.

health.parallelWriteTimeout
SNMP: healthParallelWriteTimeout

2592000

In seconds; defaults to 1 month. When to time out an uncompleted multipart upload so that Swarm can clean up the unused parts. 0 disables; do not disable if using SwarmFS.

health.persistentUnderreplicationAlertPercent
SNMP: healthPersistentUnderreplicationAlertPercent

2

Percentage, 0-100; set 0 to disable. Creates an alert when this percentage (or more) of objects are persistently under-replicated.

health.recursiveDeleteDelay

604800

In seconds; defaults to 1 week. The length of the grace period before the health processor begins reclaiming the space for a deleted domain or bucket. During this grace period, you can restore the domain or bucket without losing any of its content. No grace period is granted if you use recursive=now.

health.relocationVolumeFillRate
SNMP: hpRelocationVolumeFillRate

10

Percentage, 0-100. How much available space on new volumes may be filled for object relocation during one cluster health processor (HP) cycle, to prevent the HP on existing nodes from overwhelming a new, empty node.

health.replicationMulticastFrequency
SNMP: repMulticastFrequency

1

Percentage, 0-100. The frequency, as an approximate percentage, that UUIDs are multicast to verify replicas. Set this parameter to the same value for all nodes in the cluster.

health.replicationUnicastFrequency
SNMP: repUnicastFrequency

100

Percentage, 0-100. The frequency, as an approximate percentage, that a unit is forced to verify hints.

health.underreplicationAlertPercent
SNMP: healthUnderreplicationAlertPercent

10

Percentage, 0-100; set 0 to disable. Generates an under-replication alert when the percentage of under-replicated objects exceeds this value.

health.underreplicationTolerance
SNMP: healthUnderreplicationTolerance

100

Count. The number of under-replicated objects below which to suppress the alerts triggered by health.underreplicationAlertPercent.

index.optimize404
SNMP: overlayOptimize404

true

Enables the Optimize 404 feature in the overlay index, which returns 404 without multicast where possible.

index.ovMinNodes
SNMP: overlayMinNodes

3

Count. The minimum number of cluster nodes needed to activate use of the overlay index.

index.overlayEnabled
SNMP: overlayIndexEnabled

true

Enables the overlay index.

log.host
SNMP: logHost

 

The IP address of the remote Syslog server. Logging must be used for production environments. Set to '' to stop logging in test environments.
Example:
10.10.33.12

log.level
SNMP: logLevel

30

The log level, from most to least verbose, each including everything below it: 10, 20, 30, 40, 50, 0. 10 Debug (all information plus stack traces), 15 Audit (replication and object movement), 20 Info (informational, including non-errors), 30 Warn (user and application errors, plus SCSP 4xx/5xx codes), 40 Error (server hardware and software errors, plus abnormal conditions), 50 Critical (errors that can result in data loss, such as disk I/O errors), 0 Disable logging.

log.obscureUUIDs
SNMP: logObscureUUIDs

false

Whether to obscure UUIDs from displaying in INFO and higher level logs (does not affect AUDIT and lower levels). Set to True to abbreviate the UUID, if indicated by your security requirements.

log.port
SNMP: logPort

514

The port for the remote syslog host to use.

metrics.diskUtilizationCheck