Changes for Storage 9.0

New Features

• New Storage Management UI - The Storage UI (website) brings all new dynamic visualization to Swarm's storage configuration management and monitoring. The Dashboard communicates overall status (such as cluster health, summary, usage, overlay index, and data protection), feed statuses and recoveries, and trends in usage, bandwidth, and memory. The Cluster view allows you to manage your hardware, subclusters, and data feeds, and the Reports view offers details about cluster health and history. The Settings view lists and explains your current cluster setting values, and it allows you to manage your Swarm license. The functionality of the CSN Console (http://{csn-external-ip}:8090) and the Swarm Admin Console (http://{cluster}:90) are in the process of being unified and replaced by the Storage UI; these interfaces remain supported and available.
  See https://perifery.atlassian.net/wiki/spaces/public/pages/2443813554. 

• Encryption at Rest - Swarm provides the ability to encrypt all user data on disk volumes. Swarm encrypts the data as it writes it to disk and decrypts it on access. Provided the correct authentication and permissions, there is no difference the way clients access encrypted versus unencrypted objects. Such encryption offers security for the data on removed and failed disks as well as the ability to eradicate all data in a cluster by purging the encryption key. You enable encryption through new  [disk] settings.
  See https://perifery.atlassian.net/wiki/spaces/public/pages/2443811847.

• Metadata Annotation  - In addition to updating object metadata directly (via COPY), you can now effectively append additional metadata to existing objects without altering the original. Immutable objects, including historical versions, can be extended with metadata, because each object's create date, original metadata, and version sequence remain undisturbed. Metadata annotation makes use of a new persisted header,  Castor-System-Decorates , which is the ETag of the original object being extended by the metadata object. Annotations provide a new method for finding and managing objects, such as storing S3 object-level ACLs for the Gateway to enforce.
  See https://perifery.atlassian.net/wiki/spaces/public/pages/2443822101.

• Usable Capacity Tracking - Swarm now tracks and reports storage use by object to support calculations of usable capacity. This approach sums logical objects (the unique content of uploaded and versioned files) rather than actual streams (the raw space consumed by all Swarm components, including replicas, EC segments, and manifests). Swarm aggregates statistics from each volume and publishes them via SNMP and REST as logicalObjects, logicalSpace, and logicalUnprocessed.
  See .

• Elasticsearch 2 - Swarm now uses Elasticsearch 2 (2.3.3), which includes advances in speed, security, scalability, and hardware efficiency and supports compatibility with Linux standards and newer tool releases. Elasticsearch 2 is a fundamental leap from Elasticsearch 1. Upgrading is actually a migration. The easiest path is to provision and install a new Elasticsearch cluster and allow it build the new data on a new feed, in parallel with the old ES cluster. Swarm 9 continues to support the old primary feed while it builds the index data for the new feed, so searching is uninterrupted. Once the new feed is built, you make it the primary and delete the old one. At that point, you may upgrade to Content Gateway, which requires ES 2.3.3.

Upgrade Impacts

These items are changes to the product function that may require operational or development changes for integrated applications.

Additional Changes

These items are other changes and improvements including those that come from testing and user feedback.

• OSS Updates: The Linux kernel is updated to 4.4.14. Firmware updates now include bnx2x (7.12.30.0) and hpsa (3.4.14-0). (SWAR-7049)

• Improved: The Storage Management UI includes localization into French and Spanish. The popover and chart labels follow your default browser language setting, but they are translated on refresh. (UIS-418)

• Improved: Contexts (domains and buckets) can no longer be deleted by lifepoint directives, and any delete lifepoints written on these objects are ignored. Use an SCSP recursive delete to remove domains and buckets. (SWAR-6626)

• Improved: The procedure for updating the administrator password has been streamlined. See . (SWAR-7058)

• Improved: Swarm processes the backlogs for multiple feeds more equitably. (SWAR-7175)

• Improved: Swarm log messages include the chassis ID, making it possible to determine which nodes reside on the same chassis when examining logs. (SWAR-6999)

• Improved: Parallel writes now complete faster, and the response uses chunked transfer encoding to guarantee the client socket remains open in the event of a lengthy complete. The initial response may now be a 202 (Accepted), rather than 201 (Created). For a 202, the completion status is returned in both the body and in the trailing header 'castor-system-result.' For an error, the error message is returned in both the body and the trailing header 'castor-system-error.' (SWAR-7073)

• Fixed: Any remote replication feed defined in Swarm 8.0 did not propagate existing historical versions of versioned objects to the remote target. (SWAR-6840)

• Fixed: On cluster reboot, retired and offline drives did not appear in the Swarm Admin Console. (SWAR-7129)

• Removed: The setting disk.automaticFormat is no longer available. (SWAR-6498)

• Fixed an issue that prevented volume recovery suspension in the web console. (9.0.1: SWAR-7284)

• Fixed an issue where FVR/ECR was not kicked off properly in all circumstances. (9.0.1: SWAR-7287)

• Fixed an issue where nearly full clusters overfill some volumes, leading to 503 write responses. (9.0.2: SWAR-7321)