Changes

• Elasticsearch 6 Transition: Gateway 7.0 completes support for Elasticsearch 6. Migration from either Elasticsearch 2 or 5 should be performed at this time. Because the Elasticsearch 6 database is binary-compatible with Elasticsearch 7, upgrading without a migration is possible. Reindexing the cluster is required. (CLOUD-3194, CLOUD-3198)

• Swarm Folder Listing: With version 7.0, folder listing support across Swarm clients (such as SwarmFS and S3) has been both completely rearchitected and also newly centralized within Content Gateway. Folder listing allows Swarm clients to render virtual folders below the bucket level of Swarm Storage: it translates any delimited prefixes in Swarm object names (such as in FY2019/Q3/object.jpg) into client-side file system folders. These folders allow users to interact with objects in an intuitive hierarchical organization. (CLOUD-2975)

The new architecture brings many benefits across Swarm implementations:

• All legacy client-specific code is replaced with a central, unified, and improved approach. Centralization means future listing improvements are easier and faster to roll out.

• The pagination of large listing results is no longer bound to the Elasticsearch limit (index.max_result_window).

• The listing service is optimized for features new to Elasticsearch 6.

The scope of this release does not include unnamed objects, caching, folder locking/leasing, or client notification of namespace changes.

Upgrade Impacts

To upgrade from a version of Gateway 6, see Upgrading Gateway. See Upgrading from Gateway 5.x if migrating from Elasticsearch 2.3.3 and Gateway 5.

Address the upgrade impacts for this and each prior version since the currently running version:

See Content Gateway 6 Release Notes for impacts from prior releases.

Watch Items and Issues

These are known operational limitations and watch items existing for Gateway.

• Traffic to the Gateway is blocked unless action is taken to disable IPTABLES or to enable inbound traffic to the front-end protocol port(s) when using the default RHEL/CentOS configuration of IPTABLES, .

• Gateway is not compatible with Linux PAM modules depending upon interactive validation operations such as OTP or biometric scanners.

The following are known issues in this release.

• Recent rclone releases can make multiple PUT bucket requests fail (409 Conflict). Workaround: Add no_check_bucket=true to the rclone config, or use "rclone copy --s3-no-check-bucket ...". (CLOUD-3213)

• The AWS S3 SDK for C# does not properly sign S3-compatible requests with spaces in the name unless the domain contains ".s3." or ".s3-". See S3 request signing broken for S3-compatible services. (CLOUD-3068)

• The x-amz-storage-class header is not preserved when buckets are created. (CLOUD-3062)

• During new object creation as part of renaming with ?newname, Gateway does not verify the user has permission to create the new object name (although it is highly likely, because it is a write within the same context). (CLOUD-2966)

• An s3cmd or rclone server-side copy request may time out on a multipart copy for >5GB objects (s4cmd performs it correctly). Workaround: After verifying it is not the HTTPS proxy timing out, increase the client timeout: set s3mcd socket_timeout = 600 in ~/.s3cfg or use rclone copy --timeout=10m --contimeout=2m caringo:mybucket/5gb caringo:mybucket/subfolder/. (CLOUD-2949)

• Listings with max-keys may be shorter than expected because CommonPrefixes are included in the count of keys returned. (CLOUD-2917)

• Usernames are case-insensitive, but listings exclude a token if the username (myadmin) does not match the case used when the token was created (myAdmin). (CLOUD-2837)

• Multipart PUT requests via recent Cyberduck versions fail with 403 SignatureDoesNotMatch when using AWS Signature Version 4. Install the Caringo .cyberduckprofiles from Using the Cyberduck application with Content Gateway S3 which force V2 signatures. (CLOUD-2799)

• The policy fails to take effect without warning if a policy document includes a Principal with plural "users" or "groups" instead of "user" or "group". (CLOUD-2783)

• 403 S3 V4 Signature mismatch errors may result when using Cyberduck with the "pound" proxy in front of Gateway S3. Workaround: Disable the Expect header in the Cyberduck preferences, or (recommended) use a different proxy such as "haproxy". (CLOUD-2628)

• Errors may erroneously report being related to Storage nodes when Gateway cannot connect to Elasticsearch nodes. (CLOUD-2595)

• Because of issues with Range and ETag header handling, video playback of .mp4 streams may not work correctly when served via the Gateway S3 port. It does work when served via the Gateway SCSP port. (CLOUD-1964)

• Gateway caches the Swarm version from the "Server:" response header, so after upgrading Swarm restart Gateway to consistently see the new version. (CLOUD-1271)

• Gateway responds with a 500 (Internal Server Error) instead of 400 (Bad Request) if the size of the metadata headers sent to Swarm is too large. (CLOUD-800)

• The S3 bucket listing StorageClass response element always reports STANDARD. (CLOUD-766)

• The Gateway audit log escapes the "%" characters used by the client as escape characters if an S3 client escapes URI path characters such as "/". URI audit log processing for S3 clients requires double-unescaping when this occurs. (CLOUD-703)

See for known issues from prior releases that are still applicable, apart from those appearing above as Fixed.