Changes

• Reduced timeout for Gateway's caches to acquire a connection to Swarm Storage nodes. This avoids incoming connections triggering file handle limits with threads stuck in ScspObjectCache, requiring a "systemctl cloudgateway restart". (CLOUD-3853)

• Gateway 8.1.0 is required when using a search feed created with Swarm 16.1.2 search.perDomainIndex=True. No gateway.cfg changes are needed. Please remember to restart Gateway whenever the default search feed is changed. (CLOUD-3988)

• [Internal] Retention lock updates now use synchronous indexing to avoid potential consistency issues. This can be disabled by setting [s3] enhancedListingConsistency = False, though it disables all synchronous indexing. (CLOUD-3942)

• Added a policy action ListAllMyBuckets for AWS compatibility, to allow a listing of all the bucket names in a domain. Previously ListDomain permission was required but that can also allow for the listing of objects within buckets via SCSP. Enabling this action requires using the Content UI policy editor in "json" mode, at the tenant or domain level. (CLOUD-3526)

• Gateway S3 uses the default bucket retention duration as the minimum retention time. This will be relaxed in an upcoming release to allow an earlier time, matching AWS S3 behavior. (CLOUD-3800)

• Gateway 8.1.0 cloudgateway_audit.log now includes trailing fields on most requests that indicate the milliseconds spent in different stages. The "auth" shows the time spent in authentication and authorization. Listings show the index "refresh" time ("F" means failure), "nondelimited" query time, and for delimiter listings the "query" and "commonprefixes" query times. Writes and deletes show the time spent "indexing" into Elasticsearch. RSW requests show the "rswfeeds" status and "rswtime". (CLOUD-3902)

• Dependencies were updated to avoid all High severity security vulnerabilities. We recommend all Gateway 7 and 8.0 customers upgrade even if older versions of Swarm are still used. (CLOUD-3905)

• Better error handling around object retention updates. Upgrade to Swarm 16.1 if experiencing persistent 503 errors on some objects [ReaderNotFound7 ESR37]. [unfortunately this doesn't really help, veeam does not handle 503 SlowDowns well].

• Fixed rare "Index 0 out of bounds for length 0" error when using SAML and downloading an object. (CLOUD-3948)

• Increase [SwarmCluster] (?) indexerMaxConnections (default 30) and indexerMaxConnectionsPerRoute (default 10) to allow more open connections to Elasticsearch nodes for listing and metering queries. Set to -1 to not change the default. (CLOUD-3983)

• Respond with an error instead of empty listing results when indexerHosts [link Gateway Config] does not match the default Search Feed. (CLOUD-3219)

• Gateway 8.1.0 improves the audit logging of sub-requests of S3 DeleteObjects and CopyObject requests. The internal requests have the incoming request-id followed by -<count> or -copysource. (CLOUD-3951)

• Gateway 8.1.0 fixes the remaining Elasticsearch deprecation warnings triggered by some Portal metering queries:
  "date-interval-getter" - "[interval] on [date_histogram] is deprecated". (CLOUD-3909)

• Gateway 8.1.0 fixes Gateway 8.0 and later logs "S3ObjectRequestHandler: Unable to determine Veeam SOSAPI capacity.xml" with a NullPointerException if metering or quota is disabled. [A 200 with an XML response is returned but values are -1 which Veeam treats as unknown. (CLOUD-3941)

• An S3 HEAD request of an object could respond with a 500 Internal Error. It now responds with a 403 Forbidden like GET. (CLOUD-3987)

Upgrade Impacts

See Upgrading Gateway to upgrade from a version of Gateway 6 or 7. See Upgrading from Gateway 5.x, if migrating from Elasticsearch 2.3.3 and Gateway 5.

Starting with Gateway 7.8, Elasticsearch 6.8.6 is no longer supported. Remain on Gateway 7.7 until the rolling upgrade from Elasticsearch 6.8.6 to 7.5.2 is completed.

Address the upgrade impacts for this and each prior version since the currently running version:

Impacts for 8.0.4

• Version Requirements

  • Swarm Storage 14.1.0 or higher

  • Elasticsearch 7.5.2 or 7.17.9 (required with Swarm Storage 15.3 or higher)

  • Content UI 7.9.1

  • Storage UI 3.5.0

Impacts for 8.0.3

• Version Requirements

  • Swarm Storage 14.1.0 or higher

  • Elasticsearch 7.5.2 or 7.17.9 (required with Swarm Storage 15.3 or higher)

  • Content UI 7.9.1

  • Storage UI 3.5.0

Impacts for 8.0.2

• Version Requirements

  • Swarm Storage 14.1.0 or higher

  • Elasticsearch 7.5.2 or 7.17.9 (required with Swarm Storage 15.3 or higher)

  • Content UI 7.9.1

  • Storage UI 3.5.0

Impacts for 8.0.1

• Version Requirements

  • Swarm Storage 14.1.0 or higher

  • Elasticsearch 7.5.2 or 7.17.9 (required with Swarm Storage 15.3 or higher)

  • Content UI 7.9.1

  • Storage UI 3.5.0

See Content Gateway 7 Release Notes and Content Gateway 6 Release Notes for impacts from prior releases.

Watch Items and Issues

These are known operational limitations that exist for Gateway.

• When using the default RHEL/CentOS configuration of IPTABLES, traffic to the Gateway will be blocked unless action is taken to disable IPTABLES or to enable inbound traffic to the front-end protocol port(s).

• Gateway is not compatible with Linux PAM modules that depend on interactive validation operations such as OTP or biometric scanners.

• Gateway 8.0.4 must be restarted after creating a Search Feed to avoid the error: “ResourceUnavailableException: Application resource 'elasticsearch-storage_cluster' is unavailable”. This will be fixed in an upcoming release. (CLOUD-4003)

See Content Gateway 7 Release Notes and Content Gateway 6 Release Notes for known issues from prior releases that are still applicable, apart from those appearing above as fixed.