Release Notes for Content Gateway 5.4

Owned by Rumena Zlatkova (Deactivated)
Last updated: Apr 18, 2022 by Kyle Miller (Deactivated)
7 min read

New Features

This release includes these improvements and fixes:

  • Docker or proxy use — Gateway can now be configured for use either within a Docker environment or behind a proxy. The Gateway configuration has two new settings available (externalHTTPPort, externalHTTPSPort) per protocol: [scsp] and [cluster_admin], the Service Proxy. They only take effect when X-Forwarded-Proto is found on the request; Gateway uses X-Forwarded-Proto to determine which port to use. (CLOUD-3055)

  • Metadata Translation — Gateway seamlessly moves custom metadata between SCSP and S3. It translates the metadata formatting between the protocols. It provides S3 and SCSP applications the ability to access each other's metadata. See Metadata Translation between SCSP and S3. (CLOUD-2919)

  • Quota stability — Quota implementation has been significantly hardened, resolving general problems with frozen usage status/data and excessive emails from state changes. (CLOUD-3045)

  • Fixed issues:

    • There are issues when enabling versioning or quotas through the Content UI. The fix prevents creation of duplicate (conflicting) domains in the storage cluster. (CLOUD-3019)

    • Updating certain Swarm configuration settings through the Gateway services proxy (port 91) sometimes required the sptid query argument value. (CLOUD-2999)

    • Gateway validates on startup that [storage_cluster] managementPassword is set when the ServiceProxy ([cluster_admin]) is enabled. (CLOUD-2617)

  • New in 5.4.1:

    • Content Gateway adds support so it can be used with Swarm UI 2.3. Note: 2.3 includes a new feed type, S3 Backup feed, which requires Swarm 11.

    • Several fixes are made related to S3 multipart uploads. (CLOUD-2994, CLOUD-3074)

Upgrade Impacts

  • Swarm Storage Requirements — Gateway 5.4 requires a minimum back-end storage cluster version of 9.6. The storage cluster must be upgraded prior to installing this version of Gateway.

    • Releases of Gateway 5.1.3 and earlier are not compatible with Storage versions 9.2 or higher. For historic compatibility details, refer to the releases included with those distributions.

  • RHEL/CentOS 6 Deprecation — Support for RHEL/CentOS 6 is removed in the near future. Completing the transition to RHEL/CentOS 7 now assures a smooth upgrade path to future Gateway versions.

  • Automatic service start must be re-enabled — After the upgrade, the service may not automatically start after a system reboot. To re-enable the service, run:
    "chkconfig --add cloudgateway" for RHEL/CentOS 6, or
    "systemctl enable cloudgateway" for RHEL/CentOS 7. (CLOUD-2819)

  • ExpanDrive — ExpanDrive users must upgrade to ExpanDrive 6.1.0 with this version of Gateway; earlier versions report 403 Signature errors when creating a folder or uploading a large object. (CLOUD-2746)

Watch Items and Known Issues

These are known operational limitations and watch items that exist for Gateway.

  • When using the default RHEL/CentOS configuration of IPTABLES, traffic to the Gateway is blocked unless action is taken to disable IPTABLES or to enable inbound traffic to the front-end protocol port(s).

  • Gateway is not compatible with the fingerprint scanner module for Linux PAM. If it is installed, remove it by running:

    yum remove fprintd-pam

  • SCSP reading operations that request a Content-MD5 hash validation and for which there is a hash mismatch causes a storage node to be temporarily removed for the Gateway's connection pool due to the way Swarm reports a hash validation failure.

  • Swarm Integrity Seal upgrades cannot be performed through Gateway. They may be done directly to the back-end Swarm cluster.

  • If the HTTP cache control headers If-Modified-Since and If-Unmodified-Since are used, review the discussion of these in the Storage SCSP Development.

The following are known issues in this release.

  • The gateway error log "Unable to create phone home data" at startup can be ignored. CLOUD 3051

  • The Gateway error "Failed reading from client" on a PUT due to "EofException: Early EOF" may occur when clients do not send the full body. This may point to a bug in the client's retry logic, such as not resetting the position marker to the beginning of the file or part. CLOUD-3010

  • During new object creation as part of renaming with ?newname, Gateway does not verify the user has permission to create the new object name (although it is highly likely, because it is a write within the same context). CLOUD-2966

  • An s3cmd or rclone server-side copy request may time out on a multipart copy for >5GB objects (s4cmd performs it correctly). Workaround: After you verify it is not the HTTPS proxy timing out, increase the client timeout: set s3mcd socket_timeout = 600 in ~/.s3cfg or use rclone copy --timeout=10m --contimeout=2m caringo:mybucket/5gb caringo:mybucket/subfolder/. CLOUD-2949

  • Listings with max-keys may be shorter than expected because CommonPrefixes are included in the count of keys returned. CLOUD-2917

  • Uploading files / photos using Panic's Transmit app on iOS fails due to a 403 Invalid Signature error. CLOUD-2886

  • Gateway 5.2.2 and earlier do not output the NextMarker field in S3 listings, which can cause some S3 clients such as Caringo Drive, rclone, and Transmit to show only 1000 files in a directory or to miss some subdirectories. CLOUD-2871

  • Usernames are case-insensitive, but listings exclude a token if the username (myadmin) does not match the case used when the token was created (myAdmin). CLOUD-2837

  • Upgrade impact: After an upgrade from version 5.2.1 or earlier, the Gateway service does not automatically start after a system reboot. To re-enable the service, run "chkconfig --add cloudgateway" (RHEL 6) or "systemctl enable cloudgateway" (RHEL 7). CLOUD-2819

  • Multipart PUT requests via recent Cyberduck versions fail with 403 SignatureDoesNotMatch when using AWS Signature Version 4. Install the Caringo .cyberduckprofiles from Using the Cyberduck application with Content Gateway S3 which force V2 signatures. CLOUD-2799

  • If a policy document includes a Principal that has plural "users" or "groups" instead of "user" or "group", the policy fails to take effect without warning. CLOUD-2783

  • Versioning-enabled buckets with large numbers of objects may generate Gateway server.log warnings that can be safely ignored: "S3BucketRequestHandler: WARNING: problem with versioned bucket listing. Number of CommonPrefix (2000) exceeds max-size limit (1000)." CLOUD-2643

  • 403 S3 V4 Signature mismatch errors may result when using Cyberduck with the "pound" proxy in front of Gateway S3. Workaround: Disable the Expect header in the Cyberduck preferences, or (recommended) use a different proxy such as "haproxy". CLOUD-2628

  • When Gateway cannot connect to Elasticsearch nodes, the errors may erroneously report this as being related to Storage nodes. CLOUD-2595

  • Because of issues with Range and ETag header handling, video playback of .mp4 streams may not work correctly when served via the Gateway S3 port. It does work when served via the Gateway SCSP port. CLOUD-1964

  • Gateway caches the Swarm version from the "Server:" response header, so after upgrading Swarm you must restart Gateway to consistently see the new version. CLOUD-1271

  • Gateway responds with a 500 (Internal Server Error) instead of 400 (Bad Request) if the size of the metadata headers sent to Swarm is too large. CLOUD-800

  • The S3 bucket listing StorageClass response element always reports STANDARD. CLOUD-766

  • If an S3 client escapes URI path characters such as "/", the Gateway audit log escapes the "%" characters used by the client as escape characters. URI audit log processing for S3 clients requires double-unescaping when this occurs. CLOUD-703

Upgrading from Release 5.x

The Gateway software components packaged as RPMs in the distribution and this section describes the method to upgrade existing 5.x installations. Although these steps include instructions for routing traffic away from Gateways during the upgrade process, you may safely skip these if the deployment does not include a load balancer.

  1. Verify Swarm storage cluster is upgraded to version 9 or later. This includes completing the migration to Elasticsearch 2.3 metadata search servers in conjunction with the Swarm 9 upgrade.

  2. In the load balancer, disable traffic for one Gateway at a time and perform the following upgrade steps for that Gateway. Allow traffic to continue flowing to the other Gateways.

  3. Upgrade the traffic-disabled Gateway.

    1. Stop the Gateway service.

      RHEL/CentOS 6

      service cloudgateway stop

      RHEL/CentOS 7

      systemctl stop cloudgateway

    2. Upgrade the Gateway service:

    3. If using RHEL/CentOS 7, reload the systemd control scripts:

    4. Review and merge any configuration changes to the gateway.cfg and logging.cfg files in the /etc/caringo/cloudgateway directory.  The new distribution versions are the *.rpmnew files.

    5. Start the Gateway service:

      RHEL/CentOS 6

      RHEL/CentOS 7

    6. Verify the Gateway service is running. See the log and curl examples in "Verifying the upgrade," below.

    7. Upgrade the Content Portal:

  4. In the load balancer, re-enable client traffic to the newly upgraded Gateway and repeat the process for the remaining Gateways.

Upgrading from Release 4.x

The Gateway software components packaged as RPMs in the distribution and this section describes the method to upgrade existing 4.x installations. Although these steps include instructions for routing traffic away from Gateways during the upgrade process, you may safely skip these if the deployment does not include a load balancer. Since Gateway 4.x is only certified on RHEL/CentOS 6, the commands shown here are only applicable for RHEL/CentOS 6.

  1. Verify Swarm storage cluster is upgraded to version 9 or later. This includes completing the migration to Elasticsearch 2.3 metadata search servers in conjunction with the Swarm 9 upgrade.

  2. In the load balancer, disable traffic for one Gateway at a time and perform the following upgrade steps for that Gateway. Allow traffic to continue flowing to the other Gateways.

  3. Upgrade the traffic-disabled Gateway. 

    1. Stop the Gateway service:

    2. Upgrade the Gateway service:

    3. Review and merge any configuration changes to the gateway.cfg and logging.cfg files in the /etc/caringo/cloudgateway directory.  The new distribution versions are the *.rpmnew files.

    4. Start the Gateway service:

    5. Verify the Gateway service is running. See the log and curl examples in "Verifying the upgrade," below.

    6. Upgrade the Content Portal:

  4. In the load balancer, re-enable client traffic to the newly upgraded Gateway and repeat the process for the remaining Gateways.

Verifying the upgrade

After upgrading, you can check the Gateway's server log (default: /var/log/caringo/cloudgateway_server.log)  to verify there are no startup errors. Verify the SCSP and/or S3 protocol services are running. This is an example from a server with both protocols enabled:

You can also verify the Gateway is responding to Management API requests by requesting the API version. This command is run from the Gateway and assumes the SCSP protocol is running on port 8080. Adjust the port and/or interface IP address if the configuration is different.

You should get an HTTP 200 response and a JSON body returned with that request.



© DataCore Software Corporation. · https://www.datacore.com · All rights reserved.