| Table of Contents | ||
|---|---|---|
|
...
Docker or proxy use — Gateway can now be configured for use either within a Docker environment or behind a proxy. The Gateway configuration has two new settings available (
externalHTTPPort, externalHTTPSPort) per protocol: [scsp] and [cluster_admin], the Service Proxy. They only take effect when X-Forwarded-Proto is found on the request; Gateway uses X-Forwarded-Proto to determine which port to use. (CLOUD-3055)Metadata Translation — Gateway seamlessly moves custom metadata between SCSP and S3. It translates the metadata formatting between the protocols. It provides S3 and SCSP applications the ability to access each other's metadata. See Metadata Translation between SCSP and S3. (CLOUD-2919)
Quota stability — Quota implementation has been significantly hardened, resolving general problems with frozen usage status/data and excessive emails from state changes. (CLOUD-3045)
Fixed issues:
There were issues when enabling versioning or quotas through the Content UI. The fix prevents creation of duplicate (conflicting) domains in the storage cluster. (CLOUD-3019)
Updating certain Swarm configuration settings through the Gateway services proxy (port 91) sometimes required the sptid query argument value. (CLOUD-2999)
Gateway validates on startup that [storage_cluster] managementPassword is set whenever the ServiceProxy ([cluster_admin]) is enabled. (CLOUD-2617)
New in 5.4.1:
Content Gateway has added support so that it can be used with Swarm UI 2.3. Note that : 2.3 includes a new feed type, S3 Backup feed, which requires Swarm 11.
Several fixes were made related to S3 multipart uploads. (CLOUD-2994, CLOUD-3074)
Upgrade Impacts
Swarm Storage Requirements — Gateway 5.4 requires a minimum back-end storage cluster version of 9.6. The storage cluster must be upgraded prior to installing this version of Gateway.
Releases of Gateway 5.1.3 and earlier are not compatible with Storage versions 9.2 or higher. For historic compatibility details, refer to the releases included with those distributions.
RHEL/CentOS 6 Deprecation — Support for RHEL/CentOS 6 will be is removed in the near future. Completing the transition to RHEL/CentOS 7 now assures a smooth upgrade path to future Gateway versions.
Automatic service start must be re-enabled — After the upgrade, the service may not automatically start after a system reboot. To re-enable the service, run:
"chkconfig --add cloudgateway" for RHEL/CentOS 6, or
"systemctl enable cloudgateway" for RHEL/CentOS 7. (CLOUD-2819)ExpanDrive — ExpanDrive users must upgrade to ExpanDrive 6.1.0 with this version of Gateway; earlier versions report 403 Signature errors when creating a folder or uploading a large object. (CLOUD-2746)
...
When using the default RHEL/CentOS configuration of IPTABLES, traffic to the Gateway will be is blocked unless action is taken to disable IPTABLES or to enable inbound traffic to the front-end protocol port(s).
Gateway is not compatible with the fingerprint scanner module for Linux PAM. If it is installed, remove it by running:
Code Block yum remove fprintd-pam
SCSP reading operations that request a Content-MD5 hash validation and for which there is a hash mismatch will cause causes a storage node to be temporarily removed for the Gateway's connection pool due to the way Swarm reports a hash validation failure.
Swarm Integrity Seal upgrades cannot be performed through Gateway. They may be done directly to the back-end Swarm cluster.
If the HTTP cache control headers
If-Modified-SinceandIf-Unmodified-Sinceare used, review the discussion of these in the Storage SCSP Development.
...
The gateway error log "Unable to create phone home data" at startup can be ignored. CLOUD 3051
The Gateway error "Failed reading from client" on a PUT due to "EofException: Early EOF" may occur when clients do not send the full body. This may point to a bug in the client's retry logic, such as not resetting the position marker to the beginning of the file or part. CLOUD-3010
During new object creation as part of renaming with ?newname, Gateway does not verify that the user has permission to create the new object name (although it is highly likely, because it is a write within the same context). CLOUD-2966
An s3cmd or rclone server-side copy request may time out on a multipart copy for >5GB objects (s4cmd performs it correctly). Workaround: After you verify it is not the HTTPS proxy timing out, increase the client timeout: set s3mcd socket_timeout = 600 in ~/.s3cfg or use rclone copy --timeout=10m --contimeout=2m caringo:mybucket/5gb caringo:mybucket/subfolder/. CLOUD-2949
Listings with max-keys may be shorter than expected because CommonPrefixes are included in the count of keys returned. CLOUD-2917
Uploading files / photos using Panic's Transmit app on iOS fails due to a 403 Invalid Signature error. CLOUD-2886
Gateway 5.2.2 and earlier do not output the NextMarker field in S3 listings, which can cause some S3 clients such as Caringo Drive, rclone, and Transmit to show only 1000 files in a directory or to miss some subdirectories. CLOUD-2871
Usernames are case-insensitive, but listings will exclude a token if the username (myadmin) does not match the case used when the token was created (myAdmin). CLOUD-2837
Upgrade impact: After an upgrade from version 5.2.1 or earlier, the Gateway service will does not automatically start after a system reboot. To re-enable the service, run "chkconfig --add cloudgateway" (RHEL 6) or "systemctl enable cloudgateway" (RHEL 7). CLOUD-2819
Multipart PUT requests via recent Cyberduck versions fail with 403 SignatureDoesNotMatch when using AWS Signature Version 4. Install the Caringo .cyberduckprofiles from Using the Cyberduck application with Content Gateway S3 which force V2 signatures. CLOUD-2799
If a policy document includes a Principal that has plural "users" or "groups" instead of "user" or "group", the policy will fail fails to take effect without warning. CLOUD-2783
Versioning-enabled buckets with large numbers of objects may generate Gateway server.log warnings that can be safely ignored: "S3BucketRequestHandler: WARNING: problem with versioned bucket listing. Number of CommonPrefix (2000) exceeds max-size limit (1000)." CLOUD-2643
403 S3 V4 Signature mismatch errors may result when using Cyberduck with the "pound" proxy in front of Gateway S3. Workaround: Disable the Expect header in the Cyberduck preferences, or (recommended) use a different proxy such as "haproxy". CLOUD-2628
When Gateway cannot connect to Elasticsearch nodes, the errors may erroneously report this as being related to Storage nodes. CLOUD-2595
Because of issues with Range and ETag header handling, video playback of .mp4 streams may not work correctly when served via the Gateway S3 port. It does work when served via the Gateway SCSP port. CLOUD-1964
Gateway caches the Swarm version from the "Server:" response header, so after upgrading Swarm you must restart Gateway to consistently see the new version. CLOUD-1271
Gateway responds with a 500 (Internal Server Error) instead of 400 (Bad Request) if the size of the metadata headers sent to Swarm is too large. CLOUD-800
The S3 bucket listing StorageClass response element always reports STANDARD. CLOUD-766
If an S3 client escapes URI path characters such as "/", the Gateway audit log will escape escapes the "%" characters used by the client as escape characters. URI audit log processing for S3 clients will require requires double-unescaping when this occurs. CLOUD-703
...
The Gateway software components packaged as RPMs in the distribution and this section describes the method to upgrade existing 5.x installations. Although these steps include instructions for routing traffic away from Gateways during the upgrade process, you may safely skip these if your the deployment does not include a load balancer.
Verify Swarm storage cluster is upgraded to version 9 or later. This includes completing the migration to Elasticsearch 2.3 metadata search servers in conjunction with the Swarm 9 upgrade.
In your the load balancer, disable traffic for one Gateway at a time and perform the following upgrade steps for that Gateway. Allow traffic to continue flowing to the other Gateways.
Upgrade the traffic-disabled Gateway.
Stop the Gateway service.
RHEL/CentOS 6
Code Block language bash service cloudgateway stop
RHEL/CentOS 7
Code Block language bash systemctl stop cloudgateway
Upgrade the Gateway service:
Code Block language bash yum -y install caringo-gateway-{version}.rpmIf using RHEL/CentOS 7, reload the systemd control scripts:
Code Block language bash systemctl daemon-reload
Review and merge any configuration changes to the
gateway.cfgandlogging.cfgfiles in the/etc/caringo/cloudgatewaydirectory. The new distribution versions are the*.rpmnewfiles.Start the Gateway service:
RHEL/CentOS 6
Code Block language bash service cloudgateway start
RHEL/CentOS 7
Code Block language bash systemctl start cloudgateway
Verify the Gateway service is running. See the log and curl examples in "Verifying the upgrade," below.
Upgrade the Content Portal:
Code Block language bash yum -y install caringo-gateway-webui-{version}.rpm
In your the load balancer, re-enable client traffic to the newly upgraded Gateway and repeat the process for your the remaining Gateways.
Upgrading from Release 4.x
The Gateway software components packaged as RPMs in the distribution and this section describes the method to upgrade existing 4.x installations. Although these steps include instructions for routing traffic away from Gateways during the upgrade process, you may safely skip these if your the deployment does not include a load balancer. Since Gateway 4.x is only certified on RHEL/CentOS 6, the commands shown here are only applicable for RHEL/CentOS 6.
Verify Swarm storage cluster is upgraded to version 9 or later. This includes completing the migration to Elasticsearch 2.3 metadata search servers in conjunction with the Swarm 9 upgrade.
In your the load balancer, disable traffic for one Gateway at a time and perform the following upgrade steps for that Gateway. Allow traffic to continue flowing to the other Gateways.
Upgrade the traffic-disabled Gateway.
Stop the Gateway service:
Code Block language bash service cloudgateway stop
Upgrade the Gateway service:
Code Block language bash yum -y install caringo-gateway-{version}.rpmReview and merge any configuration changes to the
gateway.cfgandlogging.cfgfiles in the/etc/caringo/cloudgatewaydirectory. The new distribution versions are the*.rpmnewfiles.Start the Gateway service:
Code Block language bash service cloudgateway start
Verify the Gateway service is running. See the log and curl examples in "Verifying the upgrade," below.
Upgrade the Content Portal:
Code Block language bash yum -y install caringo-gateway-webui-{version}.rpm
In your the load balancer, re-enable client traffic to the newly upgraded Gateway and repeat the process for your the remaining Gateways.
Verifying the upgrade
After upgrading, you can check the Gateway's server log (default: /var/log/caringo/cloudgateway_server.log) to verify there are no startup errors. You are looking to verify Verify the SCSP and/or S3 protocol services are running. This is an example from a server with both protocols enabled:
...
You can also verify the Gateway is responding to Management API requests by requesting the API version. This command is run from the Gateway and assumes the SCSP protocol is running on port 8080. Adjust the port and/or interface IP address if your the configuration is different.
...