Overview

For more information about Swarm Gateway, its concepts, and operations, seeSwarm Content Gatewayclick here.

Prerequisites

• RHEL/CentOS 7 .x or RHEL/Rocky Linux 8 Setup

• Latest Swarm 14.0 bundle package

Setup

...

Linux for Cloud Gateway

Below is an the deployment example deployment for the site.

...

Install Swarm Gateway

1. Create a user group.

   Code Block
   groupadd clusteradmins

...

2. Add a new user.

   Code Block
   language bash
   adduser

...

1. admin -g clusteradmins -p

...

1. datacore passwd

...

1. admin

2. Update

...

1. Linux and install EPEL.

   Code Block
   language bash
   yum -y update yum -y install epel-release

2. Install NTP server and verify NTP starts on

...

1. system boot.

   Code Block
   language bash
   yum -y install

...

1. chrony systemctl enable

...

1. chronyd systemctl start

...

Install Java 8 JDK yum -y install java-1.8.0-openjdk.

1. chronyd

2. Configure the firewall to allow Swarm Gateway.

   Code Block
   vi /

...

1. etc/firewalld/services/swarm_gateway.xml

...

1. Code Block
   language xml
   <?xml version="1.0" encoding="utf-8"?

...

1. >

...

1. <service>   <short>SwarmGateway</

...

1. short>   <description>Gateway server, which is based on http traffic.</

...

1. description>   <port protocol="tcp" port="80"/

...

1. >   <port protocol="tcp" port="91"/

...

1. >   <port protocol="tcp" port="8090"/

...

1. > <port protocol="tcp" port="9095"/>   <port protocol="tcp" port="9100"/

...

1. > </

...

1. service>

2. Restart the firewall service.

3. Add Swarm Gateway firewall rule (permanent)

...

1. Code Block
   firewall-cmd --permanent --add-service swarm_gateway

...

2. Reload the firewall

...

1. service.

   Code Block
   firewall-cmd --reload

...

2. Unzip the Swarm

...

1. bundle package

...

1. .

   Code Block
   cd /root/datacore unzip Swarm-{bundle version here}.zip

2. Get the latest gateway and UI RPMs from the downloaded Swarm bundle.

   Code Block
   language bash
   ll ~/datacore/Gateway/

...

...

1. -rw-r--r--. 1 root root 58718208 Dec 16 22:37 caringo-gateway-

...

1. {version here}.noarch.rpm  -rw-r--r--. 1 root root      395 Dec 16 22:37 README.txt  -rw-r--r--. 1 root root     1927 Oct  2 04:17 RPM-GPG-KEY ll ~/datacore/UI/

...

1. -rw-r--r--. 1 root root  2835590 Oct  2 04:17 caringo-gateway-webui-

...

1. {version here}.noarch.rpm  -rw-r--r--. 1 root root 19286001 Dec 16 22:37 caringo-storage-webui-

...

1. {version here}.noarch.rpm  -rw-r--r--. 1 root root      359 Dec 16 22:37 README.txt  -rw-r--r--. 1 root root     1927 Oct  2 04:17 RPM-GPG-KEY

2. Install Swarm RPM public key included with the distribution bundle

...

1. .

   Code Block
   rpm --import RPM-GPG-KEY

...

2. Install Cloud Gateway

...

1. components.

   Code Block
   yum -y install caringo-gateway-

...

1. {version here}.noarch.rpm

...

2. Complete the IDSYS document for PAM user authentication.

   Code Block
   vi /etc/caringo/cloudgateway/idsys.json

...

1. Code Block
   language json
   {"pam": {    "name" : "idsys-pam",    "description": "PAM identity management configuration",    "cookieName": "token",    "tokenPath":  "/.TOKEN/",    "tokenAdmin": "

...

1. admin@" }}

2. Complete policy document for

...

1. access control.

   Code Block
   vi /etc/caringo/cloudgateway/policy.json

...

1. Code Block
   language json
   {    "Id": "Root policy -- grant admins full access to everything",    "Statement": [      { "Sid": "Grant admins full access",        "Resource": "*",        "Principal": {          "user": ["

...

1. admin@"],          "group": ["clusteradmins@"]        },        "Action": [ "*" ],        "Effect": "Allow"      }    ]  }

2. Install Gateway

...

1. WebUI.

   Code Block
   yum -y install caringo-gateway-webui-

...

1. {version here}.noarch.rpm

...

2. Edit

...

1. the configuration file.

   Code Block
   vi /etc/caringo/cloudgateway/gateway.cfg

...

1. 1. Set adminDomain for the administrative domain name to be created. It is recommended to always set a unique name for each Swarm cluster to prevent replication collision between the Source and Target Swarm Cluster when using replication feeds. Set a unique name for each Swarm cluster to prevent replication collision.

   2. Set hosts for the storage cluster nodes (IP address of storage nodes).

   3. Set indexerHosts to Elasticsearch Servers (IP address of Elasticsearch nodes).

   4. Enable S3.

   5. EnableCluster Admin on the gateway server to allow usage of the Swarm Storage UI.

   6. Set Management Port to 91. The admin user and password must be the same as configured in SCS.

      1. Set managementUser to the same value as admin.username

      2. Set managementPassword to the same value as admin.password

         Code Block
         language bash
         # Following are only needed when enabling cluster_

...

1. 1. 1. admin managementPort = 91  managementUser =

...

1. 1. 1. admin  managementPassword = datacore

   2. Enable metering.

Below is a sample Gateway configuration file in its entirety. This is provided only for reference.

# 
# gateway.cfg -- configuration file for Caringo CloudScaler Gateway server 
# 
# Please read the CloudScaler Gateway Administration Guide for an explanation 
# of the parameters in this configuration file. 
# 
# Basic Configuration Steps: 
# ------------------------- 
# 
#   1. Set '[gateway]adminDomain' to administrative domain's name 
# 
#   2. Use the Swarm static locator for the backend cluster: 
#        - Set '[storage_cluster]locatorType' to 'static' 
#        - Set '[storage_cluster]hosts' for your Swarm nodes 
# 
#   3. Configure the Elasticsearch metadata backend: 
#        - Set '[storage_cluster]indexerHosts' to the metadata search servers 
# 
#   4. If the S3 protocol is used: 
#        - Ensure that bindAddress:bindPort does not conflict with SCSP 
# 
# 
# NOTE: For production use, you will need to adjust the thread counts, 
#       connection limits, connection time outs, and the space/limits 
#       for the HTTP multi-part spool. 
# 

# 
# Client communications and handling 
# 
[gateway]

adminDomain = Lab1GatewayAdminDomain 
threads = 200 
[gateway]

adminDomain = Lab1GatewayAdminDomain 
threads = 100 * (Number of CPUs present in the Gateway) 
# multipartSpoolDir = /var/spool/cloudgateway 
# multipartUsageAllowed = 50 
# sanitizeErrors = false 
# enablePasswordEncryption = false 
# legacyOnlyMode = false
 
# 
# Storage cluster back-end configuration
#
[storage_cluster] 

locatorType = static 
hosts = 192.168.9.143 192.168.9.144 192.168.9.145
# port = 80
# dataProtection = immediate
# blockUndeletableWrites = true

indexerHosts = 192.168.9.21 192.168.9.22 192.168.9.23
# indexerPort = 9200
# indexerSocketTimeout = 120

# maxConnectionsPerRoute = threads / 100 2 
# maxConnections = 250(threads / 2) * (number of nodes)
# connectTimeout = 60
# socketTimeout = 120
# idleTimeout = 120
# continueWaitTimeout = 30

# Following are only needed when enabling cluster_admin 
managementPort = 91
managementUser = dcadminadmin
managementPassword = datacore

# 
# SCSP front-end protocol
#
[scsp] 

enabled = true 
bindAddress = 0.0.0.0 
bindPort = 80

# Set the following if this Gateway is the target of a push-style replication feed.
# See 'Replication Feeds' in the Swarm documentation for details.
# allowSwarmAdminIP = list,of,node,ips -or- all
allowSwarmAdminIP = all

# Set these to the external ports on your proxy, if using one 
#externalHTTPPort = 80 
#externalHTTPSPort = 443 

# 
# S3 front-end protocol
# 
[s3]

enabled = true 
bindAddress = 0.0.0.0 
bindPort = 8090

# Set these to the external ports on your proxy, if using one 
#externalHTTPPort = 80 
#externalHTTPSPort = 443

# 
# Cluster management protocol 
#
[cluster_admin] 

enabled = true 
bindAddress = 0.0.0.0 
bindPort = 91
# secretKey = yoursecretkeyhere

# Set these to the external ports on your proxy, if using one 
#externalHTTPPort = 91 
#externalHTTPSPort = 1443 

#
# Internal "folder listing service" config
# 
[folder_listings]

# enhancedListingConsistency = true     # force ES index flush before query 
# feedCheckInterval = 60000             # in milliseconds, how often to check for non-searchable domains (search feed filters) 
# suppressNonSearchableError = false    # suppress 412 error when searching non-searchable domain 

# 
# Caching timers : 0 == disabled 
#
[caching] 

# authRefresh = 300 
# tokenRefresh = 300 
# idsysRefresh = 300 
# policyRefresh = 300 
# xformRefresh = 300 
# metadataRefresh = 300 
# domainExistenceRefresh = 300

# 
# Metering Support 
#
[metering] 

enabled = true 
# flushIntervalSeconds = 300 
# retentionDays = 100 
# storageSampleIntervalSeconds = 3600 

# 
# Quota Support 
#
[quota]

enabled = false 
smtpHost = localhost 
mailFrom = donotreply@localhost 

# mailSubjectTemplate = Quota state change notification 
# mailTemplate = Metric %metric% changed to %state% state in %contextType% %contextName%.

# smtpPort = 25 
# smtpUser = 
# smtpPassword = 

# minRefreshDeadline = 60 
# maxRefreshDeadline = 3600 
# numRefreshThreads = 4 
# maxRefreshRetries = 3 
# maxQueueSize = 10000 
# queryTTL = 3600 
# refreshRetryDelay = 10 
# refreshIdleSleep = 3

# 
# Prometheus metrics capturing support 
# 
[metrics] 

metricsEnabled = true     # default enabled 
# metricsPort = 9100        # port where to scrape for metrics 

# 
# Remote synchronous write support (RSW) 
# 
[rsw]

# enabled = true            # default enabled
# maxWaitTime = -1          # timeout in seconds to wait for RSW completion, -1 means no timeout
# enableInfoLogging = true  # default true, log RSW operations at INFO level

# 
# Object Lock Support (Retention, LegalHold)
#
[object_locking] 

# scspDeleteUsesS3Logic = true  # Allows delete of a locked current object version (via delete marker)
# retentionMaxYears = 100       # Max retention duration 

Cluster Initialization

The Gateway gateway must be initialized once the configuration is complete to guarantee a proper runtime environment once the configuration is complete. This guarantees ensures the administrative domain exists within the storage cluster and also secures all plaintext passwords entered into the configuration files.

...

Use systemctl stop cloudgateway to verify the Gateway is not running

...

1. Verify Gateway is not running.

   Code Block
   systemctl stop cloudgateway

2. Perform the initialization of the environment.

   Code Block
   /opt/caringo/cloudgateway/bin/initgateway

...

2. Start

...

1. CloudGateway and verify

...

1. the gateway service is running on

...

1. system boot once initialization

...

1. is successfully completed.

   Code Block
   language bash
   systemctl start

...

1. cloudgateway systemctl enable

...

1. cloudgateway

...