Skip to end of metadata
Go to start of metadata

You are viewing an old version of this content. View the current version.

Compare with Current View Version History

« Previous Version 6 Next »

Overview

<An overview of Swarm Cloud Gateway>

Prerequisites

Setup CentOS for Cloud Gateway

Minimal Mode

Host name

cg77-01.datacore.internal

cg77-02.datacore.internal

Application Network 

IP Address

172.16.33.16/24 

172.16.33.17/24 

Gateway

172.16.33.1

NTP

172.16.33.10

Install Cloud Gateway

Info

Creating a user group/user/password applies when PAM authentication is used. You can create a user group via groupadd clusteradmins, and a new user via adduser dcadmin -g clusteradmins -p datacore & passwd datacore commands.

  1. Create a user group groupadd clusteradmins.

  2. Add a new user.

    adduser dcadmin -g clusteradmins -p datacore 
    passwd datacore 
  3. Update CentOS and install epel.

    yum -y update
    yum -y install epel-release
  4. Install NTP server and ensure that NTP starts on the system boot.

    yum -y install chrony 
    systemctl enable chronyd 
    systemctl start chronyd 
  5. Install Java 8 JDK yum -y install java-1.8.0-openjdk.

  6. Configure the firewall to allow Cloud Gateway vi /usr/lib/firewalld/services/swarm_gateway.xml.

    <?xml version="1.0" encoding="utf-8"?> 
    <service> 
      <short>SwarmGateway</short> 
      <description>Elasticsearch server REST API, which is based on http traffic.</description> 
      <port protocol="tcp" port="80"/> 
      <port protocol="tcp" port="91"/> 
      <port protocol="tcp" port="8090"/> 
      <port protocol="tcp" port="9100"/> 
    </service> 
  7. Restart the firewall service.

  8. Add Cloud Gateway firewall rule (permanent) via firewall-cmd --permanent --add-service swarm_gateway command.

  9. Reload the firewall service firewall-cmd --reload.

  10. Unzip Swarm 14.0 bundle package to /root/datacore.

  11. From the downloaded Swarm bundle, get the latest gateway and UI RPMs.

    ll ~/datacore/Gateway/ 
    total 57352
    -rw-r--r--. 1 root root 58718208 Dec 16 22:37 caringo-gateway-7.7.1-2.noarch.rpm 
    -rw-r--r--. 1 root root      395 Dec 16 22:37 README.txt 
    -rw-r--r--. 1 root root     1927 Oct  2 04:17 RPM-GPG-KEY
    
    ll ~/datacore/UI/ 
    total 21616 
    -rw-r--r--. 1 root root  2835590 Oct  2 04:17 caringo-gateway-webui-7.4.0-1.noarch.rpm 
    -rw-r--r--. 1 root root 19286001 Dec 16 22:37 caringo-storage-webui-3.4.0-1.noarch.rpm 
    -rw-r--r--. 1 root root      359 Dec 16 22:37 README.txt 
    -rw-r--r--. 1 root root     1927 Oct  2 04:17 RPM-GPG-KEY 
  12. Install Swarm RPM public key included with the distribution bundle using rpm –import RPM-GPG-KEY.

  13. Install Cloud Gateway components yum -y install caringo-gateway-7.8.0-1.noarch.rpm.

  14. Complete the IDSYS document for PAM user authentication vi /etc/caringo/cloudgateway/idsys.json

    {"pam": { 
      "name" : "idsys-pam", 
      "description": "PAM identity management configuration", 
      "cookieName": "token", 
      "tokenPath":  "/.TOKEN/", 
      "tokenAdmin": "dcadmin@"
    }} 
  15. Complete policy document for the access control vi /etc/caringo/cloudgateway/policy.json.

    { 
      "Id": "Root policy -- grant admins full access to everything", 
    
      "Statement": [ 
        { "Sid": "Grant admins full access", 
          "Resource": "*", 
          "Principal": { 
            "user": ["dcadmin@"], 
            "group": ["clusteradmins@"] 
          }, 
          "Action": [ "*" ], 
          "Effect": "Allow" 
        } 
      ] 
    } 
  16. Install Gateway WebUI yum -y install caringo-gateway-webui-7.5.0-1.noarch.rpm.

  17. Edit Cloud Gateway configure file /etc/caringo/cloudgateway/gateway.cfg.

    1. Set adminDomain for the administrative domain name to be created. It is recommended to always set a unique name for each Swarm cluster to prevent replication collision between Source and Target Swarm Cluster. Set a unique name for each Swarm cluster to prevent replication collision.

    2. Set hosts for the storage cluster nodes (IP address of storage nodes).

    3. Set indexerHosts to Elasticsearch Servers (IP address of Elasticsearch nodes).

    4. Enable S3.

    5. Enable Cluster Admin on the gateway server. It will install Swarm Storage UI.

    6. Enable Management Port 91. The admin user and password must be the same as configured in SCS.

      1. Set admin.username

      2. Set admin.password

        # Following are only needed when enabling cluster_admin 
        managementPort = 91 
        managementUser = dcadmin 
        managementPassword = datacore 
    7. Enable metering.

# 
# gateway.cfg -- configuration file for Caringo CloudScaler Gateway server 
# 
# Please read the CloudScaler Gateway Administration Guide for an explanation 
# of the parameters in this configuration file. 
# 
# Basic Configuration Steps: 
# ------------------------- 
# 
#   1. Set '[gateway]adminDomain' to administrative domain's name 
# 
#   2. Use the Swarm static locator for the backend cluster: 
#        - Set '[storage_cluster]locatorType' to 'static' 
#        - Set '[storage_cluster]hosts' for your Swarm nodes 
# 
#   3. Configure the Elasticsearch metadata backend: 
#        - Set '[storage_cluster]indexerHosts' to the metadata search servers 
# 
#   4. If the S3 protocol is used: 
#        - Ensure that bindAddress:bindPort does not conflict with SCSP 
# 
# 
# NOTE: For production use, you will need to adjust the thread counts, 
#       connection limits, connection time outs, and the space/limits 
#       for the HTTP multi-part spool. 
# 

# 
# Client communications and handling 
# 
[gateway]

adminDomain = Lab1GatewayAdminDomain 
threads = 200 
# multipartSpoolDir = /var/spool/cloudgateway 
# multipartUsageAllowed = 50 
# sanitizeErrors = false 
# enablePasswordEncryption = false 
# legacyOnlyMode = false
 
# 
# Storage cluster back-end configuration
#
[storage_cluster] 

locatorType = static 
hosts = 192.168.9.143 192.168.9.144 192.168.9.145
# port = 80
# dataProtection = immediate
# blockUndeletableWrites = true

indexerHosts = 192.168.9.21 192.168.9.22 192.168.9.23
# indexerPort = 9200
# indexerSocketTimeout = 120

# maxConnectionsPerRoute = 100 
# maxConnections = 250
# connectTimeout = 60
# socketTimeout = 120
# idleTimeout = 120
# continueWaitTimeout = 30

# Following are only needed when enabling cluster_admin 
managementPort = 91
managementUser = dcadmin
managementPassword = datacore

# 
# SCSP front-end protocol
#
[scsp] 

enabled = true 
bindAddress = 0.0.0.0 
bindPort = 80

# Set the following if this Gateway is the target of a push-style replication feed.
# See 'Replication Feeds' in the Swarm documentation for details.
# allowSwarmAdminIP = list,of,node,ips -or- all
allowSwarmAdminIP = all

# Set these to the external ports on your proxy, if using one 
#externalHTTPPort = 80 
#externalHTTPSPort = 443 

# 
# S3 front-end protocol
# 
[s3]

enabled = true 
bindAddress = 0.0.0.0 
bindPort = 8090

# Set these to the external ports on your proxy, if using one 
#externalHTTPPort = 80 
#externalHTTPSPort = 443

# 
# Cluster management protocol 
#
[cluster_admin] 

enabled = true 
bindAddress = 0.0.0.0 
bindPort = 91
# secretKey = yoursecretkeyhere

# Set these to the external ports on your proxy, if using one 
#externalHTTPPort = 91 
#externalHTTPSPort = 1443 

#
# Internal "folder listing service" config
# 
[folder_listings]

# enhancedListingConsistency = true     # force ES index flush before query 
# feedCheckInterval = 60000             # in milliseconds, how often to check for non-searchable domains (search feed filters) 
# suppressNonSearchableError = false    # suppress 412 error when searching non-searchable domain 

# 
# Caching timers : 0 == disabled 
#
[caching] 

# authRefresh = 300 
# tokenRefresh = 300 
# idsysRefresh = 300 
# policyRefresh = 300 
# xformRefresh = 300 
# metadataRefresh = 300 
# domainExistenceRefresh = 300

# 
# Metering Support 
#
[metering] 

enabled = true 
# flushIntervalSeconds = 300 
# retentionDays = 100 
# storageSampleIntervalSeconds = 3600 

# 
# Quota Support 
#
[quota]

enabled = false 
smtpHost = localhost 
mailFrom = donotreply@localhost 

# mailSubjectTemplate = Quota state change notification 
# mailTemplate = Metric %metric% changed to %state% state in %contextType% %contextName%.

# smtpPort = 25 
# smtpUser = 
# smtpPassword = 

# minRefreshDeadline = 60 
# maxRefreshDeadline = 3600 
# numRefreshThreads = 4 
# maxRefreshRetries = 3 
# maxQueueSize = 10000 
# queryTTL = 3600 
# refreshRetryDelay = 10 
# refreshIdleSleep = 3

# 
# Prometheus metrics capturing support 
# 
[metrics] 

metricsEnabled = true     # default enabled 
# metricsPort = 9100        # port where to scrape for metrics 

# 
# Remote synchronous write support (RSW) 
# 
[rsw]

# enabled = true            # default enabled
# maxWaitTime = -1          # timeout in seconds to wait for RSW completion, -1 means no timeout
# enableInfoLogging = true  # default true, log RSW operations at INFO level

# 
# Object Lock Support (Retention, LegalHold)
#
[object_locking] 

# scspDeleteUsesS3Logic = true  # Allows delete of a locked current object version (via delete marker)
# retentionMaxYears = 100       # Max retention duration 

  • No labels