Versions Compared

Version Old Version 2 New Version Current
Changes made by

Anjali Tyagi

Bala Harish A(AC)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

...

Table of Contents
minLevel1
maxLevel2
outlinefalse
typelist
printablefalse

Using Replication Feeds

Replication feeds enable the replication of objects from one or more domains or the entire cluster to another cluster. Replication provides extra protection and can be a crucial component of a high-availability design. Replication occurs as a background process that is considered "best effort" and subject to inter-cluster bandwidth. For those who require more deterministic replication, a remote synchronous write feature can be employed in gateways serving the source cluster of a replication feed.

Types of Replication

What type of replication method you should choose is chosen and how to configure it depends on whether you have a legacy Swarm implementation exists and on your the needs for securing replication traffic over untrusted networks. (v10.0)

Secure Replication — : Swarm Storage supports remote replication over a WAN, so that replication feeds can operate through the Content Gateway. When you define a replication feed is defined, you specify which replication mode to use: either the legacy bidirectional GET method of replication (which you may need be needed for specific application compatibility or network requirements) or the recommended direct POST method, which offers better performance and flow management. With Swarm Storage 10.0 and later, you can implement TLS/SSL security can be implemented as it fits your the implementation:

  • Upload a trusted certificate to Swarm

  • Replicate to an SSL offloader that services the target cluster

  • Replicate from a forward proxy on

    your

    the source cluster.

See Replicating Replication Feeds over Untrusted Networksand Adding a Trusted Certificate to Swarm.

Replication Methods — Methods: Below are two replication methods available to you, along with the configuration variants of each that are supported. For best performance, choose direct POST replication, which can go through Gateway. GET replication is the legacy method, which may be needed for application compatibility or networking requirements.

Drawio
zoom1
simple0

...

inComment0
custContentId1041006708
pageId687702205
lbox1
diagramDisplayNameReplication-direct-POST.drawio

...

contentVer1
revision1
baseUrlhttps://caringo.atlassian.net/wiki
diagramNameReplication-direct-POST.drawio
pCenter0
width466
links
tbstyle
height639.5
Drawio
zoom1
simple0

...

inComment0
custContentId1040810059
pageId687702205
lbox1
diagramDisplayNameReplication-bidirectional-GET.drawio

...

contentVer1
revision1
baseUrlhttps://caringo.atlassian.net/wiki
diagramNameReplication-bidirectional-GET.drawio
pCenter0
width406
links
tbstyle
height282

...

Note

Using the legacy Bidirectional GET for remote replication requires

...

populating the Storage configuration setting cluster.proxyIpList for any cluster using a reverse proxy. The setting is a comma-separated list of reverse proxy IP addresses or names, including ports in name:port format. If using Direct POST replication, this setting can be populated or left blank, as it has no effect.

Adding a Replication Feed

To add a feed in to the cluster, click the +Add button at the top right of the page and then select the Replication button. 

When you define a replication feed is defined, set the scope and select which type (Replication Mode) is in force and with what speed (number of concurrent Threads), if you are using direct POST:Image Removed

...

The following table describes the data entry fields in the dialog box.

...

ID (

existing

Existing feeds)

Read-only; system-assigned identifier

Status (

existing

Existing feeds)

Read-only; the current feed processing state.

Name

The name

you attach

attached to

this

a feed.

Scope

The feed filters that

you select

were selected for

your

the replication feed. The object

will only be

is replicated to the domain(s)

you indicate

indicated in this field.

Info
title

Gateway adminDomain

Never

Do not create the same domain in two clusters

:

; always create it in the source cluster and then replicate it to the target.

This means that, if you have a Gateway in front of the target cluster, that

A Gateway must use an independent adminDomain, at least temporarily, if the Gateway is in front of the target cluster. (CLOUD-2785)

If you choose to replicate specific domains, ensure that

Verify the source cluster’s adminDomain is included in the list of replicated domains if choosing to replicate specific domains.

  • Entire source cluster (global) 

— To

  • - To replicate all objects in the source cluster, leave the default selection of Entire source cluster (global)

  • Only objects in select domain(s)

—

  • - To replicate

only

  • the objects in one or more domains, select the 'Only objects in select domain(s) option. In the text box that appears, enter one or more domains:

To

    • Enter the domain to replicate

only

    • the objects within a specific domain

, enter that domain

    • .

To replicate only the objects within 

    • Enter multiple domains

, enter those domains

    • separated by commas, and/or use pattern matching to replicate objects within multiple domains.

To

    • Enter domains to exclude

domains

    • them from replication

, enter them

    • . (v10.0)

The field value allows pattern matching with the Python regular expression (RE) syntax, so

that

multiple domain names can be matched. The exception to the RE matching is that the "{m,n}"

repetitions

repetition qualifier may not be used.

An example domain list value using RE is: .*\.example\.com
This

would match

matches both of these domains: accounting.example.com, engineering.example.com.

Image Removed
Image Added

  • Include objects without a domain

—

  • - To replicate any unnamed objects that are not tenanted in any domain, enable the option.

Target Remote Cluster Name

The configuration setting for

your

the target cluster (

for example,

the cluster.name parameter in the.cfg file of the target cluster).

When using Gateway as target, be sure to

configure the Gateway setting allowSwarmAdminIP when using Gateway as a target. See Gateway Configuration.

Proxy or Host(s)

The IP address(es) or host name(s) of either:

  • One or more nodes in the target cluster. It is best to list all nodes in the target cluster, while Swarm follows redirects, it does not use those redirect targets for subsequent requests.

  • A reverse proxy host that routes to the target cluster.

To

Enter each address with a comma or space between them to enter two or more node IP addresses

, enter each address separated by a comma or spaces

.

Port

Defaults to 80.

Lets you specify

Allows specifying a custom port for the remote cluster.

Replication Mode

Defaults to Direct POST. Choose replication via direct POST (recommended) or bidirectional GET. Switching modes does not require a feed restart. (

v1

v9.

2

6)

For best performance, choose direct POST replication, which can go through Gateway. GET replication is the legacy method, which may be needed for application compatibility or networking requirements.

Threads

Replication via direct POST

only

. The default replication speed (

6

20 simultaneous threads) is best for same-sized clusters with minimal replication backlog.

 (v1.2)To

 When processing backlog, the new default push thread count is 20 per node (in releases prior to v15.0, it was 6 per volume)

Reduce the threads to avoid overwhelming a smaller target cluster

, reduce the threads

. For faster replication against a backlog, increase the threads temporarily

, but be sure to

and monitor bandwidth and cluster performance, as boosting the speed

will stress

stresses both clusters.

SSL Server

Replication via direct POST

only

. Defaults to none.

 If you are

 Enable Require trusted SSL if replicating over an untrusted network

, enable Require trusted SSL

; Allow untrusted SSL is available but not intended for production systems. (

v2

v10.0)

Remote Admin Name/Password

Inherit from source cluster: Uncheck the enabled box,

only

if  the remote cluster user name is different from the source cluster name in the same realm. Then enter:

User/Password credentials

  • The administrative user name of the target cluster.

  • The administrative password of the target cluster.

...

Propagate Deletes

Info

The legacy option Propagate Deletes is deprecated; it existed to let you set allow setting a target cluster to preserve all deleted objects. This need is now covered by Object Versioning; you can access historical versions of deleted objects can be accessed to recover mistakenly deleted content that was deleted by mistake. You can also limit versioning . Versioning to the target cluster that is serving as your the archive can be limited, to minimize space usage. (v11.1)

If you have Note these restrictions and behaviors if an existing feed that still has this option specified, note these restrictions and behaviors:

  • With Versioning

    —

    - Always propagate deletes when using Object Versioning in

    your

    the cluster.

  • Without Versioning

    — If this option is disabled, be aware that you are having your

    - The target cluster

    maintain

    maintains deleted content

    that carries

    carrying no verifiable indication of

    its

    the deleted status if this option is disabled.

As objects are written or updated, their
Excerpt

Using Feed Actions

For Clicking on an existing feed , clicking on it in the Feeds list will open its opens the Feed Settings page, with the existing settings populated. The Actions (gear) icon menu at the top right supports multiple feed actions, appropriate to the type of feed:

Image Modified

Pause / Resume

You

Manual Pause: Feed processing may occasionally

wish to pause feed processing in order

need to be paused to perform system maintenance.

For example, when you are upgrading your Elasticsearch cluster, you would pause

Pause the search feed before stopping the Elasticsearch service in

your

the search cluster

. After completing your system maintenance, return

when upgrading an Elasticsearch cluster. Return to the action menu and select the Resume action to resume feed processing after completing system maintenance.

Refresh

Automatic Pause: When the setting feeds.pauseDisconnectPerHourLimit is non-zero and the number of disconnects for a node exceeds the value (default 1000), the feed is automatically paused with a CRITICAL error message.

Refresh

Object data is sent to the feed target in near real-time (NRT) as it is written or updated. Any objects

that cannot

unable to be processed immediately are retried each HP cycle until

they succeed

successful, at which point they are marked as complete and are

never

not resent.

If a data loss failure occurs on your remote feed target and you cannot restore from backup, select

Select the Refresh option from the feed action menu, which

will verify

verifies and

rehydrate

rehydrates all

of the

previously sent content to a remote cluster if a data loss failure occurs on the remote feed target and restoration from backup is not possible. This process

will take

takes some time, as it must revisit all objects in the cluster.

Delete

When

you delete

a feed is deleted, it frees source cluster resources. This process does not affect the objects previously pushed to the remote target.

To delete a feed, select

Select the Delete option from the feed action menu and

confirm you intend

verify the intention to permanently delete the feed. The deleted feed is removed from the remaining cluster nodes within 60 seconds. 

View

feed table

Feed Table

Displays the SNMP Repository Dump for the selected node, for feed diagnostics (see below).

Troubleshooting Feeds

  • Feed

    diagnostics — To troubleshoot

    Diagnostics: Double-click a blocked feed

    , double-click it

    to open

    its

    the settings page, click the gear icon, and select View feed table, which displays the SNMP Repository Dump for the selected node to troubleshoot. (v2.0)

    Image RemovedImage Added

    Review the feedPluginState status to identify the blockage. 
    Example: feedPluginState blocked: Destination cluster onyx1 reports invalid request: Castor-System-Cluster value must refer to a remote cluster on RETRIEVE request

  • Idle

    feeds — A

    Feeds: A feed can appear to be idle with items still queued for processing. Plan for the fact that feed status reporting is a best-effort snapshot, not a low-latency or guaranteed transaction mechanism.

  • Feed

    prioritization —

    Prioritization: Domain and bucket context objects are prioritized for all types of feeds; this improves usability when

    you initiate

    remote sites are initiated.

  • Retries for

    blocked feeds —

    Blocked Feeds: Blocked feeds are retried every 20 minutes, but if

    you change

    the definition for a blocked feed is changed, it triggers an immediate attempt with the new definition, which

    might

    may clear the blockage. (v10.1)

...