Help Center/ Cloud Search Service/ User Guide (ME-Abu Dhabi Region)/ Managing Elasticsearch Clusters/ Migrating Cluster Data
Updated on 2023-06-20 GMT+08:00
View PDF
Share

Migrating Cluster Data

You can migrate data from one cluster to another. In certain scenarios, for example, if you cannot get sufficient capacity by changing the specifications of the current cluster, you can create a cluster of higher specifications and migrate all data of the current cluster to the new one. Alternatively, you can merge indexes in two clusters to one cluster to meet your business needs. CSS enables you to migrate cluster data by using the index backup and restoration function, specifically, by restoring the snapshot of a cluster to the target cluster.

In this section, assume that we need to migrate the data of cluster Es-1 to cluster Es-2. Cluster Es-2 runs a version later than that of cluster Es-1 and the number of nodes in cluster Es-2 is greater than half of that in cluster Es-1.

Prerequisites

  • The source and target clusters are in the same region.
  • The version of the target cluster is the same as or later than that of the source cluster.
  • The number of nodes in the target cluster must be greater than half of the number of nodes in the source cluster.

Suggestions

  • The number of nodes in the target cluster should be no less than the number of replicas in the source cluster.
  • The CPU, memory, and disk configurations of the target cluster should be no less than those of the source cluster. This will minimize service loss after migration.

Migration Duration

The number of nodes or index shards in the source and destination clusters determines how long the data migration will take. Data migration consists of two phases: data backup and restoration. The backup duration is determined by the source cluster and the restoration duration is determined by the destination cluster. The formula for calculating the total migration duration is as follows:

  • If the number of index shards is greater than the number of nodes:

  • If the number of index shards is smaller than the number of nodes:

The migration duration estimated using the formula is the minimal duration possible (if each node transmits data at the fastest speed, 40 MB/s). The actual duration also depends on factors such as the network and resources condition.

Procedure

  1. On the Clusters page, click Es-1.
  2. In the navigation pane on the left, choose Cluster Snapshots. Enable the cluster snapshot and set basic configuration. For details, see Manually Creating a Snapshot.
  3. Click Create to manually create a snapshot. In the displayed dialog box, enter the snapshot name and click OK.
  4. In the snapshot list, locate the target snapshot and click Restore in the Operation column to restore data to cluster Es-2.
    • In the text box next to Index, enter *, indicating that you want to restore data of all indexes in cluster Es-1.
    • From the Cluster drop-down list, select Es-2.

    Click OK.

  5. After restoration is complete, data in cluster Es-1 will be migrated to cluster Es-2.