About Elasticsearch Cluster Migration Solutions

Migrating data between Huawei Cloud Elasticsearch clusters

Huawei Cloud Logstash

• Migrate data from a Huawei Cloud Elasticsearch cluster of an earlier version to one of a later version.
• Migrate data from multiple Huawei Cloud Elasticsearch clusters to a single such cluster to form a unified data platform.

During cluster migration, do not add, delete, or modify indexes in the source cluster, or the source and destination clusters will have inconsistent data after the migration.

Migrating Data Between Elasticsearch Clusters Using Huawei Cloud Logstash

Backup and restoration

• Migrate data between Huawei Cloud Elasticsearch clusters in the same region or across regions, and under the same or different accounts.
• Migrate data from a Huawei Cloud Elasticsearch cluster of an earlier version to one of a later version.
• Migrate data from multiple Huawei Cloud Elasticsearch clusters to a single such cluster to form a unified data platform.

The version of the destination cluster must not be earlier than that of the source cluster. For details, see Snapshot version compatibility.

Migrating Data Between Huawei Cloud Elasticsearch Clusters Using Backup and Restoration (GUI)

Read/write splitting plugin

• Migrate data between Huawei Cloud Elasticsearch clusters in the same region or across regions, and under the same or different accounts.
• Migrate data from multiple Huawei Cloud Elasticsearch clusters to a single such cluster to form a unified data platform.

The versions of the source and destination clusters must both be 7.6.2 or 7.10.2.

Migrating Data Between Huawei Cloud Elasticsearch Clusters Using the Read/Write Splitting Plugin

Reindex API

Migrate data from multiple Huawei Cloud Elasticsearch clusters to a single such cluster to form a unified data platform.

During cluster migration, do not add, delete, or modify indexes in the source cluster, or the source and destination clusters will have inconsistent data after the migration.

Migrating Data Between Elasticsearch Clusters Using the Reindex API

ESM

• Migrate data from a Huawei Cloud Elasticsearch cluster of an earlier version to one of a later version.
• Migrate data from multiple Huawei Cloud Elasticsearch clusters to a single such cluster to form a unified data platform.

During cluster migration, do not add, delete, or modify indexes in the source cluster, or the source and destination clusters will have inconsistent data after the migration.

Migrating Data Between Elasticsearch Clusters Using ESM

Elasticdump

Migrate small volumes of data and a limited number of indexes. For large-scale migrations, consider other more suitable methods.

During cluster migration, do not add, delete, or modify indexes in the source cluster, or the source and destination clusters will have inconsistent data after the migration.

Migrating Data Between Elasticsearch Clusters Using Elasticdump

Migrating data from an in-house built Elasticsearch cluster to Huawei Cloud

Huawei Cloud Logstash

• Migrate data from a self-managed Elasticsearch cluster of an earlier version to a Huawei Cloud Elasticsearch cluster of a later version.
• Migrate data from multiple self-managed Elasticsearch clusters to a single Huawei Cloud Elasticsearch cluster to form a unified data platform.

• During cluster migration, do not add, delete, or modify indexes in the source cluster, or the source and destination clusters will have inconsistent data after the migration.
• The Elasticsearch clusters must be accessible via the public network.

Migrating Data Between Elasticsearch Clusters Using Huawei Cloud Logstash

Backup and restoration

• Migrate data from a self-managed Elasticsearch cluster of an earlier version to a Huawei Cloud Elasticsearch cluster of a later version.

• Migrate data from multiple self-managed Elasticsearch clusters to a single Huawei Cloud Elasticsearch cluster to form a unified data platform.

• The version of the destination cluster must not be earlier than that of the source cluster. For details, see Snapshot version compatibility.
• This method does not support incremental data synchronization. You need to pause data update before starting to back up data.
• A self-managed Elasticsearch cluster needs public network access to back up snapshots to OBS.

Migrating Data from an In-House Built Elasticsearch Cluster to Huawei Cloud Using the S3 Plugin

Reindex API

Migrate data from multiple self-managed Elasticsearch clusters to a single Huawei Cloud Elasticsearch cluster to form a unified data platform.

During cluster migration, do not add, delete, or modify indexes in the source cluster, or the source and destination clusters will have inconsistent data after the migration.

Migrating Data Between Elasticsearch Clusters Using the Reindex API

ESM

• Migrate data from a self-managed Elasticsearch cluster of an earlier version to a Huawei Cloud Elasticsearch cluster of a later version.
• Migrate data from multiple self-managed Elasticsearch clusters to a single Huawei Cloud Elasticsearch cluster to form a unified data platform.

During cluster migration, do not add, delete, or modify indexes in the source cluster, or the source and destination clusters will have inconsistent data after the migration.

Migrating Data Between Elasticsearch Clusters Using ESM

Elasticdump

Migrate small volumes of data and a limited number of indexes. For large-scale migrations, consider other more suitable methods.

During cluster migration, do not add, delete, or modify indexes in the source cluster, or the source and destination clusters will have inconsistent data after the migration.

Migrating Data Between Elasticsearch Clusters Using Elasticdump

CDM

This is a cloud migration tool provided by Huawei Cloud to migrate data between clusters that belong to different cloud services.

• A VPN or Direct Connect connection needs to be established between the customer's IDC and Huawei Cloud.
• During cluster migration, do not delete any of the source cluster's indexes, or something might go wrong.

-

Migrating data from a third-party Elasticsearch cluster to Huawei Cloud

Huawei Cloud Logstash

• Migrate Elasticsearch clusters across major versions, for example, from 6.X to 7.X.
• Migrate data from multiple third-party Elasticsearch clusters to a single Huawei Cloud Elasticsearch cluster to form a unified data platform.

• During cluster migration, do not add, delete, or modify indexes in the source cluster, or the source and destination clusters will have inconsistent data after the migration.
• A VPN or Direct Connect connection needs to be established between the customer's IDC and Huawei Cloud.

Migrating Data Between Elasticsearch Clusters Using Huawei Cloud Logstash

Backup and restoration

• Migrate data from a third-party Elasticsearch cluster of an earlier version to a Huawei Cloud Elasticsearch cluster of a later version.

• Migrate data from multiple third-party Elasticsearch clusters to a single Huawei Cloud Elasticsearch cluster to form a unified data platform.

• The version of the destination cluster must not be earlier than that of the source cluster. For details, see Snapshot version compatibility.
• This method does not support incremental data synchronization. You need to pause data update before starting to back up data.
• Snapshots can be migrated only when public access is enabled for third-party storage repositories.

Migrating Data from a Third-Party Elasticsearch Cluster to Huawei Cloud Using Backup and Restoration

Reindex API

Migrate data from multiple third-party Elasticsearch clusters to a single Huawei Cloud Elasticsearch cluster to form a unified data platform.

During cluster migration, do not add, delete, or modify indexes in the source cluster, or the source and destination clusters will have inconsistent data after the migration.

Migrating Data Between Elasticsearch Clusters Using the Reindex API

ESM

• Migrate data from a third-party Elasticsearch cluster of an earlier version to a Huawei Cloud Elasticsearch cluster of a later version.
• Migrate data from multiple third-party Elasticsearch clusters to a single Huawei Cloud Elasticsearch cluster to form a unified data platform.

During cluster migration, do not add, delete, or modify indexes in the source cluster, or the source and destination clusters will have inconsistent data after the migration.

Migrating Data Between Elasticsearch Clusters Using ESM

Elasticdump

Migrate small volumes of data and a limited number of indexes. For large-scale migrations, consider other more suitable methods.

During cluster migration, do not add, delete, or modify indexes in the source cluster, or the source and destination clusters will have inconsistent data after the migration.

Migrating Data Between Elasticsearch Clusters Using Elasticdump

CDM

This is a cloud migration tool provided by Huawei Cloud to migrate data between clusters that belong to different cloud services.

• A VPN or Direct Connect connection needs to be established between the customer's IDC and Huawei Cloud.
• During cluster migration, do not delete any of the source cluster's indexes, or something might go wrong.

-