Diese Seite ist in Ihrer lokalen Sprache noch nicht verfügbar. Wir arbeiten daran, weitere Sprachversionen hinzuzufügen. Vielen Dank für Ihre Unterstützung.
- What's New
- Function Overview
- Service Overview
- Getting Started
-
User Guide
- Getting Started
- Creating and Accessing a Cluster
- Scaling In/Out a Cluster
- Upgrading Versions
- Importing Data to Elasticsearch
-
Managing Elasticsearch Clusters
- Cluster and Storage Capacity Statuses
- Introduction to the Cluster List
- Index Backup and Restoration
- Binding an Enterprise Project
- Restarting a Cluster
- Migrating Cluster Data
- Deleting a Cluster
- Managing Tags
- Public Network Access
- Managing Logs
- Managing Plugins
- Hot and Cold Data Storage
- Configuring Parameters
- VPC Endpoint Service
- Kibana Public Access
- Vector Retrieval
- Working with Kibana
- Elasticsearch SQL
- Connecting a Cluster to a Dedicated Load Balancer
- Enhanced Features
- Monitoring
- Auditing
- Change History
- Best Practices
-
API Reference
- Before You Start
- API Overview
- Calling APIs
- Getting Started
-
Cluster Management
- Creating a cluster
- Querying the Cluster List
- Querying Cluster Details
- Deleting a Cluster
- Renaming a Cluster
- Changing the Password of a Cluster
- Restarting a Cluster
- Scaling Out a Cluster
- Adding Instances and Expanding Instance Storage Capacity
- Changing Specifications
- Obtaining the Instance Specifications List
- Querying All Tags
- Querying Tags of a Specified Cluster
- Adding Tags to a Cluster
- Deleting a Cluster Tag
- Adding or Deleting Cluster Tags in Batches
- Changing the Specifications of a Specified Node Type
- Scaling In a Cluster by Removing a Specific Node
- Scaling In Nodes of a Specific Type
- Downloading a Security Certificate
- Replacing a Node
- Configuring the Security Mode.
- Adding Independent Masters and Clients
- Changing the Security Group
- Kibana Public Network Access
- Log Management
- Public Network Access
-
Snapshot Management
- (Not Recommended) Automatically Setting Basic Configurations of a Cluster Snapshot
- Modifying Basic Configurations of a Cluster Snapshot
- Manually Creating a Snapshot
- Restoring a Snapshot
- Deleting a Snapshot
- Configuring the Automatic Snapshot Creation Policy
- Querying the Automatic Snapshot Creation Policy
- Querying a Snapshot List
- Disabling the Snapshot Function
- VPC Endpoint
- Parameter Configuration
- Common Parameters
- Change History
- SDK Reference
-
FAQs
- General Consulting
-
Accessing CSS Clusters
- How Do I Reset the Administrator Password of a Security-mode Cluster in CSS?
- Are Ports 9200 and 9300 Open for Access to Elasticsearch Clusters?
- How Do I Use a NAT Gateway to Access CSS from the Internet?
- How Do I Connect In-house Developed Kibana to an Elasticsearch Cluster in CSS?
- How Do I Connect In-house Developed OpenSearch Dashboards to an OpenSearch Cluster in CSS?
- Migrating CSS Clusters
-
Using CSS Cluster Search Engines
- Why Are Newly Created Index Shards Allocated to a Single Node in CSS?
- How Do I Create a Type Under an Index in an Elasticsearch 7.x Cluster of CSS?
- How Do I Change the Number of Replicas for Elasticsearch Indexes in CSS?
- What Are the Impacts If an Elasticsearch Cluster of CSS Has Too Many Shards?
- How Do I Check the Number of Shards and Replicas in a CSS Cluster?
- What Does the Value i for node.roles Mean for Nodes in an Elasticsearch Cluster of CSS?
- How Do I Change the Maximum Number of Results Returned for Searches to an Index in an Elasticsearch Cluster of CSS?
- How Do I Update Index Lifecycle Policies for an Elasticsearch Cluster of CSS?
- How Do I Set Slow Query Log Thresholds for an Elasticsearch Cluster of CSS?
- How Do I Clear Elasticsearch Indexes in CSS?
- How Do I Clear Elasticsearch Cache in CSS?
- Why Does the Disk Usage Increase After the delete_by_query Command Was Executed to Delete Data in an Elasticsearch Cluster?
- Do CSS Elasticsearch Clusters Support script dotProduct?
-
Managing CSS Clusters
- How Do I Check the AZ Where a CSS Cluster Is Located?
- What Is the Relationship Between the Filebeat Version and Cluster Version in CSS?
- How Do I Obtain the Security Certificate of CSS?
- How Do I Convert the Format of a CER Security Certificate in CSS?
- Can I Modify the Security Group for Elasticsearch and OpenSearch Clusters in CSS?
- How Do I Set search.max_buckets for an Elasticsearch Cluster of CSS?
- Can I Modify the TLS Algorithm of an Elasticsearch or OpenSearch Cluster in CSS?
- How Do I Enable Audit Logs for an Elasticsearch or OpenSearch Cluster of CSS?
- Can I Stop a CSS Cluster?
- How Do I Query the Index Size on OBS After the Freezing of Indexes for a CSS Cluster?
- How Do I Check the List of Default Plugins for Elasticsearch and OpenSearch Clusters?
- CSS Cluster Backup and Restoration
- CSS Cluster Monitoring and O&M
- Troubleshooting
- Videos
Show all
Migrating Cluster Data Using Logstash
Logstash is an official data migration tool provided by Elasticsearch.
- Apply for an ECS, preferably with at least 8 vCPUs and 16 GB memory.
- Install Logstash on the ECS.
- Install JDK, because Logstash depends on Java. Run the following command to install JDK using yum:
yum install java yum install python
- Download Logstash. Choose a Logstash version close to the Elasticsearch version. They do not have to use exactly the same version.
Logstash 7.10.2 OSS is recommended. You can download it from https://www.elastic.co/downloads/past-releases/logstash-oss-7-10-2
- Run the following command to install Logstash using yum:
yum install logstash-oss-7.10.0-x86_64.rpm
Replace logstash-oss-7.10.0-x86_64.rpm with the actual Logstash installation package name.
- Install JDK, because Logstash depends on Java. Run the following command to install JDK using yum:
- Modify the JVM configuration of Logstash to improve the cluster data migration efficiency.
Run the following command to modify the JVM configuration. The default heap memory of Logstash is 1 GB. You are advised to change the heap memory to half of the cluster node memory.
vim /etc/logstash/jvm.options -Xms4g -Xmx4g
- Modify the conf configuration file of Logstash and configure cluster migration settings.
- Go to the /etc/logstash/conf.d/ directory where the Logstash configuration file is stored.
cd /etc/logstash/conf.d/
- Create the logstash-es-es-all.conf file.
vim logstash-es-es-all.conf
- Add the following content to the logstash-es-es-all.conf file and save the file.
Modify the hosts, user, password, index fields as needed.
input{ elasticsearch{ #IP address of the source cluster hosts => ["http://172.16.xxx.xxx:9200", "http://172.16.xxx.xxx:9200"] # #For a security cluster, configure the username and password for cluster login. For a non-security cluster, you can use the number sign (#) to comment out the user and password fields. # user => "xxxx" # password => "xxxx" # #List of indexes to be migrated. Multiple indexes are separated by commas (,). Set this parameter based on the actual host information. -.* indicates that indexes starting with a period (.) are excluded. index => "abmau_edi*,business_test,goods_deploy*, -.*" # Retain the default values of the following three items, including the number of threads, the size of migrated data, and Logstash JVM configurations. docinfo=>true # Retain the default value. To increase the migration speed, you can increase the values of the following two parameters, but to a proper extent. slices => 3 size => 3000 } } filter { # Delete some fields added by Logstash. mutate { remove_field => ["@timestamp", "@version"] } } output{ elasticsearch{ # Destination cluster address. hosts => ["http://10.100.xx.xx:9200", "http://10.100.xx.xx:9200"] # Username and password for logging in to the target cluster. If you do not need to configure them, use the number sign (#) to comment them out. user => "admin" password => "*****" # Index name of the target cluster. The following configurations must be the same as that of the source cluster. index => "%{[@metadata][_index]}" # #Index type of the target cluster. The following configurations must be the same as that of the source cluster. document_type => "%{[@metadata][_type]}" # _id of the target data. If the original _id does not need to be retained, you can delete it. After the deletion, the cluster performance can be better. document_id => "%{[@metadata][_id]}" ilm_enabled => false manage_template => false } # Debugging information. You are advised to delete this information before migration. # stdout { codec => rubydebug { metadata => true }} }
- Go to the /etc/logstash/conf.d/ directory where the Logstash configuration file is stored.
- Start Logstash to migrate cluster data.
- Run the following command to start Logstash:
/usr/share/logstash/bin/logstash --path.settings /etc/logstash
- View the Logstash log file to check the task progress. The Logstash log directory is /var/log/logstash/.
- Wait until the data migration is complete.
- Run the following command to start Logstash:
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.