- What's New
- Function Overview
- Service Overview
- Getting Started
-
User Guide
- Getting Started
- Creating and Accessing a Cluster
- Scaling In/Out a Cluster
- Upgrading Versions
- Importing Data to Elasticsearch
-
Managing Elasticsearch Clusters
- Cluster and Storage Capacity Statuses
- Introduction to the Cluster List
- Index Backup and Restoration
- Binding an Enterprise Project
- Restarting a Cluster
- Migrating Cluster Data
- Deleting a Cluster
- Managing Tags
- Public Network Access
- Managing Logs
- Managing Plugins
- Hot and Cold Data Storage
- Configuring Parameters
- VPC Endpoint Service
- Kibana Public Access
- Vector Retrieval
- Working with Kibana
- Elasticsearch SQL
- Connecting a Cluster to a Dedicated Load Balancer
- Enhanced Features
- Monitoring
- Auditing
- Change History
- Best Practices
-
API Reference
- Before You Start
- API Overview
- Calling APIs
- Getting Started
-
Cluster Management
- Creating a cluster
- Querying the Cluster List
- Querying Cluster Details
- Deleting a Cluster
- Renaming a Cluster
- Changing the Password of a Cluster
- Restarting a Cluster
- Scaling Out a Cluster
- Adding Instances and Expanding Instance Storage Capacity
- Changing Specifications
- Obtaining the Instance Specifications List
- Querying All Tags
- Querying Tags of a Specified Cluster
- Adding Tags to a Cluster
- Deleting a Cluster Tag
- Adding or Deleting Cluster Tags in Batches
- Changing the Specifications of a Specified Node Type
- Scaling In a Cluster by Removing a Specific Node
- Scaling In Nodes of a Specific Type
- Downloading a Security Certificate
- Replacing a Node
- Configuring the Security Mode.
- Adding Independent Masters and Clients
- Changing the Security Group
- Kibana Public Network Access
- Log Management
- Public Network Access
-
Snapshot Management
- (Not Recommended) Automatically Setting Basic Configurations of a Cluster Snapshot
- Modifying Basic Configurations of a Cluster Snapshot
- Manually Creating a Snapshot
- Restoring a Snapshot
- Deleting a Snapshot
- Configuring the Automatic Snapshot Creation Policy
- Querying the Automatic Snapshot Creation Policy
- Querying a Snapshot List
- Disabling the Snapshot Function
- VPC Endpoint
- Parameter Configuration
- Common Parameters
- Change History
- SDK Reference
-
FAQs
- General Consulting
-
Accessing CSS Clusters
- How Do I Reset the Administrator Password of a Security-mode Cluster in CSS?
- Are Ports 9200 and 9300 Open for Access to Elasticsearch Clusters?
- How Do I Use a NAT Gateway to Access CSS from the Internet?
- How Do I Connect In-house Developed Kibana to an Elasticsearch Cluster in CSS?
- How Do I Connect In-house Developed OpenSearch Dashboards to an OpenSearch Cluster in CSS?
- Migrating CSS Clusters
-
Using CSS Cluster Search Engines
- Why Are Newly Created Index Shards Allocated to a Single Node in CSS?
- How Do I Create a Type Under an Index in an Elasticsearch 7.x Cluster of CSS?
- How Do I Change the Number of Replicas for Elasticsearch Indexes in CSS?
- What Are the Impacts If an Elasticsearch Cluster of CSS Has Too Many Shards?
- How Do I Check the Number of Shards and Replicas in a CSS Cluster?
- What Does the Value i for node.roles Mean for Nodes in an Elasticsearch Cluster of CSS?
- How Do I Change the Maximum Number of Results Returned for Searches to an Index in an Elasticsearch Cluster of CSS?
- How Do I Update Index Lifecycle Policies for an Elasticsearch Cluster of CSS?
- How Do I Set Slow Query Log Thresholds for an Elasticsearch Cluster of CSS?
- How Do I Clear Elasticsearch Indexes in CSS?
- How Do I Clear Elasticsearch Cache in CSS?
- Why Does the Disk Usage Increase After the delete_by_query Command Was Executed to Delete Data in an Elasticsearch Cluster?
- Do CSS Elasticsearch Clusters Support script dotProduct?
-
Managing CSS Clusters
- How Do I Check the AZ Where a CSS Cluster Is Located?
- What Is the Relationship Between the Filebeat Version and Cluster Version in CSS?
- How Do I Obtain the Security Certificate of CSS?
- How Do I Convert the Format of a CER Security Certificate in CSS?
- Can I Modify the Security Group for Elasticsearch and OpenSearch Clusters in CSS?
- How Do I Set search.max_buckets for an Elasticsearch Cluster of CSS?
- Can I Modify the TLS Algorithm of an Elasticsearch or OpenSearch Cluster in CSS?
- How Do I Enable Audit Logs for an Elasticsearch or OpenSearch Cluster of CSS?
- Can I Stop a CSS Cluster?
- How Do I Query the Index Size on OBS After the Freezing of Indexes for a CSS Cluster?
- How Do I Check the List of Default Plugins for Elasticsearch and OpenSearch Clusters?
- CSS Cluster Backup and Restoration
- CSS Cluster Monitoring and O&M
- Troubleshooting
- Videos
How Can I Improve Filebeat Performance?
Symptom
Filebeat is a high-performance file collection tool. By default, one core is allocated to Filebeat, and it writes 1 MB data to Elasticsearch per second. However, in practice, when a large number of service logs are generated, Filebeat cannot promptly collect and write them to Elasticsearch. In this case, you can optimize parameter settings in the filebeat.yml file to improve the Filebeat performance.
Procedure
- Optimize the parameters involved in input of the filebeat.yml configuration file.
Increase the value of harvester_buffer_size based on actual requirements. This parameter defines the buffer size used by every harvester.
harvester_buffer_size: 40,960,000
Increase the value of filebeat.spool_size based on actual requirements. This parameter defines the number of log records that can be uploaded by the spooler at a time.
filebeat.spool_size: 250,000
Adjust the value of filebeat.idle_timeout based on actual requirements. This parameter defines how often the spooler is flushed. After the idle_timeout is reached, the spooler is flushed regardless of whether the spool_size has been reached.
filebeat.idle_timeout: 1s
- Optimize the parameters involved in output.elasticsearch in the filebeat.yml configuration file.
Set the value of worker to the number of Elasticsearch clusters based on actual requirements. This parameter indicates the number of Elasticsearch clusters. The default value is 1.
worker: 1
Increase the value of bulk_max_size based on actual requirements. This parameter defines the maximum number of events to bulk in a single Elasticsearch bulk API index request. The default is 50.
bulk_max_size: 15,000
Adjust the value of flush_interval based on the actual requirements. This parameter defines the number of seconds to wait for new events between two bulk API index requests. If bulk_max_size is reached before this interval expires, additional bulk index requests are made.
flush_interval: 1s
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.