What's New

1

Added best practices for Elasticsearch data migration

Added best practices for migrating data between Elasticsearch clusters using Reindex API and Elasticsearch Migration Tool (ESM).

Commercial use

Migrating Data Between Elasticsearch Clusters Using the Reindex API

Migrating Data Between Elasticsearch Clusters Using ESM

2

Archive storage support by Elasticsearch clusters that have enabled storage-compute decoupling

Elasticsearch clusters with decoupled storage and compute support the change of data storage classes in OBS. Changing the storage class from standard to archive in OBS helps cut the cost of storing historical data.

Commercial use

Configuring Storage-Compute Decoupling for an Elasticsearch Cluster

Decoupling Index Storage and Compute in an Elasticsearch Cluster Through Index Lifecycle Management

3

Added monitoring metrics for vector indexes

Metrics that monitor the off-heap memory usage/utilization of vector indexes are added for Elasticsearch and OpenSearch clusters. They help quickly identify and locate issues and improve diagnosis efficiency.

Commercial use

Monitoring Metrics for Elasticsearch Clusters in Cloud Eye

Monitoring Metrics for OpenSearch Clusters in Cloud Eye

4

Support for pre-filtering query by Elasticsearch

Elasticsearch clusters support pre-filtering queries on vector indexes. Pre-filtering query works like this: First, filters are applied to indexes to generate an intermediate result set. Then, vector similarity-based search is performed on these results to find the most relevant vectors. This improves search efficiency and accuracy.

Commercial use

Using Vector Indexes for Data Search in an Elasticsearch Cluster

5

Added description about the performance impact of Elasticsearch data migration

The impact of data migration on the performance of the source Elasticsearch cluster is described, helping users plan the data before and after the migration to minimize this impact and ensure service continuity.

Commercial use

Migrating Data Between Elasticsearch Clusters Using Huawei Cloud Logstash

1

Load balancers and VPC Endpoint Service support IPv6 networks

Elasticsearch and OpenSearch clusters in CSS can be accessed via IPv6 addresses through a dedicated load balancer or VPC endpoint.

Commercial use

Configuring VPC Endpoint Service for an Elasticsearch Cluster

Configuring a Dedicated Load Balancer for an Elasticsearch Cluster

Configuring VPC Endpoint Service for an OpenSearch Cluster

Configuring a Dedicated Load Balancer for an OpenSearch Cluster

2

Support for vector search by OpenSearch

Added the operation guide and code examples for using vector search with an OpenSearch cluster.

Commercial use

Configuring Vector Search for OpenSearch Clusters

3

Node-level disk usage monitoring

The disk usage metric has been added for Elasticsearch and OpenSearch clusters.

Commercial use

Monitoring Metrics for Elasticsearch Clusters in Cloud Eye

Monitoring Metrics for OpenSearch Clusters in Cloud Eye

4

Disk I/O monitoring

Eight metrics for I/O monitoring have been added for Elasticsearch and OpenSearch clusters.

Commercial use

Monitoring Metrics for Elasticsearch Clusters in Cloud Eye

Monitoring Metrics for OpenSearch Clusters in Cloud Eye

5

Accessing clusters through LDAP

Enable Lightweight Directory Access Protocol (LDAP) authentication for an Elasticsearch or OpenSearch cluster in CSS to allow access by LDAP users of specific roles.

Commercial use

Accessing an Elasticsearch Cluster Using LDAP

Accessing an OpenSearch Cluster Using LDAP

6

Index lifecycle management for OpenSearch clusters

For an OpenSearch cluster, index lifecycle policies can be configured to decouple storage and computation for index data.

Commercial use

Decoupling Index Storage and Compute Through Index Lifecycle Management

7

Minimizing permissions for IAM agencies

Permissions can be minimized for IAM agencies automatically created for CSS. If there is an IAM agency automatically created earlier, you can also click One-click authorization to minimize its permissions.

Commercial use

All operation documents that contain steps about agency configuration

8

Modified the tag details page

The cluster tag details page is redesigned, so the operation guide for tag management is updated accordingly.

Commercial use

Setting Tags for an Elasticsearch Cluster

Setting Tags for an OpenSearch Cluster

Setting Tags for a Logstash Cluster

9

Adapting CSS to the latest CTS specifications

Updated the list of key operations that can be audited through CTS based on the latest CTS specifications.

Commercial use

Viewing Elasticsearch Cluster Audit Logs

Viewing OpenSearch Cluster Audit Logs

Viewing Logstash Cluster Audit Logs

10

Added performance testing data for enhanced aggregation

Added performance testing data for the enhanced aggregation function for Elasticsearch clusters.

Commercial use

Configuring Enhanced Aggregation for an Elasticsearch Cluster

11

CSS Dashboard displays monitoring metrics

On the Dashboard page of the CSS console, you can check monitoring metrics and configure alarm rules.

Commercial use

CSS Resource Monitoring

12

Read/write splitting enables automatic metadata synchronization

The read/write splitting function enables automatic metadata synchronization for Elasticsearch clusters.

Commercial use

Configuring Read/Write Splitting Between Two Elasticsearch Clusters

13

Disk type change

Change the node storage type of an Elasticsearch or OpenSearch cluster by node type. To do that, you need to delete old disks when rebuilding a node, and then attach disks of the new type.

Commercial use

Changing the Node Specifications of an Elasticsearch Cluster

Changing the Node Specifications of an OpenSearch Cluster

1

Optimized FAQs

Made the documents easier to understand and more searchable.

• Optimized topic titles to describe issues from the perspective of users.

• Retired questions that are no longer relevant and rarely visited.

Commercial use

FAQs

2

Updated User Guide

Made the documents easier to understand and more searchable.

• Rearranged the table of contents to make information more intuitive and searchable.

• Included scenario names in topic titles so that users can find needed information more quickly.

• Put documents for the same task together to offer all information needed.

• Added application scenarios and use constraints in operation guides.

Commercial use

Procedure for Using Elasticsearch

Procedure for Using OpenSearch

Procedure for Using Logstash

3

Optimization of Best Practices

Made the documents easier to understand and more searchable.

• Described best practices in terms of application scenario, solution architecture, and advantages.

• Enriched best practices on cluster migration.

Commercial use

Elasticsearch Cluster Migration

1

Optimized the Getting Started guide

Made the documents easier to understand and more searchable.

• Added scenario-specific cases for OpenSearch and Logstash clusters to help beginners get started quickly.

• Optimized topic titles to better identify specific tasks.

Commercial use

Using OpenSearch for Data Search

Using Logstash for Data Migration

2

OBS cold storage monitoring in the case of storage-compute decoupling

When storage-compute decoupling is enabled, the real-time OBS rates are recorded in the system index.

Commercial use

Configuring Storage-Compute Decoupling for an Elasticsearch Cluster

1

Pipeline monitoring for Logstash clusters

Logstash clusters in CSS support pipeline-level event monitoring. Alarm can be customized based on pipeline monitoring metrics.

Commercial use

Monitoring Metrics for Logstash Clusters in Cloud Eye

2

Real-time log collection

The real-time logs of an Elasticsearch or OpenSearch cluster can be stored within that cluster, facilitating log search and analysis using Kibana.

Commercial use

Querying and Managing Elasticsearch Cluster Logs

Querying and Managing OpenSearch Cluster Logs

3

Modifying preset word dictionaries

The preset Static Main Word Dictionary, Static Stop Word Dictionary, Extra Main Word Dictionary, and Extra Stop Word Dictionary can be added, deleted, and modified for Elasticsearch and OpenSearch clusters.

Commercial use

Configuring Custom Word Dictionaries for an Elasticsearch Cluster

Configuring Custom Word Dictionaries for an OpenSearch Cluster

4

Showing the causes of log backup failures

On the log backup page of CSS, information about failed log backup tasks and failure causes are displayed, facilitating troubleshooting.

Commercial use

Querying and Managing Elasticsearch Cluster Logs

Querying and Managing OpenSearch Cluster Logs

Querying and Managing Logstash Cluster Logs

5

Index recycle bin for Elasticsearch clusters

To allow users to restore data after misdeletion, CSS provides an index recycle bin for Elasticsearch clusters. Similar to other recycle bins, the index recycle bin temporarily stores deleted indexes so that users can restore them before they are finally removed from the recycle bin. This helps to improve cluster data reliability.

Commercial use

Configuring an Index Recycle Bin for an Elasticsearch Cluster

1

Cluster status check during cluster specification modification

When the specifications of a cluster are changed, the cluster status can be checked. The specifications of nodes are changed one by one to ensure success and data security.

Commercial use

Changing the Node Specifications

2

Cross-engine upgrade of Elasticsearch clusters

Elasticsearch clusters of version 7.10.2 can be upgraded to OpenSearch clusters of version 1.3.6.

Commercial use

Upgrading the Cluster Version

1

Support for general AC computing-plus ECSs

CSS supports general AC computing-plus ECSs equipped with scalable processors, providing enhanced computing, security, stability, and high cost-effectiveness.

Commercial use

Cloud Search Service Price Calculator

1

Selecting a shared VPC during cluster creation

A VPC is required when you create a cluster. You can select the name and ID of the VPC shared with the current account.

Commercial use

Creating an Elasticsearch Cluster

Creating a Logstash Cluster

Creating an OpenSearch Cluster

2

Opensearch Alert Plugin

By default, CSS has installed the open-source alert plugin opensearch-alerting for OpenSearch clusters to send notifications when data meets specific conditions.

Commercial use

Using the OpenSearch Alert Plugin

1

OpenSearch clusters

You can create and manage OpenSearch clusters in CSS.

Commercial use

Creating an OpenSearch Cluster

2

Auto-renewal for yearly/monthly clusters

For yearly/monthly clusters with auto-renewal enabled, you can modify the auto-renewal function.

Commercial use

Modifying Auto-Renewal

1

Batch export of Logstash cluster configuration files

The configuration files of Logstash clusters can be exported in batches.

Commercial use

Exporting Configuration Files

1

Migrating data before scaling in or replacing a specified node

You can select the Whether to perform data migration option to migrate data. If the target node have disabled indexes or indexes that have no replicas, this option must be selected.

Commercial use

Replacing Specified Nodes for an Elasticsearch Cluster

Scaling In an Elasticsearch Cluster

2

Querying clusters by enterprise project or private IP address

In the cluster list, you can quickly query clusters by enterprise project and private IP address.

Commercial use

Viewing Elasticsearch Cluster Information

3

Checking cluster load when upgrading an Elasticsearch cluster

If a cluster is overloaded, the upgrade task may suspend or fail. Cluster load check can effectively prevent upgrade task failures.

Commercial use

Creating an Upgrade Task

4

Logstash cluster force restart

If a Logstash cluster is faulty due to long-term running or other unknown reasons, you can forcibly restart the cluster to restore it.

Commercial use

Forcibly Restarting VMs in a Cluster

5

Extreme SSD storage

Elasticsearch and Logstash clusters support the extreme SSD storage.

Commercial use

Creating an Elasticsearch Cluster

Creating a Logstash Cluster

1

Intelligent O&M

CSS provides intelligent O&M to detect potential cluster risks and provide risk handling suggestions.

Commercial use

Intelligent Risk Detection for Elasticsearch Clusters

2

Enhanced aggregation

The enhanced aggregation is an optimization feature for service awareness. With this feature, you can optimize the aggregation analysis capability of observable services.

Commercial use

Enhanced Aggregation

3

Enhanced import performance

Developed by CSS, this feature effectively improves import performance and reduces write rejection by optimizing bulk routing, text index acceleration, and word segmentation acceleration.

Commercial use

Enhanced Import Performance

4

Read/Write splitting

CSS supports read/write splitting. Data written to the primary cluster (Leader) can be automatically synchronized to the secondary cluster (Follower). In this way, data is written to the primary cluster and queried in the secondary cluster.

Commercial use

Read/Write Splitting

5

Binding load balancers on the CSS management console

When interconnecting with a dedicated load balancer, you can bind a load balancer and add a listener on the CSS management console.

Commercial use

Connecting to a Dedicated Load Balancer

6

Logstash cluster

CSS supports Logstash clusters. You can modify the Logstash configuration files in the configuration center to import multiple data sources, such as edis, elasticsearch, and jdbc, to Elasticsearch clusters.

Commercial use

Logstash

7

Upgrading the service to version 6.8.23

CSS clusters of the 6.2.3 and 6.5.4 versions can be upgraded to the version 6.8.23.

Commercial use

Upgrading Versions

1

Node monitoring

You can query the monitoring metrics and alarms generated for CSS nodes on the Cloud Eye console or using CSS APIs.

Commercial use

Monitoring Metrics

2

Version upgrading

Same version upgrade and cross-version upgrade are supported. Same version upgrade is to upgrade the kernel patch of a cluster to fix problems or optimize performance. Cross-version upgrade is to upgrade the cluster version to enhance functions or incorporate versions.

Commercial use

Upgrading Versions

3

Supported the open-source OpenDistro alarm plug-in

By default, the open-source OpenDistro alarm plugin (opendistro_alerting) is integrated into CSS to send notifications when data meets specific conditions.

Commercial use

Using the OpenDistro Alarm Plugin

4

Supported switching between AZs

CSS supports cross-AZ deployment. You can add an AZ to obtain more resources or improve cluster availability, and can migrate your current AZ to one with higher specifications.

Commercial use

Changing AZs

1

Decreasing Cluster Specifications

You can decrease cluster specifications. After decreasing cluster specifications, the cluster performance will deteriorate and service capabilities will be affected. Exercise caution when performing this operation.

Commercial use

Changing the Node Specifications

2

Yearly/Monthly billing mode

CSS clusters can be purchased in yearly/monthly mode. This mode is recommended for long-term users.

Commercial use

Yearly/Monthly

3

OBS Operations Monitoring

The real-time OBS rate metric is added to CSS and recorded in the system index. You can view the usage of OBS in real time.

Commercial use

Monitoring OBS Operations

1

Improved Cold Data Query Performance

To accelerate data queries for the first time on the Discover page of Kibana, you can cache data locally.

Commercial use

Enhanced Cold Data Query Performance

2

Independent Master/Client Nodes

If the workloads on the data plane of a cluster increase, you can dynamically scale the cluster by adding master or client nodes.

Commercial use

Adding Master/Client Nodes

3

Added Traffic Control 2.0

After the client write traffic backpressure and control is enabled, large requests will be rejected when too much node heap memory is occupied. This function prevents nodes from breaking down and reduces the risk of node unavailability.

Commercial use

Flow Control 2.0

4

Connecting a Cluster to a Dedicated Load Balancer

You can connect a cluster to a dedicated load balancer. CSS integrates shared load balancers and allows you to bind public network access and enable the VPC Endpoint service. Dedicated load balancers provide more functions and higher performance than shared load balancers.

Commercial use

Connecting a Cluster to a Dedicated Load Balancer

5

Replacing a Specified Node

If a node in the cluster is faulty, you can create a new node with the same specifications to replace it.

Commercial use

Replacing a Specified Node

6

Changing the Security Mode in One-click

After a cluster is created, you can change its security mode in one-click.

Commercial use

Changing the Security Mode

7

Switching the Security Group of Elasticsearch Cluster

After an Elasticsearch cluster is created, you can modify its security group.

Commercial use

Viewing Cluster Information

1

Best Practices

This document summarizes common best practices for CSS. Operation details are described for each practice.

Commercial use

Best Practices

1

HTTP Status Code Monitoring

CSS allows you to monitor the HTTP status codes of clusters so that you can accurately view the actual status of calling Elasticsearch APIs.

Commercial use

HTTP Status Code Monitoring

2

P99 Latency Monitoring

CSS allows you to monitor the P99 latency of search requests in clusters to effectively reflect the actual search performance.

Commercial use

P99 Latency Monitoring

1

Vector retrieval

CSS supports k-nearest neighbor (KNN) and approximate nearest neighbor (ANN) vector search.

Commercial use

Feature Description

1

Storage-compute decoupling

CSS allows you to store historical data in OBS to reduce data storage costs.

Commercial use

Feature Description

1

Logstash cluster

Deleted documents related to Logstash clusters.

Open beta testing

What's New

1

Deleted the Poisson Analyzer

Deleted the description about the Poisson Analyzer in CSS.

Commercial use

What's New

2

Logstash cluster

Logstash clusters can be created in CSS.

Open beta testing

What's New

1

Added cluster version 7.9.3

Clusters in version 7.9.3 can be created in CSS.

Commercial use

Creating an Elasticsearch Cluster in Security Mode

1

Fine-grained authorization

CSS supports fine-grained authorization.

Commercial use

Creating a User and Granting Permissions

1

Kibana public access

For CSS clusters with security mode enabled, you can access Kibana through the Internet after configuring Kibana public access.

Commercial use

Kibana Public Access

1

Modifying a cluster name

You can change the cluster name when a cluster is created.

Commercial use

Creating an Elasticsearch Cluster in Security Mode

2

VPC endpoint service

CSS allows you to access clusters using private domain names.

Commercial use

VPC Endpoint Service

1

Managing plugins

CSS allows you to view default and custom plugins. You can upload, install, uninstall, or delete plugins as required.

Commercial use

How Do I Check the List of Default Plugins for Elasticsearch and OpenSearch Clusters?

1

Multi-AZ HA

CSS supported multi-AZ HA.

Commercial use

Deploying a Cross-AZ Cluster

1

Configuring parameters

CSS allows you to modify configurations in the elasticsearch.yml file to implement corresponding functions, such as cross-domain access and cluster reindexing.

Commercial use

Configuring Parameters

1

Managing logs

CSS provides log backup and query functions to help you locate faults. You can back up cluster logs to an OBS bucket or view node logs on the log query page.

Commercial use

Managing Logs

2

Hot and cold data storage

You can store data that requires query response in seconds on hot data nodes with high performance and store historical data that requires query response in minutes on cold data nodes with large capacity and low specifications.

Commercial use

Hot and Cold Data Storage

1

Modifying the node specifications

CSS allows you to modify the node specifications of clusters.

Commercial use

Scaling Out an Elasticsearch Cluster

2

Managing cluster tags

Adding tags to clusters can help you identify and manage your cluster resources.

Commercial use

Managing Tags

3

Public network access

You can configure the public network access for clusters with security mode enabled. In this case, you can access the cluster through the public IP address assigned by CSS.

Commercial use

Public Network Access

1

Clusters in security mode

You can enable security mode for clusters. After enabling it, identity verification, authorization, and encryption are required.

Commercial use

Elasticsearch Cluster Planning Suggestions

1

Binding enterprise projects

If you enable Enterprise Project Management, you can use it to manage cloud service resources on HUAWEI CLOUD.

Commercial use

Binding an Enterprise Project

1

Poisson analyzer

Poisson Analyzer is a plugin developed based on the open-source Jieba analyzer and provides both word segmentation and synonym extension functions. Compared with Jieba, this plugin optimizes the multi-granularity word segmentation function and enhances non-login word identification and word segmentation for Chinese-English hybrid phrases. In addition, the highlighting and phrase query functions are available for synonyms.

Commercial use

What's New

1

Changing the service name to Cloud Search Service

CSS is a fully managed, distributed search engine service. It is fully compatible with open-source Elasticsearch and provides you with structured and unstructured data search, statistics, and report capabilities. CSS works in the similar way as a database.

Commercial use

CSS Service Overview

1

Cluster backup and restoration

You can back up index data in clusters to avoid data loss. If data loss occurs or you want to retrieve data of a specified duration, you can restore the index data to obtain the data quickly.

Commercial use

Commercial use