El contenido no se encuentra disponible en el idioma seleccionado. Estamos trabajando continuamente para agregar más idiomas. Gracias por su apoyo.

Compute
Elastic Cloud Server
Huawei Cloud Flexus
Bare Metal Server
Auto Scaling
Image Management Service
Dedicated Host
FunctionGraph
Cloud Phone Host
Huawei Cloud EulerOS
Networking
Virtual Private Cloud
Elastic IP
Elastic Load Balance
NAT Gateway
Direct Connect
Virtual Private Network
VPC Endpoint
Cloud Connect
Enterprise Router
Enterprise Switch
Global Accelerator
Management & Governance
Cloud Eye
Identity and Access Management
Cloud Trace Service
Resource Formation Service
Tag Management Service
Log Tank Service
Config
OneAccess
Resource Access Manager
Simple Message Notification
Application Performance Management
Application Operations Management
Organizations
Optimization Advisor
IAM Identity Center
Cloud Operations Center
Resource Governance Center
Migration
Server Migration Service
Object Storage Migration Service
Cloud Data Migration
Migration Center
Cloud Ecosystem
KooGallery
Partner Center
User Support
My Account
Billing Center
Cost Center
Resource Center
Enterprise Management
Service Tickets
HUAWEI CLOUD (International) FAQs
ICP Filing
Support Plans
My Credentials
Customer Operation Capabilities
Partner Support Plans
Professional Services
Analytics
MapReduce Service
Data Lake Insight
CloudTable Service
Cloud Search Service
Data Lake Visualization
Data Ingestion Service
GaussDB(DWS)
DataArts Studio
Data Lake Factory
DataArts Lake Formation
IoT
IoT Device Access
Others
Product Pricing Details
System Permissions
Console Quick Start
Common FAQs
Instructions for Associating with a HUAWEI CLOUD Partner
Message Center
Security & Compliance
Security Technologies and Applications
Web Application Firewall
Host Security Service
Cloud Firewall
SecMaster
Anti-DDoS Service
Data Encryption Workshop
Database Security Service
Cloud Bastion Host
Data Security Center
Cloud Certificate Manager
Edge Security
Managed Threat Detection
Blockchain
Blockchain Service
Web3 Node Engine Service
Media Services
Media Processing Center
Video On Demand
Live
SparkRTC
MetaStudio
Storage
Object Storage Service
Elastic Volume Service
Cloud Backup and Recovery
Storage Disaster Recovery Service
Scalable File Service Turbo
Scalable File Service
Volume Backup Service
Cloud Server Backup Service
Data Express Service
Dedicated Distributed Storage Service
Containers
Cloud Container Engine
SoftWare Repository for Container
Application Service Mesh
Ubiquitous Cloud Native Service
Cloud Container Instance
Databases
Relational Database Service
Document Database Service
Data Admin Service
Data Replication Service
GeminiDB
GaussDB
Distributed Database Middleware
Database and Application Migration UGO
TaurusDB
Middleware
Distributed Cache Service
API Gateway
Distributed Message Service for Kafka
Distributed Message Service for RabbitMQ
Distributed Message Service for RocketMQ
Cloud Service Engine
Multi-Site High Availability Service
EventGrid
Dedicated Cloud
Dedicated Computing Cluster
Business Applications
Workspace
ROMA Connect
Message & SMS
Domain Name Service
Edge Data Center Management
Meeting
AI
Face Recognition Service
Graph Engine Service
Content Moderation
Image Recognition
Optical Character Recognition
ModelArts
ImageSearch
Conversational Bot Service
Speech Interaction Service
Huawei HiLens
Video Intelligent Analysis Service
Developer Tools
SDK Developer Guide
API Request Signing Guide
Terraform
Koo Command Line Interface
Content Delivery & Edge Computing
Content Delivery Network
Intelligent EdgeFabric
CloudPond
Intelligent EdgeCloud
Solutions
SAP Cloud
High Performance Computing
Developer Services
ServiceStage
CodeArts
CodeArts PerfTest
CodeArts Req
CodeArts Pipeline
CodeArts Build
CodeArts Deploy
CodeArts Artifact
CodeArts TestPlan
CodeArts Check
CodeArts Repo
Cloud Application Engine
MacroVerse aPaaS
KooMessage
KooPhone
KooDrive
On this page
Help Center/ Log Tank Service/ FAQs/ Consultation/ What Are the Advantages of LTS Compared with Self-built ELK Stack?

What Are the Advantages of LTS Compared with Self-built ELK Stack?

Updated on 2025-02-08 GMT+08:00

This section describes the main functions and advantages of Huawei Cloud LTS by comparing it with self-built ELK Stack.

Background

The open-source ELK Stack, comprising Elasticsearch, Logstash, and Kibana, is extensively used for log search, with a variety of content and use cases available within its community.

LTS is a fully managed log analysis platform that covers application O&M, security compliance, and service operations. You can use it to collect, store, query, process, analyze, and report logs with ease. For details, see Infographics.

Functions

LTS outperforms ELK in terms of feature completeness and log search and analysis performance.

Feature

Subfeature

LTS

ELK

Description

Log collection

Cloud service log collection

☆☆☆☆☆

N/A

ELK: does not collect cloud service logs.

LTS: collects all logs of the cloud service tenant plane.

VM and container log collection

☆☆☆☆☆

☆☆☆☆

ELK: uses open-source collectors such as Logstash or Filebeat.

LTS: uses ICAgent to collect logs and provides easy-to-use wizard pages.

Collection via multi-language SDKs

☆☆☆

N/A

ELK: not supported.

LTS: provides a Java SDK to directly report logs to LTS.

Host group management (dynamic scaling of hosts)

☆☆☆☆☆

N/A

ELK: not supported.

LTS: supports host and host group management. You can add custom identifiers to host groups and scale host groups in or out.

Log structuring parsing

☆☆☆☆

☆☆☆☆☆

ELK: enables custom structuring parsing based on the collectors.

LTS: enables structuring parsing with regular expressions, JSON, delimiters, or custom templates.

Log search

Keyword search, fuzz match, and quick analysis

☆☆☆☆☆

☆☆☆☆☆

ELK and LTS: provide similar keyword search functions.

Real-time log viewing

☆☆☆☆☆

N/A

ELK: does not provide the page for viewing real-time logs.

LTS: provides the page for viewing real-time logs.

Search of tens of billions of logs in seconds

☆☆☆☆☆

☆☆

ELK: Limited by the server resources, it takes a long time to search for massive logs.

LTS: With the extensive scalable computing resources of Huawei Cloud, search results can be returned in 3 seconds.

Iterative search of hundreds of billions of logs

☆☆☆☆☆

N/A

ELK: Response timeout occurs when hundreds of billions of logs are searched.

LTS: Iterative search enables search of hundreds of billions of logs.

Log management scale

100 PB level

100 TB level

ELK: It is often time-consuming to keep an eye on server scaling.

LTS: automatically manages 100 PB of logs. You do not need to worry about the underlying resource consumption and will be charged on a pay-per-use basis.

Log search

SQL analysis

☆☆☆☆☆

☆☆

ELK: does not support nested SQL statements in syntax due to poor SQL performance.

LTS: provides high SQL performance and supports nested SQL statements.

Log search

SQL functions

☆☆☆☆☆

☆☆

ELK: supports only basic SQL statistics functions.

LTS: Besides basic SQL functions, LTS offers various extended functions, such as IP, statistics, chain and parallel comparison, and URL functions, to support more scenarios.

Log search

Charts

☆☆☆☆

☆☆☆

LTS: provides various visual charts, such as tables, and line, pie, and bar charts.

Log search

Dashboards

☆☆☆☆☆

☆☆

ELK: There is no ready-to-use dashboard for cloud service logs.

LTS: provides ready-to-use dashboards for common cloud services, such as ELB, APIG, Document Database Service (DDS), DCS, and Cloud Firewall (CFW).

Log alarms

Keyword and SQL alarms

☆☆☆☆☆

ELK: No log alarm function is available.

LTS: Quasi-real-time log keyword and SQL alarms are available.

Alarm notification channels (such as email, SMS, and HTTPS)

☆☆☆☆☆

ELK: does not send alarms to users through DingTalk, WeCom, or SMS messages.

LTS: interconnects with Huawei Cloud Simple Message Notification (SMN) to notify users through channels such as email, SMS, WeCom, DingTalk, Lark, and HTTP.

Log transfer

Transfer to OBS

☆☆☆☆☆

N/A

ELK: cannot transfer logs to OBS directly.

LTS: allows you to transfer logs to OBS with simple page configurations.

Log transfer

Transfer to Kafka

☆☆☆☆☆

☆☆

ELK: requires you to deploy a program.

LTS: allows you to transfer logs to Kafka in real time with simple page configurations.

Log transfer

Transfer to data warehouses

☆☆☆☆☆

N/A

ELK: cannot transfer logs to data warehouses.

LTS: allows you to transfer logs to data warehouses with simple page configurations.

Log jobs

Scheduled SQL jobs

☆☆☆☆☆

N/A

ELK: does not support scheduled SQL jobs.

LTS: allows you to configure scheduled SQL jobs to convert raw logs into summarized results.

Log processing with functions

☆☆☆☆☆

N/A

ELK: does not support log processing.

LTS: provides function triggers. You can write custom scripts in FunctionGraph to process logs flexibly.

Costs

Scenario 1:

Assume that a total of 3,000 GB of raw logs are generated in 30 days (100 GB per day) at an average log rate of 1.16 MB/s, and those logs are retained for an average of 30 days in dual storage (primary and standby).

Elasticsearch recommends that the total storage space for raw logs, backup data, and index data in dual storage mode be about 2.2 times the size of raw logs. Considering the Elasticsearch cluster's uneven write distribution and partial disk utilization, storing 3,000 GB of raw logs needs at least 13,200 GB of disk space, calculated as 3,000 GB x 2.2 (storage expansion) x 2 (allowing for 50% disk redundancy).

Typically at least three ECSs (16 vCPUs, 64 GB memory, and 5 TB capacity) and two Kafka replicas are required for Elasticsearch to store logs of the past 12 hours.

Table 1 Self-built ELK

Category

Subcategory

Monthly Cost (Total: $1,764 USD)

Expense Proportion

Setting up Elasticsearch

3 Elastic Cloud Servers (ECSs) (C6 16 vCPUs | 64 GB)

3 x 1,999 x 0.1401 = $840 USD

47.6%

Elastic Volume Service (EVS) (high I/O 15 TB)

0.35 x 15 x 1,024 x 0.1401= $753 USD

42.7%

Setting up Kafka

3 ECSs (2 vCPUs | 4 GB)

3 x 208 x 0.1401 = $87 USD

4.9%

EVS (ultra-high I/O 3 x 200 GB)

600 x 0.1401 = $84 USD

4.7%

According to the price calculator, the monthly cost of LTS is around $539.89 USD, just 16.7% of the cost of self-built ELK. This significant saving is attributed to LTS's pay-per-use billing mode, contrasting with the high initial resource cost of ELK in scenarios with few logs.

Scenario 2:

Assume that a total of 7 TB of raw logs are generated in 7 days (1 TB per day) at an average log rate of 11.6 MB/s, and those logs are retained for an average of 30 days in dual storage (primary and standby). Elasticsearch recommends that the total storage space for raw logs, backup data, and index data in dual storage mode be about 2.2 times the size of raw logs. Considering the Elasticsearch cluster's uneven write distribution and partial disk utilization, storing 7 TB of raw logs needs at least 31 TB of disk space, calculated as 7 TB x 2.2 (storage expansion) x 2 (allowing for 50% disk redundancy).

Typically at least three ECSs (16 vCPUs, 64 GB memory, and 10 TB capacity) and two Kafka replicas are required for Elasticsearch to store logs of the past 12 hours.

Table 2 Self-built ELK

Category

Subcategory

Monthly Cost (Total: $2,652 USD)

Expense Proportion

Setting up Elasticsearch

3 ECSs (C6 16 vCPUs | 64 GB)

3 x 1,999 x 0.1401 = $840 USD

31.7%

EVS (high I/O 31 TB)

0.35 x 31 x 1,024 x 0.1401= $1,557 USD

58.7%

Setting up Kafka

3 ECSs (2 vCPUs | 4 GB)

3 x 208 x 0.1401 = $87 USD

3.3%

EVS (ultra-high I/O 3 x 400 GB)

1,200 x 0.1401 = $168 USD

6.3%

According to the price calculator, the monthly cost of LTS is around $3,409.92 USD, just 71% of the cost of self-built ELK. This significant saving is attributed to LTS's pay-per-use billing mode, contrasting with the extensive disk requirements for maintaining smooth running in self-built ELK clusters.

Scenario 3:

Assume that a total of 150 TB of raw logs are generated in 30 days (5 TB per day) at an average log rate of 58 MB/s, and those logs are retained for an average of 30 days in dual storage (primary and standby).

Elasticsearch recommends that the total storage space for raw logs, backup data, and index data in dual storage mode be about 2.2 times the size of raw logs. Considering the Elasticsearch cluster's uneven write distribution and partial disk utilization, storing 150 TB of raw logs needs at least 660 TB of disk space, calculated as 150 TB x 2.2 (storage expansion) x 2 (allowing for 50% disk redundancy).

Typically at least 66 ECSs (16 vCPUs, 64 GB memory, and 10 TB capacity) and two Kafka replicas are required for Elasticsearch to store logs of the past 12 hours.

Table 3 Self-built ELK

Category

Subcategory

Monthly Cost (Total: $52,440 USD)

Expense Proportion

Setting up Elasticsearch

66 ECSs (C6 16 vCPUs | 64 GB)

66 x 1,999 x 0.1401 = $18,489 USD

35.3%

EVS (high I/O 660 TB)

0.35 x 660 x 1,024 x 0.1401 = $33,149 USD

63.2%

Setting up Kafka

3 ECSs (2 vCPUs | 4 GB)

3 x 208 x 0.1401 = $87 USD

0.2%

EVS (ultra-high I/O 3 x 1,700 GB)

5,100 x 0.1401 = $715 USD

1.4%

According to the price calculator, the monthly cost of LTS is around $27,648 USD, just 28.8% of the cost of self-built ELK. This significant saving is attributed to LTS's pay-per-use billing mode, contrasting with the extensive disk requirements for maintaining smooth running in self-built ELK clusters.

Summary

LTS beats ELK in functions, performance, and costs. You are advised to use fully managed LTS instead of self-built ELK.

Utilizamos cookies para mejorar nuestro sitio y tu experiencia. Al continuar navegando en nuestro sitio, tú aceptas nuestra política de cookies. Descubre más

Feedback

Feedback

Feedback

0/500

Selected Content

Submit selected content with the feedback