Ingestion Center Overview
The Ingestion Center is a centralized hub for configuring LTS log sources. It supports seamless ingestion from business and component layers, self-built middleware, runtime environments, cloud services, APIs, protocols, and others.
Accessing the Ingestion Center
- Log in to the LTS console.
- In the navigation pane, choose Log Ingestion > Ingestion Center.
- Use the search and filter tools to locate the specific service type from which you want to collect logs.
- Filter: Narrow down log sources by type.
- Property: Click the search box and choose Keyword or Types as your search criteria. You can also enter a keyword for rapid identification.
- Recommended: This area features six cards. By default, it displays popular log sources. Over time, it automatically adapts to show your recently used sources for faster access.
- Hover your cursor over the card of a log resource to reveal the following options:
- Ingest Logs (LTS): Configure log ingestion to LTS as prompted.
- Documentation: View the specific ingestion guide for that log source.
Figure 1 Ingestion cards
Businesses
| Card | Description |
|---|---|
| Web & HTML5 | Obtain metrics, traces, and logs of web & HTML5 apps. For details, see LTS Web SDK. |
| Android App | Obtain metrics, traces, and logs of Android apps. For details, see LTS Android SDK. |
| iOS App | Obtain metrics, traces, and logs of iOS apps. For details, see LTS iOS SDK. |
| WeChat Applet | Obtain metrics, traces, and logs of WeChat applets. For details, see LTS WeChat Applet SDK. |
| Alipay Applet | Obtain metrics, traces, and logs of Alipay applets. For details, see LTS Alipay Applet SDK. |
| DingTalk Applet | Obtain metrics, traces, and logs of DingTalk applets. For details, see LTS DingTalk Applet SDK. |
| Baidu Applet | Obtain metrics, traces, and logs of Baidu applets. For details, see LTS Baidu Applet SDK. |
| Quick App | Obtain metrics, traces, and logs of quick apps. For details, see LTS Quick App SDK. |
Components
| Card | Description |
|---|---|
| Java Component | Obtain metrics, traces, and logs of Java apps. For details, see LTS Java SDK. |
| GO Component | Obtain metrics, traces, and logs of Go apps. For details, see LTS Go SDK. |
| Python Component | Obtain metrics, traces, and logs of Python component apps. For details, see LTS Python SDK. |
| .NET Component | Obtain metrics, traces, and logs of .NET component apps. For details, see LTS Net SDK. |
| C++ Component | Obtain metrics, traces, and logs of C++ apps. For details, see LTS C++ SDK. |
Self-built Middleware
| Card | Description |
|---|---|
| MySQL | A stable, efficient relational database for managing large data volumes. It is widely used in website and app development. For details, see MySQL Logs. |
| Redis | An in-memory storage system for multiple data structure types. It is used as a database, cache, and message broker. For details, see Redis Logs. |
| Kafka | A distributed stream processing platform with high throughput and low latency. It is used for real-time data processing and log aggregation. For details, see Kafka Logs. |
| Nginx | A high-performance HTTP/reverse proxy server. It handles up to 50,000 concurrent requests with minimal memory usage and high stability. For details, see Nginx Logs. |
| Elasticsearch | A distributed full-text search engine with PB-level data storage. It is used for full-text search, analysis, and monitoring. For details, see Elasticsearch Logs. |
| ZooKeeper | A distributed coordination service with leader election, configuration management, and distributed locks to ensure data consistency. For details, see ZooKeeper Logs. |
| IIS | Microsoft web server software that offers high performance, stability, and scalability for HTTP and other protocols. For details, see IIS Logs. |
| DNS | Translates domain names into IP addresses and supports load balancing to speed up network access. For details, see DNS Logs. |
| Flink | A distributed real-time compute engine with bounded and unbounded data streams processing, efficient memory performance, and exactly-once semantics. For details, see Flink Logs. |
Runtime Environments
| Card | Description |
|---|---|
| Elastic Cloud Server (ECS) | Provides secure, scalable, on-demand compute resources with flexible billing modes and minimal O&M, greatly reducing your costs. For details, see Ingesting ECS Text Logs to LTS. |
| Bare Metal Server (BMS) | A high-performance, high-security physical server on the cloud. For details, see Ingesting BMS Text Logs to LTS. |
| Cloud Container Engine (CCE) | A high-performance, high-reliability service for managing containerized applications. It supports native Kubernetes apps and tools and makes it easy to establish a container runtime environment on the cloud. For details, see Ingesting CCE Application Logs to LTS. |
| Self-built Kubernetes Cluster | An open-source container orchestration system for automating deployment, scaling, and management of containerized applications, enhancing application reliability and scalability. For details, see Ingesting Self-Built Kubernetes Application Logs to LTS. |
Cloud Services
| Type | Supported Cloud Service | Description |
|---|---|---|
| Logs | Application Operations Management (AOM), API Gateway (APIG), Bare Metal Server (BMS), Cloud Container Engine (CCE), Content Delivery Network (CDN), Cloud Firewall (CFW), Cloud Trace Service (CTS), and others | LTS collects log data of many cloud services, such as compute, storage, security, and database services. You can use LTS to search cloud service logs by keyword, analyze operations data, and monitor the running status and alarms. For details, see Ingesting Cloud Service Logs to LTS. |
APIs/Protocols/Others
| Card | Description |
|---|---|
| LTS APIs | Use LTS open APIs to report log data. For details, see Using APIs to Ingest Logs to LTS. |
| Kafka Protocol | Ingest logs using the Kafka binary protocol, a high-throughput, TCP-based distributed messaging system. For details, see Using Kafka to Report Logs to LTS. |
| Syslog Protocol | Exchange logs between devices using standard UDP- or TCP-based Syslog protocols. For details, see Using Flume to Report Logs to LTS. |
| Flume | Collect and upload logs to LTS using the open-source tool Flume. For details, see Using Flume to Report Logs to LTS. |
| Beats | Collect and upload logs to LTS using the open-source tool Beats. For details, see Using Kafka to Report Logs to LTS. |
| Logstash | Collect and upload logs to LTS using the open-source tool Logstash. For details, see Using Kafka to Report Logs to LTS. |
| SNMP Protocol | Simple Network Management Protocol (SNMP) is a standard protocol for remotely managing and monitoring network devices. It has three versions: v1, v2, and v3. Via SNMP, Flume collects management data reported by network devices and forwards the data to LTS. For details, see Using Flume to Report Logs to LTS. |
| Logback SDK | Configure Huawei Cloud Appender in Logback and then report generated logs to LTS. For details, see LTS Logback SDK. |
| Cross-Account Ingestion - Log Stream Mapping | Create an agency to map log streams from a delegator account to a delegated account. For details, see Ingesting Logs to LTS Across IAM Accounts. |
What is your overall rating for this page?
Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot