- What's New
- Function Overview
- Service Overview
- Billing
- Getting Started
-
User Guide
- Granting LTS Permissions to IAM Users
- Purchasing LTS Resource Packages
- Log Management
-
Log Ingestion
- Overview
-
Using ICAgent to Collect Logs
- Overview
- Installing ICAgent (Intra-Region Hosts)
- Installing ICAgent (Extra-Region Hosts)
- Managing ICAgent
- Managing Host Groups
- Ingesting BMS Text Logs to LTS
- Ingesting CCE Application Logs to LTS
- Ingesting ECS Text Logs to LTS
- Ingesting ServiceStage Containerized Application Logs to LTS
- Ingesting ServiceStage Cloud Host Logs to LTS
- Ingesting Self-Built Kubernetes Application Logs to LTS
- Setting ICAgent Structuring Parsing Rules
-
Ingesting Cloud Service Logs to LTS
- Overview
- Ingesting AOM Logs to LTS
- Ingesting APIG Logs to LTS
- Ingesting CBH Logs to LTS
- Ingesting CFW Logs to LTS
- Ingesting CTS Logs to LTS
- Ingesting DDS Logs to LTS
- Ingesting DMS for Kafka Logs to LTS
- Ingesting DRS Logs to LTS
- Ingesting GaussDB(DWS) Logs to LTS
- Ingesting ELB Logs to LTS
- Ingesting Enterprise Router Logs to LTS
- Ingesting FunctionGraph Logs to LTS
- Ingesting GaussDB Logs to LTS
- Ingesting GES Logs to LTS
- Ingesting TaurusDB Logs to LTS
- Ingesting GeminiDB Logs to LTS
- Ingesting GeminiDB Mongo Logs to LTS
- Ingesting GeminiDB Cassandra Logs to LTS
- Ingesting IoTDA Logs to LTS
- Ingesting ModelArts Logs to LTS
- Ingesting MRS Logs to LTS
- Ingesting RDS for MySQL Logs to LTS
- Ingesting RDS for PostgreSQL Logs to LTS
- Ingesting RDS for SQL Server Logs to LTS
- Ingesting ROMA Connect Logs to LTS
- Ingesting SMN Logs to LTS
- Ingesting SecMaster Logs to LTS
- Ingesting OBS Files to LTS (Beta)
- Ingesting VPC Logs to LTS
- Ingesting WAF Logs to LTS
- Using APIs to Ingest Logs to LTS
- Ingesting Logs to LTS Across IAM Accounts
- Using Kafka to Report Logs to LTS
- Using Flume to Report Logs to LTS
- Log Search and Analysis
-
Log Visualization
- Overview
- Visualizing Logs in Statistical Charts
-
Visualizing Logs in Dashboards
- Creating a Dashboard
- Adding a Dashboard Filter
-
Dashboard Templates
- APIG Dashboard Templates
- CCE Dashboard Templates
- CDN Dashboard Templates
- CFW Dashboard Templates
- CSE Dashboard Templates
- DCS Dashboard Template
- DDS Dashboard Template
- DMS Dashboard Template
- DSL Dashboard Template
- ER Dashboard Template
- METRIC Dashboard Template
- Nginx Dashboard Templates
- VPC Dashboard Template
- WAF Dashboard Templates
- Log Alarms
- Log Transfer
- Log Processing
- Configuration Center
- Querying Real-Time LTS Traces
-
Best Practices
- Overview
-
Log Ingestion
- Collecting Logs from Third-Party Cloud Vendors, Internet Data Centers, and Other Huawei Cloud Regions to LTS
- Collecting Kubernetes Logs from Third-Party Clouds, IDCs, and Other Huawei Cloud Regions to LTS
- Collecting Syslog Aggregation Server Logs to LTS
- Importing Logs of Self-built ELK to LTS
- Using Flume to Report Logs to LTS
- Collecting Zabbix Data Through ECS Log Ingestion
- Collecting Logs from Multiple Channels to LTS
- Log Search and Analysis
- Log Transfer
- Billing
- Developer Guide
-
API Reference
- Before You Start
- API Overview
- Calling APIs
- API Calling Examples
- Examples
-
APIs
- Host Group Management
- Log Group Management
- Log Stream Management
- Log Management
- Log Ingestion
- Log Transfer
- Log Collection Beyond Free Quota
- Cloud Log Structuring
- Container Log Ingestion from AOM to LTS
- Alarm Topics
- Message Template Management
- SQL Alarm Rules
- Keyword Alarm Rules
- Alarm List
- Tag Management
- Dashboard Management
- Log Charts
- Quick Search
- Multi-Account Log Aggregation
- Permissions Policies and Supported Actions
- Appendix
- SDK Reference
-
FAQs
- Overview
- Consultation
- Log Management
-
Host Management
- What Do I Do If ICAgent Installation Fails in Windows and the Message "SERVICE STOP" Is Displayed?
- What Do I Do If ICAgent Upgrade Fails on the LTS Console?
- What Do I Do If I Could Not Query New Logs on LTS?
- What Do I Do If ICAgent Restarts Repeatedly After Being Installed?
- What Do I Do If ICAgent Is Displayed as Offline on the LTS Console After Installation?
- What Do I Do If I Do Not See a Host with ICAgent Installed on the LTS Console?
- How Do I Create a VPC Endpoint on the VPCEP Console?
- How Do I Obtain an AK/SK Pair?
- How Do I Install ICAgent by Creating an Agency?
-
Log Ingestion
- What Do I Do If LTS Cannot Collect Logs After I Configure Host Log Ingestion?
- Will LTS Stop Collecting Logs After the Free Quota Is Used Up If I Disable "Continue to Collect Logs When the Free Quota Is Exceeded" in AOM?
- What Do I Do If the CPU Usage Is High When ICAgent Is Collecting Logs?
- What Kinds of Logs and Files Does LTS Collect?
- How Do I Disable the Function of Collecting CCE Standard Output Logs to AOM on the LTS Console?
- What Log Rotation Scheme Should I Use for ICAgent to Collect Logs?
- Does LTS Use the Log4j Plug-in to Report Logs?
- How Long Does It Take to Generate Logs After Configuring Log Ingestion?
- What Do I Do If LTS Cannot Collect Logs After I Configure Log Ingestion with ICAgent?
- Log Search and Analysis
-
Log Transfer
- Does LTS Delete Logs That Have Been Transferred to OBS Buckets?
- What Are the Common Causes of LTS Log Transfer Abnormalities?
- How Do I Transfer CTS Logs to an OBS Bucket?
- What Do I Do If I Cannot View Historical Data in an OBS Bucket After Transferring Data from LTS to OBS?
- What Do I Do If I Cannot Find a New Partition in a DLI Table After Logs Are Transferred to DLI?
-
More Documents
- User Guide (ME-Abu Dhabi Region)
- API Reference (ME-Abu Dhabi Region)
- User Guide(Paris Regions)
- API Reference(Paris Regions)
- User Guide (Kuala Lumpur Region)
- API Reference (Kuala Lumpur Region)
- User Guide (Ankara Region)
-
API Reference (Ankara Region)
- Before You Start
- Calling APIs
- API Calling Examples
- APIs
- Permissions and Supported Actions
- Appendix
- Change History
- Videos
- General Reference
Copied.
Built-in Reserved Fields
During log collection, LTS adds information such as the collection time, log type, and host IP address to logs in the form of Key-Value pairs. These fields are built-in reserved fields of LTS.
- When using APIs to write log data or add ICAgent configurations, do not set field names to built-in reserved fields. Otherwise, problems such as duplicate field names and inaccurate query may occur.
- The name of a custom log field cannot contain double underscores (_). Otherwise, the index cannot be configured.
Log Example
In the following log example, the value of the content field is the original log text, and other fields are common built-in reserved fields.
{ "hostName":"epstest-xx518", "hostIP":"192.168.0.31", "clusterId":"c7f3f4a5-xxxx-11ed-a4ec-0255ac100b07", "pathFile":"stdout.log", "content":"level=error ts=2023-04-19T09:21:21.333895559Z", "podIp":"10.0.0.145", "containerName":"config-reloader", "clusterName":"epstest", "nameSpace":"monitoring", "hostIPv6":"", "collectTime":"1681896081334", "appName":"alertmanager-alertmanager", "hostId":"318c02fe-xxxx-4c91-b5bb-6923513b6c34", "lineNum":"1681896081333991900", "podName":"alertmanager-alertmanager-54d7xxxx-wnfsh", "__time__":"1681896081334", "serviceID":"cf5b453xxxad61d4c483b50da3fad5ad", "category":"LTS" }
Built-in Reserved Fields Description
Built-in Reserved Fields |
Number Format |
Index and Statistics Settings |
Description |
---|---|---|---|
collectTime |
Integer, Unix timestamp (ms) |
Index setting: After this function is enabled, a field index is created for collectTime by default. The index data type is long. Enter collectTime: xxx during the query. |
Indicates the time when logs are collected by ICAgent. In the example, "collectTime":"1681896081334" is 2023-04-19 17:21:21 when converted into standard time. |
__time__ |
Integer, Unix timestamp (ms) |
Index setting: After this function is enabled, a field index is created for time by default. The index data type is long. This field cannot be queried. |
Log time refers to the time when a log is displayed on the console. In the example, "__time__":"1681896081334" is 2023-04-19 17:21:21 when converted into standard time. By default, the collection time is used as the log time. You can also customize the log time. |
lineNum |
Integer |
Index setting: After this function is enabled, a field index is created for lineNum by default. The index data type is long. |
Line number (offset), which is used to sort logs. Non-high-precision logs are generated based on the value of collectTime. The default value is collectTime * 1000000 + 1. For high-precision logs, the value is the nanosecond value reported by users. Such as "lineNum":"1681896081333991900" in the example. |
category |
String |
Index setting: After this function is enabled, a field index is created for category by default. The index data type is string, and the delimiters are empty. Enter category: xxx during the query. |
Log type, indicating the source of the log. For example, the field value of logs collected by ICAgent is LTS, and that of logs reported by VPC is VPC. |
clusterName |
String |
Index setting: After this function is enabled, a field index is created for clusterName by default. The index data type is string, and the delimiters are empty. Enter clusterName: xxx during the query. |
Cluster name, used in the Kubernetes scenario. Such as "clusterName":"epstest" in the example. |
clusterId |
String |
Index setting: After this function is enabled, a field index is created for clusterId by default. The index data type is string, and the delimiters are empty. Enter clusterId: xxx during the query. |
Cluster ID, used in the Kubernetes scenario. Such as "clusterId":"c7f3f4a5-xxxx-11ed-a4ec-0255ac100b07" in the example. |
nameSpace |
String |
Index setting: After this function is enabled, a field index is created for nameSpace by default. The index data type is string, and the delimiters are empty. Enter nameSpace: xxx during the query. |
Namespace used in the Kubernetes scenario. Such as "nameSpace":"monitoring" in the example. |
appName |
String |
Index setting: After this function is enabled, a field index is created for appName by default. The index data type is string, and the delimiters are empty. Enter appName: xxx during the query. |
Component name, used as the name of the workload in the Kubernetes scenario. Such as "appName":"alertmanager-alertmanager" in the example. |
serviceID |
String |
Index setting: After this function is enabled, a field index is created for serviceID by default. The index data type is string, and the delimiters are empty. Enter serviceID: xxx during the query. |
Workload ID in the Kubernetes scenario. Such as "serviceID":"cf5b453xxxad61d4c483b50da3fad5ad" in the example. |
podName |
String |
Index setting: After this function is enabled, a field index is created for podName by default. The index data type is string, and the delimiters are empty. Enter podName: xxx during the query. |
Pod name in the Kubernetes scenario. Such as "podName":"alertmanager-alertmanager-0" in the example. |
podIp |
String |
Index setting: After this function is enabled, a field index is created for podIp by default. The index data type is string, and the delimiters are empty. Enter podIp: xxx during the query. |
Pod IP in the Kubernetes scenario. Such as "podIp":"10.0.0.145" in the example. |
containerName |
String |
Index setting: After this function is enabled, a field index is created for containerName by default. The index data type is string, and the delimiters are empty. Enter containerName: xxx during the query. |
Container name used in the Kubernetes scenario. Such as "containerName":"config-reloader" in the example. |
hostName |
String |
Index setting: After this function is enabled, a field index is created for hostName by default. The index data type is string, and the delimiters are empty. Enter hostName: xxx during the query. |
Indicates the host name where ICAgent resides. Such as "hostName":"epstest-xx518" in the example. |
hostId |
String |
Index setting: After this function is enabled, a field index is created for hostId by default. The index data type is string, and the delimiters are empty. Enter hostId: xxx during the query. |
Indicates the host ID where ICAgent resides. The ID is generated by ICAgent. Such as "hostId":"318c02fe-xxxx-4c91-b5bb-6923513b6c34" in the example. |
hostIP |
String |
Index setting: After this function is enabled, a field index is created for hostIP by default. The index data type is string, and the delimiters are empty. Enter hostIP: xxx during the query. |
Host IP address where the log collector resides (applicable to IPv4 scenario) Such as "hostIP":"192.168.0.31" in the example. |
hostIPv6 |
String |
Index setting: After this function is enabled, a field index is created for hostIPv6 by default. The index data type is string, and the delimiters are empty. Enter hostIPv6: xxx during the query. |
Host IP address where the log collector resides (applicable to IPv6 scenario) Such as "hostIPv6":"" in the example. |
pathFile |
String |
Index setting: After this function is enabled, a field index is created for pathFile by default. The index data type is string, and the delimiters are empty. Enter pathFile: xxx during the query. |
File path is the path of the collected log file. Such as "pathFile":"stdout.log" in the example. |
content |
String |
Index setting: After Index Whole Text is enabled, the delimiter defined by the full-text index is used to segment the value of the content field. The content field cannot be configured in the field index. |
Original log content Such as "content":"level=error ts=2023-04-19T09:21:21.333895559Z" in the example. |
logContent |
String |
The logContent field cannot be configured in the field index. |
Not involved |
logContentSize |
Integer |
The logContentSize field cannot be configured in the field index. |
Not involved |
logIndexSize |
Integer |
The logIndexSize field cannot be configured in the field index. |
Not involved |
groupName |
String |
The groupName field cannot be configured in the field index. |
Not involved |
logStream |
String |
The logStream field cannot be configured in the field index. |
Not involved |
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot