- What's New
- Function Overview
- Service Overview
- Billing
- Getting Started
-
User Guide
- Granting LTS Permissions to IAM Users
- Purchasing LTS Resource Packages
- Log Management
-
Log Ingestion
- Overview
-
Using ICAgent to Collect Logs
- Overview
- Installing ICAgent (Intra-Region Hosts)
- Installing ICAgent (Extra-Region Hosts)
- Managing ICAgent
- Managing Host Groups
- Ingesting BMS Text Logs to LTS
- Ingesting CCE Application Logs to LTS
- Ingesting ECS Text Logs to LTS
- Ingesting ServiceStage Containerized Application Logs to LTS
- Ingesting ServiceStage Cloud Host Logs to LTS
- Ingesting Self-Built Kubernetes Application Logs to LTS
- Setting ICAgent Structuring Parsing Rules
-
Ingesting Cloud Service Logs to LTS
- Overview
- Ingesting AOM Logs to LTS
- Ingesting APIG Logs to LTS
- Ingesting CBH Logs to LTS
- Ingesting CFW Logs to LTS
- Ingesting CTS Logs to LTS
- Ingesting DDS Logs to LTS
- Ingesting DMS for Kafka Logs to LTS
- Ingesting DRS Logs to LTS
- Ingesting GaussDB(DWS) Logs to LTS
- Ingesting ELB Logs to LTS
- Ingesting Enterprise Router Logs to LTS
- Ingesting FunctionGraph Logs to LTS
- Ingesting GaussDB Logs to LTS
- Ingesting GES Logs to LTS
- Ingesting TaurusDB Logs to LTS
- Ingesting GeminiDB Logs to LTS
- Ingesting GeminiDB Mongo Logs to LTS
- Ingesting GeminiDB Cassandra Logs to LTS
- Ingesting IoTDA Logs to LTS
- Ingesting ModelArts Logs to LTS
- Ingesting MRS Logs to LTS
- Ingesting RDS for MySQL Logs to LTS
- Ingesting RDS for PostgreSQL Logs to LTS
- Ingesting RDS for SQL Server Logs to LTS
- Ingesting ROMA Connect Logs to LTS
- Ingesting SMN Logs to LTS
- Ingesting SecMaster Logs to LTS
- Ingesting OBS Files to LTS (Beta)
- Ingesting VPC Logs to LTS
- Ingesting WAF Logs to LTS
- Using APIs to Ingest Logs to LTS
- Ingesting Logs to LTS Across IAM Accounts
- Using Kafka to Report Logs to LTS
- Using Flume to Report Logs to LTS
- Log Search and Analysis
-
Log Visualization
- Overview
- Visualizing Logs in Statistical Charts
-
Visualizing Logs in Dashboards
- Creating a Dashboard
- Adding a Dashboard Filter
-
Dashboard Templates
- APIG Dashboard Templates
- CCE Dashboard Templates
- CDN Dashboard Templates
- CFW Dashboard Templates
- CSE Dashboard Templates
- DCS Dashboard Template
- DDS Dashboard Template
- DMS Dashboard Template
- DSL Dashboard Template
- ER Dashboard Template
- METRIC Dashboard Template
- Nginx Dashboard Templates
- VPC Dashboard Template
- WAF Dashboard Templates
- Log Alarms
- Log Transfer
- Log Processing
- Configuration Center
- Querying Real-Time LTS Traces
-
Best Practices
- Overview
-
Log Ingestion
- Collecting Logs from Third-Party Cloud Vendors, Internet Data Centers, and Other Huawei Cloud Regions to LTS
- Collecting Kubernetes Logs from Third-Party Clouds, IDCs, and Other Huawei Cloud Regions to LTS
- Collecting Syslog Aggregation Server Logs to LTS
- Importing Logs of Self-built ELK to LTS
- Using Flume to Report Logs to LTS
- Collecting Zabbix Data Through ECS Log Ingestion
- Collecting Logs from Multiple Channels to LTS
- Log Search and Analysis
- Log Transfer
- Billing
- Developer Guide
-
API Reference
- Before You Start
- API Overview
- Calling APIs
- API Calling Examples
- Examples
-
APIs
- Host Group Management
- Log Group Management
- Log Stream Management
- Log Management
- Log Ingestion
- Log Transfer
- Log Collection Beyond Free Quota
- Cloud Log Structuring
- Container Log Ingestion from AOM to LTS
- Alarm Topics
- Message Template Management
- SQL Alarm Rules
- Keyword Alarm Rules
- Alarm List
- Tag Management
- Dashboard Management
- Log Charts
- Quick Search
- Multi-Account Log Aggregation
- Permissions Policies and Supported Actions
- Appendix
- SDK Reference
-
FAQs
- Overview
- Consultation
- Log Management
-
Host Management
- What Do I Do If ICAgent Installation Fails in Windows and the Message "SERVICE STOP" Is Displayed?
- What Do I Do If ICAgent Upgrade Fails on the LTS Console?
- What Do I Do If I Could Not Query New Logs on LTS?
- What Do I Do If ICAgent Restarts Repeatedly After Being Installed?
- What Do I Do If ICAgent Is Displayed as Offline on the LTS Console After Installation?
- What Do I Do If I Do Not See a Host with ICAgent Installed on the LTS Console?
- How Do I Create a VPC Endpoint on the VPCEP Console?
- How Do I Obtain an AK/SK Pair?
- How Do I Install ICAgent by Creating an Agency?
-
Log Ingestion
- What Do I Do If LTS Cannot Collect Logs After I Configure Host Log Ingestion?
- Will LTS Stop Collecting Logs After the Free Quota Is Used Up If I Disable "Continue to Collect Logs When the Free Quota Is Exceeded" in AOM?
- What Do I Do If the CPU Usage Is High When ICAgent Is Collecting Logs?
- What Kinds of Logs and Files Does LTS Collect?
- How Do I Disable the Function of Collecting CCE Standard Output Logs to AOM on the LTS Console?
- What Log Rotation Scheme Should I Use for ICAgent to Collect Logs?
- Does LTS Use the Log4j Plug-in to Report Logs?
- How Long Does It Take to Generate Logs After Configuring Log Ingestion?
- What Do I Do If LTS Cannot Collect Logs After I Configure Log Ingestion with ICAgent?
- Log Search and Analysis
-
Log Transfer
- Does LTS Delete Logs That Have Been Transferred to OBS Buckets?
- What Are the Common Causes of LTS Log Transfer Abnormalities?
- How Do I Transfer CTS Logs to an OBS Bucket?
- What Do I Do If I Cannot View Historical Data in an OBS Bucket After Transferring Data from LTS to OBS?
- What Do I Do If I Cannot Find a New Partition in a DLI Table After Logs Are Transferred to DLI?
-
More Documents
- User Guide (ME-Abu Dhabi Region)
- API Reference (ME-Abu Dhabi Region)
- User Guide(Paris Regions)
- API Reference(Paris Regions)
- User Guide (Kuala Lumpur Region)
- API Reference (Kuala Lumpur Region)
- User Guide (Ankara Region)
-
API Reference (Ankara Region)
- Before You Start
- Calling APIs
- API Calling Examples
- APIs
- Permissions and Supported Actions
- Appendix
- Change History
- Videos
- General Reference
Copied.
Changing File Time Zones for Log Transfer in a Batch
For recurring tasks like configuring log ingestion, alarm rules, and log transfer, batch operations are not supported on the LTS console. You can use Python scripts and LTS APIs to perform custom operations in a batch.
Scenario
If you have created 1,000 rules for log transfer to OBS but set all file time zones to (UTC) Coordinated Universal Time, you need to change it to UTC+08:00. Currently, the LTS console does not allow batch modifying log transfer rules. Manually modifying each transfer rule will be time-consuming.
Prerequisites
- Prepare a Linux host.
- Check the API document.
- Obtain information about all transfer tasks by calling the log transfer querying API.
- Change the time zones configured for transfer tasks by calling the log transfer updating API.
- Test API functions in API Explorer, which provides API search and platform debugging capabilities.
- Install the Python SDK on the host by referring to the API Explorer sample code.
- Python SDK dependency address and SDK usage description:
pip install huaweicloudsdklts
- API Explorer provides sample code for calling APIs using Python. Example:
# coding: utf-8 import os from huaweicloudsdkcore.auth.credentials import BasicCredentials from huaweicloudsdklts.v2.region.lts_region import LtsRegion from huaweicloudsdkcore.exceptions import exceptions from huaweicloudsdklts.v2 import * if __name__ == "__main__": # The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security. # In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment /* Hardcoded or plaintext AK and SK are risky. For security, encrypt your AK and SK and store them in the configuration file or as environment variables. ak = os.environ["CLOUD_SDK_AK"] sk = os.environ["CLOUD_SDK_SK"] credentials = BasicCredentials(ak, sk) client = LtsClient.new_builder() \ .with_credentials(credentials) \ .with_region(LtsRegion.value_of("xx")) \ .build() try: request = ListTransfersRequest() response = client.list_transfers(request) print(response) except exceptions.ClientRequestException as e: print(e.status_code) print(e.request_id) print(e.error_code) print(e.error_msg)
- Python SDK dependency address and SDK usage description:
Changing File Time Zones for Log Transfer in a Batch
- Obtain the parameter and replace it with the actual value in the code.
- Obtain an AK/SK pair of the Huawei account.
- Obtain the project ID. For details, see API Credentials.
Figure 1 Obtaining the project ID
- Obtain the value of Region&iam_Endpoint from Regions and Endpoints.
Table 1 Endpoints Region Name
Region
Endpoint
Protocol
AP-Bangkok
ap-southeast-2
lts.ap-southeast-2.myhuaweicloud.com
HTTPS
AP-Singapore
ap-southeast-3
lts.ap-southeast-3.myhuaweicloud.com
HTTPS
CN-Hong Kong
ap-southeast-1
lts.ap-southeast-1.myhuaweicloud.com
HTTPS
- Obtain the time zone and time zone ID.
Table 2 Common time zones Time Zone
Time Zone ID
UTC-12:00
Etc/GMT+12
UTC-11:00
Etc/GMT+11
UTC-10:00
Pacific/Honolulu
UTC-09:00
America/Anchorage
UTC-08:00
America/Santa_Isabel
UTC-07:00
America/Chihuahua
UTC-06:00
America/Chicago
UTC-05:00
America/New_York
UTC-04:00
America/Santiago
UTC-03:00
America/Sao_Paulo
UTC-02:00
Etc/GMT+2
UTC-01:00
Atlantic/Azoresjavik
UTC+00:00
Europe/London
UTC+01:00
Europe/Parist
UTC+02:00
Europe/Istanbul
UTC+03:00
Europe/Minsk
UTC+04:00
Europe/Moscow
UTC+05:00
Asia/Tashkent
UTC+06:00
Asia/Almaty
UTC+07:00
Asia/Bangkok
UTC+08:00
Asia/Shanghai
UTC+09:00
Asia/Tokyo
UTC+10:00
Asia/Yakutsk
UTC+11:00
Asia/Vladivostok
UTC+12:00
Pacific/Fiji
UTC+13:00
Pacific/Apia
- Run the following command on the host to check whether the huaweicloudsdkcore and huaweicloudsdklts packages have been installed:
pip list | grep huaweicloudsdk
- If they have been installed, the command output displays information about huaweicloudsdk.
- If not, no information is returned. Run the following command to install them:
pip install huaweicloudsdkcore huaweicloudsdklts
- Run the vi lts_python.py command on the host to create the lts_python.py file. Then, copy the following code to the file to batch change the time zones of files to be transferred to OBS.
# coding: utf-8 from huaweicloudsdkcore.auth.credentials import BasicCredentials from huaweicloudsdkcore.exceptions import exceptions from huaweicloudsdklts.v2 import * from huaweicloudsdklts.v2.region.lts_region import LtsRegion /* Hardcoded or plaintext AK and SK are risky. For security, encrypt your AK and SK and store them in the configuration file or as environment variables. if __name__ == "__main__": AK = "your ak" SK = "your sk" PROJECT_ID = "your project id" REGION = "your region" IAM_ENDPOINT = "iam_endpoint" OBS_TIME_ZONE = "the time_zone you want to change" OBS_TIME_ZONE_ID = "time_zone_id" credentials = BasicCredentials(AK, SK, PROJECT_ID).with_iam_endpoint(IAM_ENDPOINT) client = LtsClient.new_builder() \ .with_credentials(credentials) \ .with_region(LtsRegion.value_of(REGION)) \ .build() # 1.get obs transfer task try: request = ListTransfersRequest() request.log_transfer_type = "OBS" response = client.list_transfers(request) obs_transfer_num = len(response.log_transfers) task_list = response.log_transfers print("#### get {} obs transfer task ####".format(obs_transfer_num)) except exceptions.ClientRequestException as e: print(e.status_code) print(e.request_id) print(e.error_code) print(e.error_msg) # 2.set obs transfer task obs_time_zone to UTC+08:00 CNT = 1 while len(task_list): transfer_task = task_list.pop() print("There are still {} progress: \n".format(len(task_list)), transfer_task) try: if transfer_task.log_transfer_info.log_transfer_detail.obs_time_zone == OBS_TIME_ZONE: CNT += 1 continue request = UpdateTransferRequest() transfer_task.log_transfer_info.log_transfer_detail.obs_time_zone = OBS_TIME_ZONE transfer_task.log_transfer_info.log_transfer_detail.obs_time_zone_id = OBS_TIME_ZONE_ID request.body = UpdateTransferRequestBody( log_transfer_info=transfer_task.log_transfer_info, log_transfer_id=transfer_task.log_transfer_id ) response = client.update_transfer(request) CNT += 1 except exceptions.ClientRequestException as e: print(e.status_code) print(e.request_id) print(e.error_code) print(e.error_msg) task_list.append(transfer_task) except exceptions.ServerResponseException as e: print({ "target": transfer_task.log_streams, "reason": e }) task_list.append(transfer_task)
- Run the Python script on the host to batch change the time zones of files to be transferred to OBS.
nohup python lts_python.py > lts_python.log &
- View execution logs to check whether the Python script has been executed and whether the files' time zones have been changed.
tail -f lts_python.log
Figure 2 Viewing execution logs
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot