Using Scripts to Invoke LTS APIs for Custom Operations

Introduction

Some configuration-related operations on the LTS console, such as ingesting logs, creating alarms, and configuring log transfer, need to be performed repeatedly. However, the LTS console does not support batch operations. In this case, you can use Python scripts and LTS APIs to perform custom batch operations.

Scenario

• If you have created 1000 rules for log transfer to OBS but set the file time zone to UTC during the transfer, you need to change the time zone to UTC+08:00.
• Currently, the LTS console does not allow batch modifying log transfer rules. Manually modifying each transfer rule will be time-consuming.

Prerequisites

1. Linux
2. Querying API-related documents
   • Obtain information about all transfer tasks by calling the log transfer API.
   • Change the time zone configured for the transfer task by updating the log transfer API.
3. Testing API functions in API Explorer

   API Explorer provides the API search and platform debugging capabilities.

4. Installing the Python SDK by referring to the API Explorer sample code
   • Python SDK dependency address and SDK usage description

     pip install huaweicloudsdklts

   • API Explorer provides sample code for calling APIs using Python. The following is an example:

     # coding: utf-8   
     from huaweicloudsdkcore.auth.credentials import BasicCredentials 
     from huaweicloudsdklts.v2.region.lts_region import LtsRegion 
     from huaweicloudsdkcore.exceptions import exceptions 
     from huaweicloudsdklts.v2 import *  
     if __name__ == "__main__":   
         AK = "your ak"   
         SK = "your sk"   
         PROJECT_ID = "your project id"   
         REGION = "your region"   
         IAM_ENDPOINT = "iam_endpoint"      
        
         credentials = BasicCredentials(AK, SK, PROJECT_ID).with_iam_endpoint(IAM_ENDPOINT)      
         client = LtsClient.new_builder()       
         .with_credentials(credentials)       
         .with_region(LtsRegion.value_of(REGION))       
         .build()      
     
         try:      
             request = ListTransfersRequest()      
             request.log_transfer_type = "OBS"       
             response = client.list_transfers(request)       
             print(response)   
         except exceptions.ClientRequestException as e:       
             print(e.status_code)       
             print(e.request_id)       
             print(e.error_code)       
             print(e.error_msg)

Procedure

1. Obtain the parameter and replace it with the actual value in the code.
   • Obtain an AK/SK Pair
   • Obtain the project ID. For details, see API Credentials.
     Figure 1 Obtaining the project ID
     
   • Obtain the value of Region&iam_Endpoint from Regions and Endpoints.

     Table 1 Endpoints

     Region Name	Region	Endpoint	Protocol
     AP-Bangkok	ap-southeast-2	lts.ap-southeast-2.myhuaweicloud.com	HTTPS
     AP-Singapore	ap-southeast-3	lts.ap-southeast-3.myhuaweicloud.com	HTTPS
     CN-Hong Kong	ap-southeast-1	lts.ap-southeast-1.myhuaweicloud.com	HTTPS

   • Time Zone and Time Zone ID

     Table 2 Common time zones

     Time Zone	Time Zone ID
     UTC-12:00	Etc/GMT+12
     UTC-11:00	Etc/GMT+11
     UTC-10:00	Pacific/Honolulu
     UTC-09:00	America/Anchorage
     UTC-08:00	America/Santa_Isabel
     UTC-07:00	America/Chihuahua
     UTC-06:00	America/Chicago
     UTC-05:00	America/New_York
     UTC-04:00	America/Santiago
     UTC-03:00	America/Sao_Paulo
     UTC-02:00	Etc/GMT+2
     UTC-01:00	Atlantic/Azoresjavik
     UTC+00:00	Europe/London
     UTC+01:00	Europe/Parist
     UTC+02:00	Europe/Istanbul
     UTC+03:00	Europe/Minsk
     UTC+04:00	Europe/Moscow
     UTC+05:00	Asia/Tashkent
     UTC+06:00	Asia/Almaty
     UTC+07:00	Asia/Bangkok
     UTC+08:00	Asia/Shanghai
     UTC+09:00	Asia/Tokyo
     UTC+10:00	Asia/Yakutsk
     UTC+11:00	Asia/Vladivostok
     UTC+12:00	Pacific/Fiji
     UTC+13:00	Pacific/Apia

2. Check whether the huaweicloudsdkcore and huaweicloudsdklts packages have been installed.

   pip list | grep huaweicloudsdk

   If not, perform the following operations:

   pip install huaweicloudsdkcore huaweicloudsdklts

3. Create a file named lts_python.py and copy the following code to the file:

   # coding: utf-8
   
   from huaweicloudsdkcore.auth.credentials import BasicCredentials
   from huaweicloudsdkcore.exceptions import exceptions
   from huaweicloudsdklts.v2 import *
   from huaweicloudsdklts.v2.region.lts_region import LtsRegion
   
   if __name__ == "__main__":
   AK = "your ak"
   SK = "your sk"
   PROJECT_ID = "your project id"
   REGION = "your region"
   IAM_ENDPOINT = "iam_endpoint"
   
   OBS_TIME_ZONE = "the time_zone you want to change"
   OBS_TIME_ZONE_ID = "time_zone_id"
   
   credentials = BasicCredentials(AK, SK, PROJECT_ID).with_iam_endpoint(IAM_ENDPOINT)
   
   client = LtsClient.new_builder() \
       .with_credentials(credentials) \
       .with_region(LtsRegion.value_of(REGION)) \
       .build()
   
   # 1.get obs transfer task
   try:
       request = ListTransfersRequest()
       request.log_transfer_type = "OBS"
       response = client.list_transfers(request)
       obs_transfer_num = len(response.log_transfers)
       task_list = response.log_transfers
       print("#### get {} obs transfer task ####".format(obs_transfer_num))
   except exceptions.ClientRequestException as e:
       print(e.status_code)
       print(e.request_id)
       print(e.error_code)
       print(e.error_msg)
   
   # 2.set obs transfer task obs_time_zone to UTC+08:00
   CNT = 1
   while len(task_list):
       transfer_task = task_list.pop()
       print("There are still {} progress: \n".format(len(task_list)), transfer_task)
       try:
           if transfer_task.log_transfer_info.log_transfer_detail.obs_time_zone == OBS_TIME_ZONE:
               CNT += 1
               continue
           request = UpdateTransferRequest()
           transfer_task.log_transfer_info.log_transfer_detail.obs_time_zone = OBS_TIME_ZONE
           transfer_task.log_transfer_info.log_transfer_detail.obs_time_zone_id = OBS_TIME_ZONE_ID
           request.body = UpdateTransferRequestBody(
               log_transfer_info=transfer_task.log_transfer_info,
               log_transfer_id=transfer_task.log_transfer_id
           )
           response = client.update_transfer(request)
           CNT += 1
       except exceptions.ClientRequestException as e:
           print(e.status_code)
           print(e.request_id)
           print(e.error_code)
           print(e.error_msg)
           task_list.append(transfer_task)
       except exceptions.ServerResponseException as e:
           print({
               "target": transfer_task.log_streams,
               "reason": e
           })
           task_list.append(transfer_task)

4. Run the Python script to change the time zone of log transfer to OBS in batches.

   nohup python lts_python.py > lts_python.log &

   View execution logs.

   tail -f lts_python.log

   Figure 2 Viewing execution logs