Changing File Time Zones for Log Transfer in a Batch

Scenario

If you have created 1,000 rules for log transfer to OBS but set all file time zones to (UTC) Coordinated Universal Time, you need to change it to UTC+08:00. Currently, the LTS console does not allow batch modifying log transfer rules. Manually modifying each transfer rule will be time-consuming.

Prerequisites

1. Prepare a Linux host.
2. Check the API document.
   • Obtain information about all transfer tasks by calling the log transfer querying API.
   • Change the time zones configured for transfer tasks by calling the log transfer updating API.
3. Test API functions in API Explorer, which provides API search and platform debugging capabilities.
4. Install the Python SDK on the host by referring to the API Explorer sample code.
   • Python SDK dependency address and SDK usage description:

     pip install huaweicloudsdklts

   • API Explorer provides sample code for calling APIs using Python. Example:

     # coding: utf-8
     import os
     from huaweicloudsdkcore.auth.credentials import BasicCredentials
     from huaweicloudsdklts.v2.region.lts_region import LtsRegion
     from huaweicloudsdkcore.exceptions import exceptions
     from huaweicloudsdklts.v2 import *
     if __name__ == "__main__":
         # The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security.
         # In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment
     /* Hardcoded or plaintext AK and SK are risky. For security, encrypt your AK and SK and store them in the configuration file or as environment variables.
         ak = os.environ["CLOUD_SDK_AK"]
         sk = os.environ["CLOUD_SDK_SK"]
         credentials = BasicCredentials(ak, sk)
         client = LtsClient.new_builder() \
             .with_credentials(credentials) \
             .with_region(LtsRegion.value_of("xx")) \
             .build()
         try:
             request = ListTransfersRequest()
             response = client.list_transfers(request)
             print(response)
         except exceptions.ClientRequestException as e:
             print(e.status_code)
             print(e.request_id)
             print(e.error_code)
             print(e.error_msg)

Changing File Time Zones for Log Transfer in a Batch

1. Obtain the parameter and replace it with the actual value in the code.
   • Obtain an AK/SK pair of the Huawei account.
   • Obtain the project ID. For details, see API Credentials.
     Figure 1 Obtaining the project ID
     
   • Obtain the value of Region&iam_Endpoint from Regions and Endpoints.

     Table 1 Endpoints

     Region Name	Region	Endpoint	Protocol
     AP-Bangkok	ap-southeast-2	lts.ap-southeast-2.myhuaweicloud.com	HTTPS
     AP-Singapore	ap-southeast-3	lts.ap-southeast-3.myhuaweicloud.com	HTTPS
     CN-Hong Kong	ap-southeast-1	lts.ap-southeast-1.myhuaweicloud.com	HTTPS

   • Obtain the time zone and time zone ID.

     Table 2 Common time zones

     Time Zone	Time Zone ID
     UTC-12:00	Etc/GMT+12
     UTC-11:00	Etc/GMT+11
     UTC-10:00	Pacific/Honolulu
     UTC-09:00	America/Anchorage
     UTC-08:00	America/Santa_Isabel
     UTC-07:00	America/Chihuahua
     UTC-06:00	America/Chicago
     UTC-05:00	America/New_York
     UTC-04:00	America/Santiago
     UTC-03:00	America/Sao_Paulo
     UTC-02:00	Etc/GMT+2
     UTC-01:00	Atlantic/Azoresjavik
     UTC+00:00	Europe/London
     UTC+01:00	Europe/Parist
     UTC+02:00	Europe/Istanbul
     UTC+03:00	Europe/Minsk
     UTC+04:00	Europe/Moscow
     UTC+05:00	Asia/Tashkent
     UTC+06:00	Asia/Almaty
     UTC+07:00	Asia/Bangkok
     UTC+08:00	Asia/Shanghai
     UTC+09:00	Asia/Tokyo
     UTC+10:00	Asia/Yakutsk
     UTC+11:00	Asia/Vladivostok
     UTC+12:00	Pacific/Fiji
     UTC+13:00	Pacific/Apia

2. Run the following command on the host to check whether the huaweicloudsdkcore and huaweicloudsdklts packages have been installed:

   pip list | grep huaweicloudsdk

   • If they have been installed, the command output displays information about huaweicloudsdk.
   • If not, no information is returned. Run the following command to install them:

     pip install huaweicloudsdkcore huaweicloudsdklts

3. Run the vi lts_python.py command on the host to create the lts_python.py file. Then, copy the following code to the file to batch change the time zones of files to be transferred to OBS.

   # coding: utf-8
   
   from huaweicloudsdkcore.auth.credentials import BasicCredentials
   from huaweicloudsdkcore.exceptions import exceptions
   from huaweicloudsdklts.v2 import *
   from huaweicloudsdklts.v2.region.lts_region import LtsRegion
   /* Hardcoded or plaintext AK and SK are risky. For security, encrypt your AK and SK and store them in the configuration file or as environment variables.
   if __name__ == "__main__":
   AK = "your ak"
   SK = "your sk"
   PROJECT_ID = "your project id"
   REGION = "your region"
   IAM_ENDPOINT = "iam_endpoint"
   
   OBS_TIME_ZONE = "the time_zone you want to change"
   OBS_TIME_ZONE_ID = "time_zone_id"
   
   credentials = BasicCredentials(AK, SK, PROJECT_ID).with_iam_endpoint(IAM_ENDPOINT)
   
   client = LtsClient.new_builder() \
       .with_credentials(credentials) \
       .with_region(LtsRegion.value_of(REGION)) \
       .build()
   
   # 1.get obs transfer task
   try:
       request = ListTransfersRequest()
       request.log_transfer_type = "OBS"
       response = client.list_transfers(request)
       obs_transfer_num = len(response.log_transfers)
       task_list = response.log_transfers
       print("#### get {} obs transfer task ####".format(obs_transfer_num))
   except exceptions.ClientRequestException as e:
       print(e.status_code)
       print(e.request_id)
       print(e.error_code)
       print(e.error_msg)
   
   # 2.set obs transfer task obs_time_zone to UTC+08:00
   CNT = 1
   while len(task_list):
       transfer_task = task_list.pop()
       print("There are still {} progress: \n".format(len(task_list)), transfer_task)
       try:
           if transfer_task.log_transfer_info.log_transfer_detail.obs_time_zone == OBS_TIME_ZONE:
               CNT += 1
               continue
           request = UpdateTransferRequest()
           transfer_task.log_transfer_info.log_transfer_detail.obs_time_zone = OBS_TIME_ZONE
           transfer_task.log_transfer_info.log_transfer_detail.obs_time_zone_id = OBS_TIME_ZONE_ID
           request.body = UpdateTransferRequestBody(
               log_transfer_info=transfer_task.log_transfer_info,
               log_transfer_id=transfer_task.log_transfer_id
           )
           response = client.update_transfer(request)
           CNT += 1
       except exceptions.ClientRequestException as e:
           print(e.status_code)
           print(e.request_id)
           print(e.error_code)
           print(e.error_msg)
           task_list.append(transfer_task)
       except exceptions.ServerResponseException as e:
           print({
               "target": transfer_task.log_streams,
               "reason": e
           })
           task_list.append(transfer_task)

4. Run the Python script on the host to batch change the time zones of files to be transferred to OBS.

   nohup python lts_python.py > lts_python.log &

5. View execution logs to check whether the Python script has been executed and whether the files' time zones have been changed.

   tail -f lts_python.log

   Figure 2 Viewing execution logs