LTS Java SDK

Transport Protocol

HTTPS

Prerequisites

• You have registered a Huawei account and enabled Huawei Cloud services.
• You have obtained the name of the region where LTS is deployed.
• You have obtained the AK/SK of your HUAWEI ID.
• You have obtained a project ID of your Huawei Cloud account. For details, see API Credentials.
• You have obtained the IDs of the target log group and log stream in LTS.
• You have installed the Java Development Kit (JDK). LTS Java SDK supports JDK 1.8 or later. You can run the java –version command to check the installed JDK version. If JDK is not installed, download the installation package from the Java official website and install it.
• The Java SDK supports cross-cloud and local log reporting. This SDK is intended for development and debugging purposes only and does not guarantee reporting performance. It is available only in regions CN North-Beijing4, CN East-Shanghai1, CN South-Guangzhou, and CN Southwest-Guiyang1.
• When logs are reported to LTS via the Java SDK, the time difference between the log reporting time you set and the current time must not exceed two days. Otherwise, the reported logs will be deleted by LTS.

Log Reporting Modes

Logs can be reported in standard, structured (new version), or structured mode. The structured (new version) mode is recommended for its enhanced flexibility and performance.

1. Standard logs:
   • Logs can be reported in batches.
   • The log field indicates a raw log. Each raw log is a string.
   • The log_time_ns field indicates the time when a log is reported, in nanoseconds. It enables you to sort logs by time on the LTS console.
   • Logs can be reported by label (labels).
   The structure for reporting logs in a batch is as follows:

   [{
   	"contents": [{
   		"log": "log content1",
   		"log_time_ns": 1737527157333902200
   	}, {
   		"log": "log content2",
   		"log_time_ns": 1737527157333914100
   	}],
   	"labels": "{\"lts-test-count\":\"2\"}"
   }, {
   	"contents": [{
   		"log": "log content3",
   		"log_time_ns": 1737527157333986200
   	}, {
   		"log": "log content4",
   		"log_time_ns": 1737527157333987400
   	}],
   	"labels": "{\"lts-test-count\":\"2\"}"
   }]

2. Structured logs (new version) are recommended. This mode is available only to SDK 1.1.3 or later.
   

   Currently, this mode is available only to whitelisted users. To enable it, submit a service ticket.

   • Logs can be reported in batches or one by one.
   • The mContents field indicates a log.
   • The mKey field indicates a key in the log.
   • The mValue field indicates the value of a key in the log.
   • The mLogTime field indicates the time when the log is reported, in milliseconds.

   The structure for reporting logs in a batch is as follows:

   [{
   	"mContents": [{
   		"mKey": "content_key_1",
   		"mValue": "content_value_1"
   	}, {
   		"mKey": "content_key_2",
   		"mValue": "content_value_2"
   	}, {
   		"mKey": "content",
   		"mValue": "sdk-new-struct-log1"
   	}],
   	"mLogTime": 1744159440780
   }, {
   	"mContents": [{
   		"mKey": "content_key_1",
   		"mValue": "content_value_1"
   	}, {
   		"mKey": "content_key_2",
   		"mValue": "content_value_2"
   	}, {
   		"mKey": "content",   // (Optional) Internal field of LTS, which indicates a raw log.
   		"mValue": "sdk-new-struct-log2"
   	}],
   	"mLogTime": 1744159440780
   }]

3. Structured logs:
   

   Currently, this mode is available only to whitelisted users. To enable it, submit a service ticket.

   • Logs can be reported in batches.
   • The contents field indicates several logs. Each log consists of a JSON body with key-value pairs. content is a reserved field of LTS. It indicates raw log content and is optional.
   • The time field indicates the timestamps (in milliseconds) when these logs are reported. LTS converts these timestamps to nanoseconds when saving these logs. This conversion may affect the order in which these logs are displayed in the LTS console.
   • The labels field indicates the common labels for this batch of logs.
   • The path field indicates the path of this batch of logs.
   • The source field indicates the source of this batch of logs.

   The structure for reporting logs in a batch is as follows:

   {
   	"labels": {
   		"label": "label"
   	},
   	"logs": [{
   		"contents": [{
   			"k1": "v1",
   			"k2": "v2",
   			"content": "log content1"
   		}, {
   			"k3": "v3",
   			"k4": "v4",
   			"content": "log content2"
   		}],
   		"time": 1721784021037
   	}, {
   		"contents": [{
   			"k5": "v5",
   			"k6": "v6",
   			"content": "log content3"
   		}],
   		"time": 1721784021038
   	}, ],
   	"path": "path",
   	"source": "source"
   }

Step 1: Install the SDK

Add dependencies to the Maven project.

1. To use Maven to build a project, add the Huawei external open-source repository as a mirror in the settings.xml file. Example:

   <mirror>
       <id>huaweicloud</id>
       <mirrorOf>*</mirrorOf>
       <url>https://repo.huaweicloud.com/repository/maven/</url>
   </mirror>

2. Obtain the latest version of the LTS Java SDK dependencies from the Huawei external open-source repository.
3. To use the LTS Java SDK in a Maven project, you only need to add the corresponding dependencies to the pom.xml file. The Maven project management tool automatically downloads the related JAR packages. Add the following content to <dependencies>.
   The latest version (1.1.3) is recommended. You can check the SDK version in step 2.

   <dependency> 
        <groupId>io.github.huaweicloud</groupId> 
        <artifactId>lts-sdk-common</artifactId> 
        <version>version</version> 
    </dependency> 
    <dependency> 
        <groupId>io.github.huaweicloud</groupId> 
        <artifactId>lts-sdk-java</artifactId> 
        <version>version</version> 
    </dependency>

Step 2: Create a Producer

Producers use ECS resources. Therefore, use one producer and make it a public resource. When sending logs, you can simply call the producer.send() method. Appenders support permanent AK/SK.

The following is the sample code for creating a producer. Modify it by referring to Table 1.

import com.huaweicloud.lts.appender.JavaSDKAppender;
import com.huaweicloud.lts.producer.Producer;

public class ProducerUtil {

    private static final Producer producer;

    static {
         /* Hard-coded or plaintext AK and SK are risky. For security, encrypt your AK and SK, store them in the configuration file or as environment variables, and decrypt them during usage.
         In this example, the AK and SK stored as environment variables are used. Configure the environment variables HUAWEICLOUD_SDK_AK and HUAWEICLOUD_SDK_SK in the local environment first.
        String ak = System.getenv("HUAWEICLOUD_SDK_AK");
        String sk = System.getenv("HUAWEICLOUD_SDK_SK");
        // Build an appender.
        JavaSDKAppender appender = JavaSDKAppender.custom()
            // Project ID of the Huawei Cloud account.
            .setProjectId("xxx")
            // AK of the Huawei Cloud account.
            .setAccessKeyId(ak)
            // SK of the Huawei Cloud account.
            .setAccessKeySecret(sk)
            // Huawei Cloud STS token. If a temporary STS token is used, use the temporary AK and SK matching the STS token.
            // .setSecurityToken("")
            // Region where LTS is deployed.
            .setRegionName("xxx")
            // Maximum size of logs that can be cached by a single appender.
            .setTotalSizeInBytes(104857600)
            // Blocking time when the producer sends logs.
            .setMaxBlockMs(0L)
            // Size of the thread pool for executing log sending tasks.
            .setIoThreadCount(8)
            // Maximum size of logs sent by the producer in a batch.
            .setBatchSizeThresholdInBytes(524288)
            // Maximum number of logs sent by the producer in a batch.
            .setBatchCountThreshold(4096)
            // Waiting time for the producer to send logs in a batch.
            .setLingerMs(2000)
            // Number of retries after the producer fails to send logs.
            .setRetries(5)
            // Backoff time for the first retry.
            .setBaseRetryBackoffMs(500L)
            // Maximum backoff time for retry.
            .setMaxRetryBackoffMs(500L)
            // true: Logs can be reported across clouds. false (default): Logs can be reported only from Huawei Cloud ECSs.
            // .setEnableLocalTest(false)
            .builder();
        // Obtain the producer for sending logs.
        producer = appender.getProducer();
    }

    public static Producer getProducer() {
        return producer;
    }
}

// If a temporary AK, SK, and STS token are used, create a ProjectConfig periodically (before the temporary AK, SK, and STS token expire) and call the Producer.putProjectConfig() method to update the ProjectConfig.
ProjectConfig projectConfig = new ProjectConfig("projectId", "regionName", "accessKeyId", "accessKeySecret", "securityToken");
producer.putProjectConfig(projectConfig);

Step 3: Build a Log Sending Structure

• The standard logs and structured logs use the following structure:

  import java.util.ArrayList;
  import java.util.HashMap;
  import java.util.List;
  import java.util.Map;
  import org.slf4j.Logger;
  import org.slf4j.LoggerFactory;
  import com.alibaba.fastjson.JSONObject;
  import com.huaweicloud.lts.producer.model.log.LogContent;
  import com.huaweicloud.lts.producer.model.log.LogItem;
  import com.huaweicloud.lts.producer.model.log.StructLog;
  import com.huaweicloud.lts.producer.model.log.StructLogItem;
  
  public class LogUtil {
  
      private static final Logger LOGGER = LoggerFactory.getLogger(ProducerUtil.class);
  
      public static void doSomething() throws InterruptedException {
          LOGGER.info("Before doSomething");
          Thread.sleep(30000);
          LOGGER.info("After doSomething");
      }
  
      // Standard log model.
      public static List<LogItem> getLogItems(int size) {
          List<LogItem> logItemList = new ArrayList<>();
          logItemList.add(getLogItem(size));
          return logItemList;
      }
  
      private static LogItem getLogItem(int count) {
          LogItem logItem = new LogItem();
  
          // Configure custom log labels. If log labels are not required, pass an empty map.
          Map<String, String> labels = new HashMap<>();
          labels.put("labels-k", "labels-v");
          logItem.setLabels(JSONObject.toJSONString(labels));
  
          List<LogContent> contents = new ArrayList<>();
          for (int i = 1; i <= count; i++) {
              LogContent logContent = new LogContent();
              // Log generation time, which is the current time in nanoseconds.
              long logTimeNs = System.currentTimeMillis() * 1000000L + System.nanoTime() % 1000000L;
              logContent.setLogTimeNs(logTimeNs);
              // Log content.
              logContent.setLog("java-sdk-log-" + i);
              contents.add(logContent);
          }
          logItem.setContents(contents);
  
          return logItem;
      }
  
      // Structured log model.
      public static StructLogItem getStructLog(int size) {
          // Build a log structure.
          StructLogItem structLogItem = new StructLogItem();
          // Set the common labels of this batch of logs.
          structLogItem.setLabels(getPostLabels());
          // Add a log list.
          structLogItem.setLogs(getLogs(size));
          // Set the common path of this batch of logs.
          structLogItem.setPath("test-path");
          // Set the common source of this batch of logs.
          structLogItem.setSource("test-source");
          return structLogItem;
      }
  
      private static JSONObject getPostLabels() {
          JSONObject jsonObject = new JSONObject();
          jsonObject.put("common-k", "common-v");
          return jsonObject;
      }
  
      private static List<StructLog> getLogs(int size) {
          List<StructLog> logs = new ArrayList<>();
          for (int i = 1; i <= size; i++) {
              logs.add(getLog(size));
          }
          return logs;
      }
  
      private static StructLog getLog(int size) {
          StructLog structLog = new StructLog();
          // Log generation time, which is the current time in milliseconds.
          long time = System.currentTimeMillis();
          structLog.setTime(time);
  
          List<JSONObject> contents = new ArrayList<>();
          for (int i = 1; i <= size; i++) {
              // Create a log.
              JSONObject jsonLog = new JSONObject();
  
              jsonLog.put("log-k", "log-v");
              // content is a reserved field in LTS. It indicates raw logs.
              jsonLog.put("content", "java-struct-log-" + i);
  
              contents.add(jsonLog);
          }
  
          structLog.setContents(contents);
          return structLog;
      }
  
  }

• The structured logs of the new version (supported only by SDK 1.1.3 or later) use the following structure:

  // Batch logs.
  public static List<LtsStructLog> getNewBatchStructLog(int size) {
      List<LtsStructLog> batchStructLog = new ArrayList<>();
      for (int i = 1; i <= size; i++) {
          // Create a log.
          LtsStructLog ltsStructLog = new LtsStructLog();
  
          ltsStructLog.pushBack("content_key_1", "content_value_1");
          ltsStructLog.pushBack("content_key_2", "content_value_2");
          // content is a reserved field in LTS. It indicates raw logs.
          ltsStructLog.pushBack("content", "sdk_new_struct_log");
          // Log generation time, which is the current time in milliseconds.
          ltsStructLog.setLogTime(System.currentTimeMillis());
  
          batchStructLog.add(ltsStructLog);
      }
      return batchStructLog;
  }
  
  // A single log.
  public static LtsStructLog getNewStructLog() {
      // Create a log.
      LtsStructLog ltsStructLog = new LtsStructLog();
  
      ltsStructLog.pushBack("content_key_1", "content_value_1");
      ltsStructLog.pushBack("content_key_2", "content_value_2");
          // content is a reserved field in LTS. It indicates raw logs.
      ltsStructLog.pushBack("content", "sdk_new_struct_log");
          // Log generation time, which is the current time in milliseconds.
      ltsStructLog.setLogTime(System.currentTimeMillis());
      return ltsStructLog;
  }

Step 4: Send Logs

Producers send logs asynchronously and have local caches. Therefore, a single producer cannot report standard and structured logs at the same time.

• Send logs asynchronously (no return value is required for additional operations):

  import java.util.concurrent.CountDownLatch;
  import java.util.concurrent.ExecutorService;
  import java.util.concurrent.Executors;
  import org.slf4j.Logger;
  import org.slf4j.LoggerFactory;
  import com.huaweicloud.lts.producer.Producer;
  import com.huaweicloud.lts.producer.exception.ProducerException;
  
  public class SamplePerformance {
  
      private static final Logger LOGGER = LoggerFactory.getLogger(SamplePerformance.class);
  
      public static void main(String[] args) throws InterruptedException {
          // Obtain the producer for sending logs.
          Producer producer = ProducerUtil.getProducer();
  
          int sendThreadCount = Math.max(Runtime.getRuntime().availableProcessors(), 1);
          ExecutorService executorService = Executors.newFixedThreadPool(sendThreadCount);
          final CountDownLatch latch = new CountDownLatch(sendThreadCount);
  
          LOGGER.info("Test started.");
          long t1 = System.currentTimeMillis();
          int n = 100;
          for (int i = 1; i <= sendThreadCount; ++i) {
              executorService.submit(new Runnable() {
                  @Override
                  public void run() {
                      try {
                          for (int i = 0; i < n; ++i) {
             
                              // Report standard logs.
                              producer.send("log_group_id", "log_stream_id", LogUtil.getLogItems(1));
                              // Send logs based on log group and log stream names. This function is supported only by SDK 1.1.0 or later.
                              // producer.send("log_group_name", "log_stream_name", true, logItemList);
                              // Note: Set the third parameter to true, indicating that logs are sent by name. If the value is false, logs are sent by ID.
  
                              // Report structured logs.
                              // producer.send("log_group_id", "log_stream_id", LogUtil.getStructLog(1));
                              // Send logs based on log group and log stream names. This function is supported only by SDK 1.1.0 or later.
                              // producer.send("log_group_name", "log_stream_name", true, structLog);
                              // Note: Set the third parameter to true, indicating that logs are sent by name. If the value is false, logs are sent by ID.
  
                              // Report structured logs of the new version.
                              // Note: By default, log_group indicates the log group ID and log_stream indicates the log stream ID.
                              // To report logs based on log group and log stream names, set setLogGroupStreamByName to true when initializing the appender.
                              // Report a single log.
                              // producer.sendStructLog("log_group", "log_stream", LogUtil.getNewStructLog());
                              // Report a batch of logs.
                              // producer.sendStructLog("log_group", "log_stream", LogUtil.getNewBatchStructLog(1));
                          }
                      } catch (Exception e) {
                          LOGGER.error("Failed to send log, e=", e);
                      } finally {
                          latch.countDown();
                      }
                  }
              });
          }
          latch.await();
          Thread.sleep(30000);
          long t2 = System.currentTimeMillis();
          LOGGER.info("Test end.");
          LOGGER.info("======Summary======");
          LOGGER.info("Total count " + sendThreadCount * n + ".");
          long timeCost = t2 - t1;
          LOGGER.info("Time cost " + timeCost + " millis");
          try {
              producer.close();
          } catch (ProducerException e) {
              LOGGER.error("Failed to close producer, e=", e);
          }
          executorService.shutdown();
      }
  
  }

• Execute additional operations using Futures.

  The send method returns a ListenableFuture. You can use it as a standard Future to call the get method to obtain the sending result. You can also register a callback method, which will be called after the Future is set. The following code example shows how to use ListenableFuture. You need to register a FutureCallback for it and deliver it to the EXECUTOR_SERVICE thread pool provided by the application for execution.

  import java.util.List;
  import java.util.concurrent.ExecutorService;
  import java.util.concurrent.Executors;
  import java.util.concurrent.TimeUnit;
  import java.util.concurrent.atomic.AtomicLong;
  import org.slf4j.Logger;
  import org.slf4j.LoggerFactory;
  import com.google.common.util.concurrent.FutureCallback;
  import com.google.common.util.concurrent.Futures;
  import com.google.common.util.concurrent.ListenableFuture;
  import com.huaweicloud.lts.producer.Producer;
  import com.huaweicloud.lts.producer.Result;
  import com.huaweicloud.lts.producer.exception.ProducerException;
  import com.huaweicloud.lts.producer.exception.ResultFailedException;
  import com.huaweicloud.lts.producer.model.log.LogItem;
  import com.huaweicloud.lts.producer.model.log.StructLogItem;
  
  public class SampleProducerWithFuture {
  
      private static final Logger LOGGER = LoggerFactory.getLogger(SampleProducerWithFuture.class);
  
      private static final ExecutorService EXECUTOR_SERVICE = Executors.newFixedThreadPool(
          Math.max(Runtime.getRuntime().availableProcessors(), 1));
  
      public static void main(String[] args) throws InterruptedException {
          Producer producer = ProducerUtil.getProducer();
          int n = 100;
          int size = 20;
  
          final AtomicLong completed = new AtomicLong(0);
  
          for (int i = 0; i < n; ++i) {
              try {
                  // Report standard logs.
                  List<LogItem> logItems = LogUtil.getLogItems(size);
                  ListenableFuture<Result> f = producer.send("log_group_id", "log_stream_id", logItems);
                  Futures.addCallback(f, new SampleFutureCallback("log_group_id", "log_stream_id", logItems, completed),
                      EXECUTOR_SERVICE);
  
                  // Report structured logs.
                  // StructLogItem structLog = LogUtil.getStructLog(size);
                  // ListenableFuture<Result> f = producer.send("log_group_id", "log_stream_id", structLog);
                  // Futures.addCallback(f, new SampleFutureCallback("log_group_id", "log_stream_id", structLog, completed),
                  //     EXECUTOR_SERVICE);
  
                  // Report structured logs of the new version.
                  // List<LtsStructLog> newBatchStructLog = LogUtil.getNewBatchStructLog(1);
                  // ListenableFuture<Result> f = producer.sendStructLog("log_group", "log_stream", newBatchStructLog);
                  // Futures.addCallback(f, new SampleFutureCallback("log_group", "log_stream", completed, newBatchStructLog),
                  //     EXECUTOR_SERVICE);
  
              } catch (Exception e) {
                  LOGGER.error("Failed to send logs, e=", e);
              }
          }
  
          LogUtil.doSomething();
  
          try {
              producer.close();
          } catch (ProducerException e) {
              LOGGER.info("Failed to close producer, e=", e);
          }
  
          EXECUTOR_SERVICE.shutdown();
          while (!EXECUTOR_SERVICE.isTerminated()) {
              EXECUTOR_SERVICE.awaitTermination(100, TimeUnit.MILLISECONDS);
          }
          LOGGER.info("All log complete, completed={}", completed.get());
      }
  
      private static final class SampleFutureCallback implements FutureCallback<Result> {
  
          private static final Logger LOGGER = LoggerFactory.getLogger(SampleFutureCallback.class);
  
          private final String logGroupId;
  
          private final String logStreamId;
  
          private List<LogItem> logItems = null;
  
          private StructLogItem structLog = null;
  
          private List<LtsStructLog> newBatchStructLog = null;
  
          private final AtomicLong completed;
  
          SampleFutureCallback(String logGroupId, String logStreamId, List<LogItem> logItems, AtomicLong completed) {
              this.logGroupId = logGroupId;
              this.logStreamId = logStreamId;
              this.logItems = logItems;
              this.completed = completed;
          }
  
          SampleFutureCallback(String logGroupId, String logStreamId, StructLogItem structLog, AtomicLong completed) {
              this.logGroupId = logGroupId;
              this.logStreamId = logStreamId;
              this.structLog = structLog;
              this.completed = completed;
          }
  
          SampleFutureCallback(String logGroupId, String logStreamId, AtomicLong completed, List<LtsStructLog> newBatchStructLog) {
              this.logGroupId = logGroupId;
              this.logStreamId = logStreamId;
              this.newBatchStructLog = newBatchStructLog;
              this.completed = completed;
          }
  
          @Override
          public void onSuccess(Result result) {
              LOGGER.info("Send logs successfully.");
              completed.getAndIncrement();
          }
  
          @Override
          public void onFailure(Throwable t) {
              if (t instanceof ResultFailedException) {
                  Result result = ((ResultFailedException) t).getResult();
                  LOGGER.error("Failed to send logs, logGroupId={}, logStreamId={}, result={}", logGroupId, logStreamId,
                      result);
              } else {
                  LOGGER.error("Failed to send log, e=", t);
              }
              completed.getAndIncrement();
          }
      }
  }

• Execute additional operations using callbacks.

  Callbacks are executed by internal threads of producers. The space occupied by data is not released until the execution is complete. To prevent the overall throughput from decreasing due to producer blocking, avoid performing time-consuming operations in callbacks. It is not recommended to call the send method in callbacks for retry, as the SDK provides a built-in retry mechanism.

  import java.util.List;
  import java.util.concurrent.CountDownLatch;
  import java.util.concurrent.ExecutorService;
  import java.util.concurrent.Executors;
  import java.util.concurrent.atomic.AtomicLong;
  import org.slf4j.Logger;
  import org.slf4j.LoggerFactory;
  import com.huaweicloud.lts.producer.Callback;
  import com.huaweicloud.lts.producer.Producer;
  import com.huaweicloud.lts.producer.Result;
  import com.huaweicloud.lts.producer.exception.ProducerException;
  import com.huaweicloud.lts.producer.model.log.LogItem;
  import com.huaweicloud.lts.producer.model.log.StructLogItem;
  
  public class SampleProducerWithCallback {
  
      private static final Logger LOGGER = LoggerFactory.getLogger(SampleProducerWithCallback.class);
  
      private static final ExecutorService EXECUTOR_SERVICE = Executors.newFixedThreadPool(
          Math.max(Runtime.getRuntime().availableProcessors(), 1));
  
      public static void main(String[] args) throws InterruptedException {
          Producer producer = ProducerUtil.getProducer();
          int nTask = 100;
          int size = 20;
  
          final AtomicLong completed = new AtomicLong(0);
  
          final CountDownLatch latch = new CountDownLatch(nTask);
  
          for (int i = 0; i < nTask; ++i) {
              EXECUTOR_SERVICE.submit(() -> {
                  try {
                      // Report standard logs.
                      List<LogItem> logItems = LogUtil.getLogItems(size);
                      producer.send("log_group_id", "log_stream_id", logItems,
                          new SampleCallback("log_group_id", "log_stream_id", logItems, completed));
  
                      // Report structured logs.
                      // StructLogItem structLog = LogUtil.getStructLog(size);
                      // producer.send("log_group_id", "log_stream_id", structLog,
                      //     new SampleCallback("log_group_id", "log_stream_id", structLog, completed));
  
                      // Report structured logs of the new version.
                      // List<LtsStructLog> newBatchStructLog = LogUtil.getNewBatchStructLog(1);
                      // producer.sendStructLog("log_group", "log_stream", newBatchStructLog,
                      //     new SampleCallback("log_group", "log_stream", completed, newBatchStructLog));
                  } catch (Exception e) {
                      LOGGER.error("Failed to send log, e=", e);
                  } finally {
                      latch.countDown();
                  }
              });
          }
          latch.await();
          EXECUTOR_SERVICE.shutdown();
  
          LogUtil.doSomething();
  
          try {
              producer.close();
          } catch (InterruptedException e) {
              LOGGER.warn("The current thread has been interrupted from close.");
          } catch (ProducerException e) {
              LOGGER.info("Failed to close producer, e=", e);
          }
  
          LOGGER.info("All log complete, completed={}", completed.get());
      }
  
      private static final class SampleCallback implements Callback {
  
          private static final Logger LOGGER = LoggerFactory.getLogger(SampleCallback.class);
  
          private final String logGroupId;
  
          private final String logStreamId;
  
          private List<LogItem> logItems = null;
  
          private StructLogItem structLog = null;
  
          private List<LtsStructLog> newBatchStructLog = null;
  
          private final AtomicLong completed;
  
          SampleCallback(String logGroupId, String logStreamId, List<LogItem> logItems, AtomicLong completed) {
              this.logGroupId = logGroupId;
              this.logStreamId = logStreamId;
              this.logItems = logItems;
              this.completed = completed;
          }
  
          SampleCallback(String logGroupId, String logStreamId, StructLogItem structLog, AtomicLong completed) {
              this.logGroupId = logGroupId;
              this.logStreamId = logStreamId;
              this.structLog = structLog;
              this.completed = completed;
          }
  
          SampleCallback(String logGroupId, String logStreamId, AtomicLong completed, List<LtsStructLog> newBatchStructLog) {
              this.logGroupId = logGroupId;
              this.logStreamId = logStreamId;
              this.completed = completed;
              this.newBatchStructLog = newBatchStructLog;
          }
  
          @Override
          public void onCompletion(Result result) {
              try {
                  if (result.isSuccessful()) {
                      LOGGER.info("Send log successfully.");
                  } else {
                      LOGGER.error("Failed to send logs, logGroupId={}, logStreamId={}, result={}", logGroupId, logStreamId,
                          result);
                  }
              } finally {
                  completed.getAndIncrement();
              }
          }
      }
  }

Step 5: Close the Producer

When no data needs to be sent or the current process is about to exit, you need to stop the producer to ensure that all data cached in the producer is processed. Currently, the producer provides two closing modes: safe close and limited close.

1. Safe close (recommended): The safe close method is close(). Using this method, the SDK closes the producer only after all data cached in the producer has been processed, all threads have been stopped, all registered callbacks have been executed, and all returned Futures have been set.
2. Limited close: If your callback may be blocked during execution but you want the close method to return quickly, you can use the limited close method, which is close(long timeoutMs). If the producer is not fully closed within the specified timeoutMs period, an IllegalStateException is thrown. This means that the cached data may be discarded before being processed, and any user-registered callbacks may not be executed.

Producer Performance Baseline

If one of the following performance baseline values is exceeded, log reporting may be abnormal.

Recommended ECS configurations:

• Instance flavor: General computing-plus c7.xlarge.2
• CPU: 4 vCPUs
• Memory: 8 GB
• Assured bandwidth: 100 Mbit/s
• OS: Huawei Cloud EulerOS release 2.0
• JVM: OpenJDK 64-Bit Server VM (build 17.0.7+7, mixed mode)

Test procedure (for a single producer):

• totalSizeInBytes: 104857600
• maxBlockMs: 0
• batchSizeThresholdInBytes: 1048576
• batchCountThreshold: 40960
• lingerMs: 2000
• ioThreadCount: specific to test cases
• Initial/Maximum heap size of the JVM: 1 GB
• Total number of sent logs: 100,000,000
• Total size of sent logs: about 50 GB

After the parameters are set based on the baseline values, use a Huawei Cloud ECS as the log reporting environment and report logs via the Huawei Cloud intranet service entry.

Example test results of the standard log reporting performance baseline of the SDK:

• This test involves reporting a batch of 4 logs, with a total size of about 2.2 KB. To simulate random data, the test uses random strings as log data, with each log being about 510 bytes.

  [
      {
          "contents" : [{
                  "log": "Random string of 510 bytes",
                  "log_time_ns" : 1637527157333902200
              }, {
                  "log": "Random string of 510 bytes",
                  "log_time_ns" : 1637527157333902200
              }
          ]
      }, {
          "contents" : [{
                  "log": "Random string of 510 bytes",
                  "log_time_ns" : 1637527157333902200
              }, {
                  "log": "Random string of 510 bytes",
                  "log_time_ns" : 1737527157333987400
              }
          ]
      }
  ]

• The following table lists the reference values of the performance baseline.

  Table 2 Standard log reporting performance baseline of the SDK

  I/O Threads	Data Throughput	Data Throughput Rate	CPU Usage
  2	7.8 MB/s	15,000 logs/s	9%
  4	15.4 MB/s	31,000 logs/s	19%
  6	19.7 MB/s	39,000 logs/s	27%

Example test results of the structured log reporting performance baseline of the SDK:

• This test involves reporting a batch of 4 logs, with a total size of about 2.2 KB. Each log contains 10 key-value pairs and the content and time fields. To simulate random data, the test uses random strings as log data, with each log being about 550 bytes.

  {
      "logs" : [
          {
              "contents" : [
                  {
                      "content": "sdk-log-new-struct-1",
                      "content_key_1":  "XmGFubcemwrceBWbZYRBTgohfxfFih",
                      "..." : "...",
                      "content_key_10":  "amchrqwPdigHopmAkNLvJtNxgiPUzh"
                  },{
                      "content": "sdk-log-new-struct-2",
                      "content_key_1":  "zOPDFRCNYsVznSgtnFejWFbaxklkMQ",
                      "..." : "...",
                      "content_key_10":  "mLzpbYcumXsIgYtQIbzizoACLtUgwS"
                  }
              ],
              "time": 1645374890235
          },{
              "contents" : [
                  {
                      "content": "sdk-log-new-struct-3",
                      "content_key_1":  "SaGsfDrQskJaHlciNAUXFyxiqCAqXe",
                      "..." : "...",
                      "content_key_10":  "wMQNuoVWonxVSsRsocQoDkEjcjiPio"
                  },{
                      "content": "sdk-log-new-struct-4",
                      "content_key_1":  "bHDjNmAvdiLAvWdxoETANqCYxhVMMk",
                      "..." : "...",
                      "content_key_10":  "scsxtrXrPUFYVARzOvbCxSofYZBsFV"
                  }
              ],
              "time": 1645374890235
          }
      ]
  }

• The following table lists the reference values of the performance baseline.

  Table 3 Structured log reporting performance baseline of the SDK

  I/O Threads	Data Throughput	Data Throughput Rate	CPU Usage
  2	18.7 MB/s	35,000 logs/s	13%
  4	34.6 MB/s	64,000 logs/s	25%
  6	42.7 MB/s	78,000 logs/s	32%

Example test results of the new-version structured log reporting performance baseline of the SDK:

• Log format: This test involves reporting a log that contains 10 key-value pairs and the content and logTime fields. To simulate random data, the test uses random strings as log data. Each log is about 540 bytes.

  content: sdk-log-new-struct-1
  content_key_1:  sshyaqKCfrAPCMpdlxroPuCedeuJ
  content_key_2:  RVFtqFBjBtTAHVvdHBYQsDsoogJc
  ...
  content_key_9:  zLKeuxDnzGtupeZrQKKIlkQemXvX  
  content_key_10:  fYxmtYxKNfBhRfqMbZEOfimlsAIo
  logTime: 1645390242169

• The following table lists the reference values of the performance baseline.

  Table 4 New-version structured log reporting performance baseline of the SDK

  I/O Threads	Data Throughput	Data Throughput Rate	CPU Usage
  2	23.7 MB/s	45,000 logs/s	11%
  4	45.6 MB/s	87,000 logs/s	23%
  6	54.7 MB/s	103,000 logs/s	30%

Summary:

• Most of the CPU time is spent on object serialization and compression, and CPU usage increases as throughput rises. However, in a typical daily environment, the average data traffic of a single ECS is 100 KB/s, resulting in negligible CPU consumption.
• Increasing the number of I/O threads significantly improves throughput, especially when the number of I/O threads is less than the number of available processors.
• Increasing the totalSizeInBytes value has minimal impact on the throughput but can increase CPU usage. You are advised to use the default value.
• If the reported logs exceed the capacity of a single producer:
  1. To ensure steady SDK log reporting, you are advised to separate logs into multiple log streams and use multiple producers to distribute the traffic.
  2. If maxBlockMs is set to 0, the SDK will be in a non-blocking state, which will trigger the protection mechanism to automatically downgrade. As a result, some logs may be discarded.
  3. If maxBlockMs is set to a value greater than 0, the SDK will be in a blocked state with a blocking duration of maxBlockMs, potentially blocking the producer.send() method used for log sending.

Obtaining Parameters

• Regions

  Table 5 Regions

  Region Name	Region
  CN North-Beijing2	cn-north-2
  CN North-Beijing4	cn-north-4
  CN North-Beijing1	cn-north-1
  CN East-Shanghai2	cn-east-2
  CN East-Shanghai1	cn-east-3
  CN South-Guangzhou	cn-south-1
  CN South-Shenzhen	cn-south-2
  CN Southwest-Guiyang1	cn-southwest-2
  AP-Singapore	ap-southeast-3

• To obtain a log group's ID, choose Log Management in the navigation pane of the LTS console and hover over the target log group's name.
• To obtain a log stream's ID, click next to the corresponding log group and hover over the target log stream's name.