Scenario 8: Converting Data to Microsecond-Level Standard ISO 8601 Timestamps

You can use LTS's data processing feature to enable high-precision timestamps. For raw logs that contain standard ISO 8601 time fields, use the e_set operation function to parse these fields into microsecond-level timestamps.

• Raw log

  {
    "source": "1.2.3.4",
    "time": 1704983810,
    "topic": "test",
    "log_time":"2024-01-11 23:10:43.992847200"
  }

• Processing rule

  e_set(
      "time", dt_parsetimestamp(v("log_time"), tz="Asia/Shanghai"), mode="overwrite",
  )
  e_set("tmp_ms", dt_prop(v("log_time"), "microsecond"))
  e_set(
      "time_ns_part", op_mul(ct_int(v("tmp_ms")), 1000),
  )

• Processing result

  {
  	"time_ns_part": 992847000,
  	"tmp_ms": 992847,
  	"topic": "test",
  	"source": "1.2.3.4",
  	"time": 1704985843,
  	"log_time": "2024-01-11 23:10:43.992847200"
  }