Updated on 2024-11-06 GMT+08:00

Run Logs (Old Version)

IoTDA can record the connections with devices and applications and report logs to Log Tank Service (LTS). LTS provides real-time query, massive log storage, log structuring, visualization, and analysis capabilities. It provides a free quota of 500 MB per month. When this quota is used up, you will be billed for any excess usage on a pay-per-use basis. For details, see What Is Log Tank Service?

Currently, only service run logs of MQTT devices can be recorded. For details, see the following table.

Table 1 Service types

Service Type

Service Flow

Device status

Device going online

Device going offline

Device message

Application requesting message delivery to devices

Platform delivering messages to devices

Device reporting messages

Device command

Application requesting command delivery to devices

Platform delivering commands to devices

Platform receiving device response to commands

Device property

Application requesting device property modification

Platform delivering property modification to devices

Device reporting properties

Gateway reporting device properties in batches

Device event

Platform notifying a gateway of new child device connection

Platform notifying a gateway of child device deletion

Gateway synchronizing child device information

Gateway updating child device statuses

Gateway requesting for adding child devices

Platform responding to a request for adding child devices

Gateway requesting for deleting child devices

Platform responding to a request for deleting child devices

Gateway updating child device statuses

Platform responding to a request for updating child device statuses

Platform delivering a command to obtain version information

Device reporting the software and firmware versions

Platform delivering an upgrade command

Device reporting the upgrade status

Device requesting a URL for file upload

Platform delivering a temporary URL for file upload

Device reporting file upload results

Device requesting a URL for file download

Platform delivering a temporary URL for file download

Device reporting file download results

Device requesting time synchronization

Platform responding to a request for time synchronization

Device reporting information

Platform delivering a log collection notification

Device reporting log content

Platform delivering a configuration notification

Device reporting the configuration response

Device downloading upgrade package

Batch task

Subtask execution result of a batch task

Device self-registration

Device self-registration result.

Device authentication

Custom device authentication result

Table 2 Format of batch task run logs

Field

Description

appId

Application ID.

deviceId

Device ID.

categoryName

Log type: batch.task.

operation

Action. Set this parameter to the ID of a batch task. For details, see Create a Batch Task.

request

Request content in JSON format.

{

"task_type": "createDevices", / / Task type

"package_id": "f2303267a6e8f0053037c2a9", // Software/Firmware upgrade package

"package_ids": ["65f3ebe2682b9f4bcc38baad"] // Software/Firmware upgrade package

}

NOTE:

When the batch task type is softwareUpgrade or firmwareUpgrade, the package_id and package_ids parameters are supported.

response

Response content in JSON format.

Success:

{

"output" : "xxxxxxxxxxxx"

}

Failure:

{

"error" : {

"error_code" : "IOTDA.XXXXX",

"error_msg" : "XXXXX."

}

}

status

Execution result. Subtask status. Options: Success, Fail, Stopped, or Removed.

Table 3 Format of non-batch tasks

Field

Description

recordTime

Log collection time. The time format is yyyy-MM-dd'T'HH:mm:ss,SSS'Z'.

Example: 2020-06-16T09:24:45,708Z

deviceId

Device ID.

requestId

Request ID.

categoryName

Options:

device.status

device.message

device.command

device.property

device.event

device.auth

device.provisioning

operation

Operation name. Example: API URL or MQTT message topic.

request

Request parameter of an operation. Example: API request body.

response

Operation result. Example: API response body or error information.

result

Operation status code.

Procedure

  1. Access the IoTDA service page and click Access Console. Click the target instance card.
  2. In the navigation pane, choose O&M > Run Logs, and click Configure Run Log.

    Figure 1 Run log - Old version configuration

  3. On the displayed dialog box, select the service type for data collection and click OK.

    Figure 2 Run log - Log switch of old version configuration

  4. Create a run log forwarding rule to forward the collected log data to other cloud services, so that you can view and process it. You are advised to forward log data to LTS. The following uses LTS as an example.

    1. In the navigation pane, choose Rules > Data Forwarding, and click Create Rule.
    2. Configure parameters by referring to the following table, and click Create Rule.
      Table 4 Creating a rule

      Parameter

      Description

      Rule Name

      Specify the name of a rule to create.

      Description

      Describe the rule.

      Data Source

      Select Run log.

      Trigger

      After the data source is selected, the platform automatically matches the trigger event.

      Resource Space

      You can select a single resource space or all resource spaces.

    3. Click the Set Forwarding Target tab, and then click Add to set a forwarding target.
      Table 5 Setting the forwarding target

      Parameter

      Description

      Forwarding Target

      Select Log Tank Service (LTS).

      Region

      Currently, log data can be forwarded only to LTS in the same region.

      Log Group/log Stream

      Select a log group and log stream of LTS. If no log group or log stream is available, create them by referring to Managing Log Groups and Managing Log Streams.

      Logs are retained in the log group for 7 days by default. Logs older than the retention period will be automatically deleted. For long-term storage, you can transfer logs to Object Storage Service (OBS) buckets. For details, see Transferring Logs to OBS.

      LTS provides a free quota of 500 MB per month. By default, it continues to collect logs when the quota is used up. You will be billed for the excess usage on a pay-per-use basis. You can log in to the LTS console and choose Configuration Center to disable this function.

    4. Click OK, and then click Enable Rule to forward run logs to LTS.

  5. Log in to the LTS console and choose Log Management.
  6. Select the log group and log stream created in 3 to view the logs reported by IoTDA. Search for raw and target logs by referring to Log Search. For example, search for logs by device ID and service type.

    Figure 3 Run logs - Checking the log list
    Figure 4 Run logs - Searching for run logs

  7. On the log stream details page, click the Log Analysis tab. On the log structuring page that is displayed, select JSON.

    Figure 5 Run logs - Log analysis

  8. Select a sample log that contains time, device ID, request ID, service type, operation, request parameter, result information, and execution status. Click the button of intelligent extraction to modify the field names, for example, recordTime, requestId, deviceId, categoryName, operation, request, response and result. Change the field type to string. Click Save to complete the log structuring configuration. For details, see Structuring Logs.

    Figure 6 Run logs - Configuring log structuring

  9. After the raw logs are structured, the service will respond. On the Log Management page, select the log group and log stream created in 3, and click the button for log analysis to check the structured target logs. You can query logs using SQL statements. For example, to query logs by device ID and service type, run select * where deviceId = '{{deviceId}}' and categoryName = 'device.status'.

    Figure 7 Run logs - Log analysis and search