Help Center> Cloud Stream Service> Eco Development> Cloud Ecosystem: CloudTable> OpenTSDB Sink Stream
View PDF

OpenTSDB Sink Stream

Overview

CS exports the job output data to OpenTSDB of CloudTable. OpenTSDB is a distributed, scalable time series database based on HBase. It stores time series data. Time series data refers to the data collected at different time points. This type of data reflects the change status or degree of an object over time. OpenTSDB supports data collection and monitoring in seconds, permanent storage, index, and queries. It can be used for system monitoring and measurement as well as collection and monitoring of IoT data, financial data, and scientific experimental results.

CloudTable is a distributed, scalable, and fully-hosted key-value data storage service based on Apache HBase. It provides CS with high-performance random read and write capabilities, which are helpful when applications need to store and query a massive amount of structured data, semi-structured data, and time series data. CloudTable applies to IoT scenarios and storage and query of massive volumes of key-value data. For more information about CloudTable, see the CloudTable Service User Guide.

Prerequisites

Ensure that OpenTSDB has been enabled on the CloudTable clusters. For details about how to enable OpenTSDB, see Enabling OpenTSDB in the CloudTable Service User Guide.

Syntax

Syntax

CREATE SINK STREAM stream_id (attr_name attr_type (',' attr_name attr_type)* )WITH (type = "opentsdb",region = "",cluster_id = "",tsdb_metrics = "", tsdb_timestamps = "",tsdb_values = "",tsdb_tags = "",batch_insert_data_num = "")

Description

Table 1 Syntax description

Parameter

Mandatory

Description

type

Yes

Output channel type. Value opentsdb indicates that data is stored to CloudTable (OpenTSDB).

region

Yes

Region to which CloudTable belongs.

cluster_id

Yes

ID of the cluster to which data is to be inserted.

tsdb_metrics

Yes

Metric of a data point, which can be specified through parameter configurations.

tsdb_timestamps

Yes

Timestamp of a data point. The data type can be TIMESTAMP, LONG, or INT. Only dynamic columns are supported.

tsdb_values

Yes

Value of a data point. The data type can be SHORT, INT, LONG, FLOAT, DOUBLE, or STRING. Dynamic columns or constant values are supported.

tsdb_tags

Yes

Tags of a data point. Each of tags contains at least one tag value and up to eight tag values. Tags of the data point can be specified through parameter configurations.

batch_insert_data_num

No

Number of data records to be written in batches at a time. The value must be a positive integer. The upper limit is 100. The default value is 8.

Precautions

If a configuration item can be specified through parameter configurations, one or more columns in the record can be used as part of the configuration item. For example, if the configuration item is set to car_$ {car_brand} and the value of car_brand in a record is BMW, the value of this configuration item is car_BMW in the record.

Example

Output data of stream weather_out to CloudTable (OpenTSDB).

CREATE SINK STREAM weather_out (
  timestamp_value LONG, /* Time */
  temperature FLOAT, /* Temperature value */
  humidity FLOAT, /* Humidity */
  location STRING /* Location */
)
WITH (
  type = "opentsdb",
  region = "cn-north-1" ,
  cluster_id = "e05649d6-00e2-44b4-b0ff-7194adaeab3f",
  tsdb_metrics = "weather",
  tsdb_timestamps = "${timestamp_value}",
  tsdb_values = "${temperature}; ${humidity}",
  tsdb_tags = "location:${location},signify:temperature; location:${location},signify:humidity",
  batch_insert_data_num = "10"
);