Help Center> Data Ingestion Service> API Reference> API Description> Creating a DIS Stream
View PDF

Creating a DIS Stream

Function

This API is used to create a stream.

  • When creating a channel, you need to specify the stream type (common or advanced), number of partitions, and source data type.
  • A maximum of 10 advanced stream partitions and 50 common stream partitions can be created for an account by default. You can submit a service ticket to increase the quota.

URI

  • URI format

    POST /v2/{project_id}/streams

  • Parameter description

    None

Request

  • Example request
    • Create a stream whose source data type is BLOB. 
      POST https://{endpoint}/v2/{project_id}/streams                
{  
	"stream_name": "dis-DLpR",
	"partition_count": 1,
	"stream_type": "COMMON",
	"data_type": "BLOB",
	"data_duration": 24
}
    • Create a stream whose source data type is JSON or CSV. 
      POST https://{endpoint}/v2/{project_id}/streams       
{  
	"stream_name": "dis-DLpR",
	"partition_count": 1,
	"stream_type": "COMMON",
        "data_type": "JSON",
	"data_duration": 24,
        "data_schema":"{\"type\":\"record\",\"name\":\"RecordName\",\"fields\":[{\"name\":\"id\",\"type\":\"string\",\"doc\":\"Type inferred from '\\\"2017/10/11 11:11:11\\\"'\"},{\"name\":\"info\",\"type\":{\"type\":\"array\",\"items\":{\"type\":\"record\",\"name\":\"info\",\"fields\":[{\"name\":\"date\",\"type\":\"string\",\"doc\":\"Type inferred from '\\\"2018/10/11 11:11:11\\\"'\"}]}},\"doc\":\"Type inferred from '[{\\\"date\\\":\\\"2018/10/11 11:11:11\\\"}]'\"}]}"}
}
  • Parameter description
Table 1 Parameter description

Parameter

Mandatory

Type

Description

stream_name

Yes

String

Name of the DIS stream.

Each DIS stream has a unique name. A stream name is 1 to 64 characters in length. Only letters, digits, hyphens (-), and underscores (_) are allowed.

stream_type

No

String

Stream type. Possible values:

  • Common: a common stream. The bandwidth is 1 MB/s.
  • ADVANCED: an advanced stream. The bandwidth is 5 MB/s.

Default value: COMMON

partition_count

Yes

Int

Quantity of the partitions into which data records in the newly created DIS stream will be distributed.

Partitions are the base throughput unit of a DIS stream.

The value range varies depending on the value of stream_type.

  • If stream_type is not specified or set to COMMON, the value of partition_count is an integer from 1 to 50. If the tenant has created N common partitions, the maximum value of partition_count is 50-N.
  • If stream_type is set to ADVANCED, the value of partition_count is an integer from 1 to 10. If the tenant has created N advanced partitions, the maximum value of partition_count is 10-N.

data_duration

No

Int

Period of time for which data is retained in the DIS stream.

Value range: 24 to 72

Unit: hour

Default value: 24

If this parameter is left unspecified, the default value will be used.

data_type

No

String

Source data type.

Possible values:
  • BLOB: a collection of binary data stored as a single entity in a database management system.
  • JSON: an open-standard file format that uses human-readable text to transmit data objects consisting of attribute–value pairs and array data types.
  • CSV: a simple text format for storing tabular data in a plain text file.

    Default value: BLOB

data_schema

No

String

Source data structure that defines JOSN and CSV formats. It is described in the syntax of Avro. For details about Avro, see http://avro.apache.org/docs/current/#schemas.

NOTE:

This parameter is mandatory when Dump File Format is Parquet or CarbonData.

Response

  • If the DIS stream was successfully created, a 201 response with an empty response body is returned.
  • If the DIS stream failed to be created, identify the failure cause according to the response body and the instructions in Error Codes.

Status Code

  • Normal

    201 Created

  • Failed

    For more information, see Error Codes.

Parent topic: API Description