Help Center> Data Ingestion Service> SDK Reference> Getting Started with SDK> Using the C SDK> Downloading Streaming Data
View PDF

Downloading Streaming Data

Sample Code

Data can be uploaded in base64 or protobuf mode.

  • To set the upload mode to base64, use DISSetSerializedMode("base64").
  • To set the upload mode to protobuf, use DISSetSerializedMode("protobuf").
  • To obtain the upload mode set by the user, use DISGetSerializedMode().

The main codes for downloading streaming data are as follows: The value of streamName must be the same as the value of Stream Name in section "Creating a Stream."

char *streamName = "myStream";
char *projectId = "d575b0b740e54221aeb9a165653b103d";
char *region = "southchina";
char *host = "XXX.XXX.XXX.XXX:XXX";
int ret = 0;
DISResponseInfo RspInfo = { 0 };
DISCursor  NestCursor = { 0 };
printf("===================%s Begin=======================\n", __FUNCTION__);
DISGetCursor tempCursor = { 0 };
tempCursor.streamName = streamName;
tempCursor.partitionId = "shardId-0000000002";
tempCursor.cursorType = DISCursorTypeAtSeqNumber;
tempCursor.startingSequenceNumber = "1";
ret = DisGetCursor(host, projectId, region, &tempCursor, &RspInfo);
if (ret != 0 || RspInfo.HttpResponseCode >= 300)
{
printf("GetCursor Error: %d, the http code id %d, Error Code :%s, message: %s\r\n", ret, RspInfo.HttpResponseCode, RspInfo.ErrorCode, RspInfo.ErrorDetail);
printf("===================%s End=======================\n", __FUNCTION__);
return;
}
printf("===================Got Cursor=======================\n");
RspInfo.HttpResponseCode = 0;
ret = GetRecords(host, projectId, region, tempCursor.streamName, tempCursor.cursorResult.partitionCursor, 0, &NestCursor, GetRecordCallBack, &RspInfo);
if (ret != 0 || RspInfo.HttpResponseCode >= 300)
{
printf("GetRecords Error: %d, the http code id %d, Error Code :%s, message: %s\r\n", ret, RspInfo.HttpResponseCode, RspInfo.ErrorCode, RspInfo.ErrorDetail);
printf("===================%s Begin=======================\n", __FUNCTION__);
return;
}

Before downloading streaming data, you need to obtain cursor data. Obtaining a Data Cursor describes the cursor parameters.

Table 1 Cursor parameters

Parameter

Mandatory

Type

Description

partitionCursor

Yes

Char *

Data cursor.

Value: 1 to 512 characters.

NOTE:

The validity period of a cursor is 5 minutes.

The GetRecords method has a callback function GetRecordCallBack, which is used to process responses. This function can be customized. An example is provided for reference only.

DISStatus GetRecordCallBack(char *streamName, const DISGetRecords *record)
{
if (NULL == streamName || NULL == record || NULL == record->seqNumber || NULL == record->partitionId)
{
printf("%s the input param invalid\r\n", __FUNCTION__);
return 0;
}

printf("the partition key is %s\r\n", record->partitionId);
printf("the seqNumber is %s\r\n", record->seqNumber);
//record->data.data indicates the start address of the data, and record->data.stringLen indicates the data length.
printf("the data is %d: %s\r\n", record->data.stringLen, record->data.data);
return 0;
}

The following table describes the response parameters:

Parameter

Type

Description

streamName

char *

Stream name.

record

DISGetRecords *

Record details.
Table 2 DISGetRecords parameter description

Parameter

Type

Description

partition_key

String

Partition key set when data is being uploaded.

NOTE:

If partition_key is specified when data is uploaded, partition_key is returned when data is downloaded. If partition_id instead of partition_key is specified when data is uploaded, no partition_key is returned.

seqNumber

Char *

Sequence number. A sequence number is the unique identifier of each record. DIS automatically allocates a sequence number when the data producer calls the PutRecords operation to add data to the DIS stream. The sequence number of the same partition key usually changes with time. A longer interval between PutRecords requests results in a larger sequence number.

timestamp

long long

Timestamp when the record is written to DIS.

timestampType

Char *

Type of the timestamp.

The value is CreateTime, specifying the creation time.

pucReserved

void *

Empty pointer, used to extend the response body.
Table 3 DISString parameter description

Parameter

Mandatory

Type

Description

stringLen

Yes

long

Length of the data to be uploaded.

data

Yes

Char *

Data to be uploaded.

Execution Result

Information similar to the following is displayed on the console:

the partition key is 1
the seqNumber is 0
the data is 10: aaaaaaaaaa
the partition key is 1
the seqNumber is 1
the data is 55: 6o96l4h0fG05Zm4sXUKEI1fyFHz68pYISa7AAdYsi373CHi2W5gl6S6
the partition key is 1
the seqNumber is 2
the data is 62: A72p5Cw04c6dy4M756Rmehm2DyKzx4Cas1OJSUliv52Q6x3lxxCi6z0Zs1b0la
Parent topic: Using the C SDK