Downloading Streaming Data

Downloading Stream Data in JSON Format

Initialize a DIS SDK client instance as instructed in Initializing a DIS SDK Client Instance.

Configure the following parameters:

   
        streamname=" "   #Name of the stream.
startSeq=' 0' # Sequence number.
partitionId="shardId-0000000000"

Run getCursor_test to change the value of cursorType to AT_SEQUENCE_NUMBER.

After configuring the parameters, run the getRecords_sample.py file to call getRecords_test. The following is an example response:

   
        200
{'next_partition_cursor': 'eyJnZXRJdGVyYXRvclBhcmFtIjp7InN0cmVhbS1uYW1lIjoiZGlzX3Rlc3QxIiwicGFydGl0aW9uLWlkIjoic2hhcmRJZC0wMDAwMDAwMDAwIiwiY3Vyc29yLXR5cGUiOiJBVF9TRVFVRU5DRV9OVU1CRVIiLCJzdGFydGluZy1zZXF1ZW5jZS1udW1iZXIiOiI2In0sImdlbmVyYXRlVGltZXN0YW1wIjoxNTU0NzA2NTc5MzA5fQ', 'records': [{'sequence_number': '4', 'data': b'xxxxx', 'partitionKey': '0', 'timestamp': 1554705842558, 'timestamp_type': 'CreateTime'}, {'sequence_number': '5', 'data': b'xxxxx', 'partitionKey': '0', 'timestamp': 1554705842558, 'timestamp_type': 'CreateTime'}]}

Downloading Streaming Data in Protobuf Format

Initialize a DIS SDK client instance as instructed in Initializing a DIS SDK Client Instance.

Initialize the DIS client and add the bodySerializeType parameter as follows:
cli = disclient(endpoint='', ak='', sk='', projectid='', region='',bodySerializeType='protobuf')

Configure the following parameters:

   
        streamname=" "   #Name of the stream.
partitionId="shardId-0000000000"

Run getCursor_test to change the value of cursorType to AT_SEQUENCE_NUMBER.

The test_0 method is used to obtain a cursor and the test method is used to download data. Compared to test_0, test has the bodySerializeType="protobuf" parameter, which is not provided in test_0.

After configuring the parameters, run the protobuf_getrecords_sample.py file to call getRecords_test. The following is an example response:

   
        200
{'next_partition_cursor': 'eyJnZXRJdGVyYXRvclBhcmFtIjp7InN0cmVhbS1uYW1lIjoiZGlzX3Rlc3QxIiwicGFydGl0aW9uLWlkIjoic2hhcmRJZC0wMDAwMDAwMDAwIiwiY3Vyc29yLXR5cGUiOiJBVF9TRVFVRU5DRV9OVU1CRVIiLCJzdGFydGluZy1zZXF1ZW5jZS1udW1iZXIiOiI2In0sImdlbmVyYXRlVGltZXN0YW1wIjoxNTU0NzA2NTc5MzA5fQ', 'records': [{'sequence_number': '4', 'data': b'xxxxx', 'partitionKey': '0', 'timestamp': 1554705842558, 'timestamp_type': 'CreateTime'}, {'sequence_number': '5', 'data': b'xxxxx', 'partitionKey': '0', 'timestamp': 1554705842558, 'timestamp_type': 'CreateTime'}]}