Help Center> >Glossary

A-E

    • A-E
      Data Ingestion Service

      Data Ingestion Service (DIS) addresses the challenge of transmitting data within the cloud and from outside the cloud to inside the cloud. With DIS, you can build custom applications to process or analyze streaming data for specific needs.

      data record

      A data record is the unit of data stored in a DIS stream. A data record is composed of a sequence number, partition key, and data blob. Data blobs are key data added by data producers to DIS streams. The payload of a data blob can be up to 1 MB before Base64 encoding.

      DIS

      See Data Ingestion Service

    P-T

      • P
        partition

        Data records in DIS streams are distributed into partitions. Partitions are the base throughput unit of a DIS stream. The total capacity of a stream is the sum of the capacities of its partitions. One partition supports the data read rate of 2 MB/sec and the data write rate of 1000 records/sec and 1 MB/sec. When creating a DIS stream, you are expected to specify the number of partitions needed within your stream.

        partition key

        A partition key is used to segregate and route records to different partitions of a DIS stream. A partition key is specified by your data producer while adding data to a DIS stream. For example, assuming you have a stream with two partitions (partition 1 and partition 2). You can configure your data producer to use two partition keys (key A and key B) so that all records with key A are added to partition 1 and all records with key B are added to partition 2.

      • S
        sequence number

        Each data record has a sequence number that is unique within its partition. The sequence number is assigned by DIS when a data producer calls PutRecord or PutRecords operation to add data to a DIS stream. Sequence numbers for the same partition key generally increase over time; the longer the time period between write requests (PutRecord or PutRecords requests), the larger the sequence numbers become.