Updated on 2022-02-22 GMT+08:00

Adaptation to the Native KafkaConsumer API

Table 1 API adaptation description

Native KafkaConsumer

Type

DISKafkaConsumer

Description

Set<TopicPartition> assignment()

API

Supported

Obtain information about the consumed stream and partition.

Set<String> subscription()

API

Supported

Obtain the name of the stream that has been subscribed to by the consumer.

void assign(Collection<TopicPartition> var1)

API

Supported

Allocate a specified partition.

void subscribe(Collection<String> var1)

API

Supported

Subscribe to a specified stream.

void subscribe(Collection<String> var1, ConsumerRebalanceListener var2)

API

Supported

Subscribe to a specified stream and supports callback of ConsumerRebalanceListener.

void subscribe(Pattern var1, ConsumerRebalanceListener var2)

API

Supported

Subscribe to all streams that match wildcards and supports callback of ConsumerRebalanceListener.

void unsubscribe()

API

Supported

Cancels all subscription.

ConsumerRecords<K, V> poll(long var1)

API

Supported

The message is obtained.

Checksum (CRC32 check value of the message), serializedKeySize (byte length after key serialization), and serializedValueSize (byte length after key serialization).

void commitSync()

API

Supported

Commit the consumed offsets synchronously.

void commitSync(final Map<TopicPartition, OffsetAndMetadata> offsets)

API

Supported

Commit the specified offset synchronously.

void commitAsync()

API

Supported

Commit the consumed offsets asynchronously.

public void commitAsync(OffsetCommitCallback callback)

API

Supported

Commit the current consumed offset asynchronously and supports callback of OffsetCommitCallback.

void commitAsync(Map<TopicPartition, OffsetAndMetadata> offsets, OffsetCommitCallback callback)

API

Supported

Commit the specified offset asynchronously and supports callback of OffsetCommitCallback.

void seek(TopicPartition partition, long offset)

API

Supported

Set the specified offset for a partition.

void seekToBeginning(Collection<TopicPartition> partitions)

API

Supported

Set the earliest value for the partition offset.

void seekToEnd(Collection<TopicPartition> partitions)

API

Supported

Set the latest value for the partition offset.

long position(TopicPartition partition)

API

Supported

Obtain the offset of the current consumed data in the partition.

OffsetAndMetadata committed(TopicPartition partition)

API

Supported

Obtain the committed offset of the partition.

List<PartitionInfo> partitionsFor(String topic)

API

Supported

Obtain the partition information of the stream, but leader, replicas, and inSyncReplicas in PartitionInfo are not implemented.

Map<String, List<PartitionInfo>> listTopics()

API

Supported

Obtain the stream information, but leader, replicas, and inSyncReplicas in PartitionInfo are not implemented.

void pause(Collection<TopicPartition> partitions)

API

Supported

Suspends the partition consumption.

void resume(Collection<TopicPartition> partitions)

API

Supported

Resumes the partition consumption.

Set<TopicPartition> paused()

API

Supported

Obtain all partitions that stop being consumed.

close()

API

Supported

Disable the consumer.

Map<MetricName, ? extends Metric> metrics()

API

Not supported

Obtain statistics.

wakeup()

API

Not supported

The internal implementation principles are different.

group.id

Parameter

Supported

Consumer group ID.

client.id

Parameter

Supported

client.id of each consumer must be unique. If client.id is not specified, the dis kafka consumer will generate a UUID as a client.id.

key.deserializer

Parameter

Supported

The meaning of this parameter is the same as that in Kafka. The default value is StringDeserializer. In Kafka, StringDeserializer has no default value, and you must configure a value for it.

value.deserializer

Parameter

Supported

The meaning of this parameter is the same as that in Kafka. The default value is StringDeserializer. In Kafka, this parameter has no default value, and you must configure a value for it.

enable.auto.commit

Parameter

Supported

The default value is the same as that in Kafka, which is true.

  • true: The automatic offset committing is enabled and offsets are automatically committed at an interval of auto.commit.interval.ms.
  • false: Offsets are not automatically committed.

auto.commit.interval.ms

Parameter

Supported

Interval for automatically committing offsets, in milliseconds. The default value is 5000.

auto.offset.reset

Parameter

Supported

The default value is the same as that in Kafka, which is latest.

This parameter is used to automatically set the offset position when there is no initial offset or the offset is incorrect.

  • earliest: The offset is automatically reset to the earliest value.
  • latest: The offset is automatically reset to the latest value.
  • none: If the previous offset is not found in the consumer group, an exception is issued to the consumer.

Others

Parameter

Not supported

-