Adding or Deleting Configurations for a Topic

Symptom

Configure or modify a specific topic when using Kafka.

Parameters that can be modified at the topic level:

cleanup.policy
compression.type
delete.retention.ms
file.delete.delay.ms
flush.messages
flush.ms
index.interval.bytes
max.message.bytes
min.cleanable.dirty.ratio
min.insync.replicas
preallocate
retention.bytes
retention.ms
segment.bytes
segment.index.bytes
segment.jitter.ms
segment.ms
unclean.leader.election.enable

Procedure

• Prerequisites

  The Kafka client has been installed.

• Procedure
  1. Log in to the node where the Kafka client is installed as the client installation user.
  2. Switch to the Kafka client installation directory, for example, /opt/client.

     cd /opt/client

  3. Run the following command to configure environment variables:

     source bigdata_env

  4. Run the following command to perform user authentication (skip this step for a cluster in common mode):

     kinit Component service user

  5. Run the following command to switch to the Kafka client installation directory:

     cd Kafka/kafka/bin

  6. Run the following commands to configure and delete a topic:

     kafka-topics.sh --alter --topic <topic_name> --zookeeper <zookeeper_host:port>/kafka --config <name=value>

     kafka-topics.sh --alter --topic <topic_name> --zookeeper <zookeeper_host:port>/kafka --delete-config <name>

     Example:

     kafka-topics.sh --alter --topic test1 --zookeeper 192.168.100.100:2181/kafka --config retention.ms=86400000

     kafka-topics.sh --alter --topic test1 --zookeeper 192.168.100.100:2181/kafka --delete-config retention.ms

  7. Run the following command to query topic information:

     kafka-topics.sh --describe -topic <topic_name> --zookeeper <zookeeper_host:port>/kafka