Updated on 2022-12-14 GMT+08:00

Adding or Deleting Configurations for a Topic

Symptom

Configure or modify a specific topic when using Kafka.

Parameters that can be modified at the topic level:

cleanup.policy
compression.type
delete.retention.ms
file.delete.delay.ms
flush.messages
flush.ms
index.interval.bytes
max.message.bytes
min.cleanable.dirty.ratio
min.insync.replicas
preallocate
retention.bytes
retention.ms
segment.bytes
segment.index.bytes
segment.jitter.ms
segment.ms
unclean.leader.election.enable

Procedure

  • Prerequisites

    The Kafka client has been installed.

  • Procedure
    1. Log in to the node where the Kafka client is installed as the client installation user.
    2. Switch to the Kafka client installation directory, for example, /opt/client.

      cd /opt/client

    3. Run the following command to configure environment variables:

      source bigdata_env

    4. Run the following command to perform user authentication (skip this step for a cluster in common mode):

      kinit Component service user

    5. Run the following command to switch to the Kafka client installation directory:

      cd Kafka/kafka/bin

    6. Run the following commands to configure and delete a topic:

      kafka-topics.sh --alter --topic <topic_name> --zookeeper <zookeeper_host:port>/kafka --config <name=value>

      kafka-topics.sh --alter --topic <topic_name> --zookeeper <zookeeper_host:port>/kafka --delete-config <name>

      Example:

      kafka-topics.sh --alter --topic test1 --zookeeper 192.168.100.100:2181/kafka --config retention.ms=86400000

      kafka-topics.sh --alter --topic test1 --zookeeper 192.168.100.100:2181/kafka --delete-config retention.ms

    7. Run the following command to query topic information:

      kafka-topics.sh --describe -topic <topic_name> --zookeeper <zookeeper_host:port>/kafka