Help Center/ MapReduce Service/ Troubleshooting/ Using Kafka/ Kafka Topics Cannot Be Deleted
Updated on 2023-01-11 GMT+08:00
View PDF

Kafka Topics Cannot Be Deleted

Symptom

When running the following command on the Kafka client to delete topics, it is found that the topics cannot be deleted.

kafka-topics.sh --delete --topic test --zookeeper 192.168.234.231:2181/kafka

Possible Causes

  • The command for connecting the client to ZooKeeper is incorrect.
  • Kafka is abnormal and some Kafka nodes are stopped.
  • Perform the following operations when Kafka server configurations cannot be deleted.
  • Perform the following operations when Kafka configurations are automatically created and the Producer is not stopped.

Cause Analysis

  1. After the client command is run, the "ZkTimeoutException" exception is reported. 
    [2016-03-09 10:41:45,773] WARN Can not get the principle name from server 192.168.234.231 (org.apache.zookeeper.ClientCnxn)
Exception in thread "main" org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 30000
at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:880)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
at kafka.admin.TopicCommand$.main(TopicCommand.scala:51)
at kafka.admin.TopicCommand.main(TopicCommand.scala)

    For details about the solution, see 1.

  2. Run the following query command on the client:
    kafka-topics.sh --list --zookeeper 192.168.0.122:2181/kafka 
    test - marked for deletion

    On Manager, check the running status of Kafka Broker instances.

    Run the cd /var/log/Bigdata/kafka/broker command to go to the log directory of node RunningAsController. Locate ineligible for deletion: test in the controller.log file.

    2016-03-09 11:11:26,228 | INFO  | [main] | [Controller 1]: List of topics to be deleted:  | kafka.controller.KafkaController (Logging.scala:68)
2016-03-09 11:11:26,229 | INFO  | [main] | [Controller 1]: List of topics ineligible for deletion: test | kafka.controller.KafkaController (Logging.scala:68)
  3. On Manager, view the delete.topic.enable status of Broker.

    For details about the solution, see 2.

  4. Run the following query command on the client: 
    kafka-topics.sh --describe –topic test --zookeeper 192.168.0.122:2181/kafka

    Go to the log directory of node RunningAsController. Locate marked ineligible for deletion in the controller.log file.

    2016-03-10 11:11:17,989 | INFO  | [delete-topics-thread-3] | [delete-topics-thread-3], Handling deletion for topics test | kafka.controller.TopicDeletionManager$DeleteTopicsThread (Logging.scala:68)
2016-03-10 11:11:17,990 | INFO  | [delete-topics-thread-3] | [delete-topics-thread-3], Not retrying deletion of topic test at this time since it is marked ineligible for deletion | kafka.controller.TopicDeletionManager$DeleteTopicsThread (Logging.scala:68)
  5. On Manager, query the Broker status.

    It can be seen that a Broker is in the Stopped state. In this case, delete the topic and ensure that Brokers where partitions of the topic reside must be in the Good state.

    For details about the solution, see 3.

  6. Go to the log directory of node RunningAsController. Locate Deletion successfully in the controller.log file. If New topics:[Set(test)] is displayed again, it indicates that the topic is created again. 
    2016-03-10 11:33:35,208 | INFO  | [delete-topics-thread-3] | [delete-topics-thread-3], Deletion of topic test successfully completed | kafka.controller.TopicDeletionManager$DeleteTopicsThread (Logging.scala:68)

2016-03-10 11:33:38,501 | INFO  | [ZkClient-EventThread-19-192.168.0.122:2181,160.172.0.52:2181,160.172.0.51:2181/kafka] | [TopicChangeListener on Controller 3]: New topics: [Set(test)], deleted topics: [Set()], new partition replica assignment
  7. Use Manager to query the topic creation configuration of Broker.

    It is confirmed that the application that performs operations on the topic is not stopped.

    For details about the solution, see 4.

Solution

  1. Perform the following operations when connection to ZooKeeper fails.

    When the connection between the Kafka client and ZooKeeper times out, run the ping command to check whether the Kafka client can connect to ZooKeeper. Check the network connection between the client and ZooKeeper.

    If the network connection fails, check the ZooKeeper service information on Manager.

    Figure 1 ZooKeeper service information

    If ZooKeeper is improperly configured, change the ZooKeeper IP address in the client command.

  2. Perform the following operations when Kafka server configurations cannot be deleted.

    On Manager, change the value of delete.topic.enable to true. Save the configurations and restart the service.

    Figure 2 Modifying delete.topic.enable.

    The client query command does not contain Topic:test.

    kafka-topics.sh --list --zookeeper 192.168.0.122:24002/kafka

    Go to the log directory of node RunningAsController. Locate Deletion of topic test successfully in the controller.log file.

    2016-03-10 10:39:40,665 | INFO  | [delete-topics-thread-3] | [Partition state machine on Controller 3]: Invoking state change to OfflinePartition for partitions [test,2],[test,15],[test,6],[test,16],[test,12],[test,7],[test,10],[test,13],[test,9],[test,19],[test,3],[test,5],[test,1],[test,0],[test,17],[test,8],[test,4],[test,11],[test,14],[test,18] | kafka.controller.PartitionStateMachine (Logging.scala:68)
2016-03-10 10:39:40,668 | INFO  | [delete-topics-thread-3] | [Partition state machine on Controller 3]: Invoking state change to NonExistentPartition for partitions [test,2],[test,15],[test,6],[test,16],[test,12],[test,7],[test,10],[test,13],[test,9],[test,19],[test,3],[test,5],[test,1],[test,0],[test,17],[test,8],[test,4],[test,11],[test,14],[test,18] | kafka.controller.PartitionStateMachine (Logging.scala:68)
2016-03-10 10:39:40,977 | INFO  | [delete-topics-thread-3] | [delete-topics-thread-3], Deletion of topic test successfully completed | kafka.controller.TopicDeletionManager$DeleteTopicsThread (Logging.scala:68)

  3. Some Kafka nodes are stopped or faulty.

    Start the stopped Broker instances.

    The client query command does not contain Topic:test.

    kafka-topics.sh --list --zookeeper 192.168.0.122:24002/kafka

    Go to the log directory of node RunningAsController. Locate Deletion of topic test successfully in the controller.log file.

    2016-03-10 11:17:56,463 | INFO  | [delete-topics-thread-3] | [Partition state machine on Controller 3]: Invoking state change to NonExistentPartition for partitions [test,4],[test,1],[test,8],[test,2],[test,5],[test,9],[test,7],[test,6],[test,0],[test,3] | kafka.controller.PartitionStateMachine (Logging.scala:68)
2016-03-10 11:17:56,726 | INFO  | [delete-topics-thread-3] | [delete-topics-thread-3], Deletion of topic test successfully completed | kafka.controller.TopicDeletionManager$DeleteTopicsThread (Logging.scala:68)

  4. Perform the following operations when Kafka configurations are automatically created and the Producer is not stopped.

    Stop related applications, change the value of auto.create.topics.enable to false on Manager. Save the configuration and restart the service.

    Figure 3 Modifying auto.create.topics.enable

  5. Perform the delete operation again.
Parent topic: Using Kafka