Creating a Kafka Topic

Constraints

• The total number of partitions in topics is limited. When the partition quantity limit is reached, you can no longer create topics. The total number of partitions varies by instance specifications. For details, see Specifications.
• If an instance node is faulty, an internal service error may be reported when you query messages in a topic with only one replica. Therefore, you are not advised using a topic with only one replica.

Method 1: Creating a Topic on the Console

1. Log in to the console.
2. Click in the upper left corner to select a region.
   

   Select the region where your Kafka instance is located.

3. Click and choose Application > Distributed Message Service for Kafka to open the console of DMS for Kafka.
4. Click the desired Kafka instance to view the instance details.
5. In the navigation pane, choose Topics. Then click Create Topic.
   Figure 1 Creating a topic
   

6. Specify the topic parameters listed in the following table.

   Table 1 Topic parameters

   Parameter	Description
   Topic Name	When creating a topic, you can modify the automatically generated topic name. Once the topic is created, you cannot modify its name.
   Partitions	A larger number of partitions for a topic indicates more messages retrieved concurrently. If this parameter is set to 1, messages will be retrieved in the FIFO order. Value range: 1 to 200 Default value: 3
   Replicas	A higher number of replicas delivers higher reliability. Data is automatically backed up on each replica. When one Kafka broker becomes faulty, data is still available on other brokers. If this parameter is set to 1, only one set of data is available. Value range: 1 to number of brokers NOTE: If an instance node is faulty, an internal service error may be reported when you query messages in a topic with only one replica. Therefore, you are not advised using a topic with only one replica.
   Aging Time (h)	The period that messages are retained for. Consumers must retrieve messages before this period ends. Otherwise, the messages will be deleted and can no longer be consumed. Value range: 1 to 720 Default value: 72
   Synchronous Replication	A message is returned to the client only after the message creation request has been received and the message has been acknowledged by all replicas. After enabling synchronous replication, set acks to all or –1 on the client. Otherwise, this function will not take effect. If there is only one replica, synchronous replication cannot be enabled.
   Synchronous Flushing	Enabled: A message is immediately flushed to disk once it is created, resulting in higher reliability. Disabled: A message is stored in the memory instead of being immediately flushed to disk once created.
   Message Timestamp	Timestamp type of a message. Options: CreateTime: time when the producer created the message. LogAppendTime: time when the broker appended the message to the log.
   Max. Message Size	Maximum batch processing size allowed by Kafka. If message compression is enabled, this parameter indicates the size after compression. If this is increased and there are consumers older than 0.10.2, the consumers' fetch size must also be increased so that they can fetch record batches this large. Value range: 0 to 10,485,760

7. Click OK.

Method 2: Creating a Topic on Kafka Manager

Log in to Kafka Manager, choose Topic > Create, and set parameters as prompted.

Figure 2 Creating a topic on Kafka Manager

If a topic name starts with a special character, for example, an underscore (_) or a number sign (#), monitoring data cannot be displayed.

Method 3: Creating a Topic by Using Kafka CLI

If your client is v2.2 or later, you can use kafka-topics.sh to create topics and manage topic parameters.

• If a topic name starts with a special character, for example, a number sign (#), monitoring data cannot be displayed.
• For an instance with SASL enabled, if allow.everyone.if.no.acl.found is set to false, topics cannot be created through the client.

• If SASL is not enabled for the Kafka instance, run the following command in the /{directory where the CLI is located}/kafka_{version}/bin/ directory to create a topic:

  ./kafka-topics.sh --create --topic {topic_name} --bootstrap-server {broker_ip}:{port} --partitions {partition_num} --replication-factor {replication_num}

• If SASL has been enabled for the Kafka instance, perform the following steps to create a topic:
  1. (Optional) If the username and password, and the SSL certificate configuration has been set, skip this step. Otherwise, perform the following operations:

     Create the ssl-user-config.properties file in the /config directory of the Kafka client. Add the username and password, and the SSL certificate configuration by referring to 3.

  2. Run the following command in the /{directory where the CLI is located}/kafka_{version}/bin/ directory to create a topic:

     ./kafka-topics.sh --create --topic {topic_name} --bootstrap-server {broker_ip}:{port} --partitions {partition_num} --replication-factor {replication_num} --command-config ./config/ssl-user-config.properties