Configuring Kafka Quotas

Scenario

On the console, you can control the message production and consumption rate limits for users, clients, or topics.

Rate limits for users and clients work on the entire broker, while topic rate limits work on a specific topic.

Operation Impact

• When the quota is reached, production/consumption latency increases.
• If the quota is small and the production rate is high, production may time out and messages may be lost. As a result, some messages fail to be produced.
• If the initial production/consumption traffic is heavy, and a small quota is set, the production/consumption latency increases and some messages fail to be produced. To ensure stable production and consumption, you are advised to first set the quota to half the traffic, and then half the quota each time you set it until the target quota is reached. For example, if the initial production traffic is 100 MB/s, you can set the production limit to 50 MB/s first. After production becomes stable, change the production limit to 25 MB/s until the target limit is reached.

Prerequisites

• To control user traffic, enable SASL_SSL when creating a Kafka instance and then obtain the username on the Users page on the console.
• To control client traffic, obtain the client ID from the client configuration.
• To control topic traffic, obtain the topic name from the Topics page.

Creating a User or Client Quota

1. Log in to the console.
2. Click in the upper left corner to select a region.
   

   Select the region where your Kafka instance is located.

3. Click and choose Application > Distributed Message Service for Kafka to open the console of DMS for Kafka.
4. Click the desired Kafka instance to view the instance details.
5. In the navigation pane, choose Kafka Quotas > Quotas.
6. Click the User/Client tab.
7. In the upper left, click Create Quota. The Create Quota slide panel is displayed.
8. Set quota parameters.

   Table 1 Quota parameters

   Parameter	Description
   Username	Enter the name of the user to apply the quota to. To apply the quota to all users, click Use Default next to Username. After the quota is created, the username cannot be changed.
   Client ID	Enter the ID of the client to apply the quota to. To apply the quota to all clients, click Use Default next to Client ID. After the quota is created, the client ID cannot be changed.
   Production Limit	Set an upper limit on the production rate. The unit is MB/s. If this parameter is left blank, no limit is set.
   Consumption Limit	Set an upper limit on the consumption rate. The unit is MB/s. If this parameter is left blank, no limit is set.

   

   • Username is not displayed in the Create Quota dialog box for instances with SASL disabled.
   • Username and Client ID cannot be both empty.
   • Production Limit and Consumption Limit cannot be both empty.

9. Click OK. The Background Tasks page is displayed. If the status of the quota creation task is Successful, the quota has been created.

   Go to the Kafka Quotas > Quotas page. On the User/Client tab page, select User quotas, Client quotas, or User and client quotas, then press Enter to view the created quota.

   Figure 1 Viewing the new quota
   

Creating a Topic Quota

1. Log in to the console.
2. Click in the upper left corner to select a region.
   

   Select the region where your Kafka instance is located.

3. Click and choose Application > Distributed Message Service for Kafka to open the console of DMS for Kafka.
4. Click the desired Kafka instance to view the instance details.
5. In the navigation pane, choose Kafka Quotas > Quotas.
6. Click the Topic tab.
7. In the upper left, click Create Quota. The Create Quota slide panel is displayed.
8. Set quota parameters.

   Table 2 Quota parameters

   Parameter	Description
   Topic Name	Enter the name of the topic to apply the quota to. After the quota is created, the topic cannot be changed.
   Production Limit	Set an upper limit on the production rate. The unit is MB/s. If this parameter is left blank, no limit is set.
   Consumption Limit	Set an upper limit on the consumption rate. The unit is MB/s. If this parameter is left blank, no limit is set.

   

   Production Limit and Consumption Limit cannot be both empty.

9. Click OK. The Background Tasks page is displayed. If the status of the quota creation task is Successful, the quota has been created.

   Go to the Kafka Quotas > Quotas page. On the Topic tab page, enter the name of the new quota in the upper left corner, then press Enter to view the created quota.

Modifying a Quota

1. Log in to the console.
2. Click in the upper left corner to select a region.
   

   Select the region where your Kafka instance is located.

3. Click and choose Application > Distributed Message Service for Kafka to open the console of DMS for Kafka.
4. Click the desired Kafka instance to view the instance details.
5. In the navigation pane, choose Kafka Quotas > Quotas.
6. In the row containing the desired quota, click Edit.
7. Change the production limit or consumption limit, and click OK. The Background Tasks page is displayed. If the status of the quota modification task is Successful, the quota has been modified.
   Choose Kafka Quotas > Quotas and view the new production or consumption rate limit.

   

   Production Limit and Consumption Limit cannot be both empty.

Exporting Quotas

1. Log in to the console.
2. Click in the upper left corner to select a region.
   

   Select the region where your Kafka instance is located.

3. Click and choose Application > Distributed Message Service for Kafka to open the console of DMS for Kafka.
4. Click the desired Kafka instance to view the instance details.
5. In the navigation pane, choose Kafka Quotas > Quotas.
6. Export quotas.
   • On the User/Client tab page: Choose Export > Export all data to an XLSX file or Export selected data to an XLSX file.
   • On the Topic tab page: Choose Export > Export all data to an XLSX file or Export selected data to an XLSX file.

Deleting a Quota

1. Log in to the console.
2. Click in the upper left corner to select a region.
   

   Select the region where your Kafka instance is located.

3. Click and choose Application > Distributed Message Service for Kafka to open the console of DMS for Kafka.
4. Click the desired Kafka instance to view the instance details.
5. In the navigation pane, choose Kafka Quotas > Quotas.
6. In the row containing the desired quota, click Delete.
7. Click OK. The Background Tasks page is displayed. If the status of the quota deletion task is Successful, the quota has been deleted.