Updated on 2024-03-15 GMT+08:00

Creating a Quota

Scenario

On the console, you can control the message production and consumption rate limits for users, clients, or topics.

Rate limits for users and clients work on the entire broker, while topic rate limits work on a specific topic.

  • This function is supported for instances created on or after November 10, 2022.
  • This function is unavailable for single-node instances.

Operation Impact

  • When the quota is reached, production/consumption latency increases.
  • If the quota is small and the production rate is high, production may time out and messages may be lost. As a result, some messages fail to be produced.
  • If the initial production/consumption traffic is heavy, and a small quota is set, the production/consumption latency increases and some messages fail to be produced. To ensure stable production and consumption, you are advised to first set the quota to half the traffic, and then half the quota each time you set it until the target quota is reached. For example, if the initial production traffic is 100 MB/s, you can set the production limit to 50 MB/s first. After production becomes stable, change the production limit to 25 MB/s until the target limit is reached.

Prerequisites

  • To control user traffic, enable SASL_SSL when creating a Kafka instance and then obtain the username on the Users page on the console.
  • To control client traffic, obtain the client ID from the client configuration.
  • To control topic traffic, obtain the topic name from the Topics page.

Creating a User or Client Quota

  1. Log in to the management console.
  2. Click in the upper left corner to select a region.

    Select the region where your Kafka instance is located.

  3. Click and choose Middleware > Distributed Message Service (for Kafka) to open the console of DMS for Kafka.
  4. Click the desired Kafka instance to view the instance details.
  5. In the navigation pane, choose Kafka Quotas > Quotas.
  6. Click the User/Client tab.
  7. In the upper left corner, click Create Quota. The Create Quota slide panel is displayed.
  8. Set quota parameters.

    Table 1 Quota parameters

    Parameter

    Description

    Username

    Enter the name of the user to apply the quota to. To apply the quota to all users, click Use Default next to Username.

    After the quota is created, the username cannot be changed.

    Client ID

    Enter the ID of the client to which the quota applies. To apply the quota to all clients, click Use Default next to Client ID.

    After the quota is created, the client ID cannot be changed.

    Production Limit

    Set an upper limit on the production rate. The unit is MB/s. If this parameter is left blank, no limit is set.

    Consumption Limit

    Set an upper limit on the consumption rate. The unit is MB/s. If this parameter is left blank, no limit is set.

    • If SASL is not enabled for the instance, Username is not displayed in the Create Quota slide panel.
    • Username and Client ID cannot be both empty.
    • Production Limit and Consumption Limit cannot be both empty.

  9. Click OK. The Background Tasks page is displayed. If the status of the quota creation task is Successful, the quota has been created.

    Go to the Kafka Quotas > Quotas page. On the User/Client tab page, select User quotas, Client quotas, or User and client quotas, then click to view the created quota.

    Figure 1 Viewing the new quota

Creating a Topic Quota

  1. Log in to the management console.
  2. Click in the upper left corner to select a region.

    Select the region where your Kafka instance is located.

  3. Click and choose Middleware > Distributed Message Service (for Kafka) to open the console of DMS for Kafka.
  4. Click the desired Kafka instance to view the instance details.
  5. In the navigation pane, choose Kafka Quotas > Quotas.
  6. Click the Topic tab.
  7. In the upper left, click Create Quota. The Create Quota slide panel is displayed.
  8. Set quota parameters.

    Table 2 Quota parameters

    Parameter

    Description

    Topic Name

    Enter the name of the topic to apply the quota to.

    After the quota is created, the topic cannot be changed.

    Production Limit

    Set an upper limit on the production rate. The unit is MB/s. If this parameter is left blank, no limit is set.

    Consumption Limit

    Set an upper limit on the consumption rate. The unit is MB/s. If this parameter is left blank, no limit is set.

    Production Limit and Consumption Limit cannot be both empty.

  9. Click OK. The Background Tasks page is displayed. If the status of the quota creation task is Successful, the quota has been created.

    Go to the Kafka Quotas > Quotas page. On the Topic tab page, enter the name of the new quota in the upper right corner, then click to view the created quota.