Updated on 2024-03-15 GMT+08:00

Modifying Kafka Parameters

Scenario

Your Kafka instances, topics, and consumers come with default configuration parameter settings. You can modify common parameters on the Kafka console. For details about parameters that are not listed on the console, see the Kafka official website.

Kafka instances have dynamic and static parameters:

  • Dynamic parameters: Modifying dynamic parameters will not restart the instance.
  • Static parameters: After static parameters are modified, you must manually restart the instance.
  • Configuration parameters of some old instances cannot be modified. Check whether your instance parameters can be modified on the console. If they cannot be modified, contact customer service.
  • This function is not available for single-node instances.

Prerequisites

You can modify configuration parameters of a Kafka instance when the instance is in the Running state.

Procedure

  1. Log in to the management console.
  2. Click in the upper left corner to select a region.

    Select the region where your Kafka instance is located.

  3. Click and choose Middleware > Distributed Message Service (for Kafka) to open the console of DMS for Kafka.
  4. Click the desired Kafka instance to view the instance details.
  5. On the Parameters page, click Edit in the row containing the parameter to modify.

    Parameters of v1.1.0 instances are described in Table 2 and Table 1. Parameters of v2.3.0/v2.7 instances are described in Table 3 and Table 4.

    Table 1 Dynamic parameters (v1.1.0 instances)

    Parameter

    Description

    Value Range

    Default Value

    auto.create.groups.enable

    Whether to automatically create consumer groups.

    You can modify this parameter on the console only for instances created on or after April 25, 2023. For instances created before April 25, 2023, the function of automatically creating consumer groups is enabled by default and cannot be disabled on the console.

    true/false

    true

    offsets.retention.minutes

    The longest period a consumption position can be retained starts from the time of submission. Positions retained beyond this duration will be deleted. Each time a consumption position is submitted to a topic partition, its retention period resets to 0. The unit is minute.

    This is a static parameter for instances created before May 1, 2023.

    1440–30240

    20160

    Table 2 Static parameters (v1.1.0 instances)

    Parameter

    Description

    Value Range

    Default Value

    min.insync.replicas

    If a producer sets the acks parameter to all (or -1), the min.insync.replicas parameter specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful.

    1–3

    1

    message.max.bytes

    Maximum length of a single message, in bytes.

    0–10,485,760

    10,485,760

    unclean.leader.election.enable

    Indicates whether to allow replicas not in the ISR set to be elected as the leader as a last resort, even though doing so may result in data loss.

    true or false

    true

    connections.max.idle.ms

    Idle connection timeout (in ms). Connections that are idle for the duration specified by this parameter will be closed.

    5000–600,000

    600,000

    log.retention.hours

    Duration (in hours) for retaining a log file.

    This parameter takes effect only for topics that have no aging time configured. If there is aging time configured for topics, it overrides this parameter.

    1–168

    72

    max.connections.per.ip

    The maximum number of connections allowed from each IP address. Request for new connections will be rejected once the limit is reached.

    100–20,000

    1000

    group.max.session.timeout.ms

    The maximum session timeout (in ms) for consumers. A longer timeout gives consumers more time to process messages between heartbeats but results in a longer time to detect failures.

    6000–1,800,000

    1,800,000

    default.replication.factor

    The default number of replicas configured for an automatically created topic.

    1–3

    3

    allow.everyone.if.no.acl.found

    When this parameter is set to true, all users can access resources without ACL rules.

    This parameter is displayed only when SASL is enabled for the instance or ciphertext access is used.

    This parameter cannot be modified for instances created before September 15, 2023.

    true/false

    true

    num.partitions

    The default number of partitions configured for each automatically created topic.

    1 ~ 200

    3

    group.min.session.timeout.ms

    The minimum session timeout (in ms) for consumers. A shorter timeout enables quicker failure detection but results in more frequent consumer heartbeating, which can overwhelm broker resources.

    6000–300,000

    6000

    Table 3 Dynamic parameters (2.3.0/2.7 instances)

    Parameter

    Description

    Value Range

    Default Value

    min.insync.replicas

    If a producer sets the acks parameter to all (or -1), the min.insync.replicas parameter specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful.

    1–3

    1

    message.max.bytes

    Maximum length of a single message, in bytes.

    0–10,485,760

    10,485,760

    auto.create.groups.enable

    Whether to automatically create consumer groups.

    You can modify this parameter on the console only for instances created on or after April 25, 2023. For instances created before April 25, 2023, the function of automatically creating consumer groups is enabled by default and cannot be disabled on the console.

    true/false

    true

    max.connections.per.ip

    The maximum number of connections allowed from each IP address. Request for new connections will be rejected once the limit is reached.

    100–20,000

    1000

    unclean.leader.election.enable

    Indicates whether to allow replicas not in the ISR set to be elected as the leader as a last resort, even though doing so may result in data loss.

    true or false

    true

    offsets.retention.minutes

    The longest period a consumption position can be retained starts from the time of submission. Positions retained beyond this duration will be deleted. Each time a consumption position is submitted to a topic partition, its retention period resets to 0. The unit is minute.

    This is a static parameter for instances created before May 1, 2023.

    1440–30240

    20160

    Table 4 Static parameters (2.3.0/2.7 instances)

    Parameter

    Description

    Value Range

    Default Value

    connections.max.idle.ms

    Idle connection timeout (in ms). Connections that are idle for the duration specified by this parameter will be closed.

    5000–600,000

    600,000

    log.retention.hours

    Duration (in hours) for retaining a log file.

    This parameter takes effect only for topics that have no aging time configured. If there is aging time configured for topics, it overrides this parameter.

    1–168

    72

    group.max.session.timeout.ms

    The maximum session timeout (in ms) for consumers. A longer timeout gives consumers more time to process messages between heartbeats but results in a longer time to detect failures.

    6000–1,800,000

    1,800,000

    default.replication.factor

    The default number of replicas configured for an automatically created topic.

    1–3

    3

    allow.everyone.if.no.acl.found

    When this parameter is set to true, all users have access to resources without ACL.

    This parameter is displayed only when SASL is enabled for the instance or ciphertext access is used.

    This parameter cannot be modified for instances created before September 15, 2023.

    true/false

    true

    num.partitions

    The default number of partitions configured for each automatically created topic.

    1 ~ 200

    3

    group.min.session.timeout.ms

    The minimum session timeout (in ms) for consumers. A shorter timeout enables quicker failure detection but results in more frequent consumer heartbeating, which can overwhelm broker resources.

    6000–300,000

    6000

    • To modify multiple dynamic or static parameters at a time, click Modify above the parameter list.
    • If you want to restore the default values, click Restore Default in the row containing the desired parameter.

  6. Click Save.

    Modifying dynamic parameters will not restart the instance. Static parameter modification requires manual restart of the instance.