Modifying Doris Parameters to Optimize Cluster Performance

Unmodified

The parameter has not been modified.

Unapplied

The parameter has been modified but has not been applied.

Applying

This is a static parameter and you need to restart the cluster after saving the modification.

Applied

The modification of the parameter takes effect.

• After the modification of a static parameter is saved, the cluster needs to be restarted for the modification to take effect.
• The modification of a dynamic parameter takes effect after being saved.

max_routine_load_job_num

No

• Description: The total number of routine import jobs, including the jobs in the NEED_SCHEDULED, RUNNING, and PAUSE states. After this parameter value is exceeded, no new jobs can be submitted.
• Type: int
• Value range: (0,300)
• Default value: 100

use_new_tablet_scheduler

Yes

• Description: Whether to enable the new replica scheduling mode.
• Type: boolean
• Value range: [true,false]
• Default value: true

max_broker_concurrency

No

• Description: Maximal concurrency of broker scanners.
• Type: int
• Value range: (5,30)
• Default value: 10

tablet_repair_delay_factor_second

No

• Description: For different scheduling priorities, we will delay different time to start repairing in order to prevent a large number of unnecessary replica repair tasks from occurring in the process of routine restart and upgrade. This parameter is a reference coefficient. For HIGH priority, the delay is the reference coefficient multiplied by 1; for NORMAL priority, the delay is the reference coefficient multiplied by 2; for LOW priority, the delay is the reference coefficient multiplied by 3. That is, the lower the priority, the longer the delay waiting time. If the user wants to repair the replica as soon as possible, this parameter can be reduced appropriately.
• Type: long
• Value range: (0,9223372036854775807)
• Default value: 60

balance_load_score_threshold

No

• Description: Threshold of the cluster balance percentage. The default value is 0.1, that is, 10%. When the load core of a BE node is not higher than or less than 10% of the average load core, we think that the node is balanced. If you want to make the cluster load more even, you can adjust this parameter appropriately.
• Type: double
• Value range: (0,1)
• Default value: 0.1

dynamic_partition_check_interval_seconds

No

• Description: The execution frequency of dynamic partition threads. The default value is 600 (10 minutes), that is, scheduling is performed every 10 minutes.

  Unit: second

• Type: long
• Value range: (0,9223372036854775807)
• Default value: 600

max_routine_load_task_concurrent_num

No

• Description: Limit on the maximum number of concurrent subtasks for a routine import job. It is advised to keep the default value. Setting it too large may result in too many concurrent tasks and consume cluster resources.
• Type: int
• Value range: (0,10)
• Default value: 5

dynamic_partition_enable

No

• Description: Whether to enable Doris' dynamic partition feature. The default value is false, indicating that the dynamic partition feature is disabled. This parameter only affects the partitioning operation of dynamic partition tables, not normal tables.
• Type: boolean
• Value range: [true,false]
• Default value: true

max_bytes_per_broker_scanner

No

• Description:

  Maximum bytes a broker scanner can process in one broker load job. Generally, each BE has a broker scanner. Maximum amount of data processed by a single BE. The default value is 500 GB. If the amount of data imported by a single BE is greater than the max_bytes_per_broker_scanner value, an error is reported. In this case, you need to increase the value of max_bytes per_broker_scanner or adjust the number of concurrent import tasks. The calculation logic is as follows: Number of concurrent import tasks = Math.min (Source file size/Minimum processing volume min_bytes_per_broker scanner, Maximum number of concurrent tasks max broker concurrency, Number of current BE nodes). Processing volume of a single BE = Source file size/Number of concurrent import tasks. The default value of min_bytes_per_broker scanner is 64 MB, and the default value of max broker concurrency is 10.

  Unit: byte

• Type: long
• Value range: (0,9223372036854775807)
• Default value: 3221225472

max_consumer_num_per_grou

Yes

• Description: Maximum number of consumers in a data consumer group, used for routine load.
• Type: int
• Value range: (0,10)
• Default value: 3

clone_worker_count

Yes

• Description: Number of threads used to perform cloning tasks. The value affects the speed of replica equalization. In the case of low disk pressure, you can speed up replica balancing by adjusting this parameter.
• Type: int
• Value range: (0,10)
• Default value: 3