Load Management

Description

Load management balances system compute resources through service concurrency control to prevent resource contention between services, achieving harmonious coexistence of jobs and optimal resource utilization. In addition, the cgroup technology is introduced to manage CPU quotas.

Technical Principles

Load management is classified into static load management and dynamic load management, which are controlled by the enable_dynamic_workload parameter. In addition, GaussDB(DWS) provides priority control to control the priorities of tenant jobs.

• Static Load Management

  Each CN controls the memory and concurrency separately. The actual concurrency of a cluster is the sum of the concurrency of all CNs. The actual memory used by the cluster is the sum of the memory occupied by the jobs running on each CN. Because the memory of each CN is controlled separately, the actual memory used by a tenant on the DN may exceed the memory quota of the tenant.

• Dynamic Load Management

  The CCN is added to control the concurrency and memory of complex jobs. Each CN requests queuing information from the CCN. The CCN accumulates the estimated memory used by tenants to obtain the used memory of each tenant. When the available memory of a tenant is insufficient, the CCN triggers queuing. After the job running is complete, the CCN attempts to wake up the jobs in the queue. The sum of used memory of all tenants is the used memory of the cluster. In addition, the CCN periodically collects the memory usage information on DNs to update the available memory of the cluster. If the available memory is insufficient, jobs are queued. The job running must meet the memory limits of both the tenant and cluster.

• Priority Control

  GaussDB(DWS) implements load identification and intra-queue priority control based on query_band. It provides more flexible load identification methods and identifies load queues based on job types, application names, and script names. Users can flexibly configure query_band identification queues based on service scenarios. In addition, priority control of job delivery in the queue is implemented. In the future, priority control of resources in the queue will be gradually implemented.

Benefits

Load management implements resource isolation and priority control between tenants. It ensures that jobs of tenants with high priorities run preferentially without affecting resources of tenants with low priorities, achieving effective and optimal resource utilization.

For more information, see Resource Management.