Help Center/ GaussDB(DWS)/ User Guide/ GaussDB(DWS) Cluster Management/ GaussDB(DWS) Resource Load Management/ Overview
Updated on 2024-10-21 GMT+08:00
View PDF
Share

Overview

The system resources (CPU, memory, I/O, and storage resources) of a database are limited. When multiple types of services (such as data loading, batch analysis, and real-time query) are running at the same time, they may compete for resources and hinder operations. As a result, the throughput decreases and the overall query performance deteriorates. To avoid this problem, resources must be properly allocated.

GaussDB(DWS) provides the resource management function. You can put resources into different resource pools, which are isolated from each other. Then, you can associate database users with these resource pools. When a user starts a SQL query, the query will be transferred to the resource pool associated with the user. You can specify the number of queries that can be concurrently executed in a resource pool, the upper limit of memory used for a single query, and the memory and CPU resources that can be used by a resource pool. In this way, you can limit and isolate the resources occupied by different workloads, properly utilizing resources to process hybrid database loads and achieve high query performance. After a cluster is converted into a logical cluster, you can create, modify, or delete a resource pool in the logical cluster.

  • This feature is supported only by clusters of version 8.0 or later.
  • Resources cannot be managed during offline scale-out. If a resource management plan is enabled, stop it before performing offline scale-out.
  • The hybrid data warehouse (standalone) does not support resource management.

Enabling or Disabling Resource Management

You can enable or disable resource management, and configure the maximum global concurrency. Max. Concurrent Queries refers to the maximum concurrent queries on a single CN. If you disable Resource Management, all resource management functions will be unavailable.

Figure 1 Enabling Resource Management Configuration

Resource Management Functions

The resource management functions of GaussDB(DWS) can be classified into the following types based on managed resources:

  • Computing resource management. It is implemented using resource pools. Computing resources are isolated and controlled to prevent cluster-level issues caused by abnormal SQL queries. Computing resource management includes concurrency management, memory management, CPU management, and exception rules. For details, see Resource Pool.
  • Storage space management: Storage is managed at user and schema level to prevent disk exhaustion, which makes the database read only. For details, see Workspace Management.
  • Resource management plan: Resources are managed automatically based on a preconfigured plan, which can flexibly cope with complex scenarios. For details, see Importing or Exporting a Resource Management Plan.

The resource management functions of GaussDB(DWS) can be classified into the following types based on when they are implemented:

  • Management before a query

    The service checks whether there are sufficient resources for a query. If there are, the query can be executed. If there are not, the query waits in a queue, and can be executed only after resources are released by other queries. Concurrency and memory are managed in this phase.

  • Management during a query

    During query execution, resources used by the query are managed and controlled to prevent cluster exceptions caused by time-consuming SQL statements. Memory, CPU, storage space, and exception rules are managed in this phase.

Simple and Complex Queries

GaussDB(DWS) supports fine-grained resource management. Before workload management is implemented, queries are classified into complex queries (with long execution time and high resource consumption) and simple queries (with short execution time and low resource consumption). Simple and complex queries also differ in their estimated memory usage.

  • The estimated memory usage of a simple query is less than 32 MB.
  • The estimated memory usage of a complex query is 32 MB or higher.

In a hybrid load database, complex queries often occupy a large number of resources for a long time. A simple query queued after a complex query is time consuming, because it has to wait for the complex query to complete and resources to be freed up. To improve execution efficiency and system throughput, GaussDB(DWS) provides the short query acceleration function, managing simple queries separately.

  • If short query acceleration is enabled, simple queries and complex queries are managed separately. Simple queries do not need to compete with complex queries for resources.
  • If short query acceleration is disabled, simple and complex queries are under the same resource management rules.

To prevent a large number of simple queries from consuming too many resources during acceleration, concurrency management is performed on the queries. Resource management is not performed, because it may affect query performance and system throughput.

Queries are categorized based on estimated memory usage, but the estimation does not equal the actual usage, nor does it reflect the query duration or CPU usage. In resource pools that are insensitive to performance and only run specific services, you can disable short query acceleration to manage resources and handle exceptions for simple queries.