2.x Versions

2022-11

Concurrent DML and TRUNCATE operations between different sessions of a global temporary table

Concurrent DML and TRUNCATE operations are supported between different sessions of a global temporary table.

Snapshot backups for standby nodes

Standby nodes support snapshot backups.

With typical public cloud resources, it takes less than 8 hours to fully back up and restore 12 TB of data. PITR is also supported.

Emergency measures for high-latency at the IaaS layer

If the latency increases due to underlying resources, many connections will be created on the service side. As a result, the DN thread pool and dynamic memory will be full. Emergency measures are provided to quickly rectify the fault, preventing long-time impact on services.

No logic decoding for specific user operations

Logical decoding logs are filtered by user. During the decoding, logical logs about database operations performed by blacklist users are not returned. This ensures that downstream data is not affected by logical replication of blacklist user operations.

Username or user OID is required for blacklist user configuration. Logical log transactions contain user information, which provides the basis for downstream services to filter different users.

Read-only nodes in the cascaded standby server

Read-only nodes provide inventory query services to offload the query load of the primary node.

1. There is one primary node, two standbys nodes, and many read-only nodes in the HA architecture. The read-only nodes can be flexibly deleted. At least 3 read-only nodes are required. (One read-only node is connected to external services, and the other read-only nodes are for high availability.)
2. When network resources are normal, the log replication delay between the primary and read-only nodes is less than 5 seconds.
3. The query SQL statement can be executed for several hours.

Viewing SQL statements that are being executed in the current stored procedure

The SQL statements that are being executed in the current stored procedure can be viewed, enhancing the DFX capability.

Security hardening

The following security vulnerabilities are fixed:

CVE-2022-32221

CVE-2022-42916

CVE-2022-42915

CVE-2022-2097

CVE-2022-32205

CVE-2022-32206

CVE-2022-32207

CVE-2022-32208

CVE-2022-2068

CVE-2022-24302

2022-06

Logical replication capability enhanced

The logical replication capability is enhanced.

Memory management is enhanced based on:

1. Parameter that controls the memory threshold of transactions. When the threshold is exceeded, logical logs are automatically flushed to disks.
2. Parameter that controls the memory threshold of logical replication slots. When the threshold is exceeded, logical logs are automatically flushed to disks.
3. Views or functions that display memory structure of parallel logic decoding.

Ustore DFx capability enhanced

1. Online verification: Important data and fields are verified during service running.
2. Offline verification: For onsite problems and data exceptions, if the processes do not exit, analyze verify can be used to obtain and output error page information.
3. Module verification: The undo, redo, upage, or ubtree module is classified into three levels: fast, skip recovery, and complete. Key verification analysis can be provided by module.
4. Fault location information enhanced: There is more detailed fault location information in logs (pg_log) and views. The uheap, undo, ubtree, and flashback modules support views.
5. Error message "Snapshot too old" resolved: The message is displayed in a new transaction due to forcible reclamation of undo space.

SQL PATCH

SQL patches are designed for database administrators, O&M personnel, and other personnel that need to optimize SQL statements. If the execution plan or execution mode of a query statement does not meet expectations, a SQL patch can adjust the query execution plan or mode without directly modifying the statement, to meet the expected effect. In addition, you can flexibly create, delete, or change a SQL patch as needed. In actual GaussDB O&M scenarios, unique SQL IDs are used to locate query statements (normal-stmt and prepared-stmt) and hints are used to optimize query performance. The scenarios for adjusting the statement execution mode are as follows:

Scenario 1: Adding hints (such as hint-index and hint-GUC) to statements to adjust the query compilation mode of the optimizer.

In scenario 2: Performing plain-text rewriting on query statements in special emergency cases.

Security hardening

The following security vulnerabilities are fixed:

CVE-2022-31799

CVE-2022-27781

CVE-2022-27782

CVE-2022-1292

CVE-2022-29824

CVE-2022-27774

CVE-2022-27776

CVE-2022-22576

CVE-2022-27775

CVE-2021-3520

2022-04

Go driver supported by primary/standby instances

1. Go driver supports SSL connections to the database.
2. Go driver supports database handles.
3. Go driver supports Stmt.
4. Go driver supports database transactions.
5. Go driver supports query results (Rows and Row).
6. Go driver supports column types.

SMP for stored procedures

SMP parallel execution plans can be generated and executed for query statements in stored procedures.

Ustore hardening and optimization

The Ustore adapts to the framework of the parallel index creation.

WDR report enhanced

The ASP capability is enhanced.

The WDR capability is enhanced.

View O&M capacity enhanced

You can view disk usage details by pg_ls_waldir() for wal diagnosis and pg_ls_tmpdir() for tmp diagnosis.

2022-02

Ustore commercial use capability improved

1. The Ustore resources can be better managed. The undo memory is optimized and zones are loaded as required. When Ustore is not used, the memory overhead and undo file space are less than 1 MB.
2. Ustore supports WDR reports which include:
   1. Number of accesses to UHeap pages (including QUERY, MODIFY, DELETE, and PRUNE operations).
   2. Number of accesses to undo records
   3. Average access length of undo chains
   4. Rate at which undo and translot files are flushed to disks
   5. Rate at which undo files are generated in a specified period.
   6. Rate at which undo files are reclaimed in a specified period
   7. Index-related information (number of layers and page visits)
3. Ustore flashback: DROP operations, TRUNCATE operations, and partitioned tables can be flashed back. The baseline for flashback query performance is provided.

Driver and syntax compatibility enhanced

1. The select for update wait n syntax is supported.
2. The conversion rule of the Decode type is compatible with O*.
3. New syntax needs to be connected to GK Smith (a pressure test tool).
4. The following regular expressions are supported: REGEXP_COUNT, REGEXP_INSTR, REGEXP_SUBSTR, and REGEXP_REPLACE.
5. execute direct on in query statements supports precompilation.
6. The criteria for updating or inserting data can be specified.

Online database maintenance without downtime

1. Indexes can be created online without downtime.
2. Adding a standby node does not restart the primary node.

Row- and column-store engine capability improved

1. Row-store table query execution is converted to vectorized execution, improving performance in the case of hybrid loads. Under the same resource environments and workloads, the TPC-H performance of vectorized execution is 30% higher than that of the row-store engine and row-store execution framework.
2. Column storage supports MERGE INTO subqueries.

Logic decoding performance and functions optimized

1. The logic decoding performance is optimized. The DML decoding performance reaches 100 Mbit/s (DDL decoding is not supported).
2. Logical decoding supports parsing DML operations on a specified table. In JDBC, a new parameter is added to support log parsing. (You can call the existing libpq logical walsender API to parse logs).

Performance improvement for enterprise-class application scenarios

1. When the same amount of data is updated concurrently, the average update latency and latency difference on a single DN are almost the same as those on MySQL.
2. There is no obvious periodic fluctuation within 2 hours in the TPC-C benchmark with 8,000 warehouses on an Arm-powered single-node instance with 128 vCPUs and 512 GB SAS SSD or equivalent specifications.
3. walwriter optimization: The performance is optimized under heavy load (the vCPU usage is 60% or higher).

Enterprise-level capability optimized

1. The IP address, port number, and host name of a dynamic server can be changed.
2. A client driver can be configured with multiple IP addresses of the primary and standby nodes, but is only connected to the primary node. In libpq, after an primary/standby switchover occurs, a standby node can be automatically promoted to primary.
3. The compatibility of gsql with SQLPlus is enhanced.
4. The JDBC interface supports uppercase and lowercase letters.

4 GB for CLOB/BLOB

CLOBs/BLOBs can store 4 GB of data. The performance of the CLOBs/BLOBs smaller than 1 GB is not inferior to that of the previous version. Constraints: More than 1 GB of data can be read and processed only by advanced package functions. An error is reported when more than 1 GB of data is transferred by system functions. Operator and string functions do not support more than 1 GB of data. The maximum buffer size in a stored procedure is 32 KB. DISTINCT, GROUP BY, or ORDER BY operations are not allowed in LOB columns. An advanced package supports a maximum of 4 GB data. To update tables, you need to add update in lob_write.

Deletion and update of specified partitions for primary/standby instances

Primary/standby instances support deletion and update of specified partitions (level-1 partitions for level-1 partitioned tables and level-2 partitions for level-2 partition tables). For the deletion and update of the same amount of data (including point-to-point deletion and batch deletion scenarios), the performance of partitioned tables is the same as that of non-partition tables. After a partition is specified, the global index scan cannot be selected.

Global cache of syscache system tables

This feature reduces the high memory usage in high-concurrency and complex query scenarios. In the case of any concurrent load, the total memory usage of the GSC or LSC cannot exceed the upper limit specified by the corresponding GUC parameters (GSC: global_syscache_threshold; LSC: local_syscache_threshold). After GSC/LSC is enabled, the loss in the standard benchmark scenario (tpcc/sysbench) does not exceed 5%.

Module function and reliability hardening

1. Capacity expansion hardening:
   1. Redistribution adaptive lock timeout and lock levels are optimized.
   2. Redistribution escape method: The kernel can trigger fast service failure.
   3. Redistribution supports different scenarios, and the mode can be dynamically configured.
   4. Abnormal residual data can be cleared during redistribution.
   5. Resumable data transfer is supported.
2. Data reliability
   1. The primary node restores damaged pages or files from the standby node: main fork is used for common row-store tables and init fork is used for unlogged tables. After the clog and csnlog files are damaged, a function is called. Currently, the file-level does not support the segment-page format.
   2. The standby node repairs damaged pages or files from the primary node: Table data files can be automatically repaired during replay. Currently, the file-level does not support the segment-page format.
   3. If the verification page and file are damaged during the backup, the damaged page and file can be automatically recovered on the standby node by calling a function.
   4. Logical decoding of a specified LSN segment is supported when the meta information is not changed.
3. Ultimate RTO enabled by default:
   1. PITR adaptation
   2. The ultimate RTO flow control is configurable, and the replay speed views are provided.
4. DFX and quality hardening:
   1. Data consistency mechanism between pgxc_node and cn/dn-handler is hardened.
5. UUID authentication: A verification mechanism is used when the primary and standby nodes are connected. The UUID is generated based on the dbname character string for verification.

PITR for distributed instances

Stored procedures

1. Autonomous transactions support global variables.
2. Cursors can be closed in exceptions. When the JDBC executes stored procedures, the input and output parameters do not need to be registered.

Permissions

Fine-grained permissions and ANY permissions are supported. The following ANY permissions and syntax are supported:

• CREATE ANY TABLE
• ALTER ANY TABLE
• DROP ANY TABLE
• SELECT ANY TABLE
• INSERT ANY TABLE
• UPDATE ANY TABLE
• DELETE ANY TABLE
• CREATE ANY FUNCTION
• EXECUTE ANY FUNCTION
• CREATE ANY PACKAGE
• EXECUTE ANY PACKAGE
• CREATE ANY TYPE

Security

1. When an audit file is damaged, audit logs generated after the damage can be queried.
2. An error in an SSL certificate revocation list (CRL) does not affect normal authentication.
3. Audit logs contain transaction IDs, which are used to associate data changes with audit operations.
4. The password of the O&M account is encrypted and flushed to disks.

Software component lifecycle matches the product lifecycle

New lower-layer software components (such as open-source software, third-party software, and self-developed platforms) are used and will be updated in a timely manner within their lifecycle to match the product lifecycle. Security vulnerabilities can be fixed in a timely manner within the lifecycle of product versions.

ODBC commercial use

1. Distributed load balancing is supported.
2. The primary/standby switchover is supported.

CM enterprise-level capability improved

1. CMS records key arbitration event logs.
2. CMS depends on Huawei-developed DCC component, instead of ETCD, to select the primary node.
3. The arbitration logic is reconstructed to expand the arbitration capability.

Performance improved

The performance of a full-table scan based on linetitem 1X is improved by 30%, and the response latency is shorter than 1s.

O&M monitoring capability improved

1. Normalized SQL processing is added to session statistics for diagnosing historical slow SQL statements.
2. The mem trace performance is enhanced to enrich memory fault locating methods.
3. Full-link trace from JDBC to the kernel is supported.
4. Kernel metric collection is optimized to support single-node collection and reporting.
5. The get_node_disk_and_log_status interface can be used in primary/standby instances.

Paxos enterprise-level capability improved

1. The DCF protocol can prevent frequent switching.
2. You can configure standby Paxos nodes are promoted to primary by priority.

Security hardening

The following security vulnerabilities are fixed:

CVE-2022-23308

CVE-2020-28473

CVE-2021-22947

CVE-2021-22946

CVE-2021-22945

CVE-2021-37750

CVE-2021-36222