V2.0-3.x Versions

2024-05

PITR modular decoupling

The startup of GaussDB processes is decoupled from the loading of the barrier file. Even if the barrier file contains errors, it will not affect the startup or availability of GaussDB.

Improved DFX capability of the PITR module

PITR-related system views and system functions are added to provide convenient O&M methods for O&M personnel. The PITR O&M efficiency is improved. The information collection duration is shortened from hours or minutes to seconds.

Automatic repair based on physical bad blocks (Pages on the standby node can be repaired from the primary node in seconds.)

When a standby DN detects a page CRC error, it obtains the correct page from the primary node and overwrites the error page at a proper time for repair.

Enhanced rebuilding capability of the standby node

To accelerate the recovery of faulty standby nodes, it is beneficial to implement incremental builds between standby nodes for transmitting incremental logs and data. Additionally, creating slots during the build process ensures normal log recycling, preventing log accumulation. When log inconsistencies are significant, the primary node retains sufficient logs after the build to avoid prolonged local log replay and prevent repeated build triggers caused by log recycling.

DR cluster RTO optimization

According to the current mechanism, if an instance in the DR cluster is disconnected, the connection is re-initiated only after all logs are replayed. The disconnection time is significantly affected by the replay speed and the volume of logs to be replayed. Under heavy load, the instances in the DR cluster may be disconnected for a long time and logs in the primary cluster may be accumulated.

After the optimization, the CM launches a pre-parsing thread for disconnected instances to quickly scan the full logs of the instance. Then, the connection is initiated from the latest log, significantly reducing the recovery time for primary/standby synchronization.

Logging of memory over-limit sessions

Building on the existing multi-level memory management mechanism, this feature adds the capability to log SQL statements when memory usage exceeds predefined limits. This enables the reproduction and analysis of memory over-limit issues, facilitating faster problem diagnosis and resolution. The main functions are as follows:

• When the memory usage of a single SQL statement exceeds the upper limit, an error is reported and the SQL statement details are logged.
• When the memory usage of a single session exceeds the upper limit, an error is reported and the SQL statement details are logged.
• A wait event is added and the current SQL statement is logged before data is flushed to disks when the value of work_mem exceeds the upper limit.
• After the global memory usage exceeds the upper limit, the top 10 sessions and threads that occupy the memory are logged. The corresponding thread types and SQL statements are logged as well.

Logical decoding that adapts to ultimate RTO

The logical decoding function of the standby node needs to access the system catalogs of the historical version. When ultimate RTO is enabled, logical decoding for the standby node needs to be adapted to prevent page recycling and use correct snapshots to access system catalogs.

Hot patch management

The list interface is connected to the management plane so that the management plane can query the patches that take effect on the kernel side.

Full SQL data supported by standby nodes of distributed instances

Full SQL data and slow SQL statements can be recorded and checked on the standby DN in a distributed instance. Full SQL data and slow SQL statements can be checked on the standby DN or on the CN by using the get_global_full/slow_sql_by_timestamp function.

Repairing corrupted pages

A manual repair operation is provided for corrupted pages.

EXECUTE DIRECT ON supported for querying standby nodes in distributed instances

Currently, distributed instances support readable standby nodes, with standby nodes handling read workloads that require operational monitoring. However, the current O&M operations from the CN cannot directly access the standby DN to query views or functions. As a result, it is difficult to locate faults. Additionally, in the distributed environment, WDR-persisted global views cannot access data from the standby DN, and thus, there is a need to extend access to standby nodes. Therefore, it is necessary to enhance the EXECUTE DIRECT ON and distqry_remote_function_result_handler functionalities to support connections to one or more standby nodes in a distributed system for performing DFX queries, improving DFX capabilities.

Enhanced fault locating and demarcating

The system resource check is added to the CM. If the resource usage exceeds the threshold, a log is recorded to facilitate subsequent fault locating.

Proactive clearing of physical replication slots

Physical replication slots used in primary/standby instances and streaming replication-based disaster recovery solutions may have their logs recycled when the streaming replication connection is broken, as the logs are protected by max_size_for_xlog_prune. In such cases, the slots associated with streaming replication become inactive. Even if streaming replication is restored later, the removal of WAL segments can force a full rebuild of the standby node. To prevent this, proactive cleanup of these physical replication slots is performed when log recycling is triggered by reaching the max_size_for_xlog_prune limit.

Multiple SQL throttling rules

In a database system, a certain type of SQL statements may be executed abnormally or the number of concurrent service requests increases sharply, which will occupy a large number of system resources or affect the execution of other workloads, or even the entire database system cannot respond to other service requests. To solve this problem, GaussDB implements SQL throttling to limit the number of concurrent SQL statements of a certain type from multiple dimensions.

Managing logs of logical replication slots

WAL logs retained in logical replication slots are managed.

Security hardening

The following security vulnerabilities are fixed:

CVE-2024-2511

CVE-2024-1013

CVE-2024-25062

CVE-2024-0727

CVE-2023-52323

CVE-2023-6237

CVE-2023-46219

CVE-2023-50472

CVE-2023-50471

2023-11

CM disk detection capability enhancement

The capability is enhanced in the following aspect: The CMA uses an independent module for disk detection so that the module can detect the disk status and report the status to the CMS for arbitration.

Security hardening

The following security vulnerabilities are fixed:

CVE-2023-46218

CVE-2023-5678

CVE-2023-5363

CVE-2023-45853

CVE-2023-45322

CVE-2023-38546

CVE-2023-38545

CVE-2023-4807

CVE-2022-34038

CVE-2023-36054

CVE-2023-3817

CVE-2023-3446

CVE-2023-2975

CVE-2023-28370

CVE-2023-2650

CVE-2023-28320

CVE-2023-28321

CVE-2023-28322

CVE-2023-32082

CVE-2023-1255

CVE-2023-29469

CVE-2023-28484

CVE-2021-28235

CVE-2023-27538

CVE-2023-27536

CVE-2023-27535

CVE-2023-27534

CVE-2023-0466

CVE-2023-0465

CVE-2023-0464

CVE-2023-27533

CVE-2022-4304

CVE-2022-4450

CVE-2023-0215

CVE-2023-0217

CVE-2023-0216

CVE-2022-4203

CVE-2023-0401

CVE-2023-0286

CVE-2022-42898

CVE-2022-3996

CVE-2022-40303

CVE-2022-40304

CVE-2022-29824

CVE-2021-37750

2023-08

Optimization on memory usage of stored procedures in global PL/SQL cache

The shared cache based on the stored procedure addresses the high memory usage identified by the stored procedure compatibility. The existing PL/SQL session-level cache is optimized in the following aspects:

1. Global PL/SQL Cache: All sessions share a PL/SQL compilation intermediate product, and their execution status is maintained independently.
2. Syntax node optimization: The data structure of syntax nodes is optimized to reduce the memory usage of a single compilation product.
3. Global PL/SQL cache is adapted for high-concurrency calls.
   • The reference count (ref count) is accurate.
   • The data structure splitting of compilation products is enhanced.
   • The high-concurrency handling capability is enhanced for calling stored procedures.

Specifications:

1. If the size of the PL/SQL compilation product cache was M, the size is now reduced to M/N (N indicates the number of long connections).
2. The specifications are equivalent to the existing GPC performance specifications.

Restrictions:

Applicable only in centralized mode.

Processing of stored procedure parameters and recompilation upon expiration

1. Exceptions caused by long Out parameter values are handled.
2. Recompilation upon expiration in the SQL syntax is supported.
   • SQL statements included in a stored procedure must be checked.
   • Expressions included in a stored procedure must be checked.
3. The validity of values on the right-hand side of the "=" sign are checked.
4. GaussDB is compatible with the syntax that a plus sign (+) is used when a function with multiple input parameters or the input parameter in a stored procedure is wrapped in a new line.
5. The FFIC information is added when a core dump occurs in the stored procedure.
6. The set type capability is enhanced. Element length verification and element precision conversion are supported. The index length of the set type with indexes can be verified.

Specifications and restrictions:

Invalid objects can be recompiled using scripts. The schema/compile_all parameter is supported, but the reuse_settings parameter is not supported.

Stored procedure cursors and historical slow SQL statements

1. The INSERT xxx SELECT xxx statement is supported. The SELECT target list expression contains both functions without parameters and functions without parameters as input parameters. (Both centralized and distributed instances are supported.)
2. If the WHERE condition in a SQL statement contains col=func(xxx), the performance is the same as that of O databases.
3. Historical slow SQL statements in stored procedures can be viewed (available for centralized instances).

Delivery mode: centralized and distributed GaussDB instances

Single-replica DR for distributed instances

Single-replica DR is put into commercial use. The specifications and restrictions are as follows:

1. The primary cluster supports only the 1 primary + 2 standby deployment model.
2. The SLA of high availability is not promised.
3. After a DR switchover, the DR cluster cannot be upgraded to the 1 primary + 2 standby deployment model.
4. Only single-node, single-replica clusters that use the Quorum protocol can be used as DR clusters.

Delivery mode: centralized and distributed GaussDB instances

Type conversion, UNION operation, and package adaptation

1. The GaussDB type conversion capability is enhanced.
2. This solves the problem that data and indexes cannot be properly matched when equality matching is performed between the bpchar type (containing multiple spaces) and the text type.
3. This solves the problem that the sorting information is lost when the UNION operation is rewritten in the SQL statement.
4. This solves the problem that the number of lines in the package error message is incorrect.
5. This solves the problem that when an element of the array type exceeds the defined array length, no exception is thrown and only null is returned.

Delivery mode: centralized GaussDB instances

Support of GB18030

1. GB18030 level 3 is supported, including 87,887 Chinese characters. The new government procurement standard that requires the support of GB18030 was enforced on August 1, 2023.
2. GaussDB supports the GB18030-2022 character set.
3. The new national standard GB18030-2022 character set is supported.

Streaming DR supported for Dorado-based dual-cluster deployment

The GaussDB dual-cluster (Dorado-based) deployment solution supports streaming DR.

1. Two Dorado-based clusters function as the primary cluster for remote streaming replication to support disaster recovery.
2. Two Dorado-based clusters function as the DR cluster for remote streaming replication to support disaster recovery.
3. In three-cluster mode, functions such as backup, node repair, node replacement, and version upgrade are supported.
4. In the Dorado-based dual-cluster DR switchover scenario, the kernel RTO is less than or equal to 2 minutes.
5. The scenario where the DR and upgrade processes overlap are supported.

Delivery mode: centralized GaussDB instances

Enhanced fault tolerance capability of gs_loader

1. Not all data is rolled back when gs_loader violates constraints.
2. If gs_loader contains incorrect data, not all data needs to be rolled back. The rows parameter is supported to specify the number of rows committed in a transaction.
3. If the imported data contains \n, a truncated character, or a character that cannot be found in the character set, the original fault tolerance mechanism does not correctly write the error data to the bad file or error table. This issue is rectified by the enhanced mechanism.

Minor version upgrade supported in the upgrade to be committed scenario of small-scale deployment

In small-scale deployment, minor versions can be upgraded during the service observation period, and rollback by minor version is supported.

System object privilege hardening for upgrade

During the upgrade process, users' system object privileges may be lost. To prevent this problem, this feature backs up the privileges before the upgrade and restores them afterwards, ensuring that users' system object privileges remain consistent before and after the upgrade.

Security hardening

The following security vulnerabilities are fixed:

CVE-2023-3817

CVE-2023-32001

CVE-2023-3446

CVE-2023-2975

CVE-2023-28370

CVE-2023-2650

CVE-2023-28320

CVE-2023-28321

CVE-2023-28322

CVE-2023-32082

CVE-2023-1255

CVE-2023-29469

CVE-2023-28484

CVE-2021-28235

CVE-2023-27536

CVE-2023-27538

CVE-2023-27535

CVE-2023-27534

CVE-2023-0466

CVE-2023-0465

CVE-2023-0464

CVE-2023-27533

CVE-2022-4304

CVE-2022-4450

CVE-2023-0215

CVE-2023-0217

CVE-2023-0216

CVE-2022-4203

CVE-2023-0401

CVE-2023-0286

CVE-2022-42898

CVE-2022-3996

CVE-2022-40303

CVE-2022-40304

CVE-2022-29824

CVE-2021-37750

2023-04

START WITH recursion performance optimized

The performance of the following statements is optimized.

• START WITH...CONNECT BY statements
• START WITH...CONNECT BY statements with the connect_by_isleaf field

Viewing running SQL statements in the current stored procedure

View SQL statements that are being executed in the current stored procedure.

Session-level HA attributes

1. Session-level standby node connection detection and strong synchronization of session-level logs are supported.
   1. Session-level standby node connection detection: After this function is enabled, the system checks whether the standby node can be connected. If the standby node cannot be connected, the primary node reports an error.
   2. Strong synchronization of session-level logs: After this function is enabled, sessions between the primary and standby nodes are strongly synchronized.
2. Single-node mode:
   1. Parameters for session-level standby node connection detection can be set, but the function does not take effect. After data is written to the primary node, a success message is returned.
   2. Parameters for strong synchronization of session-level logs can be set, but the function does not take effect. After data is written to the primary node, a success message is returned.
3. A full build of the standby node is supported.
   1. During a full build (the standby node cannot be connected):

      Parameters for session-level standby node connection detection can be set, but the function does not take effect. After data is written to the primary node, a success message is returned.

      After the parameters for strong synchronization of session-level logs are configured, the function takes effect. After most_available_sync is enabled and keep_sync_window is configured, new sessions for strong synchronization enter the maximum availability mode if the disconnection duration of the standby node exceeds the value of keep_sync_window.

   2. When the standby node is restoring the logs generated during the full build:

      Parameters for session-level standby node connection detection can be set, and the function can be enabled.

      Parameters for forced synchronization of session-level logs can be set, and the function can be enabled.

4. When the network is disconnected, the detection timeliness depends on parameters tcp_user_timeout and wal_sender_timeout. When the primary and standby nodes are disconnected (including scenarios where the wal receiver process on the standby node exits normally or crashes), the connection status of the standby node is updated synchronously.

Specifications:

1. One primary node and one standby node are deployed. The data in the primary node is asynchronously synchronized to the standby node.
2. If the standby node cannot be connected, write request errors are reported during and the service needs to be retried.

CURRENT OF CURSOR

The CURRENT OF CURSOR syntax is supported.

Restrictions for CURSOR statements are as follows:

1. Only simple queries of a single table in the SELECT statement are supported.
   1. Subqueries are not supported.
   2. LIMIT/OFFSET is not supported.
   3. START WITH is not supported.
   4. WITH statements are not supported.
2. Multiple tables are not supported.
3. Partitioned tables are not supported.
4. Column store is not supported.
5. Hash bucket tables are not supported.
6. ORDER BY is not supported.

Restrictions for UPDATE statements are as follows:

1. Only a single table can be updated.
2. UPDATE statements can be used only in stored procedures.
3. Multiple tables are not supported.
4. Multiple WHERE clauses are not supported.
5. WITH/USING and ORDER BY are not supported.

Read-Only detection capability of CM disks enhanced

The capability is enhanced in the following aspects:

1. The read-only status is obtained from the database to ensure accuracy.
2. Read-only quorum arbitrates only the nodes that exceed the threshold. Other nodes are not affected.
3. When the primary node is read-only, a primary/standby switchover is automatically performed. An available standby node is promoted to primary to ensure that the instance can work properly.

O&M database connection optimized

Peer authentication is not used. In addition, concurrency conflicts are resolved and O&M performance is improved.

Unified name GaussDB

The name is changed to GaussDB.

Automatic start of redistribution after lock timeout during scale-out

The automatic retry mechanism is provided for scale-out. If the lock times out for multiple times, the mechanism automatically exits. After a period of time, scale-out is retried to switch tables during off-peak hours, reducing the impact on user services.

Software component lifecycle matching the product lifecycle

The software component lifecycle can match the product lifecycle.

1. Update underlying software components (such as open-source, third-party, and self-developed software).
2. Fix security vulnerabilities promptly within the product version lifecycle.

Single-replica deployment enhanced (for non-production environments)

1. Monitoring metrics are supported.
2. An alarm can be triggered when a node is set to read-only due to the fully occupied disk.
   1. O&M interface reporting adapts to the single-node mode.
   2. CM makes the read-only alarm information persistent on DCC.

Optimizer adaptation and robustness improved

The operator preference provides assurance if the cost model fails, and multi-version plan selection capability well as intelligent statistics are enhanced, improving the adaptability and robustness of the optimizer.

1. In the cost convergence scenario, the operator preference mechanism is provided to prevent the optimizer from selecting a poor plan that is incorrectly estimated. In this way, the plan performance is improved by 50% when the statistics are invalid.
2. The robustness of intelligent statistics is enhanced, and statistics of multiple columns are integrated to create a statistics model, improving the evaluation performance of high-frequency values by 50%. The overall performance is improved by 20% to 30%.
3. Currently, many slow queries can be resolved only by using service hints. Multi-version plans are required to support more scenarios and provide an autonomous escape mechanism to automatically resolve more than 10% slow queries in specific scenarios. The performance deterioration in the TPC-C test model is less than 5%. This is only available for centralized instances.

Data restoration in extreme scenarios

In extreme scenarios, users can restore their data by outputting disk data files which need to be identified by the COPY command. Then, data can be imported to a new DB instance as needed.

Generally, data restoration is based on existing redundancy technologies. This solution is used only in the following scenarios:

1. Node recovery and primary/standby build
2. Disaster recovery
3. Backup and restoration
4. Hot patch
5. Other cases where the cluster needs to meet the expected requirements

Constraints:

1. Only flushed data is parsed.
2. Data consistency cannot be ensured during offline data parsing.
3. Xlog replay cannot be guaranteed.
4. The database is offline.
5. Only heap table data in Astore and Ustore can be parsed.

Undo files of Ustore managed by the segment-page mechanism

The undo files of Ustore are managed by the segment-page mechanism. This prevents file handle leakage when a large number of 1 MB undo files are generated.

Global secondary indexes for distributed instances

Global secondary indexes for distributed instances

1. Global unique indexes can be created on non-distribution keys.
2. The performance of point queries on non-base table distribution keys is three times that of common index queries.

1. Astore is supported.

Constraints:

• Common UB-tree indexes can be created. Local indexes and expression indexes are not supported.
• GSIs whose distribution is the same as that of base tables cannot be created.
• The GSI distribution key in a base table cannot be updated.
• CREATE GSI CONCURRENTLY and PARALLEL are not supported.
• Hash-based GSIs can be created for row-store Astore tables and partitioned tables whose base tables are hash-based. Base tables cannot be replicated tables, bucketed tables, segment-page tables, list/range-based tables, Ustore tables, or column-store tables. GSIs only support hash-based distribution.
• GSI can only be used in distributed instances.
• If _new$$ or _NEW$$ is added to a column name of a base table or an identifier (including ctid, xc_node_hash, xmin, xmax or tableoid), a GSI fails to be created for the base table whose column names are repeated.
• The GSI can be created only in GTM-Lite mode.
• IUD returning is not supported for base tables with GSIs.
• TABLE ACCESS BY INDEX ROWID is not supported.
• VACUUM FULL, COPY, GDS, and REINDEX TABLE/DATABASE are not supported. All GSIs will be skipped during execution.
• The following models are not supported: UPSERT, MERGE INTO, HOT UPDATE, max_datanode_for_plan (in PBE mode), logical replication, cluster, and For Update/share/nowait/skip Locked
• Compatibility syntax such as Start With is not supported.
• A maximum of 27 columns (including index keys and distribution keys) are supported.
• Other constraints are the same as those of hash distribution, UBTree, LP/FQS, and common indexes.

CURRENT OF CURSOR

The CURRENT OF CURSOR syntax supports ORDER BY.

SQL features supported by Ustore in centralized instances

SQL features supported by Ustore in centralized instances:

1. Global temporary tables
2. Column constraints
3. Parallel query
4. Statistics enhancement: Ustore index active page estimation and cost adaptation.

Out parameter overloading and capability enhancement for stored procedures

1. SQL PATCH scope: DML within a stored procedure
2. Out function overloading scope: Functions with Out parameters can be created and invoked. Only Out parameters in direct value assignment can be returned. For example, in a = func(b), b is an out parameter and is returned.

Overload escape in distributed scenarios

Overload escape is supported in distributed scenarios.

1. After a certain proportion of slow SQL statements in a thread pool are executed, the CANCEL operation is triggered.
2. Control or circuit breaker measures are taken for SQL statements that occupy a large number of resources to prevent normal SQL statements from being executed due to slow SQL statements.

SQL PATCH in distributed instances

SQL PATCH in distributed instances

Constraints:

1. SQL PATCH operations cannot be synchronized between CNs.
2. SQL PATCH cannot be used for DNs.
3. If the lightproxy plan delivered by a CN to a single node triggers the execution of SQL Patch, the plan is degraded to the FQS plan delivered by the CN to a single node, and SQL PATCH takes effect.
4. Similar to HINT, SQL PATCH affects the capability and specifications of distributed plans.
5. SQL PATCH cannot be used for DNs. For distributed instances, if a stored procedure is pushed down, the DML in the stored procedure generates plans on DNs and is not affected by SQL PATCH.

Ultimate RTO when standby nodes can process read requests

Based on the ultimate RTO technology, standby nodes can process read requests.

Constraints:

1. The DDL statement replay conflicts with the read-only mode of standby nodes.
2. When standby nodes process read requests, they consume CPU, memory, and disk space. The memory and disk space are configured based on GUC parameters. If used memory and disk space exceed the configured GUC values, the system will forcibly reclaim the overused resources.

Syntax extension and cost model optimized

Syntax extension and cost model optimized

1. Auto analyze:
   1. Auto analyze is triggered when a large amount of data is inserted and updated in a short period of time to quickly process recycle task tables.
   2. By default, the sampling ratio and histogram precision are automatically increased based on the table size to improve the analysis accuracy of large tables.
   3. The adaptive estimation algorithm is provided to improve the NDV estimation accuracy in typical scenarios.
2. Hints can be used:
   1. In parameterized paths.
   2. In Bitmap Scan operations.
   3. Across query blocks.

1. The Plan Trace function can display the complete process of generating a query plan.

   This is only available for centralized instances.

Index creation and rebuild online

[Requirement description]

1. Indexes (including common indexes, global indexes, and local indexes) can be concurrently created and rebuild in Astore.
2. Indexes (including common indexes, global indexes, and local indexes) can be concurrently created and rebuild in Ustore.

Specifications:

• The online DML performance deteriorates by no more than 10%.
• Online index fields cannot be added, deleted, or modified.
• Only common tables and level-1 partitioned tables are supported.

DFX and other features enhanced for Ustore in centralized instances

Centralized instances support the following Ustore features after related whitelist is enabled:

1. DFX capabilities are supplemented, and auxiliary verification for DML operations and REDO/UNDO functions is enhanced.
2. VACUUM FULL is supported.
3. UNDOMETA reduces the frequency and fields
4. FSC structure is embedded and its performance is optimized.

Basic operator performance improved

The performance of basic operators is improved.

1. SRF
2. Expression framework flattened
3. AGG optimized
4. INNER UNIQUE optimized
5. Expression operator optimized
6. Parser optimized
7. Printtup optimized
8. Communication performance optimized
9. Index scanning operators optimized
10. Predicates optimized
11. Subquery optimized
12. Noise floor elimination for basic operators
13. Indexes optimized

Performance improvement of basic operators: The end-to-end TPC-H performance is improved by 20%, and the TPC-H performance is improved by 40% (including the full table scan operators, expression operators, and aggregation operators).

SQL M* syntax compatibility

The following M* syntax is supported:

1. Centralized instances support the GROUP_CONCAT function.
2. User-defined variables in a session can be set through SET statements.
3. SET is enhanced in global variable configuration.
4. Prefix indexes are supported.
5. Delimiters are supported.
6. DELETE statements can be used for deleting multiple tables.

Fully-encrypted database framework with software and hardware integrated

Fully-encrypted database framework with software and hardware integrated: Based on the virtual TEE, the fully-encrypted framework can encrypt memory and ensure security isolation.

1. Secure key transmission channel is supported.
2. The fully-encrypted execution framework integrates software and hardware and supports the virtual TEE execution.
3. Server-side encryption operator support range query and sorting operations. A whitelist is used to restrict syntax operations.
4. The JDBC client driver is supported.

This is only available for centralized instances.

Logical decoding capability enhanced

1. Logical replication supports distributed strong consistency.
   1. Sequence preserving is ensured when distributed transactions are committed.
   2. Sequence preserving is ensured in the same transaction on a single DN.
   3. Sequence preserving is not supported between different DNs of the same transaction.

   Constraints: This feature is only suitable for GTM-Lite mode.

2. Logical decoding performance views are provided.
   1. Log sending speed (MB/s).
   2. Log read speed (MB/s).
   3. Log parsing speed (MB/s).

   Constraints: For parallel decoding, only the parallel decoding performance views of the primary nodes for centralized instances are provided.

3. Distributed logical decoding is supported. DRS can directly connect to each standby DN in the service cluster to perform logical decoding that is replayed in the cluster scale-out.
4. The following two monitoring metrics are added:
   1. Maximum number of transaction lines
   2. Number of large transactions (with more than 4096 lines)
5. A heartbeat message is added so that external tools can accurately calculate the synchronization latency.
   1. The latest WAL timestamp is provided for external tools to calculate the latency.
   2. Parallel decoding provides functions to query the latest WAL timestamp. Serial and parallel decoding provide heartbeat messages.
6. The logical decoding performance can reach 300 MB/s, which optimizes:
   1. The performance of saving data to the database is optimized.
   2. Single-slot decoding performance of primary nodes
   3. Concurrent replay performance of standby nodes
   4. Single-slot decoding performance of standby nodes

Partitioned table functions and performance optimized

1. LIST and RANGE multi-column partitioning are supported.
   1. The number of LIST partition keys can be increased from 1 to 16.
   2. The number of RANGE partition keys can be increased from 4 to 16.

1. Range/List/Hash (RLH) partitions:
   1. RLH partitioned tables mean LIST, RANGE, and HASH tables.
   2. LIST/RANGE tables support node group distribution.

1. The partition syntax is optimized as follows:
   1. LIST and level-2 partitions support SPLIT and MERGE functions.
   2. LIST, HASH, and level-2 partitions support CREATE TABLE ... LIKE statements.

   Constraints: DDL and DML concurrency across partitions is not supported in distributed instances.

1. Continuous improvement of partitioned table performance:
   1. The partitioned table performance in services is improved by 30%.
   2. Partition implicit type and function pruning are optimized.

   This is only available for centralized instances.

SQL O* syntax compatibility improved

1. There are new 167 compatibility items for system views, system functions, and advanced packages.
   1. System functions: 39
   2. System views: 124
   3. Advanced package: 4
2. Distributed instances support the Go driver.
3. The dblink is supported.
4. PIVOT/UNPIVOT is supported.
5. The XML data type and related interfaces are supported.
6. Distributed instances support packages.
7. Centralized and distributed instances support ROWNUM.
8. The following 11 XMLDOM interfaces are supported:
   1. XMLDOM.GETFIRSTCHILD
   2. XMLDOM.ITEM
   3. XMLDOM.DOMNODE
   4. XMLDOM.MAKEELEMENT
   5. XMLDOM.GETCHILDNODES
   6. XMLDOM.DOMNODELIST
   7. XMLDOM.GETNODEVALUE
   8. XMLDOM.GETLENGTH
   9. XMLDOM.GETNODENAME
   10. XMLDOM.GETELEMENTSBYTAGNAME
   11. XMLDOM.GETTAGNAME
   12. XMLPARSER.NEWPARSER
9. The functions of tableof are enhanced.
   1. The array can be nested with tableof.
   2. Distributed instances support tableof.

WDR optimization and DFX support

1. WDR snapshots:
   1. A statement continues to be executed after it fails to be executed due to timeout, which does not affect the overall execution of a WDR snapshot.
   2. A WDR snapshot fails during the vacuum process.
   3. The retention period of WDR snapshots is not 8 days.
2. WDR reports:
   1. The WDR report generation speed is further accelerated.
   2. The WDR report format is optimized.
   3. WDR objects are sorted based on multiple dimensions (such as dead tuples, live tuples, vacuum, analyze, scanned rows).
3. There is an interface for querying GUC parameters such as search_path configured in a session.
4. WDR reports can be read by the standby nodes.
5. L0 FULL SQL plans can be viewed.

Optimization on memory usage of stored procedures in global PL/SQL cache

The shared cache based on the stored procedure addresses the high memory usage identified by the stored procedure compatibility. The existing PL/SQL session-level cache is optimized.

1. Global PL/SQL Cache: All sessions share PL/SQL compilation intermediate products, and their execution status is maintained independently.
2. Syntax node optimization: The data structure of syntax nodes is optimized to reduce the memory usage of a single compilation product.

This is only available for centralized instances.

ODBC Gplan reuse

ODBC supports the JDBC Statement Pool function and Gplan reuse to improve SQL performance.

Go drivers for fully-encrypted databases

A fully-encrypted database can:

1. Insert 60,000 rows per second.
2. Query 90,000 rows per second.
3. Update temporary tables.
4. Support HIS key management.
5. Support server-side encryption and decryption functions.
6. Update CMK keys.
7. Support load balancing for JDBC.

Go language drivers are available for fully-encrypted databases.

Minimum RTO due to improved primary/standby synchronization

The restriction that a standby node can reconnect to the primary node only after the replay is complete is removed, and the RTO performance is optimized.

User-level audit settings

The audit function is enhanced as follows:

1. User-level audit is supported. The GUC parameter full_audit_users is added to set the list of audited users. Audit logs are recorded for all SQL statements executed by users in the list. Administrators' operations can be fully audited to audit administrators' access to user data.
2. The audit operation can be configured based on client information. The GUC parameter no_audit_client is added to configure the list of clients that do not need to be audited. Audit logs are not recorded for specified client apps and IP addresses (such as cm_agent@127.0.0.1).
3. Audit logs are recorded for high-risk system function calls.

Strong security authentication supported by Roach

Roach supports SSL and Kerberos security certificate authentication.

Security hardening

The following security vulnerabilities are fixed:

CVE-2023-30608

CVE-2023-29469

CVE-2023-28484

CVE-2021-28235

CVE-2023-27538

CVE-2023-27536

CVE-2023-27535

CVE-2023-27534

CVE-2023-0466

CVE-2023-0465

CVE-2023-0464

CVE-2023-27533

CVE-2023-23931

CVE-2023-23915

CVE-2023-23914

CVE-2023-23916

CVE-2022-4304

CVE-2022-4450

CVE-2023-0215

CVE-2023-0286

CVE-2022-42898

CVE-2022-43552

CVE-2022-43551

CVE-2022-40303

CVE-2022-40304

CVE-2022-32221

CVE-2022-42916

CVE-2022-42915

CVE-2022-1941

CVE-2022-35252

CVE-2022-37434

CVE-2022-32205

CVE-2022-32208

CVE-2022-32206

CVE-2022-2097

CVE-2022-32207

CVE-2022-2068

CVE-2022-27782

CVE-2022-27781

CVE-2021-22570

CVE-2022-29824

CVE-2022-1292

CVE-2022-27775

CVE-2022-27774

CVE-2022-27776

CVE-2022-22576

CVE-2021-3520

CVE-2021-37750

2023.3

WDR optimization

Sort top SQL statements by the average duration of a single SQL statement execution. Full SQL statements can be sorted by avg based on the top SQL statements in the current WDR report.

Online specifications change

Change instance specifications online with no downtime. DNs are rebooted in rolling mode, without the need to reboot the instance. This mitigates the impact on services.

Full SQL tracing on standby nodes and statement-level wait events

1. Support full SQL tracing on standby nodes.
2. Measure time consumption statistics on statement-level wait events for full SQL tracing.

Constraints:

1. Full SQL tracing on standby nodes is available only for centralized instances.
2. Time consumption statistics of wait events are measured for L0 full SQL tracing. The performance loss of wait events is less than 3%.

Domain name configuration for gsql and gsloader

Domain name configuration for gsql and gsloader ensures that multiple domain names can be accessed in HA mode.

1. Multiple domain names can be configured for distributed instances.
2. Centralized instances support leader election.

Automatic creation of multi-column statistics

Advantages:

1. Solve the problem of composite index cost calculation.
2. Enhance multi-column statistics.
3. Solve the problem that an incorrect index is selected in typical scenarios.

Backups created by standby nodes

Move some backup operations from primary nodes to standby nodes to reduce the usage of resources, such as the CPU and disk I/O, of primary nodes.

SMP supported by global temporary tables

Global temporary tables support SMP.

Backup and restoration for a single-node instance

Backup and restoration for a single-node instance is supported. This feature meets the requirements of R&D and joint commissioning scenarios. It is used only for SQL development and function joint commissioning.

Constraints:

1. This is only available for centralized instances.
2. Service production environments are not supported.
3. Availability, reliability, and DR (the RTO cannot be ensured) are not committed.
4. O&M functions are not supported, including but not limited to node replacement, scale-out, upgrade, and specification change.
5. Fault injection tests and special tests on performance, pressure, and long-term stability are not supported.
6. There is no requirement on performance or impact on service performance.

SQL compatibility enhancement

SQL compatibility is enhanced in the following aspects:

1. Support the following advanced packages:
   1. UTL_FILE.FILE_TYPE
   2. DBMS_SQL.VARIABLE_VALUE
   3. UTL_FILE.FGETATTR
   4. DBMS_LOB.READ
   5. DBE_LOB
   6. DBE_SQL
   7. DBE_FILE
   8. DBE_RAW
   9. DBE_UTILITY
2. Cancel the limitation on concurrent sessions in an autonomous transaction.
3. Check the compilation integrity in stored procedures.
   1. Check the compilation integrity.
   2. View the dependency of compiled objects. Invalid objects can be recompiled.
4. Support the following custom types:
   1. Array
   2. Tableof
   3. TableofIndex
   4. Combination of the preceding three types
5. Support the following 33 system functions:

   TO_CHAR, SYSDATE, TO_NUMBER, UPPER, CHR, SYS_CONTEXT, TO_DATE, INSTRB, LPAD, ROUND, CAST, TRIM, SUM, ADD_MONTHS, LAST_DAY, MOD, LOWER, RTRIM, MONTHS_BETWEEN, TRUNC, RPAD, LEAST, GREATEST, REGEXP_SUBSTR, LTRIM, TRANSLATE, TREAT, EMPTY_CLOB, USER, REGEXP_REPLACE, AVG, NVL2, REGEXP_COUNT

6. Support the following system views:

   ALL_ARGUMENTS, ALL_IND_PARTITIONS, ALL_IND_SUBPARTITIONS, ALL_INDEXES, ALL_OBJECTS, ALL_SOURCE, ALL_TAB_COLUMNS, ALL_TAB_PARTITIONS, ALL_TABLES, DBA_ARGUMENTS, DBA_CONS_COLUMNS, DBA_CONSTRAINTS, DBA_DEPENDENCIES, DBA_DIRECTORIES, DBA_HIST_SNAPSHOT, DBA_HIST_SQL_PLAN, DBA_HIST_SQLSTAT, DBA_HIST_SQLTEXT, DBA_IND_COLUMNS, DBA_IND_EXPRESSIONS, DBA_IND_PARTITIONS, DBA_IND_SUBPARTITIONS, DBA_OBJECTS, DBA_PROCEDURES, DBA_SCHEDULER_JOBS, DBA_SCHEDULER_PROGRAMS, DBA_SCHEDULER_RUNNING_JOBS, DBA_SEGMENTS, DBA_SOURCE, DBA_TAB_COLS, DBA_TAB_COLUMNS, DBA_TAB_PARTITIONS, DBA_TABLES, DBA_TRIGGERS, DBA_TYPE_ATTRS, GV$INSTANCE, GV$SESSION, PLAN_TABLE, USER_JOBS, USER_OBJECTS, USER_PROCEDURES, USER_TABLES, V$INSTANCE, V$MYSTAT, V$SESSION, ALL_DEPENDENCIES

7. Enhance output parameters of stored procedures and functions:
   1. Functions can return the record type.
   2. The stored procedure that contains the default value and out parameter can be invoked.
8. Improve the performance in stored procedures.
   1. Large loop scenarios of stored procedures, for example, for i in 1 .. 100000 loop
   2. Autonomous transactions (10 packages, each containing 10 variables, and 100,000 calls)
9. Commit+exception performance optimization: Optimize performance bottlenecks in the commit+exception scenarios and optimize the resource release related to EXCEPTION

Readable standby nodes in a distributed instance

Offload read requests from primary nodes to standby nodes in a distributed instance.

Constraints:

• Weak read consistency is provided. However, only read consistency is guaranteed only for incremental snapshots within a session, not for those among sessions.
• Session-level parameters are used to distinguish workloads on primary nodes from read workloads on standby nodes. If a standby node is faulty, its workloads cannot be switched to the primary node.
• In abnormal scenarios such as primary/standby switchovers and standby node faults, an error will be reported for the ongoing read service on the concerned standby node, and a service retry is required.
• After the resumption of a standby node or after adding replicas, an error will be reported for the ongoing read service on the concerned standby node, and a service retry is required.
• If no proper standby node can be selected, an error is reported and a service retry is required.
• After standby nodes process read requests, the RTO of the standby nodes is affected. If the RTO exceeds the specified threshold, traffic limiting or circuit breaker needs to be triggered after an error is reported on the service side.
• In DDL data replay scenarios, an API is provided for applications to check whether DDL operations on the standby node are consistent with those on the primary node. If consistency is confirmed, standby nodes can process read requests. Otherwise, a circuit breaker is triggered.

Online index creation and ultimate RTO supported by Ustore-based centralized instances

The Ustore-based centralized instances support online index creation and ultimate RTO.

1. Indexes can be created online without interrupting services.
2. An ultimate RTO improves the performance of primary/standby instances (The standby nodes in instances cannot process read requests).

Upsert and stored procedures supported by GPC

GPC supports upsert and stored procedures.

1. In the pgxc framework, upsert can be executed through gplan.
2. Stored procedures, functions, and packages are supported.

Performance improvement of basic operators

The performance of basic operators is improved.

1. The seqscan and PI operators are optimized.
2. The cost model is optimized and adjusted.

Codegen supported by row-store expressions

The Codegen capability of row-store tables is enhanced: Codegen is enabled by default. Users can query whether Codegen is enabled the specified number of rows. By default, all queries are not split.

Optimizer cost model optimization

The following basic capabilities are optimized:

1. cplan/gplan selection
2. coalesce selectivity estimation
3. nestloop/merge join in some scenarios
4. semi/anti join cost

This is only available for centralized instances.

Optimization on concurrent partitioned table query

Optimize concurrent query on partitioned tables in the following aspects:

• DML/DDL concurrency across partitions is supported.
• Static pruning and dynamic pruning are supported for expressions in partitioned tables.
• Partitioned tables can be used as parameterized paths of internal tables.
• MergeSort query plans are supported.

MySQL database syntax compatibility improved

GaussDB is compatible with common syntax and APIs of MySQL databases.

DELETE and UPDATE statements support the ORDER BY and LIMIT functions.

Multiple IP addresses for Python drivers

Support multiple IP addresses for Python drivers, so that databases can be properly connected after a primary/standby switchover.

PITR enhancement

Enhance PITR capabilities in the following aspects:

• Performance deterioration caused by PITR backups is reduced for two-phase distributed transactions in distributed GTM-Lite mode.
• PITR backups are not affected if the external storage device is faulty.

Embedded C preprocessor

Embedded C preprocessor is supported.

Overload escape in concurrency scenarios

After a certain proportion of slow SQL statements in a thread pool are executed, the CANCEL operation is triggered.

Data page restoration

Data restoration is enhanced in the following aspects:

1. Data files on the standby node can be restored to prevent file deletion or damage.
2. Damaged pages can be automatically repaired during the creation of backup. There is no impact on the backup.

GCM encryption and client sorting for fully-encrypted databases

Fully-encrypted databases support GCM encryption and client sorting.

1. The sorting operation is implemented based on clients.
2. GCM encryption and bit transmission are supported.

Constraints:

Only gsql and JDBC clients are supported.

SHA256 and SSL that uses SM series cryptographic algorithms

1. The data channel between the client and server supports SSL-encrypted transmission using SM series cryptographic algorithms.

   Constraints: Only gsql clients are supported.

2. SHA256 encrypts and saves sensitive data in the Shanghai data base project.

ANY permission management

Support the following 12 ANY permissions for five objects:

• ALTER ANY TYPE, DROP ANY TYPE
• ALTER ANY SEQUENCE, DROP ANY SEQUENCE, SELECT ANY SEQUENCE
• ALTER ANY INDEX, DROP ANY INDEX
• CREATE ANY TRIGGER, ALTER ANY TRIGGER, DROP ANY TRIGGER
• CREATE ANY SYNONYM, DROP ANY SYNONYM

Efficient data compression algorithm

With the efficient data compression algorithm, the read and write performance deterioration in the TPC-C scenario is less than 5%, and the data compression ratio of transactional databases reaches 2:1.

ABO

ABO supports the following service scenarios:

1. Intelligent cardinality estimation: improves the cardinality estimation accuracy of multi-column equality queries.
2. Adaptive plan selection: provides cache multi-plan management and adaptive selection.

Specifications:

1. Intelligent cardinality estimation: the ABO statistical algorithm enhanced (efficiency improved by 1x and performance improved by 50% in typical scenarios)
2. Adaptive plan selection: the efficiency improved by 1x in typical scenarios

Display of query plans in running state

Display the execution plan specifications using views for specified slow SQL statements.

Security hardening

The following security vulnerabilities are fixed:

CVE-2022-29824

CVE-2022-27781

CVE-2022-27775

CVE-2022-35252

CVE-2021-37750

CVE-2022-32205

CVE-2022-27776

CVE-2022-32206

CVE-2022-37434

CVE-2022-22576

CVE-2022-27782

CVE-2022-27774

CVE-2022-32207

CVE-2022-2097

CVE-2022-32208

CVE-2022-32221

CVE-2022-42916

CVE-2022-42915