New Features in Version 8.1.3
For details about new features in 8.1.3.x, see New Features. For details about resolved issues in each patch, see New Features and Resolved Issues in 8.1.3.x.
[Release date of V8.1.3.325]: November 2, 2023
[Release date of V8.1.3.323]: October 16, 2023
[Release date of V8.1.3.322]: August 4, 2023
[Release date of V8.1.3.321]: June 26, 2023
[Release date of V8.1.3.320]: May 19, 2023
[Release date of V8.1.3.310]: March 9, 2023
[Release date of V8.1.3.300]: December 19, 2022
[Release date of V8.1.3.200]: October 31, 2022
[Release date of V8.1.3.110]: August 23, 2022
[Release date of V8.1.3.100]: June 20, 2022
[Release date of V8.1.3]: April 15, 2022
New Features
Feature |
Description |
Reference |
---|---|---|
Partition addition and partition table management are supported. |
PERIOD (interval type) and TTL (interval type) are now available in partition management. |
|
JSON foreign tables are supported. |
|
|
Cross-Cluster interconnection |
|
|
Enhanced cold and hot table functions |
Automatic migration of cold and hot tables |
|
Enhanced stream data warehouse functions |
The time series database stores stream data in real time. |
|
The row-store ring buffer is controlled by a switch. |
The ring buffer is controlled by a switch and its threshold is adjustable. Batch update of large tables no longer uses the ring buffer. |
|
Default storage mode of a table |
It is controlled by the parameter default_orientation. If the storage mode is not specified during table creation, this parameter specifies the default storage mode. |
|
GDS supports the euro sign. |
The GBK character set supports the euro sign. |
|
OBS foreign tables support XSKY S3. |
OBS foreign tables are adapted to XSKY and the parameter eol is added. |
|
SQL supports Roaring Bitmaps. |
In the Internet, new retail, education, and gaming industries, target crowds need to be quickly selected based on their profiles. The SQL syntax now supports Roaring Bitmaps, which can be used for user profiling in Internet operations. |
|
Case-insensitive is supported to improve Teradata compatibility. |
In the data migration from Teradata to DWS, the source data contains a large amount of duplicate data with different letter cases. If DWS uses the distinct syntax to deduplicate the data, after deduplication, there is still a large amount of duplicate data remaining. If DWS uses the upper function to do that, the source data will be changed, and it requires a lot of procedures and complex logics. To reduce the workload, the case-insensitive feature is added to DWS. |
|
List partitioning |
List partitioned tables are supported. |
|
Enhanced MySQL compatibility |
ALTER TABLE can modify multiple columns at a time. ALTER TABLE supports COMMENT. ALTER TABLE supports CHANGE, DROP KEY, DROP PARTITION, and FIRST/AFTER. |
|
Projection pushdown |
Unnecessary columns in a multi-column view are automatically deleted during reference to avoid redundant calculation. |
Added the description for projection pushdown to rewrite_rule. |
Share scan |
Share scan |
Stream plan CTEs support share scan. Added enable_stream_ctescan. The value STREAM_CTESCAN is added to logging_module. The optional [NOT] MATERIALIZED is added to the WITH clause in the SELECT syntax. |
Bloom Filter for column-store tables |
In a HASH JOIN, if the thread of the foreign table contains HDFS tables or column-store tables, the Bloom filter is triggered. |
|
Column-store replication tables support UPDATE. |
DWS customers have a large number of MERGE INTO operations on replication tables. Like row-store replication tables, column-store replication tables now support UPDATE, DELETE, and MERGE INTO operations. |
|
Fine-grained permission management |
|
|
The problem of the INSERT OVERWRITE deadlock on multiple CNs is fixed. |
No deadlock occurs when INSERT OVERWRITE is triggered on multiple CNs at the same time. |
- |
Execution plan optimization |
|
Adjusting Key Parameters During SQL Tuning Added projection_pushdown and or_conversion to rewrite_rule. |
Relfile-based space statistics and space control |
|
|
Enhanced performance fault locating and demarcation |
|
|
Resource load management optimization and reconstruction |
|
|
Modified partition table features |
|
|
Redistribution column hint in the AGG process |
During data migration from Oracle data warehouses, you can manually specify the distribution column using enhanced hints. |
|
Added the function of returning NULL when the divisor is 0. (Added in 8.1.3.110) |
To specify whether to report an error when the divisor is 0 in MySQL compatibility mode, the compatibility configuration item enable_division_by_zero_mysql is added to the behavior_compat_options parameter. |
|
Added the DDL lock timeout configuration. (Added in 8.1.3.200) |
Added the GUC parameter ddl_lock_timeout, which specifies the DDL lock timeout interval. |
|
Enhanced the view PG_LOCKS. (Added in 8.1.3.200) |
|
|
Data masking compatibility configuration (Added in 8.1.3.310) |
The GUC parameter redact_compat_options is added to configure the compatibility of the data masking techniques. |
|
Concurrent truncate, exchange, and select operations (added in 8.1.3.320) |
Added the GUC parameter ddl_select_concurrent_mode. This parameter is used to solve the problem that the query statement lasts for a long time and blocks DDL in the scenario where the data volume is large or the query is complex. The effect is the same as that of the Oracle database. Application scenarios:
Scenario constraints:
Connection constraints:
|
|
Enabling of the early stop optimization for Limit statements (added in 8.1.3.320) |
Added the GUC parameter enable_limit_stop to specify whether to enable the early stop optimization for LIMIT statements. |
|
Early stop behavior change (Added in 8.1.3.322) |
The default value of the GUC parameter enable_limit_stop is changed to on. |
SQL Syntax
Change Type |
No. |
Name |
Change Description |
---|---|---|---|
Added |
1 |
CREATE FOREIGN TABLE (SQL on Hadoop or OBS) |
Added json for the parameter Format of OPTION. |
2 |
CREATE FOREIGN TABLE (SQL on Hadoop or OBS) |
Added force_mapping for OPTION. |
|
3 |
Column type kvtype |
Specifies the column type of time series tables in the CREATE TABLE and ALTER COLUMN syntax. |
|
4 |
Table-level parameter: sub_partition_count |
Configures the number of level-2 partitions in a time sequence table. |
|
5 |
Table-level parameter ttl |
Specifies the expiration time of auto-added partitions. |
|
6 |
Table-level parameter period |
Specifies the interval for triggering partition adding and dropping tasks and the time range of a single partition. |
|
7 |
List partitioning support |
The list partition operation syntax is added, including the CREATE and ALTER operations. The CREATE syntax is a separate syntax branch. In the AT syntax, except for the SPLIT syntax, other syntax are similar to those of the range partitioning. |
|
8 |
ALTER TABLE ADD INDEX/DROP INDEX |
The ALTER TABLE syntax branch is added to add and delete indexes. |
|
9 |
CREATE TABLE ... LIKE |
The CREATE TABLE... LIKE syntax without brackets is added. |
|
10 |
ALTER TABLE ... ADD/MODIFY COLUMN |
The statements for adding and modifying columns are added to ALTER TABLE. COMMENT and DEFAULT can be set. |
|
11 |
Added the [NOT] MATERIALIZED syntax after with cte as. |
Added the [NOT] MATERIALIZED syntax after with cte as. |
|
12 |
alter index ... comment |
The COMMENT clause is now available in the ALTER INDEX syntax. |
|
13 |
ALTER TABLE ... add index comment |
The COMMENT clause is now available in the alter table add index syntax. |
|
14 |
create index index_name on table_name comment |
The COMMENT clause is now available in the CREATE INDEX syntax. |
|
15 |
copy to |
Added server, bom, fileprefix, and maxrow to the OPTION of the COPY TO statement. |
|
16 |
CREATE FOREIGN TABLE (OBS import and export) |
Added bom to the OPTION of the CREATE FOREIGN TABLE statement (OBS import and export). |
|
17 |
Operator = |
If two Roaring Bitmaps are equal, true is returned. Otherwise, false is returned. |
|
18 |
Operator <> |
If two Roaring Bitmaps are not equal, true is returned. Otherwise, false is returned. |
|
19 |
Operator & |
Calculates the intersection of two Roaring Bitmaps. |
|
20 |
Operator | |
Calculates the union of two Roaring Bitmaps. |
|
21 |
Operator | |
Calculates the result of adding an ID to a Roaring Bitmap. |
|
22 |
Operator # |
Calculates the XOR result of two Roaring Bitmaps. |
|
23 |
Operator - |
Obtains the set of elements that are in a Roaring Bitmap but not in another Roaring Bitmap. |
|
24 |
Operator - |
Removes a specified ID from a Roaring Bitmap. |
|
25 |
Operator @> |
If the Roaring Bitmap before the operator contains the elements after the operator, true is returned. Otherwise, false is returned. |
|
26 |
Operator <@ |
If the Roaring Bitmap after the operator contains the elements before the operator, true is returned. Otherwise, false is returned. |
|
27 |
Operator && |
If two Roaring Bitmaps overlap, true is returned. Otherwise, false is returned. |
|
28 |
ALTER TABLE DROP KEY |
Added the syntax for deleting indexes. The usage of this syntax is the same as that of ALTER TABLE DROP INDEX. |
|
29 |
ALTER TABLE CHANGE |
Compatible with the CHANGE syntax in MySQL. Column names and column attributes can be modified at the same time. |
|
30 |
ALTER TABLE ... FIRST/AFTER colname |
The syntax layer supports the FIRST/AFTER syntax, but does not implement the actual semantics. It is applicable to the ALTER TABLE ADD/MODIFY/CHANGE COLUMN scenario and is controlled by the parameter skip_first_after_mysql. |
|
31 |
AGG HINT |
Added Agg hints for performance optimization. |
|
32 |
ALTER TABLE ... ALTER COLUMN cstore_cu_sample_ratio |
To reduce the work of CU decompression, samples are concentrated in randomly selected CUs. Therefore, it is difficult to obtain data features. This field attribute is used to select more CUs without increasing the sampling ratio to facilitate data feature obtaining. |
|
Modified |
33 |
Table-level parameter deltarow_threshold |
The default value of deltarow_threshold is set to 10000, which specifies the upper limit of rows when column-store tables are imported to the Delta table. This parameter is valid only when the table-level parameter enable_delta is enabled. |
34 |
More data types are supported by Btree indexes. |
The data types supported by the B-tree indexes are extended to support row store tables, column-store tables, and time series tables. btree_gin provides data types int2, int4, int8, float4, float8, timestamp with time zone, timestamp without time zone, time with time zone, time without time zone, date, interval, oid, money, "char", varchar, text, bytea, bit, varbit, macaddr, macaddr8, inet, cidr, uuid, name, bool, bpchar, and all enum types. |
|
35 |
GRANT/REVOKE/ALTER DEFAULT PRIVILEGES |
The ALTER/DROP/VACUUM permissions are added. |
|
36 |
ALTER TABLE ... DROP PARTITION |
The DROP PARTITION syntax can be used to delete multiple partitions. |
|
37 |
DEFAULT clause in the CREATE/ALTER TABLE statement |
The DEFAULT clause in the CREATE/ALTER TABLE statement does not support suffix operators (currently, only the factorial suffix operator "!" is used). For example, create table t (a int default 3!) is not allowed. |
Keywords
Change Type |
No. |
Name |
Change Description |
---|---|---|---|
Added |
1 |
MATERIALIZED |
Added the [NOT] MATERIALIZED syntax after with cte as. As a non-reserved keyword, it can be still used as an object name. When it is used as a column alias, AS must be added. |
2 |
time_fill |
It is used as the keyword of a time filling expression to output the time_fill column. It cannot be used as a function name or user-defined data type name. |
|
3 |
fill_first/fill_last/fill_avg |
It is used as the keyword of a time filling expression to output the filled column. It cannot be used as a function name or user-defined data type name. |
|
4 |
list |
Specifies the type of a partitioned table. As a non-reserved keyword, it can still be used as an object name. When it is used as a column alias, AS must be added. |
|
5 |
tsfield/tstag/tstime |
Specifies the type of a partitioned table. As a non-reserved keyword, it can still be used as an object name. When it is used as a column alias, AS must be added. |
System Catalogs
Change Type |
No. |
Name |
Change Description |
---|---|---|---|
Added |
1 |
rb_added |
A value is added to RoaringBitmap. |
2 |
pg_partition |
Added the boundexprs column to pg_partition. |
|
3 |
pg_relfilenode_size |
New system catalog |
|
4 |
pg_attribute |
The attkvtype column is added to pg_attribute to record the kvtype of columns. |
|
5 |
pg_collation |
The case_insensitive record is added to support case-insensitive behaviors. |
System Functions
Change Type |
No. |
Name |
Change Description |
---|---|---|---|
Added |
1 |
rb_build |
Converts an int array into a bitmap. |
2 |
rb_to_array |
Reverse operation of rb_build. It converts a Roaring Bitmap into an int array. |
|
3 |
rb_and |
Obtains the intersection of two Roaring Bitmaps. |
|
4 |
rb_or |
Obtains the union of two Roaring Bitmaps. |
|
5 |
rb_xor |
Obtains the XOR result of two Roaring Bitmaps. |
|
6 |
rb_andnot |
Performs AND then negation operation on two Roaring Bitmaps. |
|
7 |
rb_cardinality |
Calculates the cardinality of a Roaring Bitmap. |
|
8 |
rb_and_cardinality |
Calculates the cardinality of the And result of two Roaring Bitmaps. |
|
9 |
rb_or_cardinality |
Calculates the cardinality of the OR result of two Roaring Bitmaps. |
|
10 |
rb_xor_cardinality |
Calculates the cardinality of the XOR result of two Roaring Bitmaps. |
|
11 |
rb_andnot_cardinality |
Calculates the cardinality of the andNot result of two Roaring Bitmaps. |
|
12 |
rb_is_empty |
Determines whether a Roaring Bitmap is empty. |
|
13 |
rb_equals |
Determines whether two Roaring Bitmaps are equal. |
|
14 |
rb_intersect |
Determines whether two Roaring Bitmaps intersect. |
|
15 |
rb_contain |
Determines whether the first Roaring Bitmap contains a specified value. |
|
16 |
rb_add |
Adds a value to a Roaring Bitmap. |
|
17 |
rb_remove |
Deletes a value from a Roaring Bitmap. |
|
18 |
rb_flip |
Reverses the Roaring Bitmap within a specified range. |
|
19 |
rb_min |
Obtains the minimum value of a Roaring Bitmap. |
|
20 |
rb_max |
Obtains the maximum value of a Roaring Bitmap. |
|
21 |
rb_rank |
Returns the cardinality of values in a bitmap that are less than or equal to the specified Offset. |
|
22 |
rb_contain_rb |
Check whether the first Roaring Bitmap contains the second Roaring Bitmap. |
|
23 |
rb_containedby_rb |
Check whether the second Roaring Bitmap contains the first Roaring Bitmap. |
|
24 |
rb_containedby |
Determines whether a specified value is included in a specified Roaring Bitmap. |
|
25 |
rb_iterate |
Returns the ints corresponding to a Roaring Bitmap. |
|
26 |
rb_and_agg |
Aggregates a Roaring Bitmap column based on the AND logic. |
|
27 |
rb_or_agg |
Aggregates a Roaring Bitmap column based on the OR logic. |
|
28 |
rb_xor_agg |
Aggregates a Roaring Bitmap column based on the XOR logic. |
|
29 |
rb_and_cardinality_agg |
Obtains the cardinality of the AND result of a Roaring Bitmap column. |
|
30 |
rb_or_cardinality_agg |
Obtains the cardinality of the OR result of a Roaring Bitmap column. |
|
31 |
rb_xor_cardinality_agg |
Obtains the cardinality of the XOR result of a Roaring Bitmap column. |
|
32 |
rb_build_agg |
Aggregates the int column into RoaringBitmap data. |
|
33 |
pgxc_wlm_readjust_relfilenode_size_table() |
Space statistics calibration function. It does not recreate the PG_RELFILENODE_SIZE system catalog but re-calibrates the user and schema space. |
|
34 |
gs_table_distribution() |
Quickly queries a table size. |
|
35 |
pg_obs_cold_refresh_time |
Modifies the time for automatic switchover of OBS multi-temperature tables. |
|
36 |
gs_clean_tag_relation |
Deletes the useless data in the tagid row of the tag table. The input parameter of the function is the OID of the time series table. Each partition traverses the minimum value of the tagid column in the cudesc table to obtain the minimum tagid value in the entire time series table. |
|
37 |
proc_drop_partition |
Drops a partition whose boundary time exceeds the TTL. |
|
38 |
proc_add_partition |
Creates partitions for a partitioned table. |
|
39 |
pg_collation_actual_version |
Returns the actual version number of an ICU collation. |
|
40 |
first |
Indicates the first element in a group. |
|
41 |
last |
Indicates the last element in a group. |
|
42 |
mode |
Indicates the value with the highest occurrence frequency in a group. |
|
43 |
delta |
Indicates the difference between two adjacent rows. |
|
44 |
percentile_of_value |
Indicates the approximate percentile value. |
|
45 |
value_of_percentile |
Indicates the approximate percentile. |
|
46 |
spread |
Indicates the difference between the maximum value and minimum value in a group. |
|
47 |
pg_flush_buffers |
Flushes all row-store dirty pages. |
|
Modified |
48 |
pg_stat_activity series views |
The stmt_type and lwtid columns are added to the pg_stat_get_activity_with_conninfo, pg_stat_activity, pgxc_stat_activity and pg_stat_get_activity views. |
49 |
Added two rows to the pg_authid system catalog. |
Added the preset roles pg_role_signal_backend and pg_role_read_all_stats. |
|
50 |
vac_fileclear_relation |
Returns the number of files to be cleared in a specified table. If no files are to be cleared, 0 is returned. |
|
51 |
vac_fileclear_all_relation |
Returns the number of files to be cleared in all column store tables. If no files are to be cleared, 0 is returned. |
System Views
Change Type |
No. |
Name |
Change Description |
---|---|---|---|
Added |
1 |
pgxc_session_wlmstat |
New view |
2 |
pg_comm_query_speed |
New view. It is used to query sending information based on query_ids. |
|
3 |
pgxc_respool_resource_info pgxc_respool_runtime_info pgxc_respool_resource_history gs_respool_resource_info gs_respool_runtime_info |
New views related to resource monitoring. |
|
4 |
pgxc_wait_detail pgxc_lockwait_detail |
Added the enhanced view pg_locks. |
|
Modified |
5 |
DBA_PART_INDEXES DBA_PART_TABLES DBA_TAB_PARTITIONS USER_IND_PARTITIONS USER_PART_INDEXES USER_TAB_PARTITIONS |
Added description for list partitions. |
6 |
pgxc_wlm_session_statistics |
The logic for querying real-time information about Top SQL statements from all CNs is changed from serial to parallel to improve performance. The function remains unchanged. |
|
7 |
all_indexes |
The definition of the view all_indexes in sys and pg_catalog is incorrect. If objects with the same name exist in different schemas, the result set expands. |
Behavior Changes
Change Type |
No. |
Name |
Change Description |
---|---|---|---|
Modified |
1 |
In CREATE INDEX, the target table is a time series table. |
Any index created in the time series table is converted to a double index in the tag table. The index column of the double index is the specified column that the index is created for. |
2 |
Secondary load management |
Added secondary management to load management. Functions and multi-statements that contain complex queries may trigger multiple controls. You can set enable_transaction_parctl to off to disable secondary management. If you do so, transaction block statements and multi-statement management will also be disabled. |
|
3 |
Load management (autoanalyze management) |
The control logic for queries triggering autoanalyze is changed from no control to control. You can set enable_transaction_parctl=off to disable the autoanalyze control. |
|
4 |
User monitoring view pg_total_user_resource_info |
|
|
5 |
Audit logs |
1. Statements in a transaction are audited when the statement type is set to be audited, even if the transaction is not set to be audited. 2. The DECLARE CURSOR statement is audited when the guc parameter audit_operation_exec is set to select. |
GUC Parameters
Change Type |
No. |
Name |
Change Description |
---|---|---|---|
Added |
1 |
space_readjust_schedule |
Indicates whether to enable automatic calibration for users and schema spaces. The options are off, auto, and auto (xxx K/M/G). The default value is auto. |
2 |
space_once_adjust_num |
Threshold of the number of files processed each time during slow build and fine-grained calibration in the space management and statistics functions. The default value is 300. |
|
3 |
tag_cache_max_number |
Indicates the maximum number of items cached in the tag cache of a global hash table. The default value is 10000000. |
|
4 |
enable_tagbucket_auto_adapt |
Indicates whether to enable tagbucket adaptive adjustment. The default value is on. |
|
5 |
cache_tag_value_num |
Number of cached tag tuples during tag column lateread. The default value is 60000. |
|
6 |
buffer_ring_ratio |
Controls the threshold and switch of the ring buffer in a row-store table. The default value is 250, that is, 1/4 (250/1000), which is the same as the previous logic. |
|
7 |
enable_stream_ctescan |
This parameter is added. By default, this parameter is set to OFF after an upgrade and is set to ON after a new installation. Specifies whether to enable the share scan function for the stream plan. |
|
8 |
behavior_compat_options |
The disable_including_all_mysql option is added to control whether the CREATE TABLE LIKE syntax is in INCLUDING ALL mode. By default, this option is not set and the CREATE TABLE LIKE syntax is in INCLUDING ALL mode. |
|
9 |
profile_logging_module |
Added the GUC parameter profile_logging_module to configure the type of performance logs. By default, OBS, Hadoop, and REMOTE_DATANODE are enabled and MD is disabled after upgrade or installation. The method of setting this parameter is the same as that of setting logging_module. |
|
10 |
object_mtime_record_mode |
Controls the behaviors of pg_object recording modification time. By default, the behavior of the old version is retained. The new option can be set to not record the TRUNCATE/GRANT/REVOKE operations on partitions. |
|
11 |
skew_option |
Specifies whether to enable an optimization policy. |
|
12 |
ddl_lock_timeout |
DDL lock timeout interval. |
|
Modified |
13 |
rewrite_rule |
|
14 |
default_orientation |
The userset guc parameter default_orientation is added to control the default type of tables users can create (orientation is not set). The options are as follows: Row-store table (row), column-store table (column), and column-store table with delta enabled (column enabledelta) |
|
15 |
max_process_memory |
It is an OM setting item configured during initial installation. The original formula is: Available memory x 0.7/(1 + Number of DNs). The coefficient 0.7 is changed to 0.8. It remains unchanged in scenarios such as upgrade and scale-out of the old cluster. |
|
16 |
enable_bloom_filter |
Reused parameter. It is now used to control BloomFilter in a column-store table. The default value remains unchanged (Enabled). |
|
17 |
retry_ecode_list |
Added the default value 45003 to solve the problem that wrong partitions may be queried when partitions are added and queried concurrently in a list partitioned table. |
|
18 |
auth_iteration_count |
The default value is reduced from 50000 to 10000. The change applies to both installation and upgrade. Indicates the number of hash iterations of the passwords of the client and server. |
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.