Parallel Data Import
GaussDB(DWS) provides a parallel data import function that enables a large amount of data to be imported in a fast and efficient manner. This section describes parameters for importing data in parallel.
raise_errors_if_no_files
Parameter description: Specifies whether distinguish between the problems "the number of imported file records is empty" and "the imported file does not exist". If this parameter is set to true and the problem "the imported file does not exist" occurs, GaussDB(DWS) will report the error message "file does not exist".
Type: SUSET
Value range: Boolean
- on indicates the messages of "the number of imported file records is empty" and "the imported file does not exist" are distinguished when files are imported.
- off indicates the messages of "the number of imported file records is empty" and "the imported file does not exist" are not distinguished when files are imported.
Default value: off
partition_mem_batch
Parameter description: Specifies the number of buffers. In order to optimize the inserting of column-store partitioned table in batches, the data is buffered during the inserting process and then written in the disk. If the value is too large, much memory will be consumed. If it is too small, the performance of inserting column-store partitioned tables in batches will deteriorate.
Type: USERSET
Value range: 1 to 65535
Default value: 256
partition_max_cache_size
Parameter description: Specifies the size of a data buffer. To optimize batch insertion to column-store partitioned tables, data is buffered during the insertion and then written into disks in batches. If the value is too large, much memory will be consumed. If it is too small, the performance of data insertion to column-store partitioned tables will deteriorate.
Type: USERSET
Value range:
- Column-store partitioned table: 4096 to INT_MAX/2. The unit is KB.
Default value: 2 GB
gds_debug_mod
Parameter description: Specifies whether to enable the debug function of Gauss Data Service (GDS). This parameter is used to better locate and analyze GDS faults. After the debug function is enabled, types of packets received or sent by GDS, peer end of GDS during command interaction, and other interaction information about GDS are written into the logs of corresponding nodes. In this way, state switching on the GaussDB state machine and the current state are recorded. If this function is enabled, additional log I/O resources will be consumed, affecting log performance and validity. You are advised to enable this function only when locating GDS faults.
Type: USERSET
Value range: Boolean
- on indicates that the GDS debug function is enabled.
- off indicates that the GDS debug function is disabled.
Default value: off
Last Article: Asynchronous I/O Operations
Next Article: Write Ahead Logs
Did this article solve your problem?
Thank you for your score!Your feedback would help us improve the website.