Help Center/ DataArts Studio/ User Guide/ DataArts Migration (Real-Time Jobs)/ Job Performance Optimization/ Optimizing the Parameters of a Job for Migrating Data from PostgreSQL to GaussDB(DWS)
Updated on 2025-08-05 GMT+08:00
View PDF

Optimizing the Parameters of a Job for Migrating Data from PostgreSQL to GaussDB(DWS)

Optimizing Source Parameters

Optimization of data extraction from PostgreSQL

No optimization configuration items are available.

Optimizing Destination Parameters

Optimization of data writing to GaussDB(DWS)

You can modify writing parameters in the GaussDB(DWS) destination configuration or click View and Edit in the advanced configuration to add advanced attributes.

Figure 1 Adding advanced attributes
Table 1 Parameters for optimizing GaussDB(DWS) writing

Parameter

Type

Default Value

Description

Write Mode

enum

UPSERT

Mode for writing data to GaussDB(DWS), which can be set in the destination configuration. COPY MODE is recommended for real-time migration jobs.

  • UPSERT: batch update
  • COPY: GaussDB(DWS)-dedicated high-performance batch import

Maximum Data Volume for Batch Write

int

50000

Maximum number of data records that can be written to GaussDB(DWS) at a time. You can set this parameter in the destination configuration.

If Maximum Data Volume for Batch Write or Scheduled Batch Write Interval is met, data will be written.

Increasing the number of data records written at a time can reduce the number of DWS requests but may increase the duration of a single request and the amount of cached data, which affects memory usage. Adjust the value based on the GaussDB(DWS) specifications and load.

Scheduled Batch Write Interval

int

3

Interval for writing data to GaussDB(DWS). You can set this parameter in the destination configuration.

If the interval is reached, cached data will be written.

Increasing the value of this parameter increases the number of data records cached in a single write, but it takes a longer time for DWS data to become visible.

sink.buffer-flush.max-size

int

512

Amount of the data that can be written to GaussDB(DWS) at a time. The default value is 512 MB. You can set this parameter in the advanced settings of the destination configuration.

If the size of cached data reaches the upper limit, data will be written.

Similar to Maximum Data Volume for Batch Write, increasing the amount of data written at a time can reduce the number of DWS requests but may increase the duration of a single request and the amount of cached data, which affects memory usage. Adjust the value based on the DWS specifications and load.