Updated on 2024-07-10 GMT+08:00

Constraints

FDI

Table 1 Constraints

Function

Constraint

Data synchronization (covering all data source types)

  • A single data record cannot exceed 10 MB.
  • The fields in time format support down-to-second precision.

FTP data source

When the parsing mode is enabled, each file cannot exceed 200 MB and can contain a maximum of 1,500,000 records. When the parsing mode is disabled, a maximum of 20,000 files are supported, with each file equal to or less than 6 MB.

OBS data source

If the parsing mode is enabled, the file cannot exceed 200 MB. Otherwise, the file cannot exceed 10 MB.

MRS data source

Only MRS clusters authenticated by Kerberos can be connected.

MRS Hive data source

  • Hive supports only RCFile and TEXTFILE read and write.
  • When MRS Hive serves as the source, only tables of up to 1 million records can be synchronized.

API data source

  • If the server does not respond in 60 seconds, an error is reported during task execution.
  • When the API serves as the source, every request can read up to 20 MB data. Otherwise, paging must be enabled.
  • When the API serves as the source, only constant parameters are supported. Dynamic parameter transfer is not supported.
  • When the API serves as the destination, the source data cannot be mapped to the destination headers.

Kafka data source

SASL can be used only to connect ROMA Connect MQS. The Kafka service developed by users themselves supports only plaintext connection.

Oracle database

Only the following field types are supported:

CHAR, VARCHAR, DATE, NUMBER, FLOAT, LONG, NCHAR, NVARCHAR2, RAW, TIMESTAMP

SQL Server database

Only the following field types are supported:

BIT, CHAR, DATE, DATETIME, DECIMAL, FLOAT, IMAGE, INT, MONEY, NUMERIC

MySQL database

Only the following field types are supported:

INT, BIGINT, TINYINT, MEDIUMINT, FLOAT, DOUBLE, DECIMAL, CHAR, VARCHAR, TINYTEXT, TEXT, MEDIUMTEXT, LONGTEXT, DATETIME, TIMESTAMP, SMALLINT, YEAR, BINARY, BITDECIMAL, JSON, DOUBLE

PostgreSQL/DWS database

Only the following field types are supported:

BOOL, CIDR, CIRCLE, DATE, NUMERIC, FLOAT4, FLOAT8, MONEY, PATH, POINT, INT, TIMESTAMP, TIMETZ, UUID, VARBIT, VARCHAR

Relational database

A maximum of 10 million data records can be synchronized.

Composite task (CDC)

  • The source supports only MySQL, SQL Server, Oracle, and GaussDB(for MySQL).
  • The destination supports only MySQL, SQL Server, Oracle, PostgreSQL, Kafka, and GaussDB(for MySQL).
  • The destination table must have a primary key. Otherwise, data synchronization will be affected.