Updated on 2024-05-29 GMT+08:00

Overview

This section describes how you can migrate data to DLI in an efficient way. You can use Cloud Data Migration Service (CDM) to migrate data from other cloud services or platforms to DLI.

DLI is a serverless data processing and analysis service. It processes streaming data and batch data and supports interactive analysis. Its high-scalability framework supports the convergence of batch and streaming data analysis, and provides real-time, efficient, and diversified compute resources for TB-to EB-level data processing.

Best Practices of Data Migration

Data Type Mapping

If you migrate data from other cloud services or platforms to DLI, data types need to be converted and source and destination data must be mapped by type. Table 1 lists the mapping relationships.

Table 1 Data type mapping

MySQL

Hive

GaussDB(DWS)

Oracle

PostgreSQL

Hologres

DLI Spark

CHAR

CHAR

CHAR

CHAR

CHAR

CHAR

CHAR

VARCHAR

VARCHAR

VARCHAR

VARCHAR

VARCHAR

VARCHAR

VARCHAR/STRING

DECIMAL

DECIMAL

NUMERIC

NUMERIC

NUMERIC

DECIMAL

DECIMAL

INT

INT

INTEGER

NUMBER

INTEGER

INTEGER

INT

BIGINT

BIGINT

BIGINT

NUMBER

BIGINT

BIGINT

BIGINT/LONG

TINYINT

TINYINT

SMALLINT

NUMBER

SMALLINT

SMALLINT

TINYINT

SMALLINT

SMALLINT

SMALLINT

NUMBER

SMALLINT

SMALLINT

SMALLINT/SHORT

BINARY

BINARY

BYTEA

RAW

BYTEA

BYTEA

BINARY

VARBINARY

BINARY

BYTEA

RAW

BYTEA

BYTEA

BINARY

FLOAT

FLOAT

FLOAT4

FLOAT

DOUBLE

FLOAT4

FLOAT

DOUBLE

DOUBLE

FLOAT8

FLOAT

REAL/DOUBLE

FLOAT8

DOUBLE

DATE

DATE

TIMESTAMP

DATE

DATE

DATE

DATE

TIME

Not supported (use String instead)

TIME

DATE

TIME

TIME

Not supported (use String instead)

DATETIME

TIMESTAMP

TIMESTAMP

TIME

TIME

TIMESTAMP

TIMESTAMP

TINYINT

TINYINT

BOOLEAN

Not supported

TINYINT

BOOLEAN

BOOLEAN

Not supported (use TEXT instead)

Not supported (use String instead)

Not supported (use TEXT instead)

Not supported (use VARCHAR instead)

Not supported (use TEXT instead)

Not supported (use TEXT instead)

ARRAY

Not supported (use TEXT instead)

Not supported (use String instead)

Not supported (use TEXT instead)

Not supported (use VARCHAR instead)

Not supported (use TEXT instead)

Not supported (use TEXT instead)

MAP

Not supported (use TEXT instead)

Not supported (use String instead)

Not supported (use TEXT instead)

Not supported (use VARCHAR instead)

Not supported (use TEXT instead)

Not supported (use TEXT instead)

STRUCT

If a service does not support a standard data type, you can use the recommended data type.