Common Development Methods for DLI Cross-Source Analysis

Cross-Source Analysis

If DLI needs to access external data sources, you need to establish enhanced datasource connections to enable the network between DLI and the data sources, and then develop different types of jobs to access the data sources. This is the process of DLI cross-source analysis.

This section describes how to develop data sources supported by DLI for cross-source analysis.

Notes

Flink jobs can directly access DIS, OBS, and SMN data sources without using datasource connections.
You are advised to use enhanced datasource connections to connect DLI to data sources.

Cross-Source Analysis Development Methods

Table 1 lists the data sources supported by DLI and the corresponding development methods.

**Table 1** Syntax reference for cross-source analysis
Service	Spark SQL Job	Spark Jar Job	Flink OpenSource SQL Job	Flink Jar Job
CloudTable HBase	Create an HBase association table Insert data Query data	Scala sample code PySpark sample code Java sample code	HBase source table HBase result table HBase dimension table	-
CloudTable OpenTSDB	Create an OpenTSDB association table Insert data Query data	Scala sample code PySpark sample code Java sample code	-	-
CSS	Create a CSS association table Insert data Query data	Scala sample code PySpark sample code Java sample code	Elasticsearch result table	-
DCS Redis	Create a DCS association table Insert data Query data	Scala sample code PySpark sample code Java sample code	Redis source table Redis result table Redis dimension table	Flink job sample
DDS	Create a DDS association table Insert data Query data	Scala sample code PySpark sample code Java sample code	-	-
DMS	-	-	Kafka source table Kafka result table	-
GaussDB(DWS)	Create a GaussDB(DWS) association table Insert data Query data	Scala sample code PySpark sample code Java sample code	GaussDB(DWS) source table GaussDB(DWS) result table GaussDB(DWS) dimension table	Flink job sample
MRS HBase	Create an HBase association table Insert data Query data	Scala sample code PySpark sample code Java sample code	HBase source table HBase result table HBase dimension table	Flink job sample
MRS Kafka	-	-	Kafka source table Kafka result table	Flink job sample
MRS OpenTSDB	Create an OpenTSDB association table Insert data Query data	Scala sample code PySpark sample code Java sample code	-	-
RDS for MySQL	Create an RDS association table Insert data Query data	Scala sample code PySpark sample code Java sample code	MySQL CDC source table	-
RDS PostGre	Create an RDS association table Insert data Query data	Scala sample code PySpark sample code Java sample code	Postgres CDC source table	-