Dependency

Overview

dws-connector-flink is a tool developed by the DWS team for interconnecting with Flink based on dws-client. The tool is a wrapper of dws-client and will be continuously optimized based on the DWS database. The overall data import capability is the same as that of dws-client.

DWS connector of dws-flink-connector supports only single concurrent query of inventory data and does not support concurrent read.

Addresses for Downloading Dependency Packages

dws-connector-flink is released through the third-party Maven repository. You can select the latest version.

• The 1.x version does not distinguish SQL from API. The dependency packages are released in a unified manner. You can visit Download Address to obtain the packages. Figure 1 shows the 1.x version. Select the corresponding Scala and Flink versions.

  Software package naming rule: dws-connector-flink_${scala.version}_${flink.version}.

  Figure 1 1.x version
  

• From version 2.x, there are two versions: with dependency packages and without dependency packages.
  • The version without dependency packages can be used only in the Stream API scenario because the software package does not contain the API code that is incompatible with the Flink version.

    The software package is named dws-connector-flink and different environments are distinguished by version number. You can obtain the software package from the download address, as shown in Figure 2. Select the software package based on the Scala and Flink versions.

    Software package naming rule: ${flink.version}_${scala.version}_${version}

    Figure 2 2.x version without dependency packages
    
  • Version with dependency packages supports Flink SQL and is compatible with Stream API scenarios. Visit Download Address to obtain the package, as shown in Figure 3. Select the package based on the Flink version, and then select the final package based on the Scala version, as shown in Figure 4.

    Software package naming rule: dws-connector-flink-sql-${flink.version}, and the inner version rule is ${scala.version}_${version}.

    Figure 3 2.x version without dependency packages
    
    Figure 4 Selecting the corresponding Scala version
    

Submitting a Job Using a JAR Package and Importing the Job Using Maven

<dependency>
    <groupId>com.huaweicloud.dws</groupId>
    <artifactId>dws-connector-flink_${scala.version}_${flink.version}</artifactId>
    <version>${version}</version>
</dependency>

<dependency>
    <groupId>com.huaweicloud.dws</groupId>
    <artifactId>dws-connector-flink-sql-${flink.version}</artifactId>
    <version>${scala.version}_${version}</version>
</dependency>

2.x Stream API scenario

<dependency>
    <groupId>com.huaweicloud.dws</groupId>
    <artifactId>dws-connector-flink</artifactId>
    <version>${flink.version}_${scala.version}_${version}</version>
</dependency>

Importing Non-JAR Packages Using the Flink SQL by a Third-Party Platform or the SQL Client

When using Flink SQL to implement dws-connector-flink, you need to place the dws-connector-flink package and its dependencies in the Flink class loading directory. In 1.0.3 and later versions, the packages with dependencies have been released to the Maven repository. You can download the packages from the repository.

1. Select the package that matches the Flink environment.
   Figure 5 Flink package
   
2. Switch to the software package details page.
   Figure 6 Flink package details page
   
3. Select the required version.
   Figure 7 Selecting the version to be downloaded
   
4. Click to download files.