Dependency

Overview

dws-connector-flink is a tool developed by the DWS team for interconnecting with Flink based on dws-client. The tool is a wrapper of dws-client and will be continuously optimized based on the DWS database. The overall data import capability is the same as that of dws-client.

dws connector of dws-flink-connector supports only single concurrent query of inventory data and does not support concurrent read.

How to Use

dws-connector-flink is released through the third-party Maven repository. You can select the latest version. For details, visit https://mvnrepository.com/artifact/com.huaweicloud.dws.

In version 1.x, SQL and API are not differentiated. The dependency package is released uniformly, following the naming convention: dws-connector-flink_${scala.version}${flink.version}.

From version 2.x onwards, there are versions with and without dependencies. The dependency-free package, named dws-connector-flink, is suitable for stream API scenarios due to the absence of incompatible API code for the Flink version. Different environments are identified by version numbers (${flink.version}${scala.version}${version}). Download link: https://mvnrepository.com/artifact/com.huaweicloud.dws/dws-connector-flink.

The version with dependencies is named dws-connector-flink-sql-${flink.version}, following the rule: ${scala.version}${version}.

• Submit a job in JAR package mode and import the job in Maven mode.
  1.x version

  <dependency>
      <groupId>com.huaweicloud.dws</groupId>
      <artifactId>dws-connector-flink_${scala.version}_${flink.version}</artifactId>
      <version>${version}</version>
  </dependency>

  2.x Flink SQL scenario

  <dependency>
      <groupId>com.huaweicloud.dws</groupId>
      <artifactId>dws-connector-flink-sql-${flink.version}</artifactId>
      <version>${scala.version}_${version}</version>
  </dependency>

  2.x Stream API scenario

  <dependency>
      <groupId>com.huaweicloud.dws</groupId>
      <artifactId>dws-connector-flink</artifactId>
      <version>${flink.version}_${scala.version}_${version}</version>
  </dependency>

• Use the SQL client to import non-JAR packages or use Flink SQL to import non-JAR packages on a third-party platform.

  When using Flink SQL to implement dws-connector-flink, you need to place the dws-connector-flink package and its dependencies in the Flink class loading directory. In 1.0.3 and later versions, the packages with dependencies have been released to the Maven repository. You can download the packages from the repository.

  1. Select the package that matches the Flink environment.
     Figure 1 Flink package
     
  2. Switch to the software package details page.
     Figure 2 Flink package details page
     
  3. Select the required version.
     Figure 3 Selecting the version to be downloaded
     
  4. Click to download files.