Dependency

Overview

The dws-connector-flink tool connects the GaussDB(DWS) client to Flink by encapsulating the GaussDB(DWS) client. Its import capability is equivalent to that of the GaussDB(DWS) client. The GaussDB(DWS) team created dws-connector-flink, and it will be continually improved to align with the GaussDB(DWS) database.

dws connector of dws-flink-connector supports only single concurrent query of inventory data and does not support concurrent read.

How to Use

dws-connector-flink is released through the third-party Maven repository. You can select the latest version. For details, visit https://mvnrepository.com/artifact/com.huaweicloud.dws.

In version 1.x, SQL and API are not differentiated. The dependency package is released uniformly, following the naming convention: dws-connector-flink_${scala.version}${flink.version}.

From version 2.x onwards, there are versions with and without dependencies. The dependency-free package, named dws-connector-flink, is suitable for stream API scenarios due to the absence of incompatible API code for the Flink version. Different environments are identified by version numbers (${flink.version}${scala.version}${version}). Download link: https://mvnrepository.com/artifact/com.huaweicloud.dws/dws-connector-flink.

The version with dependencies is named dws-connector-flink-sql-${flink.version}, following the rule: ${scala.version}${version}.

• Submit a job in JAR package mode and import the job in Maven mode.
  1.x version

  <dependency>
      <groupId>com.huaweicloud.dws</groupId>
      <artifactId>dws-connector-flink_${scala.version}_${flink.version}</artifactId>
      <version>${version}</version>
  </dependency>

  2.x Flink SQL scenario

  <dependency>
      <groupId>com.huaweicloud.dws</groupId>
      <artifactId>dws-connector-flink-sql-${flink.version}</artifactId>
      <version>${scala.version}_${version}</version>
  </dependency>

  2.x Stream API scenario

  <dependency>
      <groupId>com.huaweicloud.dws</groupId>
      <artifactId>dws-connector-flink</artifactId>
      <version>${flink.version}_${scala.version}_${version}</version>
  </dependency>

• Use the SQL client to import non-JAR packages or use Flink SQL to import non-JAR packages on a third-party platform.

  When using Flink SQL to implement dws-connector-flink, you need to place the dws-connector-flink package and its dependencies in the Flink class loading directory. In 1.0.3 and later versions, the packages with dependencies have been released to the Maven repository. You can download the packages from the repository.

  1. Select the package that matches the Flink environment.
     Figure 1 Flink package
     
  2. Switch to the software package details page.
     Figure 2 Flink package details page
     
  3. Select the required version.
     Figure 3 Selecting the version to be downloaded
     
  4. Click to download files.