Help Center/ Data Ingestion Service/ User Guide/ Using DIS/ Using SparkStreaming SDK to Download Data/ Preparing a DIS Spark Streaming Environment
Updated on 2023-06-21 GMT+08:00
View PDF
Share

Preparing a DIS Spark Streaming Environment

Preparing a DIS Application Development Environment

  1. Set up DIS application development environments by following the procedure in Step 1: Creating a DIS Stream.
  2. Install Maven. Configure a local repository address.
  3. Install scala-sdk.

Configuring the DIS Spark Streaming Dependency

The DIS Spark Streaming dependency can be introduced to the project through the following configuration:

<dependency>
    <groupId>com.cloud.dis</groupId>
    <artifactId>cloud-dis-spark-streaming_2.11</artifactId>
    <version>1.2.1</version>
    <scope>compile</scope>
</dependency>

Checking Authentication Information

  • AK/SK file

    Access Key ID/Secret Access Key (AK/SK) files are created by the Identity and Access Management (IAM) service to authenticate calls to application programming interfaces (APIs) on the cloud. To obtain AK/SK, choose My Credentials > Access Keys.

  • Project ID

    A project is a group of tenant resources. A tenant can have multiple projects, one for each region. Each region has a unique Project ID. To view the project IDs of different regions, choose .