Help Center> MapReduce Service> Developer Guide (Normal_Earlier Than 3.x)> Spark Application Development> FAQs> How Do I Add a Dependency Package with Customized Codes?
Updated on 2022-06-01 GMT+08:00

How Do I Add a Dependency Package with Customized Codes?

Question

When developing Spark applications, I can add customized dependency packages except for the sample application. How do I use the IDEA to add the dependency package with customized codes to the project?

Answer

  1. On the IDEA homepage, choose File > Project Structures... to go to the Project Structure page.
  2. Click the Libraries tab. On the page that is displayed, click + to add a local dependency package.

    Figure 1 Adding the dependency package.

  3. Click Apply to load the dependency package and click OK.
  4. Because there is no custom dependency package in the running environment, you also need to add this dependency package when compiling the package, so that the generated JAR file contains the custom dependency package, ensuring that the Spark application can run properly.

    1. On the Project Structure page, click the Artifacts tab.
    2. In the pane on the right, click + and choose Library Files to add a dependency package.
      Figure 2 Adding library files
    3. Select the dependency package that you want to add and click OK.
      Figure 3 Choosing the library
    4. Click Apply to load the dependency package and click OK.