Help Center/ MapReduce Service/ Developer Guide (Normal_Earlier Than 3.x)/ Spark Development Guide/ Preparing a Spark Application Development Environment/ Spark Application Development Environment
Updated on 2024-08-16 GMT+08:00
View PDF
Share

Spark Application Development Environment

Table 1 describes the environment required for application development. You also need to prepare a Linux environment for verifying whether the application is running properly.

Table 1 Development environment

Item

Description

JDK installation

Basic configurations of the development environment. JDK 1.7 or 1.8 is required.

NOTE:

For security purpose, MRS servers support only TLS 1.1 and TLS 1.2 encryption protocols. IBM JDK supports only TSL 1.0 by default. If you use IBM JDK, set com.ibm.jsse2.overrideDefaultTLS to true. After the parameter setting, TLS1.0/1.1/1.2 can be supported at the same time.

For details, visit https://www.ibm.com/support/knowledgecenter/en/SSYKE2_8.0.0/com.ibm.java.security.component.80.doc/security-component/jsse2Docs/matchsslcontext_tls.html#matchsslcontext_tls.

IDEA installation and configuration

Tool used for developing Spark applications. Version 13.1.6 or later is required.

Scala installation

Basic configuration for the Scala development environment. Version 2.11.0 or later is required.

Scala plugin installation

Basic configuration for the Scala development environment. Version 0.35.683 or later is required.

Python installation

Basic configuration for the Python development environment. Python 2.7.x or later is required.