Updated on 2023-08-31 GMT+08:00

Preparing Development and Operating Environment

Preparing the Development Environment

Table 1 describes the environments required for application development.

Table 1 Environment

Preparation Item

Description

OS

  • Development environment: Windows OS
  • Operating environment: Linux system

    If the program needs to be commissioned locally, the operating environment must be able to communicate with the cluster service plane.

JDK

Basic configuration for the Java/Scala development and running environment. The version requirements are as follows:

If you are using the MRS server and client, only built-in OpenJDK 1.8.0_272 is supported. Other JDKs are not allowed.

If the JAR packages of the SDK classes that need to be referenced by the customer applications run in the application process, the JDK requirements are as follows:

  • x86 client: Oracle JDK 1.8; IBM JDK 1.8.5.11.
  • TaiShan client: OpenJDK 1.8.0_272

IDEA

Used for developing Flink applications. The required version is 14.1.7.

Scala installation

It is the basic configuration for the Scala development environment. The required version is 2.11.7.

Scala plug-in

Basic configuration for the Scala development environment. The required version is 1.5.4.

Maven installation

Basic configuration of the development environment. It can be used for project management throughout the lifecycle of software development.

7-zip

Tool used to decompress *.zip and *.rar files. 7-Zip 16.04 is supported.

Preparing the Operating Environment

During application development, prepare the code running and commissioning environment to verify that the application is running properly.

  • If you use the Linux environment for commissioning, you need to prepare the Linux node where the cluster client is to be installed and obtain related configuration files.
    1. Install the client in a directory, for example, /opt/client, on the node.

      The difference between the client time and the cluster time must be less than 5 minutes.

      For details about how to use a client on a master or core node in a cluster, see Installing a Client. For details about how to use a client on a node outside a cluster, see Installing a Client.

    2. Log in to FusionInsight Manager. Download the cluster client software package to the active management node and decompress it. Then, log in to the active management node as user root. Go to the decompression path of the cluster client and copy all configuration files in the FusionInsight_Cluster_1_Services_ClientConfig\Flink\config directory to the conf directory where the compiled JAR file is stored for subsequent commissioning, for example, /opt/client/conf.

      For example, if the client software package is FusionInsight_Cluster_1_Services_Client.tar and the download path is /tmp/FusionInsight-Client on the active OMS node, run the following commands:

      cd /tmp/FusionInsight-Client

      tar -xvf FusionInsight_Cluster_1_Services_Client.tar

      tar -xvf FusionInsight_Cluster_1_Services_ClientConfig.tar

      cd FusionInsight_Cluster_1_Services_ClientConfig

      scp Flink/config/* root@IP address of the client node:/opt/client/conf

      Table 2 describes the main configuration files.

      Table 2 Configuration files

      File

      Description

      core-site.xml

      Configures Flink parameters.

      hdfs-site.xml

      Configures HDFS parameters.

      yarn-site.xml

      Configures YARN parameters.

      flink-conf.yaml

      Flink client configuration file.

    3. Check the network connection of the client node.

      During the client installation, the system automatically configures the hosts file on the client node. You are advised to check whether the /etc/hosts file contains the host names of the nodes in the cluster. If there is no required information, copy the content of the hosts file in the decompression directory to the hosts file on the node where the client is deployed, to ensure that the local host can communicate with each host in the cluster.