Compiling and Running an Application With the Client Installed

Scenario

The Hadoop distributed file system (HDFS) application can run in the Linux operating system (OS) with the HDFS client installed. After the application code has been developed, you can upload the jar packages to the HDFS client and run the application.

Prerequisite

The HDFS client has been installed.
When the host where the Linux OS runs is not a node of the cluster, you are required to set the mapping between the host name and IP address in the hosts file of the node where the client is installed. The host name must be correctly mapped to the IP address.

Procedure

Go to the local root directory of the project, copy the required configuration file to the conf folder of the local project, and run the following command in Windows cmd to compress the package:

mvn -s "{maven_setting_path}" clean package
- In the preceding command, {maven_setting_path} is the path of the settings.xml file of the local Maven.
- After the package is successfully packed, obtain the JAR package, for example, HDFSTest-XXX.jar, from the target subdirectory in the root directory of the project. The name of the JAR package varies according to the actual package.
Upload the exported jarpackages to any directory in the running environment of the client, for example, /opt/client.
Configure the environment variables:

cd /opt/client

source bigdata_env
Run the following commands to execute the jar packages.

hadoop jar HDFSTest-XXX.jar com.huawei.bigdata.hdfs.examples.HdfsExample

hadoop jar HDFSTest-XXX.jar com.huawei.bigdata.hdfs.examples.ColocationExample

When com.huawei.bigdata.hdfs.examples.ColocationExample is run, the HDFSparameter fs.defaultFS cannot be set to viewfs://ClusterX.