Help Center/ MapReduce Service/ Component Operation Guide (ME-Abu Dhabi Region)/ Using Spark/ Using the Spark Client
Updated on 2024-07-19 GMT+08:00
View PDF
Share

Using the Spark Client

After an MRS cluster is created, you can create and submit jobs on the client. The client can be installed on nodes inside or outside the cluster.

  • Node in the cluster: After an MRS cluster is created, the client has been installed on the master and core nodes in the cluster by default.
  • Node outside the cluster: You can install the client on a node outside the cluster.

Using the Spark Client

  1. Log in to the node where the client is installed.
  2. Run the following command to go to the client installation directory:

    cd /opt/client

  3. Run the following command to configure environment variables:

    source bigdata_env

  4. If the cluster is in security mode, run the following command to authenticate the user. In normal mode, user authentication is not required.

    kinit Component service user

  5. Run the Spark shell command. The following provides an example:

    spark-beeline

Parent topic: Using Spark