Help Center/ MapReduce Service/ Component Operation Guide (Normal)/ Using Spark (for Versions Earlier Than MRS 3.x)/ Using the Spark Client
Updated on 2024-12-11 GMT+08:00
View PDF

Using the Spark Client

After an MRS cluster is created, you can create and submit jobs on the client. The client can be installed on nodes inside or outside the cluster.

  • Node in the cluster: After an MRS cluster is created, the client has been installed on the master and core nodes in the cluster by default. For details about how to log in to a node where the client has been installed, see Installing a Client on a Node Inside a Cluster.
  • Node outside the cluster: You can install the client on a node outside the cluster. For details about how to log in to a node where the client has been installed, see Installing a Client on a Node Outside the Cluster.

Using the Spark Client

  1. Log in to the node where the client is installed.
  2. Run the following command to go to the client installation directory:

    cd /opt/client

  3. Run the following command to configure environment variables:

    source bigdata_env

  4. If the cluster is in security mode, run the following command to authenticate the user. In normal mode, user authentication is not required.

    kinit Component service user

  5. Run the Spark shell command. The following provides an example:

    spark-beeline