Help Center/ MapReduce Service/ Component Operation Guide (Paris Region)/ Using Spark/ Using the Spark Client
Updated on 2022-12-14 GMT+08:00
View PDF
Share

Using the Spark Client

After an MRS cluster is created, you can create and submit jobs on the client. The client can be installed on nodes inside or outside the cluster.

  • Nodes inside the cluster: After an MRS cluster is created, the client has been installed on the master and core nodes in the cluster by default. For details, see . Then, log in to the node where the MRS client is installed..
  • Nodes outside the cluster: You can install the client on nodes outside a cluster. For details about how to install a client, see , and log in to the node where the MRS client is installed..

Using the Spark Client

  1. Based on the client location, log in to the node where the client is installed. For details, see , or .
  2. Run the following command to go to the client installation directory:

    cd /opt/client

  3. Run the following command to configure environment variables:

    source bigdata_env

  4. If the cluster is in security mode, run the following command to authenticate the user. In normal mode, user authentication is not required.

    kinit Component service user

  5. Run the Spark shell command. The following provides an example:

    spark-beeline

Parent topic: Using Spark