Updated on 2024-08-10 GMT+08:00

Flink Client CLI

Common CLIs

Common Flink CLIs are as follows:

  1. yarn-session.sh
    • You can run yarn-session.sh to start a standing Flink cluster to receive tasks submitted by clients. Run the following command to start a Flink cluster with three TaskManager instances:
      bin/yarn-session.sh
    • Run the following command to obtain other parameters of yarn-session.sh:
      bin/yarn-session.sh -help
  2. flink
    • You can run Flink commands to submit a Flink job to a standing Flink cluster or to execute the job in single-server mode.
      • Run the following command to submit a Flink job to a standing Flink cluster:
        bin/flink run ../examples/streaming/WindowJoin.jar

        Before using the command to submit a task, you need to run yarn-session.sh to start the Flink cluster.

      • Run the following command to execute a per-job YARN Cluster mode:
        bin/flink run -m yarn-cluster ../examples/streaming/WindowJoin.jar

        The -m yarn-cluster parameter is used to specify a job to independently start a Flink cluster.

      • List scheduled and running jobs (including their JobIDs):
        bin/flink list
      • Cancel a job:
        bin/flink cancel <jobID>
      • Stop a job (streaming jobs only):
        bin/flink stop <jobID>

        The difference between cancelling and stopping a (streaming) job is the following:

        • Cancel a job: On a cancel call, the operators in a job immediately receive a cancel() method call to cancel them as soon as possible. If operators are not stopping after the cancel call, Flink will start interrupting the thread periodically until it stops.
        • Stop a job: Stop is only available for jobs which use sources that implement the StoppableFunction interface. The job will keep running until all sources properly shut down. Stop a job is more graceful than cancel, but it may cause the job to stop failing.
    • Run the following command to obtain other parameters of Flink commands:
      bin/flink --help

Precautions

  • If yarn-session.sh uses -z to configure the specified ZooKeeper NameSpace, you need to use -yid to specify the application ID and use -yz to specify the ZooKeeper NameSpace when using flink run. The NameSpaces must the same.

    Example:

    bin/yarn-session.sh -z YARN101
    bin/flink run -yid application_****_**** -yz YARN101 examples/streaming/WindowJoin.jar
  • If yarn-session.sh does not use -z to configure the specified ZooKeeper NameSpace, do not use -yz to specify the ZooKeeper NameSpace when using flink run.

    Example:

    bin/yarn-session.sh
    bin/flink run examples/streaming/WindowJoin.jar
  • You can use -yz to specify a ZooKeeper NameSpace when using flink run -m yarn-cluster to start a cluster.
  • A NameSpace cannot be shared by multiple clusters.
  • If you use -z to specify a ZooKeeper NameSpace when starting a cluster or submitting a job, you need to use -z again to specify the NameSpace when deleting, stopping, or querying the job or triggering the savepoint.