Help Center/ MapReduce Service/ Component Operation Guide (LTS)/ Using Flink/ Enterprise-Class Enhancements of Flink/ Configuring Flink SQL Client to Support SQL Verification
Updated on 2024-12-13 GMT+08:00
View PDF
Share

Configuring Flink SQL Client to Support SQL Verification

This topic is available for MRS 3.3.0 or later only.

Configuration Method

You can verify SQL syntax during SQL job development on the SQL Client. Running SQL commands in verification mode does not start Flink jobs.

  • Verifying SQL statements
    • When you run the SQL shell command, add -v or --validate to enter the verification mode.

      sql-client.sh -v

    • When you run the SQL shell command, you can run SET to enter or exit the verification mode.
      • Enter the verification mode: SET table.validate = true;
      • Exit the verification mode: SET table.validate = false;
  • Verifying SQL scripts

    When you use the -f parameter to specify a SQL script, add -v to enter the verification mode.

    sql-client.sh -f test.sql -v

Submitting a Job Using the Flink SQL Client

  1. Install the cluster client, for example, in /opt/hadoopclient.
  2. Log in to the node where the client is installed as the client installation user.
  3. Run the following command to go to the client installation directory:

    cd /opt/hadoopclient

  4. Run the following command to initialize environment variables:

    source /opt/hadoopclient/bigdata_env

  5. Log in to the Flink SQL client and submit a job.

    1. Start a yarn-session by referring to Using the Flink Client and record the yarn-session ID (yid).

      yarn-session.sh -nm "session-name"

    2. Run the following command to access the Flink SQL Client:

      cd /opt/hadoopclient/Flink/flink/bin

      ./sql-client.sh

      Figure 1 Accessing the Flink SQL Client
    3. Set high-availability.cluster-id to the yarn-session ID.

      SET high-availability.cluster-id=yarn-session ID;

    1. Run the following SQL statement. If the execution is successful, the following information is displayed on the console.

      SELECT name, COUNT(*) AS cnt FROM ( VALUES ('Bob'), ('Alice'), ('Greg'), ('Bob') ) AS NameTable(name) GROUP BY name;

      Figure 2 Execute result
    2. View the executed job on Yarn.

      Log in to FusionInsight Manager, choose Cluster > Services > Yarn > Dashboard, and click the link next to ResourceManager WebUI to go to the Yarn web UI and view the job.

      Figure 3 Job