Scenario

This section describes how to configure a Flink SQL job in the default application.

This section demonstrates how to create a data connection with Authentication Mode set to KERBEROS, a stream table with Type set to Source, a data connection with Authentication Mode set to SIMPLE, and a stream table with Type set to Sink.

Prerequisites

• You have created a user with FlinkServer Admin Privilege for accessing the Flink web UI. For details, see Authentication Based on Users and Roles.
• You have obtained the client configuration file and credential of the user who accesses the Flink web UI.
  • To obtain the configuration file of the cluster client, log in to FusionInsight Manager, choose Cluster > Dashboard > More > Download Client > Configuration Files Only, select a platform type, and click OK.
  • To obtain the user credential, log in to FusionInsight Manager and click System. In the Operation column of the user, choose More > Download Authentication Credential, select a cluster, and click OK.
• Create an HDFS directory and upload the test.txt test file.
  • Prepare the test.txt test file. The file content is in the following format:

    id1,user_name1
    id2,user_name2
    id3,user_name3

  • Create an HDFS directory, for example, /zwtext, and upload the text.txt file.

    Log in to FusionInsight Manager and choose Cluster > Services > HDFS. On the Basic Information page, click the link next to NameNode WebUI to go to the HDFS web UI page. Select Utilities > Browse the file system and click . In the displayed dialog box, enter zwtext and click Create. Click the name of the created directory to go to the directory, click , select the text.txt file, and click Upload to upload the file.

Procedure

1. Access the Flink web UI. For details, see Accessing the Flink Web UI.
2. Choose System Management > Cluster Connection Management. The Cluster Connection Management page is displayed.
3. Click Create Cluster Connection, create a cluster connection by referring to Figure 1, and click OK.
   Figure 1 Creating a cluster connection
   

4. Choose System Management > Data Connection Management and click Create Data Connection. On the displayed page, select a data connection type, create an HDFS data connection by referring to Figure 2, and click OK.
   Figure 2 Creating an HDFS connection
   

5. Choose Table Management > Create Stream Table. On the Edit Stream Table page, create a sink stream table by referring to Figure 3, create a source stream table by referring to Figure 4, and click OK.
   Figure 3 Creating a sink stream table
   
   Figure 4 Creating a source stream table
   

6. Choose Job Management > Create Job. On the displayed page, create a Flink SQL job by referring to Figure 5 and click OK. The Flink SQL job development page is displayed.
   Figure 5 Creating a Flink SQL job
   

7. On the Flink SQL job development page, develop SQL statements and set parameters by referring to Figure 6. After the configuration is complete, click Save to save the job, and click Submit to submit the Flink SQL job.
   Figure 6 Flink SQL job development
   
   

   When submitting or running jobs on Flink, the user must have the following permissions:

   • If Ranger authentication is enabled, the current user must belong to the hadoop group or the user has been granted the /flink read and write permissions in Ranger.
   • If Ranger authentication is disabled, the current user must belong to the hadoop group.

8. Return to the job management list. You can view information about the created job, including job name, type, status, kind, and description. In the Operation column, you can start, develop, edit, stop, or delete a job.
   Figure 7 Job management
   

9. Click Start to run the job. After the job is successfully run, choose More > Job Monitoring to view the running result, as shown in Figure 8.
   Figure 8 Job details