An Error Is Reported When a Flink Job Is Submitted in a Cluster with Kerberos Authentication Enabled

Issue

An error is reported when a user submits a Flink job with Kerberos authentication enabled.

Symptom

When a user submits the official Flink case ./flink run /opt/client/Flink/flink/examples/streaming/WordCount.jar, the error message "unable to establish the security context" is displayed.

Cause Analysis

1. Kerberos authentication is enabled but jobs cannot be submitted. Therefore, the permission configuration is checked first. It is found that the parameters in /opt/client/Flink/flink/conf/flink-conf.yaml are not correctly configured.
   Figure 1 flink-conf.yaml configuration
   
2. After the configuration is modified and refreshed, the job can be submitted again, but the "log4j:ERROR setFile(null,true) call failed" error is reported.
   Figure 2 log4j error
   
3. Check on log4j indicates that the name of the log4j.properties file is changed to log4g-cli.properties (the name of log4j.properties is fixed and cannot be changed). Therefore, an error is reported.
   Figure 3 Viewing the log4j file
   
4. After the modification, the job can be submitted properly.
   Figure 4 Job submission
   

Procedure

1. Determine the place to submit jobs using the client, outside the cluster or in the cluster.
   1. If you want to submit jobs using the client in the cluster, switch to user omm before submitting jobs.
   2. If you want to submit jobs using the client outside the cluster, switch to user root before submitting jobs.

2. Check whether parameters in flink-conf.yaml are correctly set.
3. For clusters with Kerberos authentication enabled, the configuration items include keytab and principal of Kerberos.
   • Download the user keytab file from the KDC server, and place the keytab file to a folder on the host of the Flink client (for example, /home/flinkuser/keytab).
   • Configure the following parameters in the ${FLINK_HOME}/conf/flink-conf.yaml file:
     1. Keytab path (note that there is a space before the parameter):

        security.kerberos.login.keytab:  /home/flinkuser/keytab/uer.keytab

     2. Principal name (developer username):

        security.kerberos.login.principal:flinkuser

4. Submit the job ./flink run /opt//client/Flink/flink/examples/streaming/WordCount.jar again and check whether the job can be submitted.
   • If the job can be submitted, the permission authentication is correct. In this case, you can check other errors. In this example, the name of log4j.properties is changed. After the name is restored, the job can be submitted normally.
   • If the job fails to be submitted, submit a service ticket.

Related Information

For details about how to use Flink, see Using Flink from Scratch.