An Error Is Reported When a Flink Job Is Submitted in an MRS Cluster with Kerberos Authentication Enabled

Issue

An error is reported when a user submits a Flink job in an MRS cluster with Kerberos authentication enabled.

Symptom

Error unable to establish the security context is reported when a user submits a Flink sample program ./flink run /opt/client/Flink/flink/examples/streaming/WordCount.jar.

Cause Analysis

1. Kerberos authentication is enabled for the MRS cluster but jobs cannot be submitted. Therefore, the permission configuration is checked first. It is found that the parameters in /opt/client/Flink/flink/conf/flink-conf.yaml are not correctly configured.
   Figure 1 flink-conf.yaml configuration
   
2. After the configuration is modified and updated, the job can be submitted again, but the "log4j:ERROR setFile(null,true) call failed" error is reported.
   Figure 2 log4j error
   
3. The log4j file shows that the user has changed the log4j.properties file to log4g-cli.properties (the name of log4j.properties cannot be changed randomly). As a result, an error is reported.
   Figure 3 Viewing the log4j file
   
4. After the modification, the job can be submitted properly.
   Figure 4 Job submission
   

Procedure

1. Determine the place to submit jobs using the client, outside the cluster or in the cluster.
   1. If you want to submit jobs using the client in the cluster, switch to user omm before submitting jobs.
   2. If you want to submit jobs using the client outside the cluster, switch to user root before submitting jobs.

2. Check whether parameters in flink-conf.yaml are correctly set.
3. For clusters with Kerberos authentication enabled, the configuration items include keytab and principal of Kerberos.
   • Download the user keytab file from the KDC server, and place the keytab file to a folder on the host of the Flink client (for example, /home/flinkuser/keytab).
   • Configure the following parameters in the ${FLINK_HOME}/conf/flink-conf.yaml file:
     1. Keytab path (note that there is a space before the parameter):

        security.kerberos.login.keytab:  /home/flinkuser/keytab/uer.keytab

     2. Principal name (developer username):

        security.kerberos.login.principal:flinkuser

4. Submit the job ./flink run /opt//client/Flink/flink/examples/streaming/WordCount.jar again and check whether the job can be submitted.
   • If the job can be submitted, the permission authentication is correct. In this case, you can check other errors. In this example, the name of log4j.properties is changed. After the name is restored, the job can be submitted normally.
   • If the job fails to be submitted, submit a service ticket.

Related Information

For details about how to use Flink, see Using Flink from Scratch.