Error Message "Error While Parsing YAML Configuration File: Security.kerberos.login.keytab" Is Displayed When a Command Is Executed on the Flink Client

Symptom

The client is successfully installed. When a user runs a client command, for example, yarn-session.sh, the following error message is displayed:

[root@host01 bin]# yarn-session.sh
2018-10-25 01:22:06,454 | ERROR | [main] | Error while trying to split key and value in configuration file /opt/flinkclient/Flink/flink/conf/flink-conf.yaml:80: "security.kerberos.login.keytab: " | org.apache.flink.configuration.GlobalConfiguration (GlobalConfiguration.java:160)
Exception in thread "main" org.apache.flink.configuration.IllegalConfigurationException: Error while parsing YAML configuration file :80: "security.kerberos.login.keytab: "

Cause Analysis

In a secure cluster environment, Flink requires security authentication. The security authentication is not configured on the current client.

1. The following two authentication modes are available for Flink.
   • Kerberos authentication: Flink Yarn client, Yarn ResourceManager, JobManager, HDFS, TaskManager, Kafka, and ZooKeeper
   • Internal authentication mechanism of Yarn: The internal authentication used between YarnResource Manager and Application Master (AM).
2. If a security cluster is required, the Kerberos authentication and security cookie authentication are mandatory. As shown in the logs, it is found that the security.kerberos.login.keytab setting in the configuration file is incorrect and the security configuration is not performed.

Solution

1. Download the user keytab file from MRS and save it to a directory on the node where the Flink client is deployed.
2. Configure the flink-conf.yaml file.
   1. Keytab path

      security.kerberos.login.keytab: /home/flinkuser/keytab/abc222.keytab

      

      • /home/flinkuser/keytab/abc222.keytab indicates the user directory, which is the directory saves the keytab file in 1.
      • Ensure that the client user has the permission on the corresponding directory.
   2. Principal name

      security.kerberos.login.principal: abc222

   3. In HA mode, if Zookeeper is configured, the ZooKeeper Kerberos authentication configuration items must be configured as follows:

      zookeeper.sasl.disable: false 
      security.kerberos.login.contexts: Client

   4. If Kerberos authentication is required between the Kafka client and Kafka broker, configure it as follows:

      security.kerberos.login.contexts: Client,KafkaClient