"IllegalConfigurationException: Error while parsing YAML configuration file: "security.kerberos.login.keytab" Is Displayed When a Command Is Executed on an Installed Client

Symptom

After the client is successfully installed, an error message "IllegalConfigurationException: Error while parsing YAML configuration file:"security.kerberos.login.keytab" is displayed when the command (for example, yarn-session.sh) on the client is executed.

[root@8-5-131-10 bin]# yarn-session.sh
2018-10-25 01:22:06,454 | ERROR | [main] | Error while trying to split key and value in configuration file /opt/flinkclient/Flink/flink/conf/flink-conf.yaml:80: "security.kerberos.login.keytab: " | org.apache.flink.configuration.GlobalConfiguration (GlobalConfiguration.java:160)
Exception in thread "main" org.apache.flink.configuration.IllegalConfigurationException: Error while parsing YAML configuration file :80: "security.kerberos.login.keytab: "
        at org.apache.flink.configuration.GlobalConfiguration.loadYAMLResource(GlobalConfiguration.java:161)
        at org.apache.flink.configuration.GlobalConfiguration.loadConfiguration(GlobalConfiguration.java:112)
        at org.apache.flink.configuration.GlobalConfiguration.loadConfiguration(GlobalConfiguration.java:79)
        at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:482)
[root@8-5-131-10 bin]#

Cause Analysis

In a secure cluster environment, Flink requires security authentication. The security authentication is not configured on the current client.

1. The following two authentication modes are available for Flink.
   • Kerberos authentication: Flink Yarn client, Yarn ResourceManager, JobManager, HDFS, TaskManager, Kafka, and ZooKeeper
   • Internal authentication mechanism of Yarn: The internal authentication used between YarnResource Manager and Application Master (AM).
2. If a security cluster is required, the Kerberos authentication and security cookie authentication are mandatory. As shown in the logs, it is found that the security.kerberos.login.keytab setting in the configuration file is incorrect and the security configuration is not performed.

Solution

1. Download the keytab file from MRS and save it in a folder on a host where the Flink client resides.
2. Configure following parameters in the flink-conf.yaml file:
   1. Keytab path

      security.kerberos.login.keytab: /home/flinkuser/keytab/abc222.keytab

      

      • /home/flinkuser/keytab/abc222.keytab indicates the user directory, which is the directory saves the keytab file in 1.
      • Ensure that the client user has the permission on the corresponding directory.
   2. Principal name

      security.kerberos.login.principal: abc222

   3. In HA mode, if Zookeeper is configured, the ZooKeeper Kerberos authentication configuration items must be configured as follows:

      zookeeper.sasl.disable: false 
      security.kerberos.login.contexts: Client

   4. If Kerberos authentication is required between the Kafka client and Kafka broker, configure it as follows:

      security.kerberos.login.contexts: Client,KafkaClient