Help Center/ MapReduce Service/ Troubleshooting/ Using Kafka/ Failed to Start Kafka Due to Account Lockout
Updated on 2024-12-09 GMT+08:00
View PDF
Share

Failed to Start Kafka Due to Account Lockout

Symptom

The Kafka service fails to be started in the newly created MRS cluster.

The service startup log shows that the authentication fails.

/home/omm/kerberos/bin/kinit -k -t ${BIGDATA_HOME}/etc/2_15_ Broker /kafka.keytab kafka/hadoop.hadoop.com -c ${BIGDATA_HOME}/etc/2_15_ Broker /11846 failed.
export key tab file for kafka/hadoop.hadoop.com failed.export and check keytab file failed, errMsg=]}] for Broker #192.168.1.92@192-168-1-92.
[2015-07-11 02:34:33] RoleInstance started failure for ROLE[name: Broker].
[2015-07-11 02:34:34] Failed to complete the instances start operation. Current operation entities: [Broker #192.168.1.92@192-168-1-92], Failure entites : [Broker #192.168.1.92@192-168-1-92].Operation Failed.Failed to complete the instances start operation. Current operation entities: [Broker#192.168.1.92@192-168-1-92], Failure entites: [Broker #192.168.1.92@192-168-1-92].

Cause Analysis

The Kerberos log /var/log/Bigdata/kerberos/krb5kdc.log shows that IP addresses outside the cluster set up connections using a Kafka account, resulting in consecutive authentication failures and account lockout. 
Jul 11 02:49:16 192-168-1-91 krb5kdc[1863](info): AS_REQ (2 etypes {18 17}) 192.168.1.93: NEEDED_PREAUTH: kafka/hadoop.hadoop.com@HADOOP.COM for krbtgt/HADOOP.COM@HADOOP.COM, Additional pre-authentication required
Jul 11 02:49:16 192-168-1-91 krb5kdc[1863](info): preauth (encrypted_timestamp) verify failure: Decrypt integrity check failed
Jul 11 02:49:16 192-168-1-91 krb5kdc[1863](info): AS_REQ (2 etypes {18 17}) 192.168.1.93: PREAUTH_FAILED: kafka/hadoop.hadoop.com@HADOOP.COM for krbtgt/HADOOP.COM@HADOOP.COM, Decrypt integrity check failed

Solution

  1. Check the IP address of the node that connects to Kafka outside the cluster, for example, 192.168.1.93 in the example.
  2. Log in to the node outside the cluster and disable Kafka authentication on the node.
  3. Wait 5 minutes for the account to be unlocked.
  4. Restart the Kafka service.
Parent topic: Using Kafka