Producer Failed to Send Data and Threw "NullPointerException"

Symptom

An MRS cluster has ZooKeeper and Kafka installed.

Possible Causes

1. The Kafka service is abnormal.
2. The jass and keytab files configured on the Producer client are incorrect.

Cause Analysis

The possible causes may be related to Producer or Kafka.

1. Check the Kafka service status and monitoring metrics on Manager.
   • MRS Manager: Log in to MRS Manager and choose Services > Kafka. Check the Kafka status. The status is Good, and the monitoring metrics are correctly displayed.
   • FusionInsight Manager: Log in to FusionInsight Manager and choose Cluster. Click the name of the target cluster and choose Service > Kafka.

     Check the Kafka status. The status is good, and the monitoring metrics are correctly displayed.

2. Check the Producer client log. The log contains "NullPointerException", as shown in Figure 1.
   Figure 1 Producer client log
   

   Alternatively, the log contains only "NullPointerException" but no stack information. The problem is caused by JDK self-protection. If much information is printed for the same stack, the JDK self-protection is triggered and stack information is no longer printed, as shown in Figure 2.

   Figure 2 Error information
   

3. Check the Producer client log. Error information "Failed to configure SaslClientAuthenticator" is displayed, as shown in Figure 3.
   Figure 3 Error log
   

4. The authentication failure causes the failure to create the KafkaChannel. The KafkaChannel obtained through the channel(key) method is empty and "NullPointerException" is excessively printed. According to the preceding log, the authentication fails due to an incorrect password which does not match the username.
5. Check the jaas and keytab files. The principal is set to stream in the jaas file.
   Figure 4 Checking the jaas file
   

   The principal is set to zmk_kafka in the user.keytab file.

   Figure 5 Checking the user.keytab file
   

   The principal in the jaas file is inconsistent with that in the user.keytab file.

   The application automatically and periodically updates the jaas file. However, when two processes of the application update the jaas file, one process writes a correct principal whereas the other process writes an incorrect one. As a result, the application is abnormal sometimes.

Procedure

1. Modify the jaas file to ensure that its principal exists in the keytab file.