Help Center/ MapReduce Service/ Component Operation Guide (LTS) (Ankara Region)/ Using Hudi/ Common Issues About Hudi/ Data Collection/ IllegalArgumentException Is Reported When Kafka Is Used to Collect Data
Updated on 2024-11-29 GMT+08:00

IllegalArgumentException Is Reported When Kafka Is Used to Collect Data

Question

The error "org.apache.kafka.common.KafkaException: Failed to construct kafka consumer" is reported in the main thread, and the following error is reported.

java.lang.IllegalArgumentException: Could not find a 'KafkaClient' entry in the JAAS configuration. System property 'java.security.auth.login.config' is not set

Answer

This error may occur when you try to collect data from the Kafka source with SSL enabled and the installation program cannot read the jars.conf file and its properties.

To solve this problem, pass the required property as part of the command submitted through Spark. Example: --files jaas.conf,failed_tables.json --conf 'spark.driver.extraJavaOptions=-Djava.security.auth.login.config=jaas.conf' --conf 'spark.executor .extraJavaOptions=-Djava.security.auth.login.config=jaas.conf'