What Should I Do If An App ID Cannot Be Obtained When Spark Load Imports Data?

Symptom

When Spark Load is used to import data, error message "Waiting too much time to get appId from handle" is displayed.

Cause Analysis

The Doris reads logs and parses each line of log information to obtain appId and state. If the INFO log information is not printed, the obtained appId and state values are null. If the task times out, the task is canceled. As a result, data fails to be imported.

Procedure

1. Change the values of the following parameters in the log4j2.properties and log4j.properties files in the Client installation directory/Spark/spark/conf directory to INFO and save the changes:
   • log4j2.properties file:
     • rootLogger.level
     • logger.repl.level
     • logger.thriftserver.level
     • logger.jetty1.level
     • logger.jetty2.level
     • logger.parquet1.level
     • logger.parquet2.level
     • logger.RetryingHMSHandler.level
     • logger.FunctionRegistry.level
     • logger.hiveconf.level
   • log4j.properties file:
     • log4j.rootCategory
     • log4j.logger.org.apache.spark.repl.Main
     • log4j.logger.org.spark_project.jetty
     • log4j.logger.org.spark_project.jetty.util.component.AbstractLifeCycle
     • log4j.logger.org.apache.parquet
     • log4j.logger.parquet
     • log4j.logger.org.apache.hadoop.hive.metastore.RetryingHMSHandler
     • log4j.logger.org.apache.hadoop.hive.ql.exec.FunctionRegistry
     • log4j.logger.org.apache.hadoop.hive.ql.metadata.multiversion.MultiVersionFactory
     • log4j.logger.org.apache.hadoop.hive.conf.HiveConf
     • log4j.logger.org.apache.ranger.authorization.hadoop.config
     • log4j.logger.org.apache.ranger.audit.provider.AuditProviderFactory
     • log4j.logger.com.xxx.bigdata.om.agent.alarmcommon.SuppressionAlarmUtils

2. Execute the Spark Load task on the Doris client again.