What Should I Do If a Hudi Destination Job Fails Due to Intermittent Disconnection, Timeout, or Connection Termination?

Symptom

A job fails due to intermittent disconnection, timeout, or connection termination of the source link.

Possible Cause

The log contains error messages such as read timeout or terminate by xxx.

Solution

• If the network connectivity of the source link is unstable, you can use the shard retry capability to execute the job multiple times. In this case, you may need to modify the job configuration.
• If the number of shards has been configured for the job, the source is a partitioned table, and extraction by partition has been configured for the job, choose More > Shard Retry to rerun failed shards. (For example, if 100 shards are configured and an error occurs when 50 segments are executed, only the remaining 50 shards will be executed.)
• If the source is not a partitioned table and the number of shards is not configured for the job, you are advised to increase the number of shards and execute the job again. If an exception occurs, you can retry the job.
• If the source is a partitioned table and extraction by table partition is not configured, you are advised to configure extraction by table partition and execute the job again. If an exception occurs, you can retry the job.