Data Still Fails to Be Written After an FDI Task Failure Is Rectified

Cause Analysis

When the writer writes data to a database, it consumes data from a topic in Kafka. After the data is successfully saved to the database, the writer submits the consumption offset to Kafka. If an exception occurs when data is written, the consumption offset is not submitted. After the fault is rectified, restart the task to process the data that fails to be saved to the database again. Therefore, if the data does not meet the requirements of the destination after the task configuration is modified, the task still fails to be executed.

Solution

You can use any of the following methods to rectify the fault:

• As data in a Kafka topic will age six hours later, wait until the dirty data is cleared and then start the task.
• Delete the original task and create a task. In this way, topics in Kafka will be deleted and dirty data will be cleared.
• For a composite task, delete the table and start the task. Wait until the Writer consumes the data in this table and then add the table to collect data again.