Why Data Load Performance Decreases due to Bad Records?

Question

Why data load performance decreases due to bad records?

Answer

If bad records are present in the data and BAD_RECORDS_LOGGER_ENABLE is true or BAD_RECORDS_ACTION is redirect then load performance will decrease due to extra I/O for writing failure reason in log file or redirecting the records to raw CSV.

Parent topic: CarbonData FAQ

Previous topic: Why Is the Data Logged in Bad Records Different from the Original Input Data with Escape Characters?

Next topic: Why INSERT INTO/LOAD DATA Task Distribution Is Incorrect and the Opened Tasks Are Less Than the Available Executors when the Number of Initial ExecutorsIs Zero?