How Do I Merge Small Files?

If a large number of small files are generated during SQL execution, job execution and table query will take a long time. In this case, you should merge small files.

You are advised to use temporary tables for data transfer. There is a risk of data loss in self-read and self-write operations during unexpected exceptional scenarios.

INSERT OVERWRITE TABLE tablename
select  * FROM  tablename
DISTRIBUTE BY floor(rand()*20)