How Do I Migrate Data from Hive/HDFS to ClickHouse?

Question

How do I migrate Hive/HDFS data to ClickHouse?

Answer

You can export data from Hive as CSV files and import the CSV files to ClickHouse.

1. Export data from Hive as CSV files.

   hive -e "select * from db_hive.student limit 1000"| tr "\t" "," > /data/bigdata/hive/student.csv;

2. Import the CSV files to the student_hive table in the default database of ClickHouse. There can be security risks if a command contains the authentication password. You are advised to disable the command recording function (history) before running the command.

   clickhouse --client --port 9002 --password xxx -m --query='INSERT INTO default.student_hive FORMAT CSV' < /data/bigdata/hive/student.csv