Help Center/ MapReduce Service/ Component Operation Guide (LTS)/ Using ClickHouse/ Common Issues About ClickHouse/ How Do I Migrate Data from Hive/HDFS to ClickHouse?
Updated on 2024-10-25 GMT+08:00
View PDF
Share

How Do I Migrate Data from Hive/HDFS to ClickHouse?

Question

How do I migrate Hive/HDFS data to ClickHouse?

Answer

You can export data from Hive as CSV files and import the CSV files to ClickHouse.

  1. Export data from Hive as CSV files.

    hive -e "select * from db_hive.student limit 1000"| tr "\t" "," > /data/bigdata/hive/student.csv;

  2. Import the CSV files to the student_hive table in the default database of ClickHouse.

    clickhouse --client --port 9002 --password xxx -m --query='INSERT INTO default.student_hive FORMAT CSV' < /data/bigdata/hive/student.csv

    Commands containing authentication passwords pose security risks. Disable the command recording function (history) before running such commands to prevent information leakage.