How Do I Migrate Data from OBS/S3 to ClickHouse?

Question

How do I migrate data from OBS/S3 to ClickHouse?

Answer

1. Query data stored in S3.
   select * from s3(path [,aws_access_key_id, aws_secret_access_key] [,format] [,structure])

   

   Parameter description:

   • path: The bucket URL with a file path.
   • format: The file format.
   • aws_access_key_id and aws_secret_access_key: Long-term credentials of an AWS account. Credentials can be used to authenticate requests. These parameter are optional. If no credential is specified, credentials are read from the configuration file.
   • structure: The table schema.

   

2. ClickHouse obtains data from S3 to create a table.

   CREATE TABLE test1_s3 (name String, value UInt32) ENGINE = S3(path, [aws_access_key_id, aws_secret_access_key,] format)

   

3. View the created table.

   select * from test1_s3