How Do I Migrate Data from OBS/S3 to ClickHouse?

Question

How do I migrate data from OBS/S3 to ClickHouse?

Answer

1. Query data stored in S3.
   select * from s3(path [,access_key_id, secret_access_key] [,format] [,structure])

   

   • path: bucket URL with a file path.
   • format: The file format.
   • access_key_id and secret_access_key: long-term credentials of an account. Credentials can be used to authenticate requests. These parameters are optional. If no credential is specified, credentials are read from the configuration file.
   • structure: The table schema.

   

2. Obtain data from S3 to create a table.

   CREATE TABLE test1_s3 (name String, value UInt32) ENGINE = S3(path, [access_key_id, secret_access_key,] format)

   

3. View the created table.

   select * from test1_s3