Interconnecting ClickHouse with OBS

Updated on 2024-10-08 GMT+08:00

View PDF

Using S3 Table Functions

Log in to the active OMS node.
Run the following command to obtain the data stored in OBS:
select * from S3(path, [ak, sk,] format, structure, [compression])
NOTE:
- path: Access domain name/OBS file path. Log in to OBS console and choose Parallel File Systems. On the page that is displayed, click the file system name. On the Files tab page, click the file name. The value of Link is the path.
  Figure 1 File path
- ak: Optional. The AK that has the permission to access OBS.
- sk: Optional. The SK that has the permission to access OBS.
- format: The file format.
- structure: The table schema.
- compression: Optional. The compression type.

Using the S3 Table Engine

Log in to the active OMS node.
Run the following commands to create a table:

CREATE TABLE test1_s3 ('name' String, 'age' int)

ENGINE = S3(path, [ak, sk,] format, [compression])
Run the following command to query the table:

select * from test1_s3;

Modifying Manager Configurations

Log in to FusionInsight Manager and choose Cluster > Services > ClickHouse. Click Configurations then All Configurations. Search for the clickhouse-config-customize parameter and add values. The following table lists the parameter values.

Parameter	Value
s3.endpoint-name.endpoint	OBS bucket address
s3.endpoint-name.access_key_id	OBS AK. For details on how to obtain the AK, see How Do I Obtain the Access Key AK/SK?
s3.endpoint-name.secret_access_key	OBS SK. For details about how to obtain the OBS SK, see How Do I Obtain the AK/SK?

Typically, the URL shared by OBS contains HTTPS. If the URL is inaccessible directly, perform the following operations to modify the configuration:

Parameter	Value
openSSL.client.loadDefaultCAFile	true
openSSL.client.cacheSessions	true
openSSL.client.disableProtocols	sslv2,sslv3
openSSL.client.preferServerCiphers	true
openSSL.client.invalidCertificateHandler.name	AcceptCertificateHandler

After the modification, click Save.

Parent topic: ClickHouse Data Import

Previous topic: Interconnecting ClickHouse with RDS for MySQL

Next topic: Synchronizing Kafka Data to ClickHouse

Feedback

Was this page helpful?

Helpful Not helpful

Provide feedback

Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.

The system is busy. Please try again later.

Which of the following issues have you encountered?

Content is inconsistent with the product UI

Unclear descriptions

Lack of examples or code

Incorrect steps

Can't find what I need

Lack of best practices

Feedback (optional)

0/500

Select at least one type of issue, and enter your comments or suggestions.

Enter a maximum of 500 characters.

Submit Cancel

For any further questions, feel free to contact us through the chatbot.

Chatbot