Updated on 2024-10-08 GMT+08:00

Interconnecting ClickHouse with OBS

Using S3 Table Functions

  1. Log in to the active OMS node.
  2. Run the following command to obtain the data stored in OBS:

    select * from S3(path, [ak, sk,] format, structure, [compression])
    • path: Access domain name/OBS file path. Log in to OBS console and choose Parallel File Systems. On the page that is displayed, click the file system name. On the Files tab page, click the file name. The value of Link is the path.
      Figure 1 File path
    • ak: Optional. The AK that has the permission to access OBS.
    • sk: Optional. The SK that has the permission to access OBS.
    • format: The file format.
    • structure: The table schema.
    • compression: Optional. The compression type.

Using the S3 Table Engine

  1. Log in to the active OMS node.
  2. Run the following commands to create a table:

    CREATE TABLE test1_s3 ('name' String, 'age' int)

    ENGINE = S3(path, [ak, sk,] format, [compression])

  3. Run the following command to query the table:

    select * from test1_s3;

Modifying Manager Configurations

Log in to FusionInsight Manager and choose Cluster > Services > ClickHouse. Click Configurations then All Configurations. Search for the clickhouse-config-customize parameter and add values. The following table lists the parameter values.

Parameter

Value

s3.endpoint-name.endpoint

OBS bucket address

s3.endpoint-name.access_key_id

OBS AK. For details on how to obtain the AK, see How Do I Obtain the Access Key AK/SK?

s3.endpoint-name.secret_access_key

OBS SK. For details about how to obtain the OBS SK, see How Do I Obtain the AK/SK?

Typically, the URL shared by OBS contains HTTPS. If the URL is inaccessible directly, perform the following operations to modify the configuration:

Log in to FusionInsight Manager and choose Cluster > Services > ClickHouse. Click Configurations then All Configurations. Search for the clickhouse-config-customize parameter and add values. The following table lists the parameter values.

Parameter

Value

openSSL.client.loadDefaultCAFile

true

openSSL.client.cacheSessions

true

openSSL.client.disableProtocols

sslv2,sslv3

openSSL.client.preferServerCiphers

true

openSSL.client.invalidCertificateHandler.name

AcceptCertificateHandler

After the modification, click Save.