Help Center/ Object Storage Service/ Best Practices/ Using OBS to Decouple Storage from Compute in Big Data Scenarios/ Connecting OBS to Big Data Components/ Connecting Druid to OBS
Updated on 2024-10-17 GMT+08:00
View PDF
Share

Connecting Druid to OBS

Overview

Druid is specially designed for workflows where fast data query and ingestion are required. It performs well in instant data visibility, ad hoc query, operations analytics, and high concurrency.

You can use OBSA-HDFS to connect OBS to Druid. In this way, you do not need to recompile Druid. OBS should be configured as deep storage.

Procedure

  1. Configure Druid.

    1. Modify the configurations: 
      conf/druid/single-server/micro-quickstart/_common/common.runtime.properties

      Add druid-hdfs-storage to druid.extensions.loadList.

    2. Configure the Deep Storage path in OBS.

  2. Configure OBSA-HDFS.

    1. Download OBSA-HDFS from GitHub and copy it to the extensions/druid-hdfs-storage/ directory.
    2. Add hdfs-site.xml to the conf/druid/single-server/micro-quickstart/_common/ directory and configure it as follows (replace the endpoint with the one actually used).

  3. Start Druid.
Parent Topic: Connecting OBS to Big Data Components