Help Center/ Data Lake Insight/ Best Practices/ Connecting BI Tools to DLI for Data Analysis/ Configuring DBT to Connect to DLI for Data Scheduling and Analysis
Updated on 2026-09-08 GMT+08:00
View PDF
Share

Configuring DBT to Connect to DLI for Data Scheduling and Analysis

Data Build Tool (DBT) is an open-source data modeling and transformation tool that runs in a Python environment. By connecting DBT to DLI, you can define and execute SQL transformations and support full lifecycle data management, from integration and transformation to analytics. It is suitable for large-scale analytics projects and complex data analysis scenarios.

This section describes the steps to connect DBT to DLI.

Preparations

  • Environment requirements:

    Ensure your system environment meets the following requirements:

    • Operating system: Windows or Linux
    • Ensure that Python is installed as DBT is Python-based.

      Python version: Python 3.8 or later. Python 3.8 is recommended.

  • Obtaining the dli-dbt driver package:

    Download the JDBC driver huaweicloud-dli-jdbc-xxx-dependencies.jar from the DLI management console.

  • Connection information:
    Table 1 Connection information

    Item

    Description

    How to Obtain

    DLI's AK/SK

    AK/SK-based authentication uses an AK/SK pair to sign requests for identity authentication.

    Obtaining an AK/SK

    DLI endpoint address

    Endpoint of a cloud service in a region.

    Obtaining an Endpoint

    DLI project ID

    Project ID used for resource isolation.

    Obtaining a Project ID

    DLI region information

    Region to which DLI belongs.

    Regions and Endpoints

Step 1: Deploy a DBT Environment

  1. Install dbt-core.

    Use pip to install the recommended dbt-core version.

    pip install dbt-core==1.7.9

    pip is Python's package manager and is usually installed with Python.

    If pip is not installed, install it using Python's built-in ensurepip module.

    python -m ensurepip

  2. Install dli-sdk-python.

    Run:

    python setup.py install
  3. Install dli-dbt.

    Download the dli-dbt driver from the DLI management console.

    Run:

    python setup.py install

    After installation, verify DBT by running:

    dbt --version

Step 2: Configure DBT to Connect to DLI

Configure the profiles.yml file to store DBT-to-DLI connection information.

In the home directory of the server where DBT is installed, locate the .dbt directory, and create or edit the profiles.yml file.

For example, on Windows, the path may be C:\Users\Username\.dbt\profiles.yml.

The configuration file should include DBT-to-DLI connection settings, for example:

profiles:
  - name: dbt_dli
    target: dev
    outputs:
      dev:
        type: dli
        region: your-region-name
        project_id: your-project_id
        access_id: your-ak
        secret_key: your-sk
        queue: your-queue-name
        database: your-dli-database
        schema: your-dli-schema
Table 2 DBT-to-DLI parameter description

Parameter

Mandatory

Description

Example Value

type

Yes

Data source type. Set it to dli in this example.

dli

region

Yes

Region name.

ap-southeast-2

project_id

Yes

Project ID where DLI resources are located.

0b33ea2a7e0010802fe4c009bb05076d

access_id and secret_key

Yes

AK/SK credentials.

-

queue

Yes

DLI queue name.

dli_test

database

Yes

Data directory name, with dli as default.

If using LakeFormation metadata, specify the exact catalog name.

dli

schema

Yes

DLI database name used to submit jobs.

tpch

Step 3: Test Submitting Jobs to DLI with DBT

  1. Initialize a DBT project.

    Run the following command in an empty directory:

    dbt init
  2. Configure dbt_project.yml.

    In the project root directory, create or edit the dbt_project.yml file.

    Configure the project by referring to dbt_project.yml.

    Ensure that the data source name defined in profiles.yml of the project has been set in the profile file in Step 2: Connect DBT to DLI.

    Figure 1 profile file
    Figure 2 profile configured in the dbt_project.yml file
  3. Verify the configuration.

    Run the following command to check whether the DBT configuration is correct:

    dbt debug
  4. Run project jobs.
    After verification succeeds, run the following command to execute your data models: 
    dbt run