Help Center/ Huawei Cloud Astro Canvas/ User Guide/ Data Center/ Data Source Management/ DLI Data Sources
Updated on 2025-03-24 GMT+08:00
View PDF
Share

DLI Data Sources

Data Lake Insight (DLI) is fully compatible with Apache Spark, Apache Flink, and OpenLooKeng (based on Apache Presto) ecosystems. It provides a one-stop serverless solution for processing and analyzing data, including stream processing, batch processing, and interactive analysis. Users can use it without managing any servers. DLI supports standard SQL, Spark SQL, and Flink SQL, and it is compatible with various data formats. With DLI, users can explore heterogeneous data in CloudTable, RDS, DWS, CSS, custom databases on ECS, and offline databases using SQL statements or programs, without the need for data extraction, transformation, or loading.

Creating a DLI Data Source

  1. Log in to Huawei Cloud Astro Canvas by referring to Logging In to Huawei Cloud Astro Canvas.
  2. Choose Data Center from the main menu.
  3. In the navigation pane, choose Data Sources.
  4. On the Data Sources page, click Create.
  5. Choose DLI and set data source parameters.

    Figure 1 DLI data sources
    • Name: name of the data source, which is used to identify the data source. The name can contain 1 to 32 characters, including letters, digits, and underscores (_), but cannot start or end with an underscore (_).
    • Project Name: The project name in the region where the DLI service is located. To obtain the project name, perform the following steps:
      1. Log in to Huawei Cloud management console.
      2. Move the cursor over your username in the upper right corner of the management console and click My Credentials from the drop-down list.
      3. Choose My Credentials > API Credentials > Projects and obtain the value.
    • Project ID: The project ID in the region where the DLI service is located. To obtain the project ID, perform the following steps:
      1. Log in to Huawei Cloud management console.
      2. Move the cursor over your username in the upper right corner of the management console and click My Credentials from the drop-down list.
      3. Choose My Credentials > API Credentials > Projects and obtain the project ID.
    • AK: Set this parameter to the value of Access Key ID in the access key pair. For details, see Obtaining an AK/SK.
    • SK: Set this parameter to the value of Secret Access Key in the access key pair. For details, see Obtaining an AK/SK.
    • Queue Name: The queue name in DLI. Queues are compute resources required for using DLI. All tasks need compute resources to execute.
      1. Log in to the DLI management console.
      2. In the navigation pane, choose Resources > Queue Management.
      3. Check the queue name. If no queue exists, create one.
    • Description: description of the data source to be created.

  6. Click Test. If the connection is successful, the data source is ready to use.
  7. Click Save.
Parent topic: Data Source Management