Configuring Grafana to Connect to DLI for Data Query and Analysis

Preparations

• Environment requirements:

  Ensure your system environment meets the following requirements:

  • Operating system: Windows or Linux
  • System architecture: Windows AMD64, Linux AMD64, Linux ARM64, or Darwin AMD64
  • Go is installed.

    Go version: later than 1.20. Run go env to check current Go environment settings.

• Toolkits:
  • DLI's Grafana driver: dli-grafana
  • Grafana installation package:

    Download Grafana at its official website.

    Select the Grafana version appropriate for your operating system. Version later than 11.0.0 is recommended.

    Download and extract the Grafana package, and complete installation by following the official documentation.

• Connection information:

  Table 1 Connection information

  Item	Description	How to Obtain
  DLI's AK/SK	AK/SK-based authentication uses an AK/SK pair to sign requests for identity authentication.	Obtaining an AK/SK
  DLI endpoint address	Endpoint of a cloud service in a region.	Obtaining an Endpoint
  DLI project ID	Project ID used for resource isolation.	Obtaining a Project ID
  DLI region information	Region to which DLI belongs.	Regions and Endpoints

Step 1: Install the dli-grafana Driver in Grafana

1. Obtain the dli-grafana plug-in from the DLI console.
2. Save all files from the dist directory of the downloaded and extracted dli-grafana plugin to the data/plugins/dli-datasource folder under the Grafana installation directory (create this folder manually if it does not exist).
3. In the conf/defaults.ini file under the Grafana installation directory, set the following parameter to allow loading unsigned drivers:

   allow_loading_unsigned_plugins = dli-datasource

4. Restart the Grafana service to load the dli-grafana driver.

Step 2: Access Grafana

1. Access Grafana using a browser with default port 3000.

   IP address of the host where Grafana is installed:Port
   Example: http://127.x.x.x:3000

2. Enter the Grafana administrator username and password.

Step 3: Configure Grafana to Connect to DLI

1. In the Grafana side menu, click Data Sources.
2. Click Add data source and select dli-datasource as the data source type.

   Set parameters based on Figure 1.

   Table 2 Grafana-to-DLI parameter description

   Parameter	Mandatory	Description	Example Value
   Region	Yes	Region name.	ap-southeast-2
   Project Id	Yes	Project ID where DLI resources are located.	0b33ea2a7e0010802fe4c009bb05076d
   Catalog Name	Yes	Metadata catalog name. Required when using a LakeFormation catalog. Set to the corresponding LakeFormation catalog name. When querying a LakeFormation catalog, a default database must exist under the catalog. If not specified, the default DLI catalog is used. No configuration is required when using the DLI catalog.	No For example, if using a LakeFormation catalog named lfcatalog, the configuration is as follows: catalog=lfcatalog.
   Database Name	Yes	DLI database name used to submit jobs.	tpch
   Queue Name	Yes	DLI queue name.	dli_test
   Engine Type	Yes	Type of the DLI queue. You can select spark.	spark
   AK and SK	Yes	AK/SK credentials.	-

   Figure 1 Example Grafana configuration
   
3. Click Save & Test to verify if the connection is successful.

Step 4: Create a Dashboard and Submit SQL Queries

1. On the Grafana home page, click + New to create a dashboard.
2. Add a new panel and select a chart type for your data.
3. In the panel settings, select the configured dli-datasource as the data source.
4. Enter your SQL query, preview the results, and adjust the chart settings as needed.
5. Save the panel and dashboard. You can then use this dashboard to monitor and analyze data in DLI.
   Figure 2 Submitting SQL queries in the dashboard