Configuring Grafana to Connect to DLI for Data Query and Analysis

Preparations

• Environment requirements:

  Make sure that your system environment meets the following requirements:

  • Operating system: Windows or Linux
  • Operating system architecture: Windows AMD64, Linux AMD64, Linux ARM64, or Darwin AMD64
  • Go is installed.

    Go version: later than 1.20. You can run go env to query the configuration of the current Go environment.

• Toolkits:
  • DLI's Grafana driver: dli-grafana
  • Grafana installation package:

    Download Grafana at its official website.

    Choose a version that is compatible with your OS. You are advised to choose a version later than 11.0.0.

    Download and extract the Grafana installation package, and install Grafana by following its official documentation.

• Connection information:

  Table 1 Connection information

  Item	Description
  DLI's AK/SK	AK/SK-based authentication refers to the use of an AK/SK pair to sign requests for identity authentication.
  DLI's endpoint address	Endpoint of a cloud service in a region.
  DLI's project ID	Project ID, which is used for resource isolation.
  DLI's region information	DLI's region information

Step 1: Install the dli-grafana Driver in Grafana

1. Obtain the dli-grafana plug-in from the DLI console.
2. Save all files in the dist directory of the downloaded and decompressed dli-grafana plugin to the data/plugins/dli-datasource folder in the Grafana installation directory (if the folder does not exist, manually create it).
3. In the conf/defaults.ini file in the Grafana installation directory, set the following parameter to allow unsigned drivers to be loaded:

   allow_loading_unsigned_plugins = dli-datasource

4. Restart the Grafana service to load the dli-grafana driver.

Step 2: Access Grafana

1. Access Grafana using a browser with default port 3000.

   IP address of the host where Grafana is installed:Port
   Example: http://127.x.x.x:3000

2. Enter the Grafana administrator username and password.

Step 3: Connect Grafana to DLI

1. In the Grafana sidebar, click Data Sources.
2. Click Add data source and select dli-datasource as the data source type.

   Set parameters based on Figure 1.

   Table 2 Parameters for connecting Grafana to DLI

   Parameter	Mandatory	Description	Example Value
   Region	Yes	Region name.	ap-southeast-2
   Project Id	Yes	ID of the project where DLI resources are.	0b33ea2a7e0010802fe4c009bb05076d
   Catalog Name	Yes	Metadata catalog name. It is mandatory when a LakeFormation catalog is used. In this case, it indicates the name of the LakeFormation catalog used. When querying the LakeFormation catalog, there must be a default database under the catalog. If left unset, a DLI catalog is used by default. You do not need to set this parameter when using the DLI catalog.	No For example, if you use a LakeFormation catalog named lfcatalog, the configuration is as follows: catalog=lfcatalog.
   Database Name	Yes	Name of the DLI database used to submit jobs.	tpch
   Queue Name	Yes	DLI queue name.	dli_test
   Engine Type	Yes	DLI queue type. The value can be spark or hetuEngine.	spark
   AK and SK	Yes	AK/SK that acts as the authentication key.	-

   Figure 1 Example Grafana configuration
   
3. Click Save & Test to verify if the connection is successful.

Step 4: Create a Dashboard and Submit a SQL Query

1. On the Grafana home page, click + New to create a dashboard.
2. Add a new panel and select a chart type for your data.
3. In the panel settings, select the configured dli-datasource as the data source.
4. Enter your SQL query statement, preview the query result, and adjust the chart settings to meet your needs.
5. Save the panel and dashboard. Then, use the dashboard to monitor and analyze the data stored in DLI.
   Figure 2 Submitting a SQL query in the dashboard