Basic Datasource Connections
DLI basic datasource connections can be used to access HBase and OpenTSDB of CloudTable and OpenTSDB, DWS, RDS, and CSS data sources of MRS.
- For details about how to develop SQL jobs, see HBase Table, OpenTSDB Table, DWS Table, RDS Table, and CSS Table in the Data Lake Insight SQL Syntax Reference.
- For details about how to develop Spark jobs, see Using the Spark Job to Develop DLI Datasource Connections in the Data Lake Insight Developer Guide.
- Basic datasource connection is applicable only to self-created queues.
- Datasource connections cannot be created for the default queue.
- To access a datasource connection table, you need to use the queue for which a datasource connection has been created.
- The preview function is not supported for datasource tables.
The basic datasource scenario provides the following functions:
- Creating a Connection
- Searching for a Connection
- Viewing Connection Details
- Deleting a Connection
- Tag Management
Basic Datasource Connections Page
Displays all basic datasource connections. If there are a large number of connections, they are displayed on multiple pages.
| Parameter | Description |
|---|---|
| Connection Name | Name of the created datasource connection. |
| Connection Status | Status of a datasource connection. The options are as follows:
|
| Service Type | Currently, the following data sources are supported:
|
| Connection URL | After a datasource connection is created, the connection address is displayed, which is used to access other data sources in SQL or Spark jobs. |
| Destination Address | Original address of the destination data source. |
| Created | Time when a connection is created. The connections in the connection list can be displayed according to the creation time in ascending or descending order. |
| Progress | Connection creation progress, in the unit of percentage. |
| Operation |
|
Creating a Connection
The following uses the CloudTable data source as an example to describe how to create a classic datasource connection:
- Create a data source to be accessed. For example, to access CloudTable, purchase a CloudTable cluster. Figure 1 Purchasing a CloudTable cluster
If there are available CloudTable clusters, skip this step.
- On the left pane of the DLI management console, click Datasource Connections.
- Click the Basic tab and click Create.
In the displayed dialog box, specify Connection Name, Service Type, Security Group, VPC, Subnet, and Destination Address. For details about the parameter description, see Table 2.
Figure 2 Creating a connection
Table 2 Description Parameter
Description
Connection Name
Name of the created datasource connection.
- The name can contain only digits, letters, underscores (_), and hyphens (-). The parameter must be specified.
- The connection name contains up to 64 characters.
Service Type
Currently, the following six types of data sources can be connected:
- CloudTable and CloudTable.OPENTSDB
- MRS OpenTSDB
- DWS
- RDS
- CSS
Security Group
Select a security group. Security group information of different services can be provided by the system when a service cluster or instance is created. You can also view information about existing clusters or database instances.
VPC
Select the corresponding VPC. Information about virtual private clouds (VPCs) of different services can be provided by the system when a service cluster or instance is created. You can also view information about existing clusters or database instances.
Subnet
Select the corresponding subnet. Subnet information of different services can be provided by the system when a service cluster or instance is created. You can also view information about existing clusters or database instances.
Destination Address
Address for connecting to the destination data source. The addresses of different services are slightly different.
- CloudTable: ZK link.
- CloudTable OpenTSDB: IP address of the node where OpenTSDB resides: External port number of OpenTSDB
- MapReduce OpenTSDB: IP address of the node where OpenTSDB resides: External port number of OpenTSDB.
To obtain the IP address and port number of MRS OpenTSDB, perform the following steps:
- In the MRS cluster list, click the name of the target cluster. The Overview page is displayed.
- In the Basic Information area of the cluster, locate the row that contains the target cluster and click View in the Operation column.
- Enter the username and password, and click Log In to go to the MRS Manager page. (The username and password are set when you purchase an MRS cluster. The default username is admin.)
- Click the Service Management tab.
- Access the OpenTSDB service.
- Click the Instance tab to view the corresponding service IP address. You can select any service IP address.
Click the Service Configuration tab and find the external port of OpenTSDB.
NOTE:If the MRS cluster has multiple IP addresses, enter any service IP address when creating a datasource connection.
- DWS: Intranet IP address:Port. NOTE:
A DWS cluster provides two private IP addresses. Enter either of them when creating a datasource connection.
- RDS: Intranet address:Database port.
- CSS: Intranet address:Port. NOTE:
If the CSS cluster has multiple private IP addresses, enter any of them when creating a datasource connection.
Tag
Tags are used to identify cloud resources. A tag pair includes the Tag Key and Tag Value. If you want to use the same tag to identify multiple cloud resources, that is, to select the same tag from the drop-down list box for all services, you are advised to create predefined tags on the Tag Management Service (TMS). For details, see the Tag Management Service User Guide.
NOTE:- A maximum of 10 tags can be added.
- Only one tag value can be added to a tag key.
- Tag Key: Enter a tag key name in the text box. NOTE:
- A tag key contains a maximum of 36 characters. The first and last characters cannot be spaces. The following characters are not allowed: =*,<>\|/
- If there are predefined tags, you can select a tag from the drop-down list box.
- Tag Value: Enter a tag value in the text box. NOTE:
- A tag value contains a maximum of 43 characters. The first and last characters cannot be spaces. The following characters are not allowed: =*,<>\|/
- If there are predefined tags, you can select a tag from the drop-down list box.
- Click OK.
- Choose Operation > More > Bind Queue to bind the queue.
- (Optional) After creating a basic datasource connection, you can perform further datasource analysis by referring to Cross-Source Analysis Development Methods.
Searching for a Connection
On the Basic page, you can enter the keyword of a connection name in the search box to search for the matching connection.
Viewing Connection Details
On the Basic page, select a connection and click
to view its details.
The connection details include the following options: Connection Name, Connection URL, Service Type, Connection ID, Destination Address, and Log Details.
If Connection Status is Failed, the failure cause will be displayed in the Log Details area.
Deleting a Connection
On the Basic page, you can click Delete in the Operation column to delete unnecessary connections.
A connection with Connection Status of Creating cannot be deleted.
Tag Management
A tag is a key-value pair customized by users and used to identify cloud resources. It helps users to classify and search for cloud resources. A tag consists of a tag key and a tag value.
If you use tags in other cloud services, you are advised to create the same tag key-value pairs for cloud resources used by the same business to keep consistency.
DLI supports the following two types of tags:
- Resource tags: indicate non-global tags created on DLI.
- Predefined tags created on Tag Management Service (TMS). Predefined tags are global tags. For details about predefined tags, see the Tag Management Service User Guide.
DLI allows you to add, modify, or delete tags for jobs.
- In the navigation tree on the left of the DLI management console, click and click the tab.
- In the Operation column of the link, choose More > Tags.
- The tag management page is displayed, showing the tag information about the current connection.
- Click Add Tag. In the displayed Add Tag dialog box, set parameters. Figure 3 Adding a tag
Figure 4 Adding a tag
Table 3 Tag parameters Parameter
Description
Tag key
You can perform the following operations:
- Click the text box and select a predefined tag key from the drop-down list. NOTE:
To add a predefined tag, you need to create one on TMS and select it from the Tag key drop-down list. You can click View predefined tags to enter the Predefined Tag page of TMS. Then, click Create Tag to create a predefined tag. For details, see section Creating Predefined Tags in the Tag Management Service User Guide.
- Enter a tag key in the text box. NOTE:
A tag key contains a maximum of 36 characters. The first and last characters cannot be spaces. The following characters are not allowed: =*,<>\|/
Tag Value
You can perform the following operations:
- Click the text box and select a predefined tag value from the drop-down list.
- Enter a tag value in the text box. NOTE:
A tag value contains a maximum of 43 characters. The first and last characters cannot be spaces. The following characters are not allowed: =*,<>\|/
- A maximum of 10 tags can be added.
- Only one tag value can be added to a tag key.
- The key name must be unique in the same resource.
- Click the text box and select a predefined tag key from the drop-down list.
- Click OK.
- (Optional) In the tag list, locate the row where the tag whose value you want to edit resides, click Edit in the Operation column to edit the tag value.
- (Optional) In the tag list, locate the row where the tag you want to delete resides, click Delete in the Operation column to delete the tag.
Last Article: Datasource Connection and Cross-Source Analysis
Next Article: Enhanced Datasource Connections
Did this article solve your problem?
Thank you for your score!Your feedback would help us improve the website.