Help Center> Data Lake Factory> User Guide> Configuration and Management> Managing Resources
Updated on 2022-02-22 GMT+08:00
View PDF

Managing Resources

You can upload custom code or text files as resources on Manage Resource and schedule them when running nodes. Nodes that can invoke resources include DLI Spark, MRS Spark, MRS MapReduce, and DLI Flink Job.

After creating a resource, configure the file associated with the resource. Resources can be directly referenced in jobs. When the resource file is changed, you only need to change the resource reference location. You do not need to modify the job configuration. For details about resource usage examples, see Developing a Spark Job.

(Optional) Creating a Directory

If a directory exists, you do not need to create one.

  1. Log in to the DLF console.
  2. In the navigation tree of the Data Development console, choose Configuration > Manage Resource.
  3. In the directory list, click . In the displayed dialog box, configure directory parameters. Table 1 describes the directory parameters.

    Table 1 Resource directory parameters

    Parameter

    Description

    Directory Name

    Name of the resource directory. Must consist of 1 to 32 characters and contain only letters, digits, underscores (_), and hyphens (-).

    Select Directory

    Parent directory of the resource directory. The parent directory is the root directory by default.

  4. Click OK.

Creating a Resource

You have enabled OBS before creating a resource.

  1. Log in to the DLF console.
  2. In the navigation tree of the Data Development console, choose Configuration > Manage Resource.
  3. Click Create Resource. In the displayed dialog box, configure resource parameters. Table 2 describes the resource parameters.

    Table 2 Resource management parameters

    Parameter

    Mandatory

    Description

    Name

    Yes

    Name of the resource. Must consist of 1 to 32 characters and contain only letters, digits, underscores (_), and hyphens (-).

    Type

    Yes

    File type of the resource. Possible values:

    • jar
    • file
    • archive

    Resource Location

    Yes

    Location of the resource. Possible values:

    • Local
    • OBS

    Main JAR package

    Yes

    Main JAR package that has been uploaded to OBS. This parameter is required when Type is set to jar and Resource Location is set to OBS.

    Depended JAR Package

    No

    Depended JAR package that has been uploaded to OBS. This parameter is required when Type is set to jar and Resource Location is set to OBS.

    Select Resource

    Yes

    Specific resource file.

    Storage Path

    Yes

    Path to a directory where the resource is stored. This parameter is required only when Resource Location is set to Local.

    Description

    No

    Descriptive information about the resource.

    Select Directory

    Yes

    Directory to which the resource belongs. The root directory is selected by default.

  1. Click Yes.

Editing a Resource

After a resource is created, you can modify resource parameters.

  1. Log in to the DLF console.
  2. In the navigation tree of the Data Development console, choose Configuration > Manage Resource.
  3. In the Operation column of the resource, click Edit. In the displayed dialog box, modify the resource parameters as required. For details, see Table 2.
  4. Click OK.

Deleting a Resource

If you do not need to use a resource any more, perform the following operations to delete it. Before deleting a resource, ensure that it is not used by any jobs.

  1. Log in to the DLF console.
  2. In the navigation tree of the Data Development console, choose Configuration > Manage Resource.
  3. In the Operation column of the resource, click Delete. The Delete Resource dialog box is displayed.
  4. Click Yes.

Importing a Resource

To import a resource, perform the following operations:

  1. Log in to the DLF console.
  2. In the navigation tree of the Data Development console, choose Configuration > Manage Resource.
  3. In the resource directory, click and select Import Resource. The Import Resource dialog box is displayed.
  4. Import a resource file uploaded to OBS, click Next and then Close.

Exporting a Resource

To export a resource, perform the following operations:

  1. In the navigation tree of the Data Development console, choose Configuration > Manage Resource.
  2. In the Operation column of the resource, click Export. The Export Resource dialog box is displayed.
  3. Select Export resource definitions only or Export resource definitions and files, and click OK.