Help Center> Data Lake Factory> User Guide> Data Management> Data Tables> Creating a Data Table (Visualized Mode)
Updated on 2022-02-22 GMT+08:00
View PDF

Creating a Data Table (Visualized Mode)

You can create permanent data tables in visualized mode. After creating a data table, you can use it for job and script development.

The following types of data tables can be created:

  • DLI
  • DWS
  • MRS Hive
  • CloudTable

Prerequisites

  • A corresponding cloud service has been enabled and a database has been created in the cloud service. For example, before creating a DLI table, DLI has been enabled and a database has been created in DLI.
  • A data connection that matches the data table type has been created in Data Development. For details, see Creating a Data Connection.

Procedure

  1. Perform the following steps:

    1. In the navigation tree of the DLF console, choose Development > Develop Script/Development > Develop Job.
    2. In the menu on the left, click , right-click tables, and choose Create Data Table from the shortcut menu.

  2. On the displayed page, configure basic properties. Specific settings vary depending on the data connection type you select. Table 1 lists the links for viewing property parameters of each type of data connection.

    Table 1 Basic property parameters

    Data Connection Type

    Parameter Description

    DLI

    For details, see the Basic Property part in Table 3.

    DWS

    For details, see the Basic Property part in Table 4.

    MRS Hive

    For details, see the Basic Property part in Table 5.

    CloudTable

    For details, see the Basic Property part in Table 6.

  3. Click Next. On the Configure Table Structure page, configure table structure parameters. Table 2 describes the table structure parameters.

    Table 2 Table structure

    Data Connection Type

    Parameter Description

    DLI

    For details, see the Table Structure part in Table 3.

    DWS

    For details, see the Table Structure part in Table 4.

    MRS Hive

    For details, see the Table Structure part in Table 5.

    CloudTable

    For details, see the Table Structure part in Table 6.

  4. Click OK.

Parameter Description

Table 3 DLI data table

Parameter

Mandatory

Description

Basic Property

Table Name

Yes

Name of the data table. Must consist of 1 to 63 characters and contain only lowercase letters, digits, and underscores (_). Cannot contain only digits or start with an underscore.

Alias

No

Alias of the data table. Must consist of 1 to 63 characters and contain only letters, digits, and underscores (_). Cannot contain only digits or start with an underscore.

Data Connection

Yes

Data connection to which the data table belongs.

Database

Yes

Database to which the data table belongs.

Data Location

Yes

Location to save data. Possible values:

  • OBS
  • DLI

Data Format

Yes

Format of data. This parameter is available only when Data Location is set to OBS. Possible values:

  • parquet: DLF can read non-compressed parquet data and parquet data compressed using Snappy or gzip.
  • csv: DLF can read non-compressed CSV data and CSV data compressed using gzip.
  • orc: DLF can read non-compressed ORC data and ORC data compressed using Snappy.
  • json: DLF can read non-compressed JSON data and JSON data compressed using gzip.

Path

Yes

OBS path where the data is stored. This parameter is available only when Data Location is set to OBS.

Table Description

No

Descriptive information about the data table.

Table Structure

Column Name

Yes

Name of the column. Must be unique.

Type

Yes

Type of data. For details about the data types, see Data Lake Insight SQL Syntax Reference.

Column Description

No

Descriptive information about the column.

Operation

No

To add a column, click .
Table 4 DWS data table

Parameter

Mandatory

Description

Basic Property

Table Name

Yes

Name of the data table. Must consist of 1 to 63 characters and contain only letters, digits, and underscores (_). Cannot contain only digits or start with an underscore.

Alias

No

Alias of the data table. Must consist of 1 to 63 characters and contain only letters, digits, and underscores (_). Cannot contain only digits or start with an underscore.

Data Connection

Yes

Data connection to which the data table belongs.

Database

Yes

Database to which the data table belongs.

Schema

Yes

Schema of the database.

Table Description

No

Descriptive information about the data table.

Advanced Settings

No

The following advanced options are available:

  • Storage method of a data table. Possible values:
    • Row store
    • Column store
  • Compression level of a data table
    • Possible values if the storage method is row store: YES or NO.
    • Possible values if the storage method is column store: YES, NO, LOW, MIDDLE, or HIGH. For the same compression level in column store mode, you can configure compression grades from 0 to 3. Within any compression level, the higher the grade, the greater the compression ratio.

Table Structure

Column Name

Yes

Name of the column. Must be unique.

Data Classification

Yes

Classification of data. Possible values:

  • Value
  • Currency
  • Boolean
  • Binary
  • Character
  • Time
  • Geometric
  • Network address
  • Bit string
  • Text search
  • UUID
  • JSON
  • OID

Data Type

Yes

Type of data. For details about the data types, see Data Warehouse Service Developer Guide.

Column Description

No

Descriptive information about the column.

Create ES Index

No

If you click the check box, an ES index needs to be created. When creating the ES index, select the created CSS cluster from the CloudSearch Cluster Name drop-down list. For details about how to create a CSS cluster, see Cloud Search Service User Guide.

Index Data Type

No

Data type of the ES index. The options are as follows:

  • text
  • keyword
  • date
  • long
  • integer
  • short
  • byte
  • double
  • boolean
  • binary

Operation

No

To add a column, click .
Table 5 Basic property parameters of an MRS Hive data table

Parameter

Mandatory

Description

Basic Property

Table Name

Yes

Name of the data table. Must consist of 1 to 63 characters and contain only lowercase letters, digits, and underscores (_). Cannot contain only digits or start with an underscore.

Alias

No

Alias of the data table. Must consist of 1 to 63 characters and contain only letters, digits, and underscores (_). Cannot contain only digits or start with an underscore.

Data Connection

Yes

Data connection to which the data table belongs.

Database

Yes

Database to which the data table belongs.

Table Description

No

Descriptive information about the data table.

Table Structure

Column Name

Yes

Name of the column. Must be unique.

Data Classification

Yes

Classification of data. Possible values:

  • Original
  • ARRAY
  • MAP
  • STRUCT
  • UNION

Data Type

Yes

Type of data.

Column Description

No

Descriptive information about the column.

Operation

No

To add a column, click .
Table 6 Basic property parameters of a CloudTable data table

Parameter

Mandatory

Description

Basic Property

Table Name

Yes

Name of the data table. Must consist of 1 to 63 characters and contain only letters, digits, and underscores (_). Cannot contain only digits or start with an underscore.

Alias

No

Alias of the data table. Must consist of 1 to 63 characters and contain only letters, digits, and underscores (_). Cannot contain only digits or start with an underscore.

Data Connection

Yes

Data connection to which the data table belongs.

Namespace

Yes

Namespace to which the data table belongs.

Table Description

No

Descriptive information about the data table.

Table Structure

Column Family Name

Yes

Name of the column family. Must be unique.

Column Family Description

No

Descriptive information about the column family.

Operation

No

To add a column, click .
Parent topic: Data Tables