Help Center/ Data Lake Insight/ User Guide (Paris Region)/ Data Management/ Databases and Tables/ Importing Data to the Table
Updated on 2022-07-04 GMT+08:00
View PDF
Share

Importing Data to the Table

You can import data from OBS to a table created in DLI.

Precautions

  • When creating an OBS table, you must specify a folder as the directory. If a file is specified, data import may be failed.
  • Only one path can be specified during data import. The path cannot contain commas (,).
  • If a folder and a file have the same name in the OBS directory, the file path is preferred as the path of the data to be imported.
  • To import data in CSV format to a partitioned table, place the column to be partitioned in the last column of the data source.
  • You are advised not to concurrently import data in to a table. If you concurrently import data into a table, there is a possibility that conflicts occur, leading to failed data import.
  • The imported file can be in CSV, Parquet, ORC, JSON, and Avro format. The text format must be UTF-8.

Prerequisites

The data to be imported has been stored on OBS.

Procedure

  1. You can import data on either the Data Management page or the SQL Editor page.

    • To import data on the Data Management page:
      1. On the left of the management console, choose Data Management > Databases and Tables.
      2. Click the name of the database corresponding to the table where data is to be imported to switch to the Manage Tables page.
      3. Locate the row where the target table resides and choose More > Import in the Operation column to switch to the Import Data dialog box.
    • To import data on the SQL Editor page:
      1. On the left of the management console, click SQL Editor.
      2. In the navigation tree on the left of SQL Editor, click Databases to see all databases. Click the database name corresponding to the table to which data is to be imported. The tables are displayed.
      3. Click on the right of the table and choose Import from the shortcut menu. The Import Data page is displayed.

  2. On the Import Data page, set the parameters based on Table 1.

    Table 1 Description

    Parameter

    Description

    Example

    Database Name

    Database where the current table is located.

    -

    Table Name

    Name of the current table.

    -

    File Format

    Format of the data source file to be imported. The CSV, Parquet, ORC, JSON, and Avro formats are supported.

    CSV

    Queue

    Select a queue.

    -

    Path

    You can directly enter a path or click and select an OBS path. If no bucket is available, you can directly switch to the OBS management console and create an OBS bucket.

    NOTE:

    The path can be a file or folder.

    obs://DLI/sampledata.csv

    Table Header: No/Yes

    This parameter is valid only when File Format is set to CSV. Whether the data source to be imported contains the table header.

    Click Advanced Settings and select the check box next to Table Header: No. If the check box is selected, the table header is displayed. If the check box is deselected, no table header is displayed.

    -

    User-defined Delimiter

    This parameter is valid only when File Format is set to CSV and you select User-defined Delimiter.

    The following delimiters are supported:

    • Comma (,)
    • Vertical bar (|)
    • Tab character (\t)
    • Others: Enter a user-defined delimiter.

    Default value: comma (,)

    User-defined Quotation Character

    This parameter is valid only when File Format is set to CSV and User-defined Quotation Character is selected.

    The following quotation characters are supported:

    • Single quotation mark (')
    • Double quotation marks (")
    • Others: Enter a user-defined quotation character.

    Default value: double quotation marks (")

    User-defined Escape Character

    This parameter is valid only when File Format is set to CSV and you select User-defined Escape Character.

    The following escape characters are supported:

    • Backslash (\)
    • Others: Enter a user-defined escape character.

    Default value: backslash (\)

    Date Format

    This parameter is valid only when File Format is set to CSV or JSON.

    This parameter specifies the format of the date in the table and is valid only Advanced Settings is selected. The default value is yyyy-MM-dd.

    2000-01-01

    Timestamp Format

    This parameter is valid only when File Format is set to CSV or JSON.

    This parameter specifies the format of the timestamp in the table and is valid only Advanced Settings is selected. The default value is yyyy-MM-dd HH:mm:ss.

    2000-01-01 09:00:00

    Error Records Path

    This parameter is valid only when File Format is set to CSV or JSON.

    The parameter specifies the error data is stored in the corresponding OBS path and is valid only Advanced Settings is selected.

    obs://DLI/

  3. Click OK.
  4. You can view the imported data in either of the following ways:

    Currently, only the first 10 records are displayed.

    • Choose Data Management > Databases and Tables > Manage Tables, locate the row that contains the target table, click More in the Operation column, and select Properties. In the displayed Preview tab, you can view the imported data.
    • On the Databases tab page of the SQL Editor, click the database name to go to the table list. Click on the right of the table and choose Properties from the shortcut menu. In the displayed dialog box, click Preview to view the imported data.

  5. (Optional) View the status and execution result of the importing job on the Job Management page.
Parent topic: Databases and Tables