Updated on 2024-03-18 GMT+08:00

Managing Tables on the Flink Web UI

Scenario

Data tables can be used to define basic attributes and parameters of source tables, dimension tables, and output tables.

Creating a Stream Table

  1. Access the Flink web UI. For details, see Accessing the Flink Web UI.
  2. Click Table Management. The table management page is displayed.
  3. Click Create Stream Table. On the stream table creation page, set parameters by referring to Table 1 and click OK.

    Table 1 Parameters for creating a stream table

    Parameter

    Description

    Remarks

    Stream/Table Name

    Stream/Table name, which can contain 1 to 64 characters. Only letters, digits, and underscores (_) are allowed.

    Example: flink_sink

    Description

    Stream/Table description information, which can contain 1 to 1024 characters.

    -

    Mapping Table Type

    Flink SQL does not provide the data storage function. Table creation is actually the creation of mapping for external data tables or storage.

    The value can be Kafka, HDFS, or Redis.

    -

    Type

    Includes the data source table Source, data result table Sink, and data dimension table Table. Tables included in different mapping table types are as follows:

    • Kafka: Source and Sink
    • HDFS: Source and Sink
    • Redis: Sink and Table

    -

    Data Connection

    Name of the data connection.

    -

    Topic

    Kafka topic to be read. Multiple Kafka topics can be read. Use separators to separate topics.

    This parameter is available when Mapping Table Type is set to Kafka.

    -

    File Path

    HDFS directory or a single file path to be transferred.

    This parameter is available when Mapping Table Type is set to HDFS.

    Example:

    /user/sqoop/ or /user/sqoop/example.csv

    Code

    Codes corresponding to different mapping table types are as follows:

    • Kafka: CSV and JSON
    • HDFS: CSV
    • Redis:
      • If Type is set to Sink, the value can be String, List, Set, Zset, or Hash.
      • If Type is set to Table, the value can be String or Zset.

    -

    Prefix

    When Mapping Table Type is set to Kafka, Type is set to Source, and Code is set to JSON, this parameter indicates the hierarchical prefixes of multi-layer nested JSON, which are separated by commas (,).

    For example, data,info indicates that the content under data and info in the nested JSON file is used as the data input in JSON format.

    If Mapping Table Type is set to Redis, prefixes will be automatically added to the key or you can manually enter prefixes.

    For example, if the key value is key1 and the prefix is test, the key written to Redis is test:key1.

    Separator

    Meanings of this parameter corresponding to different mapping table types are as follows:

    • Kafka: This parameter is used to specify the separator between CSV fields. This parameter is available when Code is set to CSV.
    • Redis: This parameter is used to specify the field separator.

    Example: comma (,)

    Row Separator

    Line break in the file, including \r, \n, and \r\n.

    This parameter is available when Mapping Table Type is set to HDFS.

    -

    Column Separator

    Field separator in the file.

    This parameter is available when Mapping Table Type is set to HDFS.

    Example: comma (,)

    Data Validity Period

    Data validity period, which can be Permanent, Effective Duration, or Deadline.

    This parameter is available when Mapping Table Type is set to Redis and Type is set to Sink.

    -

    Stream Table Structure

    Stream/Table structure, including Name and Type.

    -

    Proctime

    System time, which is irrelevant to the data timestamp. That is, the time when the calculation is complete in Flink operators.

    This parameter is available when Type is set to Source.

    -

    Event Time

    Time when an event is generated, that is, the timestamp generated during data generation.

    This parameter is available when Type is set to Source.

    -

Editing a Stream Table

  1. Access the Flink web UI. For details, see Accessing the Flink Web UI.
  2. Click Table Management. The table management page is displayed.
  3. In the Operation column of the item to be modified, click Edit. On the displayed page, modify the stream table information by referring to Table 1 and click OK.

Searching for a stream table

  1. Access the Flink web UI. For details, see Accessing the Flink Web UI.
  2. Click Table Management. The table management page is displayed.
  3. In the upper right corner of the page, you can enter a keyword to search for stream table information.

Deleting a Stream Table

  1. Access the Flink web UI. For details, see Accessing the Flink Web UI.
  2. Click Table Management. The table management page is displayed.
  3. In the Operation column of the item to be deleted, click Delete, and click OK in the displayed page.