Help Center/ Log Tank Service/ Best Practices/ Log Jobs (Beta)/ Enriching Data/ Building Dictionaries and Tables to Enrich Data
Updated on 2025-12-04 GMT+08:00
View PDF
Share

Building Dictionaries and Tables to Enrich Data

Dictionaries and tables are two main data structures used for data enrichment. This section describes the common methods of building these two data structures and compares the advantages and disadvantages of different methods.

Building a Dictionary

The following table compares different dictionary building methods.

Table 1 Comparison of different dictionary building methods

Build Method

Advantage

Disadvantage

Direct build

Intuitive, simple, and convenient.

If there is a large amount of content, the processing rule is relatively long. In addition, it is static and inflexible.

Build from task configuration resources

Recommended when there is a large amount of content and the content is frequently modified. It is easy to maintain.

It is difficult to expand and reuse across tasks, and automatic refresh is not supported.

Build from tables

Used in advanced scenarios. The maintenance mechanism is more flexible.

You need to build and maintain the corresponding table, which is relatively complex.

Build from the dictionary function

Applicable to specific scenarios. Dictionaries are dynamically built based on logic.

It is relatively advanced and difficult to maintain.

Build from other expressions

Applicable to specific scenarios. Mappings are dynamically extracted from log event fields.

It is relatively advanced and difficult to maintain.
  • Direct build 
    e_dict_map({"400": "error", "200": "ok", "*": "other"}, "status", "message")
  • Building from advanced task configuration 
    e_dict_map(res_local("http_code_map"), "status", "message")

    http_code_map is an advanced task configuration item. Its value is {"400": "error", "200": "ok", "*": "other"}.

  • Build from tables

    Use tab_to_dict to build dictionaries from tables. For details about how to build a table, see "Building a Table" in this document.

    e_dict_map(tab_to_dict(tab_parse_csv("status_code,status_info\n400,error\n200,ok\n*,other"), "status_code", "status_info"), "status", "message")
  • Build from the dictionary function 
    e_dict_map(dct_make("400", "error", "200",  "ok", "*",  "other"), "status", "message")
  • Build from other expressions 
    e_dict_map(json_parse(v("http_code_map")), "status", "message")

    Mappings are obtained from the http_code_map field in source logs.

Building a Table

The following table compares different table building methods.

Table 2 Comparison of different table building methods

Build Method

Advantage

Disadvantage

Build from text

Intuitive, simple, and convenient.

If there is a large amount of content, the processing rule is relatively long. It is difficult to maintain, expand, and reuse.

Build from OBS resources

Recommended when there is a large amount of content and the content is not frequently modified. It is easy to maintain.

The compilation is relatively complex.
  • Build from text 
    e_table_map(tab_parse_csv("city,name,age\nshanghai,baixiao,10\ncity:nanjing,Maki,18"), "name",["city", "age"])
  • Build from OBS resources 
    e_search_table_map(tab_parse_csv(res_obs_file("https://obs.xxx.myhuaweicloud.com","dsl-test-xx","data.csv")), "name",["city", "age"])

    data.csv is a file in OBS. The value is as follows:

    e_search_table_map(tab_parse_csv(res_obs_file("https://obs.xxx.myhuaweicloud.com","dsl-test-xx","data.csv")), "name",["city", "age"])
Parent topic: Enriching Data