Field Operation Functions
This section describes field operation functions, including their syntax, parameters, and usage examples.
Function List
Function |
Description |
Obtains the value of a specific field in a log. When multiple field names are input at the same time, the value of the first existing field in the log is returned. This function can be used together with other functions. |
|
Adds a new field or sets a new value for an existing field. This function can be used together with other functions. |
|
Deletes log fields that meet conditions. This function can be used together with other functions. |
|
Retains the log fields that meet conditions. |
|
Packs log fields and outputs them to new fields. |
|
Renames the log field that meets conditions. This function can be used together with other functions. |
V
This function obtains the value of a specified log field. When multiple field names are input at the same time, the value of the first existing field in the log is returned.
- Function format
v(key, ..., default=None)
- Parameter description
Parameter
Type
Mandatory
Description
key
String
Yes
Field name.
default
Any
No
If the specified field name does not exist, the value of default is returned. The default value is None.
- Returned result
The value of the first field found in the log. If the field does not exist, the value of default is returned.
- Function example
Assign the value of the content field to the test_content field.
- Test data
{ "content": "hello" }
- Processing rule
e_set("test_content", v("content"))
- Processing result
content: hello test_content: hello
- Test data
- More
This function can be used together with other functions.
e_set
This function adds a new field or sets a new value for an existing field.
- Function format
e_set(key1, value1, key2, value2, mode="overwrite")
- key1 and value1 must be used in pairs.
- When the e_set function is used to set the time field F_TIME or __time__, the field must be set to a number or string.
e_set(F_TIME, "abc") # Incorrect e_set(F_TIME, "12345678") # Correct
- Parameter description
Parameter
Type
Mandatory
Description
key
String
Yes
Target field name, which can also be obtained through a string expression.
value
Any
Yes
New field value. Non-string values are converted into strings and stored in logs. Tuples, lists, and dictionaries are converted into JSON object strings.
Note: If the value None is transferred, the update operation is not performed.
mode
String
No
Field overwrite mode. The default value is overwrite.
- Returned result
The updated logs.
- Function example
- Example 1: Set a fixed value for a field.
Add a new field city whose value is Shanghai.
e_set("city", "Shanghai")
- Example 2: Copy the field value.
Call a single expression function to assign the value of the existing field ret to the new field result.
- Raw data
{"ret": "value"}
- Processing rule
e_set("result", v("ret"))
- Processing result
ret: value result: value
- Raw data
- Example 3: Dynamically set a value.
Call the combined expression function to obtain the value of the first existing field, return the value in lowercase, and assign the value to the result field.
e_set("result", str_lower(v("ret", "return")))
- Example 4: Set the field value multiple times.
- Raw data
{ "ret" : "fail" }
- Processing rule
e_set("event_type", "login event", "event_info", "login host")
- Processing result
ret: fail event_type: login event event_info: login host
- Raw data
- Example 1: Set a fixed value for a field.
- More
This function can be used together with other functions.
e_drop_fields
This function deletes log fields that meet the specified condition.
- Function format
e_drop_fields(key1, key2, ....,regex=false)
- Parameter description
Parameter
Type
Mandatory
Description
key
String
Yes
Log field name, which can be a regular expression. Fields are deleted if their names meet the specified condition. All other fields are retained. For more information about regular expressions, see section "Regular Expression." At least one log field must be configured.
regex
Boolean
No
If this parameter is set to false, regular expressions are not used for matching. If this parameter is not set, the default value true is used.
- Returned result
Logs with fields deleted.
- Function example
If the value of the content field is 123, the content and age fields are deleted.
- Test data
{ "age": 18, "content": 123, "name": "twiss" }
- Processing rule
e_if(e_search("content==123"), e_drop_fields("content", "age",regex=true))
- Processing result
name: twiss
- Test data
- More
This function can be used together with other functions.
e_keep_fields
This function retains log fields that meet the specified condition.
LTS contains built-in meta fields, such as __time__ and __topic__. If the __time__ field is not retained when the e_keep_fields function is called, the log time is reset to the current system time. If you do not want to reset the value of a meta field, add the meta field to the list. The common format is F_TIME, F_META, F_TAGS, "f1", "f2".
- Function format
e_keep_fields(key1, key2, ....,regex=false)
- Parameter description
Parameter
Type
Mandatory
Description
key
String
Yes
Log field name, which can be a regular expression. Fields are retained if their names meet the specified condition. All other fields are deleted. At least one field must be configured.
regex
Boolean
No
If this parameter is set to false, regular expressions are not used for matching. If this parameter is not set, the default value true is used.
- Returned result
Logs with retained fields.
- Function example
If the value of the content field is 123, the content and age fields are retained.
- Test data
{ "age": 18, "content": 123, "name": "twiss" }
- Processing rule
e_if(e_search("content==123"), e_keep_fields("content", "age"))
- Processing result
age: 18 content: 123
- Test data
e_pack_fields
This function packs log fields and exports them to new fields.
- Function format
e_pack_fields(output_fields,include=".*",exclude=None,drop_packed=true)
- Parameter description
Parameter
Type
Mandatory
Description
output_field
String
Yes
Name of the field output after packing. The value is in JSON format.
include
String
No
Whitelist configuration. Fields that meet the specified regular expression are packed. The default value is .*, indicating that all fields are matched.
exclude
String
No
Blacklist configuration. Fields that meet the regular expression are not packed. The default value is None, indicating that no matching judgment is performed.
drop_packed
Boolean
No
Whether to delete the original data after the data is packed. The default value is true.
- true (default): deletes the original data from the output result after packing.
- false: does not delete the original data from the output result after packing.
- Returned result
Packed log data.
- Function example
- Example 1: Pack all log fields into the test field. By default, the original fields are deleted.
- Test data
{ "test1":123, "test2":456, "test3":789 }
- Processing rule
e_pack_fields("test")
- Processing result
test:{"test1": 123, "test2": 456, "test3": 789}
- Test data
- Example 2: Pack all log fields into the test field. By default, the original fields are not deleted.
- Test data
{ "test1":123, "test2":456, "test3":789 }
- Processing rule
e_pack_fields("test",drop_packed=false)
- Processing result
test:{"test1": 123, "test2": 456, "test3": 789} test1:123 test2:456 test3:789
- Test data
- Example 3: Pack the test and abcd fields into the content field. The original fields are not deleted.
- Test data
{ "abcd@#%":123, "test":456, "abcd":789 }
- Processing rule
e_pack_fields("content", include="\w+", drop_packed=false)
- Processing result
abcd:789 abcd@#%:123 content:{"test": 456, "abcd": 789} test:456
- Test data
- Example 4: The test and abcd fields are not packed. Other fields are packed into the content field. The original fields are deleted.
- Test data
{ "abcd@#%":123, "test":456, "abcd":789 }
- Processing rule
e_pack_fields("content", exclude="\w+", drop_packed=true)
- Processing result
abcd:789 content:{"abcd@#%": "123"} test:456
- Test data
- Example 1: Pack all log fields into the test field. By default, the original fields are deleted.
e_rename
This function renames log fields that meet the specified condition.
- Function format
e_rename("key1", "new key1", "key2", "new key2", ..., regex=false)
The key and new key fields must be used in pairs.
- Parameter description
Parameter
Type
Mandatory
Description
key
String
Yes
Log field name, which can be a regular expression. If a field name meets the specified condition, the field name is renamed. At least one field must be configured.
new key
String
Yes
New field name after renaming.
regex
Boolean
No
If this parameter is set to false, regular expressions are not used for matching. If this parameter is not set, the default value true is used.
- Returned result
Renamed fields.
- Function example
- Example 1: Rename the host field to client_host.
- Test data
{ "host": 1006 }
- Processing rule
e_rename("host","client_host")
- Processing result
client_host: 1006
- Test data
- Example 2: If no matching field is found, no fields will be renamed.
- Test data
{ "host": 1006 }
- Processing rule
e_rename("url","rename_url")
- Processing result
host: 1006
- Test data
- Example 1: Rename the host field to client_host.
- More
This function can be used together with other functions.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot