Event Operation Functions
This section describes event operation functions, including their syntax, parameters, and usage examples.
Function List
Type |
Function |
Description |
---|---|---|
Event operations |
Determines whether to discard a log based on the specified condition. This function can be used together with other functions. |
|
Determines whether to retain a log based on the specified condition. Both the e_keep and e_drop functions discard logs. The e_keep function discards logs when the condition is not met, while the e_drop function discards logs when the condition is met. # The following four processing rules are equivalent. e_if_else(e_search("f1==v1"), e_keep(), e_drop()) e_if_else(e_search("not f1==v1"), e_drop()) e_keep(e_search("f1==v1")) e_drop(e_search("not f1==v1")) # The following processing rules are meaningless. e_if(e_search("..."), e_keep()) e_keep() This function can be used together with other functions. |
||
Event splitting |
Splits a log into multiple logs based on the values of log fields. Fields can be extracted through JMES and then split. This function can be used together with other functions. |
|
Event output |
Outputs logs to a specified log stream alias (specified when you configure the target log stream during processing task creation) and configures the tag information for output.
This function can be used together with other functions. |
e_drop
This function determines whether to discard a log based on the specified condition.
- Function format
e_drop(condition=true)
The fixed identifier DROP is supported. It is equivalent to e_drop().
- Parameter description
Parameter
Type
Mandatory
Description
condition
Bool
No
The default value is true. Generally, the result of a condition judgment function is transferred.
- Returned result
If the condition is met, the log is discarded and None is returned. Otherwise, the original log is returned.
- Function example
- Example 1: Discard a log when the __programe__ field's value is access. Otherwise, retain it.
- Test data
[ { "__programe__": "access", "age": 18, "content": 123, "name": "maki" }, { "__programe__": "error", "age": 18, "content": 123, "name": "maki" } ]
- Processing rule
e_if(e_search("__programe__==access"), DROP)
- Processing result
Logs whose value of __programe__ is access is discarded and logs whose value of __programe__ is error are retained.
__programe__: error age: 18 content: 123 name: maki
- Test data
- Example 2: Discard logs whose condition judgment result is true.
- Example 3: Retain logs whose condition judgment result is false.
- Test data
{ "k1":"v1" , "k2": "v2", "k3": "k1" }
- Processing rule
e_drop(e_search("not k1==v1"))
- Processing result
k1: v1 k2: v2 k3: k1
- Test data
- Example 4: If no judgment condition is set, the default value true is used and the log is discarded.
- Example 1: Discard a log when the __programe__ field's value is access. Otherwise, retain it.
- More
This function can be used together with other functions.
e_keep
This function determines whether to retain a log based on the specified condition.
- Function format
e_keep(condition=true)
The fixed identifier KEEP is supported. It is equivalent to e_keep().
- Parameter description
Parameter
Type
Mandatory
Description
condition
Bool
No
The default value is true. Generally, the result of a condition judgment function is transferred.
- Returned result
If the condition is met, the original log is returned. If the condition is not met, the log is discarded.
- Function example
- Example 1: Retain a log when the __programe__ field's value is access. Otherwise, discard it.
- Test data
[ { "__programe__": "access" , "age": 18, "content": 123, "name": "maki" }, { "__programe__": "error" , "age": 18, "content": 123, "name": "maki" } ]
- Processing rule
e_keep(e_search("__programe__==access")) # Equivalent to e_if(e_search("not __programe__==access"), e_drop()) # Equivalent to e_if_else(e_search("__programe__==access"), e_keep(), e_drop())
- Processing result
Logs whose value of __programe__ is access are retained.
__programe__: access age: 18 content: 123 name: maki
- Test data
- Example 2: Retain logs whose condition judgment result is true.
- Test data
{ "k1":"v1" , "k2": "v2", "k3": "k1" }
- Processing rule
e_keep(e_search("k1==v1"))
- Processing result
k1: v1 k2: v2 k3: k1
- Test data
- Example 3: Discard logs whose condition judgment result is false.
- Test data
{ "k1":"v1" , "k2": "v2", "k3": "k1" }
- Processing rule
e_keep(e_search("not k1==v1"))
- Processing result: The log is discarded.
- Test data
- Example 4: The judgment condition is false.
- Test data
{ "k1":"v1" , "k2": "v2", "k3": "k1" }
- Processing rule
e_keep(false)
- Processing result: The log is discarded.
- Test data
- Example 1: Retain a log when the __programe__ field's value is access. Otherwise, discard it.
- More
This function can be used together with other functions.
e_split
This function splits a log into multiple logs based on the values of log fields. Fields can be extracted through JMES and then split.
- Function format
e_split (field_name, sep=',', quote='"', lstrip=true, jmes=None, output=None)
Splitting rules:
- If the jmes parameter is configured, the value of the log field is converted into a JSON list, and the value extracted using JMES is used as the value of the next step. If the jmes parameter is not configured, the value of the field is used as the value of the next step.
- If the value of the previous step is a list or a string in JSON list format, the value is split according to the list and the processing ends. Otherwise, use sep, quote, or lstrip to perform CSV parsing on the value obtained in the previous step, split the value based on the parsed values, and end the processing.
- Parameter description
Parameter
Type
Mandatory
Description
field_name
String
Yes
Name of the field to be split.
sep
String
No
Delimiters for separating multiple values.
quote
String
No
Quote character for referencing multiple values.
lstrip
String
No
Whether to delete the spaces on the left of the value. The default value is true.
jmes
String
No
Convert the field value into a JSON object, extract a specific value using JMES, and then split the value.
output
String
No
Set a new field name. The old field name is overwritten by default.
- Returned result
A log list. The values of the fields in the list are the values in the source list.
- Function example
- Test data
{ "__topic__": "", "age": 18, "content": 123, "name": "maki" }
- Processing rule
e_set("__topic__", "V_SENT,V_RECV,A_SENT,A_RECV") e_split("__topic__")
- Processing result
[ { "__topic__": "V_SENT", "name": "maki", "age": 18, "content": 123 }, { "__topic__": "V_RECV", "name": "maki", "age": 18, "content": 123 }, { "__topic__": "A_SENT", "name": "maki", "age": 18, "content": 123 }, { "__topic__": "A_RECV", "name": "maki", "age": 18, "content": 123 } ]
- Test data
- More
This function can be used together with other functions.
e_output and e_coutput
These functions output logs to a specified log stream. Tag information can be configured when logs are output.
- Function format
e_output(logstream, tags=None) e_coutput(logstream, tags=None)
Logs are not output to the target log stream during preview. Instead, logs are output to the page for debugging.
- Parameter description
Parameter
Type
Mandatory
Description
logstream
String
No
The log stream alias must be specified when you configure the target log stream during job creation.
tags
Dict
No
New tag of a log. The tag is transferred in dictionary format.
Processing result- e_output: Logs are output to the specified log stream, and the subsequent processing rules are not executed.
- e_coutput: Logs are output to the specified log stream, and the subsequent processing rules are executed.
- Function example
- Example 1: Output k2 that matches the regular expression to target2.
- Test data
{ "k1": "v1", "k2":"v2", "x1":"v3" , "x5": "v4" }
- Processing rule
The e_drop() function is used to delete the data filtered by the e_if() function. If this function is not added, the filtered data is delivered to the default storage target. The first target log stream configured during task creation is the default target.
e_if(e_match("k2", r"\w+"), e_output("target2")) e_drop()
- Processing result
{ "k1": "v1", "k2": "v2", "x1": "v3", "x5": "v4" }
The results preview shows that the alias of the target log stream has changed to target2.
- Test data
- Example 2: Output k2 that matches the regular expression to target2 and set tags.
- Test data
{ "k1": "v1", "k2":"v2", "x1":"v3" , "x5": "v4" }
- Processing rule
e_if(e_match("k2", r"\w+"), e_output("target2", tags={"topic": "topic1"})) e_drop()
- Processing result
{ "k1": "v1", "__tag__": { "topic": "topic1" }, "k2": "v2", "x1": "v3", "x5": "v4" }
The results preview shows that the alias of the target log stream has changed to target2.
- Test data
- Example 1: Output k2 that matches the regular expression to target2.
- More
This function can be used together with other functions.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot