Using Search Syntax

LTS provides a set of search syntax for setting search criteria, helping you search for logs more effectively.

Search statements specify the filtering rules for log query and returns logs that meet the conditions. They can be classified into full-text search statements and field search statements based on the index configuration mode. They can also be classified into exact search statements and fuzzy search statements based on the search precision. For details, see Table 1.

Structured Query Language (SQL) is a programming language used to control database access and manage data in databases. The pipe character (|) provides the capability of analyzing data using SQL statements. For details, see the SQL-92 syntax standard.

When using SQL statements for search, you are advised to use double quotation marks (") for field names or aliases in the SQL statements, for example, select "filed1.a" as "alias" from log.

**Table 1** Search modes
Search Mode	Description	Example Value
Full text search	After the full-text index is configured, LTS splits an entire log into multiple words based on the delimiters you set. You can search for logs using specified keywords (field name and value) and rules.	hello and world: Logs containing both keywords hello and world are searched.
Field search	You can search for specified field names and values (key:value) after field indexing is configured. You can perform multiple types of basic search and combined search based on the data type set in the field index.	time>60 and region:r: Logs in which the value of time* is greater than 60 and the value of region starts with r are searched.
Exact search	Search for logs that exactly match the specified words. LTS queries logs using delimiters. In exact searches, keywords may not match exactly. For example,if the search statement is abc def, the result will contain all logs that contain abc and def, but will not contain logs that contain the exact phrase abc def. To find logs containing the exact phrase abc def, use phrase search. For details about precise log searches, see the description of phrase searches.	region:r1: Logs whose region field value exactly matches r1 are searched.
Fuzzy search	Specify a keyword with a maximum of 64 characters in the search statement and add a wildcard, that is, an asterisk () or a question mark (?), to the middle or end of the keyword. LTS searches all logs for 100 words that meet the search criteria and returns logs that contain the words. The more precise the specified word is, the more accurate the search results are. The asterisk () indicates that multiple characters are matched, and the question mark (?) indicates that one character is matched. When an asterisk () and a question mark (?) are used as delimiters, fuzzy search is not supported since the question mark is a default delimiter. To perform a fuzzy search, remove the question mark from delimiters. Words cannot start with an asterisk () or a question mark (?). Long and float data does not support fuzzy search using asterisks (*) or question marks (?). If the fuzzy condition prefix is short and more than 100 words meet the criteria, the search results will be inaccurate.	my*: 100 words starting with my are searched in all logs and logs containing these words are returned.

Delimiters
LTS splits the log content into multiple words based on delimiters. Default delimiters include ,'";=()[]{}@&<>/:\\?\n\t\r and spaces.

For example, the default delimiters divide the log 2023-01-01 09:30:00 into four parts: 2023-01-01, 09, 30, and 00.

In this case, the search statement 2023 cannot match the log. You can search for the log using 2023-01* or 2023-01-01.

If the delimiter is set to null, the field value is regarded as a whole. You can search for the corresponding log only through complete log content or fuzzy search.
Keyword sequence
Only the phrase search #"abc def" can ensure the sequence of keywords. In other search modes, multiple keywords are connected by AND.

For example, request_method:GET POST is used to query logs that contain both GET and POST, and the sequence of GET and POST is not ensured. To ensure their sequence, phrase search is recommended.
Chinese search
Fuzzy search is not required for Chinese search. Phrase search is recommended to match more accurate results.

In LTS, English content is split into words of different lengths. Therefore, you can use fuzzy search to match logs with English words with the same prefix.

Unigram segmentation is used to a Chinese string into Chinese characters. Each Chinese character is independent, and the length of each part is 1 character.

For example, the search statement Monday indicates that logs containing M, o, n, d, a, and y are searched. The search statement #"Monday" indicates that logs containing the target phrase Monday are searched.
Invalid keyword
The syntax keywords of log search statements include: && || AND OR and or NOT not in : > < = ( ) [ ]

When and AND or OR NOT not in are used as syntax keywords, separate them with a space.

If the log contains syntax keywords and needs to be searched, the search statement must be enclosed in double quotation marks. Otherwise, syntax errors may occur or incorrect results may be found.

For example, if the search statement content:and contains the syntax keyword and, change it to content:"and".

Phrase search

Phrase search is used to precisely match the target phrase. For example, the search statement abc def matches all logs that contain both abc and def regardless of the sequence. For details about the differences between phrase search and keyword search, see Table 2.

Phrase search: It is implemented based on the keyword search syntax. Phrase search can distinguish the sequence of keywords and is used to accurately match target phrases, making the search result more accurate. Phrase search is applicable to English phrases and Chinese phrases, but cannot be used together with fuzzy search.

Keyword search: Keyword search is implemented based on word segmentation. Delimiters are used to split the search content into multiple keywords for log matching. Keyword search does not distinguish the sequence of keywords. Therefore, as long as a keyword can be matched in a log based on the AND or NOT logic, the log can be found.

**Table 2** Differences between two search modes
Search Mode	Phrase Search	Keyword Search
Differences between two search modes	Distinguishes the sequence of keywords and is used to accurately match target phrases, making the search result more accurate.	Does not distinguish the sequence of keywords. The keyword is matched based on the search logic.
Examples	Assume that your log stream contains the following two raw logs: Raw log 1: this service is lts Raw log 2: lts is service
	If you search for the phrase #"is lts", one log is matched.	If you search for the keyword is lts, two logs are matched.
	If you search for the phrase #"lts is", one log is matched.	If you search for the keyword lts is, two logs are matched.

**Table 3** Search modes
Search Mode	Description
Full-text search	#"abc def" content:#"abc def" content is a built-in field corresponding to the original log text. #"abc def" is equivalent to content:#"abc def" and matches the original log content by default.
Field search	key:#"abc def" The value parameter cannot be empty. When field search is used together with the not operator, logs that do not contain the specified field are matched.

Fuzzy search cannot be used together with phrase search.
The asterisk (*) and question mark (?) in phrase search are regarded as common characters. Therefore, phrase search does not support fuzzy search and can be used to search for the asterisk (*) and question mark (?) in logs.

Phrase search does not support search by delimiter.
For example, in the search statement #"var/log", / is a delimiter. The search statement is equivalent to #"var log", and is used to search for logs containing the target phrase var log. Similarly, search statements such as #"var:log" and #"var;log" are used to search for logs that contain the target phrase var log.
Phrase search is recommended for search in Chinese.
By default, unary word segmentation is used for Chinese characters. Each Chinese character is segmented separately. During the search, logs that contain each Chinese character in the search statement are matched, which is similar to fuzzy search. When more accurate results are required, phrase search is recommended.

Operators

Search statements support the following operators:

**Table 4** Operators
Operator	Description	Example Value
and	and operator.	request_method:GET and status:200
AND	AND operator, equivalent to and.	request_method:GET AND status:200
or	or operator.	request_method:GET or status:200
OR	OR operator, equivalent to or.	request_method:GET OR status:200
not	not operator.	request_method:GET not status:200 not status:200
( )	Specifies conditions that should be matched with higher priority.	(request_method:GET or request_method:POST) and status:200
:	Searches for a specified field (key:value). Use double quotation marks ("") to enclose a field name (key) or value that contains reserved characters, such as spaces and colons (:). Examples: "request method":GET message:"This is a log" time:"09:00:00" ipv6:"2024:AC8:2ac::d09"	request_method:GET
""	Encloses a syntax keyword to convert it into common characters. For example, "and" means searching for logs that contain this word. It is not an operator. All words enclosed in double quotation marks ("") are considered as a whole.	request_method:"GET"
\	Escapes double quotation marks (""). The escaped quotation marks indicate the symbol itself. \" must appear in pairs in search statements. For example, fieldName : "\"error\"" or fieldName:"\\" is valid, but fieldName : "error\"" or fieldName : "error\" is invalid.	To search for instance_id:nginx"01", use instance_id:nginx\"01\".
*	An asterisk can match zero, single, or multiple characters.	host:ecs*c
?	A question mark matches a single character.	host:ecs?c
>	Searches for logs with the value of a specific field greater than a given value.	request_time>100
>=	Searches for logs with the value of a specific field equal to or greater than a given value.	request_time>=100
<	Searches for logs with the value of a specific field less than a given value.	request_time<100
<=	Searches for logs with the value of a specific field equal to or less than a given value.	request_time<=100
=	Searches for logs with the value of a specific field equal to a given value. For fields of double and long types, equal signs (=) and colons (:) have the same function.	request_time=100 is equivalent to request_time:100.
in	Searches for logs whose field values are in a specified range. Brackets indicate a closed interval, and parentheses indicate an open interval. Numbers are separated with spaces.	request_time in [100 200] request_time in (100 200]
#""	Searches for logs that contain the target phrase, ensuring the sequence of keywords. The asterisk () and question mark (?) in phrase search are regarded as common characters. Therefore, phrase search does not support fuzzy search and can be used to search for the asterisk () and question mark (?) in logs.	request_method:#"GET POST"

Examples

Click to enlarge

**Table 5** Search description
Search Requirement	Search Statement
Search for logs whose User-Agent field value contains Mon, 17 Apr 2023.	User-Agent:#"Mon, 17 Apr 2023"
Search for logs whose User-Agent field value contains Mozilla/5.0.	User-Agent:#"Mozilla/5.0"
Search for logs whose week field value contains Monday.	week:#"Monday"