Testing a Parser

Updated on 2024-09-30 GMT+08:00

View PDF

The ts_parse function allows direct testing of a text search parser.

   
      ts_parse(parser_name text, document text,
         OUT tokid integer, OUT token text) returns setof record

ts_parse parses the given document and returns a series of records, one for each token produced by parsing. Each record includes a tokid showing the assigned token type and a token which is the text of the token. For example:

   
      SELECT * FROM ts_parse('default', '123 - a number');
 tokid | token
-------+--------
    22 | 123
    12 |
    12 | -
     1 | a
    12 |
     1 | number
(6 rows)

   
      ts_token_type(parser_name text, OUT tokid integer,
              OUT alias text, OUT description text) returns setof record

ts_token_type returns a table which describes each type of token the specified parser can recognize. For each token type, the table gives the integer tokid that the parser uses to label a token of that type, the alias that names the token type in configuration commands, and a short description. For example:

   
    
      
      SELECT * FROM ts_token_type('default');
 tokid |      alias      |               description                
-------+-----------------+------------------------------------------
| asciiword       | Word, all ASCII
| word            | Word, all letters
| numword         | Word, letters and digits
| email           | Email address
| url             | URL
| host            | Host
| sfloat          | Scientific notation
| version         | Version number
| hword_numpart   | Hyphenated word part, letters and digits
| hword_part      | Hyphenated word part, all letters
| hword_asciipart | Hyphenated word part, all ASCII
| blank           | Space symbols
| tag             | XML tag
| protocol        | Protocol head
| numhword        | Hyphenated word, letters and digits
| asciihword      | Hyphenated word, all ASCII
| hword           | Hyphenated word, all letters
| url_path        | URL path
| file            | File or path name
| float           | Decimal notation
| int             | Signed integer
| uint            | Unsigned integer
| entity          | XML entity
(23 rows)

     

   
  