Help Center > > Developer Guide> Full Text Search> Dictionaries> Thesaurus Dictionary

Thesaurus Dictionary

Updated at: Jul 15, 2020 GMT+08:00

A Thesaurus dictionary (sometimes abbreviated as TZ) is a collection of relationships between words and phrases, such as broader terms (BT), narrower terms (NT), preferred terms, non-preferred terms, and related terms. Based on definitions in the dictionary file, a TZ replaces all non-preferred terms by one preferred term and, optionally, preserves the original terms for indexing as well. A TZ is an extension of a Synonym dictionary with added phrase support.

Precautions

  • A TZ has the capability to recognize phrases and therefore it must remember its state and interact with the parser to determine whether to handle the next token or stop accumulation. A TZ must be configured carefully. For example, if an AZ is configured to handle only asciiword tokens, a TZ definition like one 7 will not work because the token type uint is not assigned to the TZ.
  • TZs are used during indexing, so any change in the TZ's parameters requires reindexing. For most other dictionary types, small changes such as adding or removing stop words does not force reindexing.

Procedure

  1. Create a TZ named thesaurus_astro.

    thesaurus_astro is a simple astronomical TZ that defines two astronomical word combinations (word+synonym).
    1
    2
    supernovae stars : sn 
    crab nebulae : crab
    

    Run the following statement to create the TZ:

    1
    2
    3
    4
    5
    6
    CREATE TEXT SEARCH DICTIONARY thesaurus_astro (
        TEMPLATE = thesaurus,
        DictFile = thesaurus_astro,
        Dictionary = pg_catalog.english_stem,
        FILEPATH = 'obs://bucket_name/path accesskey=ak secretkey=sk region=rg'
    );
    

    The full name of the TZ file is thesaurus_astro.ths, and the TZ is stored in the obs://bucket_name/path accesskey=ak secretkey=sk region=rg directory. pg_catalog.english_stem is the subdictionary (a Snowball English stemmer) used for input normalization. The subdictionary has its own configuration (for example, stop words), which is not shown here. For details about the syntax and parameters for creating a TZ, see CREATE TEXT SEARCH DICTIONARY.

  2. Bind the TZ to the desired token types in the text search configuration.

    1
    2
    3
    ALTER TEXT SEARCH CONFIGURATION russian
        ALTER MAPPING FOR asciiword, asciihword, hword_asciipart
        WITH thesaurus_astro, english_stem;
    

  3. Use the TZ.

    • Test the TZ.
      The ts_lexize function is not very useful for testing the TZ because the function processes its input as a single token. Instead, you can use the plainto_tsquery, to_tsvector, or to_tsquery function which will break their input strings into multiple tokens.
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      SELECT plainto_tsquery('russian','supernova star');
       plainto_tsquery 
      -----------------
       'sn'
      (1 row)
      
      SELECT to_tsvector('russian','supernova star');
       to_tsvector 
      -------------
       'sn':1
      (1 row)
      
      SELECT to_tsquery('russian','''supernova star''');
       to_tsquery 
      ------------
       'sn'
      (1 row)
      

      supernova star matches supernovae stars in thesaurus_astro because the english_stem stemmer is specified in the thesaurus_astro definition. The stemmer removed e and s.

    • To index the original phrase, include it in the right-hand part of the definition.
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      supernovae stars : sn supernovae stars
      
      ALTER TEXT SEARCH DICTIONARY thesaurus_astro (
          DictFile = thesaurus_astro,
          FILEPATH = 'file:///home/dicts/');
      
      SELECT plainto_tsquery('russian','supernova star');
             plainto_tsquery       
      -----------------------------
       'sn' & 'supernova' & 'star'
      (1 row)
      

Did you find this page helpful?

Submit successfully!

Thank you for your feedback. Your feedback helps make our documentation better.

Failed to submit the feedback. Please try again later.

Which of the following issues have you encountered?







Please complete at least one feedback item.

Content most length 200 character

Content is empty.

OK Cancel