Ranking Search Results

Ranking evaluates the relevance of documents based on the query content, displaying the best matching results first. DWS provides two predefined ranking functions. These functions take into account lexical, distance, and structural information, that is, the frequency of query terms appearing in the document, their proximity to each other, and the importance of their location within the document. However, the concept of relevance is inherently ambiguous and strongly application-dependent. Different applications may require additional information for ranking, such as document modification time or built-in ranking functions. You can also develop your own ranking functions or combine the results of these ranking functions with additional factors to meet specific requirements.

The two ranking functions currently available are:

1

ts_rank([ weights float4[], ] vector tsvector, query tsquery [, normalization integer ]) returns float4

Ranks vectors based on the frequency of their matching lexemes.

1

ts_rank_cd([ weights float4[], ] vector tsvector, query tsquery [, normalization integer ]) returns float4

This function requires positional information in its input. Therefore, it will not work on "stripped" tsvector values. It will always return zero.

For both these functions, the optional weights argument offers the ability to weigh word instances more or less heavily depending on how they are labeled. The weight arrays specify how heavily to weigh each category of word, in the order:

{D-weight, C-weight, B-weight, A-weight}

If no weights are provided, then these defaults are used: {0.1, 0.2, 0.4, 1.0}

Typically weights are used to mark words from special areas of the document, like the title or an initial abstract, so they can be treated with more or less importance than words in the document body.

Since a longer document has a greater chance of containing a query term it is reasonable to take into account document size. For example, a hundred-word document with five instances of a search word is probably more relevant than a thousand-word document with five instances. Both ranking functions take an integer normalization option that specifies whether and how a document's length should impact its rank. The integer option controls several behaviors, so it is a bit mask: you can specify one or more behaviors using a vertical bar (|) (for example, 2|4).

• 0 (the default) ignores the document length
• 1 divides the rank by (1 + Logarithm of the document length)
• 2 divides the rank by the document length
• 4 divides the rank by the mean harmonic distance between extents (this is implemented only by ts_rank_cd)
• 8 divides the rank by the number of unique words in document
• 16 divides the rank by (1 + Logarithm of the number of unique words in document)
• 32 divides the rank by (itself + 1)

If more than one flag bit is specified, the transformations are applied in the order listed.

It is important to note that the ranking functions do not use any global information, so it is impossible to produce a fair normalization to 1% or 100% as sometimes desired. Normalization option 32 (rank/(rank+1)) can be used to scale all ranks into the range zero to one. This is just a cosmetic change, and it will not affect the ordering of the search results.

Here is an example that selects only the ten highest-ranked matches:

Run the following statements in a database that uses the UTF-8 or GBK encoding:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

SELECT id, title, ts_rank_cd(to_tsvector(body), query) AS rank 
FROM tsearch.pgweb, to_tsquery('science') query 
WHERE query @@ to_tsvector(body) 
ORDER BY rank DESC 
LIMIT 10;
 id |  title  | rank 
----+---------+------
 11 | Philology  |   .2
  2 | Mathematics |   .1
 12 | Geography  |   .1
 13 | Computer science  |   .1
(4 rows)

Example of using normalized ranking:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

SELECT id, title, ts_rank_cd(to_tsvector(body), query, 32 /* rank/(rank+1) */ ) AS rank 
FROM tsearch.pgweb, to_tsquery('science') query 
WHERE  query @@ to_tsvector(body) 
ORDER BY rank DESC 
LIMIT 10;
 id |  title  |   rank   
----+---------+----------
 11 | Philology  |  .166667
  2 | Mathematics | .0909091
 12 | Geography  | .0909091
 13 | Computer science  | .0909091
(4 rows)

Example of ranking queries using Chinese word segmentation:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

CREATE TABLE tsearch.ts_zhparser(id int, body text);
INSERT INTO tsearch.ts_zhparser VALUES (1, 'sort');
INSERT INTO tsearch.ts_zhparser VALUES(2, 'sort query');
INSERT INTO tsearch.ts_zhparser VALUES(3, 'query sort');
-- Accurate match
SELECT id, body, ts_rank_cd (to_tsvector ('zhparser', body), query) AS rank FROM tsearch.ts_zhparser, to_tsquery ('sort') query WHERE query @@ to_tsvector (body);
 id | body | rank 
----+------+------
  1 | sort |   .1
(1 row)

-- Fuzzy match
SELECT id, body, ts_rank_cd (to_tsvector ('zhparser', body), query) AS rank FROM tsearch.ts_zhparser, to_tsquery ('sort') query WHERE query @@ to_tsvector ('zhparser', body);
 id |   body   | rank 
----+----------+------
  3 | query sort |   .1
  1 | sort     |   .1
  2 | sort query |   .1
(3 rows)

Ranking can be expensive since it requires consulting the tsvector of each matching document, which can be I/O bound and therefore slow. Unfortunately, it is almost impossible to avoid since practical queries often result in large numbers of matches.