Performing Vector Search

Standard Query

Standard query is used to retrieve documents that are most similar to the query vector.

The following command will return k (specified by size/topk) records that are the closest matches to the query vector.

POST my_index/_search
{
  "size":2,
  "_source": false, 
  "query": {
    "vector": {
      "my_vector": {
        "vector": [1, 1],
        "topk":2
      }
    }
  }
}

Hybrid Query

Hybrid query combines vector search with traditional Elasticsearch queries, such as pre-filtering and Boolean queries.

Only Elasticsearch 7.10.2 clusters support pre-filtering queries.

In the following example, the top 10 records whose my_label value is red are returned.

• Pre-filtering query

  First, filters are applied to retrieve matching results. Then, vector search is performed on these results to retrieve the most relevant vectors based on similarity.

  The following is an example:

  POST my_index/_search
  {
    "size": 10,
    "query": {
      "vector": {
        "my_vector": {
          "vector": [1, 2],
          "topk": 10,
          "filter": {
            "term": { "my_label": "red" }
          }
        }
      }
    }
  }

  Table 2 Parameters for pre-filtering query

  Parameter	Mandatory	Type	Description
  filter	Yes	Map	Vector query filters. Standard Elasticsearch query filters are supported, such as term and range. If filter is too restrictive, leading to a small intermediate result set, you can set the index.vector.exact_search_threshold parameter, so that when the intermediate result set is smaller than this threshold, pre-filtering query automatically switches over to brute-force query (FLAT algorithm), which ensures a high recall rate. For more information, see Creating a Vector Index.
  term	No	Map	Term query is a type of exact query. Documents that contain the exact term will be returned. For example, {"term": {"my_label": "red"}} means only to return documents whose my_label value is red.

• Boolean query

  A Boolean query is in fact a post-filtering query method. Filtering and vector similarity-based search are performed separately. Then, the results of the two are combined using Boolean logic defined by clauses like must, should, and filter.

  The following is an example:

  POST my_index/_search
  {
    "size": 10,
    "query": {
      "bool": {
        "must": {
          "vector": {
            "my_vector": {
              "vector": [1, 2],
              "topk": 10
            }
          }
        },
        "filter": {
          "term": { "my_label": "red" }
        }
      }
    }
  }

  Table 3 Boolean query parameters

  Parameter	Mandatory	Type	Description
  bool	Yes	Map	A compound query clause that combines subqueries using configured Boolean logic. Parameter description: must: Clauses that must match for documents to be included in the results. filter: It is similar to must, but do not contribute to the relevance score. should: Clauses that should match, but are not required. They are like nice-to-haves. must_not: Clauses that must not match for documents to be included in the results.
  bool.must	Yes	Map	Clauses that must match for documents to be included in the results. Parameter description: vector: query vector my_vector: vector field topk: number of results to return
  bool.filter	Yes	Map	Clauses that must match, but do not contribute to the relevance score. Standard Elasticsearch query filters are supported, such as term and range.

Script Score Query

Script_score query enables custom similarity calculations for vector searches by executing a user-defined script. It works as follows:

Pre-filtering works with any query. script_score then calculates vector similarity on the pre-filtered results, and ranks the results. This query method does not use vector indexes. Its performance depends on the size of the intermediate result set after the pre-filtering. If the pre-filtering condition is set to match_all, a brute-force search is performed on all data.

The following is an example:

POST my_index/_search 
 { 
   "size":2, 
   "query": { 
   "script_score": { 
       "query": { 
         "match_all": {} 
       }, 
       "script": { 
         "source": "vector_score", 
         "lang": "vector", 
         "params": { 
           "field": "my_vector", 
           "vector": [1.0, 2.0], 
           "metric": "euclidean" 
         } 
       } 
     } 
   } 
 }

Rescore Query

Rescore query rescores and reranks the top results returned by an initial query to improve recall.

When the GRAPH_PQ or IVF_GRAPH_PQ indexing algorithm is used, query results are ranked based on the asymmetric distance calculated by PQ. Rescore query then rescores and reranks the initial search results to improve recall.

The following is an example of rescore query on a PQ index named my_index:

GET my_index/_search 
 { 
   "size": 10, 
   "query": { 
     "vector": { 
       "my_vector": { 
         "vector": [1.0, 2.0], 
         "topk": 100 
       } 
     } 
   }, 
   "rescore": { 
     "window_size": 100, 
     "vector_rescore": { 
       "field": "my_vector", 
       "vector": [1.0, 2.0], 
       "metric": "euclidean" 
     } 
   } 
 }

Painless Syntax Extension

Painless syntax extension allows the use of vector distance or similarity calculation functions in custom scripts. CSS extension supports several vector distance/similarity calculation functions, which users can use readily in custom Painless scripts to build flexible rescoring formulas.

The following is an example:

POST my_index/_search
{
  "size": 10,
  "query": {
    "script_score": {
      "query": {
        "match_all": {}
      },
      "script": {
        "source": "1 / (1 + euclidean(params.vector, doc[params.field]))",
        "params": {
          "field": "my_vector",
          "vector": [1, 2]
        }
      }
    }
  }
}