Creating and Managing OpenSearch Index Policies

Context

Index State Management (ISM) of OpenSearch is a plugin that allows you to automate periodic, administrative operations on indexes by triggering them based on changes in the index age, index size, or number of documents. With ISM, you can define custom policies to automate index rollovers and deletion, thus optimizing cluster search performance or cutting storage costs. The procedure for using ISM is as follows:

1. Creating an Index Lifecycle Policy: Create an index lifecycle policy on OpenSearch Dashboards.
2. Associating Indexes with a Lifecycle Policy: Associate indexes with a lifecycle policy.
3. Managing Index Policies: Modify, retry, and change index lifecycle policies.

For more information about ISM, see Index State Management.

Automating Index Rollover in an OpenSearch Cluster Through Index Lifecycle Management describes a common use case for index lifecycle policies.

Creating an Index Lifecycle Policy

1. Log in to the CSS management console.
2. Choose Clusters > OpenSearch, find the cluster that you want to log in to, and click Access Kibana in the Operation column.
3. Log in to OpenSearch Dashboards, and choose Index Management on the left. The index management page is displayed.
4. Click Create policy to create an index policy. In the Configuration method dialog box, select JSON editor and click Continue. The page for creating an index policy is displayed.
   • Policy ID: use a custom policy name.
   • Define policy: define a custom policy based on the reference example provided.
     Figure 1 Configuring a policy
     
5. Click Create.

Associating Indexes with a Lifecycle Policy

You can bind a policy to one or more indexes, or associate a policy with an index template, so that the policy will be applied to all indexes created using this index template.

• Method 1: OpenSearch Dashboards CLI

  On the Dev Tools page of the OpenSearch Dashboards, run the following command to associate a policy ID with an index template:

  PUT _template/<template_name> 
  {
      "index_patterns": ["index_name-*"],
      "settings": {
          "opendistro.index_state_management.policy_id": "policy_id"
      }
  }

  • <template_name>: Replace it with the name of a created index template.
  • policy_id: Replace it with a custom policy ID.

  For details about how to create an index template, see Index Templates.

• Method 2: OpenSearch Dashboards Console
  1. On the Index Management page of the OpenSearch Dashboards, choose Indices.
     Figure 2 Choosing Indices
     
  2. In the Indices list, select one or more indexes to which you want to bind a policy.
  3. Click Apply policy in the upper right corner. In the displayed dialog box, select a policy ID.
     Figure 3 Apply policy
     
     Figure 4 Selecting a policy
     
  4. Click Apply.

     After you bind a policy to an index, ISM creates a job that runs every 5 minutes by default to execute the policy, check criteria, and change index states.

Managing Index Policies

1. On the Index Management page of the OpenSearch Dashboards, choose Managed Indices.

   The displayed page shows index policies configured for the current cluster.

2. Manage policies in the index policy list.
   • For a policy whose status is abnormal, click Retry policy.
   • To update a policy for an index, select the associated index, and click Change policy. After the policy is updated, click Change to apply the new policy.
   • To remove a policy from an index, select the index, and click Remove policy. In the displayed dialog box, click Remove.

For details, see Index State Management.