Changing the Index Data Storage Mode from Local Disk to HDFS

Scenario

Solr supports collection data to be stored in local disks and HDFS. In most cases, local disk storage provides higher read and write performance and is recommended. This section describes how to change the storage mode from local disk storage to HDFS storage.

Before changing the storage mode, delete the historical indexes stored in the local disk one by one. Then, perform the following steps to change the storage mode to HDFS storage. This way, new indexes will be stored in HDFS.

Prerequisites

Solr has been installed and HDFS is running properly.

Procedure

Assume that some collections, such as collectionTest, are stored in the local disk. To change the storage mode to HDFS storage, perform the following steps:

1. Delete all collections that are stored in the local disk. The index collectionTest is used as an example.

   solrctl collection --delete collectionTest

   

   The index name collectionTest is used as an example. Change the index name based on the site requirements until all indexes stored on the local disk are deleted.

2. On the Solr configuration page, change the value of INDEX_STORED_ON_HDFS to TRUE and clear the value of SOLR_INDEX_LOCAL_STORAGE_DIR.
   

   After modifying the parameters, synchronize the configurations and restart the cluster. In this case, the index storage mode is changed to HDFS storage.

3. Create indexes as required. New index data is stored in HDFS.

If fs.defaultFS is set to viewfs://ClusterX, you need to configure the ViewFS mount directory of Solr in the HDFS parameter fs.viewfs.mounttable.ClusterX.link. Otherwise, the Solr service is unavailable.