How Do I Handle an Error When I Use Lettuce to Connect to a Redis Cluster Instance After Specification Modification?

Symptom

If the shard quantity changes during specification modification of a Redis Cluster instance, some slots are migrated to new shards. The following error occurs when you use Lettuce to connect to the instance.

Figure 1 Error

For details, see Connection to X not allowed. This connection point is not known in the cluster view.

Analysis

Specification modification process of a Redis Cluster instance:

After being started, the client obtains the cluster node topology by using the Cluster Nodes command based on RESP2, and maintains the topology in its in-memory data structure.

For data access, the client uses the CRC16 algorithm to calculate the hash slot of a key, and automatically routes requests based on the topology and slot information stored in the memory.

If the number of shards changes during scaling, the topology and slot mapping changes. In this case, the client needs to automatically update the topology. Otherwise, the request route may fail or the route location may be incorrect. As a result, an error is reported during client connection.

For example, when the number of shards in a Redis Cluster instance changes from three to six, the topology and slot mapping changes as shown in the following figures.

Figure 2 A Redis Cluster instance before scaling up

Figure 3 A Redis Cluster instance after scaling up

Solutions

Solution 1 (Recommended)

Enable automated topology refresh.

ClusterTopologyRefreshOptions topologyRefreshOptions = ClusterTopologyRefreshOptions.builder()
             // Periodic refresh: every time milliseconds.
            .enablePeriodicRefresh(Duration.ofMillis(time))
	     // Triggers of adaptive refresh: MOVED redirection, ASK redirection, reconnection, unknown node (since 5.1), and slot not in any of the current shards (since 5.2).
            .enableAllAdaptiveRefreshTriggers()
            .build(); 

For details, see an example of using Lettuce to connect to a Redis Cluster instance.

If you use Lettuce to connect to a Redis Cluster instance and automated refresh is not enabled, you need to restart the client after specification modification.

Solution 2

Disable validation of cluster node membership.

ClusterClientOptions clusterClientOptions = ClusterClientOptions.builder()  
        .validateClusterNodeMembership(false)  
        .build(); 

If validateClusterNodeMembership is true, check whether the current connection address is in the cluster topology obtained through CLUSTER NODES, before connecting to the cluster. If it is not in the topology, the error occurs.

Impact of disabling validation of cluster node membership:

• Lack of security breach detection.
• If automated topology refresh is disabled, a MOVED redirection request may be generated after the Redis Cluster specifications are changed and the shard quantity increases. Redirection increases the network load of the cluster and the time required to process a single request. If the shard quantity decreases, deleted shards cannot be connected.