Help Center/ Distributed Cache Service/ FAQs/ Big/Hot Key Analysis/ How Do I Avoid Big Keys and Hot Keys?
Updated on 2024-12-11 GMT+08:00
View PDF

How Do I Avoid Big Keys and Hot Keys?

  • Keep the size of Strings within 10 KB and the quantity of Hashes, Lists, Sets, or Zsets within 5000.
  • When naming keys, use the service name abbreviation as the prefix and do not use special characters such as spaces, line brakes, single or double quotation marks, and other escape characters.
  • Do not rely too much on Redis transactions.
  • The performance of short connections is poor. Use clients with connection pools.
  • Do not enable data persistence if you use Redis just for caching and can tolerate data losses.
  • For details about how to optimize big keys and hot keys, see the following table.

Category

Method

Big key

Split big keys.

Scenarios:

  • If the big key is a String, you can split it into several key-value pairs and use MGET or a pipeline consisting of multiple GET operations to obtain the values. In this way, the pressure of a single operation can be split. For a cluster instance, the operation pressure can be evenly distributed to multiple shards, reducing the impact on a single shard.
  • If the big key contains multiple elements, and the elements must be operated together, the big key cannot be split. You can remove the big key from Redis and store it on other storage media instead. This scenario should be avoided by design.
  • If the big key contains multiple elements, and only some elements need to be operated each time, separate the elements. Take a Hash key as an example. Each time you run the HGET or HSET command, the result of the hash value modulo N (customized on the client) determines which key the field falls on. This algorithm is similar to that used for calculating slots in Redis Cluster.

Store big keys on other storage media.

If a big key cannot be split, it is not suitable to be stored in Redis. You can store it on other storage media, such as NoSQL databases, and delete the big key from Redis.

CAUTION:

Do not use the DEL command to delete big keys. Otherwise, Redis may be blocked or even a master/standby switchover may occur. The UNLINK command can be used to delete big keys in Redis 4.0 and later.

Set appropriate expiration and delete expired data regularly.

Appropriate expiration prevents expired data from remaining in Redis. Due to Redis's lazy free, expired data may not be deleted in time. If this occurs, scan expired keys.

Hot key

Split read and write requests.

If a hot key is frequently read, configure read/write splitting on the client to reduce the impact on the master node. You can also add replicas to meet the read requirements, but there cannot be too many replicas. In DCS, replicas replicate data from the master in parallel. The replicas are independent of each other and the replication delay is short. However, if there is a large number of replicas, CPU usage and network load on the master node will be high.

Use the client cache or local cache.

If you know what keys are frequently used, you can design a two-level cache architecture (client/local cache and remote Redis). Frequently used data is obtained from the local cache first. The local cache and remote cache are updated with data writes at the same time. In this way, the read pressure on frequently accessed data can be separated. This method is costly because it requires changes to the client architecture and code.

Design a circuit breaker or degradation mechanism.

Hot keys can easily result in cache breakdown. During peak hours, requests are passed through to the backend database, causing service avalanche. To ensure availability, the system must have a circuit breaker or degradation mechanism to limit the traffic and degrade services if breakdown occurs.
Parent topic: Big/Hot Key Analysis