Scanning Expired Keys

Background

There are two ways to delete a key in Redis.

• Use the DEL command to directly delete a key.
• Use commands such as EXPIRE to set a timeout on a key. After the timeout elapses, the key becomes inaccessible but is not deleted immediately because Redis is mostly single-threaded. Redis uses the following strategies to release the memory used by expired keys:
  • Lazy free deletion: The deletion strategy is controlled in the main I/O event loop. Before a read/write command is executed, a function is called to check whether the key to be accessed has expired. If it has expired, it will be deleted and a response will be returned indicating that the key does not exist. If the key has not expired, the command execution resumes.
  • Scheduled deletion: A time event function is executed at certain intervals. Each time the function is executed, a random collection of keys are checked, and expired keys are deleted. (By default, 10 checks are executed every second. Each check randomly scans 20 keys which are set to expire.)
    

    To avoid prolonged blocks on the Redis main thread, not all keys are checked in each time event. Instead, a random collection of keys are checked each time. As a result, the memory used by expired keys cannot be released quickly.

Expired DCS Key Scan

DCS integrates these strategies and allows you to periodically release the memory used by expired keys. You can configure scheduled scans on the master nodes of your instances. The entire keyspace is traversed during the scans, triggering Redis to check whether the keys have expired and to remove expired keys if any.

• Perform expired key scans during off-peak hours to avoid 100% CPU usage.
• Released expired keys cannot be queried.

Procedure

1. Log in to the management console, and choose Application > Distributed Cache Service in the service list.
2. Click in the upper left corner of the management console and select the region where your instance is located.
3. In the navigation pane, choose Cache Manager.
4. Click the name of a DCS Redis instance.
5. Choose Analysis and Diagnosis > Cache Analysis.
6. On the Expired Key Scan tab page, scan for expired keys and release them.
   • Click Start Scanning to scan for expired keys immediately.
   • Enable Scheduled to schedule automatic scans at a specified time. For details about how to configure automatic scans, see Scheduling Automatic Scans.

7. After the expired key scan task is submitted, view it in the task list.
   Figure 1 Expired key scan tasks
   

The scan fails in the following scenarios:

• An exception occurred.
• There are too many keys, resulting in a timeout. Some keys have already been deleted before the timeout.

Scheduling Automatic Scans

To schedule automatic scans, click next to Scheduled. Set the parameters as required, and click OK.

Table 1 describes the parameters for scheduling automatic scans.

Performance

• The SCAN command is executed at the data plane every 5 ms, that is, 200 times per second. If Keys to Iterate is set to 10, 50, 100, or 1000, 2000, 10,000, 20,000, or 200,000 keys are scanned per second.
• The larger the number of keys scanned per second, the higher the CPU usage.

Reference test

A master/standby instance is scanned. There are 10 million keys that will not expire and 5 million keys that will expire. The expiration time is 1 to 10 seconds. A full scan is executed.

The following test results are for reference only. They may vary depending on the site environment and network fluctuation.

Configuration suggestions