Migration from an AWS ElastiCache for Redis Database to a GeminiDB Redis Instance

Migration Principles

After backing up and exporting the RDB file in an AWS ElastiCache for Redis database, you can use Redis-Shake to restore data to a GeminiDB Redis instance.

Usage Notes

AWS does not support the psync/sync command and data cannot be incrementally migrated.
Before the migration, ensure that the network between the ECS where Redis-shake is deployed and the destination GeminiDB Redis is normal.
Ensure that the security group configuration on the source and target ends is enabled.

Procedure

Deploy the required migration tool.
1. Obtain Redis-Shake.
  
  Download the Redis-Shake release package and decompress it.
2. Modify the Redis-Shake.conf configuration file and configuring the following items:
  log.level = info #Default log level. A printed INFO log contains migration progress information, based on which you can judge whether the migration is complete.
  
  source.rdb.input = /xx/xx.rdb # Absolute path of the source RDB file
  
  target.address = <host>:6379 # Destination instance IP address
  
  target.password_raw = ***** # Password for logging in to a destination instance
  
  target.version = 5.0 # Version of the destination Redis instance
  
  target.type = standalone # Destination instance type
  
  target.db = 0 #Data is migrated to the specified database of the destination GeminiDB Redis. The default value is db0.
  
  big_key_threshold = 1 #Setting the big key threshold
3. Specify whether data of the destination is overwritten.
  key_exists = none
If there are duplicate keys on the source and destination, specify whether data of the destination is overwritten. The options are as follows:
- rewrite indicates that the source overwrites the destination.
- none indicates that the migration process exists once duplicate keys are detected.
- ignore indicates that keys in the source are retained and keys in the destination are ignored. This value does not take effect in rump mode.
none is recommended. There will be no duplicate data because the source is an RDB file. If the migration exits unexpectedly, you can choose Service Tickets > Create Service Ticket in the upper right corner of the console and contact customer service.
Migrate data.

Migration starting command:

./redis-shake.linux -conf=redis-shake.conf -type=restore

Use the restore mode because the source is an RDB file.

Stop the migration process after the migration is complete.
Verify data.

Data is obtained from the RDB file. Therefore, you need to check the GeminiDB Redis data at the destination end from the service perspective.

Verifying Data Consistency After Migration

After the migration is complete, you can check data consistency.

Data has been migrated from Redis, or incremental migration has started.
Redis-full-check must be deployed on the ECS, and the ECS is connected to the source and destination databases.
During incremental migration, data may be inconsistent due to network latency between the source and destination databases. You are advised to stop writing data to the source and then verify data consistency.
When Redis is used, an expiration time is usually set for keys. During migration, setting a key expiration time affects data consistency. Data may be inconsistent due to inconsistent expiration time.

During migration, DTS writes temporary probing keys to Redis on the destination database. Non-service data may be detected during data verification, which is normal.

Procedure

Log in to the ECS and ensure it is connected to the source and destination Redis databases.
Deploy redis-full-check.

Verify data.

/redis-full-check -s {Source IP address}:{Source port} -p {Source password} -t {Destination IP address}:{Destination port} -a {Destination password} -m 1

**Table 1** Parameter description
Parameter	Description	Example Value
-s	Source Redis database address and port number	-s 10.0.0.1:6379
-p	Password of the source Redis database	-
-t	Destination GeminiDB Redis database address and port number	-t 10.0.0.2:6379
-a	Password of the destination GeminiDB Redis database	-
-m	Verification mode: All key-value pairs Value length only Key integrity only All key values are verified, but only the length of big keys is verified. By default, the second verification mode is used.	-m 1
-q	Maximum QPS. The default value is 15000.	-q 5000
-d	Name of the file for saving the verification result. The default value is result.db.	-d result.db

View the verification result file.

By default, three rounds of verification are performed and three verification result files are generated. Generally, you only need to view the last verification result file.
- Run the sqlite3 result.db.3 command.
- Run the select * from key command.
- Check whether there are abnormal keys.