Assessment

Before changing shards, you need to carry out a preliminary evaluation and determine the number of new shards, whether to scale up the current DDM node class, and the number of required data nodes and their specifications.

• Data volume: Run show db status to query the volume of data involved.
• DDM node class: Determine nodes of the DDM instance and vCPUs and memory size of each node.
• Data node class: Determine the number of data nodes and vCPUs and memory size of each node.
• Business scale: Analyze current service scale and growth trend. Changing shards is a critical operation for data change. You are advised to postpone this operation if the storage space of a data node is sufficient.
• Whether to add shards: Adding shards leads to sharding rule changes. All data in the current schema needs to be rebalanced based on the new sharding rule. This process is slow and requires more resources.
• Whether to perform DDL operations during shard change: Read and write services are not affected. To ensure data consistency, do not perform DDL operations during shard change.

Customer cases:

A customer has a four-node DDM instance. Each node has 8 vCPUs and 16 GB memory. The instance is associated with 6 data nodes, containing 73,000 physical tables in total where 100 billion data records are stored, with the data volume up to about 12 TB.

Changing shards will definitely cause the migration of all schema data. Each data record must be rerouted, so the computing speed is obviously slower than the computing speed when shards are unchanged. Considering service requirements, change the DDM node class to 32 vCPUs | 64 GB, increase data nodes to 12, and upgrade the DDM kernel version to the latest. You can also restore the node class to the original one as required. The shards are not changed, so only half of the shards are migrated from original data nodes to new data nodes, with no route redistribution involved. Keep shards unchanged and increase data nodes unless there is a single physical table whose storage has reached the upper limit.