Help Center/ Document Database Service/ Service Overview/ Basic Concepts/ DDS Sharding
Updated on 2026-08-17 GMT+08:00
View PDF
Share

DDS Sharding

Sharding is central to Huawei Cloud Document Database Service (DDS). This technique horizontally partitions datasets across shard nodes, enabling distributed storage and parallel processing. Each shard stores a portion of the data, and there is a shard key that determines how the data is distributed. Sharding overcomes the storage and performance limitations of a single node, ensuring seamless scalability for high-concurrency, large-scale workloads.

Why You Need DDS Sharding

  • Handling data growth: The storage capacity and performance of a single node are limited. Sharding enables horizontal scaling to improve storage and processing capabilities.
  • Load balancing: Read and write operations are evenly distributed across shards. There are no data hotspots.
  • High availability: Sharding works with replica sets to ensure that data is not lost and services are not interrupted during single node failures.
  • Elastic scaling: Shards can be added on demand to adapt to rapid service growth.

Advantages

Huawei Cloud DDS sharding delivers linear improvements in storage capacity and throughput through horizontal scaling. Leveraging an advanced cloud-native architecture, it offers automatic sharding management, elastic scaling, and fault self-healing to significantly reduce operational complexity, as described in Table 1.

Table 1 Advantages of DDS sharding

Category

Core Benefits

Description

Horizontal scalability

Linear throughput and capacity growth

Scales storage space and read/write performance proportionally by adding shard nodes to sustain service growth.

Cloud-native optimization

Automated O&M

Provides automatic shard management, elastic scaling, and fault self-healing to simplify O&M.

Performance optimization

Optimized for write-heavy workloads

Integrates the RocksDB storage engine (v4.2+), specifically optimized for write-heavy tasks like IoT logging and monitoring.

Flexible sharding policies

Adaptation to diverse business models

Supports both range-based and hash-based sharding. You can choose a sharding type based on specific service patterns.

Low O&M costs

Intelligent balancing and chunk migration

Minimizes the need for manual intervention while preserving cluster-wide load balance via automated balancers and chunk migration.

Use Cases

Table 2 Use cases of DDS sharding

Scenario

Sharding Policy

Scenario

Key Optimizations

IoT device log storage

Hash-based sharding by device ID (device_id)
  • Concurrent writes from massive devices
  • Data queries by device (for example, device track analysis).
  • Time-sensitive data that expires automatically
  • Create a compound index on device_id and timestamp.
  • Enable TTL for automatic data purging after a retention window, for example, 30 days.
  • Apply Snappy compression to minimize storage space.

Gaming statistics

Hash-based sharding by user ID (user_id)
  • Discrete user behavior distribution
  • Frequent query demands for historical user behavior (such as achievement)
  • User aggregation analysis
  • Create a compound index on user_id, action_type, and timestamp.
  • Configure read/write splitting to offload the primary node traffic.
  • Set an appropriate chunk size (for example, 64 MB) to avoid frequent migrations.

Real-time data monitoring

Sharding by timestamp range
  • Time-ordered data writes
  • Quick queries across time ranges (for example, metrics in the last hour)
  • Strong demand for hot/cold data isolation
  • Adjust the chunk size to 128 MB to balance migration frequency and efficiency.
  • Maintain an isolated single-field index on the timestamp attribute.
  • Enable storage compression paired with a cold-storage archiving policy.

How DDS Sharding Works

  1. Shard key selection policy
    • Even distribution: A field with a wide range of unique values (such as a unique user ID) is selected to ensure data is evenly distributed and avoid single-shard performance bottlenecks.
    • Query matching: The shard key aligns closely with high-frequency filtering criteria (such as time range) to guarantee targeted query routing and avoid cross-shard scans.
  2. Data distribution mechanism
    • Sharding logic
      • Hash-based sharding: Hash values are calculated for the shard key, and data is distributed across different chunks based on the hash ranges.
      • Range-based sharding: Chunks are organized sequentially based on the natural sort order of the shard key, such as timestamps and value ranges.
    • Metadata management: Config Server maintains a persistent mapping of chunks to individual shards, enabling instant routing lookups.
  3. Query routing process
    • The role of mongos: The query router parses the shard key in client query requests, evaluates the shard against Config Server metadata, and routes the requests directly to the right shard.
    • Scenarios
      • Queries containing the shard key are precisely routed to a single shard.
      • Queries without the shard key are broadcasted to all shards. mongos then aggregates the results, which impacts performance.
  4. Dynamic balancing and migration
    • Balancer monitoring: Balancer processes continuously monitor differences in the number of chunks or data volume across shards and automatically trigger migration tasks if specified thresholds are reached.
    • Migration process
      • The source shard freezes writes to the specific chunk undergoing migration.
      • Data is replicated to the target shard, and incremental modifications are synchronized.
      • Config Server metadata is updated, and the freeze is lifted.
    • Threshold control: Migration windows and chunk size thresholds can be configured to minimize the impacts on services.

Types of DDS Sharding

Table 3 Types of DDS sharding

Type

Key Feature

Typical Scenario

Range-based sharding

Data is partitioned based on the range of shard key values, for example, based on the timestamp range, to keep logically adjacent records physically close.

Time-series metrics (such as monitoring logs), and workloads requiring range queries

Hash-based sharding

Data is distributed evenly using hash values calculated from the shard key to ensure data is randomly dispersed.

Discrete fields such as user IDs and device IDs, and workloads requiring high-concurrency random read/write access.

Compound sharding

Utilizes suffix fields introduced via refineCollectionShardKey to support the optimization of multi-dimensional data distribution.

Complex queries with multiple conditions (for example, customer_id and order_id)

DDS Sharding vs. MongoDB Sharding

Although both DDS sharding and MongoDB sharding provide horizontal scaling based on the MongoDB distributed architecture, DDS, a Huawei Cloud managed database service, extends sharding with cloud-native integration, automated O&M, and targeted performance improvements. These enhancements make it better suited for large-scale enterprise production environments, as outlined in Table 4.

Table 4 Differences between DDS sharding and MongoDB sharding

Dimension

DDS Sharding

MongoDB Sharding

Cloud native support

Sharded cluster management is fully managed, with cloud native capabilities, such as auto scaling, fault self-healing, and backup and restoration.

Requires manual configuration of sharded clusters, node maintenance, and fault recovery.

O&M complexity

Automated O&M tools come built-in, including automatic balancing, monitoring and alarming, and log analysis, greatly reducing O&M costs.

Requires manual management of balancers, chunk migration, and backup.

Related Features and Operations

To learn more about DDS sharding features, refer to the following:

Best practice: To learn how to configure cluster sharding to improve database performance, see Sharding.

Deleting a shard: To learn how to release the shards of a sharded cluster instance, see Deleting Shards.

Troubleshooting shard deletion failures: To learn how to locate the failure and find the corresponding solution, see Troubleshooting Shard Deletion Failures.

Parent Topic: Basic Concepts