DDS 4.4 Function Overview

Mutable Shard Keys

In a DDS sharded cluster, a good shard key is critical because it determines whether a sharded cluster has good scalability under a specified workload. When using DDS, even if you select an appropriate shard key, there may be jumbo chunks due to workload changes, or service traffic may be sent to a single shard.

In DDS 4.2, you can change the shard key value, but you must migrate data across shards based on distributed transactions. This will increase performance overheads and is unable to prevent jumbo chunks and query hotpots. In DDS 4.4, you can run the refineCollectionShardKey command to add one or more suffix fields to an existing shard key to improve the distribution of existing data on chunks. The refineCollectionShardKey command has low performance overheads because this command does not involve any data migration. A shard key must be supported by an index. You must create an index that supports the new shard key before you run the refineCollectionShardKey command.

The following operations demonstrate how to use mutable shard keys on a DDS 4.4 cluster instance:

1. Run the shardCollection command to perform range-based sharding on the coll collection in the test database based on the customer_id field.

   use admin
   db.adminCommand({ 
     shardCollection: "test.coll",  
     key: { "customer_id": 1 }		
   })

2. To change the shard key of the coll collection to {"customer_id": 1, "order_id": 1}, create an index.

   use test
   db.coll.createIndex({
     "customer_id": 1,
     "order_id": 1
   })

3. Run the refineCollectionShardKey command to add order_id as a suffix field to change the shard key. (You can run the sh.status() command to verify the change.)

   use admin
   db.adminCommand( {
      refineCollectionShardKey: "test.coll",
      key: { customer_id: 1, order_id: 1 }
   } )

Hedged Reads

Page response speed affects user experience and is closely related to economic benefits. If a page takes more than 3 seconds to load, most visitors will leave the page. To solve this problem, DDS 4.4 provides the hedged read feature. In a sharded cluster, the mongos nodes route a read request from a client to multiple replica set nodes of a shard and return results from the first respondent to the client.

The hedged read feature is enabled for specific operations using the Read Preference parameter.

• If you set the Read Preference parameter to nearest, the hedged read feature is enabled by default.
• If you set the Read Preference parameter to primary, the hedged read feature is not supported.
• If you set the Read Preference parameter to a value other than nearest or primary, you must set the hedgeOptions parameter to true to enable the hedged read feature.

Example:

db.collection.find({ }).readPref(
   "secondary",                      // mode
   [ { "usage": "read" },  { } ],    // tag
   { enabled: true }                 // hedgeOptions
)

Default Read and Write Concerns

In versions earlier than DDS 4.4, if you do not specify readConcern or writeConcern for a specific operation, the default value is used. For example, the default value of readConcern is local, and the default value of writeConcern is {w: 1}. These default values cannot be changed. If you want to ensure strong data consistencies and set writeConcern for all insert operations to {w: majority} and readConcern for all read operations to majority, you must specify this configuration in all DDS access code.

In DDS 4.4, you can run the setDefaultRWConcern command to specify the global default readConcern and writeConcern. Example:

db.adminCommand({
  "setDefaultRWConcern" : 1,
  "defaultWriteConcern" : {
    "w" : "majority"
  },
  "defaultReadConcern" : { "level" : "majority" }
})

You can also run the getDefaultRWConcern command to obtain the current default values of readConcern and writeConcern.

Compound Hashed Shard Keys

In versions earlier than DDS 4.4, you can specify only the hash key with a single field. This may lead to uneven data distribution in collections across data shards.

DDS 4.4 supports compound hashed indexes. You can specify a single hashed field in compound indexes as the prefix or suffix field.

Example:

sh.shardCollection(
  "test.coll",
  { "fieldA" : 1, "fieldB" : 1, "fieldC" : "hashed" }  // suffix field
)

sh.shardCollection(
  "test.coll",
  { "_id" : "hashed", "fieldA" : 1}		       // prefix field
)

The flexible compound hashed indexes have many advantages and simplify the database table design. For example, a collection uses a monotonically increasing value as the shard key. Data from the latest access request is also written to the same shard, which results in a large shard and uneven distribution of data across shards. If compound hashed shard keys are not supported, you must compute the hash value of a single field, store the hash value in a special field of a document as the index value, and then use the range-based sharding feature to specify the index value as your shard key.

In DDS 4.4, you need only to specify the field as hashed index. This simplifies the business logic.

Other Usability Enhancements

1. Jumbo chunks are automatically balanced.

   In earlier versions, jumbo chunks can be eliminated only by manually migrating chunks. In DDS 4.4, jumbo chunks can be automatically migrated and balanced. This function is performed in the background, reducing unnecessary alarms and relieving the pressure of O&M personnel.

2. Distributed transactions allow the size of a single document to exceed 16 MB.

   In earlier versions, when you attempt to insert a document larger than 16 MB or update an existing document in a way that makes it larger than 16 MB, the DDS server returns an error. In DDS 4.4, this restriction is removed for distributed transactions to better meet actual service requirements.

3. The projection function is enhanced.
   DDS 4.4 is fully compatible with the new projection syntax and usage of MongoDB 4.4. For example:

   • In projection, aggregation syntax is supported, for example, using aggregation operators.
   • In projection, data is encoded in JSON format and nested to map specified fields.
   • In projection, the $ character can be used to specify a specific index subelement for a mapped array element.
4. The allowDiskUse option is added to the find command.

   In versions earlier than DDS 4.4, if the memory usage for sort operations exceeds the limit, the query operations with blocked sorting will fail. In DDS 4.4, when allowDiskUse is set to true, the find command uses temporary files on the disk to support the non-indexed sorting operations that exceed the memory usage limit 100 MB.

   Example:

   db.coll.find({"location" : "unit12" })
       .sort({"time" : 1})
       .allowDiskUse()

Summary

DDS 4.4 enhances existing capabilities and improves usability. For more information about other optimizations than the preceding features, see Compatibility Details.