Development Rules

Database Connections

If the maximum number of mongod or mongos connections is reached, your client cannot connect to the DDS instances. Each connection received by mongod or mongos is processed by a single thread of 1 MB stack space. As the connections increase, too many threads will increase the context switching overhead and memory usage.

If you connect to databases from clients, calculate the number of clients and the size of the connection pool configured for each client. The total number of connections cannot exceed 80% of the maximum number of connections allowed by the current instance.
The connection between the client and the database must be stable. It is recommended that the number of new connections per second be less than 10.
You are advised to set the connection timeout interval of the client to at least three times the maximum service execution duration.
For a replica set, the IP addresses of both the primary and standby nodes must be configured on the client. For a cluster, at least two mongos IP addresses must be configured.
DDS uses user rwuser by default. When you log in as user rwuser, the authentication database must be admin.

Reliability

Rules for setting write concern: For mission-critical services, set write concern to {w:n},n>0. A larger value is better consistency but poorer performance.

w:1 means that a confirmation message was returned after data was written to the primary node.
w:1,journal:true means that the result was returned after data was written to the primary node and logs.
w:majority means that the result was returned after data was written to more than half of the total standby nodes.

If data is not written using w:majority, the data that is not synchronized to the standby node may be lost when a primary/standby switchover occurs.

If high reliability is required, deploy a cluster in three AZs.

Performance

Specification

The service program is not allowed to perform full table scanning.
During the query, select only the fields that need to be returned. In this way, the network and thread processing loads are reduced. If you need to modify data, modify only the fields that need to be modified. Do not directly modify the entire object.
Do not use $not. DDS does not index missing data. The $not query requires that all records be scanned in a single result collection. If $not is the only query condition, a full table scan will be performed on the collection.
If you use $and, put the conditions with the fewest matches before other conditions. If you use $or, put the conditions with the more matches first.
In a single instance, the total number of databases cannot exceed 200, and the total number of collections cannot exceed 500.
Before bringing a service online, perform a load test to measure the performance of the database in peak hours.
Do not execute a large number of concurrent transactions at the same time or leave a transaction uncommitted for a long time.
Before rolling out services, check the performance of all query types through the execution of query plans.

Suggestion

Each connection is processed by an independent thread in the background. Each thread is allocated with 1 MB stack memory. The number of connections should not be too large. Otherwise, too much memory is occupied.
Use the connection pool to avoid frequent connection and disconnection. Otherwise, the CPU usage is too high.
Reduce disk read and write operations: Reduce unnecessary upsert operations.
Optimize data distribution: Data is sharded and hot data is distributed evenly between shards.
Reduce lock conflicts: Do not perform operations on the same key too frequently.
Reduce lock wait time: Do not create indexes on the frontend.

Notice

During the development process, each execution on a collection must be checked using explain() to view its execution plan. Example:

db.T_DeviceData.find({"deviceId":"ae4b5769-896f"}).explain();

db.T_DeviceData.find({"deviceId":"77557c2-31b4"}).explain("executionStats");

A covered query does not have to read a document and returns a result from an index, so using a covered query can greatly improve query efficiency. If the output of explain() shows that indexOnly is true, the query is covered by an index.

Execution plan parsing:

Check the execution time. The smaller the values of the following parameters, the better the performance: executionStats.executionStages.executionTimeMillisEstimate and executionStats.executionStages.inputStage. executionTimeMillisEstimate
- executionStats.executionTimeMillis specifies how much time the database took to both select and execute the winning plan.
- executionStats.executionStages.executionTimeMillisEstimate is the execution completion time of the winning plan.
- executionStats.executionStages.inputStage. executionTimeMillisEstimate is the execution completion time of the child stage of the winning plan.
Check the number of scanned records. If the three items are the same, the index is best used.
- executionStats. nReturned is the number of documents that match the query condition.
- executionStats .totalKeysExamined indicates the number of scanned index entries.
- executionStats .totalDocsExamined indicates the number of scanned document entries.

Check the stage status. The following combinations of stages can provide good performance.

Fetch+IDHACK
Fetch+ixscan
Limit+ (Fetch+ixscan)
PROJECTION+ixscan

**Table 1** Status description
Status Name	Description
COLLSCAN	Full table scan
SORT	In-memory sorting
IDHACK	_id-based query
TEXT	Full-text index
COUNTSCAN	Number of unused indexes
FETCH	Index scanning
LIMIT	Using Limit to limit the number of returned records
SUBPLA	$or query stage without using an index
PROJECTION	Restricting the return of stage when a field is returned.
COUNT_SCAN	Number of used indexes

Cursor Usage Rules

If a cursor is inactive for 10 minutes, it will be automatically closed. You can also manually close it to save resources.

Rules for Using Distributed Transactions in Version 4.2

Spring Data MongoDB does not support the retry mechanism after a transaction error is reported. If the client uses Spring Data MongoDB as the client to connect to MongoDB, you need to use Spring Retry to retry the transaction based on the reference file of Spring Data MongoDB. For details, see the official document.