Updated on 2024-05-30 GMT+08:00

Test Method

This section describes performance testing of GeminiDB Mongo replica sets (version 4.0), including the test environment, procedure, and results.

Test Environment

  • Region: CN-Hong Kong
  • AZ: AZ1
  • Elastic Cloud Server (ECS): s3.2xlarge.2 flavor with 8 vCPUs, 16 GB of memory, and CentOS 7.5 64-bit image
  • Replica set specifications: All specifications described in Table 1
    Table 1 Replica set specifications

    Replica Set No.

    DB Version

    Specifications

    Replica set 1

    4.0

    4 vCPUs | 16 GB

    Replica set 2

    4.0

    8 vCPUs | 32 GB

    Replica set 3

    4.0

    16 vCPUs | 64 GB

    Replica set 4

    4.0

    32 vCPUs | 128 GB

Test Tool

YCSB is an open-source tool for testing performance of databases. In this test, YCSB 0.12.0 is used.

For details on how to use this tool, see YCSB.

Test Metrics

Operations per Second (OPS): operations executed by a database per second

Test Procedure

  1. Configure the workload file.

    Set values for fields readproportion, insertproportion, and updateproportion in the workload file by referring to Table 2.

    Set a value for field recordcount in the workload file by referring to Table 3.

  2. Use workload_s1 as an example. Run the following command to prepare test data:

    ./bin/ycsb load mongodb -s -P workloads/workload_s1 -p mongodb.url=mongodb://${userName}:${password}@${mongodIP}:${port},${mongodIP}:${port},${mongodIP}:${port}/ycsb?authSource=admin&replicaSet=replica&readPreference=secondary -threads ${threadNum} 1>workload_s1_load.result 2> workload_s1_load.log

  3. Run the following command to test performance:

    ./bin/ycsb run mongodb -s -P workloads/workload_s1 -p mongodb.url=mongodb://${userName}:${password}@${mongodIP}:${port},${mongodIP}:${port},${mongodIP}:${port}/ycsb?authSource=admin&replicaSet=replica&readPreference=secondary -threads ${threadNum} -p maxexecutiontime=1800 1>workload_s1_run.result 2> workload_s1_run.log

    • ${mongodIP} indicates private IP addresses of primary and secondary nodes in the GeminiDB Mongo replica set.
    • ${password} is the administrator password of the GeminiDB Mongo replica set.
    • ${threadNum} indicates concurrent threads for running the test. In this test, the number of concurrent threads is 128.

Test Models

  • Workload model
    Table 2 Workload models

    No.

    Workload model

    S1

    100% insert

    S2

    90% update, 10% read

    S3

    65% read, 25% insert, 10% update

    S4

    90% read, 5% insert, 5% update

    S5

    50% update, 50% read

    S6

    100% read

  • Concurrent threads: 128
  • Document model

    Use the default settings of YCSB: The size of each document is 1 KB, and the default index is _id.

  • Preset data volume

    Two types of preset data volumes were used to test performance of replica sets of each type of specifications.

    The following table describes preset data volumes for different replica set specifications.

    Table 3 Preset data volumes

    Replica Set No.

    Specifications

    Small Data Volume

    Large Data Volume

    Replica set 1

    4 vCPUs | 16 GB

    Storage space: 10 GB

    RecordCount: 10000000

    Storage space: 100 GB

    RecordCount: 100000000

    Replica set 2

    8 vCPUs | 32 GB

    Storage space: 10 GB

    RecordCount: 10000000

    Storage space: 100 GB

    RecordCount: 100000000

    Replica set 3

    16 vCPUs | 64 GB

    Storage space: 10 GB

    RecordCount: 10000000

    Storage space: 100 GB

    RecordCount: 100000000

    Replica set 4

    32 vCPUs | 128 GB

    Storage space: 10 GB

    RecordCount: 10000000

    Storage space: 100 GB

    RecordCount: 100000000

  • Data consistency model

    Weak consistency: If Write Concern is set to {w : 1, j : false}, a response is returned after data is written to disks on a single node. If the default setting is kept Write Concern, data is asynchronously stored on disks.