Test Method

This section describes performance testing of GeminiDB Redis instances, including the test environment, tools, metrics, models, and procedure.

Test Environment

Region: CN-Hong Kong
AZ: AZ1
Elastic Cloud Server (ECS): c6.4xlarge.2 with 16 vCPUs, 32 GB of memory, and CentOS 7.5 64-bit image
Nodes per instance: 3

Instance specifications: Specifications described in Table 1

**Table 1** Instance specifications
No.	Specifications
Cluster 1	4 vCPUs x 3 nodes
Cluster 2	8 vCPUs x 3 nodes

Test Tool

This test used a multi-thread load test tool, memtier_benchmark, developed by Redis Labs. For details about how to use this tool, see memtier_benchmark. The following describes some functions of memtier_benchmark.

Usage: memtier_benchmark [options]

A memcache/redis NoSQL traffic generator and performance benchmarking tool.

Connection and General Options:
    -s, --server=ADDR                         Server address (default: localhost)
    -p, --port=PORT                           Server port (default: 6379)
    -a, --authenticate=PASSWORD               Authenticate to redis using PASSWORD
    -o, --out-file=FILE                       Name of output file (default: stdout)

Test Options:
    -n, --requests=NUMBER                     Number of total requests per client (default: 10000)
    -c, --clients=NUMBER                      Number of clients per thread (default: 50)
    -t, --threads=NUMBER                      Number of threads (default: 4)
        --ratio=RATIO                         Set:Get ratio (default: 1:10)
        --pipeline=NUMBER                     Number of concurrent pipelined requests (default: 1)
        --distinct-client-seed                Use a different random seed for each client
        --randomize                           Random seed based on timestamp (default is constant value)

Object Options:
    -d --data-size=SIZE                       Object data size (default: 32)
    -R --random-data                          Indicate that data should be randomized

Key Options:
    --key-prefix=PREFIX                       Prefix for keys (default: memtier-)
    --key-minimum=NUMBER                      Key ID minimum value (default: 0)
    --key-maximum=NUMBER                      Key ID maximum value (default: 10000000)

Test Metrics

**Table 2** Test metrics
Metric Abbreviation	Description
QPS	Number of read and write operations executed per second.
Avg Latency	Average latency of read and write operations, in milliseconds.
p99 Latency	p99 latency of read and write operations. 99% of operations can be completed within this latency. Only 1% of operations have a latency longer. Unit: ms.

Test Models

Workload model

**Table 3** Workload models
Workload Model	Description
100% Write	100% write operations (string set)
100% Read	100% read operations (string get). The even random access model is used in strict performance tests.
50% Read+50% Write	50% read operations (string get) plus 50% write operations (string set)

Data model

**Table 4** Data model description
Data Model	Description
value length	A value in 100 bytes is generated randomly.

Test scenarios

**Table 5** Test scenario description
Test Scenario	Description
The data volume is less than memory.	All data can be cached in memory.
The data volume is larger than memory.	Some data can be cached in memory, and some data can be accessed from the DFV storage pool.

Test Procedure

Use a DB instance with 3 nodes and 4 vCPUs for each node as an example:

Scenario 1: When the data volume is less than memory, data is written to and read data from the instance respectively and then concurrently, and record OPS, average latency, and P99 latency of each operation. Workload models and methods of testing performance metrics are as follows:

Workload model: 100% write
Use 30 threads and 3 client connections for each thread. That is, 100-byte data is written 60,000,000 times on total 90 connections. The data is generated randomly by all the clients using different seeds within the range of [1, 60,000,000]. Based on the specified range of keys, the total size of data written this time is less than the memory of the database cluster.
```
./memtier_benchmark -s ${ip} -a ${passwd} -p ${port} -c 3 -t 30 -n 1000000 --random-data --randomize --distinct-client-seed -d 100 --key-maximum=60000000 --key-minimum=1 --key-prefix= --ratio=1:0 --out-file=./output_filename
```

Workload model: 100% read

Use 30 threads and 3 client connections for each thread. That is, data is randomly and concurrently read 60,000,000 times over 90 connections. The key is within the range of [1, 60,000,000].

./memtier_benchmark -s ${ip} -a ${passwd} -p ${port} -c 3 -t 30 -n 1000000 --random-data --randomize --distinct-client-seed --key-maximum=60000000 --key-minimum=1 --key-prefix= --ratio=0:1 --out-file=./output_filename

Workload model: 50% read and 50% write
Use 30 threads and 3 client connections for each thread. That is, data is randomly and concurrently written and read 60,000,000 times over 90 connections. The key is within the range of [1, 60,000,000]. The write-read ratio is 1:1. Based on the specified range of keys, the total size of data written and read this time was less than the memory of the database cluster.
```
./memtier_benchmark -s ${ip} -a ${passwd} -p ${port} -c 3 -t 30 -n 1000000 --random-data --randomize --distinct-client-seed -d 100 --key-maximum=60000000 --key-minimum=1 --key-prefix= --ratio=1:1 --out-file=./output_filename
```

2. Scenario 2: When the data volume is larger than memory of the database cluster, use 30 threads and create 3 client connections for each thread. That is, 100-byte data is concurrently written 20,000,000 times over total 90 connections. The data is generated randomly by all the clients using different seeds within the range of [60,000,001, 780,000,000]. In addition, pipeline parameters were set to speed up data writes. Based on the specified range of keys and total writes, the total size of data written this time was larger than the memory of the database cluster.

./memtier_benchmark -s ${ip} -a ${passwd} -p ${port} -c 3 -t 30 -n 20000000 --random-data --randomize --distinct-client-seed -d 100 --key-maximum=780000000 --key-minimum=60000001 --pipeline=100 --key-prefix= --ratio=1:0 --out-file=./output_filename

3. When the data volume is larger than memory, data is written to and read from the database cluster respectively and then concurrently, and metrics OPS, average latency, and p99 latency of each operation were recorded. Workload models and methods of testing performance metrics are as follows:

Test model: 100% write
Use 30 threads and 3 clients for each thread. That is, 100-byte data is written 500,000 times over total 90 connections. The data is generated randomly by all the clients within the range of [1, 780,000,000].
```
./memtier_benchmark -s ${ip} -a ${passwd} -p ${port} -c 3 -t 30 -n 500000 --random-data --randomize --distinct-client-seed -d 100 --key-maximum=780000000 --key-minimum=1 --key-prefix= --ratio=1:0 --out-file=./output_filename
```

Test model: 100% read

Use 30 threads and 3 clients for each thread. That is, data is randomly and concurrently read 500,000 times over 90 connections. The key is within the range of [1, 780,000,000].

./memtier_benchmark -s ${ip} -a ${passwd} -p ${port} -c 3 -t 30 -n 500000 --random-data --randomize --distinct-client-seed --key-maximum=780000000 --key-minimum=1 --key-prefix= --ratio=0:1 --out-file=./output_filename

Test model: 50% read and 50% write
Use 30 threads and 3 clients for each thread. That is, data is written and read 500,000 times over total 90 connections. The data is generated randomly by all the clients within the range of [1, 780,000,000].
```
./memtier_benchmark -s ${ip} -a ${passwd} -p ${port} -c 3 -t 30 -n 500000 --random-data --randomize --distinct-client-seed -d 100 --key-maximum=780000000 --key-minimum=1 --key-prefix= --ratio=1:1 --out-file=./output_filename
```

Parent topic: General Performance Data

Previous topic: General Performance Data

Next topic: Test Result