Accessing a GeminiDB Redis Instance Using a Pipeline

Pipeline

GeminiDB Redis API is a request–reply model service.

Figure 1 Command execution process of GeminiDB Redis API

1. A client sends a command to a GeminiDB Redis server.
2. The GeminiDB Redis server receives the command and waits in a queue for processing.
3. The GeminiDB Redis server processes the command.
4. The GeminiDB Redis server sends the result to the client.

Steps 1 and 4 are I/O operations, which are slow and affected by network conditions. Therefore, bottlenecks may occur.

To reduce network costs and fully utilize performance of GeminiDB Redis API, you can execute multiple commands in a pipeline at once without waiting for individual command reply.

Figure 2 Accessing a GeminiDB Redis instance using a pipeline

The I/O operation is performed once to issue three commands.

Using a pipeline can reduce read() and write() system calls of a client and server and improve program execution efficiency.

Pipeline Size Selection and Precautions

Although pipelines can reduce I/O costs, a bigger pipeline size is not always better. Pipelines have a limit on improving program performance. If there are large amounts of data in a pipeline, the client has to wait longer time. If the socket buffer is full due to a large pipeline, network congestion may occur, causing performance deterioration.

Based on experience, 30 to 100 commands in a pipeline can fully utilize database performance. An optimal size is subject to the actual test result.

Other precautions:

• Pipelines do not guarantee atomicity. When processing batch commands, a server parses and executes commands in sequence. These commands are independent of each other. The server may also execute commands of other clients during this process. If a command fails, other commands are not affected. To achieve atomicity, you can execute transactions or Lua scripts.
• A GeminiDB Redis instance must cache the results before finishing processing all commands, so a large pipeline may cause out of memory (OOM) or network congestion. If a single command is too long, the pipeline size needs to be reduced.
• To fulfill stringent requirements on low latency, large pipelines are not recommended.