Compact Encoding for Small Objects

GeminiDB uses RocksDB as the storage engine. RocksDB only operates on a key-value (KV) storage model. As a result, any data structure must first be transformed into the KV format before it can be stored, necessitating the use of data encoding techniques. In a conventional encoding scheme, storing a hash object with N field-value pairs necessitates N+1 KV pairs. This means executing the HGETALL command demands N+1 read operations. In certain service contexts, like RTA, where hash data consists of a limited number of elements, each with minimal byte size, the GeminiDB hash compact encoding scheme can significantly enhance read performance. The compact encoding scheme encodes N field-value pairs and one metadata pair into a single KV pair, reducing the number of read times and greatly improving the performance of HGETALL and HMGET. Currently, the compact encoding scheme supports only small object storage optimization for the hash data type.

Scenarios

With compact encoding schemes, all field-value pairs are stored into a single KV pair. Therefore, compact encoding is not suitable for a single large hash object but is ideal for hash data that meets the following requirements:

The size of a single hash object is less than or equal to 1 KB.
The number of field-value pairs in a single hash object ranges from 10 to 50.

Usage Notes

Compact encoding supports only the hash data type.
Compact encoding is not suitable for large objects. If it is used for large objects, the latency of the HSET and HMSET commands may increase.
Encoding degradation is not supported. Compact encoding takes effect only for new objects.

Configuration and Usage

Enabling or disabling compact encoding

Run CONFIG SET to dynamically adjust the small-hashobj-encode-size parameter by following Connecting to an Instance Using a Load Balancer Address (Recommended) to determine whether to enable compact encoding.

Parameter	small-hashobj-encode-size
Unit	Byte
Default Value	0: Compact encoding is disabled. Traditional encoding is used.
Recommended Value	1024
Description	0: Compact encoding is disabled. All hash objects are stored in conventional encoding mode. 1024: Compact encoding is enabled. This configuration takes effect only when the size of a hash object is less than or equal to 1024 bytes. If the size of a hash object exceeds 1024 bytes, conventional encoding is used.

Viewing the encoding format of an object
Run the following command to view the storage format of a specified key:
```
OBJECT STORAGEFORMAT <key>
```

Manually converting the encoding format of an object
Run the following command to convert the compact encoding key back to the conventional encoding key:
```
OBJECT REFORMAT <key> NORMAL
```
Conventional encoding cannot be converted for small objects.