Compute
Elastic Cloud Server
Huawei Cloud Flexus
Bare Metal Server
Auto Scaling
Image Management Service
Dedicated Host
FunctionGraph
Cloud Phone Host
Huawei Cloud EulerOS
Networking
Virtual Private Cloud
Elastic IP
Elastic Load Balance
NAT Gateway
Direct Connect
Virtual Private Network
VPC Endpoint
Cloud Connect
Enterprise Router
Enterprise Switch
Global Accelerator
Management & Governance
Cloud Eye
Identity and Access Management
Cloud Trace Service
Resource Formation Service
Tag Management Service
Log Tank Service
Config
OneAccess
Resource Access Manager
Simple Message Notification
Application Performance Management
Application Operations Management
Organizations
Optimization Advisor
IAM Identity Center
Cloud Operations Center
Resource Governance Center
Migration
Server Migration Service
Object Storage Migration Service
Cloud Data Migration
Migration Center
Cloud Ecosystem
KooGallery
Partner Center
User Support
My Account
Billing Center
Cost Center
Resource Center
Enterprise Management
Service Tickets
HUAWEI CLOUD (International) FAQs
ICP Filing
Support Plans
My Credentials
Customer Operation Capabilities
Partner Support Plans
Professional Services
Analytics
MapReduce Service
Data Lake Insight
CloudTable Service
Cloud Search Service
Data Lake Visualization
Data Ingestion Service
GaussDB(DWS)
DataArts Studio
Data Lake Factory
DataArts Lake Formation
IoT
IoT Device Access
Others
Product Pricing Details
System Permissions
Console Quick Start
Common FAQs
Instructions for Associating with a HUAWEI CLOUD Partner
Message Center
Security & Compliance
Security Technologies and Applications
Web Application Firewall
Host Security Service
Cloud Firewall
SecMaster
Anti-DDoS Service
Data Encryption Workshop
Database Security Service
Cloud Bastion Host
Data Security Center
Cloud Certificate Manager
Edge Security
Situation Awareness
Managed Threat Detection
Blockchain
Blockchain Service
Web3 Node Engine Service
Media Services
Media Processing Center
Video On Demand
Live
SparkRTC
MetaStudio
Storage
Object Storage Service
Elastic Volume Service
Cloud Backup and Recovery
Storage Disaster Recovery Service
Scalable File Service Turbo
Scalable File Service
Volume Backup Service
Cloud Server Backup Service
Data Express Service
Dedicated Distributed Storage Service
Containers
Cloud Container Engine
SoftWare Repository for Container
Application Service Mesh
Ubiquitous Cloud Native Service
Cloud Container Instance
Databases
Relational Database Service
Document Database Service
Data Admin Service
Data Replication Service
GeminiDB
GaussDB
Distributed Database Middleware
Database and Application Migration UGO
TaurusDB
Middleware
Distributed Cache Service
API Gateway
Distributed Message Service for Kafka
Distributed Message Service for RabbitMQ
Distributed Message Service for RocketMQ
Cloud Service Engine
Multi-Site High Availability Service
EventGrid
Dedicated Cloud
Dedicated Computing Cluster
Business Applications
Workspace
ROMA Connect
Message & SMS
Domain Name Service
Edge Data Center Management
Meeting
AI
Face Recognition Service
Graph Engine Service
Content Moderation
Image Recognition
Optical Character Recognition
ModelArts
ImageSearch
Conversational Bot Service
Speech Interaction Service
Huawei HiLens
Video Intelligent Analysis Service
Developer Tools
SDK Developer Guide
API Request Signing Guide
Terraform
Koo Command Line Interface
Content Delivery & Edge Computing
Content Delivery Network
Intelligent EdgeFabric
CloudPond
Intelligent EdgeCloud
Solutions
SAP Cloud
High Performance Computing
Developer Services
ServiceStage
CodeArts
CodeArts PerfTest
CodeArts Req
CodeArts Pipeline
CodeArts Build
CodeArts Deploy
CodeArts Artifact
CodeArts TestPlan
CodeArts Check
CodeArts Repo
Cloud Application Engine
MacroVerse aPaaS
KooMessage
KooPhone
KooDrive
Help Center/ Distributed Cache Service/ FAQs/ Instance Types/Versions/ New Features of DCS for Redis 5.0

New Features of DCS for Redis 5.0

Updated on 2022-08-10 GMT+08:00

DCS for Redis 5.0 is compatible with the new features of the open-source Redis 5.0, in addition to all the improvements and new commands in Redis 4.0.

Stream Data Structure

Stream is a new data type introduced with Redis 5.0. It supports message persistence and multicast.

Figure 1 shows the structure of a Redis stream, which allows messages to be appended to the stream.

Streams have the following features:
  1. A stream can have multiple consumer groups.
  2. Each consumer group contains a Last_delivered_id that points to the last consumed item (message) in the consumer group.
  3. Each consumer group contains multiple consumers. All consumers share the last_delivered_id of the consumer group. A message can be consumed by only one consumer.
  4. pending_ids in the consumer can be used to record the IDs of items that have been sent to the client, but have not been acknowledged.
  5. For detailed comparison between stream and other Redis data structures, see Table 1.
Figure 1 Stream data structure
Table 1 Differences between streams and existing Redis data structures

Item

Stream

List, Pub/Sub, Zset

Complexity of seeking items

O(log(N))

List: O(N)

Offset

Supported. Each item has a unique ID. The ID is not changed as other items are added or evicted.

List: Not supported. If an item is evicted, the latest item cannot be located.

Persistence

Supported. Streams are persisted to AOF and RDB files.

Pub/Sub: Not supported.

Consumer group

Supported.

Pub/Sub: Not supported.

Acknowledgment

Supported.

Pub/Sub: Not supported.

Performance

Not related to the number of consumers.

Pub/Sub: Positively related to the number of clients.

Eviction

Streams are memory efficient by blocking to evict the data that is too old and using a radix tree and listpack.

Zset consumes more memory because it does not support inserting same items, blocking, or evicting data

Randomly deleting items

Not supported.

Zset: Supported.

Stream commands

Stream commands are described below in the order they are used. For details, see Table 2.
  1. Run the XADD command to add a stream item, that is, create a stream. The maximum number of messages that can be saved can be specified when adding the item.
  2. Create a consumer group by running the XGROUP command.
  3. A consumer uses the XREADGROUP command to consume messages.
  4. After the consumption, the client runs the XACK command to confirm that the consumption is successful.
Figure 2 Stream commands
Table 2 Stream commands description

Command

Description

Syntax

XACK

Deletes one or multiple messages from the pending entry list (PEL) a consumer group of the stream.

XACK key group ID [ID ...]

XADD

Adds a specified entry to the stream at a specified key. If the key does not exist, running this command will result in a key to be automatically created based on the entry.

XADD key ID field string [field string ...]

XCLAIM

Changes the ownership of a pending message, so that the new owner is the consumer specified as the command argument.

XCLAIM key group consumer min-idle-time ID [ID ...] [IDLE ms] [TIME ms-unix-time] [RETRYCOUNT count] [FORCE] [JUSTID]

XDEL

Removes the specified entries from a stream, and returns the number of entries deleted, that may be different from the number of IDs passed to the command in case certain IDs do not exist.

XDEL key ID [ID ...]

XGROUP

Manages the consumer groups associated with a stream You can use XGROUP to:

  • Create a new consumer group associated with a stream.
  • Destroy a consumer group.
  • Remove a specified consumer from a consumer group.
  • Set the consumer group last delivery ID to something else.

XGROUP [CREATE key groupname id-or-$] [SETID key id-or-$] [DESTROY key groupname] [DELCONSUMER key groupname consumername]

XINFO

Retrieves different information about the streams and associated consumer groups.

XINFO [CONSUMERS key groupname] key key [HELP]

XLEN

Returns the number of entries in a stream. If the specified key does not exist, 0 is returned, indicating an empty stream.

XLEN key

XPENDING

Obtains data from a stream through a consumer group. This command is the interface to inspect the list of pending messages in order to observe and understand what clients are active, what messages are pending to be consumed, or to see if there are idle messages.

XPENDING key group [start end count] [consumer]

XRANGE

Returns entries matching a given range of IDs.

XRANGE key start end [COUNT count]

XREAD

Reads data from one or multiple streams, only returning entries with an ID greater than the last received ID reported by the caller.

XREAD [COUNT count] [BLOCK milliseconds] STREAMS key [key ...] ID [ID ...]

XREADGROUP

A special version of the XREAD command, which is used to specify a consumer group to read from.

XREADGROUP GROUP group consumer [COUNT count] [BLOCK milliseconds] STREAMS key [key ...] ID [ID ...]

XREVRANGE

This command is exactly like XRANGE, but with the notable difference of returning the entries in reverse order, and also taking the start-end range in reverse order.

XREVRANGE key end start [COUNT count]

XTRIM

Trims the stream to a specified number of items, if necessary, evicting old items (items with lower IDs).

XTRIM key MAXLEN [~] count

Message (stream item) acknowledgement

Compared with Pub/Sub, streams not only support consumer groups, but also message acknowledgement.

When a consumer invokes the XREADGROUP command to read or invokes the XCLAIM command to take over a message, the server does not know whether the message is processed at least once. Therefore, once having successfully processed a message, the consumer should invoke the XACK command to notify the stream so that the message will not be processed again. In addition, the message is removed from PEL and the memory will be released from the Redis server.

In some cases, such as network faults, the client does not invoke XACK after consumption. In such cases, the item ID is retained in PEL. After the client is reconnected, set the start message ID of XREADGROUP to 0-0, indicating that all PEL messages and messages after last_id are read. In addition, repeated message transmission must be supported when consumers consume messages.

Figure 3 Acknowledgment mechanism

Memory Usage Optimization

The memory usage of Redis 5.0 is optimized based on the previous version.

  • Active defragmentation

    If a key is modified frequently and the value length changes constantly, Redis will allocate additional memory for the key. To achieve high performance, Redis uses the memory allocator to manage memory. Memory is not always freed up to the OS. As a result, memory fragments occur. If the fragmentation ratio (used_memory_rss/used_memory) is greater than 1.5, the memory usage is inefficient.

    To reduce memory fragments, properly plan and use cache data and standardize data writing.

    For Redis 3.0 and earlier versions, memory fragmentation problems are resolved by restarting the process regularly. It is recommended that the actual cache data does not exceed 50% of the available memory.

    For Redis 4.0, active defragmentation is supported, and memory is defragmented while online. In addition, Redis 4.0 supports manual memory defragmentation by running the memory purge command.

    For Redis 5.0, improved active defragmentation is supported with the updated Jemalloc, which is faster, more intelligent, and provides lower latency.

  • HyperLogLog implementation improvements

    A HyperLogLog is a probabilistic data structure used to calculate the cardinality of a set while consuming little memory. Redis 5.0 improves HyperLogLog by further optimizing its memory usage.

    For example: the B-tree is efficient in counting, but consumes a lot of memory. By using HyperLogLog, a lot of memory can be saved. While the B-tree requires 1 MB memory for counting, HyperLogLog needs only 1 KB.

  • Enhanced memory statistics

    The information returned by the INFO command is more detailed.

New and Better Commands

  1. Enhanced client management
    • redis-cli supports cluster management.

      In Redis 4.0 and earlier versions, the redis-trib module needs to be installed to manage clusters.

      Redis 5.0 optimizes redis-cli, integrating all cluster management functions. You can run the redis-cli --cluster help command for more information.

    • The client performance is enhanced in frequent connection and disconnection scenarios.

      This optimization is valuable when your application needs to use short connections.

  2. Simpler use of sorted sets

    ZPOPMIN and ZPOPMAX commands are added for sorted sets.

    • ZPOPMIN key [count]

      Removes and returns up to count members with the lowest scores in the sorted set stored at key. When returning multiple elements, the one with the lowest score will be the first, followed by the elements with higher scores.

    • ZPOPMAX key [count]

      Removes and returns up to count members with the highest scores in the sorted set stored at key. When returning multiple elements, the one with the lowest score will be the first, followed by the elements with lower scores.

  3. More sub-commands added to the help command

    The help command can be used to view help information, saving you the trouble of visiting redis.io every time. For example, run the following command to view the stream help information: xinfo help

    127.0.0.1:6379> xinfo help
    1) XINFO <subcommand> arg arg ... arg. Subcommands are:
    2) CONSUMERS <key> <groupname>  -- Show consumer groups of group <groupname>.
    3) GROUPS <key>                 -- Show the stream consumer groups.
    4) STREAM <key>                 -- Show information about the stream.
    5) HELP                         -- Print this help.
    127.0.0.1:6379> 
  4. redis-cli command input tips

    After you enter a complete command, redis-cli displays a parameter tip to help you memorize the syntax format of the command.

    As shown in the following figure, run the zadd command, and redis-cli displays zadd syntax in light color.

RDB Storing LFU and LRU Information

In Redis 5.0, storage key eviction policies LRU and LFU were added to the RDB snapshot file.

  • FIFO: First in, first out. The earliest stored data is evicted first.
  • LRU: Least recently used. Data that is not used for a long time is evicted first.
  • LFU: Least frequently used. Data that is least frequently used is evicted first.
NOTE:

The RDB file format of Redis 5.0 is modified and is backward compatible. Therefore, if a snapshot is used for migration, data can be migrated from the earlier Redis versions to Redis 5.0, but cannot be migrated from the Redis 5.0 to the earlier versions.

We use cookies to improve our site and your experience. By continuing to browse our site you accept our cookie policy. Find out more

Feedback

Feedback

Feedback

0/500

Selected Content

Submit selected content with the feedback