Compute
Elastic Cloud Server
Huawei Cloud Flexus
Bare Metal Server
Auto Scaling
Image Management Service
Dedicated Host
FunctionGraph
Cloud Phone Host
Huawei Cloud EulerOS
Networking
Virtual Private Cloud
Elastic IP
Elastic Load Balance
NAT Gateway
Direct Connect
Virtual Private Network
VPC Endpoint
Cloud Connect
Enterprise Router
Enterprise Switch
Global Accelerator
Management & Governance
Cloud Eye
Identity and Access Management
Cloud Trace Service
Resource Formation Service
Tag Management Service
Log Tank Service
Config
OneAccess
Resource Access Manager
Simple Message Notification
Application Performance Management
Application Operations Management
Organizations
Optimization Advisor
IAM Identity Center
Cloud Operations Center
Resource Governance Center
Migration
Server Migration Service
Object Storage Migration Service
Cloud Data Migration
Migration Center
Cloud Ecosystem
KooGallery
Partner Center
User Support
My Account
Billing Center
Cost Center
Resource Center
Enterprise Management
Service Tickets
HUAWEI CLOUD (International) FAQs
ICP Filing
Support Plans
My Credentials
Customer Operation Capabilities
Partner Support Plans
Professional Services
Analytics
MapReduce Service
Data Lake Insight
CloudTable Service
Cloud Search Service
Data Lake Visualization
Data Ingestion Service
GaussDB(DWS)
DataArts Studio
Data Lake Factory
DataArts Lake Formation
IoT
IoT Device Access
Others
Product Pricing Details
System Permissions
Console Quick Start
Common FAQs
Instructions for Associating with a HUAWEI CLOUD Partner
Message Center
Security & Compliance
Security Technologies and Applications
Web Application Firewall
Host Security Service
Cloud Firewall
SecMaster
Anti-DDoS Service
Data Encryption Workshop
Database Security Service
Cloud Bastion Host
Data Security Center
Cloud Certificate Manager
Edge Security
Situation Awareness
Managed Threat Detection
Blockchain
Blockchain Service
Web3 Node Engine Service
Media Services
Media Processing Center
Video On Demand
Live
SparkRTC
MetaStudio
Storage
Object Storage Service
Elastic Volume Service
Cloud Backup and Recovery
Storage Disaster Recovery Service
Scalable File Service Turbo
Scalable File Service
Volume Backup Service
Cloud Server Backup Service
Data Express Service
Dedicated Distributed Storage Service
Containers
Cloud Container Engine
SoftWare Repository for Container
Application Service Mesh
Ubiquitous Cloud Native Service
Cloud Container Instance
Databases
Relational Database Service
Document Database Service
Data Admin Service
Data Replication Service
GeminiDB
GaussDB
Distributed Database Middleware
Database and Application Migration UGO
TaurusDB
Middleware
Distributed Cache Service
API Gateway
Distributed Message Service for Kafka
Distributed Message Service for RabbitMQ
Distributed Message Service for RocketMQ
Cloud Service Engine
Multi-Site High Availability Service
EventGrid
Dedicated Cloud
Dedicated Computing Cluster
Business Applications
Workspace
ROMA Connect
Message & SMS
Domain Name Service
Edge Data Center Management
Meeting
AI
Face Recognition Service
Graph Engine Service
Content Moderation
Image Recognition
Optical Character Recognition
ModelArts
ImageSearch
Conversational Bot Service
Speech Interaction Service
Huawei HiLens
Video Intelligent Analysis Service
Developer Tools
SDK Developer Guide
API Request Signing Guide
Terraform
Koo Command Line Interface
Content Delivery & Edge Computing
Content Delivery Network
Intelligent EdgeFabric
CloudPond
Intelligent EdgeCloud
Solutions
SAP Cloud
High Performance Computing
Developer Services
ServiceStage
CodeArts
CodeArts PerfTest
CodeArts Req
CodeArts Pipeline
CodeArts Build
CodeArts Deploy
CodeArts Artifact
CodeArts TestPlan
CodeArts Check
CodeArts Repo
Cloud Application Engine
MacroVerse aPaaS
KooMessage
KooPhone
KooDrive
On this page
Help Center/ Distributed Cache Service/ Troubleshooting/ Troubleshooting High Memory Usage of a DCS Redis Instance

Troubleshooting High Memory Usage of a DCS Redis Instance

Updated on 2023-11-27 GMT+08:00

Symptom

Redis provides fast database services. If the memory is insufficient, keys may be frequently evicted, the response time may increase, and the QPS may be unstable, affecting service running. This is normal due to Redis functions (such as master/replica replication and lazyfree). When the memory becomes full, scale up the instance or remove unnecessary data. Generally, you need to be alerted when the memory usage exceeds 95%.

Fault Locating

  1. Query the memory usage in a specified period. For details, see Viewing Metrics. Check whether the value of Memory Usage is close to 100% continuously.
  2. If the values of Evicted Keys and Maximum Command Latency increase significantly during the period when the memory usage exceeds 95%, the memory is insufficient.

    In this case, log in to the console and analyze big keys and slow queries by referring to Cache Analysis and Viewing Redis Slow Queries. If no expiration is set for the instance, too much data will be stored in the instance, using up the memory.

  3. If the memory of a Redis instance is full but there are not many keys, the output buffer may have occupied an excessive amount of memory.

    In this case, run the redis-cli --bigkeys command to scan for big keys after connecting to the instance using redis-cli. Then, run the info command to check the output buffer size.

Solution

  1. Perform big key and hot key analysis on the DCS console, and take measures accordingly. For details, see Analyzing Big Keys and Hot Keys.
    NOTE:

    You can configure alarms to detect hot keys for DCS Redis 3.0 instances, which do not support hot key analysis.

  2. Scan for expired keys and release them, or manually delete unnecessary keys.
  3. Other suggestions:
    • The value of String must be less than or equal to 10 KB.
    • For data structures such as Hash, List, Set, and Zset, it is recommended that the number of elements in a single key be less than or equal to 5000.
    • When naming keys, use the service name abbreviation as the prefix and do not use special characters such as spaces, line brakes, single or double quotation marks, and other escape characters.
    • Do not rely too much on Redis transactions.
    • The performance of short connections ("connect" in Redis terminology) is poor. Use clients with connection pools.
    • If the data is used only for data cache and data loss is tolerated, you are advised to disable the persistence function. (Change the value of appendonly to no in the instance parameter configuration to disable the AOF persistence function.)
    • Configure alarms to detect big keys and hot keys in advance.
      • Configure an alarm for node-level memory usage. For details, see Configuring Alarm Rules for Critical Metrics.

        If a node has a big key, the memory usage of the node is much higher than that of other nodes. In this case, an alarm is triggered to help you find the problematic key.

      • Configure alarms for node-level Maximum Inbound Bandwidth, Maximum Outbound Bandwidth, and CPU Usage. For details, see Configuring Alarm Rules for Critical Metrics.

        If a node has a hot key, the bandwidth and CPU usage of the node is much higher than that of other nodes. In this case, an alarm is triggered to help you find the problematic key.

  4. If the instance memory usage remains high after you take the preceding measures, expand the instance specifications during off-peak hours. For details, see Modifying Specifications.

We use cookies to improve our site and your experience. By continuing to browse our site you accept our cookie policy. Find out more

Feedback

Feedback

Feedback

0/500

Selected Content

Submit selected content with the feedback