Decoupled Storage and Compute

Why You Need Decoupled Storage and Compute

In traditional database architectures, compute and storage are tightly coupled, making resources fixed and difficult to adjust. Enterprises must provision resources for peak workloads, resulting in idle and wasted capacity during normal operations. When traffic surges unexpectedly, system performance can degrade sharply, even causing interruptions. The mismatch between fluctuating workloads and fixed resource allocation has become a major barrier to improving IT efficiency.

Enterprises face a dilemma. They must control IT costs and avoid resource waste, while also ensuring stable performance under heavy loads to maintain service continuity and user experience. This conflict becomes critical during sudden traffic spikes, such as e-commerce promotions, holiday shopping peaks, and marketing campaigns, which directly affect business revenue and customer satisfaction.

Huawei Cloud TaurusDB adopts an innovative decoupled storage-compute architecture that eliminates the limitations of traditional tightly coupled designs. Separating the compute and storage layers allows you to scale and manage each layer independently.

• Compute-layer auto scaling: dynamically adjusts compute resources based on real-time workloads. The system scales in seconds to precisely match fluctuating business demands.
• Independent storage scaling: scales storage capacity linearly on demand, completely decoupled from compute node limitations. This architecture seamlessly accommodates massive data growth and ensures on-demand resource allocation.

This architecture improves resource utilization, reduces TCO, and minimizes operational overhead, enabling enterprises to focus on innovation and respond quickly to shifting market demands.

Advantages

• Decoupled compute and storage: The compute and storage layers are architecturally independent, allowing for separate management and scaling without impacting each other.
• Independent resource scaling: Compute and storage resources can be flexibly adjusted based on workload needs. Storage resources can be expanded independently.
• Highly available architecture: Redundant replicas and redundant storage ensure data security and reliability.
• High resource utilization: Fixed resource allocation is avoided, improving overall utilization.

Use Cases

• E-commerce applications: During promotions, e-commerce platforms face sharp traffic spikes and fluctuating daily access volumes, requiring high concurrency handling. TaurusDB's decoupled storage-compute architecture allows compute resources to scale independently to handle peak traffic while maintaining storage stability and consistency. This enables the system to flexibly adapt to burst traffic, ensuring stable operations of order processing, inventory management, and user data services. It avoids performance bottlenecks caused by coupled architectures and ensures business continuity during promotions.
• Mobile application backend services: As mobile user bases scale rapidly and unpredictably, backend services must guarantee low latency and highly responsive performance. TaurusDB's decoupled architecture supports backend services such as user data management, message push, and content storage. The storage layer scales independently to ensure high data availability, and the compute layer dynamically adjusts resources based on load. This simplifies O&M and enables quick responses to workload changes. Storage and compute resources can also be optimized separately for durability and performance, improving user experience and response speed.
• Data analysis platforms: Data analysts often process large volumes of historical data with complex and variable query patterns, causing significant fluctuations in compute demand. The decoupled architecture supports data storage and analysis for such platforms. The storage layer is optimized for large datasets to ensure durability and efficient access, and the compute layer scales dynamically based on query complexity and concurrency. Compute engines can share the same storage layer, improving data sharing and reuse. Storage and compute resources can be upgraded independently without affecting service availability, significantly enhancing processing efficiency and analytical flexibility.

How Decoupled Storage and Compute Works

The architecture consists of three layers from top to bottom: compute layer, storage abstraction layer (SAL), and storage layer.

• Compute layer: It includes one primary node and multiple read replicas. The primary node handles read/write operations, and read replicas share read load to improve performance and availability.
• SAL: Positioned between the compute layer and the storage nodes, the SAL provides log and data storage services for the InnoDB layer. It abstracts traditional table-file operations into distributed storage, integrates downwards with DFV, and delivers database storage services upwards to InnoDB for efficient scheduling. This layer is central to achieving high database performance.
• Storage layer: Built on Huawei DFV storage, this layer provides distributed, strongly consistent, high-performance storage. It supports seamless horizontal scaling and guarantees data durability of at least 99.999999999%. It includes two main sub-services:
  • Log Store: a service for log persistence
  • Page Store: a storage service providing consistent, multi-version page capabilities Page data is distributed across shards (slices), each with a fixed size of 10 GB.

Figure 1 Architecture

Decoupled Storage and Compute vs. Coupled Storage and Compute

They represent two distinct database architectures. Both provide data storage and compute capabilities, with the same core goals: ensuring data security and delivering efficient data access and processing. Their application scenarios largely overlap, both supporting online transaction processing (OLTP), online analytical processing (OLAP), and other workloads. They also share similar foundational capabilities, offering robust support for transaction processing, data persistence, and backup and recovery to sustain stable business operations.

Core Differences

1. Resource expansion mode
   • Decoupled storage and compute: Storage and compute resources are fully separated. Storage capacity or compute power can be expanded independently based on workload needs, without affecting each other.
   • Coupled storage and compute: Storage and compute resources are tightly bound and expanded as a single unit. Any resource modification must scale both components simultaneously.
2. Resource utilization
   • Decoupled storage and compute: Storage and compute resources can be optimized independently according to actual workloads, preventing idle capacity and improving overall utilization.
   • Coupled storage and compute: Infrastructure must be provisioned for peak demand, which frequently leads to idle capacity or resource starvation, resulting in lower overall utilization.
3. Fault isolation and recovery
   • Decoupled storage and compute: Faults in the storage and compute layers are isolated. A compute node failure does not make the entire system unavailable, and recovery is more flexible.
   • Coupled storage and compute: Storage and compute are tightly integrated. A component failure in either layer may affect the entire system's availability, and recovery is more complex.

Reason for the Differences

The fundamental reasons behind these differences are as follows:

1. Different design concepts: Decoupled storage and compute is based on resource separation and independent optimization, aiming to maximize resource utilization and system scalability. Coupled storage and compute follows an integrated design concept that prioritizes simplified setups and unified O&M.
2. Different technical architectures: Decoupled storage and compute relies on distributed storage and computing technologies to fully separate the two layers. Coupled storage and compute typically uses traditional monolithic designs that tightly bind storage and compute.