MOTService Basic Principles

Overview

MOTService is an in-memory table engine developed based on GaussDB(for openGauss). It features high throughput and low latency, and further improves performance based on the high-performance, high-security, and high-reliability enterprise-level relational database capabilities of GaussDB(for openGauss). It supports transactions and complete transaction ACID features. In FusionInsight RTD, MOTService provides data storage, rule calculation, and data query services for RTDService.

Principles

MOTService is an in-memory table engine developed based on GaussDB(for openGauss). It is essentially an OLTP standalone database. It optimizes execution, precompilation of stored procedures, and optimistic locking of MVCC, and achieves millisecond-level latency and thousand-level TPS in RTDService's rule calculation.

Figure 1 MOTService structure

• Stored procedure precompilation: Based on LLVM, stored procedures are precompiled to a format that can be directly invoked locally. This skips multi-layer database processing logic and significantly improves performance. The precompilation results of stored procedures are cached in the memory and can be invoked by subsequent sessions like the pointers in C. For the same stored procedure, the precompilation result can be reused even if the request comes from different sessions or parameters.
• Execution optimization: MOTService provides faster data access and more efficient transaction execution through data and indexes completely stored in memory, non-uniform memory access-aware (NUMA-aware) design, algorithms that eliminate locks and lock contention, and query native compilation. In addition, MOTService indexes are based on the state-of-the-art lock-free indexing of Masstree for fast and scalable key-value (KV) storage of multi-core systems, which is implemented through the Trie of a B+ tree. It achieves excellent performance on multi-core servers and high concurrent workloads.

• MVCC optimistic locking: An optimistic concurrency control (OCC) lock is introduced based on the Silo database. The database does not block in the read/write phase and conflict detection and retry are performed only in the transaction submission phase, greatly reducing the blocking time. Optimistic locking is less expensive and often more efficient, because transaction conflicts are not common in most applications.

Relationship with Other Components

You can define stored procedure rules and real-time query variables using the web UI provided by RTDService. The variables and rules are compiled in real time to generate compilation processes and deployed these processes on the MOTService database. After the event source dimension mapping is brought online, the corresponding BLU execution rule accesses the defined stored procedure of the MOTService in real time.