What Is Cold and Hot Data Separation?

Scenarios

If there are some tables with almost no data reads or writes in your DB instance, you can use cold and hot data separation to dump their data to OBS. This helps reduce costs by managing cold and hot data more efficiently.

Introduction Video

How It Works

TaurusDB allows you to dump infrequently used InnoDB tables to OBS. These dumped tables are called cold tables.

Figure 1 How it works

As shown in the figure, the cold and hot data separation architecture supports dynamic data migration between a TaurusDB instance and OBS to balance performance and storage costs. The core workflow is as follows:

• Data dictionary: Stores database metadata, such as table schemas, index definitions, and user permissions. It serves as the database's directory. Regardless of where the data itself resides, the metadata must remain in the data dictionary so that the database engine can locate and manage the data.
• Data files: Store active hot data that requires fast access, along with its associated indexes.

When infrequently accessed cold data needs to be archived, the system creates a cold table, migrating the corresponding data and index files from a TaurusDB instance to OBS. This frees up storage space and reduces storage costs. The metadata remains in the instance's data dictionary, allowing the database to continue recognizing and managing the table. When the application needs to modify or query the data frequently again, the system restores the cold table, migrating the data files from OBS back to the TaurusDB instance to restore high-performance access. The entire process is transparent to applications and requires no code changes, enabling intelligent tiered storage in which hot data delivers fast responses while cold data is stored at a lower cost.

Constraints

• Cold table data cannot be migrated using DRS. During migration, if you want to retain cold table data of a source database, migrate cold tables back to the source database and then migrate the database using DRS.
• It takes longer to query data in cold tables. If there are too many cold tables, a large number of slow query logs may be generated.
• To prevent an operation failure, do not create or migrate back a cold table when any other operation is being performed on the instance.

Billing

Cold data is stored on OBS and billed based on the amount of data in cold tables. It does not occupy the backup space. For details, see "Backup Storage Price" in TaurusDB Pricing Details. You can view the cold table size by referring to Collecting Cold Table Statistics.

How to Use

• Dumping cold table data

  You can create cold tables on the console. The data of the cold tables is stored on OBS, freeing up storage space and reducing storage costs. For details, see Configuring a Cold Table on the Console.

• Querying cold table data

  Just like querying data from any ordinary table, you can run SELECT statements to query cold table data. For details, see Configuring a Cold Table Using SQL Statements.

  The rds_schs_enable_partition_visible parameter controls whether data in archived partitions is visible to SELECT queries. The default value is ON. To modify its value, see Modifying Instance Parameters.

  When the parameter is set to ON, data in archived partitions is visible to SELECT queries. When set to OFF, such data is not visible to SELECT queries.

• Modifying cold table data

  To modify a cold table that has been dumped to OBS, you can migrate the table back to your instance on the console, but the data of the table is still stored in OBS. To delete data from OBS, submit a service ticket.

• Deleting cold table data

  After a cold table is created, DDL statements cannot be executed on the cold table. This means that the table cannot be deleted directly, and neither can the database it belongs to.

  To delete a cold table, migrate it back and run the drop command.