Scenarios
In the routine O&M and use of databases, data definition language (DDL) operations, such as adding columns or indexes to a table and removing indexes from a table, are common. However, for large tables, these operations may take several hours or even days, and block all DML (INSERT, UPDATE, and DELETE) operations on the original table, severely affecting workload continuity and availability.
To address this key issue, MySQL online DDL allows concurrent data reads and writes on a table during table structure changes, maximizing workload continuity and availability. This section explains how MySQL online DDL works with the InnoDB engine and the principles behind the gh-ost tool, helping you execute DDL operations more securely and efficiently.
Introduction to MySQL Online DDL
In MySQL online DDL operations, you can explicitly specify ALGORITHM to fine-tune table structure changes. The following uses adding an index as an example:
ALTER TABLE table_name ADD INDEX index_name(column_name), ALGORITHM=INPLACE;
ALGORITHM specifies the DDL execution algorithm. The options are:
- COPY: All concurrent DML operations are blocked during the DDL operation, which may cause a long database lock. So this option is not recommended.
- INPLACE: DDL operations can be performed without blocking concurrent DML operations. This option applies to most DDL operations. To check whether a DDL operation supports this algorithm, see How Do I Determine Whether a DDL Operation Supports the INSTANT or INPLACE Algorithm?.
- INSTANT: DDL operations are completed in seconds by modifying metadata in the data dictionary. There is no need to copy data or rebuild tables. These operations have almost no impact on the original table data and do not block concurrent DML operations. So this option is recommended. However, only a few DDL operations support INSTANT. To check whether a DDL operation supports this algorithm, see How Do I Determine Whether a DDL Operation Supports the INSTANT or INPLACE Algorithm?.
Introduction to gh-ost
If a DDL operation can only use the COPY algorithm (which locks a table for the entire process) and the original table is large or frequently updated, consider using gh-ost for better performance and stability.
gh-ost is an open-source tool specifically designed for MySQL databases to perform online DDL operations. It can avoid long table locks caused by traditional DDL operations, reducing the impact on workloads.
Disclaimer
gh-ost is a third-party open-source tool, not a Huawei Cloud product. To help you better use gh-ost, this document describes the usage constraints, suggestions, and principles of gh-ost. The content is for reference only. For more details, see the official document.
Constraints and Suggestions on Using gh-ost
- The table must have a unique key.
- Binlog must be enabled, the binlog format must be row, and binlog_row_image must be set to FULL. To check whether binlog is enabled, see Querying and Downloading Binlogs.
- Temporary tables and triggers are not supported.
- Use this tool during off-peak hours. Do not perform any other DDL operations while it is in use.
- If gh-ost enters the throttled state, you can increase the value of the -max-lag-millis parameter (default value: 1500, unit: ms), for example, set -max-lag-millis to 10000.
- gh-ost does not support foreign keys. You are advised to use MySQL official DDL. If you must use gh-ost, delete foreign keys first or configure the --skip-foreign-key-checks parameter for gh-ost. In this case, you can use gh-ost to execute DDL. However, after the execution is complete, the foreign keys on the table will be lost and you need to manually add them.
- gh-ost works by creating a ghost table with the same structure as the original table. It synchronizes all data in the original table to the ghost table by parsing binlogs. After the synchronization is complete, it exchanges the names of the original table and ghost table using the RENAME TABLE statement.
Due to a known issue of gh-ost, RENAME TABLE can sometimes hang and fail with the error message "ERROR Error 1205: Lock wait timeout exceeded; try restarting transaction".
To resolve or avoid this, consider the following:
- Increase the value of the gh-ost parameter --cut-over-lock-timeout-seconds (which specifies the RENAME TABLE timeout, in seconds), for example, set --cut-over-lock-timeout-seconds to 10. The value ranges from 1 to 10. Values outside this default to 3.
- Set the gh-ost parameter --cut-over to two-step to switch the table in a non-atomic manner. That is, rename the original table to a temporary table first, and then rename the ghost table to the original table. However, if the switchover fails, the original table may be lost. Exercise caution when using this method. You are advised to use the first method.
Example of Using gh-ost
Change the storage engine of the sbtest.sbtest1 table to InnoDB.
gh-ost -user="temp" -password="test" -host=**.*.*.* \
-database="sbtest" -table="sbtest1" \
-alter="engine=innodb" \
-chunk-size=2000 \
-allow-on-master \
-cut-over=default \
-default-retries=120 \
-panic-flag-file=/tmp/ghost.panic.flag \
-execute \
-debug \
-max-load=Threads_running=20 \
-critical-load=Threads_running=100 Table 1 Parameter description | Parameter Example | Definition | Example Meaning |
| -user="temp" -password="test" -host=**.*.*.* | Specifies the database connection credentials. | Use user temp and password test to connect to the instance with the specified IP address. |
| -database="sbtest" -table="sbtest1" | Specifies the data and table to be operated. | Operate table sbtest1 in database sbtest. |
| -alter="engine=innodb" | Specifies the table structure change to be executed. | Change the storage engine of the table to InnoDB. NOTE: The ALTER TABLE statement is not required. |
| -chunk-size=2000 | gh-ost copies all data of the entire table in batches. The batch size is determined by the -chunk-size parameter (default value: 1,000 rows). | Copy 2,000 rows from the original table to the ghost table each time. This affects the migration speed and database load. |
| -allow-on-master | Allows the execution on the primary node. | Allow gh-ost to perform the operation on the primary node. |
| -cut-over=default | Controls the table switchover mode. | Use the default atomic mode. The final table switchover is complete via RENAME TABLE. |
| -default-retries=120 | Sets the default number of retries. | Retryable operations will be retried up to 120 times. |
| -panic-flag-file=/tmp/ghost.panic.flag | Sets the emergency abort flag file. | If this file exists, gh-ost immediately aborts the operation. |
| -execute | Executes the operation. | If this parameter is not specified, gh-ost only performs tests. Migration is only performed when this parameter is specified. |
| -debug | Enables the debugging mode. | Output more detailed log information for troubleshooting. |
| -max-load=Threads_running=20 | Specifies the maximum load. | When Threads_running (number of running threads) exceeds 20, gh-ost suspends operations to reduce load. |
| -critical-load=Threads_running=100 | Specifies the critical load. | When Threads_running exceeds 100, gh-ost suspends operations to prevent database overload. |
For more parameters, see the official website.
Differences Between MySQL Online DDL and gh-ost
The following table lists the differences between MySQL online DDL and gh-ost.
Table 2 Differences between MySQL online DDL and gh-ost | Item | MySQL Online DDL | gh-ost |
| Use cases | - If a DDL operation supports the INSTANT or INPLACE algorithm, MySQL online DDL is recommended.
- If a DDL operation only supports the COPY algorithm, MySQL online DDL is not recommended, especially for large tables.
| If a DDL operation can only use the COPY algorithm and the original table is large or frequently updated, gh-ost is recommended for better performance and stability. |
| Principles | - INSTANT algorithm: modifies metadata only.
- INPLACE algorithm: modifies metadata and data pages in the existing tablespace and synchronizes incremental data through row logs.
- COPY algorithm: creates a temporary table and copies data.
| - Create a ghost table.
- Consume binlogs to synchronize incremental data.
- Copy data from the original table to the ghost table in batches.
- Exchange table names atomically.
- Clear resources.
|
| Constraints | - Only the COPY algorithm is supported for a few operations, such as deleting primary keys, adding full-text indexes or spatial indexes, and changing column types.
- Temporary tables are not supported.
| - The table must have a unique key.
- Binlog must be enabled, the binlog format must be row, and binlog_row_image must be set to FULL.
- Temporary tables, foreign keys, and triggers are not supported.
|
| DML blocking duration | - INSTANT: minimal blocking time
- INPLACE: brief blocking only at the start and end phases
- COPY: blocking throughout the entire process
| 1. During data synchronization, row data is locked in batches. Only DML operations on row data in a batch are blocked. 2. DML operations are briefly blocked when table names are exchanged atomically. |
| Extra space occupied | - INSTANT: minimal
- INPLACE: small (slightly larger when a table needs to be rebuilt)
- COPY: at least the same as the space occupied by the original table
| At least the same as the space occupied by the original table |
| Replication delay | - INSTANT: minimal
- INPLACE: low
- COPY: high
| Medium |
FAQs
How Do I Estimate the DDL Execution Duration?
The DDL execution duration depends on the execution algorithm, table data volume, instance specifications, and workloads. You are advised to execute DDL statements in off-peak hours.
You are advised to enable DDL Fast Timeout and Non-blocking DDL before executing DDL statements (ensure that the kernel version of the instance meets the requirements).
How Do I Determine Whether a DDL Operation Supports the INSTANT or INPLACE Algorithm?
- Method 1: Execute SQL statements.
- Set algorithm to instant. If the algorithm is not supported, an error will be reported.
- Set algorithm to inplace. If the algorithm is not supported, an error will be reported.
Example:
-- The instant algorithm is not supported.
alter table t drop primary key,algorithm=instant;
ERROR 1846 (0A000): ALGORITHM=INSTANT is not supported. Reason: Dropping a primary key is not allowed without also adding a new primary key. Try ALGORITHM=COPY/INPLACE.
-- The inplace algorithm is not supported.
alter table t drop primary key,algorithm=inplace;
ERROR 1846 (0A000): ALGORITHM=INPLACE is not supported. Reason: Dropping a primary key is not allowed without also adding a new primary key. Try ALGORITHM=COPY.
-- Only the copy algorithm is supported.
alter table t drop primary key,algorithm=copy;
Query OK, 0 rows affected (0.13 sec) Records: 0 Duplicates: 0 Warnings: 0
- Method 2: Visit the MySQL official website.
How Do I Check the Execution Progress of DDL Statements?
You can only check the progress of secondary index creation (Progress Queries for Creating Secondary Indexes).
To check the progress of other DDL operations, use the following SQL query (performance_schema must be set to ON):
SELECT EVENT_NAME AS 'current_execution_phase',WORK_COMPLETED AS 'completed_workload',WORK_ESTIMATED AS 'estimated_total_workload', ROUND(IF(WORK_ESTIMATED = 0, 0, (WORK_COMPLETED / WORK_ESTIMATED) * 100), 2) AS 'progress (%)' FROM performance_schema.events_stages_current WHERE EVENT_NAME LIKE 'stage/sql/copy to tmp table'OR EVENT_NAME LIKE 'stage/innodb/alter table%';