Help Center/ Data Replication Service/ Getting Started/ Creating a Workload Replay Task
Updated on 2024-11-13 GMT+08:00

Creating a Workload Replay Task

A workload replay task simulates the service load of the source database on the destination database so you can evaluate the effectiveness and performance of the destination database. It is typically used in the following scenarios:

  • Function testing: By creating a workload replay task, you can evaluate how the service load of the source database runs on the destination database before database migration.
  • Peak load testing: By specifying the replay thread and speed, you can simulate the peak service load of the source database and analyze the stability of the destination database when workloads increase sharply.

This section uses two RDS for MySQL DB instances in different regions as an example to describe how to replay SQL workload files over the public network.

Operation Process

Process

Description

Preparations

Sign up for a HUAWEI ID, complete real-name authentication, enable Huawei Cloud services, and prepare the workload files and the destination database, the database connection account, and a network.

Step 1: Create a Workload Replay Task

Select the source and destination databases as required and create a workload replay task.

Step 2: Query Replay Progress

During the workload replay, query the progress.

Step 3: Check Replay Reporting

After workload replay is complete, check the replay reporting to learn about the execution of each SQL statement.

Stopping a Task

After confirming that the workload replay task is no longer used, stop the DRS task.

Preparations

Before creating a workload replay task, prepare the Huawei account, source and destination databases, database connection accounts, SQL workload files, and network environment by referring to the following steps.

Prepare a Huawei account, create a user, and grant permissions to the user to use DRS.

  1. Access Huawei Cloud website.
  2. Click Sign Up and follow the instructions to create your account (your HUAWEI ID).
  3. Select the service agreement and click Enable.

    The system displays a message indicating that Huawei Cloud services have been enabled.

  4. Perform real-name authentication.

  5. For fine-grained permissions management, create an Identity and Access Management (IAM) user and user group on the IAM console and grant the user specific operation permissions. For details, see Creating a User and Granting Permissions

Before creating a workload replay task, you need to prepare the source and destination databases.

  • The source database in this example is an RDS for MySQL instance in the CN-Hong Kong region. For details, see Buy an RDS for MySQL DB Instance.

    Item

    Example Value

    Description

    Region

    CN-Hong Kong

    To reduce network latency, select the region nearest to you.

    Instance name

    rds-Replaysrc

    Specify a name that will be easy to identify.

    DB engine version

    MySQL 5.7

    -

    Instance type

    Single

    A single instance is used in this example.

    To improve service reliability, select a primary/standby instance.

    Storage type

    Cloud SSD

    -

    AZ

    AZ1

    AZ1 is selected in this example.

    To improve service reliability, select the primary/standby instance type and deploy the primary and standby instances in different AZs.

    Instance class

    General-purpose 2 vCPUs | 8 GB

    -

    EIP

    10.154.217.42

    Enter an IP address.

  • The destination database in this example is an RDS for MySQL instance in the AP-Singapore region. For details, see Buy an RDS for MySQL DB Instance.

    Item

    Example Value

    Description

    Region

    AP-Singapore

    To reduce network latency, select the region nearest to you.

    Instance name

    rds-Replaytar

    Specify a name that will be easy to identify.

    DB engine version

    MySQL 5.7

    -

    Instance type

    Single

    A single instance is used in this example.

    To improve service reliability, select a primary/standby instance.

    Storage type

    Cloud SSD

    -

    AZ

    AZ1

    AZ1 is selected in this example.

    To improve service reliability, select the primary/standby instance type and deploy the primary and standby instances in different AZs.

    Instance class

    General-purpose 2 vCPUs | 8 GB

    -

Before creating a task, you need to capture SQL workload files of the source database.

  1. Log in to the management console.
  2. Click in the upper left corner of the management console and select CN-Hong Kong.
  3. Click Service List. Under Databases, click Relational Database Service. The RDS console is displayed.
  4. On the Instances page, click the name of the source DB instance created in Databases to go to the Overview page.
  5. In the navigation pane, choose SQL Audits. On the displayed page, click Set SQL Audit above the list. In the displayed dialog box, configure information as required and click OK.

    • Audit Logging: Enable
    • Retention Period (days): 7
    Figure 1 Set SQL Audit

To ensure a smooth workload replay, you are advised to create an independent database account for connecting to the DRS task. It is recommended that the permissions of the destination database user be the same as that of the source database user.

Destination database user

  1. Connect to an RDS for MySQL instance through DAS.
  2. Run the following statement to create database user usertar. For details, see Creating a Database Account Through DAS.

    CREATE USER 'usertar'@'%' IDENTIFIED BY 'password';

  3. Run the following statement to grant permissions to usertar:

    GRANT SELECT,CREATE,ALTER,DROP,DELETE,INSERT,UPDATE,TRIGGER,REFERENCES,SHOW VIEW,EVENT,INDEX,LOCK TABLES,CREATE VIEW,CREATE ROUTINE,ALTER ROUTINE,CREATE USER,RELOAD,REPLICATION SLAVE,REPLICATION CLIENT ON *.* TO 'usertar'@'%' WITH GRANT OPTION;

Before creating a workload replay task over a public work, you need to apply for an EIP and set security group rules for the source database so that the EIP can access the source database.

  1. Create an EIP for the workload replay task by referring to Assigning an EIP.
  2. Configure security group rules for the source database.

    Add the EIP in 1 of the DRS instance to the inbound rule of the security group associated with the source RDS for MySQL database, select TCP, and set the port number to that of the source database.

Step 1: Create a Workload Replay Task

This step describes how to set parameters based on the example values in Preparations. For more information about workload replay, see Replay Overview.

  1. Go to the Create Workload Replay Task page.
  2. Configure basic information as follows:

    Figure 2 Basic information

    Parameter

    Example Value

    Description

    Region

    AP-Singapore

    The region where the current task is deployed.

    Project

    AP-Singapore

    The project corresponds to the current region.

    Task Name

    DRS-Replay

    The name of the workload replay task.

    Description

    Leave this parameter blank for now.

    Task description.

  3. Configure information about the replay instance.

    Figure 3 Replay instance details

    Parameter

    Example Value

    Description

    Data Flow

    Current cloud

    The direction of the workload replay task. The options are Current cloud and To the cloud.

    • Current cloud: Both the source and destination databases are Huawei Cloud DB instances.
    • To the cloud: The destination database is a Huawei Cloud DB instance and data needs to be transferred to the cloud.

    Source DB Engine

    MySQL

    The DB engine type of the source database.

    Workload File Source

    Download from Huawei Cloud APIs

    Specifies where the workload file in the source database is from.

    Destination DB Engine

    MySQL

    The DB engine type of the destination database.

    Network Type

    Public network

    The network type of the workload replay task.

    Destination DB Instance

    rds-Replaytar

    Select a destination DB instance.

    The destination DB instance is rds-Replaytar in Databases.

    Replay Instance Subnet

    Default subnet

    The subnet where the workload replay task is.

    Specify EIP

    -

    Select an EIP for the DRS instance. In this example, it is the EIP created in Network.

  4. Select an AZ and an enterprise project.

    Figure 4 AZ and Enterprise Project

    Parameter

    Example Value

    Description

    AZ

    AZ1

    Select the AZ where you want to create the DRS task.

    Enterprise Project

    default

    If your account is associated with an enterprise project, select the project from the Enterprise Project drop-down list.

    For more details, see Enterprise Management User Guide.

    Tags

    Leave this parameter blank for now.

    Tags for the workload replay task. Adding tags helps you better identify and manage your tasks.

  5. Click Create Now.

    Wait for 5 to 10 minutes. If the message "Replay instance created successfully" is displayed, the DRS task is successfully created.

  6. On the Configure Source and Destination Databases page, specify the source and destination database details.

    • Source database settings
      • Workload File Source: Download from Huawei Cloud APIs
      • DB Instance Name: Select the source DB instance rds-Replaysrc created in Databases.
      • Workload Type: Audit log
      • Time Range: Select the time range for audit logs in SQL Workload Files.
    • Destination database settings

      Enter the connection details about the destination database in Connection Accounts.

  7. Configure the task.

    Figure 5 Task settings

    Parameter

    Example Value

    Description

    SQL Type

    SELECT

    Select the SQL type to be replayed to the destination database. The default value is SELECT. The available options are SELECT, INSERT, UPDATE, DELETE, and DDL.

    Replay Mode

    Performance

    Mode of the workload replay task. You can select Performance or Transaction.

    • In performance mode, you can set how many concurrent connections are allowed. SQL statements are replayed to the destination database based on a set number of connections. The SQL execution sequence in the source database may be different from that in the destination database. The replay performance is better.
    • In transaction mode, you cannot set how many concurrent connections are allowed. The number of connections is dynamically adjusted based on the connections in the source database logs to ensure that transaction SQL statements in the same connection of the source database are executed in sequence.

    Filter out SQLs

    -

    The system fuzzily matches SQL statements based on the entered condition.

    Filter out SQLs Without Conditions

    -

    This option is used to filter out SQL statements of the SELECT, UPDATE, and DELETE types that do not contain conditions.

    Maximum Concurrent Connections

    8

    The number of replay threads configured for the workload replay task.

    Acceleration Configuration

    100%

    The percentage of the replayed SQLs to the SQLs executed on the source database within the same period.

  8. Click Test Connection in the Destination Database area to test the connection to the destination database. After the connection test is successful, click Next.
  9. On the Check Task page, check the workload replay task.

    • If any check fails, review the cause and rectify the fault. After the fault is rectified, click Check Again.
    • If all check items are successful, click Next.

  10. On the Confirm Task page, specify Start Time.

    Figure 6 Task startup settings

    Parameter

    Example Value

    Description

    Start Time

    Start upon task creation

    Set Start Time to Start upon task creation or Start at a specified time based on site requirements. The Start at a specified time option is recommended.

    Send Notifications

    Disable

    This parameter is optional. After enabled, select an SMN topic. If the status, latency metric, or data of the workload replay task is abnormal, DRS will send you a notification.

    Stop Abnormal Tasks After

    14

    Any task in the Abnormal state that has run for longer than the period you set here (in days) will automatically stop.

  11. Confirm that the configured information is correct and click Submit to submit the task.
  12. After the workload replay task is submitted, check the task status or query the replay progress on the Workload Replay Management page.

    • You can click in the upper right corner to view the latest task status.
    • By default, DRS retains a task in the Configuration state for three days. After three days, DRS automatically deletes background resources, but the task status remains unchanged. When you reconfigure the task, DRS applies for resources for the task again.

Step 2: Query Replay Progress

The replay progress displays the SQL execution status during workload replay, helping you learn about the task status.

  1. On the Workload Replay Management page, click the target replay task in the Task Name/ID column.
  2. In the navigation pane, choose Workload Replay Progress to check task progress.

    • In the Workload Replay Progress area, you can view the task status, start time, total number of SQL statements, and number of replayed SQL statements.
    • In the Statistics Chart area, you can view the total number of SQL statements, number of replayed SQL statements, number of abnormal SQL statements, and number of slow SQL statements in a specified period.
    • In the Abnormal SQLs in Workload Replay area, you can view the category and number of SQL statements that fail to be replayed.
    • In the Slow SQLs area, you can view the original time and replay time required for executing a SQL statement.
    • In the SQL Execution Progress are, you can view the SQL statements that are executing in the destination database during replay.

  3. After workload replay is complete, check the replay reporting to learn about the execution of each SQL statement.

Step 3: Check Replay Reporting

The replay reporting records the execution time curve of each SQL statement replayed in the destination database, number of replayed SQLs, and replay duration.

  1. On the Workload Replay Management page, click the target replay task in the Task Name/ID column.
  2. On the Replay Reporting page, check the report details about the current task.

    • In the Statistics Chart area, you can view the total number of SQL statements, number of replayed SQL statements, number of abnormal SQL statements, and number of slow SQL statements replayed in the destination database in a specified period.
      Figure 7 Replay statistics chart

    • In the Slow SQLs area, you can view the number of SQL statements of each type and the replay duration.
    • In the Abnormal SQLs in Workload Replay area, you can view the category and number of SQL statements that fail to be replayed.
      Figure 8 SQLs to be replayed

Stopping a Task

After confirming that the workload replay task is no longer used, you can stop the DRS task.

  1. On the Workload Replay Management page, locate the task and click Stop in the Operation column.
  2. In the displayed dialog box, click Yes.