Help Center/ Data Replication Service/ Best Practices/ Real-Time Synchronization/ From RDS for MySQL to Kafka/ Creating a DRS Synchronization Task
Updated on 2024-07-15 GMT+08:00
View PDF
Share

Creating a DRS Synchronization Task

This section describes how to create a DRS instance and synchronize incremental data from RDS for MySQL to Kafka.

Pre-Check

Before creating a task, check the synchronization conditions.

In this example, data is synchronized from RDS for MySQL to Kafka. For details, see Precautions.

Procedure

  1. Log in to the management console.
  2. Click in the upper left corner of the management console and select AP-Singapore.
  3. Under the service list, choose Databases > Data Replication Service.
  4. Choose Data Synchronization Management and click Create Synchronization Task.
  5. Configure synchronization task parameters.

    1. Task name

    2. Select the source database, destination database, and network information.

      Select the RDS instance created in Preparing for Source RDS for MySQL as the source database.

    3. Set Enterprise Project to default.

  6. Click Create Now.

    The synchronization instance is being created. It takes about 5 to 10 minutes.

  7. Configure source and destination database information.

    1. Configure source database information.
    2. Click Test Connection.

      If a successful test message is returned, the database is connected.

    3. Select the VPC and subnet where the destination database is located, and enter the Kafka IP address and port number.
    4. Click Test Connection.

      If a successful test message is returned, the database is connected.

  8. Click Next.
  9. Select the synchronization information, policy, message format, and object, and the format of the message sent to the Kafka.

    The following table lists the settings.

    Table 1 Sync Settings

    Type

    Setting

    Topic Synchronization Policy

    Deliver the content to a topic named testTopic.

    Synchronize Topic To

    Partitions are differentiated by the hash values of the database and table names

    Data Format in Kafka

    You can select the JSON format. For details, see Kafka Message Format.

    Synchronization Object

    Select table3_ under db_test.

  10. Click Next.
  11. Select a data processing mode. Data synchronization from RDS for MySQL to Kafka supports only column-based processing, which provides column-level query and filtering capabilities.

  12. Click Next and wait for the check results.
  13. If the check is complete and the check success rate is 100%, click Next.

  14. After confirming that the synchronization task information is correct, click Next.

    Return to the Data Synchronization Management page and check the synchronization task status.

    It takes several minutes to complete.

    If the status changes to Incremental, the synchronization task has been started.

    • In this example, Synchronization Mode is set to Incremental for the task from RDS for MySQL to Kafka. After the task is started, the status is Incremental.
    • If the created task is a full+incremental synchronization task, the task enters the full synchronization state after being started. After the full synchronization is complete, the task enters the incremental synchronization state.
    • Incremental synchronization continuously synchronizes incremental data and does not stop automatically.