Filtering Large Fields

Constraints

• Large field filtering is supported only for tasks from MySQL to Kafka and from GaussDB(for MySQL) to Kafka.
• When creating a task, you can choose to enable or disable large field filtering in object selection page. After a task is started, you can choose More > Filter Large Field in the Operation column of the task to set this function.
• For a failed task, you need to manually resume or reset the task to apply this function.

Procedure

1. On the Data Synchronization Management page, locate the target task and choose More > Filter Large Field in the Operation column.
   Figure 1 Task list
   

2. In the Filter Large Field dialog box, choose whether to use large field filtering to process special fields (blob, mediumblob, longblob, varbinary, mediumtext and longtext) in a synchronization table.
   • Enable: You need to set Field Filtering Threshold and Replace With. If the size of a field exceeds the threshold, the value is replaced based on a specified character. Note that large field filtering is used to replace the value of a field, not the entire DML record. If a DML record contains many large fields, the size of only some of these fields exceeds the filtering threshold, and the accumulated value of other fields that do not exceed the filtering threshold is greater than the value of request.max.size, when data is written to Kafka, the size of the message body in the destination Kafka database may still exceed the upper limit, resulting in a DRS error.
   • Disable: Large fields are not filtered.
     Figure 2 Filter Large Field
     

3. Click OK.

   For a failed task, you need to manually resume or reset the task to apply this function.