What Do I Do If the Error Message "value too long for type character varying" Is Displayed When I Migrate Data to DWS?

Symptom

When you use CDM to migrate data to DWS/FusionInsight LibrA, the migration fails and the error message "value too long for type character varying" is displayed in the log. See Figure 1.

Figure 1 Log output

Possible Cause

The data migrated to DWS is in Chinese, and the table is automatically created at the migration destination. The length of the varchar field of DWS is calculated by byte, and a Chinese character may occupy three bytes in UTF-8 encoding. If the length of a Chinese character exceeds that of the varchar field of DWS, an error occurs and the error message "value too long for type character varying" is displayed.

Solution

To solve this problem, you can select Extend Field Length to Yes, so that the length of the varchar field is automatically increased by three times when the destination table is created.

Edit the table/file migration job on CDM. In Destination Job Configuration, set Auto Table Creation to Auto creation, Extend Field Length is displayed in Show Advanced Attributes. Set Extend Field Length to Yes.

Figure 2 Extending field length