Data Mapper

Configuring Parameters

NOTE:

Subsequent nodes can reference the output data of a data mapper processor using ${node ID|payload.xxx}. The node ID is displayed after you click a data mapper node.

Destination - XML Data Example

If destination data source format is XML, Root Tag defaults to root. You can set this parameter to other values but cannot leave it empty.

Data in the demo.statistics table of the source MySQL database:

Mapping result:

<root>
    <custom>
        <key1>uuid1</key1>
        <key2>
            <level2>suzhou</level2>
        </key2>
    </custom>
    <custom>
        <key1>uuid2</key1>
        <key2>
            <level2>shanghai</level2>
        </key2>
    </custom>
</root>

Destination - CSV Data Example

Header: Whether to include the table header of the CSV file in the output. Options are true and false.

Delimiter: Data in each file column is separated by cells, which can be represented as tab characters, spaces, commas, or semicolons.

Scenario 1: Set the destination data source format to CSV, Header to true, and Delimiter to comma.

When a file is imported, if Header is set to true, the first line of the CSV file is the field names.

Data in the demo.statistics table of the source MySQL database:

Mapping result:

ManufacturerId, ProductName
uuid1, suzhou
uuid2, shanghai

Scenario 2: Set the destination data source format to CSV, Header to false, and Delimiter to comma.

When a file is imported, if Header is set to false, the header field of the imported file is not used. The filedN field is used by default.

Data in the demo.statistics table of the source MySQL database:

Mapping result:

uuid1, suzhou
uuid2, shanghai

Source - JSON Data Example

If the source data format is JSON, use the output of the source data source as the import file of the source, and select the encoding format (GB2312, UTF-8, or ASCII) of the file. The system automatically generates fields accordingly.

Set the destination data format to MySQL, and select a table name. The system automatically generates fields in the database table.

(Example) The following composite application calls he open API to list objects in a specified OBS bucket and map the bucket name and object names to corresponding fields in a specified MySQL table via data mapping, and then stores the bucket name and object names to MySQL. In this application, the OBS data serves as the source of data mapping, while MySQL is the destination. The open API calls the OBS connector to list bucket objects as the import file of the remote data source. Alternatively, you can also manually add fields based on the object list.

Content of the imported file:

{

"responseHeaders": {

"bucket-location": "cn-north-7",

"connection": "keep-alive",

"content-length": "504",

"content-type": "application/xml",

"date": "2024-04-24T07:07:43.000+00:00",

"id-2": "32AAAQAAEAABAAAQAAEAABAAAQAAEAABCSbOSbK ncLEgZnxffa8vFPuDgVIt7n9",

"request-id": "0000018F0EEE84734012ACF15A46395E",

"server": "OBS"

},

"originalHeaders": {

"connection": "keep-alive",

"content-length": "504",

"content-type": "application/xml",

"date": "Wed, 24 Apr 2024 07:07:43 GMT",

"server": "OBS",

"x-obs-bucket-location": "cn-north-7",

"x-obs-id-2": "32AAAQAAEAABAAAQAAEAABAAAQAAEAABCSbOSbK+ncLEgZnxffa8vFPuDgVIt7n9",

"x-obs-request-id": "0000018F0EEE84734012ACF15A46395E"

},

"statusCode": 200,

"objectSummaries": [

{

"bucketName": "test0223",

"objectKey": "test0424.txt",

"owner": {

"displayName": null,

"id": "0ab78f256080d2890feac0086234d7c0"

},

"metadata": {

"responseHeaders": {},

"originalHeaders": {},

"statusCode": 0,

"lastModified": "2024-04-24T06:51:46.566+00:00",

"contentLength": 19,

"contentType": null,

"contentEncoding": null,

"contentDisposition": null,

"cacheControl": null,

"contentLanguage": null,

"expires": null,

"etag": "\"0b26e313ed4a7ca6904b0e9369e5b957\"",

"contentMd5": null,

"crc64": null,

"storageClass": "STANDARD",

"webSiteRedirectLocation": null,

"nextPosition": -1,

"appendable": false,

"objectStorageClass": "STANDARD",

"allMetadata": {},

"metadata": {},

"requestId": ""

},

"objectContent": null

}

],

"commonPrefixes": [],

"extendCommonPrefixes": [],

"bucketName": "test0223",

"truncated": false,

"prefix": "",

"marker": "",

"maxKeys": 1000,

"delimiter": null,

"nextMarker": null,

"location": "cn-north-7",

"extenedCommonPrefixes": [],

"objects": [

{

"bucketName": "test0223",

"objectKey": "test0424.txt",

"owner": {

"displayName": null,

"id": "0ab78f256080d2890feac0086234d7c0"

},

"metadata": {

"responseHeaders": {},

"originalHeaders": {},

"statusCode": 0,

"lastModified": "2024-04-24T06:51:46.566+00:00",

"contentLength": 19,

"contentType": null,

"contentEncoding": null,

"contentDisposition": null,

"cacheControl": null,

"contentLanguage": null,

"expires": null,

"etag": "\"0b26e313ed4a7ca6904b0e9369e5b957\"",

"contentMd5": null,

"crc64": null,

"storageClass": "STANDARD",

"webSiteRedirectLocation": null,

"nextPosition": -1,

"appendable": false,

"objectStorageClass": "STANDARD",

"allMetadata": {},

"metadata": {},

"requestId": ""

},

"objectContent": null

}

],

"requestId": "0000018F0EEE84734012ACF15A46395E"

}

Data mapping result: