To DLI
If the destination link of a job is a DLI link, configure the destination job parameters based on Table 1.
When data is migrated to DLI using CDM, DLI generates data files in the dli-trans* temporary OBS bucket. Therefore, you need to grant the user who uses the AK/SK the permissions to read and write the dli-trans* bucket and create directories. Otherwise, the migration will fail. For details about how to add permission policies for temporary bucket dli-trans*, see Adding an Authorization Policy for the dli-trans* Temporary Bucket.
Parameter |
Description |
Example Value |
---|---|---|
Resource Queue |
Resource queue to which the destination table belongs The default queue of DLI cannot be used for migration jobs. You need to create a SQL queue in DLI. For details about how to create a queue, see Creating a Queue. |
cdm |
Database Name |
Name of the database to which data will be written |
dli |
Table Name |
Name of the table to which data will be written |
car_detail |
Clear Data Before Import |
Whether to clear data in the destination table before data import If this parameter is set to Yes, data in the destination table will be cleared before the task is started. |
No |
Convert empty strings to null |
If this parameter is set to Yes, an empty string is regarded as null. |
No |
Data Clearing Mode |
This parameter is available when Clear Data Before Import is set to Yes. TRUNCATE: deletes standard data. INSERT_OVERWRITE: overwrites existing data with inserted data.
NOTE:
If the source link is a Kafka link and Clear Data Before Import is set to Yes, INSERT_OVERWRITE is unavailable. |
TRUNCATE |
Partition |
This parameter is available when Clear Data Before Import is set to Yes. When you enter partitions, data in these partitions will be cleared. |
year=2020,location=sun |
Adding an Authorization Policy for the dli-trans* Temporary Bucket
- Log in to the IAM console.
- In the navigation pane, choose Permissions > Policies/Roles and click Create Custom Policy in the upper right corner.
Figure 1 Creating a custom policy
- On the Create Custom Policy page, select JSON for Policy View and create custom policy obs_dli-trans.
{ "Version": "1.1", "Statement": [ { "Effect": "Allow", "Action": [ "obs:object:GetObject", "obs:object:DeleteObjectVersion", "obs:bucket:GetBucketLocation", "obs:object:GetAccessLabel", "obs:bucket:PutEncryptionConfiguration", "obs:bucket:PutBucketStoragePolicy", "obs:object:DeleteAccessLabel", "obs:bucket:PutBucketCustomDomainConfiguration", "obs:bucket:GetLifecycleConfiguration", "obs:bucket:PutBucketInventoryConfiguration", "obs:bucket:DeleteDirectColdAccessConfiguration", "obs:object:AbortMultipartUpload", "obs:bucket:PutBucketLogging", "obs:bucket:DeleteBucketWebsite", "obs:object:DeleteObject", "obs:bucket:PutBucketVersioning", "obs:bucket:GetBucketWebsite", "obs:bucket:GetBucketLogging", "obs:bucket:DeleteBucketCustomDomainConfiguration", "obs:object:PutObject", "obs:object:RestoreObject", "obs:bucket:PutReplicationConfiguration", "obs:bucket:GetBucketQuota", "obs:object:GetObjectVersionAcl", "obs:bucket:DeleteBucket", "obs:bucket:CreateBucket", "obs:bucket:GetDirectColdAccessConfiguration", "obs:bucket:PutDirectColdAccessConfiguration", "obs:bucket:GetBucketAcl", "obs:bucket:GetBucketVersioning", "obs:bucket:GetBucketInventoryConfiguration", "obs:bucket:GetBucketStoragePolicy", "obs:bucket:GetEncryptionConfiguration", "obs:bucket:PutBucketCORS", "obs:bucket:PutBucketTagging", "obs:bucket:GetBucketTagging", "obs:bucket:PutLifecycleConfiguration", "obs:bucket:GetBucketCustomDomainConfiguration", "obs:object:ListMultipartUploadParts", "obs:object:ModifyObjectMetaData", "obs:bucket:ListBucketVersions", "obs:bucket:PutBucketQuota", "obs:object:PutAccessLabel", "obs:bucket:ListBucket", "obs:bucket:GetBucketCORS", "obs:bucket:DeleteBucketInventoryConfiguration", "obs:object:GetObjectVersion", "obs:bucket:PutBucketWebsite", "obs:bucket:DeleteReplicationConfiguration", "obs:object:GetObjectAcl", "obs:bucket:GetBucketNotification", "obs:bucket:PutBucketNotification", "obs:bucket:GetReplicationConfiguration", "obs:bucket:GetBucketPolicy", "obs:bucket:DeleteBucketTagging", "obs:bucket:GetBucketStorage" ], "Resource": [ "OBS:*:*:object:*", "OBS:*:*:bucket:dli-trans*" ] } ] }
Figure 2 Creating custom policy obs_dli-trans
- Click OK.
- In the navigation pane, choose User Groups, locate the user group to which the DLI link user using the AK/SK belongs, and click Authorize to assign the custom obs_dli-trans policy to the user.
Figure 3 Assigning the custom obs_dli-trans policy to a user group
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot