Data Content Copyright Check
Data content copyright check, a data source security protection product designed by Huawei Cloud, is used for AI model training. It aims to build copyright risk rating models, helping you quickly identify potential copyright compliance risks in AI training datasets, as well as building reliable, secure, and controllable AI applications.
Prerequisites
- Access to cloud assets has been authorized. For details, see Allowing or Disallowing Access to Cloud Assets.
- OBS assets have been added. For details, see Adding Unstructured OBS Data.
Notes and Constraints
This function is supported only in the CN North-Beijing4, CN East-Shanghai1, and CN Southwest-Guiyang1 regions.
Creating a Check Task
- Log in to the DSC console.
- Click
in the upper left corner and select a region or project. - In the navigation pane on the left, choose . The Data Copyright Compliance page is displayed.
- Select the Data Content Copyright Check tab.
- Click Create Detection Task and set the parameters listed in Table 1. Figure 1 Creating a data content copyright check task
Table 1 Parameters for creating a data content copyright check task Parameter
Description
Task
Enter a user-defined task name.
Task description (Optional)
Enter the purpose of the check task.
Review mode
The following review modes are supported:
- Text
- OCR Text
- Static Image
Select Data Format
Select the format of the file to be checked.
- Word (.docx), Excel (.xlsx), PowerPoint (.pptx), and PDF (.pdf): Only the selected mode will be checked. Static Image: Only the content will be checked. The text in the static image will not be checked. To check the text, select OCR Text.
- Only images with a resolution ranging from 640 x 640 to 3840 x 3840 are supported. A maximum file size of 100 MB is supported.
Select Data Source
Only OBS is supported.
Traverse Sub-folders
If this option is selected, files in all sub-folders will be traversed.
Bucket Name
In the displayed dialog box, select the OBS bucket, and click OK.
OBS Storage Path (Optional)
Select the cloud directory of the OBS bucket.
- Click Next to configure algorithm detection. The algorithm parameters vary with the configuration in Step 5.
- If the images will be reviewed, configure Confidence threshold.
Confidence threshold: indicates the confidence level of image identification. The higher the confidence level, the smaller the false positive rate. However, the false negative rate may increase.
- If the text will be reviewed, configure Matching length and Tolerance for matching interruption.
- Matching length: indicates the length of consecutive matching characters. If the length of consecutive matching characters in the text exceeds the specified length, the text is considered as infringing.
- Tolerance for matching interruption: indicates the number of character interruptions allowed during the matching process. Increasing this value can help detect slightly tampered text.
- If the images will be reviewed, configure Confidence threshold.
- Click Next. The first execution will be a full review at Estimated Start Time. Periodic tasks will be executed based on the configured frequency, for example, at 09:00 every day. Only new or modified files will be reviewed.
- If you select Move to specified OBS directory, select an OBS bucket name and the OBS path.
- Estimated Start Time: Set the task start time.
- Task Execution Frequency: You can select once, daily, weekly, or monthly.
Figure 2 Creating a check task
- Click OK.
Related Operations
Starting a Check Task
You can manually start a check task whose Task Status is New task, Stopped, or Abnormal and whose check report status is Not exported.
- Select the Data Content Copyright Check tab.
- Click Start in the Operation column of a check task.
Stopping a Check Task
You can manually stop a check task which is running.
- Select the Data Content Copyright Check tab.
- Click Stop in the Operation column of a check task.
If the task is completed, you can export the check report to a specified OBS path.
- Select the Data Content Copyright Check tab.
- Choose More > Export in the Operation column.
- In the displayed dialog box, specify the OBS name and path for saving the check report.
- Click Export to export the check report. In the Export Status column, click View Export Result to go to the OBS console and view the check report.
Editing or Deleting a Check Task
- Editing a check task: On the Data Content Copyright Check tab page, click Edit in the Operation column to modify a check task.
- Deleting a check task: On the Data Content Copyright Check tab page, click Delete in the Operation column to delete a check task.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot