Content Identification
Content identification enables the system to identify the text and images based on the blacklisted and whitelisted dictionary provided by users and the million-level sensitive content dictionary built in DSC. It can identify texts in formats such as HTML, TXT, Markdown, XML, and JSON, images in formats such as JPG, PNG, JPEG, JP2, BMP, and WEBP with a maximum resolution of 3840 x 3840 pixels. You can specify whitelists to allow terms that you consider acceptable. The system will generate the sensitive terms contained in each document and their types. You can decide to handle the files yourself or automatically move harmful content to specific directories.
You can define the following dictionaries:
- Blacklist dictionaries contain individual terms that are sensitive, toxic, or harmful.
- Blacklist dictionaries contain multiple terms that together form toxic or harmful content. The words are linked using the ampersand (&). For example, hate you & go die & self-harm.
- Whitelist dictionaries contain regular expressions to prevent specific formats of individual or combined terms from being falsely flagged by blacklists.
How It Works
Constraints
| Item | Description |
|---|---|
| Content | Only text and images in the OBS buckets can be checked.
|
| Size | Maximum: 100 MB. Files larger than 100 MB are not scanned. |
| Blacklist keyword type | The violation content can be detected only when their keywords are classified into fixed violation types in the blacklist. The keyword types in the blacklist cannot be customized. However, you can configure the matching threshold and whitelist. For details about the supported types, see Table 1. |
| Review | The system returns corresponding violation categories for non-compliant content but does not process the text content. You decide whether to move the non-compliant content to the specified directory. |
| Scanning | You can set an automated execution schedule for tasks or manually start scan tasks. By default, the system performs incremental scans for only new and modified files that belong to one task. The system scans every 10,000 characters as a batch and saves audit results per batch. For each scanned file, the system retains a maximum of 500 audit results, which are aggregated and displayed by keyword. The system can display results for up to 2 million files. |
| Dictionary | Each user can upload a maximum of five blacklist dictionaries and five whitelist dictionaries. Each dictionary contains a maximum of 5,000 entries, with an upper limit of 64 UTF-8 characters per entry. If the type of a word in the user-defined blacklist dictionary is different from that in the built-in dictionary, the user-defined one takes precedence. |
| Region | This function is supported only in the CN North-Beijing4, CN East-Shanghai1, and CN Southwest-Guiyang1 regions. |
| Level-1 Type | Level-2 Type |
|---|---|
| Sexual content | Pornography Bikini |
| Violence/Bloody behavior | Bloodshed Animal abuse Fights/Physical conflicts |
| Weapon/Military equipment | Aircraft carrier Cannon Chainsaw Cleaver Guillotine Hand axe Military uniform Muzzle/Gun parts Missile Revolver Rifle Scabbard Submarine Tank Military aircraft |
| Politics/National symbols | Political or illegal assembly Chinese map Party emblem Chinese flag Police van Politician |
| Extremism/terrorism/cult | Cult activity Flag of a terrorism organization Person of a terrorist or extremist organization |
| Illegal activity | Gambling activity Drugs Drugs Slot machine |
| Risky content/Content that needs to be determined by policies | Beer bottle Beer glass Red wine Wine bottle Cigarette Pill bottle Syringe QR code Playing card Fire Bad artist |
Review Tasks
In the Review Tasks tab, you can create tasks to review harmful and toxic content in files and text in the OBS buckets. You can also perform task management and result management in this tab.
- Review task management:
- Status management allows you to view the review progress and the review result of the current phase.
- Rule configuration includes the review period, scope and type of files to be reviewed, and scope and category of dictionaries.
- Result management
You can view the review result with the prohibited terms and categories. You can also export the report and download it from OBS.
- Log in to the DSC console.
- Click
in the upper left corner and select a region or project. - Choose AI Data Security > Content Identification. The Content Identification page is displayed.
- Click Create Review Task. The New Task drawer is displayed.
- Configure the data source parameters as shown in Table 2. Figure 2 Creating a review task
Table 2 Data source parameters Parameter
Description
Task
Enter a task name.
Only letters, digits, underscores (_), hyphens (-), and periods (.) are allowed. It must start with a letter.
Task description (Optional)
Describe the core content and purpose of the task.
Review mode
The following review modes are supported:
- Text: Indicates the text content in HTML, TXT, Markdown, XML, and JSON formats.
- OCR Text: Indicates the text in images.
- Static image: Indicates the images in JPG, PNG, JPEG, JP2, BMP, and WEBP formats with a resolution of up to 3840 x 3840 pixels. In this mode, only the content in the selected format will be checked. The text in the static image will not be checked. To check the text, select OCR Text.
Select Data Format
Select the data format corresponding to Review mode.
For details about the supported data formats, see Constraints.
Select Data Source
Only OBS is supported.
Traverse Sub-folders
If this option is selected, files in all sub-folders will be traversed.
Bucket Name
In the displayed dialog box, select the OBS bucket.
OBS Storage Path (Optional)
The default path is the root directory. You can also click the drop-down list box and specify a path.
- Click Next. On the Review Algorithm Configuration page, configure the algorithm by referring to Table 3.
Table 3 Review algorithm parameters Parameter
Description
Blacklist Dictionary
Huawei Cloud built-in dictionary is selected by default. Other dictionaries are optional. You can select the blacklist dictionaries from the drop-down list box. For details, see Managing the Dictionary.
Whitelist Dictionary
Add a whitelist dictionary by selecting one configured in Managing the Dictionary from the drop-down list. Skip this step if no dictionary is displayed in the list.
Added Dictionaries
This area displays the blacklist and whitelist dictionary you have selected.
Image Moderation Configuration
This parameter is mandatory when Review mode is set to Static image in Step 5.
Some review items and their matching thresholds (also called risk score threshold) are configured by default. If the thresholds are reached, the content will be flagged as risky.
You can select the review items as required. Click
to expand the configuration items and adjust the matching thresholds. - Click Next. The Other Configurations page is displayed. Set the parameters by referring to Table 4.
Table 4 Other parameters Parameter
Description
Violation Processing
Select Move to specified OBS directory. The system will move the files with violation content to a specified directory.
Bucket Name
If you select Move to specified OBS directory, select an OBS bucket name.
OBS Storage Path (Optional)
If you select Move to specified OBS directory, select an OBS bucket name and the OBS path.
Estimated Start Time
Click
to select the expected start time of the task. The task will start at the time you specify.The first execution will be a full review at Estimated Start Time.
Task Execution Frequency
Select a task execution period from the drop-down list box. Periodic tasks will be executed based on the configured frequency, for example, at 09:00 every day. Only new or modified files will be reviewed.
- Once: The task is executed only once at the specified expected start time.
- Daily: The task is executed once a day at the specified expected start time. It starts again 24 hours after the last execution. For example, if the task is executed for the first time at 09:00 on November 1, 2025, it will be executed for the second time at 09:00 on November 2, 2025.
- Weekly: The task is executed once a week at the specified expected start time. It starts again 7 days after the last execution. For example, if the task is executed for the first time at 09:00 on November 1, 2025, it will be executed for the second time at 09:00 on November 8, 2025.
- Monthly: The task is executed once a month at the specified expected start time. It starts again 30 days after the last execution. For example, if the task is executed for the first time at 09:00 on November 1, 2025, it will be executed for the second time at 09:00 on December 1, 2025.
- Click OK. If a success message is displayed in the upper right corner, the task is created.
- Log in to the DSC console.
- Click
in the upper left corner and select a region or project. - Choose AI Data Security > Content Identification. The Content Identification page is displayed.
- Click the Review Tasks tab to view the review tasks. Table 5 describes the parameters.
Table 5 Key parameters of the task list Parameter
Description
Task Name
Task name entered when the task is created.
Review Frequency
Execution period of the task, which can be once, daily, weekly, or monthly.
Review mode
The review mode can be text, static image, or OCR text.
Task configuration
The blacklist and whitelist configured when the task is created.
Planned Task
Planned execution time of the task.
Actual Start
Time when the task actually starts.
Completed
Time when the task execution is complete.
Task Status
Task status, which can be:
Tasks in the New task, Waiting, Completed, Completed (Current period), Stopped, or Abnormal states can be edited. Those in the Waiting or Running states cannot be edited.
- New task: If the configured start time has not arrived, you can click Start to queue the task.
- Waiting: The task has been submitted and is waiting for the system to allocate review resources. Tasks are processed in the order they are submitted.
- Running: The task starts and the system is checking the content.
- Completed: The task is completed. You can export the check report.
- Completed (Current period): The task in the current period is complete. You can export the check report.
- Stopped: The task is stopped. You can click Start to execute it again.
- Abnormal: The task fails, times out, or encounters a system error. You can click Details to view the detailed error information.
Export Status
The check report status, which can be:
- Not Exported: The review report is waiting to be exported. You can click Export in the Operation column to export the review report.
- Exporting: The review report is being exported.
- Exported: The review report is exported.
- Error: The review report fails to be exported.
The review reports can be exported only for tasks whose status is Completed or Completed (Current period) and whose export status is not Exporting.
- Log in to the DSC console.
- Click
in the upper left corner and select a region or project. - Choose AI Data Security > Content Identification. The Content Identification page is displayed.
- On the Review Tasks tab page, click Export in the Operation column of a task.
- In the displayed dialog box, specify Bucket Name and OBS Storage Path.
- Click Export to export the audit result to the specified OBS path.
- Log in to the OBS console and check the review report generated in the corresponding OBS bucket path.
Managing the Dictionary
In addition to the Huawei Cloud built-in dictionary, DSC also supports blacklist and whitelist dictionary. You can add blacklist and whitelist dictionary to the review algorithm as required. Each user can upload a maximum of five blacklist dictionaries and five whitelist dictionaries. Each dictionary contains a maximum of 5,000 entries, with an upper limit of 64 UTF-8 characters per entry. If the type of a word in the user-defined blacklist dictionary is different from that in the built-in dictionary, the user-defined one takes precedence.
- Log in to the DSC console.
- Click
in the upper left corner and select a region or project. - Choose AI Data Security > Content Identification. The Content Identification page is displayed.
- On the Dictionary Management tab page, click Add Blacklist/Whitelist Dictionary in the upper left corner. The Add Dictionary dialog box is displayed.
- Configure the dictionary information by referring to Table 6. Figure 3 Adding a dictionary
Table 6 Dictionary parameters Parameter
Description
Dictionary Name
Enter a dictionary name. The name can contain only letters, digits, underscores (_), hyphens (-), and periods (.). It must start with a letter.
Dictionary Type
Select a dictionary type, blacklist or whitelist, from the drop-down list.
- Blacklist: You can select a type for the keywords in the blacklist.
- Whitelist: You can use regular expressions to match the keywords in the whitelist.
Input Mode
You can select Manual or Upload CSV File.
- Manual: Add keywords on this page.
- Upload CSV File: Enter keyword information in the template and upload the template to import the keywords.
Whitelist Keywords
This parameter is displayed when Input Mode is set to Manual and Dictionary Type is set to Whitelist.
- Enter a keyword according to the selected Rule, which can be Combined match, Independent match, or Regular expression.
- Click Add in the lower left corner of the page to add a keyword record. A maximum of 5,000 keyword records can be added.
- Click Delete in the Operation column to delete a single keyword. To delete all, click Delete All in the upper left corner above the list.
Blacklist Keywords
This parameter is displayed when Input Mode is set to Manual and Dictionary Type is set to Blacklist.
- Enter a sensitive word according to the selected Rule, which can be Combined match or Independent match. Select a type from the drop-down list.
- Click Add in the lower left corner of the page to add a keyword record. A maximum of 5,000 keyword records can be added.
- Click Delete in the Operation column to delete a single keyword. To delete all, click Delete All in the upper left corner above the list.
Upload CSV File
This parameter is displayed when Input Mode is set to Upload CSV File.
- Click Download template to download a CSV template and fill in keyword information as required.
- Click Add to upload the CSV file. Multiple files can be added.
Description (Optional)
Enter a description of the dictionary.
- Click Save. If a success message is displayed in the upper right corner, the dictionary is created.
Checking the Dictionary
- Log in to the DSC console.
- Click
in the upper left corner and select a region or project. - Choose AI Data Security > Content Identification. The Content Identification page is displayed.
- On the Dictionary Management tab page, check your added blacklist and whitelist dictionaries. For details, see Table 7. Figure 4 Dictionary example
Related Operations
- Starting a task: Click Start in the Operation column to start a review task.
- Deleting a review task:
- Editing a review task:
Only tasks whose Task Status is New task, Completed, Completed (Current period), Stopped, or Abnormal can be edited.
Locate the target review task and click Edit in the Operation column. On the displayed drawer, modify the task information.
- Deleting a dictionary:
On the Dictionary Management tab page, locate the target dictionary and click Delete in the Operation column.
- Editing a dictionary:
On the Dictionary Management tab page, locate the target dictionary and click Edit in the Operation column. On the displayed page, modify the dictionary information.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot
