Configuring an Ingested Data Detection Rule
Ingested data detection rules can be used to detect sensitive information in real time in the following scenarios:
- Real-time detection of sensitive data in tables during table data migration in DataArts Migration (offline jobs). For details, see Configuring an Offline Processing Migration Job.

In the new version mode, you can configure ingested data detection rules only when you are using the enterprise version. In the old version mode, this function is supported in the basic version or an advanced version.
Prerequisites
- Identification rules have been created. For details, see Creating Identification Rules.
Notes and Constraints
- In the table data migration in DataArts Migration (offline jobs) and sensitive data discovery in DataArts Security, only MRS Hive, DWS, DLI, and RDS for MySQL data sources are supported.
- Sensitive table data detection in DataArts Migration (offline jobs) has the following requirements on the table content format:
- A maximum of 500 fields are allowed in a table.
- For table fields in string format, sensitive information in the first 1,000 characters is detected. The excess part will be truncated.
- Only built-in and custom identification rules in regular expression format are supported. A maximum of 50 detection rules are supported.
Configuring the Job Migration Strategy
To use an ingested data detection rule to detect sensitive data in tables in DataArts Migration (offline jobs) in real time, you need to enable and configure the job migration strategy
- On the DataArts Studio console, locate a workspace and click DataArts Security.
- In the navigation pane on the left, choose . On the displayed page, click the Ingested Data Detection Rules tab.
- Set the job migration strategy parameters listed in Table 1.
Figure 1 Configuring the ingested data detection rule
Table 1 Job migration strategy parameters Parameter
Description
Detect Sensitive Information in Jobs in Real Time
Whether to enable real-time detection of sensitive table data during table data migration in DataArts Migration (offline jobs). This function is disabled by default.
Sensitive Information Interception Policy
The following interception policies are available if sensitive information is detected in offline migration jobs in DataArts Migration:
- Strong interception: Jobs cannot be saved as long as sensitive information is detected.
- Week interception: Sensitive fields for which encryption and decryption have been configured or those that have been masked are not intercepted.
- No interception: Sensitive information is not intercepted.
Sampling
Number of rows sampled from the beginning during table field detection in an offline job in DataArts Migration. A maximum of 100 rows can be sampled.
Synchronise
Click Synchronise to synchronize the rule policy to DataArts Migration (offline jobs). The rule policy takes effect only after it is synchronized to DataArts Migration (offline jobs).
Configuring the File Detection Strategy
The file detection strategy is enabled by default. To customize configurations, perform the following steps:
- On the DataArts Studio console, locate a workspace and click DataArts Security.
- In the navigation pane on the left, choose . On the displayed page, click the Ingested Data Detection Rules tab.
- Set the file detection strategy parameters listed in Table 2.
Figure 2 Configuring the ingested data detection rule
Table 2 File detection strategy parameters Parameter
Description
Blank Line Strategy
How to process the blank lines in the sampled fields. The default value is Include in Calculation.
- Ignore: If M lines are sampled, and N lines among them are sensitive, the sensitive information rate is N/M x 100%.
- Include in Calculation: If M lines are sampled, m lines among them are blank, and N lines are sensitive, the sensitive information rate is N/(M – m) x 100%.
Sampling
The number of lines to be sampled during real-time detection of sensitive data in a file. The default value is 100, and the maximum value is 500.
Configuring an Identification Rule
- On the DataArts Studio console, locate a workspace and click DataArts Security.
- In the navigation pane on the left, choose . On the displayed page, click the Ingested Data Detection Rules tab.
- On the Ingested Data Detection Rules page, click Configure Rule.
Figure 3 Ingested Data Detection Rules pageFigure 4 Configuring the ingested data detection rule
- In the displayed Rule Settings dialog box, select required identification rules and click Submit.
Figure 5 Configuring rules
Related Operations
- Deleting ingested data detection rules: On the Ingested Data Detection Rules page, locate a rule and click Delete in the Operation column. To delete multiple rules, select them and click Delete above the list.
The deletion operation cannot be undone. Exercise caution when performing this operation.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot