Subscribing to Dynamic Masking Policies

Prerequisites

• A dynamic masking policy of a third-party platform has been released to the Kafka message queue, and the message format meets requirements. For details, see Reference: Kafka Message Format Requirements.
• An MRS Kafka data connection has been created in Management Center. For details, see Creating a Data Connection. The Kafka must be the Kafka where the third-party platform releases a message. The account in the data connection must have the permissions of the kafkaadmin user group.

Constraints

• Only the DAYU Administrator, Tenant Administrator, or data security administrator can create, edit, start, stop, or synchronize dynamic masking subscription tasks. Other common users do not have permission to perform these operations.
• You can only subscribe to the dynamic masking policies for MRS Hive on third-party platforms. The dynamic masking policies support only the masking rules supported by DataArts Security. The following rules are not supported: Custom/Show First x and Last y Characters and Custom/Mask First x and Last y Characters. For details, see Table 2.
• The name of the dynamic masking policy generated by the subscription is the policy name in the Kafka message. DataArts Security does not allow duplicate policy names. Ensure that no dynamic masking policy name is the same as any policy name in the Kafka message.
• After the dynamic masking policy generated by the subscription is synchronized to Ranger, the policy name is dlsMasking-Database name-Table name-Column name. Ranger does not allow duplicate policy names. Ensure that no existing policy name in Ranger is the same as the name of any generated policy.
• During dynamic masking subscription, DataArts Security uses the MRS cluster in the subscription task and the database, table, and column in the Kafka message dynamic masking policy to identify a dynamic masking policy. If a dynamic masking policy for the same table column in the same cluster's database already exists in the message queue or DataArts Security, the policy is skipped and will not be generated.

• The SM3, Custom/Show First x and Last y Characters, and Custom/Mask First x and Last y Characters masking rules for the MRS Hive data source are not provided by the MRS Ranger component. Instead, they are implemented by UDF-defined functions. Therefore, to use any of the three masking rules, you must upload the JAR package on which the algorithm depends to the MRS cluster, and grant the UDF creation permission to the account in the Ranger data connection and the UDF usage permission to all users in advance. For details, see Reference: Configuring UDF-related Permissions in the Ranger Component.

• DataArts Security can consume a Kafka message only if the message format meets the requirements described in Reference: Kafka Message Format Requirements.
  • If the Kafka message does not meet the message format requirements, the system records a synchronization failure message log and continues to consume the next message. The final status is partially failed or synchronization failed.
  • If the Kafka message is valid but fails to be consumed due to network resource issues, the consumption will be retried three times at intervals of 4, 6, and 9 seconds. If the message still fails to be consumed, a log will be recorded and the scheduling will be terminated.
  • If the Kafka message is valid and consumed properly, but a policy fails to be generated or synchronized to Ranger, the system records a synchronization failure message log and continues to consume the next message. The final status is partially failed or synchronization failed.
  • A maximum of 16 MB of failed Kafka messages can be stored.

Subscribing to Dynamic Masking Policies

1. On the DataArts Studio console, locate an instance and click Access. On the displayed page, locate a workspace and click DataArts Security.
   Figure 2 DataArts Security
   

2. In the navigation pane on the left, choose Dynamic Masking. On the displayed page, click the Dynamic Desensitization Subscription tab.
   Figure 3 Dynamic Desensitization Subscription tab
   

3. Click Create Subscription. In the displayed slide-out panel, set the parameters listed in Table 1.
   Figure 4 Parameters for creating a subscription
   
   The following table lists the parameters for creating a dynamic masking subscription.

   Table 1 Parameters

   Parameter	Description
   Connection Settings
   *Select Cluster	Select the cluster to which a dynamic masking policy of a third-party platform will be synchronized. Currently, a policy cannot be synchronized to multiple clusters. If you want to do so by creating multiple subscription tasks, Kafka messages will fail to be consumed due to duplicate policy names.
   Cluster Type	You do not need to set this parameter. The system automatically sets it based on the cluster you select. Currently, policies can only be synchronized to an MRS cluster.
   Data Connection	You do not need to set this parameter. The system automatically sets it based on the cluster you select.
   *Kafka Data Connection	Select the MRS Kafka connection created in Prerequisites. The Kafka must be the Kafka where the third-party platform releases a message. The account in the Kafka connection must have the permissions of the kafkaadmin user group.
   *Topic Subject	Select the topic of the Kafka message released for the dynamic masking policy of the third-party platform. A topic in the same MRS cluster can correspond to only one subscription task.
   Scheduling Settings
   Scheduling Time	Select the time period every day during which tasks will be scheduled. Set an appropriate time period based on the number of messages. Currently, it takes about two seconds to consume and synchronize a piece of data.
   Scheduling Period	Set whether to schedule tasks by hour or minute.
   Schedule Interval	Select the interval at which tasks are shceduled.

4. After setting all required parameters, click OK. Then click Start to start task scheduling.

Related Operations

• Starting or stopping a subscription task: On the Dynamic Desensitization Subscription tab page, locate a subscription task and click Start or Stop in the Operation column.
• Editing a subscription task: On the Dynamic Desensitization Subscription tab page, locate a subscription task, click More in the Operation column, and select Edit.
• Deleting subscription tasks: On the Dynamic Desensitization Subscription tab page, locate a subscription task, click More in the Operation column, and select Delete. To delete multiple tasks, select them and click Delete above the task list.
  

  The deletion operation cannot be undone. Exercise caution when performing this operation.

• Synchronizing a subscription task: On the Dynamic Desensitization Subscription tab page, locate a subscription task, click More in the Operation column, and select Synchronize. After that, DataArts Security consumes the message, generates a policy, and synchronizes the policy to Ranger.
• Viewing subscription task details: On the Dynamic Desensitization Subscription tab page, locate a task, and click Details in the Operation column to view the task details.
  Figure 5 Viewing task details
  

Reference: Kafka Message Format Requirements

Dynamic masking policies of third-party platforms need to be released to a Kafka message queue, and the message format must meet requirements. The following is a message template with parameters.

{ 
  "mask_policy_template": 
  {
      "create_time":1692839884000  //Synchronization time
      "name":" task1", //Name of the dynamic masking policy, which cannot be the same as the name of any existing dynamic masking policy
      "database": "1", //Database name
      "table": "1", //Data table name
      "column": "1", //Field name
      "column_type":"int", //Field type
      "data_level": "1", //Field security level, which is optional
      "algorithm_config": {
        "name": "SM3", //Dynamic masking rule name, which can be MASK, MASK_SHOW_LAST_4, MASK_SHOW_FIRST_4, MASK_HASH, MASK_DATE_SHOW_YEAR, MASK_NULL, or SM3
        "type": "HASH", //Dynamic masking rule type, which is MASK for all rules except the SM3 rule whose type is HASH
        "description": "Encryption using the SM3 algorithm", //Dynamic masking rule description
      },
      "datasource_type":"HIVE", //Data source type, which can only be Hive
      "users":"aaa,bbb",  //Masking users
      "user_groups":"ggg"  //Masking user groups
      "description":{
           "jdbc_url": "hive2://xxx" //Custom description, which is contained in a failure message
      }  
   }
}