Using the HetuEngine Dynamic Masking Function

Background

Ranger supports static masking, but data may be unavailable after static masking. The symptoms are as follows:

• Masked data cannot be filtered by predicate.
• Joint operations such as join queries cannot be performed on the masked data.
• Only columns of the string type can be masked.

Static data masking has significant functional restrictions. Data is pre-masked before the computation stage. To provide flexible data masking, HetuEngine supports masked data processing and dynamic masking by automatically synchronizing masking policies within the cluster.

Scenarios

You can configure HetuEngine masking policies through MRS Ranger. HetuEngine supports data masking capabilities, such as masked data processing and automatic synchronization of masking policies across a cluster for query and write scenarios.

1. After data masking is configured, tables can be queried. Values in the masked columns are automatically masked. Predicate matching can be performed for the columns configured for masking, and only the result is masked.
2. The columns configured with masking can be used for joint queries (such as Join, Union, In, and Exist). Only the final result is masked.
3. Data in the new table generated by the CTS operation is automatically masked. If Hive in the local cluster is the data source, the data is not masked, the masking policy is synchronized, and the query result will be masked based on the synchronized policy. Data from other data sources is masked before being output.
4. Data in new tables created after an ITS operation supports masking. If Hive in the local cluster is the data source, the data is not masked, the masking policy is synchronized, and the query result will be masked based on the synchronized policy. Data from other data sources is masked before being output. When the masking policy is synchronized, the following scenarios may occur: If the target column does not have a masking policy, the policy is automatically synchronized. If the target table has an existing masking policy that matches the source table, the policy remains unchanged. If the target table has a masking policy that conflicts with the source table, the target table policy is overwritten with ***.
5. Queries on views that reference masked columns return masked results.
6. After masking is configured, the results of calculated functions, nested functions, and nested subqueries are automatically masked.
7. Queries using aliases and CTS statements using aliases support data masking.

Notes and Constraints

• Dynamic data masking is available only in clusters in security mode.
• When a CUSTOM masking policy is configured in Ranger, dynamic masking replaces the data with asterisks (*). For example, ABCDE is displayed as *****.
• The system detects Ranger policy updates every 30 seconds. If the execution of statements such as CTS involves policy synchronization, there will be a delay before the new policy on Ranger takes effect.
• In a cross-region scenario, the remote HetuEngine functions as the data source of the local cluster. You must configure the masking policy directly on the local cluster.
• When a view is queried, the view owner accesses the table. When creating a view for a sensitive data table, you must apply masking for the view creator.

Enabling Dynamic Masking

1. Log in to FusionInsight Manager as a user who can access the HetuEngine web UI and choose Cluster > Services > HetuEngine.
2. In the Basic Information area on the Dashboard page, click the link next to HSConsole WebUI to access the HSConsole page.
3. On the Compute Instance tab, locate the row containing the tenant to which the desired instance belongs and click Configure in the Operation column.
4. In the Custom Configuration area, click Add to add the following parameters:

   Table 1 HetuEngine dynamic masking parameters

   Parameter	Example Value	File	Description
   hetu.dynamic.masked.enabled	true	coordinator.config.properties worker.config.properties	Whether to enable HetuEngine dynamic masking. Default value: false

5. Set Start Now to Yes and click OK.
   

   • Both dynamic and static data masking are performed based on Ranger data masking policies. You must enable Ranger for the HetuEngine service.
   • For details about the Ranger masking policies and configuration method, see HetuEngine Data Masking.