Compute
Elastic Cloud Server
Huawei Cloud Flexus
Bare Metal Server
Auto Scaling
Image Management Service
Dedicated Host
FunctionGraph
Cloud Phone Host
Huawei Cloud EulerOS
Networking
Virtual Private Cloud
Elastic IP
Elastic Load Balance
NAT Gateway
Direct Connect
Virtual Private Network
VPC Endpoint
Cloud Connect
Enterprise Router
Enterprise Switch
Global Accelerator
Management & Governance
Cloud Eye
Identity and Access Management
Cloud Trace Service
Resource Formation Service
Tag Management Service
Log Tank Service
Config
OneAccess
Resource Access Manager
Simple Message Notification
Application Performance Management
Application Operations Management
Organizations
Optimization Advisor
IAM Identity Center
Cloud Operations Center
Resource Governance Center
Migration
Server Migration Service
Object Storage Migration Service
Cloud Data Migration
Migration Center
Cloud Ecosystem
KooGallery
Partner Center
User Support
My Account
Billing Center
Cost Center
Resource Center
Enterprise Management
Service Tickets
HUAWEI CLOUD (International) FAQs
ICP Filing
Support Plans
My Credentials
Customer Operation Capabilities
Partner Support Plans
Professional Services
Analytics
MapReduce Service
Data Lake Insight
CloudTable Service
Cloud Search Service
Data Lake Visualization
Data Ingestion Service
GaussDB(DWS)
DataArts Studio
Data Lake Factory
DataArts Lake Formation
IoT
IoT Device Access
Others
Product Pricing Details
System Permissions
Console Quick Start
Common FAQs
Instructions for Associating with a HUAWEI CLOUD Partner
Message Center
Security & Compliance
Security Technologies and Applications
Web Application Firewall
Host Security Service
Cloud Firewall
SecMaster
Anti-DDoS Service
Data Encryption Workshop
Database Security Service
Cloud Bastion Host
Data Security Center
Cloud Certificate Manager
Edge Security
Situation Awareness
Managed Threat Detection
Blockchain
Blockchain Service
Web3 Node Engine Service
Media Services
Media Processing Center
Video On Demand
Live
SparkRTC
MetaStudio
Storage
Object Storage Service
Elastic Volume Service
Cloud Backup and Recovery
Storage Disaster Recovery Service
Scalable File Service Turbo
Scalable File Service
Volume Backup Service
Cloud Server Backup Service
Data Express Service
Dedicated Distributed Storage Service
Containers
Cloud Container Engine
SoftWare Repository for Container
Application Service Mesh
Ubiquitous Cloud Native Service
Cloud Container Instance
Databases
Relational Database Service
Document Database Service
Data Admin Service
Data Replication Service
GeminiDB
GaussDB
Distributed Database Middleware
Database and Application Migration UGO
TaurusDB
Middleware
Distributed Cache Service
API Gateway
Distributed Message Service for Kafka
Distributed Message Service for RabbitMQ
Distributed Message Service for RocketMQ
Cloud Service Engine
Multi-Site High Availability Service
EventGrid
Dedicated Cloud
Dedicated Computing Cluster
Business Applications
Workspace
ROMA Connect
Message & SMS
Domain Name Service
Edge Data Center Management
Meeting
AI
Face Recognition Service
Graph Engine Service
Content Moderation
Image Recognition
Optical Character Recognition
ModelArts
ImageSearch
Conversational Bot Service
Speech Interaction Service
Huawei HiLens
Video Intelligent Analysis Service
Developer Tools
SDK Developer Guide
API Request Signing Guide
Terraform
Koo Command Line Interface
Content Delivery & Edge Computing
Content Delivery Network
Intelligent EdgeFabric
CloudPond
Intelligent EdgeCloud
Solutions
SAP Cloud
High Performance Computing
Developer Services
ServiceStage
CodeArts
CodeArts PerfTest
CodeArts Req
CodeArts Pipeline
CodeArts Build
CodeArts Deploy
CodeArts Artifact
CodeArts TestPlan
CodeArts Check
CodeArts Repo
Cloud Application Engine
MacroVerse aPaaS
KooMessage
KooPhone
KooDrive

Subscribing to Dynamic Masking Policies

Updated on 2025-02-18 GMT+08:00

You can synchronize dynamic masking policies from third-party platforms by subscribing to the policies.

After dynamic masking policies of third-party platforms are released to Kafka message queues, you can subscribe to and consume them in DataArts Security. If the message format meets requirements, DataArts Security generates a dynamic masking policy (whose name is the policy name in the Kafka message) and synchronizes the policy to the MRS Ranger component to make the policy take effect.

Figure 1 Dynamic masking policy subscription process

Note that dynamic masking subscriptions configured for a DataArts Studio instance are visible to and take effect for all the workspaces of the instance.

Prerequisites

  • A dynamic masking policy of a third-party platform has been released to the Kafka message queue, and the message format meets requirements. For details, see Reference: Kafka Message Format Requirements.
  • An MRS Kafka data connection has been created in Management Center. For details, see Creating a DataArts Studio Data Connection. The Kafka must be the Kafka where the third-party platform releases a message. The account in the data connection must have the permissions of the kafkaadmin user group.

Constraints

  • Only the DAYU Administrator, Tenant Administrator, or data security administrator can create, edit, start, stop, or synchronize dynamic masking subscription tasks. Other common users do not have permission to perform these operations.
  • You can only subscribe to the dynamic masking policies for MRS Hive on third-party platforms. The dynamic masking policies support only the masking rules supported by DataArts Security. The following rules are not supported: Custom/Show First x and Last y Characters and Custom/Mask First x and Last y Characters. For details, see Table 2.
  • The name of the dynamic masking policy generated by the subscription is the policy name in the Kafka message. DataArts Security does not allow duplicate policy names. Ensure that no dynamic masking policy name is the same as any policy name in the Kafka message.
  • After the dynamic masking policy generated by the subscription is synchronized to Ranger, the policy name is dlsMasking-Database name-Table name-Column name. Ranger does not allow duplicate policy names. Ensure that no existing policy name in Ranger is the same as the name of any generated policy.
  • During dynamic masking subscription, DataArts Security uses the MRS cluster in the subscription task and the database, table, and column in the Kafka message dynamic masking policy to identify a dynamic masking policy. If a dynamic masking policy for the same table column in the same cluster's database already exists in the message queue or DataArts Security, the policy is skipped and will not be generated.
  • DataArts Security can consume a Kafka message only if the message format meets the requirements described in Reference: Kafka Message Format Requirements.
    • If the Kafka message does not meet the message format requirements, the system records a synchronization failure message log and continues to consume the next message. The final status is partially failed or synchronization failed.
    • If the Kafka message is valid but fails to be consumed due to network resource issues, the consumption will be retried three times at intervals of 4, 6, and 9 seconds. If the message still fails to be consumed, a log will be recorded and the scheduling will be terminated.
    • If the Kafka message is valid and consumed properly, but a policy fails to be generated or synchronized to Ranger, the system records a synchronization failure message log and continues to consume the next message. The final status is partially failed or synchronization failed.
    • A maximum of 16 MB of failed Kafka messages can be stored.

Subscribing to Dynamic Masking Policies

  1. On the DataArts Studio console, locate a workspace and click DataArts Security.
  2. In the navigation pane on the left, choose Dynamic Masking. On the displayed page, click the Dynamic Desensitization Subscription tab.

    Figure 2 Dynamic Desensitization Subscription tab

  3. Click Create Subscription. In the displayed slide-out panel, set the parameters listed in Table 1.

    Figure 3 Parameters for creating a subscription

    The following table lists the parameters for creating a dynamic masking subscription.
    Table 1 Parameters

    Parameter

    Description

    Connection Settings

    *Select Cluster

    Select the cluster to which a dynamic masking policy of a third-party platform will be synchronized.

    Currently, a policy cannot be synchronized to multiple clusters. If you want to do so by creating multiple subscription tasks, Kafka messages will fail to be consumed due to duplicate policy names.

    Cluster Type

    You do not need to set this parameter. The system automatically sets it based on the cluster you select. Currently, policies can only be synchronized to an MRS cluster.

    Data Connection

    You do not need to set this parameter. The system automatically sets it based on the cluster you select.

    *Kafka Data Connection

    Select the MRS Kafka connection created in Prerequisites. The Kafka must be the Kafka where the third-party platform releases a message. The account in the Kafka connection must have the permissions of the kafkaadmin user group.

    *Topic Subject

    Select the topic of the Kafka message released for the dynamic masking policy of the third-party platform. A topic in the same MRS cluster can correspond to only one subscription task.

    Scheduling Settings

    Scheduling Time

    Select the time period every day during which tasks will be scheduled.

    Set an appropriate time period based on the number of messages. Currently, it takes about two seconds to consume and synchronize a piece of data.

    Scheduling Period

    Set whether to schedule tasks by hour or minute.

    Schedule Interval

    Select the interval at which tasks are scheduled.

  4. After setting all required parameters, click OK. Then click Start to start task scheduling.

Related Operations

  • Starting or stopping a subscription task: On the Dynamic Desensitization Subscription tab page, locate a subscription task and click Start or Stop in the Operation column.
  • Editing a subscription task: On the Dynamic Desensitization Subscription tab page, locate a subscription task, click More in the Operation column, and select Edit.
  • Deleting subscription tasks: On the Dynamic Desensitization Subscription tab page, locate a subscription task, click More in the Operation column, and select Delete. To delete multiple tasks, select them and click Delete above the task list.
    NOTE:

    The deletion operation cannot be undone. Exercise caution when performing this operation.

  • Synchronizing a subscription task: On the Dynamic Desensitization Subscription tab page, locate a subscription task, click More in the Operation column, and select Synchronize. After that, DataArts Security consumes the message, generates a policy, and synchronizes the policy to Ranger.
  • Viewing subscription task details: On the Dynamic Desensitization Subscription tab page, locate a task, and click Details in the Operation column to view the task details.
    Figure 4 Viewing task details

Reference: Kafka Message Format Requirements

Dynamic masking policies of third-party platforms need to be released to a Kafka message queue, and the message format must meet requirements. The following is a message template with parameters.

{ 
  "mask_policy_template": 
  {
      "create_time":1692839884000  //Synchronization time
      "name":" task1", //Name of the dynamic masking policy, which cannot be the same as the name of any existing dynamic masking policy
      "database": "1", //Database name
      "table": "1", //Data table name
      "column": "1", //Field name
      "column_type":"int", //Field type
      "data_level": "1", //Field security level, which is optional
      "algorithm_config": {
        "name": "MASK", //Dynamic masking rule name, which can be MASK, MASK_SHOW_LAST_4, MASK_SHOW_FIRST_4, MASK_HASH, MASK_DATE_SHOW_YEAR, or MASK_NULL
        "type": "MASK", //Type of the dynamic masking rule, which is MASK
        "description": "Mask letters and digits.", //Description of the dynamic masking rule
      },
      "datasource_type":"HIVE", //Data source type, which can only be Hive
      "users":"aaa,bbb",  //Masking users
      "user_groups":"ggg"  //Masking user groups
      "description":{
           "jdbc_url": "hive2://xxx" //Custom description, which is contained in a failure message
      }  
   }
}

We use cookies to improve our site and your experience. By continuing to browse our site you accept our cookie policy. Find out more

Feedback

Feedback

Feedback

0/500

Selected Content

Submit selected content with the feedback