หน้านี้ยังไม่พร้อมใช้งานในภาษาท้องถิ่นของคุณ เรากำลังพยายามอย่างหนักเพื่อเพิ่มเวอร์ชันภาษาอื่น ๆ เพิ่มเติม ขอบคุณสำหรับการสนับสนุนเสมอมา

Compute
Elastic Cloud Server
Huawei Cloud Flexus
Bare Metal Server
Auto Scaling
Image Management Service
Dedicated Host
FunctionGraph
Cloud Phone Host
Huawei Cloud EulerOS
Networking
Virtual Private Cloud
Elastic IP
Elastic Load Balance
NAT Gateway
Direct Connect
Virtual Private Network
VPC Endpoint
Cloud Connect
Enterprise Router
Enterprise Switch
Global Accelerator
Management & Governance
Cloud Eye
Identity and Access Management
Cloud Trace Service
Resource Formation Service
Tag Management Service
Log Tank Service
Config
OneAccess
Resource Access Manager
Simple Message Notification
Application Performance Management
Application Operations Management
Organizations
Optimization Advisor
IAM Identity Center
Cloud Operations Center
Resource Governance Center
Migration
Server Migration Service
Object Storage Migration Service
Cloud Data Migration
Migration Center
Cloud Ecosystem
KooGallery
Partner Center
User Support
My Account
Billing Center
Cost Center
Resource Center
Enterprise Management
Service Tickets
HUAWEI CLOUD (International) FAQs
ICP Filing
Support Plans
My Credentials
Customer Operation Capabilities
Partner Support Plans
Professional Services
Analytics
MapReduce Service
Data Lake Insight
CloudTable Service
Cloud Search Service
Data Lake Visualization
Data Ingestion Service
GaussDB(DWS)
DataArts Studio
Data Lake Factory
DataArts Lake Formation
IoT
IoT Device Access
Others
Product Pricing Details
System Permissions
Console Quick Start
Common FAQs
Instructions for Associating with a HUAWEI CLOUD Partner
Message Center
Security & Compliance
Security Technologies and Applications
Web Application Firewall
Host Security Service
Cloud Firewall
SecMaster
Anti-DDoS Service
Data Encryption Workshop
Database Security Service
Cloud Bastion Host
Data Security Center
Cloud Certificate Manager
Edge Security
Situation Awareness
Managed Threat Detection
Blockchain
Blockchain Service
Web3 Node Engine Service
Media Services
Media Processing Center
Video On Demand
Live
SparkRTC
MetaStudio
Storage
Object Storage Service
Elastic Volume Service
Cloud Backup and Recovery
Storage Disaster Recovery Service
Scalable File Service Turbo
Scalable File Service
Volume Backup Service
Cloud Server Backup Service
Data Express Service
Dedicated Distributed Storage Service
Containers
Cloud Container Engine
Software Repository for Container
Application Service Mesh
Ubiquitous Cloud Native Service
Cloud Container Instance
Databases
Relational Database Service
Document Database Service
Data Admin Service
Data Replication Service
GeminiDB
GaussDB
Distributed Database Middleware
Database and Application Migration UGO
TaurusDB
Middleware
Distributed Cache Service
API Gateway
Distributed Message Service for Kafka
Distributed Message Service for RabbitMQ
Distributed Message Service for RocketMQ
Cloud Service Engine
Multi-Site High Availability Service
EventGrid
Dedicated Cloud
Dedicated Computing Cluster
Business Applications
Workspace
ROMA Connect
Message & SMS
Domain Name Service
Edge Data Center Management
Meeting
AI
Face Recognition Service
Graph Engine Service
Content Moderation
Image Recognition
Optical Character Recognition
ModelArts
ImageSearch
Conversational Bot Service
Speech Interaction Service
Huawei HiLens
Video Intelligent Analysis Service
Developer Tools
SDK Developer Guide
API Request Signing Guide
Terraform
Koo Command Line Interface
Content Delivery & Edge Computing
Content Delivery Network
Intelligent EdgeFabric
CloudPond
Intelligent EdgeCloud
Solutions
SAP Cloud
High Performance Computing
Developer Services
ServiceStage
CodeArts
CodeArts PerfTest
CodeArts Req
CodeArts Pipeline
CodeArts Build
CodeArts Deploy
CodeArts Artifact
CodeArts TestPlan
CodeArts Check
CodeArts Repo
Cloud Application Engine
MacroVerse aPaaS
KooMessage
KooPhone
KooDrive
Help Center/ DataArts Studio/ Best Practices/ Comparing Data Before and After Data Migration Using DataArts Quality

Comparing Data Before and After Data Migration Using DataArts Quality

Updated on 2024-11-12 GMT+08:00

Data comparison checks data consistency before and after migration or processing.

This section describes how to use the DataArts Quality module of DataArts Studio to check consistency of the data before and after it is migrated from DWS to an MRS Hive partitioned table.

Prerequisites

  • You have created a DWS cluster, which can communicate with the DataArts Studio instance. You have the permission to access the KMS key.
  • You have created an MRS cluster which can communicate with the DataArts Studio instance.
  • You have created a CDM cluster. For details, see Buying a DataArts Migration Incremental Package.

Creating a Data Migration Link

  1. Log in to the DataArts Studio console, locate a workspace, and click DataArts Migration.
  2. On the Cluster Management page, locate the prepared CDM cluster and click Job Management in the Operation column.

    Figure 1 Job Management page

  3. Click the Links tab and Create Link to create a DWS link. For details about the parameters, see Link to DWS.

    Figure 2 Creating a DWS link

  4. Create an MRS Hive link. For details about the parameters, see Link to Hive.

    Figure 3 Creating an MRS Hive link

Creating and Executing a Data Migration Job

  1. Log in to the DataArts Studio console, locate a workspace, and click DataArts Migration.
  2. On the Cluster Management page, locate the prepared CDM cluster and click Job Management in the Operation column.
  3. Click the Table/File Migration tab and then Create Job to create a data migration job.
  4. Select the DWS link for Source Link Name and MRS Hive link for Destination Link Name, and set required parameters. For details about the parameters, see From DWS and To Hive.

    Figure 4 Job configuration

  5. Configure the field mapping and task and click Save and Execute to execute the CDM job.
  6. On the Table/File Migration job list, view the job status.

    Figure 5 Viewing the job status

Creating a Data Connection

  1. Log in to the DataArts Studio console, locate a workspace, and click Management Center.
  2. Click Create Data Connection to create a DWS data connection. For details about the parameters, see Creating a DWS Connection.

    Figure 6 Creating a DWS data connection

  3. Create an MRS Hive data connection. For details about the parameters, see Creating an MRS Hive Connection.

    Figure 7 Creating an MRS Hive data connection

Creating a Comparison Job

  1. Log in to the DataArts Studio console, locate a workspace, and click DataArts Quality.
  2. In the left navigation pane, choose Quality Monitoring > Comparison Jobs.
  3. Click Create. On the Create Comparison Job page, set basic information.

    Figure 8 Setting basic information for the comparison job

  4. Click Next to go to the Define Rule page. Click , configure the comparison rule, select the data tables before and after the migration, and configure the alarm rule.

    Figure 9 Configuring the comparison rule

    NOTE:
    • Configure the source and destination information separately.
    • When configuring Alarm Condition, ${1_1} indicates the number of rows in the source table, and ${2_1} indicates the number of rows in the destination table. In the preceding figure, the alarm condition ${1_1}!=${2_1} indicates that an alarm is generated when the number of rows in the source table is inconsistent with that in the destination table.

  5. Click Next and set subscription information.

    Figure 10 Setting subscription information
    NOTE:

    If you enable notification, Alarm triggered indicates that a notification is sent to the SMN topic when an alarm is generated for the job, and Run successfully indicates that a notification is sent to the SMN topic when no alarm is generated for the job.

  6. Click Next and set scheduling parameters.

    Figure 11 Setting scheduling parameters
    NOTE:

    Once indicates that the job needs to be manually executed, and On schedule indicates that the job is executed automatically based on your configuration. The configuration in the preceding figure indicates that the job is automatically executed every 15 minutes.

  7. Click Submit to create the comparison job.

Executing the Comparison Job and Viewing the Result Analysis

  1. In the left navigation pane, choose Quality Monitoring > Comparison Jobs.
  2. Locate the created comparison job and click Run in the Operation column.

    Figure 12 Running the comparison job

  3. In the left navigation pane, choose Quality Monitoring > O&M.

    Figure 13 O&M page

  4. After the job is executed, click Details in the Operation column. If the source and destination tables have the same number of rows, the migration is successful.

    Figure 14 Viewing the running result
    NOTE:
    • In the running result, the left pane displays the execution result of the rule for source table rows, and the right pane displays the execution result of the rule for destination table rows.
    • The error rate indicates the difference between the number of rows of the source and destination tables. If the error rate is 0, the source and destination tables have the same number of rows.

เราใช้คุกกี้เพื่อปรับปรุงไซต์และประสบการณ์การใช้ของคุณ การเรียกดูเว็บไซต์ของเราต่อแสดงว่าคุณยอมรับนโยบายคุกกี้ของเรา เรียนรู้เพิ่มเติม

Feedback

Feedback

Feedback

0/500

Selected Content

Submit selected content with the feedback