Data Layer Migration
- Object storage migration
Object storage is suitable for storing unstructured data, such as documents, texts, images, XML and HTML files, reports, and audio and videos. The following figure shows how to migrate data at different scales.
Figure 1 Object storage migrationTable 1 Object storage migration methods Item
Method
Scenario
Remarks
Object storage migration
Copying data to OBS
The data volume is small, around a few gigabytes.
-
Migrating and retrieving data using OMS
The data volume is large, ranging from terabytes to petabytes, with significant increases.
Recommended. Huawei Cloud offers the technical support.
Copying DES disks offline
The data volume is large, with terabytes of incremental data.
-
- Migrating and retrieving data using OMS: OMS is a user-friendly and efficient online data migration service. It uses the SDK of your current object storage to quickly move, encrypt, and copy data to Huawei Cloud OBS. This makes it easy to transfer object storage data from other cloud providers to Huawei Cloud. The first full migration moves all data from your current object storage to Huawei Cloud OBS. When switching services, set up retrieval rules on Huawei Cloud to move new data to Huawei Cloud OBS. After the switch, run another full migration. In this step, already moved objects are skipped, and only the new data is transferred to Huawei Cloud OBS.
Use the OMS to view and manage migration tasks, create task groups, assess bucket data, and check audit logs. See the Object Storage Migration Service Documentation for details.
Figure 2 OMS - Copying DES disks offline
The Data Express Service (DES) helps you move data to Huawei Cloud. It can handle terabytes or petabytes of data. DES uses physical storage devices like USB drives or eSATA disks, or a tool called Teleport, to transfer large amounts of data quickly. This solves issues with high cloud bandwidth costs and long upload times.
For details, see the Data Express Service Documentation.
Huawei Cloud offers custom migration services for moving storage data. Their expert teams create tailored migration plans and offer complete support. They assist with both full and incremental data moves, ensuring accuracy at each step. This service helps transfer data from local storage to the cloud.
- File storage migration
Network Attached Storage (NAS) is a file system for sharing, scaling, and reliable, fast data access.
Scalable File Service (SFS) offers scalable, high-performance file storage over a network. It allows shared file access between multiple Elastic Cloud Servers (ECSs), Bare Metal Servers (BMSs), and containers on Cloud Container Engine (CCE) and Cloud Container Instance (CCI).
If the network is connected during NAS migration, you can mount files to multiple platforms. Use Rclone or Rsync to copy files from the source directory to the SFS or SFS Turbo file system on the transit host.
Figure 3 File migrationRclone is a CLI tool for synchronizing, uploading, and downloading data between various cloud storage and web disks. It allows users to set up multiple threads and concurrent tasks, which speeds up migrations and reduces time. Rclone also checks and synchronize data, copying files from one system to another for NAS file migration.
Remote synchronize (Rsync) is a Remote Copy Protocol (RCP)-based remote file synchronization tool that uses its own Rsync algorithm.
Figure 4 Rsync migrationRsync is enabled on each service platform. The Rsync client script is set up on the file server. A scheduled task, like Crontab on Linux, syncs files from the service server to the disk array incrementally (fully the first time). It has the following advantages:
- Incremental file synchronization is fast, efficient, and uses fewer resources.
- It supports network access via SSH and RSH.
- Stopping the Rsync service does not affect other services.
- It supports mirrored data copies, ideal for distributing static files online with good scalability.
- Database migration
Databases are classified into relational databases and non-relational databases. Relational databases are crucial because they keep data consistent. To move your database to Huawei Cloud, choose one of these methods:
- Use SMS host migration. If your source is a self-built database on an ECS, migrate the ECS database server to Huawei Cloud's ECS. SMS makes this easy but will pause database services during the process. For minimal downtime, use database backups or synchronization.
- Use backup and restoration. This method involves saving the source data, stopping the source database, performing a full backup, keeping necessary logs, and restoring the data on the target system.
- Use Data Replication Service (DRS) for online migration and real-time synchronization. DRS works across cloud platforms, on-premises to cloud, and between regions. It is a reliable, secure, and efficient service that simplifies data flow and reduces transmission costs. It simplifies the data flow between databases, significantly reducing data transmission costs.
DRS supports various data sources and offers real-time migration, backup migration, real-time synchronization, data subscription, and DR. For details, see the Data Replication Service Documentation.
Figure 5 DRS migrationThe following are common best practices for database migration. See DRS Best Practices Summary for more best practices.
Table 2 Best practices for database migration Source DB
Destination DB
Best Practice
Other cloud MySQL databases
Huawei Cloud RDS for MySQL
Migrating MySQL Databases from Other Clouds to RDS for MySQL
Huawei Cloud GaussDB(for MySQL)
Other cloud MongoDB
Huawei Cloud DDS
On-premises MySQL
Huawei Cloud RDS for MySQL
ECS-hosted MySQL
Huawei Cloud GaussDB(for MySQL)
ECS-hosted MongoDB
Huawei Cloud DDS
On-premises MySQL
Huawei Cloud RDS for MySQL
On-premises MongoDB
Huawei Cloud DDS
Huawei Cloud RDS for MySQL
Distributed Database Middleware (DDM)
MySQL schema and logic table
Distributed Database Middleware (DDM)
On-premises Microsoft SQL Server
Huawei Cloud RDS for SQL Server
Migrating Microsoft SQL Server Backup Data to RDS SQL Server DB Instance
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot