Help Center/ Migration Center/ User Guide/ Application Migration Projects/ Storage Migration/ Batch File Storage Migration
Updated on 2026-09-20 GMT+08:00
View PDF
Share

Batch File Storage Migration

What Is Batch File Storage Migration?

This solution provides dedicated migration clusters and private lines to support batch creation of file storage data migration tasks, enabling efficient migration of file storage data. It is suitable for scenarios where resources in multiple file storage addresses need to be migrated. Using exclusive clusters will incur fees. For details, see Billing.

Warnings

When you create a workflow, there are three options for Overwrite Existing: Never, Always, and If older or different size. If you choose Never, restarting a migration after an interruption or pause may lead to incomplete data migration, even though the task may appear successful. This could impact data integrity, so use the Never option with caution.

Procedure

Table 1 introduces the complete process of batch file storage migration.

Table 1 Process of batch file storage migration

Procedure

Description

Preparations
  • Before using MgC, prepare a HUAWEI ID or an IAM user that can access MgC, and grant them relevant permissions.
  • Create a server migration project on the MgC console.
  • Create an SFS file system on the destination end for data storage and management after the migration.

Step 1: Create a Migration Cluster

Bind a cluster to the migration task for creating the task. If no cluster is available, create one on the Migration Clusters page.

Step 2: Create a Migration Task

Create a batch file storage migration task. At this step, you will create a task template for configuration later.

Step 3: Configure the Migration Task

Select source resources, set migration parameters, configure the migration cluster, and confirm the settings.

Step 4: Track the Migration Task

Start the migration task and check the migration status, phases, and steps.

Preparations

Prepare resources by referring to Table 2 before creating a task.

Table 2 Preparations

Item

Requirement

Preparing accounts and permissions
  1. Sign up for a HUAWEI ID, enable Huawei Cloud services, and complete real-name authentication. If you have already enabled Huawei Cloud services and completed real-name authentication, skip this step.
    1. Visit Huawei Cloud and click Sign Up.
    2. Sign up for a HUAWEI ID and enable Huawei Cloud services.
    3. (Optional) Complete real-name authentication. Real-name authentication is the process of submitting valid identity information (such as a passport or business license) for verification when you use cloud services in the Chinese mainland. There are individual real-name authentication and enterprise real-name authentication.

      According to relevant laws and regulations, unauthenticated accounts cannot purchase cloud services in the Chinese mainland and your account and rights may not be protected. For more information, see Real-Name Authentication Overview.

      To complete real-name authentication, refer to:
  2. Create an IAM user and grant permissions to the user. Skip this step if you use a HUAWEI ID (administrator account).
    1. Create an IAM user. For details, see Creating an IAM User or Creating an IAM User (New Version).
    2. Create a custom policy and grant permissions to the IAM user. For details, see Using Roles or Policies to Assign Custom Permissions to IAM Users or Using Identity Policies to Assign Custom Permissions to IAM Users (New Version). For details about how to grant permissions, see Assigning Permissions to an IAM User or Assigning Permissions to an IAM User (New Version).
  3. (Optional) Obtain the AK/SK of the target account.

    Creating an Access Key or Access Keys (New Version).

Creating an application migration project

Create a project to separate and manage migration resources. For details, see Creating an Application Migration Project.

Creating a Target File System

Create target file systems for receiving migrated source data. For details, see Creating a General Purpose File System or Creating an SFS Turbo File System.

Step 1: Create a Migration Cluster

  1. Sign in to the MgC console. In the navigation pane, under Project, select your application migration project from the drop-down list.
  2. In the navigation pane on the left, choose Other > Migration Clusters.
  3. Click Create Cluster in the upper right corner of the page.
  4. Configure the parameters based on Table 3.

    Table 3 Parameters for creating a cluster

    Area

    Parameter

    Configuration

    Constraints

    Basic Settings

    Cluster Name

    User-defined

    The cluster name must be unique in the same account.

    Region

    Select the region to provision the cluster.

    The cluster must be provisioned in the target region you are migrating to.

    Cluster Type

    Select what the cluster will be used for.

    Currently, only storage migration is supported.

    Node Settings

    Key Pair

    It is used for identity authentication when you remotely log in to the nodes (ECSs) in the cluster. You need to create a key pair in advance. For details, see Creating a Key Pair.

    -

    Enterprise Project

    This function is provided for enterprise users. You can add the ECS to an enterprise project. If no enterprise projects are available, the system creates the default enterprise project.

    An enterprise project facilitates project-level management and grouping of cloud resources and users. The default project is default.

    For details about creating and managing enterprise projects, see the Enterprise Management User Guide.

    -

    Master Node Specifications

    It is used to manage migration nodes and list nodes.

    A cluster can only have one master node.

    Migration Node Specifications

    Migration nodes are used for executing migration and verification tasks. The recommended specifications are 8 vCPUs and 16 GB of memory.
    • The node specifications cannot be modified after the cluster is created.
    • The number of nodes must meet the following requirements:
      • Number of migration nodes + Number of list nodes + 1 ≤ 100
      • Number of migration nodes + Number of list nodes + 1 ≤ Number of unused IP addresses in the subnet

    List Node Specifications

    List nodes are used for listing tasks. The recommended specifications are 8 vCPUs and 16 GB of memory.

    Network Settings

    VPC

    Select a VPC from the drop-down list.

    -

    Subnet

    Make sure that there are enough unused IP addresses for the migration and list nodes in this cluster.

    Number of unused IP addresses in the subnet ≥ Number of migration nodes + Number of list nodes + 1

    Network Type
    • Internet: You need to select a public NAT gateway. If there is no gateway available, choose Buy Gateway from the drop-down list and select the gateway specifications and EIPs you want to associate with the gateway. A maximum of 20 EIPs can be selected at a time.
    • Intranet: This option is suitable for data migration within a region.
    • Private line: Source data is directly accessed through the private line. For details about how to set up a Direct Connect connection, see Direct Connect Best Practices.

    -

    -

    Log Collection

    If this option is enabled, storage migration logs will be collected for possible troubleshooting later.

    -

  5. Click OK to start creating a cluster. In the migration cluster list, check the cluster status. For details about the cluster status, see Cluster Statuses. If the cluster status is Creation failed, move the cursor to the status to view the failure cause. After the fault is rectified, choose More > Retry to create the cluster again.

Step 2: Create a Migration Task

  1. In the navigation pane on the left, choose Storage Migration > Bulk Migration.
  2. Choose File Storage > Create Migration Task.
  3. Use the default task name or create one. Enter the task description (optional).
  4. Click OK to complete the task creation. On the displayed page, configure the migration task by referring to Step 3: Configure the Migration Task.

Step 3: Configure the Migration Task

This step includes four phases.

You can add resources in either of the following ways.

  • Manual addition:
    1. On the Configure Source/Target page, select the region where the target file storage resides from the Target Region drop-down list. Target file systems included in a migration task must be in the same region.
    2. Click Add in the File Systems area. A row of source file storage configuration information is displayed in the list below.
    3. Configure source file systems based on Table 4.
      Table 4 Parameters for configuring source file systems

      Parameter

      Description

      Remarks

      Source Storage Type

      Supported source storage types include NAS_GFS, NAS_NFS_V3_PROTOCOL, NAS_NFS_V3_MOUNT, and NAS_SMB. Select a type based on your needs.

      NAS_GFS: Google Cloud file systems

      NAS_NFS_V3_PROTOCOL: file systems that support standard NFSv3

      NAS_NFS_V3_MOUNT: file systems that support non-standard NFSv3

      NAS_SMB: SMB/CIFS file systems

      Credential

      Select the credential from the drop-down list. If the credential has not been added, click Add Credential, enter a credential name, select an authentication mode, and enter the username, password, and domain. Click Confirm.

      • Username: Enter the username of the account that can access all files in the source file system, for example, administrator.
      • Password: Enter the password corresponding to the username.
      • (Optional) Domain: Open the console and run the following command to obtain the domain name. 
        wmic computersystem get domain
        NOTE:
        • You only need to enter the content before .com. For example, if the domain is test.com, enter test.
        • If the returned domain information is WORKGROUP, this parameter can be left empty.

      This parameter is mandatory when the source storage type is set to NAS_SMB.

      Source File System Address

      Enter the mount address of the source file system.

      • If the storage type is NAS_GFS, the file system address can include a host name, domain name, (optional) subdomain, port, and path.
      • If the storage type is NAS_NFS_V3_PROTOCOL or NAS_NFS_V3_MOUNT, the file system address must match any of the following formats:
        • <IP-address>:/
        • <IP-address>:/<path>
        • <domain-name>:/
        • <domain-name>:/<path>

      For both full and list migrations, a unique combination of source file system address and path or list path is required. In each case, the prefix matching rule applies.

      For example, if you have added the file system address 192.168.0.1:/a with a migration path /abc/a, an error will be reported if you try to add the file system address 192.168.0.1:/a with a migration path /abc. To overwrite an existing path, you need to delete the existing address and path combination first.

      Source Path

      Enter the directory that contains the files to be migrated. The format is /<folder-name>.

      For a list migration, enter / for the path.

      Target Storage Class

      Supported: NAS_NFS_V3_MOUNT (non-standard NFSv3 file systems)

      -

      Target File System Address

      Enter the address of the target file system in the region you are migrating to. Input format: <IP-address>:/<path> or <domain-name>:/<path>, for example, 192.1.1.1:/0001

      • If a General Purpose file system is used as the target, enter the mount address of the file system. To obtain the mount address, go to the file system list and click the icon next to the address in the Mount Point column.
      • If an SFS Turbo file system is used as the target, enter the shared path of the file system. To obtain the shared path, go to the SFS Turbo file system list and click next to the path in the Shared Path column.

      Combinations of target file system addresses and paths can be duplicated.

      Target Path

      Enter the directory for storing migrated files. The format is /<folder-name>.

      -
    4. Click Save in the Operation column. You can also click Save All after setting multiple records. After the settings are saved, batch modification is supported. Click Next to configure migration settings.

  • Batch import using a template:
    1. On the Configure Source/Target page, select the region where the target file storage resides from the Target Region drop-down list. Target file systems included in a migration task must be in the same region.
    2. In the area for adding resources, click Import to open the dialog box for importing file system settings.
    3. Click Download Template. Configure file system information in the Excel template based on Table 5 and save the file.
      Table 5 Configuration details

      Parameter

      Description

      Remarks

      Source Storage Type

      Supported source storage types include NAS_GFS, NAS_NFS_V3_PROTOCOL, NAS_NFS_V3_MOUNT, and NAS_SMB. Select a type based on your needs.

      NAS_GFS: Google Cloud file systems

      NAS_NFS_V3_PROTOCOL: file systems that support standard NFSv3

      NAS_NFS_V3_MOUNT: file systems that support non-standard NFSv3

      NAS_SMB: SMB/CIFS file systems

      Source Credential Name

      This parameter is optional. Enter the source credential name.

      This field is only required for NAS_SMB sources with connection credentials (usernames/passwords/domains) added to MgC.

      Source File System Address

      Enter the mount address of the source file system.

      • If the storage type is NAS_GFS, the file system address can include a host name, domain name, (optional) subdomain, port, and path.
      • If the storage type is NAS_NFS_V3_PROTOCOL or NAS_NFS_V3_MOUNT, the file system address must match any of the following formats:
        • <IP-address>:/
        • <IP-address>:/<path>
        • <domain-name>:/
        • <domain-name>:/<path>

      For both full and list migrations, a unique combination of source file system address and path or list path is required. In each case, the prefix matching rule applies.

      For example, if you have added the file system address 192.168.0.1:/a with a migration path /abc/a, an error will be reported if you try to add the file system address 192.168.0.1:/a with a migration path /abc. To overwrite an existing path, you need to delete the existing address and path combination first.

      Source Path

      Enter the directory that contains the files to be migrated. The format is /<folder-name>.

      For a list migration, enter / for the path.

      Target Storage Class

      The available options are NAS_NFS_V3_MOUNT (non-standard NFSv3 file systems) and NAS_SMB (SMB/CIFS file systems).

      -

      Target Credential Name

      This parameter is optional. Enter the target credential name.

      This field is only required for NAS_SMB targets with connection credentials (usernames/passwords/domains) added to MgC.

      Target File System Address

      Enter the address of the target file system in the region you are migrating to. Input format: <IP-address>:/<path> or <domain-name>:/<path>, for example, 192.1.1.1:/0001

      • If a General Purpose file system is used as the target, enter the mount address of the file system. To obtain the mount address, go to the file system list and click the icon next to the address in the Mount Point column.
      • If an SFS Turbo file system is used as the target, enter the shared path of the file system. To obtain the shared path, go to the SFS Turbo file system list and click next to the path in the Shared Path column.

      Combinations of target file system addresses and paths can be duplicated.

      Target Path

      Enter the directory for storing migrated files. The format is /<folder-name>.

      -

      Migration Method

      The available migration methods are DEFAULT and LIST.

      DEFAULT indicates full migration. This method enables the migration of all data in a specific path in a source file system.

      LIST indicates list-based migration. This method enables the migration of specific files and folders recorded in predefined lists.

      Migration Scope

      If the list migration method is used, enter the path to the list files.

      If the full migration method is used, leave it blank.
    4. In the displayed dialog box, click Access OBS at the bottom.
    5. Select a bucket and upload the Excel file to the bucket. For details about how to upload the file, see Uploading a File to OBS.
    6. After the upload is successful, return to the MgC console. In the Import dialog box, select the file stored in the OBS bucket from the drop-down list.
    7. Click Confirm. The file storage information will be added to the page. Batch modification is supported. Click Next to configure migration settings.
  1. In the Migration Method Configuration area, modify or remove resource. Select a migration method for each resource.

    • Full migration: This method enables the migration of all data in a specific path in a source file system.
    • List migration: This method enables the migration of specific files and folders recorded in predefined lists.

  2. In the Migration Settings area, set the parameters based on Table 6.

    Table 6 Migration settings

    Parameter

    Sub-parameter

    Description

    Migrate Metadata

    -

    Determine whether to migrate metadata.

    • If you select this option, object metadata will be migrated.
    • If you do not select this option, only the Content-Type and Content-Encoding metadata will be migrated. If the source storage service provider does not provide the metadata (for example, UCloud does not provide Content-Encoding), the metadata will not be migrated.

    Concurrent Subtasks

    -

    User-defined. There cannot be more than 10 concurrent subtasks for each online migration node. For example, if there are 2 online migration nodes, there can be up to 20 subtasks.

    Overwrite Existing

    NOTE:

    NAS systems do not allow any files or folders with identical names, regardless of case. The migration fails if a source file has the same name as a target folder or vice versa.

    Never

    Files existing at the target will never be overwritten.

    WARNING:
    • If you choose Never for the initial migration, the attributes of involved parent folders at the source will not be migrated to the target. As a result, the folder attributes may be incomplete at the target. To avoid this issue, use the Never option with caution for the initial migration.
    • If you choose Never, restarting a migration after an interruption or pause may lead to incomplete data migration, even though the task may appear successful. This could impact data integrity, so use the Never option with caution.

    Always

    Files existing at the target will always be overwritten.

    If older or different size
    • The system replaces existing target files if they are older than or differ in size from their source counterparts. Files with matching modification times and sizes remain unchanged and are skipped during migration.
    • The system verifies folders after their content is migrated. Folders that already exist at the target will be overwritten if they have different last modification times, sizes, or permissions from the paired folders at the source.
      NOTE:

      The same overwriting policy is applied to empty folders as files.

    Traffic Limiting

    -

    Set the maximum bandwidth for the migration task during different periods.

    • If you do not select this option, migration traffic will not be limited.
    • If you select this option, you can limit the migration traffic by setting Start Time, End Time, and Bandwidth Limit.
      For example, if you set Start Time to 08:00, End Time to 12:00, and Bandwidth Limit to 20 MB/s, the maximum migration speed is limited to 20 MB/s from 08:00 to 12:00. The migration speed is not limited outside this period.
      NOTE:
      • The bandwidth limit ranges from 1 MB/s to 1,048,576 MB/s.
      • Time periods in different rules cannot overlap.

        For example, if there is a rule added for the period from 8:00 to 12:00, you cannot configure rules for any overlapped periods, such as from 7:00 to 13:00, 7:00 to 8:00, and 9:00 to 12:00.

      • The start time in a rule cannot be later than the end time.

        For example, the time period from 23:00 to 01:00 of the next day is not allowed.

      • A maximum of five rules can be added.
      • The time is the local standard time of the region you are migrating to.

  3. (Optional) Configure advanced options based on Table 7.

    Table 7 Advanced options

    Parameter

    Description

    Sync Incremental Before Cutover
    • If this option is selected, incremental data will be migrated before the migration reaches the cutover phase.
    • If this option is not selected, the migration will proceed directly to the cutover phase without transferring incremental data.

    Filter Source Data

    Use exclude patterns, include patterns, and time ranges to filter files to be migrated. For details about the filters, see Source Data Filters.

    Send SMN Notifications

    Determine whether to use SMN to get notifications about migration results.

    • If you do not select this option, no SMN messages will be sent after the migration is complete.
    • If you select this option, after the migration is complete, SMN messages will be sent to the subscribers of the selected topic. You can select the language and trigger conditions for sending messages.

  4. After the migration settings are configured, click Next to open the Configure Migration Cluster page.
  1. In the Migration Cluster area, select the cluster to be used. To modify the cluster, click Modify in the Operation column and modify the cluster by referring to Managing a Migration Cluster.
  2. After the migration cluster is configured, click Next to open the confirmation page.
  1. After completing operations from Phase 1: Configure Migration Resources to Phase 3: Configure the Migration Cluster, you will be directed to the Confirm page. Check the resource and migration settings carefully to ensure that they meet your requirements. To update the settings, click Previous to return to the corresponding stage for modification.
  2. On the Confirm page, verify that the resource and migration settings are correct and that all quotas meet the requirements. Then, click Finish. The Start Migration Task dialog box is displayed. The migration task has been created. Determine whether to start the migration immediately.

    • If you want to start the migration immediately, click Confirm to run the task.
    • If you want to save the migration settings without starting the migration, click Cancel. The task status will change to To be started. To run the task, click Run in the Operation column to manually start the migration.

  3. (Optional) Add a phase or step.

    Adding checkpoints to migration tasks helps you control the migration process. For details, see Adding a Pre-/Post-Migration Stage and Step (Checkpoints and Scripts).

Step 4: Track the Migration Task

In the migration task list, click the task name to go to the Task Details page. Check the task progress and perform the following operations on it.
  • Checking the resource status: Click a resource name. On the Resource Details page displayed on the right, you can review the basic information, migration status, and migration progress of the resource.
  • Confirming steps: Some resources require manual confirmation. In this case, click Confirm in the Migration Status column or in the Operation column to continue the migration.

    To check the migration phases and steps included in a migration task, click View Task Template in the upper right corner of the task details page.

Reconfiguring a Task

Tasks that are to be started, paused, failed to be deleted, or successful can be reconfigured. The task status will change to Configuring. All configured resources and their progress will be deleted permanently.

  1. In the migration task list, choose More > Reconfigure in the Operation column of a migration task.
  2. Enter YES and click OK.
  3. Wait until the task status changes to Configuring. Click Configure to reset the task.

Deleting Resources from a Migration Task

Resources in the Failed, Paused, or Completed state can be deleted. If all resources in a migration task are deleted, the migration task will also be deleted.

  1. In the migration task list, click a migration task name to go to the task details page.
  2. In the resource list, select the resources to be deleted and click Delete above the list.
  3. Confirm the selected resources, enter DELETE, and click Confirm.

    To prevent resource deletion failures caused by disconnection of the task from related services (such as SMS, OMS, and OBS), you can select Force Delete to forcibly delete the resources. However, this operation only deletes the resource data from MgC. It may not delete the related tasks and resources in the related cloud services like SMS, OMS, and OBS. You must delete any remaining resources manually.

Parent Topic: Storage Migration