Help Center/ Object Storage Service/ User Guide/ Data Management/ Data Replication/ Same-Account, Cross-Region Replication Using OBS
Updated on 2026-09-23 GMT+08:00
View PDF
Share

Same-Account, Cross-Region Replication Using OBS

Scenarios

Cross-region replication within an account allows you to define replication rules that automatically replicate data from a source bucket in one region to a destination bucket in another region under the same account in near real time. Objects replicated to the destination bucket are exact copies of the objects in the source bucket.

This section describes how to replicate data between buckets of the same account across regions using OBS Console, OBS APIs, or obsutil.

Replication scope: objects with a specified prefix or all objects in a bucket

Replication content: object content, metadata (object name, size, last modification time, creator, version ID, and user-defined metadata), ACL, and storage class (under the default configuration)

If Change storage class for replicated objects is selected when you create a cross-region replication rule, the storage class of the object copies is different from that of the source objects.

Figure 1 Cross-region replication within an account

You can use cross-region replication within an account to meet the following requirements:

  • Compliance requirements

    OBS stores data across geographically distant AZs by default, but some compliance standards may require even greater separation. In such cases, you can use cross-region replication to meet compliance requirements.

  • Low latency

    The same OBS resources may need to be accessed from different locations. To minimize the access latency, you can use cross-region replication to create object copies in the nearest region.

  • Data replication

    You want to migrate data stored in OBS to the data center in another region.

  • Data backup and disaster recovery

    For security and availability purposes, you want to create explicit backups for all data written to OBS in the data center of another region to ensure data remains available if there is any damage.

  • Easy maintenance

    If you have compute clusters in two different OBS regions that need to analyze the same set of objects, you can maintain object replicas in both regions.

OBS helps you replicate your service data stored in OBS to a specified region. However, Huawei Cloud does not have visibility into your data and is not responsible for ensuring the legal compliance of your use of OBS. If your replication involves cross-border transfer, ensure that your use complies with relevant laws and regulations.

Constraints

Table 1 Constraints on same-account, cross-region replication

Item

Description

Bucket version

Only buckets with version 3.0 or later support cross-region replication.

You can view the version of a bucket in the Basic Information area on the bucket's Overview page of OBS Console. For details, see Viewing Bucket Information.

Functions
  • Objects in the Archive, or Deep Archive storage class cannot be replicated.
  • If the destination bucket's region does not support storage classes, all replicated objects are stored in the Standard storage class.
  • OBS supports replication only between one source bucket and one destination bucket. Replication from a single source bucket to multiple destination buckets is not supported. You can change the destination bucket, but doing so updates the destination bucket for all existing rules.
  • Cross-region replication does not replicate appendable objects.
  • A source bucket can have either a single cross-region replication rule that applies to all objects in the bucket, or up to 100 rules that apply to objects with specified prefixes.
  • Objects encrypted with SSE-C cannot be replicated across regions.

Time

Cross-region replication is not real-time. After you configure a replication rule, objects may not be replicated immediately. Please wait patiently.

Region
  • The source and destination buckets must be in two different regions.
  • The source and destination regions support replication with each other. Figure 2 lists the supported regions. "√" indicates that data can be replicated between regions. "x" indicates that data cannot be replicated between regions.

Synchronization of existing objects
  • By default, objects uploaded before cross-region replication is enabled are not replicated to the destination bucket unless the function for synchronizing existing objects is enabled.
  • Modifying cross-region replication settings may prevent existing objects from being replicated, so avoid making changes until the synchronization of existing objects is complete.
  • If an object replicated to the destination bucket is deleted and a new replication rule that allows replication of existing objects is configured within three days of the deletion, the same version of the deleted object will not be replicated to the destination bucket again.

    For example, you first configure cross-region replication rule 1 to replicate the current version of object test.txt from the source bucket to the destination bucket. Later, you delete test.txt from the destination bucket and delete rule 1. Two days afterward, you create cross-region replication rule 2 that allows replication of existing objects. In this case, test.txt will not be replicated to the destination bucket again.

Versioning
  • The source and destination buckets must have the same versioning status (enabled or disabled). If you change the versioning status after cross-region replication has been configured, any updates made when the source and destination buckets have inconsistent versioning statuses cannot be replicated. After versioning is enabled, it can only be suspended and cannot be disabled. Therefore, exercise caution when changing the versioning status while using replication rules.
  • If versioning is enabled or suspended for both the source and destination buckets and the cross-region replication rule is enabled, deleting an object in the source bucket without specifying its version will also delete the object in the destination bucket.
  • If you change the versioning status of the destination bucket while cross-region replication is enabled, object replication will fail. To change the versioning status of the source bucket, delete the replication configuration first and then make the change.

Access control
  • Bucket owners must have the read and write permissions for both source and destination buckets for successful replication. If the read/write permissions of the source or destination bucket are incorrect and the system cannot read source objects or write to the destination bucket, the affected objects will fail to replicate. Even after the permissions are corrected, these objects will not be replicated again.
  • Do not delete or overwrite object replicas in the destination bucket, or modify their ACLs. Doing so may cause inconsistencies in the latest object versions or permission settings between the source and destination buckets.

Tags

A cross-region replication rule does not replicate tags applied to the source bucket and objects to the destination bucket.

Others
  • Objects in a source bucket can be replicated to only one destination bucket and cannot be replicated further from that destination bucket to another bucket. For example, bucket A and bucket B are in two different regions. You can replicate data from bucket A to bucket B or vice versa. However, replicas in either bucket cannot be replicated again.
  • If BlockPublicAcls is enabled for the destination bucket but the ACLs of objects in the source bucket allow public access, cross-region replication will fail because of ACL conflicts.
Figure 2 Replication between regions

Important Notes

  • After a bucket is created, you are advised to wait at least 5 minutes before creating a cross-region replication rule for the bucket, or the rule creation may fail.
  • If the cross-region replication rule is enabled but the function of synchronizing existing objects is disabled, there is a delay of about 5 minutes to apply the rule. Objects uploaded during that delay may not be replicated.
  • If the cross-region replication rule and the function of synchronizing existing objects are both enabled, existing objects will be replicated 5 minutes later.
  • Objects encrypted with server-side encryption (including SSE-KMS and SSE-OBS) can be replicated. The replication traffic cost is calculated based on the length of the plaintext for SSE-KMS and SSE-OBS. For details, see Billing for Cross-Region Replication Within an Account.

Billing for Cross-Region Replication Within an Account

Figure 3 shows the items that will be billed when data is replicated from one region to another.

Figure 3 Items billed for cross-region replication
Table 2 Billing for cross-region replication

Action

Billing Item

Description

Billing Mode

Replicate data across regions

Requests

You are billed for the number of successfully replicated objects. Successfully replicating one object creates a copy request. For details, see Copying Objects.

  • For non-multipart objects, replicating an object creates a GET request to the source bucket and a PUT request to the destination bucket; deleting an object from the source bucket creates a DELETE request to the destination bucket.
  • For multipart objects, replicating each part creates a GET request to the source bucket and a PUT request to the destination bucket.
  • Modifying a replicated source object's ACL automatically synchronizes the change to the destination object.
    NOTE:

    Because the source bucket initiates this ACL replication, it incurs the resulting PUT request charges.

For details about request billing, see Requests.

Pay-per-use

Data transfer

Traffic generated when you replicate data from the source bucket to the destination bucket in another region. Billing only applies to the data transferred out of the source bucket.

If objects are encrypted using server-side encryption, the cost of their cross-region replication traffic is calculated based on the length of the plaintext for SSE-KMS and SSE-OBS.

NOTE:

Cross-account, cross-region replication does not support SSE-KMS-encrypted objects.

Pay-per-use

Storage space

Space occupied by the replicated objects in the destination bucket

If you have specified another storage class for object copies in the destination bucket, these copies are billed based on the new storage class.

If objects are encrypted using server-side encryption, their storage cost is calculated based on the length of the ciphertext.

Pay-per-use

Resource packages

Synchronize existing objects

Requests

You are billed for the number of existing objects that are successfully replicated to the destination bucket.

Billing applies to the number of objects that are successfully replicated. You are also billed for listing the existing objects.

Pay-per-use

Data transfer

Traffic generated when OBS replicates existing objects to the destination bucket in another region. Billing only applies to the data transferred out of the source bucket.

If historical objects are encrypted using server-side encryption, the cost of their cross-region replication traffic is calculated based on the length of the plaintext for SSE-KMS and SSE-OBS.

Pay-per-use

Storage space

Space occupied by the replicated objects in the destination bucket

If you have specified another storage class for object copies in the destination bucket, these copies are billed based on the new storage class.

If historical objects are encrypted using server-side encryption, their storage cost is calculated based on the length of the ciphertext.

Pay-per-use

Resource packages

Prerequisites

  • There is a source bucket in a region of an account. The bucket version is 3.0 or later, and the bucket region supports cross-region replication.
  • There is a destination bucket in another region of the same account. To create a bucket, see Creating a Bucket.

Creating a Same-Account, Cross-Region Replication Rule

You can use OBS Console, APIs, or obsutil to configure same-account, cross-region replication. You cannot use SDKs or OBS Browser+ to do so.

Using OBS Console

  1. In the navigation pane of OBS Console, choose Buckets.
  2. In the bucket list, click the desired bucket. The Objects page is displayed.
  3. In the navigation pane, choose Data Management > Cross-Region Replication.
  4. Select "I have read and agree to the disclaimer." Then, click Create Rule. The Create Cross-Region Replication Rule dialog box is displayed, as shown in Figure 4.

    Figure 4 Creating a cross-region replication rule

    • The source and destination buckets must have the same versioning status.
    • A bucket can have only one destination bucket and one IAM agency for cross-region replication. Any new replication rule overwrites the destination bucket and IAM agency defined in an earlier rule for the same bucket.

  5. Create a cross-region replication rule according to your needs. For details, see Table 3.

    Table 3 Parameters for configuring a cross-region replication rule

    Parameter

    Description

    Status

    Choose whether to enable or disable the rule after it is created. The source and destination buckets must have the same versioning status.

    Source Bucket

    Replicate

    Select the objects to replicate from the source bucket.

    • All objects: Replicates all objects in the source bucket to the destination bucket.
    • Match by prefix: Replicates objects that match the specified prefix to the destination bucket.

    Prefix

    If you select Match by prefix for Replicate, you must specify an object name prefix. Only objects in the source bucket that match the prefix will be replicated to the destination bucket.

    • The prefix cannot be empty and can contain up to 1,024 characters.
    • If the specified prefix overlaps with the prefix of an existing rule, OBS treats the two rules as one and prevents you from configuring the one you are creating. For example, if there is already a rule with prefix abc in OBS, you cannot create another rule whose prefix starts with abc.
    • To replicate a folder, end the prefix with a slash (/), for example, folder1/.

    Synchronize Existing Objects

    Choose whether to synchronize the objects that were already in the source bucket before the rule was configured to the destination bucket.

    By default, these objects are not synchronized.

    Destination Bucket

    Bucket Owner

    Select a bucket in the same account as the source bucket or in a different account. Select Current account here.

    • Current account: Select a bucket in the current account. This means the source and destination buckets belong to the same account, enabling same-account, cross-region replication.
    • Another account: Select a bucket in a different account. This means the source and destination buckets belong to different accounts, or the source bucket contains data uploaded by users other than the bucket owner, enabling cross-account, cross-region replication.

    Region

    Select the region of the destination bucket. This parameter is mandatory when you select Current account.

    The source and destination buckets belong to the same account but are in different regions.

    Bucket

    Select a destination bucket. This parameter is mandatory when you select Current account.

    NOTE:

    If a bucket is newly created, you may need to wait several minutes before it appears in the drop-down list of the destination bucket.

    Change storage class for replicated objects

    Choose whether to change the storage class of objects replicated from the source bucket to the destination bucket.

    • By default, this option is not selected, meaning that replicated objects use the same storage class as the source objects.
    • If you select this option, you can specify a storage class for objects replicated to the destination bucket.

    Permissions

    Replicate KMS encrypted objects

    Choose whether to replicate KMS encrypted objects from the source bucket to the destination bucket. OBS attempts to replicate KMS-encrypted objects regardless of whether this option is selected.

    • If this option is selected, the drop-down list displays only IAM agencies created for OBS that have permissions for the global project and have the KMS Administrator permission for both the source and destination regions.
    • If this option is not selected, the drop-down list displays only IAM agencies created for OBS that have permissions for the global project but do not have the KMS Administrator permission for either the source or destination region.

    If KMS is not enabled in the destination region or the agency does not have the KMS Administrator permission for the source and destination regions, KMS encrypted objects will fail to be replicated to the destination bucket, and the replication status will be FAILED.

    After a KMS-encrypted object is replicated to the destination bucket, the key for encrypting the object copy will change to the default key obs/default of the destination region.

    IAM agency

    Select an agency that authorizes OBS to operate your resources. This agency allows OBS to perform cross-region replication.

    If no agency is available for cross-region replication, click Create Agency to create one. If you have already created IAM agencies, select one from the drop-down list.

    The IAM agency must meet the following requirements:

    • The IAM agency must be for OBS and must have the OBS permissions listed in Table 4.
    • If Replicate KMS encrypted objects is selected, the agency must have the KMS Administrator permission. The authorization scope of the agency must be set to Region-specific projects, and the source and destination regions must be selected.
    Table 4 OBS permissions required by the agency

    Action

    Description

    obs:object:GetObject

    Obtains object content and metadata.

    obs:object:DeleteObjectVersion

    Deletes one or more objects.

    obs:object:PutObjectVersionAcl

    Configures the object version ACL.

    obs:object:AbortMultipartUpload

    Aborts a multipart upload.

    obs:object:PutObjectAcl

    Configures the object ACL.

    obs:object:DeleteObject

    Deletes one or more objects.

    obs:bucket:HeadBucket

    Obtains bucket metadata.

    obs:object:PutObject

    Uploads objects with PUT or POST, copies objects, appends data to objects, initiates a multipart upload, as well as uploads and assembles parts.

    obs:object:GetObjectVersionAcl

    Obtains the object ACL.

    obs:bucket:GetBucketVersioning

    Obtains the versioning status of a bucket.

    obs:bucket:ListBucketMultipartUploads

    Lists multipart uploads.

    obs:object:ListMultipartUploadParts

    Lists uploaded parts.

    obs:object:ModifyObjectMetaData

    Modifies object metadata.

    obs:bucket:ListBucketVersions

    Lists object versions in a bucket.

    obs:bucket:ListBucket

    Lists objects in a bucket.

    obs:object:GetObjectVersion

    Obtains object content and metadata.

    obs:object:GetObjectAcl

    Obtains the object ACL.

    obs:bucket:GetReplicationConfiguration

    Obtains the cross-region replication configuration of a bucket.

  6. (Optional) Create an IAM agency. For details, see Creating an Agency for Cross-Region Replication.
  7. Click OK. The cross-region replication rule is created.

Using OBS APIs

Configuring Cross-Region Replication for a Bucket

Using the OBS CLI Tool - obsutil

When using obsutil to replicate objects across regions within an account, you must enable cross-region replication, which means that the crr parameter is mandatory.

  • Do not change the source objects in the OBS bucket when copying a single object or objects in batches. Otherwise, the operation may fail or data may be inconsistent.
  • If the storage class of the object to be copied is cold, you must restore the object to be copied first. Otherwise, the copy fails.
  • To copy objects, you must have the read permission on the objects to be copied and the write permission on the destination bucket.
  • If the client-side cross-region replication function is not enabled, ensure that the source bucket and destination bucket are in the same region.
  • If the source bucket is a parallel file system (supporting POSIX), the destination bucket must also be a parallel file system.

Constraints

  • The source and destination paths cannot be the same.
  • The source and destination paths cannot be nested during a batch copy.
    • If the destination path has the source path as its prefix, recursive copy may occur.
    • If the source path has the destination path as its prefix, overwriting may occur.

Command Line Structure

  • Windows
    • Copying a single object 
      obsutil cp obs://srcbucket/key obs://dstbucket/[dest] [-dryRun] [-u] [-crr] [-vlength] [-vmd5] [-p=1] [-threshold=52428800] [-versionId=xxx] [-acl=xxx] [-sc=xxx] [-meta=aaa:bbb#ccc:ddd] [-ps=auto] [-cpd=xxx] [-fr] [-o=xxx] [-config=xxx] [-e=xxx] [-i=xxx] [-k=xxx] [-t=xxx]
    • Copying objects in batches 
      obsutil cp obs://srcbucket[/key] obs://dstbucket[/dest] -r [-dryRun] [-f] [-flat] [-u] [-crr] [-vlength] [-vmd5] [-j=1] [-p=1] [-threshold=52428800] [-acl=xxx] [-sc=xxx] [-meta=aaa:bbb#ccc:ddd] [-ps=auto] [-include=*.xxx] [-exclude=*.xxx] [-timeRange=time1-time2] [-mf] [-o=xxx] [-cpd=xxx] [-config=xxx] [-e=xxx] [-i=xxx] [-k=xxx] [-t=xxx]
  • macOS or Linux
    • Copying a single object 
      ./obsutil cp obs://srcbucket/key obs://dstbucket/[dest] [-dryRun] [-u] [-crr] [-vlength] [-vmd5] [-p=1] [-threshold=52428800] [-versionId=xxx] [-acl=xxx] [-sc=xxx] [-meta=aaa:bbb#ccc:ddd] [-ps=auto] [-cpd=xxx] [-fr] [-o=xxx] [-config=xxx] [-e=xxx] [-i=xxx] [-k=xxx] [-t=xxx]
    • Copying objects in batches 
      ./obsutil cp obs://srcbucket[/key] obs://dstbucket[/dest] -r [-dryRun] [-f] [-flat] [-u] [-crr] [-vlength] [-vmd5] [-j=1] [-p=1] [-threshold=52428800] [-acl=xxx] [-sc=xxx] [-meta=aaa:bbb#ccc:ddd] [-ps=auto] [-include=*.xxx] [-exclude=*.xxx] [-timeRange=time1-time2] [-mf] [-o=xxx] [-cpd=xxx] [-config=xxx] [-e=xxx] [-i=xxx] [-k=xxx] [-t=xxx]

Examples

  • In Windows, run obsutil cp obs://bucket-test/key obs://bucket-test2 to copy a single object. 
    obsutil cp obs://bucket-test/key obs://bucket-test2
Start at 2024-09-30 08:30:09.0815415 +0000 UTC

Parallel:      3                   Jobs:          3
Threshold:     50.00MB             PartSize:      auto
CheckpointDir: xxxx

[=====================================================] 100.00% 6/s 0s
Waiting for the copied key to be completed on server side.
Copy successfully, 19B, obs://bucket-test/key --> obs://bucket-test2/key
ext.txt, cost [1708], status [200], request id [00000192420D227E4017336A12F1DC22]
  • In Windows, run obsutil cp obs://bucket-test/temp/ obs://bucket-test2 -f -r to copy objects in batches. 
    obsutil cp obs://bucket-test/temp/ obs://bucket-test2 -r -f
Start at 2024-09-30 08:34:02.7819703 +0000 UTC

Parallel:      5                   Jobs:          5
Threshold:     50.00MB             PartSize:      auto
CheckpointDir: xxxx

Task id: 0476929d-9d23-4dc5-b2f8-0a0493f027c5
OutputDir: xxxx

[=============================================================] 100.00% 10/s 0s
Succeed count:   5         Failed count:    0
Metrics [max cost:298 ms, min cost:192 ms, average cost:238.00 ms, average tps:9.71, transferred size: 7.20MB]
Task id: 0476929d-9d23-4dc5-b2f8-0a0493f027c5

Parameter Description

Parameter

Optional or Mandatory

Description

srcbucket

Mandatory

The source bucket name

dstbucket

Mandatory

The destination bucket name

dest

Optional

The destination object name when copying an object, or the name prefix of destination objects when copying objects in batches

key

Mandatory for copying an object.

Optional for copying objects in batches.

The source object name when copying an object, or the name prefix of source objects when copying objects in batches

The rules are as follows:

  • This parameter cannot be left blank when copying an object.
    • If dest is left blank, the source object is copied to the root directory of the destination bucket.
    • If the value of dest ends with a slash (/), the destination object name is the value of dest plus the source object name. Otherwise, the destination object name is the value of dest.
  • If this parameter is left blank when copying objects in batches, all objects in the source bucket are copied. If not, objects whose name prefix is the set value in the source bucket are copied. The rules for confirming the name of the destination object are as follows:
    • If the value of dest ends with a slash (/), the destination object name is the value of dest plus the source object name.
    • If the value of dest does not end with a slash (/), the destination object name is the value of dest/source object name.
  • If this parameter is configured but the flat parameter is not when copying objects in batches, the name of the source object contains the name prefix of the parent object. If flat is configured, then the name of the source object does not contain the name prefix of the parent object.
NOTE:

For details about how to use this parameter, see Copy.

fr

Optional for copying an object (additional parameter)

Generates an operation result file when copying an object.

flat

Optional for copying objects in batches (additional parameter)

The name prefix of the parent object is excluded when copying objects in batches.

dryRun

Optional (additional parameter)

Conducts a dry run.

crr

Mandatory for cross-region replication within an account

Enables the client-side cross-region replication function. In this mode, data is directly copied to the destination bucket from the source bucket through data stream. The buckets can be any two OBS buckets.

  • If this parameter is configured, the configuration of client-side cross-region replication must be updated in the configuration file. For details, see Updating a Configuration File.
  • When cross-region replication is enabled, the upload/download bandwidth, CPU, and memory resources of the host where commands are executed will be occupied, which may deteriorate the host performance.
NOTE:

The configurations of the source bucket and destination bucket are respectively akCrr/skCrr/tokenCrr/endpointCrr and ak/sk/token/endpoint in the configuration file.

vlength

Optional (additional parameter)

Verifies whether the object size in the destination bucket is the same as that in the source bucket after the copy task completes.

This parameter must be used together with crr.

vmd5

Optional (additional parameter)

Verifies whether the MD5 value of the destination bucket is the same as that of the source bucket after the copy task completes.

  • This parameter must be used together with crr.
  • Objects in the source bucket must contain metadata x-obs-meta-md5chksum, or MD5 verification will be skipped. After the MD5 verification is successful, this parameter value is used for metadata x-obs-meta-md5chksum of the destination object, for later MD5 verification during download or replication.
  • If your object needs encryption, do not use this parameter.

u

Optional (additional parameter)

Indicates incremental copy. If this parameter is set, each object can be copied only when it does not exist in the destination bucket, its size is different from the namesake one in the destination bucket, or it has the latest modification time.

p

Optional (additional parameter)

The maximum number of concurrent multipart copy tasks when copying an object. The default value is the value of defaultParallels in the configuration file.

threshold

Optional (additional parameter)

The threshold for enabling multipart copy. If the size of the object to be copied is smaller than the threshold, copy the object directly. If not, a multipart copy is required.

The default value is the value of defaultBigfileThreshold in the configuration file.

Unit: byte

This parameter value can contain a unit, for example, 1MB (indicating 1,048,576 bytes).

NOTE:

If you copy an object directly, no part record is generated, and resumable transmission is not supported.

versionId

Optional for copying an object (additional parameter)

The source object version ID that can be specified when copying an object

acl

Optional (additional parameter)

The access control policies for destination objects that can be specified when copying objects.

Possible values are:

  • private: Only users granted permissions by the object ACL can access the object.
  • public-read: Anyone can read objects in the bucket.
  • public-read-write: Anyone can read, write, or delete objects in the bucket.
  • bucket-owner-full-control: Only the bucket owner has full control over objects in the bucket.

sc

Optional (additional parameter)

The storage classes of the destination objects that can be specified when copying objects.

Possible values are:

  • standard: The Standard storage class. It features low access latency and high throughput, and is suitable for storing frequently accessed data (multiple times a month on average) or data that is smaller than 1 MB.
  • warm: The Infrequent Access storage class. It is suitable for storing infrequently accessed (less than 12 times a year on average) data, but when needed, the access has to be fast. It has a slightly lower availability than Standard.
  • cold: The Archive storage class. It provides secure, durable, and inexpensive storage for rarely-accessed (once a year on average) data.
  • deep-archive: The Deep Archive storage class. It is suitable for storing data that is hardly ever accessed (once every few years on average). This storage class costs less than the Archive storage class, but takes longer (usually several hours) to restore data.

meta

Optional (additional parameter)

The standard or user-defined metadata that can be specified during object replication.

  • Standard metadata includes Content-Type, Content-Encoding, Cache-Control, Content-Disposition, Content-Language and Expires.
  • User-defined metadata must be in the format of key:value. Multiple user-defined metadata items are separated by number signs (#), for example, key1:value1#key2:value2#key3:value3, which indicates that the object copy in the destination bucket contains three groups of user-defined metadata: key1:value1, key2:value2, and key3:value3.

fs

Optional (additional parameter)

Specifies whether the method of listing parallel file systems is applied. If you are listing parallel file systems, you are recommended to add this parameter.

  • After this parameter is enabled, marker and limit will be ignored. Then, the size of the bucket, parallel file system, or directory will be calculated.
  • This parameter is only supported for obsutil 5.5.12 or later.
  • With this method, the listing time required varies largely depending on the directory structures.

ps

Optional (additional parameter)

The size of each part in a multipart copy task

  • The default value is the value of defaultPartSize in the configuration file. The parameter can be set to auto. In this case, obsutil automatically configures the part size for each multipart task based on the source file size.
  • Range: 100KB to 5GB
  • Unit: byte. This parameter value can contain a unit, for example, 1MB (indicating 1,048,576 bytes).

cpd

Optional (additional parameter)

The folder where the part records reside. The default value is .obsutil_checkpoint, the subfolder in the home directory of the user who executes obsutil commands.

A part record is generated during a multipart copy and saved to the upload subfolder. After the copy succeeds, its part record is deleted automatically. If the copy fails or is suspended, the system attempts to resume the task according to its part record when you perform the copy the next time.

r

Mandatory for copying objects in batches (additional parameter)

Copies objects in batches based on a specified name prefix of objects in the source bucket.

f

Optional for copying objects in batches (additional parameter)

Runs in force mode.

j

Optional for copying objects in batches (additional parameter)

The maximum number of concurrent tasks for copying objects in batches. The default value is the value of defaultJobs in the configuration file.

NOTE:

The value is ensured to be greater than or equal to 1.

exclude

Optional for copying objects in batches (additional parameter)

The matching patterns of source objects that are excluded. After this parameter is specified, if the name of the object to be copied matches the value of this parameter, the object is skipped. For example, if this parameter is set to *.txt, all files whose names end with .txt are skipped and not copied.

  • An asterisk (*) matches any number of characters. For instance, abc*.txt matches all files whose names start with abc, contain any number of characters in the middle, and end with .txt.
  • A question mark (?) matches any single character. For instance, abc?.txt matches all files whose names start with abc, contain any single character in the middle, and end with .txt.
  • You can use \* to represent * and \? to represent ?.
  • Multiple exclude parameters can be specified, for example, -exclude=*.xxx -exclude=*.xxx.

Constraints:

This matching pattern applies only to objects whose names do not end with a slash (/).

NOTE:
  • You are advised to use quotation marks for the matching pattern to prevent special characters from being escaped by the OS and leading to unexpected results. Use single quotation marks for Linux or macOS and double quotation marks for Windows.
  • The matching pattern applies to the absolute path of an object, including the object name prefix and object name starting from the root directory. For example, if the path of an object in the bucket is obs://bucket/src1/src2/test.txt, then the absolute path of the object is src1/src2/test.txt.

include

Optional for copying objects in batches (additional parameter)

The included matching patterns of source objects. After this parameter is specified, if the name of the object to be copied matches the value of this parameter, the object is copied. For example, if this parameter is set to *.txt, all objects whose names end with .txt are copied.

  • An asterisk (*) matches any number of characters. For instance, abc*.txt matches all files whose names start with abc, contain any number of characters in the middle, and end with .txt.
  • A question mark (?) matches any single character. For instance, abc?.txt matches all files whose names start with abc, contain any single character in the middle, and end with .txt.
  • You can use \* to represent * and \? to represent ?.
  • Multiple include parameters can be specified, for example, -include=*.xxx -include=*.xxx.

Constraints:

  • This matching pattern applies only to objects whose names do not end with a slash (/).
  • If both the exclude and include parameters are configured, the exclude rules are applied first. If the name of an object to be copied does not match exclude, OBS checks whether the object name matches the value of include. If yes, the object is copied. If no, the object is skipped.

Example:

./obsutil cp obs://src-bucket/ obs://target-bucket/ -include=/src-object/* -f -r

This command copies files whose names start with src-object/ from the src-bucket bucket to the target-bucket bucket.

NOTE:
  • You are advised to use quotation marks for the matching pattern to prevent special characters from being escaped by the OS and leading to unexpected results. Use single quotation marks for Linux or macOS and double quotation marks for Windows.
  • The matching pattern applies to the absolute path of an object, including the object name prefix and object name starting from the root directory. For example, if the path of an object in the bucket is obs://bucket/src1/src2/test.txt, then the absolute path of the object is src1/src2/test.txt.

timeRange

Optional for copying objects in batches (additional parameter)

The time range matching pattern when copying objects. Only objects whose latest modification time is within the configured time range are copied.

This pattern has a lower priority than the file matching patterns (exclude/include). That is, the time range matching pattern is executed after the configured file matching patterns.

  • Time in the matching pattern is the UTC time.
  • The matching time range is represented in time1-time2, where time1 must be earlier than or the same as time2. The time format is yyyyMMddHHmmss.
  • Automatic formatting is supported. For example, yyyyMMdd is equivalent to yyyyMMdd000000, and yyyyMM is equivalent to yyyyMM01000000.
  • If this parameter is set to *-time2, all files whose last modification time is earlier than time2 are matched. If it is set to time1-*, all files whose last modification time is later than time1 are matched.

This matching pattern applies only to objects whose names do not end with a slash (/).

mf

Optional (additional parameter)

Indicates that the name matching pattern (include or exclude) and the time matching pattern (timeRange) also take effect on objects whose names end with a slash (/).

o

Optional (additional parameter)

The folder that stores the result files. After the command is executed, result files (possibly success, failure, and warning files) will be created in the specified folder.

The default value is .obsutil_output, a subfolder in the user's home directory where obsutil commands are executed.

  • A result file should be named as follows: cp_{succeed | failed | warning}_report_time_TaskId.txt.
  • By default, the maximum size of a single result file is 30 MB. You can set the maximum size by configuring recordMaxLogSize in the configuration file.
  • By default, the maximum number of result files that can be retained is 1,024. You can set the maximum number by configuring recordBackups in the configuration file.
  • If there are multiple folders and files and you need to confirm the details of a failed task, refer to the failure result file cp_failed_report_time_TaskId.txt in the result folder and the log files in the log path.

config

Optional (additional parameter)

The user-defined configuration file for executing the current command.

To learn the parameters that can be configured in this file, see Configuration Parameters.

e

Optional (additional parameter)

The endpoint

i

Optional (additional parameter)

The user's AK

k

Optional (additional parameter)

The user's SK

t

Optional (additional parameter)

The user's security token

Response

Field

Description

Parallel

The parameter -p in the request

Jobs

The parameter -j in the request

Threshold

The parameter -threshold in the request

PartSize

The parameter -ps in the request

Exclude

The parameter -exclude in the request

Include

The parameter -include in the request

TimeRange

The parameter -timeRange in the request

VerifyLength

The parameter -vlength in the request

VerifyMd5

The parameter -vmd5 in the request

CheckpointDir

The parameter -cpd in the request

OutputDir

The parameter -o in the request

ArcDir

The parameter -arcDir in the request

Succeed count

The number of successful tasks

Failed count

The number of failed tasks

Skip count

The number of tasks that are skipped during incremental upload, download, or copy, and synchronous upload, download, or copy.

NOTE:

Skipped tasks are recorded into successful tasks.

Warning count

The number of tasks that are executed successfully but contain warnings.

NOTE:
  • The task for which a warning is generated may be a failure or a success, which needs to be further determined according to the corresponding result list.
  • The number of tasks that generate warnings is independent of the number of successful or failed tasks. The total number of tasks is the number of successful tasks plus the number of failed tasks.

Succeed bytes

The number of bytes that are successfully uploaded

max cost

The maximum duration of all tasks, in ms

min cost

The minimum duration of all tasks, in ms

average cost

The average duration of all tasks, in ms

average tps

The average number of tasks completed per second

Task id

The unique ID of an operation, which is used to search for the result file generated for a batch task

Editing, Disabling, Enabling, or Deleting a Cross-Region Replication Rule

You can use OBS Console or APIs to manage cross-region replication rules. You cannot use SDKs, OBS Browser+, or obsutil to do so.

Using OBS Console

  1. In the navigation pane of OBS Console, choose Buckets.
  2. In the bucket list, click the desired bucket. The Objects page is displayed.
  3. In the navigation pane, choose Data Management > Cross-Region Replication. The Cross-Region Replication page is displayed.
  4. (Optional) Edit a cross-region replication rule.

    1. Click Edit in the Operation column of the desired rule. The Edit Cross-Region Replication Rule dialog box is displayed.
      Figure 5 Editing a cross-region replication rule
    2. Edit the cross-region replication rule.
      Table 5 Cross-region replication parameters that you can edit

      Parameter

      Description

      Status

      Indicates the status of the cross-region replication rule. You can enable or disable the cross-region replication rule.

      Source Bucket

      Replicate

      Select the objects to replicate from the source bucket.

      • All objects: Replicates all objects in the source bucket to the destination bucket.
      • Match by prefix: Replicates objects that match the specified prefix to the destination bucket.

      Prefix

      If you select Match by prefix for Replicate, you must specify an object name prefix. Only objects in the source bucket that match the prefix will be replicated to the destination bucket.

      • The prefix cannot be empty and can contain up to 1,024 characters.
      • If the specified prefix overlaps with the prefix of an existing rule, OBS treats the two rules as one and prevents you from configuring the one you are creating. For example, if there is already a rule with prefix abc in OBS, you cannot create another rule whose prefix starts with abc.
      • To replicate a folder, end the prefix with a slash (/), for example, folder1/.

      Synchronize Existing Objects

      Choose whether to synchronize the objects that were already in the source bucket before the rule was configured to the destination bucket.

      By default, these objects are not synchronized.

      Destination Bucket

      Bucket Owner

      Select a bucket in the same account as the source bucket or in a different account.

      • Current account: Select a bucket in the current account. This means the source and destination buckets belong to the same account, enabling same-account, cross-region replication.
      • Another account: Select a bucket in a different account. This means the source and destination buckets belong to different accounts, or the source bucket contains data uploaded by users other than the bucket owner, enabling cross-account, cross-region replication.
        Before configuring a cross-account, cross-region replication rule, perform the following operations:
        • To replicate objects from the source bucket to the destination bucket, first configure a bucket policy on the destination bucket to grant the source bucket owner all permissions except DeleteBucket, PutBucketPolicy, and PutBucketAcl. For details about permission configuration, see Creating a Custom Bucket Policy (Visual Editor).
        • If the source bucket contains objects uploaded by users other than the source bucket owner, you must configure the bucket ACL on the source bucket to grant the source bucket owner the FULL_CONTROL permission and specify the object read permission (set Delivered to true in the API). Only then can objects uploaded by other accounts be replicated to the destination bucket. To achieve this, you can edit the ACL of the source bucket owner.

        • If you want to replicate the ACL of the source object to the destination bucket when replicating the source object content, you must first configure the bucket ACL on the destination bucket to grant the source bucket owner the FULL_CONTROL permission and specify the object read permission (set Delivered to true in the API).

          For details about how to configure a bucket ACL, see Configuring a Bucket ACL. For details about the FULL_CONTROL permission, see Customizing an ACL.

      Region

      Select the region of the destination bucket. The destination and source buckets must be in different regions. This parameter is mandatory when you select Current account.

      Bucket

      Select a destination bucket. This parameter is mandatory when you select Current account.

      NOTE:

      If a bucket is newly created, you may need to wait several minutes before it appears in the drop-down list of the destination bucket.

      Account ID

      Enter the ID of the account that owns the destination bucket. This parameter is mandatory when you select Another account.

      • If the source bucket contains objects uploaded by users other than the bucket owner and these objects need to be replicated to a destination bucket in another region owned by the source bucket owner, enter the account ID of the source bucket owner.
      • To replicate objects from the source bucket to a destination bucket in another region owned by another account, enter the account ID of the destination bucket owner.

      For details about how to obtain the account ID, see Obtaining the Account, IAM User, Project, User Group, Region, Delegating Account ID, Agency Name, and Agency ID.

      Bucket Name

      Enter the name of the destination bucket. This parameter is mandatory when you select Another account.

      To find the name of the destination bucket, see Listing Buckets.

      Use the destination bucket's owner as the owner of replicas

      Set the owner account for replicas in the destination bucket. This parameter is mandatory when you select Another account.

      • This option is selected by default. When selected, ownership and access permissions of the replicas are transferred to the destination bucket's owner.
      • If you do not select this option, the replicas retain the same ownership and access permissions as the source objects.

      Change storage class for replicated objects

      Choose whether to change the storage class of objects replicated from the source bucket to the destination bucket.

      • By default, this option is not selected, meaning that replicated objects use the same storage class as the source objects.
      • If you select this option, you can specify a storage class for objects replicated to the destination bucket.

      Permissions

      Replicate KMS encrypted objects

      Choose whether to replicate KMS encrypted objects from the source bucket to the destination bucket. OBS attempts to replicate KMS-encrypted objects regardless of whether this option is selected.

      • If this option is selected, the drop-down list displays only IAM agencies created for OBS that have permissions for the global project and have the KMS Administrator permission for both the source and destination regions.
      • If this option is not selected, the drop-down list displays only IAM agencies created for OBS that have permissions for the global project but do not have the KMS Administrator permission for either the source or destination region.

      If KMS is not enabled in the destination region or the agency does not have the KMS Administrator permission for both the source and destination regions, KMS encrypted objects cannot be replicated to the destination bucket, and the replication status will be FAILED.

      After a KMS-encrypted object is replicated to the destination bucket, the key for encrypting the object copy will change to the default key obs/default of the destination region.

      IAM agency

      Select an agency that authorizes OBS to operate your resources. This agency allows OBS to perform cross-region replication.

      If no agency is available for cross-region replication, click Create Agency to create one. If you have already created IAM agencies, select one from the drop-down list.

      The IAM agency must meet the following requirements:

      • The IAM agency must be for OBS and the OBS project must have the Tenant Administrator permission.
      • If Replicate KMS encrypted objects is selected, the agency must have the KMS Administrator permission. The authorization scope of the agency must be set to Region-specific projects, and the source and destination regions must be selected.
    3. Click OK to complete the edit of the cross-region replication rule.

  5. (Optional) Disable a cross-region replication rule.

    If a cross-region replication rule is enabled, you can disable it as follows:
    1. Disable Enabled or Not.

      You can also select multiple rules and click Disable above the list to disable them in batches.

    2. Click OK.

  6. (Optional) Enable a cross-region replication rule.

    If a cross-region replication rule is disabled, you can enable it as follows:
    1. Enable Enabled or Not.

      You can also select multiple rules and click Enable above the list to enable them in batches.

    2. Click OK.

  7. (Optional) Delete a cross-region replication rule.

    1. Click Delete in the Operation column of the target rule.

      You can also select multiple rules and click Delete above the list to delete them in batches.

    2. In the Delete This Cross-Region Replication Rule dialog box, click OK.

Using OBS APIs

References

Creating an Agency for Cross-Region Replication

  1. In the Create Cross-Region Replication Rule dialog box on OBS Console, click Create IAM Agencies to go to the Agencies page of the IAM console.
  2. In the Create Cross-Region Replication Rule dialog box on OBS Console, click Create Agency. The Create Agency dialog box is displayed on the right.
  3. Enter an agency name. The agency name cannot be left blank.
  4. (Optional) Create a custom policy that grants OBS permissions to the agency. If a policy that meets the requirements already exists, skip this step.

    1. In the navigation pane of the IAM console, choose Permissions > Policies/Roles.
    2. On the Policies/Roles page, click Create Custom Policy in the upper right corner.
    3. On the Create Custom Policy page, configure the parameters as described below. Keep the default settings for other parameters.
      Figure 6 Configuring a custom policy
      • Policy Name: Enter a name that is easy to remember.
      • Select service: Select Object Storage Service (OBS).
      • Actions: Select the actions listed in the table below.
        Table 6 OBS permissions required by the agency

        Action

        Description

        obs:object:GetObject

        Obtains object content and metadata.

        obs:object:DeleteObjectVersion

        Deletes one or more objects.

        obs:object:PutObjectVersionAcl

        Configures the object version ACL.

        obs:object:AbortMultipartUpload

        Aborts a multipart upload.

        obs:object:PutObjectAcl

        Configures the object ACL.

        obs:object:DeleteObject

        Deletes one or more objects.

        obs:bucket:HeadBucket

        Obtains bucket metadata.

        obs:object:PutObject

        Uploads objects with PUT or POST, copies objects, appends data to objects, initiates a multipart upload, as well as uploads and assembles parts.

        obs:object:GetObjectVersionAcl

        Obtains the object ACL.

        obs:bucket:GetBucketVersioning

        Obtains the versioning status of a bucket.

        obs:bucket:ListBucketMultipartUploads

        Lists multipart uploads.

        obs:object:ListMultipartUploadParts

        Lists uploaded parts.

        obs:object:ModifyObjectMetaData

        Modifies object metadata.

        obs:bucket:ListBucketVersions

        Lists object versions in a bucket.

        obs:bucket:ListBucket

        Lists objects in a bucket.

        obs:object:GetObjectVersion

        Obtains object content and metadata.

        obs:object:GetObjectAcl

        Obtains the object ACL.

        obs:bucket:GetReplicationConfiguration

        Obtains the cross-region replication configuration of a bucket.
      • Resources: Select Specific.
        • object: Select Specify resource path. The resource path format is OBS:*:*:object:bucket-name/object-name.

          Add all objects in the source and destination buckets. For example, set the resource path to OBS:*:*:object:piccomp/* and OBS:*:*:object:piccomp-output/*, indicating all objects in source bucket piccomp and destination bucket piccomp-output, respectively.

        • bucket: Select Specify resource path. The resource path format is OBS:*:*:bucket:Bucket name.

          Add the source and destination buckets. For example, set the resource path to OBS:*:*:bucket:piccomp and OBS:*:*:bucket:piccomp-output, indicating the source bucket piccomp and destination bucket piccomp-output, respectively.

    4. Click OK. If the created custom policy appears in the list, it was created successfully.

  5. Grant related permissions to the agency.

    Select Custom policy from the first drop-down list under Authorization. In the custom policy list, select the policy created in 4.

    If Replicate KMS encrypted objects is selected, the agency also needs the KMS CMKFullAccess permission for the regions where the source and destination buckets are located.

  6. Click OK.

    The agency drop-down list is automatically populated with the newly created agency. To view agency authorization details, click Check authorization details in the agency drop-down list.

  7. (Optional) If Replicate KMS encrypted objects is selected, the IAM agency also needs the KMS Administrator permission for the regions where the source and destination buckets are located.

    1. In the navigation pane of the IAM console, choose Agencies.
    2. Click Authorize in the Operation column of the agency created in 2.
    3. On the Select Policy/Role page, search for and select KMS Administrator. Then, click Next.
    4. Select Region-specific projects for Scope, and select the projects for the regions where the source and destination buckets are located.
    5. Click OK.

FAQs About Cross-Region Replication Within an Account

Parent Topic: Data Replication