Cross-Account, Cross-Region Replication Using OBS

To perform cross-account, cross-region replication, you must grant the destination account the read access to the source bucket and write access to the destination bucket. You can use bucket policies to grant access.

Configure a source bucket policy to allow the destination account to read data from the source bucket.

1. Log in to OBS Console using account A (the account that owns the source bucket).
2. In the bucket list, click the source bucket name to go to the details page.
3. In the navigation pane, choose Permissions > Bucket Policies.
4. Click Create. On the displayed page, create a bucket policy using either the visual editor or JSON mode.
   • Create a bucket policy in the visual editor. For details about parameter settings, see Table 3.

     Table 3 Creating a source bucket policy in the visual editor

     Parameter	Description
     Policy Name	Define a name based on your service requirements.
     Effect	Allow: The policy allows the matched requests.
     Principal	Select Other accounts. Enter the ID of the destination account for replication and the IAM user ID or IAM username.
     Resources	Select Entire bucket (including the objects in it).
     Actions	Select Use a template and Bucket Read-Only.
     Conditions	Do not configure it (defaults to all).

   • Create a bucket policy in JSON format:

     {
         "Statement": [
             {
                 "Sid": "<bucket-policy-name>",
                 "Effect": "Allow",
                 "Principal": {
                     "ID": [
                         "domain/account-ID/<IAM-user-ID>"
                     ]
                 },
                 "Action": [
                     "Get*",
                     "List*",
                     "HeadBucket"
                 ],
                 "Resource": [
                     "<source-bucket-name>",
                     "<source-bucket-name>/*"
                 ]
             }
         ]
     }

5. Click Create. The source bucket's read policy is created.

Configure a destination bucket policy to grant the destination account write access to the destination bucket.

1. Log in to OBS Console using the destination account.
2. In the bucket list, click the destination bucket name to go to the details page.
3. In the navigation pane, choose Permissions > Bucket Policies.
4. Click Create. On the displayed page, create a bucket policy using either the visual editor or JSON mode.
   • Create a bucket policy in the visual editor. For details about parameter settings, see Table 4.

     Table 4 Creating a destination bucket policy in the visual editor

     Parameter	Description
     Policy Name	Define a name based on your service requirements.
     Effect	Allow: The policy allows the matched requests.
     Principal	Select Other accounts. Enter the ID of the destination account for replication and the IAM user ID or IAM username.
     Resources	Select Entire bucket (including the objects in it).
     Actions	Select Use a template and Bucket Read/Write.
     Conditions	Do not configure it (defaults to all).

   • Create a bucket policy in JSON format:

     {
         "Statement": [
             {
                 "Sid": "<bucket-policy-name>",
                 "Effect": "Allow",
                 "Principal": {
                     "ID": [
                         "domain/account-ID/<IAM-user-ID>"
                     ]
                 },
                 "NotAction": [
                     "DeleteBucket",
                     "PutBucketPolicy",
                     "PutBucketAcl"
                 ],
                 "Resource": [
                     "destination-bucket-name",
                     "destination-bucket-name/*"
                 ]
             }
         ]
     }

5. Click Create. The write policy for the destination bucket is created.

1. In the navigation pane of OBS Console, choose Buckets.
2. In the bucket list, click the desired bucket. The Objects page is displayed.
3. In the navigation pane, choose Data Management > Cross-Region Replication.
4. Select "I have read and agree to the disclaimer." Then, click Create Rule. The Create Cross-Region Replication Rule dialog box is displayed, as shown in Figure 1.
   Figure 1 Creating a cross-region replication rule

   

   

   • The source and destination buckets must have the same versioning status.
   • A bucket can have only one destination bucket and one IAM agency for cross-region replication. Any new replication rule overwrites the destination bucket and IAM agency defined in an earlier rule for the same bucket.

5. Create a cross-region replication rule according to your needs. For details, see Table 3.

   Table 5 Parameters for configuring a cross-region replication rule

   Parameter		Description
   Enabled or Not		Choose whether to enable or disable the rule after it is created. The source and destination buckets must have the same versioning status.
   Source Bucket	Replicate	Select the objects to replicate from the source bucket. All objects: Replicates all objects in the source bucket to the destination bucket. Match by prefix: Replicates objects that match the specified prefix to the destination bucket.
   	Prefix	If you select Match by prefix for Replicate, you must specify an object name prefix. Only objects in the source bucket that match the prefix will be replicated to the destination bucket. The prefix cannot be empty and can contain up to 1,024 characters. If the specified prefix overlaps with the prefix of an existing rule, OBS treats the two rules as one and prevents you from configuring the one you are creating. For example, if there is already a rule with prefix abc in OBS, you cannot create another rule whose prefix starts with abc. To replicate a folder, end the prefix with a slash (/), for example, folder1/.
   	Synchronize Existing Objects	Choose whether to synchronize the objects that were already in the source bucket before the rule was configured to the destination bucket. By default, these objects are not synchronized.
   Destination Bucket	Bucket Owner	Select a bucket in the same account as the source bucket or in a different account. Select Another account here. Current account: Select a bucket in the current account. This means the source and destination buckets belong to the same account, enabling same-account, cross-region replication. Another account: Select a bucket in a different account. This means the source and destination buckets belong to different accounts, or the source bucket contains data uploaded by users other than the bucket owner, enabling cross-account, cross-region replication. Before configuring a cross-account, cross-region replication rule, perform the following operations: To replicate objects from the source bucket to the destination bucket, first configure a bucket policy on the destination bucket to grant the source bucket owner all permissions except DeleteBucket, PutBucketPolicy, and PutBucketAcl. For details about permission configuration, see Creating a Custom Bucket Policy (Visual Editor). If the source bucket contains objects uploaded by users other than the source bucket owner, you must configure the bucket ACL on the source bucket to grant the source bucket owner the FULL_CONTROL permission and specify the object read permission (set Delivered to true in the API). Only then can objects uploaded by other accounts be replicated to the destination bucket. To achieve this, you can edit the ACL of the source bucket owner. If you want to replicate the ACL of the source object to the destination bucket when replicating the source object content, you must first configure the bucket ACL on the destination bucket to grant the source bucket owner the FULL_CONTROL permission and specify the object read permission (set Delivered to true in the API). For details about how to configure a bucket ACL, see Configuring a Bucket ACL. For details about the FULL_CONTROL permission, see Customizing an ACL.
   	Account ID	Enter the ID of the account that owns the destination bucket. This parameter is mandatory when you select Another account. If the source bucket contains objects uploaded by users other than the bucket owner and these objects need to be replicated to a destination bucket in another region owned by the source bucket owner, enter the account ID of the source bucket owner. To replicate objects from the source bucket to a destination bucket in another region owned by another account, enter the account ID of the destination bucket owner. For details about how to obtain the account ID, see Obtaining the Account, IAM User, Project, User Group, Region, Delegating Account ID, Agency Name, and Agency ID.
   	Bucket Name	Enter the name of the destination bucket. This parameter is mandatory when you select Another account. To find the name of the destination bucket, see Listing Buckets.
   	Use the destination bucket's owner as the owner of replicas	Set the owner account for replicas in the destination bucket. This parameter is mandatory when you select Another account. This option is selected by default. When selected, ownership and access permissions of the replicas are transferred to the destination bucket's owner. If you do not select this option, the replicas retain the same ownership and access permissions as the source objects.
   	Change storage class for replicated objects	Choose whether to change the storage class of objects replicated from the source bucket to the destination bucket. By default, this option is not selected, meaning that replicated objects use the same storage class as the source objects. If you select this option, you can specify a storage class for objects replicated to the destination bucket.
   Permissions	Replicate KMS encrypted objects	Choose whether to replicate KMS encrypted objects from the source bucket to the destination bucket. OBS attempts to replicate KMS-encrypted objects regardless of whether this option is selected. If you set Bucket Owner to Another account, cross-account, cross-region replication of SSE-KMS encrypted objects is not supported. Therefore, retain the default setting and do not select this option.
   	IAM agency	Select an agency that authorizes OBS to operate your resources. This agency allows OBS to perform cross-region replication. If no agencies are available, click Create Agency to create one for cross-region replication. For details about how to create an agency, see Creating an Agency for Cross-Region Replication. If you have already created IAM agencies, select one from the drop-down list. The IAM agency must meet the following requirements: The IAM agency must be for OBS and must have the OBS permissions listed in Table 6. If Replicate KMS encrypted objects is selected, the agency must have the KMS Administrator permission. The authorization scope of the agency must be set to Region-specific projects, and the source and destination regions must be selected.

   Table 6 OBS permissions required by the agency

   Action	Description
   obs:object:GetObject	Obtains object content and metadata.
   obs:object:DeleteObjectVersion	Deletes one or more objects.
   obs:object:PutObjectVersionAcl	Configures the object ACL.
   obs:object:AbortMultipartUpload	Aborts a multipart upload.
   obs:object:PutObjectAcl	Configures the object ACL.
   obs:object:DeleteObject	Deletes one or more objects.
   obs:bucket:HeadBucket	Obtains bucket metadata.
   obs:object:PutObject	Uploads objects with PUT or POST, copies objects, appends data to objects, initiates a multipart upload, as well as uploads and assembles parts.
   obs:object:GetObjectVersionAcl	Obtains the object ACL.
   obs:bucket:GetBucketVersioning	Obtains the versioning status of a bucket.
   obs:bucket:ListBucketMultipartUploads	Lists multipart uploads.
   obs:object:ListMultipartUploadParts	Lists uploaded parts.
   obs:object:ModifyObjectMetaData	Modifies object metadata.
   obs:bucket:ListBucketVersions	Lists object versions in a bucket.
   obs:bucket:ListBucket	Lists objects in a bucket.
   obs:object:GetObjectVersion	Obtains object content and metadata.
   obs:object:GetObjectAcl	Obtains the object ACL.
   obs:bucket:GetReplicationConfiguration	Obtains the cross-region replication configuration of a bucket.

6. (Optional) Create an IAM agency. For details, see Creating an Agency for Cross-Region Replication.
7. Click OK. The cross-account, cross-region replication rule is created.

Configure AccessControlTranslation in the API request for configuring cross-region replication for a bucket.

When using obsutil to replicate objects across regions and accounts, you must enable cross-region replication, which means that the crr parameter is mandatory.

• Do not change the source objects in the OBS bucket when copying a single object or objects in batches. Otherwise, the operation may fail or data may be inconsistent.
• If the storage class of the object to be copied is cold, you must restore the object to be copied first. Otherwise, the copy fails.
• To copy objects, you must have the read permission on the objects to be copied and the write permission on the destination bucket.
• If the client-side cross-region replication function is not enabled, ensure that the source bucket and destination bucket are in the same region.
• If the source bucket is a parallel file system (supporting POSIX), the destination bucket must also be a parallel file system.

Constraints:

• The source and destination paths cannot be the same.
• The source and destination paths cannot be nested during a batch copy.
  • If the destination path has the source path as its prefix, recursive copy may occur.
  • If the source path has the destination path as its prefix, overwriting may occur.

Command Line Structure

• Windows
  • Copying a single object

    obsutil cp obs://srcbucket/key obs://dstbucket/[dest] [-dryRun] [-u] [-crr] [-vlength] [-vmd5] [-p=1] [-threshold=52428800] [-versionId=xxx] [-acl=xxx] [-sc=xxx] [-meta=aaa:bbb#ccc:ddd] [-ps=auto] [-cpd=xxx] [-fr] [-o=xxx] [-config=xxx] [-e=xxx] [-i=xxx] [-k=xxx] [-t=xxx]

  • Copying objects in batches

    obsutil cp obs://srcbucket[/key] obs://dstbucket[/dest] -r [-dryRun] [-f] [-flat] [-u] [-crr] [-vlength] [-vmd5] [-j=1] [-p=1] [-threshold=52428800] [-acl=xxx] [-sc=xxx] [-meta=aaa:bbb#ccc:ddd] [-ps=auto] [-include=*.xxx] [-exclude=*.xxx] [-timeRange=time1-time2] [-mf] [-o=xxx] [-cpd=xxx] [-config=xxx] [-e=xxx] [-i=xxx] [-k=xxx] [-t=xxx]

• macOS or Linux
  • Copying a single object

    ./obsutil cp obs://srcbucket/key obs://dstbucket/[dest] [-dryRun] [-u] [-crr] [-vlength] [-vmd5] [-p=1] [-threshold=52428800] [-versionId=xxx] [-acl=xxx] [-sc=xxx] [-meta=aaa:bbb#ccc:ddd] [-ps=auto] [-cpd=xxx] [-fr] [-o=xxx] [-config=xxx] [-e=xxx] [-i=xxx] [-k=xxx] [-t=xxx]

  • Copying objects in batches

    ./obsutil cp obs://srcbucket[/key] obs://dstbucket[/dest] -r [-dryRun] [-f] [-flat] [-u] [-crr] [-vlength] [-vmd5] [-j=1] [-p=1] [-threshold=52428800] [-acl=xxx] [-sc=xxx] [-meta=aaa:bbb#ccc:ddd] [-ps=auto] [-include=*.xxx] [-exclude=*.xxx] [-timeRange=time1-time2] [-mf] [-o=xxx] [-cpd=xxx] [-config=xxx] [-e=xxx] [-i=xxx] [-k=xxx] [-t=xxx]

Examples

• In Windows, run the following command to copy a single object:

  obsutil cp obs://bucket-test/key obs://bucket-test2 -crr
  Start at 2024-09-30 08:30:09.0815415 +0000 UTC
  
  Parallel:      3                   Jobs:          3
  Threshold:     50.00MB             PartSize:      auto
  CheckpointDir: xxxx
  
  [=====================================================] 100.00% 6/s 0s
  Copy successfully, 19B, obs://bucket-test/key --> obs://bucket-test2/key
  ext.txt, cost [1708], status [200], request id [00000192420D227E4017336A12F1DC22]

• In Windows, run the following command to copy objects in batches:

  obsutil cp obs://bucket-test/temp/ obs://bucket-test2 -r -f -crr
  Start at 2024-09-30 08:34:02.7819703 +0000 UTC
  
  Parallel:      5                   Jobs:          5
  Threshold:     50.00MB             PartSize:      auto
  CheckpointDir: xxxx
  
  Task id: 0476929d-9d23-4dc5-b2f8-0a0493f027c5
  OutputDir: xxxx
  
  [=============================================================] 100.00% 10/s 0s
  Succeed count:   5         Failed count:    0
  Metrics [max cost:298 ms, min cost:192 ms, average cost:238.00 ms, average tps:9.71, transferred size: 7.20MB]
  Task id: 0476929d-9d23-4dc5-b2f8-0a0493f027c5

• For more examples, see Common Examples.

Parameter Description

./obsutil cp obs://src-bucket/ obs://target-bucket/ -include=/src-object/* -f -r

Response