Help Center/ Object Storage Service/ User Guide/ Data Management/ Using Mirroring-Based Back to Source to Retrieve Data
Updated on 2026-08-21 GMT+08:00

Using Mirroring-Based Back to Source to Retrieve Data

Usually, if the data requested by a client using GET is not found in OBS, a 404 error will be returned. To avoid this error, OBS provides back to source that pulls the requested data from the source site (origin server) if it is not found in OBS.

Process of Back to Source by Mirroring

If the data you requested is not found in a bucket that has a back-to-source by mirroring rule configured and the request matches the rule, OBS will pull the data from its origin server and return it to you. This process does not interrupt services. Therefore, you can use this function to seamlessly migrate data from the origin server to OBS without being sensed by users, at low costs. #EN-US_TOPIC_0000001858185970/en-us_topic_0000001874795321_en-us_topic_0129288983_fig8517161311401 shows the process of back to source by mirroring.

Figure 1 Process of back to source by mirroring

During the back to source by mirroring process, OBS functions as a proxy. However, if the files on the origin server change after they are stored in OBS, OBS will not automatically update the files.

Back-to-Source Rules

Item

Description

Conditions for triggering an origin pull rule

OBS pulls data from the origin server only when a GetObject request results in a 404 response.

Rules for naming the retrieved files

If OBS retrieves a file from the origin server via http(s)://MirrorURL/ObjectName, the file is stored in OBS under the same name, ObjectName. For example, if the source address set for a bucket is https://yun.com and the requested file example.jpg is not found in the bucket, OBS will retrieve the file via https://yun.com/example.jpg and file example.jpg is then pushed to OBS.

Status code return rules in the case of retrieval failures

If OBS fails to retrieve a file from the source, it returns 404 to the client by default.

Rules for updating the retrieved files

If a file has been pushed by the origin server to OBS through mirroring, OBS does not update the file even if the source file changes.

HTTP request rules

By default, headers and query strings included in requests to OBS are not transferred to the origin server. However, you can configure the back to source settings to specify whether they should be passed along.

Constraints

Table 1 Back to source constraints

Category

Description

Bucket versions

Only buckets of version 3.0 or later support back to source by mirroring.

Time

It takes about five minutes to apply any changes to a back-to-source by mirroring rule.

Number of rules

A maximum of 10 back-to-source by mirroring rules can be configured for a bucket.

Functions

  • Parallel file systems do not support back-to-source by mirroring rules.
  • Static website hosting does not support back to source by mirroring. Specifically, if 404 is returned when you use a static website domain name to download an object, back to source by mirroring will not be triggered.
  • A bucket cannot mirror itself.
  • Currently, back to source by mirroring from private buckets is only supported for some cloud vendors.
  • In back-to-source setups, two buckets cannot act as each other's origin. For example, if bucket A is the origin for bucket B, bucket B cannot be the origin for bucket A.
  • Transfer-Encoding: chunked cannot be used for the origin server to transmit data, or the origin pull will fail. The response to the request for downloading an object from the origin server must contain the Content-Length header to specify the size of the source object.

    To achieve this, in the Create/Edit Back-to-Source Rule window, choose Pass all parameters for HTTP Header Pass Rule, enable Do not pass specified parameters, and add Accept-Encoding.

  • If both a CDN acceleration domain name and a back-to-source by mirroring rule are configured and range requests are enabled on CDN, the origin server must return at least one of the Etag and Last-Modified headers. Otherwise, the request may be interrupted.
  • If NGINX was deployed as a reverse proxy for your origin server, turn off chunked_transfer_encoding of the NGINX.
    location / {
         chunked_transfer_encoding off;
    }

Permissions

  • To configure, obtain, or delete back-to-source by mirroring rules, you must have the Tenant Administrator permission assigned through role/policy-based authorization of IAM.
  • You must create a cloud service agency using role/policy-based authorization of IAM to delegate OBS to pull data from the origin server. The agency must include the obs:object:PutObject, obs:object:GetObject, obs:bucket:ListBucket, and obs:object:AbortMultipartUpload permissions.
  • If SSE-KMS is enabled for a bucket, the kms:cmk:get, kms:cmk:list, kms:cmk:create, kms:dek:create, kms:dek:encrypt, and kms:dek:decrypt permissions must be configured for the IAM agency for OBS using role/policy-based authorization.

Others

  • Back to source by mirroring is free now.
  • An object cannot match two different back-to-source by mirroring rules.

Important Notes

  • If a file has been pushed by the origin server to OBS through mirroring, OBS does not simultaneously update the file even if the source file changes. You can use the following methods to indirectly synchronize files in OBS with those on the origin server:
    • Manually deleting pulled files: Manually delete the files cached in the bucket using the console or APIs. This ensures that the system fetches the latest data from the origin server on the next pull.
    • Configuring lifecycle rules to periodically delete pulled files: Configure lifecycle rules for pulled files so that they are automatically deleted after a specified period. This ensures that the system fetches the latest data from the origin server on the next pull.
    • Naming origin server files by version: When origin server files are updated, use version names or timestamps (for example, testv1.txt or test20260617181156.txt) to distinguish them. This helps quickly identify updated files.
  • When using back to source mirroring for data migration, ensure that the origin server has sufficient bandwidth to handle back-to-source requests. You are advised to migrate data during off-peak hours and monitor the origin server load in real time.
  • During back-to-source, the system continuously fetches data from the origin server and stores it in the bucket, which will incur storage charges. You are advised to set an expenditure alert to keep storage costs under control.
  • If the source address uses the HTTPS protocol, ensure that the certificate configured for the origin server is trusted and valid.

Creating a Back-to-Source by Mirroring Rule

You can use OBS Console, APIs to create back-to-source by mirroring rules. You cannot use SDKs, obsutil, or OBS Browser+ to do so.

Back to Source by Mirroring Examples

References