Configuring Back-to-Source by Mirroring Rules

Functions

OBS provides the back-to-source function. If the requested data cannot be found, OBS can obtain the data from the origin server based on back-to-source rules. This API is used to configure back-to-source by mirroring rules for a bucket. For more information about back to source by mirroring, Using Mirroring-Based Back to Source to Retrieve Data.

Constraints

**Table 1** Back to source constraints
Category	Description
Bucket versions	Only buckets of version 3.0 or later support back to source by mirroring.
Time	It takes about five minutes to apply any changes to a back-to-source by mirroring rule.
Number of rules	A maximum of 10 back-to-source by mirroring rules can be configured for a bucket.
Functions	Anonymous users are not allowed to configure back-to-source by mirroring rules. Federated users cannot use back to source by mirroring. For more information about federated users, see What Are the Relationships Between a Huawei Cloud Account, HUAWEI ID, IAM User, and Federated User? and Introduction to Identity Providers. Parallel file systems do not support back-to-source by mirroring rules. Static website hosting does not support back to source by mirroring. Specifically, if 404 is returned when you use a static website domain name to download an object, back to source by mirroring will not be triggered. A bucket cannot mirror itself. Currently, back to source by mirroring from private buckets is only supported for some cloud vendors. In back-to-source setups, two buckets cannot act as each other's origin. For example, if bucket A is the origin for bucket B, bucket B cannot be the origin for bucket A. Transfer-Encoding: chunked cannot be used for the origin server to transmit data, or the origin pull will fail. The response to the request for downloading an object from the origin server must contain the Content-Length header to specify the size of the source object. To achieve this, in the Create/Edit Back-to-Source Rule window, choose Pass all parameters for HTTP Header Pass Rule, enable Do not pass specified parameters, and add Accept-Encoding. If both a CDN acceleration domain name and a back-to-source by mirroring rule are configured and range requests are enabled on CDN, the origin server must return at least one of the Etag and Last-Modified headers. Otherwise, the request may be interrupted. If NGINX was deployed as a reverse proxy for your origin server, turn off chunked_transfer_encoding of the NGINX. location / { chunked_transfer_encoding off; }
Permissions	To configure, obtain, or delete back-to-source by mirroring rules, you must have the Tenant Administrator permission assigned by using role/policy-based authorization of IAM. You must create a cloud service agency using role/policy-based authorization of IAM to delegate OBS to pull data from the origin server. The agency must include the obs:object:PutObject, obs:object:GetObject, obs:bucket:ListBucket, and obs:object:AbortMultipartUpload permissions. If SSE-KMS is enabled for a bucket, the kms:cmk:get, kms:cmk:list, kms:cmk:create, kms:dek:create, and kms:dek:crypto permissions must be configured for the IAM agency for OBS by using role/policy-based authorization.
Others	Back to source by mirroring is free now. An object cannot match two different back-to-source by mirroring rules.

Authorization Information

To call this API, you must be the bucket owner or have the Tenant Administrator permission. To configure the Tenant Administrator permission, you need to use role/policy-based authorization (IAM v3 APIs in the old IAM version). For details, see Using IAM for Authorization.

Request Syntax

PUT /?mirrorBackToSource HTTP/1.1
Host: bucketname.obs.region.myhuaweicloud.com 
Authorization: authorization
Content-Type: application/json
Content-Length: length
Date: date

policy json body

URI Parameters

This request contains no message parameters.

Request Headers

This request uses common headers. For details, see Table 3.

Request Body

**Table 2** Request body parameters
Parameter	Mandatory	Type	Description
rules	Yes	Container	Definition Number of rules. rules is the parent node of id, condition, and redirect. Constraints For the same bucket, prefixes of different rules cannot contain each other or have duplicate starting characters. The same agency is recommended. Range A maximum of 10 back-to-source by mirroring rules can be created for each bucket. Therefore, the value range of the array length is from 1 to 10. For details, see Table 3. Default Value N/A

**Table 3** rules parameters
Parameter	Mandatory	Type	Description
id	Yes	String	Definition Unique ID of a back-to-source rule configured for the current bucket. Constraints The rule ID must be unique in the bucket. Range The value is a string of 1 to 256 characters. Only uppercase letters, lowercase letters, digits, underscores (_), and hyphens (-) are allowed. Default Value N/A
condition	Yes	Container	Definition Condition for triggering back-to-source. This parameter is the parent node of httpErrorCodeReturnedEquals and objectKeyPrefixEquals. Constraints N/A Range For details, see Table 4. Default Value N/A
redirect	Yes	Container	Definition Parameters for implementing the back-to-source function. This parameter is the parent node of agency, publicSource, retryConditions, passQueryString, mirrorFollowRedirect, replaceKeyWith, replaceKeyPrefixWith, vpcEndpointURN, redirectWithoutReferer, and mirrorAllowHttpMethod. Constraints N/A Range For details, see Table 5. Default Value N/A

**Table 4** condition parameters
Parameter	Mandatory	Type	Description
httpErrorCodeReturnedEquals	Yes	Integer	Definition Error code that triggers the back-to-source function. When this error is returned for a download request, the back-to-source function is triggered. Constraints N/A Range 404: The object does not exist in the OBS bucket. Default Value 404
objectKeyPrefixEquals	No	String	Definition Prefix of the object name that triggers the back-to-source function. The back-to-source function is performed only when the specified object name prefix is contained in the request. Constraints If this parameter is left blank, all objects are matched by default. For the same bucket, prefixes of different rules cannot contain each other or have duplicate beginning characters. Range The value is a string of 0 to 1023 characters and can contain any characters. Default Value N/A

**Table 5** redirect parameters
Parameter	Mandatory	Type	Description
agency	Yes	String	Definition Agency name. With an agency, the customer can grant OBS the permissions to query whether a specified object exists in the bucket and to upload objects to the bucket. Constraints N/A Range N/A Default Value N/A
publicSource	No	Container	Definition Configuration of the source that can be publicly accessed. This parameter is mandatory when the source is a publicly accessible resource. publicSource is the parent node of sourceEndpoint. Constraints N/A Range For details, see Table 6. Default Value N/A
retryConditions	No	Array	Definition Condition for switching the source address. Constraints 4XX and error codes starting with 4 cannot be configured together. 5XX and error codes starting with 5 cannot be configured together. Range 4XX, 5XX, 400–499, and 500–599 (a maximum of 20 error codes can be configured at the same time) Default Value N/A
passQueryString	Yes	Boolean	Definition Whether to carry the request character string. Constraints If this parameter is set to true but the query parameter contains the signature information, the signature information is removed and the remaining parameters are passed. Range false: The queryString value in the OBS request is not passed to the source. true: The queryString value in the OBS request is passed to the source. Default Value false
mirrorFollowRedirect	Yes	Boolean	Definition Whether to obtain resources by redirecting the request based on the 3XX response from the source Constraints N/A Range false: OBS transparently passes the 3XX response and does not return resources. true: OBS returns resources by redirecting the request based on the 3XX response. Default Value false
mirrorHttpHeader	No	Container	Definition Rules for passing HTTP headers. mirrorHttpHeader is the parent node of passAll, pass, remove, and set. Constraints N/A Range For details, see Table 8. Default Value N/A
replaceKeyWith	No	String	Definition Whether to add a prefix or suffix when downloading objects from the source. Constraints If both replaceKeyWith and ReplaceKeyPrefixWith are left blank, ReplaceKeyPrefixWith takes effect. The request is invalid if both the parameters are specified. Range prefix${key}suffix: The prefix or suffix needs to be added. ${key}: No prefix or suffix needs to be added. ${key} indicates keywords. The total length of the prefix and suffix ranges from 0 to 1023. Default Value N/A
replaceKeyPrefixWith	No	String	Definition Character string used to replace objectKeyPrefixEquals. If you need to replace the current object name prefix when downloading the object from the source, specify this parameter. Constraints If both replaceKeyWith and ReplaceKeyPrefixWith are left blank, ReplaceKeyPrefixWith takes effect. The request is invalid if both the parameters are specified. Range The value is a string of 0 to 1023 characters. Default Value N/A
vpcEndpointURN	No	String	Definition URN of VPC Endpoint service. Constraints N/A Range The value is a string of 0 to 127 characters. Default Value N/A
redirectWithoutReferer	No	Boolean	Definition Whether to skip carrying the original host as the referer header to the destination address for redirection. Constraints N/A Range false: The original host is carried as the referer header to the destination address for redirection. true: The original host is not carried as the referer header to the destination address for redirection. Default Value false
mirrorAllowHttpMethod	No	Array	Definition Request method that supports transparent transmission. Constraints N/A Range GET: requests to obtain objects in a bucket HEAD: requests to obtain the metadata of an object in a bucket Default Value N/A

**Table 6** publicSource parameters
Parameter	Mandatory	Type	Description
sourceEndpoint	No	Container	Definition Source address that can be publicly accessed. sourceEndpoint is the parent node of master and slave. Constraints N/A Range For details, see Table 7. Default Value N/A

**Table 7** sourceEndpoint parameters
Parameter	Mandatory	Type	Description
master	No	Array	Definition Master source address. If the source is a bucket that can be accessed over HTTP network, the address is the bucket domain name. If the source is a private bucket provided by other cloud vendors, the address is a region domain name. Constraints The master source address is preferentially used during the back-to-source process. If one to five master source addresses are configured, they are accessed in polling mode. If two or more master addresses are configured, when the first request fails and the retry conditions are met, an address other than the one involved in the first request is used to retry. Range The format of a single source address is https://xxx.yyy.zzz or http://xxx.yyy.zzz. The value is a string of 10 to 255 characters. Default Value N/A
slave	No	Array	Definition Slave source address. If the source is a bucket that can be accessed over HTTP network, the address is the bucket domain name. If the source is a private bucket provided by other cloud vendors, the address is a region domain name. Constraints A back-to-source request will retry a slave source address when master ones are not available. A maximum of five slave addresses can be configured. Range The format of a single source address is https://xxx.yyy.zzz or http://xxx.yyy.zzz. The value is a string of 10 to 255 characters. Default Value N/A

**Table 8** mirrorHttpHeader parameters
Parameter	Mandatory	Type	Description
passAll	No	Boolean	Definition Whether to pass all HTTP headers to the source. Constraints passAll and pass are mutually exclusive. The following HTTP header types do not support transparent transmission: Headers starting with the following prefixes: x-obs- x-amz- All standard HTTP headers, for example: Content-Length Authorization Date Range false: Do not transparently transmit all HTTP headers to the source. true: All HTTP headers are transparently transmitted to the source. Default Value false
pass	No	Array	Definition The list of HTTP headers to be transparently transmitted. Constraints Only letters, digits, hyphens (-), and underscores (_) are allowed. Range A maximum of 10 HTTP headers can be displayed. The length of each HTTP header ranges from 1 to 63. Default Value N/A
remove	No	Array	Definition List of HTTP headers that cannot be transparently transmitted. Constraints The remove operation takes precedence over the pass and passAll operations. Only letters, digits, hyphens (-), and underscores (_) are allowed. Range A maximum of 10 HTTP headers can be displayed. The length of each HTTP header ranges from 1 to 63. Default Value N/A
set	No	Array	Definition List of HTTP header values to be transparently transmitted. Constraints The set operation has a higher priority than the remove, pass, and passAll operations. If a custom header contains the Referer header, redirectWithoutReferer must be set to true. Otherwise, there will be an overwriting. Range A maximum of 10 HTTP headers can be displayed in a set. Each HTTP header contains a key and a value. For details about keys and values, see Table 9. Default Value N/A

**Table 9** set parameters
Parameter	Mandatory	Type	Description
key	No	String	Definition Keyword of the HTTP header that needs to be transparently transmitted. Constraints Each key is unique. Only letters, digits, hyphens (-), and underscores (_) are allowed. Range The value is a string of 1 to 63 characters. Default Value N/A
value	No	String	Definition Value of the HTTP header that needs to be transparently transmitted. Constraints N/A Range The value is a string of 1 to 2048 characters. Default Value N/A

Response Syntax

HTTP/1.1 status
Server: OBS
Date: date
Content-Length: length

Response Headers

This response uses common headers. For details, see Table 1.

Response Body

This response contains no elements.

Error Responses

No special error responses are returned. For details about error responses, see Table 2 and Table 3.

Sample Request

PUT /?mirrorBackToSource HTTP/1.1
Host: bucketname.obs.region.myhuaweicloud.com 
Authorization: OBS H4IPJX0TQTHTHEBQQCEC:sc2PM13Wlfcoc/YZLK0MwsI2Zpo=
Content-Type: application/json
Content-Length: 1049
Date: Tue, 21 Jul 2020 15:38:30 GMT

{
    "rules": [{
        "id": "abc123",
        "condition": {
            "httpErrorCodeReturnedEquals": "404",
            "objectKeyPrefixEquals": "video/"
        },
        "redirect": {
            "agency": "agency",
            "publicSource": {
                "sourceEndpoint": {
                    "master":["http://bucket1.xxx.yyy.com", "https://bucket2.xxx.yyy.com"],
                    "slave": ["http://bucket3.xxx.yyy.com", "https://bucket4.xxx.yyy.com"]
                }
            },
            "retryConditions": ["4XX", "5XX"],
            "passQueryString": true,
            "mirrorFollowRedirect": true,
            "redirectWithoutReferer": true,
            "mirrorAllowHttpMethod":["HEAD"],
            "mirrorHttpHeader": {
                "passAll": false,
                "pass": ["content-encoding"],
                "remove": ["content-type"],
                "set": [{
                    "key": "helloworld",
                    "value": "2222"
                }]
            },
            "replaceKeyWith": "prefix${key}suffix",
            "replaceKeyPrefixWith": "picture/",
            "vpcEndpointURN": "001"
        }
    }]
}

Sample Response

HTTP/1.1 201 Created
Server: OBS
Date: Tue, 07 Jul 2020 07:29:13 GMT
Content-Length: 0

References

For more information about back to source by mirroring, see Using Back to Source by Mirroring to Retrieve Data.
For details about the billing items involved in API operations, see Billing Items.

Parent Topic: Back to Source by Mirroring

Previous topic: Back to Source by Mirroring

Next topic: Obtaining Back-to-Source by Mirroring Rules

Feedback

Was this page helpful?

Helpful Not helpful

Provide feedback

Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.

The system is busy. Please try again later.

For any further questions, feel free to contact us through the chatbot.

Chatbot