Configuring a Back-to-Source by Mirroring Rule (SDK for Java)

Function

OBS provides back to source that pulls the requested data from the source site (origin server) if it is not found in OBS.

Restrictions

**Table 1** Restrictions
Category	Description
Bucket versions	Only buckets whose version is 3.0 or later support back to source by mirroring.
Time	It takes about five minutes to apply any changes to a back-to-source by mirroring rule.
Number of rules	A maximum of 10 back-to-source by mirroring rules can be configured for a bucket.
Functions	Anonymous users are not allowed to configure back-to-source by mirroring rules. Federated users cannot use back to source by mirroring. For more information about federated users, see What Are the Relationships Between a Huawei Cloud Account, HUAWEI ID, IAM User, and Federated User? and Introduction to Identity Providers. Parallel file systems do not support back-to-source by mirroring rules. Static website hosting does not support back to source by mirroring. Specifically, if 404 is returned when you use a static website domain name to download an object, back to source by mirroring will not be triggered. A bucket cannot mirror itself. Currently, back to source by mirroring from private buckets is only supported for some cloud vendors. In back-to-source setups, two buckets cannot act as each other's origin. For example, if bucket A is the origin for bucket B, bucket B cannot be the origin for bucket A. Transfer-Encoding: chunked cannot be used for the origin server to transmit data, or the origin pull will fail. The response to the request for downloading an object from the origin server must contain the Content-Length header to specify the size of the source object. To achieve this, in the Create/Edit Back-to-Source Rule window, choose Pass all parameters for HTTP Header Pass Rule, enable Do not pass specified parameters, and add Accept-Encoding. If both a CDN acceleration domain name and a back-to-source by mirroring rule are configured and range requests are enabled on CDN, the origin server must return at least one of the Etag and Last-Modified headers. Otherwise, the request may be interrupted. If NGINX was deployed as a reverse proxy for your origin server, turn off chunked_transfer_encoding of the NGINX. location / { chunked_transfer_encoding off; }
Permissions	To configure, obtain, or delete back-to-source by mirroring rules, you must have the Tenant Administrator permission assigned by using role/policy-based authorization of IAM. Back to source by mirroring requires that you create an IAM agency by using role/policy-based authorization to delegate OBS to pull data from its origin server. The permissions the agency should assign to OBS include obs:object:PutObject, obs:object:GetObject, obs:bucket:ListBucket, and obs:object:AbortMultipartUpload. If SSE-KMS is enabled for a bucket, the kms:cmk:get, kms:cmk:list, kms:cmk:create, kms:dek:create, kms:dek:encrypt, and kms:dek:decrypt permissions must be configured for the IAM agency for OBS by using role/policy-based authorization.
Others	Back to source by mirroring is free now. If a request matches both back-to-source by mirroring and by redirection rules, OBS first attempts to pull the data using the mirroring rule. If the data is not found at the origin server, OBS then proceeds with the redirection rule. An object cannot match two different back-to-source by mirroring rules. When you use Range-based requests to retrieve data from origin servers whose back-to-source by mirroring configurations do not comply with the standards of RFC Range Requests, the Range header must be included in HTTP requests and its value must be validated. The Range header defines the content to be retrieved, and its value cannot exceed the size of the whole file. For instance, to retrieve an object whose size is 1024 bytes using a Range request, you can specify a Range value (from 0 to 511) to retrieve the first 512 bytes. If you specify a Range value (from 0 to 2047) to retrieve 2,048 bytes, the value is invalid and the retrieval fails.

Method

obsClient.setBucketMirrorBackToSource(SetBucketMirrorBackToSourceRequest request)

Request Parameters

**Table 2** List of request parameters
Parameter	Type	Mandatory (Yes/No)	Description
request	Table 3	Yes	Explanation: Request parameters for configuring a back-to-source by mirroring rule. For details, see Table 3.

**Table 3** SetBucketMirrorBackToSourceRequest
Parameter	Type	Mandatory (Yes/No)	Description
bucketName	String	Yes	Explanation: Bucket name Restrictions: A bucket name must be unique across all accounts and regions. A bucket name: Must be 3 to 63 characters long and start with a digit or letter. Lowercase letters, digits, hyphens (-), and periods (.) are allowed. Cannot be formatted as an IP address. Cannot start or end with a hyphen (-) or period (.). Cannot contain two consecutive periods (..), for example, my..bucket. Cannot contain a period (.) and a hyphen (-) adjacent to each other, for example, my-.bucket or my.-bucket. If you repeatedly create buckets with the same name in the same region, no error will be reported and the bucket properties comply with those set in the first creation request. Default value: None
mirrorBackToSourceConfiguration	Table 4	Yes	Explanation: Back-to-source by mirroring configuration. For details, see Table 4.

**Table 4** MirrorBackToSourceConfiguration
Parameter	Type	Mandatory (Yes/No)	Description
rules	List<Table 5>	Yes	Explanation: List of back-to-source by mirroring rules Restrictions: For the same bucket, prefixes of different rules cannot be the same or contain each other. The same agency is recommended. Value range: A maximum of 10 back-to-source by mirroring rules can be created for each bucket. Therefore, the array length ranges from 1 to 10. For details, see Table 5. Default value: N/A

**Table 5** MirrorBackToSourceRule
Parameter	Type	Mandatory (Yes/No)	Description
id	String	Yes	Explanation: Rule ID. It is the unique identifier of the back-to-source by mirroring rule configured for the current bucket. Restrictions: The rule ID allows letters (a–z, A–Z), digits (0–9), hyphens (-), and underscores (_). Value range: The value can be up to 256 characters long. Default value: None
condition	Table 6	Yes	Explanation: Condition for triggering back to source Restrictions: N/A Value range: For details, see Table 6. Default value: N/A
redirect	Table 7	Yes	Explanation: Parameters for implementing the back to source function Restrictions: N/A Value range: For details, see Table 7. Default value: N/A

**Table 6** MirrorBackCondition
Parameter	Type	Mandatory (Yes/No)	Description
httpErrorCodeReturnedEquals	String	Yes	Explanation: Error code that triggers the back to source function. When this error code is returned for a download request, the back to source function is triggered. Restrictions: N/A Value range: 404: The object does not exist in the OBS bucket. Default value: 404
objectKeyPrefixEquals	String	No	Explanation: Prefix of the object name that triggers the back to source function. Back to source is performed only when the specified object name prefix is contained in the request. Restrictions: If this parameter is left blank, all objects can trigger the back to source function. For the same bucket, prefixes of different rules cannot be the same or contain each other. Value range: The value can contain 0 to 1,023 characters. Any characters are allowed. Default value: N/A

**Table 7** MirrorBackRedirect
Parameter	Type	Mandatory (Yes/No)	Description
agency	String	Yes	Explanation: Agency name. With an agency, you can grant OBS the permissions to query whether a specified object exists in a bucket and to upload objects to a bucket. Restrictions: N/A Value range: N/A Default value: N/A
publicSource	Table 8	No	Explanation: Configuration of the source site that can be publicly accessed. This parameter is mandatory when the source site is a publicly accessible resource. publicSource is the parent node of sourceEndpoint. Restrictions: N/A Value range: For details, see Table 8. Default value: N/A
vpcEndpointURN	String	No	Explanation: URN of the VPC endpoint service Restrictions: N/A Value range: The value can contain 0 to 127 characters. Default value: N/A
retryConditions	List<String>	No	Explanation: Condition for switching the source site address Restrictions: 4XX and error codes starting with 4 cannot be configured together. 5XX and error codes starting with 5 cannot be configured together. Value range: 4XX, 5XX, 400–499, and 500–599 (a maximum of 20 error codes can be configured at the same time) Default value: N/A
passQueryString	Boolean	Yes	Explanation: Whether to include the request character string Restrictions: If this parameter is set to true but the query parameter contains the signature information, the signature information is removed and the remaining parameters are passed. Value range: false: queryString in the OBS request is not passed to the source site. true: queryString in the OBS request is passed to the source site. Default value: false
mirrorFollowRedirect	Boolean	Yes	Explanation: Whether to obtain resources by redirecting the request based on the 3XX response from the source site Restrictions: N/A Value range: false: OBS transparently transmits the 3XX response and does not return resources. true: OBS returns resources by redirecting the request based on the 3XX response. Default value: false
mirrorHttpHeader	Table 10	No	Explanation: HTTP header pass rule Restrictions: N/A Value range: For details, see Table 10. Default value: N/A
replaceKeyWith	String	No	Explanation: Whether to add a prefix or suffix when downloading objects from the source site Restrictions: If both replaceKeyWith and replaceKeyPrefixWith are left blank, replaceKeyPrefixWith takes effect. The request is invalid if both the parameters are specified. Value range: prefix${key}suffix: A prefix or suffix needs to be added. ${key}: No prefix or suffix needs to be added. ${key} indicates the keyword. The total length of the prefix and suffix ranges from 0 to 1,023 characters. Default value: N/A
replaceKeyPrefixWith	String	No	Explanation: Character string used to replace objectKeyPrefixEquals. If you need to replace the current object name prefix when downloading the object from the source site, specify this parameter. Restrictions: If both replaceKeyWith and replaceKeyPrefixWith are left blank, replaceKeyPrefixWith takes effect. The request is invalid if both the parameters are specified. Value range: The value can contain 0 to 1,023 characters. Default value: N/A
redirectWithoutReferer	Boolean	No	Explanation: Whether to carry the original host as the referer header to the destination address for redirection Restrictions: N/A Value range: false: The original host is carried as the referer header to the destination address for redirection. true: The original host is not carried as the referer header to the destination address for redirection. Default value: false
mirrorAllowHttpMethod	List<String>	No	Explanation: Request method that supports transparent transmission Restrictions: N/A Value range: GET: requests to obtain objects in a bucket HEAD: requests to obtain the metadata of an object in a bucket Default value: N/A

**Table 8** MirrorBackPublicSource
Parameter	Type	Mandatory (Yes/No)	Description
sourceEndpoint	Table 9	No	Explanation: Address of the source site that can be publicly accessed Restrictions: N/A Value range: For details, see Table 9. Default value: N/A

**Table 9** MirrorBackSourceEndpoint
Parameter	Type	Mandatory (Yes/No)	Description
master	List<String>	No	Explanation: Primary source site address. If the source site is a bucket that can be accessed over HTTP, the address is the bucket domain name. If the source site is a private bucket provided by other cloud vendors, the address is the region domain name. Restrictions: The primary source site address is preferentially used during the back-to-source process. If one to five primary addresses are configured, they are accessed in polling mode. If two or more primary addresses are configured, when the first request to a primary address fails and the retry conditions are met, the request will retry another primary address. Value range: A single source site address is in the format of https://xxx.yyy.zzz or http://xxx.yyy.zzz and contains 10 to 255 characters. Default value: N/A
slave	List<String>	No	Explanation: Secondary source site address. If the source site is a bucket that can be accessed over HTTP, the address is the bucket domain name. If the source site is a private bucket provided by other cloud vendors, the address is the region domain name. Restrictions: A back-to-source request will retry the secondary source site address when the primary source site address is not available. A maximum of five secondary addresses can be configured. Value range: A single source site address is in the format of https://xxx.yyy.zzz or http://xxx.yyy.zzz and contains 10 to 255 characters. Default value: N/A

**Table 10** MirrorBackHttpHeader
Parameter	Type	Mandatory (Yes/No)	Description
passAll	Boolean	No	Explanation: Whether to transparently transmit all HTTP headers to the source site Restrictions: passAll and pass are mutually exclusive. The following HTTP header types do not support transparent transmission: Headers starting with the following prefixes: x-obs- x-amz- All standard HTTP headers, for example: Content-Length Authorization Date Value range: false: Do not transparently transmit all HTTP headers to the source site. true: Transparently transmit all HTTP headers to the source site. Default value: false
pass	List<String>	No	Explanation: List of HTTP headers to be transparently transmitted Restrictions: Only letters, digits, hyphens (-), and underscores (_) are allowed. Value range: A maximum of 10 HTTP headers can be displayed. The length of each HTTP header ranges from 1 to 63 characters. Default value: N/A
remove	List<String>	No	Explanation: List of HTTP headers that cannot be transparently transmitted Restrictions: The priority of remove is higher than that of pass and passAll. Only letters, digits, hyphens (-), and underscores (_) are allowed. Value range: A maximum of 10 HTTP headers can be displayed. The length of each HTTP header ranges from 1 to 63 characters. Default value: N/A
set	List<Table 11>	No	Explanation: List of HTTP header values to be transparently transmitted Restrictions: The priority of set is higher than that of remove, pass, and passAll. If the user-defined header contains Referer, redirectWithoutReferer must be set to true. Otherwise, Referer will be overwritten. Value range: A maximum of 10 HTTP headers can be displayed in set. Each HTTP header contains a key and a value. For details about keys and values, see Table 11. Default value: N/A

**Table 11** MirrorBackHttpHeaderSet
Parameter	Type	Mandatory (Yes/No)	Description
key	String	No	Explanation: Keyword of the HTTP header that needs to be transparently transmitted Restrictions: Each key is unique. Only letters, digits, hyphens (-), and underscores (_) are allowed. Value range: The value can contain 1 to 63 characters. Default value: N/A
value	String	No	Explanation: Value of the HTTP header that needs to be transparently transmitted Restrictions: N/A Value range: The value can contain 1 to 2,048 characters. Default value: N/A

Responses

**Table 12** Common response headers
Parameter	Type	Description
statusCode	int	Explanation: HTTP status code. Value range: A status code is a group of digits that can be 2xx (indicating successes) or 4xx or 5xx (indicating errors). It indicates the status of a response. For more information, see Status Code. Default value: None
responseHeaders	Map<String, Object>	Explanation: HTTP response header list, composed of tuples. In a tuple, the String key indicates the name of the header, and the Object value indicates the value of the header. Default value: None

Sample Code

This example configures back-to-source by mirroring rules for bucket examplebucket.

import com.obs.services.ObsClient;
import com.obs.services.exception.ObsException;
import com.obs.services.model.mirrorback.MirrorBackCondition;
import com.obs.services.model.mirrorback.MirrorBackHttpHeader;
import com.obs.services.model.mirrorback.MirrorBackHttpHeaderSet;
import com.obs.services.model.mirrorback.MirrorBackPublicSource;
import com.obs.services.model.mirrorback.MirrorBackRedirect;
import com.obs.services.model.mirrorback.MirrorBackSourceEndpoint;
import com.obs.services.model.mirrorback.MirrorBackToSourceConfiguration;
import com.obs.services.model.mirrorback.MirrorBackToSourceRule;
import com.obs.services.model.mirrorback.SetBucketMirrorBackToSourceRequest;
import java.util.Arrays;
import java.util.Collections;
public class SetBucketMirrorBackToSource {
    public static void main(String[] args) {
        // Obtain an AK/SK pair using environment variables or import the AK/SK pair in other ways. Using hard coding may result in leakage.
        // Obtain an AK/SK pair on the management console.
        String ak = System.getenv("ACCESS_KEY_ID");
        String sk = System.getenv("SECRET_ACCESS_KEY_ID");
        // (Optional) If you are using a temporary AK/SK pair and a security token to access OBS, you are not advised to use hard coding, which may result in information leakage.
        // Obtain an AK/SK pair and a security token using environment variables or import them in other ways.
        String securityToken = System.getenv("SECURITY_TOKEN");
        // Enter the endpoint corresponding to the bucket. CN North-Beijing4 is used here as an example. Replace it with the one currently in use.
        // Obtain an endpoint using environment variables or import it in other ways.
        String endPoint = System.getenv("ENDPOINT");
        ObsConfiguration obsConfiguration = new ObsConfiguration();
        obsConfiguration.setEndPoint(endPoint);
        ObsClient obsClient = new ObsClient(ak, sk, securityToken, obsConfiguration);
        try {
            String exampleBucket = "examplebucket";

            // Step 1: Specify the conditions for triggering back to source.
            MirrorBackCondition condition = new MirrorBackCondition();
            // Error code that triggers back to source. When this error code is returned for a download request, back to source is triggered.
            condition.setHttpErrorCodeReturnedEquals("404");
            // Object name prefix that triggers back to source. Back to source is performed only when the specified object name prefix is included in the request that triggers back to source.
            condition.setObjectKeyPrefixEquals("images/");

            // Step 2 (Optional): Specify a publicly accessible source address.
            MirrorBackSourceEndpoint endpoint = new MirrorBackSourceEndpoint();
            // Primary source address. If the source is an HTTP bucket that is publicly accessible, this address is the bucket domain name. If the source is a private bucket provided by other cloud vendors, the address is the region domain name.
            endpoint.setMaster(Collections.singletonList("https://source-bucket.obs.example.com"));
            // Standby source address. If the source is an HTTP bucket that is publicly accessible, this address is the bucket domain name. If the source is a private bucket provided by other cloud vendors, the address is the region domain name.
            endpoint.setSlave(Collections.singletonList("https://backup-bucket.obs.example.com"));
            MirrorBackPublicSource publicSource = new MirrorBackPublicSource();
            publicSource.setSourceEndpoint(endpoint);

            // Step 3: Specify the HTTP header pass rules.
            MirrorBackHttpHeader httpHeader = new MirrorBackHttpHeader();
            // Whether to pass all HTTP headers to the source.
            httpHeader.setPassAll(false);
            // List of HTTP headers to be passed
            httpHeader.setPass(Arrays.asList("content-type", "cache-control"));
            // List of HTTP headers that are not passed
            httpHeader.setRemove(Collections.singletonList("authorization"));
            // Set the list of HTTP header values to be passed. The set priority is higher than that of remove, pass, and passAll.
            httpHeader.setSet(Collections.singletonList(
                new MirrorBackHttpHeaderSet("x-custom-header", "custom-value")));

            // Step 4: Specify the parameters for implementing back to source.
            MirrorBackRedirect redirect = new MirrorBackRedirect();
            // Agency name. The agency can grant OBS permission to query whether a specified object exists in the bucket and to upload objects to the bucket.
            redirect.setAgency("your-agency");
            // Public source configuration. This parameter is mandatory when the source is a public resource. publicSource is the parent node of sourceEndpoint.
            redirect.setPublicSource(publicSource);
            // Whether to include the request character string.
            redirect.setPassQueryString(true);
            // Whether to obtain resources by redirecting the request based on the 3XX response from the source.
            redirect.setMirrorFollowRedirect(false);
            // Specify the HTTP header pass rules defined in step 3.
            redirect.setMirrorHttpHeader(httpHeader);
            // Replace the string of objectKeyPrefixEquals. If the prefix of the object name needs to be replaced when an object is downloaded from the source, modify this configuration.
            redirect.setReplaceKeyPrefixWith("backup/");
            // Request method that supports transparent passing.
            redirect.setMirrorAllowHttpMethod(Arrays.asList("GET", "HEAD"));

            // Step 5: Specify a single back-to-source rule.
            MirrorBackToSourceRule rule = new MirrorBackToSourceRule();
            rule.setId("mirror-rule-001");
            // Specify the conditions for triggering back to source, which are defined in step 1.
            rule.setCondition(condition);
            // Specify the parameters for implementing back to source, which are defined in step 4.
            rule.setRedirect(redirect);

            // Step 6: Specify the list of the back-to-source by mirroring rules.
            MirrorBackToSourceConfiguration policyConfig = new MirrorBackToSourceConfiguration();
            policyConfig.setRules(Collections.singletonList(rule));

            // Step 7: Send the request.
            SetBucketMirrorBackToSourceRequest request =
                new SetBucketMirrorBackToSourceRequest(exampleBucket, policyConfig);
            obsClient.setBucketMirrorBackToSource(request);
            System.out.println("SetBucketMirrorBackToSource successfully");
        } catch (ObsException e) {
            System.out.println("SetBucketMirrorBackToSource failed");
            // Request failed. Print the HTTP status code.
            System.out.println("HTTP Code:" + e.getResponseCode());
            // Request failed. Print the server-side error code.
            System.out.println("Error Code:" + e.getErrorCode());
            // Request failed. Print the error details.
            System.out.println("Error Message:" + e.getErrorMessage());
            // Request failed. Print the request ID.
            System.out.println("Request ID:" + e.getErrorRequestId());
            System.out.println("Host ID:" + e.getErrorHostId());
            e.printStackTrace();
        } catch (Exception e) {
            System.out.println("SetBucketMirrorBackToSource failed");
            // Print other error information.
            e.printStackTrace();
        }
    }
}

Helpful Links

Parent topic: Back to Source by Mirroring (SDK for Java)

Previous topic: Back to Source by Mirroring (SDK for Java)

Next topic: Obtaining Back-to-Source by Mirroring Rules (SDK for Java)