Updated on 2024-09-29 GMT+08:00

Using a Pre-Signed URL

Function

OBS allows you to construct a URL for a specific operation. In such a URL, you use Query parameters to provide authentication information including the user AK, signature, and validity period. Anyone who has the URL can perform the specified operation. After receiving a request made using such a URL, OBS treats the requester as the user who issued the URL and processes the request. For example, if you construct a pre-signed URL for downloading an object and provide it to various users, they can use the URL to download the object without authentication, but they must do so within the validity period specified by the Expires parameter. One use case for a pre-signed URL is granting temporary access to your OBS resources without providing them with the SK.

The following is an example pre-signed URL:

GET /ObjectKey?AccessKeyId=AccessKeyID&Expires=ExpiresValue&Signature=signature HTTP/1.1
Host: bucketname.obs.region.myhuaweicloud.com

The request format of downloading an object with a pre-signed URL containing a temporary AK/SK pair and security token:

GET /ObjectKey?AccessKeyId=AccessKeyID&Expires=ExpiresValue&Signature=signature&x-obs-security-token=securitytoken HTTP/1.1
Host: bucketname.obs.region.myhuaweicloud.com

Query Parameters in a Pre-signed URL

To access OBS with a URL, you must include the Query parameters shown in Table 1 in the URL.

Table 1 Query parameters

Parameter

Type

Mandatory (Yes/No)

Description

AccessKeyId

String

Yes

Explanation:

The access key ID (AK) of the URL issuer. OBS authenticates the identity based on the provided AK and, if verified, treats the requester as the issuer. For details about how to obtain an AK, see Access Keys.

Restrictions:

None

Value range:

None

Default value:

None

Expires

String

Yes

Explanation:

When a pre-signed URL expires, measured as a UNIX timestamp (how many seconds elapsed since 00:00:00 on January 1, 1970). After the specified time elapses, the URL expires.

Restrictions:

None

Value range:

Current time<Expires<20 years after the current time. Unit: second.

Default value:

None

Signature

String

Yes

Explanation:

The signature calculated based on the SK, Expires, and other parameters.

OBS provides the following signature calculation methods for a URL:

x-obs-security-token

String

No

Explanation:

This parameter must be added as a request header if a temporary AK/SK is used.

Restrictions:

None

Value range:

For details about how to obtain a temporary AK/SK pair and security token, see Obtaining a Temporary AK/SK Pair and Security Token.

Default value:

None

Using SDKs for Signing

Table 2 Using SDKs for signing

SDK

Signature Source File

Java

AbstractClient.java

Python

client.py

Go

auth.go

C

request.c

Node.Js

utils.js

Browser.Js

utils.js

PHP

SendRequestTrait.php

.NET

ObsClient.temporary.cs

Using a Signature Generator

OBS provides a graphical tool to make it easier to generate signatures. You can find the tool here. To learn how to use the tool, see Using Signature Generators.

Manually Calculating a Signature

Use this algorithm to calculate a signature:

Signature = URL-Encode( Base64( HMAC-SHA1( YourSecretAccessKeyID, UTF-8-Encoding-Of( StringToSign ) ) ) )

The process of calculating a signature is as follows:

Figure 1 Calculating a signature carried in a URL
  1. Construct a StringToSign in the format shown below. Table 3 describes the required parameters, and Example StringToSign provides some examples.

    StringToSign = 
         HTTP-Verb + "\n" +   
         Content-MD5 + "\n" +   
         Content-Type + "\n" +   
         Expires + "\n" +   
         CanonicalizedHeaders + "\n" +    
         CanonicalizedResource
    Table 3 Parameters required for constructing a StringToSign

    Parameter

    Type

    Mandatory (Yes/No)

    Description

    HTTP-Verb

    String

    Yes

    Explanation:

    The HTTP method used to make a request (also called an operation). For RESTful APIs, HTTP methods include PUT, GET, DELETE, and other operations. Select a method based on the API to be called.

    Restrictions:

    None

    Value range:

    • GET: Requests that a server return a specific resource, for example, obtaining a bucket list or downloading an object.
    • PUT: Requests that a server update a specific resource, for example, creating a bucket or uploading an object.
    • POST: Requests that a server add a resource or perform special operations such as initiating a multipart upload or assembling parts.
    • DELETE: Requests that a server delete a specific resource such as an object.
    • HEAD: Requests that a server return the description of a specific resource, for example, obtaining object metadata.
    • OPTIONS (not supported for signature generators): Requests that a server check whether the user has the permissions to perform an operation on a resource. CORS must be configured for the bucket.

    Default value:

    None

    Content-MD5

    String

    Yes

    Explanation:

    The base64-encoded 128-bit MD5 digest of the request body based on RFC 1864. This header can be used as a message integrity check to verify that the data was not tampered with in transit.

    Restrictions:

    If you want to allow users to access your OBS resources with a pre-signed URL in the browser, do not include Content-MD5, Content-Type, and CanonicalizedHeaders headers in signature calculation, because the browser cannot carry these headers. Requests with these headers will trigger a signature error on the server side.

    Value range:

    0–24 characters (0 included, 24 excluded)

    Default value:

    This parameter is left blank by default.

    Content-Type

    String

    Yes

    Explanation:

    The file type of an object—for example, text/plain—which determines what format and encoding a browser uses to read the file.

    Restrictions:

    None

    Value range:

    See What Is Content-Type (MIME)?

    Default value:

    If this header is not contained in the request, an empty string is used. For details, see Table 4. If this header is contained but not specified, its value is automatically specified based on the file name extension. If the file has no extension, application/octet-stream is used by default.

    Expires

    String

    Yes

    Explanation:

    When a pre-signed URL expires, measured as a UNIX timestamp (how many seconds elapsed since 00:00:00 on January 1, 1970). After the specified time elapses, the URL expires.

    Restrictions:

    None

    Value range:

    Current time<Expires<20 years after the current time. Unit: second.

    Default value:

    None

    CanonicalizedHeaders

    String

    Yes

    Explanation:

    Additional headers defined by OBS that include the x-obs- prefix, for example, x-obs-date, x-obs-acl, and x-obs-meta-*. For each additional header, separate its name and value by a colon (:). In x-obs-storage-class:STANDARD, for example, x-obs-storage-class is the header name, and STANDARD is the header value.

    Restrictions:

    1. Header names must be lowercase. The header value is case sensitive. An example header is x-obs-storage-class:STANDARD.
    2. A header name cannot contain non-ASCII or unrecognizable characters, which are also not recommended for header values. If such characters are necessary, they must be encoded or decoded in URL or Base64 on the client side, because the server side does not perform any decoding.
    3. A header cannot contain meaningless tabs or spaces. For example, x-obs-meta-name: name (with a meaningless space before name) must be changed to x-obs-meta-name:name.
    4. If multiple headers are involved, they must be sorted in ascending lexicographic order by header name.
    5. If a header has multiple values, these values need to be written together under their shared header name, separated by commas (,). For example, x-obs-meta-name:name1 and x-obs-meta-name:name2 must be combined into x-obs-meta-name:name1,name2.
    6. Each header requires a new line, and each line ends with a newline character (\n).

    Value range:

    Determined by the API to be called

    Default value:

    None

    CanonicalizedResource

    String

    Yes

    Explanation:

    OBS resources specified in an HTTP request. The structure is as follows:

    CanonicalizedResource = /bucket-name/object-name?sub-resource

    For example, if you want to call GetObject to obtain version xxx of object object-test stored in bucket-test and change Content-Type to text/plain, then CanonicalizedResource would be as follows:

    /bucket-test/object-test?response-content-type=text/plain&versionId=xxx
    • bucket-name:

      If the bucket does not have a custom domain name associated, use its own name.

      Otherwise, use its associated custom domain name. In /obs.ccc.com/object, for example, obs.ccc.com is a custom bucket domain name.

      If an API operation does not require a bucket to be specified, for example, listing all buckets under an account, omit both the bucket name and object name by using, for example, /.

    • object-name:

      The name of the required object. Follow the object naming rules.

    • sub-resource: Arrange multiple sub-resources in ascending lexicographic order and use ampersands (&) to separate them.

      sub-resource identifiers: CDNNotifyConfiguration, acl, append, attname, backtosource, cors, customdomain, delete, deletebucket, directcoldaccess, encryption, inventory, length, lifecycle, location, logging, metadata, modify, name, notification, partNumber, policy, position, quota, rename, replication, restore, storageClass, storagePolicy, storageinfo, tagging, torrent, truncate, uploadId, uploads, versionId, versioning, versions, website,x-obs-security-token, object-lock, retention

      Response header sub-resources: response-cache-control, response-content-disposition, response-content-encoding, response-content-language, response-content-type, response-expires

      Image processing sub-resources: x-image-process, x-image-save-bucket, x-image-save-object

    Restrictions:

    A sub-resource usually has only one value. Listing multiple values for the same resource key—for example, key=value1&key=value2—is not recommended. If you do so, only the first sub-resource value is used to calculate the signature.

    Value range:

    None

    Default value:

    If this parameter is not specified, / is used.

    If you want to open a pre-defined URL using your browser, you must not use Content-MD5, Content-Type, or CanonicalizedHeaders headers to calculate a signature, because the browser cannot carry them. Otherwise, the server that received the request will return a signature error.

  2. UTF-8 encode the result from step 1.
  3. Use the SK to calculate the HMAC-SHA1 of the result from step 2.
  4. Base64 encode the result from step 3.
  5. URL encode the result from step 4 to obtain the signature.

Code Examples

The following are some code examples for calculating a signature carried in a pre-signed URL:

Java

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
import java.io.UnsupportedEncodingException;
import java.net.URLEncoder;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Base64;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.TreeMap;
import java.util.regex.Pattern;

import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;

public class SignDemo {

    private static final String SIGN_SEP = "\n";
    private static final String OBS_PREFIX = "x-obs-";
    private static final String DEFAULT_ENCODING = "UTF-8";
    private static final List<String> SUB_RESOURCES = Collections.unmodifiableList(Arrays.asList(
            "CDNNotifyConfiguration", "acl", "append", "attname", "backtosource", "cors", "customdomain", "delete",
            "deletebucket", "directcoldaccess", "encryption", "inventory", "length", "lifecycle", "location", "logging",
            "metadata", "mirrorBackToSource", "modify", "name", "notification", "obscompresspolicy", 
            "partNumber", "policy", "position", "quota","rename", "replication", "response-cache-control", 
            "response-content-disposition","response-content-encoding", "response-content-language", "response-content-type", 
            "response-expires","restore", "storageClass", "storagePolicy", "storageinfo", "tagging", "torrent", "truncate",
            "uploadId", "uploads", "versionId", "versioning", "versions", "website", "x-image-process",
            "x-image-save-bucket", "x-image-save-object", "x-obs-security-token", "object-lock", "retention"));

    private String ak;
    private String sk;

    private boolean isBucketNameValid(String bucketName) {
        if (bucketName == null || bucketName.length() > 63 || bucketName.length() < 3) {
            return false;
        }

        if (!Pattern.matches("^[a-z0-9][a-z0-9.-]+$", bucketName)) {
            return false;
        }

        if (Pattern.matches("(\\d{1,3}\\.){3}\\d{1,3}", bucketName)) {
            return false;
        }

        String[] fragments = bucketName.split("\\.");
        for (int i = 0; i < fragments.length; i++) {
            if (Pattern.matches("^-.*", fragments[i]) || Pattern.matches(".*-$", fragments[i])
                    || Pattern.matches("^$", fragments[i])) {
                return false;
            }
        }

        return true;
    }

    // UTF-8 encode the string.
    public String encodeUrlString(String path) throws UnsupportedEncodingException {
        return URLEncoder.encode(path, DEFAULT_ENCODING)
                .replaceAll("\\+", "%20")
                .replaceAll("\\*", "%2A")
                .replaceAll("%7E", "~");
    }

    public String encodeObjectName(String objectName) throws UnsupportedEncodingException {
        StringBuilder result = new StringBuilder();
        String[] tokens = objectName.split("/");
        for (int i = 0; i < tokens.length; i++) {
            result.append(this.encodeUrlString(tokens[i]));
            if (i < tokens.length - 1) {
                result.append("/");
            }
        }
        return result.toString();
    }

    private String join(List<?> items, String delimiter) {
        StringBuilder sb = new StringBuilder();
        for (int i = 0; i < items.size(); i++) {
            String item = items.get(i).toString();
            sb.append(item);
            if (i < items.size() - 1) {
                sb.append(delimiter);
            }
        }
        return sb.toString();
    }

    private boolean isValid(String input) {
        return input != null && !input.equals("");
    }

    public String hmacSha1(String input) throws NoSuchAlgorithmException, InvalidKeyException, UnsupportedEncodingException {
        SecretKeySpec signingKey = new SecretKeySpec(this.sk.getBytes(DEFAULT_ENCODING), "HmacSHA1");
        // Obtain a Mac instance and use the getInstance method to specify the HMAC-SHA1 for the algorithm.
        Mac mac = Mac.getInstance("HmacSHA1");
        // Use the SK to initialize the Mac object.
        mac.init(signingKey);
        return Base64.getEncoder().encodeToString(mac.doFinal(input.getBytes(DEFAULT_ENCODING)));
    }
  
    // Construct a StringToSign.
    private String stringToSign(String httpMethod, Map<String, String[]> headers, Map<String, String> queries,
                                String bucketName, String objectName, long expires) throws Exception { 
        String contentMd5 = ""; 
        String contentType = "";  
        TreeMap<String, String> canonicalizedHeaders = new TreeMap<String, String>(); 
        String key; 
        List<String> temp = new ArrayList<String>(); 
        for (Map.Entry<String, String[]> entry : headers.entrySet()) { 
            key = entry.getKey(); 
            if (key == null || entry.getValue() == null || entry.getValue().length == 0) { 
                continue; 
            } 
            key = key.trim().toLowerCase(Locale.ENGLISH); 
            if (key.equals("content-md5")) { 
                contentMd5 = entry.getValue()[0]; 
                continue; 
            } 
            if (key.equals("content-type")) { 
                contentType = entry.getValue()[0]; 
                continue; 
            } 
            if (key.startsWith(OBS_PREFIX)) { 
                for (String value : entry.getValue()) { 
                    if (value != null) { 
                        temp.add(value.trim()); 
                    } 
                } 
                canonicalizedHeaders.put(key, this.join(temp, ",")); 
                temp.clear(); 
            } 
        } 

        // Construct the StringToSign by concatenating HTTP-Verb, Content-MD5, Content-Type, and Expires.
        StringBuilder stringToSign = new StringBuilder();
        stringToSign.append(httpMethod).append(SIGN_SEP)
                .append(contentMd5).append(SIGN_SEP)
                .append(contentType).append(SIGN_SEP)
                .append(expires).append(SIGN_SEP);


        // Construct the StringToSign by concatenating CanonicalizedHeaders.
        for (Map.Entry<String, String> entry : canonicalizedHeaders.entrySet()) {
            stringToSign.append(entry.getKey()).append(":").append(entry.getValue()).append(SIGN_SEP);
        }


        // Construct the StringToSign by concatenating CanonicalizedResource.
        stringToSign.append("/");
        if (this.isValid(bucketName)) {
            stringToSign.append(bucketName).append("/");
            if (this.isValid(objectName)) {
                stringToSign.append(this.encodeObjectName(objectName));
            }
        }

        TreeMap<String, String> canonicalizedResource = new TreeMap<String, String>();
        for (Map.Entry<String, String> entry : queries.entrySet()) {
            key = entry.getKey();
            if (key == null) {
                continue;
            }

            if (SUB_RESOURCES.contains(key)) {
                canonicalizedResource.put(key, entry.getValue());
            }
        }

        if (canonicalizedResource.size() > 0) {
            stringToSign.append("?");
            for (Map.Entry<String, String> entry : canonicalizedResource.entrySet()) {
                stringToSign.append(entry.getKey());
                if (this.isValid(entry.getValue())) {
                    stringToSign.append("=").append(entry.getValue());
                }
                stringToSign.append("&");
            }
            stringToSign.deleteCharAt(stringToSign.length() - 1);
        }
        // system.out.println(String.format("StringToSign:%s%s", SIGN_SEP, stringToSign.toString()));

        return stringToSign.toString();
    }

    public String querySignature(String httpMethod, Map<String, String[]> headers, Map<String, String> queries, String bucketName, String objectName, long expires) throws Exception { 
         if (!isBucketNameValid(bucketName)) { 
             throw new IllegalArgumentException("the bucketName is illegal"); 
         } 
    // Construct a StringToSign.
         String stringToSign = this.stringToSign(httpMethod, headers, queries, bucketName, objectName, expires); 
  
        // Calculate the signature.
         return this.encodeUrlString(this.hmacSha1(stringToSign)); 
     } 

    public String getURL(String endpoint, Map<String, String> queries, String bucketName, String objectName, String signature, long expires) throws UnsupportedEncodingException {
        StringBuilder URL = new StringBuilder(); 
        URL.append("https://").append(bucketName).append(".").append(endpoint).append("/").append(this.encodeObjectName(objectName)).append("?");
        String key;
        for (Map.Entry<String, String> entry : queries.entrySet()) {
            key = entry.getKey();
            if (key == null) {
                continue;
            }
            if (SUB_RESOURCES.contains(key)) {
                String value = entry.getValue();
                URL.append(key);
                if (value != null) {
                    URL.append("=").append(value).append("&");
                } else {
                    URL.append("&");
                }
            }
        }
        URL.append("AccessKeyId=").append(this.ak).append("&Expires=").append(expires).append("&Signature=").append(signature);
        return URL.toString();
    }

    public static void main(String[] args) throws Exception {
        SignDemo demo = new SignDemo();

        /* Hard-coded or plaintext AK and SK are risky. For security purposes, encrypt your AK and SK and store them in the configuration file or environment variables.
        In this example, the AK and SK are stored in environment variables for identity authentication. Before running the code in this example, configure environment variables HUAWEICLOUD_SDK_AK and HUAWEICLOUD_SDK_SK. */
	demo.ak = System.getenv("HUAWEICLOUD_SDK_AK");
	demo.sk = System.getenv("HUAWEICLOUD_SDK_SK");
        String endpoint = "<your-endpoint>";

        String bucketName = "bucket-test";
        String objectName = "hello.jpg";

        // If you use a URL to access OBS through a browser, headers cannot be included because this will lead to a signature mismatch. To use headers, process it on the client.
        Map<String, String[]> headers = new HashMap<String, String[]>();
        Map<String, String> queries = new HashMap<String, String>();

        // Use Expires to configure the signature to expire 24 hours after its creation.
        long expires = (System.currentTimeMillis() + 86400000L) / 1000;
        String signature = demo.querySignature("GET", headers, queries, bucketName, objectName, expires);
        System.out.println(signature);
        String URL = demo.getURL(endpoint, queries, bucketName, objectName, signature, expires);
        System.out.println(URL);
    }
}

Signature Algorithm in the C Programming Language

Download the sample code for calculating the signature in the C programming language.

  1. The API for calculating the signature is contained in the sign.h header file.
  2. The sample code for calculating the signature is contained in the main.c header file.

Using a Pre-signed URL to Generate a Pre-defined Access URL

Combine the calculated signature with the host prefix to generate a pre-defined URL. Below is an example URL. Users obtaining this URL can enter it in the browser to download object objectkey from bucket examplebucket. 1532779451 (Sat Jul 28 20:04:11 CST 2024) indicates the expiration time of this URL.

http(s)://examplebucket.obs.region.myhuaweicloud.com/objectkey?AccessKeyId=AccessKeyID&Expires=1532779451&Signature=0Akylf43Bm3mD1bh2rM3dmVp1Bo%3D

In a Linux system, if you want to use curl to access the URL, escape the ampersand (&) with a backslash (\). The following example downloads object objectkey to file output:

curl  http(s)://examplebucket.obs.region.myhuaweicloud.com/objectkey?AccessKeyId=AccessKeyID\&Expires=1532779451\&Signature=0Akylf43Bm3mD1bh2rM3dmVp1Bo%3D  -X GET -o output

Addressing a Signature Mismatch

During an OBS API call, if the following error is reported:

Status code: 403 Forbidden

Error code: SignatureDoesNotMatch

Error message: The request signature we calculated does not match the signature you provided. Check your key and signing method.

Address the problem by referring to Why Don't the Signatures Match?