Using a Pre-Signed URL
Function
OBS allows you to construct a URL for a specific operation. In such a URL, you use Query parameters to provide authentication information including the user AK, signature, and validity period. Anyone who has the URL can perform the specified operation. After receiving a request made using such a URL, OBS treats the requester as the user who issued the URL and processes the request. For example, if you construct a pre-signed URL for downloading an object and provide it to various users, they can use the URL to download the object without authentication, but they must do so within the validity period specified by the Expires parameter. One use case for a pre-signed URL is granting temporary access to your OBS resources without providing them with the SK.
The following is an example pre-signed URL:
GET /ObjectKey?AccessKeyId=AccessKeyID&Expires=ExpiresValue&Signature=signature HTTP/1.1 Host: bucketname.obs.region.myhuaweicloud.com
The request format of downloading an object with a pre-signed URL containing a temporary AK/SK pair and security token:
GET /ObjectKey?AccessKeyId=AccessKeyID&Expires=ExpiresValue&Signature=signature&x-obs-security-token=securitytoken HTTP/1.1 Host: bucketname.obs.region.myhuaweicloud.com
Query Parameters in a Pre-signed URL
To access OBS with a URL, you must include the Query parameters shown in Table 1 in the URL.
Parameter |
Type |
Mandatory (Yes/No) |
Description |
---|---|---|---|
AccessKeyId |
String |
Yes |
Explanation: The access key ID (AK) of the URL issuer. OBS authenticates the identity based on the provided AK and, if verified, treats the requester as the issuer. For details about how to obtain an AK, see Access Keys. Restrictions: None Value range: None Default value: None |
Expires |
String |
Yes |
Explanation: When a pre-signed URL expires, measured as a UNIX timestamp (how many seconds elapsed since 00:00:00 on January 1, 1970). After the specified time elapses, the URL expires. Restrictions: None Value range: Current time<Expires<20 years after the current time. Unit: second. Default value: None |
Signature |
String |
Yes |
Explanation: The signature calculated based on the SK, Expires, and other parameters. OBS provides the following signature calculation methods for a URL: |
x-obs-security-token |
String |
No |
Explanation: This parameter must be added as a request header if a temporary AK/SK is used. Restrictions: None Value range: For details about how to obtain a temporary AK/SK pair and security token, see Obtaining a Temporary AK/SK Pair and Security Token. Default value: None |
Using SDKs for Signing
SDK |
Signature Source File |
---|---|
Java |
|
Python |
|
Go |
|
C |
|
Node.Js |
|
Browser.Js |
|
PHP |
|
.NET |
Using a Signature Generator
OBS provides a graphical tool to make it easier to generate signatures. You can find the tool here. To learn how to use the tool, see Using Signature Generators.
Manually Calculating a Signature
Use this algorithm to calculate a signature:
Signature = URL-Encode( Base64( HMAC-SHA1( YourSecretAccessKeyID, UTF-8-Encoding-Of( StringToSign ) ) ) )
The process of calculating a signature is as follows:
- Construct a StringToSign in the format shown below. Table 3 describes the required parameters, and Example StringToSign provides some examples.
StringToSign = HTTP-Verb + "\n" + Content-MD5 + "\n" + Content-Type + "\n" + Expires + "\n" + CanonicalizedHeaders + "\n" + CanonicalizedResource
Table 3 Parameters required for constructing a StringToSign Parameter
Type
Mandatory (Yes/No)
Description
HTTP-Verb
String
Yes
Explanation:
The HTTP method used to make a request (also called an operation). For RESTful APIs, HTTP methods include PUT, GET, DELETE, and other operations. Select a method based on the API to be called.
Restrictions:
None
Value range:
- GET: Requests that a server return a specific resource, for example, obtaining a bucket list or downloading an object.
- PUT: Requests that a server update a specific resource, for example, creating a bucket or uploading an object.
- POST: Requests that a server add a resource or perform special operations such as initiating a multipart upload or assembling parts.
- DELETE: Requests that a server delete a specific resource such as an object.
- HEAD: Requests that a server return the description of a specific resource, for example, obtaining object metadata.
- OPTIONS (not supported for signature generators): Requests that a server check whether the user has the permissions to perform an operation on a resource. CORS must be configured for the bucket.
Default value:
None
Content-MD5
String
Yes
Explanation:
The base64-encoded 128-bit MD5 digest of the request body based on RFC 1864. This header can be used as a message integrity check to verify that the data was not tampered with in transit.
Restrictions:
If you want to allow users to access your OBS resources with a pre-signed URL in the browser, do not include Content-MD5, Content-Type, and CanonicalizedHeaders headers in signature calculation, because the browser cannot carry these headers. Requests with these headers will trigger a signature error on the server side.
Value range:
0–24 characters (0 included, 24 excluded)
Default value:
This parameter is left blank by default.
Content-Type
String
Yes
Explanation:
The file type of an object—for example, text/plain—which determines what format and encoding a browser uses to read the file.
Restrictions:
None
Value range:
See What Is Content-Type (MIME)?
Default value:
If this header is not contained in the request, an empty string is used. For details, see Table 4. If this header is contained but not specified, its value is automatically specified based on the file name extension. If the file has no extension, application/octet-stream is used by default.
Expires
String
Yes
Explanation:
When a pre-signed URL expires, measured as a UNIX timestamp (how many seconds elapsed since 00:00:00 on January 1, 1970). After the specified time elapses, the URL expires.
Restrictions:
None
Value range:
Current time<Expires<20 years after the current time. Unit: second.
Default value:
None
CanonicalizedHeaders
String
Yes
Explanation:
Additional headers defined by OBS that include the x-obs- prefix, for example, x-obs-date, x-obs-acl, and x-obs-meta-*. For each additional header, separate its name and value by a colon (:). In x-obs-storage-class:STANDARD, for example, x-obs-storage-class is the header name, and STANDARD is the header value.
Restrictions:
- Header names must be lowercase. The header value is case sensitive. An example header is x-obs-storage-class:STANDARD.
- A header name cannot contain non-ASCII or unrecognizable characters, which are also not recommended for header values. If such characters are necessary, they must be encoded or decoded in URL or Base64 on the client side, because the server side does not perform any decoding.
- A header cannot contain meaningless tabs or spaces. For example, x-obs-meta-name: name (with a meaningless space before name) must be changed to x-obs-meta-name:name.
- If multiple headers are involved, they must be sorted in ascending lexicographic order by header name.
- If a header has multiple values, these values need to be written together under their shared header name, separated by commas (,). For example, x-obs-meta-name:name1 and x-obs-meta-name:name2 must be combined into x-obs-meta-name:name1,name2.
- Each header requires a new line, and each line ends with a newline character (\n).
Value range:
Determined by the API to be called
Default value:
None
CanonicalizedResource
String
Yes
Explanation:
OBS resources specified in an HTTP request. The structure is as follows:
CanonicalizedResource = /bucket-name/object-name?sub-resource
For example, if you want to call GetObject to obtain version xxx of object object-test stored in bucket-test and change Content-Type to text/plain, then CanonicalizedResource would be as follows:
/bucket-test/object-test?response-content-type=text/plain&versionId=xxx
- bucket-name:
If the bucket does not have a custom domain name associated, use its own name.
Otherwise, use its associated custom domain name. In /obs.ccc.com/object, for example, obs.ccc.com is a custom bucket domain name.
If an API operation does not require a bucket to be specified, for example, listing all buckets under an account, omit both the bucket name and object name by using, for example, /.
- object-name:
The name of the required object. Follow the object naming rules.
- sub-resource: Arrange multiple sub-resources in ascending lexicographic order and use ampersands (&) to separate them.
sub-resource identifiers: CDNNotifyConfiguration, acl, append, attname, backtosource, cors, customdomain, delete, deletebucket, directcoldaccess, encryption, inventory, length, lifecycle, location, logging, metadata, modify, name, notification, partNumber, policy, position, quota, rename, replication, restore, storageClass, storagePolicy, storageinfo, tagging, torrent, truncate, uploadId, uploads, versionId, versioning, versions, website,x-obs-security-token, object-lock, retention
Response header sub-resources: response-cache-control, response-content-disposition, response-content-encoding, response-content-language, response-content-type, response-expires
Image processing sub-resources: x-image-process, x-image-save-bucket, x-image-save-object
Restrictions:
A sub-resource usually has only one value. Listing multiple values for the same resource key—for example, key=value1&key=value2—is not recommended. If you do so, only the first sub-resource value is used to calculate the signature.
Value range:
None
Default value:
If this parameter is not specified, / is used.
If you want to open a pre-defined URL using your browser, you must not use Content-MD5, Content-Type, or CanonicalizedHeaders headers to calculate a signature, because the browser cannot carry them. Otherwise, the server that received the request will return a signature error.
- UTF-8 encode the result from step 1.
- Use the SK to calculate the HMAC-SHA1 of the result from step 2.
- Base64 encode the result from step 3.
- URL encode the result from step 4 to obtain the signature.
Code Examples
The following are some code examples for calculating a signature carried in a pre-signed URL:
Java
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 |
import java.io.UnsupportedEncodingException; import java.net.URLEncoder; import java.security.InvalidKeyException; import java.security.NoSuchAlgorithmException; import java.util.ArrayList; import java.util.Arrays; import java.util.Base64; import java.util.Collections; import java.util.HashMap; import java.util.List; import java.util.Locale; import java.util.Map; import java.util.TreeMap; import java.util.regex.Pattern; import javax.crypto.Mac; import javax.crypto.spec.SecretKeySpec; public class SignDemo { private static final String SIGN_SEP = "\n"; private static final String OBS_PREFIX = "x-obs-"; private static final String DEFAULT_ENCODING = "UTF-8"; private static final List<String> SUB_RESOURCES = Collections.unmodifiableList(Arrays.asList( "CDNNotifyConfiguration", "acl", "append", "attname", "backtosource", "cors", "customdomain", "delete", "deletebucket", "directcoldaccess", "encryption", "inventory", "length", "lifecycle", "location", "logging", "metadata", "mirrorBackToSource", "modify", "name", "notification", "obscompresspolicy", "partNumber", "policy", "position", "quota","rename", "replication", "response-cache-control", "response-content-disposition","response-content-encoding", "response-content-language", "response-content-type", "response-expires","restore", "storageClass", "storagePolicy", "storageinfo", "tagging", "torrent", "truncate", "uploadId", "uploads", "versionId", "versioning", "versions", "website", "x-image-process", "x-image-save-bucket", "x-image-save-object", "x-obs-security-token", "object-lock", "retention")); private String ak; private String sk; private boolean isBucketNameValid(String bucketName) { if (bucketName == null || bucketName.length() > 63 || bucketName.length() < 3) { return false; } if (!Pattern.matches("^[a-z0-9][a-z0-9.-]+$", bucketName)) { return false; } if (Pattern.matches("(\\d{1,3}\\.){3}\\d{1,3}", bucketName)) { return false; } String[] fragments = bucketName.split("\\."); for (int i = 0; i < fragments.length; i++) { if (Pattern.matches("^-.*", fragments[i]) || Pattern.matches(".*-$", fragments[i]) || Pattern.matches("^$", fragments[i])) { return false; } } return true; } // UTF-8 encode the string. public String encodeUrlString(String path) throws UnsupportedEncodingException { return URLEncoder.encode(path, DEFAULT_ENCODING) .replaceAll("\\+", "%20") .replaceAll("\\*", "%2A") .replaceAll("%7E", "~"); } public String encodeObjectName(String objectName) throws UnsupportedEncodingException { StringBuilder result = new StringBuilder(); String[] tokens = objectName.split("/"); for (int i = 0; i < tokens.length; i++) { result.append(this.encodeUrlString(tokens[i])); if (i < tokens.length - 1) { result.append("/"); } } return result.toString(); } private String join(List<?> items, String delimiter) { StringBuilder sb = new StringBuilder(); for (int i = 0; i < items.size(); i++) { String item = items.get(i).toString(); sb.append(item); if (i < items.size() - 1) { sb.append(delimiter); } } return sb.toString(); } private boolean isValid(String input) { return input != null && !input.equals(""); } public String hmacSha1(String input) throws NoSuchAlgorithmException, InvalidKeyException, UnsupportedEncodingException { SecretKeySpec signingKey = new SecretKeySpec(this.sk.getBytes(DEFAULT_ENCODING), "HmacSHA1"); // Obtain a Mac instance and use the getInstance method to specify the HMAC-SHA1 for the algorithm. Mac mac = Mac.getInstance("HmacSHA1"); // Use the SK to initialize the Mac object. mac.init(signingKey); return Base64.getEncoder().encodeToString(mac.doFinal(input.getBytes(DEFAULT_ENCODING))); } // Construct a StringToSign. private String stringToSign(String httpMethod, Map<String, String[]> headers, Map<String, String> queries, String bucketName, String objectName, long expires) throws Exception { String contentMd5 = ""; String contentType = ""; TreeMap<String, String> canonicalizedHeaders = new TreeMap<String, String>(); String key; List<String> temp = new ArrayList<String>(); for (Map.Entry<String, String[]> entry : headers.entrySet()) { key = entry.getKey(); if (key == null || entry.getValue() == null || entry.getValue().length == 0) { continue; } key = key.trim().toLowerCase(Locale.ENGLISH); if (key.equals("content-md5")) { contentMd5 = entry.getValue()[0]; continue; } if (key.equals("content-type")) { contentType = entry.getValue()[0]; continue; } if (key.startsWith(OBS_PREFIX)) { for (String value : entry.getValue()) { if (value != null) { temp.add(value.trim()); } } canonicalizedHeaders.put(key, this.join(temp, ",")); temp.clear(); } } // Construct the StringToSign by concatenating HTTP-Verb, Content-MD5, Content-Type, and Expires. StringBuilder stringToSign = new StringBuilder(); stringToSign.append(httpMethod).append(SIGN_SEP) .append(contentMd5).append(SIGN_SEP) .append(contentType).append(SIGN_SEP) .append(expires).append(SIGN_SEP); // Construct the StringToSign by concatenating CanonicalizedHeaders. for (Map.Entry<String, String> entry : canonicalizedHeaders.entrySet()) { stringToSign.append(entry.getKey()).append(":").append(entry.getValue()).append(SIGN_SEP); } // Construct the StringToSign by concatenating CanonicalizedResource. stringToSign.append("/"); if (this.isValid(bucketName)) { stringToSign.append(bucketName).append("/"); if (this.isValid(objectName)) { stringToSign.append(this.encodeObjectName(objectName)); } } TreeMap<String, String> canonicalizedResource = new TreeMap<String, String>(); for (Map.Entry<String, String> entry : queries.entrySet()) { key = entry.getKey(); if (key == null) { continue; } if (SUB_RESOURCES.contains(key)) { canonicalizedResource.put(key, entry.getValue()); } } if (canonicalizedResource.size() > 0) { stringToSign.append("?"); for (Map.Entry<String, String> entry : canonicalizedResource.entrySet()) { stringToSign.append(entry.getKey()); if (this.isValid(entry.getValue())) { stringToSign.append("=").append(entry.getValue()); } stringToSign.append("&"); } stringToSign.deleteCharAt(stringToSign.length() - 1); } // system.out.println(String.format("StringToSign:%s%s", SIGN_SEP, stringToSign.toString())); return stringToSign.toString(); } public String querySignature(String httpMethod, Map<String, String[]> headers, Map<String, String> queries, String bucketName, String objectName, long expires) throws Exception { if (!isBucketNameValid(bucketName)) { throw new IllegalArgumentException("the bucketName is illegal"); } // Construct a StringToSign. String stringToSign = this.stringToSign(httpMethod, headers, queries, bucketName, objectName, expires); // Calculate the signature. return this.encodeUrlString(this.hmacSha1(stringToSign)); } public String getURL(String endpoint, Map<String, String> queries, String bucketName, String objectName, String signature, long expires) throws UnsupportedEncodingException { StringBuilder URL = new StringBuilder(); URL.append("https://").append(bucketName).append(".").append(endpoint).append("/").append(this.encodeObjectName(objectName)).append("?"); String key; for (Map.Entry<String, String> entry : queries.entrySet()) { key = entry.getKey(); if (key == null) { continue; } if (SUB_RESOURCES.contains(key)) { String value = entry.getValue(); URL.append(key); if (value != null) { URL.append("=").append(value).append("&"); } else { URL.append("&"); } } } URL.append("AccessKeyId=").append(this.ak).append("&Expires=").append(expires).append("&Signature=").append(signature); return URL.toString(); } public static void main(String[] args) throws Exception { SignDemo demo = new SignDemo(); /* Hard-coded or plaintext AK and SK are risky. For security purposes, encrypt your AK and SK and store them in the configuration file or environment variables. In this example, the AK and SK are stored in environment variables for identity authentication. Before running the code in this example, configure environment variables HUAWEICLOUD_SDK_AK and HUAWEICLOUD_SDK_SK. */ demo.ak = System.getenv("HUAWEICLOUD_SDK_AK"); demo.sk = System.getenv("HUAWEICLOUD_SDK_SK"); String endpoint = "<your-endpoint>"; String bucketName = "bucket-test"; String objectName = "hello.jpg"; // If you use a URL to access OBS through a browser, headers cannot be included because this will lead to a signature mismatch. To use headers, process it on the client. Map<String, String[]> headers = new HashMap<String, String[]>(); Map<String, String> queries = new HashMap<String, String>(); // Use Expires to configure the signature to expire 24 hours after its creation. long expires = (System.currentTimeMillis() + 86400000L) / 1000; String signature = demo.querySignature("GET", headers, queries, bucketName, objectName, expires); System.out.println(signature); String URL = demo.getURL(endpoint, queries, bucketName, objectName, signature, expires); System.out.println(URL); } } |
Signature Algorithm in the C Programming Language
Download the sample code for calculating the signature in the C programming language.
- The API for calculating the signature is contained in the sign.h header file.
- The sample code for calculating the signature is contained in the main.c header file.
Using a Pre-signed URL to Generate a Pre-defined Access URL
Combine the calculated signature with the host prefix to generate a pre-defined URL. Below is an example URL. Users obtaining this URL can enter it in the browser to download object objectkey from bucket examplebucket. 1532779451 (Sat Jul 28 20:04:11 CST 2024) indicates the expiration time of this URL.
http(s)://examplebucket.obs.region.myhuaweicloud.com/objectkey?AccessKeyId=AccessKeyID&Expires=1532779451&Signature=0Akylf43Bm3mD1bh2rM3dmVp1Bo%3D
In a Linux system, if you want to use curl to access the URL, escape the ampersand (&) with a backslash (\). The following example downloads object objectkey to file output:
curl http(s)://examplebucket.obs.region.myhuaweicloud.com/objectkey?AccessKeyId=AccessKeyID\&Expires=1532779451\&Signature=0Akylf43Bm3mD1bh2rM3dmVp1Bo%3D -X GET -o output
Addressing a Signature Mismatch
During an OBS API call, if the following error is reported:
Status code: 403 Forbidden
Error code: SignatureDoesNotMatch
Error message: The request signature we calculated does not match the signature you provided. Check your key and signing method.
Address the problem by referring to Why Don't the Signatures Match?
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot