Help Center/ Object Storage Service/ SDK Reference/ Python/ APIs Related to Multipart Upload (SDK for Python)/ Initiating a Multipart Upload (SDK for Python)
Updated on 2024-09-05 GMT+08:00
View PDF
Share

Initiating a Multipart Upload (SDK for Python)

Function

This API initiates a multipart upload and returns a globally unique upload ID. You can use the returned upload ID when you later upload, assemble, or list parts. A single object can be involved in multiple multipart uploads. Each multipart upload initiation request can contain additional headers such as acl, metadata, contentType, and encoding_type. These headers are recorded in the multipart upload metadata.

Restrictions

  • To initiate a multipart upload, you must be the bucket owner or have the required permission (obs:object:PutObject in IAM or PutObject in a bucket policy). For details, see Introduction to OBS Access Control, IAM Custom Policies, and Configuring an Object Policy.
  • The mapping between OBS regions and endpoints must comply with what is listed in Regions and Endpoints.
  • After initiating a multipart upload and uploading one or more parts, you must assemble the parts or abort the multipart upload. Only after that can OBS stops billing you for storing the uploaded parts.

Method

ObsClient.initiateMultipartUpload(bucketName, objectKey, acl, storageClass, metadata, websiteRedirectLocation, contentType, sseHeader, expires, extensionGrants)

Request Parameters

Table 1 List of request parameters

Parameter

Type

Mandatory (Yes/No)

Description

bucketName

str

Yes

Explanation:

Bucket name

Restrictions:

  • A bucket name must be unique across all accounts and regions.
  • A bucket name:
    • Must be 3 to 63 characters long and start with a digit or letter. Lowercase letters, digits, hyphens (-), and periods (.) are allowed.
    • Cannot be formatted as an IP address.
    • Cannot start or end with a hyphen (-) or period (.).
    • Cannot contain two consecutive periods (..), for example, my..bucket.
    • Cannot contain periods (.) and hyphens (-) adjacent to each other, for example, my-.bucket or my.-bucket.
  • If you repeatedly create buckets of the same name in the same region, no error will be reported and the bucket attributes comply with those set in the first creation request.

Default value:

None

objectKey

str

Yes

Explanation:

Object name. An object is uniquely identified by an object name in a bucket. An object name is a complete path that does not contain the bucket name.

For example, if the address for accessing the object is examplebucket.obs.ap-southeast-1.myhuaweicloud.com/folder/test.txt, the object name is folder/test.txt.

Value range:

The value must contain 1 to 1,024 characters.

Default value:

None

acl

str

No

Explanation:

Access control list (ACL) that can be pre-defined when a bucket is created. For details about ACLs, see ACLs.

Value range:

See Table 2.

Default value:

PRIVATE

storageClass

str

No

Explanation:

Object storage class. If this parameter is not set, the object inherits the storage class of its bucket.

Value range:

See Table 3.

Default value:

STANDARD

metadata

dict

No

Explanation:

Custom metadata of the object to be uploaded. You can add a header starting with x-obs-meta- in the request to define metadata. The custom metadata will be returned in the response when you retrieve the object or query the object metadata.

Restrictions:

  • The custom metadata cannot exceed 8 KB. To measure the custom metadata, sum the number of bytes in the UTF-8 encoding of each key and value.
  • The custom metadata keys are case insensitive, but are stored in lowercase in OBS. The key values are case sensitive.
  • Both custom metadata keys and their values must conform to US-ASCII standards. If non-ASCII or unrecognizable characters are required, they must be encoded or decoded in URL or Base64 on the client, because the server does not perform such operations.

Default value:

None

websiteRedirectLocation

str

No

Explanation:

If the bucket is configured with website hosting, the request for obtaining the object can be redirected to another object in the bucket or an external URL.

The request is redirected to object anotherPage.html in the same bucket:

WebsiteRedirectLocation:/anotherPage.html

The request is redirected to an external URL http://www.example.com/:

WebsiteRedirectLocation:http://www.example.com/

OBS obtains the specified value from the header and stores it in the object metadata WebsiteRedirectLocation.

Restrictions:

  • The value must start with a slash (/), http://, or https:// and cannot exceed 2 KB.
  • OBS only supports redirection for objects in the root directory of a bucket.

Default value:

None

contentType

str

No

Explanation:

MIME type of the file to be uploaded. MIME type is a standard way of describing a data type and is used by the browser to decide how to display data.

Value range:

See What Is Content-Type (MIME)? (Python SDK)

Default value:

If you do not specify this parameter when uploading an object, the SDK determines the object type based on the suffix of the specified object name and automatically assigns a value to this parameter.

sseHeader

SseCHeader or SseKmsHeader

No

Explanation:

Server-side encryption header. If SSE-C is used, see Table 4. If SSE-KMS is used, see Table 5.

Default value:

None

expires

int

No

Explanation:

Expiration time of the object (calculated from the latest modification time of the object). Expired objects are automatically deleted.

Restrictions:

This parameter can be configured only when uploading the object. It cannot be modified by calling a metadata modification API.

Value range:

An integer greater than or equal to 0, in days

Default value:

None

extensionGrants

list of ExtensionGrant

No

Explanation:

List of the extended permissions for the object. For details, see Table 6.

Default value:

None

encoding_type

str

No

Explanation:

Encoding type for Key in the response. If Key in the response contains control characters that are not supported by the XML 1.0 standard, you can specify this parameter to encode Key.

Value range:

url

Default value:

None. If you leave this parameter blank, encoding is not applied to Key.
Table 2 HeadPermission

Constant

Default Value

Description

HeadPermission.PRIVATE

private

Private read/write

A bucket or object can only be accessed by its owner.

HeadPermission.PUBLIC_READ

public-read

Public read and private write

If this permission is granted on a bucket, anyone can read the object list, multipart uploads, metadata, and object versions in the bucket.

If it is granted on an object, anyone can read the content and metadata of the object.

HeadPermission.PUBLIC_READ_WRITE

public-read-write

Public read/write

If this permission is granted on a bucket, anyone can read the object list, multipart tasks, metadata, and object versions in the bucket, and can upload or delete objects, initiate multipart upload tasks, upload parts, assemble parts, copy parts, and abort multipart upload tasks.

If it is granted on an object, anyone can read the content and metadata of the object.

HeadPermission.PUBLIC_READ_DELIVERED

public-read-delivered

Public read on a bucket as well as objects in the bucket

If this permission is granted on a bucket, anyone can read the object list, multipart tasks, metadata, and object versions, and read the content and metadata of objects in the bucket.

NOTE:

PUBLIC_READ_DELIVERED cannot be applied to objects.

HeadPermission.PUBLIC_READ_WRITE_DELIVERED

public-read-write-delivered

Public read/write on a bucket as well as objects in the bucket

If this permission is granted on a bucket, anyone can read the object list, multipart uploads, metadata, and object versions in the bucket, and can upload or delete objects, initiate multipart upload tasks, upload parts, assemble parts, copy parts, and abort multipart uploads. They can also read the content and metadata of objects in the bucket.

NOTE:

PUBLIC_READ_WRITE_DELIVERED cannot be applied to objects.

HeadPermission.BUCKET_OWNER_FULL_CONTROL

public-read-write-delivered

If this permission is granted on an object, only the bucket and object owners have the full control over the object. By default, if you upload an object to a bucket of any other user, the bucket owner does not have the permissions on your object. After you grant this policy to the bucket owner, the bucket owner can have full control over your object.
Table 3 StorageClass

Parameter

Type

Description

STANDARD

Standard storage class

Explanation:

Features low access latency and high throughput and is used for storing massive, frequently accessed (multiple times a month) or small objects (< 1 MB) requiring quick response.

WARM

Infrequent Access storage class

Explanation:

Used for storing data that is semi-frequently accessed (fewer than 12 times a year) but is instantly available when needed.

COLD

Archive storage class

Explanation:

Used for storing rarely accessed (once a year) data.
Table 4 SseCHeader

Parameter

Type

Mandatory (Yes/No)

Description

encryption

str

Yes

Explanation:

SSE-C used for encrypting objects

Value range:

AES256

Default value:

None

key

str

Yes

Explanation:

Key used in SSE-C encryption. It corresponds to the encryption method. For example, if encryption is set to AES256, the key is calculated using the AES-256 algorithm.

Value range:

The value must contain 32 characters.

Default value:

None
Table 5 SseKmsHeader

Parameter

Type

Mandatory (Yes/No)

Description

encryption

str

Yes

Explanation:

SSE-KMS used for encrypting objects

Value range:

kms

Default value:

None

key

str

No

Explanation:

Master key used in SSE-KMS

Value range:

The following two formats are supported:

  • regionID:domainID:key/key_id
  • key_id

In the preceding formats:

Default value:

  • If this parameter is not specified, the default master key will be used.
  • If there is no such a default master key, OBS will create one and use it by default.
Table 6 ExtensionGrant

Parameter

Type

Mandatory (Yes/No)

Description

granteeId

str

No

Explanation:

Account (domain) ID of the grantee

Value range:

To obtain the account ID, see How Do I Get My Account ID and IAM User ID? (SDK for Python)

Default value:

None

permission

str

No

Explanation:

Granted permissions. For details, see Table 7.

Default value:

None
Table 7 Permission

Constant

Description

READ

Read permission

A grantee with this permission for a bucket can obtain the list of objects, multipart uploads, bucket metadata, and object versions in the bucket.

A grantee with this permission for an object can obtain the object content and metadata.

WRITE

Write permission

A grantee with this permission for a bucket can upload, overwrite, and delete any object or part in the bucket.

Such permission for an object is not applicable.

READ_ACP

Permission to read ACL configurations

A grantee with this permission can obtain the ACL of a bucket or object.

A bucket or object owner has this permission for the bucket or object permanently.

WRITE_ACP

Permission to modify ACL configurations

A grantee with this permission can update the ACL of a bucket or object.

A bucket or object owner has this permission for the bucket or object permanently.

A grantee with this permission can modify the access control policy and thus the grantee obtains full access permissions.

FULL_CONTROL

Full control access, including read and write permissions for a bucket and its ACL, or for an object and its ACL.

A grantee with this permission for a bucket has READ, WRITE, READ_ACP, and WRITE_ACP permissions for the bucket.

A grantee with this permission for an object has READ, READ_ACP, and WRITE_ACP permissions for the object.

Responses

Table 8 List of returned results

Type

Description

GetResult

Explanation:

SDK common results
Table 9 GetResult

Parameter

Type

Description

status

int

Explanation:

HTTP status code

Value range:

A status code is a group of digits ranging from 2xx (indicating successes) to 4xx or 5xx (indicating errors). It indicates the status of a response. For more information, see Status Code.

Default value:

None

reason

str

Explanation:

Reason description.

Default value:

None

errorCode

str

Explanation:

Error code returned by the OBS server. If the value of status is less than 300, this parameter is left blank.

Default value:

None

errorMessage

str

Explanation:

Error message returned by the OBS server. If the value of status is less than 300, this parameter is left blank.

Default value:

None

requestId

str

Explanation:

Request ID returned by the OBS server

Default value:

None

indicator

str

Explanation:

Error indicator returned by the OBS server.

Default value:

None

hostId

str

Explanation:

Requested server ID. If the value of status is less than 300, this parameter is left blank.

Default value:

None

resource

str

Explanation:

Error source (a bucket or an object). If the value of status is less than 300, this parameter is left blank.

Default value:

None

header

list

Explanation:

Response header list, composed of tuples. Each tuple consists of two elements, respectively corresponding to the key and value of a response header.

Default value:

None

body

object

Explanation:

Result content returned after the operation is successful. If the value of status is larger than 300, this parameter is left blank. The value varies with the API being called. For details, see Bucket-Related APIs (SDK for Python) and Object-Related APIs (SDK for Python).

Default value:

None
Table 10 GetResult.body

GetResult.body Type

Description

InitiateMultipartUploadResponse

Explanation:

Response to the request for initiating a multipart upload. For details, see Table 11.
Table 11 InitiateMultipartUploadResponse

Parameter

Type

Description

bucketName

str

Explanation:

Name of the bucket involved in the multipart upload

Restrictions:

  • A bucket name must be unique across all accounts and regions.
  • A bucket name:
    • Must be 3 to 63 characters long and start with a digit or letter. Lowercase letters, digits, hyphens (-), and periods (.) are allowed.
    • Cannot be formatted as an IP address.
    • Cannot start or end with a hyphen (-) or period (.).
    • Cannot contain two consecutive periods (..), for example, my..bucket.
    • Cannot contain periods (.) and hyphens (-) adjacent to each other, for example, my-.bucket or my.-bucket.
  • If you repeatedly create buckets of the same name in the same region, no error will be reported and the bucket attributes comply with those set in the first creation request.

Default value:

None

objectKey

str

Explanation:

Name of the object to be uploaded. An object is uniquely identified by an object name in a bucket. An object name is a complete path that does not contain the bucket name.

For example, if the address for accessing the object is examplebucket.obs.ap-southeast-1.myhuaweicloud.com/folder/test.txt, the object name is folder/test.txt.

Value range:

The value must contain 1 to 1,024 characters.

Default value:

None

uploadId

str

Explanation:

Multipart upload ID, for example, 000001648453845DBB78F2340DD460D8

Value range:

The value must contain 1 to 32 characters.

Default value:

None

sseKms

str

Explanation:

SSE-KMS is used for encrypting objects on the server side.

Value range:

kms

Default value:

None

sseKmsKey

str

Explanation:

ID of the KMS master key when SSE-KMS is used

Value range:

Valid value formats are as follows:

  1. regionID:domainID:key/key_id
  2. key_id

In the preceding formats:

Default value:

  • If this parameter is not specified, the default master key will be used.
  • If there is no such a default master key, OBS will create one and use it by default.

sseC

str

Explanation:

Algorithm used to encrypt and decrypt objects with SSE-C

Value range:

AES256

Default value:

None

sseCKeyMd5

str

Explanation:

MD5 value of the key for encrypting objects when SSE-C is used. This value is used to check whether any error occurs during the transmission of the key.

Restrictions:

The value is encrypted by MD5 and then encoded by Base64, for example, 4XvB3tbNTN+tIEVa0/fGaQ==.

Default value:

None

encoding_type

str

Explanation:

Encoding type for Key in the response. If Key in the response contains control characters that are not supported by the XML 1.0 standard, you can specify this parameter to encode Key.

Value range:

url

Default value:

None. If you leave this parameter blank, encoding is not applied to Key.

Code Examples

This example initiates a multipart upload for bucket examplebucket.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
 
from obs import ObsClient
import os
import traceback

# Obtain an AK and SK pair using environment variables or import the AK and SK pair in other ways. Using hard coding may result in leakage.
# Obtain an AK and SK pair on the management console. For details, see https://support.huaweicloud.com/intl/en-us/usermanual-ca/ca_01_0003.html.
ak = os.getenv("AccessKeyID")
sk = os.getenv("SecretAccessKey")
# (Optional) If you use a temporary AK and SK pair and a security token to access OBS, obtain them from environment variables.
security_token = os.getenv("SecurityToken")
# Set server to the endpoint corresponding to the bucket. Here uses CN-Hong Kong as an example. Replace it with the one in use.
server = "https://obs.ap-southeast-1.myhuaweicloud.com" 

# Create an obsClient instance.
# If you use a temporary AK and SK pair and a security token to access OBS, you must specify security_token when creating an instance.
obsClient = ObsClient(access_key_id=ak, secret_access_key=sk, server=server)
try:
    bucketName = "examplebucket"
    objectKey = "objectname"
    # Specify a pre-defined ACL (PRIVATE as an example).
    acl = 'PRIVATE'
    # Specify a storage class (STANDARD as an example) for the object.
    storageClass = 'STANDARD'
    # Specify a custom metadata of the object.
    metadata = {'key': 'value'}
    # Specify the MIME type for the object.
    contentType = 'text/plain'
    # Specify the lifecycle (7 as an example) for the object, in days.
    expires = 7
    # Initiate a multipart upload.
    resp = obsClient.initiateMultipartUpload(bucketName, objectKey, acl, storageClass, metadata,
                                             contentType=contentType, expires=expires, encoding_type='url')

    # If status code 2xx is returned, the API is called successfully. Otherwise, the API call fails.
    if resp.status < 300:
        print('Initiate Multipart Upload Succeeded')
        print('requestId:', resp.requestId)
        print('bucketName:', resp.body.bucketName)
        print('objectKey:', resp.body.objectKey)
        print('uploadId:', resp.body.uploadId)
    else:
        print('Initiate Multipart Upload Failed')
        print('requestId:', resp.requestId)
        print('errorCode:', resp.errorCode)
        print('errorMessage:', resp.errorMessage)
except:
    print('Initiate Multipart Upload Failed')
    print(traceback.format_exc())

Helpful Links