Help Center/ Object Storage Service/ SDK Reference/ Python/ APIs Related to Multipart Upload (SDK for Python)/ Uploading a Part (SDK for Python)
Updated on 2024-09-05 GMT+08:00
View PDF
Share

Uploading a Part (SDK for Python)

Function

After a multipart upload is initiated, this API uploads a part to a specified bucket. In the upload request, the multipart upload ID must be included. Except for the part lastly being uploaded whose size ranges from 0 to 5 GB, sizes of the other parts range from 100 KB to 5 GB. Part numbers can be any number from 1 to 10,000.

When uploading a part, you must specify its upload ID and part number. A part number uniquely identifies a part and its position in the object you are uploading. If you upload a new part with the same part number as that of a previous part, the previously uploaded part will be overwritten. Whenever you upload a part, OBS returns the ETag header in the response. For each part upload task, you must record the part number and ETag value. These values are required in subsequent requests for you to complete a multipart upload.

Restrictions

  • To upload a part, you must be the bucket owner or have the required permission (obs:object:PutObject in IAM or PutObject in a bucket policy). For details, see Introduction to OBS Access Control, IAM Custom Policies, and Configuring an Object Policy.
  • The mapping between OBS regions and endpoints must comply with what is listed in Regions and Endpoints.
  • After initiating a multipart upload and uploading one or more parts, you must assemble the parts or abort the multipart upload. Only after that can OBS stops billing you for storing the uploaded parts.
  • partNumber in a multipart upload must be unique. When the same partNumber of the same object is concurrently uploaded, last write wins policy is applied. The time of last write is defined as the time when the part metadata is created. To ensure data accuracy, the client must be locked to ensure concurrent uploads of the same part of the same object. Concurrent uploads for different parts of the same object do not need to be locked.

Method

ObsClient.uploadPart(bucketName, objectKey, partNumber, uploadId, object, isFile, partSize, offset, sseHeader, isAttachMd5, md5, progressCallback, autoClose)

Request Parameters

Table 1 List of request parameters

Parameter

Type

Mandatory (Yes/No)

Description

bucketName

str

Yes

Explanation:

Bucket name

Restrictions:

  • A bucket name must be unique across all accounts and regions.
  • A bucket name:
    • Must be 3 to 63 characters long and start with a digit or letter. Lowercase letters, digits, hyphens (-), and periods (.) are allowed.
    • Cannot be formatted as an IP address.
    • Cannot start or end with a hyphen (-) or period (.).
    • Cannot contain two consecutive periods (..), for example, my..bucket.
    • Cannot contain periods (.) and hyphens (-) adjacent to each other, for example, my-.bucket or my.-bucket.
  • If you repeatedly create buckets of the same name in the same region, no error will be reported and the bucket attributes comply with those set in the first creation request.

Default value:

None

objectKey

str

Yes

Explanation:

Object name. An object is uniquely identified by an object name in a bucket. An object name is a complete path that does not contain the bucket name.

For example, if the address for accessing the object is examplebucket.obs.ap-southeast-1.myhuaweicloud.com/folder/test.txt, the object name is folder/test.txt.

Value range:

The value must contain 1 to 1,024 characters.

Default value:

None

NOTE:

The object URL is in the following format: https://Bucket name.Domain name/Folder directory level/Object name. If this object is stored in the root directory of the bucket, its URL does not contain the folder directory level.

partNumber

int

Yes

Explanation:

Part number

Value range:

[1,10000]

Default value:

None

uploadId

str

Yes

Explanation:

Multipart upload ID which can be returned by initiating a multipart upload, for example, 000001648453845DBB78F2340DD460D8

Restrictions:

The value must contain 32 characters.

Default value:

None

object

str or readable object

Yes

Explanation:

Part content to be uploaded

Value range:

A string or a readable object

NOTE:

If object is a readable object that contains the read attribute, data is read from the readable object. Otherwise, the object content is a string.

Default value:

None

isFile

bool

No

Explanation:

Whether object indicates the file path.

Value range:

True: object indicates the file path.

False: object does not indicate the file path.

Default value:

False

offset

int

No

Explanation:

Start offset of a part in the source file

Value range:

A non-negative integer not exceeding the size of the object to be uploaded, in bytes

Default value:

0

partSize

int

No

Explanation:

Part size

Value range:

The value ranges from 100 KB to 5 GB, in bytes.

Default value:

102400

sseHeader

SseCHeader

No

Explanation:

Server-side encryption header. For details, see Table 2.

Default value:

None

isAttachMd5

bool

No

Explanation:

Whether to automatically calculate the MD5 value of the data to be uploaded.

Restrictions:

If isAttachMd5 and md5 are used at the same time, isAttachMd5 is invalid.

Value range:

True: The MD5 value of the data to be uploaded is automatically calculated.

False: The MD5 value of the data to be uploaded is not automatically calculated.

Default value:

False

md5

str

No

Explanation:

Base64-encoded MD5 value of the part to be uploaded, which uniquely identifies the content of the uploaded part and can be used to identify whether the object content is changed.

Restrictions:

If isAttachMd5 and md5 are used at the same time, isAttachMd5 is invalid.

Value range:

The value must contain 32 characters.

Default value:

None

crc64

int

or

str

or

long

No

Explanation:

CRC64 value of the data to be uploaded.

Restrictions:

This parameter is not supported for POSIX or SFS objects.

Value range:

A 64-bit CRC value calculated based on the ECMA-182 standard.

Default value:

None

isAttachCrc64

bool

No

Explanation:

Whether to automatically calculate the CRC64 value of the data to be uploaded.

Restrictions:

This parameter is not supported for POSIX or SFS objects.

Value range:

  • True: The SDK calculates CRC64 and submits it to the server for verification.
  • False: CRC64 is not calculated.

Default value:

Falsed

progressCallback

callable

No

Explanation:

Callback function for obtaining the upload progress

Default value:

None

NOTE:

This callback function contains the following parameters in sequence: number of uploaded bytes, total bytes, and used time (in seconds).

autoClose

bool

No

Explanation:

Whether to automatically close data streams after the upload is complete

Value range:

True: The data stream is automatically closed.

False: The data stream is not automatically closed.

Default value:

True
Table 2 SseCHeader

Parameter

Type

Mandatory (Yes/No)

Description

encryption

str

Yes

Explanation:

SSE-C used for encrypting objects

Value range:

AES256

Default value:

None

key

str

Yes

Explanation:

Key used in SSE-C encryption. It corresponds to the encryption method. For example, if encryption is set to AES256, the key is calculated using the AES-256 algorithm.

Value range:

The value must contain 32 characters.

Default value:

None

Responses

Table 3 List of returned results

Type

Description

GetResult

Explanation:

SDK common results
Table 4 GetResult

Parameter

Type

Description

status

int

Explanation:

HTTP status code

Value range:

A status code is a group of digits ranging from 2xx (indicating successes) to 4xx or 5xx (indicating errors). It indicates the status of a response. For more information, see Status Code.

Default value:

None

reason

str

Explanation:

Reason description.

Default value:

None

errorCode

str

Explanation:

Error code returned by the OBS server. If the value of status is less than 300, this parameter is left blank.

Default value:

None

errorMessage

str

Explanation:

Error message returned by the OBS server. If the value of status is less than 300, this parameter is left blank.

Default value:

None

requestId

str

Explanation:

Request ID returned by the OBS server

Default value:

None

indicator

str

Explanation:

Error indicator returned by the OBS server.

Default value:

None

hostId

str

Explanation:

Requested server ID. If the value of status is less than 300, this parameter is left blank.

Default value:

None

resource

str

Explanation:

Error source (a bucket or an object). If the value of status is less than 300, this parameter is left blank.

Default value:

None

header

list

Explanation:

Response header list, composed of tuples. Each tuple consists of two elements, respectively corresponding to the key and value of a response header.

Default value:

None

body

object

Explanation:

Result content returned after the operation is successful. If the value of status is larger than 300, this parameter is left blank. The value varies with the API being called. For details, see Bucket-Related APIs (SDK for Python) and Object-Related APIs (SDK for Python).

Default value:

None
Table 5 GetResult.body

GetResult.body Type

Description

UploadPartResponse

Explanation:

Response to the request for uploading a part
Table 6 UploadPartResponse

Parameter

Type

Description

etag

str

Explanation:

Base64-encoded 128-bit MD5 digest of a part. ETag is the unique identifier of the part content. It can be used to determine whether the part content is changed.

Value range:

The value must contain 32 characters.

Default value:

None

crc64

str

Explanation:

A 64-bit CRC value calculated for the part based on the ECMA-182 standard. It uniquely identifies a part and can be used to check the integrity of the content.

Restrictions:

  • This parameter is returned when the CRC64 value was verified for the uploaded part, or when the CRC64 feature has been enabled for the bucket.
  • This parameter is not supported for POSIX or SFS objects.

Value range:

A 64-bit CRC value calculated based on the ECMA-182 standard.

Default value:

None

sseKms

str

Explanation:

SSE-KMS is used for encrypting objects on the server side.

Value range:

kms

Default value:

None

sseKmsKey

str

Explanation:

ID of the KMS master key when SSE-KMS is used

Value range:

Valid value formats are as follows:

  1. regionID:domainID:key/key_id
  2. key_id

In the preceding formats:

Default value:

  • If this parameter is not specified, the default master key will be used.
  • If there is no such a default master key, OBS will create one and use it by default.

sseC

str

Explanation:

Algorithm used to encrypt and decrypt objects with SSE-C

Value range:

AES256

Default value:

None

sseCKeyMd5

str

Explanation:

MD5 value of the key for encrypting objects when SSE-C is used. This value is used to check whether any error occurs during the transmission of the key.

Restrictions:

The value is encrypted by MD5 and then encoded by Base64, for example, 4XvB3tbNTN+tIEVa0/fGaQ==.

Default value:

None

Code Examples

This example uploads a part.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
 
from obs import ObsClient
import os
import traceback

# Obtain an AK and SK pair using environment variables or import the AK and SK pair in other ways. Using hard coding may result in leakage.
# Obtain an AK and SK pair on the management console. For details, see https://support.huaweicloud.com/intl/en-us/usermanual-ca/ca_01_0003.html.
ak = os.getenv("AccessKeyID")
sk = os.getenv("SecretAccessKey")
# (Optional) If you use a temporary AK and SK pair and a security token to access OBS, obtain them from environment variables.
security_token = os.getenv("SecurityToken")
# Set server to the endpoint corresponding to the bucket. Here uses CN-Hong Kong as an example. Replace it with the one in use.
server = "https://obs.ap-southeast-1.myhuaweicloud.com" 

# Create an obsClient instance.
# If you use a temporary AK and SK pair and a security token to access OBS, you must specify security_token when creating an instance.
obsClient = ObsClient(access_key_id=ak, secret_access_key=sk, server=server)
try:
    bucketName = "examplebucket"
    # Specify the name of the object to be uploaded to the bucket.
    objectKey = "objectname"
    # Specify the part number, which ranges from 1 to 10,000
    partNumber = "your partNumber"
    # Specify the ID of the multipart upload.
    uploadId = "your uploadid"
    # Specify the content of the part to be uploaded as a string or readable object.
    object = 'Hello OBS'
    # Specify whether object indicates the file path. The default value is False.
    isFile = False
    # Specify the start offset (in bytes) of a part in the source file. The default value is 0.
    offset = 0
    # Specify the size (in bytes) of a part in the source file. The default value is the file size minus offset.
    partSize = 9 * 1024 * 1024
    # Specify whether to automatically calculate the MD5 value of the data to be uploaded. The default value is False.
    isAttachMd5 = True
    # Upload the part to a specified bucket using the multipart upload ID.
    resp = obsClient.uploadPart(bucketName, objectKey, partNumber, uploadId, object, isFile, partSize,
                                offset, isAttachMd5=isAttachMd5)

    # If status code 2xx is returned, the API is called successfully. Otherwise, the API call fails.
    if resp.status < 300:
        print('Upload Part Succeeded')
        print('requestId:', resp.requestId)
        print('etag:', resp.body.etag)
    else:
        print('Upload Part Failed')
        print('requestId:', resp.requestId)
        print('errorCode:', resp.errorCode)
        print('errorMessage:', resp.errorMessage)
except:
    print('Upload Part Failed')
    print(traceback.format_exc())

Helpful Links