Cette page n'est pas encore disponible dans votre langue. Nous nous efforçons d'ajouter d'autres langues. Nous vous remercions de votre compréhension.

On this page

Show all

Multipart Upload Overview (SDK for Python)

Updated on 2024-11-26 GMT+08:00

You can upload large files using multipart upload. Multipart upload is applicable to many scenarios, including:

  • Files to be uploaded are larger than 100 MB.
  • The network condition is poor. Connection to the OBS server is constantly down.
  • Sizes of files to be uploaded are uncertain.

A multipart upload consists of the following steps:

  1. Initiate a multipart upload (ObsClient.initiateMultipartUpload).
  2. Upload parts one by one or concurrently (ObsClient.uploadPart).
  3. Assemble parts (ObsClient.completeMultipartUpload) or abort the multipart upload (ObsClient.abortMultipartUpload).

This example shows a complete multipart upload, including initiating a multipart upload, uploading parts, and assembling parts.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
# -*- coding:utf-8 -*-
from obs import ObsClient,CompleteMultipartUploadRequest, CompletePart
import os
import traceback
# Obtain an AK and SK pair using environment variables or import the AK and SK pair in other ways. Using hard coding may result in leakage.
# Obtain an AK and SK pair on the management console. For details, see https://support.huaweicloud.com/eu/usermanual-ca/ca_01_0003.html.
ak = os.getenv("AccessKeyID")
sk = os.getenv("SecretAccessKey")
# (Optional) If you use a temporary AK and SK pair and a security token to access OBS, obtain them from environment variables.
# security_token = os.getenv("SecurityToken")
# Set server to the endpoint corresponding to the bucket. EU-Dublin is used here as an example. Replace it with the one in use.
server = "https://obs.eu-west-101.myhuaweicloud.eu" 

# Create an obsClient instance.
# If you use a temporary AK and SK pair and a security token to access OBS, you must specify security_token when creating an instance.
obsClient = ObsClient(access_key_id=ak, secret_access_key=sk, server=server)
try:
    #Bucket name
    bucketName = "examplebucket"
    #Object name
    objectKey = "objectname"
    # Specify the MIME type for the object.
    contentType = 'text/plain'
    # Initiate a multipart upload.
    resp = obsClient.initiateMultipartUpload(bucketName, objectKey,
                                             contentType=contentType)
    #Obtain the uploadId.
    uploadId = resp.body["uploadId"]
    #Specify the size of the part to upload.
    partSize = 512 * 1024 * 1024
    #Specify the part number.
    partNum = 1
    # Specify whether object indicates the file path. True is used here. The default value is False.
    isFile = True
    #Specify the local object file to upload.
    filepath = r"D:\tmp\file.txt"
    contentLength = os.path.getsize(filepath)
    #Start offset of a part in the source file.
    offset = 0
    etags = {}

    while offset < contentLength:
        partSize = min(partSize, (contentLength - offset));
        # Upload parts.
        resp1 = obsClient.uploadPart(bucketName, objectKey, partNum, uploadId, filepath, isFile, partSize, offset)
        etags[partNum] = resp1.body.etag
        offset = offset + partSize
        partNum = partNum + 1

    completes = []
    for i in range(1, partNum):
        completes.append(CompletePart(i, etags[i]))
    # Assemble parts.
    completeMultipartUploadRequest = CompleteMultipartUploadRequest(parts = completes)
    resp = obsClient.completeMultipartUpload(bucketName, objectKey, uploadId, completeMultipartUploadRequest)
    # If status code 2xx is returned, the API is called successfully. Otherwise, the API call fails.
    if resp.status < 300:
        print('Upload Part Succeeded')
        print('requestId:', resp.requestId)
        print('etag:', resp.body.etag)
    else:
        print('Upload Part Failed')
        print('requestId:', resp.requestId)
        print('errorCode:', resp.errorCode)
        print('errorMessage:', resp.errorMessage)
except:
    print('multPartsUpload Failed')
    print(traceback.format_exc())

Below lists other multipart upload operations:

Feedback

Feedback

Feedback

0/500

Selected Content

Submit selected content with the feedback