Multipart Upload_Uploading an Object_Object Management_User Guide

Scenarios

In a multipart upload, an object is split into multiple parts and these parts are uploaded independently. After all parts are uploaded, OBS assembles them into a single object. Generally, if the size of an object reaches 100 MB, you should use multipart upload instead of simple upload. Multipart upload has the following advantages:

Improved throughput: Uploading parts in parallel speeds up the upload and maximizes bandwidth utilization.
Enhanced robustness against network instability: If a part fails to upload due to network issues, you only need to re-upload that specific part. There is no need to upload the entire object again, which saves both time and traffic.
Uploading objects with unknown sizes: You can use multipart upload to upload an object with an unknown size. For example, in video surveillance scenarios, video files are created and uploaded in real time, so multipart upload is a good choice for uploading these files.

Constraints

Folders do not support multipart upload.
Each request can upload only one part, but multiple requests can be initiated at the same time.

Constraints on parts:

**Table 1** Constraints on parts
Item	Constraints
Maximum object size	48.8 TB
Maximum number of parts for each upload task	10,000
Part number	1 to 10,000 (inclusive)
Part size	5 MB to 5 GB. The size of the last part is between 0 bytes to 5 GB.
Maximum number of parts returned in response to the request for listing uploaded parts	1,000
Maximum number of multipart uploads returned in response to the request for listing initiated multipart uploads	1,000

If you use SSE-C in a multipart upload, you must include the key in the requests for initiating the multipart upload, uploading parts, and copying parts. All requests in this multipart upload must use the same key. The key is not required for completing the multipart upload. The complete object is encrypted using SSE-C.
If you use SSE-KMS or SSE-OBS in a multipart upload, you only need to configure the encryption method, algorithm, and key in initiating the multipart upload.

Billing

If you want to upload a large number of Deep Archive objects, you can upload them in the Standard storage class and then transition them to the Deep Archive storage class through lifecycle rules to lower the costs on PUT requests.
After initiating a multipart upload and uploading one or more parts, you must complete or abort the multipart upload. This ensures that OBS frees up the space for storing the uploaded parts and stops billing you for the parts.

Permissions

Before performing a multipart upload, you must obtain the permissions listed in the following table.

Operation	Required Permission
Initiate a multipart upload.	You must be the bucket owner or have the PutObject permission.
Upload parts.	You must have the PutObject permission.
Complete the multipart upload.	You must have the PutObject permission.
Copy parts.	You must have the PutObject and GetObject permissions.
List uploaded parts.	You must be the bucket owner or have the ListMultipartUploadParts permission.
List multipart uploads.	You must be the bucket owner or have the ListBucketMultipartUploads permission.
Abort the multipart upload.	You must be the bucket owner or have the AbortMultipartUpload permission.

Important Notes

If the local file changes during the upload, all parts need to be uploaded again.
When concurrent upload operations are performed for the same part of an object, the server follows the Last Write Wins policy, where the Last Write is determined by the time the part's metadata was created. To ensure data accuracy, the client must use a locking mechanism to prevent concurrent uploads of the same part of an object. Locking is not required for concurrent uploads of different parts of an object.

When uploading a large number of objects, do not name objects with sequential prefixes, such as timestamps or alphabetical order. Objects with sequential prefixes may cause a large number of object indexes to be stored in a specific storage partition. In such case, if there are too many requests for the objects, the requests cannot be handled efficiently.
It is advised to enable versioning to prevent objects with the same name from being overwritten. Versioning keeps multiple versions of an object in the same bucket. You can restore any historical version at any time.

Multipart Upload Process

Figure 1 Process of a multipart upload

**Table 2** Process of a multipart upload
No.	Operation	Description
1	Split a file into parts.	On your local client, split a large file into multiple parts. Each part must be between 5 MB and 5 GB in size, except for the last part, which can be 0 bytes to 5 GB.
2	Initiate a multipart upload.	Initiate a multipart upload and obtain the upload ID. The ID is required for all subsequent operations in this multipart upload, such as uploading, assembling, and listing parts. When initiating a multipart upload, you need to specify the storage class, object ACL, and other object properties. Once specified, you do not need to specify them again when uploading or assembling parts later.
3	Upload parts.	Upload parts. You can upload parts independently and in any order. If your network is in good condition, you can upload parts concurrently to improve the upload speed and make full use of bandwidth. If a part fails to upload, you only need to upload that part again. The upload of other parts remains unaffected. To abort the upload, make an API call to abort the multipart upload and delete the uploaded parts.
4	Complete the multipart upload.	After all parts are uploaded, assemble the parts into a single, complete object.

Example of a Multipart Upload

OBS Console, OBS Browser+, and obsutil encapsulate the multipart upload procedure. You only need to know how to upload a file using a GUI or CLI. For details, see Ways to Upload. The following uses API Explorer to make OBS API calls to demonstrate how to perform a multipart upload.

Step 1: Split a File into Parts

Split a file into parts on your local client.

(Optional) Create a file. Here, we create a 20 MB file named test.file. If you have prepared a file, skip this step.

Linux or macOS:
```
dd if=/dev/urandom of=test.file bs=1M count=20
```
On Windows, open CMD and run the following command:
```
fsutil file createnew test.file 20971520
```

Split the file into parts. In this example, split the 20 MB test.file into five 4 MB parts. Each part is named test-part**, where ** is a number suffix from 00 to 04.

Linux or macOS:

split -b 4M -d test.file test-part

On Windows, run the following commands in PowerShell:

$inputFile = "test.file"
$outputFilePrefix = "test-part"
$chunkSize = 4MB

$fs = [System.IO.File]::OpenRead($inputFile)
$buffer = New-Object byte[] $chunkSize
$fileNumber = 0

while ($fs.Position -lt $fs.Length) {
$bytesRead = $fs.Read($buffer, 0, $chunkSize)
$outputFile = "{0}{1:D2}" -f $outputFilePrefix, $fileNumber
$fileStream = [System.IO.File]::Create($outputFile)
$fileStream.Write($buffer, 0, $bytesRead)
$fileStream.Close()
$fileNumber++
}

$fs.Close()

After the split is complete, verify that all parts are present in the directory where the commands were executed. Parts created in this example are as follows.

Step 2: Initiate a Multipart Upload

Make an API call to initiate a multipart upload and obtain the multipart upload ID.

Go to the InitiateMultipartUpload page of API Explorer.

Set the API call parameters.

Parameter	Example Value	Description
object_key	test.file	The name of the object uploaded to the bucket. It refers to the entire object, not to any individual part.
bucket_name	example-bucket-a-intl	The bucket name. Click to choose a bucket that stores your object.
uploads	Leave it blank.	This parameter indicates that the API call initiates a multipart upload, rather than a standard POST upload. Click Leave Blank to leave the parameter blank.

Click Debug.
After the API call succeeds, obtain the upload ID from the response in the lower right corner of the page.

As shown in the following figure, the upload ID is 000001987D77252A002424CADFB90B1C.

Step 3: Upload Parts

Upload the parts obtained in Step 1: Split a File into Parts to the bucket.

Go to the UploadPart page of API Explorer.

Set the API call parameters.

Click to enlarge

Parameter	Example Value	Description
object_key	test.file	The name of the object uploaded to the bucket. It refers to the entire object, not to any individual part. Keep this name the same as the one set in 2.
bucket_name	example-bucket-a-intl	The bucket name. Click to choose a bucket that stores your object. Keep this name the same as the one set in 2.
partNumber	1	The part number. It must be an integer from 1 to 10000. A part number uniquely identifies a part and its position. Parts must be numbered in ascending order according to their position in the object. For example, if a 20 MB object is split into five parts, the first part is numbered 1, the second part 2, and so on. If you upload a new part using the same part number as a previously uploaded part, the previously uploaded part will be overwritten.
uploadId	000001987D77252A002424CADFB90B1C	The upload ID, which must be the one obtained in 4.
Body	test-part00	The part to upload. Click to select the part to upload.

Click Debug.
After the API call succeeds, verify that a success message appears on the Debugging Result tab. The response shown in the lower right corner of the page includes the ETag value of the part.
Repeat 2 to 3 to upload all parts.

After the upload is complete, go to the ListParts page, set parameters, and click Debug to list all uploaded parts.

Parameter	Example Value	Description
object_key	test.file	The name of the object uploaded to the bucket. It refers to the entire object, not to any individual part. Keep this name the same as the one set in 2.
bucket_name	example-bucket-a-intl	The bucket name. Click to choose a bucket that stores your object. Keep this name the same as the one set in 2.
uploadId	000001987D77252A002424CADFB90B1C	The upload ID, which must be the one obtained in 4.

Verify that all parts have been uploaded successfully and obtain the part numbers and ETag values of all parts.

If the response result in 6 contains all parts, the parts have been uploaded successfully. In this case, record the part numbers and ETag values of all parts. The recorded data will be used as the request parameters for completing the multipart upload in the next step.

Step 4: Complete the Multipart Upload

This step assembles the uploaded parts into an object.

Go to the CompleteMultipartUpload page of API Explorer.

Set the API call parameters.

Parameter	Example Value	Description
object_key	test.file	The name of the object uploaded to the bucket. It refers to the entire object, not to any individual part. Keep this name the same as the one set in 2.
bucket_name	example-bucket-a-intl	The bucket name. Click to choose a bucket that stores your object. Keep this name the same as the one set in 2.
uploadId	000001987D77252A002424CADFB90B1C	The upload ID, which must be the one obtained in 4.
Body	<CompleteMultipartUpload> <Part> <PartNumber>1</PartNumber> <ETag>"ef0bff811bdb99fb44804719bd0f6877"</ETag> </Part> <Part> <PartNumber>2</PartNumber> <ETag>"ef0bff811bdb99fb44804719bd0f6877"</ETag> </Part> <Part> <PartNumber>3</PartNumber> <ETag>"ef0bff811bdb99fb44804719bd0f6877"</ETag> </Part> <Part> <PartNumber>4</PartNumber> <ETag>"ef0bff811bdb99fb44804719bd0f6877"</ETag> </Part> <Part> <PartNumber>5</PartNumber> <ETag>"ef0bff811bdb99fb44804719bd0f6877"</ETag> </Part> </CompleteMultipartUpload>	The body must include the part numbers and ETag values of all parts. You can obtain the data by making a ListParts API call after all parts have been uploaded. For details, see 7. Alternatively, you can obtain the ETag value of each part from its upload response. For details, see 4. CAUTION: If a part is omitted when a multipart upload is completed, the part will be deleted and cannot be recovered. For example, if 10 parts are uploaded but only nine are assembled, the omitted part will be deleted and cannot be recovered. Therefore, before completing a multipart upload, list all uploaded parts and make sure that no part is omitted.

Click Debug.
After the API call succeeds, verify that a success message appears on the Debugging Result tab. The response includes the object name, bucket name, and ETag value of the complete object.

Ways to Upload

You can use OBS Console, APIs, SDKs, OBS Browser+, or obsutil to upload objects.

To improve the upload success rate, OBS Console automatically uses multipart upload for objects larger than 8 MB, and OBS Browser+ and obsutil automatically use resumable upload (essentially multipart upload) for objects larger than 50 MB. You will not perceive any backend switchovers, as your operations on OBS Console or tools remain unchanged.

Using OBS Console

In the navigation pane of OBS Console, choose Object Storage.
In the bucket list, click the bucket you want to operate. The Objects page is displayed.
On the object list page, click Upload Object.

You can also go to the folder that the file is to be uploaded to and click Upload Object.

Batch upload is used as an example here. If a region only supports single upload, follow the on-screen instructions.

If the files that you want to upload to OBS are stored in Microsoft OneDrive, it is recommended that the names of these files contain a maximum of 32 characters to ensure compatibility.

Figure 2 Uploading objects
Select a storage class. If you do not specify a storage class, the objects you upload inherit the default storage class of the bucket.

An object can have a different storage class from its bucket. By default, an object inherits the storage class of the bucket where it is uploaded. You can specify a storage class for an object when uploading it, or you can change the object storage class after the object is uploaded.
Drag and drop the files or folders you want to upload to the Upload Object area.

You can also click add files in the Upload Object area to select files.
Server-Side Encryption: Disable it or choose SSE-KMS or SSE-OBS. For details, see Server-Side Encryption Overview.

If your bucket has server-side encryption enabled, you can select Inherit from bucket when uploading an object.
(Optional) To configure metadata, or retention policies, or object tags, click Next: (Optional) Configure Advanced Settings.

WORM retention policies can be configured in the advanced settings only when WORM is enabled for the bucket.

Configuring metadata: Add metadata ContentDisposition, ContentLanguage, WebsiteRedirectLocation, ContentEncoding, or ContentType as needed. For more information, see OBS Metadata. Metadata is a set of name-value pairs. The metadata value cannot be left blank. You can add multiple metadata entries by clicking Add.

Configuring retention: Choose Inherit from bucket, or choose Configure and then specify a retention period, to automatically protect new objects uploaded to the bucket from being deleted before the specified date.

Configuring tags: Tags are used to identify objects with tag keys and values. For details, see Adding Tags to an Object.

Figure 3 Configuring metadata or WORM retention
Click Upload.

Using APIs

Operations on Multipart Upload

Using SDKs

Using the GUI Tool - OBS Browser+

OBS Browser+ automatically uses resumable upload (essentially multipart upload) for objects larger than 50 MB in the background.

Log in to OBS Browser+.
Click the bucket where you want to upload files or folders.
Click Upload and then Add File or Folder, as shown in Figure 4.

Figure 4 Uploading a file or folder

For better experience when using the Add File function, you are advised to upload a maximum of 100 files at a time. If you need to upload more, place all the files in a folder and upload them by adding a folder.
1. If message "Service Unavailable" is displayed when files are being uploaded, try again later.
2. If an access deny message is displayed when you are uploading a file or folder, possible causes are as follows:
- Access to the bucket is restricted by an ACL. For example, you do not have the write permission for the bucket.
- Access to the bucket is restricted by a bucket policy. For example, you do not have the write permission for the bucket, or write operations cannot be performed on the bucket during the current period.
  If such message is displayed, check ACL and policy settings of the bucket and resolve the problem accordingly.
You must have access to the file you want to upload, or the file upload will fail.
In the displayed dialog box, select the file or folder you want to upload and click Open.

You can upload one folder or multiple files at a time. To upload multiple files, hold down Ctrl or Shift to select multiple files and batch upload them. You can also press Ctrl+A to select all files. The operations are consistent with those in Windows operating systems.
Click OK.

Using the CLI Tool - obsutil

obsutil automatically uses resumable upload (essentially multipart upload) for objects larger than 50 MB in the background.

Command Line Structure

In Windows

Uploading a file

obsutil cp file_url obs://bucket[/key] [-arcDir=xxx] [-dryRun] [-link] [-u] [-vlength] [-vmd5] [-p=1] [-threshold=5248800] [-acl=xxx] [-sc=xxx] [-meta=aaa:bbb#ccc:ddd] [-ps=auto] [-o=xxx] [-cpd=xxx] [-fr] [-o=xxx] [-config=xxx] [-e=xxx] [-i=xxx] [-k=xxx] [-t=xxx]

Uploading a folder

obsutil cp folder_url obs://bucket[/key] -r [-arcDir=xxx] [-dryRun] [-link] [-f] [-flat] [-u] [-vlength] [-vmd5] [-j=1] [-p=1] [-threshold=52428800] [-acl=xxx] [-sc=xxx] [-meta=aaa:bbb#ccc:ddd] [-ps=auto] [-include=*.xxx] [-exclude=*.xxx] [-timeRange=time1-time2] [-mf] [-o=xxx] [-cpd=xxx] [-config=xxx] [-e=xxx] [-i=xxx] [-k=xxx] [-t=xxx]

Uploading multiple files/folders

obsutil cp file1_url,folder1_url|filelist_url obs://bucket[/prefix] -msm=1 [-r] [-arcDir=xxx] [-dryRun] [-link] [-f] [-u] [-vlength] [-vmd5] [-flat] [-j=1] [-p=1] [-threshold=52428800] [-acl=xxx] [-sc=xxx] [-meta=aaa:bbb#ccc:ddd] [-ps=auto] [-include=*.xxx] [-exclude=*.xxx][-timeRange=time1-time2] [-at] [-mf] [-o=xxx] [-cpd=xxx] [-config=xxx] [-e=xxx] [-i=xxx] [-k=xxx] [-t=xxx]

In this command, /prefix is the name prefix for uploading folders. For the execution examples, see Upload.

In Linux or macOS

Uploading a file

./obsutil cp file_url obs://bucket[/key] [-arcDir=xxx] [-dryRun] [-link] [-u] [-vlength] [-vmd5] [-p=1] [-threshold=5248800] [-acl=xxx] [-sc=xxx] [-meta=aaa:bbb#ccc:ddd] [-ps=auto] [-o=xxx] [-cpd=xxx] [-fr] [-o=xxx] [-config=xxx] [-e=xxx] [-i=xxx] [-k=xxx] [-t=xxx]

Uploading a folder

./obsutil cp folder_url obs://bucket[/key] -r [-arcDir=xxx] [-dryRun] [-link] [-f] [-flat] [-u] [-vlength] [-vmd5] [-j=1] [-p=1] [-threshold=52428800] [-acl=xxx] [-sc=xxx] [-meta=aaa:bbb#ccc:ddd] [-ps=auto] [-include=*.xxx] [-exclude=*.xxx] [-timeRange=time1-time2] [-at] [-mf] [-o=xxx] [-cpd=xxx] [-config=xxx] [-e=xxx] [-i=xxx] [-k=xxx] [-t=xxx]

Uploading multiple files/folders

./obsutil cp file1_url,folder1_url|filelist_url obs://bucket[/prefix] -msm=1 [-r] [-arcDir=xxx] [-dryRun] [-link] [-f] [-u] [-vlength] [-vmd5] [-flat] [-j=1] [-p=1] [-threshold=52428800] [-acl=xxx] [-sc=xxx] [-meta=aaa:bbb#ccc:ddd] [-ps=auto] [-include=*.xxx] [-exclude=*.xxx][-timeRange=time1-time2] [-mf] [-o=xxx] [-cpd=xxx] [-config=xxx] [-e=xxx] [-i=xxx] [-k=xxx] [-t=xxx]

In this command, /prefix is the name prefix for uploading folders. For the execution examples, see Upload.

Examples

Take the Windows OS as an example. Run the obsutil cp d:\temp\test.txt obs://bucket-test/key command to upload the test.txt file in the temp directory in the D: drive to bucket bucket-test and rename the file as key.

obsutil cp d:\temp\test.txt obs://bucket-test/key
Start at 2024-09-30 08:11:41.6724827 +0000 UTC

Parallel:      5                   Jobs:          5
Threshold:     50.00MB             PartSize:      auto
VerifyLength:  false               VerifyMd5:     false
CheckpointDir: C:\Users\Administrator\.obsutil_checkpoint

[====================================================] 100.00% 1.68 MB/s 8.46MB/8.46MB 5s
Upload successfully, 8.46MB, n/a, d:\temp\test.txt --> obs://bucket-test/key, cost [5], status [200], request id [0000016979E1D2B2860BB5181229C72C]

Take the Windows OS as an example. Run the obsutil cp d:\temp obs://bucket-test -f -r command to recursively upload all files and subfolders in the temp directory in the D: drive to the temp folder in bucket bucket-test.

obsutil cp d:\temp obs://bucket-test -f -r
Start at 2024-09-30 08:14:12.1406275 +0000 UTC

Parallel:      5                   Jobs:          5
Threshold:     50.00MB             PartSize:      auto
VerifyLength:  false               VerifyMd5:     false
CheckpointDir: C:\Users\Administrator\.obsutil_checkpoint

Task id: 104786c8-27c2-48fc-bc6a-5886596fb0ed
OutputDir: C:\Users\Administrator\.obsutil_output

[========================================================] 100.00% tps:35.71 2.02 KB/s 7.20MB/7.20MB 0s
Succeed count:   5         Failed count:    0
Succeed bytes:      xxx
Metrics [max cost:90 ms, min cost:45 ms, average cost:63.80 ms, average tps:35.71, transferred size: 7.20MB]

Task id: 104786c8-27c2-48fc-bc6a-5886596fb0ed

For more examples, see Upload.

Parameter Description

Parameter	Optional or Mandatory	Description
file_url	Optional for uploading multiple files/folders Mandatory for uploading a file	The local file path Do not nest paths when uploading multiple files/folders. For example, you cannot specify /a/b/c and /a/b/ at the same time. If this parameter is configured when uploading multiple files/folders, msm must be set to 1. In this case, use commas (,) to separate multiple file paths, for example, file_url1,file_url2. Files and folders can both be included when uploading multiple files/folders. For example, file_url1,folder_url1,file_url2,folder_url2.
folder_url	Optional for uploading multiple files/folders Mandatory for uploading a folder	The local folder path If flat is not configured when uploading a folder, the entire folder is uploaded. If flat is configured, all files in the folder are uploaded. Do not nest paths when uploading multiple files/folders. For example, you cannot specify /a/b/c and /a/b/ at the same time. If this parameter is configured when uploading multiple files/folders, msm must be set to 1. In this case, use commas (,) to separate multiple folder paths, for example, folder_url1,folder_url2. Files and folders can both be included when uploading multiple files/folders. For example, file_url1,folder_url1,file_url2,folder_url2.
filelist_url	Optional for uploading multiple files/folders	The path of the file that contains the list of files/folders to be uploaded. If this parameter is configured, msm must be set to 2. The list file is in common text file formats, such as TXT and CSV. Each line in the file indicates a file or folder to be uploaded. For example: file_url1 file_url2 folder_url1 folder_url2 Do not nest paths in the list file. For example, you cannot specify /a/b/c and /a/b/ at the same time.
bucket	Mandatory	The name of the bucket where you will upload the file or folder
key	Optional	The object name or object name prefix specified when uploading a file, or the object name prefix specified when uploading a folder When uploading a file or folder, the upload path and name change depending on the value of this parameter. The specific rules are as follows: If this parameter is left blank, the object is uploaded to the root directory of the bucket, and the object name is the name of the uploaded file or folder. If this parameter is specified: If the parameter value ends with a slash (/), the object is uploaded to the directory indicated by this value, and the object name is the parameter value plus the file name, for example, folder/filename.txt or folder/foldername. If the parameter value does not end with a slash (/), the object name for the file is the parameter value, for example, filename.txt, and the object name for the folder is the parameter value plus a slash (/), for example, folder/, which is also the name of the directory where the folder is uploaded. NOTE: For details about how to use this parameter, see Upload.
fr	Optional for uploading a file (additional parameter)	Generates an operation result file when uploading a file.
flat	Optional for uploading a folder or multiple files/folders (additional parameter)	Uploads all files in a folder but not the folder itself.
arcDir	Optional (additional parameter)	The path to which the uploaded files are archived
dryRun	Optional (additional parameter)	Conducts a dry run.
link	Optional (additional parameter)	The actual path of a symbolic link for a file or folder If this parameter is not specified and the file to be uploaded is a symbolic-link file whose target file does not exist, the exception message "The system cannot find the file specified" will be displayed in Windows OS, while the exception message "No such file or directory" will be displayed in macOS or Linux OS. Avoid the symbolic link loop of a folder, otherwise, the upload will exit due to panic. If you do not want the system to panic, set panicForSymbolicLinkCircle to false in the configuration file.
u	Optional (additional parameter)	Indicates incremental upload. If this parameter is set, each file can be uploaded only when it does not exist in the bucket, its size is different from the namesake one in the bucket, or it has the latest modification time. When you compare each local file with data in the bucket, a billable HEAD request is generated. For details, see Requests.
vlength	Optional (additional parameter)	After the upload completes, checks whether the sizes of the objects in the bucket are the same as those of the local files.
vmd5	Optional (additional parameter)	After the upload completes, checks whether the MD5 values of the objects in the bucket are the same as those of the local files. If your object needs encryption, do not use this parameter. If the size of the file or folder to be uploaded is too large, using this parameter will degrade the overall performance due to MD5 calculation. After the MD5 verification is successful, this parameter value is used for metadata x-obs-meta-md5chksum, for later MD5 verification during download or copy.
p	Optional (additional parameter)	The maximum number of concurrent uploads for a multipart upload The default value is the value of defaultParallels in the configuration file.
threshold	Optional (additional parameter)	The threshold for enabling multipart upload. If the size of the file or folder to be uploaded is smaller than the threshold, upload it directly. Otherwise, a multipart upload is required. The default value is the value of defaultBigfileThreshold in the configuration file. Unit: byte This parameter value can contain a unit, for example, 1MB (indicating 1,048,576 bytes). NOTE: If you upload a file or folder directly, no part record is generated, and resumable transmission is not supported.
acl	Optional (additional parameter)	The access control policies that can be specified for objects when uploading files. Possible values are: private: Only users granted permissions by the object ACL can access the object. public-read: Anyone can read objects in the bucket. public-read-write: Anyone can read, write, or delete objects in the bucket. bucket-owner-full-control: Only the bucket owner has full control over objects in the bucket.
sc	Optional (additional parameter)	The storage classes of objects that can be specified when uploading files Possible values are: standard: Standard storage class. It features low access latency and high throughput, and is applicable to storing frequently accessed data (multiple accesses per month) or data that is smaller than 1 MB. warm: Infrequent Access storage class. It is ideal for storing infrequently accessed (less than 12 times a year) data, but when needed, the access has to be fast. cold: Archive storage class. It provides secure, durable, and inexpensive storage for rarely-accessed (once a year) data. deep-archive: Deep Archive storage class (under limited beta testing). It is suitable for storing data that is barely (once every few years) accessed. This storage class costs less than the Archive storage class, but takes longer time (usually several hours) to restore data.
meta	Optional (additional parameter)	The standard or user-defined metadata that can be specified during file upload Standard metadata includes Content-Type, Content-Encoding, Cache-Control, Content-Disposition, Content-Language and Expires. User-defined metadata must be in the format of key:value. Multiple user-defined metadata items are separated by number signs (#), for example, key1:value1#key2:value2#key3:value3, which indicates that the file uploaded to the bucket contains three groups of user-defined metadata: key1:value1, key2:value2, and key3:value3.
ps	Optional (additional parameter)	The size of each part in a multipart upload task The default value is the value of defaultPartSize in the configuration file. The parameter can be set to auto. In this case, obsutil automatically configures the part size for each multipart task based on the source file size. Range: 100KB to 5GB Unit: byte. This parameter value can contain a unit, for example, 1MB (indicating 1,048,576 bytes).
cpd	Optional (additional parameter)	The folder where the part records reside. The default value is .obsutil_checkpoint, the subfolder in the home directory of the user who executes obsutil commands. A part record is generated during a multipart upload and saved to the upload subfolder. After the upload succeeds, its part record is deleted automatically. If the upload fails or is suspended, the system attempts to resume the task according to its part record when you perform the upload the next time.
r	Mandatory for uploading a folder (additional parameter) Optional for uploading multiple files/folders	Recursively uploads all files and subfolders within a folder.
f	Optional for uploading a folder or multiple files/folders (additional parameter)	Runs in force mode.
j	Optional for uploading a folder or multiple files/folders (additional parameter)	The maximum number of concurrent tasks for uploading a folder. The default value is the value of defaultJobs in the configuration file. NOTE: The value is ensured to be greater than or equal to 1.
msm	Mandatory for uploading multiple files/folders (additional parameter)	Enables the mode for uploading multiple files/folders. Possible values are 1 and 2. If msm is set to 1, the source URL indicates a list of file/folder names separated by commas. If msm is set to 2, the source URL indicates a file containing a list of file/folder names. If the file or folder name already contains commas (,), do not set msm to 1. If parameter r is not set, the folders in the list will not be uploaded.
exclude	Optional for uploading a folder or multiple files/folders (additional parameter)	The file matching patterns that are excluded. After this parameter is specified, if the name of the file to be uploaded matches the value of this parameter, the file is skipped. For example, if this parameter is set to .txt, all files whose names end with .txt* are skipped and not uploaded. An asterisk () matches any number of characters. For instance, abc.txt matches all files whose names start with abc, contain any number of characters in the middle, and end with .txt. A question mark (?) matches any single character. For instance, abc?.txt matches all files whose names start with abc, contain any single character in the middle, and end with .txt*. You can use \ to represent * and \? to represent ?. Multiple exclude parameters can be specified, for example, *-exclude=.xxx -exclude=.xxx. Restrictions*: The matching pattern takes effect only for files in the folder. NOTE: You are advised to use quotation marks for the matching pattern to prevent special characters from being escaped by the OS and leading to unexpected results. Use single quotation marks for Linux or macOS and double quotation marks for Windows. The matching pattern applies to the absolute file path (including the file name and file directory).
include	Optional for uploading a folder or multiple files/folders (additional parameter)	The file matching patterns that are included. After this parameter is specified, if the name of the file to be uploaded matches the value of this parameter, the file is uploaded. For example, if this parameter is set to .txt, all files whose names end with .txt* are uploaded. An asterisk () matches any number of characters. For instance, abc.txt matches all files whose names start with abc, contain any number of characters in the middle, and end with .txt. A question mark (?) matches any single character. For instance, abc?.txt matches all files whose names start with abc, contain any single character in the middle, and end with .txt*. You can use \ to represent * and \? to represent ?. Multiple include parameters can be specified, for example, *-include=.xxx -include=.xxx. Restrictions: The matching pattern takes effect only for files in the folder. If both the exclude* and include parameters are configured, the exclude rules are applied first. If the name of a file to be uploaded does not match exclude, OBS checks whether the file name matches include. If yes, the file is uploaded. If no, the file is skipped. Example: ./obsutil cp /localpath/ obs://test/ -include=/localpath/2022-12-09/* -f -r This command uploads files that are under localpath and start with /localpath/2022-12-09/ to bucket test. NOTE: You are advised to use quotation marks for the matching pattern to prevent special characters from being escaped by the OS and leading to unexpected results. Use single quotation marks for Linux or macOS and double quotation marks for Windows. The matching pattern applies to the absolute file path (including the file name and file directory).
at	Optional for uploading a folder or multiple files/folders (additional parameter)	Indicates that only the files whose latest access time is within the value of timeRange are uploaded. This parameter must be used together with timeRange.
disableDirObject	Optional for uploading multiple folders (additional parameter)	Indicates the folders themselves are not uploaded as an object. Configuring this parameter can avoid uploading empty folders to a bucket. If a folder contains files, the files will be uploaded and the original path format is retained.
timeRange	Optional for uploading a folder or multiple files/folders (additional parameter)	The time range matching pattern when uploading files. Only files whose latest modification time is within the configured time range are uploaded. This pattern has a lower priority than the file matching patterns (exclude/include). That is, the time range matching pattern is executed after the configured file matching patterns. Time in the matching pattern is the UTC time. The matching time range is represented in time1-time2, where time1 must be earlier than or the same as time2. The time format is yyyyMMddHHmmss. Automatic formatting is supported. For example, yyyyMMdd is equivalent to yyyyMMdd000000, and yyyyMM is equivalent to yyyyMM01000000. If this parameter is set to -time2, all files whose last modification time is earlier than time2* are matched. If it is set to time1-, all files whose last modification time is later than time1* are matched.
mf	Optional (additional parameter)	Indicates that the name matching pattern (include or exclude) and the time matching pattern (timeRange) also take effect on folders.
o	Optional (additional parameter)	The folder that stores the result files. After the command is executed, result files (possibly success, failure, and warning files) will be created in the specified folder. The default value is .obsutil_output, a subfolder in the user's home directory where obsutil commands are executed. A result file should be named as follows: cp_{succeed \| failed \| warning}_report_time_TaskId.txt. By default, the maximum size of a single result file is 30 MB. You can set the maximum size by configuring recordMaxLogSize in the configuration file. By default, the maximum number of result files that can be retained is 1,024. You can set the maximum number by configuring recordBackups in the configuration file. If there are multiple folders and files and you need to confirm the details of a failed task, refer to the failure result file cp_failed_report_time_TaskId.txt in the result folder and the log files in the log path.
config	Optional (additional parameter)	The user-defined configuration file for executing the current command. To learn the parameters that can be configured in this file, see Configuration Parameters.
e	Optional (additional parameter)	The endpoint
i	Optional (additional parameter)	The user's AK
k	Optional (additional parameter)	The user's SK
t	Optional (additional parameter)	The user's security token

Response

Field	Description
Parallel	The parameter -p in the request
Jobs	The parameter -j in the request
Threshold	The parameter -threshold in the request
PartSize	The parameter -ps in the request
Exclude	The parameter -exclude in the request
Include	The parameter -include in the request
TimeRange	The parameter -timeRange in the request
VerifyLength	The parameter -vlength in the request
VerifyMd5	The parameter -vmd5 in the request
CheckpointDir	The parameter -cpd in the request
OutputDir	The parameter -o in the request
ArcDir	The parameter -arcDir in the request
Succeed count	The number of successful tasks
Failed count	The number of failed tasks
Skip count	The number of tasks that are skipped during incremental upload, download, or copy, and synchronous upload, download, or copy. NOTE: Skipped tasks are recorded into successful tasks.
Warning count	The number of tasks that are executed successfully but contain warnings. NOTE: The task for which a warning is generated may be a failure or a success, which needs to be further determined according to the corresponding result list. The number of tasks that generate warnings is independent of the number of successful or failed tasks. The total number of tasks is the number of successful tasks plus the number of failed tasks.
Succeed bytes	The number of bytes that are successfully uploaded or downloaded.
max cost	The maximum duration of all tasks, in ms
min cost	The minimum duration of all tasks, in ms
average cost	The average duration of all tasks, in ms
average tps	The average number of tasks completed per second
Task id	The unique ID of an operation, which is used to search for the result file generated for a batch task

Related Operations

The following are operations related to multipart upload:

Listing Initiated Multipart Uploads in a Bucket

An initiated multipart upload refers to an upload that has been initiated but not yet completed or aborted.

Principle: The ListMultipartUploads API returns all multipart uploads that have been initiated but not yet completed or aborted in a bucket. Each request can return a maximum of 1,000 multipart uploads. If more than 1,000 multipart uploads are in progress, you need to send additional requests to query the remaining uploads.

How to list initiated multipart uploads:

obsutil: Listing Multipart Upload Tasks
API: Listing Initiated Multipart Uploads in a Bucket
SDKs:

Listing Uploaded Parts That Have Not Been Assembled

Uploaded parts are the parts that have been uploaded to a bucket but have not yet been assembled.

Principle: OBS lists the uploaded parts that have not yet been assembled. Each request can return up to 1,000 parts. If there are more than 1,000 parts, you must send a series of requests to list all parts. The list of uploaded parts does not include assembled parts.

How to list uploaded parts:

OBS Console:
1. In the navigation pane of OBS Console, choose Buckets.
2. Click the name of the target bucket to go to the Objects page.
3. Click the Fragments tab to view all parts in the bucket.
OBS Browser+: Managing Fragments
API: Listing Uploaded Parts That Have Not Been Assembled
SDKs:

Copying Parts

After initiating a multipart upload, you can specify the upload ID and make a CopyPart API call to copy part or all of an uploaded object as a part.

Principle: If part1 already exists before you copy object A to part1, the new part1 will overwrite the original one. After the copy succeeds, only the new part1 can be listed, and the original part1 will be deleted. Therefore, before copying a part, ensure that the target part does not exist or is no longer needed. Otherwise, data may be deleted by mistake. Object A is not affected during the copy.

How to copy parts:

obsutil: Copying an Object
API: Copying Parts
SDKs:

If the response body contains the ETag value of the part, the part is successfully copied, or the part copy fails. A status code 200 does not indicate a successful copy; it simply means that the server has received the copy request and started to process it.

Aborting a Multipart Upload

Principle: After a multipart upload is aborted, the upload ID cannot be used to upload any part, and OBS will free up the storage space of each part. If any parts are being uploaded when you abort the upload, they may or may not be uploaded successfully. To ensure that the storage space used by all parts is freed up, you are advised to abort the multipart upload after all parts are uploaded.

How to abort a multipart upload: