Downloading Objects - In a Batch (SDK for Python)
Function
This API downloads specified objects in batches.
Restrictions
- To download an object, you must be the bucket owner or have the required permission (obs:object:GetObject in IAM or GetObject in a bucket policy). For details, see Introduction to OBS Access Control, IAM Custom Policies, and Configuring an Object Policy.
- The mapping between OBS regions and endpoints must comply with what is listed in Regions and Endpoints.
- Objects in the Archive storage class can be downloaded only when they are restored.
Method
ObsClient.downloadFiles(bucketName, prefix, downloadFolder, taskNum, taskQueueSize, headers, imageProcess, interval, taskCallback, progressCallback, threshold, partSize, subTaskNum, enableCheckpoint, checkpointFile, extensionHeaders)
Request Parameters
| Parameter | Type | Mandatory (Yes/No) | Description |
|---|---|---|---|
| bucketName | str | Yes | Explanation: Bucket name Restrictions:
Default value: None |
| prefix | str | Yes | Explanation: Prefix of the objects to be downloaded in batches Assume that you download the following objects: logs/day1, logs/day2, logs/day3, and ExampleObject.jpg. If you specify logs/ as the prefix, logs/day1, logs/day2, and logs/day3 will be downloaded. If you leave this parameter blank, all objects in the bucket will be returned. Value range: The value must contain 1 to 1,024 characters. Default value: None |
| downloadFolder | str | Yes | Explanation: Full path of the local folder that stores the objects to be downloaded in batches Default value: None |
| taskNum | int | No | Explanation: Maximum number of concurrent downloads Value range: 1~10000 Default value: 8 |
| taskQueueSize | int | No | Explanation: Size of the queue in a batch download Value range: Default value: 20000 |
| headers | No | Explanation: Headers in the request used for obtaining the storage class, redundancy policy, and other basic information about the object Value range: See Table 2. Default value: None | |
| imageProcess | str | No | Explanation: Image processing command or style. Example for resizing and rotating an image: image/resize,m_fixed,w_100,h_100/rotate,90 Value range: Command format: image/commands Style format: style/style name For details about image processing parameters, see Processing Images. Default value: If no commands are entered, the original image will be returned. |
| interval | int | No | Explanation: Byte interval Value range: Default value: 102400 |
| taskCallback | callable | No | Explanation: Task callback function Default value: None |
| progressCallback | callable | No | Explanation: Callback function for obtaining the download progress Default value: None NOTE: This function contains the following parameters in sequence: number of downloaded bytes, total number of bytes, and used time (in seconds). For details about the sample code, see Downloading an Object - Obtaining the Download Progress (SDK for Python). |
| threshold | int | No | Explanation: Batch download threshold Default value: 5 * 1024 * 1024 * 1024 |
| partSize | int | No | Explanation: Part size Value range: The value ranges from 100 KB to 5 GB, in bytes. Default value: 5MB |
| subTaskNum | int | No | Explanation: Number of concurrent resumable downloads during a batch download Value range: 1 to 10000 Default value: 1, indicating concurrent downloads are not used. |
| enableCheckpoint | bool | No | Explanation: Whether to enable the resumable download mode Value range: True: The resumable download mode is enabled. False: The resumable download mode is disabled. Default value: False |
| checkpointFile | str | No | Explanation: File used to record the download progress. This parameter is valid only in resumable downloads. Default value: If the parameter is left blank, the file is in the same local directory as the downloaded object. |
| extensionHeaders | dict | No | Explanation: Extension headers. Value range: See User-defined Headers (SDK for Python). Default value: None |
| Parameter | Type | Mandatory (Yes/No) | Description |
|---|---|---|---|
| range | str | No | Explanation: Download range Value range: 0 to the object length minus 1. Format: x-y Restrictions: The upper limit of range is the length of the object minus 1. If the specified value exceeds this limit, the length of the object minus 1 is used. Default value: None |
| if_match | str | No | Explanation: Preset ETag. If the ETag of the object to be downloaded is the same as the preset ETag, the object is returned. Otherwise, an error is returned. Value range: The value must contain 32 characters. Default value: None |
| if_none_match | str | No | Explanation: Preset ETag. If the ETag of the object to be downloaded is different from the preset ETag, the object is returned. Otherwise, an error is returned. Value range: The value must contain 32 characters. Default value: None |
| if_modified_since | str or | No | Explanation: The object is returned if it has been modified since the specified time; otherwise, an error is returned. Restrictions: You can generate a date time by referring to Table 3. For example, DateTime(year=2023, month=9, day=12) Default value: None |
| if_unmodified_since | str or | No | Explanation: The object is returned if it has not been modified since the specified time; otherwise, an error is returned. Restrictions: You can generate a date time by referring to Table 3. For example, DateTime(year=2023, month=9, day=12) Default value: None |
| origin | str | No | Explanation: Origin of the cross-domain request specified by the preflight request. Generally, it is a domain name. Restrictions: Each origin can contain only one wildcard character (*). Default value: None |
| requestHeaders | str | No | Explanation: HTTP headers in a cross-origin request Only CORS requests matching the allowed headers are valid. Restrictions: Each header can contain only one wildcard character (*). Spaces, ampersands (&), colons (:), and less-than signs (<) are not allowed. Default value: None |
| sseHeader | No | Explanation: Server-side decryption headers. For details, see Table 4. Restrictions: If the object uploaded to the server is encrypted on the server using the encryption key provided by the client, downloading the object requires including the encryption key in the message. Default value: None |
| Parameter | Type | Description |
|---|---|---|
| year | int | Explanation: Year in UTC Default value: None |
| month | int | Explanation: Month in UTC Default value: None |
| day | int | Explanation: Day in UTC Default value: None |
| hour | int | Explanation: Hour in UTC Restrictions: The value is in 24-hour format. Default value: 0 |
| min | int | Explanation: Minute in UTC Default value: 0 |
| sec | int | Explanation: Second in UTC Default value: 0 |
| Parameter | Type | Mandatory (Yes/No) | Description |
|---|---|---|---|
| encryption | str | Yes | Explanation: SSE-C used for encrypting objects Value range: AES256 Default value: None |
| key | str | Yes | Explanation: Key used in SSE-C encryption. It corresponds to the encryption method. For example, if encryption is set to AES256, the key is calculated using the AES-256 algorithm. Value range: The value must contain 32 characters. Default value: None |
Responses
| Type | Description |
|---|---|
| Explanation: Result objects of the batch download API. Value range: N/A |
Code Examples
1 2 3 4 5 6 7 | try: resp = obs_client.downloadFiles(bucket_name='bucketname', prefix='object_key', downloadFolder='localFolder') result = "DownloadFiles summary : total_task:%d, success:%d ,failure:%d" % (resp.total_tasks, resp.successful_tasks, resp.failed_tasks) print(result) except Exception: import traceback print(traceback.format_exc()) |
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot