Help Center/ Object Storage Service/ User Guide/ Object Management/ Downloading an Object
Updated on 2024-10-24 GMT+08:00
View PDF
Share

Downloading an Object

Scenarios

You can download objects from OBS to your local computer.

OBS supports batch download of objects. For details, see Does OBS Support Batch Download?

Limitations and Constraints

Ways to Download an Object

You can use OBS Console, APIs, SDKs, OBS Browser+, or obsutil to download an object.

Using OBS Console

  1. In the navigation pane of OBS Console, choose Object Storage.
  2. In the bucket list, click the bucket you want to operate. The Objects page is displayed.
  3. Select the file you want to download. Then, click Download or More > Download As on the right.

    You can also select multiple files and choose More > Download above the file list.

    In the Download As dialog box, right-click the object and choose Copy Link Address from the shortcut menu to obtain the object's download address.

Using the API

Downloading an Object

Using SDKs

Java

Python

C

Go

BrowserJS

.NET

Android

iOS

PHP

Node.js

Using the GUI Tool - OBS Browser+

  1. Log in to OBS Browser+.
  2. Go to the target bucket, select the file or folder you want to download, and click Download.

    You can select multiple files or folders and batch download them.

    • To download Archive files, you must restore them first. For details, see Restoring an Object.
    • Only files or folders that have been listed can be downloaded in batches. You can drag the scroll bar to the bottom to update the number of listed files or folders. 100 files or folders can be updated at a time.

  3. In the displayed dialog box, select a path for saving the file or folder and click Select Folder.

Using the CLI Tool - obsutil

Command Line Structure

  • In Windows
    • Downloading a single object 
      obsutil cp obs://bucket/key file_or_folder_url [-tempFileDir=xxx] [-dryRun] [-u] [-vlength] [-vmd5] [-p=1] [-threshold=52428800] [-versionId=xxx] [-ps=auto] [-cpd=xxx][-fr] [-o=xxx] [-config=xxx] [-e=xxx] [-i=xxx] [-k=xxx] [-t=xxx]
    • Downloading objects in batches 
      obsutil cp obs://bucket[/key] folder_url -r [-tempFileDir=xxx] [-dryRun] [-f] [-flat] [-u] [-vlength] [-vmd5] [-j=1] [-p=1] [-threshold=52428800] [-ps=auto] [-include=*.xxx] [-exclude=*.xxx] [-timeRange=time1-time2] [-mf] [-o=xxx] [-cpd=xxx] [-config=xxx] [-e=xxx] [-i=xxx] [-k=xxx] [-t=xxx]
  • In Linux or macOS
    • Downloading a single object 
      ./obsutil cp obs://bucket/key file_or_folder_url [-tempFileDir=xxx] [-dryRun] [-u] [-vlength] [-vmd5] [-p=1] [-threshold=52428800] [-versionId=xxx] [-ps=auto] [-cpd=xxx] [-fr] [-o=xxx] [-config=xxx] [-e=xxx] [-i=xxx] [-k=xxx] [-t=xxx]
    • Downloading objects in batches 
      ./obsutil cp obs://bucket[/key] folder_url -r [-tempFileDir=xxx] [-dryRun] [-f] [-flat] [-u] [-vlength] [-vmd5] [-j=1] [-p=1] [-threshold=52428800] [-ps=auto] [-include=*.xxx] [-exclude=*.xxx] [-timeRange=time1-time2] [-mf] [-o=xxx] [-cpd=xxx] [-config=xxx] [-e=xxx] [-i=xxx] [-k=xxx] [-t=xxx]

Examples

  • Take the Windows OS as an example. Run the obsutil cp obs://bucket-test/key d:\temp\test.txt command to download a single object. 
    obsutil cp obs://bucket-test/key d:\temp\test.txt

Parallel:      3                   Jobs:          3
Threshold:     524288000           PartSize:      5242880
Exclude:                           Include:
VerifyLength:  false               VerifyMd5:     false
CheckpointDir: xxxx

[==========================================] 100.00% 4.86 KB/s 8.46MB/8.46MB 0s
Download successfully, 8.46MB, obs://bucket-test/key --> d:\temp\test.txt
  • Take the Windows OS as an example. Run the obsutil cp obs://bucket-test/temp d:\ -f -r command to download objects in batches. 
    obsutil cp obs://bucket-test/temp d:\ -f -r

Parallel:      3                   Jobs:          3
Threshold:     524288000           PartSize:      5242880
Exclude:                           Include:
VerifyLength:  false               VerifyMd5:     false
CheckpointDir: xxxx

Task id: 3066a4b0-4d21-4929-bb84-4829c32cbd0f
OutputDir: xxxx

[======================================================] 100.00% tps:17.86 155.59 KB/s 7.20MB/7.20MB 0s
Succeed count is:   6         Failed count is:    0
Metrics [max cost:153 ms, min cost:129 ms, average cost:92.00 ms, average tps:17.86]

Task id: 3066a4b0-4d21-4929-bb84-4829c32cbd0f

Parameter Description

Parameter

Optional or Mandatory

Description

file_or_folder_url

Mandatory for downloading an object

Local file/folder path

folder_url

Mandatory for downloading objects in batches

Local folder path

bucket

Mandatory

Bucket name

key

Mandatory for downloading an object

Optional for downloading objects in a batch

Indicates the name of the object to be downloaded, or the name prefix of the objects to be downloaded in batches.

This parameter cannot be left blank when downloading an object. The saving and naming rules are as follows:

  • If this parameter specifies a file or folder path that does not exist, the tool checks whether the value ends with a slash (/) or backslash (\). If yes, a folder is created based on the path, and the object is downloaded to this newly created directory.
  • If this parameter specifies a file or folder path that does not exist and the value does not end with a slash (/) or backslash (\), the object is downloaded to your local PC with the value of key as the file name.
  • If this parameter specifies an existing file, the object is downloaded to your local PC overwriting the existing file, with the value of key as the file name.
  • If this parameter specifies an existing folder, the object is downloaded to the directory specified by file_or_folder_url with the object name as the file name.

During batch download, the saving rules are as follows:

  • If this parameter is left blank, all objects in the bucket are downloaded to the directory specified by folder_url.
  • If this parameter is configured, objects whose name prefix is the configured value in the bucket are downloaded to the directory specified by folder_url.
NOTE:
  • If this parameter is configured but the flat parameter is not configured when downloading objects in a batch, the name of the downloaded file contains the name prefix of the parent object. If flat is configured, then the name of the downloaded file does not contain the name prefix of the parent object.
  • For details about how to use this parameter, see Download.

fr

Optional for downloading an object (additional parameter)

Generates an operation result list when downloading an object.

flat

Optional for downloading objects in batches (additional parameter)

The name prefix of the parent object is excluded when downloading objects in batches.

tempFileDir

Optional (additional parameter)

Indicates the directory for storing temporary files during multipart download. The default value is the value of defaultTempFileDir in the configuration file.

NOTE:
  • If this parameter is left blank and the defaultTempFileDir parameter in the configuration file is also left blank, temporary files generated during multipart download are saved in the directory where to-be-downloaded files are located and end with the suffix of .obs.temp.
  • Temporary files generated during multipart download are stored in this directory. Therefore, ensure that the user who executes obsutil has the write permission on the path.
  • The available space of the partition where the path is located must be greater than the size of the objects to be downloaded.

dryRun

Optional (additional parameter)

Conducts a dry run.

u

Optional (additional parameter)

Indicates incremental download. If this parameter is set, each object can be downloaded only when it does not exist in the local path, its size is different from the namesake one in the local path, or it has the latest modification time.

vlength

Optional (additional parameter)

Checks whether the sizes of the local files are the same as those of the objects in the bucket after the download is complete.

vmd5

Optional (additional parameter)

Checks whether MD5 values of the local files are the same as those of the objects in the bucket after the download is complete.

NOTE:

Objects in the bucket must contain metadata x-obs-meta-md5chksum, or MD5 verification will be skipped.

p

Optional (additional parameter)

Indicates the maximum number of concurrent multipart download tasks when downloading an object. The default value is the value of defaultParallels in the configuration file.

threshold

Optional (additional parameter)

Indicates the threshold for enabling multipart download, in bytes. The default value is the value of defaultBigfileThreshold in the configuration file.

NOTE:
  • If the size of the object to be downloaded is smaller than the threshold, download the object directly. If not, a multipart download is required.
  • If you download an object directly, no part record is generated, and resumable transmission is not supported.
  • This parameter value can contain a unit, for example, 1MB (indicating 1048576 bytes).

versionId

Optional for downloading an object (additional parameter)

Source object version ID that can be specified when downloading an object

ps

Optional (additional parameter)

Indicates the size of each part in a multipart download task, in bytes. The default value is the value of defaultPartSize in the configuration file.

NOTE:
  • This parameter value can contain a unit, for example, 1MB (indicating 1048576 bytes).
  • The parameter can be set to auto. In this case, obsutil automatically sets the part size for each multipart task based on the source object size.

fs

Optional (additional parameter)

Specifies whether the method of listing parallel file systems is applied. If you are listing parallel file systems, you are recommended to add this parameter.

CAUTION:
  • With this method, the listing time required varies largely depending on the directory structures.
  • After this parameter is enabled, marker and limit will be ignored. Then, the size of the bucket or parallel file system will be calculated.
  • This parameter is only supported by obsutil 5.5.12 and later.

cpd

Optional (additional parameter)

Indicates the folder where the part records reside. The default value is .obsutil_checkpoint, the subfolder in the home directory of the user who executes obsutil commands.

NOTE:

A part record is generated during a multipart download and saved to the down subfolder. After the download succeeds, its part record is deleted automatically. If the download fails or is suspended, the system attempts to resume the task according to its part record when you perform the download the next time.

r

Mandatory for downloading objects in batches (additional parameter)

Copies objects in batches based on a specified object name prefix.

f

Optional for downloading objects in batches (additional parameter)

Runs in force mode.

j

Optional for downloading objects in batches (additional parameter)

Indicates the maximum number of concurrent tasks for downloading objects in a batch. The default value is the value of defaultJobs in the configuration file.

NOTE:

The value is ensured to be greater than or equal to 1.

exclude

Optional for downloading objects in batches (additional parameter)

Indicates the matching patterns of source objects that are excluded, for example, *.txt.

NOTE:
  • The asterisk (*) represents any group of characters, and the question mark (?) represents any single character. For instance, abc*.txt indicates any file whose name starts with abc and ends with .txt.
  • You can use \* to represent * and \? to represent ?.
  • If the name of the object to be downloaded matches the value of this parameter, the object is skipped.
NOTICE:
  • You are advised to use quotation marks for the matching pattern to prevent special characters from being escaped by the OS and leading to unexpected results. Use single quotation marks for Linux or macOS and quotation marks for Windows.
  • The matching pattern applies to the absolute path of an object, including the object name prefix and object name starting from the root directory. For example, if the path of an object in the bucket is obs://bucket/src1/src2/test.txt, then the absolute path of the object is src1/src2/test.txt.
  • This matching pattern applies only to objects whose names do not end with a slash (/).
  • Multiple exclude parameters can be specified, for example, -exclude=*.xxx -exclude=*.xxx.

include

Optional for downloading objects in batches (additional parameter)

Specifies what formats of objects can be downloaded. If this parameter is set to *.jpg, only objects in the .jpg format can be downloaded.

NOTE:
  • The asterisk (*) represents any group of characters, and the question mark (?) represents any single character.
  • You can use \* to represent * and \? to represent ?.
  • Only after identifying that the name of the file to be downloaded does not match the value of exclude, the system checks whether the file name matches the value of this parameter. If yes, the file is downloaded. If not, the file is skipped.
NOTICE:
  • You are advised to use quotation marks for the matching pattern to prevent special characters from being escaped by the OS and leading to unexpected results. Use single quotation marks for Linux or macOS and quotation marks for Windows.
  • The matching pattern applies to the absolute path of an object, including the object name prefix and object name starting from the root directory. For example, if the path of an object in the bucket is obs://bucket/src1/src2/test.txt, then the absolute path of the object is src1/src2/test.txt.
  • This matching pattern applies only to objects whose names do not end with a slash (/).
  • Multiple include parameters can be specified, for example, -include=*.xxx -include=*.xxx.

timeRange

Optional for downloading objects in batches (additional parameter)

Indicates the time range matching pattern when downloading objects. Only objects whose latest modification time is within the configured time range are downloaded.

This pattern has a lower priority than the object matching patterns (exclude/include). That is, the time range matching pattern is executed after the configured object matching patterns.

NOTE:
  • The matching time range is represented in time1-time2, where time1 must be earlier than or the same as time2. The time format is yyyyMMddHHmmss.
  • Automatic formatting is supported. For example, yyyyMMdd is equivalent to yyyyMMdd000000, and yyyyMM is equivalent to yyyyMM01000000.
  • If this parameter is set to *-time2, all files whose latest modification time is earlier than time2 are matched. If it is set to time1-*, all files whose latest modification time is later than time1 are matched.
NOTICE:
  • Time in the matching pattern is the UTC time.
  • This matching pattern applies only to objects whose names do not end with a slash (/).

mf

Optional (additional parameter)

Indicates that the name matching pattern (include or exclude) and the time matching pattern (timeRange) also take effect on objects whose names end with a slash (/).

o

Optional (additional parameter)

Indicates the folder where operation result lists reside. After the command is executed, result lists (possibly including success, failure, and warning files) are generated in the folder. The default value is .obsutil_output, the subfolder in the home directory of the user who executes obsutil commands.

NOTE:
  • The naming rule for result lists is as follows: cp_{succeed | failed | warning}_report_time_TaskId.txt
  • By default, the maximum size of a single result list is 30 MB and the maximum number of result lists that can be retained is 1024. You can set the maximum size and number by configuring recordMaxLogSize and recordBackups in the configuration file.
  • If there are multiple folders and files and you need to confirm the detailed error information about a failed task, refer to the failure list cp_failed_report_time_TaskId.txt in the result list folder and the log files in the log path.

config

Optional (additional parameter)

User-defined configuration file for executing the current command. To learn the parameters that can be configured in this file, see Configuration Parameters.

e

Optional (additional parameter)

Specifies the endpoint.

i

Optional (additional parameter)

Specifies the user's AK.

k

Optional (additional parameter)

Specifies the user's SK.

t

Optional (additional parameter)

Specifies the user's security token.

Response

Field

Description

Parallel

Parameter -p in the request

Jobs

Parameter -j in the request

Threshold

Parameter -threshold in the request

PartSize

Parameter -ps in the request

Exclude

Parameter -exclude in the request

Include

Parameter -include in the request

TimeRange

Parameter -timeRange in the request

VerifyLength

Parameter -vlength in the request

VerifyMd5

Parameter -vmd5 in the request

CheckpointDir

Parameter -cpd in the request

OutputDir

Parameter -o in the request

ArcDir

Parameter -arcDir in the request

Succeed count

Number of successful tasks

Failed count

Number of failed tasks

Skip count

Number of tasks that are skipped during incremental upload, download, or copy, and synchronous upload, download, or copy.

NOTE:

Skipped tasks are recorded into successful tasks.

Warning count

Number of tasks that are executed successfully but contain warnings.

NOTE:
  • The task for which a warning is generated may be a failure or a success, which needs to be further determined according to the corresponding result list.
  • The number of tasks that generate warnings is independent of the number of successful or failed tasks. The total number of tasks is the number of successful tasks plus the number of failed tasks.

Succeed bytes

Number of bytes that are successfully uploaded or downloaded.

max cost

Maximum duration of all tasks, in ms

min cost

Minimum duration of all tasks, in ms

average cost

Average duration of all tasks, in ms

average tps

The average number of tasks completed per second

Task id

Unique ID of an operation, which is used to search for the result list generated in a batch task

Parent Topic: Object Management