Resuming a Failed Download Task

Updated on 2025-02-10 GMT+08:00

View PDF

Function

You can use this command to resume a failed download task based on the task ID.

Command Line Structure

In Windows

obsutil cp -recover=xxx [-dryRun] [-tempFileDir=xxx] [-f] [-u] [-vlength] [-vmd5] [-j=1] [-p=1] [-threshold=52428800] [-ps=auto] [-include=*.xxx] [-exclude=*.xxx] [-timeRange=time1-time2] [-mf] [-o=xxx] [-cpd=xxx] [-clear] [-config=xxx]

In Linux or macOS

./obsutil cp -recover=xxx [-dryRun] [-tempFileDir=xxx] [-f] [-u] [-vlength] [-vmd5] [-j=1] [-p=1] [-threshold=52428800] [-ps=auto] [-include=*.xxx] [-exclude=*.xxx] [-timeRange=time1-time2] [-mf] [-o=xxx] [-cpd=xxx] [-clear] [-config=xxx]

Examples

Take the Windows OS as an example. Run the obsutil cp -recover=3066a4b0-4d21-4929-bb84-4829c32cbd0f d:\ -f -r command to download objects in batches.

obsutil cp -recover=3066a4b0-4d21-4929-bb84-4829c32cbd0f -f -r
Start at 2024-10-08 01:10:07.3809685 +0000 UTC

Parallel:      5                   Jobs:          5
Threshold:     50.00MB             PartSize:      auto
VerifyLength:  false               VerifyMd5:     false
CheckpointDir: xxxx

Task id: a628d6da-c562-4a1f-b687-4fa125de0dc3
OutputDir: xxxx
TempFileDir: xxxx

[========================================================] 100.00% tps:35.71 2.02 KB/s 7.20MB/7.20MB 0s
Succeed count:      1         Failed count:       0
Succeed bytes:      xxx
Metrics [max cost:90 ms, min cost:45 ms, average cost:63.80 ms, average tps:35.71, transferred size:70B]

Task id: a628d6da-c562-4a1f-b687-4fa125de0dc3

Parameter Description

Parameter	Optional or Mandatory	Description
recover	Mandatory (additional parameter)	ID of the download task to be resumed NOTE: You can obtain the task ID after a download task is complete, or query it based on the name of the result list. A task ID is the 36 characters, excluding the suffix .txt. You can locate the download task to be resumed in the directory where the result lists reside. For details about the directory of the result lists, see additional parameter o.
tempFileDir	Optional (additional parameter)	Indicates the directory for storing temporary files during download. The default value is the value of defaultTempFileDir in the configuration file. NOTE: Temporary files generated during multipart download are stored in this directory. Therefore, ensure that the user who executes obsutil has the write permission on the path. The available space of the partition where the path is located must be greater than the size of the objects to be downloaded.
dryRun	Optional (additional parameter)	Conducts a dry run.
u	Optional (additional parameter)	Indicates incremental download. If this parameter is set, each object can be downloaded only when it does not exist in the local path, its size is different from the namesake one in the local path, or it has the latest modification time.
vlength	Optional (additional parameter)	Checks whether the sizes of the local files are the same as those of the objects in the bucket after the download is complete.
vmd5	Optional (additional parameter)	Checks whether MD5 values of the local files are the same as those of the objects in the bucket after the download is complete. NOTE: Objects in the bucket must contain metadata x-obs-meta-md5chksum, or MD5 verification will be skipped. CAUTION: If your object needs encryption, do not use this parameter.
p	Optional (additional parameter)	Indicates the maximum number of concurrent multipart download tasks when downloading an object. The default value is the value of defaultParallels in the configuration file.
threshold	Optional (additional parameter)	Indicates the threshold for enabling multipart download, in bytes. The default value is the value of defaultBigfileThreshold in the configuration file. NOTE: If the size of the object to be downloaded is smaller than the threshold, download the object directly. If not, a multipart download is required. If you download an object directly, no part record is generated, and resumable transmission is not supported. This parameter value can contain a unit, for example, 1MB (indicating 1048576 bytes).
ps	Optional (additional parameter)	Indicates the size of each part in a multipart download task, in bytes. The default value is the value of defaultPartSize in the configuration file. NOTE: This parameter value can contain a unit, for example, 1MB (indicating 1048576 bytes). The parameter can be set to auto. In this case, obsutil automatically sets the part size for each multipart task based on the source object size.
cpd	Optional (additional parameter)	Indicates the folder where the part records reside. The default value is .obsutil_checkpoint, the subfolder in the home directory of the user who executes obsutil commands. NOTE: A part record is generated during a multipart download and saved to the down subfolder. After the download succeeds, its part record is deleted automatically. If the download fails or is suspended, the system attempts to resume the task according to its part record when you perform the download the next time.
f	Optional (additional parameter)	Runs in force mode.
j	Optional (additional parameter)	Indicates the maximum number of concurrent tasks for downloading objects in batches. The default value is the value of defaultJobs in the configuration file. NOTE: The value is ensured to be greater than or equal to 1.
exclude	Optional (additional parameter)	Indicates the matching patterns of source objects that are excluded, for example: .txt. NOTE: The asterisk () represents any group of characters, and the question mark (?) represents any single character. For instance, *abc.txt indicates any file whose name starts with abc and ends with .txt*. You can use \ to represent * and \? to represent ?. If the name of the object to be downloaded matches the value of this parameter, the object is skipped. NOTICE: You are advised to use quotation marks for the matching pattern to prevent special characters from being escaped by the OS and leading to unexpected results. Use single quotation marks for Linux or macOS and quotation marks for Windows. The matching pattern applies to the absolute path of an object, including the object name prefix and object name starting from the root directory. For example, if the path of an object in the bucket is obs://bucket/src1/src2/test.txt, then the absolute path of the object is src1/src2/test.txt. This matching pattern applies only to objects whose names do not end with a slash (/). Multiple exclude parameters can be specified, for example, *-exclude=.xxx -exclude=.xxx*.
include	Optional (additional parameter)	Indicates the matching patterns of source objects that are included, for example: .jpg. NOTE: The asterisk () represents any group of characters, and the question mark (?) represents any single character. You can use \* to represent * and \? to represent ?. Only after identifying that the name of the file to be downloaded does not match the value of exclude, the system checks whether the file name matches the value of this parameter. If yes, the file is downloaded. If not, the file is skipped. NOTICE: You are advised to use quotation marks for the matching pattern to prevent special characters from being escaped by the OS and leading to unexpected results. Use single quotation marks for Linux or macOS and quotation marks for Windows. The matching pattern applies to the absolute path of an object, including the object name prefix and object name starting from the root directory. For example, if the path of an object in the bucket is obs://bucket/src1/src2/test.txt, then the absolute path of the object is src1/src2/test.txt. This matching pattern applies only to objects whose names do not end with a slash (/). Multiple include parameters can be specified, for example, *-include=.xxx -include=.xxx*.
timeRange	Optional (additional parameter)	Indicates the time range matching pattern when downloading objects. Only objects whose latest modification time is within the configured time range are downloaded. This pattern has a lower priority than the object matching patterns (exclude/include). That is, the time range matching pattern is executed after the configured object matching patterns. NOTE: The matching time range is represented in time1-time2, where time1 must be earlier than or the same as time2. The time format is yyyyMMddHHmmss. Automatic formatting is supported. For example, yyyyMMdd is equivalent to yyyyMMdd000000, and yyyyMM is equivalent to yyyyMM01000000. If this parameter is set to -time2, all files whose latest modification time is earlier than time2* are matched. If it is set to time1-, all files whose latest modification time is later than time1* are matched. NOTICE: Time in the matching pattern is the UTC time. This matching pattern applies only to objects whose names do not end with a slash (/).
mf	Optional (additional parameter)	Indicates that the name matching pattern (include or exclude) and the time matching pattern (timeRange) also take effect on objects whose names end with a slash (/).
o	Optional (additional parameter)	Indicates the folder where operation result lists reside. After the command is executed, result lists (possibly including success, failure, and warning files) are generated in the folder. The default value is .obsutil_output, the subfolder in the home directory of the user who executes obsutil commands. NOTE: The naming rule for result lists is as follows: cp_{succeed \| failed \| warning}_report_time_TaskId.txt. By default, the maximum size of a single result list is 30 MB and the maximum number of result lists that can be retained is 1024. You can set the maximum size and number by configuring recordMaxLogSize and recordBackups in the configuration file. If there are multiple folders and files and you need to confirm the detailed error information about a failed task, refer to the failure list cp_failed_report_time_TaskId.txt in the result list folder and the log files in the log path.
clear	Optional (additional parameter)	Deletes the failure result files after the download task is resumed.
config	Optional (additional parameter)	User-defined configuration file for executing the current command. To learn the parameters that can be configured in this file, see .

Response

Refer to Response for uploading an object.

Parent topic: Object Commands

Previous topic: Resuming a Failed Copy Task

Next topic: Listing Multipart Upload Tasks

Feedback

Was this page helpful?

Helpful Not helpful

Provide feedback

Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.

The system is busy. Please try again later.

Which of the following issues have you encountered?

Content is inconsistent with the product UI

Unclear descriptions

Lack of examples or code

Incorrect steps

Can't find what I need

Lack of best practices

Feedback (optional)

0/500

Select at least one type of issue, and enter your comments or suggestions.

Enter a maximum of 500 characters.

Submit Cancel

For any further questions, feel free to contact us through the chatbot.

Chatbot