Performing a Resumable Download

If you have any questions during development, post them on the Issues page of GitHub.

Downloading large files often fails due to poor network conditions or program breakdowns. It is a waste of resources to restart the download process upon a download failure, and the restarted download process may still suffer from the unstable network. To resolve such issues, you can use the API for resumable download, whose working principle is to divide the to-be-downloaded file into multiple parts and download them separately. The download result of each part is recorded in a checkpoint file in real time. Only when all parts are successfully downloaded, the result indicating a successful download is returned. Otherwise, an exception is thrown to remind you of calling the API again for re-downloading. Based on the download status of each part recorded in the checkpoint file, the re-downloading will download the parts failed to be downloaded previously, instead of downloading all parts. By virtue of this, resources are saved and efficiency is improved.

You can use download_file to perform a resumable download. The following table describes the parameters involved in this API.

The following table describes the structure of obs_download_file_configuration.

Sample code:

void downloadFileResultCallback(obs_status status,
                                char *resultMsg,
                                int partCountReturn,
                                obs_download_file_part_info * downloadInfoList,
                                void *callbackData);
//Declare the callback function.
static void test_download_file(char *filename, char *key)
{
    obs_status ret_status = OBS_STATUS_BUTT;
    uint64_t uploadSliceSize =5L * 1024 * 1024;                     
    // Create and initialize option.
    obs_options option;
    init_obs_options(&option);
    option.bucket_options.host_name = "<your-endpoint>";
    option.bucket_options.bucket_name = "<Your bucketname>";

    // Hard-coded or plaintext AK/SK are risky. For security purposes, encrypt your AK/SK and store them in the configuration file or environment variables. In this example, the AK/SK are stored in environment variables for identity authentication. Before running this example, configure environment variables ACCESS_KEY_ID and SECRET_ACCESS_KEY.
    // Obtain an AK/SK pair on the management console. For details, see https://support.huaweicloud.com/eu/usermanual-ca/ca_01_0003.html.
    option.bucket_options.access_key = getenv("ACCESS_KEY_ID");
    option.bucket_options.secret_access_key = getenv("SECRET_ACCESS_KEY");
    // Initialize getConditions.
    obs_get_conditions getConditions;
    memset_s(&getConditions,sizeof(obs_get_conditions),0,sizeof(obs_get_conditions));
    init_get_properties(&getConditions);
    // Resumable download object information
    obs_download_file_configuration downloadFileConfig;
    memset_s(&downloadFileConfig,sizeof(obs_download_file_configuration),0,
                    sizeof(obs_download_file_configuration));
    downloadFileConfig.check_point_file = NULL;
    downloadFileConfig.enable_check_point = 1;
    downloadFileConfig.part_size = uploadSliceSize;
    downloadFileConfig.task_num = 10;
    downloadFileConfig.downLoad_file= filename;
    // Set response callback function.
    obs_download_file_response_handler Handler =
    { 
        {&response_properties_callback, &response_complete_callback_for_multi_task},
        &downloadFileResultCallback
    };
    initialize_break_point_lock();
    download_file(&option, key, 0,&getConditions,0,&downloadFileConfig, 
            &Handler, &ret_status);
    deinitialize_break_point_lock();
    if (OBS_STATUS_OK == ret_status) {
        printf("test download file successfully. \n");
    }
    else
    {
        printf("test download file faied(%s).\n", obs_get_status_name(ret_status));
    }
}
//downloadFileResultCallback is used as an example here. The printf statements can be replaced with custom logging print statements.
void downloadFileResultCallback(obs_status status,
                                char *resultMsg,
                                int partCountReturn,
                                obs_download_file_part_info * downloadInfoList,
                                void *callbackData)
{
    int i=0;
    obs_download_file_part_info * pstDownloadInfoList = downloadInfoList;    
	printf("status return is %d(%s)\n", status, obs_get_status_name(status));
    printf("%s",resultMsg);
    printf("partCount[%d]\n",partCountReturn);
    for(i=0;i<partCountReturn;i++)
    {
        printf("partNum[%d],startByte[%llu],partSize[%llu],status[%d]\n",
        pstDownloadInfoList->part_num,
        pstDownloadInfoList->start_byte,
        pstDownloadInfoList->part_size,
        pstDownloadInfoList->status_return);
        pstDownloadInfoList++;
    }	
	if (callbackData) {
		obs_status* retStatus = (obs_status*)callbackData;
		(*retStatus) = status;
	}
}

• The API for resumable download, which is implemented based on Performing a Multipart Upload, is an encapsulated and enhanced version of partial download.
• This API saves resources and improves efficiency upon the re-download, and speeds up the download process by concurrently downloading parts. Because this API is transparent to users, users are free from concerns about internal service details, such as the creation and deletion of checkpoint files, division of objects, and concurrent download of parts.
• The default value of the enable_check_point parameter is 0, which indicates that the resumable download mode is disabled. In such cases, the API for resumable download degrades to the simple encapsulation of partial download, and no checkpoint file will be generated.
• check_point_file is valid only when enable_check_point is set to 1.
• Use the task_num parameter to configure how many concurrent tasks there can be in downloading a single object.