Performing a Resumable Upload

If you have any questions during development, post them on the Issues page of GitHub.

Uploading large files often fails due to poor network conditions or program breakdowns. It is a waste of resources to restart the upload process upon an upload failure, and the restarted upload process may still suffer from the unstable network. To resolve such issues, you can use the API for resumable upload, whose working principle is to divide the to-be-uploaded file into multiple parts and upload them separately. The upload result of each part is recorded in a checkpoint file in real time. Only when all parts are successfully uploaded, the result indicating a successful upload is returned. Otherwise, an exception is thrown to remind you of calling the API again for re-uploading. Based on the upload status of each part recorded in the checkpoint file, the re-uploading will upload the parts failed to be uploaded previously, instead of uploading all parts. By virtue of this, resources are saved and efficiency is improved.

You can call upload_file to perform a resumable upload. The following table describes the parameters involved in this API.

The following table describes the structure of obs_upload_file_configuration.

void uploadFileResultCallback(obs_status status,
                              char *resultMsg,
                              int partCountReturn,
                              obs_upload_file_part_info * uploadInfoList,
                              void *callbackData);
//Declare the callback function.
static void test_upload_file()
{
    obs_status ret_status = OBS_STATUS_BUTT;
    // Create and initialize option.
    obs_options option;
    init_obs_options(&option);
    option.bucket_options.host_name = "<your-endpoint>";
    option.bucket_options.bucket_name = "<Your bucketname>";

    // Hard-coded or plaintext AK/SK are risky. For security purposes, encrypt your AK/SK and store them in the configuration file or environment variables. In this example, the AK/SK are stored in environment variables for identity authentication. Before running this example, configure environment variables ACCESS_KEY_ID and SECRET_ACCESS_KEY.
    // Obtain an AK/SK pair on the management console. For details, see https://support.huaweicloud.com/eu/usermanual-ca/ca_01_0003.html.
    option.bucket_options.access_key = getenv("ACCESS_KEY_ID");
    option.bucket_options.secret_access_key = getenv("SECRET_ACCESS_KEY");
    // Initialize the put_properties structure.
    obs_put_properties put_properties;
    init_put_properties(&put_properties);

    obs_upload_file_configuration uploadFileInfo;
    memset_s(&uploadFileInfo,sizeof(obs_upload_file_configuration),0,sizeof(obs_upload_file_configuration));
    uploadFileInfo.check_point_file = 0;
    uploadFileInfo.enable_check_point = 1;
    uploadFileInfo.part_size = "<part size>";
    uploadFileInfo.task_num = "<task num>";
    uploadFileInfo.upload_file = "<upload filename>";
    uploadFileInfo.put_properties = &put_properties;  

    // Callback function
    obs_upload_file_response_handler Handler =
    { 
        {&response_properties_callback, &response_complete_callback_for_multi_task},
        &uploadFileResultCallback
    };    
    initialize_break_point_lock();
    upload_file(&option, "<Your Key>", 0, &uploadFileInfo, Null, &Handler, &ret_status);
    deinitialize_break_point_lock();
    if (OBS_STATUS_OK == ret_status) {
        printf("test upload file successfully. \n");
    }
    else
    {
        printf("test upload file faied(%s).\n", obs_get_status_name(ret_status));
    }
}
//uploadFileResultCallback is used as an example here. The printf statements can be replaced with custom logging print statements.
void uploadFileResultCallback(obs_status status,
                              char *resultMsg,
                              int partCountReturn,
                              obs_upload_file_part_info * uploadInfoList,
                              void *callbackData)
{
    int i=0;
    obs_upload_file_part_info * pstUploadInfoList = uploadInfoList;
    printf("status return is %d(%s)\n",status,obs_get_status_name(status));
    printf("%s",resultMsg);
    printf("partCount[%d]\n",partCountReturn);
    for(i=0;i<partCountReturn;i++)
    {
        printf("partNum[%d],startByte[%llu],partSize[%llu],status[%d]\n",
        pstUploadInfoList->part_num,
        pstUploadInfoList->start_byte,
        pstUploadInfoList->part_size,
        pstUploadInfoList->status_return);
        pstUploadInfoList++;
    }
	if (callbackData) {
		obs_status* retStatus = (obs_status*)callbackData;
		(*retStatus) = status;
	}
}

• The API for resumable upload, which is implemented based on Performing a Multipart Upload, is an encapsulated and enhanced version of multipart upload.
• This API saves resources and improves efficiency upon the re-upload, and speeds up the upload process by concurrently uploading parts. Because this API is transparent to users, users are free from concerns about internal service details, such as the creation and deletion of checkpoint files, division of objects, and concurrent upload of parts.
• enable_check_point: The value 0 indicates that the resumable upload mode is disabled. In this case, the resumable upload API functions as an encapsulated version of multipart upload and does not generate checkpoint files. The value 1 indicates that the resumable upload mode is enabled.