多段相关接口说明(C SDK)
开发过程中,您有任何问题可以在github上提交issue,或者在华为云对象存储服务论坛中发帖求助。
对于较大文件上传,可以切分成段上传。用户可以在如下的场景内(但不仅限于此)使用分段上传的模式:
- 上传超过100MB大小的文件。
- 网络条件较差,和OBS服务端之间的连接经常断开。
- 上传前无法确定将要上传文件的大小。
分段上传分为如下3个步骤:
分段上传的主要目的是解决大文件上传或网络条件较差的情况。下面的代码示例展示了如何使用分段上传并发上传大文件:
static void test_concurrent_upload_part(char *filename, char *key, uint64_t uploadSliceSize)
{
obs_status ret_status = OBS_STATUS_BUTT;
// 创建并初始化option
obs_options option;
init_obs_options(&option);
option.bucket_options.host_name = "<your-endpoint>";
option.bucket_options.bucket_name = "<Your bucketname>";
//从环境变量读取ak/sk
option.bucket_options.access_key = getenv("ACCESS_KEY_ID");
option.bucket_options.secret_access_key = getenv("SECRET_ACCESS_KEY");
char *concurrent_upload_id;
uint64_t uploadSize = uploadSliceSize;
uint64_t filesize =0;
//初始化结构体put_properties
obs_put_properties put_properties;
init_put_properties(&put_properties);
//大文件信息:文件指针,文件大小,按照分段大小的分段数
test_upload_file_callback_data data;
memset(&data, 0, sizeof(test_upload_file_callback_data));
filesize = get_file_info(filename,&data);
data.noStatus = 1;
data.part_size = uploadSize;
data.part_num = (filesize % uploadSize == 0) ? (filesize / uploadSize) : (filesize / uploadSize +1);
// 初始化上传段回调函数
obs_response_handler Handler =
{
&response_properties_callback, &response_complete_callback
};
// 合并段回调函数
obs_complete_multi_part_upload_handler complete_multi_handler =
{
{&response_properties_callback,
&response_complete_callback},
&CompleteMultipartUploadCallback
};
//初始化上传段任务返回uploadId: uploadIdReturn
char uploadIdReturn[256] = {0};
int upload_id_return_size = 255;
initiate_multi_part_upload(&option,key,upload_id_return_size,uploadIdReturn, &putProperties,
0,&Handler, &ret_status);
if (OBS_STATUS_OK == ret_status) {
printf("test init upload part return uploadIdReturn(%s). \n", uploadIdReturn);
strcpy(concurrent_upload_id,uploadIdReturn);
}
else
{
printf("test init upload part failed(%s).\n", obs_get_status_name(ret_status));
}
// 并发上传
test_concurrent_upload_file_callback_data *concurrent_upload_file;
concurrent_upload_file =(test_concurrent_upload_file_callback_data *)malloc(
sizeof(test_concurrent_upload_file_callback_data)*(data.part_num+1));
if(concurrent_upload_file == NULL)
{
printf("malloc test_concurrent_upload_file_callback_data failed!!!");
return ;
}
test_concurrent_upload_file_callback_data *concurrent_upload_file_complete =
concurrent_upload_file;
start_upload_threads(data, concurrent_upload_id,filesize, key, option, concurrent_upload_file_complete);
// 合并段
obs_complete_upload_Info *upload_Info = (obs_complete_upload_Info *)malloc(
sizeof(obs_complete_upload_Info)*data.part_num);
for(i=0; i<data.part_num; i++)
{
upload_Info[i].part_number = concurrent_upload_file_complete[i].part_num;
upload_Info[i].etag = concurrent_upload_file_complete[i].etag;
}
complete_multi_part_upload(&option, key, uploadIdReturn, data.part_num,upload_Info,&putProperties,&complete_multi_handler,&ret_status);
if (ret_status == OBS_STATUS_OK) {
printf("test complete upload successfully. \n");
}
else
{
printf("test complete upload failed(%s).\n", obs_get_status_name(ret_status));
}
if(concurrent_upload_file)
{
free(concurrent_upload_file);
concurrent_upload_file = NULL;
}
if(upload_Info)
{
free(upload_Info);
upload_Info = NULL;
}
}
大文件分段上传时,使用Offset参数和PartSize参数配合指定每段数据在文件中的起始结束位置。
并发数过大,可能会因为网络不稳定等原因,产生Timeout错误,需要限制并发数。