Limiting Upload Speed (SDK for C)
If you have any questions during development, post them on the Issues page of GitHub.
Function
Limiting upload speed refers to limiting the bandwidth of a single connection request during object upload. This prevents a single upload request from occupying too much bandwidth and affecting other services. You can set the upload speed limit using the upload_limit field in the obs_put_properties structure. The SDK writes the limit to the x-obs-traffic-limit request header, and the OBS server performs precise bandwidth control.
The following APIs support upload speed limiting:
Restrictions
- To limit the upload speed, you must be the bucket owner or have the object upload permission (obs:object:PutObject granted using IAM or PutObject granted using a bucket policy). For details, see Introduction to OBS Access Control, IAM Custom Policies, and Configuring an Object Policy.
- The mapping between OBS regions and endpoints must comply with what is listed in Regions and Endpoints.
- If the value of upload_limit is 0, there is no limit.
- The server enforces the speed limit and ensures the accuracy. Once the limit is set, the maximum upload bandwidth will not exceed the specified value.
- Multipart upload (upload_file) also supports speed limiting, which can be set using the upload_limit field in the obs_upload_file_configuration structure.
Method
You can use the upload_limit field in the obs_put_properties structure to limit upload speed, and pass the structure as a parameter of the upload API.
void put_object(const obs_options *options, char *key, uint64_t content_length,
obs_put_properties *put_properties,
server_side_encryption_params *encryption_params,
obs_put_object_handler *handler, void *callback_data);
void put_object_content(const obs_options *options, const char *key,
const char *content, uint64_t content_length,
obs_put_properties *put_properties,
server_side_encryption_params *encryption_params,
obs_put_object_handler *handler, void *callback_data);
void put_file(const obs_options *options, const char *key,
const char *file_path,
obs_put_properties *put_properties,
server_side_encryption_params *encryption_params,
obs_put_object_handler *handler, void *callback_data); Request Parameters
The parameters related to upload speed limiting are in the obs_put_properties structure. You can use init_put_properties to initialize the structure.
| Parameter | Type | Mandatory (Yes/No) | Description |
|---|---|---|---|
| upload_limit | uint64_t | No | Explanation: Server bandwidth limit for a single connection request. The SDK writes the value to the x-obs-traffic-limit request header, and the OBS server limits the bandwidth. Restrictions: None Value range: 0 indicates that the bandwidth is not limited. A value ranging from 819200 to 838860800 specifies a bandwidth limit, measured in bit/s. 819200 bit/s equals 100 KB/s and 838860800 bit/s equals 100 MB/s. Default value: 0 (no limit) |
Speed Limit Setting Guide
| Scenario | Recommended Value of upload_limit | Description |
|---|---|---|
| Unlimited (default) | 0 | The default bandwidth is used. This setting is suitable when no bandwidth limit is required. |
| Low-bandwidth | 819200 (100 KB/s) | This setting can prevent uploads from consuming too much bandwidth and is applicable to the environment with insufficient bandwidth. |
| Moderate-bandwidth | 8192000 (1 MB/s) | This setting can balance the upload speed with other service requirements. |
| High-bandwidth | 81920000 (10 MB/s) | This setting can limit the peak bandwidth for uploading large files and is applicable to large file scenarios. |
| Near-unlimited | 838860800 (100 MB/s) | This setting sets an upper limit and has little impact in practice. |
- If the limit is too low, uploading large files will take a significantly longer time. Set the limit based on your service requirements.
- The speed limit controls only the bandwidth of a single upload request. During concurrent uploads, the bandwidth of each request is limited independently. The total bandwidth is the sum of the bandwidth of all requests.
- You are advised to use the default value 0 (no limit) when the network bandwidth is sufficient and set a speed limit only when necessary.
Sample Code: Setting a Speed Limit During Streaming Upload
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 | #include "eSDKOBS.h" #include <stdio.h> // The response callback function. The content of properties in the callback can be recorded in callback_data (custom callback data). obs_status response_properties_callback(const obs_response_properties *properties, void *callback_data); void put_buffer_complete_callback(obs_status status, const obs_error_details *error, void *callback_data); typedef struct put_buffer_object_callback_data { char *put_buffer; uint64_t buffer_size; uint64_t cur_offset; obs_status ret_status; } put_buffer_object_callback_data; int put_buffer_data_callback(int buffer_size, char *buffer, void *callback_data); char* generate_upload_buffer(uint64_t buffer_size); int main() { // The following code shows how to use put_object to upload an object in the stream: // Call the obs_initialize method at the program entry to initialize global resources such as the network and memory. obs_initialize(OBS_INIT_ALL); obs_options options; // Create and initialize options, including the access domain name (host_name), access keys (access_key_id and access_key_secret), bucket name (bucket_name), and bucket storage class (storage_class). init_obs_options(&options); // Enter the endpoint corresponding to the bucket for host_name. CN-Hong Kong is used here as an example. Replace it with the one currently in use. options.bucket_options.host_name = "obs.ap-southeast-1.myhuaweicloud.com"; // Hard-coded or plaintext AK and SK are risky. For security purposes, encrypt your AK and SK and store them in the configuration file or environment variables. // In this example, the AK and SK are stored in environment variables for identity authentication. Before running the code in this example, configure local environment variables ACCESS_KEY_ID and SECRET_ACCESS_KEY. options.bucket_options.access_key = getenv("ACCESS_KEY_ID"); options.bucket_options.secret_access_key = getenv("SECRET_ACCESS_KEY"); // Specify the bucket name, for example, example-bucket-name. char * bucketName = "example-bucket-name"; options.bucket_options.bucket_name = bucketName; // Initialize the properties of the object to be uploaded. obs_put_properties put_properties; init_put_properties(&put_properties); // Set the speed limit to 100 KB/s (819200 bit/s). put_properties.upload_limit = 819200; // Initialize the structure for storing the uploaded data. put_buffer_object_callback_data data; memset(&data, 0, sizeof(put_buffer_object_callback_data)); // Set the buffer size. data.buffer_size = 10 * 1024 * 1024; // Create a simulated data buffer for the streaming upload and assign the value to the data upload structure. data.put_buffer = generate_upload_buffer(data.buffer_size); if (NULL == data.put_buffer) { printf("generate put buffer failed. \n"); return -1; } // Name of the object to be uploaded char *key = "example_put_buffer_test.txt"; obs_put_object_handler putobjectHandler = { { &response_properties_callback, &put_buffer_complete_callback }, &put_buffer_data_callback }; put_object(&options, key, data.buffer_size, &put_properties, 0, &putobjectHandler, &data); if (OBS_STATUS_OK == data.ret_status) { printf("put object from buffer successfully. \n"); } else { printf("put object from buffer failed(%s).\n", obs_get_status_name(data.ret_status)); } // Release the buffer. free(data.put_buffer); // Release the allocated global resources. obs_deinitialize(); return 0; } // The response callback function. The content of properties in the callback can be recorded in callback_data (custom callback data). obs_status response_properties_callback(const obs_response_properties *properties, void *callback_data) { if (properties == NULL) { printf("error! obs_response_properties is null!"); if (callback_data != NULL) { obs_sever_callback_data *data = (obs_sever_callback_data *)callback_data; printf("server_callback buf is %s, len is %llu", data->buffer, data->buffer_len); return OBS_STATUS_OK; } else { printf("error! obs_sever_callback_data is null!"); return OBS_STATUS_OK; } } // Print the response. #define print_nonnull(name, field) \ do { \ if (properties-> field) { \ printf("%s: %s\n", name, properties->field); \ } \ } while (0) print_nonnull("request_id", request_id); print_nonnull("request_id2", request_id2); print_nonnull("content_type", content_type); if (properties->content_length) { printf("content_length: %llu\n", properties->content_length); } print_nonnull("server", server); print_nonnull("ETag", etag); print_nonnull("expiration", expiration); print_nonnull("website_redirect_location", website_redirect_location); print_nonnull("version_id", version_id); print_nonnull("allow_origin", allow_origin); print_nonnull("allow_headers", allow_headers); print_nonnull("max_age", max_age); print_nonnull("allow_methods", allow_methods); print_nonnull("expose_headers", expose_headers); print_nonnull("storage_class", storage_class); print_nonnull("server_side_encryption", server_side_encryption); print_nonnull("kms_key_id", kms_key_id); print_nonnull("customer_algorithm", customer_algorithm); print_nonnull("customer_key_md5", customer_key_md5); print_nonnull("bucket_location", bucket_location); print_nonnull("obs_version", obs_version); print_nonnull("restore", restore); print_nonnull("obs_object_type", obs_object_type); print_nonnull("obs_next_append_position", obs_next_append_position); print_nonnull("obs_head_epid", obs_head_epid); print_nonnull("reserved_indicator", reserved_indicator); int i; for (i = 0; i < properties->meta_data_count; i++) { printf("x-obs-meta-%s: %s\n", properties->meta_data[i].name, properties->meta_data[i].value); } return OBS_STATUS_OK; } void put_buffer_complete_callback(obs_status status, const obs_error_details *error, void *callback_data) { if (callback_data) { put_buffer_object_callback_data *data = (put_buffer_object_callback_data *)callback_data; data->ret_status = status; } else { printf("Callback_data is NULL"); } if (error && error->message) { printf("Error Message: \n %s\n", error->message); } if (error && error->resource) { printf("Error Resource: \n %s\n", error->resource); } if (error && error->further_details) { printf("Error further_details: \n %s\n", error->further_details); } if (error && error->extra_details_count) { int i; for (i = 0; i < error->extra_details_count; i++) { printf("Error Extra Detail(%d):\n %s:%s\n", i, error->extra_details[i].name, error->extra_details[i].value); } } if (error && error->error_headers_count) { int i; for (i = 0; i < error->error_headers_count; i++) { const char *errorHeader = error->error_headers[i]; printf("Error Headers(%d):\n %s\n", i, errorHeader == NULL ? "NULL Header" : errorHeader); } } } int put_buffer_data_callback(int buffer_size, char *buffer, void *callback_data) { put_buffer_object_callback_data *data = (put_buffer_object_callback_data *)callback_data; int toRead = 0; if (data->buffer_size) { toRead = ((data->buffer_size > (unsigned)buffer_size) ? (unsigned)buffer_size : data->buffer_size); memcpy_s(buffer, buffer_size, data->put_buffer + data->cur_offset, toRead); } uint64_t originalContentLength = data->buffer_size; data->buffer_size -= toRead; data->cur_offset += toRead; if (data->buffer_size) { printf("%llu bytes remaining ", (unsigned long long)data->buffer_size); printf("(%d%% complete) ...\n", (int)(((originalContentLength - data->buffer_size) * 100) / originalContentLength)); } return toRead; } // Create a simulated streaming upload buffer. char* generate_upload_buffer(uint64_t buffer_size) { void* upload_buffer = NULL; if (buffer_size > 0) { upload_buffer = malloc(buffer_size); if (upload_buffer != NULL) { memset(upload_buffer, 't', buffer_size); } } return upload_buffer; } |
Helpful Links
- For details about the API for limiting the upload speed, see Uploading an Object with PUT.
- OBS Error Codes
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot