Uploading an Object - Resumable, Asynchronous (SDK for Java)
Function
Asynchronous resumable upload is a resumable upload that supports asynchronous operations and can be paused, canceled, and resumed. It works by creating a sub-thread to perform resumable uploads. If the upload fails, the exception can be checked in a callback. If the upload is successful, an encapsulated Optional object can be obtained from the task.
If you have any questions during development, post them on the Issues page of GitHub.
Restrictions
- To upload an object, you must be the bucket owner or have the object upload permission. To use this API, permissions are required. You can use IAM policies to grant the obs:object:PutObject permission or bucket policies to grant the PutObject permission. For details, see Introduction to OBS Access Control, IAM Custom Policies, and Configuring an Object Policy.
- The mapping between OBS regions and endpoints must comply with what is listed in Regions and Endpoints.
- The file uploaded by the resumable upload API must exceed 100 KB.
- You must enable the resumable upload option when you use this API, so the progress of the last upload can be obtained for upload resumption.
Method
obsClientAsync.uploadFileAsync(UploadFileRequest request, TaskCallback<CompleteMultipartUploadResult, UploadFileRequest> completeCallback)
Request Parameters
| Parameter | Type | Mandatory (Yes/No) | Description |
|---|---|---|---|
| request | Yes | Explanation: Request parameters for uploading an object. For details, see Table 2. | |
| completeCallback | Yes | Explanation: Callbacks executed to obtain results (success or exception) for an asynchronous upload. |
| Parameter | Type | Mandatory (Yes/No) | Description |
|---|---|---|---|
| bucketName | String | Yes | Explanation: Bucket name. Restrictions:
Default value: None |
| objectKey | String | Yes | Explanation: Object name. An object is uniquely identified by an object name in a bucket. An object name is a complete path that does not contain the bucket name. For example, if the address for accessing the object is examplebucket.obs.ap-southeast-1.myhuaweicloud.com/folder/test.txt, the object name is folder/test.txt. Value range: The value must contain 1 to 1,024 characters. Default value: None |
| objectMetadata | No | Explanation: Object metadata. For details, see Table 14. Default value: None | |
| acl | No | Explanation: An ACL that can be specified at bucket creation. You can use either a pre-defined or a user-defined ACL. For more information about ACLs, see ACLs. Value range:
Default value: AccessControlList.REST_CANNED_PRIVATE | |
| sseKmsHeader | No | Explanation: Server-side encryption header. For details, see Table 9. Default value: None | |
| sseCHeader | No | Explanation: Server-side encryption header. For details, see Table 8. Default value: None | |
| enableCheckpoint | boolean | No | Explanation: Whether to enable the resumable mode. Value range: true: The resumable mode is enabled. false: The resumable mode is disabled. Default value: false |
| checkpointFile | String | No | Explanation: Path of a file generated for recording the progress of a resumable upload. The file contains the information about parts and the upload progress. Restrictions: This parameter is valid only for resumable uploads. Default value: If this parameter is left blank, the progress file will be in the same directory as the local file to be uploaded. |
| uploadFile | String | Yes | Explanation: Full path of the file or folder to be uploaded, for example, aa/bb.txt or aa/. Default value: None |
| extensionPermissionMap | Map<ExtensionObjectPermissionEnum, Set<String>> | No | Explanation: A permission map for granting bucket ACL permissions to one or more accounts. ExtensionObjectPermissionEnum specifies the permissions to grant, and Set<String> describes the list of account IDs (indicated by domain_id) the granted permissions apply to. Value range:
Default value: None |
| progressListener | No | Explanation: Data transmission listener, used for obtaining the upload progress. For details, see Table 4. | |
| partSize | long | No | Explanation: Part size. Value range: The value ranges from 100 KB to 5 GB, in bytes. Default value: 9 MB |
| taskNum | int | No | Explanation: Maximum number of files that can be uploaded concurrently in a multipart upload. Value range: An integer from 1 to 10000 Default value: 1, indicating concurrent uploads are not used. |
| encodeHeaders | boolean | No | Explanation: Whether to enable OBS to automatically encode request headers. Due to HTTP coding restrictions, only ASCII characters can be sent. If your request headers contain full-width characters, the SDK will URL encode these characters before sending the request. When you use a browser to access the object metadata, the browser automatically decodes the data. Value range: true: Encoding with SDK is enabled. false: Encoding with SDK is disabled. Default value: true |
| needCalculateCRC64 | boolean | No | Explanation: Whether to automatically calculate the CRC64 value of the data to be uploaded and submit the CRC64 value to the server for verification. Restrictions: This parameter is not supported for POSIX or SFS objects. Value range: true: The SDK calculates CRC64 and submits it to the server for verification. false: CRC64 is not calculated. Default value: false |
| Method | Return Value | Type | Description |
|---|---|---|---|
| onSuccess | void | result (generic) | Called if the request is completed successfully. |
| onException | void | ObsException, request (generic) | Called if the request failed. The exception and corresponding request information are returned. |
Responses
| Method | Return Value | Type | Description |
|---|---|---|---|
| isTaskFinished | boolean | None | Whether the task is finished. |
| waitUntilFinished | void | None | Blocks the current thread and waits until the asynchronous upload completes. |
| cancel | boolean | None | Cancels an asynchronous upload. If the upload was successfully canceled, true is returned; otherwise, false is returned. |
| getResult | java.util.Optional | None | Blocks the current thread and waits until the asynchronous upload completes before returning the result. If the upload failed, the value of Optional.isPresent() is false. |
| Parameter | Type | Description |
|---|---|---|
| statusCode | int | Explanation: HTTP status code. Value range: A status code is a group of digits that can be 2xx (indicating successes) or 4xx or 5xx (indicating errors). It indicates the status of a response. For more information, see Status Code. Default value: None |
| responseHeaders | Map<String, Object> | Explanation: Response header list, composed of tuples. In a tuple, the String key indicates the name of the header, and the Object value indicates the value of the header. Default value: None |
| bucketName | String | Explanation: Bucket name. Restrictions:
Default value: None |
| objectKey | String | Explanation: Object name. An object is uniquely identified by an object name in a bucket. An object name is a complete path that does not contain the bucket name. For example, if the address for accessing the object is examplebucket.obs.ap-southeast-1.myhuaweicloud.com/folder/test.txt, the object name is folder/test.txt. Value range: The value must contain 1 to 1,024 characters. Default value: None |
| etag | String | Explanation: ETag of an object, which is a Base64-encoded 128-bit MD5 digest. ETag is the unique identifier of the object content. It can be used to determine whether the object content is changed. For example, if the ETag is A when an object is uploaded and is B when the object is downloaded, the object content is changed. The ETag reflects changes only to the contents of the object, not its metadata. An object created by an upload or copy operation has a unique ETag. Restrictions: If an object is encrypted using server-side encryption, the ETag is not the MD5 value of the object. Value range: The value must contain 32 characters. Default value: None |
| versionId | String | Explanation: Object version ID. If versioning is enabled for the bucket, the object version number will be returned. Value range: The value must contain 32 characters. Default value: None |
Code Examples
import com.obs.services.ObsClientAsync;
import com.obs.services.exception.ObsException;
import com.obs.services.internal.task.UploadFileTask;
import com.obs.services.internal.utils.CallCancelHandler;
import com.obs.services.model.CompleteMultipartUploadResult;
import com.obs.services.model.TaskCallback;
import com.obs.services.model.UploadFileRequest;
import java.io.IOException;
import java.util.Map;
import java.util.Optional;
public class TestUploadFileCancel
{
public static void main(String[] args) {
// Obtain an AK/SK pair using environment variables or import the AK/SK pair in other ways. Using hard coding may result in leakage.
// Obtain an AK/SK pair on the management console.
String ak = System.getenv("ACCESS_KEY_ID");
String sk = System.getenv("SECRET_ACCESS_KEY_ID");
// (Optional) If you are using a temporary AK/SK pair and a security token to access OBS, you are advised not to use hard coding, which may result in information leakage.
// Obtain an AK/SK pair and a security token using environment variables or import them in other ways.
String securityToken = System.getenv("SECURITY_TOKEN");
// Enter the endpoint corresponding to the bucket. CN-Hong Kong is used here as an example. Replace it with the one currently in use.
String endPoint = "https://obs.ap-southeast-1.myhuaweicloud.com";
// Obtain an endpoint using environment variables or import it in other ways.
//String endPoint = System.getenv("ENDPOINT");
// Create an ObsClient instance.
try (ObsClientAsync obsClientAsync = new ObsClientAsync(ak, sk, securityToken, endPoint)) {
String bucketName = "examplebucket";
String objectKey = "objectKey";
String localFile = "localfile";
UploadFileRequest uploadFileRequest1 = new UploadFileRequest(bucketName, objectKey, localFile);
uploadFileRequest1.setProgressListener(status -> {
// Obtain the average upload speed.
System.out.println("AverageSpeed:" + status.getAverageSpeed());
// Obtain the upload progress (%).
System.out.println("TransferPercentage:" + status.getTransferPercentage());
});
CallCancelHandler callCancelHandler = new CallCancelHandler();
uploadFileRequest1.setCancelHandler(callCancelHandler);
// Refresh the upload progress each time 5 MB of data is uploaded.
uploadFileRequest1.setProgressInterval(5 * 1024 * 1024L);
// Enable the recording of progress, which is used for resuming upload.
uploadFileRequest1.setEnableCheckpoint(true);
// Set the number of concurrent upload threads to 5.
uploadFileRequest1.setTaskNum(5);
// Set the part size to 5 MB.
uploadFileRequest1.setPartSize(5 * 1024 * 1024L);
//Pause the task without aborting it. The parts that have been uploaded are not deleted.
uploadFileRequest1.setNeedAbortUploadFileAfterCancel(false);
// Invoke a callback to show the upload result.
TaskCallback<CompleteMultipartUploadResult,UploadFileRequest> completeCallback = new TaskCallback<CompleteMultipartUploadResult, UploadFileRequest>()
{
@Override
public void onSuccess(CompleteMultipartUploadResult result)
{}
@Override
public void onException(ObsException e, UploadFileRequest singleRequest)
{
System.out.println("uploadFileAsync failed");
if(singleRequest.getCancelHandler()!=null) {
System.out.println("UploadFileRequest isCanceled ? " + singleRequest.getCancelHandler().isCancelled());
}
// Request failed. Print the HTTP status code.
System.out.println("HTTP Code:" + e.getResponseCode());
// Request failed. Print the server-side error code.
System.out.println("Error Code:" + e.getErrorCode());
// Request failed. Print the error details.
System.out.println("Error Message:" + e.getErrorMessage());
// Request failed. Print the request ID.
System.out.println("Request ID:" + e.getErrorRequestId());
System.out.println("Host ID:" + e.getErrorHostId());
// Check all Map entries and print all headers associated with the error.
Map<String, String> headers = e.getResponseHeaders();
if(headers != null){
for (Map.Entry<String, String> header : headers.entrySet()) {
if(header.getKey().contains("error")){
System.out.println("errorHeaderKey:"+header.getKey()+", errorHeaderValue:"+header.getValue());
}
}
}
e.printStackTrace();
}
};
// Start the asynchronous resumable upload.
UploadFileTask uploadFileTask = obsClientAsync.uploadFileAsync(uploadFileRequest1, completeCallback);
Thread.sleep(1000);
// Pause the asynchronous upload after it is ongoing for a while. The parts that have been uploaded are not affected.
uploadFileTask.cancel();
// Wait for the upload to finish.
uploadFileTask.waitUntilFinished();
// Resume the upload from the previously recorded checkpoint.
uploadFileTask = obsClientAsync.uploadFileAsync(uploadFileRequest1, completeCallback);
Optional<CompleteMultipartUploadResult> result = uploadFileTask.getResult();
if (result.isPresent()) {
System.out.println("uploadFileAsync Successfully!");
System.out.println("HTTP StatusCode:"+result.get().getStatusCode());
System.out.println("ObjectUrl:"+result.get().getObjectUrl());
System.out.println("Etag:"+result.get().getEtag());
}
}
catch (IOException | InterruptedException e)
{
e.printStackTrace();
}
}
} This example uses the asynchronous resumable method to upload localfile to bucket examplebucket as object objectKey and to pause and abort the upload.
import com.obs.services.ObsClientAsync;
import com.obs.services.exception.ObsException;
import com.obs.services.internal.task.UploadFileTask;
import com.obs.services.internal.utils.CallCancelHandler;
import com.obs.services.model.CompleteMultipartUploadResult;
import com.obs.services.model.TaskCallback;
import com.obs.services.model.UploadFileRequest;
import java.io.IOException;
import java.util.Map;
public class TestUploadFileCancelAndAbort
{
public static void main(String[] args) {
// Obtain an AK/SK pair using environment variables or import the AK/SK pair in other ways. Using hard coding may result in leakage.
// Obtain an AK/SK pair on the management console.
String ak = System.getenv("ACCESS_KEY_ID");
String sk = System.getenv("SECRET_ACCESS_KEY_ID");
// (Optional) If you are using a temporary AK/SK pair and a security token to access OBS, you are advised not to use hard coding, which may result in information leakage.
// Obtain an AK/SK pair and a security token using environment variables or import them in other ways.
String securityToken = System.getenv("SECURITY_TOKEN");
// Enter the endpoint corresponding to the bucket. CN-Hong Kong is used here as an example. Replace it with the one currently in use.
String endPoint = "https://obs.ap-southeast-1.myhuaweicloud.com";
// Obtain an endpoint using environment variables or import it in other ways.
//String endPoint = System.getenv("ENDPOINT");
// Create an ObsClient instance.
try (ObsClientAsync obsClientAsync = new ObsClientAsync(ak, sk, securityToken, endPoint)) {
String bucketName = "examplebucket";
String objectKey = "objectKey";
String localFile = "localfile";
UploadFileRequest uploadFileRequest1 = new UploadFileRequest(bucketName, objectKey, localFile);
uploadFileRequest1.setProgressListener(status -> {
// Obtain the average upload speed.
System.out.println("AverageSpeed:" + status.getAverageSpeed());
// Obtain the upload progress (%).
System.out.println("TransferPercentage:" + status.getTransferPercentage());
});
CallCancelHandler callCancelHandler = new CallCancelHandler();
uploadFileRequest1.setCancelHandler(callCancelHandler);
// Refresh the upload progress each time 5 MB of data is uploaded.
uploadFileRequest1.setProgressInterval(5 * 1024 * 1024L);
// Set the number of concurrent upload threads to 5.
uploadFileRequest1.setTaskNum(5);
// Set the part size to 5 MB.
uploadFileRequest1.setPartSize(5 * 1024 * 1024L);
//Pause and abort the task. The parts that have been uploaded are deleted.
uploadFileRequest1.setNeedAbortUploadFileAfterCancel(true);
// Invoke a callback to show the upload result.
TaskCallback<CompleteMultipartUploadResult,UploadFileRequest> completeCallback =
new TaskCallback<CompleteMultipartUploadResult, UploadFileRequest>()
{
@Override
public void onSuccess(CompleteMultipartUploadResult result)
{}
@Override
public void onException(ObsException e, UploadFileRequest singleRequest)
{
System.out.println("uploadFileAsync failed");
if(singleRequest.getCancelHandler()!=null) {
System.out.println("UploadFileRequest isCanceled ? " + singleRequest.getCancelHandler().isCancelled());
}
// Request failed. Print the HTTP status code.
System.out.println("HTTP Code:" + e.getResponseCode());
// Request failed. Print the server-side error code.
System.out.println("Error Code:" + e.getErrorCode());
// Request failed. Print the error details.
System.out.println("Error Message:" + e.getErrorMessage());
// Request failed. Print the request ID.
System.out.println("Request ID:" + e.getErrorRequestId());
System.out.println("Host ID:" + e.getErrorHostId());
// Check all Map entries and print all headers associated with the error.
Map<String, String> headers = e.getResponseHeaders();
if(headers != null){
for (Map.Entry<String, String> header : headers.entrySet()) {
if(header.getKey().contains("error")){
System.out.println("errorHeaderKey:"+header.getKey()+", errorHeaderValue:"+header.getValue());
}
}
}
e.printStackTrace();
}
};
// Start the asynchronous resumable upload.
UploadFileTask uploadFileTask = obsClientAsync.uploadFileAsync(uploadFileRequest1, completeCallback);
Thread.sleep(1000);
// Pause and abort the ongoing upload. The parts that have been uploaded are deleted. If the upload is complete, it cannot be aborted.
uploadFileTask.cancel();
// Wait for the upload to finish.
uploadFileTask.waitUntilFinished();
}
catch (IOException | InterruptedException e)
{
e.printStackTrace();
}
}
} Helpful Links
What is your overall rating for this page?
Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot