Uploading an Object - Resumable (SDK for Node.js)

If you have any questions during development, post them on the Issues page of GitHub.

Function

The resumable upload is an encapsulated and enhanced version of the multipart upload used for dealing with possible upload failures of large files when the network connection is unstable or a program crashes.

Uploading large files often fails due to poor network conditions or program breakdowns. It is a waste of resources to restart the upload process upon an upload failure, and the restarted upload process may still suffer from the unstable network. To resolve such issues, you can use the API for resumable upload, whose working principle is to divide the to-be-uploaded file into multiple parts and upload them separately. The upload result of each part is recorded in a checkpoint file in real time. Only when all parts are successfully uploaded, the result indicating a successful upload will be returned. Otherwise, an error is returned in callback function to remind you of calling the API again for re-uploading. Based on the upload status of each part recorded in the checkpoint file, the re-uploading will upload the parts failed to be uploaded previously, instead of uploading all parts. By virtue of this, resources are saved and efficiency is improved.

The resumable upload interface helps save resources and improve efficiency by restarting an upload from the point of failure and concurrently uploading parts. You do not need to worry about internal service details, such as the creation and deletion of checkpoint files, division of objects, or concurrent uploads of parts.

Restrictions

The total size of files uploaded by the resumable upload API must be larger than 100 KB.

Method

ObsClient.uploadFile(params)

Request Parameters

**Table 1** List of request parameters
Parameter	Type	Mandatory (Yes/No)	Description
Bucket	string	Yes	Explanation: Bucket name. Restrictions: A bucket name must be unique across all accounts and regions. A bucket name: Must be 3 to 63 characters long and start with a digit or letter. Lowercase letters, digits, hyphens (-), and periods (.) are allowed. Cannot be formatted as an IP address. Cannot start or end with a hyphen (-) or period (.). Cannot contain two consecutive periods (..), for example, my..bucket. Cannot contain a period (.) and a hyphen (-) adjacent to each other, for example, my-.bucket or my.-bucket. If you repeatedly create buckets with the same name in the same region, no error will be reported, and the bucket attributes comply with those set in the first creation request. Value range: The value can contain 3 to 63 characters. Default value: None
Key	string	Yes	Explanation: Object name. An object is uniquely identified by an object name in a bucket. An object name is a complete path of the object that does not contain the bucket name. For example, if the address for accessing the object is examplebucket.obs.ap-southeast-1.myhuaweicloud.com/folder/test.txt, the object name is folder/test.txt. Restrictions: None Value range: The value can contain 1 to 1,024 characters. Default value: None
UploadFile	string	No	Explanation: Source file path of the object to be uploaded Restrictions: None Value range: None Default value: None
PartSize	number	No	Explanation: Size of the current part. Restrictions: None Value range: The value ranges from 100 KB to 5 GB, in bytes. Default value: 102400
TaskNum	number	No	Explanation: Maximum number of parts that can be uploaded concurrently Restrictions: None Value range: 1~10000 Default value: 1, indicating concurrent uploads are not used
ProgressCallback	function	No	Explanation: Callback function for obtaining the upload progress NOTE: This callback function contains the following parameters in sequence: number of uploaded bytes, total bytes, and used time (in seconds). Restrictions: None Value range: None Default value: None
ResumeCallback	function	No	Explanation: Callback function used to obtain the control parameter for canceling a resumable download Restrictions: None Value range: None Default value: None NOTE: This callback function contains a control parameter used for pausing or aborting a resumable upload. By calling the cancel method of this control parameter, you can pause a resumable upload. By calling the abort method of this control parameter, you can abort a resumable upload.
EnableCheckpoint	boolean	No	Explanation: Whether to enable the resumable mode. Restrictions: None Value range: true: The resumable mode is enabled. false: The resumable mode is disabled. In this case, this API works as a multipart upload API, and no checkpoint files are generated. Default value: false
CheckpointFile	string	No	Explanation: Path of a file generated for recording the progress of a resumable upload. The file contains the information about parts and progress. Restrictions: This parameter is valid only in the resumable mode. Value range: None Default value: If this parameter is left blank, the checkpoint file will be in the same directory as the local file to be uploaded.
EnableCheckSum	boolean	No	Explanation: Whether to verify the file to upload. If this function is enabled, before each task restarts, the system verifies whether the file to upload is the one used during task initialization. Restrictions: This parameter is valid only in the resumable mode. Value range: true: The file is verified. false: The file is not verified. Default value: false
ContentType	string	No	Explanation: MIME type of the file to be uploaded. MIME type is a standard way of describing a data type and is used by the browser to decide how to display data. Restrictions: None Value range: See What Is Content-Type (MIME)? (SDK for Node.js) Default value: If you do not specify this parameter when uploading an object, the SDK determines the object type based on the suffix of the specified object name and automatically assigns a value to this parameter.
ACL	AclType	No	Explanation: Access control list (ACL) that can be pre-defined when an object is created. For details about ACLs, see ACLs. Restrictions: None Value range: See Table 2. Default value: None
WebsiteRedirectLocation	string	No	Explanation: If the bucket is configured with website hosting, the request for obtaining the object can be redirected to another object in the bucket or an external URL. The request is redirected to object anotherPage.html in the same bucket: WebsiteRedirectLocation:/anotherPage.html The request is redirected to an external URL: WebsiteRedirectLocation:http://www.example.com/ OBS obtains the specified value from the header and stores it in the object metadata WebsiteRedirectLocation. Restrictions: The value must start with a slash (/), http://, or https:// and cannot exceed 2 KB. OBS only supports redirection of objects that are in the root directory. Value range: None Default value: None
SseKmsKey	string	No	Explanation: ID of the KMS master key when SSE-KMS is used. Restrictions: Valid value formats are as follows: regionID:domainID:key/key_id key_id In the preceding formats: regionID indicates the ID of the region where the key is used. You can obtain it from Regions and Endpoints. domainID indicates the ID of the account that the key is for. To obtain it, see How Do I Get My Account ID and User ID? (SDK for Node.js) key_id indicates the ID of the key created on Data Encryption Workshop (DEW). To obtain it, see Viewing a CMK. Value range: None Default value: If this parameter is not specified, the default master key will be used. If there is not a default master key, OBS will create one and use it.
SseC	string	Yes when SSE-C is used	Explanation: SSE-C is used for encrypting objects on the server side. Restrictions: Only AES256 is supported. Value range: AES256 Default value: None
SseCKey	string	Yes if used as a request parameter	Explanation: Key used for encrypting the object when SSE-C is used Restrictions: A Base64-encoded, 256-bit value, for example, K7QkYpBkM5+hca27fsNkUnNVaobncnLht/rCB2o/9Cw= Value range: None Default value: None
Metadata	object	No	Explanation: Custom metadata of the object to be uploaded. You can add a header starting with x-obs-meta- in the request to define metadata. The custom metadata will be returned in the response when you retrieve the object or query the object metadata. Restrictions: The custom metadata cannot exceed 8 KB in total. To measure the size, calculate the sum of bytes of all UTF-8 encoded keys and values. The custom metadata keys are case insensitive, but are stored in lowercase in OBS. The key values are case sensitive. Both custom metadata keys and their values must conform to US-ASCII standards. If non-ASCII or unrecognizable characters are required, they must be encoded and decoded in URL or Base64 on the client, because the server does not perform such operations. Value range: None Default value: None

**Table 2** AclType
Constant	Default Value	Description
ObsClient.enums.AclPrivate	private	Private read and write A bucket or object can only be accessed by its owner.
ObsClient.enums.AclPublicRead	public-read	Public read and private write If this permission is granted on a bucket, anyone can read the object list, multipart uploads, metadata, and object versions in the bucket. If this permission is granted on an object, everyone can obtain the content and metadata of the object.
ObsClient.enums.AclPublicReadWrite	public-read-write	Public read and write If this permission is granted on a bucket, anyone can read the object list, multipart tasks, metadata, and object versions in the bucket and can upload or delete objects, initiate multipart upload tasks, upload parts, assemble parts, copy parts, and abort multipart upload tasks. If this permission is granted on an object, everyone can obtain the content and metadata of the object.
ObsClient.enums.AclPublicReadDelivered	public-read-delivered	Public read on a bucket as well as the objects in the bucket. If this permission is granted on a bucket, anyone can read the object list, multipart tasks, metadata, and object versions and read the content and metadata of objects in the bucket. NOTE: AclPublicReadDelivered does not apply to objects.
ObsClient.enums.AclPublicReadWriteDelivered	public-read-write-delivered	Public read and write on a bucket as well as the objects in the bucket. If this permission is granted on a bucket, anyone can read the object list, multipart uploads, metadata, and object versions in the bucket and can upload or delete objects, initiate multipart upload tasks, upload parts, assemble parts, copy parts, and abort multipart uploads. They can also read the content and metadata of objects in the bucket. NOTE: AclPublicReadWriteDelivered does not apply to objects.
ObsClient.enums.AclBucketOwnerFullControl	bucket-owner-full-control	If this permission is granted on an object, only the bucket and object owners have the full control over the object. By default, if you upload an object to a bucket of any other user, the bucket owner does not have the permissions on your object. After you grant this policy to the bucket owner, the bucket owner can have full control over your object. For example, if user A uploads object x to user B's bucket, user B does not have the control over object x. If user A sets the bucket-owner-full-control policy for object x, user B then has the control over object x.

Responses

**Table 3** Responses
Type	Description
Table 4 NOTE: This API returns a Promise response, which requires the Promise or async/await syntax.	Explanation: Returned results. For details, see Table 4.

**Table 4** Response
Parameter	Type	Description
CommonMsg	ICommonMsg	Explanation: Common information generated after an API call is complete, including the HTTP status code and error code. For details, see Table 5.
InterfaceResult	Table 6	Explanation: Results outputted for a successful call. For details, see Table 6. Restrictions: This parameter is not included if the value of Status is greater than 300.

**Table 5** ICommonMsg
Parameter	Type	Description
Status	number	Explanation: HTTP status code returned by the OBS server. Value range: A status code is a group of digits indicating the status of a response. It ranges from 2xx (indicating successes) to 4xx or 5xx (indicating errors). For details, see Status Codes.
Code	string	Explanation: Error code returned by the OBS server.
Message	string	Explanation: Error description returned by the OBS server.
HostId	string	Explanation: Request server ID returned by the OBS server.
RequestId	string	Explanation: Request ID returned by the OBS server.
Id2	string	Explanation: Request ID2 returned by the OBS server.
Indicator	string	Explanation: Error code details returned by the OBS server.

**Table 6** UploadFileOutput
Parameter	Type	Description
RequestId	string	Explanation: Request ID returned by the OBS server
ETag	string	Explanation: Base64-encoded, 128-bit MD5 value of an assembled object calculated based on the ETag of each part. The ETag is a unique identifier for the object's contents, used to determine if the object has been updated. For example, if the ETag value is A when an object is uploaded and B when the object is downloaded, this indicates the contents of the object have been changed. The ETag reflects changes to an object's contents, not its metadata. Objects created by the upload and copy operations have unique ETags after being encrypted using MD5. Restrictions: If an object is encrypted using server-side encryption, the ETag is not the MD5 value of the object.
Bucket	string	Explanation: Name of the bucket in which parts are assembled
Key	string	Explanation: Object name. An object is uniquely identified by an object name in a bucket. An object name is a complete path of the object that does not contain the bucket name. For example, if the address for accessing the object is examplebucket.obs.ap-southeast-1.myhuaweicloud.com/folder/test.txt, the object name is folder/test.txt.
Location	string	Explanation: URL of the object obtained from assembling the parts. Example: https://example-Bucket.obs.regions.myhuaweicloud.com/example-Object
VersionId	string	Explanation: Version ID of the object obtained from assembling the parts. If versioning is enabled for the bucket, the object version ID will be returned.
SseKms	string	Explanation: SSE-KMS is used for encrypting objects on the server side.
SseKmsKey	string	Explanation: ID of the KMS master key when SSE-KMS is used.
SseC	string	Explanation: SSE-C is used for encrypting objects on the server side.
SseCKeyMd5	string	Explanation: MD5 value of the key used for encrypting objects when SSE-C is used. This value is used to check whether any error occurs during the transmission of the key. Restrictions: Base64-encoded MD5 value of the key, for example, 4XvB3tbNTN+tIEVa0/fGaQ==

Code Examples

This example uploads example/objectname to examplebucket using resumable upload.

    
     
       
       
         // Import the OBS library.
// Use npm to install the client.
const ObsClient = require("esdk-obs-nodejs");
// Use the source code to install the client.
// var ObsClient = require('./lib/obs');

// Create an instance of ObsClient.
const obsClient = new ObsClient({
  // Obtain an AK/SK pair using environment variables or import an AK/SK pair in other ways. Using hard coding may result in leakage.
  // Obtain an AK/SK pair on the management console. For details, see https://support.huaweicloud.com/intl/en-us/usermanual-ca/ca_01_0003.html.
  access_key_id: process.env.ACCESS_KEY_ID,
  secret_access_key: process.env.SECRET_ACCESS_KEY,
  // (Optional) If you use a temporary AK/SK pair and a security token to access OBS, you are advised not to use hard coding, which may result in information leakage. You can obtain an AK/SK pair using environment variables or import an AK/SK pair in other ways.
  // security_token: process.env.SECURITY_TOKEN,
  // Enter the endpoint corresponding to the region where the bucket is located. CN-Hong Kong is used here in this example. Replace it with the one currently in use.
  server: "https://obs.ap-southeast-1.myhuaweicloud.com"
});

async function uploadFile() {
  try {
    const params = {
      // Specify the bucket name.
      Bucket: 'examplebucket',
      // Specify the object to be created (example/objectname in this example).
      Key: 'example/objectname',
      //Specify the local file to upload (/tmp/objectname in this example).
      UploadFile: 'localfile',
      // Specify whether to enable resumable transmission. Value true is used in this example. The default value is false.
      EnableCheckpoint: true,
      // Specify a part size, in bytes. This example sets each part to 9 MB.
      PartSize: 9 * 1024 * 1024,
      // Specify the maximum number of parts that can be concurrently transmitted. 5 is used in this example.
      TaskNum: 5
    };
    // Upload the object using resumable upload.
    const result = await obsClient.uploadFile(params);
    if (result.CommonMsg.Status <= 300) {
      console.log("Upload file(%s) under the bucket(%s) successful!", params.Key, params.Bucket);
      console.log("RequestId: %s", result.CommonMsg.RequestId);
      fmt.Printf("ETag:%s\n", result.InterfaceResult.ETag)
      return;
    };
    console.log("An ObsError was found, which means your request sent to OBS was rejected with an error response.");
    console.log("Status: %d", result.CommonMsg.Status);
    console.log("Code: %s", result.CommonMsg.Code);
    console.log("Message: %s", result.CommonMsg.Message);
    console.log("RequestId: %s", result.CommonMsg.RequestId);
  } catch (error) {
    console.log("An Exception was found, which means the client encountered an internal problem when attempting to communicate with OBS, for example, the client was unable to access the network.");
    console.log(error);
  };
};

uploadFile();

        

      

    
   