Help Center> Object Storage Service> Node.js> Object Upload> Performing a Resumable Upload
Updated on 2023-11-09 GMT+08:00

Performing a Resumable Upload

If you have any questions during development, post them on the Issues page of GitHub. For details about parameters and usage of each API, see the API Reference.

Uploading large files often fails due to poor network conditions or program breakdowns. It is a waste of resources to restart the upload process upon an upload failure, and the restarted upload process may still suffer from the unstable network. To resolve such issues, you can use the API for resumable upload, whose working principle is to divide the to-be-uploaded file into multiple parts and upload them separately. The upload result of each part is recorded in a checkpoint file in real time. Only when all parts are successfully uploaded, the result indicating a successful upload will be returned. Otherwise, an error is returned in callback function to remind you of calling the API again for re-uploading. Based on the upload status of each part recorded in the checkpoint file, the re-uploading will upload the parts failed to be uploaded previously, instead of uploading all parts. By virtue of this, resources are saved and efficiency is improved.

  • The total size of files uploaded by the resumable upload API must be larger than 100 KB.

You can call ObsClient.uploadFile to perform a resumable upload. The following table describes the main parameters involved in this API.

Parameter

Description

Bucket

(Mandatory) Bucket name

Key

(Mandatory) Object name

UploadFile

(Mandatory) Local file to be uploaded

PartSize

Part size, in bytes. The value ranges from 100 KB to 5 GB and defaults to 5 MB.

TaskNum

Maximum number of threads that can be concurrently executed for upload. The default value is 20.

EnableCheckpoint

Whether to enable the resumable upload mode. The default value is false, which indicates that this mode is disabled.

CheckpointFile

File used to record the upload progress. This parameter is effective only in the resumable upload mode. If this parameter is null, the file will be in the same directory as the local file to be uploaded.

EnableCheckSum

Whether to verify the content of the to-be-uploaded file. This parameter is effective only in the resumable upload mode. The default value is false, which indicates that the content will not be verified.

Sample code:

// Import the OBS library.
// Use npm to install the client.
var ObsClient = require('esdk-obs-nodejs');
// Use the source code to install the client.
// var ObsClient = require('./lib/obs');

// Create an ObsClient instance.
var obsClient = new ObsClient({
       //Obtain an AK/SK pair using environment variables or import the AK/SK pair in other ways. Using hard coding may result in leakage.
       //Obtain an AK/SK pair on the management console. For details, see https://support.huaweicloud.com/intl/en-us/usermanual-ca/ca_01_0003.html.
       access_key_id: process.env.ACCESS_KEY_ID,
       secret_access_key: process.env.SECRET_ACCESS_KEY,
       server : 'https://your-endpoint'
});

obsClient.uploadFile({
       Bucket : 'bucketname',
       Key : 'objectname',
       // Set the large file to be uploaded. localfile is the path of the local file to be uploaded. You need to specify the file name.
       UploadFile : 'localfile',
       // Set the part size to 10 MB.
       PartSize : 10 * 1024 * 1024,
       // Enable the resumable upload mode.
       EnableCheckpoint : true
}, (err, result) => {
    if(err){
        console.error('Error-->' + err);
    }else{
        console.log('RequestId-->' + result.InterfaceResult.RequestId);
        console.log('Bucket-->' + result.InterfaceResult.Bucket);
        console.log('Key-->' + result.InterfaceResult.Key);
        console.log('Location-->' + result.InterfaceResult.Location);
    }
});
  • The API for resumable upload, which is implemented based on multipart upload, is an encapsulated and enhanced version of multipart upload.
  • This API saves resources and improves efficiency upon the re-upload, and speeds up the upload process by concurrently uploading parts. Because this API is transparent to users, users are free from concerns about internal service details, such as the creation and deletion of checkpoint files, division of objects, and concurrent upload of parts.
  • The default value of the EnableCheckpoint parameter is false, which indicates that the resumable upload mode is disabled. In such cases, this API degrades to the simple encapsulation of multipart upload, and no checkpoint file will be generated.
  • CheckpointFile and EnableCheckSum are effective only when EnableCheckpoint is true.