Help Center/ Object Storage Service/ SDK Reference/ Java/ Object Download (SDK for Java)/ Downloading an Object - Resumable (SDK for Java)
Updated on 2024-06-18 GMT+08:00

Downloading an Object - Resumable (SDK for Java)

Function

  1. Downloading large files often fails due to an unstable network or program breakdown. It is a waste of resources to download files again. Moreover, the restarted download may still fail due to an unstable network. To resolve such issues, the resumable download API splits the file to be downloaded into multiple parts and downloads them separately. The download result of each part is recorded in a checkpoint file in real time. Only when all parts are downloaded is a message indicating the download is successful returned. If any parts fail to be downloaded, a message is returned telling you to call the API again to download the failed parts. Since the checkpoint file contains the progress of all parts, it helps you avoid downloading all parts in re-downloads, so that you can enjoy a cost-effective, efficient download.
  2. The resumable download API is an encapsulated and enhanced version of the API for partial download. For details about the partial download, see Downloading an Object - Partial (SDK for Java).
  3. This API saves resources and improves efficiency upon the re-download, and speeds up the download process by concurrently downloading parts. You do not need to worry about internal service details, such as the creation and deletion of checkpoint files, division of objects, or concurrent downloads of parts.

If you have any questions during development, post them on the Issues page of GitHub.

Restrictions

Method

obsClient.downloadFile(DownloadFileRequest request)

Request Parameters

Table 1 List of request parameters

Parameter

Type

Mandatory (Yes/No)

Description

request

DownloadFileRequest

Yes

Explanation:

Request parameters for downloading a large object. For details, see Table 2.

Table 2 DownloadFileRequest

Parameter

Type

Mandatory (Yes/No)

Description

bucketName

String

Yes

Explanation:

Bucket name.

Restrictions:

  • A bucket name must be unique across all accounts and regions.
  • A bucket name:
    • Must be 3 to 63 characters long and start with a digit or letter. Lowercase letters, digits, hyphens (-), and periods (.) are allowed.
    • Cannot be formatted as an IP address.
    • Cannot start or end with a hyphen (-) or period (.).
    • Cannot contain two consecutive periods (..), for example, my..bucket.
    • Cannot contain periods (.) and hyphens (-) adjacent to each other, for example, my-.bucket or my.-bucket.
  • If you repeatedly create buckets of the same name in the same region, no error will be reported and the bucket attributes comply with those set in the first creation request.

Default value:

None

objectKey

String

Yes

Explanation:

Object name. An object is uniquely identified by an object name in a bucket. An object name is a complete path that does not contain the bucket name.

For example, if the address for accessing the object is examplebucket.obs.ap-southeast-1.myhuaweicloud.com/folder/test.txt, the object name is folder/test.txt.

Value range:

The value must contain 1 to 1,024 characters.

Default value:

None

downloadFile

String

No

Explanation:

Full path of the local directory to which the object is downloaded.

Default value:

If the value of this parameter is null, the downloaded object is saved in the directory where the program is executed.

partSize

long

No

Explanation:

Part size.

Value range:

The value ranges from 100 KB to 5 GB, in bytes.

Default value:

9 MB

taskNum

int

No

Explanation:

Maximum number of parts that can be downloaded concurrently in a multipart download

Value range:

The value must be greater than 0, but not exceed the file size divided by the part size, rounded up.

Default value:

1, indicating concurrent downloads are not used.

enableCheckpoint

boolean

No

Explanation:

Whether to enable the resumable mode.

Value range:

true: The resumable mode is enabled.

false: The resumable mode is disabled.

Default value:

false

checkpointFile

String

No

Explanation:

Path of a file generated for recording the progress of a resumable download. The file contains the information about parts and the progress.

Restrictions:

This parameter is valid only in the resumable mode.

Default value:

If this parameter is left blank, the checkpoint file will be saved in the current directory.

enableCheckSum

boolean

No

Explanation:

Whether to verify the file to upload. If this function is enabled, before each task restarts, the system verifies whether the file to upload is the one used during task initialization.

Restrictions:

This parameter is valid only in the resumable mode.

Value range:

true: The file is verified.

false: The file is not verified.

Default value:

false

versionId

String

No

Explanation:

Object version ID, for example, G001117FCE89978B0000401205D5DC9.

Value range:

The value must contain 32 characters.

Default value:

None. If this parameter is left blank, the latest version of the object is obtained.

encodeHeaders

boolean

No

Explanation:

Whether to automatically decode the response headers.

Default value:

None

ifModifiedSince

Date

No

Explanation:

Returns the object if it has been modified since the specified time; otherwise, an error is returned.

Default value:

None

ifUnmodifiedSince

Date

No

Explanation:

Returns the object if it has not been modified since the specified time; otherwise, an error is returned.

Default value:

None

ifMatchTag

String

No

Explanation:

Returns the object content if the object's ETag is the same as the value of this parameter. Otherwise, an error is reported.

Default value:

None

ifNoneMatchTag

String

No

Explanation:

Returns the object content if the object's ETag is different from the value of this parameter. Otherwise, an error is reported.

Default value:

None

progressListener

ProgressListener

No

Explanation:

Data transmission listener for obtaining the download progress. For details, see Table 3.

encodeHeaders

boolean

No

Explanation:

Whether to enable OBS to automatically encode request headers.

Due to HTTP coding restrictions, only ASCII characters can be sent. If your request headers contain full-width characters, the SDK will URL encode these characters before sending the request. When you use a browser to access the object metadata, the browser automatically decodes the data.

Value range:

true: Encoding with SDK is enabled.

false: Encoding with SDK is disabled.

Default value:

true

Table 3 ProgressListener

Method

Return Value Type

Mandatory (Yes/No)

Description

progressChanged

void

Yes

Explanation:

Used for obtaining the progress. For details, see Table 4.

Default value:

None

Table 4 progressChanged

Parameter

Type

Mandatory (Yes/No)

Description

status

ProgressStatus

Yes

Explanation:

Progress data. For details, see Table 5.

Default value:

None

Table 5 ProgressStatus

Method

Return Value Type

Description

getAverageSpeed()

double

Average transmission rate.

getInstantaneousSpeed()

double

Instantaneous transmission rate.

getTransferPercentage()

int

Transmission progress, in percentage.

getNewlyTransferredBytes()

long

Number of the newly transmitted bytes.

getTransferredBytes()

long

Number of bytes that have been transmitted.

getTotalBytes()

long

Number of the bytes to be transmitted.

Table 6 StorageClassEnum

Constant

Default Value

Description

STANDARD

STANDARD

Standard storage class

WARM

WARM

Infrequent Access storage class.

COLD

COLD

Archive storage class.

Table 7 ObjectMetadata

Parameter

Type

Mandatory (Yes/No)

Description

contentLength

Long

No

Explanation:

Object size.

Default value:

If this parameter is not specified, the SDK automatically calculates the size of the object.

contentType

String

No

Explanation:

MIME type of the object file. MIME type is a standard way of describing a data type and is used by the browser to decide how to display data.

Value range:

See What Is Content-Type (MIME)? (Java SDK)

Default value:

If this parameter is not specified, the SDK determines the file type based on the suffix of the object name and assigns a value to the parameter. For example, if the suffix of the object name is .xml, the object is an application/xml file. If the suffix is .html, the object is a text/html file.

contentEncoding

String

No

Explanation:

Content-Encoding header in the response. It specifies which encoding is applied to the object.

Default value:

None

contentDisposition

String

No

Explanation:

Provides a default file name for the requested object. When the object with the default file name is being downloaded or accessed, the content is displayed as part of a web page in the browser or as an attachment in a download dialog box.

Default value:

None

cacheControl

String

No

Explanation:

Cache-Control header in the response. It specifies the cache behavior of the web page when an object is downloaded.

Default value:

None

contentLanguage

String

No

Explanation:

Language or language combination for visitors to customize and use. For details, see the definition of ContentLanguage in the HTTP protocol.

Default value:

None

expires

String

No

Explanation:

The time a cached web page object expires.

Restrictions:

The time must be in the GMT format.

Default value:

None

contentMd5

String

No

Explanation:

Base64-encoded MD5 value of the object data. It is provided for the OBS server to verify data integrity. The OBS server will compare this MD5 value with the MD5 value calculated based on the object data. If the two values are not the same, HTTP status code 400 is returned.

Restrictions:

  • The MD5 value of the object must be Base64 encoded.
  • If the MD5 value is not specified, the OBS server will not verify the MD5 value of the object.

Value range:

Base64-encoded 128-bit MD5 value of the request body calculated according to RFC 1864.

Example: n58IG6hfM7vqI4K0vnWpog==

Default value:

None

storageClass

StorageClassEnum

No

Explanation:

Storage class of an object that can be specified at object creation. If this header is not set, the default storage class of the bucket is used as the storage class of the object.

Value range:

See Table 6.

Default value:

None

webSiteRedirectLocation

String

No

Explanation:

If the bucket is configured with website hosting, the request for obtaining the object can be redirected to another object in the bucket or an external URL. This parameter specifies the address the request for the object is redirected to.

The request is redirected to an object anotherPage.html in the same bucket:

WebsiteRedirectLocation:/anotherPage.html

The request is redirected to an external URL http://www.example.com/:

WebsiteRedirectLocation:http://www.example.com/

Restrictions:

  • The value must start with a slash (/), http://, or https:// and cannot exceed 2 KB.
  • OBS only supports redirection for objects in the root directory of a bucket.

Default value:

None

nextPosition

long

No

Explanation:

Start position for the next append upload.

Value range:

0 to the object length, in bytes.

Default value:

None

appendable

boolean

No

Explanation:

Whether the object is appendable.

Value range:

true: The object is appendable.

false: The object is not appendable.

Default value:

None

userMetadata

Map<String, Object>

No

Explanation:

User-defined metadata of the object. To define it, you can add a header starting with x-obs-meta- in the request. In Map, the String key indicates the name of the user-defined metadata that starts with x-obs-meta-, and the Object value indicates the value of the user-defined metadata. To obtain the user-defined metadata of an object, use ObsClient.getObjectMetadata. For details, see Obtaining Object Metadata (SDK for Java).

Restrictions:

  • An object can have multiple pieces of metadata. The size of the metadata cannot exceed 8 KB in total.
  • When you call ObsClient.getObject to download an object, its user-defined metadata will also be downloaded.

Default value:

None

Responses

Table 8 DownloadFileResult

Parameter

Type

Description

objectMetadata

ObjectMetadata

Explanation:

Object metadata. For details, see Table 7.

Code Examples

This example downloads object objectname from bucket examplebucket to localfile using the resumable download.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
import com.obs.services.ObsClient;
import com.obs.services.exception.ObsException;
import com.obs.services.model.DownloadFileRequest;
import com.obs.services.model.DownloadFileResult;
public class DownloadFile001 {
    public static void main(String[] args) {
        // Obtain an AK/SK pair using environment variables or import the AK/SK pair in other ways. Using hard coding may result in leakage.
        // Obtain an AK/SK pair on the management console.
        String ak = System.getenv("ACCESS_KEY_ID");
        String sk = System.getenv("SECRET_ACCESS_KEY_ID");
        // (Optional) If you are using a temporary AK/SK pair and a security token to access OBS, you are advised not to use hard coding, which may result in information leakage.
        // Obtain an AK/SK pair and a security token using environment variables or import them in other ways.
        // String securityToken = System.getenv("SECURITY_TOKEN");
        // Enter the endpoint corresponding to the bucket. CN-Hong Kong is used here as an example. Replace it with the one in your actual situation.
        String endPoint = "https://obs.ap-southeast-1.myhuaweicloud.com";
        // Obtain an endpoint using environment variables or import it in other ways.
        //String endPoint = System.getenv("ENDPOINT");
        
        // Create an ObsClient instance.
        // Use the permanent AK/SK pair to initialize the client.
        ObsClient obsClient = new ObsClient(ak, sk,endPoint);
        // Use the temporary AK/SK pair and security token to initialize the client.
        // ObsClient obsClient = new ObsClient(ak, sk, securityToken, endPoint);

        try {
            DownloadFileRequest request = new DownloadFileRequest("examplebucket", "objectname");
            // Set the local path the object will be downloaded to.
            request.setDownloadFile("localfile");
            // Set the maximum number of parts that can be concurrently downloaded.
            request.setTaskNum(5);
            // Set the part size to 10 MB.
            request.setPartSize(10 * 1024 * 1024);
            // Enable the resumable upload.
            request.setEnableCheckpoint(true);
            // Perform a resumable download.
            DownloadFileResult result = obsClient.downloadFile(request);
            System.out.println("downloadFile successfully");
            System.out.println("Etag:" + result.getObjectMetadata().getEtag());
        } catch (ObsException e) {
            System.out.println("downloadFile failed");
            // Request failed. Print the HTTP status code.
            System.out.println("HTTP Code:" + e.getResponseCode());
            // Request failed. Print the server-side error code.
            System.out.println("Error Code:" + e.getErrorCode());
            // Request failed. Print the error details.
            System.out.println("Error Message:" + e.getErrorMessage());
            // Request failed. Print the request ID.
            System.out.println("Request ID:" + e.getErrorRequestId());
            System.out.println("Host ID:" + e.getErrorHostId());
            e.printStackTrace();
        } catch (Exception e) {
            System.out.println("downloadFile failed");
            // Print other error information.
            e.printStackTrace();
        }
    }
}