Verifying Data Consistency During Download

Prerequisites

The object to be downloaded has an MD5 value. Data consistency is not verified, if the object does not have an MD5 value. The MD5 value of an object needs to be computed and set when the object is uploaded. For details, see Verifying Data Consistency During Upload.

Method 1: Using obsutil to Verify Data Consistency During Download

obsutil supports data consistency verification through the additional parameter vmd5.

For example, to download the test.txt file from bucket mytestbucket to a Windows PC, perform the following steps to enable data consistency verification:

1. Run the following command to check whether the object to be downloaded has MD5 information:

   obsutil stat obs://test-bucket/test.txt

   • If the returned information contains MD5 information, as shown in the following figure, go to 2.

     

   • If MD5 information is not contained, consistency verification cannot be performed when the object is downloaded.

2. Run the following command to download the object:

   obsutil cp obs://mytestbucket/test.txt D:\test.txt -vmd5

   • The message highlighted in the following figure is displayed in the command output when the object is downloaded and the consistency verification passes.

     

   • If the object does not have an MD5 value, the object can be successfully downloaded but data consistency is not verified. The message highlighted in the following figure is displayed in the command output.

     

Method 2: Using OBS Browser+ to Verify Data Consistency During Download

By default, MD5 verification is disabled on OBS Browser+. To enable MD5 verification and download objects from OBS Browser+, perform the following steps:

1. Log in to OBS Browser+.
2. Click in the upper right corner of displayed page and click Advanced Settings.
3. Select MD5 Verification. For details, see Figure 1.
   Figure 1 Configuring MD5 verification
   

4. Click OK.
5. Select the bucket from which you want to download the object and download the object.
   • If the MD5 verification is successful, the file will be downloaded successfully.
   • If the MD5 verification fails, the file download will fail and the failure cause will be displayed on the task management page: failed to verify the file MD5 value.

Method 3: Using OBS SDKs to Verify Data Consistency During Download

The verification compares the MD5 value in the custom metadata of the object to be downloaded with that of the downloaded object to check whether the downloaded object is consistent with the original object.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

String endPoint = "https://your-endpoint";
// Hard-coded or plaintext AK and SK are risky. For security purposes, encrypt your AK and SK and store them in the configuration file or environment variables. In this example, the AK and SK are stored in environment variables for identity authentication. Before running the code in this example, configure environment variables ACCESS_KEY_ID and SECRET_ACCESS_KEY_ID.
// Obtain an AK and SK pair on the management console. For details, see https://support.huaweicloud.com/intl/en-us/usermanual-ca/ca_01_0003.html.
String ak = System.getenv("ACCESS_KEY_ID");
String sk = System.getenv("SECRET_ACCESS_KEY_ID");
// Create an ObsClient instance.
final ObsClient obsClient = new ObsClient(ak, sk, endPoint);
// Obtain the MD5 value of the object.
ObjectMetadata metadata = obsClient.getObjectMetadata("mytestbucket", "test.txt");
String md5Origin = metadata.getUserMetadata("contentMd5");
// Compute the MD5 value of the downloaded object.
Obsobject obsobject = obsClient.getObject("mytestbucket", "test.txt");
String md5Download = obsClient.base64Md5(obsobject.getObjectContent());
// Compare the MD5 values.
if(md5Origin.contentEquals(md5Download))
    System.out.println("Object MD5 validation passes!\n");
else
    System.out.println("Object MD5 validation failed!\n"); 

In the preceding sample code, the contentMd5 parameter used to obtain the MD5 value of an object is a customized metadata specified when the object is uploaded. Replace it with the one that you customized for the actual development.