Help Center> Object Storage Service> Best Practices> Verifying Data Consistency> Verifying Data Consistency During Upload
View PDF

Verifying Data Consistency During Upload

obsutil, OBS Browser+, and OBS SDKs support verification of data consistency during upload. You can select a method that meets your requirements. This topic provides guidance on how to use these methods to verify data consistency during object upload.

Using obsutil to Verify Data Consistency During Upload

obsutil supports data consistency verification through the additional parameter vmd5.

For example, to upload the test.txt file drive D on a Windows OS to bucket mytestbucket, run the following command to enable consistency verification:

obsutil cp D:\test.txt obs://mytestbucket/test.txt -vmd5

The object is uploaded after the verification passes, and Upload successfully is displayed in the command output.

Using OBS Browser+ to Verify Data Consistency During Upload

By default, MD5 verification is disabled on OBS Browser+. To enable MD5 verification and upload objects through OBS Browser+, perform the following steps:

  1. Log in to OBS Browser+.
  2. Click in the upper right corner of displayed page and click Advanced Settings.
  3. Select MD5 Verification. For details, see Figure 1.

    Figure 1 Configuring MD5 verification

  4. Click OK.
  5. Select the bucket to which the file is to be uploaded and upload the file.

    • If the MD5 verification is successful, the file will be uploaded successfully.
    • If the MD5 verification fails, the file upload will fail and the failure cause will be displayed on the task management page: failed to verify the file MD5 value.

Using OBS SDKs to Verify Data Consistency During Upload

OBS provides SDKs for multiple programming languages such as Java and Python. You can specify the Content-MD5 parameter to enable consistency verification when uploading objects through any OBS SDK. For details about how to compute and set the MD5 value of an object, see section "Setting Object Properties" in OBS SDK of the programming language that you use.

For example, to upload file text.txt from drive D on a Windows OS to bucket mytestbucket by using OBS Java SDK, you can use the following sample code to verify data consistency through the MD5 value:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
 
String endPoint = "https://your-endpoint";
String ak = "*** Provide your Access Key ***";
String sk = "*** Provide your Secret Key ***";
// Create an instance of ObsClient.
ObsClient obsClient = new ObsClient(ak, sk, endPoint);
// Compute and set the MD5 value.
ObjectMetadata metadata = new ObjectMetadata();
File file = new File("D:\\text.txt");
FileInputStream fis = new FileInputStream(file);
InputStream is = (InputStream)fis;
String contentMd5 = obsClient.base64Md5(is);
metadata.setContentMd5(contentMd5);
// Upload a file that has the MD5 value.
obsClient.putObject("mytestbucket", "text.txt", file, metadata);
  • The MD5 value of an object must be a Base64-encoded digest.
  • The OBS server will compare this MD5 value with the MD5 value obtained by object data calculation. If the two values are not the same, the upload fails and the HTTP status code 400 is returned. If the two values are consistent, the object is successfully uploaded and the HTTP status code 200 is returned.