Verifying Data Consistency During Upload

Method 1: Using obsutil to Verify Data Consistency During Upload

obsutil supports data consistency verification using the vmd5 parameter.

For example, to upload the test.txt file drive D on a Windows OS to bucket mytestbucket, run the following command to enable consistency verification:

obsutil cp D:\test.txt obs://mytestbucket/test.txt -vmd5

The object is uploaded after the verification passes, and Upload successfully is displayed in the command output.

Method 2: Using OBS Browser+ to Verify Data Consistency During Upload

By default, MD5 verification is disabled on OBS Browser+. To enable MD5 verification and upload objects through OBS Browser+, perform the following steps:

1. Log in to OBS Browser+.
2. Click in the upper right corner of displayed page and click Advanced Settings.
3. Select MD5 Verification. For details, see Figure 1.
   Figure 1 Configuring MD5 verification
   

4. Click OK.
5. Select the bucket to which the file is to be uploaded and upload the file.
   • If the MD5 verification is successful, the file will be uploaded successfully.
   • If the MD5 verification fails, the file upload will fail and the failure cause will be displayed on the task management page: failed to verify the file MD5 value.

Method 3: Using OBS SDKs to Verify Data Consistency During Upload

OBS provides SDKs for multiple programming languages such as Java and Python. You can specify the Content-MD5 parameter to enable consistency verification when uploading objects through any OBS SDK. For details about how to calculate and set the MD5 value of an object, see the setObjectMetadata API in the OBS SDK of the programming language that you use.

For example, to upload file text.txt from drive D on a Windows OS to bucket mytestbucket by using OBS Java SDK, you can use the following sample code to verify data consistency based on the MD5 value:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

String endPoint = "https://your-endpoint";
// Hard-coded or plaintext AK and SK are risky. For security purposes, encrypt your AK and SK and store them in the configuration file or environment variables. In this example, the AK and SK are stored in environment variables for identity authentication. Before running the code in this example, configure environment variables ACCESS_KEY_ID and SECRET_ACCESS_KEY_ID.
// Obtain an AK and SK pair on the management console. For details, see https://support.huaweicloud.com/intl/en-us/usermanual-ca/ca_01_0003.html.
String ak = System.getenv("ACCESS_KEY_ID");
String sk = System.getenv("SECRET_ACCESS_KEY_ID");
// Create an ObsClient instance.
ObsClient obsClient = new ObsClient(ak, sk, endPoint);
// Compute and set the MD5 value.
ObjectMetadata metadata = new ObjectMetadata();
File file = new File("D:\\text.txt");
FileInputStream fis = new FileInputStream(file);
InputStream is = (InputStream)fis;
String contentMd5 = obsClient.base64Md5(is);
metadata.setContentMd5(contentMd5);
// Upload a file that has the MD5 value.
obsClient.putObject("mytestbucket", "text.txt", file, metadata);

• The MD5 value of an object must be a Base64-encoded digest.
• The OBS server will compare this MD5 value with the MD5 value obtained by object data calculation. If the two values are not the same, the upload fails and the HTTP status code 400 is returned. If the two values are consistent, the object is successfully uploaded and the HTTP status code 200 is returned.