Help Center> Object Storage Service> Best Practices> Verifying Data Consistency> Overview
View PDF

Overview

Background

Data inconsistency may occur due to network hijacking, cache, and other reasons during object upload and download.

Solution

OBS verifies data consistency by calculating the MD5 value when data is uploaded or downloaded. By default, OBS does not automatically verify data consistency. You can enable data consistency verification when uploading and downloading objects. Methods are explained in the following table.

  • The consistency verification methods are compatible with each other. Specifically, you can use one method to verify data consistency when uploading objects, and use another method to verify data consistency during download of the objects.
  • During object download, the MD5 verification takes effect only when the object to be downloaded has an MD5 value.
Table 1 Data consistency verification methods

Method

Description

Operation Guide

obsutil

With obsutil, a command line tool, you can run a simple command to upload or download objects and specify whether to enable MD5 verification.

Using obsutil to Verify Data Consistency During Upload

Using obsutil to Verify Data Consistency During Download

OBS Browser+

With OBS Browser, a GUI-based tool, you can enable or disable MD5 verification through a few clicks. OBS Browser also provides task management that allows you to view the verification status.

Using OBS Browser+ to Verify Data Consistency During Upload

Using OBS Browser+ to Verify Data Consistency During Download

OBS SDK

If you are a developer, you can use the OBS SDK to conduct secondary development for MD5 verification, and then proceed with the verification result according to your service needs.

Using OBS SDKs to Verify Data Consistency During Upload

Using OBS SDKs to Verify Data Consistency During Download

When an object is being uploaded, OBS calculates the MD5 value of the object on the client before uploading the object. Then OBS server calculates the MD5 value of the uploaded object and compares it with the MD5 value carried in the upload request. If the two values are consistent, the upload is successful. If the two values are inconsistent, the upload fails. Figure 1 illustrates how to use the MD5 value to verify data consistency during upload.

Figure 1 Verifying data consistency during upload

When an object is being downloaded, OBS calculates the MD5 value of the downloaded object and compares it with its original MD5 value. If the two values are consistent, the download is successful. If the two values are inconsistent, the download fails. Figure 2 illustrates how to use the MD5 value to verify data consistency during download.

Figure 2 Verifying data consistency during download

Impact

Enabling MD5 verification will affect upload and download performance.