Help Center> Object Storage Service> Best Practices> Verifying Data Consistency> Solution Overview
Updated on 2023-12-20 GMT+08:00
View PDF

Solution Overview

Application Scenario

Data inconsistency may occur due to network hijacking, cache, and other reasons during object upload and download.

Solution Architecture

OBS verifies data consistency by calculating the MD5 value when data is uploaded or downloaded. By default, OBS does not automatically verify data consistency. You can enable consistency verification when uploading and downloading objects using the methods listed in the table below.

  • The consistency verification methods are compatible with each other. Specifically, you can use one method to verify data consistency when uploading objects, and use another method to verify data consistency during download of the objects.
  • During object download, the MD5 verification takes effect only when the object to be downloaded has an MD5 value.
  • Enabling MD5 verification will affect upload and download performance.
Table 1 Data consistency verification methods

Method

Description

Operation Guide

obsutil

With obsutil, a command line tool, you can run a simple command to upload or download objects and specify whether to enable MD5 verification.

Using obsutil to Verify Data Consistency During Upload

Using obsutil to Verify Data Consistency During Download

OBS Browser+

With OBS Browser, a GUI-based tool, you can enable or disable MD5 verification through a few clicks. OBS Browser also provides task management that allows you to view the verification status.

Using OBS Browser+ to Verify Data Consistency During Upload

Using OBS Browser+ to Verify Data Consistency During Download

OBS SDK

If you are a developer, you can use the OBS SDK to conduct secondary development for MD5 verification, and then proceed with the verification result according to your service needs.

Using OBS SDKs to Verify Data Consistency During Upload

Using OBS SDKs to Verify Data Consistency During Download

When you upload an object, OBS calculates the object's MD5 value on the client and uploads the object and its MD5 value to the OBS server. The OBS server then compares the MD5 value provided in the upload request to that it calculates for the uploaded object. If the two MD5 values match, the upload succeeds, or the upload fails. Figure 1 illustrates how to use the MD5 value to verify data consistency during upload.

Figure 1 Verifying data consistency during upload

When you download an object, OBS compares the MD5 value of the downloaded object to that it calculates for the object. If the two MD5 values match, the download succeeds, or the download fails. Figure 2 illustrates how to use the MD5 value to verify data consistency during download.

Figure 2 Verifying data consistency during download