Help Center/ Dedicated Distributed Storage Service/ Service Overview/ Shared Disks and Usage Instructions
Updated on 2023-07-10 GMT+08:00
View PDF
Share

Shared Disks and Usage Instructions

DSS disks can be classified into non-shared disks and shared disks based on whether a disk can be attached to multiple servers. A non-shared disk can only be attached to one server, whereas a shared disk can be attached to multiple servers.

What Are Shared Disks?

Shared disks are block storage devices that support concurrent read/write operations and can be attached to multiple servers. Shared disks feature multiple attachments, high concurrency, high performance, and high reliability. A shared disk can be attached to a maximum of 16 servers. Figure 1 shows its application scenario.

Currently, shared disks can be used as data disks only and cannot be used as system disks.

Figure 1 Application scenario of shared disks

Application Scenarios and Precautions for Shared Disks

Shared disks are usually used for enterprise key applications that require cluster deployment and high availability (HA). These applications demand concurrent access to a disk from multiple servers. Before you attach a shared disk to multiple servers, the disk device type needs to be determined. The device type can be either VBD or SCSI.

Because most cluster applications, such as Windows MSCS, Veritas VCS, and Veritas CFS, require the usage of SCSI reservations, you are advised to use shared disks with SCSI. If a SCSI disk is attached to a Xen ECS for use, you must install the driver. For details, see Device Types and Usage Instructions.

You can create shared VBD disks or shared SCSI disks.
  • Shared VBD disks: The device type of a newly created shared disk is VBD by default. Such disks can be used as virtual block storage devices, but do not support SCSI reservations. If SCSI reservations are required for your applications, create shared SCSI disks.
  • Shared SCSI disks: These disks support SCSI reservations.
    • To improve data security, you are advised to use SCSI reservations together with the anti-affinity policy of an ECS group. That said, ensure that the shared SCSI disk is only attached to ECSs in the same anti-affinity ECS group.
    • If an ECS does not belong to any anti-affinity ECS group, you are advised not to attach shared SCSI disks to this ECS. Otherwise, SCSI reservations may not work properly, which may put your data at risk.

    Concepts of the anti-affinity ECS group and SCSI reservations:

    • The anti-affinity policy of an ECS group allows ECSs to be created on different physical servers to improve service reliability.

      For details about ECS groups, see Managing ECS Groups.

    • The SCSI reservation mechanism uses a SCSI reservation command to perform SCSI reservation operations. If an ECS sends such a command to a disk, the disk is displayed as locked to other ECSs, preventing the data damage that may be caused by simultaneous read/write operations to the disk from multiple ECSs.
    • ECS groups and SCSI reservations have the following relationship: A SCSI reservation on a single disk cannot differentiate multiple ECSs on the same physical host. For that reason, if multiple ECSs that use the same shared disk are running on the same physical host, SCSI reservations will not work properly. You are advised to use SCSI reservations only on ECSs that are in the same ECS group, thus having a working anti-affinity policy.

Advantages of Shared Disks

  • Multiple attachments: A shared disk can be attached to a maximum of 16 servers.
  • High-performance: When multiple servers concurrently access a shared ultra-high I/O disk, random read/write IOPS can reach up to 160,000.
  • High-reliability: Shared disks support both manual and automatic backup, delivering highly reliable data storage.
  • Wide application scenarios: Shared disks can be used for Linux RHCS clusters where only VBD disks are needed. Whereas, they can also be used for Windows MSCS and Veritas VCS clusters that require SCSI reservations.

Specifications of Shared Disks

Key metrics of the disk performance include read/write I/O latency, IOPS, and throughput.
  • IOPS: Number of read/write operations performed by a disk per second
  • Throughput: Amount of data read from and written into a disk per second
  • Read/write I/O latency: Minimum interval between two consecutive read/write operations of a disk
    Single-queue access latencies (4 KiB data blocks) of different types of disks are as follows:
    • Common I/O: 5 ms to 10 ms
    • High I/O: 1 ms to 3 ms
    • Ultra-high I/O: 1 ms
Table 1 Disk performance data

Parameter

Common I/O

High I/O

Ultra-high I/O

Max. capacity
  • System disk: 1,024 GB
  • Data disk: 32,768 GB
  • System disk: 1,024 GB
  • Data disk: 32,768 GB
  • System disk: 1,024 GB
  • Data disk: 32,768 GB

Max. IOPS

2,200

5,000

33,000

Max. throughput

90 MB/s

150 MB/s

350 MB/s

Burst IOPS limit

2,200

5,000

16,000

Formula used to calculate disk IOPS

NOTE:

Disk IOPS cannot exceed maximum IOPS. For example, the IOPS of an ultra-high I/O disk increases linearly as capacity grows (with a 50 IOPS increase for each GB added), but cannot exceed 33,000.

IOPS = Min. (2,200, 500 + 2 x Capacity)

IOPS = Min. (5,000, 1,200 + 6 x Capacity)

IOPS = Min. (33,000, 1,500 + 50 x Capacity)

API name

NOTE:

This API name indicates the value of the volume_type parameter in the disk API. It does not represent the type of the underlying hardware devices.

SATA

SAS

SSD

Data durability

99.9999999%

Number of servers that can be attached to

A shared disk can be attached to a maximum of 16 servers.
To test the performance of a shared disk, the following requirements must be met:
  • The shared disk must be attached to multiple servers (ECSs or BMSs).
  • If the shared disk is attached to multiple ECSs, these ECSs must belong to the same anti-affinity ECS group.

    If these ECSs fail to meet the anti-affinity requirement, the shared disk cannot reach the maximum performance.

Data Sharing Principle and Common Usage Mistakes of Shared Disks

A shared disk is essentially the disk that can be attached to multiple servers for use, which is similar to a physical disk in that the disk can be attached to multiple physical servers, and each server can read data from and write data into any space on the disk. If the data read/write rules, such as the read/write sequence and meaning, between these servers are not defined, data read/write interference between servers or other unpredictable errors may occur.

Though shared disks are block storage devices that provide shared access for servers, shared disks do not have the cluster management capability. You need to deploy a cluster system to manage shared disks. Common cluster management systems include Windows MSCS, Linux RHCS, Veritas VCS, and Veritas CFS.

If shared disks are not managed by a cluster system, the following issues may occur:
  • Data inconsistency caused by read/write conflicts

    When a shared disk is attached to two servers (server A and server B), server A cannot recognize the disk spaces allocated to server B, vice versa. That said, a disk space allocated to server A may be already used by server B. In this case, repeated disk space allocation occurs, which leads to data errors.

    For example, a shared disk has been formatted into the ext3 file system and attached to server A and server B. Server A has written metadata into the file system in space R and space G. Then server B has written metadata into space E and space G. In this case, the data written into space G by server A will be replaced. When the metadata in space G is read, an error will occur.

  • Data inconsistency caused by data caching

    When a shared disk is attached to two servers (server A and server B), the application on server A has read the data in space R and space G, then cached the data. At that time, other processes and threads on server A would then read this data directly from the cache. At the same time, if the application on server B has modified the data in space R and space G, the application on server A cannot detect this data change and still reads this data from the cache. As a result, the user cannot view the modified data on server A.

    For example, a shared disk has been formatted into the ext3 file system and attached to server A and server B. Both servers have cached the metadata in the file system. Then server A has created a new file (file F) on the shared disk, but server B cannot detect this modification and still reads data from its cached data. As a result, the user cannot view file F on server B.

Before you attach a shared disk to multiple servers, the disk device type needs to be determined. The device type can be either VBD or SCSI. Shared SCSI disks support SCSI reservations. Before using SCSI reservations, you need to install a driver in the server OS and ensure that the OS image is included in the compatibility list.