What Should I Do If the vdb Disk of a Node Is Damaged and the Node Cannot Be Recovered After Reset?

Symptom

The vdb disk of a node is damaged and the node cannot be recovered after reset.

Error Scenarios

• On a normal node, delete the LV and VG. The node is unavailable.
• Reset an abnormal node, and a syntax error is reported. The node is unavailable.

  The following figure shows the details.

  

Fault Locating

If the volume group (VG) on the node is deleted or damaged and cannot be identified, you need to manually restore the VG first to prevent your data disks from being formatted by mistake during the reset.

Solution

1. Log in to the node.
2. Create a PV and a VG again. In this example, the following error message is displayed:

   root@host1:~# pvcreate /dev/vdb
   Device /dev/vdb excluded by a filter

   This is because the added disk is created on another VM and has a partition table. The current VM cannot identify the partition table of the disk. You need to run the parted commands for three times to re-create the partition table.

   root@host1:~# parted /dev/vdb
   GNU Parted 3.2
   Using /dev/vdb
   Welcome to GNU Parted! Type 'help' to view a list of commands.
   (parted) mklabel msdos
   Warning: The existing disk label on /dev/vdb will be destroyed and all data on this disk will be lost. Do you want to continue?
   Yes/No? yes
   (parted) quit
   Information: You may need to update /etc/fstab.

   Run pvcreate again. When the system asks you whether to erase the DOS signature, enter y. The disk is created as a PV.

   root@host1:~# pvcreate /dev/vdb
   WARNING: dos signature detected on /dev/vdb at offset 510. Wipe it? [y/n]: y
   Wiping dos signature on /dev/vdb.
   Physical volume "/dev/vdb" successfully created

3. Create a VG.
   Check the Docker disks of the node. If the disks are /dev/vdb and /dev/vdc, run the following command:

   root@host1:~# vgcreate vgpaas /dev/vdb /dev/vdc

   If there is only the /dev/vdb disk, run the following command:

   root@host1:~# vgcreate vgpaas /dev/vdb

   After the creation is complete, reset the node.