Fast Import of an Image File in Linux

Scenarios

This section describes how to convert the format of a large image file on a Linux server and then quickly import it to the cloud platform. You are advised to use a EulerOS ECS for converting image file formats and generating bitmap files.

In Linux, you are advised to use qemu-img-hw to convert image formats.

Prerequisites

• The image file has been initially configured as instructed in Table 2 based on the OS in the image file.

• You have created an ECS running EulerOS on the management console and bound an EIP to the ECS.
• An OBS bucket has been created on the management console.

Procedure

1. Upload an image file.
   • If the image file is uploaded from a Linux PC, run the scp command.

     For example, to upload image01.qcow2 to the /usr/ directory of the ECS, run the following command:

     scp /var/image01.qcow2 root@xxx.xxx.xx.xxx:/usr/

     xxx.xxx.xx.xxx indicates the EIP bound to the ECS.

   • If the image file is uploaded from a Windows PC, use a file transfer tool, such as WinSCP, to upload the image file.

2. Obtain the image conversion tool (qemu-img-hw.zip) and bitmap file generation tool (createMF.zip), upload them to the ECS, and decompress the packages.

   Table 1 Tool packages

   Tool Package	How to Obtain
   qemu-img-hw.zip	https://cn-south-1-cloud-reset-pwd.obs.cn-south-1.myhuaweicloud.com/imageImportTools/qemu-img-hw.zip
   createMF.zip	https://cn-south-1-cloud-reset-pwd.obs.cn-south-1.myhuaweicloud.com/imageImportTools/createMF.zip

3. Use qemu-img-hw to convert the image format.
   1. Go to the directory where qemu-img-hw is stored, for example, /usr/qemu-img-hw.

      cd /usr/qemu-img-hw

   2. Run the following command to make qemu-img-hw executable:

      chmod +x qemu-img-hw

   3. Execute qemu-img-hw to convert the image file format to ZVHD2 (recommended) or RAW.

      Command format:

      ./qemu-img-hw convert -p -O {target-image-format} {source-image-file} {target-image-file}

      For example, run the following command to convert an image01.qcow2 file to an image01.zvhd2 file:

      ./qemu-img-hw convert -p -O zvhd2 image01.qcow2 image01.zvhd2

      • If the image file is converted to the ZVHD2 format, go to 5.
      • If the image file is converted to the RAW format, go to 4.

4. Use CreateMF.jar to generate a bitmap file.
   1. Ensure that JDK has been installed on the ECS.

      Run the following commands to check whether JDK is installed:

      source /etc/profile

      java -version

      If a Java version is displayed, JDK has been installed.

   2. Run the following command to enter the directory where CreateMF.jar is stored:

      cd /usr/createMF

   3. Run the following command to generate a bitmap file:

      java -jar CreateMF.jar /<original-RAW-file-path>/<generated-.mf-file-path>

      Example:

      java -jar CreateMF.jar image01.raw image01.mf

      

      • The generated .mf bitmap file must have the same prefix as the RAW image file. For example, if the image file name is image01.raw, the generated bitmap file name is image01.mf.

5. Use s3cmd to upload the file(s) to an OBS bucket.
   1. Install s3cmd on the ECS.

      If s3cmd has been installed, skip this step.

      1. Run the following command to install setuptools:

         yum install python-setuptools

      2. Run the following command to install wget:

         yum install wget

      3. Run the following commands to obtain the s3cmd software package:

         wget https://github.com/s3tools/s3cmd/archive/master.zip

         mv master.zip s3cmd-master.zip

      4. Run the following commands to install s3cmd:

         unzip s3cmd-master.zip

         cd s3cmd-master

         python setup.py install

   2. Configure s3cmd.

      Run the following command to configure s3cmd:

      s3cmd --configure
      Access Key: Enter an AK.
      Secret Key: Enter an SK.
      Default Region: Enter the region where the bucket is located.
      S3 Endpoint: Refer to the OBS endpoint.
      DNS-style bucket+hostname:port template for accessing a bucket: Enter a server address with a bucket name, for example, mybucket.obs.myclouds.com.
      Encryption password: Press Enter.
      Path to GPG program: Press Enter.
      Use HTTPS protocol: Specifies whether to use HTTPS. The value can be Yes or No.
      HTTP Proxy server name: Specifies the proxy address used to connect the cloud from an external network. (If you do not need it, press Enter.)
      HTTP Proxy server port: Specifies the proxy port used to connect to the cloud from an external network (If you do not need it, press Enter.)
      Test access with supplied credentials? y
      (If "Success. Your access key and secret key worked fine :-)" is displayed, the connection is successful.)
      Save settings? y (Specifies whether to save the configurations. If you enter y, the configuration will be saved.)

      

      The configurations will be stored in /root/.s3cfg. If you want to modify these configurations, run the s3cmd --configure command to configure the parameters or run the vi .s3cfg command to edit the .s3cfg file.

   3. Run the following command to upload the ZVHD2 image file (or the RAW image file and its bitmap file) to an OBS bucket.

      s3cmd put image01.zvhd2 s3://mybucket/

      

      The .mf bitmap file must be in the same OBS bucket as the RAW image file.

6. Register a private image.

   You can register a private image using the converted ZVHD2 or RAW file on the console or using an API.

   Method 1: Register a private image on the console.

   The following example describes how to create a system disk image from an external image file:

   1. Log in to the IMS console.
   2. Set image parameters as required.

      Table 2 and Table 3 list the parameters in the Image Type and Source and Image Information areas, respectively.

      Figure 1 Image type and source
      

      Table 2 Parameters for configuring the image type and source

      Parameter	Description	Example Value
      Region	Resources in different regions cannot communicate with each other over an intranet. For low latency and fast access, select a region close to where your services will be provided.	-
      Type	Private images can be created or imported.	Import Image
      Image Type	A private image contains an OS or service data, preinstalled public applications, and a user's personal applications. Private images are only available to the users who created them. A private image can be a system disk image, data disk image, ISO image, or full-ECS image. A system disk image contains an OS and preinstalled software for various services. You can use a system disk image to create ECSs and migrate your services to the cloud. A data disk image contains only service data. You can use a data disk image to create EVS disks and use them to migrate your service data to the cloud. An ISO image is created from an external ISO image file. It is a special image that is not available on the ECS console. It can only be used to provision temporary cloud servers. A full-ECS image contains an OS, preinstalled software, and service data. A full-ECS image is created using differential backups and the creation takes less time than creating a system or data disk image that has the same disk capacity.	System disk image
      Select Image File	The image file must be converted to the ZVHD2 or RAW format and optimized. Select the bucket storing the ZVHD2 or RAW image file and then select the image file. For the RAW format, you also need to upload a bitmap file with the same name as the image file. NOTE: To learn how to convert image file formats and generate bitmap files, see Fast Import of an Image File.	xxx.zvhd2
      Fast Create	This parameter is available only when you select a ZVHD2 or RAW image file. This function enables fast image creation and supports import of large files (maximum: 1 TB). If you have already prepared an image file that meets the requirements, select Enable Fast Create and select the confirmation checkbox in the Image File Preparation row.	-

      Figure 2 Image information
      

      

      Table 3 Parameters for configuring image information

      Parameter	Description	Example Value
      Enable automatic configuration	If you select this option, the system will automatically check and optimize the image file. For details, see What Will the System Do to an Image File When I Use the File to Register a Private Image?	-
      Auto sign	If you select this option, the system will sign the image for high data reliability. Key Name: Select a key from the drop-down list. The key is used to sign the image. If there is no key available, log in to the DEW console and create a key. Key ID: After you select a key, its ID will be automatically displayed. NOTE: A signed image cannot be shared with other users. Check on the console to see if Auto sign is available.	-
      Function	Indicates whether the image is used to create an ECS or BMS. The value can be ECS system disk image or BMS system disk image.	ECS system disk image
      Architecture	The value can be x86 or ARM. NOTE: If the system identifies that the architecture in the image file is different from what you set here, the identified architecture will be used. If the system fails to identify an architecture, the architecture you set will be used.	x86
      Boot Mode	This parameter is optional. Select the same boot mode as that in the image file, or the ECS created using this image cannot be booted. After you select the boot mode, the cloud platform will configure the corresponding boot settings for the image. The boot mode can be BIOS or UEFI. For details about the differences between them, see How Is BIOS Different from UEFI? For details about which OSs support UEFI boot, see OSs supporting UEFI Boot Mode. NOTE: If the image file uses the Arm architecture, only the UEFI mode is available.	BIOS
      OS	To ensure that the image can be created and used properly, select an OS consistent with that in the image file. If you do not select an OS, the system attempts to automatically identify the OS in the image file. NOTE: If the system identifies that the OS in the image file is different from the one you select here, the identified OS will be used instead. If the system fails to identify an OS, the OS you select will be used. If the OS you selected or the OS identified by the system is different from the actual OS, the performance of the ECSs created using the image file may be affected.	-
      License Type	Bring Your own license (BYOL) is supported. If you already have a license for the OS, you can directly use the license after creating an ECS and do not need to apply for a new one. This parameter is only available for Windows.	-
      System Disk	Set the system disk capacity (value range: 40 GiB to 1,024 GiB). This value must be no less than the size of the system disk in the image file. NOTE: If the uploaded VHD image is generated using qemu-img or similar tools, check the system disk capacity by referring to Why Did My VHD Upload Fail? Why Does the System Say the System Disk in the VHD Image File Is Larger Than What I Specified on the Management Console?	-
      Data Disk	You can also add data disks to the image. You need to obtain an image file containing data disks in advance. This function is used to migrate VMs and data disks from other platforms to the current platform. To add a data disk, click , configure the data disk size, and click Select Image File. In the displayed dialog box, select the target bucket and then the target image file containing the data disk. NOTE: A maximum of three data disks can be added.	-
      Name	Set an image name that is easy to identify.	-
      Encryption	This parameter is optional. If you want to encrypt the image, select KMS encryption and select the key to be used from the key list. After you select KMS encryption, the system will create a default key ims/default. You can also select a key from the key list. For details about how to encrypt an image, see Introduction.	-
      Enterprise Project	This parameter is for enterprise users. To use this function, contact customer service to enable it. An enterprise project groups cloud resources, so you can manage resources and users by project. The default project is default. Select an enterprise project from the drop-down list. For more information, see Accessing the Enterprise Center.	default
      Tag	This parameter is optional. Set tag keys and values for the image to make identification and management easier. You are advised to create predefined tags in TMS. For details, see Creating Predefined Tags. NOTE: If your organization has configured tag policies for images, you need to add tags to your images based on the policies. Otherwise, tags may fail to be added. For details about image tag format, see Image Tag Format.	-
      Description	This parameter is optional. Enter a description for the image.	-
      Agreement	Read and agree to the Image Disclaimer	-

      Click Next. Confirm the settings and click Submit.

   3. Go back to the private image list. The image is successfully registered when its status becomes Normal.

      If you add data disks during image creation, a system disk image and data disk images will be generated. The number of data disk images depends on the number of data disks you add (a maximum of 3).

      

      The time required for registering an image file as a private image varies depending on the image file size.

   Method 2: Register a private image using an API.

   The API is POST /v2/cloudimages/quickimport/action.

   For details about how to call this API, see Importing an Image File Quickly.

Appendix 1: Common qemu-img-hw Commands

• Convert the format of the image file.

  qemu-img-hw convert -p -O {target-image-format} {source-image-file} {target-image-file}

  The parameters are described as follows:

  -p: indicates the conversion progress.

  The part following -O (which must be in upper case) consists of the target image format, source image file, and target image file.

  For example, run the following command to convert a QCOW2 image file to a ZVHD2 file:

  qemu-img-hw convert -p -O zvhd2 test.qcow2 test.zvhd2

• Query image file information.

  qemu-img-hw info {image-file}

  Example:

  qemu-img-hw info test.zvhd2

• View help information.

  qemu-img-hw -help

Appendix 2: Common qemu-img-hw Errors

• Symptom:
  The following information is displayed when you run the qemu-img-hw command:

  ./qemu-img-hw: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by ./qemu-img-hw)

  Solution:

  Run the strings /lib64/libc.so.6 | grep GLIBC command to check the GLIBC version. If the version is too early, install the latest version. Run the following commands in sequence:

  wget http://ftp.gnu.org/gnu/glibc/glibc-2.15.tar.gz
  wget http://ftp.gnu.org/gnu/glibc/glibc-ports-2.15.tar.gz
  tar -xvf glibc-2.15.tar.gz
  tar -xvf glibc-ports-2.15.tar.gz
  mv glibc-ports-2.15 glibc-2.15/ports
  mkdir glibc-build-2.15
  cd glibc-build-2.15
  ../glibc-2.15/configure --prefix=/usr --disable-profile --enable-add-ons --with-headers=/usr/include --with-binutils=/usr/bin

  

  If "configure: error: no acceptable C compiler found in $PATH" is displayed, run the yum -y install gcc command.

  make
  make install

• Symptom:

  The following information is displayed when you run the qemu-img-hw command:

  ./qemu-img-hw: error while loading shared libraries: libaio.so.1: cannot open shared object file: No such file or directory

  Solution:

  Run the following command:

  yum install libaio

• Symptom:

  The following information is displayed when you run the qemu-img-hw command:

  ./qemu-img-hw: error while loading shared libraries: libcrypto.so.10: cannot open shared object file: No such file or directory

  Solution: Run openssl version to check whether the OpenSSL version is later than 1.0. If yes, install OpenSSL 1.0.

  Run the following commands to install OpenSSL 1.0:

  wget https://github.com/openssl/openssl/releases/download/OpenSSL_1_0_2k/openssl-1.0.2k.tar.gz
  tar -xvf openssl-1.0.2k.tar.gz
  cd openssl-1.0.2k
  ./config -d shared --prefix=/tmp/openssl (custom installation directory)
  make
  make install
  cp /tmp/openssl/lib/libcrypto.so.1.0.0 /usr/lib64/libcrypto.so.10

  Delete the /usr/lib64/libcrypto.so.10 file if you no longer need it.