Installing CUDA-related Software (PyTorch + CPU/GPU) on Ubuntu 20.04

Prerequisites

You have signed up for a Huawei Cloud account, completed real-name authentication, and granted permissions before using ModelArts. For details, see 1. Preparations: Accounts and Permissions.

Billing

This operation involves costs for ECS, SWR, and notebook. Release resources promptly after completion to avoid continuous billing. For details about the billing, see ECS Billing, SWR Billing, and Notebook Billing.

Specifications and Constraints for Custom Notebook Images

When creating a custom image, the base image must meet the following specifications:

• Produced based on official open-source images such as the NVIDIA official website or Docker Hub. The open-source image must meet the following operating system constraints:

  x86: Ubuntu 22.04, Ubuntu 20.04

• You are advised to install CUDA 12.0 or later. This document uses ubuntu20.04-cuda12.4 as an example. For details, see the NVIDIA official website.
• The Dockerfile must contain ma-user-related configurations and the ipykernel dependency package; otherwise, notebook-related functions cannot be used.
  

  ma-user is the system user for ModelArts notebook runtime. The image must pre-configure this user and its directory permissions.

• If the image has Jupyter static files, the notebook might not open. Run the jupyter --path command to see the relevant directories and delete the static files there.
• Training debug mode: Ensure that the curl command exists.

Custom Image Build Workflow

1. Prepare a Linux server. In this example, an ECS is used.
2. Create a custom image on the ECS. The Dockerfile sample file is provided.
3. Upload the created image to SWR.
4. Register an SWR image on ModelArts.
5. Create a notebook instance and verify the new image.

Step 1: Preparing an ECS and Configuring the Environment

Purchase an x86-based Ubuntu server in the Elastic Cloud Server (ECS) console and configure an Elastic IP (EIP).

1. Log in to the ECS console, and click Buy ECS in the upper right corner of the Elastic Cloud Server page.

   This guide uses custom purchase as an example. Set CPU Architecture to x86, select Ubuntu 20.04 or 22.04 for OS, and configure other parameters as needed. For details, see Purchasing an ECS in Custom Config Mode.

2. Log in to the ECS via Cloud Shell or VNC. For details, see Logging In to a Linux ECS.
3. In the ECS, run the following commands to install Docker and enable it to start upon system startup:

   wget https://cnnorth4-modelarts-sdk.obs.cn-north-4.myhuaweicloud.com/modelarts/custom-image-build/install_docker_on_ubuntu.sh
   bash install_docker_on_ubuntu.sh
   source /etc/profile

   After the installation is complete, run the following command to check the Docker running status:

   systemctl status docker

   If active (running) is displayed in the output, Docker is running properly.

   Figure 1 Checking the Docker status
   
4. (Optional) Run the following command to check the Docker Engine version:

   docker version | grep -A 1 Engine

   If the version number is displayed in the output, Docker Engine has been successfully installed.

   Figure 2 Viewing the Docker Engine version
   

Step 2: Creating a Custom Image

This section describes how to install the required software packages using a Dockerfile and build a new notebook image.

• Basic installation packages: torchvision, torchaudio, ipykernel, and numpy
• CUDA-related installation packages: Miniconda3 and CUDA Toolkit

1. Query the base image for writing the Dockerfile (Skip this step for third-party images.)

   For details about how to obtain the official ModelArts image address, see ModelArts Official Images.

2. Access SWR.
   1. Log in to the SWR console, choose Dashboard in the navigation pane, and click Generate Login Command in the upper right corner. On the displayed page, click the copy button.
      The login command is used to complete the authentication between the Docker client and SWR, which is required for subsequently pushing images.

      Figure 3 Obtaining the login command
      
      

      Select the validity period as needed. To obtain a long-term valid login command, see Obtaining a Long-Term Login or Image Push/Pull Command. After you obtain a long-term valid login command, your temporary login commands will still be valid as long as they are in their validity periods.

   2. Run the login command on the server where Docker is installed. The message Login Succeeded will be displayed upon a successful login.
3. Write the container image Dockerfile.

   A Dockerfile is a text file with instructions to build a Docker image. It defines the image build process, such as the base image, software to install, files to copy, and environment variables.

   • Example Dockerfile for Official ModelArts Images (Ubuntu):

     # ModelArts official image address. Replace it as needed.
     FROM swr.cn-north-4.myhuaweicloud.com/atelier/pytorch_cuda:pytorch_1.12.1-cuda_10.2-py_3.9.11-ubuntu_20.04-x86_64-20260425154604-d9a1f9f
     ENV DEBIAN_FRONTEND=noninteractive
     USER root
     # section1: Use standard focal sources (x86_64 architecture), install system dependencies, and add the NVIDIA CUDA repository
     RUN mv /etc/apt/sources.list /etc/apt/sources.list.bak && \
         printf "deb http://archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse\n\
         deb http://archive.ubuntu.com/ubuntu/ focal-updates main restricted universe multiverse\n\
         deb http://archive.ubuntu.com/ubuntu/ focal-backports main restricted universe multiverse\n\
         deb http://security.ubuntu.com/ubuntu/ focal-security main restricted universe multiverse\n" > /etc/apt/sources.list && \
         apt-get update && \
         apt-get install -y zip wget ca-certificates vim python3 python3-pip tzdata gnupg && \
         ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && \
         dpkg-reconfigure --frontend noninteractive tzdata && \
         apt update && echo Y | apt install openssh-server && \
         wget https://developer.download.nvidia.cn/compute/cuda/repos/ubuntu2004/x86_64/cuda-keyring_1.1-1_all.deb && \
         dpkg -i cuda-keyring_1.1-1_all.deb && \
         rm -f cuda-keyring_1.1-1_all.deb && \
         apt-get update
     RUN rm /bin/sh && ln -s /bin/bash /bin/sh
     # section2: Install CUDA Toolkit 12.4
     RUN apt-get install -y cuda-toolkit-12-4
     # section3: Set CUDA environment variables
     ENV CUDA_HOME=/usr/local/cuda-12.4
     ENV PATH=${CUDA_HOME}/bin:$PATH
     ENV LD_LIBRARY_PATH=${CUDA_HOME}/lib64:$LD_LIBRARY_PATH
     # section4: Configure conda and pip Tsinghua mirrors and initialize conda
     RUN mkdir -p /home/ma-user/.pip && \
         echo -e "channels:\nshow_channel_urls: true\ndefault_channels:\n  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main\n  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r\n  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2\nremote_read_timeout: 120" > /home/ma-user/.condarc && \
         echo -e "[global]\nindex-url = https://pypi.tuna.tsinghua.edu.cn/simple\n[install]\ntrusted-host = pypi.tuna.tsinghua.edu.cn" > /home/ma-user/.pip/pip.conf && \
         /home/ma-user/anaconda3/bin/conda init bash && \
         source /home/ma-user/anaconda3/bin/activate base && \
         conda config --remove-key channels || true && \
         conda config --append channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main && \
         conda config --append channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r && \
         conda config --append channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2
     # section5: Create PyTorch 2.6.0 dedicated conda virtual environment (Python 3.9) and install CUDA-based PyTorch
     RUN /bin/bash -c '\
         source /home/ma-user/anaconda3/bin/activate base && \
         conda create -y --override-channels \
         -c https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main \
         -c https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r \
         -n pytorch_2_6 python=3.9.18 && \
         conda run -n pytorch_2_6 \
         pip install --index-url https://download.pytorch.org/whl/cu124 \
         torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 && \
         conda run -n pytorch_2_6 \
         pip install --index-url https://pypi.tuna.tsinghua.edu.cn/simple --trusted-host pypi.tuna.tsinghua.edu.cn \
         ipykernel==6.7.0 numpy==1.26.4 \
     '
     USER ma-user
     WORKDIR /home/ma-user

   • Example Dockerfile for non-official ModelArts images:
     

     If pulling a non-official ModelArts image and you encounter connection refused or Client.Timeout exceeded errors when accessing https://registry-1.docker.io/v2/, you need to configure a Docker proxy.

     1. Run the following command to edit the configuration file:

        vi /etc/docker/daemon.json

     2. Add the following configuration:

        {
        "registry-mirrors":[
        "https://docker.m.daocloud.io",
        "https://docker.jianmuhub.com",
        "https://huecker.io",
        "https://dockerhub.timeweb.cloud",
        "https://dockerhub1.beget.com",
        "https://noohub.ru"]
        }

     3. Run the following commands sequentially to make the configuration take effect:

        systemctl daemon-reload

        systemctl restart docker

     FROM ubuntu:20.04
     ENV DEBIAN_FRONTEND=noninteractive
     # section1: Create the ma-user user group and user
     USER root
     RUN default_user=$(getent passwd 1000 | awk -F ':' '{print $1}') || echo "uid: 1000 does not exist" && \
         default_group=$(getent group 100 | awk -F ':' '{print $1}') || echo "gid: 100 does not exist" && \
         if [ ! -z "${default_user}" ] && [ "${default_user}" != "ma-user" ]; then \
             userdel -r ${default_user}; \
         fi && \
         if [ ! -z "${default_group}" ] && [ "${default_group}" != "ma-group" ]; then \
             groupdel -f ${default_group}; \
         fi && \
         if ! getent group 100 > /dev/null; then \
             groupadd -g 100 ma-group; \
         fi && \
         useradd -d /home/ma-user -m -u 1000 -g 100 -s /bin/bash ma-user && \
         chmod -R 750 /home/ma-user
     # section2: Use standard focal sources (x86_64 architecture), install system dependencies, and add the NVIDIA CUDA repository
     RUN mv /etc/apt/sources.list /etc/apt/sources.list.bak && \
         printf "deb http://archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse\n\
         deb http://archive.ubuntu.com/ubuntu/ focal-updates main restricted universe multiverse\n\
         deb http://archive.ubuntu.com/ubuntu/ focal-backports main restricted universe multiverse\n\
         deb http://security.ubuntu.com/ubuntu/ focal-security main restricted universe multiverse\n" > /etc/apt/sources.list && \
         apt-get update && \
         apt-get install -y zip wget ca-certificates vim python3 python3-pip tzdata gnupg && \
         ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && \
         dpkg-reconfigure --frontend noninteractive tzdata && \
         apt update && echo Y | apt install openssh-server && \
         wget https://developer.download.nvidia.cn/compute/cuda/repos/ubuntu2004/x86_64/cuda-keyring_1.1-1_all.deb && \
         dpkg -i cuda-keyring_1.1-1_all.deb && \
         rm -f cuda-keyring_1.1-1_all.deb && \
         apt-get update
     RUN rm /bin/sh && ln -s /bin/bash /bin/sh
     # section3: Install CUDA Toolkit 12.4
     RUN apt-get install -y cuda-toolkit-12-4
     # section4: Set CUDA environment variables
     ENV CUDA_HOME=/usr/local/cuda-12.4
     ENV PATH=${CUDA_HOME}/bin:$PATH
     ENV LD_LIBRARY_PATH=${CUDA_HOME}/lib64:$LD_LIBRARY_PATH
     # section5: Install Miniconda3 as ma-user (x86_64 version)
     USER ma-user
     RUN cd /home/ma-user/ && \
         wget --no-check-certificate https://mirrors.tuna.tsinghua.edu.cn/anaconda/miniconda/Miniconda3-latest-Linux-x86_64.sh && \
         bash Miniconda3-latest-Linux-x86_64.sh -b -p /home/ma-user/anaconda3 && \
         rm -rf Miniconda3-latest-Linux-x86_64.sh
     # section6: Configure conda and pip Tsinghua mirrors and initialize conda
     RUN mkdir -p /home/ma-user/.pip && \
         echo -e "channels:\nshow_channel_urls: true\ndefault_channels:\n  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main\n  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r\n  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2\nremote_read_timeout: 120" > /home/ma-user/.condarc && \
         echo -e "[global]\nindex-url = https://pypi.tuna.tsinghua.edu.cn/simple\n[install]\ntrusted-host = pypi.tuna.tsinghua.edu.cn" > /home/ma-user/.pip/pip.conf && \
         /home/ma-user/anaconda3/bin/conda init bash && \
         source /home/ma-user/anaconda3/bin/activate base && \
         conda config --remove-key channels || true && \
         conda config --append channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main && \
         conda config --append channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r && \
         conda config --append channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2 && \
         conda tos accept --override-channels -c https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main && \
         conda tos accept --override-channels -c https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r && \
         conda tos accept --override-channels -c https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2
     # section7: Create PyTorch 2.6.0 dedicated conda virtual environment (Python 3.9) and install CUDA-based PyTorch
     RUN /bin/bash -c '\
         source /home/ma-user/anaconda3/bin/activate base && \
         conda create -y --override-channels \
         -c https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main \
         -c https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r \
         -n pytorch_2_6 python=3.9.18 && \
         conda run -n pytorch_2_6 \
         pip install --index-url https://download.pytorch.org/whl/cu124 \
         torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 && \
         conda run -n pytorch_2_6 \
         pip install --index-url https://pypi.tuna.tsinghua.edu.cn/simple --trusted-host pypi.tuna.tsinghua.edu.cn \
         ipykernel==6.7.0 numpy==1.26.4 \
     '
     USER ma-user
     WORKDIR /home/ma-user

4. Create the container image.

   In the directory containing the Dockerfile, run the following command to build the container image. Modify the SWR image address in the command as needed. The build time depends on network bandwidth and image size, typically taking around 30 minutes.

   docker build -t swr.cn-north-4.myhuaweicloud.com/dev-custom/pytorch2_6-cuda12.4.0:v1 -f Dockerfile .

   If Successfully built and Successfully tagged are displayed in the output, the image has been successfully built.

   Figure 4 Container image built successfully
   

Step 3: Pushing the Image to SWR

1. Run the following command to upload the locally built image to SWR:

   docker push swr.cn-north-4.myhuaweicloud.com/dev-custom/pytorch2_6-cuda12.4.0:v1

   If digest and the SHA256 hash value are displayed in the output, the image has been successfully uploaded to SWR.

   Figure 5 Image pushed successfully
   
2. In the navigation pane of the SWR console, choose My Images. Refresh the page to view the new image. Click the image name to view its details.

Step 4: Registering the Image with ModelArts

Register the new image with ModelArts image management so that you can use it in ModelArts. For details, see Image Registration and Management.

Set SWR Source to the image pushed to SWR in step 3. Since the installation packages above were completed under an x86-based image, select x86 for the architecture type, and GPU/CPU for the type.

Figure 6 Registering an image

Step 5: Creating a Notebook Instance and Verifying the New Image

1. Create a notebook instance. (Ensure that GPU resources are available in the current region.)
   1. In the left navigation pane on the ModelArts console, perform the following operations to go to the notebook management page:
      • New console: Choose Model Build > Notebook.
      • Old console: Choose Development Space > Notebook.
   2. In the upper right corner of the Notebook page, click Create.

      In the Resource area, select GPU/CPU for the instance specification. In the Environment area, select Custom Images, click , select the image and version registered in step 4, and select JupyterLab for WebIDE.

      For details about the parameters for creating a notebook instance, see Creating a Notebook Instance.

      Figure 7 Creating a notebook instance
      

      If the notebook instance status is Running, the notebook instance has been created.

2. On the Notebook page, click Access Environment in the Operation column. In the Access Method dialog box, click Access on the right of JupyterLab Access.
3. In the Notebook area on the ModelArts Launcher page, click pytorch_2_6 to create an IPYNB file.
   Figure 8 Opening pytorch_2_6
   
4. Import the PyTorch library and check the PyTorch version:

   import torch
   print(torch.__version__)

   If the following version information is displayed, PyTorch is installed.

   Figure 9 Checking the PyTorch version
   
5. Check whether CUDA is successfully installed.
   1. Right-click in the left directory area, click New File, and create a new script file named test.py.
      Figure 10 Creating a file
      
   2. Enter the following content in the test.py file:

      import torch
      # Check whether CUDA is available
      print(f"PyTorch version: {torch.__version__}")
      print(f"CUDA available: {torch.cuda.is_available()}")
      if torch.cuda.is_available():
      	print(f"GPU device: {torch.cuda.get_device_name(0)}")
      # Create tensors and transfer them to the GPU
      x = torch.randn(2, 2).cuda()
      y = torch.randn(2, 2).cuda()
      z = x.mm(y)
      print(z)

   3. In the left navigation pane, click and choose Terminal.
      Figure 11 Terminal
      
   4. Run the following command in the terminal to run the test.py file:

      python test.py

      If the following information is displayed, CUDA is successfully installed.

      Figure 12 CUDA installed successfully