Preparing Virtualized GPU Resources

Prerequisites

Step 1: Enable GPU Virtualization

Both CCE AI Suite (NVIDIA GPU) and Volcano Scheduler must be installed in the cluster.

1. Log in to the CCE console and click the cluster name to access the cluster console. In the navigation pane, choose Settings.
2. Switch to the Heterogeneous Resources tab and enable GPU Virtualization. You can configure GPU virtualization by cluster or by node pool.
   • Cluster-level GPU virtualization:
     1. Enable GPU Settings > GPU Virtualization. After this function is enabled, GPU virtualization is enabled globally by default, and the compute power and GPU memory of a single GPU can be split.
     2. In Default Cluster Driver, select a driver that supports GPU virtualization.
     3. In the lower right corner of the page, click Confirm Settings.

   • Node pool-level GPU virtualization: If CCE AI Suite (NVIDIA GPU) of version 2.7.2 or later is installed, GPU virtualization can be configured by node pool.
     1. In Node Pool Configurations under GPU Settings, click Add.
     2. In the Node Pools list, select the node pool where you want to enable GPU virtualization and choose a driver that supports GPU virtualization from Driver. After you customize a GPU driver for a node pool, nodes in that pool will preferentially use the custom driver. Nodes for which no driver is specified will use the cluster's default driver.
        

        • The specified driver version will be installed for new nodes added to the pool. This configuration does not affect existing nodes in the pool.
        • An updated driver version applies only to new nodes added to the node pool. Existing nodes must be restarted to apply the changes.
     3. Click under GPU Virtualization to enable GPU virtualization for the node pool. To configure GPU virtualization for multiple node pools, click Add.
     4. In the lower right corner of the page, click Confirm Settings.

3. After configuring GPU virtualization, verify the settings.

   In the navigation pane, choose Cluster > Nodes. In the right pane, click the Nodes tab and find the node where GPU virtualization has been configured. In the Operation column of the target node, choose More > View YAML. If the node-status.volcano.sh/nvidia value in the YAML file is {"enableXGPU":true}, GPU virtualization has been configured on the node.

Step 2: Create a GPU Node

Create nodes that support GPU virtualization in the cluster to use the GPU virtualization function. For details, see Creating a Node or Creating a Node Pool. If there are GPU nodes in your cluster that meet the prerequisites requirements, skip this step.