Kubeflow Add-on

Fundamental Concepts

For more details, see Kubeflow Pipeline Concepts.

Prerequisites

• A CCE standard or Turbo cluster v1.29 or later is available. For details, see Buying a Standard/Turbo Cluster.
• Before installing the add-on, ensure that the cluster's pods can access the Internet to download required images. You can configure an SNAT rule for the cluster to ensure that pods can access the Internet. You will be billed for the SNAT rule. For details, see NAT Gateway Price Calculator.
• In the use case described in this section, a NodePort Service is required to access the Kubeflow web UI. Ensure an EIP is bound to any node in the cluster. The EIP will be billed. For details, see Elastic IP Price Calculator.

Notes and Constraints

• Kubeflow pipelines run only on x86 nodes.
• This add-on is in initial rollout. Check the console for supported regions.
• This add-on is in OBT. You can access the latest features, but stability is not fully verified and no CCE SLA applies.

Installing the Add-on

1. Log in to the CCE console and click the cluster name to access the cluster console.
2. In the navigation pane, choose Add-ons. On the displayed page, locate Kubeflow and click Install.
3. On the Install Add-on page, configure the add-on specifications.

   Table 2 Add-on settings

   Parameter	Description
   Version	Select a version as needed.
   Add-on Specifications	Only the default specifications are supported.

4. In the Parameters area, properly configure the parameters below based on applications.

   Parameter	Format	Description
   minioAK	Length: at least 3 characters	The client identity used for MinIO storage authentication. Use minio or a custom account.
   minioSK	Length: at least 8 characters	An important credential, which is used with the AK.
   mysqlPassword	Complex, long password	MySQL root password. The password for the MySQL root user. Use a strong password.
   mysqlUsername	root (default value)	Database user. Retain the default value.

5. Click Install in the lower right corner. If the status is Running, the add-on has been installed.

Components

• All preceding components are deployed in the kubeflow namespace.
• Components like Minio and MySQL provide dependencies for Kubeflow Pipelines and can be replaced as needed.

MySQL and Minio backend PVs are bound to emptyDir volumes. If pods are migrated or restarted, unexpected results may occur.

Use Case

This case describes how to use the Kubeflow add-on to run a simple pipeline, as shown in Figure 1. The pipeline includes a whalesay task that runs an Alpine container executing echo "hello world" to stdout.

Figure 1 Execution process of the case

1. Install kubectl on an existing ECS and use kubectl to access the target cluster. For details, see Accessing a Cluster Using kubectl.
2. After the Kubeflow add-on is installed, CCE automatically creates the ml-pipeline-ui component to provide the frontend web interface. This component is automatically bound to a ClusterIP Service named ml-pipeline-ui. To access the UI, change the Service type of ml-pipeline-ui from ClusterIP to NodePort or LoadBalancer. The following uses a NodePort Service as an example.
   Run the following command to change the ClusterIP Service to a NodePort Service:

   kubectl edit svc ml-pipeline-ui -n kubeflow

   Set type to NodePort in the file and specify a node port that does not conflict with other services in the cluster. For details about NodePort Services, see NodePort.

   ...
   spec:
     ports:
       - name: http
         protocol: TCP
         port: 80
         targetPort: 3000
         nodePort: 30083 # Node port number
     selector:
       app: ml-pipeline-ui
       application-crd-id: kubeflow-pipelines
     clusterIP: 10.247.101.116
     clusterIPs:
       - 10.247.101.116
     type: NodePort # Change ClusterIP to NodePort.
     sessionAffinity: None
   ...

3. Enter http://<EIP of the target node>:<Node port>/ in the address bar of a browser to access the Kubeflow UI. On the UI, you can create and manage machine learning pipelines, define complete workflows including data preprocessing, model training, evaluation, and deployment. You can also configure and launch runs, monitor task execution in real time, and view logs and visualizations. This enables the automatic orchestration and execution of machine learning workflows.

   If the following page is displayed, the UI has been accessed.

   Figure 2 Kubeflow add-on UI
   

4. Create a pipeline.yaml file locally to configure a pipeline. This example deploys a pipeline that runs an Alpine container outputting "hello world" to verify your environment. For more information about pipelines, see Compile a Pipeline.
   pipeline.yaml file content:

   apiVersion: argoproj.io/v1alpha1
   kind: Workflow
   metadata:
     generateName: hello-world-
   spec:
     entrypoint: whalesay
     templates:
     - name: whalesay
       container:
         image: alpine
         command: [echo]
         args: ["hello world"]

5. Return to the UI and click Upload pipeline in the upper right corner to upload the pipeline.yaml file. After the configuration is complete, click Create in the lower part to create a pipeline.
   Figure 3 Uploading a pipeline file
   

6. After the pipeline is created, the whalesay task is automatically displayed on the UI. Click Create run in the upper right corner to create a run. After the configuration is complete, click Start in the lower part to run the pipeline.
   Figure 4 Creating a run
   

7. Click whalesay to view its status.
   Figure 5 Viewing the task status
   

Release History