What Is Workflow?

A workflow is a pipeline tool developed based on service scenarios for deploying models or applications. In ML, a pipeline may involve data labeling, data processing, model development and training, model evaluation, application development, and application evaluation.

Figure 1 Workflow

Different from traditional ML-based model building, workflows can be used to develop production pipelines. Based on MLOps, workflows enable runtime recording, monitoring, and continuous running. The development and continuous iteration of a workflow are separated in products based on roles and concepts.

A pipeline consists of multiple phases. The functions required by the pipeline and the function parameters are called through workflow SDKs. When developing a pipeline, you can use SDKs to describe phases and the relationships between phases. Developing a pipeline is the development state of the workflow. After a pipeline is determined, you can consolidate and provide it for others to use. You do not need to pay attention to what algorithms are used in the pipeline or how the pipeline is implemented. Instead, you only need to check whether the models or applications produced by the pipeline meet the release requirements. If not, you need to check whether the data and parameters need to be adjusted for iteration. Using such a consolidated pipeline is the running state of the workflow.

The development and running states of a workflow are as follows:

Development state: Workflow Python SDKs are used to develop and debug a pipeline.
Running state: You can configure and run a produced pipeline in visualized mode.

Leveraging DevOps principles and practices, workflows orchestrate ModelArts capabilities to help you efficiently train, develop, and deploy AI models.

Different functions are implemented in the development and running states of a workflow.

Workflow Development State

Based on service requirements, you can use Python SDKs provided by ModelArts workflows to offer each ModelArts capability as a step in a pipeline. This is a familiar and flexible development mode for AI developers. Python SDKs support:

Debugging: partially execution, fully execution, and debugging.
Release: Release a workflow from the development state to the running state.
Experiment record: for persistence and the management of experiments.

Workflow Running State

Workflows are executed in visualized mode. You only need to pay attention to some simple parameter settings, whether the model needs to be retrained, and model deployment.

Running workflows are released from the development state or subscribed to from AI Gallery.

A running workflow supports:

Unified configuration management: The parameters and resources required for a workflow are centrally managed.
Workflow operations: include starting, stopping, copying, and deleting workflows.
Running record: records historical running parameters and statuses of the workflow.

Workflow Components

A workflow is the description of a directed acyclic graph (DAG). You can develop a DAG through a workflow. A DAG consists of phases and the relationships between phases. To define a DAG, specify the execution content and sequence on phases. A green rectangle indicates a phase, and the link between phases shows the phase relationship. A DAG is actually an ordered job execution template.

Sample Workflows

ModelArts provides abundant scenario-oriented sample workflows. You can subscribe to them in AI Gallery.

Subscribing to and Using an AI Gallery Workflow

Log in to AI Gallery workflow cases.
On the workflow asset page of AI Gallery, select and subscribe to a workflow, read and agree to Data Security and Privacy Risk and Service Agreement of AI Gallery, and click Continue.
After the subscription, click Run. You will be automatically redirected to the ModelArts console. Select an asset version, workflow name, service region, and workspace, and click Import. The workflow details page is displayed.
Click Configure in the upper right corner. On the configuration page that appears, set parameters and click Save in the upper right corner to save the configuration.
Click Start in the top right corner to start the workflow.
On the workflow execution page, wait for the workflow to start running.
On the dashboard, check the status of each phase. The workflow runs automatically from one phase to the next until it finishes all the phases.