Partial Execution

Workflows support predefined scenarios to enable partial execution. You can split the DAG into different branches based on the scenarios during workflow development. Then, you can run each branch independently as a separate workflow. The sample code is as follows:

workflow =wf.Workflow(
    name="image_cls",
    desc="this is a demo workflow",
    steps=[label_step, release_data_step, training_step, model_step, service_step],
    policy=wf.policy.Policy(
        scenes=[
            wf.policy.Scene(
                scene_name="Model training",
                scene_steps=[label_step, release_data_step, training_step]
            ),
            wf.policy.Scene(
                scene_name="Service deployment",
                scene_steps=[model_step, service_step]
            ),
        ]
    )
)

This example shows a workflow with five phases. The policy defines two preset scenarios: model training and service deployment. When the workflow is published to the running state, partial execution is disabled by default and all phases run. You can specify certain scenarios to run on the global configuration page.

You can define the same phase in different running scenarios using partial execution. However, you must ensure that the data dependency between phases is correct. Partial execution can only be configured and used in the running state and cannot be debugged in the development state.