Configuring Tasks

Background

A build task is the minimum unit and applies to simple service scenarios. However, build tasks may not meet complex requirements. For example:

• A multi-repository project needs to be built on multiple machines, and the build projects depend on each other.
• You want to split build task in a more modular and fine-grained manner and build them in the dependency sequence.

In the preceding build scenarios, BuildFlow can be used to assemble multiple interdependent build tasks in directed acyclic graph (DAG) mode. BuildFlow will parallelly execute these build tasks based on their dependency relationship.

BuildFlow Overview

The following is a BuildFlow example.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

version: 2.0 # The value must be 2.0.
params:
  - name: buildFlowParam
    value: buildFlowValue
buildflow:
 strategy: lazy # Defines the running policy of BuildFlow. The value can be lazy or eager.
jobs: # Build task
   - job: Job3 
     depends_on: # Define the job dependency. In the instance, Job3 depends on Job1 and Job2.
         - Job1
         - Job2
     build_ref: .cloudbuild/build3.yml # Define the YAML build script to run during a job build.
   - job: Job1
     build_ref: .cloudbuild/build1.yml
   - job: Job2
     build_ref: .cloudbuild/build2.yml

The BuildFlow contains the following key elements:

• version: version number, which is mandatory and unique. In the example file, the value of version is 2.0.
• params: global build parameters of BuildFlow. These parameters are shared by all jobs.
• strategy: running policy. There are two running modes. If there is no explicit definition, the Eager mode is used by default.
  • Lazy: A sub-job with a higher priority is triggered first. After successful execution, a sub-job with a lower priority is then triggered.
    

    The build takes a long time but saves resources. Therefore, you are advised to use this method when the number of parallel jobs is insufficient.

  • Eager: All sub-jobs are triggered synchronously. For sub-jobs that depend on other jobs, prepare the environment and code first and wait until the dependency jobs are successfully executed.
    

    Resources may be idle, but the build time can be shortened. You are advised to use this mode when the parallel job quota is sufficient.

• Jobs: jobs to be orchestrated. In the example file, there are three parameters under Jobs.
  • job: build task name, which can be customized.
  • depends_on: dependency build jobs.
  • build_ref: YAML build script that needs to be run during a build.

  In this example, three build jobs Job1, Job2, and Job3 are configured. The build jobs share the defined parameters (params), and Job3 depends on Job1 and Job2.

Introduction to BuildFlow Jobs

BuildFlow jobs are used to define jobs to be orchestrated in BuildFlow. Each job must have a unique name as the unique identifier.

• If sub-job A depends on sub-job B, B has a higher priority.
• Sub-jobs with the same priority are triggered synchronously.

The following is an example of BuildFlow jobs.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

buildflow:
  strategy: lazy
  jobs:
   - job: Job3
     depends_on:
         - Job1
         - Job2
     build_ref: .cloudbuild/build3.yml
   - job: Job1
     build_ref: .cloudbuild/build1.yml
   - job: Job2
     build_ref: .cloudbuild/build2.yml

As shown in the preceding information, Job3 depends on and has lower priority than Job1 and Job2, which are triggered synchronously.

Introduction to BuildFlow Parameters

BuildFlow params can define global parameters to be shared by all jobs. However, in some cases, the granularity of global parameters may be too large. You only need to define parameters on some jobs. You can also define parameters only for some jobs. Here is an example.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

buildflow:
 jobs:
   - job: Job3
     depends_on:
         - Build Job1
         - Build job2
     build_ref: .cloudbuild/build3.yml
   - job: Job1
     params:
       - name: isSubmodule
         value: true
     build_ref: .cloudbuild/build1.yml
   - job: Job2
     params:
       - name: isSubmodule
         value: true
     build_ref: .cloudbuild/build2.yml

As shown in the preceding information, global parameters (params) are not defined in BuildFlow. Instead, the isSubmodule parameter is defined in Job1 and Job2.

During build with YAML, parameters are used in the following priority:

1. Runtime parameters
2. Parameters configured for a build task
3. Parameters defined in the YAML files of BuildFlow sub-jobs
4. Parameters defined for the jobs in the YAML file of the BuildFlow parent task
5. Global parameters defined in the YAML file of the BuildFlow parent task