Configuring Tasks

Background

A build task is the minimum unit and applies to simple service scenarios. However, build tasks may not meet complex requirements. For example:

• A multi-repository project needs to be built on multiple machines, and the build projects depend on each other.
• You want to split build task in a more modular and fine-grained manner and build them in the dependency sequence.

In the preceding complex build scenarios, BuildFlow can be used to assemble multiple dependent build tasks in directed acyclic graph (DAG) mode. BuildFlow will concurrently build tasks based on the dependencies.

BuildFlow Overview

The following is a BuildFlow example.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

version: 2.0 # The value must be 2.0.
params:
  - name: buildFlowParam
    value: buildFlowValue
buildflow:
 strategy: lazy # Defines the running policy of BuildFlow. The value can be lazy or eager.
jobs: # Build task
   - job: Job3 
     depends_on: # Define the job dependency. In the instance, Job3 depends on Job1 and Job2.
         - Job1
         - Job2
     build_ref: .cloudbuild/build3.yml # Define the YAML build script to run during a job build.
   - job: Job1
     build_ref: .cloudbuild/build1.yml
   - job: Job2
     build_ref: .cloudbuild/build2.yml

The BuildFlow contains the following key elements:

• version: version number, which is mandatory and unique. In the example file, the value of version is 2.0.
• params: global build parameters of BuildFlow. This parameter is shared by all jobs.
• strategy: running policy. There are two running modes. If there is no explicit definition, the Eager mode is used by default.
  • Lazy: The build of a sub-job with a higher priority is triggered first. After the sub-job with a higher priority is successfully executed, the build of a sub-job with a lower priority is then triggered.
    

    The build takes a long time but saves build resources. Therefore, you are advised to use this method when the number of parallel jobs is insufficient.

  • Eager: Trigger the build of all sub-jobs synchronously. For sub-jobs that depend on other jobs, prepare the environment and code first and wait until the dependent jobs are successfully built.
    

    Resources may be idle, but the build time can be shortened. You are advised to use this function when the number of concurrent requests is large enough.

• Jobs: jobs to be orchestrated. In the example file, there are three parameters under Jobs.
  • job: build task name, which can be customized.
  • depends_on: build task on which the build job depends.
  • build_ref: YAML build script that needs to be run during a build.

  In this example, three build jobs Job1, Job2, and Job3 are configured. The build jobs share the defined parameters params, and Job3 depends on Job1 and Job2.

Introduction to BuildFlow Jobs

BuildFlow jobs are used to define jobs to be orchestrated in BuildFlow. Each job must have a unique name as the unique identifier.

• If sub-job A depends on sub-job B, B has a higher priority.
• Sub-jobs with the same priority are triggered synchronously.

BuildFlow jobs example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

buildflow:
  strategy: lazy
  jobs:
   - job: Job3
     depends_on:
         - Job1
         - Job2
     build_ref: .cloudbuild/build3.yml
   - job: Job1
     build_ref: .cloudbuild/build1.yml
   - job: Job2
     build_ref: .cloudbuild/build2.yml

As shown in the preceding information, Job3 depends on and has lower priority than Job1 and Job2, which are triggered synchronously.

Introduction to BuildFlow Parameters

BuildFlow params can define global parameters, that is, shared by all jobs. However, in some cases, the granularity of global parameters may be too large. You only need to define parameters on some jobs. You can also define parameters only for some jobs. Here is an example.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

buildflow:
 jobs:
   - job: Job3
     depends_on:
         - Build Job1
         - Build job2
     build_ref: .cloudbuild/build3.yml
   - job: Job1
     params:
       - name: isSubmodule
         value: true
     build_ref: .cloudbuild/build1.yml
   - job: Job2
     params:
       - name: isSubmodule
         value: true
     build_ref: .cloudbuild/build2.yml

As shown in the preceding information, global parameters (params) are not defined in BuildFlow. Instead, the isSubmodule parameter is defined in Job1 and Job2.

During Build with YAML, pay attention to the parameter priority.

Runtime parameters > Parameters configured in settings of a task > Parameters defined in the YAML file of the BuildFlow sub-jobs > Parameters defined in the job in the YAML file of the BuildFlow parent task > Global parameters defined in the YAML file of the BuildFlow parent task