Creating a Lineage Collection Task

Prerequisites

You have collected metadata.

Procedure

1. Sign in to the MgC console. In the navigation pane, under Project, select a big data migration project from the drop-down list.
2. In the navigation pane, choose Survey > Preparations.
3. Choose Metadata Management. Under the Big Data Lineage tab, click Create Collection Task.
   Figure 1 Creating a lineage collection task
   

4. Select a job type and configure the parameters shown.

   Type	Parameter	Configuration
   Lineage template	File	Download the lineage template to the local PC and set parameters in the template. The following fields are mandatory: Target Database (TargetDataset) Target Table (TargetTable) Target Connection Name (TargetConnectionName) Target Component Type (TargetComponentType) Upstream Database (SourceDataset) Upstream Table (SourceTable) Upstream Connection Name (SourceConnectionName) Upstream Component Type (SourceComponentType) Job ID (JobId) NOTICE: The value of Target Component Type and Upstream Component Type in the template can be Hive SQL or MaxCompute. Cells in the template cannot contain formulas. Otherwise, the parsing will fail. Go back to the console and click Select File to upload the saved file to MgC. CAUTION: The file size cannot exceed 100 MB.

   • Lineage template.
     1. Click Download Template to download the template to the local PC.
     2. Complete the lineage template. The following parameters are mandatory:
        • Target Database (TargetDataset)
        • Target Table (TargetTable)
        • Target Connection Name (TargetConnectionName)
        • Target Component Type (TargetComponentType)
        • Upstream Database (SourceDataset)
        • Upstream Table (SourceTable)
        • Upstream Connection Name (SourceConnectionName)
        • Upstream Component Type (SourceComponentType)
        • Job ID (JobId)
        

        • The value of Target Component Type and Upstream Component Type in the template can be Hive SQL or MaxCompute.
        • Cells in the template cannot contain formulas. Otherwise, the parsing will fail.
     3. Go back to the console and click Select File to upload the saved file to MgC.
        

        The file size cannot exceed 100 MB.

5. Click Confirm. The data lineage collection task is created. The system automatically starts collecting data lineage.
6. Click View Tasks. On the displayed page, you can view the collection task in the task list.
7. Wait until the task status changes to Completed. Then click View Lineage in the upper right corner of the page to view the Lineage Graph.