Why Is the Value of Task configured for the OGG Source Different from the Actual Number of Running Tasks When Data Is Synchronized from OGG to Hudi?

Symptom

When the CDL task for synchronizing data from OGG to Hudi is executed, the value of tasks.max specified in the source end (ThirdKafka) is different from the actual number of running tasks.

For example, on the CDL web UI, the value of tasks.max of the source job ThirdKafka is 3.

However, the tasks information is id: 0, state: xxx is, indicating that there is only 1 task.

Possible Causes

The number of running tasks in OGG Source is calculated based on the smaller value between the values of source.topics and tasks.max.

Procedure

1. Log in to FusionInsight Manager and choose Cluster > Services > CDL. Click the hyperlink next to CDLService UI to access the CDLService web UI. On the data synchronization job list page, locate the row that contains the target job and choose More > Stop. After the job is stopped, choose More > Edit.
2. Change the value of tasks.max of Thirdk Kafka to the number of topics specified by source.topics and click Save.
3. On the data synchronization job list page, locate the row containing the target job and click Start to restart the job.