Help Center/ DataArts Studio/ User Guide/ DataArts Migration (Real-Time Jobs)/ Real-Time Migration Job O&M/ Viewing Monitoring Metrics
Updated on 2025-09-09 GMT+08:00
View PDF
Share

Viewing Monitoring Metrics

Scenario

After you start a real-time migration job, Cloud Eye automatically associates monitoring metrics of the job. This helps you learn the status of the job.

It takes some time for the system to obtain and transmit monitoring data, so the job status displayed on the Job Monitoring page is not up-to-date. If you just started a real-time migration job, wait for 5 to 10 minutes and then view the monitoring data.

Prerequisites

  • You have obtained required Cloud Eye permissions.
  • The real-time migration job is running properly. If it has been stopped or is abnormal, you can only view its monitoring metrics in the last seven days.
  • The real-time migration job has been running properly for about 10 minutes.

Supported Monitoring Metrics

Table 1 lists the monitoring metrics supported for real-time processing migration jobs.

Table 1 Monitoring metrics supported for real-time processing migration jobs

Metric ID

Metric Name

Description

Value Range

Unit

Conversion Rule

Monitored Object (Dimension)

Monitoring Period (Raw Data)

extract_latency

Source Database WAL Extract Lag

Latency of extracting WALs from the source database

≥ 0 ms

ms

N/A

Real-time processing migration job

1 minute

flink_read_records_per_second

Flink job data read rate

Data input rate of a Flink job for monitoring and debugging

≥ 0 record/s

record/s

N/A

Real-time processing migration job

1 minute

flink_write_records_per_second

Flink job data write rate

Data output rate of a Flink job for monitoring and debugging

≥ 0 record/s

record/s

N/A

Real-time processing migration job

1 minute

flink_read_records_total

Flink job total data read

Total number of data inputs of a Flink job for monitoring and debugging

≥ 0 record/s

record/s

N/A

Real-time processing migration job

1 minute

flink_write_records_total

Flink job total data write

Total number of output data records of Flink jobs for monitoring and debugging.

≥ 0 record/s

record/s

N/A

Real-time processing migration job

1 minute

flink_read_bytes_per_second

Flink job byte read rate

Input bytes of a Flink job per second

≥ 0 Byte/s

Byte/s

1024(IEC)

Real-time processing migration job

1 minute

flink_write_bytes_per_second

Flink job byte write rate

Output bytes of a Flink job per second

≥ 0 Byte/s

Byte/s

1024(IEC)

Real-time processing migration job

1 minute

flink_read_bytes_total

Flink job total read byte

Total number of input bytes of a Flink job

≥ 0 Byte

Byte

1024(IEC)

Real-time processing migration job

1 minute

flink_write_bytes_total

Flink job total write byte

Total number of output bytes of a Flink job

≥ 0 Byte

Byte

1024(IEC)

Real-time processing migration job

1 minute

flink_cpu_usage

Flink job CPU usage

CPU usage of a Flink job

0% to 100%

%

N/A

Real-time processing migration job

1 minute

flink_mem_usage

Flink job memory usage

Memory usage of a Flink job

0% to 100%

%

N/A

Real-time processing migration job

1 minute

flink_max_op_latency

Flink job max operator latency

Maximum operator latency of a Flink job

≥ 0 ms

ms

N/A

Real-time processing migration job

1 minute

flink_max_op_backpressure_level

Flink job max operator back pressure

Maximum operator back pressure value of a Flink job. The value ranges from 0 to 1. A larger value indicates severer back pressure.

0 to 1

-

N/A

Real-time processing migration job

1 minute

binlog_file_read_per_second

Binlog Files Read per Second

Number of binlog files read per second

≥ 0 count/s

count/s

N/A

Real-time processing migration job

1 minute

binlog_file_read_total

Total Binlog Files Read

Total number of binlog files read

≥ 0 count

count

N/A

Real-time processing migration job

1 minute

binlog_bytes_per_second

Binlog Bytes Read per Second

Number of binlog bytes read per second

≥ 0 byte/s

bytes/s

1024(IEC)

Real-time processing migration job

1 minute

binlog_bytes_total

Total Binlog Bytes Read

Total number of binlog bytes read

≥ 0 bytes

bytes

1024(IEC)

Real-time processing migration job

1 minute

insert_sink_num

Data Records Inserted at the Destination

Number of data records inserted at the destination

≥ 0 count

count

N/A

Real-time processing migration job

1 minute

update_sink_num

Data Records Updated at the Destination

Number of data records updated at the destination

≥ 0 count

count

N/A

Real-time processing migration job

1 minute

delete_sink_num

Data Records Deleted at the Destination

Number of data records deleted at the destination

≥ 0 count

count

N/A

Real-time processing migration job

1 minute

dirty_records_out

Dirty data records total

Number of dirty data records

≥ 0 count

records

N/A

Real-time processing migration job

1 minute

dirty_bytes_out

Dirty data bytes total

Number of dirty data bytes

≥ 0 bytes

bytes

1024(IEC)

Real-time processing migration job

1 minute

unsupport_ddls_num

Unsupported ddl statement count

Total number of unsupported DDL statements

≥ 0 count

count

N/A

Real-time processing migration job

1 minute

ignored_ddls_num

Ignored ddl statement count

Total number of ignored DDL statements

≥ 0 count

count

N/A

Real-time processing migration job

1 minute

job_retry_times

Job retry times

Number of job retries

≥ 0 count

count

N/A

Real-time processing migration job

1 minute

queue_job_launching_num

Jobs being submitted

Number of jobs being submitted in a queue

≥ 0 count

Count

N/A

Real-time processing migration job

6 minutes

queue_job_running_num

Running jobs

Number of running jobs in a queue

≥ 0 count

Count

N/A

Real-time processing migration job

6 minutes

queue_job_succeed_num

Finished jobs

Number of finished jobs in a queue

≥ 0 count

Count

N/A

Real-time processing migration job

6 minutes

queue_job_failed_num

Failed jobs

Number of failed jobs in a queue

≥ 0 count

Count

N/A

Real-time processing migration job

6 minutes

queue_job_cancelled_num

Canceled jobs

Number of canceled jobs in a queue

≥ 0 count

Count

N/A

Real-time processing migration job

6 minutes

elastic_resource_pool_cpu_usage

CPU usage (elastic resource pool)

Percentage of used CPUs to total CPUs of a resource pool

0% ~ 100%

%

N/A

Real-time processing migration job

6 minutes

elastic_resource_pool_mem_usage

Memory usage (elastic resource pool)

Percentage of the used memory to the total memory of a resource group

0% ~ 100%

%

N/A

Real-time processing migration job

6 minutes

elastic_resource_pool_disk_usage

Disk usage (elastic resource pool)

Percentage of the used disks to the total disks of a resource group

0% ~ 100%

%

N/A

Real-time processing migration job

6 minutes

elastic_resource_pool_disk_max_usage

Max disk usage (elastic resource pool)

Maximum disk usage of a resource group

0% ~ 100%

%

N/A

Real-time processing migration job

6 minutes

elastic_resource_pool_cu_num

CU usage (elastic resource pool)

CUs used by an elastic resource pool

≥ 0 count

Count

N/A

Real-time processing migration job

6 minutes

elastic_resource_pool_min_cu_num

Mini CUs (elastic resource pool)

Minimum CUs allowed for an elastic resource pool

≥ 0 count

Count

N/A

Real-time processing migration job

6 minutes

elastic_resource_pool_max_cu_num

Max CUs (elastic resource pool)

Maximum CUs allowed for an elastic resource pool

≥ 0 count

Count

N/A

Real-time processing migration job

6 minutes

elastic_resource_pool_alloc_cu_num

Allocated CUs (elastic resource pool)

Total number of elastic resource pool CUs that are allocated to queues

≥ 0 count

Count

N/A

Real-time processing migration job

6 minutes

apply_ddls_number

DDLs

Total number of DDL transactions written into the destination database

≥ 0 count

Count

N/A

Real-time processing migration job

1 minute

network_incoming_bytes_rate

Network Input Throughput

Incoming traffic in bytes per second

≥ 0 byte/s

byte/s

1024(IEC)

Real-time processing migration job

1 minute

network_outgoing_bytes_rate

Network Output Throughput

Outgoing traffic in bytes per second

≥ 0 byte/s

byte/s

1024(IEC)

Real-time processing migration job

1 minute

disk_read_bytes_rate

Disk Read Throughput

Number of bytes read from the disk per second

≥ 0 byte/s

byte/s

1024(IEC)

Real-time processing migration job

1 minute

disk_write_bytes_rate

Disk Write Throughput

Number of bytes written to the disk per second

≥ 0 byte/s

byte/s

1024(IEC)

Real-time processing migration job

1 minute

disk_util

Storage Space Usage

Storage space usage of the monitored object

0% ~ 100%

%

N/A

Real-time processing migration job

1 minute

extract_bytes_rate

Source Database Read Throughput

Table data or WAL bytes read from a source database per second

≥ 0 byte/s

byte/s

1024(IEC)

Real-time processing migration job

1 minute

extract_rows_rate

Rows Read from Source Database per Second

Number of table data rows or WAL rows read from a source database per second

≥ 0 row/s

row/s

N/A

Real-time processing migration job

1 minute

apply_bytes_rate

Destination Database Write Throughput

Number of bytes written to the destination database per second

≥ 0 byte/s

byte/s

1024(IEC)

Real-time processing migration job

1 minute

apply_rows_rate

Rows Written into Destination Database per Second

Number of rows written into the destination database per second

≥ 0 row/s

row/s

N/A

Real-time processing migration job

1 minute

lag_records

Unconsumed Data

Number of unconsumed data records

≥ 0 records

records

N/A

Real-time processing migration job

1 minute

insert_num

Insert count

Number of data insertions

≥ 0 count

Count

N/A

Real-time processing migration job

1 minute

update_num

Update count

Number of data updates

≥ 0 count

Count

N/A

Real-time processing migration job

1 minute

delete_num

Delete count

Number of data deletion times

≥ 0 count

Count

N/A

Real-time processing migration job

1 minute

apply_transactions_rate

DML TPS

Number of DML transactions written into the destination database per second

≥ 0 transaction/s

transaction/s

N/A

Real-time processing migration job

1 minute

apply_ddls_rate

DDL TPS

Number of DDL transactions written into the destination database per second

≥ 0 record/s

record/s

N/A

Real-time processing migration job

1 minute

apply_latency

Data Synchronization Latency

Data synchronization latency

≥ 0 ms

ms

N/A

Real-time processing migration job

1 minute

Procedure

  1. Log in to the DataArts Studio console by following the instructions in Accessing the DataArts Studio Instance Console.
  2. On the DataArts Studio console, locate a workspace and click DataArts Factory.
  3. In the left navigation pane of DataArts Factory, choose Monitoring > Job Monitoring.
  4. On the Real-Time Migration Jobs page, click a job name.
    Figure 1 Monitoring a real-time migration job
  5. On the details page, click the Monitoring Information tab. You can view some key metrics of the job at the bottom of the page.
    Figure 2 Key metrics
  6. Click View Monitoring Metrics to go to the Cloud Eye console and view monitoring metrics.
    Figure 3 Viewing monitoring metrics

    For more information about monitoring metrics, see the Cloud Eye User Guide.