Collection Diagnosis Dashboard Templates

Prerequisites

You must enable ICAgent diagnosis: Navigate to the Configuration Center of the LTS console, switch to the ICAgent Collection tab page, enable ICAgent Diagnosis. For details, see Setting ICAgent Collection.

Viewing ICAgent Collection Monitoring

1. Log in to the management console and choose Management & Deployment > Log Tank Service.
2. In the navigation pane, choose Dashboards.
3. Choose Collection Diagnosis Dashboard Templates under Dashboard Templates and click ICAgent Collection Monitoring to view the detailed charts.
   • Filter by log group ID. The associated query and analysis statement is:

     select loggroup from log where report_topic = 'icagent_profile' or report_topic = 'icagent_alarm' group by loggroup

   • Filter by log stream ID. The associated query and analysis statement is:

     select logstream from log where report_topic = 'icagent_profile' or report_topic = 'icagent_alarm' group by logstream

   • Raw Data Traffic. The associated query and analysis statement is:

     SELECT
      case 
       when diff [ 1 ] is null then '0' 
       when diff [ 1 ] > 1024 and diff [ 1 ] <= 1024*1024 then concat(round(diff [ 1 ]*1.0/1024,4),' KB')
       when diff [ 1 ] > 1024*1024 and diff [ 1 ] < 1024*1024*1024 then  concat(round(diff [ 1 ]*1.0/1024/1024,4),' MB')
       when diff [ 1 ] > 1024*1024*1024 and diff [ 1 ] < 1024*1024*1024*1024 then  concat(round(diff [ 1 ]*1.0/1024/1024/1024,4),' GB')
       when diff [ 1 ] > 1024*1024*1024*1024 then  concat(round(diff [ 1 ]*1.0/1024,4),' TB')
       else concat(round(diff [ 1 ]*1.0,2),' B') 
      END AS "Original Traffic",
      case 
       when diff [ 3 ] is null then 'No data yesterday' 
       else round(diff [ 3 ] - 1,2)
      END AS "Day-on-Day" 
     FROM
      (
      SELECT
       report_topic,
       compare ( traffic, 86400 ) AS diff 
      FROM
       ( SELECT report_topic, sum( read_bytes ) AS traffic FROM log WHERE report_topic = 'icagent_profile' GROUP BY report_topic ) 
     GROUP BY
      report_topic)

   • Collected Files. The associated query and analysis statement is:

     select diff[1] as "Collected Files" , case when diff[3] is not null then round(diff[3] -1 ,2) else 'No data yesterday' end as "Day-on-Day" from  (select compare(uv,86400) as diff from (select report_topic,count(distinct concat(file_name,host_ip)) as uv from log where report_topic = 'icagent_profile' group by report_topic) group by report_topic)

   • ICAgents (Day-on-Day). The associated query and analysis statement is:

     select diff[1] as "ICAgents" ,  case when diff[3] is not null then round(diff[3] -1 ,2) else 'No data yesterday' end as "Day-on-Day" from (select compare(uv,86400) as diff from (select report_topic,count(distinct host_ip) as uv from log where report_topic = 'icagent_profile' group by report_topic) group by report_topic)

   • Data Sending Traffic. The associated query and analysis statement is:

     SELECT
      "time",
       case 
       when traffic is null then 0 
       else round(traffic*1.0/1024/1024,2)
      END AS "Sending traffic (MB)"
      from
      (SELECT
       time_format( time_floor ( __time, 'PT5M' ), 'yyyy-MM-dd HH:mm' ) AS "time",
       sum( read_bytes ) AS "traffic" 
      FROM
       log 
      WHERE
       report_topic = 'icagent_profile' 
      GROUP BY
       "time")

   • ICAgent Writes. The associated query and analysis statement is:

     select time_floor(__time,'PT5M') as \"time\",sum(read_count) as \"Writes\" where report_topic = 'icagent_profile'  group by \"time\"

   • ICAgents. The associated query and analysis statement is:

     select time_floor(__time,'PT5M') as "time" , count(distinct host_ip) as "ICAgents" where report_topic = 'icagent_profile' group by "time"

   • Collected File Distribution. The associated query and analysis statement is:

     SELECT
       file_name AS "Collection Path",
       host_ip AS "IP",
       case 
        when traffic is null then '0' 
        when traffic > 1024 and traffic <= 1024*1024 then concat(round(traffic*1.0/1024,2),' KB')
        when traffic > 1024*1024 and traffic < 1024*1024*1024 then  concat(round(traffic*1.0/1024,2),' MB')
        when traffic > 1024*1024*1024 and traffic < 1024*1024*1024*1024 then  concat(round(traffic*1.0/1024,2),' GB')
        when traffic > 1024*1024*1024*1024 then  concat(round(traffic*1.0/1024,2),' TB')
       else concat(round(traffic*1.0,2),' B') 
      END AS "Collection Traffic"
      FROM
      (SELECT
       file_name,
       host_ip,
       sum( read_bytes ) AS "traffic" 
      WHERE
       "report_topic" = 'icagent_profile' 
      GROUP BY
       file_name,
       host_ip)

Viewing Overall ICAgent Status

1. Choose Collection Diagnosis Dashboard Templates under Dashboard Templates and click Overall ICAgent Status to view the detailed charts.
   • Active ICAgents. The associated query and analysis statement is:

     select diff[1] as "Active ICAgents",case when diff[2] is not null then diff[2] else 'No data yesterday' end as "Active ICAgents Yesterday" from  (select report_topic,compare(uv,86400) as diff from (SELECT   report_topic,COUNT(DISTINCT ip) as uv FROM log where report_topic = 'icagent_status' group by report_topic) group by report_topic)

   • Sending Delay/Times. The associated query and analysis statement is:

     select time_floor(__time,'PT5M') as \"time\" , sum(\"metric.lts_cost.below_100_ms\") as \"below_100_ms\" , sum(\"metric.lts_cost.100to500ms\") as \"100to500ms\" , sum(\"metric.lts_cost.500msto1s\") as \"500msto1s\" , sum(\"metric.lts_cost.1sto10s\") as \"1sto10s\", sum(\"metric.lts_cost.10ston\") as \"10ston\" from log where \"report_topic\" = 'icagent_status' group by \"time\"

   • Running Status Distribution. The associated query and analysis statement is:

     select status,count(DISTINCT ip) as pv from log where report_topic = 'icagent_status' group by status

   • CPU Trend. The associated query and analysis statement is:

     select ip,time_floor(__time,'PT5M') as \"time\",avg(\"metric.cpu_usage\") as \"CPU usage\" from log where report_topic = 'icagent_status' and \"metric.cpu_usage\" is not null group by \"time\",ip order by \"time\"

   • Overall ICAgent Status. The associated query and analysis statement is:

     select host_name as "Host",ip as "IP" , version as "Version" , os as "OS" , time_format(MILLIS_TO_TIMESTAMP(ANY_VALUE("metric.start_time")),'yyyy/MM/dd HH:mm:ss ZZZ') as "Local Time When ICAgent Started to Run",avg("metric.cpu_usage") as "CPU",avg("metric.mem_used")*1.0 as "Memory (MB)",status as "Running Status" where report_topic = 'icagent_status' group by host_name,ip,version,os,status

Viewing ICAgent Exception Monitoring

1. Choose Collection Diagnosis Dashboard Templates under Dashboard Templates and click ICAgent Exception Monitoring to view the detailed charts.
   • Filter by log group ID. The associated query and analysis statement is:

     select loggroup from log where report_topic = 'icagent_profile' or report_topic = 'icagent_alarm' group by loggroup limit 10000

   • Filter by log stream ID. The associated query and analysis statement is:

     select logstream from log where report_topic = 'icagent_profile' or report_topic = 'icagent_alarm' group by logstream limit 10000

   • Critical Errors. The associated query and analysis statement is:

     select diff[1] as "Errors", case when diff[3] is not null then round(diff[3] - 1 , 2) else 'No data yesterday' end as "Errors (Day-on-Day)" from (select report_topic,compare(pv , 86400) as diff from (select report_topic,count(1) as pv from log where report_topic = 'icagent_alarm' group by report_topic) group by report_topic)

   • Discarded Oversize Lines. The associated query and analysis statement is:

     select diff[1] as "Discarded Lines" ,  case when diff[3] is not null then round(diff[3] - 1 , 2) else 'No data yesterday' end as "Discarded Lines (Day-on-Day)" from (select report_topic,compare(pv , 400) as diff from (select report_topic,count(1) as pv from log where report_topic = 'icagent_alarm' and alarm_type = 'DISCARD_BIG_LINE' group by report_topic) group by report_topic)

   • Failed Requests to LTS. The associated query and analysis statement is:

     select diff[1] as "Failed Requests" , case when diff[3] is not null then round(diff[3] - 1 , 2) else 'No data yesterday' end as "Failed Requests (Day-on-Day)" from (select report_topic,compare(pv , 86400) as diff from (select report_topic,count(1) as pv from log where report_topic = 'icagent_alarm' and alarm_type = 'HTTP_REQUEST_ALARM' group by report_topic) group by report_topic)

   • Files with Excessive Issues. The associated query and analysis statement is:

     select diff[1] as "Files with Excessive Issues" ,case when diff[3] is not null then round(diff[3] - 1 , 2) else 'Files with Excessive Issues' end as "Files with Excessive Issues (Day-on-Day)" from (select  report_topic,compare(pv , 86400) as diff from (select  report_topic,count(1) as pv from log where report_topic = 'icagent_alarm' and alarm_type = 'COLLECT_FILE_EXCEED' group by report_topic) group by report_topic)

   • Critical Errors (Must Handle). The associated query and analysis statement is:

     select TIME_FORMAT(MILLIS_TO_TIMESTAMP(ANY_VALUE(report_time/1000000)),'yyyy/MM/dd HH:mm:ss ZZ') as "Last Occurred" , loggroup as "Log Group ID" ,logstream as "Log Stream ID" ,  host_ip as "IP" , alarm_type as "Alarm Type", os as "System Type" , alarm_message as "Alarm Details" where report_topic = 'icagent_alarm' group by loggroup,logstream,host_ip,alarm_type,os,alarm_message limit 10000

   • Details of Discarded Oversize Lines. The associated query and analysis statement is:

     select TIME_FORMAT(MILLIS_TO_TIMESTAMP(ANY_VALUE(report_time/1000000)),'yyyy/MM/dd HH:mm:ss ZZ') as "Last Occurred" , loggroup as "Log Group ID" ,logstream as "Log Stream ID" ,  host_ip as "IP" , alarm_type as "Alarm Type", os as "System Type" , alarm_message as "Alarm Details" where report_topic = 'icagent_alarm' and alarm_type = 'DISCARD_BIG_LINE' group by loggroup,logstream,host_ip,alarm_type,os,alarm_message limit 10000

   • Details of Failed Requests to LTS. The associated query and analysis statement is:

     select TIME_FORMAT(MILLIS_TO_TIMESTAMP(ANY_VALUE(report_time/1000000)),'yyyy/MM/dd HH:mm:ss ZZ') as "Last Occurred" , loggroup as "Log Group ID" ,logstream as "Log Stream ID" ,  host_ip as "IP" , alarm_type as "Alarm Type", os as "System Type" , alarm_message as "Alarm Details" where report_topic = 'icagent_alarm'  and alarm_type = 'HTTP_REQUEST_ALARM' group by loggroup,logstream,host_ip,alarm_type,os,alarm_message limit 10000

   • Details of Files with Excessive Issues. The associated query and analysis statement is:

     select TIME_FORMAT(MILLIS_TO_TIMESTAMP(ANY_VALUE(report_time/1000000)),'yyyy/MM/dd HH:mm:ss ZZ') as "Last Occurred" , loggroup as "Log Group ID" ,logstream as "Log Stream ID" ,  host_ip as "IP" , alarm_type as "Alarm Type", os as "System Type" , alarm_message as "Alarm Details" where report_topic = 'icagent_alarm'  and alarm_type = 'COLLECT_FILE_EXCEED' group by loggroup,logstream,host_ip,alarm_type,os,alarm_message limit 10000