LTS仪表盘模板
LTS仪表盘模板主要介绍OBS导入任务监控仪表盘、资源用量明细仪表盘、用量Top排序仪表盘。
- OBS导入任务监控仪表盘:支持查看OBS导入LTS的任务详情,例如任务ID、任务名称、成功条数、失败条数、处理速率等关键指标,帮助用户实时监控数据导入状态,快速定位异常问题,确保数据高效、完整导入LTS。
- 资源用量明细仪表盘:需要开通“可观测For LTS”功能,详细操作请参考设置LTS日志采集配额和使用量预警。支持按时间范围、日志组ID或日志流ID筛选,查看日志流的资源用量明细(如原始日志流量、索引流量、存储量等),帮助用户快速识别资源消耗情况,及时发现异常流量,优化资源配置,提升资源利用率。
- 用量Top排序仪表盘:需要开通“可观测For LTS”功能,详细操作请参考设置LTS日志采集配额和使用量预警。支持按时间范围、日志组ID或日志流ID筛选,查看TOP N日志组或日志流的资源用量(如原始日志流量、索引流量、存储量等),帮助用户快速识别高资源消耗情况,及时发现异常流量,优化日志采集策略与资源配置,降低无效成本。
OBS导入任务监控仪表盘
- 登录云日志服务控制台,进入“日志管理”页面。
- 在左侧导航栏中选择“仪表盘 ”。
- 在仪表盘模板下方,选择“LTS仪表盘模板”。
- 在仪表盘列表页面,单击“OBS导入任务监控仪表盘”仪表盘名称,查看图表详情。
- 过滤任务ID,所关联的查询分析语句如下所示:
* | select task_set_id group by task_set_id
- 过滤任务名称,所关联的查询分析语句如下所示:
* | select task_set_name group by task_set_name
- 成功条数图表所关联的查询分析语句如下所示:
* | select sum(max_read_line) as "成功条数" from (select max_by(read_lines,__time) as max_read_line group by task_set_id,period_times limit 10000)
- 失败条数图表所关联的查询分析语句如下所示:
* | select sum(max_failure_lines) as "失败条数" from (select max_by(failure_lines,__time) as max_failure_lines group by task_set_id,period_times limit 10000)
- 丢弃条数图表所关联的查询分析语句如下所示:
* | select sum(max_ignore_lines) as "丢弃条数" from (select max_by(ignore_lines,__time) as max_ignore_lines group by task_set_id,period_times limit 10000)
- 读流量图表所关联的查询分析语句如下所示:
* | select sum(max_in_flow)/1024.0/1024 as "读流量 MB" from (select max_by(in_flow,__time) as max_in_flow group by task_set_id,period_times limit 10000)
- 写流量图表所关联的查询分析语句如下所示:
* | select sum(max_out_flow)/1024.0/1024 as "写流量 MB" from (select max_by(out_flow,__time) as max_out_flow group by task_set_id,period_times limit 10000)
- 导入进度图表所关联的查询分析语句如下所示:
* | select max_by(case when total_files = 0 or total_files is null then 'NA' else cast(complete_files*1.0/total_files*100 as VARCHAR) end ,__time ) as progress
- 处理速率图表所关联的查询分析语句如下所示:
* | SELECT t AS "统计时间", ROUND(( CASE WHEN diff [ 1 ] IS NOT NULL AND diff [ 2 ] IS NOT NULL AND diff [ 1 ] - diff [ 2 ] > 0 THEN diff [ 1 ] - diff [ 2 ] WHEN diff [ 1 ] IS NOT NULL AND diff [ 2 ] IS NULL THEN diff [ 1 ] ELSE 0 END ) / 300,0) AS "成功条数/s" FROM ( SELECT t, ts_compare ( max_read_lines, 300 ) AS diff FROM ( SELECT t, sum( max_read_lines ) AS max_read_lines FROM ( SELECT from_unixtime( __time - __time % 300000 ) AS t, max_by ( read_lines, __time ) AS max_read_lines, task_set_id, period_times GROUP BY t, task_set_id, period_times LIMIT 10000 ) GROUP BY t ORDER BY t ) GROUP BY t ORDER BY t LIMIT 10000 ) LIMIT 10000
- 运行异常图表所关联的查询分析语句如下所示:
* | select task_set_name as "任务名称",task_set_id as "任务ID" , max_by(task_set_state,__time ) as "任务状态" , from_unixtime(max(__time)) as "采集时间" , max_by(failure_lines,__time) as "失败读取条数" group by task_set_id,task_set_name HAVING "任务状态" = 'UNHEALTHY' OR "任务状态" = 'FAILURE' limit 10000
- 运行状态图表所关联的查询分析语句如下所示:
* | select task_set_name as "任务名称",task_set_id as "任务ID" , max_by(task_set_state,__time ) as "任务状态" , max_by(read_lines,__time) as "成功条数" ,max_by(failure_lines,__time) as "失败条数" , max_by(in_flow,__time)/1024/1024 as "读流量/MB" , max_by(out_flow,__time)/1024/1024 as "写流量/MB",period_times as "周期" group by task_set_id,task_set_name,period_times order by max(__time),task_set_id,period_times limit 10000
- 过滤任务ID,所关联的查询分析语句如下所示:
资源用量明细仪表盘
- 在左侧导航栏中选择“仪表盘 ”。
- 在仪表盘模板下方,选择“LTS仪表盘模板”。
- 在仪表盘列表页面,单击“资源用量明细”仪表盘名称,查看图表详情。
若仪表盘中的部分图表展示异常,您可在索引配置页面单击“自动配置”更新索引字段,详细操作请参考创建LTS日志索引。索引更新完成后,返回仪表盘页面,查看图表显示正常。
- 过滤日志组ID,所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" | select log_group_id from log group by log_group_id limit 100000
- 过滤日志组名称,所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" | select log_group_name_alias from log group by log_group_name_alias limit 100000
- 过滤日志流ID,所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" | select log_stream_id from log group by log_stream_id limit 100000
- 过滤日志流名称,所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" | select log_stream_name_alias from log group by log_stream_name_alias limit 100000
- 原始日志流量/MB图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and metric_name:"storage_traffic" | select sum(counter)/1024/1024 as "原始日志流量/MB"
- 索引流量(标准型)/MB图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and metric_name:"index_traffic" | select sum(counter)/1024/1024 as "索引流量/MB"
- 索引流量(搜索型)/MB图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and metric_name:"search_index_traffic" | select sum(counter)/1024/1024 as "索引流量/MB"
- DSL加工流量/MB图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and metric_name:"dsl_traffic" && counter:* | select sum(counter)/1024/1024 as "日志DSL加工流量/MB"
- 写流量(压缩后)/MB图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and metric_name:"write_traffic" | select sum(counter)/1024/1024 as "写流量/MB"
- 标准存储量(最新值)/MB图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and metric_name:"storage" && counter:* | select select sum(max_storage) as "标准存储量(最新值)/MB" from (select max_by(counter,__time)/1024/1024 as max_storage group by log_stream_id limit 10000 )
- 冷存储量(最新值)/MB图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and metric_name:"cold_storage" && counter:* | select sum(max_cold_storage) as "冷存储量(最新值)/MB" from (select max_by(counter,__time)/1024/1024 as max_cold_storage group by log_stream_id limit 10000 )
- 基础转储流量/MB图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and (metric_name:"obs_transfer" or metric_name:"dms_transfer" or metric_name:"dis_transfer") | select sum(counter)/1024.0/1024 as "基础转储流量/MB"
- 高级转储流量/MB图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and (metric_name:"dws_transfer" or metric_name:"dli_transfer" or metric_name:"obs_orc_transfer") | select sum(counter)/1024/1024 as "高级转储流量/MB"
- 原始日志流量图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and metric_name:"storage_traffic" | select from_unixtime(__time - __time%60000) as "时间",sum(counter/1024/1024) as "原始日志流量/MB" group by "时间" order by "时间" limit 10000
- 索引流量(标准型)图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and metric_name:"index_traffic" | select from_unixtime(__time - __time%60000) as "时间",sum(counter)/1024/1024 as "索引流量/MB" group by "时间" order by "时间" limit 10000
- 索引流量(搜索型)图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and metric_name:"search_index_traffic" | select from_unixtime(__time - __time%60000) as "时间",sum(counter)/1024/1024 as "索引流量/MB" group by "时间" order by "时间" limit 10000
- DSL加工流量图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and metric_name:"dsl_traffic" | select from_unixtime(__time - __time%60000) as "时间",sum(counter)/1024/1024 as "日志DSL加工流量/MB" group by "时间" order by "时间" limit 10000
- 写流量(压缩后)图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and metric_name:"write_traffic" | select from_unixtime(__time - __time%60000) as "时间",sum(counter)/1024/1024 as "写流量/MB" group by "时间" order by "时间" limit 10000
- 标准存储量(最新值)图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and metric_name:"storage"| select from_unixtime(__time - __time%60000) as "时间",max_by(counter,__time)/1024/1024 as "标准存储量(最新值)/MB" group by "时间" order by "时间" limit 10000
- 冷存储量(最新值)图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and metric_name:"cold_storage" | select from_unixtime(__time - __time%60000) as "时间",max_by(counter,__time)/1024/1024 as "冷存储量(最新值)/MB" group by "时间" order by "时间" limit 10000
- 基础转储流量图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and (metric_name:"obs_transfer" or metric_name:"dms_transfer" or metric_name:"dis_transfer") | select from_unixtime(__time - __time%60000) as "时间",sum(counter)/1024/1024 as "基础转储流量/MB" group by "时间" order by "时间" limit 10000
- 高级转储流量图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and (metric_name:"dws_transfer" or metric_name:"dli_transfer" or metric_name:"obs_orc_transfer") | select from_unixtime(__time - __time%60000) as "时间",sum(counter)/1024/1024 as "高级转储流量/MB" group by "时间" order by "时间" limit 10000
- 过滤日志组ID,所关联的查询分析语句如下所示:
用量Top排序仪表盘
- 在左侧导航栏中选择“仪表盘 ”。
- 在仪表盘模板下方,选择“LTS仪表盘模板”。
- 在仪表盘列表页面,单击“用量Top排序”仪表盘名称,查看图表详情。
若仪表盘中的部分图表展示异常,您可在索引配置页面单击“自动配置”更新索引字段,详细操作请参考创建LTS日志索引。索引更新完成后,返回仪表盘页面,查看图表显示正常。
- 过滤日志组ID,所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" | select log_group_id from log group by log_group_id limit 100000
- 过滤日志组名称,所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" | select log_group_name_alias from log group by log_group_name_alias limit 100000
- 过滤日志流ID,所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" | select log_stream_id from log group by log_stream_id limit 100000
- 过滤日志流名称,所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" | select log_stream_name_alias from log group by log_stream_name_alias limit 100000
- 原始日志流量/MB图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and metric_name:"storage_traffic" | select sum(counter)/1024/1024 as "原始日志流量/MB"
- 索引流量(标准型)/MB图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and metric_name:"index_traffic" | select sum(counter)/1024/1024 as "索引流量/MB"
- 索引流量(搜索型)/MB图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and metric_name:"search_index_traffic" | select sum(counter)/1024/1024 as "索引流量/MB"
- DSL加工流量/MB图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and metric_name:"dsl_traffic" && counter:* | select sum(counter)/1024/1024 as "日志DSL加工流量/MB"
- 写流量(压缩后)/MB图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and metric_name:"write_traffic" | select sum(counter)/1024/1024 as "写流量/MB"
- 标准存储量(最新值)/MB图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and metric_name:"storage" && counter:* | select sum(max_storage) as "标准存储量(最新值)/MB" from (select max_by(counter,__time)/1024/1024 as max_storage group by log_stream_id limit 10000 )
- 冷存储量(最新值)/MB图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and metric_name:"cold_storage" && counter:* | select sum(max_cold_storage) as "冷存储量(最新值)/MB" from (select max_by(counter,__time)/1024/1024 as max_cold_storage group by log_stream_id limit 10000 )
- 基础转储流量/MB图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and (metric_name:"obs_transfer" or metric_name:"dms_transfer" or metric_name:"dis_transfer") | select sum(counter)/1024/1024 as "基础转储流量/MB"
- 高级转储流量/MB图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" and (metric_name:"dws_transfer" or metric_name:"dli_transfer" or metric_name:"obs_orc_transfer") | select sum(counter)/1024/1024 as "高级转储流量/MB"
- TOP日志组用量图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" | select log_group_id as "日志组ID", sum(case when metric_name = 'storage_traffic' then counter else 0 end)*1.0/1024/1024 as "原始日志流量/MB", sum(case when metric_name = 'index_traffic' then counter else 0 end)*1.0/1024/1024 as "索引流量(标准型)/MB", sum(case when metric_name = 'search_index_traffic' then counter else 0 end)*1.0/1024/1024 as "索引流量(搜索型)/MB", sum(case when metric_name = 'write_traffic' then counter else 0 end)*1.0/1024/1024 as "写流量(压缩后)/MB", sum(case when metric_name = 'storage' then counter else 0 end)*1.0/1024/1024 as "标准存储量(最新值)/MB", sum(case when metric_name = 'cold_storage' then counter else 0 end)*1.0/1024/1024 as "冷存储量(最新值)/MB" group by log_group_id order by "原始日志流量/MB" desc
- TOP日志流用量图表所关联的查询分析语句如下所示:
not log_group_id:"__default__*" and not log_group_name_alias:"__default__*" | select log_stream_id as "日志流ID", sum(case when metric_name = 'storage_traffic' then counter else 0 end)*1.0/1024/1024 as "原始日志流量/MB", sum(case when metric_name = 'index_traffic' then counter else 0 end)*1.0/1024/1024 as "索引流量(标准型)/MB", sum(case when metric_name = 'search_index_traffic' then counter else 0 end)*1.0/1024/1024 as "索引流量(搜索型)/MB", sum(case when metric_name = 'write_traffic' then counter else 0 end)*1.0/1024/1024 as "写流量(压缩后)/MB",sum(case when metric_name = 'storage' then counter else 0 end)*1.0/1024/1024 as "标准存储量(最新值)/MB",sum(case when metric_name = 'cold_storage' then counter else 0 end)*1.0/1024/1024 as "冷存储量(最新值)/MB" group by log_stream_id order by "原始日志流量/MB" desc
- 过滤日志组ID,所关联的查询分析语句如下所示: