Collecting Logs from RDS for PostgreSQL

LTS can collect logs from RDS for PostgreSQL. For details, see Log Reporting.

Structuring Template Details of PostgreSQL Slow Query Logs

PostgreSQL slow query log example

**Table 1** Structuring template example
Template Name	Example Log
PostgreSQL slow query logs	{"log_type":"slow_log","execute_time":328.662,"user":"authoring","log_time":"2022-07-24T10:06:41.000","database":"authoring-test","statement":"SELECT * FROM ( SELECT n.user_id,n.id AS resource_id,e.create_at AS begin_time,e.create_at AS end_time ,N AS resource_spec_code,COALESCE(cast(e.flavor as varchar), cast(s.volume_size as varchar)) AS billing_unit,c.az_id,-N AS accumulate_factor_value,CONCAT(N, s.id, N) AS bss_params,n.project_id, n.domain_id, e.status , N AS resource_type , w.workspace_id,w.enterprise_project_id FROM t_resource_status_event e INNER JOIN t_notebook_evs_storage s on s.id=e.resource_id LEFT JOIN t_notebook_instance n on s.id=n.storage_id LEFT JOIN t_logic_cluster l on n.resource_cluster_id=l.id LEFT JOIN t_cce_cluster c on c.id=l.cce_id LEFT JOIN t_workspace w on w.workspace_id=n.workspace_id WHERE e.category = N AND s.resource_ownership=N AND e.create_at BETWEEN $N AND $N UNION ALL SELECT n.user_id,n.id AS resource_id,$N AS begin_time,$N AS end_time ,N AS resource_spec_code,COALESCE(cast(e.flavor as varchar), cast(s.volume_size as varchar)) AS billing_unit,c.az_id,-N AS accumulate_factor_value,CONCAT(N, s.id, N) AS bss_params,n.project_id, n.domain_id, e.status , N AS resource_type , w.workspace_id,w.enterprise_project_id FROM t_resource_status_event e INNER JOIN t_notebook_evs_storage s on s.id=e.resource_id LEFT JOIN t_notebook_instance n on s.id=n.storage_id LEFT JOIN t_logic_cluster l on n.resource_cluster_id=l.id LEFT JOIN t_cce_cluster c on c.id=l.cce_id LEFT JOIN t_workspace w on w.workspace_id=n.workspace_id INNER JOIN (SELECT resource_id,max(create_at) as create_at FROM t_resource_status_event WHERE create_at < $N AND category = N GROUP BY resource_id) x ON e.resource_id=x.resource_id AND e.create_at=x.create_at WHERE e.create_at < $N AND e.category = N AND e.status = N AND s.resource_ownership=N) m ORDER BY resource_id,begin_time ASC","host":"10...206","log_timestamp":"1658657201","operate_type":"SELECT","node_id":"d285609201534696bdcd648519fe2b8dno02","instance_id":"5b67dc63ba824145aae1f12ff51e58b8in02"}

Table 1 Structuring template example

Template Name

Example Log

PostgreSQL slow query logs

{"log_type":"slow_log","execute_time":328.662,"user":"authoring","log_time":"2022-07-24T10:06:41.000","database":"authoring-test","statement":"SELECT * FROM ( SELECT n.user_id,n.id AS resource_id,e.create_at AS begin_time,e.create_at AS end_time ,N AS resource_spec_code,COALESCE(cast(e.flavor as varchar), cast(s.volume_size as varchar)) AS billing_unit,c.az_id,-N AS accumulate_factor_value,CONCAT(N, s.id, N) AS bss_params,n.project_id, n.domain_id, e.status , N AS resource_type , w.workspace_id,w.enterprise_project_id  FROM t_resource_status_event e INNER JOIN t_notebook_evs_storage s on s.id=e.resource_id LEFT JOIN t_notebook_instance n on s.id=n.storage_id LEFT JOIN t_logic_cluster l on n.resource_cluster_id=l.id LEFT JOIN t_cce_cluster c on c.id=l.cce_id LEFT JOIN t_workspace w on w.workspace_id=n.workspace_id WHERE e.category = N AND s.resource_ownership=N AND e.create_at BETWEEN $N AND $N UNION ALL SELECT n.user_id,n.id AS resource_id,$N AS begin_time,$N AS end_time ,N AS resource_spec_code,COALESCE(cast(e.flavor as varchar), cast(s.volume_size as varchar)) AS billing_unit,c.az_id,-N AS accumulate_factor_value,CONCAT(N, s.id, N) AS bss_params,n.project_id, n.domain_id, e.status , N AS resource_type , w.workspace_id,w.enterprise_project_id  FROM t_resource_status_event e INNER JOIN t_notebook_evs_storage s on s.id=e.resource_id LEFT JOIN t_notebook_instance n on s.id=n.storage_id LEFT JOIN t_logic_cluster l on n.resource_cluster_id=l.id LEFT JOIN t_cce_cluster c on c.id=l.cce_id LEFT JOIN t_workspace w on w.workspace_id=n.workspace_id INNER JOIN (SELECT resource_id,max(create_at) as create_at FROM t_resource_status_event  WHERE create_at < $N AND category = N GROUP BY resource_id) x ON e.resource_id=x.resource_id AND e.create_at=x.create_at  WHERE e.create_at < $N AND e.category = N AND e.status = N AND s.resource_ownership=N) m ORDER BY resource_id,begin_time ASC","host":"10.*.*.206","log_timestamp":"1658657201","operate_type":"SELECT","node_id":"d285609201534696bdcd648519fe2b8dno02","instance_id":"5b67dc63ba824145aae1f12ff51e58b8in02"}

Structuring fields and description

**Table 2** Structuring fields
Field	Example	Description	Type
log_type	slow_log	Log type.	string
execute_time	328.662	Time when the SQL statement was executed	float
user	authoring	User	string
log_time	2022-07-24T10:06:41.000	Log printing time	string
database	authoring-test	Database name	string
statement	SELECT * FROM ( SELECT n.user_id,n.id AS resource_id,e.create_at AS begin_time,e.create_at AS end_time ,N AS resource_spec_code,COALESCE(cast(e.flavor as varchar), cast(s.volume_size as varchar)) AS billing_unit,c.az_id,-N AS accumulate_factor_value,CONCAT(N, s.id, N) AS bss_params,n.project_id, n.domain_id, e.status , N AS resource_type , w.workspace_id,w.enterprise_project_id FROM t_resource_status_event e INNER JOIN t_notebook_evs_storage s on s.id=e.resource_id LEFT JOIN t_notebook_instance n on s.id=n.storage_id LEFT JOIN t_logic_cluster l on n.resource_cluster_id=l.id LEFT JOIN t_cce_cluster c on c.id=l.cce_id LEFT JOIN t_workspace w on w.workspace_id=n.workspace_id WHERE e.category = N AND s.resource_ownership=N AND e.create_at BETWEEN $N AND $N UNION ALL SELECT n.user_id,n.id AS resource_id,$N AS begin_time,$N AS end_time ,N AS resource_spec_code,COALESCE(cast(e.flavor as varchar), cast(s.volume_size as varchar)) AS billing_unit,c.az_id,-N AS accumulate_factor_value,CONCAT(N, s.id, N) AS bss_params,n.project_id, n.domain_id, e.status , N AS resource_type , w.workspace_id,w.enterprise_project_id FROM t_resource_status_event e INNER JOIN t_notebook_evs_storage s on s.id=e.resource_id LEFT JOIN t_notebook_instance n on s.id=n.storage_id LEFT JOIN t_logic_cluster l on n.resource_cluster_id=l.id LEFT JOIN t_cce_cluster c on c.id=l.cce_id LEFT JOIN t_workspace w on w.workspace_id=n.workspace_id INNER JOIN (SELECT resource_id,max(create_at) as create_at FROM t_resource_status_event WHERE create_at < $N AND category = N GROUP BY resource_id) x ON e.resource_id=x.resource_id AND e.create_at=x.create_at WHERE e.create_at < $N AND e.category = N AND e.status = N AND s.resource_ownership=N) m ORDER BY resource_id,begin_time ASC	SQL information	string
host	10...206	ip	string
log_timestamp	1658657201	Log printing timestamp	string
operate_type	SELECT	Operation type	string
node_id	d285609201534696bdcd648519fe2b8dno02	Node ID	string
instance_id	5b67dc63ba824145aae1f12ff51e58b8in02	Instance ID	string

Structuring Template Details of PostgreSQL Error Logs

PostgreSQL error log example

**Table 3** Structuring template example
Template Name	Example Log
PostgreSQL error logs	{"log_type":"error_log","severity":"WARNING","log_time":"2022-08-22T06:52:08Z","raw_message":"Occur error when reading bytes from a network handler. Client actively closes the connection.","node_id":"d285609201534696bdcd648519fe2b8dno02","instance_id":"5b67dc63ba824145aae1f12ff51e58b8in02"}

Table 3 Structuring template example

Template Name

Example Log

PostgreSQL error logs

{"log_type":"error_log","severity":"WARNING","log_time":"2022-08-22T06:52:08Z","raw_message":"Occur error when reading bytes from a network handler. Client actively closes the connection.","node_id":"d285609201534696bdcd648519fe2b8dno02","instance_id":"5b67dc63ba824145aae1f12ff51e58b8in02"}

Structuring fields and description

**Table 4** Structuring fields
Field	Example	Description	Type
log_type	error_log	Log type	string
severity	WARNING	Log level	string
log_time	2022-08-22T06:52:08Z	Time when a log is generated	string
raw_message	Occur error when reading bytes from a network handler. Client actively closes the connection.	Error log content	string
node_id	d285609201534696bdcd648519fe2b8dno02	Node ID	string
instance_id	5b67dc63ba824145aae1f12ff51e58b8in02	Instance ID	string