Help Center > > Developer Guide> Query Performance Optimization> Tuning Queries> Typical SQL Optimization Methods> Optimizing SQL Self-Diagnosis

Optimizing SQL Self-Diagnosis

Updated at:Jul 15, 2020 GMT+08:00

Performance issues may occur when you query data or run the INSERT, DELETE, UPDATE, or CREATE TABLE AS statement. In this case, you can query the warning column in the gs_wlm_session_statistics, gs_wlm_session_history, and gs_wlm_session_info views to obtain reference for performance optimization.

Alarms that can trigger SQL self diagnosis depend on the settings of resource_track_level. If resource_track_level is set to query, alarms about the failures in collecting column statistics and pushing down SQL statements will trigger the diagnosis. If resource_track_level is set to operator, all alarms will trigger the diagnosis.

Whether a SQL plan will be diagnosed depends on the settings of resource_track_cost. A SQL plan will be diagnosed only if its execution cost is greater than resource_track_cost. You can use the EXPLAIN keyword to check the plan execution cost.

Alarms

Currently, the following performance alarms will be reported:

  • Some column statistics are not collected.

An alarm will be reported if some column statistics are not collected. For details about the optimization, see Updating Statistics and Optimizing Statistics.

An alarm will also be reported if column statistics are not collected for queries on OBS or HDFS foreign tables. The performance of the ANALYZE statement executed for these foreign tables is poor and seems inefficient for a simple query. Run this statement as needed.

Example alarms:

No statistics about a table are not collected.

Statistic Not Collect:
    schema_test.t1

The statistics about a single column are not collected.

Statistic Not Collect:
    schema_test.t2(c1,c2)

The statistics about multiple columns are not collected.

Statistic Not Collect:
    schema_test.t3((c1,c2))

The statistics about a single column and multiple columns are not collected.

Statistic Not Collect:
    schema_test.t4(c1,c2)    schema_test.t4((c1,c2))
  • SQL statements are not pushed down.
    The cause details are displayed in the alarms. For details about the optimization, see Optimizing Statement Pushdown.
    • If the pushdown failure is caused by functions, the function names are displayed in the alarm.
    • If the pushdown failure is caused by syntax, the alarm indicates that the syntax does not support pushdown. For example, the syntax containing the With Recursive, Distinct On, or Row expression, and syntax whose return value is of record type do not support pushdown.

Example alarms:

SQL is not plan-shipping, reason : ""enable_stream_operator" is off"
SQL is not plan-shipping, reason : ""Distinct On" can not be shipped"
SQL is not plan-shipping, reason : ""v_test_unshipping_log" is VIEW that will be treated as Record type can't be shipped"
  • In a hash join, the larger table is used as the inner table.

An alarm will be reported if the number of rows in the inner table reaches or exceeds 10 times of that in the outer table, more than 100 thousand of inner-table rows are processed on each DN in average, and the join statement has spilled to disk. You can view the query_plan column in gs_wlm_session_history to check whether the hash join is used. For details about the optimization, see Hint-based Tuning.

Example alarm:

PlanNode[7] Large Table is INNER in HashJoin "Vector Hash Aggregate"
  • nestloop is used in a large-table equivalent join.

An alarm will be reported if nestloop is used in an equivalent join where more than 100 thousand of the larger-table rows are processed on each DN in average. You can view the query_plan field of gs_wlm_session_history to check whether nestloop is used. For details about the optimization, see Hint-based Tuning.

Example alarm:

PlanNode[5] Large Table with Equal-Condition use Nestloop"Nested Loop"
  • A large table is broadcasted.

An alarm will be reported if more than 100 thousand of rows are broadcasted on each DN in average. For details about the optimization, see Hint-based Tuning.

Example alarm:

PlanNode[5] Large Table in Broadcast "Streaming(type: BROADCAST dop: 1/2)"
  • Data skew occurs.

An alarm will be reported if the number of rows processed on any DN exceeds 100 thousand, and the number of rows processed on a DN reaches or exceeds 10 times of that processed on another DN. For the optimization, see Case: Selecting an Appropriate Distribution Column and Optimizing Data Skew.

Example alarm:

PlanNode[6] DataSkew:"Seq Scan", min_dn_tuples:0, max_dn_tuples:524288
  • The index is improper.

    During base table scanning, an alarm is reported if the following conditions are met:

    • For row-store tables:
      • When the index scanning is used, the ratio of the number of output lines to the number of scanned lines is greater than 1/1000 and the number of output lines is greater than 10,000.
      • When sequential scanning is used, the number of output lines to the number of scanned lines is less than 1/1000, the number of output lines is less than or equal to 10,000, and the number of scanned lines is greater than 10,000.
    • For column-store tables:
      • When the index scanning is used, the ratio of the number of output lines to the number of scanned lines is greater than 1/10000 and the number of output lines is greater than 100.
      • When sequential scanning is used, the number of output lines to the number of scanned lines is less than 1/10,000, the number of output lines is less than or equal to 100, and the number of scanned lines is greater than 10,000.

For details about the optimization, see Optimizing Operators.

Example alarms:

PlanNode[4] Indexscan is not properly used:"Index Only Scan", output:524288, filtered:0, rate:1.00000
PlanNode[5] Indexscan is ought to be used:"Seq Scan", output:1, filtered:524288, rate:0.00000
  • Estimation is inaccurate.

An alarm will be reported if the maximum number or the estimated maximum number of rows processed on a DN is over 10 thousand, and the larger number reaches or exceeds 10 times of the smaller one. For details about the optimization, see Hint-based Tuning.

Example alarm:

PlanNode[5] Inaccurate Estimation-Rows: "Hash Join" A-Rows:0, E-Rows:52488

Restrictions

  1. An alarm contains a maximum of 2048 characters. If the length of an alarm exceeds this value (for example, a large number of long table names and column names are displayed in the alarm when their statistics are not collected), a warning instead of an alarm will be reported.
    WARNING, "Planner issue report is truncated, the rest of planner issues will be skipped"
  2. If a query statement contains the Limit operator, alarms of operators lower than Limit will not be reported.
  3. For alarms about data skew and inaccurate estimation, only alarms on the lower-layer nodes in a plan tree will be reported. This is because the same alarms on the upper-level nodes may be triggered by problems on the lower-layer nodes. For example, if data skew occurs on the Scan node, data skew may also occur in operators (for example, Hashagg) at the upper layer.

Did you find this page helpful?

Submit successfully!

Thank you for your feedback. Your feedback helps make our documentation better.

Failed to submit the feedback. Please try again later.

Which of the following issues have you encountered?







Please complete at least one feedback item.

Content most length 200 character

Content is empty.

OK Cancel