Help Center/ MapReduce Service/ Component Operation Guide (LTS) (Ankara Region)/ Using HetuEngine/ HetuEngine FAQ/ How Do I Configure HetuEngine SQL Inspection?
Updated on 2024-11-29 GMT+08:00
View PDF
Share

How Do I Configure HetuEngine SQL Inspection?

There are numerous SQL engines in the current big data field, which bring diversity to the solutions but also expose some issues such as varying quality of SQL input statements, difficult SQL problem localization, and excessive resource consumption by large SQL statements.

Poor quality SQL can have unforeseeable impacts on data analysis platforms, affecting system performance or platform stability.

Function Description

MRS allows you to configure inspection rules for mainstream SQL engines (Hive, Spark, HetuEngine, and ClickHouse). MRS can identify typical large SQL queries and low-quality SQL statements and intercepts them before execution or block them during execution. Users do not need to change how they submit SQL statements or change SQL syntax. Service modifications are not required and inspection is easy to implement.

  • You can configure SQL inspection rules on the UI that also allows you to query and modify the rules.
  • During query response and execution, each SQL engine proactively inspects SQL statements based on the rules.
  • Administrators can select to display hints on, intercept, or block SQL statements. The system logs SQL inspection events in real time for SQL audit. O&M engineers can analyze the logs, evaluate SQL statement quality on the live network, detect target statements, and take effective measures.

SQL inspection rules are classified into the following types:

  • Static interception: The system displays hints on or intercepts SQL statements based on SQL syntax rules.
  • Dynamic interception: The system displays hints on or intercepts SQL statements based on rules of data table statistics and metadata information.
  • Runtime Blocking: The system blocks SQL statements based on system states (such as CPU, memory, and I/O) during the runtime of the SQL statements.

SQL requests that meet the static and dynamic interception rules can be intercepted, and the system gives hints for processing the statements properly. If a SQL request meets the blocking rule, the system blocks the SQL task.

Parent topic: HetuEngine FAQ