Help Center/ MapReduce Service/ User Guide/ Managing Clusters/ Managing SQL Inspection Rules for an MRS Cluster/ SQL Inspector Overview
Updated on 2025-03-17 GMT+08:00
View PDF

SQL Inspector Overview

New SQL engines are constantly emerging for big data. While contributing to increased diversity of big data solutions, they also lead to many issues, for example, the varying quality of SQL statements, increased difficulty in troubleshooting SQL-related performance issues, and excessive resource consumption by large queries.

Low-quality SQL statements may have unpredictable impacts on the performance and stability of data analytics platforms.

This function is supported only by MRS 3.3.0 or later.

Introduction to SQL Inspector

MRS offers a SQL Inspector that allows you to detect, warn about, intercept, or terminate large SQL queries for mainstream SQL engines, such as Hive, Spark, HetuEngine, and ClickHouse. The implementation of SQL Inspector does not require any service changes. Users do not need to change the way they submit SQL statements or the SQL syntax they are familiar with.

  • You can configure SQL inspection rules on the UI that also allows you to query and modify the rules.
  • During query execution and response, each SQL engine proactively inspects SQL statements based on the rules.
  • You can choose one of the following actions for problematic SQL statements: Hint, Intercept, or Block. Hint means to log and warn the user about large SQL queries; Intercept means to prevent them from being executed; and Block means to terminate them while they are being executed. The system logs SQL inspection events in real time for auditing purposes. O&M engineers can analyze the logs, evaluate SQL statement quality on the live network, identify problematic statements, and take appropriate measures.

SQL inspection rules are classified into the following types:

  • Static interception: displays hints or intercepts SQL statements based on SQL syntax.
  • Dynamic interception: displays hints or intercepts SQL statements based on the rules regarding data table statistics and metadata information.
  • Runtime blocking: terminates SQL statements that are being executed based on real-time system status information, such as CPU, memory, and I/O.

SQL statements that trigger static or dynamic interception rules are intercepted. Alternatively, the system gives hints for you to handle the statements properly. SQL statements that trigger runtime blocking rules are terminated.

SQL Inspection Rules and Constraints

  • A SQL inspection rule can be associated with multiple SQL engines, and have different threshold parameters configured for each service.
  • A SQL inspection rule can be associated with multiple tenants, and takes effect only for associated tenants.