System Rules
Table 1 lists the system inspection rules supported by DLI.
Default system rules are SQL inspection rules automatically created by DLI when a queue is created. These rules are bound to the queue and cannot be deleted.
Default system rules include Scan files number, Scan partitions number, Count(distinct) occurrences, and Not in<Subquery>.
Only one inspection rule can be created for an action or a queue.
For every supported action, a default system rule is created. For example, when a queue is created, a Scan files number rule is automatically created for both the Info and Block actions.
Rule ID |
Rule Name |
Description |
Type |
Applicable Engine |
Action |
Value |
Default System Rule |
Example SQL Statement |
---|---|---|---|---|---|---|---|---|
dynamic_0001 |
Scan files number |
Maximum number of files to be scanned |
Dynamic |
Spark trino |
Info Block |
Value range: 1–2000000 Default value: 200000 |
Yes |
N/A |
dynamic_0002 |
Scan partitions number |
Maximum number of partitions involved in the operations (select, delete, update, and alter) that can be performed on a table |
Dynamic |
Spark |
Info Block |
Value range: 1–500000 Default value: 5000 |
Yes |
DELETE FROM table_name WHERE column_name = value |
running_0002 |
Memory used(MB) |
Peak memory usage of the SQL statement |
Running |
Spark |
Circuit Breaker |
Value range: 1–8388608 |
No |
N/A |
running_0003 |
Run time(S) |
Maximum running duration of the SQL statement |
Running |
Spark |
Circuit Breaker |
Unit: second Value range: 1–43200 |
No |
N/A |
running_0004 |
Scan data(GB) |
Maximum amount of data to be scanned |
Running |
Spark |
Circuit Breaker |
Unit: GB Value range: 1–10240 |
No |
N/A |
static_0001 |
Count(distinct) occurrences |
Maximum number of occurrences of count(distinct) in the SQL statement |
Static |
Spark |
Info Block |
Value range: 1–100 Default value: 10 |
Yes |
SELECT COUNT(DISTINCT deviceId), COUNT(DISTINCT collDeviceId) FROM table GROUP BY deviceName, collDeviceName, collCurrentVersion; |
static_0002 |
Not in<Subquery> |
Check whether not in <subquery> is used in the SQL statement. |
Static |
Spark |
Info Block |
Value range: Yes, No Default value: Yes |
Yes |
SELECT * FROM Orders o WHERE Orders.Order_ID not in (Select Order_ID FROM HeldOrders h where h.order_id = o.order_id); |
static_0003 |
Join occurrences |
Maximum number of joins in the SQL statement |
Static |
Spark |
Info Block |
Value range: 1–50 |
No |
SELECT name, text FROM table_1 JOIN table_2 ON table_1.Id = table_2.Id |
static_0004 |
Union occurrences |
Maximum number of union all times in the SQL statement |
Static |
Spark |
Info Block |
Value range: 1–100 |
No |
select * from tables t1 union all select * from tables t2 union all select * from tables t3 |
static_0005 |
Subquery nesting layers |
Maximum number of subquery nesting layers |
Static |
Spark |
Info Block |
Value range: 1–20 |
No |
select * from ( with temp1 as (select * from tables) select * from temp1); |
static_0006 |
Sql size(KB) |
Maximum size of a SQL file |
Static |
Spark |
Info Block |
Unit: KB Value range: 1–1024 |
No |
N/A |
static_0007 |
Cartesian product |
Limitation of Cartesian products when multiple tables are being associated |
Static |
Spark |
Info Block |
Value range: 0–1 |
No |
select * from A,B; |
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot