Data Lake Insight
Data Lake Insight
All results for "
" in this service
All results for "
" in this service
What's New
Function Overview
Product Bulletin
Product Bulletin
EOL Announcement for DLI Yearly/Monthly and Pay-per-Use Queues as Well as Queue CUH Packages
EOS Announcement for DLI Flink 1.10 and Flink 1.11
EOS Announcement for DLI Spark 2.3.2
EOS Announcement for DLI Flink 1.7
Version Support Bulletin
Lifecycle of DLI Compute Engine Versions
What's New in Flink 1.15
What's New in Flink 1.12
What's New in Spark 3.3.1
What's New in Spark 3.1.1
What's New in Spark 2.4.5
Service Overview
Infographics
What Is Data Lake Insight
Advantages
Application Scenarios
Notes and Constraints
Permissions Management
Quotas
Related Services
Basic Concepts
Billing
Billing Overview
Billing for Compute Resources
Queues
Elastic Resource Pools
Billing for Storage Resources
Billing for Scanned Data
Yearly/Monthly Billing
Billing Examples
Renewing Subscriptions
Bills
Arrears
Billing Termination
Billing FAQ
What Billing Modes Does DLI Offer?
Would a Pay-Per-Use DLI Queue Not Be Billed if No Job Is Submitted for Execution?
When Is a Data Lake Queue Idle?
How Do I Troubleshoot DLI Billing Issues?
Why Am I Still Being Billed on a Pay-per-Use Basis After I Purchased a Yearly/Monthly Package?
How Do I View the Usage of a Package?
How Do I Purchase a 30-CU Queue?
How Will I Be Billed If I Purchased a Pay-per-Use 16-CU DLI Queue but Only Used 2 CUs to Run Jobs?
How Do I View a Job's Scanned Data Volume?
Would a Pay-Per-Use Elastic Resource Pool Not Be Billed if No Job Is Submitted for Execution?
Do I Need to Pay Extra Fees for Purchasing a Queue Billed Based on the Scanned Data Volume?
How Is the Usage Beyond the Package Limit Billed?
What Are the Actual CUs, CU Range, and Specifications of an Elastic Resource Pool?
Change History
Getting Started
Using DLI to Submit a SQL Job to Query OBS Data
Using DLI to Submit a SQL Job to Query RDS for MySQL Data
Using DLI to Submit a Flink OpenSource SQL Job to Query RDS for MySQL Data
Using DLI to Submit a Flink Jar Job
Using DLI to Submit a Spark Jar Job
Practices
User Guide
DLI Console Overview
Creating an Elastic Resource Pool and Queues Within It
Overview of DLI Elastic Resource Pools and Queues
Creating an Elastic Resource Pool
Creating Queues in an Elastic Resource Pool
Creating a Standard Queue (Discarded and Not Recommended)
Managing Elastic Resource Pools
Checking Basic Information
Managing Permissions
Binding a Queue
Setting CUs
Modifying Specifications
Managing Tags
Viewing Scaling History
Allocating to an Enterprise Project
Managing Queues
Queue Permission Management
Adjusting Scaling Policies for Queues in an Elastic Resource Pool
Allocating a Queue to an Enterprise Project
Creating an SMN Topic
Managing Queue Tags
Setting Queue Properties
Testing Address Connectivity
Modifying Queue Specifications
Deleting a Queue
Elastic Scaling of Standard Queues (Discarded and Not Recommended)
Setting a Scheduled Elastic Scaling Task for a Standard Queue (Discarded and Not Recommended)
Changing the CIDR Block of a Standard Queue (Discarded and Not Recommended)
Example Use Case: Creating an Elastic Resource Pool and Running Jobs
Example Use Case: Configuring Scaling Policies for Queues in an Elastic Resource Pool
Job Management
Overview
Creating and Managing SQL Jobs
Submitting a SQL Job Using the SQL Editor
Managing SQL Jobs
Exporting SQL Job Results
Creating and Managing Flink Jobs
Flink Job Management Overview
Managing Flink Job Permissions
Preparing Flink Job Data
Creating a Flink OpenSource SQL Job
Creating a Flink SQL Job
Creating a Flink Jar Job
Performing Operations on a Flink Job
Flink Job Details
Tag Management
Enabling Dynamic Scaling for Flink Jobs
Creating and Managing Spark Jobs
Spark Job Management
Creating a Spark Job
Setting the Priority for a Job
Data Management
Databases and Tables
Overview
Managing Database Permissions
Managing Table Permissions
Creating a Database or a Table
Deleting a Database or a Table
Modifying the Owners of Databases and Tables
Importing Data to the Table
Exporting Data from DLI to OBS
Viewing Metadata
Previewing Data
Managing Tags
Package Management
Overview
Managing Permissions on Packages and Package Groups
Creating a Package
Deleting a Package
Modifying the Owner
Built-in Dependencies
Enhanced Datasource Connections
Overview
Cross-Source Analysis Development Methods
Creating an Enhanced Datasource Connection
Deleting an Enhanced Datasource Connection
Modifying Host Information in an Elastic Resource Pool
Binding and Unbinding a Queue
Adding a Route
Deleting a Route
Enhanced Connection Permission Management
Enhanced Datasource Connection Tag Management
Datasource Authentication
Overview
Creating a CSS Datasource Authentication
Creating a Kerberos Datasource Authentication
Creating a Kafka_SSL Datasource Authentication
Creating a Password Datasource Authentication
Datasource Authentication Permission Management
Job Templates
Managing SQL Templates
Managing Flink Templates
Managing Spark SQL Templates
Appendix
TPC-H Sample Data in the SQL Template
DLI Agency Permissions
DLI Agency Overview
Updating the Permissions of a DLI Agency
Customizing DLI Agency Permissions
Agency Permission Policies in Common Scenarios
Guidelines for Configuring DLI Agency Permissions in Specific Scenarios
Creating a DLI Global Variable
Configuring a DLI Job Bucket
SQL Inspector
Overview
Creating a SQL Inspection Rule
System Rules
Creating a Custom Image
Permissions Management
Overview
Creating an IAM User and Granting Permissions
Creating a Custom Policy
DLI Resources
DLI Request Conditions
Common Operations Supported by DLI System Policy
Other Common Operations
Importing Data to a DLI Table
Viewing Monitoring Metrics
DLI Operations That Can Be Recorded by CTS
Quota Management
Permission Management for Global Variables
Best Practices
Overview
Data Migration
Overview
Migrating Data from Hive to DLI
Migrating Data from MRS Kafka to DLI
Migrating Data from Elasticsearch to DLI
Migrating Data from RDS to DLI
Migrating Data from GaussDB(DWS) to DLI
Data Analysis
Analyzing Driving Behavior Data
Converting Data Format from CSV to Parquet
Analyzing E-commerce BI Reports
Analyzing DLI Billing Data
Interconnecting Yonghong BI with DLI to Submit Spark Jobs
Preparing for Yonghong BI Interconnection
Adding Yonghong BI Data Source
Creating Yonghong BI Data Set
Creating a Chart in Yonghong BI
Connections
Configuring the Connection Between a DLI Queue and a Data Source in a Private Network
Configuring the Connection Between a DLI Queue and a Data Source in the Internet
Change History
Developer Guide
SQL Jobs
Using Spark SQL Jobs to Analyze OBS Data
Calling UDFs in Spark SQL Jobs
Calling UDTFs in Spark SQL Jobs
Calling UDAFs in Spark SQL Jobs
Submitting a Spark SQL Job Using JDBC
Obtaining the Server Connection Address
Downloading the JDBC Driver Package
Performing Authentication
Submitting a Job Using JDBC
JDBC API Reference
Flink OpenSource SQL Jobs
Reading Data from Kafka and Writing Data to RDS
Reading Data from Kafka and Writing Data to GaussDB(DWS)
Reading Data from Kafka and Writing Data to Elasticsearch
Reading Data from MySQL CDC and Writing Data to GaussDB(DWS)
Reading Data from PostgreSQL CDC and Writing Data to GaussDB(DWS)
Configuring High-Reliability Flink Jobs (Automatic Restart upon Exceptions)
Flink Jar Jobs
Stream Ecosystem
Flink Jar Job Examples
Writing Data to OBS Using Flink Jar
Using Flink Jar to Connect to Kafka that Uses SASL_SSL Authentication
Using Flink Jar to Read and Write Data from and to DIS
Spark Jar Jobs
Using Spark Jar Jobs to Read and Query OBS Data
Using the Spark Job to Access DLI Metadata
Using Spark-submit to Submit a Spark Jar Job
Submitting a Spark Jar Job Using Livy
Using Spark Jobs to Access Data Sources of Datasource Connections
Overview
Connecting to CSS
CSS Security Cluster Configuration
Scala Example Code
PySpark Example Code
Java Example Code
Connecting to GaussDB(DWS)
Scala Example Code
PySpark Example Code
Java Example Code
Connecting to HBase
MRS Configuration
Scala Example Code
PySpark Example Code
Java Example Code
Troubleshooting
Connecting to OpenTSDB
Scala Example Code
PySpark Example Code
Java Example Code
Troubleshooting
Connecting to RDS
Scala Example Code
PySpark Example Code
Java Example Code
Connecting to Redis
Scala Example Code
PySpark Example Code
Java Example Code
Troubleshooting
Connecting to Mongo
Scala Example Code
PySpark Example Code
Java Example Code
Change History
Spark SQL Syntax Reference
Common Configuration Items
Spark SQL Syntax
Spark Open Source Commands
Databases
Creating a Database
Deleting a Database
Viewing a Specified Database
Viewing All Databases
Tables
Creating an OBS Table
Creating an OBS Table Using the DataSource Syntax
Creating an OBS Table Using the Hive Syntax
Creating a DLI Table
Creating a DLI Table Using the DataSource Syntax
Creating a DLI Table Using the Hive Syntax
Deleting a Table
Viewing a Table
Viewing All Tables
Viewing Table Creation Statements
Viewing Table Properties
Viewing All Columns in a Specified Table
Viewing All Partitions in a Specified Table
Viewing Table Statistics
Modifying a Table
Adding a Column
Modifying Column Comments
Enabling or Disabling Multiversion Backup
Partition-related Syntax
Adding Partition Data (Only OBS Tables Supported)
Renaming a Partition (Only OBS Tables Supported)
Deleting a Partition
Deleting Partitions by Specifying Filter Criteria (Only Supported on OBS Tables)
Altering the Partition Location of a Table (Only OBS Tables Supported)
Updating Partitioned Table Data (Only OBS Tables Supported)
Updating Table Metadata with REFRESH TABLE
Backing Up and Restoring Data of Multiple Versions
Setting the Retention Period for Multiversion Backup Data
Checking Multiversion Backup Data
Restoring Multiversion Backup Data
Configuring the Trash Bin for Expired Multiversion Data
Deleting Multiversion Backup Data
Table Lifecycle Management
Specifying the Lifecycle of a Table When Creating the Table
Modifying the Lifecycle of a Table
Disabling or Restoring the Lifecycle of a Table
Data
Importing Data
Inserting Data
Clearing Data
Exporting Query Results
Datasource Connections
Creating a Datasource Connection with an HBase Table
Creating a DLI Table and Associating It with HBase
Inserting Data to an HBase Table
Querying an HBase Table
Creating a Datasource Connection with an OpenTSDB Table
Creating a DLI Table and Associating It with OpenTSDB
Inserting Data to the OpenTSDB Table
Querying an OpenTSDB Table
Creating a Datasource Connection with a DWS Table
Creating a DLI Table and Associating It with DWS
Inserting Data to the DWS Table
Querying the DWS Table
Creating a Datasource Connection with an RDS Table
Creating a DLI Table and Associating It with RDS
Inserting Data to the RDS Table
Querying the RDS Table
Creating a Datasource Connection with a CSS Table
Creating a DLI Table and Associating It with CSS
Inserting Data to the CSS Table
Querying the CSS Table
Creating a Datasource Connection with a DCS Table
Creating a DLI Table and Associating It with DCS
Inserting Data to a DCS Table
Querying the DCS Table
Creating a Datasource Connection with a DDS Table
Creating a DLI Table and Associating It with DDS
Inserting Data to the DDS Table
Querying the DDS Table
Creating a Datasource Connection with an Oracle Table
Creating a DLI Table and Associating It with Oracle
Inserting Data to an Oracle Table
Querying an Oracle Table
Views
Creating a View
Deleting a View
Viewing the Execution Plan
Data Permissions
Data Permissions List
Creating a Role
Deleting a Role
Binding a Role
Unbinding a Role
Displaying a Role
Granting a Permission
Revoking a Permission
Displaying the Granted Permissions
Displaying the Binding Relationship Between All Roles and Users
Data Types
Overview
Primitive Data Types
Complex Data Types
User-Defined Functions
Creating a Function
Deleting a Function
Displaying Function Details
Displaying All Functions
Built-In Functions
Date Functions
Overview
add_months
current_date
current_timestamp
date_add
dateadd
date_sub
date_format
datediff
datediff1
datepart
datetrunc
day/dayofmonth
from_unixtime
from_utc_timestamp
getdate
hour
isdate
last_day
lastday
minute
month
months_between
next_day
quarter
second
to_char
to_date
to_date1
to_utc_timestamp
trunc
unix_timestamp
weekday
weekofyear
year
String Functions
Overview
ascii
concat
concat_ws
char_matchcount
encode
find_in_set
get_json_object
instr
instr1
initcap
keyvalue
length
lengthb
levenshtein
locate
lower/lcase
lpad
ltrim
parse_url
printf
regexp_count
regexp_extract
replace
regexp_replace
regexp_replace1
regexp_instr
regexp_substr
repeat
reverse
rpad
rtrim
soundex
space
substr/substring
substring_index
split_part
translate
trim
upper/ucase
Mathematical Functions
Overview
abs
acos
asin
atan
bin
bround
cbrt
ceil
conv
cos
cot1
degrees
e
exp
factorial
floor
greatest
hex
least
ln
log
log10
log2
median
negative
percentlie
percentlie_approx
pi
pmod
positive
pow
radians
rand
round
shiftleft
shiftright
shiftrightunsigned
sign
sin
sqrt
tan
Aggregate Functions
Overview
avg
corr
count
covar_pop
covar_samp
max
min
percentile
percentile_approx
stddev_pop
stddev_samp
sum
variance/var_pop
var_samp
Window Functions
Overview
cume_dist
first_value
last_value
lag
lead
percent_rank
rank
row_number
Other Functions
Overview
decode1
javahash
max_pt
ordinal
trans_array
trunc_numeric
url_decode
url_encode
SELECT
Basic Statements
Sort
ORDER BY
SORT BY
CLUSTER BY
DISTRIBUTE BY
Grouping
Column-Based GROUP BY
Expression-Based GROUP BY
Using HAVING in GROUP BY
ROLLUP
GROUPING SETS
Joins
INNER JOIN
LEFT OUTER JOIN
RIGHT OUTER JOIN
FULL OUTER JOIN
IMPLICIT JOIN
Cartesian JOIN
LEFT SEMI JOIN
NON-EQUIJOIN
Clauses
FROM
OVER
WHERE
HAVING
Multi-Layer Nested Subquery
Alias
Table Alias
Column Alias
Set Operations
UNION
INTERSECT
EXCEPT
WITH...AS
CASE...WHEN
Basic CASE Statement
CASE Query Statement
Identifiers
aggregate_func
alias
attr_expr
attr_expr_list
attrs_value_set_expr
boolean_expression
class_name
col
col_comment
col_name
col_name_list
condition
condition_list
cte_name
data_type
db_comment
db_name
else_result_expression
file_format
file_path
function_name
groupby_expression
having_condition
hdfs_path
input_expression
input_format_classname
jar_path
join_condition
non_equi_join_condition
number
num_buckets
output_format_classname
partition_col_name
partition_col_value
partition_specs
property_name
property_value
regex_expression
result_expression
row_format
select_statement
separator
serde_name
sql_containing_cte_name
sub_query
table_comment
table_name
table_properties
table_reference
view_name
view_properties
when_expression
where_condition
window_function
Operators
Relational Operators
Arithmetic Operators
Logical Operators
Flink SQL Syntax Reference
Flink OpenSource SQL 1.15 Syntax Reference
Constraints and Definitions
Supported Data Types
Reserved Keywords
Data Definition Language (DDL)
CREATE TABLE
CREATE CATALOG
CREATE DATABASE
CREATE VIEW
CREATE FUNCTION
Data Manipulation Language (DML)
Overview
Flink OpenSource SQL 1.15 Usage
Formats
Overview
Avro
Canal
Confluent Avro
CSV
Debezium
JSON
Maxwell
Ogg
Orc
Parquet
Raw
Connectors
Overview
BlackHole
ClickHouse
DataGen
Doris
Overview
Source Table
Result Table
Dimension Table
GaussDB(DWS)
Overview
GaussDB(DWS) Source Table (Not Recommended)
GaussDB(DWS) Result Table (Not Recommended)
GaussDB(DWS) Dimension Table (Not Recommended)
Elasticsearch
FileSystem
Source Table
Result Table
HBase
Source Table
Result Table
Dimension Table
Hive
Creating a Hive Catalog
Hive Dialect
Source Table
Result Table
Hive Dimension Table
Using Temporal Join to Associate the Latest Partition of a Dimension Table
Using Temporal Join to Associate the Latest Version of a Dimension Table
JDBC
Kafka
Print
Redis
Source Table
Result Table
Dimension Table
Upsert Kafka
DML Snytax
SELECT
INSERT INTO
Set Operations
Window
Window Functions
Window Aggregation
Window Top-N
Window Deduplication
Window Join
Group Aggregation
Over Aggregation
JOIN
OrderBy & Limit
Top-N
Deduplication
Functions
UDFs
Type Inference
Parameter Transfer
Built-In Functions
Comparison Functions
Logical Functions
Arithmetic Functions
String Functions
Temporal Functions
Conditional Functions
Type Conversion Functions
Collection Functions
JSON Functions
Value Construction Functions
Value Retrieval Functions
Grouping Functions
Hash Functions
Aggregate Functions
Table-Valued Functions
string_split
Flink OpenSource SQL 1.12 Syntax Reference
Constraints and Definitions
Supported Data Types
Syntax
Data Definition Language (DDL)
CREATE TABLE
CREATE VIEW
CREATE FUNCTION
Data Manipulation Language (DML)
Overview
DDL Syntax
Creating Source Tables
DataGen Source Table
GaussDB(DWS) Source Table
HBase Source Table
JDBC Source Table
Kafka Source Table
MySQL CDC Source Table
Postgres CDC Source Table
Redis Source Table
Upsert Kafka Source Table
FileSystem Source Table
Creating Result Tables
BlackHole Result Table
ClickHouse Result Table
GaussDB(DWS) Result Table
Elasticsearch Result Table
HBase Result Table
JDBC Result Table
Kafka Result Table
Print Result Table
Redis Result Table
Upsert Kafka Result Table
FileSystem Result Table
Creating Dimension Tables
GaussDB(DWS) Dimension Table
HBase Dimension Table
JDBC Dimension Table
Redis Dimension Table
Format
Avro
Canal
Confluent Avro
CSV
Debezium
JSON
Maxwell
Raw
DML Snytax
SELECT
Set Operations
Window
JOIN
OrderBy & Limit
Top-N
Deduplication
Functions
User-Defined Functions (UDFs)
Type Inference
Parameter Transfer
Built-In Functions
Mathematical Operation Functions
String Functions
Temporal Functions
Conditional Functions
Type Conversion Functions
Collection Functions
Value Construction Functions
Value Access Functions
Hash Functions
Aggregate Functions
Table-Valued Functions
string_split
Flink Opensource SQL 1.10 Syntax Reference
Constraints and Definitions
Supported Data Types
Syntax Definition
Data Definition Language (DDL)
CREATE TABLE
CREATE VIEW
CREATE FUNCTION
Data Manipulation Language (DML)
Flink OpenSource SQL 1.10 Syntax
Data Definition Language (DDL)
Creating a Source Table
Kafka Source Table
DIS Source Table
JDBC Source Table
GaussDB(DWS) Source Table
Redis Source Table
HBase Source Table
userDefined Source Table
Creating a Result Table
ClickHouse Result Table
Kafka Result Table
Upsert Kafka Result Table
DIS Result Table
JDBC Result Table
GaussDB(DWS) Result Table
Redis Result Table
SMN Result Table
HBase Result Table
Elasticsearch Result Table
OpenTSDB Result Table
User-defined Result Table
Print Result Table
File System Result Table
Creating a Dimension Table
JDBC Dimension Table
GaussDB(DWS) Dimension Table
HBase Dimension Table
Data Manipulation Language (DML)
SELECT
Set Operations
Window
JOIN
OrderBy & Limit
Top-N
Deduplication
Functions
User-Defined Functions
Built-In Functions
Mathematical Operation Functions
String Functions
Temporal Functions
Conditional Functions
Type Conversion Function
Collection Functions
Value Construction Functions
Value Access Functions
Hash Functions
Aggregate Function
Table-Valued Functions
split_cursor
string_split
Historical Version
Flink SQL Syntax (This Syntax Will Not Evolve. Use FlinkOpenSource SQL Instead.)
Constraints and Definitions
Overview
Creating a Source Stream
CloudTable HBase Source Stream
DIS Source Stream
DMS Source Stream
MRS Kafka Source Stream
Open-Source Kafka Source Stream
OBS Source Stream
Creating a Sink Stream
CloudTable HBase Sink Stream
CloudTable OpenTSDB Sink Stream
MRS OpenTSDB Sink Stream
CSS Elasticsearch Sink Stream
DCS Sink Stream
DDS Sink Stream
DIS Sink Stream
DMS Sink Stream
DWS Sink Stream (JDBC Mode)
DWS Sink Stream (OBS-based Dumping)
MRS HBase Sink Stream
MRS Kafka Sink Stream
Open-Source Kafka Sink Stream
File System Sink Stream (Recommended)
OBS Sink Stream
RDS Sink Stream
SMN Sink Stream
Creating a Temporary Stream
Creating a Dimension Table
Creating a Redis Table
Creating an RDS Table
Custom Stream Ecosystem
Custom Source Stream
Custom Sink Stream
Data Manipulation Language (DML)
SELECT
Condition Expression
Window
JOIN Between Stream Data and Table Data
Data Types
User-Defined Functions
Built-In Functions
Mathematical Operation Functions
String Functions
Temporal Functions
Type Conversion Functions
Aggregate Functions
Table-Valued Functions
Other Functions
Geographical Functions
Configuring Time Models
Pattern Matching
StreamingML
Anomaly Detection
Time Series Forecasting
Real-Time Clustering
Deep Learning Model Prediction
Reserved Keywords
API Reference
Before You Start
Overview
API Calling
Endpoints
Constraints
Basic Concepts
Overview
Calling APIs
Making an API Request
Authentication
Returned Values
Getting Started
Creating a Queue
Creating and Submitting a SQL Job
Creating and Submitting a Spark Job
Creating and Submitting a Flink Job
Creating and Using a Datasource Connection
Permission-related APIs
Granting Data Access Control to Users or Projects
Checking the Permissions Granted to a User
Global Variable-related APIs
Creating a Global Variable
Deleting a Global Variable
Modifying a Global Variable
Querying All Global Variables
APIs Related to Enhanced Datasource Connections
Creating an Enhanced Datasource Connection
Deleting an Enhanced Datasource Connection
Listing Enhanced Datasource Connections
Querying an Enhanced Datasource Connection
Binding a Queue
Unbinding a Queue
Modifying Host Information
Querying Authorization of an Enhanced Datasource Connection
APIs Related to Elastic Resource Pools
Creating an Elastic Resource Pool
Querying All Elastic Resource Pools
Deleting an Elastic Resource Pool
Modifying Elastic Resource Pool Information
Querying All Queues in an Elastic Resource Pool
Associating a Queue with an Elastic Resource Pool
Viewing Scaling History of an Elastic Resource Pool
Modifying the Scaling Policy of a Queue Associated with an Elastic Resource Pool
Queue-related APIs (Recommended)
Creating a Queue
Deleting a Queue
Querying All Queues
Viewing Details of a Queue
Restarting, Scaling Out, and Scaling In Queues
Creating an Address Connectivity Test Request
Querying Connectivity Test Details of a Specified Address
SQL Job-related APIs
Submitting a SQL Job (Recommended)
Canceling a Job (Recommended)
Querying All Jobs
Previewing SQL Job Query Results
Exporting Query Results
Querying Job Status
Querying Job Details
Checking SQL Syntax
Querying the Job Execution Progress
SQL Template-related APIs
Saving a SQL template
Checking All SQL Templates
Updating a SQL template
Deleting a SQL template
Flink Job-related APIs
Creating a SQL Job
Updating a SQL Job
Creating a Flink Jar job
Updating a Flink Jar Job
Running Jobs in Batches
Listing Jobs
Querying Job Details
Querying the Job Execution Plan
Stopping Jobs in Batches
Deleting a Job
Deleting Jobs in Batches
Exporting a Flink Job
Importing a Flink Job
Generating a Static Stream Graph for a Flink SQL Job
APIs Related to Flink Job Templates
Creating a Template
Updating a Template
Deleting a Template
Listing Templates
Spark Job-related APIs
Creating a Batch Processing Job
Listing Batch Processing Jobs
Querying Batch Job Details
Querying a Batch Job Status
Canceling a Batch Processing Job
APIs Related to Spark Job Templates
Creating a Job Template
Listing Job Templates
Modifying a Job Template
Obtaining a Job Template
Permissions Policies and Supported Actions
Out-of-Date APIs
Agency-related APIs (Discarded)
Obtaining DLI Agency Information (Discarded)
Creating a DLI Agency (Discarded)
Package Group-related APIs (Discarded)
Uploading a Package Group (Discarded)
Listing Package Groups (Discarded)
Uploading a JAR Package Group (Discarded)
Uploading a PyFile Package Group (Discarded)
Uploading a File Package Group (Discarded)
Querying Resource Packages in a Group (Discarded)
Deleting a Resource Package from a Group (Discarded)
Changing the Owner of a Group or Resource Package (Discarded)
APIs Related to Spark Batch Processing (Discarded)
Querying Batch Job Logs (Discarded)
SQL Job-related APIs (Discarded)
Importing Data (Discarded)
Exporting Data (Discarded)
Resource-related APIs (Discarded)
Database-related APIs (Discarded)
Creating a Database (Discarded)
Deleting a Database (Discarded)
Querying All Databases (Discarded)
Modifying a Database Owner (Discarded)
Table-related APIs (Discarded)
Creating a Table (Discarded)
Deleting a Table (Discarded)
Querying All Tables (Discarded)
Describing Table Information (Discarded)
Previewing Table Content (Discarded)
Listing Partitions (Discarded)
Permission-related APIs (Discarded)
Granting Queue Permissions to a User (Discarded)
Querying Queue Users (Discarded)
Granting Data Permission to Users (Discarded)
Querying Database Users (Discarded)
Querying Table Users (Discarded)
Querying a User's Table Permissions (Discarded)
Queue-related APIs (Discarded)
Creating a Scheduled CU Change (Discarded)
Viewing a Scheduled CU Change (Discarded)
Deleting Scheduled CU Changes in Batches (Discarded)
Deleting a Scheduled CU Change (Discarded)
Modifying a Scheduled CU Change (Discarded)
Datasource Authentication-related APIs (Discarded)
Creating Datasource Authentication (Discarded)
Listing Datasource Authentication Information (Discarded)
Updating Datasource Authentication (Discarded)
Deleting Datasource Authentication (Discarded)
APIs Related to Enhanced Datasource Connections (Discarded)
Creating a Route (Discarded)
Deleting a Route (Discarded)
Template-related APIs (Discarded)
Querying All Sample SQL Templates (Discarded)
APIs Related to Flink Jobs (Discarded)
Querying Job Monitoring Information (Discarded)
Granting OBS Permissions to DLI
Public Parameters
Status Codes
Error Codes
Obtaining a Project ID
Obtaining an Account ID
SDK Reference
Introduction to DLI SDKs
What Is DLI SDK
Content Navigation
Preparing the SDK Environment
Prerequisites
Configuring the Java Environment
Configuring the Python Environment
Mapping Between DLI SDKs and APIs
Java SDK
Instructions
Initializing the DLI Client
OBS Authorization
Queue-Related SDKs
Resource-Related SDKs
SDKs Related to SQL Jobs
Database-Related SDKs
Table-Related SDKs
Job-related SDKs
SDKs Related to Flink Jobs
SDKs Related to Spark Jobs
SDKs Related to Flink Job Templates
Python SDK
Instructions
Initializing the DLI Client
Queue-Related SDKs
Resource-Related SDKs
SDKs Related to SQL Jobs
Database-Related SDKs
Table-Related SDKs
Job-related SDKs
SDKs Related to Spark Jobs
Change History
FAQs
Flink Jobs
Usage
What Data Formats and Data Sources Are Supported by DLI Flink Jobs?
How Do I Authorize a Subuser to View Flink Jobs?
How Do I Set Auto Restart upon Exception for a Flink Job?
How Do I Save Flink Job Logs?
How Can I Check Flink Job Results?
Why Is Error "No such user. userName:xxxx." Reported on the Flink Job Management Page When I Grant Permission to a User?
How Do I Know Which Checkpoint the Flink Job I Stopped Will Be Restored to When I Start the Job Again?
Which Flink Version Does DLI Support? Is Flink 1.13 Supported? Which Version Is the Next?
Why Is a Message Displayed Indicating That the SMN Topic Does Not Exist When I Use the SMN Topic in DLI?
Flink SQL
How Much Data Can Be Processed in a Day by a Flink SQL Job?
Does Data in the Temporary Stream of Flink SQL Need to Be Cleared Periodically? How Do I Clear the Data?
Why Is a Message Displayed Indicating That the OBS Bucket Is Not Authorized When I Select an OBS Bucket for a Flink SQL Job?
How Do I Create an OBS Partitioned Table for a Flink SQL Job?
How Do I Change the Number of Kafka Partitions of a Flink SQL Job Without Stopping It?
How Do I Dump Data to OBS and Create an OBS Partitioned Table?
Why Is Error Message "DLI.0005" Displayed When I Use an EL Expression to Create a Table in a Flink SQL Job?
Why Is No Data Queried in the DLI Table Created Using the OBS File Path When Data Is Written to OBS by a Flink Job Output Stream?
Why Does a Flink SQL Job Fails to Be Executed, and Is "connect to DIS failed java.lang.IllegalArgumentException: Access key cannot be null" Displayed in the Log?
Why Is Error "Not authorized" Reported When a Flink SQL Job Reads DIS Data?
Data Writing Fails After a Flink SQL Job Consumed Kafka and Sank Data to the Elasticsearch Cluster
How Does Flink Opensource SQL Parse Nested JSON?
Why Is the RDS Database Time Read by a Flink Opensource SQL Job Different from RDS Database Time?
What Are the Syntax Differences Between Flink SQL and Flink Opensource SQL?
Why Does Job Submission Fail When the failure-handler Parameter of the Elasticsearch Result Table for a Flink Opensource SQL Job Is Set to retry_rejected?
Flink Jar Jobs
How Do I Configure Checkpoints for Flink Jar Jobs and Save the Checkpoints to OBS?
Does a Flink JAR Job Support Configuration File Upload? How Do I Upload a Configuration File?
Why Does the Submission Fail Due to Flink JAR File Conflict?
Why Does a Flink Jar Job Fail to Access GaussDB(DWS) and a Message Is Displayed Indicating Too Many Client Connections?
Why Is Error Message "Authentication failed" Displayed During Flink Jar Job Running?
Why Is Error Invalid OBS Bucket Name Reported After a Flink Job Submission Failed?
Why Does the Flink Submission Fail Due to Hadoop JAR File Conflict?
How Do I Connect a Flink jar Job to SASL_SSL?
Performance Tuning
What Is the Recommended Configuration for a Flink Job?
How Do I Optimize Performance of a Flink Job?
How Do I Write Data to Different Elasticsearch Clusters in a Flink Job?
How Do I Prevent Data Loss After Flink Job Restart?
O&M Guide
How Do I Locate a Flink Job Submission Error?
How Do I Locate a Flink Job Running Error?
How Do I Know Whether a Flink Job Can Be Restored from a Checkpoint After Being Restarted?
Why Does DIS Stream Not Exist During Job Semantic Check?
Why Is the OBS Bucket Selected for Job Not Authorized?
Why Are Logs Not Written to the OBS Bucket After a DLI Flink Job Fails to Be Submitted for Running?
How Do I Configure Connection Retries for Kafka Sink If it is Disconnected?
Why Is Information Displayed on the FlinkUI/Spark UI Page Incomplete?
Why Is the Flink Job Abnormal Due to Heartbeat Timeout Between JobManager and TaskManager?
Why Is Error "Timeout expired while fetching topic metadata" Repeatedly Reported in Flink JobManager Logs?
Problems Related to SQL Jobs
Usage
SQL Jobs
Job Development
How Do I Merge Small Files?
How Do I Use DLI to Access Data in an OBS Bucket?
How Do I Specify an OBS Path When Creating an OBS Table?
How Do I Create a Table Using JSON Data in an OBS Bucket?
How Can I Use the count Function to Perform Aggregation?
How Do I Synchronize DLI Table Data from One Region to Another?
How Do I Insert Table Data into Specific Fields of a Table Using a SQL Job?
How Do I Delete Table Data?
Job O&M Errors
Why Is Error "path obs://xxx already exists" Reported When Data Is Exported to OBS?
Why Is Error "SQL_ANALYSIS_ERROR: Reference 't.id' is ambiguous, could be: t.id, t.id.;" Displayed When Two Tables Are Joined?
Why Is Error "The current account does not have permission to perform this operation,the current account was restricted. Restricted for no budget." Reported when a SQL Statement Is Executed?
Why Is Error "There should be at least one partition pruning predicate on partitioned table XX.YYY" Reported When a Query Statement Is Executed?
Why Is Error "IllegalArgumentException: Buffer size too small. size" Reported When Data Is Loaded to an OBS Foreign Table?
Why Is Error "DLI.0002 FileNotFoundException" Reported During SQL Job Running?
Why Is a Schema Parsing Error Reported When I Create a Hive Table Using CTAS?
Why Is Error "org.apache.hadoop.fs.obs.OBSIOException" Reported When I Run DLI SQL Scripts on DataArts Studio?
Why Is Error "UQUERY_CONNECTOR_0001:Invoke DLI service api failed" Reported in the Job Log When I Use CDM to Migrate Data to DLI?
Why Is Error "File not Found" Reported When I Access a SQL Job?
Why Is Error "DLI.0003: AccessControlException XXX" Reported When I Access a SQL Job?
Why Is Error "DLI.0001: org.apache.hadoop.security.AccessControlException: verifyBucketExists on {{bucket name}}: status [403]" Reported When I Access a SQL Job?
Why Is Error "The current account does not have permission to perform this operation,the current account was restricted. Restricted for no budget" Reported During SQL Statement Execution? Restricted for no budget.
O&M Guide
How Do I Troubleshoot Slow SQL Jobs?
How Do I View DLI SQL Logs?
How Do I View SQL Execution Records?
How Do I Eliminate Data Skew by Configuring AE Parameters?
What Can I Do If a Table Cannot Be Queried on the DLI Console?
The Compression Ratio of OBS Tables Is Too High
How Can I Avoid Garbled Characters Caused by Inconsistent Character Codes?
Do I Need to Grant Table Permissions to a User and Project After I Delete a Table and Create One with the Same Name?
Why Can't I Query Table Data After Data Is Imported to a DLI Partitioned Table Because the File to Be Imported Does Not Contain Data in the Partitioning Column?
How Do I Fix the Data Error Caused by CRLF Characters in a Field of the OBS File Used to Create an External OBS Table?
Why Does a SQL Job That Has Join Operations Stay in the Running State?
The on Clause Is Not Added When Tables Are Joined. Cartesian Product Query Causes High Resource Usage of the Queue, and the Job Fails to Be Executed
Why Can't I Query Data After I Manually Add Data to the Partition Directory of an OBS Table?
Why Is All Data Overwritten When insert overwrite Is Used to Overwrite Partitioned Table?
Why Is a SQL Job Stuck in the Submitting State?
Why Is the create_date Field in the RDS Table Is a Timestamp in the DLI query result?
What Can I Do If datasize Cannot Be Changed After the Table Name Is Changed in a Finished SQL Job?
Why Is the Data Volume Changes When Data Is Imported from DLI to OBS?
Problems Related to Spark Jobs
Usage
Spark Jobs
Job Development
How Do I Use Spark to Write Data into a DLI Table?
How Do I Set the AK/SK for a Queue to Operate an OBS Table?
How Do I View the Resource Usage of DLI Spark Jobs?
How Do I Use Python Scripts to Access the MySQL Database If the pymysql Module Is Missing from the Spark Job Results Stored in MySQL?
How Do I Run a Complex PySpark Program in DLI?
How Does a Spark Job Access a MySQL Database?
How Do I Use JDBC to Set the spark.sql.shuffle.partitions Parameter to Improve the Task Concurrency?
How Do I Read Uploaded Files for a Spark Jar Job?
Job O&M Errors
What Can I Do When Receiving java.lang.AbstractMethodError in the Spark Job?
Why Are Errors "ResponseCode: 403" and "ResponseStatus: Forbidden" Reported When a Spark Job Accesses OBS Data?
Why Is Error "verifyBucketExists on XXXX: status [403]" Reported When I Use a Spark Job to Access an OBS Bucket That I Have Access Permission?
Why Is a Job Running Timeout Reported When a Spark Job Runs a Large Amount of Data?
Why Does the Job Fail to Be Executed and the Log Shows that the File Directory Is Abnormal When I Use a Spark Job to Access Files in SFTP?
Why Does the Job Fail to Be Executed Due to Insufficient Database and Table Permissions?
Why Can't I Find the global_temp Database in the Spark 3.x Job Log?
Why Does the DataSource Syntax Fail to Create an OBS Table in Avro Format When Spark 2.3.x Is Used?
O&M Guide
Why Can't I Find the Specified Python Environment After Adding the Python Package?
Why Is a Spark Jar Job Stuck in the Submitting State?
Product Consultation
Usage
What Is DLI?
What Are the Application Scenarios of DLI?
Which Data Formats Does DLI Support?
What Are the Differences Between DLI Flink and MRS Flink?
What Are the Differences Between MRS Spark and DLI Spark?
Where Can DLI Data Be Stored?
What Are the Differences Between DLI Tables and OBS Tables?
How Can I Use DLI If Data Is Not Uploaded to OBS?
Can I Import OBS Bucket Data Shared by Other Tenants into DLI?
Why Is Error "Failed to create the database. {"error_code":"DLI.1028";"error_msg":"Already reached the maximum quota of databases:XXX"." Reported?
Can a Member Account Use Global Variables Created by Other Member Accounts?
Job Management
How Do I Manage Tens of Thousands of Jobs Running on DLI?
How Do I Change the Name of a Field in a Created Table?
Privacy and Security
Does DLI Have the Apache Spark Command Injection Vulnerability (CVE-2022-33891)?
Billing
What Is the Billing Mode of DLI?
Can I Change a Yearly/Monthly Queue to a Pay-per-Use Queue?
Can I Change a Pay-Per-Use Queue to a Yearly/Monthly Queue?
When Is a Data Lake Queue Idle?
How Do I Check the Billing?
What Is the Difference Between the Following Two Payment Modes: One Is to Purchase 4,000-CU Resources for Three Months at a Time, the Other Is to Purchase 4,000-CU Resources for One Month for Three Times?
How Do I Purchase 30-CU Resources?
How Will I Be Billed if Only Two CUs Are Used to Run a Flink Job on a Queue of 16 CUs?
Do I Need to Pay Extra Fees for Purchasing a Queue Billed Based on the Scanned Data Volume?
Quota
What Is User Quota?
How Do I View My Quotas?
How Do I Increase a Quota?
Permission
Usage
How Do I Manage Fine-Grained DLI Permissions?
What Is Column Permission Granting of a DLI Partition Table?
O&M Guide
Why Does My Account Have Insufficient Permissions Due to Arrears?
Why Does the System Display a Message Indicating Insufficient Permissions When I Update a Program Package?
Why Is Error "DLI.0003: Permission denied for resource..." Reported When I Run a SQL Statement?
Why Can't I Query Table Data After I've Been Granted Table Permissions?
Will an Error Be Reported if the Inherited Permissions Are Regranted to a Table That Inherits Database Permissions?
Why Can't I Query a View After I'm Granted the Select Table Permission on the View?
Queue
Usage
Does the Description of a DLI Queue Can Be Modified?
Will Table Data in My Database Be Lost If I Delete a Queue?
How Does DLI Ensure the Reliability of Spark Jobs When a Queue Is Abnormal?
How Do I Monitor Queue Exceptions?
O&M Guide
How Do I View DLI Queue Load?
How Do I Determine Whether There Are Too Many Jobs in the Current Queue?
How Do I Switch an Earlier-Version Spark Queue to a General-Purpose Queue?
Why Cannot I View the Resource Running Status of DLI Queues on Cloud Eye?
How Do I Allocate Queue Resources for Running Spark Jobs If I Have Purchased 64 CUs?
Why Is Error "Queue plans create failed. The plan xxx target cu is out of quota" Reported When I Schedule CU Changes?
Why Is a Timeout Exception Reported When a DLI SQL Statement Fails to Be Executed on the Default Queue?
Datasource Connections
Datasource Connections
Why Do I Need to Create a VPC Peering Connection for an Enhanced Datasource Connection?
Failed to Bind a Queue to an Enhanced Datasource Connection
DLI Failed to Connect to GaussDB(DWS) Through an Enhanced Datasource Connection
How Do I Do if the Datasource Connection Is Created But the Network Connectivity Test Fails?
How Do I Configure the Network Between a DLI Queue and a Data Source?
What Can I Do If a Datasource Connection Is Stuck in Creating State When I Try to Bind a Queue to It?
How Do I Bind a Datasource Connection to a Pay-per-Use Queue?
How Do I Connect DLI to Data Sources?
Cross-Source Analysis
How Can I Perform Query on Data Stored on Services Rather Than DLI?
How Can I Access Data Across Regions?
How Do I Set the Auto-increment Primary Key or Other Fields That Are Automatically Filled in the RDS Table When Creating a DLI and Associating It with the RDS Table?
Datasource Connection O&M
Why Is the Error Message "communication link failure" Displayed When I Use a Newly Activated Datasource Connection?
Connection Times Out During MRS HBase Datasource Connection, and No Error Is Recorded in Logs
Why Can't I Find the Subnet When Creating a DLI Datasource Connection?
Error Message "Incorrect string value" Is Displayed When insert overwrite Is Executed on a Datasource RDS Table
Null Pointer Error Is Displayed When the System Creates a Datasource RDS Table
Error Message "org.postgresql.util.PSQLException: ERROR: tuple concurrently updated" Is Displayed When the System Executes insert overwrite on a Datasource GaussDB(DWS) Table
RegionTooBusyException Is Reported When Data Is Imported to a CloudTable HBase Table Through a Datasource Table
A Null Value Is Written Into a Non-Null Field When a DLI Datasource Connection Is Used to Connect to a GaussDB(DWS) Table
An Insert Operation Failed After the Schema of the GaussDB(DWS) Source Table Is Updated
APIs
How Do I Obtain the AK/SK Pair?
How Do I Obtain the Project ID?
Why Is Error "unsupported media Type" Reported When I Subimt a SQL Job?
Are Project IDs of Different Accounts the Same When They Are Used to Call APIs?
What Can I Do If an Error Is Reported When the Execution of the API for Creating a SQL Job Times Out?
What Can I Do If Chinese Characters Returned by an API Are Garbled Characters?
SDKs
How Do I Set the Timeout Duration for Querying SQL Job Results Using SDK?
How Do I Handle the dli.xxx,unable to resolve host address Error?
Change History
Videos
SQL Syntax Reference (To Be Offline)
Notice on Taking This Syntax Reference Offline
Spark SQL Syntax Reference (Unavailable Soon)
Common Configuration Items of Batch SQL Jobs
SQL Syntax Overview of Batch Jobs
Databases
Creating a Database
Deleting a Database
Viewing a Specified Database
Viewing All Databases
Creating an OBS Table
Creating an OBS Table Using the DataSource Syntax
Creating an OBS Table Using the Hive Syntax
Creating a DLI Table
Creating a DLI Table Using the DataSource Syntax
Creating a DLI Table Using the Hive Syntax
Deleting a Table
Viewing Tables
Viewing All Tables
Viewing Table Creation Statements
Viewing Table Properties
Viewing All Columns in a Specified Table
Viewing All Partitions in a Specified Table
Viewing Table Statistics
Modifying a Table
Adding a Column
Modifying Column Comments
Enabling or Disabling Multiversion Backup
Syntax for Partitioning a Table
Adding Partition Data (Only OBS Tables Supported)
Renaming a Partition (Only OBS Tables Supported)
Deleting a Partition
Deleting Partitions by Specifying Filter Criteria (Only OBS Tables Supported)
Altering the Partition Location of a Table (Only OBS Tables Supported)
Updating Partitioned Table Data (Only OBS Tables Supported)
Updating Table Metadata with REFRESH TABLE
Importing Data to the Table
Inserting Data
Clearing Data
Exporting Search Results
Backing Up and Restoring Data of Multiple Versions
Setting the Retention Period for Multiversion Backup Data
Viewing Multiversion Backup Data
Restoring Multiversion Backup Data
Configuring the Trash Bin for Expired Multiversion Data
Deleting Multiversion Backup Data
Table Lifecycle Management
Specifying the Lifecycle of a Table When Creating the Table
Modifying the Lifecycle of a Table
Disabling or Restoring the Lifecycle of a Table
Creating a Datasource Connection with an HBase Table
Creating a DLI Table and Associating It with HBase
Inserting Data to an HBase Table
Querying an HBase Table
Creating a Datasource Connection with an OpenTSDB Table
Creating a DLI Table and Associating It with OpenTSDB
Inserting Data to the OpenTSDB Table
Querying an OpenTSDB Table
Creating a Datasource Connection with a DWS table
Creating a DLI Table and Associating It with DWS
Inserting Data to the DWS Table
Querying the DWS Table
Creating a Datasource Connection with an RDS Table
Creating a DLI Table and Associating It with RDS
Inserting Data to the RDS Table
Querying the RDS Table
Creating a Datasource Connection with a CSS Table
Creating a DLI Table and Associating It with CSS
Inserting Data to the CSS Table
Querying the CSS Table
Creating a Datasource Connection with a DCS Table
Creating a DLI Table and Associating It with DCS
Inserting Data to a DCS Table
Querying the DCS Table
Creating a Datasource Connection with a DDS Table
Creating a DLI Table and Associating It with DDS
Inserting Data to the DDS Table
Querying the DDS Table
Creating a Datasource Connection with an Oracle Table
Creating a DLI Table and Associating It with Oracle
Inserting Data to an Oracle Table
Querying an Oracle Table
Views
Creating a View
Deleting a View
Viewing the Execution Plan
Data Permissions Management
Data Permissions List
Creating a Role
Deleting a Role
Binding a Role
Unbinding a Role
Displaying a Role
Granting a Permission
Revoking a Permission
Showing Granted Permissions
Displaying the Binding Relationship Between All Roles and Users
Data Types
Overview
Primitive Data Types
Complex Data Types
User-Defined Functions
Creating a Function
Deleting a Function
Displaying Function Details
Displaying All Functions
Built-in Functions
Date Functions
Overview
add_months
current_date
current_timestamp
date_add
dateadd
date_sub
date_format
datediff
datediff1
datepart
datetrunc
day/dayofmonth
from_unixtime
from_utc_timestamp
getdate
hour
isdate
last_day
lastday
minute
month
months_between
next_day
quarter
second
to_char
to_date
to_date1
to_utc_timestamp
trunc
unix_timestamp
weekday
weekofyear
year
String Functions
Overview
ascii
concat
concat_ws
char_matchcount
encode
find_in_set
get_json_object
instr
instr1
initcap
keyvalue
length
lengthb
levenshtein
locate
lower/lcase
lpad
ltrim
parse_url
printf
regexp_count
regexp_extract
replace
regexp_replace
regexp_replace1
regexp_instr
regexp_substr
repeat
reverse
rpad
rtrim
soundex
space
substr/substring
substring_index
split_part
translate
trim
upper/ucase
Mathematical Functions
Overview
abs
acos
asin
atan
bin
bround
cbrt
ceil
conv
cos
cot1
degrees
e
exp
factorial
floor
greatest
hex
least
ln
log
log10
log2
median
negative
percentlie
percentlie_approx
pi
pmod
positive
pow
radians
rand
round
shiftleft
shiftright
shiftrightunsigned
sign
sin
sqrt
tan
Aggregate Functions
Overview
avg
corr
count
covar_pop
covar_samp
max
min
percentile
percentile_approx
stddev_pop
stddev_samp
sum
variance/var_pop
var_samp
Window Functions
Overview
cume_dist
first_value
last_value
lag
lead
percent_rank
rank
row_number
Other Functions
Overview
decode1
javahash
max_pt
ordinal
trans_array
trunc_numeric
url_decode
url_encode
Basic SELECT Statements
Filtering
WHERE Filtering Clause
HAVING Filtering Clause
Sorting
ORDER BY
SORT BY
CLUSTER BY
DISTRIBUTE BY
Grouping
Column-Based GROUP BY
Expression-Based GROUP BY
GROUP BY Using HAVING
ROLLUP
GROUPING SETS
JOIN
INNER JOIN
LEFT OUTER JOIN
RIGHT OUTER JOIN
FULL OUTER JOIN
IMPLICIT JOIN
Cartesian JOIN
LEFT SEMI JOIN
NON-EQUIJOIN
Subquery
Subquery Nested by WHERE
Subquery Nested by FROM
Subquery Nested by HAVING
Multi-Layer Nested Subquery
Alias
AS for Table
AS for Column
Set Operations
UNION
INTERSECT
EXCEPT
WITH...AS
CASE...WHEN
Basic CASE Statement
CASE Query Statement
OVER Clause
Flink OpenSource SQL 1.12 Syntax Reference
Constraints and Definitions
Supported Data Types
Syntax
Data Definition Language (DDL)
CREATE TABLE
CREATE VIEW
CREATE FUNCTION
Data Manipulation Language (DML)
Overview
DDL Syntax
Creating Source Tables
DataGen Source Table
GaussDB(DWS) Source Table
HBase Source Table
JDBC Source Table
Kafka Source Table
MySQL CDC Source Table
Postgres CDC Source Table
Redis Source Table
Upsert Kafka Source Table
Creating Result Tables
BlackHole Result Table
ClickHouse Result Table
GaussDB(DWS) Result Table
Elasticsearch Result Table
HBase Result Table
JDBC Result Table
Kafka Result Table
Print Result Table
Redis Result Table
Upsert Kafka Result Table
FileSystem Result Table
Creating Dimension Tables
GaussDB(DWS) Dimension Table
HBase Dimension Table
JDBC Dimension Table
Redis Dimension Table
Format
Avro
Canal
Confluent Avro
CSV
Debezium
JSON
Maxwell
Raw
DML Snytax
SELECT
Set Operations
Window
JOIN
OrderBy & Limit
Top-N
Deduplication
Functions
User-Defined Functions (UDFs)
Built-In Functions
Mathematical Operation Functions
String Functions
Temporal Functions
Conditional Functions
Type Conversion Functions
Collection Functions
Value Construction Functions
Value Access Functions
Hash Functions
Aggregate Functions
Table-Valued Functions
string_split
Flink Opensource SQL 1.10 Syntax Reference
Constraints and Definitions
Supported Data Types
Syntax Definition
Data Definition Language (DDL)
CREATE TABLE
CREATE VIEW
CREATE FUNCTION
Data Manipulation Language (DML)
Flink OpenSource SQL 1.10 Syntax
Data Definition Language (DDL)
Creating a Source Table
Kafka Source Table
DIS Source Table
JDBC Source Table
GaussDB(DWS) Source Table
Redis Source Table
HBase Source Table
userDefined Source Table
Creating a Result Table
ClickHouse Result Table
Kafka Result Table
Upsert Kafka Result Table
DIS Result Table
JDBC Result Table
GaussDB(DWS) Result Table
Redis Result Table
SMN Result Table
HBase Result Table
Elasticsearch Result Table
OpenTSDB Result Table
User-defined Result Table
Print Result Table
File System Result Table
Creating a Dimension Table
JDBC Dimension Table
GaussDB(DWS) Dimension Table
HBase Dimension Table
Data Manipulation Language (DML)
SELECT
Set Operations
Window
JOIN
OrderBy & Limit
Top-N
Deduplication
Functions
User-Defined Functions
Built-In Functions
Mathematical Operation Functions
String Functions
Temporal Functions
Conditional Functions
Type Conversion Function
Collection Functions
Value Construction Functions
Value Access Functions
Hash Functions
Aggregate Function
Table-Valued Functions
split_cursor
string_split
Historical Versions (Unavailable Soon)
Flink SQL Syntax
SQL Syntax Constraints and Definitions
SQL Syntax Overview of Stream Jobs
Creating a Source Stream
CloudTable HBase Source Stream
DIS Source Stream
DMS Source Stream
MRS Kafka Source Stream
Open-Source Kafka Source Stream
OBS Source Stream
Creating a Sink Stream
CloudTable HBase Sink Stream
CloudTable OpenTSDB Sink Stream
MRS OpenTSDB Sink Stream
CSS Elasticsearch Sink Stream
DCS Sink Stream
DDS Sink Stream
DIS Sink Stream
DMS Sink Stream
DWS Sink Stream (JDBC Mode)
DWS Sink Stream (OBS-based Dumping)
MRS HBase Sink Stream
MRS Kafka Sink Stream
Open-Source Kafka Sink Stream
File System Sink Stream (Recommended)
OBS Sink Stream
RDS Sink Stream
SMN Sink Stream
Creating a Temporary Stream
Creating a Dimension Table
Creating a Redis Table
Creating an RDS Table
Custom Stream Ecosystem
Custom Source Stream
Custom Sink Stream
Data Type
Built-In Functions
Mathematical Operation Functions
String Functions
Temporal Functions
Type Conversion Functions
Aggregate Functions
Table-Valued Functions
Other Functions
User-Defined Functions
Geographical Functions
SELECT
Condition Expression
Window
JOIN Between Stream Data and Table Data
Configuring Time Models
Pattern Matching
StreamingML
Anomaly Detection
Time Series Forecasting
Real-Time Clustering
Deep Learning Model Prediction
Reserved Keywords
Identifiers
aggregate_func
alias
attr_expr
attr_expr_list
attrs_value_set_expr
boolean_expression
col
col_comment
col_name
col_name_list
condition
condition_list
cte_name
data_type
db_comment
db_name
else_result_expression
file_format
file_path
function_name
groupby_expression
having_condition
input_expression
join_condition
non_equi_join_condition
number
partition_col_name
partition_col_value
partition_specs
property_name
property_value
regex_expression
result_expression
select_statement
separator
sql_containing_cte_name
sub_query
table_comment
table_name
table_properties
table_reference
when_expression
where_condition
window_function
Operators
Relational Operators
Arithmetic Operators
Logical Operators
Change History