Developing User-Defined Impala Functions

When built-in functions of Impala cannot meet requirements, you can compile user-defined functions (UDFs) and use them for query.

According to implementation methods, UDFs are classified as follows:

• Common UDFs: used to perform operations on a single data row and export a single data row.
• User-Defined Aggregate Functions (UDAFs): These functions process multiple input rows and return a single output value.
• User-Defined Table-Generating Functions (UDTFs): These functions process a single input row and return multiple output rows. Impala does not support this type of UDFs.

UDFs are classified as follows based on their usage:

• Temporary functions: used only in the current session and must be recreated after a session restarts.
• Permanent functions: used in multiple sessions. You do not need to create them every time a session restarts.

Impala supports both the development of native Java UDFs and the reuse of UDFs originally created for Hive. However, this compatibility is dependent on using data types that are supported by Impala. For details about the data types supported by Impala, visit https://impala.apache.org/docs/build3x/html/topics/impala_datatypes.html.

In addition, Impala supports UDFs written in C++, which provides better performance than Java UDFs.

[localhost:21000] > create database udfs;
[localhost:21000] > use udfs;
[localhost:21000] > create function my_lower(string)
returns string location '/user/hive/udfs/hive.jar'
symbol='org.apache.hadoop.hive.ql.udf.UDFLower';
[localhost:21000] > select my_lower('Some String NOT ALREADY LOWERCASE');
+----------------------------------------------------+
| udfs.my_lower('some string not already lowercase') |
+----------------------------------------------------+
| some string not already lowercase |
+----------------------------------------------------+
Returned 1 row(s) in 0.11s
[localhost:21000] > create table t2 (s string);
[localhost:21000] > insert into t2 values ('lower'),('UPPER'),('Init cap'),('CamelCase');
Inserted 4 rows in 2.28s
[localhost:21000] > select * from t2;
+-----------+
| s |
+-----------+
| lower |
| UPPER |
| Init cap |
| CamelCase |
+-----------+
Returned 4 row(s) in 0.47s
[localhost:21000] > select my_lower(s) from t2;
+------------------+
| udfs.my_lower(s) |
+------------------+
| lower |
| upper |
| init cap |
| camelcase |
+------------------+
Returned 4 row(s) in 0.54s