Scalar Python UDF Registration Parameters
Registering a Scalar Python UDF involves registering an original Python function into the database.
Whether it is explicit or implicit registration, for registering a Python-type Scalar UDF, you can currently pass the following parameters:
Registration Parameter |
Description |
Type |
Default Value |
---|---|---|---|
name |
Specifies the actual storage name of the UDF in the database. |
str | None |
None |
database |
Specifies the LakeFormation database where the UDF resides. |
str | None |
None |
fn |
Specifies the original Python function of the UDF. |
Callable |
None |
signature (currently unavailable) |
Specifies the UDF function signature and return value type. |
ibis.common.annotations.Signature | None |
None |
replace (currently unavailable) |
Specifies whether the UDF supports in-place modification. |
bool |
False |
temporary (currently unavailable) |
Specifies whether the UDF has a session-level lifecycle. |
bool |
False |
if_not_exist (currently unavailable) |
Specifies whether to skip errors for existing UDFs. |
bool |
False |
strict |
Specifies whether the UDF automatically filters NULL values. |
bool |
True |
volatility |
Specifies the stability of the UDF. |
VolatilityType.VOLATILE | VolatilityType.STABLE | VolatilityType.IMMUTABLE |
VolatilityType.VOLATILE |
runtime_version (currently unavailable) |
Specifies the Python version for executing the UDF. |
str |
sys.version_info |
imports |
Specifies the external code files on which the UDF depends. |
List[str] |
None |
packages |
Specifies the Python modules on which the UDF depends. |
List[Union[str, module]] |
None |
register_type (currently, only RegisterType.OBS is supported) |
Specifies the registration form of the UDF. |
RegisterType.TEXT | RegisterType.OBS |
RegisterType.OBS |
comment |
Specifies user comments for the UDF. |
str | None |
None |
Notes:
- For the imports parameter, only file paths in the same directory or subdirectories as the .py file containing the current Python function are allowed.
- For the fn parameter, if fn is not in the .py file where the Scalar Python UDF is being registered, then the file path defining fn must also be added to the imports parameter, for example:
from process import outer con = ibis.fabric.connect(...) # Register a UDF. udf = con.udf.python.register( outer(), # fn introduced externally imports=["process.py"] # Add the file path for fn )
- For the signature parameter, user input is currently not allowed. Only automatic inference of parameter/return value types is supported. See Type Inference of the signature Parameter for details.
- For the volatility parameter, the meanings of the three enumeration types are:
- VolatilityType.VOLATILE: Function results may change at any time.
- VolatilityType.STABLE: For fixed inputs, the function's result does not change during a single scan.
- VolatilityType.IMMUTABLE: The function always produces the same result for identical inputs.
This parameter, along with SHIPPABLE, affects the push-down execution of functions. Specifically: Functions of the IMMUTABLE type can always be pushed down to DNs for execution. For STABLE or VOLATILE type functions, they can only be pushed down to DNs for execution if their property is SHIPPABLE.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot