Help Center/ Data Lake Insight/ Spark SQL Syntax Reference/ Built-In Functions/ Aggregate Functions/ corr
Updated on 2023-10-25 GMT+08:00
View PDF
Share

corr

This function is used to return the correlation coefficient between two columns of numerical values.

Syntax

corr(col1, col2)

Parameters

Table 1 Parameters

Parameter

Mandatory

Type

Description

col1

Yes

DOUBLE, BIGINT, INT, SMALLINT, TINYINT, FLOAT, or DECIMAL

Columns with a data type of numeric. If the value is of any other type, NULL is returned.

col2

Yes

DOUBLE, BIGINT, INT, SMALLINT, TINYINT, FLOAT, or DECIMAL

Columns with a data type of numeric. If the values are of any other type, NULL is returned.

Return Values

The return value is of the DOUBLE type.

Example Code

  • Calculates the correlation coefficient between the inventory (items) and the price of all products. An example command is as follows: 
    select corr(items,price) from warehouse;

    The command output is as follows:

    _c0        
1.242355
  • When used with group by, it groups all offerings by warehouse (warehourseId) and returns the correlation coefficient between the inventory (items) and the price of offerings within each group. An example command is as follows: 
    select warehourseId, corr(items,price) from warehourse group by warehourseId;

    The command output is as follows:

    warehouseId _c1        
city1   0.43124   
city2   0.53344   
city3   0.73425
Parent topic: Aggregate Functions