Configuring CDL ENV Variables

Scenario

To capture data to or from Hudi, create and manage Hudi environment variables by performing the operations in this section.

Prerequisites

A user with the CDL management permission has been created for the cluster with Kerberos authentication enabled. For details, see CDL User Permission Management.

Procedure

1. Access the CDLService web UI as a user with the CDL management permissions or the admin user (for the cluster where Kerberos authentication is not enabled). For details, see Logging In to the CDLService WebUI.
2. Choose ENV Management and click Add Env. In the displayed dialog box, set related parameters.

   Table 1 Parameters for adding an ENV

   Parameter	Description	Example Value
   Name	ENV name	spark-env
   Type	ENV type	spark
   Driver Memory	Memory for the driver process, in GB ( by default).	1 GB
   Executor Memory	Memory size for each Executor process, in GB by default. Its string format is the same as that of JVM.	1 GB
   Executor Cores	Number of CPU cores occupied by each Executor	1
   Number Executors	Number of Executors	1
   Queue	Name of the Yarn tenant queue. Jobs are submitted to the default queue if this parameter is not specified.	-
   Description	ENV description	-

3. Click OK.

   After the ENV is created, you can click Edit or Delete in the Operation column to edit or delete the ENV, respectively.