Creating a HetuEngine Compute Instance

Scenarios

This section describes how to create a HetuEngine compute instance. To stop a cluster with successfully created compute instances, you must first manually stop the compute instances. After the cluster is restarted, you must manually start the compute instance if you want to use it within the cluster.

Prerequisites

• You have created a user for accessing the HetuEngine web UI, for example, hetu_user. For details, see Creating a HetuEngine Permission Role.
• A tenant has been created in the target cluster. Ensure the tenant has sufficient CPU and memory resources before modifying the HetuEngine compute instance configuration.
  • You must use a leaf tenant when you create a HetuEngine compute instance because YARN jobs can only be submitted to the queues of a leaf tenant.
  • To avoid uncertainties caused by resource competition, you are advised to create dedicated resource pools for HetuEngine tenants.

Procedure

1. Log in to MRS Manager of the cluster as user hetu_user.

   For details about how to log in to MRS Manager, see Accessing MRS Manager.

2. Choose Cluster > Services > HetuEngine to go to the HetuEngine service page.
3. In the Basic Information area on the Dashboard page, click the link next to HSConsole WebUI to access the HSConsole page.
4. Click Compute Instance and then Create Configuration, and configure the compute instance parameters.
   1. Set parameters in the Basic Configuration area. For details about the parameters, see Table 1.

      Table 1 Basic configuration

      Parameter	Description	Example Value
      Tenant	Tenant to which the instance belongs. Only tenants without compute instances can be selected for new compute instances.	Select a value from the Tenant drop-down list.
      Instance Deployment Timeout Period (s)	Timeout interval for starting a compute instance deployed using the YARN service. The system tracks the duration of the compute instance startup process. If the instance remains in the Creating or Starting state after the specified timeout expires, its status changes to Error and the compute instance that is being created or started on YARN is stopped.	Default value: 300 Value range: 1 to 2,147,483,647
      Instance Count	The number of compute instances created under the current tenant.	Default value: 1

   2. Set parameters in the Coordinator Container Resource Configuration area. For details about the parameters, see Table 2.

      Table 2 Parameters for configuring coordinator container resources

      Parameter	Description	Example Value
      Container Memory (MB)	The memory size (MB) that YARN allocates to a single coordinator container of the compute instance.	Default value: 5120 Value range: 1 to 2,147,483,647
      vcore	The number of vCPUs (vCores) that YARN allocates to a single coordinator container of a compute instance.	Default value: 1 Value range: 1 to 2,147,483,647
      Quantity	The number of containers that YARN allocates to the coordinator of a compute instance.	Default value: 2 Value range: 1 to 3
      JVM	Log in to FusionInsight Manager and choose Cluster > Services > HetuEngine > Configurations. On the All Configurations tab page, search for extraJavaOptions. The value of this parameter in the coordinator.jvm.config file is the JVM value.	-

   3. Set parameters in the Worker Container Resource Configuration area. For details about the parameters, see Table 3.

      Table 3 Parameters for configuring Worker container resources

      Parameter	Description	Example Value
      Container Memory (MB)	The memory size (MB) that YARN allocates to a single worker container of a compute instance.	Default value: 10240 Value range: 1 to 2,147,483,647
      vcore	The number of vCPUs (vCores) that YARN allocates to a single worker container of a compute instance.	Default value: 1 Value range: 1 to 2,147,483,647
      Quantity	The number of containers that YARN allocates to the worker of a compute instance.	Default value: 2 Value range: 1 to 256
      JVM	Log in to FusionInsight Manager and choose Cluster > Services > HetuEngine > Configurations. On the All Configurations tab page, search for extraJavaOptions. The value of this parameter in the worker.jvm.config file is the JVM value.	-

   4. Set parameters in the Advanced Configuration area. For details about the parameters, see Table 4.

      Table 4 Advanced configuration parameters

      Parameter	Description	Example Value
      Ratio of Query Memory	The ratio of query memory to JVM heap memory on a node. The default value is 0.7. Setting this value to 0 disables the compute function. In this case, you can start the compute instance only if the -Xmx value in the JVM configuration is greater than or equal to the sum of memory.heap-headroom-per-node and query.max-memory-per-node values for the coordinator or worker.	0.7
      Scaling	If auto scaling is enabled, you can increase or decrease the number of workers without restarting the instance. However, the instance performance may deteriorate. In multi-instance mode, auto scaling cannot be enabled. For details about the parameters for enabling auto scaling, see Configuring the Number of HetuEngine Worker Nodes.	-
      Maintenance Instance	To enable automatic refresh for materialized views, you must configure a single, globally unique compute instance as the maintenance instance. Only one compute instance can serve as the maintenance instance, even if multiple compute instances exist.	-

   5. Configure Custom Configuration parameters. You can add custom parameters to a specified parameter file. Select the specified parameter file from the Parameter File drop-down list.
      • Click Add to add custom configuration parameters.
      • You can click Delete to delete custom configuration parameters.
      • You can set Parameter File to resource-groups.json to configure the resource group mechanism. Table 5 describes the resource group configuration parameter. For details about how to configure a resource group, see Configuring HetuEngine Resource Groups.

        Table 5 Resource group configuration parameter

        Parameter	Description	Example Value
        resourcegroups	Resource management group configuration of the cluster. Select resource-groups.json from the drop-down list of the parameter file.	{ "rootGroups": [{ "name": "global", "softMemoryLimit": "100%", "hardConcurrencyLimit": 1000, "maxQueued": 10000, "killPolicy": "no_kill" }], "selectors": [{ "group": "global" }] }

      

      • If a custom parameter configured in the coordinator.config.properties, worker.config.properties, log.properties, or resource-groups.json file already exists in another specified file, the custom parameter value will override the existing value. If the custom parameter does not exist in another specified file, the custom parameter is added to that file.
      • killPolicy: After a query is submitted to the worker, if the total memory usage exceeds the softMemoryLimit value, select one of the following policies to terminate running queries:
        • no_kill (default value): Do not terminate the queries.
        • recent_queries: Terminate the queries in descending order of execution sequence.
        • oldest_queries: Terminate the queries in ascending order of execution sequence.
        • finish_percentage_queries: Terminate the queries in ascending order of completion percentage. The query with the lowest completion percentage is terminated first.
        • high_memory_queries: Terminate the queries in descending order of memory usage. Queries with the highest memory usage are terminated first to release maximum memory with the fewest query terminations. If the memory usage difference between two queries is less than 10%, the query with the lower completion percentage is terminated. If the completion percentage difference between two queries is less than 5%, the query with the higher memory usage is terminated.
   6. Determine whether to start the instance immediately after the configuration is finished.
      • If yes, the system automatically starts the instance immediately after the configuration is finished.
      • If no, you must manually start the instance after the configuration is finished.

5. Click OK and wait until the instance configuration is finished.

Precautions for Maintaining Compute Instances

• During the restart or rolling restart of the HetuEngine service, do not create, start, stop, or delete HetuEngine compute instances on HSConsole.
• By default, a maximum of 10 compute instances can be in the starting, creating, deleting, stopping, scaling out, scaling in, or rolling restart state at the same time. O&M tasks that exceed this limit will wait to be executed in the background. To change the number of concurrent tasks, log in to MRS Manager, choose HetuEngine and click Configurations and then All Configurations. On the displayed page, search for hsbroker.event.task.executor.threads and change its value.
• Precautions for restarting HetuEngine compute instances
  • During the restart or rolling restart of HetuEngine compute instances, do not perform any change operations on data sources via the HetuEngine service or HetuEngine web UI, including restarting the service or changing configurations.
  • If a compute instance has only one coordinator or worker, do not perform a rolling restart of the instance.
  • If the number of workers exceeds 10, the rolling restart of instances may take more than 200 minutes. During this period, do not perform other O&M operations.
  • During the rolling restart of compute instances, HetuEngine releases YARN resources and applies for them again. Ensure that YARN CPU and memory resources are sufficient to start 20% of the workers and that these resources are not preempted by other jobs. Otherwise, the rolling restart will fail.
    • Viewing YARN resources: Log in to MRS Manager, choose Tenant Resources > Tenant Resources Management, and check the available queue resources of YARN in the Resource Quota area.
    • Viewing the CPU and memory resources of a worker container: Log in to MRS Manager as a user who can access the HetuEngine web UI and choose Cluster > Services > HetuEngine. In the Basic Information area, click the link next to HSConsole WebUI to go to the HSConsole page. Locate the target instance, click Configure in the Operation column, and view the memory size and number of vCores in the Worker Container Resource Configuration area.
  • During the rolling restart, ensure that the ApplicationMaster for the coordinator or worker in the YARN queue remains stable.
• HetuEngine compute instance restart exception handling
  • If the ApplicationMaster for the coordinator or worker in the YARN queue restarts during the rolling restart, the compute instance state may become abnormal. In this case, you must stop the compute instance and then start it to restore the service.
  • If the rolling restart of a compute instance fails, the instance enters a subhealthy state. Consequently, the configurations or count of coordinators or workers may become inconsistent. In this case, the system cannot automatically recover the subhealthy state of the compute instance. You must manually verify and rectify the fault, perform the rolling restart again, or stop and then start the compute instance.

Compute Instance Statuses

After a compute instance is created, you can view information about the created instance on the Compute Instance tab page, including the tenant name, number of instances, instance status, and total resources. Instance statuses are as follows:

Figure 1 Compute instance statuses

• Green icon: The instance is in the running or subhealthy state.
• Red icon: The instance is faulty.
• Gray icon: The instance is stopped or is ready to be started.
• Blue icon: The instance is in other states, including scaling out, scaling in, rolling restart, creating, starting, safely starting, shutting down, safely shutting down, terminating, terminated, and stopping.