Using Resource Labels to Specify on Which Node Coordinators Should Run
By default, coordinator and worker nodes randomly start on Yarn NodeManager nodes, and you have to open all ports on all NodeManager nodes. Using resource labels of Yarn, HetuEngine allows you to specify NodeManager nodes to run coordinators.
Prerequisites
You have created a user for accessing the HetuEngine web UI. For details, see Creating a HetuEngine User.
Procedure
- Log in to FusionInsight Manager as a user who can access the HetuEngine web UI.
- Set Yarn parameters to specify the scheduler to handle PlacementConstraints.
- Choose Cluster > Services > Yarn. Click the Configurations tab and then All Configurations. On the displayed page, search for yarn.resourcemanager.placement-constraints.handler, set Value to scheduler, and click Save.
- Click the Instance tab, select the active and standby ResourceManager instances, click More, and select Restart Instance to restart the ResourceManager instances of Yarn. Then wait until they are restarted successfully.
- Configure resource labels.
- Choose Tenant Resources > Resource Pool. On the displayed page, click Add Resource Pool.
- Select a cluster, and enter a resource pool name and a resource label name, for example, pool1. Select the desired hosts, click to add the selected hosts to the new resource pool, and click OK.
- Set HetuEngine parameters to enable the coordinator placement policy and enter the node resource label.
- Choose Cluster > Service > HetuEngine. Click the Configurations tab and then All Configurations. On the displayed page, set parameters and click Save.
Table 1 Setting HetuEngine parameters Parameter
Setting
yarn.hetuserver.engine.coordinator.placement.enabled
true
yarn.hetuserver.engine.coordinator.placement.label
Node resource label created in 3, for example, pool1
- Click Dashboard, click More, and select Service Rolling Restart. Wait until the HetuEngine service is restarted successfully.
- Choose Cluster > Service > HetuEngine. Click the Configurations tab and then All Configurations. On the displayed page, set parameters and click Save.
- Restart the HetuEngine compute instance.
- In the Basic Information area on the Dashboard page, click the link next to HSConsole WebUI. The HSConsole page is displayed.
- Locate the row that contains the target instance and click Start in the Operation column.
- Check the node on which the coordinator is running.
- Return to FusionInsight Manager.
- Choose Cluster > Services > Yarn. In the Basic Information area on the Dashboard page, click the link next to ResourceManager WebUI.
- In the navigation pane on the left, choose Cluster > Nodes. You can view that the coordinator has been started on the node in the resource pool created in 3.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot