Enabling Dynamic Filtering
This section applies to MRS 3.2.0 or later.
Scenario
HetuEngine provides the dynamic filtering function, which significantly improves the performance in join scenarios.
This section describes how to enable dynamic filtering.
Procedure
- Log in to FusionInsight Manager as a user who can access the HetuEngine web UI and choose Cluster > Services > HetuEngine. The HetuEngine service page is displayed.
- In the Basic Information area on the Dashboard tab page, click the link next to HSConsole WebUI. The HSConsole page is displayed.
- On the Compute Instance page, locate the row containing your desired instance and click Configure in the Operation column.
- In the Custom Configuration area, click Add to add the following parameters:
Table 1 Dynamic filtering parameters Parameter
Value
ConFile
Parameter Description
hetu.seed-store.enabled
true
coordinator.config.properties and worker.config.properties
Whether to enable the seed-store function. The default value is false. To enable dynamic filtering, set this parameter to true.
hetu.embedded-state-store.enabled
true
coordinator.config.properties and worker.config.properties
Whether to enable the state-store function. The default value is false. To enable dynamic filtering, set this parameter to true.
enable-dynamic-filtering
true
coordinator.config.properties and worker.config.properties
Whether to enable the dynamic filtering function. The default value is false.
dynamic-filtering-wait-time
1s
coordinator.config.properties and worker.config.properties
Maximum waiting time for generating dynamic filter criteria. The default value is 1s.
dynamic-filtering-max-size
1000000
coordinator.config.properties and worker.config.properties
Maximum size of each dynamic filter. If the estimated size exceeds the specified value, the cost-based optimizer does not generate the dynamic filter. The default value is 1000000.
dynamic-filtering-max-per-driver-size
100M
coordinator.config.properties and worker.config.properties
Maximum amount of data that can be collected by each driver when dynamic filtering starts. The default value is 1M.
dynamic-filtering-max-per-driver-row-count
20000
coordinator.config.properties and worker.config.properties
Number of data rows stored in each driver during dynamic filtering. The default value is 20000.
- After the parameters are added, select Start Now and click OK.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot