Enabling Dynamic Filtering

This section applies to MRS 3.2.0 or later.

Scenario

HetuEngine provides the dynamic filtering function, which significantly improves the performance in join scenarios.

This section describes how to enable dynamic filtering.

Procedure

Log in to FusionInsight Manager as a user who can access the HetuEngine web UI and choose Cluster > Services > HetuEngine. The HetuEngine service page is displayed.
In the Basic Information area on the Dashboard tab page, click the link next to HSConsole WebUI. The HSConsole page is displayed.
On the Compute Instance page, locate the row containing your desired instance and click Configure in the Operation column.

In the Custom Configuration area, click Add to add the following parameters:

**Table 1** Dynamic filtering parameters
Parameter	Value	ConFile	Parameter Description
hetu.seed-store.enabled	true	coordinator.config.properties and worker.config.properties	Whether to enable the seed-store function. The default value is false. To enable dynamic filtering, set this parameter to true.
hetu.embedded-state-store.enabled	true	coordinator.config.properties and worker.config.properties	Whether to enable the state-store function. The default value is false. To enable dynamic filtering, set this parameter to true.
enable-dynamic-filtering	true	coordinator.config.properties and worker.config.properties	Whether to enable the dynamic filtering function. The default value is false.
dynamic-filtering-wait-time	1s	coordinator.config.properties and worker.config.properties	Maximum waiting time for generating dynamic filter criteria. The default value is 1s.
dynamic-filtering-max-size	1000000	coordinator.config.properties and worker.config.properties	Maximum size of each dynamic filter. If the estimated size exceeds the specified value, the cost-based optimizer does not generate the dynamic filter. The default value is 1000000.
dynamic-filtering-max-per-driver-size	100M	coordinator.config.properties and worker.config.properties	Maximum amount of data that can be collected by each driver when dynamic filtering starts. The default value is 1M.
dynamic-filtering-max-per-driver-row-count	20000	coordinator.config.properties and worker.config.properties	Number of data rows stored in each driver during dynamic filtering. The default value is 20000.