Stream
Description
Stream is a technology used in the symmetric multiprocessing (SMP) of GaussDB. SMP uses the multi-thread parallel algorithm to execute operators in parallel, making full use of the multi-core feature of a single server to improve the execution efficiency. SMP consists of two parts: plan generation and execution.
- SMP plan generation: For one-phase plan generation, in the path generation phase, parallel paths are added, and the selected plan is determined based on the cost. For two-phase plan generation, the first step is to generate the original serial plan, and the second step is to reconstruct the serial plan into a parallel plan.
- SMP execution: Data is allocated, exchanged, and summarized between parallel execution threads.
SMP execution depends on stream operators. Different types of stream operators work together to complete the SMP execution process. Stream operators are classified into the following types:
- Local Gather: Data is allocated within a DN to aggregate data of parallel threads within the DN.
- Local Redistribute: Data is allocated within a DN to redistribute data based on the distributed key across threads within a DN
- Local Broadcast: Data is allocated within a DN to broadcast to each thread within the DN.
- Local RoundRobin: Data is allocated within a DN to distribute data in polling mode among threads in a DN.
GaussDB uses broadcast and redistribute to process data in multiple threads, and then uses gather to summarize, process, and return data.
Typical Scenarios
Set the query parallelism degree when query_dop is greater than 1.
Examples
Example: Set the query parallelism degree when query_dop is greater than 1.
-- Prepare data. gaussdb=# CREATE TABLE t1 ( id int, number int); CREATE TABLE gaussdb=# INSERT INTO t1 VALUES (generate_series(1, 500000), 1); INSERT 0 500000 gaussdb=# CREATE TABLE t2 (id int, number int); CREATE TABLE gaussdb=# INSERT INTO t2 VALUES (generate_series(250000, 750000), 1); INSERT 0 500001 gaussdb=# SET query_dop = 4; SET -- Execution result. gaussdb=# EXPLAIN SELECT * FROM t1,t2 WHERE t1.id = t2.id; QUERY PLAN ----------------------------------------------------------------------------------------------------------- Streaming(type: LOCAL GATHER dop: 1/4) (cost=14971.50..33394.63 rows=408830 width=16) -> Hash Join (cost=14971.50..32097.07 rows=408830 width=16) Hash Cond: (t1.id = t2.id) -> Streaming(type: LOCAL REDISTRIBUTE dop: 4/4) (cost=0.00..15303.62 rows=500000 width=8) -> Seq Scan on t1 (cost=0.00..4725.50 rows=500000 width=8) -> Hash (cost=13693.90..13693.90 rows=408830 width=8) -> Streaming(type: LOCAL REDISTRIBUTE dop: 4/4) (cost=0.00..13693.90 rows=408830 width=8) -> Seq Scan on t2 (cost=0.00..4497.57 rows=408830 width=8) (8 rows) gaussdb=# explain select * from t1,t2 where t1.id > 250000; QUERY PLAN ------------------------------------------------------------------------------------------------- Streaming(type: LOCAL GATHER dop: 1/4) (cost=2518.31..643988111.08 rows=102235709270 width=16) -> Nested Loop (cost=2518.31..319509541.62 rows=102235709270 width=16) -> Streaming(type: BROADCAST dop: 4/4) (cost=2518.31..18197.05 rows=1000276 width=8) -> Seq Scan on t1 (cost=2518.31..5038.00 rows=250069 width=8) Filter: (id > 250000) -> Materialize (cost=0.00..5008.61 rows=408830 width=8) -> Seq Scan on t2 (cost=0.00..4497.57 rows=408830 width=8) (7 rows) -- Drop. gaussdb=# DROP TABLE t1,t2;
In the preceding example, the output of the stream operator is as follows.
Item |
Description |
---|---|
Streaming |
Operator name. |
Type |
Type of the stream operator. Gather summarizes data of parallel threads. Redistribute redistributes data across threads based on the distribution key. Broadcast broadcasts data to each thread within a DN. |
dop |
Parallelism of stream operators. In the example, dop: 4/4 indicates that all four threads are used. |
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot