Help Center/ MapReduce Service/ Component Development Specifications/ Flink/ FlinkSQL Connector Development/ Flink SQL Redis Development Suggestions
Updated on 2025-04-15 GMT+08:00
View PDF
Share

Flink SQL Redis Development Suggestions

Setting Proper Batch Write Parameters for Sink Tables

  • sink.batch.max-size: enables batch write to Redis and sets the maximum number (a positive integer) of records to be written in batches. The value -1 indicates that batch write to Redis is disabled.
    • Enabling this function can improve the performance in big data scenarios, but it is not suitable for scenarios that have high requirements on real-time performance. It is recommended that the number of batch writes be no more than 30,000.
    • If this parameter is enabled, Checkpoint must also be enabled.
  • sink.flush-buffer.timeout: indicates after enabling batch write to Redis, data in the queue can be flushed to Redis at a specified time in millisecond.
  • Example 1 
    # Enable batch write to Redis and set the number of batch writes to 5. If batch write is enabled, Checkpoint must also be enabled.
Set sink.batch.max-size to 5.
# The maximum waiting time of data in the buffer is 1s.
Set sink.flush-buffer.timeout to 1000.
  • Example 2 
    # The value -1 indicates that batch write to Redis is disabled.
Set sink.batch.max-size to -1.
Set sink.flush-buffer.timeout to 1000.