Updated on 2025-04-15 GMT+08:00

Flink SQL Redis Development Suggestions

Setting Proper Batch Write Parameters for Sink Tables

  • sink.batch.max-size: enables batch write to Redis and sets the maximum number (a positive integer) of records to be written in batches. The value -1 indicates that batch write to Redis is disabled.
    • Enabling this function can improve the performance in big data scenarios, but it is not suitable for scenarios that have high requirements on real-time performance. It is recommended that the number of batch writes be no more than 30,000.
    • If this parameter is enabled, Checkpoint must also be enabled.
  • sink.flush-buffer.timeout: indicates after enabling batch write to Redis, data in the queue can be flushed to Redis at a specified time in millisecond.
  • Example 1
    # Enable batch write to Redis and set the number of batch writes to 5. If batch write is enabled, Checkpoint must also be enabled.
    Set sink.batch.max-size to 5.
    # The maximum waiting time of data in the buffer is 1s.
    Set sink.flush-buffer.timeout to 1000.
  • Example 2
    # The value -1 indicates that batch write to Redis is disabled.
    Set sink.batch.max-size to -1.
    Set sink.flush-buffer.timeout to 1000.