FAQ About Partitioned Tables

How Many Partitions Can a TaurusDB Table Have?

A table supports up to 8,192 partitions. If subpartitions are defined, the total number of all partitions cannot exceed 8,192.

How Many Partitions Are Recommended for a Partitioned Table?

As long as the total count does not exceed 8,192, the number of partitions should be determined based on your workload scenarios and data volume.

How Do I Determine the Appropriate Number of Partitions?

• RANGE partitioning

  RANGE partitioning is typically used for time-series data. Partition granularity can be defined by day, month, quarter, or year. Determine the data retention period based on business needs, and then calculate the number of partitions using the chosen partition granularity.

• HASH partitioning

  HASH partitioning evenly distributes data across a specified number of partitions using a HASH function. You can try several partition counts when creating the table and select a value that achieves a balanced data distribution. The following statements help you evaluate partition counts.

  • Check the number of rows in each partition when data is divided by ID across N partitions:

    select id%N as part_id , count(1) from table_name group by id%N; 

  • Check the data volume for each ID:

    select id, count(1) as cnt from table_name group by id order by cnt desc;

  Key factors for determining the number of HASH partitions include:

  • Write throughput and the upper limit of writes per partition

    N_min = max(Total write TPS/Maximum TPS per partition, Total data volume/Maximum capacity per partition)

    It is recommended that writes per partition be less than or equal to 2,000 TPS.

  • Total data volume and data volume per partition

    Even with evenly distributed writes, an extremely large total data volume will result in long scan times for each partition. It is recommended that each partition contain fewer than 50 million rows.

    The calculation formula is as follows:

    Number of partitions = Expected total data volume/Recommended capacity per partition

  • Parallel query efficiency

    HASH partitioning allows all partitions to be scanned in parallel. However, excessive partitioning increases scheduling overhead, so choose your partition count carefully.