Hudi Read Overview

Hudi provides three logical views for read operations: real-time view, incremental view, and read-optimized view. You can select the appropriate query view based on your specific requirements.

• Real-time view: Provides the most current data. It is ideal for use cases requiring immediate access to the latest data. It combines base files and incremental logs to deliver current data records, ensuring users access the most recent updates.
• Incremental view: Displays data changes since the last query, including inserts, updates, and deletes.
• Read-optimized view: Uses optimized columnar formats, such as Parquet or ORC, to accelerate data read performance. This view is ideal for batch processing and analytics requiring high-throughput access to large datasets. Read-optimized views minimize I/O overhead and enhance data compression to boost query performance, making them ideal for big data analytics and report generation.

By providing different views, Hudi addresses various query patterns, enhancing data processing efficiency and accuracy.

Hudi supports multiple query engines, including Spark, Hive, and HetuEngine. For details, see Table 1 and Table 2.

• The Spark DataSource API does not currently support automatic partition inference when reading Hudi tables. For example, when you use the DataSource API to query a bootstrap table, partition fields may appear as null or may not be returned.
• For an incremental view, set hoodie.hudicow.consume.mode to INCREMENTAL. This parameter applies only to queries on the incremental view and cannot be used for queries on other types of Hudi tables or queries on other tables. You can set hoodie.hudicow.consume.mode to SNAPSHOT or any value to restore the configuration.