Help Center/ MapReduce Service/ Component Development Specifications/ Hudi/ Spark on Hudi Stream Scenario Operation Guide/ Determining the Write Operation Type
Updated on 2026-06-26 GMT+08:00
View PDF
Share

Determining the Write Operation Type

Write operation type description

Write Operation Type

Supported Table Types

Feature

How to Use

Preferred Scenario

insert into

cow/mor

The data written using INSERT INTO will overwrite the inventory data with the same primary key in the table.
  • insert into SinkTable values()
  • insert into SinkTable select * from SourceTable

Writing the result of multi-source table association query to the Hudi table

insert overwrite

cow/mor
  • For a partitioned table, only the partition to which the data is written is overwritten.
  • For a non-partitioned table, the entire table is overwritten.
  • insert overwrite SinkTable values()
  • insert overwrite SinkTable select * from SourceTable

COW table, partition overwriting

insert overwrite table

cow/mor

The entire table or partitions can be overwritten.
  • insert overwrite table SinkTable values()
  • insert overwrite table SinkTable select * from SourceTable
  • insert overwrite table SinkTable partition (partition field=partition value) values ()
  • insert overwrite table SinkTable partition (partition field=partition value) select * from SourceTable

Overwriting the entire table

bulk_insert

cow/mor
  • Set the bulk_insert mode for INSERT INTO. Hudi tables can be quickly initialized in this mode.
  • bulk_insert can be used only once during inventory migration and does not perform deduplication.

For details, see Common Parameters and Application Scenarios.

Hudi table initialization

update

cow/mor

Updating columns in batches
  • update SinkTable set Field name=Value where Field name=Value

Update the values of a column in batches.

merge into

cow/mor

Perform update, insert, and delete using one statement.
  • merge into SinkTable as t1 using (subquery) as t2 on t1. field = t2. field when matched then

You are not advised to use the MergeInto capability of the Hudi community, as it may lead to data consistency issues. Identified issue scenarios include performing Delete before MergeInto, and there may be other undiscovered scenarios with potential risks. It is advised to use InsertInto/InsertOverwrite/Update/Delete instead.

delete

cow/mor

Deleting a small amount of data
  • delete from SinkTable where field name=value
  • delete from SinkTable where field name in (subquery)

Delete a small amount of data, for example, less than 30% of the data in a table.

Precautions

  • The delete command is not used to delete a partition. To delete a partition, run the drop partition command. For details, see Introduction to Common SQL Statements.
  • For bucket indexes, the error message Find multiple files at partition xxx belongs to the same bucket id = xxx may be displayed when a duplicate bucket ID exists.

    Common error scenarios:

    1. The INSERT INTO job conflicts with the TRUNCATE job. The TRUNCATE job is re-initializing the Hudi table, but the INSERT INTO job is writing files. As a result, the file added in the last INSERT INTO job remains in the re-initialized Hudi table.
    2. The insert overwrite job is switched to the insert into job. When the insert overwrite job writes data to Hudi, the clean and archive configurations must be correctly used according to Selecting a Proper Table Service Execution Mode. Otherwise, the replacecommit metadata of the insert overwrite job has been archived, but the data file still exists.