Help Center/ MapReduce Service/ Component Operation Guide (LTS)/ Using Hudi/ Hudi Schema Evolution/ Configuring Spark for Hudi Schema Evolution
Updated on 2026-06-27 GMT+08:00
View PDF
Share

Configuring Spark for Hudi Schema Evolution

Configure Hudi in Spark to enable schema evolution, allowing you to modify the table schema dynamically without recreating the entire table.

Notes and Constraints

  • Schema evolution cannot be disabled once being enabled.
  • This section applies only to MRS 3.2.0-LTS or earlier.

Configuring Hudi Schema Evolution

  • Use Spark Beeline to configure Hudi schema evolution.

    Log in to MRS Manager, choose Cluster > Services > Spark2x, and click Configurations and then All Configurations. Search for spark.sql.extensions in the search box and change its value of JDBCServer to org.apache.spark.sql.hive.FISparkSessionExtension,org.apache.spark.sql.hudi.HoodieSparkSessionExtension,org.apache.spark.sql.hive.CarbonInternalExtensions.

  • To enable Hudi schema evolution using Spark SQL, run the following command before executing your SQL statements: 
    set hoodie.schema.evolution.enable=true
  • To enable Hudi schema evolution using the API, set the following parameter in your DataFrame configuration: 
    hoodie.schema.evolution.enable -> true
Parent topic: Hudi Schema Evolution