DLI Delta Table Overview

Delta tables use the Delta Lake technology to offer a robust data storage solution. By extending Parquet data files with file-based transaction logs, they support ACID transactions and scalable metadata. Delta Lake seamlessly integrates with Apache Spark APIs and is designed to work closely with structured streaming, allowing for the use of a single data backup for both batch and streaming operations, and enabling large-scale incremental processing.

Constraints on Delta in DLI

Only Spark 3.3.1 (3.0.0) or later supports Delta.
Only Delta 2.3.0 is supported by DLI.

Certain SQL statements in Spark 3.3.1 (3.0.0) do not support the open-source syntax related to Delta tables. For details, see Table 1.

**Table 1** Open source syntax related to Delta tables not supported by Spark 3.3.1-3.0.0 SQL
Unsupported Statement	Example
ALTER TABLE REPLACE COLUMNS: replaces columns	alter table table0 replace columns(id1 int,name1 string);
SHOW CREATE TABLE: shows table creation statements	show create table table1;
INSERT INTO/OVERWRITE: inserts data into a table in a specified static partition	insert into table1 partition(part='part1') select * from table2;
ALTER TABLE ADD/DROP PARTITION: manages partitions	alter table test_delta_parts1 add partition('2024-10-28');
CONVERT TO DELTA: does not support tables in parquet.'tablePath' format	convert to delta parquet.`obs://bucket0/db0/table0`;