Configuring LakeFormation Connection Properties

When using DataArts Migration, you can configure additional properties of the connection to the LakeFormation data source to optimize the data migration process and resolve possible problems.

The following table lists the configurable connection properties.

Optional Connection Properties

Property	Type	Description
orc.writer.compress	string	Compression format for the ORC files written to LakeFormation The following formats are supported: NONE, ZLIB, SNAPPY, LZO, and LZ4.
orc.writer.batch.size	int	Number of rows to be committed in batches during the writing of ORC files to LakeFormation The default value is 10,000.
parquet.writer.pageSize	int	Page size of a Parquet file written to LakeFormation The default value is 1,048,576 bytes (1,024 x 1,024 = 1 MB).
parquet.writer.dictionaryPageSize	int	Dictionary page size of a Parquet file written to LakeFormation The default value is 1,048,576 bytes (1,024 x 1,024 = 1 MB).
parquet.writer.blockSize	int	Row group size of a Parquet file written to LakeFormation The default value is 134,217,728 bytes (128 x 1,024 x 1,024 = 128 MB).
parquet.compression	string	Compression format for the Parquet files written to LakeFormation The following formats are supported: UNCOMPRESSED, SNAPPY, GZIP, LZO, BROTLI, LZ4, and ZSTD.
writer.target.file.size	int	Size (in bytes) of a file written to LakeFormation The default value is 536,870,912 bytes (512 x 1,024 x 1,024 = 512 MB).
partition.writer.limit	int	Maximum number of writers that can be used in a single task to write data to specified dynamic partitions. The default value is 10.