Updated on 2025-12-22 GMT+08:00

Log Transfer Constraints

This section describes the constraints on log transfer.

**Table 1** Log transfer constraints
Category	Item	Description	Remarks
Log transfer to OBS	Transfer tasks for a log stream	A log stream can have only one task for transferring logs to OBS.	N/A
	Log transfer interval	2 minutes, 5 minutes, 30 minutes, 1 hour, 3 hours, 6 hours, 12 hours	N/A
	Data size of each log transfer task	0 MB to 2 GB	N/A
	Transfer rate threshold	100 MB/s. The transfer may fail if this limit is exceeded.	N/A
	Log transfer delay	10 minutes For example, if the transfer interval is 30 minutes and the transfer starts at 8:30, transfer files will be generated at 8:40 at the latest.	N/A
	Destination bucket	Standard buckets are supported. Parallel file systems are not supported.	N/A
Log transfer to DIS	Transfer tasks for a log stream	A log stream can have only one task for transferring logs to DIS.	N/A
	Log transfer interval	Real time	N/A
	Data size of each log transfer task	N/A	N/A
	Log transfer delay	N/A	N/A
	Transfer rate threshold	Same as the maximum write rate of the relevant DIS stream. The transferred data will be unstable if this limit is exceeded.	N/A
Log transfer to DMS	Transfer tasks for a log stream	A log stream can have only one task for transferring logs to DMS.	N/A
	Log transfer interval	Real time	N/A
	Data size of each log transfer task	N/A	N/A
	Log transfer delay	N/A	N/A
	Transfer rate threshold	Same as the upper traffic limit of the relevant DMS (Kafka) cluster. The transferred data will be unstable if this limit is exceeded.	N/A
Log transfer to DWS	Transfer tasks for a log stream	A log stream can have only one task for transferring logs to DWS.	N/A
	Log transfer interval	1 minute	N/A
	Data size of each log transfer task	< 5 MB	N/A
	Log transfer delay	5 minutes For example, if the transfer starts at 8:30, transfer files will be generated at 8:35 at the latest.	N/A
	Transfer rate threshold	40 MB/s The transferred data will be unstable if this limit is exceeded.	N/A
	Data reliability	If the format of a batch of data is valid, transfer is initiated at least once (guaranteeing At Least Once delivery). However, heavy DWS cluster load or network failures can cause a write response timeout, leading to data re-sends. This results in data duplication and compromises Exactly Once accuracy.	N/A
	Table structure change	Adding non-mandatory columns to DWS tables does not affect log transfer. Adding mandatory columns to DWS tables during log delivery will cause a data write failure. Deleting columns that contain transfer rules from DWS tables during log delivery will cause a data write failure.	N/A
	Invalid data columns	The common scenarios include mismatch and type conversion failure. This batch of data will not be written to DWS, while other batches will be written normally.	N/A
	Oversized data columns	The common scenarios include long string and varchar type data. This batch of data will not be written to DWS, while other batches will be written normally.	N/A
Log transfer to DLI	Transfer tasks for a log stream	A log stream can have only one task for transferring logs to DLI.	N/A
	Log transfer interval	5 minutes	N/A
	Data size of each log transfer task	0 MB to 2 GB	N/A
	Transfer rate threshold	100 MB/s. The transfer may fail if this limit is exceeded.	N/A
	Log transfer delay	10 minutes For example, if the transfer interval is 30 minutes and the transfer starts at 8:30, transfer files will be generated at 8:40 at the latest.	N/A

Parent topic: Notes and Constraints

Log Transfer Constraints

What is your overall rating for this page?