Compute
Elastic Cloud Server
Huawei Cloud Flexus
Bare Metal Server
Auto Scaling
Image Management Service
Dedicated Host
FunctionGraph
Cloud Phone Host
Huawei Cloud EulerOS
Networking
Virtual Private Cloud
Elastic IP
Elastic Load Balance
NAT Gateway
Direct Connect
Virtual Private Network
VPC Endpoint
Cloud Connect
Enterprise Router
Enterprise Switch
Global Accelerator
Management & Governance
Cloud Eye
Identity and Access Management
Cloud Trace Service
Resource Formation Service
Tag Management Service
Log Tank Service
Config
OneAccess
Resource Access Manager
Simple Message Notification
Application Performance Management
Application Operations Management
Organizations
Optimization Advisor
IAM Identity Center
Cloud Operations Center
Resource Governance Center
Migration
Server Migration Service
Object Storage Migration Service
Cloud Data Migration
Migration Center
Cloud Ecosystem
KooGallery
Partner Center
User Support
My Account
Billing Center
Cost Center
Resource Center
Enterprise Management
Service Tickets
HUAWEI CLOUD (International) FAQs
ICP Filing
Support Plans
My Credentials
Customer Operation Capabilities
Partner Support Plans
Professional Services
Analytics
MapReduce Service
Data Lake Insight
CloudTable Service
Cloud Search Service
Data Lake Visualization
Data Ingestion Service
GaussDB(DWS)
DataArts Studio
Data Lake Factory
DataArts Lake Formation
IoT
IoT Device Access
Others
Product Pricing Details
System Permissions
Console Quick Start
Common FAQs
Instructions for Associating with a HUAWEI CLOUD Partner
Message Center
Security & Compliance
Security Technologies and Applications
Web Application Firewall
Host Security Service
Cloud Firewall
SecMaster
Anti-DDoS Service
Data Encryption Workshop
Database Security Service
Cloud Bastion Host
Data Security Center
Cloud Certificate Manager
Edge Security
Managed Threat Detection
Blockchain
Blockchain Service
Web3 Node Engine Service
Media Services
Media Processing Center
Video On Demand
Live
SparkRTC
MetaStudio
Storage
Object Storage Service
Elastic Volume Service
Cloud Backup and Recovery
Storage Disaster Recovery Service
Scalable File Service Turbo
Scalable File Service
Volume Backup Service
Cloud Server Backup Service
Data Express Service
Dedicated Distributed Storage Service
Containers
Cloud Container Engine
SoftWare Repository for Container
Application Service Mesh
Ubiquitous Cloud Native Service
Cloud Container Instance
Databases
Relational Database Service
Document Database Service
Data Admin Service
Data Replication Service
GeminiDB
GaussDB
Distributed Database Middleware
Database and Application Migration UGO
TaurusDB
Middleware
Distributed Cache Service
API Gateway
Distributed Message Service for Kafka
Distributed Message Service for RabbitMQ
Distributed Message Service for RocketMQ
Cloud Service Engine
Multi-Site High Availability Service
EventGrid
Dedicated Cloud
Dedicated Computing Cluster
Business Applications
Workspace
ROMA Connect
Message & SMS
Domain Name Service
Edge Data Center Management
Meeting
AI
Face Recognition Service
Graph Engine Service
Content Moderation
Image Recognition
Optical Character Recognition
ModelArts
ImageSearch
Conversational Bot Service
Speech Interaction Service
Huawei HiLens
Video Intelligent Analysis Service
Developer Tools
SDK Developer Guide
API Request Signing Guide
Terraform
Koo Command Line Interface
Content Delivery & Edge Computing
Content Delivery Network
Intelligent EdgeFabric
CloudPond
Intelligent EdgeCloud
Solutions
SAP Cloud
High Performance Computing
Developer Services
ServiceStage
CodeArts
CodeArts PerfTest
CodeArts Req
CodeArts Pipeline
CodeArts Build
CodeArts Deploy
CodeArts Artifact
CodeArts TestPlan
CodeArts Check
CodeArts Repo
Cloud Application Engine
MacroVerse aPaaS
KooMessage
KooPhone
KooDrive
On this page

Show all

Log Transfer Constraints

Updated on 2025-01-17 GMT+08:00

This section describes the constraints on log transfer.

Table 1 Log transfer constraints

Category

Item

Description

Remarks

Log transfer to OBS

Transfer tasks for a log stream

A log stream can have only one task for transferring logs to OBS.

N/A

Log transfer interval

2 minutes, 5 minutes, 30 minutes, 1 hour, 3 hours, 6 hours, 12 hours

N/A

Data size of each log transfer task

0 MB to 2 GB

N/A

Transfer rate threshold

100 MB/s The transfer may fail if this limit is exceeded.

N/A

Log transfer delay

10 minutes

For example, if the transfer interval is 30 minutes and the transfer starts at 8:30, transfer files will be generated at 8:40 at the latest.

N/A

Destination bucket

Standard buckets are supported. Parallel file systems are not supported.

N/A

Log transfer to DIS

Transfer tasks for a log stream

A log stream can have only one task for transferring logs to DIS.

N/A

Log transfer interval

Real time

N/A

Data size of each log transfer task

N/A

N/A

Log transfer delay

N/A

N/A

Transfer rate threshold

Same as the maximum write rate of the relevant DIS stream.

The transferred data will be unstable if this limit is exceeded.

N/A

Log transfer to DMS

Transfer tasks for a log stream

A log stream can have only one task for transferring logs to DMS.

N/A

Log transfer interval

Real time

N/A

Data size of each log transfer task

N/A

N/A

Log transfer delay

N/A

N/A

Transfer rate threshold

Same as the upper traffic limit of the relevant DMS (Kafka) cluster.

The transferred data will be unstable if this limit is exceeded.

N/A

Log transfer to GaussDB(DWS)

Transfer tasks for a log stream

A log stream can have only one task for transferring logs to GaussDB(DWS).

N/A

Log transfer interval

1 minute

N/A

Data size of each log transfer task

< 5 MB

N/A

Log transfer delay

5 minutes

For example, if the transfer starts at 8:30, transfer files will be generated at 8:35 at the latest.

N/A

Transfer rate threshold

40 MB/s

The transferred data will be unstable if this limit is exceeded.

N/A

Data reliability

If the format of a batch of data is valid, the data is transferred at least once. However, if the GaussDB(DWS) cluster is heavily loaded or a network error occurs, the write response will time out, which may cause duplicate data. In this case, data accuracy (Exactly Once) cannot be ensured.

N/A

Table structure change

  • Adding non-mandatory columns to DWS tables does not affect log transfer.
  • Adding mandatory columns to DWS tables during log delivery will cause a data write failure.
  • Deleting columns that contain transfer rules from DWS tables during log delivery will cause a data write failure.

N/A

Invalid data columns

The common scenarios include mismatch and type conversion failure. This batch of data will not be written to GaussDB(DWS), while other batches will be written normally.

N/A

Oversized data columns

The common scenarios include long string and varchar type data. This batch of data will not be written to GaussDB(DWS), while other batches will be written normally.

N/A

We use cookies to improve our site and your experience. By continuing to browse our site you accept our cookie policy. Find out more

Feedback

Feedback

Feedback

0/500

Selected Content

Submit selected content with the feedback