Appendable Upload

Scenarios

After a streaming upload, browser-based upload, or multipart upload is complete, the uploaded object can be read but cannot be modified. If the object's content changes, you need to upload a new object with the same name to overwrite the original one.

There will be real-time streams when you upload logs, video surveillance data, live streaming data, and others. Using any of the foregoing uploads may cause problems such as complex software architecture, high server loads from frequent client requests, and high data latency. To solve such problems, you can use the appendable upload, which allows you to append data to the end of an object in a specific bucket. If there is no object with the same key value in the bucket, a new object will be created.

With appendable upload, newly generated video data can be appended to the same object in a timely manner. A client only needs to periodically obtain the length of the object and compare it with the length read last time. If there is new data that can be read, the client will read the newly uploaded data. In this way, the architecture can be simplified and its scalability can be enhanced.

Limits

The last modification time of the object is updated each time an appendable upload is performed.
If SSE-C encryption is used on the server side, you need to specify the request headers such as x-obs-server-side-encryption when you perform any appendable upload, which is the same as initiating multipart upload.
For the server-side encryption (SSE-KMS), you only need to specify the request header such as x-obs-server-side-encryption when the file is uploaded for the first time and there is no object with the same name in the bucket.
The length of each appendable upload cannot exceed the upper limit (5 GB) of the object length.
Each appendable object can be appended for up to 10,000 times.
If the object storage class is COLD (Archive) or DEEP_ARCHIVE (Deep Archive), appendable upload API cannot be called.
If cross-region replication is configured for a bucket, appendable upload API cannot be called.
Appendable upload is not available for parallel file systems.
If you perform the PUT operation on an existing appendable object, the appendable object is overwritten by the newly uploaded object and the object type changes to normal. If you perform the PUT operation on a normal object, an error occurs.
An appendable object will be changed to a normal object after being copied. An appendable object cannot be copied and saved as an appendable object.

Important Notes

If the file you want to append data to does not exist, calling the AppendObject API will create an appendable file.
If a bucket has WORM enabled, an append operation on this bucket will fail, with a 403 error returned.
The ETag returns the hash value of the data to be uploaded, not the hash value of the entire object.
Suppose there is a file you want to append data to:
- If the file is appendable and the position where data will be appended matches the file length, the data can be directly appended to the end of the file. If the position does not match the file length, the PositionNotEqualToLength exception will be returned.
- If the file is not appendable, for example, the file type is Normal, the ObjectNotAppendable exception will be returned.