Updated on 2024-03-22 GMT+08:00

Appending an Object

Functions

The AppendObject operation adds data to the end of an object in a specified bucket. If there is no namesake object in the bucket, a new object is created.

The object created using the AppendObject operation is an appendable object, and the object uploaded using the PUT operation is a normal object.

Uploaded objects must be stored in buckets. Only the users who have the write permission to a bucket can upload objects to the bucket. The name of each object in the same bucket must be unique.

To ensure that data is not damaged during transmission, you can add the Content-MD5 parameter to the request header. After receiving the data, OBS performs MD5 verification for the data. If the data is inconsistent, OBS returns an error message.

This operation allows you to specify the x-obs-acl parameter when creating an appendable object and set the permission control policy for the object.

This operation supports server-side encryption.

Relationship with Other Operations

  1. If you perform the PUT operation on an existing appendable object, the appendable object is overwritten by the newly uploaded object and the object type changes to normal. If you perform the other way around, an error occurs.
  2. An appendable object will be changed to a normal object after being copied. An appendable object cannot be copied and saved as an appendable object.

WORM

If a bucket has WORM enabled, an append operation on this bucket will fail, with a 403 error returned.

Constraints

  1. The last modification time of the object is updated each time an appending upload is performed.
  2. If the SSE-C encryption mode is used on the server side, the appending upload is the same as the initialization segment. In this case, the request headers such as x-obs-server-side-encryption must be carried.
  3. For the server-side encryption (SSE-KMS), the request header such as x-obs-server-side-encryption is specified only when the file is uploaded for the first time and no object with the same name exists in the bucket.
  4. The length of each appended upload cannot exceed the upper limit (5 GB) of the object length.
  5. The maximum number of append-only writes for each appendable object is 10,000.
  6. If the object storage class is COLD (Archive) or DEEP_ARCHIVE (Deep Archive), this API cannot be called.
  7. If cross-region replication is configured for a bucket, this API operation cannot be used.
  8. Object appending is not available for parallel file systems.

Request Syntax

POST /ObjectName?append&position=Position HTTP/1.1 
Host: bucketname.obs.region.myhuaweicloud.com 
Content-Type: application/xml 
Content-Length: length
Authorization: authorization
Date: date
<Optional Additional Header> 
<object Content>

Request Parameters

The request needs to specify parameters in the message, indicating that the request is for appending upload and the upload location must be specified. For details about the parameters, see Table 1.

Table 1 Request parameters

Parameter

Description

Mandatory

append

Indicates that the file is uploaded in appending mode.

Type: string

Yes

position

Location for the appending upload For an object to be appended, the value of position must be set to 0 when the object is uploaded for the first time. The value of position will be carried in the x-obs-next-append-position header of the response returned by the server when the object is successfully uploaded next time.

Type: integer

Yes

Request Headers

This request uses common headers. For details, see Table 3.

Table 2 describes the additional message headers that a request can use when the position=0 parameter is requested.

This request can use the server-side encryption request header. For details, see Table 3.

Table 2 Request headers

Header

Description

Mandatory

x-obs-acl

For the first appending, the message header can be added to set the permission control policy of the object. The predefined common policies are used, including: private, public-read, public-read-write. For details about each policy, see the ACL configuration using header fields in ACLs.

Type: string

Note: This header is a predefined policy expressed in a character string.

No

x-obs-grant-read

For the first write, you can use this header to grant all users in an account the permissions to read the object and obtain the object metadata.

Type: string

No

x-obs-grant-read-acp

For the first write, you can use this header to grant all users in an account the permission to obtain object ACL information.

Type: string

No

x-obs-grant-write-acp

For the first write, you can use this header to grant all users in an account the permission to write the object ACL.

Type: string

No

x-obs-grant-full-control

For the first write, you can use this header to grant all users in an account the permissions to read the object, obtain the object metadata, obtain the object ACL information, and write the object ACL.

Type: string

No

x-obs-storage-class

For the first write, you can use this header field to configure the object storage class. If you do not use this header, the object storage class is the default storage class of the bucket.

Type: string

Because Archive (COLD) and Deep Archive (DEEP_ARCHIVE) objects do not support append upload, the configurable values are as follows: STANDARD (Standard), WARM (Infrequent Access), which are case sensitive.

Example: x-obs-storage-class:STANDARD

No

x-obs-meta-*

For the first write, you can use a header starting with x-obs-meta- to define object metadata in an HTTP request. Custom metadata will be returned in the response header when you retrieve or query the metadata of the object. The size of the HTTP request excluding the request body must be equal to or smaller than 8 KB.

Type: string

Example: x-obs-meta-test:test metadata

No

x-obs-persistent-headers

For the first appending, you can add the x-obs-persistent-headers header in an HTTP request to specify one or more user-defined response headers. User-defined response headers will be returned in the response header when you retrieve the object or query the object metadata.

Type: string

Format: x-obs-persistent-headers: key1:base64_encode(value1),key2:base64_encode(value2)...

Note: Items, such as key1 and key2, are user-defined headers. If they contain non-ASCII or unrecognizable characters, they can be encoded using URL or Base64. The server processes these headers as character strings, but does not decode them. Items, such as value1 and value2 are the values of the corresponding headers. base64_encode indicates that the value is encoded using Base64. A user-defined header and its Base64-encoded value are connected using a colon (:) to form a key-value pair. All key-value pairs are separated with a comma (,) and are placed in the x-obs-persistent-headers header. The server then decodes the uploaded value.

Example: x-obs-persistent-headers: key1:dmFsdWUx,key2:dmFsdWUy

The returned header for downloading the object or obtaining the object metadata is key1:value1 or key2:value2 respectively.

Constraints:

  1. Response headers customized in this way cannot be prefixed with x-obs-. For example, you should use key1, instead of x-obs-key1.
  2. Standard HTTP headers, such as host, content-md5, origin, range, and Content-Disposition, cannot be specified as custom headers.
  3. The total length of this header and the custom metadata cannot exceed 8 KB.
  4. If the same keys are passed, values are separated with commas (,) and then returned in one key.
  5. If the decoded value contains non-US-ASCII or unrecognizable characters, the server processes the value as a string and encapsulates it using ?UTF-8?B?<(str)>?=, but does not decode the value. For instance, value key1:abbc will be returned as key1: =?UTF-8?B?abbc?=.
  6. The values cannot contain spaces, equal signs (=), commas (,), semicolons (;), colons (:), or periods (.). If such characters are required, use URL or Base64 encoding.

No

x-obs-website-redirect-location

If a bucket is configured with the static website hosting function, it will redirect requests for this object to another object in the same bucket or to an external URL. OBS stores the value of this header in the object metadata.

Type: string

Default value: none

Constraint: The value must be prefixed by a slash (/), http://, or https://. The length of the value cannot exceed 2 KB.

No

x-obs-expires

Specifies when an object expires. It is measured in days. Once the object expires, it is automatically deleted. (The calculation starts from when the object was last modified).

Type: integer

Example: x-obs-expires:3

No

Table 3 Server encryption request headers

Header

Description

Mandatory

x-obs-server-side-encryption

Indicates that SSE-KMS is used.

Type: string

Example: x-obs-server-side-encryption:kms

No. This header is required when SSE-KMS is used.

x-obs-server-side-encryption-kms-key-id

Indicates the master key when SSE-KMS is used. If this header is not provided, the default master key will be used. If there is no such a default master key, OBS will create one and use it by default.

Type: string

The following two formats are supported:

- regionID:domainID:key/key_id

- key_id

regionID indicates the ID of the region where the key belongs. domainID indicates the ID of the tenant where the key belongs. key_id indicates the ID of the key created in DEW.

Examples:

- x-obs-server-side-encryption-kms-key-id:region:domainiddomainiddomainiddoma0001:key/4f1cd4de-ab64-4807-920a-47fc42e7f0d0

- x-obs-server-side-encryption-kms-key-id:4f1cd4de-ab64-4807-920a-47fc42e7f0d0

No

x-obs-server-side-encryption-customer-algorithm

Indicates the encryption algorithm when SSE-C is used.

Type: string

Example: x-obs-server-side-encryption-customer-algorithm:AES256

Constraint: This header must be used together with x-obs-server-side-encryption-customer-key and x-obs-server-side-encryption-customer-key-MD5.

No. This header is required when SSE-C is used.

x-obs-server-side-encryption-customer-key

Indicates the key for encrypting objects when SSE-C is used.

Type: string

Example: x-obs-server-side-encryption-customer-key:K7QkYpBkM5+hca27fsNkUnNVaobncnLht/rCB2o/9Cw=

Constraint: This header is a Base64-encoded 256-bit key and must be used together with x-obs-server-side-encryption-customer-algorithm and x-obs-server-side-encryption-customer-key-MD5.

No. This header is required when SSE-C is used.

x-obs-server-side-encryption-customer-key-MD5

Indicates the MD5 value of the encryption key when SSE-C is used. The MD5 value is used to check whether any error occurs during the transmission of the key.

Type: string

Example: x-obs-server-side-encryption-customer-key-MD5:4XvB3tbNTN+tIEVa0/fGaQ==

Constraint: This header is a Base64-encoded 128-bit MD5 value and must be used together with x-obs-server-side-encryption-customer-algorithm and x-obs-server-side-encryption-customer-key.

No. This header is required when SSE-C is used.

Request Elements

This request involves no elements.

Response Syntax

1
2
3
4
HTTP/1.1 status_code
Date: date
ETag: etag
Content-Length: length

Response Headers

The response to the request uses common headers. For details, see Table 1.

The ETag returns the hash value of the data to be uploaded, not the hash value of the entire object.

Table 4 Additional response headers

Header

Description

x-obs-version-id

Object version ID. If versioning is enabled for the bucket, the object version ID will be returned.

Type: string

x-obs-server-side-encryption

This header is included in a response if SSE-KMS is used.

Type: string

Example: x-obs-server-side-encryption:kms

x-obs-server-side-encryption-kms-key-id

Indicates the master key ID. This header is included in a response when SSE-KMS is used.

Type: string

Format: regionID:domainID:key/key_id

regionID indicates the ID of the region where the key belongs. domainID indicates the ID of the tenant where the key belongs. key_id indicates the key ID used in this encryption.

Example: x-obs-server-side-encryption-kms-key-id:region:domainiddomainiddomainiddoma0001:key/4f1cd4de-ab64-4807-920a-47fc42e7f0d0

x-obs-server-side-encryption-customer-algorithm

Indicates the encryption algorithm. This header is included in a response when SSE-C is used.

Type: string

Example: x-obs-server-side-encryption-customer-algorithm:AES256

x-obs-server-side-encryption-customer-key-MD5

Indicates the MD5 value of the key for encrypting objects. This header is included in a response when SSE-C is used.

Type: string

Example: x-obs-server-side-encryption-customer-key-MD5:4XvB3tbNTN+tIEVa0/fGaQ==

x-obs-next-append-position

Indicates the position to be provided for the next request.

Type: integer

Response Elements

This response contains no elements.

Error Responses

  1. If the object length exceeds the limit due to the appending upload, OBS returns 400 Bad Request and the error code is AppendTooLarge.
  2. If the value of position is different from the original length of the current object, OBS returns 409 Conflict and the error code is PositionNotEqualToLength.
  3. If an object with the same object name exists in a bucket and the object type is not Appendable, OBS returns 409 Conflict and the error code is ObjectNotAppendable.
  4. If the number of write times of an object exceeds 10000, OBS returns 409 Conflict and the error code is ObjectNotAppendable.
  5. If the object storage class is COLD (Archive) or DEEP_ARCHIVE (Deep Archive), this API cannot be called. If you still call this API, OBS returns 409 Conflict with the error code of ObjectNotAppendable.
  6. If cross-region replication is configured for a bucket, this API operation cannot be used. Otherwise, OBS returns 400 Bad Request and the error code is OperationNotSupported.

Other errors are included in Table 2.

Sample Request: Append Upload

POST /object?append&position=0 HTTP/1.1
Host: examplebucket.obs.region.myhuaweicloud.com
Expires: Wed, 27 Jun 2015 13:45:50 GMT
Date: Wed, 08 Jul 2015 06:57:01 GMT
Content-Type: image/jpg
Content-Length: 1458
Authorization: OBS H4IPJX0TQTHTHEBQQCEC:kZoYNv66bsmc10+dcGKw5x2PRrk=

[1458 bytes of object data]

Sample Response: Append Upload

1
2
3
4
5
6
7
8
HTTP/1.1 200 OK
Date: Wed, 27 Jun 2015 13:45:50 GMT
ETag: "d41d8cd98f00b204e9800998ecf8427e"
Content-Length: 0  
Server: OBS
x-obs-request-id: 8DF400000163D3F0FD2A03D2D30B0542
x-obs-id-2: 32AAAUgAIAABAAAQAAEAABAAAQAAEAABCTjCqTmsA1XRpIrmrJdvcEWvZyjbztdd 
x-obs-next-append-position: 1458

Sample Request: Append Upload (with redirect and a User-Defined Header Used)

The bucket examplebucket exists but the object obj001 does not exist. Create an object by making the API call for the append operation. Set the redirection header field as follows: "x-obs-website-redirect-location":"http://www.example.com/", and set the user-defined header field to: "x-obs-meta-redirect":"redirect". The request is as follows:

POST /obj001?append&position=0 HTTP/1.1
Host: examplebucket.obs.region.myhuaweicloud.com
Expires: Wed, 27 Jun 2015 13:45:50 GMT
Date: Wed, 08 Jul 2015 06:57:01 GMT
x-obs-website-redirect-location: http://www.example.com/
x-obs-meta-redirect: redirect
Content-Length: 6
Authorization: OBS H4IPJX0TQTHTHEBQQCEC:kZoYNv66bsmc10+dcGKw5x2PRrk=

[6 bytes of object data]

Sample Response: Append Upload (with redirect and a User-Defined Header Used)

1
2
3
4
5
6
7
8
HTTP/1.1 200 OK
Date: Wed, 27 Jun 2015 13:45:50 GMT
ETag: "9516dfb15f51c7ee19a4d46b8c0dbe1d"
Content-Length: 0  
Server: OBS
x-obs-request-id: 5DEB00000164A3150AC36F8F0C120D50
x-obs-id-2: 32AAAUgAIAABAAAQAAEAABAAAQAAEAABCSrVlTYwsA4p9GEW+LYqotSl5BYDxHfT 
x-obs-next-append-position: 6

Sample Request: Appending Data to an Object in a Versioning-enabled Bucket

POST /object01?append&position=0 HTTP/1.1
PUT /?lifecycle HTTP/1.1
Authorization: OBS H4IPJX0TQTHTHEBQQCEC:iqSPeUBl66PwXDApxjRKk6hlcN4=
User-Agent: curl/7.29.0
Host: examplebucket.obs.region.myhuaweicloud.com
Date: WED, 01 Jul 2015 02:37:22 GMT
Content-Type: application/octet-stream
 
[1458 bytes of object data]

Sample Response: Appending Data to an Object in a Versioning-enabled Bucket

x-obs-id-2: 32AAAQAAEAABSAAgAAEAABAAAQAAEAABCSZbDadL1f7fYU44bvRLvc0l6D10+wzG
x-obs-request-id: 0000018A2BCBB3ABD3046B99E3ED2E30
Server: OBS
Content-Length: 0
Date: WED, 01 Jul 2015 02:37:22 GMT
x-obs-next-append-position: 4
ETag: "56468d5607a5aaf1604ff5e15593b003"
x-obs-version-id: G001118A6803675AFFFFD3043F7F91D0