Help Center/ Object Storage Service/ SDK Reference/ Go/ Objects (SDK for Go)/ Object Upload Overview (SDK for Go)/ Uploading an Object - Append (SDK for Go)
Updated on 2024-12-09 GMT+08:00
View PDF
Share

Uploading an Object - Append (SDK for Go)

Function

This API uploads a file or folder to an existing OBS bucket. You can upload texts, videos, or any other types of files.

The AppendObject operation adds data to the end of an object in a specified bucket. If there is no namesake object in the bucket, a new object is created.

Restrictions

  • To upload an object, you must be the bucket owner or have the required permission (obs:object:PutObject in IAM or PutObject in a bucket policy). For details, see Introduction to OBS Access Control, IAM Custom Policies, and Configuring an Object Policy.
  • The mapping between OBS regions and endpoints must comply with what is listed in Regions and Endpoints.
  • Uploaded objects are stored in buckets. Only the users who have the write permission can upload objects to buckets. The name of each object in a bucket must be unique.
  • The last modification time of the object is updated each time an append upload is performed.
  • If the server-side encryption (SSE-C) is used, the append upload is the same as the multipart initialization. The request header such as x-obs-server-side-encryption should be carried for each append upload.
  • If the KMS server-side encryption (SSE-KMS) is used, the request header such as x-obs-server-side-encryption should be carried only when the object is uploaded for the first time and no object with the same name exists in the bucket.
  • The length of each append upload cannot exceed the upper limit (5 GB) of the object length.
  • The maximum number of append writes for each appendable object is 10,000.
  • If the storage class is COLD (Archive), this API cannot be called.
  • If cross-region replication is configured for a bucket, this API cannot be called.
  • Object appending is not available for parallel file systems.

Method

func (obsClient ObsClient) AppendObject(input *AppendObjectInput) (output *AppendObjectOutput, err error)

Request Parameters

Table 1 List of request parameters

Parameter

Type

Mandatory (Yes/No)

Description

input

*AppendObjectInput

Yes

Explanation:

Request parameters for an append upload. For details, see Table 2.
Table 2 AppendObjectInput

Parameter

Type

Mandatory (Yes/No)

Description

Bucket

string

Yes

Explanation:

Bucket name

Restrictions:

  • A bucket name must be unique across all accounts and regions.
  • A bucket name:
    • Must be 3 to 63 characters long and start with a digit or letter. Lowercase letters, digits, hyphens (-), and periods (.) are allowed.
    • Cannot be formatted as an IP address.
    • Cannot start or end with a hyphen (-) or period (.).
    • Cannot contain two consecutive periods (..), for example, my..bucket.
    • Cannot contain a period (.) and a hyphen (-) adjacent to each other, for example, my-.bucket or my.-bucket.
  • If you repeatedly create buckets of the same name in the same region, no error will be reported and the bucket attributes comply with those set in the first creation request.

Default value:

None

Key

string

Yes

Explanation:

Object name. An object is uniquely identified by an object name in a bucket. An object name is a complete path that does not contain the bucket name.

For example, if the address for accessing the object is examplebucket.obs.ap-southeast-1.myhuaweicloud.com/folder/test.txt, the object name is folder/test.txt.

Value range:

The value must contain 1 to 1,024 characters.

Default value:

None

Position

int64

Yes

Explanation:

Position where the object data is appended

Restrictions:

For an object to be appended, the value of position must be set to 0 when the object is uploaded for the first time. For the second append upload, the value of position should be set to the value of NextAppendPosition returned in the response when the first upload is successful.

Value range:

0 to (263 – 1), in bytes

Default value:

0

ContentMD5

string

No

Explanation:

Base64-encoded MD5 value of the object data to be uploaded. It is provided for the OBS server to verify data integrity.

Value range:

ContentMD5 contains the Base64-encoded 128-bit MD5 digest calculated according to RFC 1864.

Example: n58IG6hfM7vqI4K0vnWpog==

Default value:

None

ACL

AclType

No

Explanation:

Pre-defined access policy that can be specified during object creation. For details about the ACL, see ACLs.

Value range:

See Table 3.

Default value:

None

WebsiteRedirectLocation

string

No

Explanation:

If the bucket is configured with website hosting, the request for obtaining the object can be redirected to another object in the bucket or an external URL.

The request is redirected to an object anotherPage.html in the same bucket:

WebsiteRedirectLocation:/anotherPage.html

The request is redirected to an external URL http://www.example.com/:

WebsiteRedirectLocation:http://www.example.com/

OBS obtains the specified value from the header and stores it in the object metadata WebsiteRedirectLocation.

Restrictions:

  • The value must start with a slash (/), http://, or https:// and cannot exceed 2 KB.
  • OBS only supports redirection for objects in the root directory of a bucket.

Default value:

None

ContentType

string

No

Explanation:

MIME type of the file to be uploaded. MIME type is a standard way of describing a data type and is used by the browser to decide how to display data.

Value range:

See What Is Content-Type (MIME)?

Default value:

If you do not specify this parameter when uploading an object, the SDK determines the object type based on the suffix of the specified object name and automatically assigns a value to this parameter.

ContentLength

int64

No

Explanation:

Size of the object to be uploaded

Restrictions:

  • The object size in a single upload ranges from 0 to 5 GB.
  • To upload files larger than 5 GB, multipart uploads should be used.

Default value:

If this parameter is not specified, the SDK automatically calculates the size of the object.

SseHeader

SseCHeader or SseKmsHeader

No

Explanation:

Server-side encryption header information. If SSE-C is used, see Table 4. If SSE-KMS is used, see Table 5.

StorageClass

StorageClassType

No

Explanation:

Object storage class

Value range:

See Table 6.

Default value:

If you do not specify this header, the object inherits the storage class of the bucket.

Metadata

map[string]string

No

Explanation:

Custom metadata of the object to be uploaded. You can add a header starting with x-obs-meta- in the request to define metadata. The custom metadata will be returned in the response when you retrieve the object or query the object metadata.

Restrictions:

  • The custom metadata cannot exceed 8 KB. To measure the custom metadata, sum the number of bytes in the UTF-8 encoding of each key and value.
  • The custom metadata keys are case insensitive, but are stored in lowercase in OBS. The key values are case sensitive.
  • Both custom metadata keys and their values must conform to US-ASCII standards. If non-ASCII or unrecognizable characters are required, they must be encoded and decoded in URL or Base64 on the client, because the server does not perform such operations.

Default value:

None

Body

io.Reader

No

Explanation:

Data stream of the object to be uploaded

Restrictions:

  • The object size in a single upload ranges from 0 to 5 GB.
  • To upload files larger than 5 GB, multipart uploads should be used.

Default value:

None

GrantReadId

string

No

Explanation:

ID (domain_id) of an account the READ permission is granted to. The account with the READ permission can read the current object and obtain its metadata.

Value range:

To obtain the account ID, see How Do I Get My Account ID and User ID?

Default value:

None

GrantReadAcpId

string

No

Explanation:

ID (domain_id) of an account the READ_ACP permission is granted to. The account with the READ_ACP permission can read the ACL of the current object.

Value range:

To obtain the account ID, see How Do I Get My Account ID and User ID?

Default value:

None

GrantWriteAcpId

string

No

Explanation:

ID (domain_id) of an account the WRITE_ACP permission is granted to. The account with the WRITE_ACP permission can write the ACL of the current object.

Value range:

To obtain the account ID, see How Do I Get My Account ID and User ID?

Default value:

None

GrantFullControlId

string

No

Explanation:

ID (domain_id) of an account the FULL_CONTROL permission is granted to. The account with the FULL_CONTROL permission can read the current object, obtain its metadata, and obtain and write its ACL.

Value range:

To obtain the account ID, see How Do I Get My Account ID and User ID?

Default value:

None

Expires

int64

No

Explanation:

Expiration time of the object (calculated from the latest modification time of the object). Expired objects are automatically deleted.

Value range:

1 to (263 - 1), in days

Default value:

None
Table 3 AclType

Constant

Default Value

Description

AclPrivate

private

Private read/write

A bucket or object can only be accessed by its owner.

AclPublicRead

public-read

Public read and private write

If this permission is granted on a bucket, anyone can read the object list, multipart tasks, metadata, and object versions in the bucket.

If it is granted on an object, anyone can read the content and metadata of the object.

AclPublicReadWrite

public-read-write

Public read/write

If this permission is granted on a bucket, anyone can read the object list, multipart tasks, metadata, and object versions in the bucket, and can upload or delete objects, initiate multipart upload tasks, upload parts, assemble parts, copy parts, and abort multipart upload tasks.

If it is granted on an object, anyone can read the content and metadata of the object.

AclPublicReadDelivered

public-read-delivered

Public read on a bucket as well as objects in the bucket

If this permission is granted on a bucket, anyone can read the object list, multipart tasks, metadata, and object versions, and read the content and metadata of objects in the bucket.

NOTE:

AclPublicReadDelivered does not apply to objects.

AclPublicReadWriteDelivered

public-read-write-delivered

Public read/write on a bucket as well as objects in the bucket

If this permission is granted on a bucket, anyone can read the object list, multipart uploads, metadata, and object versions in the bucket, and can upload or delete objects, initiate multipart upload tasks, upload parts, assemble parts, copy parts, and abort multipart uploads. They can also read the content and metadata of objects in the bucket.

NOTE:

AclPublicReadWriteDelivered does not apply to objects.

AclBucketOwnerFullControl

bucket-owner-full-control

If this permission is granted on an object, only the bucket and object owners have the full control over the object.

By default, if you upload an object to a bucket of any other user, the bucket owner does not have the permissions on your object. After you grant this policy to the bucket owner, the bucket owner can have full control over your object.
Table 4 SseCHeader

Parameter

Type

Mandatory (Yes/No)

Description

Encryption

string

Yes if used as a request parameter

Explanation:

SSE-C used for encrypting objects

Value range:

AES256, indicating objects are encrypted using SSE-C

Default value:

None

Key

string

Yes if used as a request parameter

Explanation:

Key for encrypting the object when SSE-C is used

Restrictions:

The value is a Base64-encoded 256-bit key, for example, K7QkYpBkM5+hca27fsNkUnNVaobncnLht/rCB2o/9Cw=.

Default value:

None

KeyMD5

string

No if used as a request parameter

Explanation:

MD5 value of the key for encrypting objects when SSE-C is used. This value is used to check whether any error occurs during the transmission of the key.

Restrictions:

The value is encrypted by MD5 and then encoded by Base64, for example, 4XvB3tbNTN+tIEVa0/fGaQ==.

Default value:

None
Table 5 SseKmsHeader

Parameter

Type

Mandatory (Yes/No)

Description

Encryption

string

Yes if used as a request parameter

Explanation:

SSE-KMS used for encrypting objects

Value range:

kms, indicating objects are encrypted using SSE-KMS

Default value:

None

Key

string

No if used as a request parameter

Explanation:

ID of the KMS master key when SSE-KMS is used

Value range:

Valid value formats are as follows:

  1. regionID:domainID:key/key_id
  2. key_id

In the preceding formats:

Default value:

  • If this parameter is not specified, the default master key will be used.
  • If there is no such a default master key, OBS will create one and use it by default.
Table 6 StorageClassType

Constant

Default Value

Description

StorageClassStandard

STANDARD

OBS Standard

Features low access latency and high throughput and is used for storing massive, frequently accessed (multiple times a month) or small objects (< 1 MB) requiring quick response.

StorageClassWarm

WARM

OBS Infrequent Access

Used for storing data that is semi-frequently accessed (fewer than 12 times a year) but is instantly available when needed.

StorageClassCold

COLD

OBS Archive

Used for storing rarely accessed (once a year) data.

Responses

Table 7 List of returned results

Parameter

Type

Description

output

*AppendObjectOutput

Explanation:

Returned results. For details, see Table 8.

err

error

Explanation:

Error messages returned by the API
Table 8 AppendObjectOutput

Parameter

Type

Description

StatusCode

int

Explanation:

HTTP status code

Value range:

A status code is a group of digits that can be 2xx (indicating successes) or 4xx or 5xx (indicating errors). It indicates the status of a response. For more information, see Status Code.

Default value:

None

RequestId

string

Explanation:

Request ID returned by the OBS server

Default value:

None

ResponseHeaders

map[string][]string

Explanation:

HTTP response headers

Default value:

None

VersionId

string

Explanation:

Object version ID. If versioning is enabled for the bucket, the object version ID will be returned.

Value range:

The value must contain 32 characters.

Default value:

None

ETag

string

Explanation:

Base64-encoded, 128-bit MD5 value of the object. ETag is the unique identifier of the object content. It can be used to determine whether the object content is changed. For example, if ETag value is A when an object is uploaded but changes to B when the object is downloaded, it indicates that the object content is changed. The ETag reflects changes to the object content, rather than the object metadata. An uploaded object or copied object has a unique ETag.

Restrictions:

If an object is encrypted using server-side encryption, the ETag is not the MD5 value of the object.

Value range:

The value must contain 32 characters.

Default value:

None

SseHeader

SseCHeader or SseKmsHeader

Explanation:

Server-side encryption header information. If SSE-C is used, see Table 9. If SSE-KMS is used, see Table 10.

NextAppendPosition

int64

Explanation:

Position from which the next append upload starts

Restrictions:

For an object to be appended, the value of position must be set to 0 when the object is uploaded for the first time. For the second append upload, the value of position should be set to the value of NextAppendPosition returned in the response when the first upload is successful.

Value range:

0 to (263 – 1), in bytes

Default value:

0
Table 9 SseCHeader

Parameter

Type

Mandatory (Yes/No)

Description

Encryption

string

Yes if used as a request parameter

Explanation:

SSE-C used for encrypting objects

Value range:

AES256, indicating objects are encrypted using SSE-C

Default value:

None

Key

string

Yes if used as a request parameter

Explanation:

Key for encrypting the object when SSE-C is used

Restrictions:

The value is a Base64-encoded 256-bit key, for example, K7QkYpBkM5+hca27fsNkUnNVaobncnLht/rCB2o/9Cw=.

Default value:

None

KeyMD5

string

No if used as a request parameter

Explanation:

MD5 value of the key for encrypting objects when SSE-C is used. This value is used to check whether any error occurs during the transmission of the key.

Restrictions:

The value is encrypted by MD5 and then encoded by Base64, for example, 4XvB3tbNTN+tIEVa0/fGaQ==.

Default value:

None
Table 10 SseKmsHeader

Parameter

Type

Mandatory (Yes/No)

Description

Encryption

string

Yes if used as a request parameter

Explanation:

SSE-KMS used for encrypting objects

Value range:

kms, indicating objects are encrypted using SSE-KMS

Default value:

None

Key

string

No if used as a request parameter

Explanation:

ID of the KMS master key when SSE-KMS is used

Value range:

Valid value formats are as follows:

  1. regionID:domainID:key/key_id
  2. key_id

In the preceding formats:

Default value:

  • If this parameter is not specified, the default master key will be used.
  • If there is no such a default master key, OBS will create one and use it by default.

Code Examples

This example appends content to example/objectname in examplebucket.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
 
package main
import (
    "fmt"
    "os"
    "strings"
    obs "github.com/huaweicloud/huaweicloud-sdk-go-obs/obs"
)
func main() {
    //Obtain an AK/SK pair using environment variables or import an AK/SK pair in other ways. Using hard coding may result in leakage.
    //Obtain an AK/SK pair on the management console. For details, see https://support.huaweicloud.com/intl/en-us/usermanual-ca/ca_01_0003.html.
    ak := os.Getenv("AccessKeyID")
    sk := os.Getenv("SecretAccessKey")
    // (Optional) If you use a temporary AK/SK pair and a security token to access OBS, you are advised not to use hard coding to reduce leakage risks. You can obtain an AK/SK pair using environment variables or import an AK/SK pair in other ways.
    securityToken := os.Getenv("SecurityToken")
    // Enter the endpoint corresponding to the bucket. CN-Hong Kong is used here as an example. Replace it with the one currently in use.
    endPoint := "https://obs.ap-southeast-1.myhuaweicloud.com"
"
    // Create an obsClient instance.
    // If you use a temporary AK/SK pair and a security token to access OBS, use the obs.WithSecurityToken method to specify a security token when creating an instance.
    obsClient, err := obs.New(ak, sk, endPoint, obs.WithSecurityToken(securityToken))
    if err != nil {
        fmt.Printf("Create obsClient error, errMsg: %s", err.Error())
    }
    input := &obs.AppendObjectInput{}
    // Specify a bucket name.
    input.Bucket = "examplebucket"
    // Specify the object (example/objectname as an example) to append content to.
    input.Key = "example/objectname"
    // Specify the position where content is appended. Position must be set to 0 for the first append upload.
    input.Position = 0
    // Specify data streams of the object to upload.
    input.Body = strings.NewReader("Hello OBS")
    // Append content to the object.
    output, err := obsClient.AppendObject(input)
    if err == nil {
        fmt.Printf("Append object(%s) under the bucket(%s) successful!\n", input.Key, input.Bucket)
        fmt.Printf("ETag:%s, NextAppendPosition:%d\n", output.ETag, output.NextAppendPosition)
        return
    }
    fmt.Printf("Append objects under the bucket(%s) fail!\n", input.Bucket)
    if obsError, ok := err.(obs.ObsError); ok {
        fmt.Println("An ObsError was found, which means your request sent to OBS was rejected with an error response.")
        fmt.Println(obsError.Error())
    } else {
        fmt.Println("An Exception was found, which means the client encountered an internal problem when attempting to communicate with OBS, for example, the client was unable to access the network.")
        fmt.Println(err)
    }
}

Helpful Links