Uploading an Object - Resumable (SDK for Go)_Object Storage Service

Function

The resumable upload is an encapsulated and enhanced version of the multipart upload used for dealing with possible upload failures of large files when the network connection is unstable or a program crashes. This API splits the file into multiple parts and uploads them individually. The upload result of each part is recorded in a checkpoint file in real time. A success message is returned only when all parts are uploaded. If any parts fail, an error message is returned telling you to call the API again to upload the failed parts. Since the checkpoint file contains the progress of each part, it saves you uploading all parts again in the event of an error.

You can call ObsClient.UploadFile to perform a resumable upload.

Restrictions

To upload an object, you must be the bucket owner or have the required permission (obs:object:PutObject in IAM or PutObject in a bucket policy). For details, see Introduction to OBS Access Control, IAM Custom Policies, and Configuring an Object Policy.
The mapping between OBS regions and endpoints must comply with what is listed in Regions and Endpoints.
The file uploaded by the resumable upload API must be larger than 100 KB in size.
When the resumable upload API is used, this option should be enabled so that the progress of the last upload can be read for entering the process again.

Method

func (obsClient ObsClient) UploadFile(input *UploadFileInput) (output *CompleteMultipartUploadOutput, err error)

Request Parameters

**Table 1** List of request parameters
Parameter	Type	Mandatory (Yes/No)	Description
input	*UploadFileInput	Yes	Explanation: Input parameters for a resumable upload. For details, see Table 2.

**Table 2** UploadFileInput
Parameter	Type	Mandatory (Yes/No)	Description
Bucket	string	Yes	Explanation: Bucket name Restrictions: A bucket name must be unique across all accounts and regions. A bucket name: Must be 3 to 63 characters long and start with a digit or letter. Lowercase letters, digits, hyphens (-), and periods (.) are allowed. Cannot be formatted as an IP address. Cannot start or end with a hyphen (-) or period (.). Cannot contain two consecutive periods (..), for example, my..bucket. Cannot contain a period (.) and a hyphen (-) adjacent to each other, for example, my-.bucket or my.-bucket. If you repeatedly create buckets of the same name in the same region, no error will be reported and the bucket attributes comply with those set in the first creation request. Default value: None
Key	string	Yes	Explanation: Object name. An object is uniquely identified by an object name in a bucket. An object name is a complete path that does not contain the bucket name. For example, if the address for accessing the object is examplebucket.obs.ap-southeast-1.myhuaweicloud.com/folder/test.txt, the object name is folder/test.txt. Value range: The value must contain 1 to 1,024 characters. Default value: None
UploadFile	string	Yes	Explanation: Local file path of the object to be uploaded Default value: None
PartSize	int64	Yes	Explanation: Part size Value range: The value ranges from 100 KB to 5 GB, in bytes. Default value: 102400
TaskNum	int	No	Explanation: Maximum number of parts that can be uploaded concurrently Value range: An integer from 1 to 10000 Default value: 1, indicating concurrent uploads are not used
EnableCheckpoint	bool	No	Explanation: Whether to enable the resumable upload Value range: true: The resumable upload mode is enabled. false: The resumable upload mode is disabled. Default value: false
CheckpointFile	string	No	Explanation: File used to record the upload progress. This parameter is valid only in the resumable upload mode. Default value: If this parameter is left blank, the progress file will be in the same directory as the local file to be uploaded.
ACL	AclType	No	Explanation: Pre-defined access policy that can be specified during object creation. For details about the ACL, see ACLs. Value range: See Table 3. Default value: None
WebsiteRedirectLocation	string	No	Explanation: If the bucket is configured with website hosting, the request for obtaining the object can be redirected to another object in the bucket or an external URL. The request is redirected to object anotherPage.html in the same bucket: WebsiteRedirectLocation:/anotherPage.html The request is redirected to an external URL http://www.example.com/: WebsiteRedirectLocation:http://www.example.com/ OBS obtains the specified value from the header and stores it in the object metadata WebsiteRedirectLocation. Restrictions: The value must start with a slash (/), http://, or https:// and cannot exceed 2 KB. OBS only supports redirection for objects in the root directory of a bucket. Default value: None
ContentType	string	No	Explanation: MIME type of the file to be uploaded. MIME type is a standard way of describing a data type and is used by the browser to decide how to display data. Value range: See What Is Content-Type (MIME)? Default value: If you do not specify this parameter when uploading an object, the SDK determines the object type based on the suffix of the specified object name and automatically assigns a value to this parameter.
SseHeader	SseCHeader or SseKmsHeader	No	Explanation: Server-side encryption header information. If SSE-C is used, see Table 4. If SSE-KMS is used, see Table 5.
StorageClass	StorageClassType	No	Explanation: Object storage class Value range: See Table 6. Default value: If this parameter is not specified, the object inherits the storage class of the bucket.
Metadata	map[string]string	No	Explanation: Custom metadata of the object to be uploaded. You can add a header starting with x-obs-meta- in the request to define metadata. The custom metadata will be returned in the response when you retrieve the object or query the object metadata. Restrictions: The custom metadata cannot exceed 8 KB. To measure the custom metadata, sum the number of bytes in the UTF-8 encoding of each key and value. The custom metadata keys are case insensitive, but are stored in lowercase in OBS. The key values are case sensitive. Both custom metadata keys and their values must conform to US-ASCII standards. If non-ASCII or unrecognizable characters are necessary, they must be encoded and decoded in URL or Base64 on the client, because the server side does not perform any decoding. Default value: None
Expires	int64	No	Explanation: Expiration time of the object (calculated from the latest modification time of the object). Expired objects are automatically deleted. Value range: 1 to (2⁶³ - 1), in days Default value: None
GrantReadId	string	No	Explanation: ID (domain_id) of an account the READ permission is granted to. The account with the READ permission can read the current object and obtain its metadata. Value range: To obtain the account ID, see How Do I Get My Account ID and User ID? Default value: None
GrantReadAcpId	string	No	Explanation: ID (domain_id) of an account the READ_ACP permission is granted to. The account with the READ_ACP permission can read the ACL of the current object. Value range: To obtain the account ID, see How Do I Get My Account ID and User ID? Default value: None
GrantWriteAcpId	string	No	Explanation: ID (domain_id) of an account the WRITE_ACP permission is granted to. The account with the WRITE_ACP permission can write the ACL of the current object. Value range: To obtain the account ID, see How Do I Get My Account ID and User ID? Default value: None
GrantFullControlId	string	No	Explanation: ID (domain_id) of an account the FULL_CONTROL permission is granted to. The account with the FULL_CONTROL permission can read the current object, obtain its metadata, and obtain and write its ACL. Value range: To obtain the account ID, see How Do I Get My Account ID and User ID? Default value: None
EncodingType	string	No	Explanation: Encoding type for Key in the response. If Key in the response contains control characters that are not supported by the XML 1.0 standard, you can specify this parameter to encode Key. Value range: url Default value: None. If you leave this parameter blank, encoding is not applied to Key.

**Table 3** AclType
Constant	Default Value	Description
AclPrivate	private	Private read/write A bucket or object can only be accessed by its owner.
AclPublicRead	public-read	Public read and private write If this permission is granted on a bucket, anyone can read the object list, multipart tasks, metadata, and object versions in the bucket. If it is granted on an object, anyone can read the content and metadata of the object.
AclPublicReadWrite	public-read-write	Public read/write If this permission is granted on a bucket, anyone can read the object list, multipart tasks, metadata, and object versions in the bucket, and can upload or delete objects, initiate multipart upload tasks, upload parts, assemble parts, copy parts, and abort multipart upload tasks. If it is granted on an object, anyone can read the content and metadata of the object.
AclPublicReadDelivered	public-read-delivered	Public read on a bucket as well as objects in the bucket If this permission is granted on a bucket, anyone can read the object list, multipart tasks, metadata, and object versions, and read the content and metadata of objects in the bucket. NOTE: AclPublicReadDelivered does not apply to objects.
AclPublicReadWriteDelivered	public-read-write-delivered	Public read/write on a bucket as well as objects in the bucket If this permission is granted on a bucket, anyone can read the object list, multipart uploads, metadata, and object versions in the bucket, and can upload or delete objects, initiate multipart upload tasks, upload parts, assemble parts, copy parts, and abort multipart uploads. They can also read the content and metadata of objects in the bucket. NOTE: AclPublicReadWriteDelivered does not apply to objects.
AclBucketOwnerFullControl	bucket-owner-full-control	If this permission is granted on an object, only the bucket and object owners have the full control over the object. By default, if you upload an object to a bucket of any other user, the bucket owner does not have the permissions on your object. After you grant this policy to the bucket owner, the bucket owner can have full control over your object.

**Table 4** SseCHeader
Parameter	Type	Mandatory (Yes/No)	Description
Encryption	string	Yes if used as a request parameter	Explanation: SSE-C used for encrypting objects Value range: AES256, indicating objects are encrypted using SSE-C Default value: None
Key	string	Yes if used as a request parameter	Explanation: Key for encrypting the object when SSE-C is used Restrictions: The value is a Base64-encoded 256-bit key, for example, K7QkYpBkM5+hca27fsNkUnNVaobncnLht/rCB2o/9Cw=. Default value: None
KeyMD5	string	No if used as a request parameter	Explanation: MD5 value of the key for encrypting objects when SSE-C is used. This value is used to check whether any error occurs during the transmission of the key. Restrictions: The value is encrypted by MD5 and then encoded by Base64, for example, 4XvB3tbNTN+tIEVa0/fGaQ==. Default value: None

**Table 5** SseKmsHeader
Parameter	Type	Mandatory (Yes/No)	Description
Encryption	string	Yes if used as a request parameter	Explanation: SSE-KMS used for encrypting objects Value range: kms, indicating objects are encrypted using SSE-KMS Default value: None
Key	string	No if used as a request parameter	Explanation: ID of the KMS master key when SSE-KMS is used Value range: Valid value formats are as follows: regionID:domainID:key/key_id key_id In the preceding formats: regionID indicates the ID of the region where the key is used. You can obtain it from Regions and Endpoints. domainID indicates the ID of the account where the key is used. To obtain it, see How Do I Get My Account ID and User ID? key_id indicates the ID of the key created on Data Encryption Workshop (DEW). To obtain it, see Viewing a CMK. Default value: If this parameter is not specified, the default master key will be used. If there is no such a default master key, OBS will create one and use it by default.

**Table 6** StorageClassType
Constant	Default Value	Description
StorageClassStandard	STANDARD	OBS Standard Features low access latency and high throughput and is used for storing massive, frequently accessed (multiple times a month) or small objects (< 1 MB) requiring quick response.
StorageClassWarm	WARM	OBS Infrequent Access Used for storing data that is semi-frequently accessed (fewer than 12 times a year) but is instantly available when needed.
StorageClassCold	COLD	OBS Archive Used for storing rarely accessed (once a year) data.

Responses

**Table 7** List of returned results
Parameter	Type	Description
output	*CompleteMultipartUploadOutput	Explanation: Returned results. For details, see Table 8.
err	error	Explanation: Error messages returned by the API

**Table 8** CompleteMultipartUploadOutput
Parameter	Type	Description
StatusCode	int	Explanation: HTTP status code Value range: A status code is a group of digits that can be 2xx (indicating successes) or 4xx or 5xx (indicating errors). It indicates the status of a response. For more information, see Status Code. Default value: None
RequestId	string	Explanation: Request ID returned by the OBS server Default value: None
ResponseHeaders	map[string][]string	Explanation: HTTP response headers Default value: None
ETag	string	Explanation: Base64-encoded, 128-bit MD5 value of the object. ETag is the unique identifier of the object content. It can be used to determine whether the object content is changed. For example, if ETag value is A when an object is uploaded but changes to B when the object is downloaded, it indicates that the object content is changed. The ETag reflects changes to the object content, rather than the object metadata. An uploaded object or copied object has a unique ETag. Restrictions: If an object is encrypted using server-side encryption, the ETag is not the MD5 value of the object. Value range: The value must contain 32 characters. Default value: None
Bucket	string	Explanation: Bucket in which parts are assembled Restrictions: A bucket name must be unique across all accounts and regions. A bucket name: Must be 3 to 63 characters long and start with a digit or letter. Lowercase letters, digits, hyphens (-), and periods (.) are allowed. Cannot be formatted as an IP address. Cannot start or end with a hyphen (-) or period (.). Cannot contain two consecutive periods (..), for example, my..bucket. Cannot contain a period (.) and a hyphen (-) adjacent to each other, for example, my-.bucket or my.-bucket. If you repeatedly create buckets of the same name in the same region, no error will be reported and the bucket attributes comply with those set in the first creation request. Default value: None
Key	string	Explanation: Object name obtained after part combination An object is uniquely identified by an object name in a bucket. An object name is a complete path that does not contain the bucket name. Value range: The value must contain 1 to 1,024 characters. Default value: None
Location	string	Explanation: URL of the generated object after part assembling Example: https://example-Bucket.obs.regions.myhuaweicloud.com/example-Object Default value: None
VersionId	string	Explanation: Version ID of the object obtained after part combination If versioning is enabled for the bucket, the object version ID will be returned. Value range: The value must contain 32 characters. Default value: None
SseHeader	SseCHeader or SseKmsHeader	Explanation: Server-side encryption header information. If SSE-C is used, see Table 9. If SSE-KMS is used, see Table 10.
EncodingType	string	Explanation: Encoding type for some elements in the response. If Key contains control characters (special characters) that are not supported by the XML 1.0 standard, set this parameter to url. Value range: url Default value: None. If you leave this parameter blank, encoding is not applied to elements.

**Table 9** SseCHeader
Parameter	Type	Mandatory (Yes/No)	Description
Encryption	string	Yes if used as a request parameter	Explanation: SSE-C used for encrypting objects Value range: AES256, indicating objects are encrypted using SSE-C Default value: None
Key	string	Yes if used as a request parameter	Explanation: Key for encrypting the object when SSE-C is used Restrictions: The value is a Base64-encoded 256-bit key, for example, K7QkYpBkM5+hca27fsNkUnNVaobncnLht/rCB2o/9Cw=. Default value: None
KeyMD5	string	No if used as a request parameter	Explanation: MD5 value of the key for encrypting objects when SSE-C is used. This value is used to check whether any error occurs during the transmission of the key. Restrictions: The value is encrypted by MD5 and then encoded by Base64, for example, 4XvB3tbNTN+tIEVa0/fGaQ==. Default value: None

**Table 10** SseKmsHeader
Parameter	Type	Mandatory (Yes/No)	Description
Encryption	string	Yes if used as a request parameter	Explanation: SSE-KMS used for encrypting objects Value range: kms, indicating objects are encrypted using SSE-KMS Default value: None
Key	string	No if used as a request parameter	Explanation: ID of the KMS master key when SSE-KMS is used Value range: Valid value formats are as follows: regionID:domainID:key/key_id key_id In the preceding formats: regionID indicates the ID of the region where the key is used. You can obtain it from Regions and Endpoints. domainID indicates the ID of the account where the key is used. To obtain it, see How Do I Get My Account ID and User ID? key_id indicates the ID of the key created on Data Encryption Workshop (DEW). To obtain it, see Viewing a CMK. Default value: If this parameter is not specified, the default master key will be used. If there is no such a default master key, OBS will create one and use it by default.

Code Examples

This example uploads example/objectname to examplebucket in a resumable upload.

    
     
       
       package main
import (
    "fmt"
    "os"
    obs "github.com/huaweicloud/huaweicloud-sdk-go-obs/obs"
)
func main() {
    //Obtain an AK/SK pair using environment variables or import an AK/SK pair in other ways. Using hard coding may result in leakage.
    //Obtain an AK/SK pair on the management console. For details, see https://support.huaweicloud.com/intl/en-us/usermanual-ca/ca_01_0003.html.
    ak := os.Getenv("AccessKeyID")
    sk := os.Getenv("SecretAccessKey")
    // (Optional) If you use a temporary AK/SK pair and a security token to access OBS, you are advised not to use hard coding to reduce leakage risks. You can obtain an AK/SK pair using environment variables or import an AK/SK pair in other ways.
    // securityToken := os.Getenv("SecurityToken")
    // Enter the endpoint corresponding to the bucket. CN-Hong Kong is used here as an example. Replace it with the one currently in use.
    endPoint := "https://obs.ap-southeast-1.myhuaweicloud.com"
    // Create an obsClient instance.
    // If you use a temporary AK/SK pair and a security token to access OBS, use the obs.WithSecurityToken method to specify a security token when creating an instance.
    obsClient, err := obs.New(ak, sk, endPoint/*, obs.WithSecurityToken(securityToken)*/)
    if err != nil {
        fmt.Printf("Create obsClient error, errMsg: %s", err.Error())
    }
    input := &obs.UploadFileInput{}
    // Specify a bucket name.
    input.Bucket = "examplebucket"
    // Specify the object (example/objectname as an example) to upload.
    input.Key = "example/objectname"
    // Specify your local file (/tmp/objectname as an example) to upload.
    input.UploadFile = "/tmp/objectname"
    // Specify whether to enable resumable upload (true as an example). The default value is false, indicating that resumable upload is disabled.
    input.EnableCheckpoint = true
    // Specify a part size, in bytes. This example sets each part to 9 MB.
    input.PartSize = 9 * 1024 * 1024
    // Specify the maximum number of parts that can be concurrently uploaded. 5 is used as an example.
    input.TaskNum = 5
    // Upload the object using resumable upload.
    output, err := obsClient.UploadFile(input)
    if err == nil {
        fmt.Printf("Upload file(%s) under the bucket(%s) successful!\n", input.UploadFile, input.Bucket)
        fmt.Printf("ETag:%s\n", output.ETag)
        return
    }
    fmt.Printf("Upload file(%s) under the bucket(%s) fail!\n", input.UploadFile, input.Bucket)
    if obsError, ok := err.(obs.ObsError); ok {
        fmt.Println("An ObsError was found, which means your request sent to OBS was rejected with an error response.")
        fmt.Println(obsError.Error())
    } else {
        fmt.Println("An Exception was found, which means the client encountered an internal problem when attempting to communicate with OBS, for example, the client was unable to access the network.")
        fmt.Println(err)
    }
}

      

    
   

Uploading an Object - Resumable (SDK for Go)