Help Center/ Object Storage Service/ Operation Guide (Leaving soon. Moving to User Guide.)/ Object Management/ Uploading Objects/ Uploading an Object
Updated on 2024-05-31 GMT+08:00
View PDF
Share

Uploading an Object

Scenarios

You can upload files or folders to an existing OBS bucket. If you want to classify files to be uploaded, create a folder and upload related files to the folder.

For details about how to create a folder, see Creating a Folder.

These files can be texts, images, videos, or any other type of files.

OBS allows you to upload objects to buckets in a specified region, but Huawei Cloud does not detect the object content you uploaded. If your object uploads involve cross-border transfer, ensure that your use complies with relevant laws and regulations.

Constraints

Size and number of objects to upload

  • OBS Console puts limits on the size and number of files you can upload.
    • In regions that support batch uploads, a maximum of 100 files can be uploaded at a time, with a total size of no more than 5 GB. If you upload only one file in a batch, this file cannot exceed 5 GB in size.
    • In regions that do not support batch uploads, only one file can be uploaded at a time, with a size of no more than 50 MB.
  • With OBS Browser+ or obsutil, you can upload files smaller than 48.8 TB. OBS Browser+ allows you to upload a maximum of 500 files at a time. There is no limit on the number of files you can upload using obsutil at a time.
  • With PUT, POST, or append methods of the OBS SDKs or API, you can upload files smaller than 5 GB.
  • With multipart uploads of the OBS SDKs or API or resumable uploads of the OBS SDKs, you can upload files smaller than 48.8 TB.
  • You can specify a quota for a bucket to limit the bucket's capacity for storing the uploaded objects. For details, see Managing Bucket Quotas.

Object naming

Batch operations

  • Batch uploads are available only when:
    1. The bucket is in any of the following regions: CN-Hong Kong, AP-Bangkok, and AP-Singapore.
    2. The bucket version is 3.0. To view the bucket version, see Checking OBS Version (OBS 2.0 or OBS 3.0).

Folder uploads

  • You can upload folders using OBS Console, OBS Browser+, or obsutil.

Others

  • If versioning is disabled and the name of a newly uploaded file is the same as that of a file in the bucket, the newly uploaded file automatically overwrites the existing file and does not retain the ACL information of the existing file. If the name of the newly uploaded folder is the same as that of a folder in the bucket, the two folders will be merged, and files in the new folder will overwrite namesake files in the old folder.
  • If versioning is enabled and the name of a newly uploaded file is the same as that of a file in the bucket, a new version is added to the existing file. For details about versioning, see Versioning.

Upload Methods

Upload Method

Description

PUT

(Streaming upload)

Use the PUT or POST method when the size of the file to be uploaded is less than 5 GB.

For details about the differences between the two upload methods, see What Are the Differences Between PUT and POST Upload Methods?

POST

(browser-based upload)

Multipart upload

Use this method when the size of the file to be uploaded is greater than 5 GB and less than 48.8 TB.

For details, see Uploading Objects Using a Multipart Upload.

NOTE:

If you have over 48.8 TB data to upload, refer to Migrating Local Data to OBS.

Resumable transfer

Uploading large files often fails due to poor network conditions or program breakdowns. It is a waste of resources to restart the upload process upon an upload failure, and the restarted upload process may still suffer from the unstable network. To resolve such issues, you can use the API for resumable upload, whose working principle is to divide the to-be-uploaded file into multiple parts and upload them separately. This method saves resources and improves efficiency upon the re-upload, and speeds up the upload process by concurrently uploading parts.

For details, see Does OBS Support Resumable Data Transfer?

Synchronous upload of incremental objects

This method synchronizes all content in the local source path to the specified target bucket in OBS, ensuring that the content is consistent between the local path and the target bucket. Incremental synchronization has the following meanings: 1) Increment: Compare the source file with the target object and upload only the source file that has changes. 2) Synchronization: After the command is executed, ensure that the local source path is a subset of the target bucket specified by OBS. That is, any file in the local source path has its corresponding object in the target bucket on OBS.

For details, see Synchronously Uploading Incremental Objects.

Appendable upload

The AppendObject operation adds data to the end of an object in a specified bucket. If there is no namesake object in the bucket, a new object is created.

For details, see Appending an Object.

How to Use

You can use OBS Console, SDKs, OBS Browser+, obsutil, or APIs to upload objects.

Tool

Reference

OBS Console

Uploading a File

SDKs

OBS supports software development kits (SDKs) in multiple languages. For details, see the corresponding developer guide on the SDK Overview page.

API

Operations on Objects

OBS Browser+

Drag-and-Drop Upload

obsutil

Uploading an Object

Related Operations

When uploading an object, you can specify a storage class for it. After the object is uploaded, you can also change its storage class.

  • You can manually change objects among the Standard, Infrequent Access, Archive, and Deep Archive storage classes. Objects in the Archive storage class must be restored before they can be manually changed to the Standard, or Infrequent Access storage class. Objects in the Deep Archive storage class must be restored before they can be manually changed to the Standard, Infrequent Access, or Archive storage class. Changing the storage class of objects in the Infrequent Access or Archive or Deep Archive storage class involves retrieval costs, so you are advised to change their storage classes based on the access frequency and scenario.
  • The minimum storage duration is 180 days for Deep Archive storage, 90 days for Archive storage, and 30 days for Infrequent Access storage. If an object is transitioned to another storage class before it has been stored for the required minimum storage duration, you need to pay for the remaining days.
  • You can also configure a lifecycle rule to transition the storage class of an object. For details, see Lifecycle Management.

Granting Other Huawei Accounts Permission to Upload Objects

This example describes how to grant other Huawei accounts (including their IAM users) permission to upload objects to OBS.

Bucket policies are recommended to grant such a permission.

Before you grant the object upload permission to an IAM user under another Huawei account, the IAM user must have been authorized to perform specified operations on the bucket by its account. The allowed operations must be the same as those specified in the bucket policy. For details, see Configure an IAM Permission That Allows Specified Operations.

If the Huawei account you want to grant permission to has the Tenant Administrator role that contains the administrator permissions for all services except IAM, skip the configuration mentioned here.

  1. Log in to OBS Console. In the navigation pane, choose Object Storage.
  2. In the bucket list, click the bucket name you want to go to the Objects page.
  3. In the navigation pane, choose Permissions > Bucket Policies.
  4. On the Bucket Policies page, click Create.
  5. Configure parameters for a bucket policy.
    Figure 1 Configuring a bucket policy
    Table 1 Parameters for configuring a bucket policy

    Parameter

    Description

    Configuration method

    Choose a method you like, for example, Visual Editor.

    Policy Name

    Enter a custom policy name.

    Policy content

    Effect

    Select Allow.

    Principals
    • Select Other accounts.
      NOTE:

      You can obtain the account ID and IAM user ID from the My Credentials page.

      Accounts should be configured in the Domain ID/IAM user ID format, with each one on a separate line.

      Account ID/* indicates that permission is granted to all IAM users under the account.

    Resources
    • Select Specified objects.
    • Enter an object name prefix for the resource path.
      NOTE:
      • You can click Add to specify multiple resource paths.
      • You can specify a prefix to apply the policy to a specific object, a set of objects, or directories, leave the prefix blank to apply the policy only to the current bucket, or set the prefix to a wildcard (*) to apply the policy to all objects in the current bucket.

        To specify a specific object, enter the object name.

        To specify a set of objects, enter Object name prefix*, *Object name suffix, or *.

    Actions
    • Choose Customize.
    • Select action PutObject (to upload objects).
      NOTE:

      If an account (including its IAM users) wants to access the bucket from OBS Browser+ by adding an external bucket, Current bucket is also needed for Resources and action ListBucket should be selected too.
  6. Ensure all the configurations are correct and click Create.
  7. Verify that the authorized account can upload objects using an API or SDKs, or can upload objects using the external bucket added to OBS Browser+ when the ListBucket permission is available. Currently, using OBS Console to access buckets of other accounts is not allowed.

    After the ListBucket permission is configured, when you use OBS Browser+ to access the added external bucket, a message may still be displayed indicating that you do not have required permissions.

    This happens because loading the bucket's details page on OBS Browser+ calls some other OBS APIs whose operations are not allowed by the granted permissions. However, existing permissions are not affected.

Causes of Object Upload Failures and Solutions

For details, see Why Am I Unable to Upload an Object?

Parent topic: Uploading Objects