Copied.
Streaming Upload (PUT)
Scenarios
If the size of an object, such as a text file, image, or video, to be uploaded is smaller than 5 GB, you can use streaming upload (PUT).
Parameters sent by PUT are passed in the request header. You need to specify the object name in the URL. An example request is as follows:
PUT /ObjectName HTTP/1.1
OBS allows you to upload objects to buckets in a specified region, but Huawei Cloud does not detect the object content you uploaded. If your object uploads involve cross-border transfer, ensure that your use complies with relevant laws and regulations.
Prerequisites
- You have the write permissions to a bucket.
- If you want to classify files, you can create folders and upload files to different folders. For details, see Creating a Folder.
Constraints
Size and number of objects to upload
- If a file is smaller than 5 GB, use streaming upload. To upload a file larger than 5 GB, refer to Multipart Upload.
- You can specify a quota for a bucket to limit its storage capacity. For details, see Managing Bucket Quotas.
Object naming
See Guidelines on Naming Object Keys.
Batch operations
- The bucket is in CN-Hong Kong, AP-Bangkok, or AP-Singapore.
- The bucket version is 3.0. To view a bucket version, see Checking OBS Version (OBS 2.0 or OBS 3.0).
Folder uploads
You can upload folders using OBS Console, OBS Browser+, or obsutil.
Other
- Versioning is not enabled for a bucket:
- If you upload a new file with the same name as the one in the bucket, the new file will automatically overwrite the previous one and the ACL information of the previous file will not be retained.
- If you upload a new folder with the same name as the one in the bucket, the two folders will be merged, and files in the new folder will overwrite those with the same name in the previous folder.
- Versioning is enabled for a bucket: If you upload a new file with the same name as the one you previously uploaded to the bucket, a new file version will be added to the bucket. For details, see Versioning.
Ways to Upload
You can use OBS Console, APIs, SDKs, OBS Browser+, or obsutil to upload objects.
Using OBS Console
- In the navigation pane of OBS Console, choose Object Storage.
- In the bucket list, click the bucket you want to operate.
- Go to the folder that you want to upload files to and click Upload Object. The Upload Object dialog box is displayed.
Batch upload is used as an example here. If a region only supports single upload, follow the on-screen instructions.
If the files that you want to upload to OBS are stored in Microsoft OneDrive, it is recommended that the names of these files contain a maximum of 32 characters to ensure compatibility.
Figure 1 Object Upload
- Select a storage class. If you do not specify a storage class, the objects you upload inherit the default storage class of the bucket.
An object can have a different storage class from its bucket. You can specify a storage class for an object when uploading it, or you can change the object storage class after the object is uploaded.
- Drag and drop the files or folders you want to upload to the Upload Object area.
You can also click add files in the Upload Object area to select files.
- Server-Side Encryption: Choose Disable, SSE-KMS, or SSE-OBS. For details, see Server-Side Encryption.
If the bucket has Server-Side Encryption configured, an object you upload will inherit encryption from the bucket by default, but you can change the encryption option as required.
- (Optional) To configure WORM retention policies, or metadata, click Next: (Optional) Configure Advanced Settings.
WORM retention policies can be configured in the advanced settings only when WORM is enabled for the bucket.
Configuring metadata: Add metadata ContentDisposition, ContentLanguage, WebsiteRedirectLocation, ContentEncoding, or ContentType as needed. For more information, see OBS Metadata. Metadata is a set of name-value pairs. The metadata value cannot be left blank. You can add two or more metadata entries by clicking Add.
Configuring WORM retention: Choose Inherit from bucket, or choose Configure and then specify a retention period, to automatically protect new objects uploaded to the bucket from being deleted.
Figure 2 Configuring metadata or WORM retention
- Click Upload.
Using APIs
Using the GUI Tool - OBS Browser+
- Log in to OBS Browser+.
- Click the bucket where you want to upload files or folders.
- Click Upload and then Add File or Folder, as shown in Figure 3.
For better experience when using the Add File function, you are advised to upload a maximum of 100 files at a time. If you need to upload more, place all the files in a folder and upload them by adding a folder.
- If message "Service Unavailable" is displayed when files are being uploaded, try again later.
- If an access deny message is displayed when you are uploading a file or folder, possible causes are as follows:
- Access to the bucket is restricted by an ACL. For example, you do not have the write permission for the bucket.
- Access to the bucket is restricted by a bucket policy. For example, you do not have the write permission for the bucket, or write operations cannot be performed on the bucket during the current period.
If such message is displayed, check ACL and policy settings of the bucket and resolve the problem accordingly.
You must have access to the file you want to upload, or the file upload will fail.
- In the displayed dialog box, select the file or folder you want to upload and click Open.
You can upload one folder or multiple files at a time. To upload multiple files, hold down Ctrl or Shift to select multiple files and batch upload them. You can also press Ctrl+A to select all files. The operations are consistent with those in Windows operating systems.
- Click OK.
Using the CLI Tool - obsutil
Command Line Structure
- In Windows
- Uploading a file
obsutil cp file_url obs://bucket[/key] [-arcDir=xxx] [-dryRun] [-link] [-u] [-vlength] [-vmd5] [-p=1] [-threshold=5248800] [-acl=xxx] [-sc=xxx] [-meta=aaa:bbb#ccc:ddd] [-ps=auto] [-o=xxx] [-cpd=xxx] [-fr] [-o=xxx] [-config=xxx] [-e=xxx] [-i=xxx] [-k=xxx] [-t=xxx]
- Uploading a folder
obsutil cp folder_url obs://bucket[/key] -r [-arcDir=xxx] [-dryRun] [-link] [-f] [-flat] [-u] [-vlength] [-vmd5] [-j=1] [-p=1] [-threshold=52428800] [-acl=xxx] [-sc=xxx] [-meta=aaa:bbb#ccc:ddd] [-ps=auto] [-include=*.xxx] [-exclude=*.xxx] [-timeRange=time1-time2] [-mf] [-o=xxx] [-cpd=xxx] [-config=xxx] [-e=xxx] [-i=xxx] [-k=xxx] [-t=xxx]
- Uploading multiple files/folders
obsutil cp file1_url,folder1_url|filelist_url obs://bucket[/prefix] -msm=1 [-r] [-arcDir=xxx] [-dryRun] [-link] [-f] [-u] [-vlength] [-vmd5] [-flat] [-j=1] [-p=1] [-threshold=52428800] [-acl=xxx] [-sc=xxx] [-meta=aaa:bbb#ccc:ddd] [-ps=auto] [-include=*.xxx] [-exclude=*.xxx][-timeRange=time1-time2] [-at] [-mf] [-o=xxx] [-cpd=xxx] [-config=xxx] [-e=xxx] [-i=xxx] [-k=xxx] [-t=xxx]
In this command, /prefix is the name prefix for uploading folders. For the execution examples, see Upload.
- Uploading a file
- In Linux or macOS
- Uploading a file
./obsutil cp file_url obs://bucket[/key] [-arcDir=xxx] [-dryRun] [-link] [-u] [-vlength] [-vmd5] [-p=1] [-threshold=5248800] [-acl=xxx] [-sc=xxx] [-meta=aaa:bbb#ccc:ddd] [-ps=auto] [-o=xxx] [-cpd=xxx] [-fr] [-o=xxx] [-config=xxx] [-e=xxx] [-i=xxx] [-k=xxx] [-t=xxx]
- Uploading a folder
./obsutil cp folder_url obs://bucket[/key] -r [-arcDir=xxx] [-dryRun] [-link] [-f] [-flat] [-u] [-vlength] [-vmd5] [-j=1] [-p=1] [-threshold=52428800] [-acl=xxx] [-sc=xxx] [-meta=aaa:bbb#ccc:ddd] [-ps=auto] [-include=*.xxx] [-exclude=*.xxx] [-timeRange=time1-time2] [-at] [-mf] [-o=xxx] [-cpd=xxx] [-config=xxx] [-e=xxx] [-i=xxx] [-k=xxx] [-t=xxx]
- Uploading multiple files/folders
./obsutil cp file1_url,folder1_url|filelist_url obs://bucket[/prefix] -msm=1 [-r] [-arcDir=xxx] [-dryRun] [-link] [-f] [-u] [-vlength] [-vmd5] [-flat] [-j=1] [-p=1] [-threshold=52428800] [-acl=xxx] [-sc=xxx] [-meta=aaa:bbb#ccc:ddd] [-ps=auto] [-include=*.xxx] [-exclude=*.xxx][-timeRange=time1-time2] [-mf] [-o=xxx] [-cpd=xxx] [-config=xxx] [-e=xxx] [-i=xxx] [-k=xxx] [-t=xxx]
In this command, /prefix is the name prefix for uploading folders. For the execution examples, see Upload.
- Uploading a file
Examples
- Take the Windows OS as an example. Run the obsutil cp d:\temp\test.txt obs://bucket-test/key command to upload the test.txt file in the temp directory in the D: drive to bucket bucket-test and rename the file as key.
obsutil cp d:\temp\test.txt obs://bucket-test/key Parallel: 5 Jobs: 5 Threshold: 50.00MB PartSize: auto VerifyLength: false VerifyMd5: false CheckpointDir: C:\Users\Administrator\.obsutil_checkpoint [====================================================] 100.00% 1.68 MB/s 8.46MB/8.46MB 5s Upload successfully, 8.46MB, n/a, d:\temp\test.txt --> obs://bucket-test/key, cost [5], status [200], request id [0000016979E1D2B2860BB5181229C72C]
- Take the Windows OS as an example. Run the obsutil cp d:\temp obs://bucket-test -f -r command to recursively upload all files and subfolders in the temp directory in the D: drive to the temp folder in bucket bucket-test.
obsutil cp d:\temp obs://bucket-test -f -r Parallel: 5 Jobs: 5 Threshold: 50.00MB PartSize: auto VerifyLength: false VerifyMd5: false CheckpointDir: C:\Users\Administrator\.obsutil_checkpoint Task id: 104786c8-27c2-48fc-bc6a-5886596fb0ed OutputDir: C:\Users\Administrator\.obsutil_output [========================================================] 100.00% tps:35.71 2.02 KB/s 7.20MB/7.20MB 0s Succeed count is: 5 Failed count is: 0 Metrics [max cost:90 ms, min cost:45 ms, average cost:63.80 ms, average tps:35.71] Task id: 104786c8-27c2-48fc-bc6a-5886596fb0ed
- For more examples, see Upload.
Parameter Description
Parameter |
Optional or Mandatory |
Description |
---|---|---|
file_url |
Optional for uploading multiple files/folders Mandatory for uploading a file |
Local file path
NOTE:
|
folder_url |
Optional for uploading multiple files/folders Mandatory for uploading a folder |
Local folder path
NOTE:
|
filelist_url |
Optional for uploading multiple files/folders |
Indicates the path of the file that contains the list of files/folders to be uploaded. If this parameter is configured, msm must be set to 2. |
bucket |
Mandatory |
Bucket name |
key |
Optional |
Indicates the object name or object name prefix specified when uploading a file, or the object name prefix specified when uploading a folder. The rules are as follows:
NOTE:
For details about how to use this parameter, see Upload. |
fr |
Optional for uploading a file (additional parameter) |
Generates an operation result list when uploading a file. |
flat |
Optional for uploading a folder or multiple files/folders (additional parameter) |
Uploads all files in a folder but not the folder itself. |
arcDir |
Optional (additional parameter) |
Path to which the uploaded files are archived |
dryRun |
Optional (additional parameter) |
Conducts a dry run. |
link |
Optional (additional parameter) |
Uploads the actual path of the symbolic-link file/folder
NOTICE:
|
u |
Optional (additional parameter) |
Indicates incremental upload. If this parameter is set, each file can be uploaded only when it does not exist in the bucket, its size is different from the namesake one in the bucket, or it has the latest modification time.
CAUTION:
When you compare each local file with data in the bucket, a billable HEAD request is generated. For details, see Requests. |
vlength |
Optional (additional parameter) |
After the upload completes, check whether the sizes of the objects in the bucket are the same as those of the local files. |
vmd5 |
Optional (additional parameter) |
After the upload completes, check whether the MD5 values of the objects in the bucket are the same as those of the local files.
NOTE:
|
p |
Optional (additional parameter) |
Indicates the maximum number of concurrent multipart upload tasks when uploading a file. The default value is the value of defaultParallels in the configuration file. |
threshold |
Optional (additional parameter) |
Indicates the threshold for enabling multipart upload, in bytes. The default value is the value of defaultBigfileThreshold in the configuration file.
NOTE:
|
acl |
Optional (additional parameter) |
Access control policies that can be specified when uploading files. Possible values are:
NOTE:
The preceding four values indicate private read and write, public read, public read and write, and bucket owner full control. |
sc |
Optional (additional parameter) |
Indicates the storage classes of objects that can be specified when uploading files. Possible values are:
|
meta |
Optional (additional parameter) |
Indicates the standard and custom metadata that can be specified during file upload. This parameter should be configured in the following format: key1:value1#key2:value2#key3:value3.
NOTE:
|
ps |
Optional (additional parameter) |
Indicates the size of each part in a multipart upload task, in bytes. The value ranges from 100KB to 5GB. The default value is the value of defaultPartSize in the configuration file.
NOTE:
|
cpd |
Optional (additional parameter) |
Indicates the folder where the part records reside. The default value is .obsutil_checkpoint, the subfolder in the home directory of the user who executes obsutil commands.
NOTE:
A part record is generated during a multipart upload and saved to the upload subfolder. After the upload succeeds, its part record is deleted automatically. If the upload fails or is suspended, the system attempts to resume the task according to its part record when you perform the upload the next time. |
r |
Mandatory for uploading a folder (additional parameter) Optional for uploading multiple files/folders |
Indicates files and subfolders within the folder when uploading a folder recursively. |
f |
Optional for uploading a folder or multiple files/folders (additional parameter) |
Runs in force mode. |
j |
Optional for uploading a folder or multiple files/folders (additional parameter) |
Indicates the maximum number of concurrent tasks for uploading a folder. The default value is the value of defaultJobs in the configuration file.
NOTE:
The value is ensured to be greater than or equal to 1. |
msm |
Mandatory for uploading multiple files/folders (additional parameter) |
Enables the mode for uploading multiple files/folders. Possible values are 1 and 2.
NOTE:
|
exclude |
Optional for uploading a folder or multiple files/folders (additional parameter) |
Indicates the file matching patterns that are excluded, for example: *.txt.
NOTE:
NOTICE:
|
include |
Optional for uploading a folder or multiple files/folders (additional parameter) |
Indicates the file matching patterns that are included, for example: *.jpg.
NOTE:
NOTE:
Example of uploading files in a request with the include parameter contained: ./obsutil cp /localpath/ obs://test/ -include=/localpath/2022-12-09/* -f -r This command uploads files that are under localpath and start with /localpath/2022-12-09/ to bucket test.
NOTICE:
|
at |
Optional for uploading a folder or multiple files/folders (additional parameter) |
Indicates that only the files whose latest access time is within the value of timeRange are uploaded.
NOTE:
|
disableDirObject |
Optional for uploading multiple folders (additional parameter) |
Indicates the folders themselves are not uploaded as an object. Configuring this parameter can avoid uploading empty folders to a bucket. If a folder contains files, the files will be uploaded and the original path format is retained. |
timeRange |
Optional for uploading a folder or multiple files/folders (additional parameter) |
Indicates the time range matching pattern when uploading files. Only files whose latest modification time is within the configured time range are uploaded. This pattern has a lower priority than the file matching patterns (exclude/include). That is, the time range matching pattern is executed after the configured file matching patterns.
NOTE:
NOTICE:
Time in the matching pattern is the UTC time. |
mf |
Optional (additional parameter) |
Indicates that the name matching pattern (include or exclude) and the time matching pattern (timeRange) also take effect on folders. |
o |
Optional (additional parameter) |
Indicates the folder where operation result lists reside. After the command is executed, result lists (possibly including success, failure, and warning files) are generated in the folder. The default value is .obsutil_output, the subfolder in the home directory of the user who executes obsutil commands.
NOTE:
|
config |
Optional (additional parameter) |
User-defined configuration file for executing the current command. To learn the parameters that can be configured in this file, see Configuration Parameters. |
e |
Optional (additional parameter) |
Specifies the endpoint. |
i |
Optional (additional parameter) |
Specifies the user's AK. |
k |
Optional (additional parameter) |
Specifies the user's SK. |
t |
Optional (additional parameter) |
Specifies the user's security token. |
Response
Field |
Description |
---|---|
Parallel |
Parameter -p in the request |
Jobs |
Parameter -j in the request |
Threshold |
Parameter -threshold in the request |
PartSize |
Parameter -ps in the request |
Exclude |
Parameter -exclude in the request |
Include |
Parameter -include in the request |
TimeRange |
Parameter -timeRange in the request |
VerifyLength |
Parameter -vlength in the request |
VerifyMd5 |
Parameter -vmd5 in the request |
CheckpointDir |
Parameter -cpd in the request |
OutputDir |
Parameter -o in the request |
ArcDir |
Parameter -arcDir in the request |
Succeed count |
Number of successful tasks |
Failed count |
Number of failed tasks |
Skip count |
Number of tasks that are skipped during incremental upload, download, or copy, and synchronous upload, download, or copy.
NOTE:
Skipped tasks are recorded into successful tasks. |
Warning count |
Number of tasks that are executed successfully but contain warnings.
NOTE:
|
Succeed bytes |
Number of bytes that are successfully uploaded or downloaded. |
max cost |
Maximum duration of all tasks, in ms |
min cost |
Minimum duration of all tasks, in ms |
average cost |
Average duration of all tasks, in ms |
average tps |
The average number of tasks completed per second |
Task id |
Unique ID of an operation, which is used to search for the result list generated in a batch task |
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot