Uploading an Object

file_url

Optional for uploading multiple files/folders

Mandatory for uploading a file

The local file path

• Do not nest paths when uploading multiple files/folders. For example, you cannot specify /a/b/c and /a/b/ at the same time.
• If this parameter is configured when uploading multiple files/folders, msm must be set to 1. In this case, use commas (,) to separate multiple file paths, for example, file_url1,file_url2.
• Files and folders can both be included when uploading multiple files/folders. For example, file_url1,folder_url1,file_url2,folder_url2.

folder_url

Optional for uploading multiple files/folders

Mandatory for uploading a folder

The local folder path

• If flat is not configured when uploading a folder, the entire folder is uploaded. If flat is configured, all files in the folder are uploaded.
• Do not nest paths when uploading multiple files/folders. For example, you cannot specify /a/b/c and /a/b/ at the same time.
• If this parameter is configured when uploading multiple files/folders, msm must be set to 1. In this case, use commas (,) to separate multiple folder paths, for example, folder_url1,folder_url2.
• Files and folders can both be included when uploading multiple files/folders. For example, file_url1,folder_url1,file_url2,folder_url2.

filelist_url

Optional for uploading multiple files/folders

The path of the file that contains the list of files/folders to be uploaded. If this parameter is configured, msm must be set to 2.

• The list file is in common text file formats, such as TXT and CSV. Each line in the file indicates a file or folder to be uploaded. For example:

  file_url1

  file_url2

  folder_url1

  folder_url2

• Do not nest paths in the list file. For example, you cannot specify /a/b/c and /a/b/ at the same time.

bucket

Mandatory

The name of the bucket where you will upload the file or folder

key

Optional

The object name or object name prefix specified when uploading a file, or the object name prefix specified when uploading a folder

When uploading a file or folder, the upload path and name change depending on the value of this parameter. The specific rules are as follows:

• If this parameter is left blank, the object is uploaded to the root directory of the bucket, and the object name is the name of the uploaded file or folder.
• If this parameter is specified:
  • If the parameter value ends with a slash (/), the object is uploaded to the directory indicated by this value, and the object name is the parameter value plus the file name, for example, folder/filename.txt or folder/foldername.
  • If the parameter value does not end with a slash (/), the object name for the file is the parameter value, for example, filename.txt, and the object name for the folder is the parameter value plus a slash (/), for example, folder/, which is also the name of the directory where the folder is uploaded.

fr

Optional for uploading a file (additional parameter)

Generates an operation result file when uploading a file.

flat

Optional for uploading a folder or multiple files/folders (additional parameter)

Uploads all files in a folder but not the folder itself.

arcDir

Optional (additional parameter)

The path to which the uploaded files are archived

dryRun

Optional (additional parameter)

Conducts a dry run.

link

Optional (additional parameter)

The actual path of a symbolic link for a file or folder

• If this parameter is not specified and the file to be uploaded is a symbolic-link file whose target file does not exist, the exception message "The system cannot find the file specified" will be displayed in Windows OS, while the exception message "No such file or directory" will be displayed in macOS or Linux OS.
• Avoid the symbolic link loop of a folder, otherwise, the upload will exit due to panic. If you do not want the system to panic, set panicForSymbolicLinkCircle to false in the configuration file.

u

Optional (additional parameter)

Indicates incremental upload.

If this parameter is set, each file can be uploaded only when it does not exist in the bucket, its size is different from the namesake one in the bucket, or it has the latest modification time.

When you compare each local file with data in the bucket, a billable HEAD request is generated. For details, see Requests.

vlength

Optional (additional parameter)

After the upload completes, checks whether the sizes of the objects in the bucket are the same as those of the local files.

vmd5

Optional (additional parameter)

After the upload completes, checks whether the MD5 values of the objects in the bucket are the same as those of the local files.

• If your object needs encryption, do not use this parameter.
• If the size of the file or folder to be uploaded is too large, using this parameter will degrade the overall performance due to MD5 calculation.
• After the MD5 verification is successful, this parameter value is used for metadata x-obs-meta-md5chksum, for later MD5 verification during download or copy.

p

Optional (additional parameter)

The maximum number of concurrent uploads for a multipart upload

The default value is the value of defaultParallels in the configuration file.

threshold

Optional (additional parameter)

The threshold for enabling multipart upload. If the size of the file or folder to be uploaded is smaller than the threshold, upload it directly. Otherwise, a multipart upload is required.

The default value is the value of defaultBigfileThreshold in the configuration file.

Unit: byte

This parameter value can contain a unit, for example, 1MB (indicating 1,048,576 bytes).

acl

Optional (additional parameter)

The access control policies that can be specified for objects when uploading files.

Possible values are:

• private: Only users granted permissions by the object ACL can access the object.
• public-read: Anyone can read objects in the bucket.
• public-read-write: Anyone can read, write, or delete objects in the bucket.
• bucket-owner-full-control: Only the bucket owner has full control over objects in the bucket.

sc

Optional (additional parameter)

The storage classes of objects that can be specified when uploading files

Possible values are:

• standard: Standard storage class. It features low access latency and high throughput, and is applicable to storing frequently accessed data (multiple accesses per month) or data that is smaller than 1 MB.
• warm: Infrequent Access storage class. It is ideal for storing infrequently accessed (less than 12 times a year) data, but when needed, the access has to be fast.
• cold: Archive storage class. It provides secure, durable, and inexpensive storage for rarely-accessed (once a year) data.

meta

Optional (additional parameter)

The standard or user-defined metadata that can be specified during file upload

• Standard metadata includes Content-Type, Content-Encoding, Cache-Control, Content-Disposition, Content-Language and Expires.
• User-defined metadata must be in the format of key:value. Multiple user-defined metadata items are separated by number signs (#), for example, key1:value1#key2:value2#key3:value3, which indicates that the file uploaded to the bucket contains three groups of user-defined metadata: key1:value1, key2:value2, and key3:value3.

ps

Optional (additional parameter)

The size of each part in a multipart upload task

• The default value is the value of defaultPartSize in the configuration file. The parameter can be set to auto. In this case, obsutil automatically configures the part size for each multipart task based on the source file size.
• Range: 100KB to 5GB
• Unit: byte. This parameter value can contain a unit, for example, 1MB (indicating 1,048,576 bytes).

cpd

Optional (additional parameter)

The folder where the part records reside. The default value is .obsutil_checkpoint, the subfolder in the home directory of the user who executes obsutil commands.

A part record is generated during a multipart upload and saved to the upload subfolder. After the upload succeeds, its part record is deleted automatically. If the upload fails or is suspended, the system attempts to resume the task according to its part record when you perform the upload the next time.

r

Mandatory for uploading a folder (additional parameter)

Optional for uploading multiple files/folders

Recursively uploads all files and subfolders within a folder.

f

Optional for uploading a folder or multiple files/folders (additional parameter)

Runs in force mode.

j

Optional for uploading a folder or multiple files/folders (additional parameter)

The maximum number of concurrent tasks for uploading a folder. The default value is the value of defaultJobs in the configuration file.

encoding-type

Optional (additional parameter)

If the object name contains special characters, set this parameter to url.

This parameter is only supported by obsutil 5.5.12 and later.

msm

Mandatory for uploading multiple files/folders (additional parameter)

Enables the mode for uploading multiple files/folders. Possible values are 1 and 2.

• If msm is set to 1, the source URL indicates a list of file/folder names separated by commas.
• If msm is set to 2, the source URL indicates a file containing a list of file/folder names.
• If the file or folder name already contains commas (,), do not set msm to 1.
• If parameter r is not set, the folders in the list will not be uploaded.

exclude

Optional for uploading a folder or multiple files/folders (additional parameter)

The file matching patterns that are excluded. After this parameter is specified, if the name of the file to be uploaded matches the value of this parameter, the file is skipped. For example, if this parameter is set to *.txt, all files whose names end with .txt are skipped and not uploaded.

• An asterisk (*) matches any number of characters. For instance, abc*.txt matches all files whose names start with abc, contain any number of characters in the middle, and end with .txt.
• A question mark (?) matches any single character. For instance, abc?.txt matches all files whose names start with abc, contain any single character in the middle, and end with .txt.
• You can use \* to represent * and \? to represent ?.
• Multiple exclude parameters can be specified, for example, -exclude=*.xxx -exclude=*.xxx.

Restrictions:

The matching pattern takes effect only for files in the folder.

include

Optional for uploading a folder or multiple files/folders (additional parameter)

The file matching patterns that are included. After this parameter is specified, if the name of the file to be uploaded matches the value of this parameter, the file is uploaded. For example, if this parameter is set to *.txt, all files whose names end with .txt are uploaded.

• An asterisk (*) matches any number of characters. For instance, abc*.txt matches all files whose names start with abc, contain any number of characters in the middle, and end with .txt.
• A question mark (?) matches any single character. For instance, abc?.txt matches all files whose names start with abc, contain any single character in the middle, and end with .txt.
• You can use \* to represent * and \? to represent ?.
• Multiple include parameters can be specified, for example, -include=*.xxx -include=*.xxx.

Restrictions:

• The matching pattern takes effect only for files in the folder.
• If both the exclude and include parameters are configured, the exclude rules are applied first. If the name of a file to be uploaded does not match exclude, OBS checks whether the file name matches include. If yes, the file is uploaded. If no, the file is skipped.

Example:

./obsutil cp /localpath/ obs://test/ -include=/localpath/2022-12-09/* -f -r

This command uploads files that are under localpath and start with /localpath/2022-12-09/ to bucket test.

at

Optional for uploading a folder or multiple files/folders (additional parameter)

Indicates that only the files whose latest access time is within the value of timeRange are uploaded.

This parameter must be used together with timeRange.

disableDirObject

Optional for uploading multiple folders (additional parameter)

Indicates the folders themselves are not uploaded as an object. Configuring this parameter can avoid uploading empty folders to a bucket. If a folder contains files, the files will be uploaded and the original path format is retained.

timeRange

Optional for uploading a folder or multiple files/folders (additional parameter)

The time range matching pattern when uploading files. Only files whose latest modification time is within the configured time range are uploaded.

This pattern has a lower priority than the file matching patterns (exclude/include). That is, the time range matching pattern is executed after the configured file matching patterns.

• Time in the matching pattern is the UTC time.
• The matching time range is represented in time1-time2, where time1 must be earlier than or the same as time2. The time format is yyyyMMddHHmmss.
• Automatic formatting is supported. For example, yyyyMMdd is equivalent to yyyyMMdd000000, and yyyyMM is equivalent to yyyyMM01000000.
• If this parameter is set to *-time2, all files whose last modification time is earlier than time2 are matched. If it is set to time1-*, all files whose last modification time is later than time1 are matched.

mf

Optional (additional parameter)

Indicates that the name matching pattern (include or exclude) and the time matching pattern (timeRange) also take effect on folders.

o

Optional (additional parameter)

The folder that stores the result files. After the command is executed, result files (possibly success, failure, and warning files) will be created in the specified folder.

The default value is .obsutil_output, a subfolder in the user's home directory where obsutil commands are executed.

• A result file should be named as follows: cp_{succeed | failed | warning}_report_time_TaskId.txt.
• By default, the maximum size of a single result file is 30 MB. You can set the maximum size by configuring recordMaxLogSize in the configuration file.
• By default, the maximum number of result files that can be retained is 1,024. You can set the maximum number by configuring recordBackups in the configuration file.
• If there are multiple folders and files and you need to confirm the details of a failed task, refer to the failure result file cp_failed_report_time_TaskId.txt in the result folder and the log files in the log path.

config

Optional (additional parameter)

The user-defined configuration file for executing the current command.

To learn the parameters that can be configured in this file, see Configuration Parameters.

e

Optional (additional parameter)

The endpoint

i

Optional (additional parameter)

The user's AK

k

Optional (additional parameter)

The user's SK

t

Optional (additional parameter)

The user's security token

Parallel

The parameter -p in the request

Jobs

The parameter -j in the request

Threshold

The parameter -threshold in the request

PartSize

The parameter -ps in the request

Exclude

The parameter -exclude in the request

Include

The parameter -include in the request

TimeRange

The parameter -timeRange in the request

VerifyLength

The parameter -vlength in the request

VerifyMd5

The parameter -vmd5 in the request

CheckpointDir

The parameter -cpd in the request

OutputDir

The parameter -o in the request

ArcDir

The parameter -arcDir in the request

Succeed count

The number of successful tasks

Failed count

The number of failed tasks

Skip count

The number of tasks that are skipped during incremental upload, download, or copy, and synchronous upload, download, or copy.

Warning count

The number of tasks that are executed successfully but contain warnings.

Succeed bytes

The number of bytes that are successfully uploaded or downloaded.

max cost

The maximum duration of all tasks, in ms

min cost

The minimum duration of all tasks, in ms

average cost

The average duration of all tasks, in ms

average tps

The average number of tasks completed per second

Task id

The unique ID of an operation, which is used to search for the result file generated for a batch task