Configuration Parameters

Updated on 2024-12-11 GMT+08:00

View PDF

You can set obsutil parameters in the .obsutilconfig file.

Configuration file format:

endpoint=<value>
ak=<value>
sk=<value>
token=<value>
endpointCrr=<value>
akCrr=<value>
skCrr=<value>
tokenCrr=<value>
connectTimeout=<value>
socketTimeout=<value>
maxRetryCount=<value>
maxConnections=<value>
defaultBigfileThreshold=<value>
defaultPartSize=<value>
defaultParallels=<value>
defaultJobs=<value>
defaultJobsCacheCount=<value>
rateLimitThreshold=<value>
sdkLogBackups=<value>
sdkLogLevel=<value>
sdkLogPath=<value>
sdkMaxLogSize=<value>
utilLogBackups=<value>
utilLogLevel=<value>
utilLogPath=<value>
utilMaxLogSize=<value>
writeBufferIoSize=<value>
readBufferIoSize=<value>
recordMaxLogSize=<value>
recordBackups=<value>
humanReadableFormat=<value>
showProgressBar=<value>
showStartTime=<value>
helpLanguage=<value>
defaultTempFileDir=<value>
checkSourceChange=<value>
skipCheckEmptyFolder=<value>
fsyncForDownload=<value>
memoryEconomicalScanForUpload=<value>
forceOverwriteForDownload=<value>
panicForSymbolicLinkCircle=<value>
autoChooseSecurityProvider=<value>
fastFailThreshold=<value>
abortHttpStatusForResumableTasks=<value>
showBytesForCopy=<value>
proxyUrl=<value>
faultTolerantMode=<value>

Table 1 describes the parameters.

**Table 1** obsutil configuration parameters
Parameter	Optional or Mandatory	Description	Recommended Value
endpoint	Mandatory	Endpoint for accessing OBS, which can contain the protocol type, domain name, and port number (optional), for example, https://your-endpoint:443. (For security purposes, you are advised to use HTTPS. The port number 443 can be omitted.) If you want to access OBS in the CN-Hong Kong region, the address you actually visit is https://obs.ap-southeast-1.myhuaweicloud.com. NOTE: To view the endpoints and regions available for OBS, see Regions and Endpoints. If the configured endpoint does not contain any protocol, the HTTPS protocol is used by default.	N/A
ak	Optional	Access key ID. This parameter is optional when autoChooseSecurityProvider is set to true. NOTE: For more information, visit the access key management console. After you run obsutil for the first time, the tool encrypts the AK to ensure the key security.	N/A
sk	Optional	Secret access key. This parameter is optional when autoChooseSecurityProvider is set to true. NOTE: For more information, visit the access key management console. After you run obsutil for the first time, the tool encrypts the SK to ensure the key security.	N/A
token	Optional	Security token. If this parameter is empty, the security token is not set. When temporary AK and SK are used to access OBS, the token must be carried.	N/A
endpointCrr	Optional	Endpoint for accessing OBS in the region where the source bucket resides when the client-side cross-region replication function is enabled, which can contain the protocol type, domain name, and port number, for example, https://your-endpoint:443. (For security purposes, you are advised to use HTTPS. The port number 443 can be omitted.) NOTE: To view the endpoints and regions available for OBS, see Regions and Endpoints. If the configured endpoint does not contain any protocol, the HTTPS protocol is used by default.	N/A
akCrr	Optional	AK for the source bucket when the client-side cross-region replication function is enabled	N/A
skCrr	Optional	SK for the source bucket when the client-side cross-region replication function is enabled	N/A
tokenCrr	Optional	Security token used by the source bucket when the client-side cross-region replication is used. If this parameter is left blank, no security token is set. This parameter is mandatory when temporary access keys are used for access.	N/A
connectTimeout	Optional	Timeout interval for establishing an HTTP/HTTPS connection, in seconds. The default value is 30.	[5, 120]
socketTimeout	Optional	Timeout interval for reading and writing data, in seconds. The default value is 310.	[5, 600]
maxRetryCount	Optional	Maximum number of retry attempts. The default value is 3. NOTE: When an OBS request completes but HTTP status code 408 or 5XX is returned, or when a timeout error occurs in an OBS request, the request is retried.	[0, 5]
maxConnections	Optional	Maximum number of HTTP connections that can be accessed. The default value is 1000.	N/A
defaultBigfileThreshold	Optional	Threshold for triggering a multipart upload, in bytes. If the size of a file to be uploaded, downloaded, or copied is greater than the specified threshold, the file will be uploaded, downloaded, or copied using a multipart upload. The default value is 50MB. NOTE: This parameter value can contain a unit, for example, 1MB (indicating 1048576 bytes).	Greater than 5MB
defaultPartSize	Optional	Size of each part, in bytes. The default value is auto. NOTE: For a multipart upload and copy, the value ranges from 100KB to 5GB. For multipart download, the value is unrestricted. This parameter value can contain a unit, for example, 1MB (indicating 1048576 bytes). If this parameter is set to auto. In this case, obsutil automatically sets the part size for each multipart task based on the source object size.	[9MB, 100MB]
defaultParallels	Optional	Maximum number of concurrent tasks in the multipart mode. The default value is 5.	Set this parameter according to Fine-Tuning obsutil Performance.
defaultJobs	Optional	Maximum number of concurrent tasks in batches. The default value is 5. NOTE: Batch tasks include uploading, downloading, and copying folders, as well as restoring and deleting objects in batches.	[1, 50]
defaultJobsCacheCount	Optional	Cache size of a batch task queue, indicating the maximum number of tasks that can be cached. The default value is 1000000. NOTE: More cached tasks consume more memory resources. Therefore, you are advised to adjust the value of this parameter based on site requirements.	Default value
rateLimitThreshold	Optional	Traffic control threshold of an upload or a download request, in bytes per second. The default value is 0, indicating that traffic is not limited. The minimum value is 10KB. NOTE: This parameter value can contain a unit, for example, 1MB (indicating 1048576 bytes).	Greater than 100KB
sdkLogBackups	Optional	Maximum number of SDK log files that can be retained. The default value is 10.	N/A
sdkLogLevel	Optional	SDK log level. Possible values are: DEBUG INFO WARN ERROR The default value is WARN.	N/A
sdkLogPath	Optional	Absolute path of SDK logs. The value must be a file path. The default value is the path of the obssdk.log file in the subfolder obsutil_log of the user's home directory (HOME in Linux or macOS and C:\Users\<Username> in Windows). NOTE: If this parameter is left blank, no SDK log is generated. The path must be a file path and cannot be a folder path. After the SDK log function is enabled, all logs of requests to OBS are saved in the SDK log file for problem analysis and location. Ensure that the user who runs the command has the read and write permissions on the path. NOTICE: If multiple obsutil processes are running at the same time, log files may fail to be written concurrently or may be lost. In this case, add parameter -config when running commands to configure an independent configuration file for each process.	N/A
sdkMaxLogSize	Optional	Size of an SDK log file, in bytes. The default value is 30MB. NOTE: This parameter value can contain a unit, for example, 1MB (indicating 1048576 bytes).	[10MB, 100MB]
utilLogBackups	Optional	Maximum number of obsutil log files that can be retained. The default value is 10.	N/A
utilLogLevel	Optional	obsutil log level. Possible values are: DEBUG INFO WARN ERROR The default value is INFO.	N/A
utilLogPath	Optional	Absolute path of obsutil logs. The value must be a file path. The default value is the path of the obsutil.log file in the subfolder .obsutil_log of the user's home directory (HOME in Linux or macOS and C:\Users\<Username> in Windows). NOTE: If this parameter is left blank, no obsutil log is generated. The path must be a file path and cannot be a folder path. After the obsutil log function is enabled, all logs generated during commands executing are saved in the obsutil log file for problem analysis and location. Ensure that the user who runs the command has the read and write permissions on the path. NOTICE: If multiple obsutil processes are running at the same time, log files may fail to be written concurrently. In this case, add parameter -config when running commands to configure an independent configuration file for each process.	N/A
utilMaxLogSize	Optional	Size of an obsutil log file, in bytes. The default value is 30MB. NOTE: This parameter value can contain a unit, for example, 1MB (indicating 1048576 bytes).	[10MB, 100MB]
writeBufferIoSize	Optional	Size of the cache for downloading data, in bytes. The default value is 65536. NOTE: Set this parameter based on site requirements. If the size of the file to be downloaded is large, set this parameter to a large value. This parameter value can contain a unit, for example, 1MB (indicating 1048576 bytes).	N/A
readBufferIoSize	Optional	Size of the cache for uploading data, in bytes. The default value is 8192. NOTE: Set this parameter based on site requirements. If a large number of small files are uploaded, set this parameter to a small value. If large files are uploaded, set this parameter to a large value. This parameter value can contain a unit, for example, 1MB (indicating 1048576 bytes).	[4096, 65536]
recordMaxLogSize	Optional	Size of a result list containing success, failure, or warning lists in a batch task, in bytes. The default value is 30MB. NOTE: This parameter value can contain a unit, for example, 1MB (indicating 1048576 bytes).	[5MB, 100MB]
recordBackups	Optional	Maximum number of result lists of successful or failed batch tasks that can be retained. The default value is 1024.	N/A
humanReadableFormat	Optional	Indicates whether to convert the number of bytes in the object listing result and result list content to the human-readable format. The default value is true.	N/A
showProgressBar	Optional	Indicates whether to display the progress bar on the console. The value true indicates that the progress bar is displayed. The default value is true.	N/A
showStartTime	Optional	Indicates whether to display the start time on the console. The value true indicates that start time is displayed. The default value is true.	N/A
helpLanguage	Optional	Language of the help documents. Options are as follows: Chinese English The default value is English.	N/A
defaultTempFileDir	Optional	Indicates the directory for storing temporary files during download. The default value is the .obsutil_tempfile subfolder in the user directory (HOME in Linux or macOS and C:\Users\<Username> in Windows). NOTE: Temporary files generated during multipart download are stored in this directory. Therefore, ensure that the user who executes obsutil has the write permission on the path. The available space of the partition where the path is located must be greater than the size of the objects to be downloaded.	N/A
checkSourceChange	Optional	Indicates whether to check the change of source files or objects during upload/download/copy. The value true indicates that the function is enabled.	N/A
skipCheckEmptyFolder	Optional	Indicates whether to skip checking empty folders on the OBS server during download. true indicates to skip the check. The default value is false. NOTICE: If this parameter is set to true, the directory structure downloaded to your local PC may be different from that in OBS.	N/A
fsyncForDownload	Optional	Indicates whether to forcibly synchronize memory data to disks during download. The value true indicates to enable forcible synchronization. The default value is false. NOTE: Set this parameter to true for scenarios that require high data reliability. If this parameter is set to true, the download performance will be deteriorated. Therefore, exercise caution when using this parameter.	N/A
memoryEconomicalScanForUpload	Optional	Indicates whether to use the scanning mode that occupies less memory space when uploading a folder. The value true indicates using this method. The default value is true.	N/A
forceOverwriteForDownload	Optional	Indicates to forcibly overwrite the local executable file (even if the local executable file is running) when downloading objects to the Linux OS or macOS. The value true means to overwrite, and the default value is true.	N/A
panicForSymbolicLinkCircle	Optional	Indicates the processing method after a symbolic link loop is detected during upload. The value false indicates that errors are only recorded. The value true indicates that panic is triggered. The default value is false.	N/A
autoChooseSecurityProvider	Optional	Indicates whether to automatically obtain the access keys from the configuration file, environment variables, and ECS APIs in sequence. The default value is false. NOTE: For details about this parameter, see Configuring Auto Obtaining of Access Keys for obsutil.	N/A
fastFailThreshold	Optional	Threshold for fast failure upon 4XX errors of batch tasks. When the number of 4XX errors exceeds the threshold, the fast failure process is triggered. All tasks that are not executed or being scanned are suspended. The default value is 5. NOTE: The fast failure mechanism is to avoid excessive traffic generated during batch task execution. To start a fast failure as soon as possible, set this parameter to 0 or -1, indicating that the fast failure process starts immediately whenever a 4XX error occurs.	N/A
abortHttpStatusForResumableTasks	Optional	HTTP status codes for fast interruption of multipart upload, download, and copy tasks. If a sub-task of a multipart task receives an HTTP code that falls into this range, the multipart task is immediately interrupted. The default value is 401,403,404,405,409. NOTE: Multiple HTTP status codes can be carried and separated by commas (,), for example, 401,403,404. The status code must be a 4XX HTTP status code. Other status codes are ignored.	Default value
showBytesForCopy	Optional	Indicates whether the progress bar displays the rate in bytes when objects are copied between buckets. The default value is false.	N/A
proxyUrl	Optional	HTTP proxy example: http://username:password@your-proxy:8080 NOTE: HTTP proxy format: http://[Username:Password@]Proxy server address:Port number. The Username and Password are optional. The proxyUrl parameter and system environment variables are in the following priority order: proxyUrl > HTTPS_PROXY > HTTP_PROXY. The user name and password cannot contain colons (:) and at signs (@), which will result in parsing errors.	N/A
faultTolerantMode	Optional	Indicates whether to ignore the panic caused by read operations. Default value false indicates that the panic is not ignored.	N/A

NOTE:

Set parameters with N/A as the recommended value based on your needs.
You are advised to specify sdkLogPath and utilLogPath to enable SDK logging and obsutil logging.
The values of defaultBigfileThreshold, defaultPartSize, rateLimitThreshold, sdkMaxLogSize, utilMaxLogSize, recordMaxLogSize, readBufferIoSize, and writeBufferIoSize can contain a unit, for example, 1MB (indicating 1048576 bytes).

Previous topic: Can I Rename an Object or a Folder?

Feedback

Was this page helpful?

Helpful Not helpful

Provide feedback

Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.

The system is busy. Please try again later.

Which of the following issues have you encountered?

Content is inconsistent with the product UI

Unclear descriptions

Lack of examples or code

Incorrect steps

Can't find what I need

Lack of best practices

Feedback (optional)

0/500

Select at least one type of issue, and enter your comments or suggestions.

Enter a maximum of 500 characters.

Submit Cancel

For any further questions, feel free to contact us through the chatbot.

Chatbot