Parameter Description
You can set obsutil parameters in the .obsutilconfig file.
Configuration file format:
endpoint=<value> ak=<value> sk=<value> token=<value> endpointCrr=<value> akCrr=<value> skCrr=<value> tokenCrr=<value> connectTimeout=<value> socketTimeout=<value> maxRetryCount=<value> maxConnections=<value> defaultBigfileThreshold=<value> defaultPartSize=<value> defaultParallels=<value> defaultJobs=<value> defaultJobsCacheCount=<value> rateLimitThreshold=<value> sdkLogBackups=<value> sdkLogLevel=<value> sdkLogPath=<value> sdkMaxLogSize=<value> utilLogBackups=<value> utilLogLevel=<value> utilLogPath=<value> utilMaxLogSize=<value> writeBufferIoSize=<value> readBufferIoSize=<value> recordMaxLogSize=<value> recordBackups=<value> humanReadableFormat=<value> showProgressBar=<value> showStartTime=<value> helpLanguage=<value> defaultTempFileDir=<value> checkSourceChange=<value> skipCheckEmptyFolder=<value> fsyncForDownload=<value> memoryEconomicalScanForUpload=<value> forceOverwriteForDownload=<value> panicForSymbolicLinkCircle=<value> fastFailThreshold=<value> abortHttpStatusForResumableTasks=<value> showBytesForCopy=<value> proxyUrl=<value> faultTolerantMode=<value>
Table 1 describes the parameters.
Parameter |
Optional or Mandatory |
Description |
Recommended Value |
---|---|---|---|
endpoint |
Mandatory |
Endpoint for accessing OBS, which can contain the protocol type, domain name, and port number (optional), for example, https://your-endpoint:80. For security purposes, you are advised to use HTTPS.
NOTE:
|
N/A |
ak |
Mandatory |
Access key ID.
NOTE:
|
N/A |
sk |
Mandatory |
Secret access key.
NOTE:
|
N/A |
token |
Optional |
Security token. If this parameter is empty, the security token is not set. When temporary AK and SK are used to access OBS, the token must be carried. |
N/A |
endpointCrr |
Optional |
Endpoint for accessing OBS in the region where the source bucket resides when the client-side cross-region replication function is enabled, which can contain the protocol type, domain name, and port number, for example, https://your-endpoint:80.
NOTE:
|
N/A |
akCrr |
Optional |
AK for the source bucket when the client-side cross-region replication function is enabled |
N/A |
skCrr |
Optional |
SK for the source bucket when the client-side cross-region replication function is enabled |
N/A |
tokenCrr |
Optional |
Security token used by the source bucket when the client-side cross-region replication is used. If this parameter is left blank, no security token is set. This parameter is mandatory when temporary access keys are used for access. |
N/A |
connectTimeout |
Optional |
Timeout interval for establishing an HTTP/HTTPS connection, in seconds. The default value is 30. |
[5, 120] |
socketTimeout |
Optional |
Timeout interval for reading and writing data, in seconds. The default value is 310. |
[5, 600] |
maxRetryCount |
Optional |
Maximum number of retry attempts. The default value is 3.
NOTE:
When an OBS request completes but HTTP status code 408 or 5XX is returned, or when a timeout error occurs in an OBS request, the request is retried. |
[0, 5] |
maxConnections |
Optional |
Maximum number of HTTP connections that can be accessed. The default value is 1000. |
N/A |
defaultBigfileThreshold |
Optional |
Threshold for triggering a multipart upload, in bytes. If the size of a file to be uploaded, downloaded, or copied is greater than the specified threshold, the file will be uploaded, downloaded, or copied using a multipart upload. The default value is 50MB.
NOTE:
This parameter value can contain a unit, for example, 1MB (indicating 1048576 bytes). |
Greater than 5MB |
defaultPartSize |
Optional |
Size of each part, in bytes. The default value is auto.
NOTE:
|
[9MB, 100MB] |
defaultParallels |
Optional |
Maximum number of concurrent tasks in the multipart mode. The default value is 5. |
Set this parameter according to Fine-Tuning obsutil Performance. |
defaultJobs |
Optional |
Maximum number of concurrent tasks in batches. The default value is 5.
NOTE:
Batch tasks include uploading, downloading, and copying folders, as well as deleting objects in batches. |
[1, 50] |
defaultJobsCacheCount |
Optional |
Cache size of a batch task queue, indicating the maximum number of tasks that can be cached. The default value is 1000000.
NOTE:
More cached tasks consume more memory resources. Therefore, you are advised to adjust the value of this parameter based on site requirements. |
Default value |
rateLimitThreshold |
Optional |
Traffic control threshold of an upload or a download request, in bytes per second. The default value is 0, indicating that traffic is not limited. The minimum value is 10KB.
NOTE:
This parameter value can contain a unit, for example, 1MB (indicating 1048576 bytes). |
Greater than 100KB |
sdkLogBackups |
Optional |
Maximum number of SDK log files that can be retained. The default value is 10. |
N/A |
sdkLogLevel |
Optional |
SDK log level. Possible values are:
The default value is WARN. |
N/A |
sdkLogPath |
Optional |
Absolute path of SDK logs. The value must be a file path. The default value is the path of the obssdk.log file in the subfolder obsutil_log of the user's home directory (HOME in Linux or macOS and C:\Users\<Username> in Windows).
NOTE:
NOTICE:
If multiple obsutil processes are running at the same time, log files may fail to be written concurrently or may be lost. In this case, add parameter -config when running commands to configure an independent configuration file for each process. |
N/A |
sdkMaxLogSize |
Optional |
Size of an SDK log file, in bytes. The default value is 30MB.
NOTE:
This parameter value can contain a unit, for example, 1MB (indicating 1048576 bytes). |
[10MB, 100MB] |
utilLogBackups |
Optional |
Maximum number of obsutil log files that can be retained. The default value is 10. |
N/A |
utilLogLevel |
Optional |
obsutil log level. Possible values are:
The default value is INFO. |
N/A |
utilLogPath |
Optional |
Absolute path of obsutil logs. The value must be a file path. The default value is the path of the obsutil.log file in the subfolder .obsutil_log of the user's home directory (HOME in Linux or macOS and C:\Users\<Username> in Windows).
NOTE:
NOTICE:
If multiple obsutil processes are running at the same time, log files may fail to be written concurrently. In this case, add parameter -config when running commands to configure an independent configuration file for each process. |
N/A |
utilMaxLogSize |
Optional |
Size of an obsutil log file, in bytes. The default value is 30MB.
NOTE:
This parameter value can contain a unit, for example, 1MB (indicating 1048576 bytes). |
[10MB, 100MB] |
writeBufferIoSize |
Optional |
Size of the cache for downloading data, in bytes. The default value is 65536.
NOTE:
|
N/A |
readBufferIoSize |
Optional |
Size of the cache for uploading data, in bytes. The default value is 8192.
NOTE:
|
[4096, 65536] |
recordMaxLogSize |
Optional |
Size of a result list containing success, failure, or warning lists in a batch task, in bytes. The default value is 30MB.
NOTE:
This parameter value can contain a unit, for example, 1MB (indicating 1048576 bytes). |
[5MB, 100MB] |
recordBackups |
Optional |
Maximum number of result lists of successful or failed batch tasks that can be retained. The default value is 1024. |
N/A |
humanReadableFormat |
Optional |
Indicates whether to convert the number of bytes in the object listing result and result list content to the human-readable format. The default value is true. |
N/A |
showProgressBar |
Optional |
Indicates whether to display the progress bar on the console. The value true indicates that the progress bar is displayed. The default value is true. |
N/A |
showStartTime |
Optional |
Indicates whether to display the start time on the console. The value true indicates that start time is displayed. The default value is true. |
N/A |
helpLanguage |
Optional |
Language of the help documents. Options are as follows:
The default value is English. |
N/A |
defaultTempFileDir |
Optional |
Indicates the directory for storing temporary files during download. The default value is the .obsutil_tempfile subfolder in the user directory (HOME in Linux or macOS and C:\Users\<Username> in Windows).
NOTE:
|
N/A |
checkSourceChange |
Optional |
Indicates whether to check the change of source files or objects during upload/download/copy. The value true indicates that the function is enabled. |
N/A |
skipCheckEmptyFolder |
Optional |
Indicates whether to skip checking empty folders on the OBS server during download. true indicates to skip the check. The default value is false.
NOTICE:
If this parameter is set to true, the directory structure downloaded to your local PC may be different from that in OBS. |
N/A |
fsyncForDownload |
Optional |
Indicates whether to forcibly synchronize memory data to disks during download. The value true indicates to enable forcible synchronization. The default value is false.
NOTE:
|
N/A |
memoryEconomicalScanForUpload |
Optional |
Indicates whether to use the scanning mode that occupies less memory space when uploading a folder. The value true indicates using this method. The default value is true. |
N/A |
forceOverwriteForDownload |
Optional |
Indicates to forcibly overwrite the local executable file (even if the local executable file is running) when downloading objects to the Linux OS or macOS. The value true means to overwrite, and the default value is true. |
N/A |
panicForSymbolicLinkCircle |
Optional |
Indicates the processing method after a symbolic link loop is detected during upload. The value false indicates that errors are only recorded. The value true indicates that panic is triggered. The default value is false. |
N/A |
fastFailThreshold |
Optional |
Threshold for fast failure upon 4XX errors of batch tasks. When the number of 4XX errors exceeds the threshold, the fast failure process is triggered. All tasks that are not executed or being scanned are suspended. The default value is 5.
NOTE:
The fast failure mechanism is to avoid excessive traffic generated during batch task execution. To start a fast failure as soon as possible, set this parameter to 0 or -1, indicating that the fast failure process starts immediately whenever a 4XX error occurs. |
N/A |
abortHttpStatusForResumableTasks |
Optional |
HTTP status codes for fast interruption of multipart upload, download, and copy tasks. If a sub-task of a multipart task receives an HTTP code that falls into this range, the multipart task is immediately interrupted. The default value is 401,403,404,405,409.
NOTE:
|
Default value |
showBytesForCopy |
Optional |
Indicates whether the progress bar displays the rate in bytes when objects are copied between buckets. The default value is false. |
N/A |
proxyUrl |
Optional |
HTTP proxy example: http://username:password@your-proxy:8080
NOTE:
|
N/A |
faultTolerantMode |
Optional |
Indicates whether to ignore the panic caused by read operations. Default value false indicates that the panic is not ignored. |
N/A |
![](https://support.huaweicloud.com/intl/en-us/tr-central-201-utiltg-obs/public_sys-resources/note_3.0-en-us.png)
- Set parameters with N/A as the recommended value based on your needs.
- You are advised to specify sdkLogPath and utilLogPath to enable SDK logging and obsutil logging.
- The values of defaultBigfileThreshold, defaultPartSize, rateLimitThreshold, sdkMaxLogSize, utilMaxLogSize, recordMaxLogSize, readBufferIoSize, and writeBufferIoSizecan can contain a unit, for example, 1MB (indicating 1048576 bytes).
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot