Listing Objects in a Bucket (SDK for Python)
Function
This API lists some or all of the objects in a bucket. You can use parameters such as the prefix, number of returned objects, and start position to list objects that meet specified criteria. Returned objects are listed in alphabetical order by object name.
Restrictions
- A maximum of 1,000 objects can be listed for each API call.
- To list objects in a bucket, you must be the bucket owner or have the required permission (obs:bucket:ListBucket in IAM or ListBucket in a bucket policy). For details, see Introduction to OBS Access Control, IAM Custom Policies, and Creating a Custom Bucket Policy.
- The mapping between OBS regions and endpoints must comply with what is listed in Regions and Endpoints.
Method
ObsClient.listObjects(bucketName, prefix, marker, max_keys, delimiter)
Request Parameters
Parameter |
Type |
Mandatory (Yes/No) |
Description |
---|---|---|---|
bucketName |
str |
Yes |
Explanation: Bucket name Restrictions:
Default value: None |
prefix |
str |
No |
Explanation: Name prefix that the objects to be listed must contain. Assume that you have the following objects: logs/day1, logs/day2, logs/day3, and ExampleObject.jpg. If you specify logs/ as the prefix, logs/day1, logs/day2, and logs/day3 will be returned. If you leave this parameter blank, all objects in the bucket will be returned. Value range: The value must contain 1 to 1,024 characters. Default value: None |
marker |
str |
No |
Explanation: Name of the object to start with when listing objects in a bucket. All objects following this object are listed in alphabetical order by object name. Assume that you have the following objects: test/a, test/b, test/c, and test/d. If you specify test/b as the marker, test/c and test/d are returned. Restrictions: This parameter is only used for listing objects with a single version. Value range: The value must contain 1 to 1,024 characters. Default value: None |
max_keys |
int |
No |
Explanation: The maximum number of objects returned in the response in alphabetical order Value range: The value ranges from 1 to 1000. If the specified value is greater than 1000, only 1,000 objects are returned. Default value: 1000 |
delimiter |
str |
No |
Explanation: This parameter is used to group object names. If a prefix is specified, objects with the same string from the prefix to the first delimiter are grouped into one CommonPrefix. If no prefix is specified, objects with the same string from the first character to the first delimiter are grouped into one CommonPrefix. Assume that a bucket has objects abcd, abcde, and bbcde in it. If delimiter is set to d and prefix is set to a, objects abcd and abcde are grouped into a CommonPrefix with abcd as the prefix. If only delimiter is set to d, objects abcd and abcde are grouped into a CommonPrefix with abcd as the prefix, and bbcde is grouped separately into another CommonPrefix with bbcd as the prefix. For a parallel file system, if this parameter is not specified, all the content in the directory is recursively listed by default, including the content in subdirectories. In big data scenarios, parallel file systems usually have multiple directory levels and each directory level has a large number of files. In such case, you are advised to configure [delimiter=/] to list the content in the current directory but exclude the content in subdirectories, thereby making the listing more efficient. Value range: The value must contain 1 to 1,024 characters. Default value: None |
encoding_type |
str |
No |
Explanation: Encoding type for some elements in the response. If delimiter, key_marker, prefix, nextKeyMarker, and key contain control characters that are not supported by the XML 1.0 standard, you can set encoding_type to encode delimiter, key_marker, prefix (including the Prefix in commonPrefixes), nextKeyMarker, and key in the response. Value range: url Default value: None. If you leave this parameter blank, encoding is not applied to elements. |
Responses
Type |
Description |
---|---|
Explanation: SDK common results |
Parameter |
Type |
Description |
---|---|---|
status |
int |
Explanation: HTTP status code Value range: A status code is a group of digits ranging from 2xx (indicating successes) to 4xx or 5xx (indicating errors). It indicates the status of a response. For more information, see Status Code. Default value: None |
reason |
str |
Explanation: Reason description. Default value: None |
errorCode |
str |
Explanation: Error code returned by the OBS server. If the value of status is less than 300, this parameter is left blank. Default value: None |
errorMessage |
str |
Explanation: Error message returned by the OBS server. If the value of status is less than 300, this parameter is left blank. Default value: None |
requestId |
str |
Explanation: Request ID returned by the OBS server Default value: None |
indicator |
str |
Explanation: Error indicator returned by the OBS server. Default value: None |
hostId |
str |
Explanation: Requested server ID. If the value of status is less than 300, this parameter is left blank. Default value: None |
resource |
str |
Explanation: Error source (a bucket or an object). If the value of status is less than 300, this parameter is left blank. Default value: None |
header |
list |
Explanation: Response header list, composed of tuples. Each tuple consists of two elements, respectively corresponding to the key and value of a response header. Default value: None |
body |
object |
Explanation: Result content returned after the operation is successful. If the value of status is larger than 300, this parameter is left blank. The value varies with the API being called. For details, see Bucket-Related APIs (SDK for Python) and Object-Related APIs (SDK for Python). Default value: None |
GetResult.body Type |
Description |
---|---|
Explanation: See Table 5. |
Parameter |
Type |
Description |
---|---|---|
name |
str |
Explanation: Bucket name Restrictions:
Default value: None |
location |
str |
Explanation: Region where a bucket is located Restrictions: If the used endpoint is obs.myhuaweicloud.com, this parameter is not required. If any other endpoint is used, this parameter is required. Value range: To learn about valid regions and endpoints, see Regions and Endpoints. An endpoint is the request address for calling an API. Endpoints vary depending on services and regions. To obtain the regions and endpoints, contact the enterprise administrator. Default value: If obs.myhuaweicloud.com is used as the endpoint and no region is specified, cn-north-1 (the CN North-Beijing1 region) is used by default. |
prefix |
str |
Explanation: Name prefix that the objects to be listed must contain. Assume that you have the following objects: logs/day1, logs/day2, logs/day3, and ExampleObject.jpg. If you specify logs/ as the prefix, logs/day1, logs/day2, and logs/day3 will be returned. If you leave this parameter blank, all objects in the bucket will be returned. Value range: The value must contain 1 to 1,024 characters. Default value: None |
marker |
str |
Explanation: Name of the object to start with when listing objects in a bucket. All objects following this object are listed in alphabetical order by object name. Assume that you have the following objects: test/a, test/b, test/c, and test/d. If you specify test/b as the marker, test/c and test/d are returned. Restrictions: This parameter is only used for listing objects with a single version. Value range: The value must contain 1 to 1,024 characters. Default value: None |
delimiter |
str |
Explanation: This parameter is used to group object names. If a prefix is specified, objects with the same string from the prefix to the first delimiter are grouped into one CommonPrefix. If no prefix is specified, objects with the same string from the first character to the first delimiter are grouped into one CommonPrefix. Assume that a bucket has objects abcd, abcde, and bbcde in it. If delimiter is set to d and prefix is set to a, objects abcd and abcde are grouped into a commonPrefix with abcd as the prefix. If only delimiter is set to d, objects abcd and abcde are grouped into a commonPrefix with abcd as the prefix, and bbcde is grouped separately into another commonPrefix with bbcd as the prefix. Value range: The value must contain 1 to 1,024 characters. Default value: None |
max_keys |
int |
Explanation: The maximum number of objects returned in the response in alphabetical order Value range: The value ranges from 1 to 1000. If the specified value is greater than 1000, only 1,000 objects are returned. Default value: 1000 |
is_truncated |
bool |
Explanation: Whether all results are returned in the response Value range:
Default value: None |
next_marker |
str |
Explanation: Object name to start with upon next request for listing objects Value range: An object name string Default value: None |
contents |
list of Content |
Explanation: List of objects in the bucket Value range: See Table 6. |
commonPrefixs |
list of CommonPrefix |
Explanation: List of object name prefixes grouped according to the delimiter parameter (if specified) Value range: See Table 8. |
encoding_type |
str |
Explanation: Encoding type for some elements in the response. If delimiter, key_marker, prefix, nextKeyMarker, and key contain control characters that are not supported by the XML 1.0 standard, you can set encoding_type to encode delimiter, key_marker, prefix (including the Prefix in commonPrefixes), nextKeyMarker, and key in the response. Value range: url Default value: None. If you leave this parameter blank, encoding is not applied to elements. |
Parameter |
Type |
Description |
---|---|---|
key |
str |
Explanation: Object name. An object is uniquely identified by an object name in a bucket. An object name is a complete path that does not contain the bucket name. For example, if the address for accessing the object is examplebucket.obs.ap-southeast-1.myhuaweicloud.com/folder/test.txt, the object name is folder/test.txt. Value range: The value must contain 1 to 1,024 characters. Default value: None |
lastModified |
str |
Explanation: Time when the last modification was made to the object Restrictions: The time must be in the ISO8601 format. Example: 2018-01-01T00:00:00.000Z Default value: None |
etag |
str |
Explanation: Base64-encoded, 128-bit MD5 value of an object. ETag is the unique identifier of the object contents and is used to determine whether the contents of an object are changed. For example, if the ETag value is A when an object is uploaded and is B when the object is downloaded, this indicates the contents of the object are changed. The ETag reflects changes only to the contents of an object, not its metadata. Objects created by the upload and copy operations have unique ETags after being encrypted using MD5. Restrictions: If an object is encrypted using server-side encryption, the ETag is not the MD5 value of the object. Value range: The value must contain 32 characters. Default value: None |
size |
int |
Explanation: Object size Value range: The value ranges from 0 TB to 48.8 TB, in bytes. Default value: None |
owner |
Explanation: Object owner Value range: See Table 7. |
|
storageClass |
str |
Explanation: Storage class of the object Value range:
Default value: None |
isAppendable |
bool |
Explanation: Whether the object is appendable Value range: True: The object is appendable. False: The object is not appendable. Default value: None |
Parameter |
Type |
Mandatory (Yes/No) |
Description |
---|---|---|---|
owner_id |
str |
Yes if used as a request parameter |
Explanation: Account (domain) ID of the owner Value range: To obtain the account ID, see How Do I Get My Account ID and IAM User ID? (SDK for Python) Default value: None |
owner_name |
str |
No if used as a request parameter |
Explanation: Account name of the owner Value range: To obtain the account ID, see How Do I Get My Account ID and IAM User ID? (SDK for Python) Default value: None |
Parameter |
Type |
Description |
---|---|---|
prefix |
str |
Explanation: List of object name prefixes grouped according to the delimiter parameter (if specified) Value range: The value must contain 1 to 1,024 characters. Default value: None |
Parameter |
Type |
Description |
---|---|---|
STANDARD |
Standard storage class |
Explanation: Features low access latency and high throughput and is used for storing massive, frequently accessed (multiple times a month) or small objects (< 1 MB) requiring quick response. |
WARM |
Infrequent Access storage class |
Explanation: Used for storing data that is semi-frequently accessed (fewer than 12 times a year) but is instantly available when needed. |
COLD |
Archive storage class |
Explanation: Used for storing rarely accessed (once a year) data. |
Code Examples
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 |
from obs import ObsClient import os import traceback # Obtain an AK and SK pair using environment variables or import the AK and SK pair in other ways. Using hard coding may result in leakage. # Obtain an AK and SK pair on the management console. For details, see https://support.huaweicloud.com/intl/en-us/usermanual-ca/ca_01_0003.html. ak = os.getenv("AccessKeyID") sk = os.getenv("SecretAccessKey") # (Optional) If you use a temporary AK and SK pair and a security token to access OBS, obtain them from environment variables. security_token = os.getenv("SecurityToken") # Set server to the endpoint corresponding to the bucket. Here uses CN-Hong Kong as an example. Replace it with the one in use. server = "https://obs.ap-southeast-1.myhuaweicloud.com" # Create an obsClient instance. # If you use a temporary AK and SK pair and a security token to access OBS, you must specify security_token when creating an instance. obsClient = ObsClient(access_key_id=ak, secret_access_key=sk, server=server) try: bucketName = "examplebucket" # Specify an object prefix. prefix = 'test/' # Specify the maximum number of objects to be listed at a time. 100 is used as an example. max_keys = 100 # List objects in the bucket. resp = obsClient.listObjects(bucketName, prefix, max_keys=max_keys, encoding_type='url') # If status code 2xx is returned, the API is called successfully. Otherwise, the API call fails. if resp.status < 300: print('List Objects Succeeded') print('requestId:', resp.requestId) print('name:', resp.body.name) print('prefix:', resp.body.prefix) print('max_keys:', resp.body.max_keys) print('is_truncated:', resp.body.is_truncated) index = 1 for content in resp.body.contents: print('object [' + str(index) + ']') print('key:', content.key) print('lastModified:', content.lastModified) print('etag:', content.etag) print('size:', content.size) print('storageClass:', content.storageClass) print('owner_id:', content.owner.owner_id) print('owner_name:', content.owner.owner_name) index += 1 else: print('List Objects Failed') print('requestId:', resp.requestId) print('errorCode:', resp.errorCode) print('errorMessage:', resp.errorMessage) except: print('List Objects Failed') print(traceback.format_exc()) |
Sample Code (for Listing over 1,000 Objects)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
try: max_num = 1000 mark = None index = 1 while True: resp = obsClient.listObjects('bucketname', marker=mark, max_keys=max_num, encoding_type='url') if resp.status < 300: print('requestId:', resp.requestId) print('name:', resp.body.name) print('prefix:', resp.body.prefix) print('max_keys:', resp.body.max_keys) print('is_truncated:', resp.body.is_truncated) for content in resp.body.contents: print('object [' + str(index) + ']') print('key:', content.key) print('lastModified:', content.lastModified) print('etag:', content.etag) print('size:', content.size) print('storageClass:', content.storageClass) print('owner_id:', content.owner.owner_id) print('owner_name:', content.owner.owner_name) index += 1 if resp.body.is_truncated is True: mark = resp.body.next_marker else: break else: print('errorCode:', resp.errorCode) print('errorMessage:', resp.errorMessage) break except: import traceback print(traceback.format_exc()) |
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot