Listing Objects in a Bucket (SDK for Python)_Object Storage Service

Function

This API lists some or all of the objects in a bucket. You can use parameters such as the prefix, number of returned objects, and start position to list objects that meet specified criteria. Returned objects are listed in alphabetical order by object name.

Restrictions

A maximum of 1,000 objects can be listed for each API call.
To list objects in a bucket, you must be the bucket owner or have the required permission (obs:bucket:ListBucket in IAM or ListBucket in a bucket policy). For details, see Introduction to OBS Access Control, IAM Custom Policies, and Creating a Custom Bucket Policy.
The mapping between OBS regions and endpoints must comply with what is listed in Regions and Endpoints.

Method

ObsClient.listObjects(bucketName, prefix, marker, max_keys, delimiter, extensionHeaders)

Request Parameters

**Table 1** List of request parameters
Parameter	Type	Mandatory (Yes/No)	Description
bucketName	str	Yes	Explanation: Bucket name Restrictions: A bucket name must be unique across all accounts and regions. A bucket name: Must be 3 to 63 characters long and start with a digit or letter. Lowercase letters, digits, hyphens (-), and periods (.) are allowed. Cannot be formatted as an IP address. Cannot start or end with a hyphen (-) or period (.). Cannot contain two consecutive periods (..), for example, my..bucket. Cannot contain periods (.) and hyphens (-) adjacent to each other, for example, my-.bucket or my.-bucket. If you repeatedly create buckets of the same name in the same region, no error will be reported and the bucket properties comply with those set in the first creation request. Default value: None
prefix	str	No	Explanation: Name prefix that the objects to be listed must contain. Assume that you have the following objects: logs/day1, logs/day2, logs/day3, and ExampleObject.jpg. If you specify logs/ as the prefix, logs/day1, logs/day2, and logs/day3 will be returned. If you leave this parameter blank, all objects in the bucket will be returned. Value range: The value must contain 1 to 1,024 characters. Default value: None
marker	str	No	Explanation: Name of the object to start with when listing objects in a bucket. All objects following this object are listed in alphabetical order by object name. Assume that you have the following objects: test/a, test/b, test/c, and test/d. If you specify test/b as the marker, test/c and test/d are returned. Restrictions: This parameter is only used for listing objects with a single version. Value range: The value must contain 1 to 1,024 characters. Default value: None
max_keys	int	No	Explanation: The maximum number of objects returned in the response in alphabetical order Value range: The value ranges from 1 to 1000. If the specified value is greater than 1000, only 1,000 objects are returned. Default value: 1000
delimiter	str	No	Explanation: This parameter is used to group object names. If a prefix is specified, objects with the same string from the prefix to the first delimiter are grouped into one CommonPrefix. If no prefix is specified, objects with the same string from the first character to the first delimiter are grouped into one CommonPrefix. Assume that a bucket has objects abcd, abcde, and bbcde in it. If delimiter is set to d and prefix is set to a, objects abcd and abcde are grouped into a CommonPrefix with abcd as the prefix. If only delimiter is set to d, objects abcd and abcde are grouped into a CommonPrefix with abcd as the prefix, and bbcde is grouped separately into another CommonPrefix with bbcd as the prefix. For a parallel file system, if this parameter is not specified, all the content in the directory is recursively listed by default, including the content in subdirectories. In big data scenarios, parallel file systems usually have multiple directory levels and each directory level has a large number of files. In such case, you are advised to configure [delimiter=/] to list the content in the current directory but exclude the content in subdirectories, thereby making the listing more efficient. Value range: The value must contain 1 to 1,024 characters. Default value: None
extensionHeaders	dict	No	Explanation: Extension headers. Value range: See User-defined Header (SDK for Python). Default value: None
encoding_type	str	No	Explanation: Encoding type for some elements in the response. If delimiter, key_marker, prefix, nextKeyMarker, and key contain control characters that are not supported by the XML 1.0 standard, you can set encoding_type to encode delimiter, key_marker, prefix (including the Prefix in commonPrefixes), nextKeyMarker, and key in the response. Value range: url Default value: None. If you leave this parameter blank, encoding is not applied to elements.

Responses

**Table 2** List of returned results
Type	Description
GetResult	Explanation: SDK common results

**Table 3** GetResult
Parameter	Type	Description
status	int	Explanation: HTTP status code Value range: A status code is a group of digits ranging from 2xx (indicating successes) to 4xx or 5xx (indicating errors). It indicates the status of a response. For more information, see Status Code. Default value: None
reason	str	Explanation: Reason description. Default value: None
errorCode	str	Explanation: Error code returned by the OBS server. If the value of status is less than 300, this parameter is left blank. Default value: None
errorMessage	str	Explanation: Error message returned by the OBS server. If the value of status is less than 300, this parameter is left blank. Default value: None
requestId	str	Explanation: Request ID returned by the OBS server Default value: None
indicator	str	Explanation: Error indicator returned by the OBS server. Default value: None
hostId	str	Explanation: Requested server ID. If the value of status is less than 300, this parameter is left blank. Default value: None
resource	str	Explanation: Error source (a bucket or an object). If the value of status is less than 300, this parameter is left blank. Default value: None
header	list	Explanation: Response header list, composed of tuples. Each tuple consists of two elements, respectively corresponding to the key and value of a response header. Default value: None
body	object	Explanation: Result content returned after the operation is successful. If the value of status is larger than 300, the value of body is null. The value varies with the API being called. For details, see Bucket-Related APIs (SDK for Python) and Object-Related APIs (SDK for Python). Default value: None

**Table 4** GetResult.body
GetResult.body Type	Description
ListObjectsResponse	Explanation: See Table 5.

**Table 5** ListObjectsResponse
Parameter	Type	Description
name	str	Explanation: Bucket name Restrictions: A bucket name must be unique across all accounts and regions. A bucket name: Must be 3 to 63 characters long and start with a digit or letter. Lowercase letters, digits, hyphens (-), and periods (.) are allowed. Cannot be formatted as an IP address. Cannot start or end with a hyphen (-) or period (.). Cannot contain two consecutive periods (..), for example, my..bucket. Cannot contain periods (.) and hyphens (-) adjacent to each other, for example, my-.bucket or my.-bucket. If you repeatedly create buckets of the same name in the same region, no error will be reported and the bucket properties comply with those set in the first creation request. Default value: None
location	str	Explanation: Region where a bucket is located Restrictions: If the used endpoint is obs.myhuaweicloud.com, this parameter is not required. If any other endpoint is used, this parameter is required. Value range: To learn about valid regions and endpoints, see Regions and Endpoints. An endpoint is the request address for calling an API. Endpoints vary depending on services and regions. To obtain the regions and endpoints, contact the enterprise administrator. Default value: If obs.myhuaweicloud.com is used as the endpoint and no region is specified, cn-north-1 (the CN North-Beijing1 region) is used by default.
prefix	str	Explanation: Name prefix that the objects to be listed must contain. Assume that you have the following objects: logs/day1, logs/day2, logs/day3, and ExampleObject.jpg. If you specify logs/ as the prefix, logs/day1, logs/day2, and logs/day3 will be returned. If you leave this parameter blank, all objects in the bucket will be returned. Value range: The value must contain 1 to 1,024 characters. Default value: None
marker	str	Explanation: Name of the object to start with when listing objects in a bucket. All objects following this object are listed in alphabetical order by object name. Assume that you have the following objects: test/a, test/b, test/c, and test/d. If you specify test/b as the marker, test/c and test/d are returned. Restrictions: This parameter is only used for listing objects with a single version. Value range: The value must contain 1 to 1,024 characters. Default value: None
delimiter	str	Explanation: This parameter is used to group object names. If a prefix is specified, objects with the same string from the prefix to the first delimiter are grouped into one CommonPrefix. If no prefix is specified, objects with the same string from the first character to the first delimiter are grouped into one CommonPrefix. Assume that a bucket has objects abcd, abcde, and bbcde in it. If delimiter is set to d and prefix is set to a, objects abcd and abcde are grouped into a commonPrefix with abcd as the prefix. If only delimiter is set to d, objects abcd and abcde are grouped into a commonPrefix with abcd as the prefix, and bbcde is grouped separately into another commonPrefix with bbcd as the prefix. Value range: The value must contain 1 to 1,024 characters. Default value: None
max_keys	int	Explanation: The maximum number of objects returned in the response in alphabetical order Value range: The value ranges from 1 to 1000. If the specified value is greater than 1000, only 1,000 objects are returned. Default value: 1000
is_truncated	bool	Explanation: Whether all results are returned in the response Value range: True indicates that not all results are returned. False indicates that all results are returned. Default value: None
next_marker	str	Explanation: Object name to start with upon next request for listing objects Value range: An object name string Default value: None
contents	list of Content	Explanation: List of objects in the bucket Value range: See Table 6.
commonPrefixs	list of CommonPrefix	Explanation: List of object name prefixes grouped according to the delimiter parameter (if specified) Value range: See Table 8.
encoding_type	str	Explanation: Encoding type for some elements in the response. If delimiter, key_marker, prefix, nextKeyMarker, and key contain control characters that are not supported by the XML 1.0 standard, you can set encoding_type to encode delimiter, key_marker, prefix (including the Prefix in commonPrefixes), nextKeyMarker, and key in the response. Value range: url Default value: None. If you leave this parameter blank, encoding is not applied to elements.

**Table 6** Content
Parameter	Type	Description
key	str	Explanation: Object name. An object is uniquely identified by an object name in a bucket. An object name is a complete path that does not contain the bucket name. For example, if the address for accessing the object is examplebucket.obs.ap-southeast-1.myhuaweicloud.com/folder/test.txt, the object name is folder/test.txt. Value range: The value must contain 1 to 1,024 characters. Default value: None
lastModified	str	Explanation: Time when the last modification was made to the object Restrictions: The time must be in the ISO8601 format. Example: 2018-01-01T00:00:00.000Z Default value: None
etag	str	Explanation: Base64-encoded, 128-bit MD5 value of an object. ETag is the unique identifier of the object contents and is used to determine whether the contents of an object are changed. For example, if the ETag value is A when an object is uploaded and is B when the object is downloaded, this indicates the contents of the object are changed. The ETag reflects changes only to the contents of an object, not its metadata. Objects created by the upload and copy operations have unique ETags after being encrypted using MD5. Restrictions: If an object is encrypted using server-side encryption, the ETag is not the MD5 value of the object. Value range: The value must contain 32 characters. Default value: None
size	int	Explanation: Object size Value range: The value ranges from 0 TB to 48.8 TB, in bytes. Default value: None
owner	Owner	Explanation: Object owner Value range: See Table 7.
storageClass	str	Explanation: Storage class of the object Value range: If the storage class is Standard, leave this parameter blank. For details about the available storage classes, see Table 9. Default value: None
isAppendable	bool	Explanation: Whether the object is appendable Value range: True: The object is appendable. False: The object is not appendable. Default value: None

**Table 7** Owner
Parameter	Type	Mandatory (Yes/No)	Description
owner_id	str	Yes if used as a request parameter	Explanation: Account (domain) ID of the owner Value range: To obtain the account ID, see How Do I Get My Account ID and IAM User ID? (SDK for Python) Default value: None
owner_name	str	No if used as a request parameter	Explanation: Account name of the owner Value range: To obtain the account ID, see How Do I Get My Account ID and IAM User ID? (SDK for Python) Default value: None

**Table 8** CommonPrefix
Parameter	Type	Description
prefix	str	Explanation: List of object name prefixes grouped according to the delimiter parameter (if specified) Value range: The value must contain 1 to 1,024 characters. Default value: None

**Table 9** StorageClass
Parameter	Type	Description
STANDARD	Standard storage class	Explanation: Features low access latency and high throughput and is used for storing massive, frequently accessed (multiple times a month) or small objects (< 1 MB) requiring quick response.
WARM	Infrequent Access storage class	Explanation: Used for storing data that is semi-frequently accessed (fewer than 12 times a year) but is instantly available when needed.
COLD	Archive storage class	Explanation: Used for storing rarely accessed (once a year) data.

Code Examples

All elements stored in OBS buckets are called objects. Folders are objects whose sizes are 0 and whose names end with a slash (/). You can use the folder name as the prefix to list objects in a folder. This example lists objects with test/ as the prefix in bucket examplebucket. A maximum of 100 objects can be returned at a time.

     
      
        
        
          from obs import ObsClient
import os
import traceback

# Obtain an AK and SK pair using environment variables or import the AK and SK pair in other ways. Using hard coding may result in leakage.
# Obtain an AK and SK pair on the management console. For details, see https://support.huaweicloud.com/intl/en-us/usermanual-ca/ca_01_0003.html.
ak = os.getenv("AccessKeyID")
sk = os.getenv("SecretAccessKey")
# (Optional) If you use a temporary AK and SK pair and a security token to access OBS, obtain them from environment variables.
# security_token = os.getenv("SecurityToken")
# Set server to the endpoint corresponding to the bucket. CN-Hong Kong is used here as an example. Replace it with the one in use.
server = "https://obs.ap-southeast-1.myhuaweicloud.com" 

# Create an obsClient instance.
# If you use a temporary AK and SK pair and a security token to access OBS, you must specify security_token when creating an instance.
obsClient = ObsClient(access_key_id=ak, secret_access_key=sk, server=server)
try:
    bucketName = "examplebucket"
    # Specify an object prefix.
    prefix = 'test/'
    # Specify the maximum number of objects to be listed at a time. 100 is used as an example.
    max_keys = 100
    # List objects in the bucket.
    resp = obsClient.listObjects(bucketName, prefix, max_keys=max_keys, encoding_type='url')

    # If status code 2xx is returned, the API is called successfully. Otherwise, the API call fails.
    if resp.status < 300:
        print('List Objects Succeeded')
        print('requestId:', resp.requestId)
        print('name:', resp.body.name)
        print('prefix:', resp.body.prefix)
        print('max_keys:', resp.body.max_keys)
        print('is_truncated:', resp.body.is_truncated)
        index = 1
        for content in resp.body.contents:
            print('object [' + str(index) + ']')
            print('key:', content.key)
            print('lastModified:', content.lastModified)
            print('etag:', content.etag)
            print('size:', content.size)
            print('storageClass:', content.storageClass)
            print('owner_id:', content.owner.owner_id)
            print('owner_name:', content.owner.owner_name)
            index += 1
    else:
        print('List Objects Failed')
        print('requestId:', resp.requestId)
        print('errorCode:', resp.errorCode)
        print('errorMessage:', resp.errorMessage)
except:
    print('List Objects Failed')
    print(traceback.format_exc())

         

       

     
    

Sample Code (for Listing over 1,000 Objects)

    
     
       
       
         try:
    max_num = 1000
    mark = None
    index = 1 
    while True:
        resp = obsClient.listObjects('bucketname', marker=mark, max_keys=max_num, encoding_type='url')
        if resp.status < 300: 
            print('requestId:', resp.requestId) 
            print('name:', resp.body.name) 
            print('prefix:', resp.body.prefix) 
            print('max_keys:', resp.body.max_keys) 
            print('is_truncated:', resp.body.is_truncated) 
            for content in resp.body.contents: 
                print('object [' + str(index) + ']') 
                print('key:', content.key) 
                print('lastModified:', content.lastModified) 
                print('etag:', content.etag) 
                print('size:', content.size) 
                print('storageClass:', content.storageClass) 
                print('owner_id:', content.owner.owner_id) 
                print('owner_name:', content.owner.owner_name) 
                index += 1 
            if resp.body.is_truncated is True:
                mark = resp.body.next_marker
            else:
                break
        else: 
            print('errorCode:', resp.errorCode) 
            print('errorMessage:', resp.errorMessage)
            break
except:
    import traceback
    print(traceback.format_exc())

        

      

    
   

Listing Objects in a Bucket (SDK for Python)