Updated on 2024-09-05 GMT+08:00

Listing Objects in a Bucket (SDK for Python)

Function

This API lists some or all of the objects in a bucket. You can use parameters such as the prefix, number of returned objects, and start position to list objects that meet specified criteria. Returned objects are listed in alphabetical order by object name.

Restrictions

Method

ObsClient.listObjects(bucketName, prefix, marker, max_keys, delimiter)

Request Parameters

Table 1 List of request parameters

Parameter

Type

Mandatory (Yes/No)

Description

bucketName

str

Yes

Explanation:

Bucket name

Restrictions:

  • A bucket name must be unique across all accounts and regions.
  • A bucket name:
    • Must be 3 to 63 characters long and start with a digit or letter. Lowercase letters, digits, hyphens (-), and periods (.) are allowed.
    • Cannot be formatted as an IP address.
    • Cannot start or end with a hyphen (-) or period (.).
    • Cannot contain two consecutive periods (..), for example, my..bucket.
    • Cannot contain periods (.) and hyphens (-) adjacent to each other, for example, my-.bucket or my.-bucket.
  • If you repeatedly create buckets of the same name in the same region, no error will be reported and the bucket properties comply with those set in the first creation request.

Default value:

None

prefix

str

No

Explanation:

Name prefix that the objects to be listed must contain.

Assume that you have the following objects: logs/day1, logs/day2, logs/day3, and ExampleObject.jpg. If you specify logs/ as the prefix, logs/day1, logs/day2, and logs/day3 will be returned. If you leave this parameter blank, all objects in the bucket will be returned.

Value range:

The value must contain 1 to 1,024 characters.

Default value:

None

marker

str

No

Explanation:

Name of the object to start with when listing objects in a bucket. All objects following this object are listed in alphabetical order by object name.

Assume that you have the following objects: test/a, test/b, test/c, and test/d. If you specify test/b as the marker, test/c and test/d are returned.

Restrictions:

This parameter is only used for listing objects with a single version.

Value range:

The value must contain 1 to 1,024 characters.

Default value:

None

max_keys

int

No

Explanation:

The maximum number of objects returned in the response in alphabetical order

Value range:

The value ranges from 1 to 1000. If the specified value is greater than 1000, only 1,000 objects are returned.

Default value:

1000

delimiter

str

No

Explanation:

This parameter is used to group object names. If a prefix is specified, objects with the same string from the prefix to the first delimiter are grouped into one CommonPrefix. If no prefix is specified, objects with the same string from the first character to the first delimiter are grouped into one CommonPrefix.

Assume that a bucket has objects abcd, abcde, and bbcde in it. If delimiter is set to d and prefix is set to a, objects abcd and abcde are grouped into a CommonPrefix with abcd as the prefix. If only delimiter is set to d, objects abcd and abcde are grouped into a CommonPrefix with abcd as the prefix, and bbcde is grouped separately into another CommonPrefix with bbcd as the prefix.

For a parallel file system, if this parameter is not specified, all the content in the directory is recursively listed by default, including the content in subdirectories. In big data scenarios, parallel file systems usually have multiple directory levels and each directory level has a large number of files. In such case, you are advised to configure [delimiter=/] to list the content in the current directory but exclude the content in subdirectories, thereby making the listing more efficient.

Value range:

The value must contain 1 to 1,024 characters.

Default value:

None

encoding_type

str

No

Explanation:

Encoding type for some elements in the response. If delimiter, key_marker, prefix, nextKeyMarker, and key contain control characters that are not supported by the XML 1.0 standard, you can set encoding_type to encode delimiter, key_marker, prefix (including the Prefix in commonPrefixes), nextKeyMarker, and key in the response.

Value range:

url

Default value:

None. If you leave this parameter blank, encoding is not applied to elements.

Responses

Table 2 List of returned results

Type

Description

GetResult

Explanation:

SDK common results

Table 3 GetResult

Parameter

Type

Description

status

int

Explanation:

HTTP status code

Value range:

A status code is a group of digits ranging from 2xx (indicating successes) to 4xx or 5xx (indicating errors). It indicates the status of a response. For more information, see Status Code.

Default value:

None

reason

str

Explanation:

Reason description.

Default value:

None

errorCode

str

Explanation:

Error code returned by the OBS server. If the value of status is less than 300, this parameter is left blank.

Default value:

None

errorMessage

str

Explanation:

Error message returned by the OBS server. If the value of status is less than 300, this parameter is left blank.

Default value:

None

requestId

str

Explanation:

Request ID returned by the OBS server

Default value:

None

indicator

str

Explanation:

Error indicator returned by the OBS server.

Default value:

None

hostId

str

Explanation:

Requested server ID. If the value of status is less than 300, this parameter is left blank.

Default value:

None

resource

str

Explanation:

Error source (a bucket or an object). If the value of status is less than 300, this parameter is left blank.

Default value:

None

header

list

Explanation:

Response header list, composed of tuples. Each tuple consists of two elements, respectively corresponding to the key and value of a response header.

Default value:

None

body

object

Explanation:

Result content returned after the operation is successful. If the value of status is larger than 300, this parameter is left blank. The value varies with the API being called. For details, see Bucket-Related APIs (SDK for Python) and Object-Related APIs (SDK for Python).

Default value:

None

Table 4 GetResult.body

GetResult.body Type

Description

ListObjectsResponse

Explanation:

See Table 5.

Table 5 ListObjectsResponse

Parameter

Type

Description

name

str

Explanation:

Bucket name

Restrictions:

  • A bucket name must be unique across all accounts and regions.
  • A bucket name:
    • Must be 3 to 63 characters long and start with a digit or letter. Lowercase letters, digits, hyphens (-), and periods (.) are allowed.
    • Cannot be formatted as an IP address.
    • Cannot start or end with a hyphen (-) or period (.).
    • Cannot contain two consecutive periods (..), for example, my..bucket.
    • Cannot contain periods (.) and hyphens (-) adjacent to each other, for example, my-.bucket or my.-bucket.
  • If you repeatedly create buckets of the same name in the same region, no error will be reported and the bucket properties comply with those set in the first creation request.

Default value:

None

location

str

Explanation:

Region where a bucket is located

Restrictions:

If the used endpoint is obs.myhuaweicloud.com, this parameter is not required. If any other endpoint is used, this parameter is required.

Value range:

To learn about valid regions and endpoints, see Regions and Endpoints. An endpoint is the request address for calling an API. Endpoints vary depending on services and regions. To obtain the regions and endpoints, contact the enterprise administrator.

Default value:

If obs.myhuaweicloud.com is used as the endpoint and no region is specified, cn-north-1 (the CN North-Beijing1 region) is used by default.

prefix

str

Explanation:

Name prefix that the objects to be listed must contain.

Assume that you have the following objects: logs/day1, logs/day2, logs/day3, and ExampleObject.jpg. If you specify logs/ as the prefix, logs/day1, logs/day2, and logs/day3 will be returned. If you leave this parameter blank, all objects in the bucket will be returned.

Value range:

The value must contain 1 to 1,024 characters.

Default value:

None

marker

str

Explanation:

Name of the object to start with when listing objects in a bucket. All objects following this object are listed in alphabetical order by object name.

Assume that you have the following objects: test/a, test/b, test/c, and test/d. If you specify test/b as the marker, test/c and test/d are returned.

Restrictions:

This parameter is only used for listing objects with a single version.

Value range:

The value must contain 1 to 1,024 characters.

Default value:

None

delimiter

str

Explanation:

This parameter is used to group object names. If a prefix is specified, objects with the same string from the prefix to the first delimiter are grouped into one CommonPrefix. If no prefix is specified, objects with the same string from the first character to the first delimiter are grouped into one CommonPrefix.

Assume that a bucket has objects abcd, abcde, and bbcde in it. If delimiter is set to d and prefix is set to a, objects abcd and abcde are grouped into a commonPrefix with abcd as the prefix. If only delimiter is set to d, objects abcd and abcde are grouped into a commonPrefix with abcd as the prefix, and bbcde is grouped separately into another commonPrefix with bbcd as the prefix.

Value range:

The value must contain 1 to 1,024 characters.

Default value:

None

max_keys

int

Explanation:

The maximum number of objects returned in the response in alphabetical order

Value range:

The value ranges from 1 to 1000. If the specified value is greater than 1000, only 1,000 objects are returned.

Default value:

1000

is_truncated

bool

Explanation:

Whether all results are returned in the response

Value range:

  • True indicates that not all results are returned.
  • False indicates that all results are returned.

Default value:

None

next_marker

str

Explanation:

Object name to start with upon next request for listing objects

Value range:

An object name string

Default value:

None

contents

list of Content

Explanation:

List of objects in the bucket

Value range:

See Table 6.

commonPrefixs

list of CommonPrefix

Explanation:

List of object name prefixes grouped according to the delimiter parameter (if specified)

Value range:

See Table 8.

encoding_type

str

Explanation:

Encoding type for some elements in the response. If delimiter, key_marker, prefix, nextKeyMarker, and key contain control characters that are not supported by the XML 1.0 standard, you can set encoding_type to encode delimiter, key_marker, prefix (including the Prefix in commonPrefixes), nextKeyMarker, and key in the response.

Value range:

url

Default value:

None. If you leave this parameter blank, encoding is not applied to elements.

Table 6 Content

Parameter

Type

Description

key

str

Explanation:

Object name. An object is uniquely identified by an object name in a bucket. An object name is a complete path that does not contain the bucket name.

For example, if the address for accessing the object is examplebucket.obs.ap-southeast-1.myhuaweicloud.com/folder/test.txt, the object name is folder/test.txt.

Value range:

The value must contain 1 to 1,024 characters.

Default value:

None

lastModified

str

Explanation:

Time when the last modification was made to the object

Restrictions:

The time must be in the ISO8601 format.

Example: 2018-01-01T00:00:00.000Z

Default value:

None

etag

str

Explanation:

Base64-encoded, 128-bit MD5 value of an object. ETag is the unique identifier of the object contents and is used to determine whether the contents of an object are changed. For example, if the ETag value is A when an object is uploaded and is B when the object is downloaded, this indicates the contents of the object are changed. The ETag reflects changes only to the contents of an object, not its metadata. Objects created by the upload and copy operations have unique ETags after being encrypted using MD5.

Restrictions:

If an object is encrypted using server-side encryption, the ETag is not the MD5 value of the object.

Value range:

The value must contain 32 characters.

Default value:

None

size

int

Explanation:

Object size

Value range:

The value ranges from 0 TB to 48.8 TB, in bytes.

Default value:

None

owner

Owner

Explanation:

Object owner

Value range:

See Table 7.

storageClass

str

Explanation:

Storage class of the object

Value range:

  • If the storage class is Standard, leave this parameter blank.
  • For details about the available storage classes, see Table 9.

Default value:

None

isAppendable

bool

Explanation:

Whether the object is appendable

Value range:

True: The object is appendable.

False: The object is not appendable.

Default value:

None

Table 7 Owner

Parameter

Type

Mandatory (Yes/No)

Description

owner_id

str

Yes if used as a request parameter

Explanation:

Account (domain) ID of the owner

Value range:

To obtain the account ID, see How Do I Get My Account ID and IAM User ID? (SDK for Python)

Default value:

None

owner_name

str

No if used as a request parameter

Explanation:

Account name of the owner

Value range:

To obtain the account ID, see How Do I Get My Account ID and IAM User ID? (SDK for Python)

Default value:

None

Table 8 CommonPrefix

Parameter

Type

Description

prefix

str

Explanation:

List of object name prefixes grouped according to the delimiter parameter (if specified)

Value range:

The value must contain 1 to 1,024 characters.

Default value:

None

Table 9 StorageClass

Parameter

Type

Description

STANDARD

Standard storage class

Explanation:

Features low access latency and high throughput and is used for storing massive, frequently accessed (multiple times a month) or small objects (< 1 MB) requiring quick response.

WARM

Infrequent Access storage class

Explanation:

Used for storing data that is semi-frequently accessed (fewer than 12 times a year) but is instantly available when needed.

COLD

Archive storage class

Explanation:

Used for storing rarely accessed (once a year) data.

Code Examples

This example lists objects with test/ as the prefix in bucket examplebucket. A maximum of 100 objects can be returned at a time.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
from obs import ObsClient
import os
import traceback

# Obtain an AK and SK pair using environment variables or import the AK and SK pair in other ways. Using hard coding may result in leakage.
# Obtain an AK and SK pair on the management console. For details, see https://support.huaweicloud.com/intl/en-us/usermanual-ca/ca_01_0003.html.
ak = os.getenv("AccessKeyID")
sk = os.getenv("SecretAccessKey")
# (Optional) If you use a temporary AK and SK pair and a security token to access OBS, obtain them from environment variables.
security_token = os.getenv("SecurityToken")
# Set server to the endpoint corresponding to the bucket. Here uses CN-Hong Kong as an example. Replace it with the one in use.
server = "https://obs.ap-southeast-1.myhuaweicloud.com" 

# Create an obsClient instance.
# If you use a temporary AK and SK pair and a security token to access OBS, you must specify security_token when creating an instance.
obsClient = ObsClient(access_key_id=ak, secret_access_key=sk, server=server)
try:
    bucketName = "examplebucket"
    # Specify an object prefix.
    prefix = 'test/'
    # Specify the maximum number of objects to be listed at a time. 100 is used as an example.
    max_keys = 100
    # List objects in the bucket.
    resp = obsClient.listObjects(bucketName, prefix, max_keys=max_keys, encoding_type='url')

    # If status code 2xx is returned, the API is called successfully. Otherwise, the API call fails.
    if resp.status < 300:
        print('List Objects Succeeded')
        print('requestId:', resp.requestId)
        print('name:', resp.body.name)
        print('prefix:', resp.body.prefix)
        print('max_keys:', resp.body.max_keys)
        print('is_truncated:', resp.body.is_truncated)
        index = 1
        for content in resp.body.contents:
            print('object [' + str(index) + ']')
            print('key:', content.key)
            print('lastModified:', content.lastModified)
            print('etag:', content.etag)
            print('size:', content.size)
            print('storageClass:', content.storageClass)
            print('owner_id:', content.owner.owner_id)
            print('owner_name:', content.owner.owner_name)
            index += 1
    else:
        print('List Objects Failed')
        print('requestId:', resp.requestId)
        print('errorCode:', resp.errorCode)
        print('errorMessage:', resp.errorMessage)
except:
    print('List Objects Failed')
    print(traceback.format_exc())

Sample Code (for Listing over 1,000 Objects)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
try:
    max_num = 1000
    mark = None
    index = 1 
    while True:
        resp = obsClient.listObjects('bucketname', marker=mark, max_keys=max_num, encoding_type='url')
        if resp.status < 300: 
            print('requestId:', resp.requestId) 
            print('name:', resp.body.name) 
            print('prefix:', resp.body.prefix) 
            print('max_keys:', resp.body.max_keys) 
            print('is_truncated:', resp.body.is_truncated) 
            for content in resp.body.contents: 
                print('object [' + str(index) + ']') 
                print('key:', content.key) 
                print('lastModified:', content.lastModified) 
                print('etag:', content.etag) 
                print('size:', content.size) 
                print('storageClass:', content.storageClass) 
                print('owner_id:', content.owner.owner_id) 
                print('owner_name:', content.owner.owner_name) 
                index += 1 
            if resp.body.is_truncated is True:
                mark = resp.body.next_marker
            else:
                break
        else: 
            print('errorCode:', resp.errorCode) 
            print('errorMessage:', resp.errorMessage)
            break
except:
    import traceback
    print(traceback.format_exc())