Listing Objects in a Parallel File System (SDK for Python)

Function

This API lists some or all of the objects in a parallel file system (PFS). You can configure the prefix, number, and start position to list objects that meet specified criteria. Returned objects are listed in alphabetical order by object name.

Restrictions

A maximum of 1,000 objects can be listed for each API call.
To list objects in a PFS, you must be the PFS owner or have the required permission (obs:bucket:ListBucket in IAM or ListBucket in a policy). For details, see Introduction to OBS Access Control, IAM Custom Policies, and Creating a Custom Bucket Policy.
The mapping between OBS regions and endpoints must comply with what is listed in Regions and Endpoints.

Method

ObsClient.listObjects(bucketName, prefix, marker, max_keys, delimiter, extensionHeaders)

Request Parameters

**Table 1** List of request parameters
Parameter	Type	Mandatory (Yes/No)	Description
bucketName	str	Yes	Explanation: PFS name. Restrictions: A PFS name must be unique across all accounts and regions. A PFS name: Must be 3 to 63 characters long and start with a digit or letter. Lowercase letters, digits, hyphens (-), and periods (.) are allowed. Cannot be formatted as an IP address. Cannot start or end with a hyphen (-) or period (.). Cannot contain two consecutive periods (..), for example, my..bucket. Cannot contain periods (.) and hyphens (-) adjacent to each other, for example, my-.bucket or my.-bucket. If you repeatedly create PFSs with the same name in the same region, no error will be reported and the PFS attributes comply with those set in the first creation request. Default value: None
prefix	str	No	Explanation: Name prefix that the objects to be listed must contain. Assume that you have the following objects: logs/day1, logs/day2, logs/day3, and ExampleObject.jpg. If you specify logs/ as the prefix, logs/day1, logs/day2, and logs/day3 will be returned. If this parameter is left blank and there are no other filter criteria, all objects in the PFS will be returned. Value range: The value can contain 1 to 1,024 characters. Default value: None
marker	str	No	Explanation: All objects following the specified object marker are returned in alphabetical order by object name. Assume that you have the following objects: test/a, test/b, test/c, and test/d. If you specify test/b as the marker, test/c and test/d are returned. Restrictions: This parameter is only used for listing objects with a single version. Value range: The value can contain 1 to 1,024 characters. Default value: None
max_keys	int	No	Explanation: The maximum number of objects returned in the response in alphabetical order Value range: The value ranges from 1 to 1000. If the specified value is greater than 1000, only 1,000 objects are returned. Default value: 1000
delimiter	str	No	Explanation: This parameter is used to group object names. If a prefix is specified, objects with the same string from the prefix to the first delimiter are grouped into one CommonPrefix. If no prefix is specified, objects with the same string from the first character to the first delimiter are grouped into one CommonPrefix. Assume that a PFS has objects abcd, abcde, and bbcde in it. If delimiter is set to d and prefix is set to a, objects abcd and abcde are grouped into a CommonPrefix with abcd as the prefix. If only delimiter is set to d, objects abcd and abcde are grouped into a CommonPrefix with abcd as the prefix, and bbcde is grouped separately into another CommonPrefix with bbcd as the prefix. For a PFS, if this parameter is not specified, all the contents in the current directory are recursively listed by default, and subdirectories are also listed. In big data scenarios, PFSs usually have multiple directory levels and each directory level has a large number of objects. In such case, you are advised to configure [delimiter=/] to list only the content in the current directory, without the content in subdirectories, so the listing can be more efficient. Value range: The value can contain 1 to 1,024 characters. Default value: None
extensionHeaders	dict	No	Explanation: Extension headers. Value range: See User-defined Headers (SDK for Python). Default value: None
encoding_type	str	No	Explanation: Encoding type for some elements in the response. If delimiter, key_marker, prefix, nextKeyMarker, and key contain control characters that are not supported by the XML 1.0 standard, you can set encoding_type to encode delimiter, key_marker, prefix (including the Prefix in commonPrefixes), nextKeyMarker, and key in the response. Value range: url Default value: None. If you leave this parameter blank, encoding is not applied.

Responses

**Table 2** Responses
Type	Description
GetResult	Explanation: SDK common results

**Table 3** GetResult
Parameter	Type	Description
status	int	Explanation: HTTP status code Value range: A status code is a group of digits indicating the status of a response. It ranges from 2xx (indicating successes) to 4xx or 5xx (indicating errors). For more information, see Status Code. Default value: None
reason	str	Explanation: Reason description. Default value: None
errorCode	str	Explanation: Error code returned by the OBS server. If the value of status is less than 300, this parameter is left blank. Default value: None
errorMessage	str	Explanation: Error message returned by the OBS server. If the value of status is less than 300, this parameter is left blank. Default value: None
requestId	str	Explanation: Request ID returned by the OBS server Default value: None
indicator	str	Explanation: Error indicator returned by the OBS server. Default value: None
hostId	str	Explanation: Requested server ID. If the value of status is less than 300, this parameter is left blank. Default value: None
resource	str	Explanation: Error source (a bucket or an object). If the value of status is less than 300, this parameter is left blank. Default value: None
header	list	Explanation: Response header list, composed of tuples. Each tuple consists of two elements, respectively corresponding to the key and value of a response header. Default value: None
body	object	Explanation: Result content returned after the operation is successful. If the value of status is larger than 300, the value of body is null. The value varies with the API being called. For details, see Bucket-Related APIs (SDK for Python) and Object-Related APIs (SDK for Python). Default value: None

**Table 4** GetResult.body
GetResult.body Type	Description
ListObjectsResponse	Explanation: Returned results for the request. For details, see Table 5.

**Table 5** ListObjectsResponse
Parameter	Type	Description
name	str	Explanation: PFS name. Restrictions: A PFS name must be unique across all accounts and regions. A PFS name: Must be 3 to 63 characters long and start with a digit or letter. Lowercase letters, digits, hyphens (-), and periods (.) are allowed. Cannot be formatted as an IP address. Cannot start or end with a hyphen (-) or period (.). Cannot contain two consecutive periods (..), for example, my..bucket. Cannot contain periods (.) and hyphens (-) adjacent to each other, for example, my-.bucket or my.-bucket. If you repeatedly create PFSs with the same name in the same region, no error will be reported and the PFS attributes comply with those set in the first creation request. Default value: None
location	str	Explanation: Region where the PFS is located. Restrictions: If the specified endpoint is obs.myhuaweicloud.com, this parameter is not required. If any other endpoints are specified, this parameter is required. Value range: To learn about valid regions and endpoints, see Regions and Endpoints. An endpoint is the request address for calling an API. Endpoints vary depending on services and regions. To obtain the regions and endpoints, contact the enterprise administrator. Default value: If obs.myhuaweicloud.com is used as the endpoint and no region is specified, cn-north-1 (the CN North-Beijing1 region) is used by default.
prefix	str	Explanation: Name prefix that the objects to be listed must contain. Assume that you have the following objects: logs/day1, logs/day2, logs/day3, and ExampleObject.jpg. If you specify logs/ as the prefix, logs/day1, logs/day2, and logs/day3 will be returned. If this parameter is left blank and there are no other filter criteria, all objects in the PFS will be returned. Value range: The value can contain 1 to 1,024 characters. Default value: None
marker	str	Explanation: All objects following the specified object marker are returned in alphabetical order by object name. Assume that you have the following objects: test/a, test/b, test/c, and test/d. If you specify test/b as the marker, test/c and test/d are returned. Restrictions: This parameter is only used for listing objects with a single version. Value range: The value can contain 1 to 1,024 characters. Default value: None
delimiter	str	Explanation: This parameter is used to group object names. If a prefix is specified, objects with the same string from the prefix to the first delimiter are grouped into one CommonPrefix. If no prefix is specified, objects with the same string from the first character to the first delimiter are grouped into one CommonPrefix. Assume that a PFS has objects abcd, abcde, and bbcde in it. If delimiter is set to d and prefix is set to a, objects abcd and abcde are grouped into a commonPrefix with abcd as the prefix. If only delimiter is set to d, objects abcd and abcde are grouped into a commonPrefix with abcd as the prefix, and bbcde is grouped separately into another commonPrefix with bbcd as the prefix. Value range: The value can contain 1 to 1,024 characters. Default value: None
max_keys	int	Explanation: The maximum number of objects returned in the response in alphabetical order Value range: The value ranges from 1 to 1000. If the specified value is greater than 1000, only 1,000 objects are returned. Default value: 1000
is_truncated	bool	Explanation: Whether all results are returned in the response. Value range: True indicates that not all results were returned. False indicates that all results were returned. Default value: None
next_marker	str	Explanation: Where in the bucket the next listing begins. Value range: An object name string Default value: None
contents	list of Content	Explanation: List of objects in the PFS. Value range: See Table 6.
commonPrefixs	list of CommonPrefix	Explanation: List of object name prefixes grouped according to the delimiter parameter (if specified) Value range: See Table 8.
encoding_type	str	Explanation: Encoding type for some elements in the response. If delimiter, key_marker, prefix, nextKeyMarker, and key contain control characters that are not supported by the XML 1.0 standard, you can set encoding_type to encode delimiter, key_marker, prefix (including the Prefix in commonPrefixes), nextKeyMarker, and key in the response. Value range: url Default value: None. If you leave this parameter blank, encoding is not applied.

**Table 6** Content
Parameter	Type	Description
key	str	Explanation: Object name. An object is uniquely identified by an object name in a PFS. An object name is the complete path of the object without the PFS name. For example, if the access path is examplebucket.obs.ap-southeast-1.myhuaweicloud.com/folder/test.txt, the object name is folder/test.txt. Value range: The value can contain 1 to 1,024 characters. Default value: None
lastModified	str	Explanation: Time when the object was last modified. Restrictions: The time must be in the ISO8601 format. Example: 2018-01-01T00:00:00.000Z Default value: None
etag	str	Explanation: Base64-encoded, 128-bit MD5 value of an object. ETag is the unique identifier of the object contents, used to determine whether the contents of an object are changed. For example, if the ETag value is A when an object is uploaded and becomes B when the object is downloaded, this indicates the contents of the object were changed. The ETag reflects changes of an object, not of the metadata. Objects created by the upload and copy operations have unique ETags after being encrypted using MD5. Restrictions: If an object is encrypted using server-side encryption, the ETag is not the MD5 value of the object. Value range: The value must contain 32 characters. Default value: None
size	int	Explanation: Object size Value range: The value ranges from 0 TB to 48.8 TB, in bytes. Default value: None
owner	Owner	Explanation: Object owner Value range: See Table 7.
storageClass	str	Explanation: Object storage class. Value range: If the storage class is Standard, leave this parameter blank. For details about the available storage classes, see Table 9. Default value: None
isAppendable	bool	Explanation: Whether the object is appendable Value range: True: The object is appendable. False: The object is not appendable. Default value: None

**Table 7** Owner
Parameter	Type	Mandatory (Yes/No)	Description
owner_id	str	Yes if used as a request parameter	Explanation: Account (domain) ID of the owner Value range: To obtain the account ID, see How Do I Get My Account ID and IAM User ID? (SDK for Python) Default value: None
owner_name	str	No if used as a request parameter	Explanation: Account name of the owner Value range: To obtain the account ID, see How Do I Get My Account ID and IAM User ID? (SDK for Python) Default value: None

**Table 8** CommonPrefix
Parameter	Type	Description
prefix	str	Explanation: List of object name prefixes grouped according to the delimiter parameter (if specified) Value range: The value must contain 1 to 1,024 characters. Default value: None

**Table 9** StorageClass
Parameter	Type	Description
STANDARD	Standard storage class	Explanation: Features low access latency and high throughput and is used for storing massive, frequently accessed (multiple times a month) or small objects (< 1 MB) requiring quick response.
WARM	Infrequent Access storage class	Explanation: Used for storing data that is semi-frequently accessed (fewer than 12 times a year) but is instantly available when needed.
COLD	Archive storage class	Explanation: Used for storing rarely accessed (once a year) data.
INTELLIGENT_TIERING	Intelligent Tiering	Explanation: Is designed to optimize storage costs by automatically moving data to a more economical access tier when data access patterns change. This storage class is ideal for data with constantly changing or unpredictable access patterns.

Code Examples

This example lists objects with test/ as the prefix in parallel file system examplebucket and specifies a maximum of 100 objects to return at a time.

    
     
       
       
         from obs import ObsClient
import os
import traceback

# Obtain an AK and SK pair using environment variables or import the AK and SK pair in other ways. Using hard coding may result in leakage.
# Obtain an AK and SK pair on the management console. For details, see https://support.huaweicloud.com/intl/en-us/usermanual-ca/ca_01_0003.html.
ak = os.getenv("AccessKeyID")
sk = os.getenv("SecretAccessKey")
# (Optional) If you use a temporary AK and SK pair and a security token to access OBS, obtain them from environment variables.
# security_token = os.getenv("SecurityToken")
# Set server to the endpoint corresponding to the bucket. CN-Hong Kong is used here as an example. Replace it with the one in use.
server = "https://obs.ap-southeast-1.myhuaweicloud.com" 

# Create an obsClient instance.
# If you use a temporary AK and SK pair and a security token to access OBS, you must specify security_token when creating an instance.
obsClient = ObsClient(access_key_id=ak, secret_access_key=sk, server=server)
try:
    bucketName = "examplebucket"
    # Specify an object prefix.
    prefix = 'test/'
    # Specify the maximum number of objects to be listed at a time. 100 is used as an example.
    max_keys = 100
    # Delimiter used to group object names
    delimiter = "/"
    # List objects in the parallel file system.
    resp = obsClient.listObjects(bucketName, prefix, max_keys=max_keys,delimiter=delimiter, encoding_type='url')

    # If status code 2xx is returned, the API was called successfully. Otherwise, the call failed.
    if resp.status < 300:
        print('List Objects Succeeded')
        print('requestId:', resp.requestId)
        print('name:', resp.body.name)
        print('prefix:', resp.body.prefix)
        print('max_keys:', resp.body.max_keys)
        print('is_truncated:', resp.body.is_truncated)
        index = 1
        for content in resp.body.contents:
            print('object [' + str(index) + ']')
            print('key:', content.key)
            print('lastModified:', content.lastModified)
            print('etag:', content.etag)
            print('size:', content.size)
            print('storageClass:', content.storageClass)
            print('owner_id:', content.owner.owner_id)
            print('owner_name:', content.owner.owner_name)
            index += 1
    else:
        print('List Objects Failed')
        print('requestId:', resp.requestId)
        print('errorCode:', resp.errorCode)
        print('errorMessage:', resp.errorMessage)
except:
    print('List Objects Failed')
    print(traceback.format_exc())

        

      

    
   