Listing Objects in a Parallel File System (SDK for Go)

Function

This API lists some or all of the objects in a parallel file system (PFS). You can configure the prefix, number, and start position to list objects that meet specified criteria. Returned objects are listed in alphabetical order by object name.

Restrictions

A maximum of 1,000 objects can be listed for each API call.
To list objects in a PFS, you must be the PFS owner or have the required permission (obs:bucket:ListBucket in IAM or ListBucket in a policy). For details, see Introduction to OBS Access Control, IAM Custom Policies, and Creating a Custom Bucket Policy.
The mapping between OBS regions and endpoints must comply with what is listed in Regions and Endpoints.

Method

func (obsClient ObsClient) ListObjects(input *ListObjectsInput) (output *ListObjectsOutput, err error)

Request Parameters

**Table 1** List of request parameters
Parameter	Type	Mandatory (Yes/No)	Description
input	*ListObjectsInput	Yes	Explanation: Input parameters for listing objects in a PFS. For details, see Table 2.

**Table 2** ListObjectsInput
Parameter	Type	Mandatory (Yes/No)	Description
Bucket	string	Yes	Explanation: PFS name. Restrictions: A PFS name must be unique across all accounts and regions. A PFS name: Must be 3 to 63 characters long and start with a digit or letter. Lowercase letters, digits, hyphens (-), and periods (.) are allowed. Cannot be formatted as an IP address. Cannot start or end with a hyphen (-) or period (.). Cannot contain two consecutive periods (..), for example, my..bucket. Cannot contain a period (.) and a hyphen (-) adjacent to each other, for example, my-.bucket or my.-bucket. If you repeatedly create PFSs with the same name in the same region, no error will be reported and the PFS attributes comply with those set in the first creation request. Default value: None
Marker	string	No	Explanation: Object name to start with when listing objects in a bucket. Objects after the specified object are returned in alphabetical order. Restrictions: This parameter is only used for listing objects with a single version. Value range: The value can contain 1 to 1,024 characters. Default value: None
Prefix	string	No	Explanation: Name prefix that the objects to be listed must contain Assume that you have the following objects: logs/day1, logs/day2, logs/day3, and ExampleObject.jpg. If you specify logs/ as the prefix, logs/day1, logs/day2, and logs/day3 will be returned. If this parameter is left blank and there are no other filter criteria, all objects in the PFS will be returned. Value range: The value can contain 1 to 1,024 characters. Default value: None
MaxKeys	int	No	Explanation: The maximum number of objects returned in the response in alphabetical order Value range: The value ranges from 1 to 1000. If the specified value is greater than 1000, only 1,000 objects are returned. Default value: 1000
Delimiter	string	No	Explanation: Object names are grouped by this parameter, which is often used with Prefix. If a prefix is specified, objects with the same string from the prefix to the first delimiter are grouped into one CommonPrefixes. If no prefix is specified, objects with the same string from the first character to the first delimiter are grouped into one CommonPrefixes. Assume that a PFS has objects abcd, abcde, and bbcde in it. If delimiter is set to d and prefix is set to a, objects abcd and abcde are grouped into a CommonPrefixes with abcd as the prefix. If only delimiter is set to d, files abcd and abcde are grouped into a CommonPrefixes with abcd as the prefix, and bbcde is grouped separately into another CommonPrefixes with bbcd as the prefix. For a PFS, if this parameter is not specified, all the contents in the current directory are recursively listed by default, and subdirectories are also listed. In big data scenarios, PFSs usually have multiple directory levels and each directory level has a large number of objects. In such case, you are advised to configure [delimiter=/] to list only the content in the current directory, without the content in subdirectories, to make the listing more efficient. Value range: The value can contain 1 to 1,024 characters. Default value: None
EncodingType	string	No	Explanation: Encoding type for some elements in the response. If Delimiter, Marker, Prefix, CommonPrefixes, NextMarker, and Key contain control characters (special characters) that are not supported by the XML 1.0 standard, set this parameter to url. Value range: url Default value: None. If you leave this parameter blank, encoding is not applied.

Responses

**Table 3** List of returned results
Parameter	Type	Description
output	*ListObjectsOutput	Explanation: Response parameters for listing objects in a PFS. For details, see Table 4.
err	error	Explanation: Error messages returned by the API

**Table 4** ListObjectsOutput
Parameter	Type	Description
StatusCode	int	Explanation: HTTP status code Value range: A status code is a group of digits indicating the status of a response. It ranges from 2xx (indicating successes) to 4xx or 5xx (indicating errors). For more information, see Status Code. Default value: None
RequestId	string	Explanation: Request ID returned by the OBS server Default value: None
ResponseHeaders	map[string][]string	Explanation: HTTP response headers Default value: None
Location	string	Explanation: Region where the PFS is located. Restrictions: If the specified endpoint is obs.myhuaweicloud.com, this parameter is not required. If any other endpoints are specified, this parameter is required. Value range: To learn about valid regions and endpoints, see Regions and Endpoints. An endpoint is the request address for calling an API. Endpoints vary depending on services and regions. To obtain the regions and endpoints, contact the enterprise administrator. Default value: If obs.myhuaweicloud.com is used as the endpoint and no region is specified, cn-north-1 (the CN North-Beijing1 region) is used by default.
Name	string	Explanation: PFS name. Restrictions: A PFS name must be unique across all accounts and regions. A PFS name: Must be 3 to 63 characters long and start with a digit or letter. Lowercase letters, digits, hyphens (-), and periods (.) are allowed. Cannot be formatted as an IP address. Cannot start or end with a hyphen (-) or period (.). Cannot contain two consecutive periods (..), for example, my..bucket. Cannot contain a period (.) and a hyphen (-) adjacent to each other, for example, my-.bucket or my.-bucket. If you repeatedly create PFSs with the same name in the same region, no error will be reported and the PFS attributes comply with those set in the first creation request. Value range: The value can contain 3 to 63 characters.
Prefix	string	Explanation: Object name prefix, which is consistent with that set in the request. Assume that you have the following objects: logs/day1, logs/day2, logs/day3, and ExampleObject.jpg. If you specify logs/ as the prefix, logs/day1, logs/day2, and logs/day3 will be returned. If this parameter is left blank and there are no other filter criteria, all objects in the PFS will be returned. Value range: The value can contain 1 to 1,024 characters. Default value: None
Marker	string	Explanation: Object name to start with for listing objects. All objects following the specified object are returned in alphabetical order by object name. Restrictions: This parameter is only used for listing objects with a single version. Value range: The value can contain 1 to 1,024 characters. Default value: None
Delimiter	string	Explanation: Object names are grouped by this parameter, which is often used with Prefix. If a prefix is specified, objects with the same string from the prefix to the first delimiter are grouped into one CommonPrefixes. If no prefix is specified, objects with the same string from the first character to the first delimiter are grouped into one CommonPrefixes. Assume that a PFS has objects abcd, abcde, and bbcde in it. If delimiter is set to d and prefix is set to a, objects abcd and abcde are grouped into a CommonPrefixes with abcd as the prefix. If only delimiter is set to d, files abcd and abcde are grouped into a CommonPrefixes with abcd as the prefix, and bbcde is grouped separately into another CommonPrefixes with bbcd as the prefix. For a PFS, if this parameter is not specified, all the contents in the current directory are recursively listed by default, and subdirectories are also listed. In big data scenarios, PFSs usually have multiple directory levels and each directory level has a large number of objects. In such case, you are advised to configure [delimiter=/] to list only the content in the current directory, without the content in subdirectories, to make the listing more efficient. Value range: The value can contain 1 to 1,024 characters. Default value: None
MaxKeys	int	Explanation: The maximum number of objects returned in the response in alphabetical order. Value range: The value ranges from 1 to 1000. If the specified value is greater than 1000, only 1,000 objects are returned. Default value: 1000
IsTruncated	bool	Explanation: Whether all results are returned in the response. A maximum of 1,000 objects can be listed at a time. If the number of objects is greater than 1,000, the objects beyond 1,000 cannot be returned. Value range: true: Not all results are returned. false: All results are returned. Default value: None
NextMarker	string	Explanation: Where in the bucket the next listing begins. If not all results are returned, the response contains this parameter to mark the last object listed in the request. In a subsequent request, you can set Marker to the value of this parameter to list the remaining objects. Value range: The value can contain 1 to 1,024 characters. Default value: None
Contents	[]Content	Explanation: List of objects in the PFS. For details, see Table 5.
CommonPrefixes	[]string	Explanation: List of object name prefixes grouped according to the Delimiter parameter (if specified) Value range: The value can contain 1 to 1,024 characters. Default value: None
EncodingType	string	Explanation: Encoding type for some elements in the response. If Delimiter, Marker, Prefix, CommonPrefixes, NextMarker, and Key contain control characters (special characters) that are not supported by the XML 1.0 standard, set this parameter to url. Value range: url Default value: None

**Table 5** Content
Parameter	Type	Description
Key	string	Explanation: Object name. An object is uniquely identified by an object name in a PFS. An object name is the complete path of the object without the PFS name. For example, if the address for accessing the object is examplebucket.obs.ap-southeast-1.myhuaweicloud.com/folder/test.txt, the object name is folder/test.txt. Value range: The value can contain 1 to 1,024 characters. Default value: None
LastModified	time.Time	Explanation: Time when the object was last modified. Value range: UTC time Default value: None
ETag	string	Explanation: Base64-encoded, 128-bit MD5 value of an object. ETag is the unique identifier of the object contents that is used to determine whether the contents of an object are changed. For example, if the ETag value is A when an object is uploaded and becomes B when the object is downloaded, this indicates the contents of the object were changed. The ETag reflects changes of an object, not of the metadata. Objects created by the upload and copy operations have unique ETags after being encrypted using MD5. Restrictions: If an object is encrypted using server-side encryption, the ETag is not the MD5 value of the object. Value range: The value must contain 32 characters. Default value: None
Size	int64	Explanation: Object size in bytes Value range: The value ranges from 0 TB to 48.8 TB, in bytes. Default value: None
Owner	Owner	Explanation: Object owner. This parameter contains the domain ID and name of the object owner. For details, see Table 7. Default value: None
StorageClass	string	Explanation: Object storage class Value range: See Table 6. Default value: None

**Table 6** StorageClassType
Constant	Default Value	Description
StorageClassStandard	STANDARD	OBS Standard Features low access latency and high throughput and is used for storing massive, frequently accessed (multiple times a month) or small objects (< 1 MB) requiring quick response.
StorageClassWarm	WARM	OBS Infrequent Access Used for storing data that is semi-frequently accessed (fewer than 12 times a year) but is instantly available when needed.
StorageClassCold	COLD	OBS Archive Used for storing rarely accessed (once a year) data.

**Table 7** Owner
Parameter	Type	Mandatory (Yes/No)	Description
ID	string	Yes if used as a request parameter	Explanation: Account (domain) ID of the owner Value range: To obtain the account ID, see How Do I Get My Account ID and User ID? Default value: None

Code Examples

This example lists all files in a PFS examplebucket using pagination.

    
     
       
       
         // This example lists all files in examplebucket using pagination.
package main
import (
    "fmt"
    "os"
    obs "github.com/huaweicloud/huaweicloud-sdk-go-obs/obs"
)
func main() {
    // Obtain an AK/SK pair using environment variables or import an AK/SK pair in other ways. Using hard coding may result in leakage.
    // Obtain an AK/SK pair on the management console. For details, see https://support.huaweicloud.com/intl/en-us/usermanual-ca/ca_01_0003.html.
    ak := os.Getenv("AccessKeyID")
    sk := os.Getenv("SecretAccessKey")
    // (Optional) If you use a temporary AK/SK pair and a security token to access OBS, you are advised not to use hard coding, which may result in information leakage. You can obtain an AK/SK pair using environment variables or import an AK/SK pair in other ways.
    securityToken := os.Getenv("SecurityToken")
    // Enter the endpoint corresponding to the region where the bucket is located. CN-Hong Kong is used here as an example. Replace it with the one currently in use.
    endPoint := "https://obs.ap-southeast-1.myhuaweicloud.com"
    // Create an obsClient instance.
    // If you use a temporary AK/SK pair and a security token to access OBS, use the obs.WithSecurityToken method to specify a security token when creating an instance.
    obsClient, err := obs.New(ak, sk, endPoint, obs.WithSecurityToken(securityToken))
    if err != nil {
        fmt.Printf("Create obsClient error, errMsg: %s", err.Error())
    }
    input := &obs.ListObjectsInput{}
    // Specify the name of the PFS.
    input.Bucket = "examplebucket"
    // Specify the maximum number of objects to be returned in alphabetic order. The default value is 1000. 100 is used as an example.
    input.MaxKeys = 100
    // Specify an encoding type. url is used as an example. If the objects to list contain special characters, this parameter must be passed.
    input.EncodingType = "url"
    for {
        // List objects in the PFS.
        output, err := obsClient.ListObjects(input)
        if err != nil {
            fmt.Printf("List objects under the bucket(%s) fail!\n", input.Bucket)
            if obsError, ok := err.(obs.ObsError); ok {
                fmt.Println("An ObsError was found, which means your request sent to OBS was rejected with an error response.")
                fmt.Println(obsError.Error())
            } else {
                fmt.Println("An Exception was found, which means the client encountered an internal problem when attempting to communicate with OBS, for example, the client was unable to access the network.")
                fmt.Println(err)
            }
            os.Exit(-1)
        }
        // Print the result. By default, 100 records are returned at a time.
        fmt.Printf("List objects under the bucket(%s) successful!\n", input.Bucket)
        for index, val := range output.Contents {
            fmt.Printf("Content[%d]-OwnerId:%s, ETag:%s, Key:%s, LastModified:%s, Size:%d\n",
                index, val.Owner.ID, val.ETag, val.Key, val.LastModified, val.Size)
        }
        if output.IsTruncated {
            input.Marker = output.NextMarker
        } else {
            break
        }
    }
}

        

      

    
   