Updated on 2026-08-21 GMT+08:00

Bucket Inventories

Listing millions or even tens of billions of objects in a bucket is slow and expensive. OBS provides a bucket inventory feature that periodically generates inventory files containing object details, such as size, storage class, and last modification time, and uploads those files to the bucket you specify. This gives you a more efficient way to manage large-scale object sets.

Scenarios

A bucket inventory periodically generates a CSV file that contains information about objects in a bucket and uploads the file to the bucket you specify.

  • Inventories can be generated for objects with a specific prefix.
  • Each inventory file includes the object version, size, storage class, encryption status, and last modification time.
  • Inventory files can be encrypted with SSE-KMS.
  • Inventory files can be generated daily or weekly.
  • Inventory files can be stored in a bucket, including the source bucket.

Source and destination buckets are described as follows:

  • Source bucket: the bucket for which the inventory is configured. The inventory file lists objects stored in this bucket.
  • Destination bucket: the bucket that stores the generated inventory files. The source and destination buckets must belong to the same account and region, and they can be the same bucket.

How an Inventory Works

An inventory works as follows:

  1. Inventory configuration: Specify the destination bucket, generation frequency, object version, and object information fields. Add a bucket policy that allows OBS to upload inventory files to the destination bucket.
  2. Object scan: OBS periodically scans all objects that match the configured rules, including filters such as object prefix and object version. Scans run daily or weekly.
  3. Inventory file generation and storage: OBS generates a full inventory file from the scan results and stores it in the destination bucket as a CSV file.

OBS performs these scans and generates inventory files in the background. This does not affect normal bucket operations.

Constraints

Table 1 Constraints on bucket inventories

Item

Description

Bucket version

Inventories can be generated only for OBS 3.0 buckets, but the bucket that stores the inventory files can be any OBS version. To check the bucket version, see Checking OBS Version (OBS 2.0 or OBS 3.0)

Maximum number of bucket inventories

A bucket can have a maximum of 10 inventories.

Source and destination buckets

  • The source bucket (for which a bucket inventory rule is configured) and the destination bucket (where the generated inventory files are stored) must belong to the same account.
  • The source and destination buckets must be in the same region.
  • The destination bucket cannot have server-side encryption enabled.

Functions

  • Inventory files must be in the CSV format.
  • Inventories can apply to all objects in a bucket or to a set of objects with the same name prefix.
  • The filter criteria of multiple inventory rules within the same bucket must not overlap.
    • If there is already an inventory rule for all objects in the bucket, any other inventory rule with an object name prefix specified cannot be created. To create a rule for only a set of objects, delete the inventory rule that applies to all objects first.
    • If there is already an inventory rule for a set of objects, a rule for all objects in the bucket cannot be created. To create a rule for all objects, delete all inventory rules that use a prefix filter.
    • Assume there is an inventory rule that uses the ab prefix as a filter. Any new inventory rule whose prefix overlaps with ab, such as a or abc, cannot be created. To create a rule with prefix a or abc, delete the overlapping rule first.
  • Bucket inventories can collect encryption information only for objects encrypted with SSE-KMS.

Permissions

OBS must be authorized to upload inventory files to the destination bucket. The destination bucket needs a policy that grants OBS write permissions, with the Principal set to {"Service": "obs"}. For details, see Adding a Bucket Policy for the Destination Bucket.

Others

  • The bucket inventory feature itself is free. You are billed only for the storage used by the generated inventory files.
  • The bucket inventory feature is not available to federated users.
    NOTE:

    Federated users are created in an enterprise management system. After the account administrator creates an identity provider on the IAM console, federated users can log in to Huawei Cloud and use cloud services based on assigned permissions.

Billing

Inventory files are free, but the following related charges apply:

  • Request costs: Uploading inventory files to the destination bucket incurs PUT request costs. Downloading or reading inventory files incurs GET request costs. For details, see Billing for Requests.
  • Storage costs: Storing inventory files in the destination bucket incurs storage costs. For details, see Billing for Storage.
  • Outbound Internet traffic costs: Downloading inventory files over the Internet incurs outbound Internet traffic costs.

    Traffic transferred from OBS to Huawei Cloud ECSs, CCEs, and BMSs in the same region over an intranet is free.

To avoid unnecessary costs, delete unneeded inventory files promptly (manually or through lifecycle rules). If you no longer need the inventory, delete its configuration.

Access Control

When configuring a bucket inventory, you must configure a policy on the destination bucket to allow OBS to upload generated inventory files. If you configure the inventory on OBS Console, OBS automatically creates this policy. If you configure the inventory through the REST API, you must add the policy manually. For details, see Configuring a Bucket Policy.

The following is an example bucket policy. Replace destbucket with the actual destination bucket name.

{
	"Statement": [
		{
			"Effect": "Allow",
			"Sid": "1",
			"Principal": {"Service": "obs"},
			"Resource": ["destbucket/*"],
			"Action": ["PutObject"]
		}
	]
}

Inventory File Path

The destination bucket stores each inventory file in the following path:

destinationPrefix/sourceBucketName/inventoryId/yyyy-MM-dd'T'HH-mm'Z'/files/UUID_number.csv

Example:

obj/test-zcy-bucket-002/inventory-861c/2026-06-11T03-04Z/files/0000019EB4A3D9C7B8EA68E19D649A41_1.csv
Figure 1 Path of an inventory file
Table 2 Parameters in the inventory file path

Parameter

Description

destinationPrefix

The inventory file prefix in the configuration. Generated inventory files start with this prefix, which helps group them.

If this parameter is not specified, BucketInventory is used by default.

sourceBucketName

The source bucket for which the inventory is configured. This prevents conflicts when inventory files from multiple source buckets are stored in the same destination bucket.

inventoryId

The inventory name in the configuration. This prevents conflicts when multiple inventories of the same bucket store files in the same destination bucket.

This directory contains the hive/dt=yyyy-MM-dd-HH-mm/symlink.txt file, for example, hive/dt=2026-06-11-00-00/symlink.txt.

yyyy-MM-dd'T'HH-mm'Z'

The date and time when the bucket scan started. Example: 2026-06-11T03-04Z

The time is in GMT. Add 8 hours to convert it to Beijing time.

The inventory file does not contain objects uploaded after this scan time, so the number of objects in the file may differ from the actual number in the bucket.

This directory contains the manifest.json file.

files

A fixed directory. Each inventory file path includes this directory. It ensures that the manifest.json file and the inventory file are stored separately.

UUID_number.csv

The inventory file name. Inventory files are stored as CSV files in the destination bucket.

Generated Files

OBS generates three types of files for each inventory run: CSV inventory files, manifest.json files, and symlink.txt files.

The content of an inventory file is determined by the inventory configuration. The following is an example inventory file. For details about all possible fields, see Table 3.

Figure 2 An inventory file
Table 3 Object metadata fields in an inventory file

Metadata

Description

Bucket

The source bucket for which the inventory task is executed

Key

The object name. Each object in a bucket has a unique key.

Object names in an inventory file are encoded using UTF-8 URL encoding and must be decoded before use. In Figure 2:

  • testfolder%2F indicates the object testfolder/, which is shown as a folder on the console.
  • testfolder%2Ftest.txt indicates the object testfolder/test.txt, which is shown as test.txt under the folder testfolder.
  • test%2D01.txt indicates the object test-01.txt.

VersionId

The object version ID

This field appears only when Object Versions is set to Include all versions in the configuration.

  • If versioning is disabled for the source bucket, this field is left blank.
  • If versioning is enabled for the source bucket, this field shows the object's version ID.

IsLatest

Indicates whether the object version is the current version. This field appears only when Object Versions is set to Include all versions in the configuration.

  • If versioning is disabled for the source bucket, this field shows TRUE.
  • If versioning is enabled for the source bucket and the object version is the current version, this field shows TRUE; if the object version is a historical version, this field shows FALSE.

IsDeleteMarker

Indicates whether the object version is a delete marker. After versioning is enabled for a bucket, deleting an object inserts a delete marker. The delete marker becomes the current version and indicates that the object is in the deleted state.

This field appears only when Object Versions is set to Include all versions in the configuration.

  • If versioning is disabled for the source bucket, this field shows FALSE.
  • If versioning is enabled for the source bucket and the object version is a delete marker, this field shows TRUE; if the object version is not a delete marker, this field shows FALSE.

Size

The object size, in bytes

This field corresponds to Size in the Optional Fields area of the configuration.

LastModifiedDate

The last time the object was modified before the inventory file was generated. If the object has never been modified, this field shows the object's creation time.

The time is in GMT. Add 8 hours to convert it to Beijing time. For example, 2026-04-01T07:33:48Z corresponds to 15:33:48 on April 1, 2026 in Beijing time.

This field corresponds to Last modified date in the Optional Fields area of the configuration.

ETag

The unique identifier of the object content. It reflects whether the content has changed. For example, if the ETag is A when the object is uploaded but becomes B when it is downloaded, the content has changed.

  • If the object is uploaded using PUT, the ETag is the MD5 hash of the object.
  • If the object is uploaded using multipart upload, the ETag is a multipart-MD5 value, regardless of the encryption method. This value is calculated from the MD5 checksums of the individual parts, so it is not the MD5 hash of the complete object.

ETag reflects changes to object content, not to object metadata.

Objects uploaded or created by copy operations each have a unique ETag generated from an MD5 checksum.

This field corresponds to ETag in the Optional Fields area of the configuration.

StorageClass

The object's storage class. Its value can be:

  • STANDARD: the Standard storage class
  • STANDARD_IA (also WARM): the Infrequent Access storage class
  • GLACIER (also COLD): the Archive storage class
  • DEEP_ARCHIVE: the Deep Archive storage class

This field corresponds to Storage class in the Optional Fields area of the configuration.

IsMultipartUploaded

Indicates whether the object was uploaded using multipart upload. Its value can be:

  • TRUE: The object was uploaded using multipart upload.
  • FALSE: The object was not uploaded using multipart upload.

This field corresponds to Multipart upload in the Optional Fields area of the configuration.

ReplicationStatus

The cross-region replication status of the object. Its value can be:

  • NOREP: The object was not replicated from another region.
  • REP: The object was replicated from another region.

This field corresponds to Replication status in the Optional Fields area of the configuration.

EncryptionStatus

The encryption status of the object. Its value can be:

  • SSE-KMS: The object is encrypted using SSE-KMS.
  • NOT-SSE: The object is not encrypted or is not encrypted using SSE-KMS.

This field corresponds to Encryption status in the Optional Fields area of the configuration.

BucketKeyStatus

Indicates whether the bucket key is enabled for the source bucket. Its value can be:

  • ENABLED: The bucket key is enabled for the source bucket.
  • DISABLED: The bucket key is disabled for the source bucket.

This field corresponds to Bucket key status in the Optional Fields area of the configuration.

If a bucket contains a large number of objects, a single inventory configuration may generate multiple inventory files. Generating these files takes time. For example, generating inventory files for 200,000 objects takes about 1.5 minutes. After all inventory files are generated, OBS creates a manifest.json file one to two hours later. The manifest.json file contains information about all inventory files generated in that cycle.

  • Storage path of the manifest.json file

    The destination bucket stores each manifest.json file in the following path:

    destinationPrefix/sourceBucketName/inventoryId/yyyy-MM-dd'T'HH-mm'Z'/manifest.json

    For details about the parameters in the file path, see Inventory File Path.

    Example:

    obj/test-zcy-bucket-002/inventory-861c/2026-06-11T03-04Z/manifest.json
    Figure 3 Storage path of the manifest.json file
  • manifest.json file example and the fields in the file
    {
       "sourceBucket": "test-zcy-bucket-002",
       "destinationBucket": "test-zcy-bucket-001",
       "version": "2026-06-11",
       "fileFormat": "CSV",
       "fileSchema": "Bucket,Key,VersionId,IsLatest,IsDeleteMarker,Size,LastModifiedDate,StorageClass,ETag,IsMultipartUploaded,ReplicationStatus,EncryptionStatus,BucketKeyStatus,CRC64",
       "files": [{
          "key": "obj%2Ftest-zcy-bucket-002%2Finventory-861c%2F2026-06-11T03-04Z%2Ffiles%2F0000019EB4A3D9C7B8EA68E19D649A41_1.csv",
          "size": 76782,
          "inventoriedRecord": 376
       }]
    }

    Table 4 describes the fields in the file.

    Table 4 Fields in the manifest file

    Parameter

    Description

    sourceBucket

    The name of the source bucket

    destinationBucket

    The name of the destination bucket

    version

    The inventory version

    fileFormat

    The inventory file format

    fileSchema

    The object metadata fields included in the inventory file. These listed match those listed in Table 3.

    files

    The list of all inventory files. Each inventory file contains the key, size, and inventoriedRecord fields.

    key

    The name of an inventory file, which is described in Inventory File Path. The name is encoded using UTF-8 URL encoding and must be decoded before display.

    For example, obj%2Ftest-zcy-bucket-002%2Finventory-861c%2F2026-06-11T03-04Z%2Ffiles%2F0000019EB4A3D9C7B8EA68E19D649A41_1.csv decodes to obj/test-zcy-bucket-002/inventory-861c/2026-06-11T03-04Z/files/0000019EB4A3D9C7B8EA68E19D649A41_1.csv.

    size

    The size of an inventory file, in bytes

    inventoriedRecord

    The number of object records included in an inventory file

The symlink.txt file records the path of each inventory file. It helps quickly locate inventory files in big data scenarios. The symlink.txt file is Apache Hive-compatible. Hive can automatically detect the symlink.txt file and the inventory files listed in it. symlink.txt files are sorted by object name in lexicographical order.

  • Storage path of the symlink.txt file

    The destination bucket stores each symlink.txt file in the following path:

    destinationPrefix/sourceBucketName/inventoryId/hive/dt=yyyy-MM-dd-HH-mm/symlink.txt

    For details about the parameters in the file path, see Inventory File Path.

    Example:

    obj/test-zcy-bucket-002/inventory-861c/hive/dt=2026-06-11-00-00/symlink.txt
    Figure 4 Storage path of the symlink.txt file
  • symlink.txt file example and the fields in the file
    obs://test-zcy-bucket-001/obj/test-zcy-bucket-002/inventory-861c/2026-06-11T03-04Z/files/0000019EB4A3D9C7B8EA68E19D649A41_1.csv

    In the file, test-zcy-bucket-001 indicates the name of the destination bucket that stores the inventory file. For details about other fields, see Inventory File Path.

Important Notes

If you operate objects in the source bucket while a bucket inventory is being generated, the number of objects in the inventory file may differ from the actual number of objects in the bucket. An object is excluded from the inventory file if its last modification time is later than the version timestamp recorded in the manifest.json file.

Recommendations

  • Least privilege: Before configuring bucket inventory using REST APIs, manually configure a bucket policy on the destination bucket. Follow the principle of least privilege and grant only the required PutObject permission. Do not assign broader or unnecessary permissions.
  • Performance: When selecting a bucket to store inventory files, choose one with low service traffic. Uploading a large number of inventory files to a high-traffic bucket may cause bandwidth contention and affect service access.
  • Cost reduction: Bucket inventory can be generated daily or weekly. For buckets with a large number of objects (for example, millions or tens of billions), configure weekly generation. In addition, configure a lifecycle rule to automatically delete inventory files older than a specified number of days (such as 30 or 60 days) to reduce storage costs.
  • Monitoring and alarms: Configure storage-usage alarms for the destination bucket and periodically check its usage. This helps prevent unexpected cost increases caused by uncontrolled growth of inventory files.
  • Inventory modification: Changes to bucket inventory configurations can affect downstream data analysis. Plan configuration updates in advance and notify all key stakeholders to avoid unintended impacts.

Ways to Configure a Bucket Inventory

You can use OBS Console, APIs, or SDKs to configure a bucket inventory. You cannot use OBS Browser+ or obsutil to do so.

References