Logging Overview

Scenarios

Bucket access generates a large number of logs, but OBS does not collect them by default. To analyze the characteristics, types, or trends of requests to a bucket, you can enable logging for the bucket. OBS will automatically create the log files according to certain naming rules and upload them to the specified log storage bucket (the current bucket or another bucket in the same region).

Constraints

You can store logs in the logged bucket or another bucket. However, the log storage bucket and logged bucket must belong to the same account and region.

Important Notes

After logging is configured for a bucket, its operation logs will appear in the log storage bucket in about fifteen minutes. Log files for a given period may not contain all requests made during that period. Some requests may appear in the log files of the previous or next period.

Log File Naming Rules

• If <Log file name prefix> ends with a slash (/), the bucket's log files are stored in the folder named <Log file name prefix> in the target bucket. Each log file is named YYYY-mm-DD-HH-MM-SS-<UniqueString>.
  Figure 1 Log folder name
  
  Figure 2 Log file names
  
• If <Log file name prefix> does not end with a slash (/), the bucket's log files are stored in the root directory of the target bucket. Each log file is named <Log file name prefix>YYYY-mm-DD-HH-MM-SS-<UniqueString>.
  Figure 3 Log file names
  

Log Fields

• Example of log content

  The following shows an example access log stored in the target bucket:

  787f2f92b20943998a4fe2ab75eb09b8 bucket [13/Aug/2015:01:43:42 +0000] xx.xx.xx.xx 
  787f2f92b20943998a4fe2ab75eb09b8 281599BACAD9376ECE141B842B94535B  REST.GET.BUCKET.LOCATION 
  - "GET /bucket?location HTTP/1.1" 200 - 211 - 6 6 "-"  "HttpClient" - - - - "-" 089fe8c2c380f4031f6dc0197fe99d4d HPTAAFZfyW0yD80idvjw

• Log content format

  The log content contains the entries listed in the following table.

  Table 2 Bucket log format

  Parameter	Description	Example Value
  BucketOwner	Account ID of the bucket owner	787f2f92b20943998a4fe2ab75eb09b8
  BucketName	Bucket name	bucket
  Time	UTC timestamp when OBS received the request	[13/Aug/2015:01:43:42 +0000]
  Remote IP	IP address of the request source	192.168.7.132
  Requester	Requester ID. The value can be: Anonymous: indicating that the request was made by an anonymous user Account ID of the requester: indicating that the request was made by an account or an IAM user	787f2f92b20943998a4fe2ab75eb09b8
  RequestID	ID of the request for performing operations on the bucket	281599BACAD9376ECE141B842B94535B
  Operation	Operation type of the request For common operations and their description, see Table 3.	REST.GET.BUCKET.LOCATION
  Key	Name of the requested object	-
  Request-URI	URI used in the request for operating OBS resources. For details about URIs, see Constructing a Request. NOTE: If query_string is used for signature, the Request-URI will contain the signature information; otherwise, it will not.	GET /bucket?location HTTP/1.1
  HTTPStatus	HTTP status code returned by OBS	200
  ErrorCode	Error code returned by OBS. - indicates that no error code was returned.	-
  BytesSent	Size of the HTTP response body Unit: byte - indicates that the HTTP response does not contain a body.	211
  ObjectSize	Object size Unit: byte When OBS deletes an object, it does not log the object's size. In the object deletion log, the value of ObjectSize is 0. If error code 4XX is returned, the value of ObjectSize is -, indicating that the specific object size is not displayed.	-
  TotalTime	Total request duration Total request duration = Time when the last byte of the response was received – Time when the first byte of the request was sent	6
  Turn-AroundTime	Time required by the OBS server to process the request, in ms Time required by the OBS server to process the request = Time when the first byte of the response was sent– Time when the last byte of the request was received	6
  Referer	Referer header of the request The Referrer header is part of the HTTP request header. It indicates the URL from which the user navigated, helping the server identify the source of the request.	-
  User-Agent	User-Agent header of the request The User-Agent header is part of the HTTP request header and identifies the client software that made the request. This header includes information about the browser, operating system, and device type. Servers can use that information to optimize response content or perform statistical analysis.	HttpClient
  VersionID	Object version ID in the request. This parameter specifies the version of the object on which the operation is performed. In a bucket with versioning enabled, an object can have multiple versions. Each version has a unique version ID. For details, see Versioning.	-
  STSLogUrn	Information about federated identity authentication and agency authorization	-
  StorageClass	Current storage class of the object. The value can be: STANDARD: the Standard storage class STANDARD_IA (also WARM): the Infrequent Access storage class COLD: the Archive storage class DEEP_ARCHIVE: the Deep Archive storage class	STANDARD_IA
  TargetStorageClass	Storage class of the object after the transition triggered by the lifecycle rule. The value can be: STANDARD: the Standard storage class WARM: the Infrequent Access storage class COLD: the Archive storage class DEEP_ARCHIVE: the Deep Archive storage class	GLACIER
  DentryName	For a parallel file system, this field indicates an internal identifier of a file or directory. Its value consists of a parent directory inode number and a file or directory name. For a bucket, the value of this field is -.	12456/file.txt
  IAMUserID	IAM user ID. The value can be: IAM user ID: indicating that the request was made by a non-anonymous user Anonymous: indicating that the request was made by an anonymous user	8f3b8c53d29244a780084f2b8c106c32
  AccessKeyID	Access key ID of the requester. The value can be: -: indicating that the request was made by an anonymous user Access key ID of the requester: indicating that the request was made by a non-anonymous user	UDSIAMSTUBTEST002852

  Table 3 Common operations

  Operation	Description	Operation	Description
  REST.GET.SERVICE	Lists buckets.	REST.GET.ENCRYPTION	Obtains the bucket encryption configuration.
  REST.PUT.BUCKET	Creates a bucket.	REST.DELETE.ENCRYPTION	Deletes the bucket encryption configuration.
  REST.HEAD.BUCKET	Views the bucket information.	REST.PUT.OTM_DIRECT_COLD_ACCESS	Configures direct reading for Archive objects in a bucket.
  REST.GET.BUCKETVERSIONS	Lists objects in a bucket.	REST.GET.OTM_DIRECT_COLD_ACCESS	Obtains the direct reading configuration of a bucket.
  REST.GET.BUCKET	Obtains the bucket metadata.	REST.DELETE.OTM_DIRECT_COLD_ACCESS	Deletes the direct reading configuration of a bucket.
  REST.GET.BUCKET.LOCATION	Obtains the bucket location.	REST.PUT.BUCKET.WEBSITE	Configures static website hosting for a bucket.
  REST.DELETE.BUCKET	Deletes a bucket.	REST.GET.BUCKET.WEBSITE	Obtains the static website hosting configuration of a bucket.
  REST.PUT.POLICY	Configures a bucket policy.	REST.DEL.BUCKET.WEBSITE	Deletes the static website hosting configuration of a bucket.
  REST.GET.POLICY	Obtains a bucket policy.	REST.PUT.BUCKET.CORS	Configures CORS for a bucket.
  REST.DELETE.POLICY	Deletes a bucket policy.	REST.GET.BUCKET.CORS	Obtains the CORS configuration of a bucket.
  REST.PUT.ACL	Configures an ACL for a bucket or an object.	REST.DEL.BUCKET.CORS	Deletes the CORS configuration of a bucket.
  REST.GET.ACL	Obtains a bucket ACL or an object ACL.	REST.OPTIONS.BUCKET	Sends an OPTIONS request to a bucket.
  REST.PUT.LOGGING_STATUS	Configures logging for a bucket.	REST.OPTIONS.OBJECT	Sends an OPTIONS request to an object.
  REST.GET.LOGGING_STATUS	Obtains the logging configuration of a bucket.	REST.PUT.OBJECT	Uploads an object with PUT.
  REST.PUT.BUCKET.LIFECYCLE	Configures a lifecycle rule for a bucket.	REST.POST.OBJECT	Uploads an object with POST.
  REST.GET.LIFECYCLE	Obtains the lifecycle configuration of a bucket.	REST.COPY.OBJECT	Copies an object.
  REST.DEL.LIFECYCLE	Deletes the lifecycle configuration of a bucket.	REST.GET.OBJECT	Obtains the object content.
  REST.PUT.VERSIONING	Configures versioning for a bucket.	REST.HEAD.OBJECT	Obtains the object metadata.
  REST.GET.VERSIONING	Obtains the versioning status of a bucket.	REST.DELETE.OBJECT	Deletes an object.
  REST.GET.BUCKET.STORAGE.POLICY	Obtains the storage class of a bucket.	REST.TRANSITION.STORAGECLASS.OBJECT	Changes the storage class of an object.
  REST.PUT.BUCKET.STORAGE.POLICY	Configures a storage class for a bucket.	OP_MULTIPLE_DELETEOBJECT	Batch deletes objects (the batch operation itself).
  REST.PUT.REPLICATION	Configures cross-region replication for a bucket.	REST.POST.RESTORE	Restores an Archive object.
  REST.DELETE.REPLICATION	Deletes the cross-region replication configuration of a bucket.	REST.APPEND.OBJECT	Appends data to an object.
  REST.GET.REPLICATION	Obtains the cross-region replication configuration of a bucket.	REST.MODIFY.OBJECT.META	Modifies the object metadata.
  REST.PUT.TAGGING	Adds tags to a bucket.	REST.TRUNCATE.OBJECT	Truncates an object.
  REST.GET.TAGGING	Obtains the tags of a bucket.	REST.RENAME.OBJECT	Renames an object.
  REST.DEL.TAGGING	Deletes the tags of a bucket.	REST.GET.UPLOADS	Lists the initiated multipart uploads in a bucket.
  REST.PUT.BUCKET_QUOTA	Configures a storage quota for a bucket.	REST.POST.UPLOADS	Initiates a multipart upload.
  REST.GET.BUCKET.QUOTA	Obtains the storage quota of a bucket.	REST.PUT.PART	Uploads a part.
  REST.GET.BUCKET.STORAGEINFO	Queries the storage usage of a bucket.	REST.COPY.PART	Copies a part.
  REST.PUT.BUCKET.INVENTORY	Configures inventories for a bucket.	REST.GET.UPLOAD	Lists uploaded parts.
  REST.GET.BUCKET.INVENTORY	Obtains or lists inventories of a bucket.	REST.POST.UPLOAD	Assembles parts.
  REST.DELETE.BUCKET.INVENTORY	Deletes inventories of a bucket.	REST.DELETE.UPLOAD	Aborts a multipart upload.
  REST.PUT.CUSTOMDOMAIN	Configures a user-defined domain name for a bucket.	REST.CLEAR.EXPIRE.UPLOAD	Deletes expired parts.
  REST.GET.CUSTOMDOMAIN	Obtains the user-defined domain name of a bucket.	REST.DELETE.CUSTOMDOMAIN	Deletes the user-defined domain name of a bucket.
  REST.PUT.ENCRYPTION	Configures encryption for a bucket.	-	-

Billing for Bucket Logs

• Uploading and storing bucket logs will incur costs for PUT requests and storage. For details, see OBS Billing.
• If log files are stored in the logged bucket, OBS creates additional logs for writing log files to the bucket. The additional logs will take up extra storage. This leads to higher costs and makes it harder to locate required logs. Therefore, you are advised to store log files in a bucket other than the logged bucket.
• You can delete unnecessary log files to reduce storage costs. Configuring lifecycle rules for scheduled deletion is recommended.

Permissions Related to Bucket Logging

• Enabling logging will automatically grant log delivery users the write and ACL read permissions for the log storage bucket.

  If you manually disable such permissions, bucket logging will fail.

  

• OBS creates log files and uploads them to a specified bucket. To perform these operations, OBS must be granted required permissions. Therefore, before configuring logging for a bucket, you need to create an IAM agency for OBS and add this agency when configuring logging for the bucket. To create an agency, see Creating an Agency for Uploading Logs.
  • By default, you only need to grant the agency the upload permission (obs:object:PutObject) for the log storage bucket. The following is an example of a custom IAM policy in JSON. mybucketlogs indicates the name of the log storage bucket.

    {
        "Version": "1.1",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                    "obs:object:PutObject"
                ],
                "Resource": [
                    "OBS:*:*:object:mybucketlogs/*"
                ]
            }
        ]
    }

• If the log storage bucket has Server-Side Encryption enabled, the agency also requires the KMS Administrator permission for the region where the bucket is located.
• If you want other users to access log files stored in the log storage bucket, grant permissions to them in either of the following ways:
  • Configuring the bucket ACL

    This method is applicable to accounts, but not to IAM users. To grant permissions to IAM users, use bucket policies.

    To access log files in a bucket, an account only requires the read permissions for the bucket and objects in it. For details, see Configuring a Bucket ACL.

    

  • Configuring a bucket policy
    • If you want other IAM users under your account (also the account of the log storage bucket owner) to read log files, grant read permissions for specific objects to the IAM users.
    • If you want IAM users under other accounts to read log files, grant read permissions for specific objects to other accounts.