Logging Overview

Scenarios

A large number of access logs are generated during bucket access. By default, OBS does not collect access logs for your bucket. If you need to analyze the property, type, or trend of requests to a bucket, you can enable logging for the bucket. OBS will automatically name access logs according to certain rules, generate log files, and upload the log files to the specified log storage bucket (the current bucket or another bucket in the same region).

Constraints

Logs can be stored in the logged bucket or another bucket. However, the log storage bucket and logged bucket must belong to the same account and region.

Important Notes

After logging is configured for a bucket, you can view the bucket's operation logs in the log storage bucket in approximately fifteen minutes. Log files generated in a specified period may not contain all requests made during that period. Some requests may be recorded in the log files of the previous or next period. Therefore, the log files generated in a specified period cannot record all logs generated during that period in real time.

Log File Naming Rules

• If <Log file name prefix> ends with a slash (/), the log files generated for the bucket are stored in the folder named <Log file name prefix> in the target bucket. The log file name is YYYY-mm-DD-HH-MM-SS-<UniqueString>.
  Figure 1 Log folder name
  
  Figure 2 Log file name
  
• If <Log file name prefix> does not end with a slash (/), the log files generated for the bucket are stored in the root directory of the target bucket. The log file name is <Log file name prefix>YYYY-mm-DD-HH-MM-SS-<UniqueString>.
  Figure 3 Log file name
  

Log Fields

• Example of log content

  The following shows an access log delivered to the target bucket:

  787f2f92b20943998a4fe2ab75eb09b8 bucket [13/Aug/2015:01:43:42 +0000] xx.xx.xx.xx 
  787f2f92b20943998a4fe2ab75eb09b8 281599BACAD9376ECE141B842B94535B  REST.GET.BUCKET.LOCATION 
  - "GET /bucket?location HTTP/1.1" 200 - 211 - 6 6 "-"  "HttpClient" - - - - "-" 089fe8c2c380f4031f6dc0197fe99d4d HPTAAFZfyW0yD80idvjw

• Log content format

  The access log of each bucket contains the following information.

  Table 2 Bucket log format

  Parameter	Description	Example Value
  BucketOwner	Account ID of the bucket owner	787f2f92b20943998a4fe2ab75eb09b8
  BucketName	Bucket name	bucket
  Time	UTC timestamp when OBS received the request	[13/Aug/2015:01:43:42 +0000]
  Remote IP	IP address of the request source	192.168.7.132
  Requester	Requester ID. The value can be: Anonymous: Indicates that the request was made by an anonymous user. Account ID of the requester: Indicates that the request was made by an account or an IAM user.	787f2f92b20943998a4fe2ab75eb09b8
  RequestID	ID of the request for performing operations on the bucket	281599BACAD9376ECE141B842B94535B
  Operation	Operation type of the request For common operations and their description, see Table 3.	REST.GET.BUCKET.LOCATION
  Key	Name of the requested object	-
  Request-URI	URI used to request operations on OBS resources. For details about URIs, see Constructing a Request. NOTE: If query_string is used for signature, the Request-URI will contain the signature information. Otherwise, the Request-URI will not contain the signature information.	GET /bucket?location HTTP/1.1
  HTTPStatus	HTTP status code returned by OBS	200
  ErrorCode	Error code returned by OBS. - indicates that no error code was returned.	-
  BytesSent	Size of the HTTP response body Unit: byte - indicates that the HTTP response does not contain a body.	211
  ObjectSize	Object size Unit: byte When OBS deletes an object, it does not log the object's size. In the object deletion log, the value of ObjectSize is 0. If error code 4XX is returned, the value of ObjectSize is -, indicating that the specific object size is not displayed.	-
  TotalTime	Total request duration, in ms. It is the time taken by the OBS server to process the request. Total request duration = Time when the last byte of the response was received – Time when the first byte of the request was sent	6
  Turn-AroundTime	Time required for sending a request to the OBS server, in ms. Time required for sending a request to the OBS server = Time when the first byte of the response was received – Time when the last byte of the request was sent	6
  Referer	Referer header field of the request The Referrer header field is a part of the HTTP request header. It helps the server understand the request source, specifically indicating which URL the user navigated from to access the requested resource.	-
  User-Agent	User-Agent header field of the request The User-Agent header field is a part of the HTTP request header and is used to identify the client software that made the request. This field provides information about the browser, operating system, and device type, which the server can use to optimize the response content or perform statistical analysis.	HttpClient
  VersionID	Object version ID in the request. This parameter specifies the version of the object on which the operation is performed. In a bucket with versioning enabled, an object can have multiple versions. Each version has a unique version ID. For details, see Versioning.	-
  STSLogUrn	Information about federated identity authentication and agency authorization	-
  StorageClass	Current storage class of the object. The value can be: STANDARD: the Standard storage class STANDARD_IA (also WARM): the Infrequent Access storage class COLD: the Archive storage class DEEP_ARCHIVE: the Deep Archive storage class	STANDARD_IA
  TargetStorageClass	Storage class of the object after the lifecycle rule is applied. The value can be: STANDARD: the Standard storage class WARM: the Infrequent Access storage class COLD: the Archive storage class DEEP_ARCHIVE: the Deep Archive storage class	GLACIER
  DentryName	For a parallel file system, this field indicates an internal identifier of a file or directory. Its value consists of a parent directory inode number and a file or directory name. For a bucket, the value of this field is -.	12456/file.txt
  IAMUserID	IAM user ID. The value can be: IAM user ID: Indicates that the request was made by a non-anonymous user. Anonymous: Indicates that the request was made by an anonymous user.	8f3b8c53d29244a780084f2b8c106c32
  AccessKeyID	Access key ID of the requester. The value can be: -: Indicates that the request was made by an anonymous user. Access key ID of the requester: Indicates that the request was made by a non-anonymous user.	UDSIAMSTUBTEST002852

  Table 3 Common operations

  Operation	Description	Operation	Description
  REST.GET.SERVICE	Lists buckets.	REST.GET.ENCRYPTION	Obtains the bucket encryption configuration.
  REST.PUT.BUCKET	Creates a bucket.	REST.DELETE.ENCRYPTION	Deletes the bucket encryption configuration.
  REST.HEAD.BUCKET	Views the bucket information.	REST.PUT.OTM_DIRECT_COLD_ACCESS	Configures direct reading for Archive objects in a bucket.
  REST.GET.BUCKETVERSIONS	Lists objects in a bucket.	REST.GET.OTM_DIRECT_COLD_ACCESS	Obtains the direct reading configuration of a bucket.
  REST.GET.BUCKET	Obtains the bucket metadata.	REST.DELETE.OTM_DIRECT_COLD_ACCESS	Deletes the direct reading configuration of a bucket.
  REST.GET.BUCKET.LOCATION	Obtains the bucket location.	REST.PUT.BUCKET.WEBSITE	Configures static website hosting for a bucket.
  REST.DELETE.BUCKET	Deletes a bucket.	REST.GET.BUCKET.WEBSITE	Obtains the static website hosting configuration of a bucket.
  REST.PUT.POLICY	Configures a bucket policy.	REST.DEL.BUCKET.WEBSITE	Deletes the static website hosting configuration of a bucket.
  REST.GET.POLICY	Obtains a bucket policy.	REST.PUT.BUCKET.CORS	Configures CORS for a bucket.
  REST.DELETE.POLICY	Deletes a bucket policy.	REST.GET.BUCKET.CORS	Obtains the CORS configuration of a bucket.
  REST.PUT.ACL	Configures an ACL for a bucket or an object.	REST.DEL.BUCKET.CORS	Deletes the CORS configuration of a bucket.
  REST.GET.ACL	Obtains a bucket ACL or an object ACL.	REST.OPTIONS.BUCKET	Sends an OPTIONS request to a bucket.
  REST.PUT.LOGGING_STATUS	Configures logging for a bucket.	REST.OPTIONS.OBJECT	Sends an OPTIONS request to an object.
  REST.GET.LOGGING_STATUS	Obtains the logging configuration of a bucket.	REST.PUT.OBJECT	Uploads an object with PUT.
  REST.PUT.BUCKET.LIFECYCLE	Configures a lifecycle rule for a bucket.	REST.POST.OBJECT	Uploads an object with POST.
  REST.GET.LIFECYCLE	Obtains the lifecycle configuration of a bucket.	REST.COPY.OBJECT	Copies an object.
  REST.DEL.LIFECYCLE	Deletes the lifecycle configuration of a bucket.	REST.GET.OBJECT	Obtains the object content.
  REST.PUT.VERSIONING	Configures versioning for a bucket.	REST.HEAD.OBJECT	Obtains the object metadata.
  REST.GET.VERSIONING	Obtains the versioning status of a bucket.	REST.DELETE.OBJECT	Deletes an object.
  REST.GET.BUCKET.STORAGE.POLICY	Obtains the storage class of a bucket.	REST.TRANSITION.STORAGECLASS.OBJECT	Changes the storage class of an object.
  REST.PUT.BUCKET.STORAGE.POLICY	Configures a storage class for a bucket.	OP_MULTIPLE_DELETEOBJECT	Batch deletes objects (the batch operation itself).
  REST.PUT.REPLICATION	Configures cross-region replication for a bucket.	REST.POST.RESTORE	Restores an Archive object.
  REST.DELETE.REPLICATION	Deletes the cross-region replication configuration of a bucket.	REST.APPEND.OBJECT	Appends data to an object.
  REST.GET.REPLICATION	Obtains the cross-region replication configuration of a bucket.	REST.MODIFY.OBJECT.META	Modifies the object metadata.
  REST.PUT.TAGGING	Adds tags to a bucket.	REST.TRUNCATE.OBJECT	Truncates an object.
  REST.GET.TAGGING	Obtains the tags of a bucket.	REST.RENAME.OBJECT	Renames an object.
  REST.DEL.TAGGING	Deletes the tags of a bucket.	REST.GET.UPLOADS	Lists the initiated multipart uploads in a bucket.
  REST.PUT.BUCKET_QUOTA	Configures a storage quota for a bucket.	REST.POST.UPLOADS	Initiates a multipart upload.
  REST.GET.BUCKET.QUOTA	Obtains the storage quota of a bucket.	REST.PUT.PART	Uploads a part.
  REST.GET.BUCKET.STORAGEINFO	Queries the storage usage of a bucket.	REST.COPY.PART	Copies a part.
  REST.PUT.BUCKET.INVENTORY	Configures inventories for a bucket.	REST.GET.UPLOAD	Lists uploaded parts.
  REST.GET.BUCKET.INVENTORY	Obtains or lists inventories of a bucket.	REST.POST.UPLOAD	Assembles parts.
  REST.DELETE.BUCKET.INVENTORY	Deletes inventories of a bucket.	REST.DELETE.UPLOAD	Aborts a multipart upload.
  REST.PUT.CUSTOMDOMAIN	Configures a user-defined domain name for a bucket.	REST.CLEAR.EXPIRE.UPLOAD	Deletes expired parts.
  REST.GET.CUSTOMDOMAIN	Obtains the user-defined domain name of a bucket.	REST.DELETE.CUSTOMDOMAIN	Deletes the user-defined domain name of a bucket.
  REST.PUT.ENCRYPTION	Configures encryption for a bucket.	-	-

Billing for Bucket Logs

• Uploading bucket logs to and storing them in the log storage bucket will incur PUT request and storage costs. For details, see OBS Billing.
• If log files are stored in the logged bucket, OBS creates additional logs for writing log files to the bucket, which takes up extra storage space that will increase your costs and makes it more difficult for you to locate required logs. Therefore, you are advised to store log files in a bucket other than the logged bucket.
• You can delete unnecessary log files to reduce storage costs. Configuring lifecycle rules for scheduled deletion is recommended.

Permissions Related to Bucket Logging

• After logging is enabled, log delivery users of the log storage bucket will be automatically granted the permissions to read the bucket ACL and write logs to the bucket.

  If you manually disable such permissions, bucket logging will fail.

  

• OBS creates log files and uploads them to a specified bucket. To perform these operations, OBS must be granted required permissions. Therefore, before configuring logging for a bucket, you need to create an IAM agency for OBS and add this agency when configuring logging for the bucket. To create an agency, see Creating an Agency for Uploading Logs.
  • By default, you only need to grant the agency the upload permission (obs:object:PutObject) for the log storage bucket. The following is an example of a custom policy in the JSON view for IAM. mybucketlogs indicates the name of the log storage bucket.

    {
        "Version": "1.1",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                    "obs:object:PutObject"
                ],
                "Resource": [
                    "OBS:*:*:object:mybucketlogs/*"
                ]
            }
        ]
    }

• If the log storage bucket has Server-Side Encryption enabled, the agency also requires the KMS Administrator permission for the region where the bucket is located.
• If you want other users to access log files stored in the log storage bucket, grant permissions to them in either of the following ways:
  • Configuring the bucket ACL

    Only account-level ACL permissions are supported; IAM user-level ACL permissions are not supported. If you want to grant IAM users permissions to access log files, use bucket policies.

    Grant an account the read permissions for the log storage bucket and objects in it. For details, see Configuring a Bucket ACL.

    

  • Configuring a bucket policy
    • If you want other IAM users under your account (owner of the log storage bucket) to read log files, grant read permissions for specific objects to an IAM user.
    • If you want IAM users under other accounts to read log files, grant read permissions for specific objects to other accounts.