Using CDN to Accelerate OBS File Download

This section uses the game download and update services on a game website as examples to describe how to use CDN on HUAWEI CLOUD to rapidly download game installation packages and update packages from OBS.

Scenario Introduction

A game website has subscribed to OBS and stores a large number of files such as game software packages, pictures, and videos on OBS. As the number of users increases, responses of game download and image loading became slower, especially for users who are far away from the file storage area. To address this issue, the website decided to use the CDN service to accelerate game download at the lowest cost and improve user experience.

Data Preparations

Table 1 Required data

Item

Description

Example

Website domain name

Domain name of the game website. According to Regulation on Internet Information Service of the People's Republic of China, this domain name must be licensed by the Ministry of Industry and Information Technology (MIIT) and within the valid period. Otherwise, it cannot use the service of CDN

download.game-apk.com

OBS Bucket

Bucket for storing static resources such as images and software packages. Set the storage class to Standard and the bucket policy to private.

game-apk

Prerequisites

Static resources, such as images and software packages, have been stored in a bucket on OBS.

If the required files are not yet uploaded to OBS, you can create buckets and upload files through OBS Console, OBS Browser+, or the SDK. For details, see the help documents of each method.

Procedure

  1. (Optional) Purchase pull traffic packages.

    CDN consumes traffic when it pulls data from OBS. OBS provides pull traffic packages for CDN to pull contents from OBS, reducing the costs on data traffic.
    1. Log in to the HUAWEI CLOUD console. On the homepage, choose Storage > Object Storage Service. The OBS Console page is displayed.
    2. Click Buy OBS Package in the upper right corner of the page.
    3. Configure the following parameters based on the service demands.
      • Region: Select the region where the bucket for which CDN acceleration is to be configured is located.
      • Package Type: Select Pull traffic.
      • Monthly Quota: Select a quota based on your actual needs.
      • Quantity: Enter the number of pull traffic packages that you want to buy. For example, if you purchase two pull traffic packages with 1-TB monthly quota each, you will get 2 TB pull traffic each month.
      • Required Duration: Select a duration for the packages.
      • Effective Time: You can specify the package to be effective Immediately after payment, or Custom an effective time.
    4. Click Add.
    5. In the list of Selected Resource Packages on the right, click Next to submit the order and pay for it.

  2. Configure CDN file download acceleration.

    OBS supports domain name management. After a user-defined domain name is bound to OBS, the OBS can be accessed using this domain name. In addition, CDN acceleration can be enabled during the binding without configuration on the CDN page.
    1. Log in to the HUAWEI CLOUD console. On the homepage, choose Storage > Object Storage Service. The OBS Console page is displayed.
    2. Click the bucket where the software package is stored, for example, bucket game-apk.
    3. In the navigation pane on the left, choose Domain Name Mgmt and click Bind User Domain Name.
    4. In the Bind User Domain Name dialog box, configure the domain name and CDN acceleration information. For details, see Figure 1.
      • User Domain Name: Enter the domain name of the game website, for example, download.game-apk.com.
      • CDN Acceleration: Enable CDN acceleration.
      • Service Type: Select File download.
      Figure 1 Binding a user domain name
    5. Click OK.

  3. Configure CNAME.

    After the CDN acceleration function is enabled when the domain name is bound to OBS, CDN automatically generates a CNAME for the bound domain name. Configure the CNAME record at the domain name service provider so that the acceleration domain name is pointed to its corresponding CNAME in CDN. After the domain name resolution takes effect, all requests for the acceleration domain name will be directed to CDN nodes. In this example, the automatically generated CNAME domain name is download.game-apk.com.c.cdnhwc1.com.

    The CNAME configuration method varies depending on the DNS service provider. In this document, DNS provided by HUAWEI CLOUD is used as an example. For details about how to configure CNAME records with other DNS service providers, see Overview.
    1. Log in to the HUAWEI CLOUD console. On the homepage of the management console, choose Network > Domain Name Service. The DNS console is displayed.
    2. In the navigation pane on the left, choose DNS Resolution > Public Zones. The domain name list of the public zones is displayed.
    3. Click the domain name to which you want to add a record set. In this example, the domain name is game-apk.com.
    4. Click Add Record Set on the right and go to the Record Sets page.
    5. Configure the parameters as instructed. Table 2 lists the parameters. Retain the default values for the parameters that are not listed in the table.
      Table 2 Parameter description

      Parameter

      Description

      Example

      Name

      Domain name prefix

      download

      Type

      Type of the record set, which should be the CNAME type.

      CNAME-Map one domain to another

      Alias

      Whether you want this record set to be used as an alias for a HUAWEI CLOUD service resource.

      No

      Line

      Used when the DNS server is resolving a domain name. It returns the IP address of the server according to the visitor source.

      You must add a Default line to ensure that the website is accessible to all users.

      Default

      TTL (s)

      TTL is the valid cache time of the resolution records on the local DNS server. If your service address is changed frequently, it is recommended that you set TTL to a smaller value. Otherwise, you are advised to set TTL to a larger value.

      The default interval is 5 min, 300 seconds.

      Value

      Domain name to be pointed to.

      If CDN acceleration is not enabled, the value is the bucket domain name. If CDN acceleration is enabled, the value of this parameter is the CNAME domain name allocated by CDN.

      download.game-apk.com.c.cdnhwc1.com

    6. Click OK.
    7. Check whether the CNAME configuration takes effect.

      Open the cmd program in your Windows system, and run the following command:

      nslookup -qt=cname User-defined domain name bound to the bucket

      In this example, the user-defined domain name bound to the bucket is download.game-apk.com. If the CNAME domain name allocated by CDN is displayed, the CNAME configuration has taken effect.

  4. Enable private bucket retrieval on CDN.

    Because the bucket for storing the software package is a private bucket, CDN can retrieve data from the OBS only after the function for private bucket retrieval is enabled on the CDN.

    1. Log in to the HUAWEI CLOUD console. On the homepage of the management console, choose Storage > CDN. The CDN console is displayed.
    2. In the navigation tree on the left, click Domain Name Management.
    3. In the domain name list, click the domain name to be modified or click Settings at the end of the row of the domain name. In this example, the domain name is download.game-apk.com.
    4. Click the Retrieval Settings tab.
    5. In the Private Bucket Retrieval area, authorize CDN the permission to access your buckets.

      If it is the first time you set a private OBS bucket as the origin server of this domain name, you need to authorize CDN the permission to access your OBS private buckets.

      1. (Required only for initial configuration) Click Authorize Now. The Authorize Access dialog box is displayed.
      2. (Required only for the initial configuration) Click Agree. The system creates an agency named CDNAccessPrivateOBS for you in IAM to allow CDN to access your OBS private buckets.

        Do not delete the agency relationship between CDN and OBS. Otherwise, CDN cannot obtain resources from OBS private buckets.

      3. Switch on Private Bucket Retrieval.

  5. Configure the file download URL.

    Set the URL of the file to be downloaded in the code as follows: Domain name of the game website + Storage path of the file in the OBS bucket + File name.

    In the following example, the game website domain name is download.game-apk.com, as configured in 2, and the android.apk file under the game/3.2.1/ folder in the game-apk is used. Then the file download URL is as follows:

    https://download.game-apk.com/game/3.2.1/android.apk

  6. Configure a custom bucket policy to prevent objects in a private bucket from being listed by anonymous users.

    After private bucket retrieval is enabled, any anonymous user who accesses the bucket through the user-defined domain name (CDN acceleration domain name) can successfully list objects in the root directory of the bucket. As a result, the object list is exposed to the public network. If you do not want anyone to list objects in the bucket, you need to configure another two custom bucket policies to allow only specified users to list objects in the bucket.

    1. Log in to the HUAWEI CLOUD console. On the homepage, choose Storage > Object Storage Service. The OBS Console page is displayed.
    2. Click the bucket where the software package is stored (the bucket used as the source for content retrieval by CDN). In this example, click the game-apk bucket.
    3. In the navigation pane on the left, choose Permissions > Bucket Policy.
    4. Click Create Bucket Policy.
    5. In the first row of the bucket policy template, click Create Custom Policy.
    6. Configure the following parameters.
      This policy denies the bucket list (List*) permission of users except the specified users.
      Table 3 Bucket policy parameters

      Parameter

      Description

      Policy View

      Select Visual editor.

      Policy Name

      Custom

      Policy Content

      Effect

      Select Deny.

      Principal

      • Principal: Current account
      • Sub-user: Select the users who are allowed to list objects in the bucket. You can select the IAM user with the account name or other IAM users based on service requirements.
      • User Policy: Exclude specified users.

      Resources

      • Resource: Select Current bucket.
      • Resource Policy: Include specified resources.

      Actions

      • Select List*.
      • Operation Strategy: Include selected actions.
    7. Click Next in the lower right corner.
    8. Click Create.
    9. Click Create Bucket Policy and configure the parameters of the second custom bucket policy.
    10. In the first row of the bucket policy template, click Create Custom Policy.
    11. Configure the following parameters.
      This policy grants the bucket list (List*) permission to specified users. The users specified here must be the same as those specified in the previous policy.

      Question: Why do I need to configure the second bucket policy that grants the bucket list permission?

      A: When creating a bucket policy in the common mode instead of using the code, you cannot configure the policy to take effect on the current account and its IAM users. So the first bucket policy does not grants the bucket list permission to the specified users under the current account. These users can list objects in the bucket before configuring the first bucket policy because they have been granted the permission through IAM configurations. However, the previous IAM configurations become invalid after the first bucket policy is configured, so the bucket list permission of these users is denied by default. According to the principle that explicit deny statements have the highest priority, then the allow statements, and then the default deny statements, we need to configure another bucket policy to grant the bucket list permission to these IAM users, so that they can list objects in the bucket.

      Table 4 Bucket policy parameters

      Parameter

      Description

      Policy View

      Select Visual editor.

      Policy Name

      Custom

      Policy Content

      Effect

      Select Allow.

      Principal

      • Principal: Current account
      • Sub-user: Select the users who are allowed to list objects in the bucket. Specify the same users that you have specified in the first bucket policy.
      • User Policy: Include specified users.

      Resources

      • Resource: Select Current bucket.
      • Resource Policy: Include specified resources.

      Actions

      • Select List*.
      • Operation Strategy: Include selected actions.
    12. Click Next in the lower right corner.
    13. Click Create.
    14. Log in to the CDN console, choose Preheating & Refresh > Cache Refreshing, and refresh the CDN cache so that the bucket policy can take effect.

      When refreshing the cache, select the URL type. Entered the URL added when configuring the file download URL. In this example, the URL is as follows:

      https://download.game-apk.com/game/3.2.1/android.apk

  7. Verify the services.

    After the game website is redeployed, log in to the game website, browse web pages, and download games.

    If images are displayed properly and the game can be downloaded successfully, the acceleration configuration is successful.