Help Center/ Object Storage Service/ User Guide/ Data Security/ Configuring URL Validation to Prevent Unauthorized Access to Your Data
Updated on 2026-11-02 GMT+08:00
View PDF
Share

Configuring URL Validation to Prevent Unauthorized Access to Your Data

OBS blocks access requests from blacklisted URLs and allows those from whitelisted URLs.

Scenarios

Some malicious websites hotlink resources from other websites to expand their content without incurring additional costs. This not only infringes on the interests of the original websites but also increases their server load. URL validation is designed to address this issue.

In HTTP, the Referer header field allows websites to identify the source webpage of a request. By checking the Referer value, websites can determine whether a request comes from an authorized source and block unauthorized requests or redirect them to a specified page. URL validation works by matching the Referer field against a whitelist or blacklist.

To prevent unauthorized hotlinking of data stored in OBS, OBS supports Referer-based URL validation with configurable whitelists and blacklists.

How It Works

When a browser sends a request to OBS, the Referer field in the HTTP request header records the URL of the page from which the request originated. OBS verifies the value of the Referer field to determine whether the request source is trusted. By configuring a Referer whitelist or Referer blacklist, you can allow or deny requests from specific sources.

After URL validation is configured, OBS performs access control according to the following rules:

  1. Check whether the Referer field is empty. OBS first determines whether the Referer field in the request is empty.
    • If the Referer field is empty, the following rules apply:
      • If Null referer is allowed is enabled, the request is allowed.
      • If Null referer is allowed is disabled and Whitelisted Referers is empty, the request is allowed.
      • If Null referer is allowed is disabled and Whitelisted Referers is not empty, the request is denied.
    • If the Referer field is not empty, the following rules apply:
  2. If the Referer field is not empty, OBS checks the value of the Referer field against the configured whitelist or blacklist in URL validation.
    • If Whitelisted Referers and Blacklisted Referers are both empty, all requests are allowed by default.
    • If Whitelisted Referers is not empty and Blacklisted Referers is empty, the request is allowed if the Referer value matches an entry in the whitelist. Otherwise, the request is denied.
    • If Whitelisted Referers is empty and Blacklisted Referers is not empty, the request is denied if the Referer value matches an entry in the blacklist. Otherwise, the request is allowed.
    • If Whitelisted Referers and Blacklisted Referers are not empty, the request is denied if the Referer value matches an entry in the blacklist. If the Referer value matches an entry in the whitelist and does not match any entry in the blacklist, the request is allowed.

Constraints

  • Effective scope of URL validation: URL validation rules are configured and applied at the bucket level. They cannot be applied to specific objects.

Important Notes

Before determining whether a user has the four types of permissions (read, write, ACL read, and ACL write) for a bucket and its objects, check whether this user complies with the URL validation rules defined in the Referer field.

Because HTTP Referer headers can be easily spoofed, URL validation is only a supplementary defense. If you have strict data security requirements, use other OBS data protection features, such as server-side encryption and WORM. For details, see OBS Data Security Features.

Ways to Configure URL Validation

You can configure URL validation using OBS Console or APIs, but not using SDKs, OBS Browser+, or obsutil.

Using OBS Console

  1. In the navigation pane of OBS Console, choose Buckets.
  2. In the bucket list, click the desired bucket. The Objects page is displayed.
  3. In the navigation pane, choose Data Security > URL Validation.
  4. Click next to the text box of Whitelisted Referers or Blacklisted Referers and enter the Referers.

    Figure 1 Configuring URL validation
    Table 1 Parameters for configuring a URL validation rule

    Parameter

    Example

    Description

    Whitelisted Referers

    http://www.example.com

    Specifies the list of domain names that are allowed to access data in the target bucket.

    Input rules:

    • The total length of the Referers in a whitelist or blacklist cannot exceed 5,000 characters.
    • You can specify multiple Referers, with each one on a separate line.
    • The asterisk (*) is supported, which represents zero or more characters.

      Example: *.example.com matches http://www.example.com, http://test.example.com, and so on.

    • The question mark (?) is supported, which represents a single character.

      Example: http://www.example?.com matches http://www.examplet.com, http://www.exampley.com, and so on.

    • If the Referer header in a download request contains http or https, the configured Referer must also contain http or https.
    • Wildcard domain names are supported.
    • Domain names or IP addresses with port numbers are supported,

      for example, http://www.example.com:8080 and 10.10.10.10:8080.

    • When matching against the Referer whitelist or blacklist, the protocol cannot be ignored. To avoid mismatches, configure both http and https.

      For example, http://www.example.com does not match https://www.examplet.com.

    Null referer is allowed

    Disabled
    Specifies whether to allow requests with an empty Referer field, such as requests made by entering a URL directly in the browser address bar.
    • Option enabled: Requests with an empty Referer are allowed.
    • Option disabled and Whitelisted Referers is empty: Requests with an empty Referer are allowed.
    • Option disabled and Whitelisted Referers is not empty: Requests with an empty Referer are denied.

    Blacklisted Referers

    http://www.test.com

    Specifies the list of domain names that are not allowed to access data in the target bucket.

    Input rules:

    • The total length of the Referers in a whitelist or blacklist cannot exceed 5,000 characters.
    • You can specify multiple Referers, with each one on a separate line.
    • The asterisk (*) is supported, which represents zero or more characters.

      Example: *.example.com matches http://www.example.com, http://test.example.com, and so on.

    • The question mark (?) is supported, which represents a single character.

      Example: http://www.example?.com matches http://www.examplet.com, http://www.exampley.com, and so on.

    • If the Referer header in a download request contains http or https, the configured Referer must also contain http or https.
    • Wildcard domain names are supported.
    • Domain names or IP addresses with port numbers are supported,

      for example, http://www.example.com:8080 and 10.10.10.10:8080.

    • When matching against the Referer whitelist or blacklist, the protocol cannot be ignored. To avoid mismatches, configure both http and https.

      For example, http://www.example.com does not match https://www.examplet.com.

    The matching rules for URL validation are as follows:

    • When the Referer whitelist and blacklist are both empty, requests from all websites are allowed to access data in the target bucket by default.
    • When the Referer whitelist is not empty and the blacklist is empty, requests from websites in the whitelist are allowed to access data in the target bucket.
    • When the Referer whitelist is empty and the blacklist is not empty, requests from websites that are not in the blacklist are allowed to access data in the target bucket.
    • When the Referer whitelist and blacklist are both not empty, requests from websites that are in the whitelist and not in the blacklist are allowed to access data in the target bucket. Requests from websites in the blacklist are not allowed to access data in the target bucket, regardless of whether the websites are in the whitelist.

  5. Click to save the settings.

Using APIs

Configure a URL validation whitelist

Example Scenarios

The following are common application scenarios of URL validation:

Allowing Only Trusted Websites to Access OBS

Use the Referer whitelist of URL validation to allow access to OBS only from trusted Referers that frequently access OBS buckets.

  1. View the bucket logs and find the value of the Referer field in the bucket log file. For details about bucket logging, see Log Storage Overview.

    If the value of the Referer field in the bucket log file is -, the Referer is null.

  2. Configure URL validation rules for the bucket.

    For details about the configuration procedure, see Using OBS Console. Table 2 describes the parameters.

    Table 2 URL validation whitelist rules

    Parameter

    Value

    Description

    Whitelisted Referers

    		 
    https://www.example.com
http://www.example.com
https://console.huaweicloud.com/*
    • Add the domain names of trusted websites that frequently access OBS buckets.
    • You are advised to add both HTTPS and HTTP domain names.
    • Add the console domain name and use a wildcard (*) to ensure that previews work properly in consoles across different regions.

    Null referer is allowed

    Toggle it on.

    Ensure that both direct access and console preview functions work properly.

    Blacklisted Referers

    No configuration is required.

    You do not need to block specific sources.

  3. Verify the configuration.

    • Simulate a request with a whitelisted Referer to OBS. The request is expected to be successful. 
      curl -e "https://www.example.com" https://test-bucket.obs.cn-north-4.myhuaweicloud.com/test.txt
    • Simulate a request with a non-whitelisted Referer to OBS. The request is expected to fail, and AccessDenied is returned. 
      curl -e "https://www.example01.com" https://test-bucket.obs.cn-north-4.myhuaweicloud.com/test.txt
    • Simulate a request with a null Referer to OBS. The request is expected to be successful. 
      curl https://test-bucket.obs.cn-north-4.myhuaweicloud.com/test.txt

Preventing Malicious Websites from Stealing OBS Data

Access from all sources is allowed, but some malicious websites must be prohibited from stealing OBS data. You can blacklist specific Referers to precisely block malicious access sources.

  1. View logs about abnormal traffic of the OBS bucket and find the value of the Referer field in the bucket log file. For details about bucket logging, see Log Storage Overview.
  2. Configure URL validation rules for the bucket.

    For details about the configuration procedure, see Using OBS Console. Table 3 describes the parameters.

    Table 3 URL validation blacklist rules

    Parameter

    Value

    Description

    Whitelisted Referers

    *

    Allow access from any source to OBS.

    Null referer is allowed

    Toggle it on.

    Ensure that both direct access to OBS and access via normal clients are allowed.

    Blacklisted Referers

    		 
    https://badmail.example.com
http://badmail.example.com
*badsite.example

    Block access from malicious website domain names. The wildcard (*) indicates that the entire domain name and its subdomains are blocked.

  3. Verify the configuration.

    • Simulate a request from a trusted website to OBS. The request is expected to be successful. 
      curl -e "https://www.example.com" https://test-bucket.obs.cn-north-4.myhuaweicloud.com/test.txt
    • Simulate a request with a blacklisted Referer to OBS. The request is expected to fail, and AccessDenied is returned. 
      curl -e "https://badmail.example.com" https://test-bucket.obs.cn-north-4.myhuaweicloud.com/test.txt
    • Simulate a request with a null Referer to OBS. The request is expected to be successful. 
      curl https://test-bucket.obs.cn-north-4.myhuaweicloud.com/test.txt

Accessing OBS Using Mini Programs

The URL validation whitelist allows access to OBS exclusively from a specific mini program (for example, https://appexample.com/{appid}/{version}/page-example.html). You can whitelist specific Referers to allow access to OBS from the mini program.

  1. Obtain the domain name of the mini program and use the wildcard (*) to match all subdomains of the mini program.

    You can view the logs about the mini program's access to the OBS bucket and find the value of the Referer field in the bucket log file. For details about bucket logging, see Log Storage Overview.

  2. Configure URL validation rules for the bucket.

    For details about the configuration procedure, see Using OBS Console. Table 4 describes the parameters.

    Table 4 URL validation whitelist rules

    Parameter

    Value

    Description

    Whitelisted Referers

    		 
    *appexample.com
https://console.huaweicloud.com/*
    • Use the wildcard (*) to match all subdomains of the mini program.
    • Add the console domain name and use a wildcard (*) to ensure that previews work properly in consoles across different regions.

    Null referer is allowed

    Toggle it on.

    In some cases, the mini program may not send a Referer. Enabling this function ensures compatibility.

    Blacklisted Referers

    Leave it blank.

    You do not need to block specific sources.

  3. Verify the configuration.

    • Use the mini program to access OBS. The access is expected to be successful. 
      curl -e "https://appexample.com/kbaS3wM/page.html" https://test-bucket.obs.cn-north-4.myhuaweicloud.com/test.txt
    • Simulate a request with a non-whitelisted Referer to OBS. The request is expected to fail, and AccessDenied is returned. 
      curl -e "https://www.example.com" https://test-bucket.obs.cn-north-4.myhuaweicloud.com/test.txt
    • Use the mini program to access OBS (without a Referer). The access is expected to be successful. 
      curl https://test-bucket.obs.cn-north-4.myhuaweicloud.com/test.txt

Allowing Object Preview and Download on the OBS Console

With the URL validation whitelist, you can add the OBS console domain name to the whitelisted Referers to ensure normal management functions.

  1. Configure URL validation rules for the bucket.

    For details about the configuration procedure, see Using OBS Console. Table 5 describes the parameters.

    Table 5 URL validation whitelist rules

    Parameter

    Value

    Description

    Whitelisted Referers

    		 
    https://console.huaweicloud.com/*

    Add the console domain name and use a wildcard (*) to support all subdomains, ensuring that previews work properly in consoles across different regions.

    Null referer is allowed

    Toggle it on.

    Ensure that both direct access and console preview functions work properly.

    Blacklisted Referers

    Leave it blank.

    You do not need to block specific sources.

  2. Verify the configuration.

    Simulate access to the console. The access is expected to be successful. You can also preview and download files on the OBS console for verification.

    curl -e "https://console.huaweicloud.com" https://test-bucket.obs.cn-north-4.myhuaweicloud.com/test.txt

Prohibiting Direct Access to OBS Via URLs

All access to OBS must come from specified websites. Direct access via URLs is prohibited.

This configuration will block null Referers, indicating that all direct access requests are prohibited. Since bookmarks, email links, and some browser plug-ins, download tools, or video players typically do not include Referers in their requests, these components may not function properly.

  1. Determine the Referer of a specified website.

    Identify all possible combinations of domain names and protocols for the website to ensure that all legitimate access scenarios are covered.

  2. Configure URL validation rules for the bucket.

    For details about the configuration procedure, see Using OBS Console. Table 6 describes the parameters.

    Table 6 URL validation whitelist rules

    Parameter

    Value

    Description

    Whitelisted Referers

    		 
    https://www.example.com
http://www.example.com
https://console.huaweicloud.com/*
    • Identify all possible combinations of domain names and protocols for the website to ensure that all legitimate access scenarios are covered.
    • You are advised to add both HTTPS and HTTP domain names.
    • Add the console domain and use a wildcard (*) to ensure that objects can be previewed properly in consoles across different regions.

    Null referer is allowed

    Toggle it off.

    Disabling this function can prevent direct access to OBS resources through URLs in the address bar.

    Blacklisted Referers

    Leave it blank.

    You do not need to block specific sources.

  3. Verify the configuration.

    • Simulate a request from a specified website to access OBS. The request is expected to be successful. 
      curl -e "https://www.example.com" https://test-bucket.obs.cn-north-4.myhuaweicloud.com/test.txt
    • Simulate a request to access OBS directly through a URL in the address bar. The request is expected to fail, and AccessDenied is returned. 
      curl https://test-bucket.obs.cn-north-4.myhuaweicloud.com/test.txt
    • Simulate a request from another website to OBS. The request is expected to fail, and AccessDenied is returned. 
      curl -e "https://other.example.com" https://test-bucket.obs.cn-north-4.myhuaweicloud.com/test.txt

Best Practices

  • Risk of bypassing URL validation via CDN caching: If both URL validation and CDN acceleration domain names are configured for an OBS bucket, an unauthorized request to OBS may directly hit the CDN cache node and obtain the cached content without triggering OBS validation. To ensure complete and effective protection, you must also configure URL validation rules on CDN. For details, see Filtering Requests from Specific Referers.
  • Impact of browser Referrer-Policy: Modern browsers support the Referrer-Policy response header, which allows websites to control how the Referer header is sent in cross-domain requests. Under certain policies, such as no-referrer, the browser suppresses the Referer header, and OBS processes it as a null Referer. When configuring URL validation, you need to take this behavior into consideration (for example, when configuring whitelisted Referers, enable Null referer is allowed).
  • Video playback compatibility: When a browser uses the native <video> tag to play videos, it sends requests to the video server to obtain media data. The Referers of these requests may be null or point to other pages. Therefore, OBS must allow access from null Referers (when configuring whitelisted Referers, enable Null referer is allowed). Otherwise, video playback will fail.

References

Why Doesn't URL Validation Take Effect After I Configure It?

Perform the following checks:

  1. Check that the Referer values in the URL validation rules are correct. In some browser environments, the Referer header may be modified or set to a specific value. Check the actual Referer of the request in the OBS bucket logs or the browser developer tools, and then update the Referer values in the rules accordingly.
  2. Check that the Referer format is correct. Verify the format against the following Referer input rules.

    Referer input rules:

    • The total length of the Referers in a whitelist or blacklist cannot exceed 5,000 characters.
    • You can specify multiple Referers, with each one on a separate line.
    • The asterisk (*) is supported, which represents zero or more characters.

      Example: *.example.com matches http://www.example.com, http://test.example.com, and so on.

    • The question mark (?) is supported, which represents a single character.

      Example: http://www.example?.com matches http://www.examplet.com, http://www.exampley.com, and so on.

    • If the Referer header in a download request contains http or https, the configured Referer must also contain http or https.
    • Wildcard domain names are supported.
    • Domain names or IP addresses with port numbers are supported,

      for example, http://www.example.com:8080 and 10.10.10.10:8080.

    • When matching against the Referer whitelist or blacklist, the protocol cannot be ignored. To avoid mismatches, configure both http and https.

      For example, http://www.example.com does not match https://www.examplet.com.

  3. Check whether requests can bypass URL validation by hitting the CDN cache. If a bucket has both URL validation and a CDN acceleration domain name configured, an unauthorized request may be served directly from a CDN cache node that holds the requested content. The request then never reaches OBS, so the URL validation rules on OBS are bypassed. To ensure complete and effective protection, you must also configure URL validation rules on CDN. For details, see Filtering Requests from Specific Referers.

Access Denied When Accessing OBS Resources from a Mini Program

The Referer format of requests initiated by mini programs may vary slightly. For example, requests from mini program A may have a Referer starting with https://example.com/, but requests from its sub-programs may contain different subpaths, such as https://example.com/app/page1.html and https://example.com/app/page2.html.

To ensure that the URL validation rules are compatible with all mini program requests, use wildcards when you configure the Referer whitelist. The asterisk (*) matches zero or more characters, and the question mark (?) matches a single character.

In the preceding example, you can set the Referer whitelist to https://example.com/*.

Access Denied When Entering an Object URL Directly in a Browser

Entering a URL directly in a browser address bar sends a request with an empty Referer. If Null referer is allowed is disabled in the URL validation rules, the request is denied.

To resolve this issue, use either of the following methods:

  • Method 1: Enable Null referer is allowed.
  • Method 2: Keep Null referer is allowed disabled and leave the Whitelisted Referers empty.
Parent Topic: Data Security