How Do I Troubleshoot 404/502/504 Errors?

404 Not Found

Scenario 1: When a visitor accesses your website, the page shown in Figure 1 is displayed.

Figure 1 404 page

Scenario 2: When a visitor accesses your website, another 404 error page is displayed instead of the page shown in Figure 1.

Cause: The website does not exist or has been deleted.

Solution: Check your website.

502 Bad Gateway

Scenario: Website access is normal after the WAF configuration is complete. However, after a certain period of time, a 502 Bad Gateway error is reported frequently.

If your web server is not deployed on the cloud, consult your server provider about whether the server has default block settings. If there are default block settings, ask the service provider to remove them.

Possible causes are as follows:

• Cause 1: Your website is using another security protection software. The software considers back-to-source IP addresses of WAF as malicious and blocks the requests forwarded by WAF. As a result, the site becomes inaccessible.

• Cause 2: Multiple backend servers are configured. However, one backend server is unreachable.
  Perform the following steps to check whether the origin server configuration is correct:

  1. Log in to the management console, click Service List in the upper part of the page, and choose Security > Web Application Firewall.
  2. In the navigation pane, choose Website Settings.
  3. In the Domain Name column, click the domain name. Its information is displayed.
  4. In the Server Information area, click . On the displayed page, check whether the client protocol, server protocol, origin server address, and port used by the origin server are correct.
     Figure 2 Server configuration
     
  5. Run the curl command on the host to check whether each origin server can be properly accessed.

     curl http://xx.xx.xx.xx:yy -kvv

     xx.xx.xx.xx indicates the IP address of the origin server. yy indicates the port of the origin server. xx.xx.xx.xx and yy must belong to the same origin server.

     

     • The host where the curl command can be run must meet the following requirements:
       • The network communication is normal.
       • The curl command has been installed. curl must be manually installed on the host running a Windows operating system. curl is installed along with other operating systems.
     • You can also enter http://origin server address:origin server port in the address bar of the browser to check whether the origin server can be properly accessed.
     Figure 3 Command output
     

     If connection refused is displayed, the origin server is unreachable and website cannot be accessed. Perform the following operations:

     • Check whether the server is running properly. If it is not, restart the server.
     • Add the WAF IP address ranges to the whitelist of the firewall (hardware or software), security protection software, and rate limiting module.
• Cause 3: Origin server performance

  Solution: Contact your website owner to rectify the fault.

504 Gateway Timeout

Scenario: After the configuration of connecting a domain name to WAF is complete, your website works properly. However, with the increasing traffic volume, the number of 504 errors also increases. If you directly access the IP address of the origin server, the 504 error code is returned sometimes.

The possible causes are as follows:

• Cause 1: Backend server performance issues (such as too many connections or high CPU usage)
  Solution:

  1. Optimize the server configuration, including TCP network parameters and ulimit parameters.
  2. To handle large-scale service increase, use method 1 or method 2 to perform the processing.

     Method 1: Add a backend server group to the ELB load balancer.

     Method 2: Create an ELB. Use the EIP of ELB as the IP address of the server to connect to WAF.

     1. Log in to the management console, click Service List in the upper part of the page, and choose Security > Web Application Firewall.
     2. In the navigation pane, choose Website Settings.
     3. In the Domain Name column, click the domain name. Its information is displayed.
     4. In the Server Information area, click . On the displayed page, click Add.
  3. If the Client Protocol is HTTPS, you can use HTTPS on the WAF side. However, it is recommended that HTTP (Server Protocol) to forward the requests to your web server, lowering the computational demands on backend servers.
• Cause 2: The WAF back-to-source IP addresses are not whitelisted or your origin server port is not enabled.

  Solution: Whitelist the WAF back-to-source IP addresses in the corresponding ECS security groups.

• Cause 3: The origin server has a firewall and the firewall blocks the WAF IP addresses.

  Solution: Whitelist the WAF back-to-source IP addresses in the corresponding ECS security groups or uninstall the firewall software except WAF.

• Cause 4: Connection timeout and read timeout

  Solution

  • Database queries are slow.
    • Tune services to shorten the query duration and improve user experience.
    • Modify the request interaction mode so that the persistent connection can have some data transmitted within 60 seconds, such as ACK packets, heartbeat packets, keep-alive packets, and other packets that can keep the session alive.
  • It takes a long time to upload large files.
    • Tune services to shorten the file upload time.
    • An FTP server is recommended for file upload.
    • Upload the file through an IP address or a domain name that is not protected by WAF.
    • The default timeout for a dedicated WAF instance to respond origin servers is 180s.
  • The origin server is faulty.

    Check whether the origin server works properly.

• Cause 5: The bandwidth of the origin server exceeds the upper limit.

  Solution: Increase the bandwidth of the origin server.

• Cause 6: In dedicated mode, the origin server port is not enabled in the security group of the origin server or the origin server subnet is not enabled in network ACLs.

  Solution: Enable the security group ports, such as ports 80 and 443, and configure a network ACL to allow access from the origin server subnet.

404 Not Found Troubleshooting Process and Suggestions

Refer to Figure 4 to fix the 404 Not Found error occurred after your website is connected to WAF.

Figure 4 Troubleshooting for 404 Not Found error

• If the page shown in Figure 5 is displayed, the possible causes and solutions are as follows:
  Figure 5 404 page
  
  Cause 1: A non-standard port is configured when you add the domain name to WAF, but the visitors use the domain name and standard port or use only the domain name to access the website. For example, a non-standard port is configured as shown in Figure 6. A visitor uses https://www.example.com or https://www.example.com:80 to access the website. As a result, 404 error page is displayed.

  Figure 6 Configuration of a non-standard port
  

  Solution: Add the non-standard port to the URL and access the origin server again, for example, https://www.example.com:8080.

  Cause 2: No non-standard port is configured when the domain name is added to WAF. The visitors use the domain name and a non-standard port or the non-standard port configured for origin server port to access the website. For example, access https://www.example.com:8080 when the protection service shown in Figure 7 is configured.

  Figure 7 Non-standard port not configured
  

  

  If no non-standard port is configured, WAF protects services on port 80/443 by default. To protect services on other ports, re-configure domain settings.

  Solution: Use only the domain name to access the website. For example, https://www.example.com.

  Cause 3: The domain name is incorrectly resolved.

  Solution:

  • If the domain name has been added to WAF, resolve the domain name to WAF by referring to .
  • If the domain name is no longer protected by WAF, resolve it to the origin server IP address on the DNS hosting platform.

  Cause 4: If a WAF cluster pointed multiple domain names through HTTPS to an origin server over the same port, origin servers cannot tell which domain name a request originated from. This is because WAF uses persistent connections to forward requests to origin servers and Nginx identifies domain names based on Host and SNI. So, there might be a probability that requests destined for domain name A was mistakenly forwarded to domain name B, which causes 404 not found errors.

  Solution: Modify the server configuration in WAF to route different domain names over different origin server ports.

• If the response page is not similar the one shown in Figure 5, the possible causes and solutions are as follows:

  Cause: The website does not exist or has been deleted.

  Solution: Check the website.