Obtaining Real Client IP Addresses

Background

Generally, a browser request does not directly reach the web server. Proxy servers, such as CDN, WAF, and advanced anti-DDoS, may be deployed between the browser and the origin server. Using WAF as an example, see Figure 1.

Figure 1 WAF deployment diagram

• DNS resolves your domain name to the origin server IP address before your website is connected to WAF. Therefore, web visitors can directly access the server.
• After your website is connected to WAF, DNS resolves your domain name to the CNAME record of WAF. In this way, the traffic passes through WAF. WAF then filters out illegitimate traffic and only routes legitimate traffic back to the origin server.

In this case, the access request may be forwarded by multiple layers of security or acceleration proxies before reaching the origin server. So, how does the server obtain the real IP address of the client that initiates the request?

When forwarding HTTP requests to the downstream server, the transparent proxy server adds an X-Forwarded-For field to the HTTP header to identify the client IP address in the format of X-Forwarded-For: client IP address, proxy 1-IP address, proxy 2-IP address, proxy 3-IP address, ........->....

Then, you can obtain the client IP address from the X-Forwarded-For field, the first IP address in which is the client IP address.

Constraints

• Ensure that Proxy Configured is configured correctly when you add the website to the WAF instance, or WAF cannot obtain the real IP address of your website visitors.

  To ensure that WAF obtains real client IP addresses and takes protective actions configured in protection policies, if your website has layer-7 proxy server such as CDN and cloud acceleration products deployed in front of WAF, select Yes for Proxy Configured. In other cases, select No for Proxy Configured.

• In normal cases, the first IP address in the X-Forwarded-For field is the real IP address of the client. If the length of an IPv6 address exceeds the length limit of the X-Forwarded-For field, the IP address cannot be read. In NAT64, the load balancer uses IPv4 listeners, which cannot read IPv6 addresses.

Obtaining the Client IP Address from WAF

After a website is connected to WAF, WAF is deployed between the client and server as a reverse proxy to protect the website.

The following describes how WAF uses the X-Forwarded-For and X-Real-IP variables to obtain the real IP address of a client:

• Using the X-Forwarded-For field to obtain the client IP address

  The client IP address is placed in the X-Forwarded-For HTTP header field. The format is as follows:

  1	X-Forwarded-For: Client IP address,Proxy 1-IP address,Proxy 2-IP address,...

  

  The first IP address included in the X-Forwarded-For field is the client IP address.

  The methods to obtain the X-Forwarded-For field by invoking the SDK interface in different programming languages are as follows:

  • ASP

    Request.ServerVariables("HTTP_X_FORWARDED_FOR")

  • ASP.NET(C#)

    Request.ServerVariables["HTTP_X_FORWARDED_FOR"]

  • PHP

    $_SERVER["HTTP_X_FORWARDED_FOR"]

  • JSP

    request.getHeader("HTTP_X_FORWARDED_FOR")

• Using the X-Real-IP field to obtain the client IP address (modifications caused by reverse proxies is considered)
  The methods to obtain the X-Real-IP field by invoking the SDK interface in different programming languages are as follows:

  • ASP

    Request.ServerVariables("HTTP_X_REAL_IP")

  • ASP.NET(C#)

    Request.ServerVariables["HTTP_X_REAL_IP"]

  • PHP

    $_SERVER["HTTP_X_REAL_IP"]

  • JSP

    request.getHeader("HTTP_X_REAL_IP")

How Does Tomcat Obtain the Client IP Address from Access Logs?

If Tomcat is deployed on your origin server, you can enable the X-Forwarded-For function of Tomcat to obtain the client IP address.

1. Open the server.xml file in the tomcat/conf/ directory. Partial information about the AccessLogValue logging function is as follows:

   <Host name="localhost"  appBase="webapps" unpackWARs="true" autoDeploy="true">
           <Valve className="org.apache.catalina.values.AccessLogValue" directory="logs"
                  prefix="localhost_access_log." suffix=".txt"
                  pattern="%h %l %u %t "%r" %s %b" />

2. Add %{X-Forwarded-For}i to pattern. Part of the modified server.xml file is as follows:

   <Host name="localhost"  appBase="webapps" unpackWARs="true" autoDeploy="true">
           <Valve className="org.apache.catalina.valves.AccessLogValue" directory="logs"
                  prefix="localhost_access_log." suffix=".txt"
                  pattern="%{X-Forwarded-For}i %h %l %u %t "%r" %s %b" />
   </Host>

3. View the localhost_access_log file to obtain the client IP address from the X-Forwarded-For field.

How Does Apache Obtain the Client IP Address from Access Logs?

If Apache HTTP Server 2.4 or later is deployed on your origin server, you can use the mod_remoteip.so file under remoteip_module in the Apache installation package to obtain the real client IP address.

• CentOS 7.6
  1. Add the following content to the httpd.conf file:

     LoadModule remoteip_module modules/mod_remoteip.so ##Load the mod_remoteip.so module.
     RemoteIPHeader X-Forwarded-For ## Set RemoteIPHeader.
     RemoteIPInternalProxy WAF IP address range##Set the WAF back-to-source IP address range.

     

     • File /etc/httpd/conf.modules.d/00-base.conf:46 has been added to the mod_remoteip.so module.
     • Use spaces to separate multiple back-to-source IP address ranges.
  2. Replace %h with %a in the log format file.

     LogFormat "%a %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined 
     LogFormat "%a %l %u %t \"%r\" %>s %b" common

  3. Restart the Apache service to make the configuration take effect.

• Ubuntu 20.04.2
  1. Add the following content to the apache2.conf file:

     ln -s ../mods-available/remoteip.load /etc/apache2/mods-enabled/remoteip.load  ##Load the mod_remoteip.so module.
     RemoteIPHeader X-Forwarded-For   ## Set RemoteIPHeader.
     RemoteIPInternalProxy WAF IP address range##Set the WAF back-to-source IP address range.

     

     • You can also add the following content to load the mod_remoteip.so module:

       LoadModule remoteip_module /usr/lib/apache2/modules/mod_remoteip.so

     • Use spaces to separate multiple back-to-source IP address ranges.
  2. Replace %h with %a in the log format file.

     LogFormat "%a %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined 
     LogFormat "%a %l %u %t \"%r\" %>s %b" common

  3. Restart the Apache service to make the configuration take effect.

If Apache 2.2 or earlier is deployed on your origin server, to obtain the real client IP address, you can run commands to install third-party module mod_rpaf of Apache and modify the http.conf file

1. Run the following commands to install third-party module mod_rpaf for Apache:

   wget https://github.com/gnif/mod_rpaf/archive/v0.6.0.tar.gz
   tar xvfz mod_rpaf-0.6.tar.gz
   cd mod_rpaf-0.6
   /usr/local/apache/bin/apxs -i -c -n mod_rpaf-2.0.so mod_rpaf-2.0.c

2. Open the httpd.conf configuration file and modify the file content as follows:

   LoadModule rpaf_module   modules/mod_rpaf-2.0.so ##Load module mod_rpaf.
   <IfModule mod_rpaf.c>
   RPAFenable On
   RPAFsethostname On
   RPAFproxy_ips 127.0.0.1 <Reverse proxy IP address>
   RPAFheader X-Forwarded-For
   </IfModule>

3. Define the log format.

   LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" common

4. Enable customized logs.

   CustomLog"[Apache server directory]/logs/$access.log"common

5. Restart the Apache server for the configuration to take effect.

   /[Apache server directory]/httpd/bin/apachectl restart

6. View the access.log file to obtain the client IP address from the X-Forwarded-For field.

How Does Nginx Obtain the Client IP Address from Access Logs?

If an Nginx reverse proxy is deployed on your origin server, you can configure location information on the Nginx reverse proxy so that the backend web server can use similar functions to obtain the client IP address

1. Configure the following information in the corresponding location of the Nginx reverse proxy to obtain the information about the client IP address:

   1 2 3 4

   Location ^ /<uri> { proxy_pass ....; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; }

2. The backend web server obtains the real IP address of your website visitors by defining the Nginx log parameter $http_x_forwarded_for.
   Example

   log_format main ' "<$http_Cdn_Src_IP>" "{$http_x_real_ip}" "[$http_x_forwarded_for]" "$remote_addr" ' '$http_user_agent - $remote_user [$time_local] "$request" '   ' $status $body_bytes_sent "$http_referer" '; 

How Does IIS 6 Obtain the Client IP Address from Access Logs?

If you have deployed an IIS 6 server on your origin server, you can install the F5XForwardedFor.dll plug-in and obtain the client IP address from the access logs recorded by the IIS 6 server.

1. Download the F5XForwardedFor module.
2. Copy the F5XForwardedFor.dll file in the x86\Release or x64\Release directory to a specified directory (for example, C:\ISAPIFilters) based on the operating system version of your server. Ensure that the IIS process has the read permission for the directory.
3. Open the IIS manager, right-click the website that is currently open, and choose Attribute from the shortcut menu. The Attribute page is displayed.
4. On the Attribute page, switch to ISAPI filter and click Add. In the dialog box that is displayed, configure the following information:
   • Filter Name: Set this parameter to F5XForwardedFor.
   • Executable file: Set this parameter to the full path of F5XForwardedFor.dll, for example, C:\ISAPIFilters\F5XForwardedFor.dll.

5. Click OK to restart the IIS 6 server.
6. View the access logs recorded by the IIS 6 server (the default log path is C:\WINDOWS\system32\LogFiles\, and the IIS log file name extension is .log). You can obtain client IP address from the X-Forwarded-For field.

How Does IIS 7 Obtain the Client IP Address from Access Logs?

If you have deployed an IIS 7 server on your origin server, you can install the F5XForwardedFor.dll module and obtain the client IP address from the access logs recorded by the IIS 7 server.

1. Download the F5XForwardedFor module.
2. Copy the F5XFFHttpModule.dll and F5XFFHttpModule.ini files in the x86\Release or x64\Release directory to a specified directory (for example, C:\x_forwarded_for\x86 or C:\x_forwarded_for\x64) based on the operating system version of your server. Ensure that the IIS process has the read permission for the directory.
3. On the server home page, double-click Modules to go to the Modules page.
4. Click Configure Native Module. In the dialog box displayed, click Register.
5. In the displayed dialog box, register the downloaded DLL file according to the operating system, and then click OK.
   • x86 operating system: registration module x_forwarded_for_x86
     • Name: x_forwarded_for_x86
     • Path: C:\x_forwarded_for\x86\F5XFFHttpModule.dll
   • x64: Register the module x_forwarded_for_x64.
     • Name: x_forwarded_for_x64
     • Path: C:\x_forwarded_for\x64\F5XFFHttpModule.dll

6. After the registration is complete, select the newly registered module (x_forwarded_for_x86 or x_forwarded_for_x64) and click OK.
7. In ISAPI and CGI restriction, add the registered DLL files by operating system and change Restriction to Permitting.
   • x86 operating system:
     • ISAPI or CGI path: C:\x_forwarded_for\x86\F5XFFHttpModule.dll
     • Description: x86
   • x64 operating system:
     • ISAPI or CGI path: C:\x_forwarded_for\x64\F5XFFHttpModule.dll
     • Description: x64

8. Restart the IIS 7 server and wait for the configuration to take effect.
9. View the access logs recorded by the IIS 7 server (the default log path is C:\WINDOWS\system32\LogFiles\, and the IIS log file name extension is .log). You can obtain the client IP address from the X-Forwarded-For field.