Using Nginx to Unify Kibana and Dashboards Management for Multiple Clusters

Scenarios

If your organization uses multiple Elasticsearch and OpenSearch clusters, you can configure a reverse proxy (such as Nginx) to serve as a single entry point for Kibana and OpenSearch Dashboards for all clusters. To distinguish between different clusters (for example, one for each business line), you need to configure a custom base path for Kibana/OpenSearch Dashboards of each cluster. Traditionally, you need to make this configuration on each node of each cluster. This process can be cumbersome and error-prone. CSS simplifies this by providing centralized configuration management. By entering the custom base path once on the console, CSS automatically synchronizes it to all relevant cluster nodes. This ensures configuration consistency, simplifies multi-cluster management, and improves security.

This topic describes how to use Nginx to unify Kibana management for multiple Elasticsearch clusters and OpenSearch Dashboards management for multiple OpenSearch clusters. The procedure also applies to the following scenarios:

• Using Nginx to unify Kibana management for multiple Elasticsearch clusters
• Using Nginx to unify OpenSearch Dashboards management for multiple OpenSearch clusters

Solution Architecture

Figure 1 Solution architecture

A reverse proxy server like Nginx enables access path isolation (routing requests to different service backends via path rules) and centralized management (ensuring configuration consistency across nodes). The technical implementation is as follows:

• CSS automatically synchronizes the base path configuration across all nodes of a cluster to ensure consistency. The server.basePath and server.rewriteBasePath parameters define path rules for Elasticsearch and OpenSearch clusters, respectively.
• Nginx is deployed on an ECS to connect clusters. Nginx uses the location block in its configuration file (such as nginx.conf) to match request paths and forward them to the appropriate service backends. Path isolation is achieved by configuring different location blocks in the Nginx configuration file, while centralized management is ensured by CSS, which automatically synchronizes the base path across all cluster nodes.
• Users access Nginx through a unified address (for example, http://203.0.113.45:5600). Nginx reads the requests' base path (for example, /css-es-test or /css-os-test) and routes them to the corresponding Kibana or OpenSearch Dashboards instance, which then forwards the requests to the corresponding Elasticsearch or OpenSearch cluster.

Advantages

• Centralized configuration management: Custom base paths can be configured on the CSS console and automatically synchronized to all relevant cluster nodes. This simplifies multi-cluster management and reduces errors.
• Flexible access control and path isolation: Base paths route requests to different clusters, enabling logical isolation for different business lines. Depending on your security requirements, you can choose to enable or disable the security mode (HTTPS/HTTP) for clusters.
• High availability and stability: Nginx, as a mature reverse proxy, can efficiently handle high concurrency to ensure reliable and stable access to your clusters.
• Scalability and ease of use: Adding a new cluster is straightforward—simply define its base path and the corresponding Nginx mapping rules. The Nginx configuration can be updated dynamically without interrupting ongoing requests.

Prerequisites

• An Elasticsearch 7.10.2 cluster has been created, and its status is Running. You have recorded its private IP address (for example, 10.10.1.53:9200).
• An OpenSearch cluster has been created and its status is Running. You have recorded its private IP address (for example, 10.10.1.117:9200).
• An ECS has been created for installing Nginx. The ECS OS is CentOS 8.0, and an EIP (for example, 203.0.113.45) has been allocated to the ECS. The ECS is in the same VPC as the Elasticsearch and OpenSearch clusters. If they are not in the same VPC, you need to configure cluster routes to connect them. For details, see Purchasing and Using a Linux ECS.

Step 1: Configuring a Custom Kibana Base Path

1. Log in to the CSS management console.
2. In the navigation pane on the left, choose Clusters > Elasticsearch.
3. In the cluster list, click the name of the target cluster. The cluster information page is displayed.
4. Choose Cluster Settings > Parameter Settings > Kibana.
5. Click Edit, and set the parameters.

   Table 1 Configuring Base Path

   Parameter	Example	Description
   server.basePath	/css-es-test	Custom value for Base Path. The value: Must start with /. Must not contain consecutive slashes (for example, //). Can only contain letters, digits, hyphens (-), underscores (_), periods (.), and slashes (/). Can contain multi-level paths, but its length cannot exceed 254 characters. Must not end with a slash (/).
   server.rewriteBasePath	true	Whether Kibana or a reverse proxy will remove the server.basePath prefix from the request path. The value can be: true: Kibana will automatically remove the server.basePath prefix from the request path. false (default): The reverse proxy (such as Nginx) will remove the server.basePath prefix from the request path.

6. After the change is complete, click Save. In the displayed dialog box, confirm the settings, and select "Modifying Kibana settings will restart Kibana, making it temporarily inaccessible." Then click OK.

   If Job Status is Succeeded in the parameter settings task list below, the change is successful.

Step 2: Configuring a Custom OpenSearch Dashboards Base Path

1. Log in to the CSS management console.
2. In the navigation pane on the left, choose Clusters > OpenSearch.
3. In the cluster list, click the name of the target cluster. The cluster information page is displayed.
4. Choose Cluster Settings > Parameter Settings > Dashboards.
5. Click Edit, and set the parameters.

   Table 2 Configuring Base Path

   Parameter	Example	Description
   server.basePath	/css-os-test	Custom value for Base Path. The value: Must start with /. Must not contain consecutive slashes (for example, //). Can only contain letters, digits, hyphens (-), underscores (_), periods (.), and slashes (/). Can contain multi-level paths, but its length cannot exceed 254 characters. Must not end with a slash (/).
   server.rewriteBasePath	false	Whether OpenSearch Dashboards or a reverse proxy will remove the server.basePath prefix from the request path. The value can be: true: OpenSearch Dashboards will automatically remove the server.basePath prefix from the request path. false (default): The reverse proxy (such as Nginx) will remove the server.basePath prefix from the request path.

6. After the change is complete, click Save. In the displayed dialog box, confirm the settings, and select "Modifying Dashboards settings will restart Dashboards, making it temporarily inaccessible." Then click OK.

   If Job Status is Succeeded in the parameter settings task list below, the change is successful.

Step 3: Installing Nginx

1. Log in to the ECS.
2. Install Nginx:
   1. Run the following command to install Nginx:

      sudo dnf -y install https://nginx.org/packages/centos/8/x86_64/RPMS/nginx-1.20.1-1.el8.ngx.x86_64.rpm

   2. Run the following command to check the Nginx version:

      nginx -v

      Information similar to the following is returned:

      nginx version: nginx/1.20.1

   3. Run the following commands to start Nginx and configure auto-start of Nginx upon system startup:

      systemctl start nginx
      systemctl enable nginx

   4. Check that Nginx is now active.

      systemctl status nginx.service

      Information similar to the following is returned:

      

   5. Enter http://EIP of the ECS in the browser address bar to access Nginx. If the following page is displayed, Nginx has been installed.
      Figure 2 Accessing Nginx
      
3. Modify the Nginx configuration file to configure Nginx as the reverse proxy for Elasticsearch and OpenSearch clusters.
   1. Edit the Nginx configuration file.

      vi /etc/nginx/nginx.conf

   2. Add a server. The IP address and port number of the server must match those used in your environment.

      server {
        # Configure a custom Nginx access port. For example, if the port number is set to 5600, you can access Nginx through http://{Nginx IP address}:5600.
        listen 5600;
      
        # Custom service name, such as kibana_proxy.
        server_name kibana_proxy;
      
        # Define a forwarding rule. The rule applies the URL prefix is server.basePath.
        location /css-os-test {
          # Remove the server.basePath prefix from the request path. This parameter is required only when server.rewriteBasePath is set to false.
          rewrite ^/css-os-test/(.*)$ /$1 break;
      
          # Kibana or Dashboards access address, which can be an internal or public network address
          proxy_pass https://10.10.1.117:5601;
        }
      
        location /css-es-test {
          proxy_pass http://10.10.1.53:5601;
        }
      }
      

   3. Reload the configuration.

      nginx -s reload

   4. Check that Nginx is now active.

      systemctl status nginx.service

Step 4: Using Nginx to Unify Cluster Access

Use Nginx to unify access to multiple clusters by providing a single entry point, while balancing the load.

Access address: http://{Nginx IP address}:{Nginx access port}{Base path}.

Nginx IP address is the public IP address of the Nginx server, that is, the EIP allocated to the ECS. Nginx access port is the server port configured in the Nginx configuration file. Base path is the server.basePath of Kibana or OpenSearch Dashboards.

Therefore, in our example, the cluster access addresses are as follows:

• Kibana access address for the Elasticsearch cluster: http://203.0.113.45:5600/css-es-test
• OpenSearch Dashboards access address for the OpenSearch cluster: http://203.0.113.45:5600/css-os-test

If you can log in to Kibana and OpenSearch Dashboards, the configuration is successful.