Help Center/ Cloud Container Engine/ Best Practices/ Networking/ Configuring LoadBalancer Ingress Load Balancing for a Persistent-Connection Service

Updated on 2026-03-10 GMT+08:00

View PDF

Configuring LoadBalancer Ingress Load Balancing for a Persistent-Connection Service

When a containerized persistent-connection service exposes traffic to external systems through a LoadBalancer ingress, pressure-testing may reveal uneven load distribution. This can cause certain backend servers to become overloaded, ultimately affecting overall service performance. To ensure balanced traffic distribution and efficient services, you can fine-tune client connection idle timeout, client request timeout, backend response timeout, and other configurations.

Client connection idle timeout: defines how long a connection can remain open when no data is being transmitted. For persistent connections, configuring an appropriate idle timeout prevents connections from occupying resources unnecessarily while still keeping them available when needed. If the timeout is too short, connections may disconnect and reconnect frequently, increasing server load. If the timeout is too long, idle connections may consume resources that could be used more efficiently elsewhere.
Client request timeout: defines how long a server waits for a client to send a complete request after the connection is established. For persistent connections, this prevents the server from waiting indefinitely for client data and preventing resources from being occupied for a long time. A reasonable client request timeout improves server responsiveness and resource utilization.
Backend response timeout: defines how long a server waits for a backend server to return a response after forwarding a request. For persistent connections, a reasonable backend response timeout prevents congestion caused by slow backend responses. This improves system performance and stability.

Prerequisites

A CCE Turbo cluster is available. The cluster has the Cloud Native Cluster Monitoring add-on installed.
A dedicated load balancer is available.
An ECS that can access the Internet is available. The ECS also has Docker and wrk installed.

Step 1: Prepare a Test Image

Prepare the required Dockerfile, go.mod, and app.go files.

The sample files are shown below.

Dockerfile

FROM golang:1.19-alpine AS builder
 
WORKDIR /app
 
# Copy the Go module file.
COPY go.mod ./
# Download dependencies.
RUN go mod download
 
# Copy the source code.
COPY *.go ./
 
# Build an application.
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o http-long-conn .
 
# Running phase
FROM alpine:3.10.2
 
RUN apk --no-cache add ca-certificates
 
WORKDIR /root/
 
# Copy the binary file from the build phase.
COPY --from=builder /app/http-long-conn .
 
# Exposed port
EXPOSE 8080
 
# Configure environment variables.
ENV PORT=8080
 
# Run the service.
CMD ["./http-long-conn"]

go.mod
```
module http-long-conn
 
go 1.19
```

app.go

package main
 
import (
                         "encoding/json"
                         "fmt"
                         "log"
                         "net/http"
                         "os"
                         "sync"
                         "time"
)
 
var (
                         activeConnections int
                         mu                sync.RWMutex
                         startTime         = time.Now()
                         totalRequests     int64
)
 
type StatsResponse struct {
                         Status            string `json:"status"`
                         ActiveConnections int    `json:"active_connections"`
                         TotalRequests     int64  `json:"total_requests"`
                         Uptime            string `json:"uptime"`
                         Hostname          string `json:"hostname"`
                         Message           string `json:"message"`
}
 
type ConnectionRequest struct {
                         ClientID  string `json:"client_id"`
                         KeepAlive int    `json:"keep_alive"` // How long a connection can remain open, in seconds
}
 
type ConnectionResponse struct {
                         Status       string `json:"status"`
                         ConnectionID string `json:"connection_id"`
                         ServerTime   string `json:"server_time"`
                         KeepAlive    int    `json:"keep_alive"`
                         Message      string `json:"message"`
}
 
func main() {
                         port := os.Getenv("PORT")
                         if port == "" {
                                                   port = "8080"
                         }
 
                         http.HandleFunc("/", rootHandler)
                         http.HandleFunc("/health", healthHandler)
                         http.HandleFunc("/stats", statsHandler)
                         http.HandleFunc("/long-connection", longConnectionHandler)
                         http.HandleFunc("/ws", websocketHandler)
 
                         log.Printf("HTTP long-connection service started at port %s", port)
                         log.Printf("Available endpoints:")
                         log.Printf("  GET  /              - Root path")
                         log.Printf("  GET  /health        - Health check")
                         log.Printf("  GET  /stats         - Service statistics")
                         log.Printf("  POST /long-connection - Establish a persistent connection.")
                         log.Printf("  GET  /ws            - WebSocket connection")
 
                         if err := http.ListenAndServe(":"+port, nil); err != nil {
                                                   log.Fatal("Failed to start the server:", err)
                         }
}
 
func rootHandler(w http.ResponseWriter, r *http.Request) {
                         mu.Lock()
                         totalRequests++
                         mu.Unlock()
 
                         hostname, _ := os.Hostname()
                         response := map[string]interface{}{
                                                   "service":   "HTTP persistent connection test service",
                                                   "version":   "1.0.0",
                                                   "hostname":  hostname,
                                                   "timestamp": time.Now().Format(time.RFC3339),
                                                   "endpoints": []string{
                                                                            "GET  /health",
                                                                            "GET  /stats",
                                                                            "POST /long-connection",
                                                                            "GET  /ws",
                                                   },
                         }
 
                         w.Header().Set("Content-Type", "application/json")
                         json.NewEncoder(w).Encode(response)
}
 
func healthHandler(w http.ResponseWriter, r *http.Request) {
                         response := map[string]string{
                                                   "status":    "healthy",
                                                   "timestamp": time.Now().Format(time.RFC3339),
                         }
 
                         w.Header().Set("Content-Type", "application/json")
                         json.NewEncoder(w).Encode(response)
}
 
func statsHandler(w http.ResponseWriter, r *http.Request) {
                         mu.RLock()
                         defer mu.RUnlock()
 
                         hostname, _ := os.Hostname()
                         response := StatsResponse{
                                                   Status:            "running",
                                                   ActiveConnections: activeConnections,
                                                   TotalRequests:     totalRequests,
                                                   Uptime:            time.Since(startTime).String(),
                                                   Hostname:          hostname,
                                                   Message:           "The HTTP persistent-connection service is running.",
                         }
 
                         w.Header().Set("Content-Type", "application/json")
                         json.NewEncoder(w).Encode(response)
}
 
func longConnectionHandler(w http.ResponseWriter, r *http.Request) {
                         mu.Lock()
                         activeConnections++
                         totalRequests++
                         mu.Unlock()
 
                         defer func() {
                                                   mu.Lock()
                                                   activeConnections--
                                                   mu.Unlock()
                         }()
 
                         // Configure the persistent connection header.
                         w.Header().Set("Content-Type", "application/json")
                         w.Header().Set("Connection", "keep-alive")
 
                         var req ConnectionRequest
                         if r.Method == "POST" {
                                                   if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
                                                                            http.Error(w, `{"error": "Invalid request body"}`, http.StatusBadRequest)
                                                                            return
                                                   }
                         }
 
                         // By default, the connection remains open for 30 seconds.
                         keepAlive := 30
                         if req.KeepAlive > 0 {
                                                   keepAlive = req.KeepAlive
                         }
 
                         clientID := req.ClientID
                         if clientID == "" {
                                                   clientID = fmt.Sprintf("client-%d", time.Now().UnixNano())
                         }
 
                         // Create a response.
                         response := ConnectionResponse{
                                                   Status:       "connected",
                                                   ConnectionID: fmt.Sprintf("conn-%d", time.Now().UnixNano()),
                                                   ServerTime:   time.Now().Format(time.RFC3339),
                                                   KeepAlive:    keepAlive,
                                                   Message:      "Persistent connection established. It will be maintained.",
                         }
 
                         // Send the initial response.
                         if err := json.NewEncoder(w).Encode(response); err != nil {
                                                   log.Printf("Failed to send the initial response: %v", err)
                                                   return
                         }
 
                         // Refresh the buffer to ensure that data can be sent to the client.
                         if flusher, ok := w.(http.Flusher); ok {
                                                   flusher.Flush()
                         }
 
                         // Keep the connection and periodically send heartbeat messages.
                         ticker := time.NewTicker(5 * time.Second)
                         defer ticker.Stop()
 
                         timeout := time.After(time.Duration(keepAlive) * time.Second)
 
                         for {
                                                   select {
                                                   case <-ticker.C:
                                                                            // Send a heartbeat message.
                                                                            heartbeat := map[string]interface{}{
                                                                                                      "type":      "heartbeat",
                                                                                                      "timestamp": time.Now().Format(time.RFC3339),
                                                                                                      "client_id": clientID,
                                                                            }
 
                                                                            // Send the JSON data separated by newline characters.
                                                                            if _, err := w.Write([]byte("\n")); err != nil {
                                                                                                      log.Printf("Failed to send newline characters: %v", err)
                                                                                                      return // The client is disconnected.
                                                                            }
 
                                                                            if err := json.NewEncoder(w).Encode(heartbeat); err != nil {
                                                                                                      log.Printf ("Failed to send the heartbeat message: %v", err)
                                                                                                      return // The client is disconnected.
                                                                            }
 
                                                                            if flusher, ok := w.(http.Flusher); ok {
                                                                                                      flusher.Flush()
                                                                            }
 
                                                   case <-timeout:
                                                                            // The connection times out.
                                                                            endMsg := map[string]string{
                                                                                                      "type":    "connection_end",
                                                                                                      "message": "Connection duration expired.",
                                                                            }
                                                                            json.NewEncoder(w).Encode(endMsg)
                                                                            return
 
                                                   case <-r.Context().Done():
                                                                            // The client is disconnected.
                                                                            log.Printf("Client disconnected: %s", clientID)
                                                                            return
                                                   }
                         }
}
 
func websocketHandler(w http.ResponseWriter, r *http.Request) {
                         // Configure a simple WebSocket compatibility response.
                         w.Header().Set("Content-Type", "application/json")
                         json.NewEncoder(w).Encode(map[string]string{
                                                   "error":      "The WebSocket endpoint needs to be implemented additionally. Currently, HTTP persistent connections are used.",
                                                   "suggestion": "Use /long-connection to test HTTP persistent connections.",
                         })
}

After the creation, check whether the dockerfile directory contains the following files:
```
app.go  Dockerfile  go.mod
```

Build the image.

docker build -t http-long-conn:v1 .

Wait until the image build is complete.

...
Successfully built fdc357e8247d
Successfully tagged http-long-conn:v1

Log in to the SWR image repository and push the image to the SWR repository. For details, see Pushing an Image.
```
docker tag http-long-conn:v1 {image-repository-address}/{organization}/http-long-conn:v1
docker push {image-repository-address}/{organization}/http-long-conn:v1
```
{image-repository-address} specifies an SWR image repository address. {organization} specifies an SWR organization name.

Step 2: Deploy a Workload in the Cluster

Access the cluster using kubectl.

Create a file named http-long-conn.yaml. You can name the file as required.

vi http-long-conn.yaml

The file content is as follows:

kind: Deployment
apiVersion: apps/v1
metadata:
  name: http-long-conn
  namespace: default
  labels:
    app: http-long-conn
spec:
  replicas: 4
  selector:
    matchLabels:
      app: http-long-conn
  template:
    metadata:
      labels:
        app: http-long-conn
    spec:
      containers:
        - name: http-long-conn
          image: {image-repository-address}/{organization}/http-long-conn:v1 # Replace it with the SWR image address you uploaded.
          ports:
            - containerPort: 8080
              protocol: TCP
          env:
            - name: PORT
              value: '8080'
          resources:
            limits:
              cpu: 100m
              memory: 128Mi
            requests:
              cpu: 100m
              memory: 128Mi
          livenessProbe:
            httpGet:
              path: /health
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 30
            timeoutSeconds: 1
            periodSeconds: 10
            successThreshold: 1
            failureThreshold: 3
          readinessProbe:
            httpGet:
              path: /health
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 5
            timeoutSeconds: 1
            periodSeconds: 5
            successThreshold: 1
            failureThreshold: 3
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: IfNotPresent
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      dnsPolicy: ClusterFirst
      securityContext: {}
      schedulerName: default-scheduler
      imagePullSecrets:
        - name: default-secret
      tolerations: null
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 25%
      maxSurge: 25%
  revisionHistoryLimit: 10
  progressDeadlineSeconds: 600

---
apiVersion: v1
kind: Service
metadata:
  name: http-long-conn
  labels:
    app: http-long-conn
  namespace: default
spec:
  selector:
    app: http-long-conn
  ports:
    - name: http-0
      targetPort: 8080
      nodePort: 0
      port: 8080
      protocol: TCP
  type: ClusterIP

Create the workload and the Service.
```
kubectl create -f http-long-conn.yaml
```
Check the workload status.
```
kubectl get pod -l app=http-long-conn
```

Step 4: Create a LoadBalancer Ingress

Create a file named elb-ingress.yaml. You can name the file as required.

vi elb-ingress.yaml

The file content is as follows:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: http-long-conn
  namespace: default
  annotations:
    kubernetes.io/elb.port: '8080'
    kubernetes.io/elb.id: 1fce4b38-c72b-4fd4-8430-62d46c0a7998   #ELB ID
    kubernetes.io/elb.class: performance
    kubernetes.io/elb.keepalive_timeout: '300'  # Timeout setting for client connections
    kubernetes.io/elb.client_timeout: '60'      # Timeout for waiting for a request from a client
    kubernetes.io/elb.member_timeout: '60'      # Timeout for waiting for a response from a backend server
spec:
  rules:
    - host: example.com # Use a custom domain name.
      http:
        paths:
          - path: /
            backend:
              service:
                name: http-long-conn
                port:
                  number: 8080
            property:
              ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH
            pathType: ImplementationSpecific
  ingressClassName: cce

Create the ingress.
```
kubectl create -f elb-ingress.yaml
```
Test the domain name connectivity.

Step 5: Perform a Pressure Test

Use wrk to perform a pressure test.

wrk -t2 -c100 -d300s -H "Connection: keep-alive" http://example.com/long-connection

Where:

-t2: specifies that two threads are enabled.
-c100: specifies that 100 concurrent connections are established.
-d300s: specifies how long the test will last.

Click to enlarge

In this example, pressure tests are performed using 100 and 200 concurrent connections, respectively. By viewing the pod monitoring metrics in the Monitoring Center, you can see that the pods are operating in a balanced load-distribution state.

Click to enlarge