Connecting Memcached Exporter to AOM for Monitoring Metrics

Scenario

When using Memcached, you need to monitor their running and locate their faults in a timely manner. The Prometheus monitoring function monitors Memcached running using Exporter in the CCE container scenario. This section describes how to monitor Memcached.

Constraints

You are advised to use CCE for unified Exporter management.

Prerequisites

A CCE cluster has been created and Memcached has been installed.
Your service has been connected for Prometheus monitoring and a CCE cluster has also been connected. For details, see Prometheus Instance for CCE.
You have uploaded the memcached_exporter image to SoftWare Repository for Container (SWR). For details, see Uploading an Image Through a Container Engine Client.

Deploying Memcached Exporter in a CCE Cluster

Log in to the CCE console.
Click the connected cluster. The cluster management page is displayed.

Perform the following operations to deploy Exporter:

Configure a secret.
In the navigation pane, choose ConfigMaps and Secrets. Then click Create from YAML in the upper right corner of the page. For details about how to configure a secret, see Creating a Secret.

The following is a YAML configuration example. The Memcached connection string is in the format of "http://{IP address}:{Port}".
```
apiVersion: v1
kind: Secret
metadata:
    name: memcached-exporter-secret
    namespace: default
type: Opaque
stringData:
    memcachedURI: 120.46.215.4:11211 # Memcached address.
```

Deploy Memcached Exporter.

In the navigation pane, choose Workloads. On the Deployments tab page, click Create from YAML in the upper right corner and then configure a YAML file to deploy Exporter. For more details about Exporter parameters, see memcached_exporter.

YAML configuration example:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    k8s-app: memcached-exporter # Change the value based on service requirements.
  name: memcached-exporter # Change the value based on service requirements.
  namespace: default 
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: memcached-exporter # Change the value based on service requirements.
  template:
    metadata:
      labels:
        k8s-app: memcached-exporter # Change the value based on service requirements.
    spec:
      containers:
      - env:
          - name: Memcached_Url
            valueFrom:
              secretKeyRef:
                name: memcached-exporter-secret # Secret name specified in the previous step.
                key: memcachedURI # Secret key specified in the previous step.
          - name: Memcached_ALL
            value: "true"
        image: swr.cn-east-3.myhuaweicloud.com/aom-org/bitnami/memcached-exporter:0.13.0 # Image uploaded to SWR, as described in "Prerequisites".
        imagePullPolicy: IfNotPresent
        name: memcached-exporter
        ports:
        - containerPort: 9150
          name: metric-port
        securityContext:
          privileged: false
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
    dnsPolicy: ClusterFirst
    imagePullSecrets:
    - name: default-secret
    restartPolicy: Always
    schedulerName: default-scheduler
    securityContext: {}
    terminationGracePeriodSeconds: 30
---
apiVersion: v1
kind: Service
metadata:
  name: memcached-exporter
spec:
  type: NodePort
  selector:
    k8s-app: memcached-exporter
  ports:
    - protocol: TCP
      nodePort: 30122
      port: 9150
      targetPort: 9150

Check whether Memcached Exporter is successfully deployed.
1. On the Deployments tab page, click the Deployment created in 3.b. In the pod list, choose More > View Logs in the Operation column. The Exporter is successfully started and its access address is exposed.
2. Run the following commands to check whether Memcached Exporter is successfully deployed. If metric data is returned, it is successfully deployed. Perform verification using one of the following methods:
  - Log in to a cluster node and run either of the following commands:
```
curl http://{Cluster IP address}:9150/metrics
```
```
curl http://{Private IP address of any node in the cluster}:30122/metrics
```
  - Access http://{Public IP address of any node in the cluster}:30122/metrics.
    Figure 1 Access address
  - In the instance list, choose More > Remote Login in the Operation column and run the following command:
```
curl http://localhost:9150/metric
```
    Figure 2 Executing the command

Configuring a CCE Cluster Metric Collection Rule

Add PodMonitor to configure a Prometheus collection rule for monitoring the service data of applications deployed in the CCE cluster.

Log in to the AOM 2.0 console.
In the navigation pane on the left, choose Prometheus Monitoring > Instances.
In the instance list, click a Prometheus instance for CCE.
In the navigation pane on the left, choose Metric Management. On the Settings tab page, click PodMonitor.

Click Add PodMonitor. In the displayed dialog box, set parameters and click OK.

In the following example, metrics are collected every 30s. Therefore, you can check the reported metrics on the AOM page about 30s later.

apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
  name: memcached-exporter
  namespace: default
spec:
  namespaceSelector: # Select the namespace where the target Exporter pod is located.
    matchNames:
      - default # Namespace where Exporter is located.
  podMetricsEndpoints:
  - interval: 30s # Set the metric collection period.
    path: /metrics # Enter the path corresponding to Prometheus Exporter. Default: /metrics.
    port: metric-port # Enter the name of ports in the YAML file corresponding to Prometheus Exporter.
  selector: # Enter the label of the target Exporter pod.
    matchLabels:
      k8s-app: memcached-exporter

Verifying that CCE Cluster Metrics Can Be Reported to AOM

Log in to the AOM 2.0 console.
In the navigation pane on the left, choose Prometheus Monitoring > Instances.
Click the Prometheus instance connected to the CCE cluster. The instance details page is displayed.
On the Metrics tab page of the Metric Management page, select your target cluster.
Select job {namespace}/memcached-exporter to query metrics starting with go_memstats.

Setting a Dashboard and Alarm Rule on AOM

By setting a dashboard, you can monitor CCE cluster data on the same screen. By setting an alarm rule, you can detect cluster faults and implement warning in a timely manner.

Setting a dashboard
1. Log in to the AOM 2.0 console.
2. In the navigation pane, choose Dashboard > Dashboard. On the displayed page, click Add Dashboard to add a dashboard. For details, see Creating a Dashboard.
3. On the Dashboard page, select a Prometheus instance for CCE and click Add Graph. For details, see Adding a Graph to a Dashboard.
Setting an alarm rule
1. Log in to the AOM 2.0 console.
2. In the navigation pane, choose Alarm Center > Alarm Rules.
3. On the Prometheus Monitoring tab page, click Create Alarm Rule to create an alarm rule. For details, see Creating a Metric Alarm Rule.