Ingesting Data to AOM Using Open-Source APIs and Protocols

Configuring Data Ingestion to AOM Using Open-Source APIs and Protocols

1. Log in to the AOM 2.0 console.
2. In the navigation pane on the left, choose Access Center > Access Center to go to the new access center.

   If the old access center is displayed, click Try New Version in the upper right corner.

3. Select the check box next to APIs/protocols... under Types to filter out target cards.
4. Click Ingest Metric (AOM) or Ingest Trace (APM) to quickly ingest metrics or traces, or click Ingest Log (LTS) Details to ingest logs or check documents related to log ingestion.
   • Ingest Metric (AOM): AOM supports metric ingestion using open-source APIs and protocols. By clicking Ingest Metric (AOM), you can quickly ingest metrics using open-source APIs and protocols.
   • Ingest Trace (APM): AOM provides an entry for ingesting traces to APM using open-source APIs and protocols. By clicking Ingest Trace (AOM), you can quickly ingest traces using open-source APIs and protocols.
   • Ingest Log (LTS) Details: AOM provides an entry for ingesting logs to LTS using open-source APIs and protocols.
     • By clicking Details on Ingest Log (LTS) Details, you can check documents related to log ingestion. You can ingest logs according to the documents.
     • For some components, you can quickly ingest their logs by clicking Ingest Log (LTS). For example, the Cross-Account Ingestion - Log Stream Mapping card.

   Table 1 Ingesting metrics using open-source APIs and protocols

   Card	Related Operation
   AOM APIs	Use the open APIs of AOM to report metric data. For details, see Adding Monitoring Data.
   LTS APIs	Use the open APIs of LTS to report log data. For details, see Using APIs to Ingest Logs to LTS.
   APM APIs	Use the open APIs of APM to report application performance monitoring data. For details, see Using APIs to Ingest Traces to APM.
   Common Prometheus Instance	Suitable for customers who have self-built Prometheus servers, but need Prometheus storage availability and scalability through remote write. For details, see Connecting Common Prometheus Instances (AOM).
   Kafka Protocol	TCP-based binary protocol used by Kafka (a high-throughput, distributed message system). For details, see Using Kafka to Report Logs to LTS.
   OpenTelemetry	Report application performance monitoring data using the OpenTelemetry protocol. For details, see Using OpenTelemetry to Ingest Traces to APM.
   SkyWalking	Report application performance monitoring data using the SkyWalking protocol. For details, see Using SkyWalking to Ingest Traces to APM.
   Syslog Protocol	Exchange logs between devices based on UDP/TCP. For details, see Using Flume to Report Logs to LTS.
   Flume	Collect logs and upload them to LTS using Flume. For details, see Using Flume to Report Logs to LTS.
   Beats	Collect logs and upload them to LTS using Beats. For details, see Using Kafka to Report Logs to LTS.
   Logstash	Collect logs and upload them to LTS using Logstash. For details, see Using Kafka to Report Logs to LTS.
   SNMP Protocol	Remotely monitor network devices. Three versions (v1, v2, and v3) available. For details, see Using Flume to Report Logs to LTS.
   Java SDK (log4j2)	Configure Huawei Cloud Appender in Log4j2 and then report generated logs to LTS. For details, see LTS Log4j 2 SDK.
   Logback SDK	Configure Huawei Cloud Appender in logback and then report generated logs to LTS.
   Cross-Account Ingestion - Log Stream Mapping	Create an agency to map the delegator account's log stream to the delegated account's log stream. For details, see Ingesting Logs to LTS Across IAM Accounts.
   Custom Prometheus Metrics	Ingest custom Prometheus metrics. For details, see Ingesting Custom Prometheus Metrics.

Ingesting Custom Prometheus Metrics

You can ingest custom Prometheus metrics. They can be automatically reported to AOM.

• Prerequisites
  • You have connected a Prometheus instance for ECS or a common Prometheus instance. For details, see Managing Prometheus Instances or Connecting Open-Source Monitoring Systems to AOM.
  • A host group has been created. For details, see (New) Managing Host Groups.
• Procedure

1. Log in to the AOM 2.0 console.
2. In the navigation pane, choose Access Center > Access Center. Click Try New Version in the upper right corner of the page.
3. Click Custom Prometheus Metrics under APIs/Protocols..., and then click Ingest Metric (AOM) on the card to enter the configuration page.
4. Configure parameters for ingesting custom Prometheus metrics.
   1. Select a Prometheus instance.
      1. Instance Type: Select a Prometheus instance type. Options: Prometheus for ECS and Common Prometheus instance.
      2. Instance Name: Select a Prometheus instance from the drop-down list.

         If no Prometheus instance is available, click Create Instance to create one.

   2. Select a host group.
      In the host group list, select a target host group.

      • If no host group is available, click Create Host Group to create one.
      • You can also perform editing, deletion, and other operations on the host group as needed. For details, see (New) Managing Host Groups.

      Collection configurations are delivered by host group. Therefore, it is easy for you to configure data collection for multiple hosts. When there is a new host, simply add it to a host group and the host will automatically inherit the log ingestion configurations associated with the host group.

   3. Configure the collection.
      Under Configure Collection, set parameters by referring to the following table.

      Table 2 Parameters for configuring a collection task

      Operation	Parameter	Description
      Basic Settings	Configuration Name	Name of a custom metric ingestion rule. Enter 1 to 50 characters starting with a letter. Only letters, digits, underscores (_), and hyphens (-) are allowed.
      Metric Collection Rule	Collection Target	Enter the target IP address and port number for collecting Prometheus metrics, for example, 10.0.0.1:3306.
      	Metric Collection Interval (s)	Interval for collecting metrics, in seconds. Options: 10, 30, and 60 (default).
      	Metric Collection Timeout (s)	Timeout period for executing a metric collection task, in seconds. Options: 10, 30, and 60 (default). The timeout period cannot exceed the collection interval.
      	Executor	User who executes the metric ingestion rule, that is, the user of the selected host group. By default, the executor is root.
      Other	Custom Dimensions	Dimensions (key-value pairs) added to specify additional metric attributes. You can click Add Dimension to add multiple custom dimensions (key-value pairs). Key: key of the additional attribute of a metric. Enter 1 to 64 characters starting with a letter or underscore (_). Only letters, digits, and underscores are allowed. Value: corresponds to the key of the additional attribute of a metric. A maximum of 10 custom dimensions can be added. Example: Set the key to app and value to abc.

      After the parameters are configured, you can click YAML to view the configuration data in YAML format.

5. After the configuration is complete, click Next. The custom Prometheus metrics can then be ingested.
   After ingesting custom Prometheus metrics, you can perform the following operations:

   • Go to the Metric Browsing page to analyze metrics. For details, see Observability Metric Browsing.
   • Go to the Access Management page to view, edit, or delete the configured ingestion rule. For details, see Managing Metric and Log Ingestion.