Help Center/ MapReduce Service/ User Guide/ MRS Cluster O&M/ Viewing MRS Cluster Monitoring Metrics/ Dumping MRS Cluster Monitoring Data
Updated on 2026-10-21 GMT+08:00
View PDF

Dumping MRS Cluster Monitoring Data

During MRS cluster O&M, administrators can export cluster monitoring metrics to an external system for long-term archiving and secondary analysis. By default, MRS stores monitoring data in a local database. Over time, accumulated data occupies significant disk space, and the system cannot directly connect with third-party monitoring systems. Configuring monitoring data dumping allows the system to write collected metric data into text files and periodically upload them to an external server via FTP or SFTP. This enables long-term data archiving and seamless integration with third-party systems. This section describes how to configure monitoring data dumping for an MRS cluster.

  • Monitoring data dumping: Writes cluster monitoring data collected by MRS Manager into text files at specified intervals and uploads them to an external server via FTP or SFTP. After dumping, third-party systems can parse monitoring data from these files for secondary processing.
  • Collection Period: The interval at which MRS Manager collects monitoring data. Supported options are 30s, 60s, and 300s. Metric data for different collection periods is saved in separate monitoring files.
  • Metric ID: The unique identifier for each monitoring metric. IDs are categorized into real-time metric IDs (for 30s or 60s collection periods) and 5-minute metric IDs (for the 300s collection period). Third-party systems use this ID to query metric details.

Manager supports the collection of all the monitoring data in the managed clusters. The collection period is 30 seconds, 60 seconds, or 300 seconds. The monitoring data is stored in different monitoring files on the FTP server by collection period.

Prerequisites

  • The ECS corresponding to the dump server must be in the same VPC as the Master node of the MRS cluster, and the Master node can access the IP address and a specified port of the dump server.
  • The FTP service on the dump server is running properly.

Dumping Monitoring Data of an MRS 3.x Cluster

  1. Log in to FusionInsight Manager of the MRS cluster.

    For details about how to log in to FusionInsight Manager, see Accessing MRS Manager.

  2. Choose System > Interconnection > Upload Performance Data.
  3. Toggle on Upload Performance Data.
  4. Set the upload parameters according to Table 1.

    Table 1 Upload parameters

    Parameter

    Description

    FTP IP Address Mode

    Specifies the server IP address mode. This parameter is mandatory. The value can be IPV4 or IPV6.

    FTP IP Address

    Specifies the IP address of the FTP server for storing monitoring files after the monitoring metric data is interconnected. This parameter is mandatory.

    FTP Port

    Specifies the port for connecting to the FTP server. This parameter is mandatory.

    FTP Username

    Specifies the username for logging in to the FTP server. This parameter is mandatory.

    FTP Password

    Specifies the password for logging in to the FTP server. This parameter is mandatory.

    Save Path

    Specifies the path for storing monitoring files on the FTP server. This parameter is mandatory. The path must be valid.

    Dump Interval (second)

    Specifies the interval at which monitoring files are periodically stored on the FTP server, in seconds. This parameter is mandatory. The system will periodically upload files to the corresponding FTP server at the specified interval.

    Dump Mode

    Specifies the protocol used for sending monitoring files. This parameter is mandatory. The value can be SFTP or FTP. You are advised to use the SFTP mode based on SSH v2. Otherwise, security risks may be incurred.

    SFTP Service Public Key

    Specifies the public key of the FTP server. This parameter is optional. It is valid only when Dump Mode is set to SFTP.

    You are advised to configure the SFTP service public key to prevent data security risks.

  5. Click OK.

    After the monitoring metric data dump is configured, the system uploads the files to the corresponding server directory at the specified interval.

    Table 2 Monitoring data files

    Monitoring Data

    Description

    Collection period
    • 30s: real-time metrics that are collected every 30s by default
    • 60s: real-time metrics that are collected every 60s by default
    • 300s: all metrics that are not collected every 30s or 60s

    File name format
    • MRS 3.x and later:

      metric_Monitoring data collection period_File creation time.log

      Examples: metric_60_20160908085915.log and metric_300_20160908085613.log

    • Versions earlier than MRS 3.x:

      Cluster name_metric_Monitoring data collection period_File creation time.log

    Content format (MRS 3.x as an example)
    • Cluster ID|Cluster name|Display name|Service name|Metric ID|Collection time|Collection host@m@Sub-metric|Unit|Metric value
    • Example: 
      1|xx1|Host|Host|10000413|2019/06/18 10:05:00|10-66-254-146|KB/s|309.910
1|xx1|Host|Host|10000413|2019/06/18 10:05:00|10-66-254-152|KB/s|72.870
2|xx2|Host|Host|10000413|2019/06/18 10:05:00|10-66-254-163|KB/s|100.650

    After obtaining the Metric ID from the exported file, third-party systems can retrieve metric details by querying the metric set file located at Cluster installation path/om-server/om/etc/om/all-shown-metric-en_US on the active or standby OMS node. This file contains detailed information for all metrics, as shown in the following example (using MRS 3.x):

    Real-Time Metric ID,5-Minute Metric ID,Metric Name,Metric Collection Period (s),Collected by Default,Service Belonged To,Role Belonged To
00101,10000101,JobHistoryServer non-heap memory usage,30,false,Mapreduce,JobHistoryServer
00102,10000102,JobHistoryServer non-heap memory allocation volume,30,false,Mapreduce,JobHistoryServer
00103,10000103,JobHistoryServer heap memory usage,30,false,Mapreduce,JobHistoryServer
00104,10000104,JobHistoryServer heap memory allocation volume,30,false,Mapreduce,JobHistoryServer
00105,10000105,Number of blocked threads,30,false,Mapreduce,JobHistoryServer
00106,10000106,Number of running threads,30,false,Mapreduce,JobHistoryServer
00107,10000107,GC time,30,false,Mapreduce,JobHistoryServer
00110,10000110,JobHistoryServer CPU usage,30,false,Mapreduce,JobHistoryServer
...
    • Metric ID meaning:

      For metrics whose collection period is 30s/60s, you can find the corresponding metric description by referring to the first column, that is, Real-Time Metric ID.

      For metrics whose collection period is 300s, you can find the corresponding metric description by referring to the second column, that is, 5-Minute Metric ID.

    • Field description:

      Real-Time Metric ID: indicates the ID of the metric whose collection period is 30s or 60s.

      5-Minute Metric ID: indicates the ID of a 5-minute (300s) metric.

      Metric Collection Period (s): indicates the collection period of real-time metrics. The value can be 30 or 60.

      Service Belonged To: indicates the name of the service to which a metric belongs, for example, HDFS and HBase.

      Role Belonged To: indicates the name of the role to which a metric belongs, for example, JobServer and RegionServer.

Verification:

After completing the configuration, wait for one dump interval, log in to the target FTP or SFTP server, and check the specified save directory for a metric data file matching the metric_*.log pattern. If the file exists and contains metric data, the dump configuration is successful.

Dumping MRS Cluster Monitoring Data (MRS 2.x and Earlier)

  1. Log in to FusionInsight Manager of the MRS cluster.

    For details about how to log in to FusionInsight Manager, see Accessing MRS Manager.

  2. Click System.
  3. In Configuration, click Configure Monitoring Metric Dump under Monitoring and Alarm.
  4. Table 3 describes the parameters for dumping.

    Table 3 Parameters

    Parameter

    Description

    Mandatory

    Dump Monitoring Metric

    Whether to enable monitoring metric data interconnection.

    Yes

    FTP IP Address

    FTP server for storing monitoring files after monitoring metric data is interconnected.

    Yes

    FTP Port

    Port for connecting to the FTP server.

    Yes

    FTP Username

    Username for logging in to the FTP server.

    Yes

    FTP Password

    Password for logging in to the FTP server.

    Yes

    Save Path

    Path for storing monitoring files on the FTP server.

    Yes

    Dump Interval (second)

    Interval at which monitoring files are periodically stored on the FTP server, in seconds. The system will periodically upload files to the corresponding FTP server at the specified interval.

    Yes

    Dump Mode

    Protocol used for sending monitoring files. The options are FTP and SFTP.

    Yes

    SFTP Service Public Key

    Public key of the FTP server. This parameter is available only when Dump Mode is set to SFTP. You are advised to configure a public key. Otherwise, security risks may arise.

    No

  5. Click OK.

Verification:

After completing the configuration, wait for one dump interval, log in to the target FTP server, and verify that a metric log file is generated in the specified directory. If the file exists and is not empty, the configuration dump was successful.

FAQs

  • Q: What should I do if the monitoring data dump fails?

    Common causes of dump failures: The FTP or SFTP server is unreachable, or the port is blocked. The FTP username or password is incorrect. The specified save path does not exist, or the user lacks write permissions. The configured dump protocol does not match the server configuration (for example, selecting SFTP when the server supports only FTP).

    Verify network connectivity between the Master node and the dump server, ensuring the IP address and port number are correct. Confirm that the FTP username and password are valid, the save directory exists, and the user account has write permissions. Additionally, ensure that the dump protocol matches the server configuration. After verifying these settings, wait for the next dump interval for the system to automatically retry.

  • Q: What should I do if the metric data in a dump file is empty?

    Possible causes: The cluster was recently created and insufficient monitoring data has been collected, or high disk usage on the MRS Manager storage caused monitoring data to be cleared.

    Ensure that the cluster has run long enough to collect monitoring data. Check the storage disk usage and delete unnecessary files if space is low. Afterward, wait for the next dump period and verify the file content again.

Helpful Links

Parent Topic: Viewing MRS Cluster Monitoring Metrics