Help Center/ MapReduce Service/ User Guide/ MRS Cluster O&M/ Common Configuration Files and Log Directories of an MRS Cluster

Updated on 2025-08-09 GMT+08:00

View PDF

Common Configuration Files and Log Directories of an MRS Cluster

This section applies only to MRS 3.x or later.

Cluster Installation Directory

After an MRS cluster is created, the default installation directory of components in the cluster is /opt/Bigdata. Table 1 describes the main directories in the installation directory.

**Table 1** Installation directories
Directory	Description
/opt/Bigdata/common	MRS cluster public resource directory, including the JDK/JRE directory and key storage directory
/opt/Bigdata/components	Directory for storing preset component information
/opt/Bigdata/FusionInsight_Component name_Version number	Installation directory of the registered component in an MRS cluster, for example, /opt/Bigdata/FusionInsight_HD_8.1.2.2
/opt/Bigdata/FusionInsight_Module name_Version number/install/FusionInsight-Component name-XXX	Running directory of the installed component in an MRS cluster, for example, /opt/Bigdata/FusionInsight_HD_8.1.2.2/install/FusionInsight-Hadoop-3.1.1
/opt/Bigdata/FusionInsight_Current	Workspace guidance directory of the installed component in an MRS cluster
/opt/Bigdata/om-agent_Version number	Running directory of the OMA module of MRS Manager
/opt/Bigdata/om-server_Version number	Running directory of the OMS module of MRS Manager
/opt/Bigdata/packaged-distributables	Installation directory of the MRS cluster component installation package
/opt/Bigdata/third_lib/Component name	Directory for storing user-defined JAR packages of components

Component Log Directory

MRS cluster logs are stored in /var/log/Bigdata. For the run log directories of each component, see Table 2.

For details about how to view and download MRS cluster logs, see Viewing Logs of an MRS Cluster.

**Table 2** Log directories
File Directory	Log Content
/var/log/Bigdata/audit	Component audit log.
/var/log/Bigdata/controller	Log collecting script log. Controller process log. Controller monitoring log.
/var/log/Bigdata/dbservice	DBService log.
/var/log/Bigdata/flume	Flume log.
/var/log/Bigdata/hbase	HBase log.
/var/log/Bigdata/hdfs	HDFS log.
/var/log/Bigdata/hive	Hive log.
/var/log/Bigdata/hetuengine	HetuEngine logs.
/var/log/Bigdata/httpd	HTTPD log.
/var/log/Bigdata/hue	Hue log.
/var/log/Bigdata/kerberos	Kerberos log.
/var/log/Bigdata/ldapclient	LDAP client log.
/var/log/Bigdata/ldapserver	LDAP server log.
/var/log/Bigdata/loader	Loader log.
/var/log/Bigdata/logman	logman script log management log.
/var/log/Bigdata/mapreduce	MapReduce log.
/var/log/Bigdata/nodeagent	NodeAgent log.
/var/log/Bigdata/okerberos	OMS Kerberos log.
/var/log/Bigdata/oldapserver	OMS LDAP log.
/var/log/Bigdata/metric_agent	MetricAgent run log.
/var/log/Bigdata/omm	oms: complex event processing log, alarm service log, HA log, authentication and authorization management log, and monitoring service run log of the omm server. oma: installation log and run log of the omm agent. core: dump log generated when the omm agent and the HA process are suspended.
/var/log/Bigdata/spark2x	Spark2x log.
/var/log/Bigdata/sudo	Log generated when the sudo command is executed by user omm.
/var/log/Bigdata/timestamp	Time synchronization management log.
/var/log/Bigdata/tomcat	Tomcat log.
/var/log/Bigdata/watchdog	Watchdog log.
/var/log/Bigdata/yarn	YARN log.
/var/log/Bigdata/zookeeper	ZooKeeper log.
/var/log/Bigdata/oozie	Oozie log.
/var/log/Bigdata/kafka	Kafka log.
/var/log/Bigdata/storm	Storm log.
/var/log/Bigdata/iotdb	IoTDB log.
/var/log/Bigdata/cdl	CDL log.
/var/log/Bigdata/upgrade	OMS upgrade log.
/var/log/Bigdata/update-service	Upgrade service log.
/var/log/Bigdata/patch	Patch log.

Common Configuration Files of the Client

During upper-layer application development and external system interconnection, you need to obtain MRS cluster configuration files using the MRS cluster client. For details about the common client configuration files of each component, see Table 3.

**Table 3** Common client configuration files of MRS components
Component	File Path	File description
Flink	Client installation directory/Flink/flink/conf/flink-conf.yaml	Primary configuration file for Flink, defining global settings such as cluster resource allocation, high availability, state backends, and network parameters. These configurations directly impact the execution environment of all submitted Flink jobs.
HBase	Client installation directory/HBase/hbase/conf/hbase-site.xml	Basic configuration file used to start and manage the HBase cluster. It includes settings such as the data storage path, ZooKeeper connection details, and RegionServer configurations.
HDFS	Client installation directory/HDFS/hadoop/etc/hadoop/hdfs-site.xml	Configuration file for HDFS. It defines critical parameters including data storage settings, redundancy policies, and node behavior.
HDFS	Client installation directory/HDFS/hadoop/etc/hadoop/core-site.xml	Central configuration file for Hadoop. It defines key parameters such as the cluster's communication protocol, default file system, and global security settings. This file is essential and shared across all Hadoop components.
HetuEngine	Client installation directory/HetuEngine/hetuserver/conf/hetuserver-client.properties	Configuration file for HetuEngine client connection parameters.
Hive	Client installation directory/Hive/config/hive-site.xml	Primary configuration file for Hive, defining essential settings for both the client and server.
Kafka	Client installation directory/Kafka/kafka/config/client.properties	Fundamental configuration file for the Kafka client, specifying general connection parameters used to communicate with a Kafka cluster.
	Client installation directory/Kafka/kafka/config/consumer.properties	Kafka consumer configuration file that governs message consumption behavior, including offset management and partition assignment strategies.
	Client installation directory/Kafka/kafka/config/producer.properties	Kafka producer configuration file that governs message sending behavior, including settings for retry policies, serialization formats, and partitioning strategies.
	Client installation directory/Kafka/kafka/config/server.properties	Kafka server configuration file that defines essential cluster settings such as operating mode, network communication, storage policies, and performance tuning parameters.
MapReduce	Client installation directory/Yarn/config/mapred-site.xml	Configuration file for MapReduce that specifies how jobs are executed, how resources are allocated, and how MapReduce integrates with YARN.
Spark	Client installation directory/Spark2x/spark/conf/spark-defaults.conf	Configuration file for setting default parameters in Spark applications. It defines key runtime settings, including integration with cluster managers (such as YARN), resource allocation policies, memory management, network configurations, and performance tuning options.
Yarn	Client installation directory/Yarn/config/yarn-site.xml	Configuration file for YARN that defines key settings for cluster resource management and task scheduling. It includes runtime parameters for core components like ResourceManager, NodeManager, and ApplicationMaster, which influence resource allocation, container management, and application scheduling across the cluster.