Metadata Basic Principles

Introduction to Metadata

Metadata Management (MDM) provides metadata extraction capabilities for MRS data warehouse components (Hive and HBase), and allows users to label each metadata for data analysis, search, and other extended functions.

Metadata Principles

MDM extracts metadata from Hive and HBase in the MRS system and dumps the metadata. By using the MRS framework installation process, MDM obtains the Hive and HBase connection mode and valid access authentication, and finally obtains the metadata from the Hive and HBase databases.

Figure 1 Logical architecture of MDM

The MDM working principle is as follows:

1. MDM obtains the Hadoop cluster basic information from Manager. The basic information includes HBase RegionServer node deployment information and information about DBService that saves Hive metadata.
2. According to the information obtained by 1, MDM extracts metadata from Hive and HBase and saves the metadata in DBService. You can log in to the FusionInsight Manager system from a client and view the metadata.
3. Upload the extracted metadata to a third-party metadata management system by using an external FTP server. The uploaded metadata can be used to support higher-level metadata management.