What Is a Database, Data Warehouse, Data Lake, and Huawei FusionInsight Intelligent Data Lake? What Are the Differences and Relationships Between Them?

Database

A database is a warehouse where data is organized, stored, and managed based on the data structure.

In a broad sense, databases have been used in computers since the 1960s. However, hierarchical and network models were prevailing at that time, and they lacked structural independency between data and applications. As a result, database application was limited.

Nowadays, a database usually refers to a relational database. A relational database organizes data based on relational models and stores data in a set of tables with columns and rows. Therefore, data is well-structured and independent with low redundancy. In 1970, relational databases were born to completely separate data from applications in software and since then have become an indispensable part of mainstream computer systems. Relational databases have become one of the most important database products. Almost all new database products from database vendors support relational databases. Some non-relational database products also have APIs that support relational databases.

Relational databases process basic and routine transactions using online transaction processing (OLTP), such as bank transactions.

Data Warehouse

The large-scale application of databases has facilitated the exponential growth of data. Online analytical processing (OLAP) is desired more than ever to explore the relationship between data and unveil the untapped data value. However, it is difficult to share data between different databases, and data integration and analysis also face great challenges.

To overcome these challenges, Bill Inmon, the father of the data warehouse, proposed the idea of data warehousing in 1990. The data warehouse runs on a unique data storage architecture to perform OLAP on a large amount of data accumulated over the years. In this way, enterprises can obtain valuable information from massive data quickly and effectively to make informed decisions. The appearance of data warehouses has prompted the information industry to develop from operational systems based on relational databases to decision support systems.

Unlike a database, a data warehouse has the following features:

• It is theme-oriented. It supports various services and operational data. Therefore, the required data needs to be extracted from multiple heterogeneous data sources, processed and integrated, and reorganized by the theme.
• A data warehouse mainly supports enterprise decision analysis and operations involved are mainly data queries. Therefore, it improves the query speed and cuts down the total cost of ownership (TCO) by optimizing the table structures and storage modes.

Data Lake

Data is an important asset for enterprises. As production and operations data piles up, enterprises hope to save the data for effective management and centralized governance and explore data values.

The data lake provides a good answer to these requirements. It is a large data warehouse that centrally stores structured and unstructured data. It can store raw data from multiple sources and of multiple types. The data can be accessed, processed, analyzed, and transmitted without being structured. The data lake helps enterprises quickly complete federated analysis of heterogeneous data sources and explore data value.

With the development of big data and AI, the value of data in the data lake is increasing gradually. The data lake enables enterprises to build more optimized operation models by realizing centralized data management. It also helps enterprise with prediction analysis and recommendation models, stimulating further growth of enterprise capabilities.

The difference between a data warehouse and a data lake is analogous to that between a warehouse and a lake: A warehouse stores goods from a specific source while the water (raw data) in a lake comes from rivers, streams and other sources.

Huawei FusionInsight Intelligent Data Lake

Huawei's DAYU is a data enablement solution that helps large government agencies and enterprises customize their own intelligent data resource management solutions. This solution can import all-domain data into the data lake, eliminating data silos, unleashing the value of data, and empowering data-driven digital transformation.

DAYU, with the FusionInsight Intelligent Data Lake as the core, contains computing engines such as the database, data warehouse, and data lake, as well as platforms such as DataArts Studio. DAYU provides comprehensive data enablement capabilities, covering data collection, aggregation, computing, asset management, and data openness.

The FusionInsight Intelligent Data Lake solution provides the following services:

• Database
  • Relational database: RDS, TaurusDB, GaussDB, and PostgreSQL.
  • Non-relational database: Document Database Service (DDS) and GeminiDB (compatible with Influx, Redis, Mongo, and Cassandra protocols)
• Data warehouse: GaussDB (DWS)
• Data lake: MapReduce Service (MRS) and Data Lake Insight (DLI).
• Data governance platform: DataArts Studio