Function Overview

LakeFormation lets you manage data lake metadata with a GUI and APIs. It works with Hive and Ranger metadata and permission models, and connects to various compute engines and cloud services for big data. LakeFormation helps you to easily and efficiently build data lakes and operation services, accelerating the release of service data value.

LakeFormation is a serverless service that uses underlying resources to implement cross-AZ deployment, high reliability, auto scaling, unified metadata management, association between metadata and file directories, and interconnection with multiple compute engines.

Before using LakeFormation, create an instance. Then, you can perform operations on the instance, such as managing metadata and setting metadata permissions.

LakeFormation provides basic functions such as instance creation, overview, and deletion, helping you easily manage instances and accelerate the planning and deployment of services carried by the data lake.

• Catalog

A top-level resource in the metadata resources of a LakeFormation instance and multiple catalogs can be created in a LakeFormation instance. Metadata information such as name, description, and location are included in catalogs. Catalogs can be created, modified, and deleted.Location indicates the file directory of the OBS parallel file system mapped to the catalog

• Database

Databases are stored in the data catalogs of a LakeFormation instance and multiple databases can be created under a catalog. Metadata information such as name, description, and location are included in databases. You can create, modify, and delete databases, as well as grant and check databases permissions.
Location indicates the file directory of the OBS parallel file system mapped to the databases.

• Table

You can create multiple tables in a database. Metadata such as basic information, format and serialization information, fields, and attributes are included in tables. You can create, modify, and delete tables, as well as grant and check permissions.

• Function

Functions are used to perform specific processing on data in SQL queries, including built-in functions and user-defined functions (UDFs).

• Partition
  

Partitioning is to split a data table by row to reduce the total amount of data read and write operations in specific SQL operations, and therefore shortening the response time.

LakeFormation allows you to authorize, cancel, and view data resources such as catalogs, databases, and tables. It helps you implement convenient and unified data permission management for the data lake.

LakeFormation supports full or incremental migration of metadata and permissions from external services to the current LakeFormation instance for unified management

On the Clients page, you can quickly create and manage access clients. You can also view information such as access IP address, which is required for connect multiple services to LakeFormation instances.