Updated on 2022-08-12 GMT+08:00

Resource Overview

The resources of the MRS big data platform are divided into computing resources and storage resources. Multi-tenant enables resource isolation:

  • Computing Resource

    Computing resources include CPUs and memory. One tenant cannot occupy the computing resources of another tenant.

  • Storage Resource

    Storage resources include disks and third-party storage systems. One tenant cannot access the data of another tenant.

Computing Resource

Computing resources are divided into static service resources and dynamic resources.

  • Static service resources

    Static service resources are computing resources allocated to each service. The total volume of computing resources allocated to each service is fixed. Such services include Flume, HBase, HDFS and Yarn.

  • Dynamic resources

    Dynamic resources are computing resources dynamically scheduled to a task queue by distributed resource management service Yarn. Yarn dynamically schedules resources for the task queues of Mapreduce, Spark2x, Flink, and Hive.

The resources allocated to Yarn in a big data cluster are static service resources and can be dynamically allocated to task queues by Yarn.

Storage Resource

Storage resources are data storage resources that can be allocated by distributed file storage service HDFS. Directories are used as the basic unit of HDFS storage resource allocation. Tenants can obtain storage resources by specifying directories in the HDFS file system.