Deployment

Big Data Platform Deployment

You can deploy the big data platform as follows:

• Big data cluster deployment

  Following the established architecture design principles, big data clusters in the cloud environment are typically provisioned utilizing cloud-based services. Huawei Cloud MapReduce Service (MRS) offers a streamlined solution for deploying and managing Hadoop ecosystems on Huawei Cloud. This service enables the one-click deployment of fully controllable, enterprise-grade big data clusters. Within these clusters, users can readily operate key big data components such as Hadoop, Spark, HBase, and Kafka.

  For details, see the MRS official website.

• Big data task scheduling platform deployment

  If the target architecture incorporates Huawei Cloud DataArts Studio for task scheduling, the platform can be deployed and configured according to the guidelines provided on its official website.

  For target architectures employing a self-built big data task scheduling platform, the scheduling software can be deployed on Huawei Cloud ECSs. Alternatively, Huawei Cloud SMS can be utilized to migrate an existing on-premises scheduling platform to a Huawei Cloud ECS instance.

• Big data application deployment

  Big data applications can be deployed on Huawei Cloud ECSs. Alternatively, Huawei Cloud SMS can be employed to migrate existing big data applications to Huawei Cloud ECS instances.

Platform Permission Configuration

• Platform permission configuration

  Following the deployment of the target big data platform, ensuring accurate and consistent permission settings is crucial. To achieve this, you can reference the permission configurations of the source platform and implement the following steps:

• Reviewing source permission settings

  Conduct a thorough review of the source platform's permission settings. This includes a detailed examination of users, roles, organizational structures, and associated permission levels. The objective is to fully understand the scope of permissions and access rights granted to each user, enabling the accurate replication of these settings on the target platform.

• Creating users and roles

  Based on the reviewed source platform permissions, create corresponding users and roles on the target platform. It is imperative to ensure that user identities and role assignments on the target platform precisely mirror those on the source platform.

• Adjusting permission levels and scopes

  On the target platform, meticulously adjust permission levels and scopes to align with the configurations identified on the source platform. The goal is to establish identical permission settings on the target platform, guaranteeing that users can only access the resources they are authorized to view and interact with.

• Assigning and inheriting permissions

  Implement permission assignment and inheritance mechanisms on the target platform based on the source platform's permission structure. This ensures that users inherit the same permissions and retain their corresponding roles and permission settings in the new environment.

• Reviewing and adjusting access control

  Review the access control mechanisms implemented on the target platform and make necessary adjustments to align with the source platform's permission settings. The objective is to ensure that access controls effectively restrict user access to appropriate resources, adhering to the permission rules established on the source platform.

• Implementing security audit and monitoring

  Establish comprehensive security audit and monitoring mechanisms to ensure the ongoing and effective oversight of permission settings on the target platform. This proactive approach facilitates the detection and prevention of unauthorized access attempts, enabling timely and appropriate incident response.