A Large Number of Jobs Occupying Resources After Yarn Is Started in a Cluster

Symptom

In an MRS 2.x cluster or earlier, a large number of jobs are generated after Yarn is started, occupying computing resources of the cluster.

Cause Analysis

If the source IP address of the Any protocol in the inbound direction of the cluster's security group is set to 0.0.0.0/0, the cluster may be attacked from external networks.

Procedure

1. Log in to the MRS console. On the Active Clusters page, click the cluster name to access the cluster details page.
2. Click Access Manager next to MRS Manager. The Access MRS Manager page is displayed.
3. Click Manage Security Group Rule to check the security group rule configuration.

   

4. Check whether the source address of the Any protocol in the inbound direction is 0.0.0.0/0.

   If it is, change the remote end of the Any protocol in the inbound direction to a specified IP address. If it is not, no change is required.

5. After the value is changed successfully, restart the cluster.

Summary and Suggestions

Disable the Any protocol in the inbound direction, or set the remote end of the Any protocol in the inbound direction to a specified IP address.