Scaling In a Logstash Cluster
If a cluster can process current traffic without fully utilizing its resources, you can scale in the cluster to cut costs.
Scaling in a Logstash cluster means to randomly remove some of its nodes, so that it is less costly to keep it running.
Scale-in process: Random remove cluster nodes and modify the cluster configuration.
Prerequisites
The cluster status is Available, and there are no ongoing tasks.
Impact on Billing
For a pay-per-use cluster, you can see its new price when confirming the scale-in on the console. After the scale-out or scale-up is complete, the new price will apply. For pricing details, see Cloud Search Service Price Calculator.
For a yearly/monthly cluster, a refund request will be triggered after you confirm the scale-in. Then you can check the new price.
Constraints
- To reduce the impact on services, you are advised to perform scale-in during off-peak hours.
- In a cross-AZ cluster, the difference between the numbers of the same-type nodes in different AZs cannot exceed 1.
Change Impact
Before the change, learn about possible impacts and operation suggestions, and develop a plan to minimize these impacts.
- Impact on cluster throughput
Taking Logstash nodes offline usually does not interrupt services. However, jobs that are running on these will be stopped, causing the cluster throughput to decrease.
- Characteristics of this process
Once started, a scaling task cannot be stopped until it succeeds or fails.
Scale-in Duration
The following formula can be used to estimate how long a scale-in operation will take:
Scale-in duration (min) = 5 (min) x Number of nodes to be removed
where, 5 minutes is the duration per node. It is an empirical value.
Reducing the Nodes of a Logstash Cluster
- Log in to the CSS management console.
- In the navigation pane on the left, choose Clusters > Logstash.
- In the cluster list, find the target cluster, and choose More > Modify Configuration in the Operation column. The Modify Configuration page is displayed.
- Select the Scale Cluster tab and click Scale in to set parameters.
Table 1 Removing nodes randomly Parameter
Description
Action
Select Scale in.
Resources
Quantities of resources reduced.
Agency
When a node is deleted, NICs are released. This means you need to have VPC permissions. Select an IAM agency to grant the current account the permission to access and use VPC.
- This parameter is available only when the new IAM plane is connected.
- If you are configuring an agency for the first time, click Automatically Create IAM Agency to create css-upgrade-agency.
- If there is an IAM agency automatically created earlier, you can click One-click authorization to have the permissions associated with the VPC Administrator role or the VPC FullAccess system policy deleted automatically, and have the following custom policies added automatically instead to implement more refined permissions control.
"vpc:subnets:get", "vpc:ports:*"
- To use Automatically Create IAM Agency and One-click authorization, the following minimum permissions are required:
"iam:agencies:listAgencies", "iam:roles:listRoles", "iam:agencies:getAgency", "iam:agencies:createAgency", "iam:permissions:listRolesForAgency", "iam:permissions:grantRoleToAgency", "iam:permissions:listRolesForAgencyOnProject", "iam:permissions:revokeRoleFromAgency", "iam:roles:createRole"
- To use an IAM agency, the following minimum permissions are required:
"iam:agencies:listAgencies", "iam:agencies:getAgency", "iam:permissions:listRolesForAgencyOnProject", "iam:permissions:listRolesForAgency"
Nodes
Reduce the number of nodes in the Nodes column.
Figure 1 Reducing nodes - Click Next.
- Confirm the information and click Submit.
- Click Back to Cluster List to go back to the Clusters page. Task Status is Scaling in. When Cluster Status changes to Available, the cluster has been successfully scaled in.
Related Documents
To optimize costs, you may also reduce the capacity of a Logstash cluster by downgrading node specifications. For details, see Changing the Node Specifications of a Logstash Cluster.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot