Scaling In a Logstash Cluster

Prerequisites

The cluster status is Available, and there are no ongoing tasks.

Impact on Billing

For a pay-per-use cluster, you can see its new price when confirming the scale-in on the console. After the scale-out or scale-up is complete, the new price will apply. For pricing details, see Cloud Search Service Price Calculator.

For a yearly/monthly cluster, a refund request will be triggered after you confirm the scale-in. Then you can check the new price.

Constraints

• To reduce the impact on services, you are advised to perform scale-in during off-peak hours.
• In a cross-AZ cluster, the difference between the numbers of the same-type nodes in different AZs cannot exceed 1.

Change Impact

Before the change, learn about possible impacts and operation suggestions, and develop a plan to minimize these impacts.

• Impact on cluster throughput

  Taking Logstash nodes offline usually does not interrupt services. However, jobs that are running on these will be stopped, causing the cluster throughput to decrease.

• Characteristics of this process

  Once started, a scaling task cannot be stopped until it succeeds or fails.

Scale-in Duration

The following formula can be used to estimate how long a scale-in operation will take:

Scale-in duration (min) = 5 (min) x Number of nodes to be removed

where, 5 minutes is the duration per node. It is an empirical value.

Reducing the Nodes of a Logstash Cluster

1. Log in to the CSS management console.
2. In the navigation pane on the left, choose Clusters > Logstash.
3. In the cluster list, find the target cluster, and choose More > Modify Configuration in the Operation column. The Modify Configuration page is displayed.
4. Select the Scale Cluster tab and click Scale in to set parameters.

   Table 1 Removing nodes randomly

   Parameter	Description
   Action	Select Scale in.
   Resources	Quantities of resources reduced.
   Agency	When a node is deleted, NICs are released. This means you need to have VPC permissions. Select an IAM agency to grant the current account the permission to access and use VPC. This parameter is available only when the new IAM plane is connected. If you are configuring an agency for the first time, click Automatically Create IAM Agency to create css-upgrade-agency. If there is an IAM agency automatically created earlier, you can click One-click authorization to have the permissions associated with the VPC Administrator role or the VPC FullAccess system policy deleted automatically, and have the following custom policies added automatically instead to implement more refined permissions control. "vpc:subnets:get", "vpc:ports:*" To use Automatically Create IAM Agency and One-click authorization, the following minimum permissions are required: "iam:agencies:listAgencies", "iam:roles:listRoles", "iam:agencies:getAgency", "iam:agencies:createAgency", "iam:permissions:listRolesForAgency", "iam:permissions:grantRoleToAgency", "iam:permissions:listRolesForAgencyOnProject", "iam:permissions:revokeRoleFromAgency", "iam:roles:createRole" To use an IAM agency, the following minimum permissions are required: "iam:agencies:listAgencies", "iam:agencies:getAgency", "iam:permissions:listRolesForAgencyOnProject", "iam:permissions:listRolesForAgency"
   Nodes	Reduce the number of nodes in the Nodes column.

   Figure 1 Reducing nodes
   
5. Click Next.
6. Confirm the information and click Submit.
7. Click Back to Cluster List to go back to the Clusters page. Task Status is Scaling in. When Cluster Status changes to Available, the cluster has been successfully scaled in.

Related Documents

To optimize costs, you may also reduce the capacity of a Logstash cluster by downgrading node specifications. For details, see Changing the Node Specifications of a Logstash Cluster.