Performance Automation Test Cases

Authorizing Performance Automation Delegation

You need to authorize HaydnCSF to use CodeArts PerfTest to manage test resources for automated performance tests. HaydnCSF will display a dialog box when you use performance automation. You only need to click Authorize in the dialog box. You do not need to manually create an agency.

Figure 1 CodeArts PerfTest delegation

What can I do if authorization fails?

If a message is displayed indicating that the authorization fails due to lack of permissions, check whether your account has the authorization permission. (By default, IAM users do not have the permission and cannot perform authorization.) There are two ways to handle this issue:

• Method 1: Send the link in the dialog box to a user who has permissions under your account for authorization.
• Method 2: Contact the administrator of your account to grant permissions.

1. Log in to the IAM console using your Huawei Cloud account, HUAWEI ID, or an account with administrator permissions.
2. Create a user group with a custom name for easy identification and add IAM users to the user group.
3. On the User Groups page, select the user group you created, and click Authorize in the Operation column.
   Figure 2 Authorizing a user
   
4. Search for and select Security Administrator.
   Figure 3 Granting the Security Administrator permission to a user
   
5. Set Scope to All resources.
   Figure 4 Selecting a scope
   
6. Click OK.

Adding the Performance Automation Permission

To use HaydnCSF performance automation, you must have the required permission.

By default, the Huawei Cloud account or HUAWEI ID has all permissions, but IAM users do not have any permissions. If you use an IAM user, contact your account administrator to grant you the permission to use performance automation.

1. Log in to the IAM console using your Huawei Cloud account, HUAWEI ID, or an account with administrator permissions.
2. Create a user group with a custom name for easy identification, for example, CodeArts PerfTest Permissions, or use an existing user group, and add IAM users to which the permissions need to be granted to the user group.
3. On the User Groups page, select the user group you created, and click Authorize in the Operation column.
4. Search for and select ServiceStage Administrator.
   Figure 5 Granting permissions to a user group
   
5. Set Scope to All resources.
6. Click OK.

Creating a Performance Automation Test Case

Test case designers can create performance automation test cases.

1. On the Case Management page, click the Performance Automation tab. Select a performance test requirement and click Create.
   Figure 6 Creating a performance automation test case
   
2. Fill in the required information. If you need to manually supplement the execution content, enable Manual Test Procedure and enter the steps and expected results. Click OK.

   Table 3 Parameters for creating a performance automation test case

   Parameter	Description
   Requirement Description	(Mandatory) Select the requirement that the test case belongs to from the drop-down list.
   Case Name	(Mandatory) Enter a name for the test case. It is recommended that you standardize the test case name so that the function can be reflected in the test case name.
   Deployment Environments	(Mandatory) Select the environment for deploying the test case from the drop-down list based on the associated requirement.
   Priority	(Mandatory) Select the importance level of the test case from the drop-down list.
   Description	Describe the test case.
   Prerequisites	Enter the prerequisites for executing the test case.
   Expected Result	Enter the expected result of each step.
   Attachments	Upload the associated files required by the test case.

3. In the displayed dialog box, click OK to compile the script.
   Figure 7 Going to CodeArts PerfTest for compiling a script
   
4. Configure the test case script.
   • Packet: indicates the data sent between applications. For details, see the CodeArts PerfTest User Guide.
   • Think Time: indicates the waiting time between operations. For details, see the CodeArts PerfTest User Guide.
   • Response Extraction: The output of the previous packet is extracted as the input of the next packet. For details, see the CodeArts PerfTest User Guide.
   • Checkpoint: Check whether the content returned by the server is correct. For details, see the CodeArts PerfTest User Guide.
   Figure 8 Script compilation (1)
   
5. Edit the pressure configuration and save it. For details, see the CodeArts PerfTest User Guide.
   Figure 9 Script compilation (2)
   

Adding Indicator Parameters

Indicator parameters, such as the latency, error rate, and RPS, are used to measure whether the automatic performance tests are passed. You need to add indicator parameters before executing performance automation test cases.

1. On the Performance Automation tab page, click View Indicator Parameters.
2. In the displayed dialog box, click Add Indicator Parameter, enter the parameter name and description.
3. Select an indicator parameter and add an indicator for the parameter, for example, Success Rate.
   If there are multiple indicator parameters, these marked with a pin are default indicator parameters.

   Figure 10 Parameters required for adding indicator parameters for performance automation
   

Executing a Performance Automation Test Case

After performance automation test cases are designed, the test case executor can execute the performance automation test cases. Locate the test case and choose More > Execute in the Operation column.

Prerequisites: You have purchased the PerfTest package so that the performance automation project can be started.

Figure 11 Executing a performance automation test case

Click the link in the preceding figure to buy a PerfTest package. If the service is deployed in a region other than Beijing 4, the test is not affected as long as the service can be accessed.

Procedure

• Method 1: Use the default parameters.

  Click Execute. By default, the selected indicator parameters are used for execution.

  Figure 12 Executing a performance automation test cases using default parameters
  
• Method 2: Specify an indicator parameter group. Locate the test case and choose More > Execute Environment Parameters by Group. Select the indicator parameters and click OK.
  Figure 13 Executing a performance automation test case using an indicator parameter group
  
  Figure 14 Selecting an indicator parameter group
  

• To use CodeArts PerfTest for performance testing, you need to purchase a pay-per-use or yearly/monthly package PerfTest package in advance.
• If you have executed performance automation test cases on CodeArts PerfTest, you can synchronize the execution records from CodeArts PerfTest to HaydnCSF.

Viewing Execution Results

1. Click the test case name to go to the test case details page.
2. Click the Execution History to view the execution results.
3. Click the execution report link to obtain the real-time execution results. (The execution page is refreshed in real time.)
   Figure 15 Execution results