Test Report Description

CodeArts PerfTest provides comprehensive pressure test reports that cover multiple dimensions, including over 20 performance metrics such as TPS, RT, SuccessRate, TPxx, StatusCode, and execution logs. By analyzing real-time access resources and call chain relationships through visualized data, CodeArts PerfTest provides a thorough evaluation of performance metrics and compares multiple rounds of reports. This enables quick and accurate identification of performance bottlenecks, ensuring smooth service rollout.

CodeArts PerfTest provides real-time and offline test reports for you to view and analyze test data anytime.

Report Overview

For details about the test report, see Table 1.

This report shows the response performance of the tested system in a scenario with a large number of concurrent users. To understand the report, see the following information:

Statistical dimension: In this report, RPS, response time, and concurrency are measured in a single case. If a request has multiple packets, the request is considered successful only when all the packets are responded to. The response time of the request is the sum of the response time of the packets.
Response timeout: If the corresponding TCP connection does not return the response data within the set response timeout (5s by default), the case request is counted as a response timeout. Possible causes include: the tested server is busy or crashes, or the network bandwidth is fully occupied.
Verification failure: The response packet content and response code returned from the server do not meet the expectation (the default expected response code of HTTP/HTTPS is 200), such as code 404 or 502. A possible cause is that the tested service cannot be processed normally in scenarios with a large number of concurrent users. For example, a database bottleneck exists in the distributed system or the backend application returns an error.
Parsing failure: All response packets are received, but some packets are lost. As a result, the entire case response is incomplete. This may be caused by network packet loss.
Bandwidth: This report collects statistics on the bandwidth of the execution end of CodeArts PerfTest. The uplink indicates the traffic sent from CodeArts PerfTest, and the downlink indicates the received traffic. In the external pressure test scenario, you need to check whether the EIP bandwidth of the executor meets the uplink bandwidth requirement and whether the bandwidth exceeds the downlink bandwidth of 1 GB.
Requests per second (RPS): indicates the number of requests per second. Average RPS = Total number of requests in a statistical period/Statistical period.
SLA Result: If you have configured SLA rules for a test case, you can view the SLA report to check whether the SLA rules are triggered during testing. In the SLA report, Request Name is the name of the request in the test case. Rules indicates the configured SLA rules. Average Value (displayed as the current value for a running task) indicates the average value of service metrics and the total number of times that SLA rules have been triggered since the pressure test starts. Trigger Events displays the request name and when and how many times the SLA rules were triggered during testing.
How to determine the quality of tested applications: According to the service quality definition of an application, the optimal status is that there is no response or verification failure. If there is any response or verification failure, it must be within the defined service quality range. Generally, the value does not exceed 1%. The shorter the response time is, the better. User experience is considered good if the response time is within 2s, acceptable if it is within 5s and needs optimization when it is over 5s. TP90 and TP99 can objectively reflect the response time experienced by 90% to 99% of users.

**Table 1** Test report description
Parameter	Description
Total Number of Metrics	Total number of metrics of all cases. Maximum Concurrency: indicates the maximum number of concurrent virtual users. RPS: requests per second. Response Time: duration from the time when a client sends a request to the time when the client receives a response from the server. Response Code: distribution of response codes during the running of a pressure test task. Bandwidth: records the real-time bandwidth usage during the running of a pressure test task. SLA Events: occurrence of an event defined in the SLA. Normal Response: indicates the number of case responses that pass the set checkpoints. If no checkpoints are set, the number of case responses with code 2XX will be calculated by default. Abnormal Response: indicates the number of parsing failures, verification failures, response timeout, 3XX, 4XX, 5XX, and connection failures. Success Rate: Number of requests that are normally returned/Total number of requests.
Average RPS	Total number of requests in a statistical period/Statistical period
Average RT	Average response time of all requests sent in a second
Concurrent Users	Changes on the number of concurrent virtual users during testing
Bandwidth (kbit/s)	Records the real-time bandwidth usage during the running of a pressure test task. Uplink bandwidth: speed at which the CodeArts PerfTest execution node sends out data. Downlink bandwidth: speed at which the CodeArts PerfTest execution node receives data
Response distribution statistics	Number of cases processed per second for normal return, parsing failure, verification failure, response timeout, connection rejection, and other errors. This metric is related to the think time, concurrent users, and server response capability. For example, if the think time is 500 ms and the response time of the last request for the current user is less than 500 ms, the user requests twice per second. Normal Response: indicates the number of case responses that pass the set checkpoints. If no checkpoints are set, the number of case responses with code 2XX will be calculated by default. Abnormal Response: indicates the number of parsing failures, verification failures, response timeout, 3XX, 4XX, 5XX, and connection failures. Parsing Failure: indicates the number of HTTP responses that fail to be parsed. Verification Failure: indicates the number of case responses that do not pass the set checkpoints. If no checkpoints are set, the number of case responses returning contents other than 2XX will be calculated by default. Response Timeout: indicates the number of case requests that do not receive responses from the server within 5 seconds after a request packet is sent. Rejected: indicates the number of rejected connection requests. Others: indicates the number of other errors.
Response Code statistics	1XX/2XX/3XX/4XX/5XX
Response Time Ratio	Ratio of response time of cases.
Maximum Response Time of TP (ms)	If you want to calculate the top percentile XX (TPXX) for a request, collect all the response time values for the request over a time period (such as 10s) and sort them in an ascending order. Remove the top (100–XX)% from the list, and the highest value left is the value of TPXX. TP50: Remove the top 50% from the list, and the highest value left is the value of TP50. TP75: Remove the top 25% from the list, and the highest value left is the value of TP75. TP90: Remove the top 10% from the list, and the highest value left is the value of TP90. TP95: Remove the top 5% from the list, and the highest value left is the value of TP95. TP99: Remove the top 1% from the list, and the highest value left is the value of TP99. TP99.9: Remove the top 0.1% from the list, and the highest value left is the value of TP99.9. TP99.99: Remove the top 0.01% from the list, and the highest value left is the value of TP99.99.