Evaluating Test Quality
Custom test reports let you flexibly evaluate test quality across requirement and defect completion, test case execution, and risk assessment. You can download reports in multiple formats to efficiently share test progress.
Notes and Constraints
Up to 15 attachments, max. 10 MB each, can be uploaded to Custom Information of a test report.
Creating a Test Report
On the Quality Report Page
- In the navigation pane, choose .
- In the upper part of the page, select the target version and test plan from the drop-down lists. (Test case library and Baseline are selected by default.)
- Click Generate Report.
- In the displayed dialog box, enter the report name (default format: Project name_Selected version name_Selected test plan name_Random code generated by the system), select a version (defaults to the version selected on the quality report page), test plan (defaults to the plan selected on the quality report page. You can select multiple test plans), test conclusion, and reviewer.
- Click OK.
- After the creation is complete, choose to view the created test report.
On the Quality Assessment Page
- Log in to the CodeArts homepage, search for your target project, and enter the project.
- In the navigation pane, choose Testing > Quality Assessment.
- Click Generate Report in the upper left corner of the page. Enter a report name, select a version, test plan (or multiple test plans), and reviewer, enter a test conclusion and description, and click OK.
Overview
Click the report name. The Overview page of the test report is displayed by default. The Overview page displays the test data of the selected test plan.
| Report Item | Description |
|---|---|
| Requirement Coverage (of all requirements) | Percentage of tested requirements in selected plans. This reflects the test coverage of function points. Completed requirements: Requirements where all associated test cases are marked as completed. Total requirements: The total number of requirements associated with the test plan. Requirement coverage = Number of completed requirements/Total number of requirements |
| Project Total Legacy DI | Weighted sum of uncompleted defects in the project, calculated by severity. The recommended formula is as follows: DI = Critical × 10 + Major × 3 + Minor × 1 + Trivial × 0.1 NOTE: All defects are calculated based on their severity levels. The DI value of each defect is as follows: Trivial: 0.1 Minor: 1 Major: 3 Critical: 10 |
| New DI | Weighted sum of uncompleted defects that are newly added to the test plan. This includes defects directly associated with the test plan and those associated with its test cases. Defects are weighted by severity using the following recommended formula: DI = Critical × 10 + Major × 3 + Minor × 1 + Trivial × 0.1 |
| Case Execution Rate | Statistics on the execution of test cases. Case execution rate = Number of test cases that are executed as planned and have execution results/Total number of test cases as planned |
| Case Automation Rate | Proportion of automated test cases. Case automation rate in the test plan = (Total number of test cases – Number of manual test cases)/Total number of test cases |
| Valid Case Rate | Proportion of executed test cases where defects are found to all executed test cases. Valid case rate = Number of executed test cases where defects are found/Number of executed test cases |
| Requirement Pass Status | Passed requirements: Requirements whose associated test cases have all passed and been completed Requirements not passed: Requirements whose associated test cases have not all passed and been completed Requirement pass rate = Number of passed requirements/(Number of passed requirements + Number of requirements not passed) |
| Case Pass Rate | The pass rate reflects product quality. The pass rate covers all test cases in selected plans. Another chart displays data by result. Pass rate = Number of successful cases/Total number of cases |
| Defect Severity | Displays the number of defects associated with the selected plan. |
| Automation Rate | Displays the proportion of automated test cases in a pie chart. Case automation rate in the test plan = (Total number of test cases – Number of manual test cases)/Total number of test cases |
| Case Pass Rate by Type | Displays the pass rate of test cases of different test types. |
| Bugs by Module | Displays the number of defects by module. |
Details
On the Details page, you can view the outstanding and completion status of requirements and defects in the test plan.
Enter a title or number in the search box to search for the corresponding requirement or defect.
Evaluating Risks
On the Risk Evaluation page, enter your evaluation of the test quality and risks, or add custom information modules to the report.
- Quality attribute evaluation
Select a test type on the top, click the text box area, enter evaluation information, and click Save.
- Risk analysis
Evaluate risks based on the test progress. Click the text box, enter risk analysis information, and click Save.
- Custom information
Click Create Custom Information and add an information module.
Click the module name to redefine the module title, enter the related description, and click
to save the modification. To delete the module, click 
.
Click Upload Attachment to upload a local file to the customized module.
1. The size of a single file to be uploaded cannot exceed 10 MB.
2. A maximum of 15 attachments can be uploaded.
What is your overall rating for this page?
Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot
