Writing a Test Report

Test Overview

A test report briefs the test activity, clarifies target audience, reference test standards, test background and requirements, summarizes test object analysis, test requirements, test content, and test process, and draws test conclusions.

The test conclusions must be prepared based on the focuses of the target audience of the report, that is, the target stakeholders. Product managers should focus on risk disclosure and product quality conclusions. Test managers should focus on test costs and test outputs. Developers should focus on defect results and product quality information.

Test Scope and Function List

• A test report describes the functions, application scenarios, benefits, and functions of test objects.
• A test report describes the test scope specified in the test plan, including the name, version, features, requirements, environment, and test items of the tested system (test objects).

Test Strategy and Method Description

• A test report reviews the test policy and solution, such as the test types, scenarios, and methods. It strategically describes how to perform the test. It also introduces the solution used in the test, such as the integration procedure and sequence, test procedure and sequence, test method, test tool, and test case design and execution method.
• A test report describes the test environment, such as the name, specifications, quantity, version, and account of the hardware, software, and test tools used in the test.
• A test report summarizes the test period and tester input, that is, the planned start time and end time of the test, overall test progress, key progress check points, number of testers, work division, and labor hours.

Test Metric Statistics and Analysis

• Statistics on key test metrics: quantifiable metrics of the tested system quality, such as the speed, throughput, temperature, time, and resource usage.
• Defect statistics and analysis: total number of defects, defects by severity, defect resolution rate, defect repeat rate, number of pending defect tickets, defect distribution by module, and defect source distribution. Techniques such as defect orthogonal analysis and four-quadrant defect analysis can be used.
• Test execution statistics: number and proportion of designed test cases, number of executed test cases, pass rate of test case execution, number of regression tests, manpower input for test execution, and test execution period.
• Statistics on test adequacy and test capability: coverage of requirements and functional features, test execution completion rate, code test coverage rate, test automation rate, and defect hit rate of test cases.

Test Risk Analysis and Disclosure

Based on the test process and result, a report analyzes whether the product has quality risks, and lists the risks, risk basis, risk level, and risk mitigation suggestions. Risks do not mean that the quality does not meet requirements. The risk handling strategy depends on the risk occurrence probability and the estimated loss after the occurrence. If the product of the two is very low, the risk can be accepted.

Risks are an important basis for major stakeholders to determine the overall product quality and whether the release conditions are met. Risks must be filled in logically.

Quality Evaluation and Release Suggestions

A test report provides objective quality ratings and evaluations based on company or industry standards, as well as quality results and risk analysis for reference.

It provides release suggestions such as "Release", "Delay", or "Partial release" based on the quality evaluation. The release suggestions can be specific to features. Features with high risks are not recommended for release or can be released with restrictions.