Tailoring a Test Plan

Significance of Making a Test Plan

• Ensures that test activities are carried out based on test objectives and serve specific test objectives.
• Determines the test features, requirement list, and test scope of the test object.
• Selects test policy and methods suitable for the team's technical capabilities and tool portfolio.
• Identifies possible risk factors during test activities as early as possible and resolves them in time.
• Properly estimates the test workload, personnel, and resource requirements, and prepares the plan for each test item.
• Helps testers break down test activities and tasks and orchestrate personal work plans.
• Guides test execution activities, and corrects and remedies execution deviations in time.
• Provides related documents to report and communicate with stakeholders.

Time for Making a Test Plan

Test activities include test plan development, test design, and test execution. Test plan development ranks first and is carried out in the early stages of the test period. The test plan may be carried out at multiple time points based on the development mode and team organization mode used by different projects. For example:

• A system test plan can be made after the test department receives the test demand for a customer-oriented mobile app before its first version is released.
• In an internal software project, a test plan can be made before the product enters the development stage when the product initiation requirements are confirmed, the architect has designed the product architecture solution and design solution, and the developers begin to make a development plan.
• For commercial products that require routine special security tests, a test plan can be made after the security test department evaluates feature changes of new products based on historical security test plans and test reports.
• Different types of test plans can be made by experts of function tests, performance tests, security tests, and more after the test team leader makes the overall test plan.

It is recommended that the test plan be started as early as possible to guide and standardize the product quality activities in the early stages and improve the product testability. The testers should guide the quality elements in the product architecture and design from the perspective of attackers, which complies with the idea of shift-left testing.

However, in the early stages of the product, the granularity of the test plan is coarser. There is a lack of test cases at the executable level and available test environments. Therefore, the test plan needs to be refined as the project progresses. The test plan is not constant. As the test project is carried out, the test plan is gradually detailed and contains more and more information. During the refinement and improvement of the test plan, the initial test objectives, scope, design, and policy should be reviewed.

Test Plan Makers

• Owners of test projects and test teams
• Owners of security, performance, and reliability tests
• Experienced test engineers and test architects

Test Plan Content

According to ISO and IEEE standards related to test documents, the content to be included in a test plan can vary from project to project. The content depends on the project and team scale. The test plans can be simplified for small teams.

• Objectives

  Describe why a test is performed and what test objectives need to be achieved. The test objectives are the beginning of a test plan. The test needs to focus on the service value of the product. An overall product test plan should integrate the function, security, performance, usability, compatibility, and scalability into the test objectives based on the service attributes of the product. For example, financial products have high requirements on security.

• Test scope

  Describe the name, version, features, requirements, environments, and test items of the tested system (test objects).

• Test policy

  Specify the test types, scenarios, and methods, and strategically describe how to perform the test.

• Test solution

  Describe the test solution, such as the integration procedure and sequence, test procedure and sequence, test methods, test tools, and test case design and execution methods.

• Test environments

  Describe the names, specifications, quantities, versions, and accounts of the hardware, software, and test tools required for the test, as well as the management policy for preparing, reserving, restoring, and releasing the test environments.

• Testers

  Describe the number of testers, work division, and responsibilities, such as test architects, test development engineers, performance test engineers, and test environment management personnel.

• Test schedule

  Describe the planned start time and end time of a test, overall test schedule, and key phased progress check points. The test schedule should be combined with the development plan. The constraints and dependencies between tasks and resources such as the test solution, environments, and personnel must be considered.

• Test entry conditions

  Specify the entry conditions for starting a test. For example, the functions specified in the product specifications have been implemented, and the basic process and entry test cases have been passed. This prevents the test plan from being affected by the lack of basic test conditions.

• Test release standards and deliverables

  Specify the conditions to be met for completing a test, the criteria for passing or failing the test, and the deliverables to be generated after the test, for example, a test report whose content is specified.

• Risks

  Analyze the potential risks in the current project operation and the measures to mitigate and resolve the risks. Examples of risks: human resource availability risks, personnel skill and domain knowledge risks, and development-to-test time risks.

Test Plan Review

After a test plan is created, key stakeholders, such as the project manager, test manager, product manager, architect, and O&M manager, should be invited to review the correctness, comprehensiveness, and feasibility of the test plan.