Every stage of the software development lifecycle is equally important. However, if there is one part of development that has the biggest impact on the customer experience, it is probably testing. Missed bugs that turn up in the production environment make the worst impression on end users. And if you miss critical errors, the application can stop working at the most inopportune time which might cost your users, and your own business, a lot of money.
Thorough software testing can prevent all this from happening. But if you want to make the testing stage of your development workflow truly effective, you need to treat it like a battle with bugs. No battle has ever been won without a good plan. So what is a software testing plan, and how do you create a test plan? Let’s learn everything about it.
What is a test plan?
First and foremost, a test plan is an important document that contains all the information about how your team is going to perform the testing of your project. It starts with general information about the testing objectives but then goes deeper, to a practical level, outlining the test environment, schedule, and test deliverables. The test plan is not set in stone; it can grow and change during the project. Best practice is to update it regularly, taking into account the latest project developments.
What is in a test plan?
A comprehensive test plan includes all the useful details on how the testing will be done, where to start, and how to define when the software testing is completed successfully. It should include some data on the resources that will be required, which components or features will be tested, and which team members are responsible for conducting the tests.
Who takes part in it?
It might appear that you create a test plan specifically for the QA team, and only the testers and the test manager will use it. In fact, it is important to work on test documentation together with external teams, such as project managers, developers, business analysts, and others, to make the testing more transparent and help stakeholders better understand its details. As a result, it gets easier to meet the project deadlines and fulfill the product requirements.
Writing a test plan
Test plans should include details on what you are going to test, how you are going to do that, and what results you expect to achieve. There are several steps that can help you understand how to write a test plan that will make the life of your QA team easier and ensure the success of your product with customers.
Understand the product
You should start test planning by learning more about the software. You need to understand how the product is supposed to work and what the end users and project stakeholders expect from it so that you can make sure your testing will cover all the important features. You can do the following to learn more about the software:
- Do a walkthrough to see the product functionality for yourself.
- Check the available product and process documentation, including the project requirements, for a better understanding of the testing scope.
- Speak to the project stakeholders, from developers to the client, to learn more about the business goals and set the correct testing priorities.
Outline the test strategy
A test strategy is a part of the preparations for writing the test plan, but these two terms are often mistakenly used interchangeably. You need to design the testing strategy as a separate document so that your team can clearly see the goals they need to achieve.
Typically, the strategy includes general information on what is going to be tested, who will do it, and when (test scope and test logistics). It also points out the possible risks and ways to mitigate them.
The test strategy will depend on the testing type, which could be unit testing, end-to-end testing, performance testing, API testing, security testing, etc. Different test types can uncover different bugs, so choose the test approach that is closest to your project objectives.
Finally, consider the costs of testing. The test strategy document should address the available testing resources since some types of software testing can be quite pricey, especially if you go for test automation.
Set the test objectives for software testing
A successful system testing is a test that achieves its goal. You can’t do that if you don’t really know what you are trying to accomplish with your testing. A test objective should detail the features that will be tested, defining the system/software under test (SUT). It is important to set boundaries for what is going to be tested and what is not by specifying SUT because it can easily become confusing in modern applications with multiple integrated systems.
Test objectives can also include the metrics (for the results of non-binary tests e.g. performance tests). These metrics, or expected outcomes, will be used for the comparison with the actual results of the testing. For better visualization of testing objectives and results, you might want to include charts and graphs in your test plan document.
Describe the test cases
The objectives of your test cover its global targets, but the next step is to go into more detail about what exactly will be done. A test case is a step-by-step description of all the actions required to test a certain feature. Test cases also include information on what results are expected in each situation. It can be difficult to provide exact descriptions before the testing project starts, so it might be better to separate test cases in a different document or even multiple documents for each specific feature/functionality organized in a tree structure, to facilitate updating them as needed without re-approving the master test plan.
Allocate the resources and set up the test environment
An important part of creating a test plan is deciding how best to use the human and system resources that can be provisioned for the project. You need to determine the tasks that each member of the testing team will perform. This information will help you to estimate the time required for the particular project and draw up a realistic schedule.
This is also where you compile the list of hardware and software resources that your team needs. If anything is required but not available immediately, including it in the test plan will remind you to provide that resource ahead of time.
Plan the resources you need to set up the test environment as well. You want the conditions of software testing to emulate real-life conditions as much as possible, which might require some specific test tools. It helps to discuss the needs of the test team to better understand what is missing.
Specify the test criteria
You can’t determine the success or failure of your testing effort if there are no parameters by which you can judge the test results. The effectiveness of your testing also depends on when you start and end the testing process. That is why, in your test plans, you should outline the test criteria that you use to govern your testing. Typically, there are three types of test criteria:
1. Entry criteria
This is what defines when you can start testing. It goes without saying that the particular feature or element of the software you are about to test needs to be ready for testing. Furthermore, before you begin, you need to have everything ready for the actual work, which includes the complete test plan and all the resources that you require. The test environment should be set up, and every engineer in the testing team should be assigned to specific modules, components, or features they are going to test.
2. Suspension criteria
Sometimes testing needs to be paused before it is completed. Suspension criteria determine when exactly that might happen. For example, too many bugs might be encountered during the testing, which makes it inadvisable to continue. So, for example, you can set a limit of 40-50%, and when that many test cases fail and the suspension criteria are met, you suspend testing until the issues get resolved.
3. Exit criteria
These are the metrics you will look at to determine the testing outcome and also the targets you want to achieve, as well as the actual moment when the testing is complete. There are different methods that can be used to set exit criteria, but the most widespread two are:
- Run rate. This method uses the ratio between the number of executed test cases and the total number of test cases as defined in the test specification. Typically, the exit criteria rate is set as 100% unless there is some specific reason to set it lower.
- Pass rate. This one is based on the ratio between the number of successfully passed test cases and the total number of executed test cases. The exit criteria can be set at 80-90%, as a rule; the higher, the better, but the exact benchmark depends on the scope of your project. Also, it is important to single out critical functionality that has to pass all the tests in order for the release to happen, regardless of the general test rate.
List the test deliverables
Every test phase results in a number of tangible resources, some of which might be later forwarded to the test stakeholders. All these documents should be listed in your test plan to help your development team understand when they need to be delivered. The test deliverables are usually divided into three categories, depending on the project’s test phase:
1. Before the testing starts
First of all, you need to make sure your test plan for this particular phase of the project is ready. There is also usually a more detailed lower-level test plan that describes test design specifications and test cases. If possible the test data and candidates for automation should be specified beforehand.
2. During the testing
When you run the tests, you should fill out bug ticket reports that must contain any deliverables that help in bug reproduction and analysis, e.g. logs, test data used if it is not prepared beforehand, screencasts, screenshots, etc. Besides this, if some or all of your testing is automated, you will have to deliver the test scripts and their documentation.
3. After the testing is complete
Test results are outlined as testing reports, which include a testing summary with test results analysis and findings on defects. The data from these reports is further used when creating product guidelines and release notes. It is an important factor for decision making by all stakeholders and can be used in judging whether the exit criteria have been fulfilled.
Include the time for bug fixes
Your schedule in the test plan should have some time set apart specifically for fixing the bugs that are discovered. In an agile environment, testing may happen in sprints just like any other part of the software development lifecycle, and then after each sprint, the bugs found should be fixed at once. It is a well-known fact, confirmed by a study conducted on behalf of the US Department of Commerce’s National Institute of Standards and Technology in 2003, that the earlier an issue gets fixed, the cheaper it is for the project budget. A bug discovered in the production environment can cost the company $10,000, while if the same bug gets fixed in the QA phase the cost can be a hundred times less, and it might cost as little as $100 to fix if you find the bug when still gathering the requirements. So it is crucial to resolve all the potential problems as soon as possible.
To facilitate bug fixing, you should include issue classification in your test plan. This will help the developers to prioritize fixing critical bugs over minor ones. The exact metrics and set of values for assigning priority levels should reflect your company policy and can be different in each organization.
How to keep your test plan useful
Software development projects can be rather flexible. A lot of things, even the very project requirements, can change along the way. These changes, inevitably, will require editing the test plan as well. Is it worth it even trying to plan the testing in detail?
The thing is, different parts of the test plan can be more prone to changes than others. The testing scope and test objectives are less likely to be modified during the project, so you can describe them in detail directly in the master test plan document. As regards the things that change more often, like specific people assigned to certain testing tasks, estimates, schedules, and so on, it is usually better to only reference them in your main document, providing links to detailed test plan documents which you can change without rewriting the actual master test plan.
There are various test case management tools, like Zephyr, QA Touch, TestRail, PractiTest, TestLink, or Testpad, where you can conveniently manage separate parts of your test plan on a single platform. This way, you can ensure your test plan is up-to-date and valid at all times.
Software test plan templates
If you are still not sure where to start, you don’t actually need to write a test plan from scratch. There are multiple test plan templates available online that can be a good first step on the way to effective testing. If you decide to use one, make sure you modify the test plan template you find to reflect your real project specifics. It might be simpler to start from a template rather than from a blank page - that is up to you to decide.
If you are worried about the quality of a test plan template, the easy solution is to go with the international standard ISO/IEC/IEEE 29119-3:2013. This standard contains documentation templates for software testing that work for both traditional and agile SDLC models. The samples include a test plan, test procedures, and cases that can be used for any testing activities, projects, or organizations.
Conclusion
Creating a good software test plan prevents chaos in your QA team. A comprehensive test plan tells everyone what to do and when while also ensuring that only the necessary tasks are completed, with less overhead. Investing some time into a tight, well-thought-out test plan is a great way to get your product ready for users as quickly and safely as possible. Just remember that testing shouldn’t be treated as something that can be done along the way, treat it with respect as the important part of your software development workflow that it is.
