Jory MacKay
Jory is a writer, content strategist and award-winning editor of the Unsplash Book. He contributes to Inc., Fast Company, Quartz, and more.
February 19, 2020 · 10 min read

5 Steps to Create a Test Plan for Your New Feature Release (Free test plan template)

🎁 Bonus Material: Free Test Plan Template

Create a Test Plan for Your New Feature Release

In 1962, NASA launched the Mariner 1 as their first attempt to send a spacecraft to Venus. However, shortly after it launched, the rocket veered off course and was forced to self destruct.

The cost? $135 million (in today’s dollars).

The issue? A missing hyphen in the code.

You might think software testing won’t result in as serious consequences for your company. However, a 2017 study found that software failures cost the U.S. economy $1.7 trillion in financial losses (and more than 268 years in downtime) that could’ve been avoided with proper testing.

Before any piece of software or new feature goes out to your users, you need to thoroughly put it through its paces. Test it. Try to break it. And make sure that whatever your users do, it responds as designed.

In short, you need a test plan.

A test plan is one of the most important parts of any software development process. It outlines how you’ll make sure your product or feature will do what it’s supposed to and not break when your users need it most.

But what should your test plan include? How deep do you really need to go to ensure your product holds up and your users get what they expect?

This guide will cover everything you need to know about defining and documenting your test plan and choosing the right test strategies that will ensure your users, development team, and stakeholders are all happy.

Ready to write your own test plan? Download our free test plan template and follow along with the post.

What is a test plan (and why do you need one)?

A test plan is a detailed document that outlines the test strategy, objectives, resources needed, schedule, and success criteria for testing a specific new feature or piece of software.

The main goal, of course, is to discover defects, errors, and any other gap that might cause the software to not act as intended or provide a bad experience for your users. More specifically, a test plan ensures your software:

The cost of a bug found after release is nearly 30 times more expensive than if it was found during the design phase.

While these sound like pretty straight-forward criteria, in practice, they rarely are. The problem is that a “test” means you’re naturally testing against something. In most cases, this will mean your specifications and success criteria you put down in your SOW or planning doc. (But could also include things like comparable products, past versions, user expectations, standards or laws).

It’s nearly impossible to test every scenario, environment, or use case your software will encounter during its lifecycle. Instead, software errors, bugs, and defects could pop up from:

This makes writing a clear, yet comprehensive test plan a difficult balance. You want to include as much detail as possible to make sure you’re not missing any glaring errors. But you also don’t want to drown your team in testing tasks, delay your release, or add in new errors from your “fixes.”

What should be included in your test plan template?

So what should go in your test plan then? Well, that depends.

Each product and feature will have its own specific testing criteria, strategies, and needs. Also, the goal of your test will change how you approach it. For example, User Acceptance Testing (UAT) is completely different from stress and load testing and your plan will need to be tailored to your ultimate goal.

Template for your new Test Plan

However, this doesn’t mean you want to start from scratch each time you’re testing a new piece of software. Creating different test plan templates for different products is a great way to quickly guide your approach to testing new product releases, updates, and features.

So what should (or could) you include in your test plan? Broadly speaking, there are a few main areas you’ll want to include in your test plan that will act as the foundation of your test plan document:

1. Coverage: What exactly are you testing?

As we said before, creating a test plan is all about balance. You want to be comprehensive, but not overwhelming, which means getting specific about what will (and won’t) be included in the test plan.

After a brief introduction highlighting your test plan objectives, high-level scope, and schedule you need to define what you will or won’t be testing.

This is your test scope and it can quickly get out of hand if you don’t take time to be specific with it and answer both what you will test and why you’re going to test it.

Everyone needs to be on the same page with the test criteria and scope. As a best practice, make sure you use industry-standard or at least agreed-upon standards and terminology to describe your tests and why they were (or weren’t) completed. This way, there’s no grey area or confusion about what you tested.

2. Methods: How are you going to carry out these tests?

Next, you need to clearly explain what your test strategy is. Go into as much detail as possible.

You also need to know when your test has been successful. In other words, what are the pass/fail criteria for each test?

This isn’t the only criteria you need to be aware of. There are a few other common situations you need to outline in your test plan, including:

It’s also a good idea at this point to list your assumptions and risks. In other words, what are you assuming is going to happen and what are some of the risks you’re going to face during the test?

Lastly, you need to outline your test project’s resource needs and schedule. Who is in charge of testing and what resources do they need (both technical and human)? When is the testing going to take place and for how long?

3. Responsibilities: What are your desired outcomes?

What are your required test deliverables? This means the data you want to collect, how you’re going to compile them in reports, and the issues and tasks that will be passed back to the development team.

To make sure nothing gets missed, each test deliverable should be assigned to a specific person on your team in a section on roles and responsibilities.

It’s important to remember that this is just a basic framework of what to include in a test plan. Over time, you’ll create your own library of test plan templates that will serve as guides for new product releases, updates, and features.

Download our free Test Plan Template to get started.

5 steps to create (and execute) a test plan for your new product or feature

Now that we have a high-level idea of what to include in our test plan template, let’s dig into the specifics. To make sure your testing scope doesn’t get way out of proportion, it’s important to have a step-by-step process for creating your test plan and executing on it properly.

Here’s where you should start:

1. Analyze the product or feature you’re testing

You need to have a deep understanding of the product or feature before you can start creating a test plan for it. For example, let’s say you’ve just gone through a website redesign and want to test it before launch. What information do you need?

This step is what gives you the context to write your test plan introduction and objectives and start to plan out the resources you’ll need to complete it.

Truly understanding your product is the first step to creating a productive and successful test plan.

2. Design the test strategies (and approach) you’re going to use

Next, it’s time to decide the scope of your test plan. What’s included in the scope of your testing will depend on a number of factors beyond just the product or feature. You need to dig in and think about:

For our website redesign example, we might want to say that functionality, UX, and checkout flow are in scope. While stress, performance, and database testing are out of scope.

You might also want to think of this in terms of commonly used testing approaches, such as:

Deciding what to test and documenting your test strategy are the most critical parts of your test plan. Don’t rush through it. Take the time to really understand your goals and needs and balance them against the resources you have for testing.

3. Define the test objectives and pass/fail criteria

As you define each different test you’re going to run, you need to know when your test is “done.” This means defining the pass and fail criteria for each specific test, as well as some of the things we mentioned above, such as exit and suspension criteria.

To do this, you’ll want to identify individual system metrics that you’re checking and decide what success means for each one. For example, if you were doing a performance test you might look at metrics such as:

Remember, you can continue testing and iterating forever. So you need to decide what’s “good enough” to get your software out and in the hands of users.

Don’t rush. Take the time to really understand your goals and needs and balance them against the resources you have for testing.

4. Plan the test environment

The results of your test plan will depend as much on the feature you’re testing as the environment you’re testing it in. As part of the scope, you need to determine what hardware, software, operating system, and device combinations you’re going to test.

This is a situation where it pays to be specific. For example, if you’re going to specify an operating system to be used during the test plan, mention the OS edition/version as well, not just the name.

5. Execute your test plan and track progress in your project management tool

Once your test plan is in place, there’s a specific process you need to follow. Think of this as the Software Testing Life Cycle (STLC). Similar to the Software Development Life Cycle, the STLC follows each phase of testing and usually looks something like this:

This is roughly the path that we’ve described so far. But what about actually executing on your test plan and tracking/reporting the results?

Using a tool like Planio, its easy to set up and track any number of testing scenarios.

Planio’s customizable trackers and workflows can be used to track and create issues or repeatable tasks related to each test. Each issue associated with a tracker has a fixed selection of statuses for the test phases to work through.

Workflow in Planio for your feature testing

Using a workflow like this, you can identify when a test has failed, decide what to do next and prevent the issue being closed. Even better, once you’ve created these workflows they can be repeated in Planio every time you need to test a new feature or piece of software.

For example, let’s say you’re going to test the checkout functionality of your new e-commerce software. Using Planio, you could:

Issues to track feature tests in Planio

Want to learn more about Planio’s powerful workflows? Check out our post on How to create a project workflow in Planio that will save your team hours a week.

Why you should start testing as early as possible during your development process

If you’ve read our Guide on Software Development Processes, you know that when you test features, products, and specific code is dependent on what style you’re using. The biggest difference probably being between traditional (Waterfall) and Agile development.

When to start testing

In traditional software development (aka Waterfall), your test plan starts once you’ve completed the project. Whereas in Agile, requirements, development, and testing are all done simultaneously as you develop new pieces of usable software.

So which is right? And when should you start testing?

While there are debates on both sides, it’s probably safe to say that the earlier you start testing the better.

In fact, IBM commissioned a report almost a decade ago that discovered that the further into the Software Development Life Cycle that a bug or issue is discovered, the more expensive it is to fix.

In other words, if you leave your testing until just before release, you’re adding a huge strain on your resources to deal with it. Even worse, the cost of a bug found after release is nearly 30 times more expensive than if it was found during the design phase.

Testing is an iterative process. When one bug is found and fixed, it can illuminate other, deeper bugs or even create new ones. The earlier you can start dealing with those, the less of an impact testing will have on your launch and go-to-market strategy.

Don’t treat your test plan as an afterthought

Testing isn’t just another thing to check off your list. It’s an important and potentially project-altering phase that needs to be carefully thought through and planned.

Your test plan will guide you through that process from start to finish, helping you understand the objectives, define the scope of testing, create pass/fail criteria, document your process, and deliver the documents and artifacts you need to make your product or feature the best it can be.