Test management tools – why they are essential for you

Test management tools allow you to plan, monitor, and audit your test executions. They are an essential element in any modern software development tool stack. In this blog, we look at why they are so useful and introduce 5 of our favorite tools.

Introduction

Testing isn’t just checking for bugs and regressions. Test management is also vital. In any company, it’s essential that you can audit test coverage, execution, and outcomes. This is where test management tools come into play. Here at Functionize, we are agnostic to which tools you use. In this blog, we show you why test management is so important, and how using the correct tool can help you.

Testing is just about checking for bugs, right?

Managing tests efficiently

Managing your testing is about more than just finding bugs. You also need to be able to track and record test outcomes.

Clearly, testing is about checking your software works. That is to say, checking it does what it should do, proving it is bug-free, and ensuring it is reliable. The first two are achieved with unit testing, integration testing, system testing, and acceptance testing. The third is done using regression testing, load testing, and API testing. However, how do you verify that your testing is adequate? And how do you check the outcomes of your testing?

Understand test coverage

Test coverage is a rough metric for assessing how much of your code is being directly tested. Typically, it refers to unit testing, so it’s measuring whether each function in your code has associated unit tests. Many build tools are able to report the test coverage within the build results or prepend it to the top of the source.

But test coverage needs to be thought of in a broader sense. Test coverage should measure that you are testing every interaction in your UI. That you have tested all those weird corner cases that might hide a bug. This is much harder to measure than what percentage of your functions have unit tests. But it is also much more valuable.

Track test outcomes

Testing is all very well, but unless you record test outcomes correctly it’s pretty useless. This is especially true when a regression test triggers a failure. If you can’t compare the current result with the old result, it’s nigh impossible to tell if this is a new bug or a regression. I remember working with QA engineers who just relied on their memory for this. Or would search through Slack chats to find discussions about the old failure. Memorably, in one case I remember being told: “It’s OK, that test often fails, we just ignore it.” Tracking these outcomes allows you to be more efficient.

Prove that you tested

While Continuous Integration is growing in popularity, many companies still release new builds periodically. Often this means there is a rush before release to complete testing. But how do you ensure you’re not just duplicating existing test results? Being able to show when each part of your system was tested can greatly improve the efficiency of releases. This is especially true when you have a large and complex regression suite.

Audits and the blame game

We’d all like to think that testing is infallible. But of course, it isn’t. Software is now so complex that it may be impossible to test every single possible interaction. But however idiot proof your UI is, Murphy’s Law dictates that someone will manage to break it. And at that stage, the blame game will start. Being able to check and verify your test coverage will be critical at this stage.

Of course, for some apps (such as healthcare apps) this requirement for auditability may be mandatory. For instance, ISO 13485 (“Medical devices -- Quality management systems -- Requirements for regulatory purposes”) mandates that every stage of development is documented and subject to quality management oversight. Measuring the quality of your quality assurance system may seem a bit meta, but it is vitally important

What is test management?

Wherever there is a problem in the software development world, someone will have come up with a solution. In this case, the solution is test management tools. Just as a CI tool helps coordinate and control builds, a test management tool coordinates and monitors your testing. Wikipedia says test management tools:

“store information on how testing is to be done, plan testing activities and report the status of quality assurance activities.”

There are many tools to choose from and which one is right for you will depend on your circumstances.

Test management tools usually provide several core functionalities. These are:

• Test Planning and Design. This is about planning your test strategy. Understanding which tests can be automated and which are manual.
• Requirements Capture. Here you check that your tests are covering all that they need to. This includes verifying that you understand what your software is meant to be doing.
• Test Execution and analysis. This is the heart of test management. It involves coordinating test resources and ensuring test plans are executed properly. It should also be alerting you to any bugs and making sure it keeps a record of all tests and the outcomes.

How to choose a test management tool

As well as the core functions mentioned above, there are other features that will make your life much easier.

Integration

It may be stating the obvious, but you should check whether the tool integrates with your existing tool stack. In particular, can it integrate with your bug/issue tracking software?

Flexibility

The system should be flexible enough to handle a variety of styles of development and a range of different projects. You may have a specific project in mind when you select your tool, but don’t fall into the trap of over-specializing!

Task tracking and assignment

A key benefit of test management tools is the ability to assign and track tasks across your team. You may already be using a system like Jira or Trello to do this, but it pays to be able to do this in your test management tool too.

Traceability

Following on in some ways from the previous feature, it is vital to be able to trace test executions and outcomes. This means tracking test runs, tracking issues, verifying bug fixes, and recording successful tests.

Some test management tools

There are literally dozens of test management tools to choose from, and we can’t tell which is the right one for you. At Functionize we are completely agnostic to which tool you use. But the following is a brief overview of 5 good tools that offer a range of useful features.

Adaptavist Test Management for Jira

Adaptavist describe themselves as the world’s largest provider of Atlassian services. As part of this, they develop plugins for well-known Atlassian products. Their Test Management plugin for Jira is one of the best on the market. It offers seamless integration, a REST API, live statistics, and clear traceability between requirements, test cases, and defects.

PractiTest

PractiTest sell themselves with the strapline “More Tests? More Builds? More Devices?

Simply accomplish more with PractiTest Test Case Management.” They claim to turn your test data into actionable intelligence. Among other things, they allow seamless integration of manual and automated testing with your CI processes.

QA Coverage

QA Coverage offers a modular test management platform. Modules include Requirements, Test Design, Test Execution, and Ticketing. They also offer an impressive Reporting and Dashboard module.

Test Collab

Test Collab offers integrations with a number of issue management systems. This includes Jira, Mantis, Pivotal Tracker, and Redmine. For Jira and Redmine, this integration is bi-directional, allowing you to create and manage test cases directly from the issue manager.

Xqual

Xqual Xstudio bills itself as an Application Life-Cycle Management (ALM) solution. That means that it offers a huge range of tasks over and above simple test management. These include release management, requirements capture, specifications, test management, and bug tracking. In addition, it can be integrated with systems like Jira, ClearQuest, and Mantis.

Conclusions

Test management tools will greatly simplify the task of testing complex software. They are designed to integrate seamlessly into your tool stack and will help you become far more productive and efficient. Some test automation companies try to get you to use their own test management tool. Here at Functionize, we prefer to leave that up to you. You know best what your requirements are and what your tool stack looks like. But we strongly encourage everyone to start using test management tools.