Rediscovering the true purpose of QA

Quality assurance is about ensuring that every piece of software you release is reliable, error-free, and fit for purpose. At least, that’s what it used to be about. Increasingly, we see QA teams and test engineers focusing the majority of their entire time on creating and maintaining tests. While this may seem OK, it means they are often losing sight of their main goal. Suddenly, getting the test script to work becomes more important than ensuring the quality of the software. In this blog, we explore why this is the case and show how smart testing platforms like Functionize redress the QA balance.

The evolution of testing

Testing used to be simple. You employed a team of QA testers and they manually worked through test plans to check your software was bug-free. However, testing has evolved a lot since those days.

Manual testing

Manual testing was and is a key part of any testing strategy. It involves a QA tester working through a test plan step by step. At each step, she compares the result with the expected one. If they differ, then the test fails. Manual testing is also used for tracking down newly-reported bugs or for developing tests for new features (sometimes called progression testing).

Manual testing has some advantages, such as the fact a human can apply her experience to make sure she doesn’t do the wrong thing. However, repeating the same test multiple times is time-consuming, boring, and it’s easy to make mistakes.

The motivation for test automation

Over the years, the problems with manual testing have motivated people to develop test automation systems. Test automation was seen as a panacea, allowing you to execute repetitive tests quickly and reliably. At first, test automation systems had to be custom-built for every application. That was slow, cumbersome, and only worked for the biggest companies. As a result, people began to look for general test automation frameworks.

Selenium and test scripting

Selenium was the first general-purpose test automation framework to become popular. Selenium uses a Web Driver to interact with your application’s UI. You tell the system what to do by writing a test script. Each test step asks the Web Driver to select a given element in the UI and interact with it. Then it checks whether the outcome is as expected. Selenium-based test scripts were intended to speed up and simplify testing, but they have a fatal flaw—test maintenance.

The drawbacks of test scripting

Test scripting has been the mainstay of test automation for almost two decades. So much so that many people believe the two terms are synonymous. But the reality is test scripting has become obsolete without people even realizing. Here’s just a few of the reasons why:

• Slow to create tests. Test scripts are just another piece of software. As a result, they pose just the same challenges for your team. Tests are slow to create, require testing, and often need repeated rounds of debugging.
• Time consuming to fix. As we mentioned, test scripts rely on locating elements in your UI using a selector. Sadly, these selectors are prone to changing randomly any time you change your code or redesign your UI. That means, you often spend time having to fix your tests. In fact, we often see teams building up so much test debt, they can no longer create any new tests.
• Hard to run efficiently. Most test automation systems need to be run on local hardware. Tools like Selenium Grid allow you to run tests in parallel. However, you can’t scale up the execution like you would in a cloud environment. That poses an upper limit on how many tests you can run at a time.
• Requires constant updating. One of the subtler problems with Selenium and related systems is that they constantly need updating. That’s because you need to keep up with the frequent releases of new browser versions. Moreover, you also need to keep your infrastructure updated. All in all, this impacts on your sys admins or DevOps team, who are probably tasked with sorting this out.

What is test debt?

Test debt is an insidious problem that can see QA team productivity head to zero. Test debt is a function of application complexity, frequency of application changes, and the degree of test coverage. As your application becomes more complex, you need to increase the number of tests you automate (coverage). However, the more tests you automate, the more time you spend on test maintenance as a result of application changes. The following graph illustrates how this leads to an increase in test debt.

The impact on QA teams

Test debt has a huge impact on QA teams. They go from creating and executing new tests to just playing catch-up as they try to keep up with the pace of new features. In the end, this can overwhelm even the best teams.

Loss of focus 

Test debt leads to a real loss of focus. Your test engineers start to concentrate on fixing broken tests, rather than developing new ones. QA should be about testing and improving your application. But instead, it becomes all about maintaining your tests and trying not to sink under the burden. 

Wasted time 

Test maintenance is almost by definition wasted time. In the ideal world, you wouldn’t ever need test maintenance because tests shouldn’t break just because your UI changed. Imagine if you faced a situation where every change by one of your developers completely broke every other part of the application. You’d try to fix that situation pretty quickly right? However, test engineers have been programmed to view test maintenance as a necessary evil. 

The test paradox 

Test debt is a manifestation of what we call the test paradox. This paradox reflects the fact that the more tests you try to automate, the less time you have available to automate more tests. Often, QA managers believe the solution is to throw more resources at the problem. But the worst part of this paradox is that the more resources you throw at it, the worse the problem becomes. So, you need to work smarter, not harder.

How Functionize helps

Functionize offers you a smart solution to the test paradox. Our AI-based test automation platform slashes the need for test maintenance. It also solves most of the other issues that affect test scripting.

• Simpler test creation. Anyone can create fully-functional tests using our low-code Architect tool. Meanwhile, test engineers can use advanced features like test data management, customized validations, and accessing data from the underlying DOM, CSS, and javascript.
• Cross-browser and cross-platform straight out the box. Unlike test scripts, Functionize tests will immediately work on virtually any browser or platform. That makes them much more flexible and further speeds up test creation.
• Virtually maintenance free. Our tests are built on top of a powerful AI system. This records millions of data points while you create your tests and each time you run them. This allows tests to self heal whenever the UI or site code changes. If a fundamental change in the logic triggers a failure, SmartFix offers you several options that may solve the problem.
• Cloud-based solution. Functionize is delivered as a cloud-based solution. This can be using the Functionize Test Cloud or your own custom installation. Either way, you reap the full benefits of a cloud-scale platform. Run as many tests as you need, run tests from anywhere in the world, and allow anyone on the team to access test results for free.

Conclusions

Test debt and the test paradox have seen QA teams lose sight of their main purpose. As we have seen, they start to focus more on building and fixing tests than checking your product. Fortunately, Functionize’s AI-powered platform completely removes your test debt. If you wan to wave goodbye to the test paradox, sign up for a demo today.