Test Maintenance needn’t be hard

Keeping your scripts current without wasting time

Test maintenance is essential. At Functionize we make it much easier

Here at Functionize we leverage AI to help you reduce the time needed for test maintenance. From Self-Healing Tests to Root Cause Analysis, our system is designed to make your life easier and more productive.

Introduction

Ask any test automation engineer what they hate about their job and chances are the reply will be test maintenance. The problem with frameworks like Selenium is they are extremely brittle. Even the slightest UI change or styling change can break them, and they are not good at coping with responsive or dynamic sites.

At Functionize one of our key aims is to make life simpler for test engineers. So, we put a lot of effort into reducing the burden of test maintenance. In this blog we will discuss some of the things we do to help and highlight the importance of regularly reviewing your scripts. 

Test maintenance is a key part of the test cycle[/caption]

Why is test maintenance needed?

The downside to test automation

Test automation always brings with it a requirement to up your game when it comes to test maintenance. Every tester will be familiar with the pain of test maintenance. Especially, those test failures that take hours to track back to their root cause. The problem is, when you create a test script, you are baking in assumptions about the UI. And as we all know, UIs develop and evolve over time.

Systems such as Selenium are especially brittle. Even something as simple as your designer changing the look of a button can break all your tests. This means all too often an apparent test failure isn’t actually caused by a bug. As a result, you often have to spend more time on test maintenance than on test creation.

How Functionize can help

Here at Functionize, we specialize in applying artificial intelligence in order to simplify the lives of testers. Our ML Engine has a feature we call self-healing. Rather than use dumb one-dimensional selectors, all the selectors in our tests are fingerprinted using machine learning. This means the system learns what the selector is doing, rather than just what it looks like or where it is on the page. As a result, if the selector moves or is re-styled, the system can automatically (and transparently) repair the test. This immediately reduces the need for test maintenance.

Root Cause Analysis (RCA) uses machine learning and an intelligent rules engine to identify the actual cause of a failure. Often failures can be traced back to an action taken many steps before a failure actually manifests itself. Once RCA has identified likely causes it uses its understanding of your tests to suggest possible solutions which you can choose. You can even use our 1-click updates feature which tests each possible solution, ranks them and asks you to select the one you want. Once you click on the solution that test step is updated for all tests. Between them, ML Engine and self-healing can deal with all but the most obscure test failures, thus significantly reducing your test maintenance workload.

How you can help yourself

Proactive maintenance can still save you time and helps keep your tests efficient.

Even with RCA and self-healing tests, you will still need to spend a bit of time doing test maintenance. Sometimes changes are so fundamental that everything on the page changes, including API calls. So it is essential that you try to help yourself. A good test engineer should maintain a great working relationship with the product team. They are the ones that can warn you when significant changes are coming and you can start to prepare for these.

You should also regularly check whether your tests are still relevant. Code changes, flows change and you may find that some of your tests are simply pointless now. Removing these obsolete tests is an essential part of test maintenance. Similarly, you may find there is so much overlap between two tests that it is better to merge them into a single test.  

In our experience, proactive maintenance like this pays real dividends. And even here we try to make your life easier. One of our newest innovations is NLP Engine. This allows you to write tests in plain English and then uses Natural Language Processing to convert these into fully functional tests. This means that when a design change comes along, you can easily update the test in collaboration with the product team and then re-generate the script. 

Test maintenance will never go away completely. However, Functionize lets you claw back most of the time you would have spent dealing with minor issues. Less time on test maintenance equals more time doing the tasks that matter!