Make test maintenance easy in Selenium?

The dos and don’ts of test scripting

Simplify your life by following these best practices

Test maintenance is one of the biggest nightmares for test automation engineers. Even the smallest change to a UI can break your tests. Here we give some helpful tips to make your life easier.

Introduction

There is no question that test automation has transformed software development. Arguably, without Selenium, we wouldn’t see such a plethora of web-applications as we do today. But going hand-in-hand with test automation is the potential nightmare of test maintenance. Below we explore some of the common maintenance issues and give some advice on how to avoid these.

The test maintenance problem

Or where did all my time go?

Mention test automation and most people immediately think of Selenium. Selenium is definitely the Granddaddy of the test automation world, but it comes with some issues. The problem is, Selenium (like related test automation tools) relies on JavaScript. When you create a script, you have to specify the selector of the element you want to interact with. When the test script is run, this selector is found on the page and the appropriate action is taken.

However, as the UI changes and develops, elements on the page may move, get restyled or even disappear. This means any test that interacts with those elements now needs to be updated to avoid spurious test failures. To make things even harder, often the initial failure is silent. It’s only several steps later that you see the actual error. Imagine for example a shopping page where you have several “Add to cart” buttons. One is for the product you are viewing, but there’s also one for each of the related products. It’s easy to see how a layout change leads to the wrong item being added to your cart. But this will only show as an error once you reach the checkout stage several steps later. Now, you have to go back through every step of the test to try and find the trigger. 

This ongoing need for test maintenance is a real problem. Anecdotally, some testers find that fixing issues with existing tests can take over half their time. You could use all that lost time to create new tests, perform exploratory testing and refine your test suite. So, what can you do to make things easier for yourself?

Creating tests for easy maintenance

How good design can make your life easier

Test maintenance will always be an issue for Selenium. But you can simplify your life to no end if you just think a bit before writing your scripts.

As mentioned above, the JavaScript selectors used in Selenium are the biggest cause of test maintenance problems. In the ideal world, every selector you specify will be unique, and will still be accessible following design changes. Let’s look in a bit more detail at what kinds of selectors are available to you.

Unique ID

Every element on a page can be assigned a unique ID. Clearly, this makes it easy for you to select the correct element. However, as a tester, you aren’t able to go into the code and change the ID of an element. Therefore, to achieve this you will need the cooperation of the developers. This means convincing the managers that it’s a good use of their time.

Name or CSS locator

Often elements will have a name or CSS locator. For example, “buy now button”, or “email”. These can be used as selectors in your test. However, as explained in the example above, multiple identical elements can exist on the same page. Selenium will just take the first such selector it finds, which may well be the wrong one.

Attribute

Potentially you can use object attributes as a selector. As an example, the button located at the top right of the screen. But this sort of selector is almost always a poor choice. Your test will fail as soon as the element moves, or as soon as the page is redesigned.

Xpath

Sometimes you may find yourself having to define your selector using Xpaths. An Xpath is simply the SML path to a specific location on your page. The advantage of using an Xpath is that it can cope with nested DOMs. Typically, Selenium cannot cope when you have nested or multiple DOMs. The problem is that Xpath can either be relative or absolute. If you use a relative Xpath your test is doomed to fail at some point. So, you need to be certain that you have given a stable absolute Xpath as your selector.

The importance of good design

One thing that can make test maintenance easier is consistent design. This means that when a UI change is made, it should affect all similar elements. In turn, this reduces the time for maintaining your tests. Let’s see an example of what I mean. Say your UX designer decides to reorder the menu. When you come to update your first test you need to find the new correct selector. But then you just have to find all tests that use that selector and copy the change across. However, this can only work if every page has been updated in the same manner.

Another way that design can make your life easier is the use of standard page layouts. Your life will be simplified if you know the login/logout button is always at the top right. Then you will be able to make assumptions about what the button at the top right is doing. Fortunately, there is a nice synergy here between designing for good UX and designing for testers.

Reusable test code

Developers are always being encouraged to reuse code, rather than write unique code for every element. By reusing code, the developer will save time and effort. Also, when they have to make a change, that change only has to happen in one place. You should be applying exactly the same rules to test code.

Automated tests consist of a set of discrete actions that the script performs in sequence. You will often find that some steps are common across multiple tests. A perfect example of this is logging in. Almost certainly, many of your tests will require a test user to be logged in. Rather than write this step for every test, it’s better if you write it once and then call it. Now when you have to update the login step in your tests, you only have to update it once.

Waiting for slow responses

Often, when you write a test you make some implicit assumptions about the completion time for each step. However, changes in the backend code can often lead to changes in responsiveness. These may show as test failures when a required step hasn’t completed before the script moves to the next step. As everyone knows, you can solve this by adding “waits” in your code. These can be implicit, explicit or can use the sleep() command. The advice is always to favor explicit waits, and you should avoid sleep() command like the plague!

How Functionize can help

Here at Functionize, we feel sorry for testers still stuck using Selenium. Sometimes you must feel like you are in Purgatory! The problem is that Selenium never makes your life easy. Even if you follow all the advice above you are bound to spend much of your time on test maintenance. There is a better way.

Functionize tests are created using our natural language test creation engine. You simply write your test in plain English and submit it to the system. The test is then parsed using NLP and is converted to a test script. You then confirm that the test is working correctly and can add it to your suite of tests. We are pretty confident this is a better approach than writing test scripts in Selenese! 

Once your test has been created, it becomes largely self-maintaining. This is because of the power of our ML Engine. It consists of 3 key parts. Self-healing tests can cope with the majority of CSS and layout changes. Root cause analysis is designed to find the actual trigger for a failure, even if it was many steps back. Finally, our smart suggestions and one-click updates features allow you to select the best solution and update your test instantly.

Conclusions

Selenium was revolutionary when it was released 15 years ago. However, if you use Selenium you are fated to spend a lot of effort on test maintenance. The advice given above can definitely save you time and effort. But we encourage you to try Functionize. We’re sure you won’t look back once you have!