The true costs of Selenium: why we chose Functionize instead

I’ve spent my career testing with Selenium; however, it no longer provides what I need. In this article, I will explain why my team has moved on from Selenium. I will also show why Functionize turned out to be the best tool possible.

My quality testing dilemma

I’m a technical quality engineer for web apps, and I have been a staunch Selenium supporter for years. In fact, it’s easy to say that I’ve built my career using it. So much so that I even contribute to the open source community regularly. Like most open source tools, the lack of fees, along with the large community of users, are very appealing to engineers in my line of work.

However, these days I find myself leading a team of six other testers. One of my new managerial responsibilities is to provide testing tools to my team. Even though easier to use tools exist for a fee, I couldn’t shake my devotion to Selenium. So naturally, I started training my testers to use Selenium. At that time I was just too “old school” to trust the new tools. I was still laboring under the idea that if a tool was too easy to use, it couldn’t possibly be effective.  

However, Selenium just isn’t the right tool for the task anymore. The fact is, Selenium is a hard tool to learn and an even harder tool to use for practical applications. For manual testers, the learning curve is much too steep. During my struggles to train my team on Selenium, I finally admitted to myself that it just wasn’t working. I needed a new, easy, and intuitive tool for my team if we had any chance of success.

Our challenges with using Selenium as a team

So first, let me explain some of the reasons Selenium was the wrong tool. There are many technical challenges when it comes to Selenium. Some are simple, and some are downright prohibitive. From both technical and manpower perspectives, Selenium wasn’t able to meet our testing demands. A few of the biggest deal-breakers are as follows.

Handling change

Selenium very much struggles when handling selector position and functionality changes. This is a well-known and oft-cited problem with Selenium. Whenever we make an even tiniest change of selector position, Selenium tests fail. The failure arises because Selenium can’t recognize when a selector has moved, even when the functionality hasn’t changed. This is also true when the functionality of a selector changes slightly. For example, adding a new menu option will break every test related to that menu.

What’s more, this struggle with selectors and functionality relates to another big problem with Selenium. Selenium scripts are built on unforgiving comparisons. If anything “looks different” between scripting and testing, the tests will fail. For example, Selenium scripts can’t handle something as simple as changing the order of elements in a menu.

Complex failures

As a consequence of these inconsistencies, Selenium tests often fail early and break late. That is to say, an action that should have failed will succeed and you won’t know it until a much later test. Countless hours (or even days!) of test time will be wasted. Not only that, Selenium doesn’t have a tool for root cause analysis of the error. You know the test eventually failed, but not when or where the cause was.

Furthermore, Selenium also doesn’t provide any form of automated test reporting. To capture test failures in Selenium, you must take a screenshot at the moment of failure. This is far from the readable format that the team needs to quickly diagnose the problem. Selenium has to rely on third-party solutions for test reporting.

Why I moved to Functionize

Having given up on Selenium, I needed to find an alternative. After some searching, I came across Functionize. There are many ways in which I found Functionize to be technically superior to Selenium. Way too many to list them all here, as much as I’d like to. Three of the most important reasons to switch to Functionize are its low-code testing, machine learning, and test reporting.

I know, low-code testing is supposed to be the bad guy. After all, if I can’t see lines of code and go in and manually change it, how can I possibly know the test worked? Turns out, this simply isn’t the case. Not only does Funcionize’s low-code testing and UI work; they are miles ahead of the tools I was using before. With Functionize, my testers don’t need a separate IDE, don’t need to know several programming languages, and don’t have to be computer engineers. 

Functionize also makes use of machine learning to solve so many of Selenium’s shortcomings. By using smart image comparison, Functionize has no trouble adapting to selector changes. Gone are the over-precise comparisons that plague Selenium. Additionally, Functionize also gives my testers smart screenshots, which show me the before and after of every test. I don’t have to fight with fixing timing errors with complicated delays in page loading. This comes in extra handy, as it provides the root error analysis that Selenium lacks. 

Lastly, Functionize automatically provides very, very detailed test reports. Even better, these test reports are available to the entire testing team. Without taking precious time away from testing, my team can view what the entire team is testing or is planning to test. Functionize provides test reports including everything from the complete test history. Functionize gives reports that ell my team who has made changes, when, and the outcome of the change. The cross-team knowledge Functionize gives eliminates so much time wasted on poor or incomplete communication.

Why I recommend Functionize to my fellow quality engineers

At the end of the day, I just had to change my testing mindset. In a stunning turn of events, testing doesn’t actually have to be “hard.” Making the switch to Functionize made more room at the “testing table.” Because of its remarkable low-code testing capabilities, my testers don’t have to be seasoned engineers. My testers don’t need to learn any new languages or syntax. What’s more, my testers don’t need to learn any new programming languages. This feature is particularly helpful when weighed against the sheer number of languages on which Selenium relies (something I saw as a benefit in my youth).

Free isn’t really free

At the end of the day, though, my decision was cost-related. Selenium is free…but is it, though? How many Selenium tutorials are there that a new tester has to watch? How long do new test engineers have to spend on these tutorials before they even become productive? How many plugins do you need to add, subtract, learn, etc.? How many plugins are in Python, C++, C#, etc? Selenium has hundreds of step-by-step YouTube tutorials, a topic on Quora with 48k followers, and 84k questions on StackOverflow. As an “old school” engineer, that sounds great! As a manager, it sounds just plain awful.

Anyone leading a team will tell you right away that their biggest expense is their people. Sure, a Functionize license isn’t free, but neither are the thousands of man-hours spent every year simply learning Selenium. Even if new testers are familiar with Selenium, they won’t be familiar with your Selenium. They have to spend days or even weeks learning how you test with Selenium with your plugins in your language. 

So this all adds up to Functionize being better for me both technically and financially. If I could get back my first few years as a manager, I would start with Functionize from day one. To see how Functionize can do the same for you, sign up for a demo today.