Alternatives to Selenium IDE

As Selenium IDE’s support within Firefox 55 and onward has been phased out this past August, IDE has found new life with Chrome. While this is great news for existing users of IDE, who understand the practicality of the tool, for those QA engineers and developers who are still hoping that IDE will be a complete, standalone automation solution, that optimism is misfounded. According to the Selenium council: “[IDE] is not designed to run your test passes nor is it designed to build all the automated tests you will need. Specifically, Selenium IDE doesn’t provide iteration or conditional statements for test scripts. At the time of writing there is no plan to add such things. The reasons are partly technical and partly based on the Selenium developers encouraging best practices in test automation which always requires some amount of programming. Selenium IDE is simply intended as a rapid prototyping tool”1

Plugins that extend IDE

There are dozens of supported plugins that extend IDE’s abilities beyond those of a rapid prototyping tool, including Flex Pilot X, Favorites, File Logging, Flow Control, Implicit Wait, SeLite Commands, SeLite Bootstrap, and many more. Additionally, there are also dozens of 3rd party plugins that are not supported or hosted by the Selenium project, further illustrating the user demand that remains surrounding extending IDE’s core functionality into a truer automation tool. As capable as some of the 3rd party plugins are, very few work off-the-shelf and are dependent on developer centric installations and runtime solutions. Open source projects like Selenium represent a tremendous asset to the testing community at large. But open source communities are often not-so-great at delivering software that’s readily consumable by people lacking -- not just coding expertise, per se, but direct project involvement. 

Limitations of linear frameworks

Linear test automation frameworks, often recognized for their recorder and playback abilities, allow for rapid creation of test scripts with no scripting and with little to no planning. Microsoft’s Visual Studio Team Edition is an example of a recorder that generates long code segments, most of which consist of instructions that examine fine-grained details of the Document Object Model. So very little of this code, if any, is functional logic, divorced from page-level detail and hence re-usable. When tests break, the usual practice is to re-record large parts of them. 

For this reason, some developers have even concluded that manual scripting (i.e., without recording, but using a more sophisticated language with real conditional branching and other features) is a better idea. At least a manually-composed test script can segregate its functional logic from object recognition, enabling some reusability. 

Keyword-driven testing

Keyword frameworks like Robot were developed in response to the conspicuous limitations of linear frameworks that focus on recording and playback like IDE. Keywords are reusable, lower-level tests that execute a function, like “Login.” These keywords usually do not need to be created and are already provided by keyword libraries, and thus made available to a testing framework. With keyword frameworks like Robot, reusable portions of code can be distributed into test function libraries, callable from conventional languages like Java and Python, enabling the reuse of keywords and abstracting away the technical encumbrances of testing automation. This kind of modularity enables much more efficient test creation and the ability to create coverage faster as those keywords are abstractly layered into the automation. For example, instead of having to script 100 unique logins with different names, with keyword modularization, only one coded statement calling for login is needed to execute that test scenario at scale. While a login is a simplistic example, a keyword is able to contain much more complex logic and apply those keywords in a reusable manner, and avoid code duplication. 

Data-driven testing

As testing frameworks have evolved, the goal to reduce test maintenance has remained paramount. A Data-Driven framework succeeds in reducing maintenance by separating the storage of data from the script itself. In using our above example of a login, the user and password of the inputted data would be stored externally, and the test script is commanded to read the stored data on command. This technique vastly reduces the need to rerecord tests from the beginning when data changes, and automation is possible as long as the stored data is properly synchronized with the control script. Unfortunately, this type of framework requires very proficient programming in scripting languages, and superior management care as it relies on accessing the stored input data. 

Scriptless testing

Most yesteryear scripting automation seeks to identify page objects through an archaic scanning process. Some portions of an application's GUI may be inventoried (in terms of all kinds of properties) and then be made available for the application of some sort of operation (e.g. point-and-click input) or test function (e.g. isEnabled, hasValue). The available types of operations and assertions per GUI element are thus part of the object map. However, scanning a page and creating a symbolic map of its objects, along with a light-duty API that lets you query page elements in Boolean ways is insufficient to build complete tests and requires a procedural or other framework (e.g. a script language) to get something useful. But that reasoning is circular, because there are other ways for automated tests to self-orient and access page objects. And these new methods are exactly what sets Functionize apart -- leveraging machine learning and related technologies -- permit all kinds of efficiencies, from self-healing tests to re-usability of functional test logic. 

1 https://www.seleniumhq.org/docs/01_introducing_selenium.jsp