Agile Testing Automation | What it is & How to Avoid Scripting

Automation testing traditionally involved writing a script to test new code commits to an application; in reality, the developer automated software testing for the QA engineers. But this is not automation from the developer’s point of view. This method involves a lot of additional manual coding work and anticipation for the developer. End users, in this case the QA testing team, view automation as a process that runs tests for them, without human assistance. But if the human assistance is provided by developers and is merely concealed from view, then QA and perhaps the enterprise may need a refreshing view of the influence on automation testing of new machine learning methods. In fact, automation that requires scripting or coding is not really automation anymore. Why? Because we crossed a nebulous threshold and entered an entirely new realm of automation with Functionize and its capability to learn testing methods and improve itself automatically. In order to fully appreciate this growing influence, let’s explore the advantages and disadvantages of several traditional frameworks of automation testing.

Coded UI 

Among the stalwart proprietary frameworks one topheavy giant stands out. In our new world of automation testing coded is the antithesis of automated. Microsoft’s Coded UI automation is an API which provides access to the user interface of other applications, thus enabling a virtually unlimited scope of testing possibilities. 1 Coded UI testing is used most often where the UI of an app contains logic for validation, such as in a web application, because in this scenario a human tester must verify that a specific required input passes. It’s a mixture of scripting and hands-full testing. Here by contrast we can see the liberating benefit of Functionize’s learning capability. After Functionize learns a sequence, it will remember the next time, even if the control moves or gets renamed. 

A Coded UI test, called a CUIT, can save time by replacing an existing manual test process. A tester can run an app while beneath the surface the CUIT Test Builder records the tester’s actions and creates code; actually the recording itself is the coded version of the testing sequence. Following a test, the recorded script can be revised by testers using a special editor. Here again, we can see how a fully automated machine learning based application will replace the manual part of this testing and free the tester from learning a specialized editor and scripting language for each technology implemented in the application under test. 

Although the test run itself has been automated, the development of the test process is just that - yet another step of development - it is far from automated. The coming wave of Functionize will deprecate this step, freeing the heavy dependency between developers and QA. This is the sharp edge of what is changing today: the development of testing automation itself is being automated through advanced techniques of machine learning. Smart automation testing will inevitably replace chunks of the current CI/CD model by attrition as it learns. What can we replace now? Let’s explore an example. 

To begin a CUIT test we need Visual Studio Enterprise. That’s a hefty prerequisite, considering that Functionize needs no IDE. If you look at a copy of Visual Studio, you will not see the Coded UI test templates. That is because only the Enterprise product level supports Coded UI. After adding a Coded UI test file to your project, in the Generate Code dialog, choose, “Record actions, edit UI map or add assertions.” Now the CUIT Test Builder appears with the red recording button, which will smack of recording a macro in Excel. After clicking the record button while working within any app, Test Builder creates lines of code, as we will see, which mimic the tester’s gestures. Because CUIT operates inside the .Net Framework testers and developers can read and revise scripts in their choice of either Visual Basic or C# programming languages. This is ideal if all teams involved are working within .Net technologies. Otherwise, it’s complicated. 

A Coded UI test contains a Map which includes the UI under test, methods, assertions, and classes, some of which can be edited, but some of which cannot. A caveat to this limitation is that, although UIMap.Designer.cs cannot be edited in VS Enterprise, it is possible to edit it with another editor. The problem is that the test scripting process is divided into a complex set of files by Test Builder, and a lot of knowledge and experience is required on the part of QA to master the process. And after doing so, the process is still not automated. 

For example, in the following C# script from a web app test recorded with Test Builder, the tester navigates to a page and clicks the search button: 

[TestMethod1]

public void CodedUITestMethodTest()

{
BrowserWindow _bw1 = BrowserWindow.FromProcess(proc);
_bw1.NavigateToUrl(new Uri("https://www.functionize.com"));
EnterSearchValue(_bw1, "CodedUI wo UIMap");
ClickSearch(_bw1);
}

private static void ClickSearch(BrowserWindow _bw1)

{
HtmlInputButton searchButton = new HtmlInputButton(_bw1);
searchButton.SearchProperties.Add(HtmlInputButton.PropertyNames.Id,"sb_forms_go");
Mouse.Click(searchButton);
}

‍

But what if that button moves or gets renamed? The script needs revision. This is a classic hassle; now testing is as complex as designing code for the product. And we have not even talked about disadvantages of Coded UI yet... 

Coded UI maintains a library of classes to handle basic Windows UI events. But there are some ordinary actions, like a mouse hover, which cannot be recorded by Test Builder. Mouse hover can be programmed by shifting methods around between UI Map and UIMap.cs, but the prospect is grim. Nearly all tech support for Coded UI on this subject is posted by MSDN, and there is little community support. Open Source programs, on the other hand, have widespread user based support forums, especially those specialized for web application testing. 

Protractor & Sahi 

Agile-based developer teams deploy engineers to build automation testing. This means testing methods are branching into specializations which include both open source and proprietary tools throughout the test cycle and pervasive in CI/CD pipelines. These testing tools are themselves further specialized toward a particular technology. An example of this acute specialization is the Protractor testing tool, which is used primarily in AngularJS development and testing. 

The focus of Protractor is to capture the action of web page elements written in AngularJS, because this is difficult to do with either Selenium or JUnit. In particular, the ng-model of the MVC and other bindings created with AngularJS cannot be found with a Selenium locator. Protractor is designed to run in the browser and act like an actual user in the testing process. 2 Protractor is a Node.JS tool and so the automation testing code is JavaScript. Returning for a moment to the testing issue of scripting a test for a mouse hover event, Protractor needs the use of the browser.actions() method to script tests on web elements, as in this example: 

‍

var EC1 = protractor.ExpectedConditions;
var productsMenu = element(by.linkText("Products Menu"));
var subMenu = element(by.linkText("test.portal"));
browser.actions().mouseMove(products).perform();
browser.wait(EC1.visibilityOf(subMenu), 5000);
subMenu.click();

‍

The new wave of machine learning based automation testing, as exemplified by Functionize, realizes testing based on computer vision and learned sequencing. This is a much more natural endeavor of testing which requires no scripting. Functionize sees the mouse hover. 

Sahi is a basic browser testing tool with an open source version. Advocates report that Sahi is simpler to use than most testing tools. Scripting is easier to introduce to testing staff, and it runs in all browsers. Sahi supports record-and-playback test scenarios, like most such tools. It supports regression testing as well. A common complaint about Sahi is that all testing scripts must be coded in JS. But as we have mentioned, if you have to script it, then it’s not automated testing. So, how does Sahi cope with web elements such as the mouse hover? 

Sahi uses assertions to verify the functionality of actions performed on a page. To record a mouse over assertion during recording in Sahi we hold the control key down and hover over any HTML element. Next, click the Assert button to generate assertions which then appear in the Evaluate expression box. Once the tester confirms the assertions, the code is added to the script. 

Truly Automated Testing 

An immediate disadvantage to all traditional automation testing methods is that each and every one is limited to testing a target technology; these methods lack true cross-platform visibility, and they are difficult and in some cases outright impossible to integrate. QTP cannot test Informatica for example. Furthermore, QTP tests can only be scripted in QTP’s own IDE, another limiting factor. Likewise MS’s Coded UI depends on the Visual Studio Enterprise IDE. Therefore users with innovative ideas about expanding the testing cycle envelope may stagnate on a learning curve on an unfamiliar IDE instead of following their insights toward improving the app under test. There is no mystery to why MS wants to lock us into its products, the same way Apple has always defeated cross-platform interoperability; but this sort of tech nepotism will only limit a company’s popularity in the future. Open source tools cannot be blamed for such mercurial deeds; but they too are the stopgap measures of an age before truly automated testing, led by smart apps like Functionize. 

Functionize runs on any site because it is a browser extension. No specialized IDE is required. QA members need not master a scripting language to run tests. There is no need for any compatibility concerns. Functionize can test any web application. If a triad of testing tools must be combined to test an application, we are happy to announce that Functionize readily replaces them all.