Mobile app testing with Functionize’s intelligent test agent

Mobile testing is critical for many businesses. You may be developing a native app or just want to know that your web application or site works on mobile devices. Either way, you need to do extensive quality testing across a range of mobile platforms. The traditional method for doing this was to manually test on each platform. More recently, platforms like Appium have allowed automated testing of mobile apps similar to that done by Selenium. Here at Functionize, we believe these methods are obsolete. Our intelligent test agent allows you to test any application on any device with no need for scripting or cross-platform modifications.

The problems with legacy mobile app testing

The legacy approach to mobile app testing has a number of major issues. Let us look at these in detail:

Large device inventories

Tests self-heal to take the burden of maintenance off the user and onto the AI.

Slow to perform tests

Manual testing is slow, even with the most efficient test engineers. This makes it unproductive and has a direct impact on your release cycle. As you need to repeat the same test across a dozen or more devices, you start to get mistakes creeping in.

Clunky test automation

Appium-style testing tools for mobile app testing is very simplistic. It shares all the problems of Selenium (slow and complex to create tests, hard to select elements, needs constant maintenance, etc.) But it also has its own issues such as the overuse of Xpath selectors hitting performance.

Slow emulation and simulation

While Appium offers the chance to automate your mobile testing, it still requires mobile hardware to run. The alternative is to use device emulators, such as those included in Apple’s Xcode. However, such emulators (and the related simulators) run slowly and are inefficient.

How can Functionize help?

The power of intelligent mobile testing

FNLP is our natural language processing engine that takes test plans and converts them into tests. Test plans are written in plain English, so anyone on your team can contribute (unlike Appium, which requires skilled developers). These plans can be unstructured text (like the user stories your product team uses). Or it can take structured test plans, such as those produced by test management systems. In both cases, it compares the test plan with your UI and works out what is meant to happen in the test.

Natural Language Processing

NLP is our natural language processing engine that takes test plans and converts them into tests. Test plans are written in plain English, so anyone on your team can contribute (unlike Appium, which requires skilled developers). These plans can be unstructured text (like the user stories your product team uses). Or it can take structured test plans, such as those produced by test management systems. In both cases, it compares the test plan with your UI and works out what is meant to happen in the test.

ML Engine

ML Engine is the brains at the heart of our intelligent test agent. It combines multiple forms of artificial intelligence to learn how your UI really works. When you pass in a set of test plans via NLP, these act as instructions to teach ML Engine about your application. It uses these to build a complex model of your entire application. This model takes account of hundreds of data points for every element within the UI. 50 test plans can be modeled in just a day, and once complete, each test can be run on any platform without modification.

Visual testing

Analyzing the results of Functionize tests couldn’t be easier. For every test step, ML Engine captures before, during, and after screenshots, highlighting any unexpected result on the screen. This approach has two big advantages. First, anyone can verify that the test is doing what it is meant to (something you can’t check in Appium). Second, anyone can instantly see any test failure and can then drill in to see more details (including the history of previous runs for that test).

Self-healing tests

All Apps and UIs go through multiple changes in their life. Often, these changes may be simple visual or style changes. Other times, they may involve moving elements on the screen or rearranging menu items. These sorts of changes cause real problems for any script-based mobile app testing system because they change the selectors for the UI elements. This results in legacy test scripts undergoing constant maintenance. Our tests are different. Elements are selected using machine learning and a complex set of descriptors that act like a fingerprint. Even when things change, the element can still be identified. This means Functionize tests are self-healing.

Root Cause Analysis and 1-click Updates

Some changes are more fundamental and affect the functionality or introduce ambiguity to the test. Here, test failures may not show up until many steps later in the test. In these cases, our Root Cause Analysis engine will identify the most likely cause of the failure. It then uses the history of that test to come up with likely fixes which it tests. These are then ranked and presented to you to choose the best one. You can then select this with 1 click, and the change will be learned and propagated through all the tests.

Nested virtualization

Virtualization is where you emulate a server within a host system. Services like Google Cloud and AWS are built on virtualization. Generally, when we refer to virtualization we actually mean paravirtualization (where you provide access to the underlying hardware via drivers within the host operating system). Virtualization used to be very inefficient, but CPU extensions like Intel VT-X significantly improved performance. However, until recently, it was hard to virtualize mobile operating systems and hardware, particularly within public clouds. It could be done, but it was inefficient, slow, and lacked performance.

Google changed all that when they launched their nested virtualization platform built on Google Compute Engine. Functionize is a key partner of GCE thanks to the Functionize test-cloud. We were instrumental in driving forward this innovative technology. The result is you can now do performant emulation of mobile devices within a virtual server. We use this to accurately emulate both Android and iOS devices, obviating the need to maintain an inventory of physical devices.