The limitations of test scripting
Automation testing based on scripts has a number of serious limitations. The upshot is that writing and maintaining tests is so effortful few companies ever achieve 100% test automated testing coverage.
Hard to create test scripts
Legacy test scripting requires you to be able to write complex code. As a result, it takes skilled Test Automation Engineers or Developers in Test to get scripts right. Even a simple test script will take many hours to write and debug.
Test recorders can simplify this process, but the scripts they produce are more brittle. Once you record the script you still have to debug it manually.
Slow to adapt for cross-browser
Web applications are expected to work on any platform and browser. So cross-browser testing is essential. Adapting legacy scripts is painstaking and slow. Essentially, the script has to be modified or rewritten for every different combination of platform and browser.
It can easily take a team of competent test engineers days to produce just a few legacy scripts that work cross-browser. And once created they are only comprehensible to other test engineers.
Unsuitable for modern UIs
Modern websites and UIs contain rich, dynamic content. They frequently import content such as a PayPal Buy Now button, or a newsletter sign up from 3rd parties. These are embedded inside a set of nested DOMs (Domain Object Models), the logical containers within your application’s UI.
Unfortunately, legacy scripts can only locate and interact with elements in their own DOM. Accessing embedded objects requires complex XPath queries that take skill and effort to construct. And if the embedded object changes at all, then every query has to be rewritten.
Need significant maintenance
Legacy automated testing is extremely unstable. Every change to your UI risks breaking all your tests. Even a simple CSS change can cause all the selectors on a page to change, making it hard for the script to select the right elements.
If a script fails to find an element at all it fails instantly. However, sometimes it can select the wrong element. This causes you real pain as these failures don’t show up until many steps later. This could even invalidate the test completely. Either way, your engineers have to track back to find the original cause of the failure and fix it by hand.
Intelligent automated software testing
Here at Functionize, we delegate automated testing to our intelligent test agent. This agent uses a combination of different forms of artificial intelligence to deliver exceptional results. Our aim was to create a system that is accessible to anyone. Automated software testing shouldn’t require highly skilled developers in test.
Our system is based on two core technologies. The first is Natural Language Processing which leverages natural language processing. NLP achieves our aim of making plain English the new language of testing. Using NLP means you can write test plans in plain English using both structured and unstructured text. These test plans are then passed to our ML Engine engine. This uses a combination of several types of AI to create a functional model of your UI. The test plans are used during the modeling phase to tell ML Engine how your site works. Finally, they are converted into fully-functional, self-healing tests.
Automated Test CreationThe benefits of Functionize
Functionize’s intelligent approach to automated software testing brings numerous benefits.
Making testing accessible to non-experts
NLP allows anyone to create a test simply by writing a test plan in plain English. This means anyone on your team can help create tests, especially those skilled manual testers who were left behind by automated testing. Furthermore, our intuitive UI and visual representation of test steps allow even lay people to understand and analyze test failures.
Increasing the velocity of test creation
Functionize’s modeling approach allows you to write and create large batches of tests in just a few days. As an example, writing 50 test plans, modeling them with ML Engine and running the tests across all browsers takes just 3 days. By contrast, doing the same with legacy automated testing systems would take a minimum of 35 days.
Testing can be done faster using the cloud
Functionize has adopted a cloud-first approach to testing. The Functionize Test Cloud is based on Google Cloud. It allows you to execute thousands of tests in parallel. This dramatically increases test productivity and has a direct impact on your overall software development process.
Closing the gap between the product and testing team
Test planning is driven by business analysts in the product team. They define exactly what tests are needed, and what the outcomes should be. With manual test plans, the BA can verify that tests are correct. However, legacy automation testing tools make this impossible.
Removing the need for test maintenance
Test maintenance is said to take up to half of a quality engineer’s time. But ML Engine includes features that reduce the need for test maintenance to almost nothing. This is because ML Engine knows how your site should work, so minor changes to the UI or styling won’t affect it. For more complex changes, it uses Root Cause Analysis to identify the likely problem and suggest how you can fix it.
Detailed statistics allow you to track performance
Every step in every Functionize test is instrumented allowing you to see key statistics and monitor performance over time. One of the most useful statistics is our proprietary Visual Completion Time. This measures the point at which a real user would be able to interact with your site. We also take screenshots before, during, and after each test step, showing you exactly what changed.
