Legacy test automation isn’t agile
Test automation that is based on Selenium and related scripting tools simply isn’t agile. Tests take a huge effort to develop, test engineers spend half their time on test maintenance, and few companies ever get close to 100% test coverage. Let’s look at these problems in more detail and show how they stop you from being agile.
Tests are slow to develop
Creating a test in a legacy test automation tool takes a long time and requires skilled quality engineers. When things are going well the process looks something like this:
1A business analyst creates the test outline from the documentation, defining exactly what needs to be tested.
2This test outline is converted to a detailed test plan by your test management software.
3A test automation engineer or developer in test now takes the plan and starts to iteratively create the test script.
4After each piece of the script is written, the engineer will test that it performs correctly and debug if necessary.
5Once the whole script is created, the engineer will fully debug it (effectively performing testing on the test script).
6Next, the script needs to be modified to work cross-browser, meaning yet more rounds of developing and debugging. This may even require a complete rewrite of the script if you are unlucky.
7Finally, the test can be run on all the required browsers and platforms.
In the perfect case, this process takes many hours, even for a simple script. And it is common to hear of large teams taking well over a year just to create a couple of hundred tests. It is easy to see that this rate of test creation will become a blocker to your agile development. This is especially true if you are doing something like Scrum or extreme programming.
Tests need constant maintenance
Legacy systems like Selenium suffer from one massive flaw. Every time your UI changes, most of your tests will break. This is true for styling changes and layout changes as well as actual functional changes. This is because all these changes will alter the selectors on the page that your test script uses to choose the correct element to interact with. As a result, all your tests need to be constantly maintained. Test maintenance means rewriting and debugging your tests again, including debugging those changes in all the cross-browser tests. This process can soak up half your test automation engineers’ time. Obviously, this is not good for agile testing.
Lack of test coverage
The two issues above combine to make it almost impossible to reach 100% test coverage for your automated tests. This is a particular problem for agile development teams because it means testing becomes the blocker to progress. Tests that aren’t automated have to be done manually. This means employing more test engineers and accepting that testing won’t be happening 24/7. Testing ends up taking 30% of the overall development cycle.
How can Functionize help?
Agile testing with an intelligent test agent
Functionize’s intelligent test agent is an enabler for true agile testing. It combines multiple forms of AI to give you the perfect regression tester: focused, tireless, and efficient. It uses two key technologies: Natural Language Processing and ML Engine. Together, these make it possible for even an unskilled person to create 50 new tests in as little as 3 days. These tests will self-heal when your UI changes, work across any browser and platform and produce results that anyone can interpret.
Plain English is the new language of testing
One of the core tenets of our Manifesto is that plain English should be the new language of testing. With NLP, we have made this a reality. NLP uses natural language processing to convert test plans written in plain English into functional tests. These test plans can be written in structured text (like the plans produced by test management systems) or in unstructured text. The upshot is, creating a new test takes just minutes – truly agile testing!
Tests should never break
Another plank of our Manifesto is that tests should never break. With ML Engine we are very close to this. ML Engine constructs a detailed ML model of your website, which allows it to learn how it is working. It uses a complex combination of factors to identify every element on the page. As a result, tests Self-heal when you make changes that don’t affect underlying functionality (e.g. CSS changes, layout changes, button naming, etc.). Even when functionality changes, and triggers a hard-to-track failure, ML Engine can cope using its Root Cause Analysis capability. This acts as an intelligent manual tester to locate the most likely cause of the failure. It then tests possible solutions and presents you with the best one. Less time spent maintaining tests means more time spent running tests, thus making the process more agile.
Testing must be cloud first
One of the most powerful features is the Functionize Test Cloud. This allows you to run thousands of tests in parallel, testing every combination of browser and platform at the same time. Furthermore, because every test runs in its own virtual server, this gives much more realistic results when interacting with your backend. The upshot is, you can now run your entire test suite in a day, essential if you are agile and need to release quickly.
