Functionize as a User Part 4: Test Management Through Machine Learning
In the final blog in this series on learning to use Functionize, I look at how I can debug and edit my tests while they are running live
Welcome to part 4 of my series on Functionize as a new user; the first part can be found here. In my previous posts, I discussed how I used Functionize to perform full-scale QA testing of my application and how I analyzed the results. Full-scale testing for me meant lots of tests generating lots of data. More tests and data than I could possibly track manually. Therefore, this final post will talk about how I used the Functionize platform to automate and maintain my tests.
Introduction
As a robotics engineer working with simulators, the applications I create can often be hard to use. Therefore, I need a user-friendly web UI frontend for my customers to use to access my software products. I found this application with robotbenchmark, but to be successful I needed to rigorously QA test the interface.
Naturally, as I continued to test features, I found myself with an ever-increasing library of tests. Fortunately for me, Functionize is built from the ground up to make test automation and maintenance easy. The Functionize platform made the care and keeping of my tests so simple, I was able to perform all the necessary tests myself—no IT support needed! Not only that, I was able to complete the full QA testing in only a few weeks with no prior testing experience. For this final article, I will show how I was able to use the Functionize platform’s test maintenance and automation features to quickly and painlessly finish my testing.
Intelligently executing my tests with ML
Plunging deeper into the Functionize platform allowed me to streamline my testing. The platform allowed me to test my application’s features much more rigorously than I ever could on my own. I was able to execute more complex testing without having to do anything more complex. Or put another way, I did not have to get any smarter at testing to make smarter tests.
Remember when my test for creating a new user failed in part 2, and the platform showed why in a matter of minutes? Well, in this part, I will show how the Functionize platform can show all that happening in real-time. What’s more, the Functionize platform live testing and debugging features will flag test failures and provide solutions as I test.
After creating my first tests, I used the Functionize platform to manage them with Test Health Reports. Over the course of many test iterations, my tests learned what to look for during each execution. And when the test didn’t find it, the Functionize platform showed me where, how, and why the test failed. As I added more tests, I made use of three main platform features, which I describe below.
Live Execution
The first feature I used to upgrade my testing was the Live Execution feature. As the name sounds, it allowed me to watch my tests run in real-time. With Live Execution, I got to see exactly when my application failed—what step or verification threw an error. I wasn’t stuck waiting for an entire test to execute, only to sift through the test’s outputs to suss out why the test failed. I simply sat back, watched the test, and let the Functionize platform tell me right away when something broke.
Another powerful feature of Live Execution is my ability to edit the test execution. For example, if an intermediate step failed, but I think the subsequent tests will pass, I can suppress the failed test. Or, I can rearrange the order of tests on the fly. Additionally, I can discover any missing steps and add them. By watching the test live, I can find and fix the bottlenecks and roadblocks in my testing—what’s slowing me down or stopping me.
The picture below shows an example of Live Execution.
Live Debug
Watching a test run in real-time both saves me time and provides me with valuable information. However, this feature is at its most effective when I can also change the tests in real-time. Not surprisingly, the Functionize platform also provides this capability, known as Live Debug. While watching a live test, I can fix the tests that failed as I go. No longer am I trapped waiting for a test to finish execution, only to find out it broke two steps in. What’s more, with ML, the Functionize platform Self Heals by finding and suggesting fixes for the broken tests, as I test.
By the time a previously successful test suddenly fails, the Functionize platform has already learned the “ins and outs” of that test. Through Self Healing, the platform not only flags the failed test but also offers me solutions. Solutions that I can input during testing. I am free to move forward with my testing without making incremental changes and hoping that this time will work. I can execute the test, watch the test, debug the test, and rerun the test, all in real-time. What’s more, I can do all this using a platform and UI that I only had to spend a couple days learning to use.
Smart screenshots
Smart screenshots are essentially the Slider View on steroids. Remember the Slider View showed me a “manual gif” of test outputs? Well, with smart screenshots, I can interact with the “frames” of my gif. Smart screenshots let me edit both the test’s attributes and actions within the Slider View. For example, I could change a verification element, or add a new verification step.
The true power of Smart Screenshots is how it lets me edit and execute tests. Because I am working with recorded screenshots, test parameters are remarkably easy to change on the fly.. To execute my new tests, I run them from within the Functionize platform. The platform creates a clean instance of my browser and runs my tests—no new VM required. This feature makes editing and running tests extraordinarily fast, which equals more testing in less time.
Below is an image of Smart Screenshots in action.
Test Maintenance
The more testing I do, the more tests and data I create. Too much to maintain the good, old-fashioned way, adding a v1, v2, v2.1,etc. behind each updated test script and then putting entirely too much faith in my memory. Therefore, I needed a way to manage my tests—their run history, changelogs, action logs, etc.
With the Functionize platform, all my tests were clearly and precisely presented inside my project. I can learn at a glance when my tests last ran, on what browsers, and with what outcome. Additionally, I could both view previous test versions and revert back to previous revisions. The test management feature further let me reinstate a deleted test. Similarly, I can lock the test to prevent further edits. Furthermore, I can customize a multitude of other test features, such as what errors to ignore, what errors to flag, and how I want to be notified of the test results.
The Functionize platform keeps all of my tests in one place and gives me quick access to all the associated information. I didn’t have to worry about local storage and tracking test results and edits. When I bring on more people to my testing team, I also don’t have to get them “up to speed.” The platform tells them right away what QA testing we’re doing, why we’re doing it, and how we’re doing it.
Why I will stick with Functionize
The last few weeks I have been on a journey triggered by realizing I needed a better, more reliable way for customers to interact with my software. I began my journey by searching for a QA testing platform. I needed something simple to use, as I didn’t have the time to learn new skills. On the other hand, I needed a tool capable of complex UI testing. I needed an easy way to do hard testing. Fortunately, I found this tool with Functionize. The Functionize platform made it easy to create tests, organize them into orchestrations, and execute the tests. Even more to my benefit, all these tests lived and ran on the cloud—I didn’t need to create any VMs or get anyone from IT involved.
During my journey, Functionize has exceeded my expectations at every step. I started with no knowledge of QA testing at all, yet creating and analyzing tests was easy. With the Slider View, I could view my test results every step of the way. I didn’t just “see” my test outputs, I could actually see my test results through screenshots of my application. Even better, I found that I watch all my tests running live, and even debug them on the go.
A few weeks ago I knew nothing about QA. Now, I have become a master QA tester. And thanks to the tools and knowledge gained using Architect and the Functionize platform, I was able to do so in less than 30 days. All of this triggered by my decision to try out their free trial.
