Functionize as a User Part 1: Creating and Running Tests
It has been my privilege to work in machine learning and AI for over a decade as a robotics engineer. In that time, I have seen machine learning grow from basic image and language processing algorithms to powering fully-fledged software tools. Thus, I was intrigued to find AI being used to enhance software test automation. Here, I describe my experiences with using Functionize as a new user.
My testing history
Like most young engineers in the 2000s, I had my tried and trusted testing method: cout statements. Need to know the value of a variable after a certain step? Print to screen. Need to know if this other variable is a NaN? Print to screen. However, the projects I execute have become more complex over the years. So much so that my approach to testing became completely worthless.
My brute force method worked fine for small problems and stand-alone projects. Nowadays, the projects I work on are much too complex to test like this. Not only that, the software I create now generates a lot more than just numbers. In particular, I find myself relying more and more on web applications these days.
Using my naive approach, each time my code runs, thousands of tests and outputs would need to be created. All of which would break every time I changed the code. Moreover, the approach is utterly unsuitable for testing web interfaces and the like. I knew something had to change—I understood the core idea of QA testing, yet I had no idea what QA really meant. Not only that, I also lacked an understanding of how to approach it in an intelligent fashion. All I knew was that I needed specialized tools for testing.
My “system under test”
Before I dive into how I started creating and running tests, let’s look at my application. Ask any self-driving car engineer where they do the bulk of their development, and the answer will almost always be “simulations.” Self-driving cars are expensive to build and test. VERY expensive. And these machines are often fragile and easy to break. Thus, most developers turn to simulations for design and testing. For my application, I am creating a web interface into a custom simulation package. The goal is for engineers to be able to set up and execute simulations from their browser. Not every engineer wants to be a modeling expert, just like I don’t want to become a test automation expert. So, I am using Gzweb, which lets me create my own custom interface for my software.
Test automation
So, how to test this new interface? I couldn’t use my old approach but I quickly realised that my savior would be test automation. This would allow me to create tests that ran each time I updated my code, and it wouldn’t be fazed by UI testing. My search for suitable utilities started with looking at the open source community. I quickly landed on Selenium. While I immediately fell in love with the idea of Selenium, I found the reality was daunting. You see, my problem is twofold. First, I am a software engineer, not a tester. I don’t want to spend weeks learning a new programming language. Second, I don’t have time to learn the “tricks of the trade” that QA engineers use.
Selenium was just too intimidating of a tool to approach from my background. For starters, I would have to learn some QA testing fundamentals to get its full benefit. Moreover, I would have to learn some QA testing programming fundamentals. To top it all off, if I tried to automate my tests the wrong way, I would do much more harm than good. More than anything, I needed a testing platform I can pick up and use without having to invest valuable time into learning, and can automate my testing intelligently.
Finding Functionize
My search for easier-to-use alternatives brought me to Functionize. Their Architect is powered by the same AI techniques my code is generating and that lets it shine where Selenium is lost in the mists. The Functionize platform empowered me to take charge of my testing using the knowledge I already had. As I explored the Functionize platform, I discovered some amazing capabilities:
- I was able to create and run tests right away with just a few clicks of the mouse.
- I could start testing on day one, without having to learn new software programming skills.
- I could test much more efficiently because it brought me straight to full test automation.
- I could rely on self-healing to detect and flag errors in my test and let me know.
- I could use the Natural Language Processing feature to create tests easily without learning new language or syntax.
- I could integrate Functionize with existing web tools and communities, in my case Slack.
In brief, Functionize used machine learning to automate my QA testing in a way that helped, not hindered. Armed with my newfound knowledge of QA testing and why Architect and the Functionize platform were right for me, I went straight into creating and running my first tests.
Creating Tests
Right out of the gate, I noticed how easy it was to use Architect to create and record tests. Architect was not only easy to download, it also runs right inside Chrome. All I had to do was sign up and install a Chrome plugin. Once installed, I could load my application, hit record, and start testing. That’s it. No IDE to learn, no scripts to write, no output files to format. Below is an example of creating a test in Architect. Of important note, my interface is still in development, so for this example I was more focused on learning the Functionize platform and the functionality of the Gzweb interface.
To create my first tests, I took the following steps:
1. I logged into my Functionize account and created my first project. In this case, the website I wanted to test was Gzweb.
2. I created a new test under my project in the Functionize app, and launched Architect from inside Chrome. Once I started creating my test, Functionize took me to the Gzweb page and started recording.
3. I saved my test, and created a few more to get the hang of test creation.4. Finally, I used the Test Orchestration feature to determine which tests to run automatically, and in what order.
And that was it; my tests were ready for running.
Running Tests
Once I installed Architect and created my tests and test orchestration, running them was just as simple. The Functionize platform made it easy to pick and choose which tests I wanted to run. I could run a single test or multiple tests without having to change any setup files. Not only that, I could control the execution of the tests. I could run them in the foreground or background, with background execution allowing users to still use the site during testing. Furthermore, I could set the tests to run either in order or concurrently. I could even choose which browser or device to run the test on, all without needing to do anything other than select a few dropdowns.
Seeing the results
Functionize also gave me a really simple interface to understand the test outcome. For example, running my first test gave the below results. As you can see, each step executed correctly, and you get an accompanying screenshot of what the site looked like at the selected step.
In my case, I was testing different actions that were not connected. Therefore, using my Orchestration, I could execute both my tests at the same time. The execution of my orchestration provided the below results.
And with that, after only an hour or so of work, I had my first tests and test orchestration created and executed.
Lessons Learned
I discovered three main takeaways from my first test creation and execution.
- Functionize made it incredibly easy to leverage test orchestrations. Test Orchestrations were a new concept to me, but I quickly learned of their importance. An orchestration is like an outline for your testing. Or, rather, a playbook by which your tests are executed automatically. Orchestrations are a powerful way to organize your tests and execute them in bulk. With the Functionize platform, I could quickly crate tests and organize them into my playbook. Once organized, I could run all my tests in the order I needed without having to babysit.
- I could test my application across browsers. Cross browser functionality is critical to the success of any modern site or application. For my needs, cross browser functionality was necessary because I never know how my partners access the internet. My engineering peers use a variety of browsers, including Chrome, Firefox, Edge, or Safari. Cross browser functionality lets us collaborate without forcing everyone onto the same browser – telling an engineer they can’t use their preferred piece of software is a surefire way to make them unhappy and/or obstinate.
- Everything in Functionize runs on the cloud. Remember how I said I was unwilling to learn new software skills? That also applies to also learning how to build and maintain virtual machines (VMs). Using Functionize precludes any need to set up my own VMs. Previously, I was used to asking IT guys to help setup VMs for testing, or to (gasp) procure physical servers for testing. So, executing using the cloud is great, especially since there’s no need to connect to a separate cloud infrastructure tool (eg. like SauceLabs) to run the tests. That also helps save cost since Functionize provides their “test cloud” as part of the subscription.
Conclusions
I have suffered through many a year of my own sad attempts at QA testing. Finally, as my software applications transcended simple math to complex applications, I gave in and realized I needed a new way to test. Now that my software has “hit the web” and has a proper UI, I had to find a way to perform “real deal” QA testing. I desperately needed a test platform that was intuitive, efficient, and did not require me to learn new skills.
I found that tool with Functionize Architect. Architect uses the ML I know and love to enable the QA testing I badly needed. In this article, I walked you through my experience creating and running my first tests. In my next article, I will share my experience analyzing my test results.
To follow along with your own free trial, visit https://www.functionize.com/free-trial today.
