Cloud Testing: What, why and how?

What it is, why you should care and how to do it.

Advice on moving your test infrastructure to the cloud and how Functionize Cloud helps

In this blog we look at the benefits of cloud testing and explore how you can create a test cloud. Then we introduce you to the Functionize Test Cloud and show how this can transform your testing.  

What is cloud testing?

Traditionally, test automation was done on your own servers and infrastructure. If your resources are limited you could run Selenium on a single server, cycling between OSes and browsers as your tests require. On the other hand, if you have more resources, you might install Selenium Grid with one server as a hub and half a dozen servers as nodes, thus allowing you to run multiple tests in parallel.

This approach works perfectly well for many companies, But, as with any other server-based infrastructure, this can be readily (and more affordably) duplicated in the cloud. Which brings us to cloud testing. Basically, any test automation solution where the testing is being run on virtual infrastructure in the cloud.

What are the benefits of cloud testing?

So, why should you move your testing to the cloud? Well, there’s actually lots of good reasons. Let’s look at a few of them here.

Cheaper. Renting cloud infrastructure from an IaaS provider is almost always cheaper than relying on your own servers. Moreover, when you have no tests to run you won’t get charged for servers that are lying idle. 

Scalable. Cloud is nearly infinitely scalable. The only limitations on scaling your tests are likely to come from the underlying test automation software (and from the resource limitations of the VMs you are using).   

Reliable. Virtual servers always offer some level of reliability. That could be cold migration (where you power off, start a new instance and recover using the backup). Or it could be hot migration of running servers. The better providers even offer forms of Disaster Recovery. This is in marked contrast to what happens if your own server fails. 

Realism. This is one of the more subtle benefits of using cloud testing. Generally, you will be able to run your tests from multiple locations and, importantly, your tests are coming from an external network. This can avoid embarrassing issues like the one I faced a few years ago, when we discovered our app suddenly stopped working when you tried to access it from outside our own network!

How can I create my own test cloud?

Setting up your own testing cloud is perfectly feasible. But doing it without the support of a dev-ops engineer may prove quite painful.

Moving your test infrastructure to the cloud does require a certain amount of effort. Setting up your own testing cloud is perfectly feasible, and we will briefly explain how you can do it here. However, you may find that without the support of a competent dev-ops engineer, it can be quite a painful experience.

The first step is to choose a provider. Needless to say, there are thousands to choose from. Not only the usual suspects (Google Cloud, AWS and Azure), but also vast numbers of smaller providers often offering very competitive prices and good customer service. The important thing is to make sure you are actually being given what you need. Namely, virtual servers (a.k.a. virtual machines or VMs) that you can configure with the correct OS for your test software. In the case of Selenium you are pretty free to choose the OS. 

Having found a provider, you then need to choose your base requirements. Things you need to consider include:  

How powerful do the VMs need to be? Most providers allow you to specify the number of (virtual) processors, the amount of RAM and the storage per-VM. 

Do you need additional storage that can be accessed by all VMs? And do you also need it to be accessible externally? 

How many VMs will you (usually) need? Remember, many providers will allow you to scale up on demand if you need additional VMs. 

How would you like to network them? Will you need your own virtual switch? Do you need to pay for external IP addresses for each VM? 

How much data will you need to transfer in and out? Think about the data needed for testing your app, as well as the data needed to access test results. 

Where physically would you like your VMs? Many operators allow you to pay a premium to gain some control over things like rack-locality of your VMs.

Setting up the software

Now you have a set of blank VMs. Usually, the provider will give you these pre-configured with the OS(es) of your choice. Occasionally you may need to actually install the OS yourself. However, you will probably need to do some admin tasks, setting up users, checking that you can mount the correct storage, etc. At this stage you also need to start thinking about your backup strategy.

Login remotely to each of your VMs and start to install Selenium and Selenium Grid. They will also need some additional dependencies such as the Java JDK, Chrome and Chrome-driver. Hopefully you remembered to allow external network access in order to download the packages… As this isn’t a tutorial on installing Selenium, we’ll skip over the actual steps. But you can find good resources online to help you.  

Finally, once everything is installed you can start the process of transferring your local setup over to the cloud you just created. But almost inevitably you will face a degree of pain getting it to work. By now you’re probably getting the idea that this may not be as straightforward as you first thought! Fortunately, there is a very easy solution. 

The Functionize Test Cloud

Functionize is leading the way in creating test automation built using genuinely intelligent AI. Our system allows you to create tests from plans written in plain English. Tests are self-healing, and you are presented with detailed results including screenshots before, during and after each test step, along with in-depth analysis of the performance of your system.

At the heart of all this is the Functionize Test Cloud. We have partnered with Google to offer a testing solution that scales globally, offering parallel execution across multiple browsers including mobile browsers (beta). You also get access to features such as our unique load testing and our visual API explorer for API testing.

Why should I use the Functionize Test Cloud?

There are several compelling reasons to use the Functionize Test Cloud. Space doesn’t let us list them all, so let’s look at few of the better ones.

First and foremost has to be the convenience. Our system is designed to be intuitive and quick to learn, and you won’t have to waste time setting up any infrastructure before starting your testing. This is especially true with our Adaptive Language Processing engine (ALP). This takes test cases written in plain English and uses Natural Language Processing to convert them to tests you can run. And running tests is equally easy with integrations possible with all the best-known CI engines. 

Then comes the reliability – because we partner with Google, our service is highly available and dependable. The last thing you need is for your test infrastructure to suddenly get flaky just before a release! And talking of reliability, we offer load testing at impressive scales, with up to 100,000 connections from multiple locations globally. This means you will be sure your servers can handle the load. Even better, the load tests generate realistic user sessions based on the tests you have defined.

Intelligent Automation

As mentioned above, our tests are truly intelligent. The Adaptive Event Analysis™ engine is designed to make test maintenance extremely simple. It is even able to analyze test failures caused by issues such as choosing the incorrect selector several steps earlier. Once it has identified the problem it will then offer you a choice of solutions. Simply click on the correct one and all your tests will be updated.

After you have completed your tests you get access to incredibly detailed reports. These include screenshots taken before, during and after each and every test step, detailed page load times (including element load times and the time taken for the page to be “visually complete”).  

The last thing we will mention is the new Functionize CLI. This allows you the convenience of running and managing your tests from the command line. Because it uses Java it works on any OS. Installation is incredibly easy, and there is no need to configure anything. Among other cool features, you can use it to test against a local instance of your application.  

So what are you waiting for? Why not give the Functionize Test Cloud a go and see how autonomous cloud testing can transform your life.