Save DevOps time with a cloud-first testing strategy
As a DevOps engineer, you have a tough job ensuring everything works. Here, we see how cloud-first testing can help you save time and avoid nasty surprises.
Agile teams seldom get a chance to stop and catch a breath. This is especially true for DevOps engineers. Many people have tried to give definitions of the role of a DevOps engineer, but it can be summed up as “make it all work.” Essentially, you are the grease that allows high-velocity CI/CD releases to happen smoothly while ensuring the end user still receives the best UX. But testing often steals time and resources from you, distracting from your core responsibility.
As you know, it’s hard enough being a DevOps engineer at the best of times. But increasingly, you are expected to help sort out the complex issues faced by the QA team. These issues range from maintaining their test infrastructure to automating the deployment of new images. And chances are, you will find yourself being asked to help out with other scripting tasks. Many of these problems come about because test automation is still stuck over a decade in the past. And in our world, that counts as prehistoric. However, it doesn’t have to be like this with a cloud-first testing strategy.
The problem with automated testing
Selenium revolutionized the world of testing when it was released almost 20 years ago. Finally, it was possible to automate the repetitive UI regression testing that is essential prior to releasing any product. Selenium works by using a script to interact with the UI, replicating the actions of a user. The original version only worked on a limited range of browsers and as a standalone instance. Selenium Webdriver and related projects led to it working on the majority of browsers. Later, Selenium Grid allowed multiple instances to be managed from a single control node. The trouble is, Selenium wasn’t really designed to scale in this way.
Testing at scale
Comprehensive regression testing for a modern web application might require 100s of different tests to be run on each of 20-30 browser/device combinations. Each of these tests takes several minutes to run. Clearly, running these one at a time is simply infeasible unless you want each release to take months. Selenium Grid tries to solve this by allowing one control node to run several dozen test nodes.
On the face of it, Selenium Grid will help you scale your testing as required. However, it has real issues. Firstly, it isn’t really optimized for running at large scale. 10 servers is completely feasible, 100 starts to become really hard, more than that and you are into the realms of the impossible. The only solution is to constantly repurpose servers during your testing. Secondly, you almost certainly need to reconfigure servers dynamically in order to ensure none are standing idle. Not all tests take the same time to complete. This is especially true when a test fails, as happens all too regularly with Selenium. Thirdly, all these servers need to be kept updated with the latest browser releases, security patches, etc. This is essential from both a security viewpoint and a test quality viewpoint.
Why is this a pain in the butt for DevOps?
Test automation engineers already have to combine two distinct skills: understanding testing and writing efficient test scripts. Finding people that are skilled in two mismatched disciplines is hard enough. Finding ones that combine these skills with sysadmin skills is effectively impossible.
As a result, the DevOps team usually ends up supporting the testing team. We often hear reports of teams needing dedicated support from DevOps engineers. And as you well know, DevOps engineers are a scarce commodity—even scarcer than developers in test! However, as a DevOps engineer, it is in your best interest to ensure all testing has been done as efficiently and effectively as possible. You don’t want testing to become a roadblock to efficient releases, especially if you are running CI/CD. But even more, you don’t want to release software with bugs. So, the solution is to move to a cloud-first testing strategy.
Moving to the cloud
The first step in any cloud-first testing strategy is moving from physical to virtual infrastructure. There are a couple of ways to do this. You can set up your own test cloud using a public IaaS provider. Effectively, you are creating a whole new backend, just for managing your testing. This will save some DevOps time by removing the need for machine maintenance. However, you will still end up doing additional tasks to automate the setup, ensure security, etc.
Alternatively, you can use a managed service. There are a number of companies that offer Selenium automation in the cloud. Often, they market themselves as cross-browser testing services. Companies such as these save even more DevOps time, but they cost thousands of bucks a month for just a few dedicated test instances.
AI-powered testing as a solution
AI-powered solutions like Functionize have revolutionized testing. We have taken the cloud-first testing strategy to its logical limits. As our Manifesto says, “Testing must be cloud-first”. This means we don’t just move your test infrastructure to the cloud. Our solution also leverages the almost-infinite resources of the cloud to power our advanced AI models.
Test creation
Architect allows absolutely anyone to create automated tests using a simple interface. The tests simply work first time and can run on any browser, including mobile. Architect even offers advanced features, such as automatic creation of test data, testing for two-factor authentication, and storing variables to use between tests.
Test maintenance
Our system uses multiple forms of artificial intelligence to learn exactly how the system under test works. This allows us to use Smart Element Recognition to identify elements in your UI. The result is, tests don’t break whenever the UI is updated or the application logic changes.
Test analysis
Running tests is all very well, but what about analyzing the results? This is where our visual testing approach comes to the fore. Every time you run a test, our system will capture screenshots before, during and after each test step. It then uses intelligent visual analysis to highlight problems. You can also choose to visually verify specific elements in the UI, or even the entire UI.
Leveraging our test cloud
All of the above rely on the Functionize Test Cloud. This allows you to run tens of thousands of tests in parallel. Each test runs in its own virtual server, meaning the load on your backend is more realistic. The test cloud can support any browser/device you need to test, including emulation of mobile devices. It also stores the history of all your tests, so you can compare results over time. This can often help identify backend issues before they become a problem.
So, how is this saving DevOps time?
Put simply, by moving your company’s testing to Functionize, you no longer need to waste DevOps resources supporting the QA team. Less test maintenance means more time for automating tests, which cuts the risk of uncaught bugs. Cloud-scale execution means more testing can be done with every release, driving up quality. Better still, Functionize tests can be easily incorporated into any CI/CD toolchain. This makes it easy to ensure smooth, bug-free automated releases. In short, this is a win-win for both the QA and DevOps teams.
Conclusions
Test automation is essential nowadays. But the classic approach to test automation has a habit of eating DevOps resources. As we have seen, the solution is to adopt a cloud-first testing strategy. If you want to find out how Functionize saves DevOps resources, book a demo today.
