How Functionize Fits into a CI/CD Pipeline

Implementing test automation into a CI/CD pipeline can be very challenging. One major challenge is the lack of tools in the industry that can truly automate application testing.

The essence of continuous integration and delivery is the ability to detect, categorize, and test a continuous flow of application/system changes. Another important factor is the need to shift testing leftward—upstream—in the development pipeline. In this article, we reminisce about life without CI/CD and also look at some of the challenges of moving test automation upstream. Then, we show you how Functionize helps you navigate these challenges.

Life before CI/CD

Once upon a time, continuous integration was a novel concept. There was much excitement about the advantages of continuous merges of small codes changes—with the goal of minimizing or eliminating code conflicts at the end of a development cycle. Many Dev and QA teams were looking forward to higher productivity and better quality, so they eagerly embraced the CI/CD methodology.

Developers prepare a deployment while DevOps and QA defend themselves. 

Though many teams envisioned a locomotive they could jump onto, it became much more like a catapult. At first, it was difficult to get the train running smoothly. Sometimes, it didn’t make it to the DevOps station. Developers would jettison their code into the CI/CD repository, washing their hands of code they confidently thought would sail straight and true into production. As application complexity continued to increase, QA and DevOps came to anticipate the continuous stream of changes with perpetual dread—trembling at the potentially lethal projectiles that kept coming over the wall. “How funny,” said the combatants, “that this differs little different from the old days.”

CI/CD: How it should work

Continuous integration and delivery (CI/CD) is the practice of automating software development, change management, and testing to ensure that the application is ready for deployment—at any time. A central activity in CI/CD is testing every change to the application. This includes new functionality, configuration changes, and bug fixes. The other major task is to package all of the new and changed components into the deployment vehicle.

Continuous delivery requires continuous integration, and continuous integration requires solid test automation. The same core principle applies to the entire framework: divide the work into small elements and carefully titrate the flow of those elements through automated tests and ensure delivery of a fully-verified product—all the way to the end of the pipeline. Your users will be waiting there—expecting high quality and no surprises.  

Yes, it can be scary to build software that isn’t ready. To justify the effort and expense of CI/CD, the team must build the framework and thoroughly test every change so that any code sent downstream has a high probability of contributing to maintaining customer satisfaction. 

Without CI/CD, the entire team frets over one thing always: deployment. The goal of CI/CD is to configure an automated framework that makes deployment so seamless that nobody feels any tremors. Think about the value of that. Take a guess at how many times your eyes blinked while reading this blog post. Yep, that’s right: you don’t know—or care—because it’s not painful when you blink. Unless you get debris under your eyelid. Kinda of like life without CI/CD, eh?

The benefits of CI/CD

Here are the major benefits that development teams—big and small—are realizing from their CI/CD frameworks.

Faster time-to-market — Product releases are much more frequent—multiple times a day if necessary. Overall cycle time, from the inception of a user-story all the way through to production, is now only several days (down from several months).  

Higher utility and value for the customer — Previously, considerable effort may have been spent on unwanted features—but this wasn’t known until well after the release. CI/CD enables frequent minor releases that deliver product updates and feature prototypes in front of users—much more often. If a feature is found to be unwanted, no further development effort is necessary. This saves time, money, and increases the overall utility of the product.  

More efficiency and productivity — Conventionally, developers might have spent 20 percent of their effort configuring their own test environments. With a solid CI/CD framework, the pipeline will set up the test environment(s) automatically. QA also benefits from a high degree of downstream test automation. Before CI/CD, Ops engineers might require several days to deploy an app to production. Now, it’s as easy as a quick cross-check and a click of a button to automatically push a release into production. 

Increase in reliability — Release risks are much lower with CI/CD since the entire release process is much more reliable. This is partly because all critical elements are well integrated and thoroughly tested before deployment. Most errors, if any, will be found prior to this stage. Moreover, it’s easy to rollback a release in the event of critical failure. 

Much higher product quality — Product quality is much higher. Some teams report that the average number of open bugs is 80% less than before—with far fewer deferrals. Perhaps the biggest factor here is that with each commit to a CI/CD code repository will initiate a series of automated tests to the entire code base. If any test fails, the development team must fix it before it can move further along the pipeline. This eliminates most of the bugs that would otherwise remain open. Previously, approximately 30% of development effort would go toward bug-fixing. With a solid CI/CD pipeline, it is rare for anyone to be found working on a bug found by the end user.

CI/CD requires shifting left

If your team is developing complex software and looking to ramp up on CI/CD, then it’s impracticable to move forward without implementing test automation to a significant extent. As applications and environments become more complex, throwing changes over the wall to another team is no longer effective. Many of the problems come from separating analytics from the development team during the specification phase. Also, it’s no longer efficient to wait long past development complete to find bugs.

Test automation enables easy handling of small changes, faster tests, more consistency, and more agility. A general concept that shapes CI/CD and testing automation is well-known as shift-left. It’s all about a progressive shift of testing, discovery, and automation as far upstream as possible. From a conventional vantage point, this is shifting leftward toward the earlier phases in the development process. Of course, implementing shift-left doesn’t come without challenges.

The impact shift left has on the company

Ideally, the entire development team—including testers—should participate in specifying user stories. This will serve to eliminate defects well upstream, way before implementation or delivery. But the crucial distinction with CI/CD testing is that QA will be keen to create a testing strategy for the entire process. Testing will no longer be driven by large, infrequent dumps of output from the development team, or constrained by specific scenarios. Rather, QA becomes part of the product definition. This can be challenging because it requires early collaboration between developers and testers.

The impact of shift left on developers and project managers

Shifting left means an end to waiting for testing to begin at the end of the development phase. A solid way forward here is to arrange for unit testing and independent testing of small elements of the application. This can be divided into testing front- and back-end components. While challenging at first, QA can divide further and automate the testing of isolated back-end components. It’s important to realize that this requires close collaboration among the testers and developers. Simply speaking, CI/CD requires that you reorganize product development so that any defects can be found collaboratively—in close synchronization on each code commit.

For some app components, it might be good to pair developers with testers. Though testers are reluctant if they don’t have development skills. But, experienced testers can bring a fresh perspective in analyzing a particular implementation. This, of course, is the passion for quality. Also, a tester is typically a strong advocate for the end-user. This is often a solid combination of domain, quality, and technical expertise that can help the developers adopt a stronger user-orientation and also learn to prevent defects in their code.

How Functionize fits into CI/CD

If you’re responsible for testing a complex application, ask yourself:

• How does your testing automation solution handle all of this?
• Does it have any capability at all to adequately automate the testing of highly complex application or systems?

Here’s how Functionize can help you maximize the value of your CI/CD pipeline:

Smoke testing — Testing should begin when development makes a submission into the code repository. Testing should happen as early as practicable, in sync with development efforts. Frequent smoke test execution is essential and is vital for each new build of each application—in both staging and production environments. Functionize makes it easy to configure and execute your smoke tests. 

Integration with CI/CD tools — Functionize weaves tightly into your pipeline since it’s seamlessly compatible with popular platforms including GitHub, Circle CI, Heroku, Jenkins, Travis CI, Pagerduty, AWS CodePipeline, and Jira. With Functionize, you get a highly-secure tunneling integration. From a development server, you can test cases in with cloud-scalability—without exposing any data. 

Orchestration control — With Functionize orchestrations, you can set the testing order sequence, or have them all run in parallel. This offers a high degree of flexibility in managing tests and easily integrates them directly into the CI/CD pipeline—irrespective of configuration or complexity. If necessary, you can leverage webhooks and API calls to automatically trigger any amount of orchestrations—with the power of in-the-cloud scalability. The team can review orchestration results and readily determine if it’s sensible to move forward with the deployment. It’s easy to execute orchestrations in multiple environments and many times throughout a release cycle. 

Build momentum further upstream — Shift-left begins with developer code. Functionize gives you maximum flexibility to configure upstream triggers that initiate critical tests. This is how velocity is built upstream, and your productivity gains can multiply from there. 

Performance and load testing — Get the ability to leverage not only functional tests but performance and load tests. It’s too easy to manage using our unified platform and centralized data. 

Flexible with various CI/CD pipelines — Since testing should start as early as possible, Functionize is flexible enough to support different CI/CD implementations, across the most elaborate pipelines. It’s easy for any team member to configure how early or late various tests should occur, and in how many stages of the pipeline. 

Functionize is the only comprehensive browser-driven, cloud-based test automation solution that can help you automate and integrate well with your CI/CD pipeline. If you’ve got questions, we’ve got answers. Don’t hesitate to give us a call or send us a message.