Progression testing matters too! Why you should expand your (regression) test automation.

Software testing is full of terminology, and some terms can be confusing. Here, we explain the key differences between progression testing and regression testing. We also show you how easy it is to automate your progression testing.

What is system testing?

Before we look at regression and progression testing, we need to understand where they fit in the overall testing picture. Software testing is traditionally divided up into stages. These start with unit testing, which is performed as functions are developed. You then move onto integration testing, where several functions are tested as a complete functional unit. Next comes system testing. This is where you test your complete software for functionality and bugs. Finally, you need some form of acceptance testing, where you verify that the software does what the end-user needs. Regression testing and progression testing are typically a part of your system testing.

What is regression testing?

Regression testing is vital any time you change or update your software. The idea is to make sure that your new changes haven’t broken any existing features. There are a few different things you are trying to answer.

1. Does the software function as expected?
2. Have any previously-fixed bugs re-emerged?
3. Have any previously undiscovered bugs been triggered?

Typically, regression testing involves running a selected set of your existing tests. The skill lies in selecting the best set of tests to fully answer the questions above.

Why is it so important?

Regression testing is one of the most important forms of testing for any software. There is nothing worse than releasing a new update with some exciting new features, only to find that your software is buggy. Users can be really quick to condemn buggy software. Especially if they see an old bug re-emerging. For a user, triggering a bug suggests two things. Either, you have just shipped broken software, which looks careless. Or, if they are not a confident user, they may think they did something wrong, which provides for a terrible user experience. In both cases, you have just damaged your reputation.

What is progression testing?

Progression testing is a less frequently used term. As the term implies, it’s to some degree, the opposite of regression testing. Progression testing is about testing new features, rather than existing features. So, it is about expanding your overall testing. Progression testing takes a few forms.

Creating tests for new features

Each time you develop a new feature or update an existing one, you need to create new tests that verify it is working. These tests focus on the specific functionality of the new feature. For instance, does it display correctly in the UI? Is the application logic for it working correctly? How well does it handle errors or unexpected inputs?

Exploratory testing

Modern software applications are among the most complex entities we have created as a species. Consequently, there are often millions of ways that a user can navigate through the application. Ideally, you’d be testing every one of these but that is simply unrealistic within reasonable time constraints. However, you need to make sure you are testing the ones most likely to cause problems. This is where you turn to exploratory testing. Here, an experienced tester takes the application and tries to trigger problems based on her knowledge of the app and common bugs. For instance, testing how a mobile application reacts when you change from cellular data to WiFi.

Bug hunting

The final form of progression testing is hunting for new bugs. All too often, your users will be the first people to trigger a bug. If you are lucky, they might submit a bug report or contact your support team. But often you will only know about the bug from the application and system logs. Here, the priority is to work out what are the steps needed to recreate the bug. Once you know those, then developers have a definite target for debugging.

How do you move from progression tests to regression tests?

Typically, progression tests end up being added to your regression test suite. For instance, if you have identified a bug, you need to test for it again in the future. This applies to bugs found during testing and bugs found in the wild. And needless to say, you always need to be testing the existing features in your app. So, tests developed for new features become regression tests for existing features. Often, during this process, the test migrates from being a manual test to being automated.

How to use FZE for progression testing

One reason progression tests tend to be manual is that automating a test is a slow and time-consuming task. Most progression testing is reactive or exploratory in nature. That means you don’t have the time or resources to automate the test until you are certain it is correct.

But now Functionize Architect makes it far quicker and easier to create new tests. You simply need to interact with your application through Chrome with the Architect extension running. Our AI backend takes what you do and converts it into a smart test. This is especially useful for developing tests for new features. However, it can also be useful for exploratory testing or bug tracking. For instance, you might suspect that a bug will be triggered if certain characters are entered in a text box. This might happen with accented characters within someone’s name. With Architect, you can quickly create a test, connect it to a data source, and test dozens of different names at once. 

Another reason why progression tests are rarely automated is that new features are inherently less stable. The newer the feature, the more likely it is that the developers will make last-minute changes. Perhaps during development, the product manager changes her mind on the style or positioning of a button. Or during beta testing, customers request the button to look entirely different. Because traditional automation scripts are time-consuming to maintain, many testers decide to defer automating tests for new features until the window of likely changes has passed. But Functionize tests are robust against these sorts of changes, so there’s no need to delay automating the testing.

How to use FZE for regression testing

Where Functionize really comes into its own is regression testing. As we saw, it’s incredibly easy to automate your tests using Architect. But we also offer other ways to create tests, such as NLP or live recording of your production system. Together, this means any team can easily automate all their regression tests. But what are the benefits? Well, here’s just a few:

No test maintenance. Our tests are powered by AI. Our intelligent system learns how your application actually works and understands what your tests are trying to achieve. That means the tests dynamically heal each time your application changes or evolves. Where there is any ambiguity, the system will offer you a number of SmartFix suggestions to choose from. 

Visual testing. Typically, automated scripted tests only check a fraction of your application’s UI. That’s because you have to explicitly code every aspect you want the script to check. By contrast, we adopt a visual testing approach. Every time a test runs, the system records hundreds of screenshots. These include before, during, and after each step of the test. In addition, the system records details like the computed CSS for every element, any hidden content, etc. If anything changes unexpectedly, the system will flag it, mark it on the screenshots, and let you know. 

Testing at scale. All our tests run in the Functionize Test Cloud. That allows you to test at an unprecedented scale. You can run thousands of tests in parallel, across any browser, including mobile. Tests can be launched from locations all over the globe. That makes it really easy to do internationalization and localization testing. Moreover, this is all powered by a rich set of orchestrations, allowing you to create complex test scenarios, integrate with CI/CD, and more. If you'd like to compare FZE to other tools, we compiled a full list of the best regression testing tools where you can see the features, benefits and drawbacks of each software and choose the one that suits your needs the most.

Ready to learn more?

Hopefully, you now understand more about regression and progression testing. Maybe you’re even feeling inspired to try using Functionize for your progression testing? Or you want to see how easy it is to automate all your regression tests? Trying out Functionize is incredibly easy. Just install the Chrome Plugin and register for our free trial. You’ll be up and running in no time at all.