How NLP reconnects you with your end users

Testing should be the connection between your developers and users

NLP helps you to reconnect you with your users

Functionize’s NLP makes pain English the new language of testing. As a result, this lets you produce reliable products that actually deliver what your end users want. Here we explain why this is the case and contrast it to traditional test automation.

Summary

Funcitonize’s NLP Engine allows you to use plain English to write all your automated tests. This has profound implications throughout the software delivery lifecycle. Now everyone from product managers to business analysts can contribute to the SDLC process. NLP engine reconnects the product and testing teams. Now, it is easy for product managers to write tests that match the real user journeys in your application. Furthermore, the gap between testing and the end customers is narrowed. Traditional test automation focuses on how the test works over what the test is trying to achieve. With NLP engine, you are telling the Functionize system exactly how the system should function from the end-user’s perspective.

Background

One of the greatest problems for software development is avoiding building software no one wants, or which is broken. Ensuring this is the job of the product and QA teams. These teams need to work closely together to ensure all necessary testing is being done, and to ensure the focus is on the most important aspects of the system.

In the traditional software development lifecycle, Business Analysts from the product team collaborate with Quality Engineers to define a set of test plans. The BA gives a rough plan to the QE and this is then refined into a detailed set of test steps. In the days before automation, these test plans were then used by manual testers to verify the product was working as expected. Other manual testers would then try to “break” the product through exploratory testing. Trying to find those paths through the system that users might stumble upon.

The problem with test automation

All this changed about a decade ago when test automation hit the mainstream. Nowadays, the aim is that all the routine functional testing of your UI should be done using automated tests. So, now the BA gives a test outline to a QE. The QE refines this into a detailed test plan and then works to convert that plan into a test script. This script is probably written in Selenese (the language of Selenium) and one or more other scripting languages (e.g. JavaScript, Python, etc.).

Writing these scripts is notoriously hard, especially if (as is likely) they have to be converted to work cross-browser. Usually, the process is incremental. The QE writes a few steps, checks that they work, debugs them and then writes some more. During this process, the QE may get distracted from the actual intended functionality and concentrate instead on creating a working script. But because it is a script, not a test plan, the BA has no way to verify that the script now accurately reflects the plan.

A different approach

Clearly, the problem here is that automated test scripts are written in a language designed for computers to understand. Computers are dumb, and so we humans have always had to develop ways to translate our language into simplified programming languages the computer can understand. But there is another way to do things. Natural language processing is a specialized field of artificial intelligence. In NLP, the aim is to teach a computer to understand human language. So, you don’t need to learn to speak to the computer, the computer can learn to understand you!

Here at Functionize, we decided to use NLP to remove the need for the script altogether. The skill of testing isn’t about writing code, it’s about knowing what to test in order to verify the system works. Nowadays NLP has reached the stage where you can just use plain English to write and create your tests. And so our engine was born.

How does NLP Engine work?

NLP engine is a key element of Functionize’s ML Engine intelligent test agent. NLP engine takes us back to the time when BAs were able to work with QEs to develop test plans that anyone could understand. With NLP engine, you can take a test plan written in plain English and can use this to generate a fully functional automated test. These plans can be structured (like the ones that are produced by test management systems), or they can be unstructured. You can choose to use keywords to remove any ambiguity, but the system will always check back with you if it didn’t understand what you meant.

NLP engine is used to translate your test plan into a form that can be understood by our ML Engine. In turn, ML engine takes all these test plans and uses these to model how your system is meant to work. Having established that, it can then run the tests you defined. As we say in our Manifesto,

Plain English is the new test language

Once the test has run, the results are then displayed graphically. Each step of the test is shown, color-coded to indicate any errors. Clicking on a step brings up a detailed set of screenshots that show what happened on the UI. ML engine annotates these screenshots to highlight any unexpected outcomes. As a result, anyone can look through the test and rapidly see what the problem was.

The benefits NLP brings to your users

So, why is NLP language so revolutionary from an end-user viewpoint?

Empowering your whole team

One of the key implications is that NLP empowers every single person in your team to be involved in the software development lifecycle process. Now, anyone from Product Managers and Business Analysts to UX Designers and even C-suite executives can understand your tests. More importantly, all these people can potentially contribute to the tests. Up till now, the people that actually designed your product had to rely on a chain of others to ensure that the product being released was working. Now, they can actually help define the tests.

This is important because these people are the ones that best understand the end users. They are the ones that came up with the user stories and customer journeys. They are the user experience experts that worked with focus groups to come up with a perfectly-designed product. So, why not let them get involved in the testing process that checks the system is delivering what was promised? That way you reconnect the people that understand your users with the users themselves.

Closing the disconnect between the product and quality teams

As we said above, there was traditionally a close connection between the quality and product teams. But with the growth of test automation, this began to break. Now, in most teams, there is a real disconnect between the Test Automation Engineers and the Business Analysts in the product team. Once upon a time, Quality Engineers were temperamentally closer to the product team than the development team. After all, their job was ensuring the product works. But modern Test Automation Engineers are much more likely to be developers that have moved across to testing.

NLP offers a way to reconnect. If you like, it offers a way to bridge the gap in understanding between QEs and BAs. Once again, the product team can be sure that the test plans are accurately representing all the user journeys through your system. They can even go into the system and visually see that the tests were doing the right thing.

Reconnecting with your customers and users

Traditional test automation approaches like Selenium brought about an absurd situation where what matters is tests that work, rather than tests that are correct. Even worse, Selenium tests are so brittle that they break almost any time you change your code. In the worst cases, a simple CSS change can break all your tests. One of the upshots of this has been the birth of a new breed of developer – the Developer in Test. This solves one problem (that of poorly-written test scripts), but it totally ignores the elephant in the room. Testing used to be about connecting with your customers and users. But test automation has actually achieved the opposite.

With NLP you can start to reconnect with your customers and users. Almost all test teams still have manual testers whose job is to understand how the user might interact with the system. This may simply be trying to second-guess the inventive ways in which they will try to break your system, but to do that requires a deep understanding of the user.

“A common mistake that people make when trying to design something completely foolproof is to underestimate the ingenuity of complete fools.” Douglas Adams, 1992

In many cases, these QEs have remained as manual testers because they lack the skills to become Test Automation Engineers. Now, NLP empowers them to leverage their customer-focusses test skills to create better automated tests.

Conclusions

NLP engine has the potential to transform automated testing. It can reconnect your product and testing teams, removing the obfuscation introduced by test scripts. Most significantly, it can reconnect you with your customers. In turn, this means you will be able to produce a product that works reliably and actually delivers what your users want.