Removing Selenium heartache: QA with Zero Engineers Required

SyTrue is a high-tech company specializing in Natural Language Processing (NLP) to abstract vast amounts of unstructured data from medical records for downstream communications and analysis. Having processed over a billion medical records, in multiple formats, SyTrue works with some of the largest healthcare providers and payers in the United States, such as Humana, Change Healthcare, and vRad, a division of MedNax. SyTrue’s NLP OSTM uses artificial intelligence and machine learning to ingest the raw medical records in multiple formats. The company also provide tools such as the Natural Language Query Search (NLQS) allowing end-users to search for specific findings within the medical record population from simple natural language. This has significant impact streamlining how healthcare payers and providers query their huge datasets and return useful, relevant results in seconds.

Challenges

As with all good software, the SyTrue platform is a dynamic ecosystem that is advancing and broadening capabilities to address market problems and client satisfaction. This causes significant challenges for testing, both in the creation of suitable tests and in running those tests. With SyTrue, the testing is made more complex by the underlying nature of the data they deal with and by the large number of constituent modules that make up their NLP OSTM. As with so many other companies, SyTrue started off using an entirely manual testing approach. However, with the company’s rapid growth it became clear SyTrue needed to move from manual testing and invest in automated test tools. Therefore, SyTrue decided they needed to invest in automating their testing.

Initially, SyTrue explored whether Selenium would provide a suitable solution. They quickly hit the classic problems with Selenium – complex scripts that didn’t seem to behave as expected and took days or weeks to write and debug. Quickly, SyTrue’s QA team realized they were banging their heads against the wall. The team had failed to create a single usable test despite many man-hours of effort and pain. As Kyle Silvestro, SyTrue’s CEO, puts it “With Selenium we were going nowhere, and I wasn’t looking to hire an automation engineer.”  

At this stage, they looked around to see if they could find a company that would be able to help. Their requirements were straightforward. They wanted a simple system that would allow them to create automated tests without any need for a deep background in test automation and Selenium. A Google search quickly turned up Functionize and we were able to step in and help them out.

How Functionize helped

The Functionize system is designed to hugely simplify the process of creating and maintaining automated tests for web applications. Unlike Selenium scripts, Functionize tests are able to cope with context-sensitive data, inconsistent selectors and minor changes to UI. Creating tests is designed to be as straightforward as possible and maintaining tests is even easier.

When SyTrue first reached out to us we were able to demonstrate to them how simple our system is to use and how powerful it is. Initially, we worked with them to help create their first tests using Functionize. Within a couple of weeks, they were confident enough to start creating tests on their own and now have a suite of well over 100 tests, with new tests being added almost daily. Many of these tests consist of 100+ steps. Given the complexity of the app, it would simply not have been possible to create reliable tests in Selenium.  

Functionize’s advanced AI capabilities mean it is able to cope with extremely complicated test cases where Selenium and other technologies would simply break down. One of the most complex modules in SyTrue’s app is the Clinical Editor. This uses nested Iframe elements for its various functionalities. Iframe elements are always difficult to target using scripting technologies like Selenium. What made this case even more complicated was the dynamic nature of these elements, appearing on demand based on the current work mode for the module. Our experts were able to work closely with SyTrue to create suitable tests using our advanced element targeting algorithm. This has resulted in an easy to use, stable set of tests that accurately target the content in these dynamic iframe elements.

Benefits/Outcomes

For SyTrue the benefits have been obvious. They were keen to avoid investing heavily in employing a large team of test automation engineers who would need to be trained up and would doubtless see a large turnover given the shortage of test automation engineers. Instead, they wanted a scalable automated solution that can be easily leveraged. Above all, they needed to be able to release bug-free software while not falling foul of the constraints imposed by the Healthcare Insurance Portability and Accountability Act (HIPAA). As Kyle Sylvestro, the SyTrue CEO, puts it:

“We needed a platform that was going to enable this level of automation … and you guys checked all the boxes.”  

Because Functionize is so simple to learn it allowed Leslie (who has no background in software development, let alone test automation) to be functional within days. It has also avoided the need for SyTrue to invest developer time in creating and debugging complex Selenium scripts, something their Chief Architect was keen to avoid.  

Leslie has been able to work closely with the Functionize team to maintain and extend the test suite. The team is now familiar enough with SyTrue’s system that they are able to create new tests just by Leslie sending a few screenshots and emails. In turn, this has meant that Leslie’s time has been freed up to allow him to concentrate on the things that Functionize cannot cover. Leslie appreciates how little effort he has to put in maintenance, stating it this way: 

“With Functionize, I spend almost no time manually maintaining our test suites.”  

Since we started working with them over a year ago, SyTrue’s experience of Functionize has been extremely positive and highlights many of the key features we offer to customers. We’ll give the last word to Kyle: 

“Too many people want to get too complicated in software and you guys have done a good job keeping it simple.”