Automotive Infotainment System Testing Automation

Automotive equipment manufacturers continue to face the challenge of providing in-vehicle infotainment (IVI) systems that delight their customers. Automobile occupants expect to interact with new infotainment systems that are intuitive, powerful and exhibit high quality. For a new IVI system to be viable in the marketplace, it must easily integrate with mobile device features and provide seamless connectivity.

Newer technologies continually increase the expectations of vehicle owners, so IVI manufacturers are under constant pressure to add more features and content. Many vehicle owners also have direct experience with the innovation in computing devices and smartphones, so naturally, they expect the same evolution in newer automobiles.  

This additional complexity increases the probability that product defects will occur. Compounding this is the steady pressure to reduce the new product development cycle down from the conventional 4-5 years to aggressive consumer expectation of about 1 year for new product features. Add it all up and it’s clear that testing the newer IVI systems can be an extremely challenging effort.

The challenges of testing complex infotainment systems

Owing to its extensive and elaborate functionality, a new infotainment system is now the most complex system in the vehicle. It has a broad-distribution architecture with a wide range of I/O types, a modern infotainment system must support multiple buses, RFs, and wireless connectivity. The IVI must also process huge data volumes and expose several MMI interfaces such as touch screen, multiple user displays, audio I/O, and various switch interfaces.

Taking all this complexity—together with the incessant product innovation and risk of product defects—automotive manufacturers and equipment suppliers have come to these two urgent realizations:

• Conventional automotive system and software testing tools are quickly becoming inadequate for evaluating and verify newer IVI systems.
• Because of the diversity and extent of asynchronous I/O, it has become virtually impossible to maintain a manual testing program to test even a subset of all use cases for a new infotainment system.

Autonomous testing platforms are emerging to accommodate highly complex systems

The Functionize testing platform can readily accommodate extensively complex systems such as IVI. Functionize has many capabilities that make it highly suitable for IVI testing, such as accommodating a wide variety of new interfaces, pattern generation from test primitives, support for concurrent testing, image capture, and environment simulation. With Functionize, users can develop function-level or feature-level test cases and automate test execution—much more rapidly than was possible before. Extensive stress, stability, and endurance tests can be built from primitives that will expose any remaining defects or inefficiencies—earlier in the development cycle and well prior to in-vehicle testing.

Addressing the challenges of infotainment system testing

IVI system software presents significant challenges. Electronic control unit (ECU) suppliers specify choose one or several operating systems and software frameworks. Many cars built today contain more than 100 Electronic Control Units (ECUs) and a total of over 10 million lines of software code. While the number and complexity of ECUs in vehicles continue to increase, pressure continues to intensify on manufacturers to ensure the safety and reliability of ECU software.

A particular IVI or ECU software feature can function independently—or it may cooperate with other components to provide more complex functionality. As a software team develops a multitude of features, late arrival of some features will often negate or neutralize some of the original assumptions about expected performance. Moreover, the integration with the IVI system can reveal issues that apparent upstream in the product pipeline. 

Achieving thorough and proper testing of the IVI system requires that the team mitigate the most difficult challenges first. Considering the technical complexities of an IVI system, it’s unsurprising that these are the top concerns of OEMs and systems suppliers:

• Huge number of features
• Numerous interfaces
• Complex user interfaces
• Mobile device support—different hardware and software
• Interactions among many ECUs
• Adding unplanned features into the current product development lifecycle

An IVI testing solution must take these and other considerations into account. The Functionize testing platform is designed with these and many other platform challenges in mind.

Computer vision can significantly improve testing automation

An area of particular concern to human testers is the visual inspection and verification of display screen feedback. For example, when a driver or passenger selects a different language that is verbose and requires additional space for legible display, most automation platforms fail to handle this correctly. The additional text may overlap another control or button and make it difficult to read properly.

Using enhanced computer vision algorithms, Functionize captures screenshots at every test case step for analysis by machine learning algorithms. Managed by a highly autonomous apparatus, many test cases will repeat and permutate hundreds of times. Even after any display elements have been relocated or altered. This is made possible through a proprietary AI subsystem known as Adaptive Event Analysis—a breakthrough technology that clearly distinguishes pages and page-elements that are new or have been relocated during the most recent development cycle.

The challenge of multiple interfaces

A vehicle may have multiple user interfaces, such as the human-machine interface (HMI) device for input and display located on the center console. Another common HMI is the instrument cluster display that is controllable by steering wheel switches. Commonly, there is also an HMI to control features in the rear of the vehicle. System control design must implement the command hierarchy throughout the vehicle and at each HMI. It should also easily resolve any contradictory instructions or inputs.

To test such a system, the test designs must cover all combinations of each normal use case. Test architecture must also cover undefined or inconsistent inputs. Permutations must be done with respect to timing, order, and cadence to ensure system resilience. The testing systems must also track and I/O and handle pass/fail resolution. Each exception and failure must capture all possible details so that the tester can readily determine the state of each participating device.

Testing automation can be so much better

Here’s something to think about: How does your testing automation solution handle all of this? Does it have any capability to test IVI, or systems of such high complexity? Functionize has been built from the ground up to accommodate all of this complexity, and so much more.

As we wrap up this article, we leave you to consider these Functionize benefits:

• Automatic test setup—no need for additional investment on test tools or hardware
• Easily reproduce test case failures
• Large-scale, highly automatable overnight test capability.
• Short learning curve for installation and use.
• Reduces multi-interface/device interoperability cycle time
• Find many issues that would not be found with manual or conventional automated tests
• Measure and report on the performance of each test case